problem description
currently, data caching is done in the data layer, mostly using memcached or redis for data caching.
but recently found that if you cache files directly, the first read is slower than mem, but the second time is much faster than mem.
Baidu for a while, the reason is:
Cache Memory (cache memory):
when reading and writing files, the Linux kernel caches files in memory in order to improve read-write performance and speed. This part of memory is Cache Memory (cache memory. , Cache Memory will not be released automatically even after your program has finished running. This will cause you to read and write files frequently in the Linux system, and you will find that there is very little physical memory available.
then take a look at the disk io, for the first time, and basically get the data in memory. And the larger the amount of data, the slower the mem, and the file reading speed is almost unaffected.
questions
so, when there are not many pieces of data (the usual number is about 100s and the file size is about 100k), should we use mem or just use file caching?
ps: is tested on the local machine and gets a piece of data for one week, and the speed of the file is more than 5 times faster than that of mem.