Subscribe: Comments on: Why (on Linux) am I seeing so much RAM usage?
http://chrisjohnston.org/2009/why-on-linux-am-i-seeing-so-much-ram-usage/feed
Added By: Feedage Forager Feedage Grade B rated
Language: English
Tags:
active total  buffers  cache  free  iloga proc  iloga  inode cache  memory  pagecache  power  proc sys  proc  root iloga  size  sys  total 
Rate this Feed
Rate this feedRate this feedRate this feedRate this feedRate this feed
Rate this feed 1 starRate this feed 2 starRate this feed 3 starRate this feed 4 starRate this feed 5 star

Comments (0)

Feed Details and Statistics Feed Statistics
Preview: Comments on: Why (on Linux) am I seeing so much RAM usage?

Comments on: Why (on Linux) am I seeing so much RAM usage?



a blog about whatever i feel like blogging about



Last Build Date: Wed, 31 Dec 2014 22:03:46 +0000

 



By: farouk chi

Tue, 12 Feb 2013 14:06:26 +0000

An outstanding share! I have just forwarded this onto a coworker who was doing a little research on this. And he actually ordered me dinner simply because I discovered it for him... lol. So allow me to reword this.... Thank YOU for the meal!! But yeah, thanks for spending the time to discuss this issue here on your blog.



By: alieblice

Sun, 24 Jul 2011 17:30:03 +0000

nice story



By: babagau

Wed, 07 Apr 2010 11:46:25 +0000

Excellent and explanatory!



By: Nate Johnston

Tue, 26 Jan 2010 22:05:42 +0000

Chris, There are some applications that will have issues when confronting a large cache size. In particular, I had a tomcat that was complaining of memory exhaustion on an 8GB linux host I was able to drop the cache size from 4015 megabytes to 50 megabytes. First, here is the status quo ante. The "cached" field indicates the combination of the pagecache, the inode cache, and the dentries cache. The pagecache is a copy of files on disk for speedier access. Since this application does not need speedier access to files on disk, the size of this can be tuned down. [natej@iloga-m02]~% free total used free shared buffers cached Mem: 8176800 8111792 65008 0 255144 4109096 -/+ buffers/cache: 3747552 4429248 Swap: 4192880 0 4192880 I checked the kernel cache, but the kernel only reported ~267 megabytes cache. See the "Active / Total Size" line in the top block of slabtop output: [natej@iloga-m02]~% slabtop -s c Active / Total Objects (% used) : 1385680 / 1402585 (98.8%) Active / Total Slabs (% used) : 76175 / 76188 (100.0%) Active / Total Caches (% used) : 85 / 133 (63.9%) Active / Total Size (% used) : 271147.38K / 273534.20K (99.1%) Minimum / Average / Maximum Object : 0.02K / 0.19K / 128.00K OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME 134296 134271 99% 0.82K 33574 4 134296K ext3_inode_cache 1006335 1006166 99% 0.09K 22363 45 89452K buffer_head 169552 169429 99% 0.23K 10597 16 42388K dentry_cache 37933 37857 99% 0.52K 5419 7 21676K radix_tree_node 2968 2968 100% 0.55K 424 7 1696K inode_cache 750 718 95% 2.00K 375 2 1500K size-2048 2016 2016 100% 0.59K 336 6 1344K proc_inode_cache 608 608 100% 1.94K 304 2 1216K task_struct 1112 529 47% 1.00K 278 4 1112K size-1024 256 256 100% 4.00K 256 1 1024K biovec-(256) 4686 4461 95% 0.17K 213 22 852K vm_area_struct 198 188 94% 4.00K 198 1 792K size-4096 5828 5619 96% 0.12K 188 31 752K size-128 Then I checked the kernel variables for cacheability: [natej@iloga-m02]/proc/sys/vm% sysctl -A | egrep "swap|cache" vm.drop_caches = 0 vm.pagecache = 100 vm.vfs_cache_pressure = 100 vm.swappiness = 60 fs.quota.cache_hits = 0 Adjusting the vfs_cache_pressure variable to more aggressively prune the dentry and inode caches doesn't do much of anything. But adjusting the maximum size of the pagecache instructing the kernel not to use 100% of memory for cache does help for new cache allocations. [root@iloga-m02]/proc/sys/vm# echo "10 20 40" > /proc/sys/vm/pagecache Finally, in order to cause the cache to be freed I used the drop_cache to drop just the clean pages in the pagecache. Always, always sync thrice before doing this, because in very rare instances it can cause a kernel panic if the number of dirty pages causes the swapout mechanism to choke. [root@iloga-m02]/proc/sys/vm# free total used free shared buffers cached Mem: 8176800 8115404 61396 0 255148 4110724 -/+ buffers/cache: 3749532 4427268 Swap: 4192880 0 4192880 [root@iloga-m02]/proc/sys/vm# sync && sync && sync [root@iloga-m02]/proc/sys/vm# echo 1 > /proc/sys/vm/drop_caches [root@iloga-m02]/proc/sys/vm# free total used free shared buffers cached Mem: 8176800 3680508 4496292 0 608 38832 -/+ buffers/cache: 3641068 4535732 Swap: 4192880 0 4192880 The pagecache can vary upwards, but hopefully it should not grow larger than 20% of RAM under [...]



By: kersurk

Fri, 18 Sep 2009 06:11:28 +0000

"OR imagine you like a song. You record it to the beginning of a cassette tape. When you want a new song, do you re-record over the first song or record after it?" Damn, it's 2009 :) Who on earth uses the cassette tapes for recording anymore. Anyhow, thanks for the explanation.



By: syamsul

Sun, 19 Jul 2009 05:56:39 +0000

Thanks Chris for the insightful article. Just one question - do OpenVZ and Xen based VPSes differ in the way they manage/allocate memory? The reason I'm asking is that on my OpenVZ VPS, I hardly see anything under Buffers or Cached after doing a free -m. OTOH, the Xen VPS seems to be allocating quite a bit to buffers and cache like you said.



By: JT

Wed, 27 May 2009 07:08:43 +0000

Re: Chris Routh's comment above about being green. More information in RAM doesn't mean more power consumption (or vise versa). Most memory technology is made up of "destructive technology" (capacitors) because it's so cheap. Whenever a bit is read it's destroyed. Picture a bottle of water with the outside painted completely black. You have to empty it to know what's inside! This happens billions of times a second. Now, pretend there's a hole in the bottle of water with the outside painted black. You'd have to empty and refill it just to make sure it retained some semblance of its contents. This is true of memory too. Capacitors are like leaky water tanks that continually drain. The computer has to read (which takes power), calculate (which takes power), and write (which takes power too) thousands of times a second just to keep information in memory. More goes into it, of course, but in the grand scheme of things, it doesn't matter whether the information is full or not in terms of power consumption. Not to mention that unallocated memory is undefined and can be all 1s, all 0s, or any combination of the two ;-).



By: Chris Routh

Wed, 13 May 2009 23:13:25 +0000

What about the green argument? Holding information in RAM requires power to keep it there, or else the computer forgets it. RAM that has nothing in it has no power cost to the system, therefore you are being more power-wise by keeping system memory clear?



By: rod

Wed, 13 May 2009 02:58:12 +0000

that was a pretty awesome article, halfway trough it i had my "oh.. duh" moment. :D



By: Andrea

Fri, 01 May 2009 22:09:44 +0000

Nice article :)