Brought to you by : Chandan.
 
HomeHome  FAQFAQ  SearchSearch  RegisterRegister  MemberlistMemberlist  UsergroupsUsergroups  Log in  
Share | 
 

 Memory usage Retrieval on Linux ( process wise and general )

View previous topic View next topic Go down 
AuthorMessage
chandan



Number of posts: 11
Age: 30
Registration date: 2007-04-13

PostSubject: Memory usage Retrieval on Linux ( process wise and general )   Sun Sep 09, 2007 5:59 pm

Note : This guide is applicable to Linux kernels 2.6.14 and above, which adds support for smaps, per-mapping data, including data on each mapping's rss usage.

Most Admins and Developers are used to retrieve memory usage information of a process with VMSTAT or RSS, but these are not accurate,

VMSIZE counts pages that are not mapped in to memory.

RSS multi-counts shared pages, and hence is also more or less useless.

But fortunately, form kernel 2.6.14 (November 2005) onwards, has support for smaps, per-mapping data, including data on each mapping's rss usage.

This mapping data resides in /proc/$pid/smaps.

But it is not easily comprehend-able and lists all private regions and public region usage separately.

we can use this following script written by Ben Maurer, undergraduate at Carnegie Mellon University ( can reach him at, bmaurer@andrew.cmu.edu), many thanks to him.

The perl script follows below, which uses the following perl modules, But before that make sure that you have the following perl module, which is perls interface for smap facility.

Linux::Smaps ( this in turn depends on Class::Member 1.3 )

So if the modules are not there in your system, than get them form the following links.

1.) http://search.cpan.org/CPAN/authors/id/O/OP/OPI/Linux-Smaps-0.01.tar.gz

2.) http://search.cpan.org/CPAN/authors/id/O/OP/OPI/Class-Member-1.3.tar.gz

to install these two , decompress both as follows :

Code:
bash # gunzip -v ./Class-Member-1.3.tar.gz ; tar xvf ./Class-Member-1.3.tar
bash # cd ./Class-Member-1.3
bash # perl Makefile.PL
bash # make
bash # make test
bash # make install

bash # gunzip -v ./Linux-Smaps-0.01.tar.gz ; tar xvf ./Linux-Smaps-0.01.tar
bash # cd ./Linux-Smaps-0.01
bash # perl Makefile.PL
bash # make
bash # make test
bash # make install

Now you can run the following script, with pid as the argument name. To get the complete memory statistics of a particular process.

Code:
#!/usr/bin/perl
#smaps.pl
#typical usage is as follows :
# smaps.pl pid

use Linux::Smaps;

my $pid=shift @ARGV;
unless ($pid) {
        print "./smem.pl <pid>\n";
        exit 1;
}
my $map=Linux::Smaps->new($pid);
my @VMAs = $map->vmas;

format STDOUT =
VMSIZE:  @######## kb
$map->size
RSS:    @######## kb total
$map->rss
        @######## kb shared
$map->shared_clean + $map->shared_dirty
        @######## kb private clean
$map->private_clean
        @######## kb private dirty
$map->private_dirty
.

write;

printPrivateMappings ();
printSharedMappings ();

sub sharedMappings () {
    return grep { ($_->shared_clean  + $_->shared_dirty) > 0 } @VMAs;
}

sub privateMappings () {
    return grep { ($_->private_clean  + $_->private_dirty) > 0 } @VMAs;
}

sub printPrivateMappings ()
{
    $TYPE = "PRIVATE MAPPINGS";
    $^ = 'SECTION_HEADER';
    $~ = 'SECTION_ITEM';
    $- = 0;
    $= = 100000000;
    foreach  $vma (sort {-($a->private_dirty <=> $b->private_dirty)}
                                  privateMappings ()) {
        $size  = $vma->size;
        $dirty = $vma->private_dirty;
        $clean = $vma->private_clean;
        $file  = $vma->file_name;
        write;
    }
}

sub printSharedMappings ()
{
    $TYPE = "SHARED MAPPINGS";
    $^ = 'SECTION_HEADER';
    $~ = 'SECTION_ITEM';
    $- = 0;
    $= = 100000000;

    foreach  $vma (sort {-(($a->shared_clean + $a->shared_dirty)
                          <=>
                          ($b->shared_clean + $b->shared_dirty))}
                  sharedMappings ()) {

        $size  = $vma->size;
        $dirty = $vma->shared_dirty;
        $clean = $vma->shared_clean;
        $file  = $vma->file_name;
        write;


    }
}

format SECTION_HEADER =
@<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
$TYPE
@>>>>>>>>>> @>>>>>>>>>>  @>>>>>>>>>  @<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
"vmsize" "rss clean" "rss dirty" "file"
.

format SECTION_ITEM =
@####### kb @####### kb @####### kb  @<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
$size $clean $dirty $file
.

save this file as SCRIPTNAME.pl . I have saved this script as smaps.pl.

Now assume that, you want to see memory usage statistics for the Firefox process running on the system ( useful to detect firefox memory leak problem .. . Very Happy ) .

Make sure smaps.pl is in your path and Just run

Code:
[bash] # pgrep firefox | xargs smaps.pl 

you get the following output :

Code:
[root@chandan-gateway chandan]# pgrep firefox | xargs ./smaps.pl
VMSIZE:    240304 kb
RSS:        97392 kb total
              6512 kb shared
            21952 kb private clean
            68928 kb private dirty
PRIVATE MAPPINGS
    vmsize  rss clean  rss dirty  file
  54624 kb        0 kb    54384 kb
  10316 kb        0 kb    10316 kb
    376 kb        0 kb      376 kb  /SYSV00000000 (deleted)
    376 kb        0 kb      376 kb  /SYSV00000000 (deleted)
    372 kb        0 kb      372 kb  /usr/lib/firefox-2.0.0.4/components/libgklayout.so
    296 kb        0 kb      296 kb  /SYSV00000000 (deleted)
      24 kb        0 kb      24 kb  /usr/lib/firefox-2.0.0.4/components/libuconv.so
      20 kb        0 kb      20 kb  /usr/lib/firefox-2.0.0.4/libmozjs.so
      16 kb        0 kb      16 kb  /usr/lib/firefox-2.0.0.4/components/libxpconnect.so
      4 kb        0 kb        4 kb  /usr/lib/firefox-2.0.0.4/components/libtxmgr.so
      4 kb        0 kb        4 kb  /lib/libnss_dns-2.6.so
      4 kb        0 kb        4 kb  /lib/libnss_dns-2.6.so
      4 kb        0 kb        4 kb  /usr/lib/firefox-2.0.0.4/components/libnecko2.so
      4 kb        0 kb        4 kb  /usr/lib/firefox-2.0.0.4/components/libremoteservice.so
      60 kb      44 kb        0 kb  /usr/lib/firefox-2.0.0.4/components/libmyspell.so
      16 kb      12 kb        0 kb  /usr/lib/firefox-2.0.0.4/libgtkxtbin.so
      24 kb      16 kb        0 kb  /usr/lib/libpangoxft-1.0.so.0.1600.4
    104 kb      68 kb        0 kb  /usr/lib/firefox-2.0.0.4/components/libchrome.so
    120 kb      80 kb        0 kb  /usr/lib/firefox-2.0.0.4/components/libcaps.so
      80 kb      60 kb        0 kb  /usr/lib/libbonobo-activation.so.4.0.0
      8 kb        8 kb        0 kb  /usr/lib/libavahi-glib.so.1.0.1
    248 kb      168 kb        0 kb  /usr/lib/firefox-2.0.0.4/components/libwidget_gtk2.so
    444 kb      268 kb        0 kb  /usr/lib/firefox-2.0.0.4/components/libtoolkitcomps.so
      16 kb      12 kb        0 kb  /usr/lib/libORBitCosNaming-2.so.0.1.0
      88 kb      56 kb        0 kb  /usr/lib/firefox-2.0.0.4/components/libpref.so
    1100 kb      624 kb        0 kb  /usr/lib/firefox-2.0.0.4/components/libnecko.so
    132 kb      64 kb        0 kb  /usr/lib/firefox-2.0.0.4/components/libappcomps.so
      52 kb      32 kb        0 kb  /usr/lib/firefox-2.0.0.4/components/libcookie.so
      44 kb      32 kb        0 kb  /usr/lib/firefox-2.0.0.4/components/libpipboot.so
    100 kb      48 kb        0 kb  /usr/lib/firefox-2.0.0.4/libjsj.so
    812 kb      124 kb        0 kb  /usr/lib/firefox-2.0.0.4/components/libuconv.so
    424 kb      260 kb        0 kb  /usr/lib/firefox-2.0.0.4/components/libhtmlpars.so
      52 kb      20 kb        0 kb  /usr/lib/libgnome-keyring.so.0.0.1
    960 kb      296 kb        0 kb  /usr/lib/firefox-2.0.0.4/components/libeditor.so
      28 kb      20 kb        0 kb  /usr/lib/firefox-2.0.0.4/components/libtxmgr.so
    112 kb      60 kb        0 kb  /usr/share/fonts/liberation/LiberationSans-Regular.ttf
    144 kb      144 kb        0 kb  /usr/share/fonts/default/Type1/n021003l.pfb
    428 kb      20 kb        0 kb  /usr/share/fonts/dejavu-lgc/DejaVuLGCSans.ttf
    100 kb      100 kb        0 kb  /usr/share/fonts/default/Type1/n019003l.pfb
    408 kb      12 kb        0 kb  /usr/share/fonts/dejavu-lgc/DejaVuLGCSans-Bold.ttf
      60 kb      16 kb        0 kb  /usr/share/fonts/liberation/LiberationMono-Regular.ttf
      72 kb      28 kb        0 kb  /usr/share/fonts/liberation/LiberationSans-Italic.ttf
      68 kb      40 kb        0 kb  /usr/share/fonts/liberation/LiberationSans-Bold.ttf
      28 kb        4 kb        0 kb  /usr/lib/gconv/gconv-modules.cache
. . . . .
. . . . .
. . . . .

SHARED MAPPINGS
    vmsize  rss clean  rss dirty  file
    1336 kb      660 kb        0 kb  /lib/libc-2.6.so
    1016 kb      524 kb        0 kb  /usr/lib/libX11.so.6.2.0
    384 kb        0 kb      384 kb  /SYSV00000000 (deleted)
    896 kb      360 kb        0 kb  /usr/lib/libstdc++.so.6.0.8
    536 kb      340 kb        0 kb  /usr/lib/libfreetype.so.6.3.15
    632 kb      336 kb        0 kb  /lib/libglib-2.0.so.0.1200.11
    248 kb      220 kb        0 kb  /lib/libgobject-2.0.so.0.1200.11
    504 kb      216 kb        0 kb  /usr/lib/libnss3.so
    384 kb        0 kb      192 kb  /SYSV00000000 (deleted)
    312 kb      188 kb        0 kb  /usr/lib/libORBit-2.so.0.1.0
    3676 kb      176 kb        0 kb  /usr/lib/libgtk-x11-2.0.so.0.1000.13
    212 kb      152 kb        0 kb  /usr/lib/libnspr4.so
    328 kb      148 kb        0 kb  /usr/lib/libsoftokn3.so
    156 kb      124 kb        0 kb  /usr/lib/libfontconfig.so.1.2.0
    204 kb      112 kb        0 kb  /usr/lib/libgconf-2.so.4.1.2
    108 kb      100 kb        0 kb  /lib/ld-2.6.so
    148 kb      100 kb        0 kb  /usr/lib/libpng12.so.0.16.0
    868 kb      92 kb        0 kb  /lib/libasound.so.2.0.0
    124 kb      88 kb        0 kb  /lib/libexpat.so.0.5.0
    1136 kb      84 kb        0 kb  /lib/libcrypto.so.0.9.8b
    428 kb      72 kb        0 kb  /usr/share/fonts/dejavu-lgc/DejaVuLGCSans.ttf
    552 kb      68 kb        0 kb  /usr/lib/libgdk-x11-2.0.so.0.1000.13
    572 kb      68 kb        0 kb  /usr/lib/libkrb5.so.3.3
      80 kb      64 kb        0 kb  /lib/libpthread-2.6.so
      64 kb      64 kb        0 kb  /var/cache/fontconfig/d066cc093611c74861a22d5ef60b042c-x86.ca
    232 kb      60 kb        0 kb  /usr/lib/libfreebl3.so
    408 kb      60 kb        0 kb  /usr/share/fonts/dejavu-lgc/DejaVuLGCSans-Bold.ttf
    2048 kb      52 kb        0 kb  /usr/lib/locale/locale-archive
    156 kb      48 kb        0 kb  /lib/libm-2.6.so
      72 kb      44 kb        0 kb  /lib/libz.so.1.2.3
    260 kb      44 kb        0 kb  /lib/libssl.so.0.9.8b
. . . . .
. . . . .
. . . . .



The starting four lines is enough to see how much memory is it actually using, The next details if useful, if you want to pin point which library or usually plugin lib, which is eating memory , and other analysis.

So just to get the Jist of process memory usage , do the following :

Code:
VMSIZE:    247496 kb
RSS:        104740 kb total
              6512 kb shared
            21956 kb private clean
            76272 kb private dirty


here u see that actually, over 90.5 MB ( that is 21956 kb private clean + 76272 kb private dirty ) Is used by firefox , as reserved private region, exclusively. and only over 6 mb is occupied by shared libs, used by many processes along with Firefox.

actually, Though this is a better solution, it is not ideal. See for example, All this shared libraries is used only by this process, or else just by 2 or 3 processes, than this one is not ideal.

Hope this is helpful.

Regards,

-- Chandan

Chandan.Maddanna@gmail.com

If any queries etc, Reply to this Message by CLICKING HERE

.
Back to top Go down
View user profile http://chandan-m.blogspot.com/
Mark
Guest



PostSubject: He thank you very much   Mon Sep 10, 2007 4:53 pm

Just what i was looking for , i am trying to use this script as a part of another memory tracer and reporter script. Will surely let you know my mileage.

Mark
Back to top Go down
chandan



Number of posts: 11
Age: 30
Registration date: 2007-04-13

PostSubject: Hey Mark, Quite Surprizing.   Tue Sep 11, 2007 10:47 am

Wow,

Quite Surprising, because even i am designing one. Also i am thinking of this script monitoring and also sending alerts through paging etc.

I will discuss it with you once i am done.

Regards,

-- Chandan


Mark wrote:
Just what i was looking for , i am trying to use this script as a part of another memory tracer and reporter script. Will surely let you know my mileage.

Mark
Back to top Go down
View user profile http://chandan-m.blogspot.com/
 

Memory usage Retrieval on Linux ( process wise and general )

View previous topic View next topic Back to top 
Page 1 of 1

 Similar topics

-
» Aklabeth 1.0 for linux
» In Memory of...Mimo (Ukere)
» Starfleet Command (Credit To Memory Alpha)
» Wise Care 365 Pro 2.25 Build 181 Incl Serials - SceneDL
» IS it wise to invest money on a game like fallen sworad??

Permissions in this forum:You cannot reply to topics in this forum
 ::  :: -