Skip navigation
6235 Views 11 Replies Latest reply: Apr 29, 2011 7:51 AM by jcurry RSS
jcurry ZenossMaster 1,021 posts since
Apr 15, 2008
Currently Being Moderated

Apr 27, 2011 11:34 AM

Timestamps on rrd files stopped updating in 3.1

This area has done the round several times before, see

 

http://community.zenoss.org/thread/9438

 

for one.

 

I have Zenoss 3.1 on OpenSuse 10.3.  My graphs are fine.  The data inside the rrd files is updating fine.  But the modified timestamps on the rrd files are not changing.  I updated from 3.0.3 to 3.1 on March 11th.  Many of my rrd file timestamps are March 28th or 29th - but not all.  Newly created rrd files have the timestamp of creation but that timestamp should change every 5 minutes as the data gets updated (and it DOES get updated).

 

I am concerned that the file cleanup code in zenperfsnmp that deletes "old" rrd files, may delete my data files after 30 days.  I know this issue used to appear on some Linux distros in the past and some mods were done to zenperfsnmp but not quite sure we are at with the code at present????

 

This Virtual Machine got forked long back from a Zenoss 2.5.2 where rrd file timestamps ARE still being updated as data is entered - the Ops Sys is identical; it's just the Zenoss that changed.

 

Anyone else seeing this?

 

Cheers,

Jane

  • nilie Rank: Green Belt 372 posts since
    May 27, 2010
    Currently Being Moderated
    1. Apr 27, 2011 1:42 PM (in response to jcurry)
    Re: Timestamps on rrd files stopped updating in 3.1

    Jane,

     

    On my Zenoss v3.1.0 the timestamps are getting updated every 5 minutes as expected. I'm running it under CentOS v5.6 64bit on a physical server.

     

    Are you by chance using RRD caching ? Did you noticed if there are any other files affected besindes RRD files ?

  • nilie Rank: Green Belt 372 posts since
    May 27, 2010
    Currently Being Moderated
    3. Apr 27, 2011 1:57 PM (in response to jcurry)
    Re: Timestamps on rrd files stopped updating in 3.1

    You should look for a daemon called rrdcached running ( ps -A | grep rrdcached ). It does not seem to come with Zenoss by default so if you or someone else did not install it on purpose I don't think you have it on your system.

  • jhelgesen Rank: White Belt 64 posts since
    Dec 10, 2007
    Currently Being Moderated
    5. Apr 27, 2011 4:47 PM (in response to jcurry)
    Re: Timestamps on rrd files stopped updating in 3.1

    Hi Jane.

     

    If the timestamps are not updated, but the file _is_ updated, I guess I'd suspect the filesystem.  If you "stat" one of the files (stat sysUpTime_sysUpTime.rrd), it ought to list out the access, change, and modify times.  When I test this (2.5.2 on opensuse11.2 and 3.1.0 on opensuse11.4) all three times are the same.  Do you see different times on your system?

     

    --Jay

  • jskeane Rank: Green Belt 67 posts since
    Sep 15, 2010
    Currently Being Moderated
    7. Apr 27, 2011 7:16 PM (in response to jcurry)
    Re: Timestamps on rrd files stopped updating in 3.1

    Jane,

     

    I wonder if the timestamps are "stuck" because some process is holding the files open. What does lsof think of your file? How does that change with Zenoss stopped?

     

    Other potentially edifying experiments:

     

    Stop Zenoss, then start it up again. Do the timestamps update to the startup time?

     

    Stop Zenoss. Rename one of the files, appending _. Rename the file again, removing the _.  This will give it a new inode. Start Zenoss, and look at the timestamps.

     

    I hope this helps.

     

    Regards,

     

    jskeane

  • jhelgesen Rank: White Belt 64 posts since
    Dec 10, 2007
    Currently Being Moderated
    8. Apr 28, 2011 10:05 AM (in response to jcurry)
    Re: Timestamps on rrd files stopped updating in 3.1

    hmm...  certainly possible that a file handle someplace didn't get closed, hanging up the 'modified' date.

     

    What happens if the zenoss user manually modifies an RRD file?

     

    zenoss@zenoss:....../os/filesystems/C__ Label_  Serial Number 5865c6ad> ls -la
    total 44
    drwxr-x--- 2 zenoss zenoss  4096 2011-04-28 09:59 .
    drwxr-x--- 5 zenoss zenoss  4096 2009-12-29 15:31 ..
    -rw-r--r-- 1 zenoss zenoss 35432 2011-04-28 09:57 usedBlocks_usedBlocks.rrd
    zenoss@zenoss:....../os/filesystems/C__ Label_  Serial Number 5865c6ad> rrdtool lastupdate usedBlocks_usedBlocks.rrd
    ds0

     

    1303999057: 2266756.0
    zenoss@zenoss:....../os/filesystems/C__ Label_  Serial Number 5865c6ad> rrdtool update usedBlocks_usedBlocks.rrd N:2266757.0
    zenoss@zenoss:....../os/filesystems/C__ Label_  Serial Number 5865c6ad> rrdtool lastupdate usedBlocks_usedBlocks.rrd
    ds0

     

    1303999247: 2266757.0
    zenoss@zenoss:....../os/filesystems/C__ Label_  Serial Number 5865c6ad> ls -la
    total 44
    drwxr-x--- 2 zenoss zenoss  4096 2011-04-28 09:59 .
    drwxr-x--- 5 zenoss zenoss  4096 2009-12-29 15:31 ..
    -rw-r--r-- 1 zenoss zenoss 35432 2011-04-28 10:00 usedBlocks_usedBlocks.rrd

     

    --Jay

  • jhelgesen Rank: White Belt 64 posts since
    Dec 10, 2007
    Currently Being Moderated
    11. Apr 29, 2011 1:12 PM (in response to jcurry)
    Re: Timestamps on rrd files stopped updating in 3.1

    Very good, when the manual update still failed you knew it wasn't a python problem.  It seems unlikely the APIs or features of RRDtool that Zenoss uses would change on a minor point upgrade... I'd be tempted to backrev the thing and see if the problem's corrected.

     

    My copy of Zenoss2.5.2 says it's got RRDtool 1.3.8, you should be able to pull that from your 2.5.2 copy over to the zenoss user's home directory (or tmp, someplace not in the search path), and try the manual update again with the older rrdtool.  If it works, backup rrdtool139 and drop in the rrdtool138?

     

    --Jay

More Like This

  • Retrieving data ...

Legend

  • Correct Answers - 4 points
  • Helpful Answers - 2 points