Skip navigation
7282 Views 3 Replies Latest reply: Aug 9, 2010 10:54 PM by cntrytwist RSS
cntrytwist Rank: White Belt 9 posts since
May 22, 2010
Currently Being Moderated

Jul 31, 2010 6:37 PM

ZenCommand fails with too many open files

I've installed and setup ZenossCore Version 3.0.0.  Overall I like it, but I'm having some trouble that I'm hoping someone can help with. My graphs quite being created.  I'll reboot and things start working as expected.  However, within 24 hours, the graphs will stop working.
There are no errors and no events being reported that indicate things are not working. so, after digging around a bit, I found the command logs and saw them filling up with series of the following errors every five minutes.

2010-07-31 16:14:01,350 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_schedule.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_schedule.rrd
2010-07-31 16:14:01,351 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_commands.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_commands.rrd
2010-07-31 16:14:01,351 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_dataPoints.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_dataPoints.rrd
2010-07-31 16:14:01,352 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_cyclePoints.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_cyclePoints.rrd
2010-07-31 16:15:01,357 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_schedule.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_schedule.rrd
2010-07-31 16:15:01,358 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_commands.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_commands.rrd
2010-07-31 16:15:01,358 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_dataPoints.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_dataPoints.rrd
2010-07-31 16:15:01,359 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_cyclePoints.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_cyclePoints.rrd
2010-07-31 16:16:01,363 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_schedule.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_schedule.rrd
2010-07-31 16:16:01,364 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_commands.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_commands.rrd
2010-07-31 16:16:01,365 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_dataPoints.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_dataPoints.rrd
2010-07-31 16:16:01,365 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_cyclePoints.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_cyclePoints.rrd
2010-07-31 16:17:01,370 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_schedule.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_schedule.rrd
2010-07-31 16:17:01,371 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_commands.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_commands.rrd
2010-07-31 16:17:01,372 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_dataPoints.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_dataPoints.rrd
2010-07-31 16:17:01,372 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_cyclePoints.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_cyclePoints.rrd
2010-07-31 16:18:01,377 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_schedule.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_schedule.rrd
2010-07-31 16:18:01,378 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_commands.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_commands.rrd
2010-07-31 16:18:01,379 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_dataPoints.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_dataPoints.rrd
2010-07-31 16:18:01,379 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_cyclePoints.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_cyclePoints.rrd
2010-07-31 16:18:02,466 ERROR zen.DaemonStats: rrdtool reported error opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_eventQueueLength.rrd': Too many open files /usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_eventQueueLength.rrd
2010-07-31 16:18:02,466 ERROR zen.MinMaxCheck: Unable to read RRD file for zencommand_eventQueueLength
Traceback (most recent call last):
  File "/usr/local/zenoss/zenoss/Products/ZenModel/MinMaxThreshold.py", line 239, in checkRaw
  File "/usr/local/zenoss/zenoss/Products/ZenModel/MinMaxThreshold.py", line 182, in rrdInfoCache
error: opening '/usr/local/zenoss/zenoss/perf/Daemons/localhost/zencommand_eventQueueLength.rrd': Too many open files

 

So, I started digging around and found the following command to see how many files I have open. 

 

ls -1 /proc/`pgrep -f zencommand.py`/fd | wc -l

 

My system shows 1024 files open.

 

Can any one help me out as to how to fix this problem?  I'm running on Ubuntu 10.  Thanks in advance.

More Like This

  • Retrieving data ...

Legend

  • Correct Answers - 4 points
  • Helpful Answers - 2 points