Skip navigation
1 2 3 Previous Next 60032 Views 31 Replies Latest reply: Nov 6, 2013 11:59 AM by John Komara RSS Go to original post
  • ebogaard Rank: White Belt 41 posts since
    Jun 9, 2011
    Currently Being Moderated
    15. Mar 28, 2012 5:10 AM (in response to linuxdynasty)
    Re: Linux memory thresholds

    Thanks for your script. Will try it out!

  • ebogaard Rank: White Belt 41 posts since
    Jun 9, 2011
    Currently Being Moderated
    16. Mar 28, 2012 5:42 PM (in response to ebogaard)
    Re: Linux memory thresholds

    Tried the script, but this isn't really what I'm searching for. It still doesn't subtract the cache and buffers in the memory from the total used memory (or adds both tot the free memory).

    And it seems to be working quite slowly and take up quite some memory, so probably isn't for a production environment with multiple monitoring subjects.

     

    Thanks anyway!

    James, as you have a script that derives the actual usage, could you make it public?

  • Shane Scott ZenossMaster 1,373 posts since
    Jul 6, 2009
    Currently Being Moderated
    17. Mar 29, 2012 1:33 AM (in response to ebogaard)
    Re: Linux memory thresholds

    ebogaard:

     

    Once you find a script that produces the result you want I highly reccomend you dump the value in to a custom net-snmp oid and poll the oid using zenperfsnmp in zenoss. Command datasources don't scale well in some cases.

     

    Best,

    --Shane

  • ebogaard Rank: White Belt 41 posts since
    Jun 9, 2011
    Currently Being Moderated
    18. Mar 30, 2012 5:11 PM (in response to rhester)
    Re: Linux memory thresholds

    Found some time today, and in hindsight, it wasn't all that difficult to gat the 'real' free memory. I did the following:

    1. Made a script 'memoryFreee.sh' that uses 'free' to get the real free memory (in KB):

    /usr/bin/free -k | awk '/cache:/ {print $2,$3,$4}' | sed 'N;s/\n/ /' | awk '{print ($3)}'

     

    2. Extended snmp with this script. In snmpd.conf:

    extend memoryFree /bin/sh /opt/scripts/memoryFree.sh

     

    3. Added the resulting OID. to zenoss and set an threashold:

    1.3.6.1.4.1.8072.1.3.2.4.1.2.10.109.101.109.111.114.121.70.114.101.101.1

  • Shane Scott ZenossMaster 1,373 posts since
    Jul 6, 2009
    Currently Being Moderated
    19. Mar 30, 2012 6:11 PM (in response to ebogaard)
    Re: Linux memory thresholds

    ebogarrd:

     

    Excellent!

     

    Best,

    --Shane

  • Michael Kemsley Newbie 1 posts since
    Jul 12, 2012
    Currently Being Moderated
    20. Jul 23, 2012 10:16 AM (in response to rhester)
    Re: Linux memory thresholds

    We setup a threshold for memory and swap that seems to be working well

     

    - Under a Device for a linux server add a threshold

    - Set name to 'available memory'

    - Add 'memAvailReal_memAvailReal' as a DataPoint

    - For 'Minimum Value' add: (alert when less then 15% available)

    (here.hw.totalMemory/1024 * 0.15) - ((here.getRRDValue('memBuffer') or 9e9) + (here.getRRDValue('memCached') or 9e9))

    - For Event Class change to '/Perf/Memory'
    - Click Save

     

    For checking swap:

    - Set name to 'available swap'
    - Add 'memAvailSwap_memAvailSwap'

    - For 'Minimum Value' add: (alert when less then 30% available)

    here.os.totalSwap/1024 * 0.3

    - For Event Class change to '/Perf/Memory'

    - Click Save

     

     

    To make the events more readable you can also create a transform for /Perf/Memory:

    if getattr(evt, 'severity', 0) > 0 \

    and evt.message.find('threshold of available memory not met') == 0:

         from Products.ZenUtils.Utils import convToUnits

     

         memAvail = (device.getRRDValue('memAvailReal') + device.getRRDValue('memBuffer') + device.getRRDValue('memCached'))*1024

         memAvailPercent = 100 - (memAvail / device.hw.totalMemory * 100)

         memAvailUnits = convToUnits(memAvail)

         memTotalUnits = convToUnits(device.hw.totalMemory)

     

         evt.summary = 'low memory : %3.1f%% used (%s of %s free)' % (memAvailPercent, memAvailUnits,memTotalUnits)

         evt.message = evt.summary

     

     

    if getattr(evt, 'severity', 0) > 0 \

    and evt.message.find('threshold of available swap not met') == 0:

         from Products.ZenUtils.Utils import convToUnits

     

         swapAvail = device.getRRDValue('memAvailSwap')*1024

         swapAvailPercent = 100 - (swapAvail / device.os.totalSwap * 100)

         swapAvailUnits = convToUnits(swapAvail)

         swapTotalUnits = convToUnits(device.os.totalSwap)

     

         evt.summary = 'swap usage: %3.1f%% used (%s of %s free)' % (swapAvailPercent, swapAvailUnits,swapTotalUnits)

         evt.message = evt.summary

  • felipe.castelar Rank: White Belt 14 posts since
    Jun 14, 2011
    Currently Being Moderated
    21. Oct 25, 2012 3:05 PM (in response to Michael Kemsley)
    Re: Linux memory thresholds

    I've implemented the Memory Threshold quite the same way as you did. Here I called it "Mem Utilization".

     

    It's working pretty cool here ( >90% -> Event). I'm getting the message "threshold of Mem Utilization not met: current value:xxxx" associated with the /Perf/Memory event class on my event console.

     

    Now I having difficulties transforming the /Perf/Memory event. I've created the defaultmapping on the /Perf/Memory event class and used the Rule evt.summary.find('threshold of Mem Utilization not met')>-1 and then the Transform evt.summary="my test text".

     

    How have you implemented the transform you've just described above? Have you  created a mapping? Which Rule or Event Class Key have you used?

     

    Thanks in advance,

     

    Felipe Castelar

  • Chris Smith Rank: White Belt 23 posts since
    Jun 4, 2013
    Currently Being Moderated
    22. Jul 16, 2013 6:05 PM (in response to felipe.castelar)
    Re: Linux memory thresholds

    Hoping to get some help on the tips above as well. Thanks in advance

     

    here.hw.totalMemory/1024 * 0.15 seems to work but I know it's not accurate.

     

    (here.hw.totalMemory/1024 * 0.95) - ((here.getRRDValue('memBuffer') or 9e9) + (here.getRRDValue('memCached') or 9e9)) does not work for me. My Solaris box is using 22% memory and I set it to 0.95 as shown above...it never triggered.

     

     

    The transform didn't seem to work at all. I placed he transform under Events > Perf > Memory > Transform and saved it. I'm still getting alarms like "threshold of High Memory Utilization not met: current value 1629304.000000"

  • Shane Scott ZenossMaster 1,373 posts since
    Jul 6, 2009
    Currently Being Moderated
    23. Jul 17, 2013 3:39 AM (in response to Chris Smith)
    Re: Linux memory thresholds

    Chris,

     

    I would highly reccomend avoiding using here.getRRDValue, it really won't scale well.

     

    Figuring the proper free memory can be best done using https://github.com/zenoss/ZenPacks.zenoss.CalculatedPerformance

     

    My recently released Improved Linux Monitor ZenPack has a good example of how to do this for calculating free memory. https://github.com/Hackman238/ZenPacks.Rackspace.LinuxMonitor

     

    In the image below (Memory graph from my Improved Linux Monitor) a new datapoint representing (((Free + Cached) - Buffers) * 1024) has been created and is represented by green. A simple Min/Max threshold can be set against that datapoint using ((here.hw.totalMemory/1024) * 0.95) or similar. In my ZenPack I've added a zProperty where one can dictate a percentage of free memory, below which a threshold is event is triggered. That threshold is represented by ((here.hw.totalMemory) * here.getFreeMemoryThresholdPercentLimit()) Helps simplify management.

    RS-LinuxMonitor-MemoryGraph.png

     

    This wil not only scale better, but will also simplify reporting on historical freeMemory (less calculations).

     

    Best,

    --Shane Scott (Hackman238)

    http://shanewilliamscott.com

    http://linkedin.com/in/shanewilliamscott

  • Chris Smith Rank: White Belt 23 posts since
    Jun 4, 2013
    Currently Being Moderated
    24. Jul 17, 2013 4:10 PM (in response to Shane Scott)
    Re: Linux memory thresholds

    You rock Shane. I will do my best to get this implemented and get back with questions. Do you have your alarms transformed to percentage or just using the raw value?

  • Shane Scott ZenossMaster 1,373 posts since
    Jul 6, 2009
    Currently Being Moderated
    25. Jul 19, 2013 1:47 PM (in response to Chris Smith)
    Re: Linux memory thresholds

    Chris,

     

    Thanks. I didn't include transforms since many people have their own transforms and it would complicate the install. I can offer you what we use if you're interested.

     

    Best,

    --Shane Scott (Hackman238)

    http://shanewilliamscott.com

    http://linkedin.com/in/shanewilliamscott

  • Chris Smith Rank: White Belt 23 posts since
    Jun 4, 2013
    Currently Being Moderated
    26. Jul 19, 2013 3:15 PM (in response to Shane Scott)
    Re: Linux memory thresholds

    I would like to see them for reference. I have the general idea down and I'm able to decipher some python but most of the time I cannot get most transforms I find on the forums to function correctly.

  • barney34 Rank: White Belt 23 posts since
    Mar 4, 2009
    Currently Being Moderated
    27. Sep 4, 2013 11:26 AM (in response to Shane Scott)
    Re: Linux memory thresholds

    Scott,

     

    Where is the EGG files I can not see them on the Wiki or github.   Thank you kindly sir.

  • ynodelman Newbie 1 posts since
    Oct 25, 2012
    Currently Being Moderated
    29. Sep 26, 2013 2:07 PM (in response to Shane Scott)
    Re: Linux memory thresholds

    Hi Shane,

     

    Would you advise me on where I can download realFreeMemory.pl script? Furthermore, is there a special setup for monitoring individual CPU Cores?

    -----

    Preparing Command...

    Executing command $ZENHOME/libexec/realFreeMemory.pl [args omitted] against servername

    /bin/sh: /opt/zenoss/libexec/realFreeMemory.pl: No such file or directory

    DONE in 0 seconds

    --------

    Thank you.

More Like This

  • Retrieving data ...