Skip navigation
9260 Views 12 Replies Latest reply: Apr 30, 2012 2:43 AM by didierh7 RSS
Dave Bucci Rank: White Belt 21 posts since
Oct 21, 2011
Currently Being Moderated

Nov 16, 2011 3:34 PM

WMI monitoring, many nan values, missing graphs

Ok, making progress.  I've come through the configuration issues to allow WMI access from the Linux Zenoss VM to the Windows 2008R2 VMs using a non-administrator account, relying on NTLMv2 (as required in my env).

 

Using egor's WMI Data Source and WMI Performance zenpacks, btw.  Core 3.2.1, RPM install into a RHEL 5.5 VM, accessing a 64-bit Windows 2008R2 SP2 VM.  Server is in /CIM/WMI device tree, and appears to model properly.  I have SNMP monitoring turned off (at the config level) for that portion of the device tree.

 

But I have the following problems with the single server I'm trying to monitor (a test case, to rough out all proper config settings, etc.):

 

  1. For the device-specific graphs, CPU utilization always shows 0%; Memory Utilization and Paging, all values show nan
  2. For Components | File Systems, the Utilization shows correct values (and a related threshold is being triggered, as i do in fact have < 10% of C:\ space available), but Queue Length and I/O Requests graphs, all values show nan (I switches the modeler plugin to community.wmi.VolumeFileSystemMap from the Core one, btw -- though the values all also looked correct with the core one [which makes sense, given that it's a simple disk, not a large volume or anything])
  3. Under Components | Interfaces, the interfaces did model correctly, but all graph values show nan
  4. Under Components | Windows Services, the services list populates with correct info, but all graph values for all services show nan
  5. Under Software, installed software list is blank

 

Where I can figure out how to, I've tested, and it's not a basic access issue ... e.g., in Advanced | Monitoring Templates, with Device | /CIM/WMI selected, selecting the sysUpTime data point, and from the gear widget, selecting View and Edit Details, then entering my device and doing "TEST", I get what looks fine to me as a result (i fuxxed the IP address, but it is correct):

 

Get WMI Instance SELECT LastBootUpTime FROM Win32_OperatingSystem from //x.x.x.x/root/cimv2

InstanceName: root/cimv2:Win32_OperatingSystem. LastBootUpTime = 2011/11/14 10:56:53.296875 GMT-5

 

(Sorry for being so explicit about the steps I'm taking, but on these forums, i've noticed it's hard for a n00b, because people say "test the datasource" and never explain how to do that -- trying to help those a 1/2 step behind me :-)

 

Doing the same test on the Processor data point, though, I get:

 

Get WMI Instance SELECT LoadPercentage FROM Win32_Processor from //10.13.0.93/root/cimv2

InstanceName: root/cimv2:Win32_Processor.DeviceID="CPU0" InstanceName: root/cimv2:Win32_Processor.DeviceID="CPU1"

 

There's mention in some of egor's blog postings of this, how if there are multiple CPUs, you'll get a list back - and maybe his pending newer perf pack (which relies on the SQLDataSource under the covers?) will combine multiple queries or something ...?  But I couldn't make heads nor tails of it, if there was anything in there that would allow a fix for the current packs.

 

Then, for the PerfOSMemory data source, when I test it, I get literally nothing back -- just the SELECT line, and no 2nd result line. Running the same query via wmic -d 99, everything looks fine until the last line of debug output, which reads:

 

[librpc/ndr/ndr.c:200:ndr_print_debug_helper()] result                         :  WERR_BADFUNC

 

(even though the final line reads "[wmi/wmic.c:212:main()] OK").

 

Running "zenperfwmi -v10 -d device" instead, I see reasonable output, including (i've fuzzed IP and uid/pw, but values were all correct):

 

2011-11-16 14:13:50,445 DEBUG zen.zenperfwmi: Polling for WMI data from x.x.x.x [x.x.x.x]

2011-11-16 14:13:50,445 DEBUG zen.collector.scheduler: Task x.x.x.x changing state from RUNNING to WMIC_QUERY

2011-11-16 14:13:50,445 DEBUG zen.WMIClient: connect to x.x.x.x, user 'xx/yy'

2011-11-16 14:13:50,662 DEBUG zen.pysamba: OK: x.x.x.x - Connect

2011-11-16 14:13:50,663 DEBUG zen.WMIClient: Query: SELECT NetConnectionStatus,DeviceID FROM Win32_NetworkAdapter

2011-11-16 14:13:50,666 DEBUG zen.pysamba: OK: x.x.x.x - ExecQuery

2011-11-16 14:13:50,670 DEBUG zen.pysamba: OK: x.x.x.x - Reset result of WMI query.

2011-11-16 14:13:50,809 DEBUG zen.pysamba: OK: x.x.x.x - Retrieve result data.

2011-11-16 14:13:50,812 DEBUG zen.pysamba: OK: x.x.x.x - Retrieve result data.

2011-11-16 14:13:50,814 DEBUG zen.pysamba: OK: x.x.x.x - Retrieve result data.

2011-11-16 14:13:50,814 DEBUG zen.WMIClient: Query: SELECT LastBootUpTime FROM Win32_OperatingSystem

2011-11-16 14:13:50,816 DEBUG zen.pysamba: OK: x.x.x.x - ExecQuery

2011-11-16 14:13:50,818 DEBUG zen.pysamba: OK: x.x.x.x - Reset result of WMI query.

2011-11-16 14:13:50,825 DEBUG zen.pysamba: OK: x.x.x.x - Retrieve result data.

2011-11-16 14:13:50,827 DEBUG zen.pysamba: OK: x.x.x.x - Retrieve result data.

2011-11-16 14:13:50,828 DEBUG zen.WMIClient: Query: SELECT Name,CurrentDiskQueueLength,DiskWriteBytesPersec,DiskReadsPersec,DiskReadBytesPersec,DiskWritesPersec FROM Win32_PerfRawData_PerfDisk_LogicalDisk WHERE Name="C:"

2011-11-16 14:13:50,832 DEBUG zen.pysamba: OK: x.x.x.x - ExecQuery

2011-11-16 14:13:50,834 DEBUG zen.pysamba: OK: x.x.x.x - Reset result of WMI query.

2011-11-16 14:13:50,843 DEBUG zen.pysamba: OK: x.x.x.x - Retrieve result data.

2011-11-16 14:13:50,843 DEBUG zen.WMIClient: Query: SELECT PacketsOutboundErrors,Name,PacketsReceivedErrors,PacketsSentPersec,PacketsReceivedPersec,BytesSentPersec,PacketsReceivedUnicastPersec,BytesReceivedPersec,PacketsSentUnicastPersec FROM Win32_PerfRawData_Tcpip_NetworkInterface

2011-11-16 14:13:50,846 DEBUG zen.pysamba: OK: x.x.x.x - ExecQuery

2011-11-16 14:13:50,848 DEBUG zen.pysamba: OK: x.x.x.x - Reset result of WMI query.

2011-11-16 14:13:50,856 DEBUG zen.pysamba: OK: x.x.x.x - Retrieve result data.

2011-11-16 14:13:50,856 DEBUG zen.WMIClient: Query: SELECT AvailableKBytes, CacheBytes, PagesPerSec FROM Win32_PerfRawData_PerfOS_Memory

2011-11-16 14:13:50,859 DEBUG zen.pysamba: OK: x.x.x.x - ExecQuery

2011-11-16 14:13:50,861 DEBUG zen.pysamba: OK: x.x.x.x - Reset result of WMI query.

2011-11-16 14:13:50,877 DEBUG zen.pysamba: OK: x.x.x.x - Retrieve result data.

 

After all the queries, I see:

 

2011-11-16 14:13:50,965 DEBUG zen.collector.scheduler: Task x.x.x.x changing state from WMIC_QUERY to WMIC_PROCESS

2011-11-16 14:13:50,965 DEBUG zen.zenperfwmi: Successful collection from x.x.x.x [x.x.x.x], results={

 

followed by an output of a table, with results - showing e.g.:

 

'x.x.x.x_Device_sysUpTime': [{'sysUpTime': DateTime('2011/11/14 10:56:53.296875 GMT-5')}], 'C_FileSystem_usedBlocks': [{'usedBlocks': 3241193472}], 'Intel_R_ PRO_1000 MT Network Connection_ethernetCsmacd_PerfRawData': [], 'x.x.x.x_Device_Processor': [{'LoadPercentage_count': 0, 'LoadPercentage_avg': 0}, {'LoadPercentage_count': 0, 'LoadPercentage_avg': 0}], 'x.x.x.x_Device_PerfOSMemory': [], 'Intel_R_ PRO_1000 MT Network Connection _2_ethernetCsmacd_PerfRawData': []

 

When I run those same queries from WBEMTEST, I get basically the same results - an object returned for uptime, no object returned for the PerfOSMemory query.  Turning on WMI event tracing on the Windows server, nothing in particular shows up - i see the queries coming in,

 

I've done what things I could find on the Windows box, e.g. run wmiadap /F, winmgmt /resyncperf, restarted WMI service, ran mofcomp and regsvr32 /s on all the .mof and .dll files in %windir%\system32\wbem.

 

So ... any thoughts?  Something I'm doing wrong?

 

thx!

  • Shane Scott ZenossMaster 1,373 posts since
    Jul 6, 2009
    Currently Being Moderated
    1. Nov 17, 2011 3:55 AM (in response to Dave Bucci)
    Re: WMI monitoring, many nan values, missing graphs

    Dave Bucci:

     

    I have these same results. I can't figure why its happening other than the templates just don't work against Windows 2008.

     

    --Shane (Hackman238)

  • Shane Scott ZenossMaster 1,373 posts since
    Jul 6, 2009
    Currently Being Moderated
    4. Nov 22, 2011 1:30 PM (in response to Dave Bucci)
    Re: WMI monitoring, many nan values, missing graphs

    Dave Bucci:

     

    This is an awesome find. Thank you for your hard work to figure this solution.

     

    Best,
    --Shane (Hackman238)

  • nabil naim Newbie 4 posts since
    Mar 14, 2012
    Currently Being Moderated
    5. Mar 29, 2012 11:06 AM (in response to Dave Bucci)
    Re: WMI monitoring, many nan values, missing graphs

    i have the same problem, although i use domain admin account

    it's WIN 2003 server 32 bit, it's under /CIM/WMI template zenoss able to list all interfaces with its ip and name and list all services and all cpu but non of its graphs give any value only nan, when i check zenperfwmi.log find below error repeated frequently

    2012-03-29 14:57:45,772 ERROR zen.zenperfwmi: Device xxx.xxx.xxx.xxx: Received NT code 0x80041017 from query: SELECT NetConnectionStatus FROM 0

     

    to check it's not configuration issue i add WMI dataSource using advanced -> devices -> CIM -> WMI -> Device (locally defined)

    Data Source

    select * from Win32_PerfRawData_Tcpip_NetworkInterface where Name="Intel[R] PRO_1000 MT Network Connection"

    where Intel[R] PRO_1000 MT Network Connection is server interface name

    and add data points

    BytesReceivedPersec

    BytesSentPersec

     

    and add it to new graph and it read successfully and works fine, so i don't know why default graphes not working ? and where i can find any graph wmi query to modify it ?

  • Shane Scott ZenossMaster 1,373 posts since
    Jul 6, 2009
    Currently Being Moderated
    6. Mar 29, 2012 11:22 AM (in response to nabil naim)
    Re: WMI monitoring, many nan values, missing graphs

    nabil:

     

    NT code 0x80041017 is a denied message. Make sure that the user has access to that item in WMI and that the item is exposed in WMI.

     

    --Shane

  • nabil naim Newbie 4 posts since
    Mar 14, 2012
    Currently Being Moderated
    7. Apr 1, 2012 6:42 AM (in response to Dave Bucci)
    Re: WMI monitoring, many nan values, missing graphs

    i check the WMI configuration, and i mention i create my own WMI datasource for same device and it works fine

    when i check advanced -> Monitor template -> devices -> CIM -> WMI -> ethernetCsmacd (define locally)

    i find the below data source "ifOperStatus"

    Name: ifOperStatus

    Name Space: root/cimv2

    Query: ${here/snmpindex}

     

    Data Point

    Name: ifOperStatus

    ID/Formula: NetConnectionStatus

     

    when i try to test this againest my server it gives the same error

    ERROR:zen.WMIClient:Received NT code 0x80041017 from query: SELECT * FROM 0 

    Received NT code 0x80041017 from query: SELECT * FROM 0 

     

    - is it normal to use snmp variable with WMI query  ?

    - how could i force default interface graph to use WMI query instead of SNMP ? or how can i know/change default graph queries appears in in Component menu ?

  • Craig Lunsford Newbie 3 posts since
    Apr 13, 2012
    Currently Being Moderated
    8. Apr 13, 2012 2:11 PM (in response to nabil naim)
    Re: WMI monitoring, many nan values, missing graphs

    Did you ever resolve this.

    I'm having the exact same problem with NetConnectionStatus on one server I am monitoring.

     

    I have deleted the server, added it back and remodeled it.

    What's strange is the interface device is no longer present in Zenoss, but Zenoss continues to generate this error for a network device.

     

    message:

    Could not get WMI Instance (Received NT code 0x80041017 from query: SELECT NetConnectionStatus FROM 0)

     

    HP NC326i PCIe Dual Port Gigabit Server Adapter|/Status/Wbem||4|Could not get WMI Instance

     

    Component:HP NC326i PCIe Dual Port Gigabit Server Adapter

     

    If I look under my interface components, I only see: HP Network Team_1, and it is returning no graphs.

     

    Any help would greatly be appreciated

  • Shane Scott ZenossMaster 1,373 posts since
    Jul 6, 2009

    Craig:

     

    I would try to restart the WMI polling daemon and see if the event stop.

     

    Best,

    --Shane

  • Craig Lunsford Newbie 3 posts since
    Apr 13, 2012

    I checked under perf/devices under the Zenoss file directory and noticed I had this device listed by both the IP address and the name.  So I deleted both, and from the Zenoss console.  Then added it back, and so far so good.

    I really need to do some in depth studying of Zenoss as to how it all works, sometimes I see some weird stuff that I just don't understand how that happened.

     

    Restarting the WMI daemon might have worked as well, I do this when Zenoss starts to run really slow to resolve the wmi memory leak.

  • Shane Scott ZenossMaster 1,373 posts since
    Jul 6, 2009

    Craig:

     

    Sounds all too much like ZenOSS.

  • didierh7 Newbie 1 posts since
    Jan 8, 2012
    Currently Being Moderated
    12. Apr 30, 2012 2:43 AM (in response to nabil naim)
    Re: WMI monitoring, many nan values, missing graphs

    Hi all, I'm using the same scenario and had same problem described by Nabil, after follow his instructions, all my graphs magically started to work (network, filesystems, memory cpu etc.).

     

    I'm using Zenoss Version 3.2.1

    Thanks Nabil

More Like This

  • Retrieving data ...

Legend

  • Correct Answers - 4 points
  • Helpful Answers - 2 points