Skip navigation
7444 Views 11 Replies Latest reply: Oct 28, 2008 6:31 PM by kkearney RSS
Currently Being Moderated

May 10, 2008 1:24 PM

event class mapping

I am trying to map syslog events to an event class. No matter how I set up the
regex it does not catch the events and classify them as it should. When I go to
the event browser and try to select a checkbox and map to event class, it returns
with "1 event does not have an event class key. Created 0 event mappings." Is
there something I am missing? It seems that all syslog events from all hosts
have no event class key. Is this the problem?

--
Shane
Independant Computer Consultant
corbin.sl@gmail.com



_______________________________________________
zenoss-users mailing list
zenoss-users@zenoss.org
http://lists.zenoss.org/mailman/listinfo/zenoss-users
  • Chet Luther ZenossEmployee 1,302 posts since
    May 22, 2007
    Currently Being Moderated
    1. May 10, 2008 6:17 PM (in response to Guest )
    event class mapping
    On May 10, 2008, at 1:24 PM, Shane Corbin wrote:

     

     

    I am trying to map syslog events to an event class. No matter how I
    set up the
    regex it does not catch the events and classify them as it should.
    When I go to
    the event browser and try to select a checkbox and map to event
    class, it returns
    with "1 event does not have an event class key. Created 0 event
    mappings." Is
    there something I am missing? It seems that all syslog events from
    all hosts
    have no event class key. Is this the problem?


    Yes, the missing eventClassKey is the problem. Zenoss comes with some
    generic parsers that extract an eventClassKey for most common types of
    devices, but it would appear that yours is not being parsed.

    Could you look in your $ZENHOME/log/zensyslog.log for any lines that
    include "parseTag failed" ? This will show us logs that zensyslog
    wasn't able to parse anything from, and should match the events you're
    seeing with no eventClassKey. If they don't include sensitive data,
    please post some examples to the list along with the type of device
    that is sending them. We should be able to build a proper parser and
    include it in future releases of Zenoss.

    _______________________________________________
    zenoss-users mailing list
    zenoss-users@zenoss.org
    http://lists.zenoss.org/mailman/listinfo/zenoss-users
  • artifact Rank: Green Belt 303 posts since
    Mar 11, 2008
    Currently Being Moderated
    2. May 11, 2008 9:01 PM (in response to Chet Luther)
    RE: event class mapping
    This is also a big annoyance for me with Zenoss.

    My zensyslog.log is chock full of the aforementioned errors. Mostly (but not exclusively) these are events coming from switches, interfaces going up and down, failed authentication attempts, DNS Client timeouts, all kinds of stuff.. here's a common example:

    2008-05-11 17:39:13 WARNING zen.Syslog: parseTag failed:'[devicename] ETHERNET_INTERFACE:eth 0/14 link down'
  • Jean-Francois Maltais Rank: White Belt 49 posts since
    May 8, 2007
    Currently Being Moderated
    3. Jun 20, 2008 3:50 PM (in response to artifact)
    RE: event class mapping
    I'll add a few to the list... Basically, entries that span 2 syslog lines:

    
    zensyslog.log:
    2008-06-20 08:54:36 WARNING zen.Syslog: parseTag failed:'offlining lun=0 (trace=0), target=9d (trace=2800004)'
    
    syslog message:
    Jun 20 08:54:29 antares scsi: [ID 243001 kern.info] /ssm@0,0/pci@1d,700000/SUNW,qlc@1,1/fp@0,0 (fcp10):
    Jun 20 08:54:29 antares         offlining lun=0 (trace=0), target=9d (trace=2800004)
    
    zensyslog.log:
    2008-06-18 08:57:48 WARNING zen.Syslog: parseTag failed:'offline'
    
    syslog message:
    Jun 18 08:57:47 sirius scsi: [ID 107833 kern.warning] WARNING: /pci@8,700000/scsi@2/sd@3,0 (sd33):
    Jun 18 08:57:47 sirius  offline
    
  • rlund Rank: White Belt 64 posts since
    Feb 19, 2008
    Currently Being Moderated
    4. Jul 22, 2008 1:42 PM (in response to Jean-Francois Maltais)
    RE: event class mapping
    update, this works on version SVN Zenoss 2.2.3 r9713

    Must be a bug in Version 2.2.0
  • artifact Rank: Green Belt 303 posts since
    Mar 11, 2008
    Currently Being Moderated
    5. Oct 8, 2008 10:18 PM (in response to rlund)
    RE: event class mapping
    This still does not work for me in 2.2.4.

    There are tons of parseTag Failed messages in the log...
  • agent00111 Rank: White Belt 9 posts since
    Sep 23, 2008
    Currently Being Moderated
    6. Oct 21, 2008 3:50 PM (in response to artifact)
    RE: event class mapping
    I had a similar problem. After reading a few other threads, here is what I ended up doing:

    1. First I enabled debug messages and raw logging for zensyslog:

    Management -> Settings -> Daemons (Edit Config for zensyslog)

    I used these parameters. The first logs the original syslog message in its entirety (pre-processed), and the second I believe enables debugging messages.
    2008-10-21 12:32:11 DEBUG zen.Syslog: tag regex: ^(?P<component>.+)\[(?P<ntseverity>\D+)\] (?P<ntevid>\d+) (?P<summary>.*)
    2008-10-21 12:32:11 DEBUG zen.Syslog: tag regex: %CARD-\S+:(SLOT\d+) %(?P<eventClassKey>\S+): (?P<summary>.*)
    2008-10-21 12:32:11 DEBUG zen.Syslog: tag regex: %(?P<eventClassKey>(?P<component>\S+)-\d-\S+): (?P<summary>.*)
    2008-10-21 12:32:11 DEBUG zen.Syslog: tag regex: ^(?P<ipAddress>\S+)\s+(?P<summary>(?P<eventClassKey>CisACS_\d\d_\S+)\s+(?P<eventKey>\S+)\s.*)
    2008-10-21 12:32:11 DEBUG zen.Syslog: tag regex: device_id=\S+\s+\[\S+\](?P<eventClassKey>\S+\d+):\s+(?P<summary>.*)\s+\((?P<originalTime>\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d)\)
    2008-10-21 12:32:11 DEBUG zen.Syslog: tag regex: (?P<component>\S+)\[(?P<pid>\d+)\]:\s*(?P<summary>.*)
    2008-10-21 12:32:11 DEBUG zen.Syslog: tag regex: (?P<component>\S+): (?P<summary>.*)

    I saved these and restarted zensyslog, then I could 'view log' for zensyslog and get more detail.

    2. From the logs I could then see the regex statements that were being used:

    
    Parameter Type Violation on server.domain.com from xxx.xxx.xxx.xxx


    3. I compared those statements to my raw syslog messages and I could see then why I wasn't getting a regex match based on the regex patterns listed.

    4. In my case I was able to reformat the syslog message from this:
    MYCOMPONENTNAME: Parameter Type Violation on server.domain.com from xxx.xxx.xxx.xxx
    


    To this:

    
    #PARAMETER      VALUE
    logorig          1
    logseverity     10
    

    That allowed the parser to match the last very basic regex pattern and that gave me the component name, which gave me the eventclasskey, which allowed me to add an event mapping.

    Things that break or mess with this:

    1. Not putting space after the colon
    2. line feeds in the event message


    In my case I was fortunate that the sending device allowed me to modify the syslog message text. In cases where that cannot be modified, I would think that the next step is to modify the file: $ZENHOME/Products/ZenEvents/SyslogProcessing.py to add a new regex, but that's deep waters for me since I'm relatively new to zenoss/python/zope so maybe someone could provide details for this.

    It would be nice if at some point the preparse regex statements could be managed and added at the UI. My concerns with adding it to code is that either a) I'll forget about it or b) it will get overwritten in future build. That way community can contribute message formats in the future to expand the current default set.

    Hope that helps....
  • artifact Rank: Green Belt 303 posts since
    Mar 11, 2008
    Currently Being Moderated
    7. Oct 21, 2008 8:04 PM (in response to agent00111)
    RE: event class mapping
    Thanks for the info...

    Unfortunately most of my syslog events are coming from switches and I can't change their format... I guess I'm going to have to forward them through syslog-ng or something and format them there.
  • agent00111 Rank: White Belt 9 posts since
    Sep 23, 2008
    Currently Being Moderated
    8. Oct 22, 2008 12:26 PM (in response to artifact)
    RE: event class mapping
    You also should be able to modify $ZENHOME/Products/ZenEvents/SyslogProcessing.py to include a python regex that would better match your syslog message string.

    That would probably be less work than forwarding message to syslog-ng for reformat. Also you could then post the regex you came up with along with information about your sending device and that could help others in the community if they have the same type of device.

    You could use an online regex tool (I used http://regex.larsolavtorvik.com/), to test your regex against your syslog message to make sure component and summary are parsed correctly.


    Hope that helps!
  • artifact Rank: Green Belt 303 posts since
    Mar 11, 2008
    Currently Being Moderated
    9. Oct 24, 2008 1:11 PM (in response to agent00111)
    RE: event class mapping
    I'm having trouble created a regex statement...

    The devices that I'm having trouble with are Adtran Netvanta switches. Interestingly, there already is a regex statement for Adtran devices in SyslogProcessing.py:
    ^(?P<deviceModel>[^\[]+)\[(?P<deviceManufacturer>ADTRAN)\]:(?P<component>[^\|]+\|\d+\|\d+)\|(?P<summary>.*)


    But that doesn't seem to fit my syslog messages at all:
    device_host_name ETHERNET_INTERFACE:eth 0/12 auto-negotiation complete


    Can someone help me create an appropriate statement for this?
  • artifact Rank: Green Belt 303 posts since
    Mar 11, 2008
    Currently Being Moderated
    10. Oct 24, 2008 2:06 PM (in response to artifact)
    RE: event class mapping
    I have the following that seems to be working, but I don't know what regions I'm supposed to be defining:
    r"^(?P<deviceModel>.*)\s(?P<eventClassKey>.*):+(?P<summary>.*)"
    

    i.e. what is deviceModel? Is that appropriate for the hostname of my device?
  • kkearney ZenossEmployee 118 posts since
    Sep 23, 2008
    Currently Being Moderated
    11. Oct 28, 2008 6:31 PM (in response to artifact)
    RE: event class mapping
    Just a little bit of housekeeping: I've added ticket "Add method to update syslog regexes inside of Zenoss" http://dev.zenoss.org/trac/ticket/3973 to Trac so that it can be prioritized and scheduled.

    From a quick look at the code, the regular expressions match the special Python 'group' names and these get turned into the fields that get provided to the event 'evt'. To take a specific example:

      r"^(?P<deviceModel>.*)\s(?P<eventClassKey>.*):+(?P<summary>.*)" 


    will populate the evt object with 'deviceModel', 'eventClassKey' and 'summary'. You don't need anything really except for 'eventClassKey' and possibly 'summary', but more is nicer. happy Check out the more detailed view of the fields in evt from the Event Console to see what's available.

    So to try to tackle a few of the regular expressions in here:

    Sample:
    device_host_name ETHERNET_INTERFACE:eth 0/12 auto-negotiation complete

    Regex:
    r"^(?P<device>\S+)\s(?P<eventClassKey>.*):(?P<component>eth\s* \d+/\d+)\s*(?P<summary>.*)"

    Sample:
    Parameter Type Violation on server.domain.com from xxx.xxx.xxx.xxx

    Regex:
    r"^(?P<eventClassKey>Parameter Type Violation)\son\s(?P<device>.*)\sfrom (?P<reportedip>\S+)"


    NB: I had to make up a field there. I *think* that should work :)

    Make a backup copy of the SyslogProcessing.py file, make your edits, restart zensyslog and look for errors in the log file.

    Hopefully that helps!

    kells

More Like This

  • Retrieving data ...