Have you recently downloaded Zenoss Core, or do you have questions about implementing the solution in your environment? If so, please register to attend our bi-weekly Getting Started with Zenoss Core Webinar. The March 9 session is still open for sign-up, and if you can’t make this session, the next March 23 one is on the schedule. You can register here:

Tuesday, March 9 1:00 p.m. EST

Tuesday, March 23 9:00 a.m. EST

Here’s what you’ll get out of the session:

• An introduction to the Zenoss Community
• Installing the software properly
• Preparing your environment
• Logging in to get started
• Adding, classifying and auto-discovering your devices
• Getting and staying organized
• Seeing the “big picture” (dashboard, network map, event console, Google Maps, etc.)
• Avoiding common mistakes

We also have a Zenoss engineer available to answer questions live – and there are usually lots of questions submitted! If you’re interested in seeing past Q&A logs, take a look at some of the previous Getting Started with Zenoss Q&A sessions where we document and upload all of the questions submitted along with answers.

We are pleased  to announce the Zenoss Core 2.5.2 maintenance  release, now available for  download from:

http://community.zenoss.org/community/download

Version 2.5.2 of Zenoss Core offers:

Prior  2.5.x versions of Zenoss Core offer these new features and  improvements:

The 2.5.2  Zenoss Core release notes are available from the Documentation page in  PDF and HTML formats:

http://community.zenoss.org/community/documentation/official_documentation/release_notes

Installation  and upgrades from earlier versions are covered in Zenoss Core  Installation, also available in PDF and HTML formats from the Documentation page: http://community.zenoss.org/community/documentation/official_documentation/installation-guide

Zenoss  thanks everyone who contributed to the testing effort for this release!

Have you recently downloaded Zenoss Core, or do you have questions  about implementing the solution in your environment? If so, please  register to attend our bi-weekly Getting Started with  Zenoss Core Webinar. The February 23 session is still open for  sign-up, and if you can’t make this session, the next ones will be on the schedule soon. You can register here:

Tuesday, February 23  9:00 a.m. EST

Here’s   what you’ll get out of the  session:

• An introduction to the Zenoss Community
• Installing  the software properly
• Preparing your environment
• Logging  in to get started
• Adding, classifying and auto-discovering your  devices
• Getting and staying organized
• Seeing the “big  picture” (dashboard, network map, event console, Google Maps, etc.)
• Avoiding  common mistakes

We also have a Zenoss engineer  available to answer questions live – and there are usually lots of  questions submitted! If you’re interested in seeing past Q&A logs,  take a look at some of the previous Getting Started  with Zenoss Q&A sessions where we document and  upload all of the questions submitted along with answers.

Have you recently downloaded Zenoss Core, or do you have questions about implementing the solution in your environment? If so, please register  to attend our bi-weekly Getting Started with Zenoss Core Webinar. The February 9 session is still open for sign-up, if you can’t make this session, the next one is already on the schedule. You can register here:

Tuesday, February 9 2:00 p.m. EST

Tuesday, February 23 9:00 a.m. EST

Here’s   what you’ll get out of the session:

• An introduction to the Zenoss Community
• Installing the software properly
• Preparing your environment
• Logging in to get started
• Adding, classifying and auto-discovering your devices
• Getting and staying organized
• Seeing the “big picture” (dashboard, network map, event console, Google Maps, etc.)
• Avoiding common mistakes

We also have a Zenoss engineer available to answer questions live – and there are usually lots of questions submitted! If you’re interested in seeing past Q&A logs, take a look at some of the previous Getting Started with Zenoss Q&A sessions where we document and upload all of the questions submitted along with answers.

We really enjoy meeting everyone in person at Zenoss Community Days but for those of you who can't attend due to distance or other restrictions we decided to video those sessions and post them to the web so you can get the benefit of community from the free training. We broke the Zenoss Community Day training from ZCD Baltimore down into 10 sessions for you to follow along at your own pace.

We included both the links to the individual videos as well as embedding the player in the post below.

Here's a sample of what you would get during community day training:

1. Introduction to Zenoss
2. Users, Roles and Alerting
3. Inventory and Configuration
4. zProperties and Device Tree
5. Event Management and Correlation
6. Availability Monitoring
   
7. Reporting and Maintenance Windows
8. Advanced Administration
   
9. TALES Expressions
10. Customizing the Zenoss User Interface

Background

Event driven network monitoring systems rely on acknowledgment for multiple reasons. These reasons include:

• To let other Zenoss administrators and users know that someone is aware of the problem and taking action.
• To suppress escalation of alerts.

To acknowledge an alert, a responder typically:

• Clicks the acknowledge link in the body of the default Zenoss alert email.
• Logs into the Zenoss server, navigates to the alert on the device or in the event manager, and acknowledges the alert.

This process works well when you're in the office or your operations are business hours only. Typically, this is not the case for most users of Zenoss. Most users have critical systems that require 24x7 monitoring and alerting. Additionally, managers and executives require escalation of messages from the on-call personnel through management and sometimes to executive management.

The acknowledgment model of Zenoss, in its current state, is limited. An alert will never escalate, as long as it is acknowledged. In a 24x7 monitoring and alerting model, an alert can and will come during non-business hours. The first responder (on-call personnel) is usually mobile and does not always have immediate access to a computer. To compound this, many Zenoss servers live on private networks that require VPN authentication to access them. There are times where the on-call personnel can not get into the system quick enough to prevent escalation from occurring. We all know how upset managers get when the on-call grunt doesn't do her job properly.

One possible solution to this issue is email based event acknowledgment. The on-call responder replies to the alert email on her PDA. The alert email is routed back to the Zenoss server for processing with Procmail. The body of the email is piped into a python script, via Procmail. The device and event id are extracted and the alert is acknowledged through Zenoss API calls. An email is sent back to the responder reporting whether or not the acknowledgment was successful.

The biggest caveat, for this model, is that incoming mail needs to be routed to your Zenoss server. As such, this will require knowledge of mail relay administration or you'll need to rope in your mail relay administrator to help you out. Since every company's mail setup is unique (Spam filters, Exchange Servers, Postfix Servers, et. al) I will leave this as an exercise for the user of this how-to. The second caveat is that your mail relay becomes a single point of failure for delivery. If your mail relay is down or in schedule maintenance, you won't be able to acknowledge events.

** This will work with plain-text, base64 encoded and multipart email types (Iphone, Blackberry, Exchange). I have tested this on Zenoss Core 2.4.5 and 2.5.0

Installation

Postfix Installation

Install Postfix from RPM or source.

Procmail Installation

Install Procmail from RPM or source and configure per the README.

Configuration

Postfix Configuration

Edit your configuration file to use procmail. This is typically located at:

vi /etc/postfix/main.cf

Edit mailbox_command to reflect the path of your Procmail installation.

mailbox_command = /usr/bin/procmail

Restart Postfix

/etc/init.d/postfix restart

Procmail Configuration

We will be filtering email at the user level, not the system level.  As the zenoss user:

vi ~/.procmailrc
chmod 600 ~/.procmailrc

Create a procmail directory to log to.

mkdir ~/procmail

Copy and paste these two sections into your .procmailrc. There can be space between them. The first section exports basic environment variables.

SHELL=/bin/bash
PMDIR=$HOME/procmail
LOGFILE=$PMDIR/pmlog
ZENHOME=/usr/local/zenoss
PYTHONPATH=$ZENHOME/lib/python
PATH=$ZENHOME/bin:$PATH

The next section is the Procmail formula. This says for any message that contains '[zenoss]' in the subject, forward the header and body (as STDIN) to /home/zenoss/zenemailack.py and redirect STDOUT and STDERR to $ZENHOME/log/zenemailack.log. This formula also locks and waits until the script has finished. You can relocate zenemailack.py anywhere on your system, just update the path in the formula accordingly.

:0
* ^Subject:.*\[zenoss\].*
{
  :0 hbW:
  | /home/zenoss/zenemailack.py >> $ZENHOME/log/zenemailack.log 2>&1
}

Zenoss UI Configuration

Under the 'settings' tab, from the 'settings' link on the main navigation menu (left navbar), change this value to reflect the valid email address to your zenoss user.

From Address For Emails        zenoss@server.mydomain.com

* You must be able to receive emails at this email address on your Zenoss server.

Zenoss Server Configuration

Download the attached script below (zenemailack.py) and place it in /home/zenoss. You can install the file to any location of your choice. If you use a different location, make sure to update the Procmail formula to reflect the new path of your installation.

Change permissions to make the script executable.

chmod 755 ~/zenemailack.py

Edit zenemailack.py and change these variables to suit your environment:

# Update for outgoing email
MAIL = "/usr/sbin/sendmail -t "
from_addr = "zenoss@server.mydomain.com"

cc_address = ""

Testing

General

The first step is to make sure you can receive emails at zenoss@server.mydomain.com. If you can't receive a basic email, resolve this or you're out of luck.

Procmail

Once you can successfully receive emails, make sure that procmail is working. You can create a test formula in your .procmailrc to test that it's working correctly.

vi ~/.procmailrc

Append this to your .procmailrc and save it.

:0
* ^Subject:.*test
/dev/null

As zenoss user, test Procmail functionality by sending yourself an email with 'test' as the subject.

echo "foo" | mailx -s "test" zenoss

Check the procmail log for success.

tail ~/procmail/pmlog
From zenoss@localhost  Wed Sep  3 16:11:26 2008
 Subject: test
  Folder: /dev/null                                595

If you have no entry for this test in your Procmail log, Procmail is not working properly. Check your installation, double check that Postfix is configured to use Procmail and that you re-started Postfix after you configured it to use Procmail.

Re-edit ~/.procmailrc and remove the test formula you added above. Your .procmailrc should look like this after.

SHELL=/bin/bash
PMDIR=$HOME/procmail
LOGFILE=$PMDIR/pmlog
ZENHOME=/usr/local/zenoss
PYTHONPATH=$ZENHOME/lib/python
PATH=$ZENHOME/bin:$PATH
:0
* ^Subject:.*\[zenoss\].*
{  
  :0 hbW:
  | /home/zenoss/zenemailack.py >> $ZENHOME/log/zenemailack.log 2>&1
}

Zenoss

Once you've verified that you can receive emails and Procmail works, test this with an actual event.

If you're impatient, you can inject an event into Zenoss. To do this, you can follow the admin guide. Just make sure your event matches an alertable condition.

Admin Guide - 7.1.8.1 – Creating Events Through the User Interface

If you're not impatient, just wait around until you get an alert. Once you receive the alert, simply reply. There is no need to type anything in your reply.

Make sure that your mail editor includes the body of the alert when you reply. I don't know of any that don't, by default, but I'm just throwing that in there.

You should receive a verification email back saying that the alert has been acknowledged or not. You’ll only receive a verification reply if the alert is in an unacknowledged state or something crapped out during the attempted acknowledgment. If you don’t get a reply email, the alert most likely cleared itself and has been moved to history, by Zenoss.  Once an event has been acknowledged, any reply-to-all email threads will not generate email noise. They will, however, be logged.

For debugging or troubleshooting, you can view the log at:

$ZENHOME/log/zenemailack.log

Further configuration and debugging information may be found at the wiki entry: Email Acknowledgment - Postfix, Procmail and Python, which will have any updates to the instructions going forward.  Thanks again to Scott Haskell for providing this tip!

Just in time for this week's SC09 High Performance Computing (HPC) conference comes the announcement from the Los Alamos National Laboratory (LANL) HPC Division that they are using Zenoss to monitor their HPC large scale clusters.  LANL currently has the #2 fastest supercomputer in the world, and they use a modified version of Zenoss to monitor it.  They are working to share their customizations and there is a High Performance Computing Development area dedicated to the work and an HPC group for further collaboration on using Zenoss in your HPC environment.

LANL HPC Deployment of Zenoss

Los Alamos National Laboratory High Performance Computing Division is currently deploying Zenoss with some modifications to monitor high performance large scale clusters.   We have created several ZenPacks that help to extend Zenoss in the areas of issue tracking, asset tracking, and scalability.  Attached is a file that gives a high level description of the enhancements LANL made to Zenoss for our deployment.

 

The basics are in place, but there are lots of opportunities for contribution. Here’s a partial list of things we think would be of great use to the HPC that we won’t get to any time soon:

• Direct feed of resource manager job allocation data

• Increased automation of event-->issue roll-up

• Performance data from the nodes

• End to end I/O subsystem view

• After-the-fact automated event/issue correlation

• Continuing filter/mapping refinement

• Better high-level reporting facilities

• Alternate visualization of data across event, performance, environmental data categories

• Appropriate and relevant monitoring data and rates for HPC Center networks.

We are currently working within our organization to authorize approval for sharing of the enhancements we have made.  Our next steps will involve working to get our changes integrated with the newer versions of Zenoss.  We look forward to working with others to add more functionality specific to high performance computing.

Zenoss Core 2.5 code-named King Crab was released on October 28th. This newest release has been the result of hard work between the Zenoss Development team and the Zenoss Community. Here are the highlighed features of Zenoss Core 2.5:

• Amazon Web Services (Amazon EC2) Monitoring
• Over 40+ New Community Developed Innovations
• New Event Console
• 65 Community Submitted Bugs and Feature Requests Addressed

Download Zenoss Core 2.5 today!

Last Chance for Free Training

On November 6th Zenoss will be conducting our third Zenoss Community Day. The event will be held in Baltimore, MD during the USENIX 23rd Large Installation Systems Administrator Conference (LISA). Our most recent event at the Ohio Linuxfest had over 80 attendees, who now are well-trained on how to monitor their networks with Zenoss Core. By all accounts these training events are an invaluable way to learn about monitoring your infrastructure with Zenoss Core and they're free!

Read More >>

Tip of the Month:Finding Unbound Devices

If you need to find all devices not currently bound to a Location, Group or System you can query the system to find these unbound devices. Read this simple tip and get your monitored devices better organized.

Read More >>

2010 Open Source Management Survey

For the last three years we have conducted an online management survey discussing the trends in open source systems management. This year we will be doing so again and hoping to gain as well as provide insight to open source users and potential users. All participants will get early access to the survey results and be entered for a chance to win a Dell Mini netbook.

Read More >>

Cloud Monitoring Contest

As enterprise infrastructure continues to evolve and move into more virtual and non-local environments, Zenoss wants to help our users expand their monitoring of their networks, devices and the services that they depend on. We've decided to help kickstart the creation of new ZenPacks for monitoring this evolving landscape with another Community ZenPack Contest, the "Zenoss in the Clouds ZenPack Contest" with not 1, but 5 great prizes!

Read More >>

Zenoss Community Partners

This month some of our community partners are providing some cool things to the Zenoss community. Jane Curry at Skills 1st is offering Zenoss workshops in the UK. And Plixer is providing integration with their free Scrutinizer NetFlow and sFlow Analyzer to all Zenoss users.

Do you provide support and services for Zenoss Core? Do you develop ZenPacks and want to do so for other income? Check out the new community partner portal today!

Read More >>

Thank you for your interest and support for Zenoss.

Mark R. Hinkle
Vice President, Community
Zenoss Inc.

Upcoming Events

• 11/04 - 11/5 – USENIX LISA 2009 – Baltimore, MD
• 11/04 – Zenoss Enterprise Demo – Webinar
  
• 11/05 – GOSCON - Washington D.C.
• 11/06 – Zenoss Community Day – Baltimore, MD
• 11/11 – Zenoss Enterprise Demo – Webinar
• 11/18- 11/19 – Zenoss Administrator Training – Distance Learning
• 11/24 - 11/25 – Zenoss Administrator Training – Distance Learning