Zenoss Core 3.x Network and System Monitoring

Table of Contents

Zenoss Core 3.x Network and System Monitoring

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers and more

Why Subscribe?

Free Access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Network and System Monitoring with Zenoss Core

Device management

Availability and performance monitors

Event management

Plugin architecture

System reports

Custom device reports

System architecture

User layer

Data layer

Collection layer

Device management daemons

Performance and availability daemons

Event daemons

Summary

2. Discovering Devices

Zenoss Core installation

Preparing devices for monitoring

SNMP

SNMP versions

Configuring SNMP on Linux

Configuring SNMP and WMI on Windows

Zenoss Plugins

Installing Zenoss Plugins

Port scan

Opening monitoring-specific ports

Configuring Linux firewalls

Configuring Windows firewall

Zenoss Core setup wizard

Step 1: Setting up users

Step 2: Specify or discover devices to monitor

Adding devices

Manually find devices

Autodiscover devices

Our device inventory: A job well done

Reviewing device creation job log

Adding a single device

Entering device attributes

Importing a list of devices with zenbatchload

Command line discovery with zendisc

Summary

3. Device Setup and Administration

Organizing devices in Zenoss Core

Locations

Systems and Groups

Organizer details

Editing organizers

Moving organizers

Classes

Viewing a list of device classes

Assigning devices to a class

Modeling devices

Modeler plugins gather device information

Assigning modeler plugins

Troubleshooting data collection

Troubleshooting SNMP problems

Running snmpwalk

Is the SNMP daemon running on Linux servers?

SNMP problems on Windows

Troubleshooting WMI problems

Zeneventlog—unable to connect to Windows

Zenoss Core does not collect WMI data

Troubleshooting Zenoss Plugins

A class of its own

Device administration

Locking or unlocking a device

Renaming a device

Resetting the IP address

Push changes

Deleting devices

zProperties defined

Summary

4. Monitor Status and Performance

Collectors collect

Configuring the performance collector

Monitoring components

Interfaces

OS Processes

Add Process

Viewing or editing the process details

Configuration properties

Monitoring OS Processes

Services

Enable monitoring for a service

Configuration properties

Monitoring exceptions for services

Interactively monitor IP services

File Systems

Ignoring File Systems with zProperties

Network Routes

Networks

Add Components

Viewing and editing component details for a device

Performance Graphs

Interface template

Performance Graphs

Working with graphs

Monitoring performance thresholds

Summary

5. Custom Monitoring Templates

Monitoring Templates

Monitoring SNMP data sources

Overriding templates

Editing the /Server/Linux template

Find OIDs for SNMP monitoring

Monitoring with Nagios plugins

Working with Nagios plugins

Nagios return codes

Nagios performance data

Adding the Nagios plugin to Monitoring Templates

Adding a Data Source

Adding a Data Point

RRDtool Data Point configurations

Defining monitoring thresholds

Graph definitions

RRDtool Graph Point configurations

Binding templates to the device class

Adding a device to monitor using the Bogo template

Monitoring with Cacti plugins

Data Source parser

Summary

6. Core Event Management

Event Console

Event severities defined

Event statuses defined

Acknowledging an event

Viewing an event log

Events consoles are everywhere

Closing events

Displaying historical events

Event Manager

Event Fields

Event commands

Creating a command

Working with events

Simulating an event

Clearing the event

Event mapping

Event Classes

Event class zProperties

Mapping an event

Event mapping sequence

Event de-duplication

Turning off event de-duplication

Summary

7. Collecting Events

Routing syslog messages to Zenoss Core

Collecting Cisco router syslogs

Testing syslog configuration with Logger

Monitoring Windows event logs

Windows event log severities

Testing the event log configuration with Eventcreate

Incorporating event reporting into third-party scripts via zensendevent

Simple backup script with zensendevent

Creating events by e-mail

Zenmail

Zenpop3

Configuring alerting rules

Alert filters

Alert escalations

Schedule

Alert messages

Event transformations

Some event transformation examples

Programming in zendmd, an interactive shell

Summary

8. Settings and Administration

Managing Zenoss Core users

Administered Objects

Event Views

Groups

Creating custom User Commands

Adding a User Command

System settings

Configuring Zenoss Core's Monitoring Dashboard

Locations portlet with Google Maps

Device Issues portlet

Zenoss Issues portlet

Watch List portlet

Root Organizers portlet

Production States portlet

Portlet permissions

Meet the Zenoss Daemons

Maintenance Windows

Adding MIBs

Backing up and restoring monitoring data

Automating backups with zenbackup

Restoring backups with zenrestore

Updating Zenoss Core

Summary

9. Extending Zenoss Core with ZenPacks

Installing community ZenPacks

Monitoring websites with HttpMonitor

Viewing a list of installed ZenPack objects

Configuring HttpMonitor

Configuring HttpMonitor settings

Creating a ZenPack

Adding files and objects to the ZenPack

Adding a new data source to the monitoring template

Adding objects to a ZenPack

Packaging the ZenPack

ZenPack development mode

Developer resources

Summary

10. Reviewing Built-in Reports

Report overview

Device Reports

New Devices

Device Changes

Model Collection Age

Software Inventory

Manufacturers and Products

SNMP Status Issues

Ping Status Issues

All Devices

All Monitored Components

Event Reports

All Event Classes

All Event Mappings

All Heartbeats

Graph Reports

Multi-Graph Reports

Adding Collections

Adding Graph Definitions

Adding Graph Groups

Performance Reports

Aggregate Report

Availability

CPU Utilization

Filesystem Utilization

Interface Utilization

Memory Utilization

Threshold Summary

User Reports

Notification Schedules

Summary

11. Writing Custom Device Reports

Creating Custom Device Reports

Custom Device Report fields

Building Custom Device Report queries

Using zendmd to test report queries

Exploring data in Zope

Using Python expressions in the columns

Convenience functions

convToUnits

Scheduling reports for e-mail delivery

Sending a CSV report

Scheduling a cron job

Summary

A. Event Attributes

B. Device Attributes

C. Example snmpd.conf

Index

Zenoss Core 3.x Network and System Monitoring

Zenoss Core 3.x Network and System Monitoring

Copyright © 2011 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: April 2011

Production Reference: 1120411

Published by Packt Publishing Ltd.

32 Lincoln Road

Olton

Birmingham, B27 6PA, UK.

ISBN 978-1-849511-58-2

www.packtpub.com

Cover Image by Asher Wishkerman (<a.wishkerman@mpic.de>)

Credits

Author

Michael Badger

Reviewers

Andrea Consadori

Jonny Gerold

Duncan M. McGreggor

Charlie Schluting

Development Editor

Neha Mallik

Technical Editor

Conrad Sardinha

Indexer

Tejal Daruwale

Editorial Team Leader

Akshara Aware

Project Team Leader

Priya Mukherji

Project Coordinator

Jovita Pinto

Proofreader

Stephen Silk

Graphics

Nilesh Mohite

Production Coordinator

Melwyn D'sa

Cover Work

Melwyn D'sa

About the Author

Michael Badger is a freelance technical communicator with a knack for helping other people understand and use their computer software and technology. In addition to writing a previous book about Zenoss Core: Zenoss Core Network and System Monitoring, Badger authored Scratch 1.4: Beginner's Guide, a Scratch programming tutorial.

He lives in north central Pennsylvania (United States) on a small farm and has recently taken to raising pastured chickens, honeybees, and pigs. Michael is searching for a way to integrate Zenoss Core into the hen house so that he can receive an alert each time an egg is laid.

For more information, visit www.badgerfiles.com/zenoss3.

There are so many people to thank, starting with my family. They tolerate my late nights and weekend work.

My team at Packt deserves a nod for finally helping me get this revision done. Thanks for the help Rakesh Shejwal and Jovita Pinto.

Then there are the reviewers. It's not easy to provide substantive critique of another person's work because it takes time and thoughtful consideration for you to want to make my work better. You should know that even though I did not incorporate all your suggestions, I considered them carefully.

About the Reviewers

Andrea Consadori is the lead technical support at Lais s.r.l. and has been working with Zenoss to monitor customer IT infrastructures for the past four years.

He has been implementing simple zenpacks to integrate all of the vendors' products he uses like Motorola Canopy/PTP, Alvarion, and so on.

Lais s.r.l. is a WISP and uses Zenoss to monitor its wireless infrastructure.

Earlier, he worked at Edslan s.p.a. (an Italian networking product distributor) where he studied lots of networking brands.

Andrea is passionate about solving hard networking issues and enjoys working with routing protocol and firewall rules.

I would like to thank Michela for her encouragement that makes my skill and knowledge grow every day.

Jonny Gerold loves Open Source, enjoys working with Linux/Unix/Solaris, and also enjoys dirt biking.

Duncan M. McGreggor started his programming career at the ripe old age of 11 in the early 80s. From his adventures in rewriting games on Kaypro's luggable CP/M machine to the open source world, programming has been his passion. When Duncan wasn't hacking, he was an Army MI linguist; worked his way up to sous chef in a Massachusetts restaurant; studied quantum mechanics and mathematics as a physics major; learned meditation while living with Tibetan monks; and started his own software consulting company. His contract work included systems management solutions for the U.S. Federal Government as well as Zenoss, Inc. After consulting for several years, Duncan joined an engineering startup as the COO and eventually left that position for Canonical where he manages teams in the Product Strategy group, improving the Ubuntu Linux distribution.

Charlie Schluting, BS CS, MBA; is first a sysadmin, and second a technology strategy connoisseur, currently working as the IS Operations Manager at Canonical (the creators of Ubuntu Linux). Charlie also wrote Network Ninja, http://stores.lulu.com/schluting, a book designed to educate sysadmins and mid-level network engineers on the fundamentals of the protocols they work with. Charlie can frequently be found dabbling in various technology startups, attempting to change the world, when he's not touring off-road on his motorcycle (http://charlierides.com).

www.PacktPub.com

Support files, eBooks, discount offers and more

You might want to visit www.PacktPub.com for support files and downloads related to your book.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

http://PacktLib.PacktPub.com

Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can access, read and search across Packt's entire library of books. 

Why Subscribe?

Fully searchable across every book published by Packt

Copy & paste, print and bookmark content

On demand and accessible via web browser

Free Access for Packt account holders

If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view nine entirely free books. Simply use your login credentials for immediate access.

Preface

For system administrators, network engineers, and security analysts, it is essential to keep a track of network traffic.

Zenoss Core is an enterprise-level systems and network monitoring solution that can be as complex as you need it to be. While just about anyone can install it, turn it on, and monitor something, Zenoss Core has a complicated interface packed with features. The interface has been drastically improved over version 2, but it's still not the type of software you can use intuitively—in other words, a bit of guidance is in order.

The role of this book is to serve as your Zenoss Core tour guide and save you hours, days, maybe weeks of time.

This book will show you how to work with Zenoss and effectively adapt Zenoss for System and Network monitoring. Starting with the Zenoss basics, it requires no existing knowledge of systems management, and whether or not you can recite MIB trees and OIDs from memory is irrelevant. Advanced users will be able to identify ways in which they can customize the system to do more, while less advanced users will appreciate the ease of use Zenoss provides. The book contains step-by-step examples to demonstrate Zenoss Core's capabilities. The best approach to using this book is to sit down with Zenoss and apply the examples found in these pages to your system.

The book covers the monitoring basics: adding devices, monitoring for availability and performance, processing events, and reviewing reports. It also dives into more advanced customizations, such as custom device reports, external event handling (for example, syslog server, zensendevent, and Windows Event Logs), custom monitoring templates using SNMP data sources, along with Nagios, and Cacti plugins. An example of a Nagios-style plugin is included and the book shows you where to get an example of a Cacti-compatible plugin for use as a command data source in monitoring templates.

In Zenoss Core, ZenPacks are modules that add monitoring functionality. Using the Nagios plugin example, you will learn how to create, package, and distribute a ZenPack. You also learn how to explore Zenoss Core's data model using zendmd so that you can more effectively write event transformations and custom device reports.

Implement Zenoss Core and fit it into your security management environment using this easy-to-understand tutorial guide.

What this book covers

Chapter 1, Network and System Monitoring with Zenoss Core, provides an overview of Zenoss Core's monitoring capabilities and system architecture.

In Chapter 2, Discovering Devices, we prepare our monitoring environment by configuring SNMP, WMI, SSH, and firewall ports. We'll add devices to Zenoss Core via the setup wizard, zenbatchload, and zendisc.

Chapter 3, Device Setup and Administration, configures devices so that we ensure we collect the proper monitoring information by organizing, configuring, and troubleshooting the monitoring properties.

Chapter 4, Monitor Status and Performance, monitors and graphs the performance of device components such as routes, windows services, IP services, processes, file systems, and network interfaces.

Chapter 5, Custom Monitoring Templates, explores custom monitoring templates by configuring various data sources, including SNMP, Nagios plugins, and Cacti plugins.

Chapter 6, Core Event Management, introduces us to processing events via the Event Console. We create custom event commands, learn how to create test events, and perform event mapping.

Chapter 7, Collecting Events, allows Zenoss Core to receive and process events from third-party sources, such as syslog, Windows Event Log, e-mail, and home-grown system administration scripts.

Chapter 8, Settings and Administration, covers common Zenoss Core administration tasks, such as managing users, the monitoring dashboard, backups, and updates.

Chapter 9, Extending Zenoss Core with ZenPacks, installs, creates, and packages add-on modules. ZenPacks extend the functionality of Zenoss Core.

Chapter 10, Reviewing Built-in Reports, reviews each of Zenoss Core's included reports to help us troubleshoot, analyze, and view our monitoring performance over time. It also creates custom graph and multi-graph reports.

Chapter 11, Writing Custom Device Reports, provides an in-depth look at Zenoss Core's custom device report functionality, including the use of zendmd to explore the Zenoss data model.

Appendix A, Event Attributes, lists the available event attributes in Zenoss Core.

Appendix B, Device Attribute, lists the attributes that we may use when working with our devices.

Appendix C, Example snmpd.conf, lists a sample snmpd.conf file.

What you need for this book

This book will work best if you have a working installation of Zenoss Core and some network servers, routers, switches, and other devices to monitor. Zenoss Core can be installed on Linux, Mac OS X, and Windows (via a virtual Zenoss Virtual Appliance and VMware).

Who this book is for

This book is written primarily for network and systems administrators who are monitoring their IT assets with Zenoss Core or who plan to monitor them. In reality, this book will benefit anyone, regardless of job title, who recognizes the importance of proactively monitoring the servers, routers, computers, websites, and devices that connect companies to customers.

Conventions

In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.

Code words in text are shown as follows: Before you make any changes, back up the snmpd.conf file.

A block of code is set as follows:

syslocation Unknown (edit /etc/snmp/snmpd.local.conf)

syscontact Root (configure /etc/snmp/snmpd.local.conf)

Any command-line input or output is written as follows:

python setup.py build python setup.py install

New terms and important words are shown in bold. Words that you see on the screen, in menus or dialog boxes for example, appear in the text like this: Select Simple Network Management Protocol and WMI.

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.

To send us general feedback, simply send an e-mail to <feedback@packtpub.com>, and mention the book title via the subject of your message.

If there is a book that you need and would like to see us publish, please send us a note in the SUGGEST A TITLE form on www.packtpub.com or e-mail .

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide on www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you would report this