OpenLMI Project - Part I

Introduction

This document is an overview of CIM/WBEM technology (DMTF Standards) implemented in the Open Source current scene. In particular the Red Hat/Fedora Open Linux Management Infrastructure Project (OpenLMI) approach - https://fedorahosted.org/openlmi/.

The document is splitted in two parts, this first part is an overview of the underlying technology used in the OpenLMI Project, focusing in the set of standards, and the tools used in the development.

Distributed Management Task Force - DMTF

DMTF is an industry organization that develops, maintains and promotes standards for systems management in enterprise IT environments. Enable systems management interoperability between IT products from different manufacturers or companies, allowing to build systems management infrastructure components in a platform-independent and technology-neutral way.

DMTF puts together many IT areas in the shape of standards, among the most important we can find:

  • CIM - Common Information Model: CIM provides a common definition of management information for systems, networks, applications and services (processors, operating systems, and users, etc), and allows for vendor extensions. CIM's common definitions enable vendors to exchange semantically rich management information between systems throughout the network.
  • WBEM - Web-Based Enterprise Management: Defines protocols for the interaction between systems management infrastructure components implemented in CIM.
  • CDM - Common Diagnostic Model: The CDM schema is a part of the CIM schema that defines how system diagnostics should be incorporated into the management infrastructure.
  • SMASH - Systems Management Architecture for Server Hardware: Initiative that include management profiles for server hardware management.
  • SMBIOS - System Management BIOS: Defines how the BIOS interface of x86 architecture systems is represented in CIM
  • ASF - Alert Standard Format: defines remote control and alerting interfaces for OS-absent environments (for instance a system board controller of a PC)
  • DEN - Directory Enabled Network: Defines how LDAP directories can be used to provide access to CIM managed elements and defines CIM to LDAP mappings for a part of the CIM schema.
  • DMI - Desktop Management Interface: Deprecated, this feature is supported within the other set of standards in DTFM.
  • DASH - Desktop and mobile Architecture for System Hardware: a management standard based on DMTF Web Services for Management (WS-Management), for desktop and mobile client systems.
  • CMDBF - Configuration Management Database Federation: CMDBf facilitates the sharing of information between configuration management databases (CMDBs) and other management data repositories (MDRs).
  • VMAN - Virtualization Management Initiative: A suite of specifications based on DMTF’s CIM that helps IT managers: Deploy virtual computer systems, Discover/inventory virtual computer systems, Manage lifecycle of virtual computer systems, Create/modify/delete virtual resources and Monitor virtual systems for health and performance.


image



CIM related standards are also developed outside of the DMTF. Some examples are:

  • The SNIA – develops and maintains the SMI-S standard that defines DMTF management profiles for Storage Area Networks.
  • The Open Group – develops and maintains the CMPI standard that defines a C/C++ API for CIM providers. The Common Manageability Programming Interface (CMPI) is an open standard that defines a programming interface between a CIM server and CIM providers. http://en.wikipedia.org/wiki/Common_Manageability_Programming_Interface
  • The Java Community Process – currently develops the JSR-48 standard that defines a Java API for CIM client applications.

The next sections are an overview of DMTF concepts used on OpenLMI Project. Some of these are WBEM, CIM, CMPI, SMASH, SMI-S, CIMOM's, and so forth.

WBEM

The Web Based Enterprise Management defines protocols for IT infrastructure management using web technologies. In particular WEBM provides access to management data, managed items and centralized data of management applications using web technology.

This WBEM is based in three foundations:

  1. CIM, the Common Information Model.
  2. XML codifications.
  3. HTTP as transport protocol

WBEM uses CIM as DTD of XML used on top of HTTP protocol.

The official references:

  1. WBEM: http://dmtf.org/standards/wbem
  2. CIM: http://dmtf.org/standards/cim

WBEM FLOSS implementations

The following is list of current open source approachs related with WBEM standard:

  1. Pegasus

    https://collaboration.opengroup.org/pegasus/

    Open source project providing a CIM Object Manager written in C++ (the central infrastructure component for CIM and WBEM)

  2. SBLIM

    http://sourceforge.net/apps/mediawiki/sblim/index.php?title=Main_Page

    Open source project providing an implementation of CIM and WBEM for Linux as well as other CIM and WBEM related components and tools

  3. PyWBEM

    http://sourceforge.net/apps/mediawiki/pywbem/index.php?title=Main_Page

    Python library for making CIM operations over HTTP using the WBEM CIM-XML protocol. It is based on the idea that a good WBEM client should be easy to use and not necessarily require a large amount of programming knowledge. PyWBEM is suitable for a large range of tasks from simply poking around to writing web and GUI applications.

  4. Ubuntu WBEM

    https://blueprints.launchpad.net/ubuntu/+spec/wbem-support

    Ubuntu is only thinking about to integrate this kind of technologies right now. It's not under development this kind of stack by the Ubuntu community.

  5. OpenWBEM

    http://openwbem.sourceforge.net/ (deprecated)

  6. WBEM Services

    http://wbemservices.sourceforge.net/ (deprecated)

CIM: The Common Information Model

CIM is the basic building block for integration and use of management data within of IT infrastructure. CIM provides the basic information model between the enterprise and technology vendors domain.

CIM is a conceptual information model for describing managed entities, their composition, and relationships. The management models (the manageable entities across an enterprise) are comprised of a Core Model and a set of Common Models than extend from the Core.

Commom Models have been defined for systems, services, networks, applications, users and databases; all the major technology domains than require management. Those common models are for “physical” entities (linke an operative system, an application, …), there are Information Models have alse been developed to specify consistent, security, event management, support and so forth.

On the other hand, the models were designed to distinguish between the physical and logical aspects of the managed entities. The Physical Model describes how components can be seen or touched, such as racks, cards are physically configured. The Logical Model, the logical aspect of CIM, describes the higher-level, more abstract, functional elements, such as computer systems, databases, services, their capabilities, settings, statics, resourses, status.

The benefits of CIM from the integrators perspective:

  • CIM is defined using a language called Managed Object Format (MOF). MOF is a textual forma (both human and machine-readable) for describing an information model usgin an object-oriented design. CIM can also be rendered using the Unified Modeling Language (UML). MOF is formally defined in the CIM Specification. You can download the specification from http://dmtf.org/standards/cim. This language allows integrators to express the exact semantics and behaivour of their managed entities with the context of CIM.
  • There are two parts to CIM: The CIM Specification and the CIM Schema.

    The CIM Specification describes the language, naming, Meta Schema and mapping techniques to other management models such as SNMP MIBs, and DMTF MIFs etc. The Meta Schema is a formal definition of the model. It defines the terms used to express the model and their usage and semantics. The elements of the Meta Schema are Classes, Properties, and Methods. The Meta Schema also supports Indications and Associations as types of Classes and References as types of Properties.

    The CIM Schema provides the actual model descriptions. The CIM Schema supplies a set of classes with properties and associations that provide a well-understood conceptual framework within which it is possible to organize the available information about the managed environment.

    The CIM Schema itself is structured into three distinct layers:

    The Core Schema is an information model that captures notions that are applicable to all areas of management
    Common Schemas are information models that capture notions that are common to particular management areas, but independent of a particular technology or implementation. The common areas are systems, devices, networks, applications, metrics, databases, the physical environment, event definition and handling, management of a CIM infrastructure (the Interoperability Model), users and security, policy and trouble ticketing/ knowledge exchange (the Support Model). These models define classes addressing each of the management areas in a vendor-neutral manner.
    Extension Schemas represent organizational or vendor-specific extensions of the Common Schema. These schemas can be specific to environments, such as operating systems (for example, UNIX® or Microsoft Windows®). Extension Schema fall into two categories, Technology-Specific areas such UNIX98 or Product-Specific areas that are unique to a particular product such as Windows.

    The formal definition of the CIM Schema is expressed in a Managed Object File (MOF).

  • CIM is extensible: Integrators can extend from common classes to include any vendor value-added content or behaivours.
  • CIM Shemas can be partitioned: Integrators don’t need to implement the complete CIM schema, just the portions that apply for their specific management requiriments.
  • CIM is protocol agnostic: Right now, CIM supports an XML-based protocol called WBEM for encoding CIM operations, in the future other protocols will be defined.

CMPI

The Common Manageability Programming Interface is defined by The Open Group initiative, as we have already said. This programming interface is used by the OpenLMI Project people in order to make the CIM providers.

The CMPI programming interface is defined for the C programming language. Its C header files are enabled for C++. In addition, there are C++ utility macros that allow accessing the interface in a way that is more typical to C++. Before the introduction of CMPI, each CIM server implementation had its own specific programming interface for CIM providers (e.g.WMI COM API, OpenPegasus C++ API, OpenWBEM C++ API, etc.). CMPI allows CIM providers to be developed that are mostly or completely agnostic to the type of CIM server they are being used with. Therefore, CMPI providers can be deployed across a variety of operating environments with no or minimal adaptation work. This protects the investment in these CIM providers.

Let's see some project supporting this technology.

CMPI FLOSS implementations

  • KonkretCMPI - (http://konkretcmpi.org/)

    An open-source tool for rapidly developing CMPI providers in the C programming language. KonkretCMPI makes CMPI provider development easier by generating type-safe concrete CIM interfaces from MOF definitions and by providing default implementations for many of the provider operations.

  • PyWebm - (http://pywbem.sf.net)

    PyWBEM is a set of tools around Python language. PyWBEM provides a Python CIM Provider and CIM Client interface, MOF compiler, and an interesting CIM classes browser based on Apache and mod-python, YAWN.

  • CIMPLE - http://www.simplewbem.org/

    This is a set for CIM providers development. A CIM provider enviroment.

SMASH

In emergency and/or ad/hoc situation, system administrators often need to interactively manage various systems using a specific command. The challenge however is that servers from different vendors use different commands to do the same task. The Systems Management Architecture for Server Hardware (SMASH) is a recent development that enables administrators to use a consistent command line interface to server monitoring and management tasks irrespective of the vendor. The Distributed Management Task Force (DMTF) released the new SMASH Command Line Protocol (CLP) standard in June 2005 to address the lack of this command line consistency to management and monitoring information in heterogeneous server environments.

In the past, there were no cross-platform standards that addressed the need for administrators to directly manage servers from multiple vendors. This led hardware manufacturers to develop varied tool sets to manage systems via in-band (ssh, VNC, …) and out-of-band (specialized device card) connections for different operating systems and system states. To address this problem, the Distributed Management Task Force (DMTF) created the Systems Management Architecture for Server Hardware (SMASH) Initiative.

The SMASH Initiative includes a suite of specifications that deliver architectural semantics, industry standard protocols and DMTF Management Profiles to unify the management of the data center. The DMTF specifications published as part of this Initiative include: the Web Services for Management (WS-Man) Specification, the Server Management (SM) Command Line Protocol (CLP) Specification, the SM Managed Element Addressing Specification, the SM CLP-to-CIM Mapping Specification, the SMASH Implementation Requirements Specification, and the SMASH Architecture White Paper, as well as over 30 DMTF Management Profiles.

SMASH specifies both a web services and command line protocols for server management. For programmatic access by management software, such as system management consoles, SMASH specifies the Web Services for Management Protocol (WS-Man). For management by a administrator using command line tools or scripts, SMASH also defines a command line protocol or SMASH CLP. Independent of these protocols, the profiles that can be implemented for SMASH remain the same, ensuring consistency of server management.

SMI-S

SMI-S, or the Storage Management Initiative – Specification, is a storage standard developed and maintained by the Storage Networking Industry Association (SNIA) http://www.snia.org/. Therefore this standard is out of DMTF organization, however is a partner organization.

SMI-S is based upon the CIM and the WBEM standards defined by the DMTF, which define management functionality via HTTP. SMI-S defines CIM management profiles for storage systems. The complete SMI Specification is categorized in profiles and subprofiles. A profile describes the behavioral aspects of an autonomous, self-contained management domain. SMI-S includes profiles for Arrays, Switches, Storage Virtualizers, Volume Management and many other domains. In DMTF parlance, a provider is an implementation for a specific profile. A subprofile describes part of the domain, which can be a common part in many profiles.

At a very basic level, SMI-S entities are divided into two categories:

  • Clients are management software applications that can reside virtually anywhere within a network, provided they have a communications link (either within the data path or outside the data path) to providers.
  • Servers are the devices under management. Servers can be disk arrays, virtualization engines, host bus adapters, switches, tape drives, etc.

Note: StorageIM is a monitoring console for storage systems and networks designed around industry standards. StorageIM supports most storage arrays, switches, HBA's and storage libraries.http://www.storageim.com/

SFCB and OpenPegasus CIMOMs

The Common Information Model Object Manager (CIMOM) is a CIM object manager or, more specifically, an application that manages objects according to the CIM standard. CIMOM manages communication between CIMOM providers and a CIM client, where the administrator manages the system.

CIMOM providers are software performing specific tasks within the CIMOM that are requested by client applications. Each provider instruments one or more aspects of the CIMOM's schema. These providers interact directly with the hardware.

There are two open source CIMOM's with quite spreading in the scene, right now:

  • The Small Footprint CIM Broker (SFCB) - http://sourceforge.net/apps/mediawiki/sblim/index.php?title=Sfcb

    SFCB is a CIM Broker from the IBM-initiated open source project SBLIM. The SBLIM project is a collection of tools designed to support Web-Based Enterprise Management (WBEM). This project try to to map to all parts of the WBEM stack: CIM Broker, CIM providers set, CIM client development in Java, and C language, and so on. Probably the SBLIM project is a superset of the OpenLMI Project, because of the OpenLMI project is focusing only in the CIM providers set and CLI CIM client development.

  • The Open Group's OpenPegasus - https://collaboration.opengroup.org/pegasus/

    This is a C++ implementation of CIM Broker from the Open Group initiative.

Koncret CMPI tools: Simple CIM provider

KonkretCMPI, is an open-source tool for developing CMPI providers in the C programming language. This set of tools is strongly used in OpenLMI Project for the set of providers availables right now, there are other core OpenLMI providers which language of development is Python (using PyWBEM tool set).

From the KonkretCMPI documentation, it is dedicated to improving the productivity of CMPI developers while satisfying the following objectives:

  • Builds on CMPI rather than introducing a new provider interface.
  • Requires no other programming language other than C.
  • Introduces no runtime library dependencies.
  • Produces providers with a very small footprint.
  • Can be used with existing CMPI providers.

A very simple example of CMPI provider using KonkretCMPI is available in:

https://github.com/javiroman/openlmi-poc/tree/master/cmpi-providers-libs...

This example could be the entry point to understand the techology involved in the OpenLMI Providers set.

PyWBEM Python tools: Simple Python CIM client

This Python library for making CIM operations over HTTP is useful either for CIM providers or CIM client development. This library is the foundation of OpenLMI samples and CLI tools for CIM OpenLMI providers exploitation.

The following is a very simple example of CIM client for working with the CMPI provider from the below section.

https://github.com/javiroman/openlmi-poc/blob/master/cmpi-providers-libs...

SBLIM CIM Client for Java: Simple Java CIM client

On the other hand, we can use the Java language for CIM client development. For this porpoise we can use the SBLIM CIM Java library. Again we can use as sample the follwing example for the above CMPI provider.

https://github.com/javiroman/openlmi-poc/tree/master/cmpi-providers-libs...

The OpenLMI Project: DMTF in Red Hat

This section is a overview at higer level of the Fedora OpenLMI Project. This is a first contact with the project, so this is not a deep functional study, probably that's the target of the second document (OpenLMI Project - Part II).

This is a good point for comment about the precursor project of OpenLMI, probably the Matahari Project. That project was an IT management stack on top of Qpid from Apache (an implementation of MOM, Message-oriented middleware server). This e-mail explains the change in the policy of the development of this technology:

https://lists.fedorahosted.org/pipermail/matahari/2012-September/002331....

All of the tests were checked using Fedora 17 as base system. The following tables are a compilation of RPM packages in order to install the current OpenLMI stack, and the relevant dependencies for the project.

Fedora package Description
OpenLMI-fan CIM provider for controlling fans.
openlmi-networking CMPI providers for network management
openlmi-powermanagement Power management CIM provider
openlmi-providers OpenLMI-providers is set of (usually) small CMPI providers (agents) for basic monitoring and management of host system
openlmi-service CIM provider for controlling system services
openlmi-storage CMPI providers for management of storage. The providers can be registered in any CMPI-aware CIMOM, both OpenPegasus and SFCB were tested

Dependencies (direct relation with OpenLMI techology):

Fedora package Description
sblim-sfcb Small Footprint CIM Broker (sfcb) is a CIM server conforming to the CIM Operations over HTTP protocol. It is robust, with low resource consumption and therefore specifically suited for embedded and resource constrained environments. sfcb supports providers written against the Common Manageability Programming Interface (CMPI)
cim-schema Common Information Model (CIM) Schema. Common Information Model (CIM) is a model for describing overall management information in a network or enterprise environment. CIM consists of a specification and a schema. The specification defines the details for integration with other management models. The schema provides the actual model descriptions.
cmpi-bindings-pywbem Adapter to write and run CMPI-type CIM providers in Python. CMPI-compliant provider interface for Python
konkretcmpi KonkretCMPI makes CMPI provider development easier by generating type-safe concrete CIM interfaces from MOF definitions and by providing default implementations for many of the provider operations.
sblim-cmpi-base SBLIM (Standards Based Linux Instrumentation for Manageability) CMPI (Common Manageability Programming Interface) Base Providers for System-Related CIM (Common Information Model) classes.


OpenLMI Project: The source code

Downloanding the SRPM packages we can look a first sight the structure of the source code project.

rpmbuild/
├── SOURCES
│   ├── openlmi-networking-0.0.2.tar.gz
│   ├── openlmi-providers-0.0.1.tar.gz
│   ├── openlmi-storage-0.1.tar.gz
│   └── README.Fedora
├── SPECS
│   ├── openlmi-networking.spec
│   ├── openlmi-providers.spec
│   └── openlmi-storage.spec
└── SRPMS
      ├── openlmi-networking-0.0.2-1.fc17.src.rpm
      ├── openlmi-providers-0.0.1-2.fc17.src.rpm
      └── openlmi-storage-0.1-3.fc17.src.rpm

Those source packages corresponds to the following Git repositories:

http://git.fedorahosted.org/cgit/openlmi-networking.git
http://git.fedorahosted.org/cgit/openlmi-providers.git
http://git.fedorahosted.org/cgit/openlmi-storage.git

openlmi-providers.git

This information is directly extracted from the REAME files availables in the source code.

openlmi-providers is a set of (usually) small providers (agents) that will provide given functionality on host system. Those are the available providers in this sub-project:

  • Fan: Fan provider implements interface for hardware monitoring of fans
    in the system. It uses lm_sensors library so it's linux specific.

    There are three interfaces provided according to CIM model:
    * Linux_Fan
    - Provides instance for every fan founded by lm_sensors with basic
    information.
    * Linux_FanSensor
    - Associated sensor for each instance of Linux_Fan.
    - Value of fan speed can be obtained from property CurrentReading.
    * Linux_FanAssociatedSensor
    - Provides association of instances of above 2 providers.

  • Power: Power management provider allows to monitor and change power state of the
    computer system. Support commands:
    - halt, reboot, suspend, hibernate
  • Service: Service Provider that allows manipulation with system services.

    Supported commands:
    - status
    - start, stop, restart
    - enable, disable

Building dependencies: Each one of those providers run on top of a set of underlayers libraries or tools. Those are the dependencies:

All providers konkretcmpi-devel, sblim-cmpi-devel, cim-schema
Fan lm_sensors, lm_sensors-libs, lm_sensors-devel
Power glib2-devel, upower-devel or pm-utils
Service chkconfig, systemd or upstart or SysVinit

From kernel-labs recommend the installation of the OpenLMI project from the source code. Following the README instructions is easy the installation in the system. In a minimal Fedora 17 installation, before the building of the project, we have to install those packages:

$ yum group install “Development Tools”
$ yum install cmake
$ yum install konkretcmpi-devel sblim-cmpi-devel cim-schema
$ yum install lm_sensors lm_sensors-libs lm_sensors-devel
$ yum install glib2-devel upower-devel
$ yum install libuser-devel
$ yum install python-setuptools (for Python cli-tools)

After the building of the source code, the installation looks like this scheme:

/usr/
├── include
│   └── openlmi
│       └── globals.h
├── lib64
│   ├── cmpi
│   │   ├── libcmpiCura_Account.so
│   │   ├── libcmpiCura_Fan.so
│   │   ├── libcmpiCura_PowerManagement.so
│   │   └── libcmpiCura_Service.so
│   ├── libcuracommon.so -> libcuracommon.so.0.0.1
│   └── libcuracommon.so.0.0.1
├── libexec
│   ├── servicedisc.sh
│   └── serviceutil.sh
├── bin
│   ├── cura-user.py
│   ├── cura-power.py
│   └── cura-service.py
`── share
    ├── cmake
    │   └── Modules
    │       ├── CuraMacros.cmake
    │       ├── FindCMPI.cmake
    │       └── FindKonkretCMPI.cmake
    `── cura-providers
        ├── LMI_Account.mof
        ├── LMI_Account.reg
        ├── LMI_Fan.mof
        ├── LMI_Fan.reg
        ├── LMI_PowerManagement.mof
        ├── LMI_PowerManagement.reg
        ├── LMI_Service.mof
        ├── LMI_Service.reg
        ├── reg2pegasus.py
        └── register.sh

openlmi-networking.git

This CIM provider is based on SMASH specification of DMTF. The underlaying technology is the NetworkManager http://projects.gnome.org/NetworkManager/ solution.

Packages in a Fedora 17 minimal install for building from source code.

$ yum install check-devel
$ yum install NetworkManager-glib-devel

openlmi-storage.git

OpenLMI-Storage is CIM provider for storage management on Linux. Currently it has SMI-S like API.

The provider is written in Python and based on Anaconda installer storage module. Therefore it is probably usable only on Red Hat based distros, right now.

Conclusion

As conclusion this is an exciting initiative for one of the big contributors of open source code in the IT scene, Red Hat and its community project Fedora. In this first article the OpenLMI Project is only revised a little bit, so the main work is delayed for the second part of this study.