Chapter 8. Customizing and Extending PCP Services

Performance Co-Pilot (PCP) has been developed to be fully extensible. The following sections summarize the various facilities provided to allow you to extend and customize PCP for your site:

This chapter describes ways of extending PCP services using tools and services made available by the pcp product. For information on how to extend and customize PCP, see Chapter 8, “Customizing and Extending PCP Services”, in Performance Co-Pilot User's and Administrator's Guide.

PMDA Customization

The summary PMDA is a special case that warrants further discussion.

Customizing the Summary PMDA

The summary PMDA exports performance metrics derived from performance metrics made available by other PMDAs. It is described completely in the pmdasummary(1) man page.

The summary PMDA consists of two processes:

pmie process 

Periodically samples the base metrics and compute values for the derived metrics. This dedicated instance of the PCP pmie inference engine is launched with special command line arguments by the main process.

main process 

Reads and buffers the values computed by the pmie process and makes them available to the Performance Metrics Collection Daemon (PMCD).

All of the metrics exported by the summary PMDA have a singular instance and the values are instantaneous; the exported value is the correct value as of the last time the corresponding expression was evaluated by the pmie process.

The summary PMDA resides in the /usr/pcp/pmdas/summary directory and may be installed with a default configuration by following the steps described in Chapter 2, “Installing and Configuring Performance Co-Pilot ”.

Alternatively, you may customize the summary PMDA to export your own derived performance metrics by following the steps in Procedure 8-1:

Procedure 8-1. Customizing the Summary PMDA

  1. Check that the symbolic constant SYSSUMMARY is defined in the /var/pcp/pmns/stdpmid file. If it is not, perform the postinstall update of this file, as superuser:

    # cd /var/pcp/pmns
            # ./Make.stdpmid

  2. Choose Performance Metric Name Space (PMNS) names for the new metrics. These must begin with summary and follow the rules described in the pmns(4) man page. For example, you might use summary.fs.cache_write and summary.fs.cache_hit.

  3. Edit the pmns file in the /usr/pcp/pmdas/summary directory to add the new metric names in the format described in the pmns(4) man page. You must choose a unique performance metric identifier (PMID) for each metric. In the pmns file, these appear as SYSSUMMARY:0:x . The value of x is arbitrary in the range 0 to 1023 and unique in this file.

    For example:

    summary {
              fs              /*new*/
    summary.fs {
              cache_write     SYSSUMMARY:0:10
              cache_hit       SYSSUMMARY:0:11

  4. Use the local test PMNS root and validate that the PMNS changes are correct.

    For example, enter this command:

    pminfo -n root -m summary.fs

    You see output similar to the following:

    summary.fs.cache_write PMID: 27.0.10
    summary.fs.cache_hit PMID: 27.0.11

  5. Edit the /usr/pcp/pmdas/summary/expr.pmie file to add new pmie expressions. If the name to the left of the assignment operator (=) is one of the PMNS names, then the pmie expression to the right will be evaluated and returned by the summary PMDA. The expression must return a numeric value.

    For example, consider this expression:

    // filesystem buffer cache hit percentages
    prefix = "";          // macro, not exported
    summary.fs.cache_write =
                    100 - 100 * $prefix.bwrite / $prefix.lwrite;
    summary.fs.cache_hit =
                    100 - 100 * $prefix.bread / $prefix.lread;

  6. Run pmie in debug mode to verify that the expressions are being evaluated correctly, and the values make sense.

    For example, enter this command:

    pmie -t 2sec -v expr.pmie

    You see output similar to the following:

    summary.fs.cache_write:      ?
    summary.fs.cache_hit:      ?
    summary.fs.cache_write:  45.83
    summary.fs.cache_hit:   83.2
    summary.fs.cache_write:  39.22
    summary.fs.cache_hit:  84.51

  7. Install the new PMDA.

    From the /usr/pcp/pmdas/summary directory, use this command:


    You see the following output:

    You need to choose an appropriate configuration for installation of
    the “summary” Performance Metrics Domain Agent (PMDA).
    collector collect performance statistics on this system
    monitor   allow this system to monitor local and/or remote systems
    both      collector and monitor configuration for this system
    Please enter c(ollector) or m(onitor) or b(oth) [b] both
    Interval between summary expression evaluation (seconds)? [10] 10
    Updating the Performance Metrics Name Space...
    Installing pmchart view(s) ...
    Terminate PMDA if already installed ...
    Installing files ..
    Updating the PMCD control file, and notifying PMCD ...
    Wait 15 seconds for the agent to initialize ...
    Check summary metrics have appeared ... 8 metrics and 8 values

  8. Check the metrics.

    For example, enter this command:

    pmval -t 5sec -s 8 summary.fs.cache_write

    You see a response similar to the following:

    metric:    summary.fs.cache_write
    host:      localhost
    semantics: instantaneous value
    units:     none
    samples:   8
    interval:  5.00 sec

    Note that the values are being sampled here by pmval every 5 seconds, but pmie is passing only new values to the summary PMDA every 10 seconds. Both rates could be changed to suit the dynamics of your new metrics.

  9. You may now create pmchart views, pmview scenes, and pmlogger configurations to monitor and archive your new performance metrics.

    PCP Tool Customization

    Stripchart Customization

    The PCP tool pmchart produces stripchart displays of performance metrics. Refer to “The pmchart Tool” in Chapter 4, for an extensive description of the capabilities of pmchart.

    Customization is centered on PCP views that may be created interactively and saved via the Save View option in the File menu.

    When pmchart is loading a view, the following directories are searched:


    The current directory.


    Views for each user.


    The system-wide catalog of views. Any view installed here becomes visible to every pmchart user.

    The X11 application resources for pmchart are in /usr/lib/X11/app-defaults/PmChart, and these may be edited to customize the appearance of the display. The default update interval and other attributes are described in the pmchart(1) man page.

    Inference Engine Customization

    The PCP inference engine is presented in Chapter 6, “Performance Metrics Inference Engine”, and documented in the pmie(1) man page.

    The following resources are available to aid customizing pmie :


    Example pmie rules that may be used as a basis for developing local rules.

    Snapshot Customization

    The PCP snapshot production facility is presented in “Making Snapshot Images from Archive Logs” in Chapter 7, and documented in the pmsnap(1) man page.

    The following global files and directories influence the behavior of pmsnap:


    Defines how to produce a snapshot, including the output filename, the PCP archive folio name to be used as input, the pmchart configuration file, and command line arguments to pmchart.


    A pmchart configuration file to produce a sample summary snapshot in conjunction with pmsnap.


    A pmlogger configuration file that can produce an archive containing performance metrics required by the sample summary snapshot.


    Prototype crontab entries that may be merged with the crontab entries for root schedule the periodic execution of the archive log management scripts, for example, pmsnap.


    An example HTML page suitable for publishing images from the pmsnap examples via a Web server.

    Icon Control Panel Customization

    The gadget specification language of pmgadgets supports the creation of arbitrary gadget layouts and bindings to hosts and performance metrics. See “The pmgadgets Command” in Chapter 4, and the pmgadgets (1) man page.

    3D Visualization Customization

    The 3D scene specification language of pmview supports the creation of block layouts and bindings to hosts and performance metrics. See Chapter 5, “System Performance Visualization Tools” and the pmview(1) man page.

    PMDA Development

    Performance Co-Pilot (PCP) is designed to be extensible at the collector site.

    Application developers are encouraged to create new PMDAs to export performance metrics from the applications and service layers that are particularly relevant to a specific site, application suite, or processing environment.

    These PMDAs use the routines of the libpcp_pmda library, which is discussed in detail by the Performance Co-Pilot Programmer's Guide.

    Source code for several PMDAs (simple, trivial, and txmon) is provided in the pcp.sw.demo subsystem. When it is installed, all of the relevant files reside in directories (one per PMDA) below the /var/pcp/pmdas directory.

    PCP Tool Development

    Performance Co-Pilot (PCP) is designed to be extensible at the monitor site.

    Application developers are encouraged to create new PCP client applications to monitor or display performance metrics in a manner that is particularly relevant to a specific site, application suite, or processing environment.

    Client applications use the routines of the PMAPI (performance metrics application programming interface) described in the Performance Co-Pilot Programmer's Guide.

    Source code for a sample PMAPI client (pmclient) is provided in the pcp.sw.demo subsystem, and when installed all of the relevant files reside in /var/pcp/demos/pmclient.