PythonG3Ma

From GEANT2-JRA1 Wiki

1 Python G3 MA

Python G3 MA

Python G3 MA accepts requests for link utilization in the same format as RRD MA. But Python G3 MA does not retrieve data from local RRD files filled-in by SNMP requests. Instead, Python G3 MA interfaces to G3 application, which is an extensive SNMP-based monitoring application developed by Tom Kosnar from CESNET.

Among many other features, G3 system (which is a next-generation version of its GTDMS II predecesor) uses a concept of "logical interfaces" (this concept is introduced by me so any imprecision is my fault and not Tom's) which can be defined by any complex condition on any objects related to physical interfaces, which can be obtained by SNMP. For instance, you can define a logical interface by specifying strings to be included in ifDescr, ifType and ifAdminStatus objects.

G3 then correctly finds interfaces going from one network node to the other even when router cards are added, removed, cables are swapped or when more physical interfaces go to the same destination for load balancing or backup. (the measured characteristics are then correctly summed up from all interfaces).

This capability significantly enhances reliability of measurement results. After all, we usually want to know what is the load on "a network connection going from city A to city B", rather than on "interface Giga 9/43.8 on router R105".

For the description of G3 user interface see CESNET Technical Report 9/2005.

[edit]

Installation

The following description assumes that you installed Python G3 MA into /usr/local/perfSONAR-MA-G3 directory.

Follow installation instructions for Python RRD MA, but download perfSONAR-MA-G3-0.1.tgz instead of PerfSONAR-MA-0.1-20.tar.gz.

Edit service/etc/service.properties file and set service.ma.linkload.commandpath property to the absolute path of the link_load.sh script, which is normally in tests subdirectory of Python G3 MA installation directory. For example:

service.ma.linkload.commandpath=/usr/local/perfSONAR-MA-G3/tests/link_load.sh

You will need working installation of G3 system including line_usage_exporter interface utility.

[edit]

Running server

cd /usr/local/perfSONAR-MA-G3
./service/bin/startup.sh -d -p8090 -l DEBUG

Note that server logs messages about its operation to service/log/perfsonar.log file.

[edit]

Running client

cd /usr/local/perfSONAR-MA-G3
./client/perfsonar_httplib_client.py -a localhost -p 8090 \
       -f client/schema/example-instances/sonar/SetupDataRequest-LinkLoadSelect.xml

The request used in the above example should produce response included in schema/example-instances/sonar/SetupDataResponse-LinkLoadSelect.xml file.

The response file includes very long lines, which are difficult to read in a terminal window. But you can nicely display this XML message by opening it in a web browser (tested with Mozilla).

Computing averages over specified time granularity is not yet implemented inside Python G3 MA (see end of service/lib/SONARpyMALinkLoad/message/services/ma/linkload_ma_message_handler.py). That is granularity parameter is ignored.

Program does not generate exceptions, errors are written to a log file and empty result is returned.

[edit]

Interface to G3 system

Python G3 MA calls a remote command on the host running G3 system to retrieve measurement results. This command looks as follows:

ssh username@host_running_g3 line_usage_exporter { arguments }

Required arguments:

--from_pop="link from this PoP"
--to_pop="link to this PoP"

Currently results for the links between the following CESNET PoPs (Points of Presence) can be requested:

Brno - Ceske Budejovice
Brno - Olomouc
Brno - Ostrava
Brno - Zlin
Ceske Budejovice - Plzen
Hradec Kralove - Liberec
Liberec - Usti nad Labem
Olomouc - Hradec Kralove
Olomouc - Ostrava
Olomouc - Zlin
Pardubice - Hradec Kralove
Praha - Brno
Praha - GEANT2
Praha - Hradec Kralove
Praha - NIX
Praha - Pardubice
Praha - Plzen
Praha - US
Praha - Usti nad Labem

Note that NIX (Czech interconnection of network providers) is connected via two physical interfaces, whose measurements are correctly summed together.

Optional arguments:

--start_time="results from this time" (default="-1 hour")
--end_time="results to this time" (default="now")

Time can be specified as UNIX timestamp (seconds since 00:00:00 UTC, January 1, 1970) or in format accepted by rrdtool fetch command.

--items_of_interest="characteristics"

This argument can be specified multiple times to request multiple characteristics separated by spaces. Currently the following characteristics are available:

bitrate (default)
pktrate
line_speed (installed capacity)
line_speed_administrative (administratively constrained installed capacity)

--granularity="seconds"

This argument specifies time period over which average values will be computed and returned as results.

Example1:

ssh username@g3.cesnet.cz line_usage_exporter \
--items_of_interest=bitrate \
--items_of_interest=pktrate \
--from_pop=Praha \
--to_pop="Usti nad Labem" \
--start_time='-24 hour' \
--end_time='-10 min'

Example2:

ssh username@g3.cesnet.cz line_usage_exporter \
--from_pop=Praha \
--to_pop=US \
--start_time=-14400 \
--end_time=-300

[edit]

CESNET G3 MA

CESNET G3 MA can be contacted by client (see example above) at host perfmon1.cesnet.cz and port 8091. You can use client included either in Python G3 MA distribution or client included in Python RRD MA distribution, it does not matter. It should probably also work with client from Java RRD MA distribution, but I did not test that.

The G3 MA accepts requests in the same format as RRD MA where <nmwgt:ifDescription>element should include some of the PoP pairs listed above (replace spaces in town names with underscores, for example, Ceske_Budejovice). You can start from a complete example of a working message with minimal needed information (replace timestamps with some recent values). Names of towns are also visible on this [CESNET2 load map] (this is map is currently produced from GTDMS II, which is a predecesor of G3).

This service is not opened to the whole Internet. If you want to connect to it, send me your subnet.

[edit]