PerfSONAR PS Perl LS Installation

From GEANT2-JRA1 Wiki

Contents 1 Information 2 Introduction 3 Caveats 4 Installation 4.1 Automatic 4.1.1 configure.PL Script 4.1.1.1 Post Install 4.1.1.2 Known Issues 4.1.1.2.1 Perl Modules 4.2 Manual 4.2.1 Prerequisites 4.2.1.1 Oracle Berkeley DB XML 4.2.1.1.1 Known Issues 4.2.1.2 Perl Modules 4.2.1.2.1 Known Issues 5 LS 5.1 Configuration 6 Testing 7 Running 8 References

Information

Version(s)	RC1 - RC6
Date	November 5, 2007
Author	Jason Zurawski

Introduction

This document describes the installation procedures of the perfSONAR-PS Lookup Service (LS). perfSONAR-PS is an implementation of the perfSONAR standard written in the Perl programming language. 1 2 perfSONAR-PS aims for complete compatibility with all other implementations of the perfSONAR standard.

This software offers the ability to receive registrations from services in the perfSONAR framework. Each service will send a request message to the deployed LS instance that contains information about itself (contact information, etc.) and information about all internal data is has access to. The data must be refreshed on a regular basis so the LS doesn't delete it for being stale. Additionally, the information can be added to, changed, or deleted by services.

Other services and client applications will make queries to the LS in an attempt to figure our where to look for specific data. These queries will be delivered in the form of XQuery/XPath expressions directly to the LS.

The LS is built to use the Oracle (Formerly Berkeley) DB XML 3. Currently, this is the ONLY XML database technology that we support. Instructions for installing this software are available later in this document.

Caveats

This software represents the first widespread release of perfSONAR-PS. As such we are still experimenting with packaging and delivery methods that will best fit the user community.

This release is being presented as a stand-alone application. Other perfSONAR-PS services will be presented the same way, and do not share the underlying libraries. Future releases WILL share the core code, and will be available through the more accessible and standardized Comprehensive Perl Archive Network (CPAN) tool.

We thank the user community in advance for bearing with this flux in installation procedure and still look forward to hearing comments that will make the process of obtaining and installing perfSONAR code much faster and easier.

Installation

Before installing the LS, be sure to check if currently have the software listed in the prerequisite sub-section. Software versions may change from prior releases of perfSONAR-PS without explicit notice.

Automatic

An automatic installation script has been developed that will take care of obtaining libraries and software needed for this application.

configure.PL Script

To execute the script:

     
(sudo) perl configure.PL

The script will check for and attempt to install Berkeley DB XML for you. This process may fail if not all of the supporting libraries for Berkeley DB XML are found. Please read the rest of this document for more information. The script can be run repeatedly to find issues with no harm to the system.

The script will also check for required perl libraries and use CPAN to install any that are missing or old. The script will report any errors that CPAN encountered due to missing components.

Post Install

Oracle Berkeley DB XML does require knowledge of certain environmental paths. If your installation was in /usr/local/dbxml-2.3.10:

      
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/dbxml-2.3.10/lib
export PATH=$PATH:/usr/local/dbxml-2.3.10/bin

Known Issues

In addition to the issues regarding the installation of Berkeley DB XML, these issues may arise:

Perl Modules

Some modules require software libraries that may not be installed on most systems. Although it is normally the job of the module designer to take care of these situations, this is not always the case. Specifically the XML modules (XML::XPath, XML::LibXML) require the the XML::Parser module that depends on several internal libraries.

If you system is equipped with a software management system (such as up2date, yum, or apt) try and locate the xml-parser-perl package via the search interface. This is the easiest way to avoid trouble with CPAN. Additionally, ensure that libxml2-dev, the LibXML version 2 development libraries, are also available.

Manual

The software can be installed without the use of the configure script. If this script also fails it is possible to follow these steps.

Prerequisites

To ensure proper installation and functioning, the LS requires two key areas of prerequisite software:

Oracle Berkeley DB XML

Oracle Berkeley DB XML is an open source, embeddable XML database with XQuery-based access to documents stored in containers and indexed based on their content. Oracle Berkeley DB XML is built on top of Oracle Berkeley DB and inherits its rich features and attributes. Like Oracle Berkeley DB, it runs in process with the application with no need for human administration. Oracle Berkeley DB XML adds a document parser, XML indexer and XQuery engine on top of Oracle Berkeley DB to enable the fastest, most efficient retrieval of data. 3

Download and unpack DBXML:

wget -c http://download.oracle.com/berkeley-db/dbxml-2.3.10.tar.gz
tar -zxvf dbxml-2.3.10.tar.gz

From inside of the install directory, install DBXML:

sudo ./buildall.sh --enable-perl --prefix=/usr/local/dbxml-2.3.10

Known Issues

Oracle Berkeley DB XML does require knowledge of certain environmental paths. If your installation was in /usr/local/dbxml-2.3.10:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/dbxml-2.3.10/lib
export PATH=$PATH:/usr/local/dbxml-2.3.10/bin

Perl Modules

There are several Perl modules that are required for the LS to function properly. Some of these are included in the base Perl installation on many machines, although it is important to check all to be sure upgrades are not available.

The most straightforward way to install Perl modules is through the CPAN administrative interface available on most machines:

sudo perl -MCPAN -e shell

If this is your first time using CPAN, there will be several questions to answer as you prepare the interface. You will eventually be presented with a shell:

cpan[1]>

Modules may be installed by typing:

      
cpan[1]>install MODULE::NAME

Instructions may appear as the installation proceeds, errors may also occur that will need to be handled. If a module is installed, a message may appear:

  
cpan[1]> install MODULE::NAME
MODULE::NAME is up to date (1.9707).   

This is a list of the necessary modules for the LS:

• warnings
• strict
• Exporter
• Cwd
• POSIX
• Fcntl
• Getopt::Long
• Log::Log4perl
• Params::Validate
• Log::Dispatch
• Log::Dispatch::Screen
• Log::Dispatch::FileRotate
• Data::Dumper
• XML::LibXML
• XML::XPath
• Time::HiRes
• Time::Local
• IO::File
• HTTP::Daemon
• LWP::UserAgent
• File::Temp
• File::Basename

Known Issues

Some modules require software libraries that may not be installed on most systems. Although it is normally the job of the module designer to take care of these situations, this is not always the case. Specifically the XML modules (XML::XPath, XML::LibXML) require the the XML::Parser module that depends on several internal libraries.

If you system is equipped with a software management system (such as apt, yum, or up2date) try and locate the xml-parser-perl package via the search interface. This is the easiest way to avoid trouble with CPAN. Additionally, ensure that libxml2-dev, the LibXML version 2 development libraries, are also available.

LS

After installing the prerequisite software, the remaining steps to getting the LS working are few. The software directory should be placed in a common location such as /usr/local or /opt. This is not a requirement.

There are three major configuration files that may be adjusted, although the software is configured to run out of the box. The configurations files can be adjusted to personalize settings.

Configuration

• DB_CONFIG - This is a list of directivies for the Berekeley DB XML environment. Another copy of this file lives in the util/ directory. This file *MUST* be included in the xmldb/ folder with each running or 'bad things will happen' (TM).
• logger.conf - This file contains information regarding logging in the perfSONAR-PS architecture. There is a choice between three methods:
  • log4perl.appender.A1=Log::Dispatch::Screen - sends log files to the output screen
  • log4perl.appender.A1=Log::Dispatch::Syslog - append the log messages directly to the Syslog system
  • log4perl.appender.A1=Log::Dispatch::FileRotate - appends log messages to a specified log file (features rotation of old files)
• ls.conf - This file contains detailed information and settings that help operate the LS. Here is a listing of the major directives, and their purpose. Defaults are included in the initial file, and for most instances will work fine:
  • METADATA_DB_TYPE - 'Type' of the metadata database, this should be 'xmldb' (and only xmldb) for this particular version.
  • METADATA_DB_NAME - Path to the location of the xmldb directory.
  • METADATA_DB_FILE - Location of the data storage collection.
  • PORT - Listen port for the application. This should be something that can be accessed from the outside world.
  • ENDPOINT - An 'endPoint' is a contact string that a service is listening on. This is used in conjunction with the port and hostname: http://HOST:PORT/end/Point/String
  • LS_REGISTRATION_INTERVAL - The interval of time (in seconds) that this service will contact the LS_INSTANCE to register itself.
  • LS_INSTANCE - The LS that will be contacted to register this service.
  • LS_TTL - The time to live interval of data in the LS.
  • LS_REAPER_INTERVAL - The frequency the LS will clean itself.
  • SERVICE_NAME - String describing the 'name' of this LS.
  • SERVICE_ACCESSPOINT - String describing the 'accessPoint' of this LS. This string is how the service can be reached: http://HOST:PORT/end/Point/String
  • SERVICE_TYPE - String describing the 'type' of this LS.
  • SERVICE_DESCRIPTION - String describing this LS, it can include location info, etc.
  • MAX_WORKER_PROCESSES - Maximum number of child processes that can be spawned at a given time.
  • MAX_WORKER_LIFETIME - Maximum lifetime a child has to do work (eliminates hung children)

A helper utility named 'configure.pl' (located in util/) can help create th-is file by providing questions regarding each value.

Testing

As stated previously, the LS can run essentially out of the box (locally at least). You will need two terminal sessions to test out the LS, one for the server and the other for a client application (included).

In the first terminal:

Enter the directory where the software is installed, run this command:

      
./ls.pl --verbose

You may see output like this:

Can't write pidfile to /var/run

This is a warning letting you know that you don't have permission to use the default pid file location (/var/run). Normally only the root user will have the ability to do this, so a new pid directory must be tried. Try this:

./ls.pl --verbose --piddir=/tmp

This will write the pid file to /tmp. You will now see output similar to this:

    
Use of uninitialized value in scalar chomp at ./ls.pl line 237.
Use of uninitialized value in string ne at ./ls.pl line 238.
2007/10/26 11:10:10 (24852) DEBUG> ls.pl:163 main:: - Starting '24852'
2007/10/26 11:10:10 (24852) DEBUG> Transport.pm:166 perfSONAR_PS::Transport::startDaemon - Starting daemon.
2007/10/26 11:10:10 (24852) ERROR> LS.pm:108 perfSONAR_PS::LS::LS::init - Setting 'LS_REAPER_INTERVAL' to '0'.
2007/10/26 11:10:10 (24852) DEBUG> ls.pl:180 main:: - Not registering with an LS instance
2007/10/26 11:10:10 (24867) DEBUG> ls.pl:322 main::lookupService - Starting '24867' as the LS.
2007/10/26 11:10:10 (24867) DEBUG> Transport.pm:187 perfSONAR_PS::Transport::acceptCall - Accepting calls.

The server is started, and it has stared its major functions: registering data with an LS and listening for calls. The LS registration functionality is disabled by default.

In the second terminal:

Enter the directory where the software is installed, change into the client directory, and run this command:

./client.pl --server=localhost --port=8080 --endpoint=perfSONAR_PS/services/LS requests/EchoRequest.xml

This will tell the client to contact the LS which is running on:

    
http://localhost:8080/perfSONAR_PS/services/LS

And will send a simple Echo message (to see if the server is active). The client will see this output if everything is running smoothly:

2007/10/26 11:12:04 (25250) DEBUG> Transport.pm:145 perfSONAR_PS::Transport::getHttpURI - Created URI: http://localhost:8080/perfSONAR_PS/services/LS
2007/10/26 11:12:04 (25250) DEBUG> Transport.pm:346 perfSONAR_PS::Transport::sendReceive - Sending information to "http://localhost:8080/perfSONAR_PS/services/LS".
2007/10/26 11:12:04 (25250) DEBUG> Transport.pm:364 perfSONAR_PS::Transport::sendReceive - Response returned.
<?xml version="1.0"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
  <SOAP-ENV:Header/>
  <SOAP-ENV:Body>
    <nmwg:message xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" id="message.7499368" messageIdRef="id1" type="EchoResponse">
      <nmwg:metadata xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" id="metadata.9659137" metadataIdRef="meta">
        <nmwg:eventType>success.echo</nmwg:eventType>
      </nmwg:metadata>
      <nmwg:data xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" id="data.14453530" metadataIdRef="metadata.9659137">
        <nmwgr:datum xmlns:nmwgr="http://ggf.org/ns/nmwg/result/2.0/">The echo request has passed.</nmwgr:datum>
      </nmwg:data>
    </nmwg:message>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

If you are not seeing a similar message, you may see something such as this:

2007/10/26 11:13:58 (25564) DEBUG> Transport.pm:145 perfSONAR_PS::Transport::getHttpURI - Created URI: http://localhost:8080/perfSONAR_PS/services/LS
2007/10/26 11:13:58 (25564) DEBUG> Transport.pm:346 perfSONAR_PS::Transport::sendReceive - Sending information to "http://localhost:8080/perfSONAR_PS/services/LS".
2007/10/26 11:13:58 (25564) DEBUG> Transport.pm:356 perfSONAR_PS::Transport::sendReceive - Send to "http://localhost:8080/perfSONAR_PS/services/LS" failed.
Empty String at ./client.pl line 106

This message indicates that the server is not properly listening on the port you are contacting, perhaps it is listening on another, the port is firewalled, or it may not be started at all. Please check the above steps, and try again.

In the first terminal:

This information will appear when the call comes in:

2007/10/26 11:12:04 (25199) INFO> Transport.pm:198 perfSONAR_PS::Transport::acceptCall - Received call from:  127.0.0.1
2007/10/26 11:12:04 (25199) DEBUG> Common.pm:281 perfSONAR_PS::Common::reMap - Setting namespace "http://schemas.xmlsoap.org/soap/envelope/" with prefix "SOAP-ENV".
2007/10/26 11:12:04 (25199) DEBUG> Common.pm:281 perfSONAR_PS::Common::reMap - Setting namespace "http://ggf.org/ns/nmwg/base/2.0/" with prefix "nmwg".
2007/10/26 11:12:04 (25199) DEBUG> Request.pm:108 perfSONAR_PS::Request::parse - 
<?xml version="1.0"?>
<nmwg:message xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" type="EchoRequest" id="id1">
  <nmwg:metadata xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" id="meta">
    <nmwg:eventType>http://schemas.perfsonar.net/tools/admin/echo/ls/2.0</nmwg:eventType>
  </nmwg:metadata>
  <nmwg:data xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" id="data" metadataIdRef="meta"/>
</nmwg:message>
2007/10/26 11:12:04 (25199) DEBUG> Request.pm:183 perfSONAR_PS::Request::setResponse - <?xml version="1.0"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
  <SOAP-ENV:Header/>
  <SOAP-ENV:Body>
    <nmwg:message xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" id="message.7499368" messageIdRef="id1" type="EchoResponse">
      <nmwg:metadata xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" id="metadata.9659137" metadataIdRef="meta">
        <nmwg:eventType>success.echo</nmwg:eventType>
      </nmwg:metadata>
      <nmwg:data xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/" id="data.14453530" metadataIdRef="metadata.9659137">
        <nmwgr:datum xmlns:nmwgr="http://ggf.org/ns/nmwg/result/2.0/">The echo request has passed.</nmwgr:datum>
      </nmwg:data>
    </nmwg:message>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>
2007/10/26 11:12:04 (25199) DEBUG> LS.pm:131 perfSONAR_PS::LS::LS::receive - Received 'shadow' request from below; no action required.
2007/10/26 11:12:04 (25199) INFO> Request.pm:271 perfSONAR_PS::Request::finish - Seconds required to handle request from 127.0.0.1: 0
2007/10/26 11:12:04 (25199) DEBUG> Request.pm:275 perfSONAR_PS::Request::finish - Closing call.
2007/10/26 11:12:04 (25199) DEBUG> Transport.pm:187 perfSONAR_PS::Transport::acceptCall - Accepting calls.

The other messages in the client/requests may also be tried, with similar results.

Running

The software can be run as:

  
./ls.pl

This will background the process automatically. Be sure to adjust your logging settings in logger.conf.

References

• 1 perfSONAR
• 2 The Perl Directory
• 3 Oracle Berkeley DB XML