RRD MA Stitching guide

From GEANT2-JRA1 Wiki

This is the guide to stitch (configure) your RRD MA to provide metadata information about your network interfaces and work with the RRD files

Contents

[edit]

Stitching for perfSONAR Bundle 2.2

Objective

  • To create a metadata configuration file for your network which will expose information about all the rrd files that you wish to make available to the measurement archive

Requirements before using this guide

  1. Sample file for you to follow - available here
  2. The metadata configuration file guide - available here
  3. For the data that you want to expose, you will need the full path and name of the RRD files that store this data.
    1. For each such file,
    2. The name(s) of the datasource in the RRD file. For each datasource, you also need to know
      1. The unit in which data is stored (bps or Bps)
      2. Name of the interface for which data is stored in the rrd file's datasource (Example: so-1/0/0 which could be an IP interface on a router)
      3. Description of the interface - usually configured on the ifConfig (ex: Link to GÉANT2)
      4. IPv4 address of the interface
      5. The traffic direction (in/out) being monitoring on each interface and being stored in each data source
      6. Capacity/Interface speed (units = bits per second or the units which are using for storing data in the rrd file)
      7. DNS entry of the Router/network equipment which contains this interface
      8. Authentication Realm or authRealm - Usually the domain name or NREN name for this router (ex: GÉANT2, RENATER)


[edit]

Step 1 - Understanding the configuration file structure

  • Take a look at the metadata configuration file guide and the sample file. Try to understand the general logic of the file
    • Remember that each metadata-data chain represents one datasource in a rrd file.
    • You will have to create as many metadata-data chains as the number of data sources that you want to expose in all the rrd files put together. Usually, a measurement of one direction of traffic flow on one interface = one metadata-data chain
    • You can have as many chains as you want in your metadata configuration file. There are currently no limits.
[edit]

Step 2 - Creating your own metadata configuration file

  • Either with the help of the the contribution scripts and programs or your own scripts, you might be able to generate these files easily. These contribution scripts are available here.
  • Remember that whenever something changes in your network, the metadata configuration file will need to be updated. It is hence worth automating the process which will periodically contact the network elements/routers and automatically generate the file or which will use the contrib tools to do that for you periodically.
  • Manually creating these files for a large number of interfaces is not recommended.
[edit]

Step 3 - Applying your metadata configuration file to your installation

  1. Generate and Verify the xml configuration file
  2. There are three ways to apply your metadata configuration
    1. With the help of the exist xml database web based UI (recommended)
      1. Open the exist xml database web interface
        1. If you installed exist database on your tomcat as a web application you should be able to find the web interface by simply typing http://server_name:port/exist (for example: mu.dante.org.uk:8080/exist) where server_name is the dns/ip address of the server where you installed tomcat, exist and the service and port is the port on which tomcat is running. Scroll down the page and on the left menu bar, you can find the Admin button. Clicking on this button will give you a login screen. Use the username and password that you provided during the installation (NOT THE ADMIN USERNAME AND PASSWORD) and login to the database. Remember to log out after you are done. If not, exist won't be happy and will create problems when you try to log in again.
        2. If you installed exist database as a standalone application, you can use the exist xml database client application (from any machine. For example, your desktop) to access this database. You will need to provide your server_name, port, username and password
      2. Once you have logged into the exist database, you should be able to locate the collection for rrdmaconfig. Double click on this collection.
      3. If you find previous xml files in the collection, remove them (maybe you want to back it up somewhere?) and add your new file with the help of the menus available on the screen.
      4. Confirm that your new xml file has been loaded and then log out
      5. This completes this step on applying your metadata configuration
    2. With the help of the installer
      1. Copy your metadata configuration file to the $perfSONAR-bundle/perfSONAR-RRD-MA-2.0/perfsonar/conf directory
      2. Start the perfSONAR bundle installer and choose the RRD MA service installation that you wish to modify
      3. Go through the 'configure' step
      4. During the configure step, you will be asked to specify the complete path of the metadata configuration file.
      5. Instead of choosing the default value for the metadata configuration, provide the complete path of your configuration file
      6. Complete the configure step and then follow the 'deploy' step via the 'modify your service' menu
      7. In the deploy step, make sure you ask the installer to update the xml database.
      8. Confirm that the xml file has been uploaded into the exist xml database by accessing its contents from the exist xml UI. See notes in the previous option (With the help of the exist xml database web based UI) described above.
    3. With the help of the service installation scripts (ant targets)
      1. Copy your metadata configuration file to the $perfSONAR-bundle/perfSONAR-RRD-MA-2.0/perfsonar/conf directory
      2. In the directory in $perfSONAR-bundle/perfSONAR-RRD-MA-2.0/perfsonar/conf, look for the service.properties file
      3. Open the file for editing and look for the property 'service.ma.conf_file'
      4. Modify its value - provide the complete path to your metadata configuration file (instead of the default configuration file)
      5. In the ant directory ($perfSONAR-bundle/perfSONAR-RRD-MA-2.0/perfsonar/ant), execute the following command
        1. ant xmldb-init-sqlma
      6. Confirm that the xml file has been uploaded into the xml database by following the first option above (With the help of the exist xml database web based UI). Instead of viewing the files in the xml database collection, you can simply view the files to see if your file has been added to the database


[edit]

Stitching for perfSONAR Bundle 2.0, 2.1

Objective

  • To create a metadata configuration file for your network which will expose information about all the rrd files that you wish to make available to the measurement archive

Requirements before using this guide

  1. Sample file for you to follow - available here
  2. The metadata configuration file guide - available here
  3. For the data that you want to expose, you will need the full path and name of the RRD files that store this data.
    1. For each such file,
    2. The name(s) of the datasource in the RRD file. For each datasource, you also need to know
      1. The unit in which data is stored (bps or Bps)
      2. Name of the interface for which data is stored in the rrd file's datasource (Example: so-1/0/0 which could be an IP interface on a router)
      3. Description of the interface - usually configured on the ifConfig (ex: Link to GÉANT2)
      4. IPv4 address of the interface
      5. The traffic direction (in/out) being monitoring on each interface and being stored in each data source
      6. Capacity/Interface speed (units = bits per second or the units which are using for storing data in the rrd file)
      7. DNS entry of the Router/network equipment which contains this interface
      8. Authentication Realm or authRealm - Usually the domain name or NREN name for this router (ex: GÉANT2, RENATER)


[edit]

Step 1 - Understanding the configuration file structure

  • Take a look at the metadata configuration file guide and the sample file. Try to understand the general logic of the file
    • Remember that each metadata-data chain represents one datasource in a rrd file.
    • You will have to create as many metadata-data chains as the number of data sources that you want to expose in all the rrd files put together. Usually, a measurement of one direction of traffic flow on one interface = one metadata-data chain
    • You can have as many chains as you want in your metadata configuration file. There are currently no limits.
[edit]

Step 2 - Creating your own metadata configuration file

  • Either with the help of the the contribution scripts and programs or your own scripts, you might be able to generate these files easily. These contribution scripts are available here.
  • Remember that whenever something changes in your network, the metadata configuration file will need to be updated. It is hence worth automating the process which will periodically contact the network elements/routers and automatically generate the file or which will use the contrib tools to do that for you periodically.
  • Manually creating these files for a large number of interfaces is not recommended.
[edit]

Step 3 - Applying your metadata configuration file to your installation

  1. Generate and Verify the xml configuration file
  2. There are three ways to apply your metadata configuration
    1. With the help of the exist xml database web based UI (recommended)
      1. Open the exist xml database web interface
        1. If you installed exist database on your tomcat as a web application you should be able to find the web interface by simply typing http://server_name:port/exist (for example: mu.dante.org.uk:8080/exist) where server_name is the dns/ip address of the server where you installed tomcat, exist and the service and port is the port on which tomcat is running. Scroll down the page and on the left menu bar, you can find the Admin button. Clicking on this button will give you a login screen. Use the username and password that you provided during the installation (NOT THE ADMIN USERNAME AND PASSWORD) and login to the database. Remember to log out after you are done. If not, exist won't be happy and will create problems when you try to log in again.
        2. If you installed exist database as a standalone application, you can use the exist xml database client application (from any machine. For example, your desktop) to access this database. You will need to provide your server_name, port, username and password
      2. Once you have logged into the exist database, you should be able to locate the collection for rrdmaconfig. Double click on this collection.
      3. If you find previous xml files in the collection, remove them (maybe you want to back it up somewhere?) and add your new file with the help of the menus available on the screen.
      4. Confirm that your new xml file has been loaded and then log out
      5. This completes this step on applying your metadata configuration
    2. With the help of the installer
      1. Copy your metadata configuration file to the $perfSONAR-bundle/perfSONAR-RRD-MA-2.0/perfsonar/conf directory
      2. Start the perfSONAR bundle installer and choose the RRD MA service installation that you wish to modify
      3. Go through the 'configure' step
      4. During the configure step, you will be asked to specify the complete path of the metadata configuration file.
      5. Instead of choosing the default value for the metadata configuration, provide the complete path of your configuration file
      6. Complete the configure step and then follow the 'deploy' step via the 'modify your service' menu
      7. In the deploy step, make sure you ask the installer to update the xml database.
      8. Confirm that the xml file has been uploaded into the exist xml database by accessing its contents from the exist xml UI. See notes in the previous option (With the help of the exist xml database web based UI) described above.
    3. With the help of the service installation scripts (ant targets)
      1. Copy your metadata configuration file to the $perfSONAR-bundle/perfSONAR-RRD-MA-2.0/perfsonar/conf directory
      2. In the directory in $perfSONAR-bundle/perfSONAR-RRD-MA-2.0/perfsonar/conf, look for the service.properties file
      3. Open the file for editing and look for the property 'service.ma.conf_file'
      4. Modify its value - provide the complete path to your metadata configuration file (instead of the default configuration file)
      5. In the ant directory ($perfSONAR-bundle/perfSONAR-RRD-MA-2.0/perfsonar/ant), execute the following command
        1. ant xmldb-init-sqlma
      6. Confirm that the xml file has been uploaded into the xml database by following the first option above (With the help of the exist xml database web based UI). Instead of viewing the files in the xml database collection, you can simply view the files to see if your file has been added to the database
Retrieved from "http://wiki.perfsonar.net/jra1-wiki/index.php/RRD_MA_Stitching_guide"
Personal tools