070430-perfSONAR-Salvador

From GEANT2-JRA1 Wiki

The meeting was hosted by RNP (Brazilian Research and Education Network) and UNIFACS (Universidade Salvador).

Contents 1 Attendees 2 Agenda 2.1 Tuesday, May 1 2.2 Wednesday, May 2 2.3 Thursday, May 3 2.4 Friday, May 4 3 Minutes 3.1 Tuesday, May 1 3.2 Wednesday, May 2 3.3 Thursday, May 3 3.4 Friday, May 4 4 Photos

Attendees

1. AH, Andreas Hanemann (DFN Munich)
2. CR, Candido Rodriguez (RedIRIS)
3. DA, Dimitri Anjos (UNIFACS/RNP)
4. EM, Edison Tadeu Lopes Melo (UFSC/RNP)
5. EO, Eliseu Oliveira (UNIFACS/RNP)
6. EP, Eriko Porto (RNP)
7. FV, Fausto Vetter (UFSC/RNP)
8. GF, Guilherme Fernandes (UFSC/RNP)
9. GR, Guilherme Eliseu Rhoden (UFSC/RNP)
10. HS, Herbert Souza (UNIFACS/RNP)
11. IK, Ivo Koga (UNIFACS/RNP)
12. IM, Iara Machado (RNP)
13. JB, Jeff Boote (Internet2)
14. JM, Joe Metzger (ESnet)
15. LK, Loukik Kudarimoti (DANTE)
16. LR, Leandro Rodrigues (RNP)
17. LS, Leobino Sampaio (UNIFACS/RNP)
18. MS, Martin Swany (UDel/Internet2)
19. MSt, Michael Stanton (RNP)
20. MV, Murilo Vetter (UFSC/RNP)
21. MY, Mark Yampolskiy (DFN Munich)
22. PM, Priscilla Moraes (UNIFACS/RNP)
23. RC, Rafael Costa (UNIFACS/RNP)
24. SM, José Augusto Suruagy Monteiro (UNIFACS/RNP)
25. ST, Szymon Trocha (PSNC)
26. VV, Verena Venus (DFN Erlangen)

Agenda

Tuesday, May 1

Start 12:00

• Roadmap Y3 (and Y4) (Szymon)
  • Give an overview of the JRA1 plans for Y3 and Y4
  • Get partners plans
    • Download RNP presentation
  • Discuss between all, common things, general strategy, etc.
• COBIT review (Fausto)
  • perfSONAR compliance IT processes review based on COBIT
  • Explanation of the review, examples, help
  • I suggest to fill in the review in free time and consult with Fausto if necessary
• Developer Support (Loukik)
  • Relation/benefits of GEANT2 In Service Support on perfSONAR development
  • Support processes (ITIL compliance)
  • jar-repository/tar-repository

End 18:00

Wednesday, May 2

Start 9:00

• NMWG schema enchancements (Martin)
  • Specification of interactions for cNIS (Martin, Andreas, Mark, RNP?)
  • Further extension for E2Emon/JRA4 (Martin, Mark)
  • Other services (Martin, Verena)
• AA (Candido)
  • What and when, information for partners
  • Issues from service and tool developers
• LS (Szymon)
  • Use cases - discuss different use cases and write down examples coming from different users of LS
    • Visualisation
    • cNIS / SA3
  • New parameters in Lookup Information
    • ServiceDomain
    • ServiceSubtype
  • Interface changes
    • Discuss and agree on MG proposal to replace all Exceptions by PerfSONARException
• MAs (Szymon)
  • Support of more than one eventType for one subject (metadata)
  • Support for a new namespace to include container structures like maps, lists, etc. for parameter element
  • New metrics
    • Ideas for new metrics to be implemented in MAs
    • L3 status interesting for CNM (Andreas)
  • Using namespaces for result codes in eventType elements (consistently through all services as we did for utilization metric)
  • Supporting different time formats in requests and transformation of different time formats (general for services)

End 18:00

Thursday, May 3

Start 9:00

• LS
  • Common service naming scheme - discuss and finally agree on naming scheme used during service registration. make a list of hierarchy of domains.
    • Service name + domain name
    • Hierarchy of domains
  • LS in the MDM pilot - how many and where
  • LS backup strategy
    • Back-up for LS achieved by Multi-LS
    • Store and access of backup information from LS (in particular for visualisation tools)
• Visualization (Andreas)
  • Integration of RNP's ICE tool into perfSONAR visualization tools "suite" (Ivo, Andreas)
  • Contribution of RNP to cNIS development (automated topology discovery) (Murilo, Andreas)
  • Use of perfsonarUI/CNM/E2Emon in RNP (Ivo, Leobino, Andreas, Mark)
  • Addressing of visualization for remaining MPs/MAs (CLI MP, L2 status MP); use of perfsonarUI plug-in mechanism for that
• CLI MP
  • Service development and deployment in Internet2 (Jeff)
  • RNP development, experience and priorities (Guliherme)
• BWCTL MP (Verena)
• Configuration service
  • Proposal of configuration service - initial discussion (Fausto)
• Release management and strategy (Loukik)
  • Services
    • Sotware auto-update functionality
    • Bundle installer
  • RNP experience on CLMP testing (Dimitri)

End 18:00

Friday, May 4

Start 10:00

• Other
  • RNP work on Web services composition through ESB (Enterprise Service Bus) (Rafael)
• Documentation (Loukik)
  • Relation between deliverables and release documents
  • Improvements of release documents
  • How to keep the documentation up-to-date
  • Web sites improvements
    • Wiki page restructuring
    • perfSONAR page restructuring
• Licence (Szymon)
  • Current status (I2, GN2)
  • Final agreement
  • What's next
  • Visualization tools
• Next release (Loukik)
  • Target timeframe
  • Services and functionalites to include
    • Proposals to prioritize the work

End 13:00

Minutes

Tuesday, May 1

• Roadmap Y3 (and Y4)
  • ST gave an overview of the JRA1 plans for Y3 and Y4, presented the idea of MDM service and asked partners about their plans.
  • SM presented an overview of RNP network and measurement activities in the context of perfSONAR (download RNP presentation) and showed pilot deployment of pS within RNP network. He said they wanted to show users that applications were attractive. They wanted to measure delay, abw, utilization.
  • JB informed I2 was deploying new core network and looking for measurement tools. The network will have 26 optical locations and 9 router locations. Due to bad experiences attempting to get members to install the pS Java version they decided to develope Perl-base services. He said that before July this year they were going to have SNMP MP, BWCTL MP, E2EMon and topology. JB also showed two new tools using pS and Google map developed by Internet2 and SLAC ([1] and [2]). He said it was the first step.
  • EP showed RedCLARA network and project. He mentioned measurements done: BWCTL, OWAMP, ping, traceroute. Cacti is used for visualisation with plugin to get data. ST asked whether there are any requirements on pS from RedCLARA. EP answered there weren't any but the definitely wanted to deliver performance indicators.
  • JM said their new network would based on the same optical core as I2 but other equipment. The goal is to provide more visibility of the network. They want to use as much as possible fo existing tools and he said the software should have been standardised but people should be able to develop what they wanted.
• COBIT review
  • FV presented perfSONAR compliance IT processes review based on COBIT. He explained parts of the template document to be filled by partners. JM said it could be a problem when an organisation depended on another organisation with no control on it. Partners were filling the questionnaires during the meeting. AH asked if FV had some experience with eTOM. He will send him a pointer.
• Developer Support
  • LK explained benefits of GEANT2 In Service Support on perfSONAR development. He explained ITIL and Service Desk. ST asked whether DANTE was going to use some tools for knowledge base. LK answered that it was under investigation by NS. JM asked who was going to run MDM. LK: DANTE. JM commenetd that EsNet used standard support procedures for services and they would not want to deploy services which would require a lot of user support e.g. VoIP. SM asked who would be supported by Service Desk. LK: users, projects, NOCs. LK added DS3.15.1 would be circulated soon to include all details.

Wednesday, May 2

• NMWG schema enchancements
  • MS showed an introduction to NMWG schema and presented topology schema update.
  • Action (MS): To send document describing topology schema
  • Discussion was held on documenting schema as it was difficult for developers to find proper documentation, explanation of namespaces, see difference between different types of chaining or filtering. JM, JB said a place is needed where we could register and validate schema extentions but we need to find resources for doing it. We could go step-by-step with next releases. LK said we needed a place where schema was. We could start building it with doc format and them move to some CGI scripts. JB answered that it would depend what was easier. He porposed to start with a simple list of what is in SVN done by CGI script which would list eventTypes and messageTypes. Then have some way of registering schema extenstions and some sort of automated validation to check syntax.
  • Action (MS, JB): Prepare a first version of website schema repository. Should be possible for the JRA1 meeting in June. It will list things (not registering functionality yet) and cover currently available eventTypes in NMWG schema which have been already checked to SVN. This site will be hosted on I2 sever. Next steps will include automatic validation and documentation generation - ONGOING, beginning of June
  • MY asked about schema for technologies like MPLS or SDH. MS asked they were going to modify the schema namespace to technology specific targets.
  • AH presented cNIS overview
  • MV presented RNP's TopS and showed message proposal for discussion. AH asked to harmonize efforts with PSNC developers of cNIS.
  • Discussion was held on having complex parameters within message and support for a new namespace to include container structure. Two proposals with regard to TopS were sent to the mailing list some time ago: one from MV with nested parameters and then its modification done by RL. MS commented that MV's proposal is valid and we didn't need to use collection items as RL had suggested. He also added that Java code already supported what MV had suggested.
  • Agreement: MV's proposal accepted, RL's proposal rejected.
  • Support for different time format was discussed as proposed by RL. MS said that RL's exampled were good and were already supported. JB asked to discuss possible implementation strategy so that changes were backward-compatible
  • Action (all): Discuss different time format implementation strategy.
  • Agreement: Use namespaces for result codes in evetType elements in all services
  • MS presented strawman for result codes namespaces.
  • MS added that more details could be included inside datum element. LK commented that we should have used as much as possible common namespaces. But should be a room for extentions per specific service e.g. when rrdh.lib is missing in RRD MA.
  • Action (MS): To harmonize proposals with the OGF WG and then send them to the mailing list for discussion - ONGOING, beginning of June
  • Agreement: Keep existing hierarchy of result codes but e.g. remove "error" instead of "level".
  • LK noticed that NMWG object structure was maintained by LZ and MS. He asked whether there was any release management process followed by them. He also proposed to use some kind of versionning of releases for Java classes.
  • Action (LK): Prepare a proposal for releasing NMWG classes.
  • LK asked if there was any info about NMWG versionning in the new NMWG schema documents MS was woring on. MS confirmed.
• AA
  • CR gave an status update (download presentation) for AA work within JRA1 and informed that the first version of AS would be available end of May and at least one service should have been used AS. AH suggested that RNP should look at eduGain if it fitted into their needs. IM asked about eduGain details. JB suggested to read AS specification document.
• MAs
  • ST asked the audience about new metrics to be stored in MA. A few suggestions were received: AH requested for L3 status (SNMP status of interface) for CNM so that he could use color coding; JB requested SNMP errors coming from interface counters and for traceroute for routing changes and ping results; GF asked for active measurement results storage in SQL MA. JB suggested that information about available metrics could be expressed through an info message sent to the MA.
  • Action (ST): Send proposal of new metrics to the mailing list for discussion and prioritize them - DONE
• LS
  • JB summarised MG's proposal to change exceptions. MG suggested to get rid of these. JB suggested not to remove but to make the change such that it does not break code - by deprecating. MG agreed. This will make other people to change the class when receiving error message.
  • Action (MG): Proceed with exceptions changes. Mark changed classs as deprecated. - DONE
  • HS and RC presented their experience in development of UDDI-based LS. UDDI LS interoperates with XML LS and is used by RNP visualisation tool - ICE. They showed performance tests which revealed that UDDI LS preformance was better than XML LS. But the tests were amde on vesrion before MG's changes of storing information in multiple files. AH asked pS partners why we didn't use UDDI. MS asnwered that we had gone for easier extensibility but worse performance.
  • JB asked what's the path forward with UDDI-LS. LK suggested to use xPath on top of it. JM asked about data aggregation in those two types of LSes. JB answered for mLS it was done with xslt summarization and the summary was send to upper level Lookup Service.
  • Action (all): Discuss summarisation issues until mLS is ready.
  • ST asked if UDDI-LS is multi-domain. LS answered that it wasn't because free UDDI was a single domain while multidomain UDDI was only commercially available. He also said they planned to investigate mappings between XNM and SQL and how to implement multidoman LS with the next version of UDDI. RNP will make UDDI LS message available.
  • ST asked to accept MG's proposal to add new Lookup Information fields to Lookup Service. JB asked what criteria should have been used in adding new parameters. He said it had to take into account other partners work e.g. RNP. RNP agreed to add new fields.
  • Agreement: MG's proposal of new Lookup Info parameters accepted.
  • LS asked what was the relation between topology information and LS. AH asnwered that topology would be in cNIS. JB mentioned that this was not the case for I2 and RNP as they would not have cNIC. SM asked how data was stored in cNIS. LK answered in relational database, with cNIS in each domain. JM asked whether it was possible to query about topology data in specific time in the past. MS answered it should have been possible.

Thursday, May 3

• LS
  • ST asked about backup solutions for LS. LS said that for MDM there would be hardware redundancy. JM mentioned that services should have capability to register in multiple LSes. JB said mLS would provide backup solution for LS. He added that it was a question of how much downtime one could accept. LK said that switching to another LS wasn't smooth now. LK was wandering whether we worried about the service available or data available and wheter we were asking domains to have a backuo server or one central LS for all domains. JB said we would need both. LK said there would be different SLAs with domains and it could differ. JM noticed that he didn't think one LS could cupport current number of interfaces from EsNet, I2 and GEANT2 in RRD MA. Only mL and data aggregation may help. He added that backup implementation should have been user driven and we couldn't convince them. LK explained from MDM perspective there would be probably a few instances of LS running but they would not be "visible". They will become only when the primary is down.
  • Agreement: Keep LK's proposal until mLS is ready: every MDM domain will have an LS (with hw backup) and possible central GEANT2 LS. With other solutions we wait until mLS is ready.
  • JB presented and explained example transformation messages
  • Action (JB): Send the proposal to the mailing list for broader discussion. - DONE
  • ST asked about use cases. AH answered from CNM point fo view. The primary interaction for their needs is that we would like to know the MAs (RRD, SQL Hades MAs) that exist. For now, it it is sufficient to return the MAs with an identification of the domain (i.e., MA URL + domain name). The domain name is useful for us to map the MA to the topology maps that we have. In the meantime ST receieved via e-mail also use cases from pSUI developers. They said in general, it would be nice to have a library (.jar) implementation of a "caching lookup service" and have the client talk only with the local caching service. It then could forward the request to a real LS, if the queried info is not found in the cache. NJ would like to have:
    • Measurement Archives (Interfaces tab): Retrieve a list of MAs (similar to the one currently configured by MA.conf). This should include service address as well as an information about the schema version supported.
    • For the traceroute functionality: 1)Get information which MA is responsible for particular IP. It will then be used to retrieve utilization about interfaces at given IP. 2)The same, but if possible, send a list of IPs to the LS and get a list of MA, responsible for each IP.
    • Hades (Circuits tab): Retrieve a list of services, supporting Hades protocol (not sure if there is an official name of the Hades schema). Note that besides the address, currently there are different eventtypes configured for RIPE and GEANT measurements.
• Visualization
  • AH presented CNM developments and status as well as the same for perfsonarUI and VisualPerfsonar. JM said VisualpS gave user an easy way to see messages.
  • IK presented RNP work on ICE tool, OSGI framework and FLAVOR (download presentation). AH mentioned that if we wanted to have the tool available for other partners we should have made RNP-related things in visualization out of this application. He asked if RNP wanted to recommend pS tools for their users. SM answered they wanted to use what was available. Only CNM is confusing them as it is maintained by DFN. AH said they could prepare a customised version of CNM to be installed in RNP. He said that CNM and pSUI would be deployed during MDM service. pSUI will include more tabs and use LS.
• CLI MP
  • GF presented RNP work on CLI MP. He mentioned expectations: service manager for the service, new BWCTL schema to support not only iperf but also other tools, resource protection which is currently done thorugh threads so you are placed in a queue. VV said they had solved AA issue using BWCTL parameters file but problem was that password and user was a clear text in XML file. ST noticed he was happy to see RNP work in pS2.0 as a result of collaboration. He asked about using it in MDM. LK answered that it would need support but RNP was on different funding. It could be future policy decisions. Now it will be supported on the best effort base.
• BWCTL MP
  • VV summarised the work on BWCTL MP: implementing pSUI plugin for BWCTL MP and BWCTL rnc schema will be incorporated into next release.
• Configuration service
  • FV presented an idea of service which would be a central point for configuration - one place where one could configure everything. JB commented that current Java services config file is parameters-based so instead of having properties file one could fetch it from central service. It could even be based on LS as it had the same database. It would help when we have a lot of services to deploy.
  • Action (FV): Send strawman proposal for this new configuration service to the mailing list for discussion - ONGOING, beginning of June
• Release management and strategy
  • MY asked whether it was possible to get some sort of test sw for developers to test his service against it. It could help developers before the service goes into mirco release. LK agreed it was a good idea. MY added that test sheets should have been available for developers.
  • Action (LK): Organise a conf call to discuss possibilities of having test sw and test sheets.
  • MY asked what it meant software update functionality. LK explained that it would be something similar to Windows Update but might be triggered. He doesn't have much idea now. ST asked about release management process for visualisation tools. AH answered thare was a document prepared and process is run in parallel to services.
  • DA and IK presented their experience with CLI MP testing. LK asked if those were out of the box tests. DS answered yes.

Friday, May 4

• Other
  • RC presented (download presentation) Enterprise Service Bus (ESB). ESB is a Middleware infrastructure used to integrate SOA. MY asked if we could use ESB for multidomain environment. RC answered yes. MY asked if he was planning to do such workflows. RC said he would do this.
• Documentation
  • LK presented documentation concept overview and asked for comments. JB complained about doc format. LK said we would move to docbook. JB said that interface specification was ok but there was no pointer to it. Should be in Readme file of the bundle. GF added that functional specification should have better introduced to what service does. LK asked partners about rnc files generation. JM and JB said it would be nice to have some sort of automatic generation because now they were manually generated. LK added that even if they are automatically generated they didn't reflect what had been programmed.
  • Action (JB, MS): Propose alternative solution to the way interface specification is written. This will involve typing up the protocol specification rnc with the NMWG base schema. Timeframe August/September.
  • LK said that detailed description of ant targets was necessary so that questions were asked the same way by all services. ST said he was not sure developers should have done it. VV agreed with LK developers should have done it as they know services the best. She added that base ant targets could be a part of pS base to separate general questions and specific for the service. CR asked who was the developer of pS base. Discussion was held to identify this person(s).
  • Agreement: The "sort of" agreed approach was: In the (future) perfsonar-base trunk, have an ant directory. In this ant directory, have the base ant targets which all developers should work with. If the developers have to write their own ant targets in addition to these or need to modify these slightly in their implementation, they should contact the release management team and get it approved. The base ant targets will be documented (i.e. interaction, structure, etc) as part of base development. This should reduce the documentation effort on the developer but for their own ant targets or modifications, they should follow the same type and provide the same information as the perfsonar-base ant targets document.
  • Agreement: PSNC and GF will take care of pS base. Details what it involves to be discussed.
  • The group discussed Wiki improvements. ST said it was difficult to find information and many information was out-of-date. All agreed.
  • Action (ST): Go through the content, contact related authors and mark such content for removal (maybe a new page which lists out all the pages which should be removed) - ONGOING
  • Action (LK): To contact EB for getting the pS webpage sync stuff working. - DONE
  • At the end LK showed SVN doc structure.
• Licence
  • ST informed that the decision was to go for modified BSD licence.
  • Action (ST): Send information about licence decision to the mailing list and discuss what's next.
• Next release
  • ST asked about timeframe for next release. LK answered it wasn't known yet. First decision of Steering Committee must be made what functionalities to include.
  • Action (ST): Start discussing with SC what to include in the next release. - DONE
• Next meeting
  • Some of the partners will meet at GN2 Technical Workshop in Cambridge
  • JM offered to host the meeting in Berkeley (US) this fall.

Photos

Are available here: http://picasaweb.google.com.br/suruagy/PerfSONARDevelopersMeetingSalvador?authkey=Ron1cXl_41A

---

Back to JRA1 events