This document is in Draft Status

Post Upgrade Functionality Checks

This document is to provide a list of checks that will test the functionality of the BDII services after an upgrade has been done. These can be done from the end user perspective and does not include admin level checks of the service which require access to the BDII hosts.

Checks Immediately after service is brough up

Checks ~5 Minutes after the service is brought up

  • From Status Page at or
    • Check freshness of Raw Incoming Data most resources should be < 5 Minutes
    • Check freshness of Data Feeds to OSG and WLCG these should also be < 5 Minutes
  • Checks from Command Line of LDAP Server Functionality
    • Run ldapsearch -h -p 2170 -x -b mds-vo-name=local,o=grid - this will return several thousand lines of GLUE Schema information
    • Run ldapsearch -h -p 2180 -x -b o=grid - this will also return several thousand lines of GLUE Schema information
    • Possible Error Examples Below

Ongoing Monitoring Checks

  • Several scripted checks run on these services including:
    • Check of BDII Freshness at CERN Top Level BDII and SAM BDII available at only WLCG resources will be listed here as they are the only resources publishing to CERN BDIIs, these tests are run each hour on the 30 minute mark. So may be behind up to an hour after upgrade has completed.
  • Email alerts are sent to the GOC-ALERTS mailing list for the following conditions
    • More than 10% of resources are not updating BDII information (either WLCG or OSG)
    • FNAL or BNL is not available from the CERN Top Level BDIIs
    • RSV Probes check timestamps of information in the BDII failure are reported via mail
  • The BDII Service also reports many system level metrics via Munin, these should be checked continuously for anomaly after an upgrade.

LDAP Errors

LDAP Server Not Running

This error will happen in the LDAP Server is not responding.

ldap_bind: Can't contact LDAP server (-1)

Data not found

This type of error will happen if no data is found matching your query, first check the ldapsearch syntax if it is correct data is missing.

# extended LDIF
# LDAPv3
# base  with scope subtree
# filter: (objectclass=*)
# requesting: ALL

# local, grid
dn: Mds-Vo-name=local,o=grid
objectClass: GlueTop
objectClass: Mds
Mds-Vo-name: local

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1
-- RobQ - 11 Aug 2010 -- RobQ - 11 Aug 2010
Topic revision: r2 - 27 Aug 2010 - 17:25:51 - ElizabethChism
Hello, TWikiGuest


TWiki | Report Bugs | Privacy Policy

This site is powered by the TWiki collaboration platformCopyright by the contributing authors. All material on this collaboration platform is the property of the contributing authors..