You are here: TWiki > Documentation Web>UsingTheGrid (06 Dec 2016, KyleGross)

An Introduction for Grid Users

OSG Connect users: please use the ConnectBook documentation, which is available from the OSG Connect homepage,

About this Document

This page is aimed at beginning users who want to use the resources on the OSG. Every user must be associated with a registered organization (called a VO). Many large VOs (such as ATLAS and CMS) already have extensive user documentation; these pages are primarily oriented at users of smaller VOs who might be porting their own or community applications to the OSG.

These pages first introduce the primary OSG concepts, then dives into the nuts and bolts of building (or porting) an application to the grid by showing demonstration applications of increasing complexity. Both job management and data management are covered.

About the Open Science Grid

  • What is OSG? This page introduces the "big-picture" concepts of what the OSG is, what it is not, and what it tries to achieve.
  • What are VOs? Virtual Organizations - "VOs" - are an important organizational concept for the OSG; this page introduces VOs for the end-user.

About Security Credentials

  • What is a certificate? A digital certificate is like a passport for the grid. This page introduces them to new users, and includes links for getting your own certificate.
  • Get a Certificate Instructions on how to get a certificate.
  • How do you get access to a site? The certificate provides you with an identity - but doesn't give you access rights to a site! This page describes how one gets authorized at an OSG site.

Joining a VO

  • You must belong to a VO to run jobs on OSG.
  • VO List includes a Membership URL for each VO for application to become a member. For example, the Trash/Engagement VO URL allows you to apply for Engage VO membership. To access these URLs and become a member, you must already have a valid certificate loaded in your browser as explained in Get a Certificate.

Using the OSG

OSG Client Software

All of the following sections will focus on job and data management on the OSG in order to build a successful grid application.

These will all assume that you have access to (an account on) an OSG Submit host provided by your VO or have the OSG client software installed.

Running Jobs

Data management

Data management is difficult on the grid. Unlike running jobs, where a consensus approach is beginning to appear, there are many techniques for data management. Some of them scale up well to hundreds of TBs a day, but scale poorly to opportunistic jobs.

The most common approach is to use a get/put method - they download input from a remote endpoint to the scratch area. The jobs run, and write output to the scratch area. After computation part of the job is finished, the result upload is uploaded to a remote endpoint.

Two common approaches to staging in data are via HTTP (many sites have local caches) or a protocol called SRM.

The de-facto way to save output is through SRM.

Advanced Topics

Beyond "traditional", basic job handling, there is a wide variety of specialized topics for using the OSG. Here are a few which may appeal to advanced users:

  • High-Throughput Parallel Computing: This page covers how to submit multi-core (which may be MPI or just traditional multi-threaded) jobs on the OSG using Condor.
  • Globus RSL language usage on the OSG: Globus RSL is a mini-language that controls advanced, specific aspects of your jobs. For some users, this might be necessary for their jobs; we recommend it against using RSL heavily because it is not portable between sites.
  • Job Submission tool comparison: A document that compares the three most common submission methods in the OSG: GlideInWMS, PanDA?, and OSG-MM. The current OSG recommendation is to use GlideInWMS for grid applications.
  • GlideinWMS VO Frontend Install Guide: Installation guide for the GlideinWMS VO Frontend.
  • Condor-G to GlideinWMS porting guide: GlideinWMS is a grid scheduler that can be used by VO's. This guide will document the best practices on moving jobs from Condor-G to GlideinWMS.
  • GlideinWMS VO Frontend Groups: Description of groups in the VO Frontend. These are used to submit different 'types' of glidein jobs.

How to get Help?


Topic revision: r73 - 06 Dec 2016 - 18:12:39 - KyleGross
Hello, TWikiGuest!


TWiki | Report Bugs | Privacy Policy

This site is powered by the TWiki collaboration platformCopyright by the contributing authors. All material on this collaboration platform is the property of the contributing authors..