Data Services Newsletter

Volume 16 : No 1 : Spring 2014

Core systems and web services running at the Auxilary Data Center

The story

For the last year the DMC has been building an offsite facility, known as the Auxiliary Data Center (ADC), with the following primary goals:

  • Host a fail-over facility for data and software managed by the DMC geographically distant from Seattle and autonomously operated (no on-site staff)
  • Provide uninterrupted access to data, and continue to collect real-time data in the event of catastrophic failure of the Seattle infrastructure
  • Increase the capacity for data access, leveraging future load balancing capability
  • Provide rapid access to data into a High Performance Computing (HPC) facility

The ADC is physically located at the Livermore Valley Open Campus (LVOC, San Francisco Bay area) in collaboration with Lawrence Livermore National Laboratory (LLNL). In addition to being geographically distant from the DMC in Seattle, this site offers excellent network connectivity, which is critical for the DMC to keep data holdings synchronized between remote sites while continuing to offer data access. Because this site is operated autonomously, where no staff are present, it will not be as robust as the DMC, but it will provide the essential functions of data assimilation and distribution that are primary operational goals.

Additionally, LLNL is host to High Performance Computing (HPC) infrastructure. Researchers using those resources will be able to access DMC data holdings “locally” instead of from Seattle. In this collaborative pilot project we aim to streamline data flow from DMC resources to forms appropriate for advanced processing in an HPC environment.

At this point, the core systems are running at the ADC, and all data and metadata holdings are synchronized. Most of the data and information systems at each center operate independently, we will continue to reduce the remaining dependencies. The long term goal, when the centers are sufficiently independent of each other, is to be able to operate with only a single data center running at any given time. Over time, it’s believed that more nodes at other sites could be operated as autonomously as this one. Not only would this mitigate down time in the case of catastrophic failure of one of the centers, but will also allow us to perform center-wide maintenance without interruption of services.

Beyond providing added capacity, by using the ADC as an operational extension of the DMC we ensure that it is in a constant state of readiness.

Web services at the ADC

As one of the first steps in making the ADC an independent data center, we have replicated our web service architecture, effectively creating a copy of the services that are hosted at As with the installation in Seattle, the systems are fully redundant and have similar capacity for handling requests.

What does this mean for DMC data users?

Except for very special scenarios, most data users will not directly interact with services and resources at the ADC. Instead, we will develop systems and practices where these additional resources are used transparently by users. For example, data access performed by the DMC (for data product generation, quality assessment, BREQ_FAST request processing, etc.) can be load-shared between the centers without any knowledge by users.

We are also investigating how to globally load-balance web service requests between the centers in a manner transparent to users. The intention is that requests sent to a single web interface can be seamlessly fulfilled by either data center.

Ultimately, the ADC enhances our data security and adds capacity to collect data and service requests.

“Breathtaking. I shall call him… Mini-Me”

LVOC move in
Figure 1: IRIS moving equipment into the LVOC-LLNL facility.
This is a temporary location until LLNL build their new state-of-the-art
facility in a few years.

by Chad Trabant and Rick Benson (IRIS Data Management Center)