Discovery and collection of data across data centers using the IRIS Federator
Overview
The IRIS Federator is an IRIS DMC service that allows users to discover data at many FDSN data centers that support FDSN web services. Behind the scenes, the DMC harvests the list of time series channels offered by each data center by requesting and storing the channel list metadata from each center daily. The irisws-fedcatalog web service then allows users to search the combined catalog of channels across all data centers.
Using the Federator
To promote usability and compatibility, the irisws-fedcatalog uses the same interface design as standardized FDSN web services. A request is submitted to the service using the same parameters and formats as FDSN data centers, and the response is a slightly enhanced version of FDSN request format. The enhanced response identifies which data should be requested from which data center and is designed to make it easy to forward the request details.
Support for the IRIS Federator
Support has been included in the DMC’s popular FetchData command line script, the DMC’s irisFetch.m script and is included in the most recent release of the community-developed ObsPy Python toolkit. Support in more IRIS tools, such as the station mapping service and other data access tools is in progress.
Regarding actual data availability versus metadata
The information harvested from various FDSN data centers is metadata describing stations, channels, etc. This metadata does not always reflect the true availability of time series availability. This means the IRIS Federator does not know the actual, fine-grained details of data availability, but only what is reported as metadata by each data center.
The future
The DMC is planning extensions to the IRIS Federator that allow it to return simple metadata for station and network levels. Essentially mimicking an FDSN metadata service, the “irisws-fedcatalog” service will be able to return the “text” formats (simple metadata) for networks, stations in addition to the already supported channels. These enhancements will make the Federator useful for discovering stations and networks, and then suitable for applications such as an FDSN-wide station map and many others.
by Chad Trabant and Mick Van Fossen (IRIS DMC)