Data Services Newsletter : Volume 12 : No 3 : Winter 2010

Web Services at the DMC

Notice: All web services are now provided via the new services interface at http://service.iris.edu
As much as possible the content of this article has been updated to reflect the new locations
IRIS DMC Webmaster, 2013-08-08

Introduction

We are pleased to introduce a suite of core web-services for the direct access of waveform and metadata held at the IRIS DMC. The web services use standard HTTP protocol technologies and are relatively simple to understand and use. They can be accessed programmatically with a wide range of programming languages (e.g. Perl, Python, Java), command line utilities such as wget and curl or with any web browser. Additionally, by using HTTP over port 80, the services are not restricted by network firewalls. We anticipate that the new web services will be used to develop novel software applications including web based applications both internally and externally to the IRIS DMC. We also anticipate that the web services will make the data that we manage more accessible to users outside of the traditional academic seismological community.

The following table summarizes the core web services.

Summary of IRIS DMC Core Web Services

Service Name Description
dataselect Retrieves individual segments of waveform time-series data; one segment per query. Returns miniSEED output.
tracedsp Digitally filters miniSEED data produced by the dataselect webservice. Returns miniSEED or SAC output.
ms2ascii Converts miniSEED data produced by dataselect or tracedsp into ASCII output. The ASCII output can be easily read without knowledge of SAC or miniSEED. Especially useful to non-seismologist. For example, weather researchers who wish to use barometric reading from seismic stations.
plotter Reads miniSEED data from dataselect or tracedsp and produces time series plots in PNG format.
timeseries Combines the functionality of: dataselect, tracedsp, ms2ascii and plotter. Retrieves individual segments of waveform time-series data; one segment per query. The data delivered can be optionally gain corrected, digitally filtered and deconvolved with standard filtering algorithms. The output data can be in several formats including: miniSEED, SAC, ASCII and plot images (png).
bulkdataselect Retrieves multiple segments of time-series waveform data in a single query. The data is sent to the caller in miniSEED output.
resp Retrieves channel instrument response in RESP format.
sacpz Retrieves channel instrument response in SAC ASCII (poles and zeros) format.
station Retrieves station and channel information in StationXML format.
availability Retrieves waveform data availability information in two formats: XML, list. The XML format is similar to StationXML. The list format can be directly fed to the bulkdataselect service.
quake (coming soon) Retrieves earthquake event information in “QuakeML”:http://quake.ethz.ch/quakeml/ format.

Each web service has a home page located at the

http://service.iris.edu/irisws/<SERVICENAME>/<VERSION>

or

http://service.iris.edu/fdsnws/<SERVICENAME>/<VERSION>

where you can find documentation on how to use the web service.

Usage

The web services use simple, and easily understood, calling conventions. They do not use complex web service mechanisms such as SOAP. If you can figure out how to craft a URL, you should be able to get going quickly. All but one of the services (bulkdataselect) use URL requests of the form:

http://service.iris.edu/irisws/<SERVICENAME>/<VERSION>/query?<PARAMETERS>

As a illustrative example, consider a request to generate and retrieve a waveform plot using the timeseries web service for the station ANMO , the date February 27th, 2010 and the time range 06:30 GMT to 10:30 GMT (coincidental with an 8.8 event in Chile). ANMO belongs the IU network. For this example we choose location and channel identifiers of 00 and BHZ respectively. The following table summarizes the query parameters:

Name Value
network IU
station ANMO
location 00
channel BHZ
starttime 2010-02-27T06:30:00
endtime 2010-02-27T10:30:00
output plot

The parameters are serialized as a set of name=value pairs separated by ampersand (&) characters. This is the standard way in which query parameters are represented for HTTP GET requests. For this example we will have:

network=IU&station=ANMO&location=00&channel=BHZ&starttime=2010-02-27T06.30.00&endtime=2010-02-27T10.30.00&output=plot

The final query becomes:

http://service.iris.edu/irisws/timeseries/1/query?network=IU&station=ANMO&location=00&channel=BHZ&starttime=2010-02-27T06.30.00&endtime=2010-02-27T10.30.00&output=plot

This will produce a plot looking like the following:

Example waveform plot from the timeseries webservice
Figure 1: Example waveform plot from the timeseries webservice

This plot shows the units as “COUNTS”. If sensor units were desired, add scale=AUTO to the parameter list like this:

http://service.iris.edu/irisws/timeseries/1/query?network=IU&station=ANMO&location=00&channel=BHZ&starttime=2010-02-27T06.30.00&endtime=2010-02-27T10.30.00&output=plot&scale=AUTO

Example waveform plot from the timeseries webservice specifying sensor units
Figure 2: Example waveform plot from the timeseries webservice, specifying sensor units

If miniSEED data is desired instead of a plot, simply change output=plot to output=miniseed like this:

http://service.iris.edu/irisws/timeseries/1/query?network=IU&station=ANMO&location=00&channel=BHZ&starttime=2010-02-27T06.30.00&endtime=2010-02-27T10.30.00&output=miniseed

To aid in the understanding of how to construct request URL’s, most of the web services have a “URL Builder” form where one can enter parameter values and see the resulting URL.

Screenshot of the time series builder
Figure 3: Shown here is the URL Builder for the timeseries web service. This web service takes a relatively complex set of parameters, some of which are mutually exclusive. The form dynamically enables and disables fields which are mutually exclusive. This can be a time saver when trying to understand call logic.

Waveform Access

The waveform access web services bulkdataselect, dataselect and timeseries retrieve data from both the waveform archive and the real time collection system (BUD) . The services seamlessly combine the two storage systems so that the user does not need to be aware of their distinction. When overlapping data exists in both systems, data is preferentially pulled from the archive.

The bulkdataselect service is useful for getting large collections of channel waveform segments. For its input it takes a list of channels and time ranges as shown here:

TA A25A -- BHZ 2010-084T00:00:00 2010-091T00:00:00
IU ANMO 00 BHZ 2010-084T00:00:00 2010-091T00:00:00
IU ANMO 10 HHZ 2010-084T00:00:00 2010-091T00:00:00
II KURK 00 BHN 2010-084T00:00:00 2010-091T00:00:00

The input list can be long. As the web service processes a request, it streams the data back to the client. By streaming data back to the client, the memory footprint on the server is greatly reduced, thus making large request processing scalable. The data is sent back as one concatenated bundle of miniSEED data. The input list is submitted with an HTTP POST request. This can be easily done with the UNIX commands wget and curl. The online documentation at http://service.iris.edu/fdsnws/dataselect explains this in more detail.

The dataselect and timeseries web services are useful for retrieving individual segments of waveform data. In some applications this maybe easier to use. As mentioned previously timeseries provides several output format options as well as signal processing capability. In general bulkdataselect is a faster download mechanism and its use is encouraged for users who simply want large blocks of data.

Meta-Data Access

The resp, sacpz, and station web services allow the user to retrieve channel response information in three different formats: RESP, SAC Poles and Zeros and StationXML. In addition, the station service returns other meta information including geographic location.

The availability web service returns information about waveform availability. The output from this service can be directly fed to the dataselect service. The following example demonstrates using the wget command to retrieve all B channel waveforms within 30 degrees of the Chilean event discussed above.

wget "http://service.iris.edu/irisws/availability/1/query?cha=B*&start=2010-02-27T06:30:00&end=2010-02-27T10:30:00&lat=-36&lon=-73&maxradius=30.0&output=bulk" -O chile.txt
wget --post-file=chile.txt -O chile.miniseed http://service.iris.edu/fdsnws/dataselect/1/query
</pre>

The first wget command retrieves the list channel time range segments and saves the results to a file named chile.txt. The second wget command sends the list to the dataselect service and saves the output to chile.miniseed.

All of the metadata services take network code, station code, location code and channel code query parameters as well as time range information. The station and availability web services also allow for geographic location queries.

ICAB Workflow System

The web services dataselect, tracedsp, ms2ascii, plotter, resp and timeseries take part in a system called ICAB: IRIS Caching Artifact Builder. ICAB allows for the passing of data between web services through a token reference system. When a client invokes an ICAB webservice, instead of downloading the the generated data item (e.g. a miniSEED file), it can return a token string that references that data item. The data items are known as “artifacts”. The artifacts persist for a period of time in a cache. Tokens, which refer to cached artifacts, can be passed from one web service to another. For example, a token generated by dataselect could be passed to tracedsp and then the token from tracedsp could be passed to the plotter web service.

The timeseries web service wraps the functionality of the ICAB services dataselect, tracedsp, ms2ascii, plotter and resp. We anticipate that more ICAB services will be developed in the future. You can read more about ICAB at the article posted at: http://www.iris.washington.edu/ws/icab.html

Perl Clients

Four web service clients written in Perl are available for download at http://service.iris.edu/clients: FetchBulkData, FetchMetadata, FetchRESP, FetchSACPZ. These scripts should run with any relatively recent Perl interpreter. Run the scripts with no arguments to see useful usage information. Please see the online documentation for more information.

Conclusion

This is the first “big” rollout of web services providing access to the IRIS DMC’s main archive and database. Despite our best testing efforts, there will undoubtedly be hiccups as we release the services to a wide audience. As we move forward we will be both refining our existing services and providing new services.

Support

This work is funded by the NSF (Award # EAR-0552316) and the USGS (Award # G10AC00533).

by Bruce Weertman (IRIS Data Management Center)