BUD - an online Buffer of Uniform Data and how it works
The Origin of BUD
The BUD system was created to provide a single point of entry into the DMC for real time data from disparate sources. The data would be in a uniform format and in a strictly defined directory structure. BUD would serve as a staging area where data could be QC’d, a buffer of real time data immediately available to data users, and a single point from which to monitor incoming data flow.
The BUD Infrastructure
BUD hardware consists of a 1 terabyte RAID on an 8 processor E4000 Sun server. The disks are available to the web server for easy data distribution through the web interface and to WILBER II. Data streams currently flow directly into BUD from Antelope and an Earthworm WaveServer client, ew2mseed. The BUD Antelope stream originates from LISS, NRTS, SeedLink, and other Antelope sources.
The BUD Directory Structure and Data Files
BUD is intended to be independent of the incoming data source, and as simple and easy to use as possible. The BUD data organization consists of a top level directory containing Network/Station directories. In the Station directories are miniSEED data files by channel day. This organization can produce a lot of files if there are many days of data online, but BUD is intended as a staging area, so this is not a problem. Future plans may include expanding the time window maintained in BUD, at which point the BUD definition may be revised to include a higher directory broken out by time with BUD directories below it. At the time of this article, the time window is 2 months.
An inventory of networks and stations in a BUD directory is immediately visible by inspection of the BUD directory structure. An estimate of whether there is data for a particular channel, day, and time in the BUD directory can be derived from file modification time and an expected latency (typically a function of logical record length, sample rate, and input source) associated with a channel. The channel day miniSEED files are fairly small and so quick to read. These features of the BUD directory/file structure make it fast and easy to find out what data are available and to extract data.
The BUD Web Interface
A BUD Web interface was developed for data QC, data flow monitoring, and ftp data distribution. Existing web based QC tools (GOAT and map making) were integrated into the interface, and a seismogram viewer applet (Wiggles) was developed for it.
Especially in the case of regional network data, a Web interface available to network operators will help maintain a high quality data stream into BUD. The Web interface to BUD is intended for use by DMC personnel, network operators, and data users who may or may not be interested in data quality or data latency issues, but do want access to the data.
Effort was made to keep the Web interface to BUD standalone, although there is some auxiliary freeware that is needed to support full functionality. If you are interested in writing data to a BUD directory structure and installing a BUD Web interface on top of it, contact us.
Other BUD Services
In addition to distributing BUD data through the Web interface with ftp, data is distributed through a LISS Server and Fissures Seismogram Server. An autoDRM which serves BUD data is under development. See more detailed information on all BUD services.
ew2mseed
An Earthworm WaveServer client, ew2mseed, which writes out continuous data to a BUD directory structure was developed by ISTI under contract to the DMC.
WILBER II
The DMC’s WILBER II accesses BUD for SPYDER® data. This approximately doubles the average number of stations from which data are retrieved.
by Sandy Stromme (IRIS Data Management Center)