Merging Quality Controlled Data with Raw Data from Real Time Data Streams at the DMC
The Federation of Digital Broadband Seismographic Networks (FDSN) approved a modification to the SEED standard earlier this year, resulting in SEED version 2.4. This modification allows merging of data from quality-controlled sources with data received without quality assurance, usually data received via real time telemetry. The modification required for the SEED format is small; in the Data Records there is a one-byte field that has always been D. With this modification, the value can now also be either R or Q. For those interested in the gory details, consult the Adobe Acrobat (pdf) version of the SEED manual on page 100. (A full description of the Data Headers Quality Indicator Field can be found in the manual.)
The IRIS DMC is presently implementing support for this field to allow us to merge real time data received in the BUD system with data received from our traditional, generally quality controlled sources such as the ASL and IDA Data Collection Centers or data that have passed through the PASSCAL Database. All data from the BUD will be tagged as R type (raw data without Quality Control) before being moved to the IRIS DMC archive. Since most of the data from our traditional sources will also be received again from the IRIS Data Collection Centers or from other Data Centers with quality control applied, this allows both versions of the data to exist in our archive. All DMC software to support this modification is now in place.
What do users have to do?
In short, you don’t have to do anything. You can continue submitting your requests in the same manner you always have. In this case you will receive the “Best” version of the data the DMC has. If we have data tagged Q it will be included in your requested data; if no Q data are available then D data will be included; and finally, if neither Q nor D data exist, you will receive R data (assuming we have that).
DMC request tools have been changed to allow you to fine tune your request instead of taking the default.
In the BREQ_FAST format you may now specify a .QUALITY field in the header portion of the BREQ_FAST request. See the BREQ_FAST manual for examples.
The values the Quality Field can assume are
| Code | Definition |
|---|---|
B |
Best Data (Q, if not Q then D, if not Q or D then R) |
E |
Everything (Q & D & R) |
Q |
Data tagged as QC’d |
D |
Data of indeterminate quality: Before June 2002 all miniSEED data was labeled D and as such some had been quality controlled and some had not. |
R |
Raw Data at the DMC: For example, data reaching the IRIS DMC via real-time telemetric methods does not have QC applied and is labeled R beginning with data in the year 2002. |
Other DMC request tools have also been extended to support this capability.
When using rdseed version 4.19 and later, users must specify a new input parameter, namely the Quality Control field. This can effect users that invoke rdseed from within a script, but will not effect those of you that invoke rdseed with command line arguments. When requesting data extraction from a SEED volume, rdseed will now prompt you for the Quality Control field and the supported options are
| Code | Definition |
|---|---|
E |
Everything (Q & D & R) |
Q |
Data tagged as QC’d |
D |
Data of indeterminate quality |
R |
Raw Data – no QC has been applied |
The names of rdseed output files now have the Quality Control Indicator of the data included in their names, the field just before the file format. This allows users to extract versions of the time series of differing quality and maintain their uniqueness within a given file. Here is an example of a new file name:
2002.064.00.00.00.0240.IU.ANMO.10.BH1.D.AH
by Tim Ahern (IRIS Data Management Center)
