Data Quality Assurance

About

The Quality Assurance (QA) team at EarthScope is tasked with monitoring the quality of the EarthScope seismic data archive and providing resources relating to data quality to the earth science community. On this page you will find information on general QA practices, MUSTANG metrics and PDFs, and links to QA related products and services at EarthScope.

Quality Assurance Mission Statement

QA Related Web Services, Products, and Software

Web Services and Client Tools

MUSTANG, Modular Utility for Statistical Knowledge Gathering: seismic data quality metrics and Probability Density Functions (PDFs).

Products

Synthetic Seismograms
Global ShakeMovie synthetics at the DMC
Global ShakeMovie synthetics at the DMC event listing

Envelope Functions

Calibration Products

Software

R Package on CRAN, IRISMustangMetrics is the official public release of R code used by MUSTANG to calculate metrics.
ISPAQ (IRIS System for Portable Assessment of Quality), software for calculating MUSTANG-style seismic data quality metrics on a local machine.
QuARG (Quality Assurance Report Generator), software for network operators to find station problems using MUSTANG metrics and user-specified threshold values, examine the generated problem list using links to QA tools, document the issues in a ticketing system, and create a formatted HTML report from the tickets.
PIQQA (PI’s Quick Quality Assessment), software for generating automated reports summarizing data availability and noise levels for PASSCAL and other temporary experiments.

Other Resources

Data Problem Reports – search submitted data problem reports by station and network. This is not an exhaustive listing of all data problems that may exist in the archive.

Citations and DOIs

To cite the MUSTANG system or reference the use of MUSTANG metrics:

  • Assuring the Quality of EarthScope Data with MUSTANG
    Robert Casey, Mary E. Templeton, Gillian Sharer, Laura Keyson, Bruce R. Weertman, Tim Ahern
    Seismological Research Letters (2018) 89 (2A): 630-639.
    DOI: https://doi.org/10.1785/0220170191

Contact Us

Questions, comments, and suggestions related to quality assurance or data issues at EarthScope can be directed to:
data-help@earthscope.org

News

New improvements include: 
1. the ability to use the absolute value of the metric
2. the ability to reset the color binning to defaults after making changes
3. updated metric unit labels in time series plots
4. improved help text

MUSTANGular was created and is maintained at the University of Washington by the Pacific Northwest Seismic Network. Thanks go to Kyla Marczewski (primary developer) and Renate Hartog (Network Manager) for making this available not only to EarthScope, but also as an open source project on GitHub (https://github.com/pnsn/mustangular).

ISPAQ is a command line application that enables you to calculate quality metrics for seismic data locally, by leveraging MUSTANG R-code within a Python client. Over 40 MUSTANG metrics can be calculated for either local miniSEED files or for data available through any Data Center that supports FDSN web services. These metrics include basic trace statistics, metrics based on miniSEED state-of-health flags (if available), metrics based on event arrivals, Power Spectral Densities (PSDs), Probability Density Functions (PDFs), and metrics derived from PSDs. All results are computed and stored on the user’s local machine.

Improvements and changes for this release include:
1. ability to write metric values to an SQLite database
2. addition of MUSTANG metrics sample_rate_resp, sample_rate_channel, max_range
3. access to EarthScope PH5 archive data, using IRISPH5 alias
4. new Jupyter notebook tutorials
5. bug fixes and usage improvements
6. code port to Python3

In addition, the new ISPAQ SQLite database is compatible with QuARG, the Quality Assurance Report Generator (https://github.com/iris-edu/quarg) and can be used as a source to import metric values into that tool.

  • (2021-08-16) We are happy to announce the release of MUSTANGular version 2.2, available at http://ds.iris.edu/mustang/mustangular. MUSTANGular is a web-based application that plots color-coded MUSTANG metric values on an interactive map.

New improvements include:
1. ‘Display single channel’ priority now includes location codes. When there are co-located station channels and the display is set to show one channel only, the user can prioritize which channel is displayed (if the first priority channel is not available at a station, it goes to the second priority, etc.). In previous versions, this priority selection did not include location code and as a result there were channel-locations that could not be displayed. This update fixes this issue.
2. Re-introducing the ‘aggregate channels’ feature, which creates a single station value by aggregating the values across co-located channels. The options are to aggregate my maximum, minimum, and most extreme values. This is secondary to the ‘channel value across timespan’ value, which has options to calculate single values for each station-channel-location for the time period selected (default=average). It will calculate a single value for each station-channel-location, and then aggregate these values across co-located channels.
3. Improvements to the station pop-up display. When selecting a station on the map, the pop-up displays the values for all the channel-locations in addition to the value displayed on the map. These are individually color-coded according to the color scale.

MUSTANGular was created and is maintained at the University of Washington by the Pacific Northwest Seismic Network. Thanks go to Kyla Marczewski (primary developer) and Paul Bodin (Network Manager) for making this available not only to EarthScope, but also as an open source project on GitHub (https://github.com/pnsn/mustangular).

  • (2021-07-20) NSF SAGE is pleased to announce the release of a new software tool for producing quick data quality reports using MUSTANG-generated metrics, designed for use with PASSCAL experiments and other temporary deployments. Named PIQQA (PI’s Quick Quality Assessment), this tool is the result of a collaborative effort between EarthScope Data Services and Instrumentation Services.

This recent Data Services Newsletter article provides a nice overview of PIQQA, including an example report:
https://ds.iris.edu/ds/newsletter/vol23/no1/533/piqqa-data-quality-reports-for-passcal-experiments/

PIQQA is open-source and will run on any dataset archived at the EarthScope DMC for which MUSTANG metrics have been generated (please bear in mind that there are limits to the amount of metric data that MUSTANG web services can deliver through this tool; multi-year requests for large networks may not be appropriate). We think PIQQA will be particularly useful for PIs of recent temporary/PASSCAL deployments, who may appreciate a quick overview of the quality of their dataset. PIQQA does not display all MUSTANG metrics, and users interested in exploring additional metrics in greater detail are encouraged to explore additional MUSTANG clients or query MUSTANG directly via web services.

People wishing to use PIQQA may download the source code from theGitHub repository https://github.com/iris-edu/piqqa

  • (2020-11-18) We are pleased to announce the release of QuARG, the Quality Assurance Report Generator, available through the iris-edu GitHub repository: https://github.com/iris-edu/quarg. This utility creates a Quality Assurance report, intended for network operators who want and need to have an understanding of the health of the stations in their network. This utility creates a Quality Assurance report, intended for network operators who want and need to have an understanding of the health of the stations in their network. The report calls attention to underperforming or broken stations so that time and resources can be prioritized as they are allocated for improving the quality of the network.


QuARG is a python-based utility that walks the user through the process of creating a quality assurance report. This process follows 4 broad steps:
1. It utilizes MUSTANG metrics available through our web services, or alternately metrics generated using ISPAQ (using ISPAQ 3.0, available soon), to find and highlight potential problems in the data by flagging days that exceed user configurable threshold values. By using the pre-computed metrics, it reduces the amount of time that an analyst has to spend scanning the data for problems. It can also find issues that would otherwise go undetected by the eye.
2. Users then analyze the list of potential issues to determine if these are data quality problems that should be included in the report. QuARG makes it easy to keep track of which issues have been investigated, keep notes on what the analyst has found, and link to a slew of QA tools, such as waveform plots, metric plots, and Probability Density Function (PDF) plots, to make it easier to understand the problem.
3. From there, users create tickets that describe the problem. Tickets can be created in QuARG, or in an external ticketing system if the analysts have one that they already use.
4. These tickets, which track problems as they arise and can be updated when they are fixed, are then used to create a nicely formatted HTML quality assurance report.

Full documentation can be read at https://iris-edu.github.io/quarg/.

Frequently Answered Questions

There are currently 2 Quality Assurance related questions:

Tutorials

There are currently 4 Quality Assurance tutorials:

11:18:38 v.9bc884e1