This November 2020, IRIS Data Services released the Quality Assurance Report Generator – or QuARG, for short – available at GitHub (https://github.com/iris-edu/quarg). QuARG is a python-based utility that walks the user through the process of generating a quality assurance report; it is intended for network operators to help keep track of problems in their network as they arise and are resolved.

QuARG is based on years of effort at the IRIS DMC to refine the process of generating network quality assurance reports. Beginning the TA network, then expanding into the GSN and a handful of other networks, a process was created to find a significant portion of the issues in the data and reduce the time analysts spent on data quality review. This four-step process has been incorporated directly into QuARG:

1. Find: Utilize MUSTANG metrics available through IRIS web services, or alternatively metrics generated using ISPAQ version 3 (projected release January 2021), to find and highlight potential problems in the data.

QuARG comes with default thresholds that combine single or multiple MUSTANG (or ISPAQ) metrics with cutoff values that have been found empirically to detect various problems in the data. There is a balance between finding real issues in the data using thresholds while minimizing false positives that can occur in normal, healthy data. QuARG has made it easy to adjust the thresholds to specific networks: a network operator can edit the existing threshold definitions, add new thresholds, create instrument groups that each have independent thresholds, or simply toggle certain thresholds (or entire groups of thresholds) on or off.

2. Examine: Users analyze the list of potential issues to determine the cause and root out false positives. QuARG makes it easy to keep track of which issues have been investigated and keep notes on what the analyst has found. Links to a variety of tools to make it easy to diagnose problems, including tools such as waveform plots, metric plots, and Probability Density Function (PDF) plots, amongst others.

3. Ticket: Using what they’ve learned by analyzing the issue, users create tickets that describe the problem. These tickets are used to track issues as they arise and are resolved, and they are the basis of the final report. The level of detail in a ticket can vary, but enough information should be included so that the field crew has a good understanding of the problem and can plan accordingly. Adding figures and links in addition to the text description adds clarity to tickets. Tickets can be created in QuARG, or in an external ticketing system if the analysts have one that they already use.

4. Report: Once all tickets have been created and/or updated for a reporting period, they are used to generate a user-friendly HTML report. The tickets from the database are incorporated into a document that can be easily read and digested. There are three general sections to the report: a summary section that provides an overview of all of the issues listed in the report; a detailed description section that describes the problem with the text, images, and links from the ticket; and a diagnostic section that provides links to a handful of potentially useful sites, as well as a list of the current threshold definitions.

In 2018, IRIS Data Services hosted network operators from across North America to provide information about the MUSTANG quality assurance resources and to introduce QuARG and the concept of our version of a quality assurance report. Since that time, QuARG’s usability has been greatly improved: we have expanded it to include an internal ticketing system, simplified the process to edit thresholds and preference files, added more diagnostics tools, and performed a large overhaul of the aesthetics of the program to be more user friendly and functional.

We are excited to finally have QuARG available to network operators and hope that you will find it useful in tracking issues, planning and prioritizing field work, and generally understanding the state of health of your network.

Software Requirements: – Linux or macOS operating system – Anaconda (https://www.anaconda.com) or Miniconda (http://conda.pydata.org/miniconda.html)