Thread: Fwd: MUSTANG pdf problem

Started: 2019-05-04 08:03:27
Last activity: 2019-05-23 07:46:29
Topics: MUSTANG Users
Carl Tape
2019-05-04 08:03:27
Dear Kyle and Gillian,

I'm shifting this discussion to mustang-users, since my understanding is
that this is the correct place for it (not mustang-qa, which is an
announcement board from the DMC to users).

I have attached Kyle's spectrograms here. Note that the missing data do not
correspond to an epoch change. There are other cases where there is a gap
in horizontal but not in vertical, even though the 3-component waveforms
are all complete.

I can reiterate that the number of PSDs that Kyle receives from MUSTANG
appears to be highly time-dependent, something that would be great to
resolve. Usually we detect a swatch of missing PSDs within his home-made
spectrograms, then he retries the download another time and everything is
there. He's been using PSDs for about 4 years now, and we're grateful for
this data product!

Thanks,
Carl

--------------------------------------------------------------
Carl Tape
Associate Professor
Geophysical Institute (office 413D)
University of Alaska Fairbanks
Email: ctape<at>alaska.edu
Web: https://sites.google.com/alaska.edu/carltape
--------------------------------------------------------------

---------- Forwarded message ---------
From: Gillian Sharer <gillian<at>iris.washington.edu>
Date: Sat, May 4, 2019 at 4:53 AM
Subject: Re: [IRIS][mustang-qa] MUSTANG pdf problem
To: MUSTANG QA <mustang-qa<at>lists.ds.iris.edu>


Hi Kyle,

The MUSTANG PDF webservice should not have missing data (except for a
recently identified issue that affects days that have a metadata-epoch
change; this is in the process of being fixed) and data availability should
not change depending on the day. The fact that you are observing this is
concerning to us.

Can you give me more information about how you make requests from the
noise-pdf service to create the spectrogram? There was a problem that
occurred between March-18 to April-19-2019 that potentially affected
multiple-day PDF requests, but it should not have affected single day
requests.

Best regards,
Gillian

On May 3, 2019, at 12:34 PM, Kyle Smith <todakozhi<at>gmail.com> wrote:

Hi MUSTANG QA folks:

I am having trouble downloading some data from the SALMON projects (ZE
network) from MUSTANG pdf webservice:

http://services.iris.edu/mustang/noise-pdf/1/

The data is used to create a spectrogram with a resolution of 1 day. My
example is shown in the attached file WFLW_pdf_spectrogram.png. However,
sometimes time periods can be missing. Data availability can change
depending on the day. I compared my downloads to the new MUSTANG
spectrogram tool in:

http://services.iris.edu/mustang/noise-spectrogram/1/

and got a complete set of data shown in
WFLW_Z_Spectrogram_IRIS_MUSTANG.png. I also experience the same issue with
TA data. My downloaded data was from a few weeks ago but today I checked
the data again and it is available.

My questions are:

Why does the MUSTANG pdf webservice have missing data when there is not
supposed to be?
Why does data availability change depending on the day?
Why does the MUSTANG spectrogram tool have a more complete set of data?

Thank You,

Kyle Smith
<WFLW_Z_Spectrogram_IRIS_MUSTANG.png><WFLW_pdf_spectrogram.png>
----------------------
MUSTANG QA
Topic home: http://ds.iris.edu/message-center/topic/mustang-qa/ |
Unsubscribe: mustang-qa-unsubscribe<at>lists.ds.iris.edu

Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
Update subscription preferences at http://ds.iris.edu/account/profile/


---------------------------------------------------
Gillian Sharer
Deputy Director, Quality Assurance, IRIS DMC
gillian<at>iris.washington.edu
206-547-0393
---------------------------------------------------






----------------------
MUSTANG QA
Topic home: http://ds.iris.edu/message-center/topic/mustang-qa/ |
Unsubscribe: mustang-qa-unsubscribe<at>lists.ds.iris.edu

Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
Update subscription preferences at http://ds.iris.edu/account/profile/

  • Kyle Smith
    2019-05-03 16:30:06
    The data is being downloaded one day at a time and a MATLAB program is
    doing it with the urlread function. My program loops over many days to get
    the data. In the past I have checked data completeness manually for one
    day and it works fine one day but there are issues for another day.

    Thanks,

    Kyle

    On Fri, May 3, 2019 at 4:04 PM Carl Tape <ctape<at>alaska.edu> wrote:

    Dear Kyle and Gillian,

    I'm shifting this discussion to mustang-users, since my understanding is
    that this is the correct place for it (not mustang-qa, which is an
    announcement board from the DMC to users).

    I have attached Kyle's spectrograms here. Note that the missing data do
    not correspond to an epoch change. There are other cases where there is a
    gap in horizontal but not in vertical, even though the 3-component
    waveforms are all complete.

    I can reiterate that the number of PSDs that Kyle receives from MUSTANG
    appears to be highly time-dependent, something that would be great to
    resolve. Usually we detect a swatch of missing PSDs within his home-made
    spectrograms, then he retries the download another time and everything is
    there. He's been using PSDs for about 4 years now, and we're grateful for
    this data product!

    Thanks,
    Carl

    --------------------------------------------------------------
    Carl Tape
    Associate Professor
    Geophysical Institute (office 413D)
    University of Alaska Fairbanks
    Email: ctape<at>alaska.edu
    Web: https://sites.google.com/alaska.edu/carltape
    --------------------------------------------------------------

    ---------- Forwarded message ---------
    From: Gillian Sharer <gillian<at>iris.washington.edu>
    Date: Sat, May 4, 2019 at 4:53 AM
    Subject: Re: [IRIS][mustang-qa] MUSTANG pdf problem
    To: MUSTANG QA <mustang-qa<at>lists.ds.iris.edu>


    Hi Kyle,

    The MUSTANG PDF webservice should not have missing data (except for a
    recently identified issue that affects days that have a metadata-epoch
    change; this is in the process of being fixed) and data availability should
    not change depending on the day. The fact that you are observing this is
    concerning to us.

    Can you give me more information about how you make requests from the
    noise-pdf service to create the spectrogram? There was a problem that
    occurred between March-18 to April-19-2019 that potentially affected
    multiple-day PDF requests, but it should not have affected single day
    requests.

    Best regards,
    Gillian

    On May 3, 2019, at 12:34 PM, Kyle Smith <todakozhi<at>gmail.com> wrote:

    Hi MUSTANG QA folks:

    I am having trouble downloading some data from the SALMON projects (ZE
    network) from MUSTANG pdf webservice:

    http://services.iris.edu/mustang/noise-pdf/1/

    The data is used to create a spectrogram with a resolution of 1 day. My
    example is shown in the attached file WFLW_pdf_spectrogram.png. However,
    sometimes time periods can be missing. Data availability can change
    depending on the day. I compared my downloads to the new MUSTANG
    spectrogram tool in:

    http://services.iris.edu/mustang/noise-spectrogram/1/

    and got a complete set of data shown in
    WFLW_Z_Spectrogram_IRIS_MUSTANG.png. I also experience the same issue with
    TA data. My downloaded data was from a few weeks ago but today I checked
    the data again and it is available.

    My questions are:

    Why does the MUSTANG pdf webservice have missing data when there is not
    supposed to be?
    Why does data availability change depending on the day?
    Why does the MUSTANG spectrogram tool have a more complete set of data?

    Thank You,

    Kyle Smith
    <WFLW_Z_Spectrogram_IRIS_MUSTANG.png><WFLW_pdf_spectrogram.png>
    ----------------------
    MUSTANG QA
    Topic home: http://ds.iris.edu/message-center/topic/mustang-qa/ |
    Unsubscribe: mustang-qa-unsubscribe<at>lists.ds.iris.edu

    Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
    Update subscription preferences at http://ds.iris.edu/account/profile/


    ---------------------------------------------------
    Gillian Sharer
    Deputy Director, Quality Assurance, IRIS DMC
    gillian<at>iris.washington.edu
    206-547-0393
    ---------------------------------------------------






    ----------------------
    MUSTANG QA
    Topic home: http://ds.iris.edu/message-center/topic/mustang-qa/ |
    Unsubscribe: mustang-qa-unsubscribe<at>lists.ds.iris.edu

    Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
    Update subscription preferences at http://ds.iris.edu/account/profile/

    ----------------------
    MUSTANG Users
    Topic home: http://ds.iris.edu/message-center/topic/mustang-users/ |
    Unsubscribe: mustang-users-unsubscribe<at>lists.ds.iris.edu

    Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
    Update subscription preferences at http://ds.iris.edu/account/profile/


    • gillian@iris.washington.edu
      2019-05-06 15:29:57
      Hi Kyle,

      Thank you for the additional information. I assume you didn’t see any errors when you ran your script (such as http failed connection or timeout messages). We were not able to retrieve the log files for your requests but we have increased the number of days that we retain logs before deleting them. When you encounter this problem again, please let us know so that we can check for errors in the logs. We have not been able to replicate this behavior yet, but we’ll keep looking for a solution.

      Best regards,
      Gillian

      On May 3, 2019, at 5:31 PM, Kyle Smith <todakozhi<at>gmail.com> wrote:

      The data is being downloaded one day at a time and a MATLAB program is doing it with the urlread function. My program loops over many days to get the data. In the past I have checked data completeness manually for one day and it works fine one day but there are issues for another day.

      Thanks,

      Kyle

      On Fri, May 3, 2019 at 4:04 PM Carl Tape <ctape<at>alaska.edu <ctape<at>alaska.edu>> wrote:
      Dear Kyle and Gillian,

      I'm shifting this discussion to mustang-users, since my understanding is that this is the correct place for it (not mustang-qa, which is an announcement board from the DMC to users).

      I have attached Kyle's spectrograms here. Note that the missing data do not correspond to an epoch change. There are other cases where there is a gap in horizontal but not in vertical, even though the 3-component waveforms are all complete.

      I can reiterate that the number of PSDs that Kyle receives from MUSTANG appears to be highly time-dependent, something that would be great to resolve. Usually we detect a swatch of missing PSDs within his home-made spectrograms, then he retries the download another time and everything is there. He's been using PSDs for about 4 years now, and we're grateful for this data product!

      Thanks,
      Carl

      --------------------------------------------------------------
      Carl Tape
      Associate Professor
      Geophysical Institute (office 413D)
      University of Alaska Fairbanks
      Email: ctape<at>alaska.edu <ctape<at>alaska.edu>
      Web: https://sites.google.com/alaska.edu/carltape
      --------------------------------------------------------------

      ---------- Forwarded message ---------
      From: Gillian Sharer <gillian<at>iris.washington.edu <gillian<at>iris.washington.edu>>
      Date: Sat, May 4, 2019 at 4:53 AM
      Subject: Re: [IRIS][mustang-qa] MUSTANG pdf problem
      To: MUSTANG QA <mustang-qa<at>lists.ds.iris.edu <mustang-qa<at>lists.ds.iris.edu>>


      Hi Kyle,

      The MUSTANG PDF webservice should not have missing data (except for a recently identified issue that affects days that have a metadata-epoch change; this is in the process of being fixed) and data availability should not change depending on the day. The fact that you are observing this is concerning to us.

      Can you give me more information about how you make requests from the noise-pdf service to create the spectrogram? There was a problem that occurred between March-18 to April-19-2019 that potentially affected multiple-day PDF requests, but it should not have affected single day requests.

      Best regards,
      Gillian

      On May 3, 2019, at 12:34 PM, Kyle Smith <todakozhi<at>gmail.com <todakozhi<at>gmail.com>> wrote:

      Hi MUSTANG QA folks:

      I am having trouble downloading some data from the SALMON projects (ZE network) from MUSTANG pdf webservice:

      http://services.iris.edu/mustang/noise-pdf/1/

      The data is used to create a spectrogram with a resolution of 1 day. My example is shown in the attached file WFLW_pdf_spectrogram.png. However, sometimes time periods can be missing. Data availability can change depending on the day. I compared my downloads to the new MUSTANG spectrogram tool in:

      http://services.iris.edu/mustang/noise-spectrogram/1/

      and got a complete set of data shown in WFLW_Z_Spectrogram_IRIS_MUSTANG.png. I also experience the same issue with TA data. My downloaded data was from a few weeks ago but today I checked the data again and it is available.

      My questions are:

      Why does the MUSTANG pdf webservice have missing data when there is not supposed to be?
      Why does data availability change depending on the day?
      Why does the MUSTANG spectrogram tool have a more complete set of data?

      Thank You,

      Kyle Smith
      <WFLW_Z_Spectrogram_IRIS_MUSTANG.png><WFLW_pdf_spectrogram.png>

      ---------------------------------------------------
      Gillian Sharer
      Deputy Director, Quality Assurance, IRIS DMC
      gillian<at>iris.washington.edu
      206-547-0393
      ---------------------------------------------------






      • Carl Tape
        2019-05-07 08:11:01
        Thanks, Gillian. With the MUSTANG-generated spectrograms, we can compare
        ours vs yours. So we're now in better position to document the issue, the
        next time it arises. Great to hear that you have log files, too.

        Carl

        On Tue, May 7, 2019 at 6:30 AM Gillian Sharer <gillian<at>iris.washington.edu>
        wrote:

        Hi Kyle,

        Thank you for the additional information. I assume you didn’t see any
        errors when you ran your script (such as http failed connection or timeout
        messages). We were not able to retrieve the log files for your requests but
        we have increased the number of days that we retain logs before deleting
        them. When you encounter this problem again, please let us know so that we
        can check for errors in the logs. We have not been able to replicate this
        behavior yet, but we’ll keep looking for a solution.

        Best regards,
        Gillian

        On May 3, 2019, at 5:31 PM, Kyle Smith <todakozhi<at>gmail.com> wrote:

        The data is being downloaded one day at a time and a MATLAB program is
        doing it with the urlread function. My program loops over many days to get
        the data. In the past I have checked data completeness manually for one
        day and it works fine one day but there are issues for another day.

        Thanks,

        Kyle

        On Fri, May 3, 2019 at 4:04 PM Carl Tape <ctape<at>alaska.edu> wrote:

        Dear Kyle and Gillian,

        I'm shifting this discussion to mustang-users, since my understanding is
        that this is the correct place for it (not mustang-qa, which is an
        announcement board from the DMC to users).

        I have attached Kyle's spectrograms here. Note that the missing data do
        not correspond to an epoch change. There are other cases where there is a
        gap in horizontal but not in vertical, even though the 3-component
        waveforms are all complete.

        I can reiterate that the number of PSDs that Kyle receives from MUSTANG
        appears to be highly time-dependent, something that would be great to
        resolve. Usually we detect a swatch of missing PSDs within his home-made
        spectrograms, then he retries the download another time and everything is
        there. He's been using PSDs for about 4 years now, and we're grateful for
        this data product!

        Thanks,
        Carl

        --------------------------------------------------------------
        Carl Tape
        Associate Professor
        Geophysical Institute (office 413D)
        University of Alaska Fairbanks
        Email: ctape<at>alaska.edu
        Web: https://sites.google.com/alaska.edu/carltape
        --------------------------------------------------------------

        ---------- Forwarded message ---------
        From: Gillian Sharer <gillian<at>iris.washington.edu>
        Date: Sat, May 4, 2019 at 4:53 AM
        Subject: Re: [IRIS][mustang-qa] MUSTANG pdf problem
        To: MUSTANG QA <mustang-qa<at>lists.ds.iris.edu>


        Hi Kyle,

        The MUSTANG PDF webservice should not have missing data (except for a
        recently identified issue that affects days that have a metadata-epoch
        change; this is in the process of being fixed) and data availability should
        not change depending on the day. The fact that you are observing this is
        concerning to us.

        Can you give me more information about how you make requests from the
        noise-pdf service to create the spectrogram? There was a problem that
        occurred between March-18 to April-19-2019 that potentially affected
        multiple-day PDF requests, but it should not have affected single day
        requests.

        Best regards,
        Gillian

        On May 3, 2019, at 12:34 PM, Kyle Smith <todakozhi<at>gmail.com> wrote:

        Hi MUSTANG QA folks:

        I am having trouble downloading some data from the SALMON projects (ZE
        network) from MUSTANG pdf webservice:

        http://services.iris.edu/mustang/noise-pdf/1/

        The data is used to create a spectrogram with a resolution of 1 day. My
        example is shown in the attached file WFLW_pdf_spectrogram.png. However,
        sometimes time periods can be missing. Data availability can change
        depending on the day. I compared my downloads to the new MUSTANG
        spectrogram tool in:

        http://services.iris.edu/mustang/noise-spectrogram/1/

        and got a complete set of data shown in
        WFLW_Z_Spectrogram_IRIS_MUSTANG.png. I also experience the same issue with
        TA data. My downloaded data was from a few weeks ago but today I checked
        the data again and it is available.

        My questions are:

        Why does the MUSTANG pdf webservice have missing data when there is not
        supposed to be?
        Why does data availability change depending on the day?
        Why does the MUSTANG spectrogram tool have a more complete set of data?

        Thank You,

        Kyle Smith
        <WFLW_Z_Spectrogram_IRIS_MUSTANG.png><WFLW_pdf_spectrogram.png>


        ---------------------------------------------------
        Gillian Sharer
        Deputy Director, Quality Assurance, IRIS DMC
        gillian<at>iris.washington.edu
        206-547-0393
        ---------------------------------------------------






        ----------------------
        MUSTANG Users
        Topic home: http://ds.iris.edu/message-center/topic/mustang-users/ |
        Unsubscribe: mustang-users-unsubscribe<at>lists.ds.iris.edu

        Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
        Update subscription preferences at http://ds.iris.edu/account/profile/


        • gillian@iris.washington.edu
          2019-05-22 15:59:38
          Hi Kyle and Carl,

          We have tracked down the problem to the rate that your script is making requests of the web service. The load balancer has set limits to the number of connections per minute that can be handled, and this is sometimes being exceeded. The simplest solution might be to slow the script down slightly by using pause or sleep functions. We don’t have a better solution on our end at this time.

          cheers,
          Gillian

          On May 6, 2019, at 5:11 PM, Carl Tape <ctape<at>alaska.edu> wrote:

          Thanks, Gillian. With the MUSTANG-generated spectrograms, we can compare ours vs yours. So we're now in better position to document the issue, the next time it arises. Great to hear that you have log files, too.

          Carl

          On Tue, May 7, 2019 at 6:30 AM Gillian Sharer <gillian<at>iris.washington.edu <gillian<at>iris.washington.edu>> wrote:
          Hi Kyle,

          Thank you for the additional information. I assume you didn’t see any errors when you ran your script (such as http failed connection or timeout messages). We were not able to retrieve the log files for your requests but we have increased the number of days that we retain logs before deleting them. When you encounter this problem again, please let us know so that we can check for errors in the logs. We have not been able to replicate this behavior yet, but we’ll keep looking for a solution.

          Best regards,
          Gillian

          On May 3, 2019, at 5:31 PM, Kyle Smith <todakozhi<at>gmail.com <todakozhi<at>gmail.com>> wrote:

          The data is being downloaded one day at a time and a MATLAB program is doing it with the urlread function. My program loops over many days to get the data. In the past I have checked data completeness manually for one day and it works fine one day but there are issues for another day.

          Thanks,

          Kyle

          On Fri, May 3, 2019 at 4:04 PM Carl Tape <ctape<at>alaska.edu <ctape<at>alaska.edu>> wrote:
          Dear Kyle and Gillian,

          I'm shifting this discussion to mustang-users, since my understanding is that this is the correct place for it (not mustang-qa, which is an announcement board from the DMC to users).

          I have attached Kyle's spectrograms here. Note that the missing data do not correspond to an epoch change. There are other cases where there is a gap in horizontal but not in vertical, even though the 3-component waveforms are all complete.

          I can reiterate that the number of PSDs that Kyle receives from MUSTANG appears to be highly time-dependent, something that would be great to resolve. Usually we detect a swatch of missing PSDs within his home-made spectrograms, then he retries the download another time and everything is there. He's been using PSDs for about 4 years now, and we're grateful for this data product!

          Thanks,
          Carl

          --------------------------------------------------------------
          Carl Tape
          Associate Professor
          Geophysical Institute (office 413D)
          University of Alaska Fairbanks
          Email: ctape<at>alaska.edu <ctape<at>alaska.edu>
          Web: https://sites.google.com/alaska.edu/carltape
          --------------------------------------------------------------

          ---------- Forwarded message ---------
          From: Gillian Sharer <gillian<at>iris.washington.edu <gillian<at>iris.washington.edu>>
          Date: Sat, May 4, 2019 at 4:53 AM
          Subject: Re: [IRIS][mustang-qa] MUSTANG pdf problem
          To: MUSTANG QA <mustang-qa<at>lists.ds.iris.edu <mustang-qa<at>lists.ds.iris.edu>>


          Hi Kyle,

          The MUSTANG PDF webservice should not have missing data (except for a recently identified issue that affects days that have a metadata-epoch change; this is in the process of being fixed) and data availability should not change depending on the day. The fact that you are observing this is concerning to us.

          Can you give me more information about how you make requests from the noise-pdf service to create the spectrogram? There was a problem that occurred between March-18 to April-19-2019 that potentially affected multiple-day PDF requests, but it should not have affected single day requests.

          Best regards,
          Gillian

          On May 3, 2019, at 12:34 PM, Kyle Smith <todakozhi<at>gmail.com <todakozhi<at>gmail.com>> wrote:

          Hi MUSTANG QA folks:

          I am having trouble downloading some data from the SALMON projects (ZE network) from MUSTANG pdf webservice:

          http://services.iris.edu/mustang/noise-pdf/1/

          The data is used to create a spectrogram with a resolution of 1 day. My example is shown in the attached file WFLW_pdf_spectrogram.png. However, sometimes time periods can be missing. Data availability can change depending on the day. I compared my downloads to the new MUSTANG spectrogram tool in:

          http://services.iris.edu/mustang/noise-spectrogram/1/

          and got a complete set of data shown in WFLW_Z_Spectrogram_IRIS_MUSTANG.png. I also experience the same issue with TA data. My downloaded data was from a few weeks ago but today I checked the data again and it is available.

          My questions are:

          Why does the MUSTANG pdf webservice have missing data when there is not supposed to be?
          Why does data availability change depending on the day?
          Why does the MUSTANG spectrogram tool have a more complete set of data?

          Thank You,

          Kyle Smith
          <WFLW_Z_Spectrogram_IRIS_MUSTANG.png><WFLW_pdf_spectrogram.png>


          ---------------------------------------------------
          Gillian Sharer
          Deputy Director, Quality Assurance, IRIS DMC
          gillian<at>iris.washington.edu
          206-547-0393
          ---------------------------------------------------






          • Carl Tape
            2019-05-23 07:46:29
            Hi Gillian,

            This is a tremendously helpful answer! Especially because it is an easy fix
            in a script. It's also possible that some kind of batch submission is
            better, but in any case, I suspect that the problem is solved. We will
            report back if not.

            Thanks,
            Carl

            On Thu, May 23, 2019 at 7:00 AM Gillian Sharer <gillian<at>iris.washington.edu>
            wrote:

            Hi Kyle and Carl,

            We have tracked down the problem to the rate that your script is making
            requests of the web service. The load balancer has set limits to the number
            of connections per minute that can be handled, and this is sometimes being
            exceeded. The simplest solution might be to slow the script down slightly
            by using pause or sleep functions. We don’t have a better solution on our
            end at this time.

            cheers,
            Gillian

            On May 6, 2019, at 5:11 PM, Carl Tape <ctape<at>alaska.edu> wrote:

            Thanks, Gillian. With the MUSTANG-generated spectrograms, we can compare
            ours vs yours. So we're now in better position to document the issue, the
            next time it arises. Great to hear that you have log files, too.

            Carl

            On Tue, May 7, 2019 at 6:30 AM Gillian Sharer <gillian<at>iris.washington.edu>
            wrote:

            Hi Kyle,

            Thank you for the additional information. I assume you didn’t see any
            errors when you ran your script (such as http failed connection or timeout
            messages). We were not able to retrieve the log files for your requests but
            we have increased the number of days that we retain logs before deleting
            them. When you encounter this problem again, please let us know so that we
            can check for errors in the logs. We have not been able to replicate this
            behavior yet, but we’ll keep looking for a solution.

            Best regards,
            Gillian

            On May 3, 2019, at 5:31 PM, Kyle Smith <todakozhi<at>gmail.com> wrote:

            The data is being downloaded one day at a time and a MATLAB program is
            doing it with the urlread function. My program loops over many days to get
            the data. In the past I have checked data completeness manually for one
            day and it works fine one day but there are issues for another day.

            Thanks,

            Kyle

            On Fri, May 3, 2019 at 4:04 PM Carl Tape <ctape<at>alaska.edu> wrote:

            Dear Kyle and Gillian,

            I'm shifting this discussion to mustang-users, since my understanding is
            that this is the correct place for it (not mustang-qa, which is an
            announcement board from the DMC to users).

            I have attached Kyle's spectrograms here. Note that the missing data do
            not correspond to an epoch change. There are other cases where there is a
            gap in horizontal but not in vertical, even though the 3-component
            waveforms are all complete.

            I can reiterate that the number of PSDs that Kyle receives from MUSTANG
            appears to be highly time-dependent, something that would be great to
            resolve. Usually we detect a swatch of missing PSDs within his home-made
            spectrograms, then he retries the download another time and everything is
            there. He's been using PSDs for about 4 years now, and we're grateful for
            this data product!

            Thanks,
            Carl

            --------------------------------------------------------------
            Carl Tape
            Associate Professor
            Geophysical Institute (office 413D)
            University of Alaska Fairbanks
            Email: ctape<at>alaska.edu
            Web: https://sites.google.com/alaska.edu/carltape
            --------------------------------------------------------------

            ---------- Forwarded message ---------
            From: Gillian Sharer <gillian<at>iris.washington.edu>
            Date: Sat, May 4, 2019 at 4:53 AM
            Subject: Re: [IRIS][mustang-qa] MUSTANG pdf problem
            To: MUSTANG QA <mustang-qa<at>lists.ds.iris.edu>


            Hi Kyle,

            The MUSTANG PDF webservice should not have missing data (except for a
            recently identified issue that affects days that have a metadata-epoch
            change; this is in the process of being fixed) and data availability should
            not change depending on the day. The fact that you are observing this is
            concerning to us.

            Can you give me more information about how you make requests from the
            noise-pdf service to create the spectrogram? There was a problem that
            occurred between March-18 to April-19-2019 that potentially affected
            multiple-day PDF requests, but it should not have affected single day
            requests.

            Best regards,
            Gillian

            On May 3, 2019, at 12:34 PM, Kyle Smith <todakozhi<at>gmail.com> wrote:

            Hi MUSTANG QA folks:

            I am having trouble downloading some data from the SALMON projects (ZE
            network) from MUSTANG pdf webservice:

            http://services.iris.edu/mustang/noise-pdf/1/

            The data is used to create a spectrogram with a resolution of 1 day. My
            example is shown in the attached file WFLW_pdf_spectrogram.png. However,
            sometimes time periods can be missing. Data availability can change
            depending on the day. I compared my downloads to the new MUSTANG
            spectrogram tool in:

            http://services.iris.edu/mustang/noise-spectrogram/1/

            and got a complete set of data shown in
            WFLW_Z_Spectrogram_IRIS_MUSTANG.png. I also experience the same issue with
            TA data. My downloaded data was from a few weeks ago but today I checked
            the data again and it is available.

            My questions are:

            Why does the MUSTANG pdf webservice have missing data when there is not
            supposed to be?
            Why does data availability change depending on the day?
            Why does the MUSTANG spectrogram tool have a more complete set of data?

            Thank You,

            Kyle Smith
            <WFLW_Z_Spectrogram_IRIS_MUSTANG.png><WFLW_pdf_spectrogram.png>



            ---------------------------------------------------
            Gillian Sharer
            Deputy Director, Quality Assurance, IRIS DMC
            gillian<at>iris.washington.edu
            206-547-0393
            ---------------------------------------------------






            ----------------------
            MUSTANG Users
            Topic home: http://ds.iris.edu/message-center/topic/mustang-users/ |
            Unsubscribe: mustang-users-unsubscribe<at>lists.ds.iris.edu

            Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
            Update subscription preferences at http://ds.iris.edu/account/profile/


04:00:57 v.22510d55