Thread: Timeseries Decimation

Started: 2017-02-08 20:14:53
Last activity: 2017-02-14 05:22:51
Topics: Web Services
jsmith@unavco.org
2017-02-08 20:14:53
Hello IRIS Web Services Folks,
We are using the timeseries web service to get our pore pressure and pore temperature data. For example:
https://service.iris.edu/irisws/timeseries/1/query?net=PB&sta=B078&cha=LKD&start=2010-02-27T06:30:00&end=2010-02-27T10:30:00&output=ascii2&loc=--

The service has been working very well, but ideally, we'd like to be able to use the web service to decimate the data down to one sample every 30s or one sample per minute (from the original 1Hz). I realize that's an eternity in the seismic world, but it's often all we need for this particular dataset. I don't think it's currently possible to do this. I can get a sample every 2 seconds by setting deci=0.5, but most other decimals don't work. For example, 1 sample per ten seconds, deci=0.1, returns an error:
https://service.iris.edu/irisws/timeseries/1/query?net=PB&sta=B078&cha=LKD&start=2010-02-27T06:30:00&end=2010-02-27T10:30:00&deci=0.1&output=ascii2&loc=--

The error is:
ratio of source sample rate [1.0] to final sample rate [0.1] not an integer: 10.0

Even if this did work, it still wouldn't be possible to specify 1 sample per minute (1/60), because it's not a rational number.

Is there some other way to downsample to one sample per minute?


Thanks,
Jeremy Smith

  • Robert Weekly
    2017-02-14 00:46:19
    Hello Jeremy,

    Unfortunately, we can confirm that the irisws-timeseries service is not able to accommodate your particular request for down-sampled temperature and pressure data. Please refer to the service documentation for more details about why your request is running into errors (http://service.iris.edu/irisws/timeseries/docs/1/help/).

    The decimation method used by the service is similar to the DECIMATE function in SAC, but requires the use of an anti-aliasing FIR filter, whereas the anti-alias filter in SAC is considered optional. This FIR filter requirement makes this problem much more complex and is not easily solved with our irisws-timeseries service. We can provide low-pass filtered data for this request, but the subsequent decimation would have to be accomplished with another program outside the scope of the IRIS web services.

    Apologies for not being able to process your request at this time, however, we will consider this use case going forward and look into making select improvements to the irisws-timeseries service in the future.

    Thank you for bringing this to our attention.

    Kindly,
    Robert

    _______________________________
    Robert Weekly
    Quality and Deployment System Engineer
    [e] rtweekly<at>iris.washington.edu


    On Feb 8, 2017, at 12:15 PM, jsmith<at>unavco.org wrote:

    Hello IRIS Web Services Folks,
    We are using the timeseries web service to get our pore pressure and pore temperature data. For example:
    https://service.iris.edu/irisws/timeseries/1/query?net=PB&sta=B078&cha=LKD&start=2010-02-27T06:30:00&end=2010-02-27T10:30:00&output=ascii2&loc=--

    The service has been working very well, but ideally, we'd like to be able to use the web service to decimate the data down to one sample every 30s or one sample per minute (from the original 1Hz). I realize that's an eternity in the seismic world, but it's often all we need for this particular dataset. I don't think it's currently possible to do this. I can get a sample every 2 seconds by setting deci=0.5, but most other decimals don't work. For example, 1 sample per ten seconds, deci=0.1, returns an error:
    https://service.iris.edu/irisws/timeseries/1/query?net=PB&sta=B078&cha=LKD&start=2010-02-27T06:30:00&end=2010-02-27T10:30:00&deci=0.1&output=ascii2&loc=--

    The error is:
    ratio of source sample rate [1.0] to final sample rate [0.1] not an integer: 10.0

    Even if this did work, it still wouldn't be possible to specify 1 sample per minute (1/60), because it's not a rational number.

    Is there some other way to downsample to one sample per minute?


    Thanks,
    Jeremy Smith

    ----------------------
    Web Services (http://ds.iris.edu/message-center/topic/webservices/)

    Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
    Update subscription preferences at http://ds.iris.edu/account/profile/






    • Philip Crotwell
      2017-02-14 05:22:51
      Hi

      Not sure if you are looking for a solution runs locally, but SOD can
      handle your download and decimation needs. The attached recipe will
      download one 4 hour chunk of data for the LKD channel and decimate it
      to 1 sample every 60 seconds, so you have 240 samples for the 4 hour
      window. To run it, download/install SOD and run
      sod -f decimate.recipe -q
      and the output text file will be in the seismograms directory. I also
      attached the output file for this particular time window.

      SOD has lots of options to preprocess data and do querying to find
      stations and channels you are interested in, but for your case this
      simple recipe may be sufficient.

      I did notice one issue in testing this, SOD outputs the ascii data in
      2 columns, which seems kind of dumb. This is easily fixed, so if SOD
      is something you are interested in using, let me know and I will get
      you a bugfix release that will allow you to choose how many columns,
      likely one is the best choice.

      More information and download for SOD is here:
      http://seis.sc.edu/SOD/

      Let me know if you have questions
      Philip



      On Mon, Feb 13, 2017 at 7:47 PM, Robert Weekly
      <rtweekly<at>iris.washington.edu> wrote:
      Hello Jeremy,

      Unfortunately, we can confirm that the irisws-timeseries service is not able to accommodate your particular request for down-sampled temperature and pressure data. Please refer to the service documentation for more details about why your request is running into errors (http://service.iris.edu/irisws/timeseries/docs/1/help/).

      The decimation method used by the service is similar to the DECIMATE function in SAC, but requires the use of an anti-aliasing FIR filter, whereas the anti-alias filter in SAC is considered optional. This FIR filter requirement makes this problem much more complex and is not easily solved with our irisws-timeseries service. We can provide low-pass filtered data for this request, but the subsequent decimation would have to be accomplished with another program outside the scope of the IRIS web services.

      Apologies for not being able to process your request at this time, however, we will consider this use case going forward and look into making select improvements to the irisws-timeseries service in the future.

      Thank you for bringing this to our attention.

      Kindly,
      Robert

      _______________________________
      Robert Weekly
      Quality and Deployment System Engineer
      [e] rtweekly<at>iris.washington.edu


      On Feb 8, 2017, at 12:15 PM, jsmith<at>unavco.org wrote:

      Hello IRIS Web Services Folks,
      We are using the timeseries web service to get our pore pressure and pore temperature data. For example:
      https://service.iris.edu/irisws/timeseries/1/query?net=PB&sta=B078&cha=LKD&start=2010-02-27T06:30:00&end=2010-02-27T10:30:00&output=ascii2&loc=--

      The service has been working very well, but ideally, we'd like to be able to use the web service to decimate the data down to one sample every 30s or one sample per minute (from the original 1Hz). I realize that's an eternity in the seismic world, but it's often all we need for this particular dataset. I don't think it's currently possible to do this. I can get a sample every 2 seconds by setting deci=0.5, but most other decimals don't work. For example, 1 sample per ten seconds, deci=0.1, returns an error:
      https://service.iris.edu/irisws/timeseries/1/query?net=PB&sta=B078&cha=LKD&start=2010-02-27T06:30:00&end=2010-02-27T10:30:00&deci=0.1&output=ascii2&loc=--

      The error is:
      ratio of source sample rate [1.0] to final sample rate [0.1] not an integer: 10.0

      Even if this did work, it still wouldn't be possible to specify 1 sample per minute (1/60), because it's not a rational number.

      Is there some other way to downsample to one sample per minute?


      Thanks,
      Jeremy Smith

      ----------------------
      Web Services (http://ds.iris.edu/message-center/topic/webservices/)

      Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
      Update subscription preferences at http://ds.iris.edu/account/profile/






      ----------------------
      Web Services (http://ds.iris.edu/message-center/topic/webservices/)

      Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
      Update subscription preferences at http://ds.iris.edu/account/profile/
      # PB.B078..LKD 240 6.0E7 microSECOND 20100227T06:30:00.160GMT COUNT
      240 60.0
      +1.499440E01 +2.229574E04
      +2.300799E04 +2.265255E04
      +2.269122E04 +2.268975E04
      +2.268969E04 +2.268898E04
      +2.268898E04 +2.268871E04
      +2.268866E04 +2.268939E04
      +2.268867E04 +2.268938E04
      +2.268824E04 +2.268930E04
      +2.268845E04 +2.268897E04
      +2.268842E04 +2.268903E04
      +2.268861E04 +2.268912E04
      +2.268906E04 +2.268860E04
      +2.268812E04 +2.268844E04
      +2.268814E04 +2.268854E04
      +2.268825E04 +2.268765E04
      +2.268835E04 +2.268823E04
      +2.268793E04 +2.268836E04
      +2.268759E04 +2.268760E04
      +2.268792E04 +2.268754E04
      +2.268762E04 +2.268802E04
      +2.268783E04 +2.268724E04
      +2.268783E04 +2.268773E04
      +2.268724E04 +2.268759E04
      +2.268738E04 +2.268743E04
      +2.268718E04 +2.268710E04
      +2.268746E04 +2.268729E04
      +2.268708E04 +2.268699E04
      +2.268700E04 +2.268689E04
      +2.268699E04 +2.268681E04
      +2.268671E04 +2.268693E04
      +2.268686E04 +2.268683E04
      +2.268646E04 +2.268673E04
      +2.268626E04 +2.268622E04
      +2.268612E04 +2.268657E04
      +2.268649E04 +2.268624E04
      +2.268639E04 +2.268646E04
      +2.268615E04 +2.268614E04
      +2.268596E04 +2.268608E04
      +2.268643E04 +2.268640E04
      +2.268559E04 +2.268588E04
      +2.268613E04 +2.268618E04
      +2.268529E04 +2.268529E04
      +2.268653E04 +2.268535E04
      +2.268568E04 +2.268557E04
      +2.268548E04 +2.268556E04
      +2.268577E04 +2.268528E04
      +2.268513E04 +2.268547E04
      +2.268513E04 +2.268571E04
      +2.268490E04 +2.268490E04
      +2.268574E04 +2.268493E04
      +2.268515E04 +2.268444E04
      +2.268462E04 +2.268474E04
      +2.268502E04 +2.268491E04
      +2.268512E04 +2.268451E04
      +2.268494E04 +2.268428E04
      +2.268447E04 +2.268419E04
      +2.268484E04 +2.268446E04
      +2.268445E04 +2.268436E04
      +2.268379E04 +2.268411E04
      +2.268410E04 +2.268429E04
      +2.268379E04 +2.268341E04
      +2.268428E04 +2.268431E04
      +2.268381E04 +2.268356E04
      +2.268440E04 +2.268355E04
      +2.268386E04 +2.268406E04
      +2.268372E04 +2.268409E04
      +2.268377E04 +2.268340E04
      +2.268351E04 +2.268413E04
      +2.268402E04 +2.268395E04
      +2.268364E04 +2.268439E04
      +2.268382E04 +2.268374E04
      +2.268381E04 +2.268356E04
      +2.268364E04 +2.268379E04
      +2.268375E04 +2.268363E04
      +2.268307E04 +2.268365E04
      +2.268382E04 +2.268272E04
      +2.268359E04 +2.268266E04
      +2.268309E04 +2.268323E04
      +2.268313E04 +2.268309E04
      +2.268314E04 +2.268299E04
      +2.268304E04 +2.268318E04
      +2.268345E04 +2.268245E04
      +2.268317E04 +2.268257E04
      +2.268273E04 +2.268271E04
      +2.268210E04 +2.268228E04
      +2.268245E04 +2.268312E04
      +2.268269E04 +2.268244E04
      +2.268276E04 +2.268240E04
      +2.268242E04 +2.268234E04
      +2.268215E04 +2.268247E04
      +2.268231E04 +2.268217E04
      +2.268260E04 +2.268215E04
      +2.268204E04 +2.268226E04
      +2.268238E04 +2.268224E04
      +2.268209E04 +2.268217E04
      +2.268204E04 +2.268211E04
      +2.268197E04 +2.268198E04
      +2.268209E04 +2.268201E04
      +2.268168E04 +2.268192E04
      +2.268192E04 +2.268195E04
      +2.268169E04 +2.268181E04
      +2.268165E04 +2.268167E04
      +2.268186E04 +2.268166E04
      +2.268175E04 +2.268177E04
      +2.268154E04 +2.268166E04
      +2.268182E04 +2.268182E04
      +2.268165E04 +2.268159E04
      +2.268149E04 +2.268147E04
      +2.268140E04 +2.268160E04
      +2.268136E04 +2.268138E04
      +2.268159E04 +2.268117E04
      +2.268117E04 +2.268159E04
      +2.268154E04 +2.268127E04
      +2.268164E04 +2.268159E04
      +2.268067E04 +2.268081E04
      +2.268160E04 +2.268120E04
      +2.268103E04 +2.268135E04
      +2.268134E04 +2.268130E04
      +2.268091E04 +2.268086E04
      +2.268059E04 +2.268063E04

      Attachments
14:25:46 v.22510d55