Thread: Re: More Stations service problems

Started: 2023-02-08 16:20:54
Last activity: 2023-02-09 11:52:03
Topics: Web Services
Sebastian Heimann
2023-02-08 16:20:54
Retrying, sorry my previous mail got incorrectly formatted probably
because of the included PGP signature.


Hi!

I think I am running into the same type of problems as has been reported
previously. I find that about 1 in 30 requests returns corrupt
StationXML. Looks like there is e.g. a stray debug print statement in
the server code, printing the size of the following block as a hex value.

Here's an excerpt from one of the corrupt results (the `2000` lines are
the problem):

```
2000
<?xml version="1.0" encoding="ISO-8859-1"?>

<FDSNStationXML xmlns="http://www.fdsn.org/xml/station/1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:iris="http://www.fdsn.org/xml/station/1/iris"
xsi:schemaLocation="http://www.fdsn.org/xml/station/1
http://www.fdsn.org/xml/station/fdsn-station-1.1.xsd" schemaVersion="1.1">
<Source>IRIS-DMC</Source>

...

<NumeratorCoefficient i="12">-9.78385E-7</NumeratorCoefficient>
<NumeratorCoefficient i="13">3.18969E-5</NumeratorCoefficient>
<NumeratorCoefficient i="14">1.68242E-4</NumeratorCoefficient>
<Nume
2000
ratorCoefficient i="15">-3.49811E-4</NumeratorCoefficient>
<NumeratorCoefficient i="16">-5.52227E-4</NumeratorCoefficient>
<NumeratorCoefficient i="17">7.31389E-4</NumeratorCoefficient>
<NumeratorCoefficient i="18">-0.00108921</NumeratorCoefficient>
<NumeratorCoefficient i="19">7.87631E-4</NumeratorCoefficient>

...

```

All blocks have size 0x2000 except for the last one.

Additionally the end a superfluous `0` is printed, I guess the size of
the print buffer is printed under some conditions to the response.

```
</Response>
</Channel>
</Station>
</Network>
</FDSNStationXML>
0
```

Strange that this only happens on about every 30th request. Repeating
the request with the same parameters usually returns a correct result.

Hope this helps to find the problem.

Best wishes
Sebastian




PS: here is some hotfix code for the client side to clean the possibly
corrupt StationXML after retrieval:


import io

def fix_iris_bug(data):
fin = io.BytesIO(data)
fout = io.BytesIO()
start = True
buggy = False
while True:
line = fin.readline()
if not line:
break

if start:
try:
iexpect = int(line, 16) + len(line)
buggy = True
ipos = 0
except ValueError:
pass

start = False

else:
ipos += len(line)
if buggy and ipos >= iexpect:
iexpect = int(line, 16) + len(line)
ipos = 0
else:
fout.write(line)

return fout.getvalue()



  • Rob Casey
    2023-02-08 13:48:29

    Hi Sebastian-

    Thank you for reporting this!

    One of our staff also saw these 2000's pop up. I thought that it might have been the intermixing of curl download counter output getting worked into the stream, but here you are using Python and I am assuming that there is not additional output being generated from your client.

    Very good analysis that this number is actually 0x2000, which would be 8192 or 8K bytes. Very odd that this happens at all, but happens seemingly randomly. Given this information, I wonder if the developers I have CC'ed have any theories as to why 8K is being sent with the payload parts?

    Regards,

    -Rob

    On Feb 8, 2023, at 7:21 AM, Sebastian Heimann (via IRIS) <webservices-bounce<at>lists.ds.iris.edu> wrote:

    Retrying, sorry my previous mail got incorrectly formatted probably
    because of the included PGP signature.


    Hi!

    I think I am running into the same type of problems as has been reported
    previously. I find that about 1 in 30 requests returns corrupt
    StationXML. Looks like there is e.g. a stray debug print statement in
    the server code, printing the size of the following block as a hex value.

    Here's an excerpt from one of the corrupt results (the `2000` lines are
    the problem):

    ```
    2000
    <?xml version="1.0" encoding="ISO-8859-1"?>

    <FDSNStationXML xmlns="http://www.fdsn.org/xml/station/1"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:iris="http://www.fdsn.org/xml/station/1/iris"
    xsi:schemaLocation="http://www.fdsn.org/xml/station/1
    http://www.fdsn.org/xml/station/fdsn-station-1.1.xsd" schemaVersion="1.1">
    <Source>IRIS-DMC</Source>

    ...

    <NumeratorCoefficient i="12">-9.78385E-7</NumeratorCoefficient>
    <NumeratorCoefficient i="13">3.18969E-5</NumeratorCoefficient>
    <NumeratorCoefficient i="14">1.68242E-4</NumeratorCoefficient>
    <Nume
    2000
    ratorCoefficient i="15">-3.49811E-4</NumeratorCoefficient>
    <NumeratorCoefficient i="16">-5.52227E-4</NumeratorCoefficient>
    <NumeratorCoefficient i="17">7.31389E-4</NumeratorCoefficient>
    <NumeratorCoefficient i="18">-0.00108921</NumeratorCoefficient>
    <NumeratorCoefficient i="19">7.87631E-4</NumeratorCoefficient>

    ...

    ```

    All blocks have size 0x2000 except for the last one.

    Additionally the end a superfluous `0` is printed, I guess the size of
    the print buffer is printed under some conditions to the response.

    ```
    </Response>
    </Channel>
    </Station>
    </Network>
    </FDSNStationXML>
    0
    ```

    Strange that this only happens on about every 30th request. Repeating
    the request with the same parameters usually returns a correct result.

    Hope this helps to find the problem.

    Best wishes
    Sebastian




    PS: here is some hotfix code for the client side to clean the possibly
    corrupt StationXML after retrieval:


    import io

    def fix_iris_bug(data):
    fin = io.BytesIO(data)
    fout = io.BytesIO()
    start = True
    buggy = False
    while True:
    line = fin.readline()
    if not line:
    break

    if start:
    try:
    iexpect = int(line, 16) + len(line)
    buggy = True
    ipos = 0
    except ValueError:
    pass

    start = False

    else:
    ipos += len(line)
    if buggy and ipos >= iexpect:
    iexpect = int(line, 16) + len(line)
    ipos = 0
    else:
    fout.write(line)

    return fout.getvalue()



    ----------------------
    Web Services
    Topic home: http://ds.iris.edu/message-center/topic/webservices/ | Unsubscribe: webservices-unsubscribe<at>lists.ds.iris.edu

    Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
    Update subscription preferences at http://ds.iris.edu/account/profile/



    • Andres Heinloo
      2023-02-08 22:58:59
      Hi,

      could it be related to HTTP chunked encoding, eg., incorrect
      Transfer-Encoding header or client not interpreting it correctly?

      Regards,
      Andres



      On 2/8/23 22:49, Rob Casey (via IRIS) wrote:

      Hi Sebastian-

      Thank you for reporting this!

      One of our staff also saw these 2000's pop up. I thought that it might have been the intermixing of curl download counter output getting worked into the stream, but here you are using Python and I am assuming that there is not additional output being generated from your client.

      Very good analysis that this number is actually 0x2000, which would be 8192 or 8K bytes. Very odd that this happens at all, but happens seemingly randomly. Given this information, I wonder if the developers I have CC'ed have any theories as to why 8K is being sent with the payload parts?

      Regards,

      -Rob

      On Feb 8, 2023, at 7:21 AM, Sebastian Heimann (via IRIS) <webservices-bounce<at>lists.ds.iris.edu> wrote:

      Retrying, sorry my previous mail got incorrectly formatted probably
      because of the included PGP signature.


      Hi!

      I think I am running into the same type of problems as has been reported
      previously. I find that about 1 in 30 requests returns corrupt
      StationXML. Looks like there is e.g. a stray debug print statement in
      the server code, printing the size of the following block as a hex value.

      Here's an excerpt from one of the corrupt results (the `2000` lines are
      the problem):

      ```
      2000
      <?xml version="1.0" encoding="ISO-8859-1"?>

      <FDSNStationXML xmlns="http://www.fdsn.org/xml/station/1"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xmlns:iris="http://www.fdsn.org/xml/station/1/iris"
      xsi:schemaLocation="http://www.fdsn.org/xml/station/1
      http://www.fdsn.org/xml/station/fdsn-station-1.1.xsd" schemaVersion="1.1">
      <Source>IRIS-DMC</Source>

      ...

      <NumeratorCoefficient i="12">-9.78385E-7</NumeratorCoefficient>
      <NumeratorCoefficient i="13">3.18969E-5</NumeratorCoefficient>
      <NumeratorCoefficient i="14">1.68242E-4</NumeratorCoefficient>
      <Nume
      2000
      ratorCoefficient i="15">-3.49811E-4</NumeratorCoefficient>
      <NumeratorCoefficient i="16">-5.52227E-4</NumeratorCoefficient>
      <NumeratorCoefficient i="17">7.31389E-4</NumeratorCoefficient>
      <NumeratorCoefficient i="18">-0.00108921</NumeratorCoefficient>
      <NumeratorCoefficient i="19">7.87631E-4</NumeratorCoefficient>

      ...

      ```

      All blocks have size 0x2000 except for the last one.

      Additionally the end a superfluous `0` is printed, I guess the size of
      the print buffer is printed under some conditions to the response.

      ```
      </Response>
      </Channel>
      </Station>
      </Network>
      </FDSNStationXML>
      0
      ```

      Strange that this only happens on about every 30th request. Repeating
      the request with the same parameters usually returns a correct result.

      Hope this helps to find the problem.

      Best wishes
      Sebastian




      PS: here is some hotfix code for the client side to clean the possibly
      corrupt StationXML after retrieval:


      import io

      def fix_iris_bug(data):
      fin = io.BytesIO(data)
      fout = io.BytesIO()
      start = True
      buggy = False
      while True:
      line = fin.readline()
      if not line:
      break

      if start:
      try:
      iexpect = int(line, 16) + len(line)
      buggy = True
      ipos = 0
      except ValueError:
      pass

      start = False

      else:
      ipos += len(line)
      if buggy and ipos >= iexpect:
      iexpect = int(line, 16) + len(line)
      ipos = 0
      else:
      fout.write(line)

      return fout.getvalue()



      ----------------------
      Web Services
      Topic home: http://ds.iris.edu/message-center/topic/webservices/ | Unsubscribe: webservices-unsubscribe<at>lists.ds.iris.edu

      Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
      Update subscription preferences at http://ds.iris.edu/account/profile/



      ----------------------
      Web Services
      Topic home: http://ds.iris.edu/message-center/topic/webservices/ | Unsubscribe: webservices-unsubscribe<at>lists.ds.iris.edu

      Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
      Update subscription preferences at http://ds.iris.edu/account/profile/


      • Sebastian Heimann
        2023-02-09 11:52:03
        Hi Andres,

        yes, this is the reason.

        If it works, the Transfer-Encoding setting is set correctly:


        ```
        HTTP/1.1 200
        Access-Control-Allow-Origin: *
        Access-Control-Expose-Headers:
        Access-Control-Allow-Origin,Access-Control-Allow-Credentials
        Vary:
        origin,access-control-request-method,access-control-request-headers,accept-encoding
        Content-Disposition: inline;
        filename=fdsn-station_2023-02-09T10:31:08.4770.xml
        Date: Thu, 09 Feb 2023 10:31:08 GMT
        X-XSS-Protection: 1; mode=block
        Cache-Control: no-cache, no-store, max-age=0, must-revalidate
        Pragma: no-cache
        Expires: 0
        X-Frame-Options: DENY
        Content-Type: application/xml
        Transfer-Encoding: chunked
        Date: Thu, 09 Feb 2023 10:31:08 GMT
        Connection: close

        2000
        <?xml version="1.0" encoding="ISO-8859-1"?>
        ```

        If it doesn't work, this header is missing:

        ```
        HTTP/1.1 200
        Access-Control-Allow-Origin: *
        Access-Control-Expose-Headers:
        Access-Control-Allow-Origin,Access-Control-Allow-Credentials
        Vary:
        origin,access-control-request-method,access-control-request-headers,accept-encoding
        Content-Disposition: inline;
        filename=fdsn-station_2023-02-09T10:31:25.7000.xml
        X-Content-Type-Options: nosniff
        X-XSS-Protection: 1; mode=block
        Cache-Control: no-cache, no-store, max-age=0, must-revalidate
        Pragma: no-cache
        Expires: 0
        X-Frame-Options: DENY
        Content-Type: application/xml
        Date: Thu, 09 Feb 2023 10:31:25 GMT
        Date: Thu, 09 Feb 2023 10:31:25 GMT
        Connection: close

        2000
        <?xml version="1.0" encoding="ISO-8859-1"?>
        ```

        Best wishes
        Sebastian




        On 08.02.23 23:00, Andres Heinloo (via IRIS) wrote:
        Hi,

        could it be related to HTTP chunked encoding, eg., incorrect
        Transfer-Encoding header or client not interpreting it correctly?

        Regards,
        Andres



        On 2/8/23 22:49, Rob Casey (via IRIS) wrote:

        Hi Sebastian-

        Thank you for reporting this!

        One of our staff also saw these 2000's pop up. I thought that it might have been the intermixing of curl download counter output getting worked into the stream, but here you are using Python and I am assuming that there is not additional output being generated from your client.

        Very good analysis that this number is actually 0x2000, which would be 8192 or 8K bytes. Very odd that this happens at all, but happens seemingly randomly. Given this information, I wonder if the developers I have CC'ed have any theories as to why 8K is being sent with the payload parts?

        Regards,

        -Rob

        On Feb 8, 2023, at 7:21 AM, Sebastian Heimann (via IRIS) <webservices-bounce<at>lists.ds.iris.edu> wrote:

        Retrying, sorry my previous mail got incorrectly formatted probably
        because of the included PGP signature.


        Hi!

        I think I am running into the same type of problems as has been reported
        previously. I find that about 1 in 30 requests returns corrupt
        StationXML. Looks like there is e.g. a stray debug print statement in
        the server code, printing the size of the following block as a hex value.

        Here's an excerpt from one of the corrupt results (the `2000` lines are
        the problem):

        ```
        2000
        <?xml version="1.0" encoding="ISO-8859-1"?>

        <FDSNStationXML xmlns="http://www.fdsn.org/xml/station/1"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns:iris="http://www.fdsn.org/xml/station/1/iris"
        xsi:schemaLocation="http://www.fdsn.org/xml/station/1
        http://www.fdsn.org/xml/station/fdsn-station-1.1.xsd" schemaVersion="1.1">
        <Source>IRIS-DMC</Source>

        ...

        <NumeratorCoefficient i="12">-9.78385E-7</NumeratorCoefficient>
        <NumeratorCoefficient i="13">3.18969E-5</NumeratorCoefficient>
        <NumeratorCoefficient i="14">1.68242E-4</NumeratorCoefficient>
        <Nume
        2000
        ratorCoefficient i="15">-3.49811E-4</NumeratorCoefficient>
        <NumeratorCoefficient i="16">-5.52227E-4</NumeratorCoefficient>
        <NumeratorCoefficient i="17">7.31389E-4</NumeratorCoefficient>
        <NumeratorCoefficient i="18">-0.00108921</NumeratorCoefficient>
        <NumeratorCoefficient i="19">7.87631E-4</NumeratorCoefficient>

        ...

        ```

        All blocks have size 0x2000 except for the last one.

        Additionally the end a superfluous `0` is printed, I guess the size of
        the print buffer is printed under some conditions to the response.

        ```
        </Response>
        </Channel>
        </Station>
        </Network>
        </FDSNStationXML>
        0
        ```

        Strange that this only happens on about every 30th request. Repeating
        the request with the same parameters usually returns a correct result.

        Hope this helps to find the problem.

        Best wishes
        Sebastian




        PS: here is some hotfix code for the client side to clean the possibly
        corrupt StationXML after retrieval:


        import io

        def fix_iris_bug(data):
        fin = io.BytesIO(data)
        fout = io.BytesIO()
        start = True
        buggy = False
        while True:
        line = fin.readline()
        if not line:
        break

        if start:
        try:
        iexpect = int(line, 16) + len(line)
        buggy = True
        ipos = 0
        except ValueError:
        pass

        start = False

        else:
        ipos += len(line)
        if buggy and ipos >= iexpect:
        iexpect = int(line, 16) + len(line)
        ipos = 0
        else:
        fout.write(line)

        return fout.getvalue()



        ----------------------
        Web Services
        Topic home: http://ds.iris.edu/message-center/topic/webservices/ | Unsubscribe: webservices-unsubscribe<at>lists.ds.iris.edu

        Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
        Update subscription preferences at http://ds.iris.edu/account/profile/



        ----------------------
        Web Services
        Topic home: http://ds.iris.edu/message-center/topic/webservices/ | Unsubscribe: webservices-unsubscribe<at>lists.ds.iris.edu

        Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
        Update subscription preferences at http://ds.iris.edu/account/profile/


        ----------------------
        Web Services
        Topic home: http://ds.iris.edu/message-center/topic/webservices/ | Unsubscribe: webservices-unsubscribe<at>lists.ds.iris.edu

        Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
        Update subscription preferences at http://ds.iris.edu/account/profile/

        • Rob Casey
          2023-02-09 09:18:37

          Thank you for doing this expert investigation, Sebastian and Andres. The fact that it flip-flops makes me wonder if certain load balanced servers are not configured correctly. I don't know why the header would turn on and off intermittently otherwise, though there could be single-server causes as well.

          I'll engage the relevant staff members to discuss this matter.

          Very much appreciated!

          -Rob



          On Feb 9, 2023, at 2:53 AM, Sebastian Heimann (via IRIS) <webservices-bounce<at>lists.ds.iris.edu> wrote:

          Hi Andres,

          yes, this is the reason.

          If it works, the Transfer-Encoding setting is set correctly:


          ```
          HTTP/1.1 200
          Access-Control-Allow-Origin: *
          Access-Control-Expose-Headers:
          Access-Control-Allow-Origin,Access-Control-Allow-Credentials
          Vary:
          origin,access-control-request-method,access-control-request-headers,accept-encoding
          Content-Disposition: inline;
          filename=fdsn-station_2023-02-09T10:31:08.4770.xml
          Date: Thu, 09 Feb 2023 10:31:08 GMT
          X-XSS-Protection: 1; mode=block
          Cache-Control: no-cache, no-store, max-age=0, must-revalidate
          Pragma: no-cache
          Expires: 0
          X-Frame-Options: DENY
          Content-Type: application/xml
          Transfer-Encoding: chunked
          Date: Thu, 09 Feb 2023 10:31:08 GMT
          Connection: close

          2000
          <?xml version="1.0" encoding="ISO-8859-1"?>
          ```

          If it doesn't work, this header is missing:

          ```
          HTTP/1.1 200
          Access-Control-Allow-Origin: *
          Access-Control-Expose-Headers:
          Access-Control-Allow-Origin,Access-Control-Allow-Credentials
          Vary:
          origin,access-control-request-method,access-control-request-headers,accept-encoding
          Content-Disposition: inline;
          filename=fdsn-station_2023-02-09T10:31:25.7000.xml
          X-Content-Type-Options: nosniff
          X-XSS-Protection: 1; mode=block
          Cache-Control: no-cache, no-store, max-age=0, must-revalidate
          Pragma: no-cache
          Expires: 0
          X-Frame-Options: DENY
          Content-Type: application/xml
          Date: Thu, 09 Feb 2023 10:31:25 GMT
          Date: Thu, 09 Feb 2023 10:31:25 GMT
          Connection: close

          2000
          <?xml version="1.0" encoding="ISO-8859-1"?>
          ```

          Best wishes
          Sebastian




          On 08.02.23 23:00, Andres Heinloo (via IRIS) wrote:
          Hi,

          could it be related to HTTP chunked encoding, eg., incorrect
          Transfer-Encoding header or client not interpreting it correctly?

          Regards,
          Andres



          On 2/8/23 22:49, Rob Casey (via IRIS) wrote:

          Hi Sebastian-

          Thank you for reporting this!

          One of our staff also saw these 2000's pop up. I thought that it might have been the intermixing of curl download counter output getting worked into the stream, but here you are using Python and I am assuming that there is not additional output being generated from your client.

          Very good analysis that this number is actually 0x2000, which would be 8192 or 8K bytes. Very odd that this happens at all, but happens seemingly randomly. Given this information, I wonder if the developers I have CC'ed have any theories as to why 8K is being sent with the payload parts?

          Regards,

          -Rob

          On Feb 8, 2023, at 7:21 AM, Sebastian Heimann (via IRIS) <webservices-bounce<at>lists.ds.iris.edu> wrote:

          Retrying, sorry my previous mail got incorrectly formatted probably
          because of the included PGP signature.


          Hi!

          I think I am running into the same type of problems as has been reported
          previously. I find that about 1 in 30 requests returns corrupt
          StationXML. Looks like there is e.g. a stray debug print statement in
          the server code, printing the size of the following block as a hex value.

          Here's an excerpt from one of the corrupt results (the `2000` lines are
          the problem):

          ```
          2000
          <?xml version="1.0" encoding="ISO-8859-1"?>

          <FDSNStationXML xmlns="http://www.fdsn.org/xml/station/1"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xmlns:iris="http://www.fdsn.org/xml/station/1/iris"
          xsi:schemaLocation="http://www.fdsn.org/xml/station/1
          http://www.fdsn.org/xml/station/fdsn-station-1.1.xsd" schemaVersion="1.1">
          <Source>IRIS-DMC</Source>

          ...

          <NumeratorCoefficient i="12">-9.78385E-7</NumeratorCoefficient>
          <NumeratorCoefficient i="13">3.18969E-5</NumeratorCoefficient>
          <NumeratorCoefficient i="14">1.68242E-4</NumeratorCoefficient>
          <Nume
          2000
          ratorCoefficient i="15">-3.49811E-4</NumeratorCoefficient>
          <NumeratorCoefficient i="16">-5.52227E-4</NumeratorCoefficient>
          <NumeratorCoefficient i="17">7.31389E-4</NumeratorCoefficient>
          <NumeratorCoefficient i="18">-0.00108921</NumeratorCoefficient>
          <NumeratorCoefficient i="19">7.87631E-4</NumeratorCoefficient>

          ...

          ```

          All blocks have size 0x2000 except for the last one.

          Additionally the end a superfluous `0` is printed, I guess the size of
          the print buffer is printed under some conditions to the response.

          ```
          </Response>
          </Channel>
          </Station>
          </Network>
          </FDSNStationXML>
          0
          ```

          Strange that this only happens on about every 30th request. Repeating
          the request with the same parameters usually returns a correct result.

          Hope this helps to find the problem.

          Best wishes
          Sebastian




          PS: here is some hotfix code for the client side to clean the possibly
          corrupt StationXML after retrieval:


          import io

          def fix_iris_bug(data):
          fin = io.BytesIO(data)
          fout = io.BytesIO()
          start = True
          buggy = False
          while True:
          line = fin.readline()
          if not line:
          break

          if start:
          try:
          iexpect = int(line, 16) + len(line)
          buggy = True
          ipos = 0
          except ValueError:
          pass

          start = False

          else:
          ipos += len(line)
          if buggy and ipos >= iexpect:
          iexpect = int(line, 16) + len(line)
          ipos = 0
          else:
          fout.write(line)

          return fout.getvalue()



          ----------------------
          Web Services
          Topic home: http://ds.iris.edu/message-center/topic/webservices/ | Unsubscribe: webservices-unsubscribe<at>lists.ds.iris.edu

          Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
          Update subscription preferences at http://ds.iris.edu/account/profile/



          ----------------------
          Web Services
          Topic home: http://ds.iris.edu/message-center/topic/webservices/ | Unsubscribe: webservices-unsubscribe<at>lists.ds.iris.edu

          Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
          Update subscription preferences at http://ds.iris.edu/account/profile/


          ----------------------
          Web Services
          Topic home: http://ds.iris.edu/message-center/topic/webservices/ | Unsubscribe: webservices-unsubscribe<at>lists.ds.iris.edu

          Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
          Update subscription preferences at http://ds.iris.edu/account/profile/

          ----------------------
          Web Services
          Topic home: http://ds.iris.edu/message-center/topic/webservices/ | Unsubscribe: webservices-unsubscribe<at>lists.ds.iris.edu

          Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
          Update subscription preferences at http://ds.iris.edu/account/profile/



22:09:19 v.22510d55