Thread: intermittent problems with station service

Started: 2024-05-28 20:18:24
Last activity: 2024-05-29 12:26:38
Topics: Web Services
John West
2024-05-28 20:18:24
Hi!

I'm calling the station service using the obspy client.get_stations()
method, and getting randomly distributed badly formed XML.

Typical error message is something like: "Start tag expected, '<' not
found, line 1, column 1 (<string>, line 1)"

I've also had a few errors with: "Not a gzipped file (b'a\r')"

The attached sample Python code sometimes runs without error, and sometimes
generates multiple errors during a run. The errors are NOT in the same
places, they appear for different networks on each run.

Any ideas?

Thanks!
-- John

  • Philip Crotwell
    2024-05-29 12:26:38
    Hi

    Not sure I have anything useful to add, but I too have seen random,
    unrepeatable errors with the fdsn web services. Often retrying the failed
    query succeeds immediately afterwards, so I do not think this is a data
    problem. If anything, it feels like maybe a threading race condition type
    of error, and the reproducibility problem is exacerbated, I assume, by
    there being more than one instance of the web service behind the load
    balancer. And of course it can be really hard to find and resolve these
    sorts of issues unless you can recover the actual raw response to the query.

    One thing you might try is getting debug output from obspy, like

    DCclient = Client('IRIS', debug=True)

    This might provide additional information if you can catch one of the
    errors with it enabled.

    I tried and caught this. Immediately afterwards, it ran fine, so .

    Downloaded http://service.iris.edu/fdsnws/event/1/application.wadl with
    HTTP code: 200
    Uncompressing gzipped response for
    http://service.iris.edu/fdsnws/dataselect/1/application.wadl
    Downloaded http://service.iris.edu/fdsnws/dataselect/1/application.wadl
    with HTTP code: 200
    Traceback (most recent call last):
    File "/Users/crotwell/Code/GitHub/obspy/JustTesting.py", line 21, in
    <module>
    DCclient = Client('IRIS', debug=True)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
    File
    "/opt/homebrew/Caskroom/miniforge/base/envs/obspy/lib/python3.12/site-packages/obspy/clients/fdsn/client.py",
    line 277, in __init__
    self._discover_services()
    File
    "/opt/homebrew/Caskroom/miniforge/base/envs/obspy/lib/python3.12/site-packages/obspy/clients/fdsn/client.py",
    line 1584, in _discover_services
    decoded_wadl = wadl.decode('utf-8')
    ^^^^^^^^^^^^^^^^^^^^
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1:
    invalid start byte

    Doubt this is very helpful, but at least you know you are not the only one?

    Philip


    On Tue, May 28, 2024 at 10:23 PM John West (via SAGE) <
    webservices-bounce<at>lists.ds.iris.edu> wrote:

    Hi!

    I'm calling the station service using the obspy client.get_stations()
    method, and getting randomly distributed badly formed XML.

    Typical error message is something like: "Start tag expected, '<' not
    found, line 1, column 1 (<string>, line 1)"

    I've also had a few errors with: "Not a gzipped file (b'a\r')"

    The attached sample Python code sometimes runs without error, and
    sometimes generates multiple errors during a run. The errors are NOT in the
    same places, they appear for different networks on each run.

    Any ideas?

    Thanks!
    -- John

    ----------------------
    Web Services
    Topic home: http://ds.iris.edu/message-center/topic/webservices/ |
    Unsubscribe: webservices-unsubscribe<at>lists.ds.iris.edu

    Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
    Update subscription preferences at http://ds.iris.edu/account/profile/


    • John West
      2024-05-29 10:50:51
      Thanks, Philip.

      I turned on debug and continued to get the xml parsing error, but no
      additional information. The invalid start byte error you got seems to be
      less common, I haven't seen it in a while.

      The specific error message I get is:
      Downloading
      http://service.iris.edu/fdsnws/station/1/query?network=SB&channel=HH%2A%2CBH%2A&level=channel&startbefore=2024-04-09T09%3A48%3A01.603000&endafter=2024-04-09T11%3A48%3A01.603000
      with requesting gzip compression
      Downloaded
      http://service.iris.edu/fdsnws/station/1/query?network=SB&channel=HH%2A%2CBH%2A&level=channel&startbefore=2024-04-09T09%3A48%3A01.603000&endafter=2024-04-09T11%3A48%3A01.603000
      with HTTP code: 200
      could not retrieve channels for datacenter IRIS: Network = SB Start tag
      expected, '<' not found, line 1, column 1 (<string>, line 1) Traceback
      (most recent call last):
      File "/home/jdw/Dropbox/SWAT/JustTesting.py", line 29, in <module>
      Channels = DCclient.get_stations(startbefore=FileStart,
      endafter=FileEnd,
      File
      "/home/jdw/.local/lib/python3.10/site-packages/obspy/clients/fdsn/client.py",
      line 756, in get_stations
      inventory = read_inventory(data_stream, format='STATIONXML')
      File "/home/jdw/.local/lib/python3.10/site-packages/decorator.py", line
      232, in fun
      return caller(func, *(extras + args), **kw)
      File
      "/home/jdw/.local/lib/python3.10/site-packages/obspy/core/util/decorator.py",
      line 297, in _map_example_filename
      return func(*args, **kwargs)
      File
      "/home/jdw/.local/lib/python3.10/site-packages/obspy/core/inventory/inventory.py",
      line 91, in read_inventory
      return _generic_reader(path_or_file_object, _read, format=format,
      File
      "/home/jdw/.local/lib/python3.10/site-packages/obspy/core/util/base.py",
      line 624, in _generic_reader
      generic = callback_func(pathname_or_url, **kwargs)
      File "/home/jdw/.local/lib/python3.10/site-packages/decorator.py", line
      232, in fun
      return caller(func, *(extras + args), **kw)
      File
      "/home/jdw/.local/lib/python3.10/site-packages/obspy/core/util/decorator.py",
      line 142, in uncompress_file
      return func(filename, *args, **kwargs)
      File
      "/home/jdw/.local/lib/python3.10/site-packages/obspy/core/inventory/inventory.py",
      line 100, in _read
      inventory, format = _read_from_plugin('inventory', filename,
      format=format,
      File
      "/home/jdw/.local/lib/python3.10/site-packages/obspy/core/util/base.py",
      line 423, in _read_from_plugin
      list_obj = read_format(filename, **kwargs)
      File
      "/home/jdw/.local/lib/python3.10/site-packages/obspy/io/stationxml/core.py",
      line 152, in _read_stationxml
      root = etree.parse(path_or_file_object).getroot()
      File "src/lxml/etree.pyx", line 3541, in lxml.etree.parse
      File "src/lxml/parser.pxi", line 1896, in lxml.etree._parseDocument
      File "src/lxml/parser.pxi", line 1916, in lxml.etree._parseMemoryDocument
      File "src/lxml/parser.pxi", line 1803, in lxml.etree._parseDoc
      File "src/lxml/parser.pxi", line 1144, in lxml.etree._BaseParser._parseDoc
      File "src/lxml/parser.pxi", line 618, in
      lxml.etree._ParserContext._handleParseResultDoc
      File "src/lxml/parser.pxi", line 728, in lxml.etree._handleParseResult
      File "src/lxml/parser.pxi", line 657, in lxml.etree._raiseParseError
      File "<string>", line 1
      lxml.etree.XMLSyntaxError: Start tag expected, '<' not found, line 1,
      column 1

      -- John


      On Wed, May 29, 2024 at 10:27 AM Philip Crotwell <crotwell<at>seis.sc.edu>
      wrote:

      Hi

      Not sure I have anything useful to add, but I too have seen random,
      unrepeatable errors with the fdsn web services. Often retrying the failed
      query succeeds immediately afterwards, so I do not think this is a data
      problem. If anything, it feels like maybe a threading race condition type
      of error, and the reproducibility problem is exacerbated, I assume, by
      there being more than one instance of the web service behind the load
      balancer. And of course it can be really hard to find and resolve these
      sorts of issues unless you can recover the actual raw response to the query.

      One thing you might try is getting debug output from obspy, like

      DCclient = Client('IRIS', debug=True)

      This might provide additional information if you can catch one of the
      errors with it enabled.

      I tried and caught this. Immediately afterwards, it ran fine, so .

      Downloaded http://service.iris.edu/fdsnws/event/1/application.wadl
      https://urldefense.com/v3/__http://service.iris.edu/fdsnws/event/1/application.wadl__;!!IKRxdwAv5BmarQ!bwcZrxC4BwireDnr1AghrLVfeX-SeBZLuM6h_avmtz0_5wzqxYdZ9le7MY9gP0DDGSbZCh-r0zCpQLMPstlPNZRx$>
      with HTTP code: 200
      Uncompressing gzipped response for
      http://service.iris.edu/fdsnws/dataselect/1/application.wadl
      https://urldefense.com/v3/__http://service.iris.edu/fdsnws/dataselect/1/application.wadl__;!!IKRxdwAv5BmarQ!bwcZrxC4BwireDnr1AghrLVfeX-SeBZLuM6h_avmtz0_5wzqxYdZ9le7MY9gP0DDGSbZCh-r0zCpQLMPshKNb7zr$>
      Downloaded http://service.iris.edu/fdsnws/dataselect/1/application.wadl
      https://urldefense.com/v3/__http://service.iris.edu/fdsnws/dataselect/1/application.wadl__;!!IKRxdwAv5BmarQ!bwcZrxC4BwireDnr1AghrLVfeX-SeBZLuM6h_avmtz0_5wzqxYdZ9le7MY9gP0DDGSbZCh-r0zCpQLMPshKNb7zr$>
      with HTTP code: 200
      Traceback (most recent call last):
      File "/Users/crotwell/Code/GitHub/obspy/JustTesting.py", line 21, in
      <module>
      DCclient = Client('IRIS', debug=True)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^
      File
      "/opt/homebrew/Caskroom/miniforge/base/envs/obspy/lib/python3.12/site-packages/obspy/clients/fdsn/client.py",
      line 277, in __init__
      self._discover_services()
      File
      "/opt/homebrew/Caskroom/miniforge/base/envs/obspy/lib/python3.12/site-packages/obspy/clients/fdsn/client.py",
      line 1584, in _discover_services
      decoded_wadl = wadl.decode('utf-8')
      ^^^^^^^^^^^^^^^^^^^^
      UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1:
      invalid start byte

      Doubt this is very helpful, but at least you know you are not the only one?

      Philip


      On Tue, May 28, 2024 at 10:23 PM John West (via SAGE) <
      webservices-bounce<at>lists.ds.iris.edu> wrote:

      Hi!

      I'm calling the station service using the obspy client.get_stations()
      method, and getting randomly distributed badly formed XML.

      Typical error message is something like: "Start tag expected, '<' not
      found, line 1, column 1 (<string>, line 1)"

      I've also had a few errors with: "Not a gzipped file (b'a\r')"

      The attached sample Python code sometimes runs without error, and
      sometimes generates multiple errors during a run. The errors are NOT in the
      same places, they appear for different networks on each run.

      Any ideas?

      Thanks!
      -- John

      ----------------------
      Web Services
      Topic home: http://ds.iris.edu/message-center/topic/webservices/
      https://urldefense.com/v3/__http://ds.iris.edu/message-center/topic/webservices/__;!!IKRxdwAv5BmarQ!bwcZrxC4BwireDnr1AghrLVfeX-SeBZLuM6h_avmtz0_5wzqxYdZ9le7MY9gP0DDGSbZCh-r0zCpQLMPsqxCRzht$>
      | Unsubscribe: webservices-unsubscribe<at>lists.ds.iris.edu

      Sent from the IRIS Message Center (http://ds.iris.edu/message-center/
      https://urldefense.com/v3/__http://ds.iris.edu/message-center/__;!!IKRxdwAv5BmarQ!bwcZrxC4BwireDnr1AghrLVfeX-SeBZLuM6h_avmtz0_5wzqxYdZ9le7MY9gP0DDGSbZCh-r0zCpQLMPsiwnGZ-I$>
      )
      Update subscription preferences at http://ds.iris.edu/account/profile/
      https://urldefense.com/v3/__http://ds.iris.edu/account/profile/__;!!IKRxdwAv5BmarQ!bwcZrxC4BwireDnr1AghrLVfeX-SeBZLuM6h_avmtz0_5wzqxYdZ9le7MY9gP0DDGSbZCh-r0zCpQLMPsrAx73qb$>



20:47:22 v.b4412d20