Thread: Re: null returns from fdsnws-station service

Started: 2014-09-12 10:38:14
Last activity: 2014-09-13 17:12:39
Topics: Web Services
Matthew.Hoyles@ga.gov.au
2014-09-12 10:38:14
Hello

Thank you for the quick response with useful information. We are querying from a program, I was just using a browser to test and see what was going wrong. I will modify the code to check for status code 204 and treat it as a special case.

I will explain what was happening in a bit more detail, in case anyone is interested. The program (written in Python) was doing three steps: first query the station service for the station and channel metadata as XML, second parse the metadata and extract a list of channels, third request waveform data from the dataselect service (for the intersection of a list of channels of interest and the list of available channels from the metadata). The first step was apparently succeeding (204 is a success code, so the url library was not throwing an exception, as it does for 404, for example). Then an empty string was passed to the second step, causing the XML parser to throw an exception, which was reported as an error in the log files and a failed data request, rather than a no data result. In this scenario the third step, which would have interpreted an empty channel list as 'no data', was never reached.

Thanks again for the help.

Matthew


Geoscience Australia Disclaimer: This e-mail (and files transmitted with it) is intended only for the person or entity to which it is addressed. If you are not the intended recipient, then you have received this e-mail by mistake and any use, dissemination, forwarding, printing or copying of this e-mail and its file attachments is prohibited. The security of emails transmitted cannot be guaranteed; by forwarding or replying to this email, you acknowledge and accept these risks.
-------------------------------------------------------------------------------------------------------------------------



  • Rob Newman
    2014-09-12 19:16:16
    Hi Matthew,

    Depending on what Python request library you are using (I highly recommend requests*, much more modular than urllib2), you can get the status response code easily and test for it in a try/except.
    Using requests you can do this way:
    r = requests.get('https://api.github.com/user', auth=('user', 'pass'))
    r.status_code
    200
    Good luck,
    - Rob Newman, IRIS DMC

    * http://docs.python-requests.org/en/latest/


    On Sep 11, 2014, at 8:38 PM, Matthew.Hoyles<at>ga.gov.au wrote:

    Hello

    Thank you for the quick response with useful information. We are querying from a program, I was just using a browser to test and see what was going wrong. I will modify the code to check for status code 204 and treat it as a special case.

    I will explain what was happening in a bit more detail, in case anyone is interested. The program (written in Python) was doing three steps: first query the station service for the station and channel metadata as XML, second parse the metadata and extract a list of channels, third request waveform data from the dataselect service (for the intersection of a list of channels of interest and the list of available channels from the metadata). The first step was apparently succeeding (204 is a success code, so the url library was not throwing an exception, as it does for 404, for example). Then an empty string was passed to the second step, causing the XML parser to throw an exception, which was reported as an error in the log files and a failed data request, rather than a no data result. In this scenario the third step, which would have interpreted an empty channel list as 'no data', was never reached.

    Thanks again for the help.

    Matthew





  • Chad Trabant
    2014-09-13 17:12:39

    Hi Matthew,

    I think Philip gave you the most important information regarding the return code. I will add that when using the 'matchtimeseries' parameter and getting confusing results you might also want to use the 'includeavailability' parameter to see the data availability directly. For example, this XML includes a DataAvailability element:

    http://service.iris.edu/fdsnws/station/1/query?loc=00&end=2014-09-10T09:47:11&sta=BJT&level=chan&net=IC&start=2014-09-10T09:27:11&includeavailability=true

    Note, for example, that the metadata for the BH1 channel in starts a little more than 2 days before time series is available.

    In your case you were matching data availability near current time. In this case the station service treats any data channels that have arrived within 36 hours of the current time as available through the current time. This is (now) documented in the 'matchtimeseries' section here:

    http://service.iris.edu/fdsnws/station/docs/1/help/

    regards,
    Chad

    On Sep 11, 2014, at 8:38 PM, Matthew.Hoyles<at>ga.gov.au wrote:

    Hello

    Thank you for the quick response with useful information. We are querying from a program, I was just using a browser to test and see what was going wrong. I will modify the code to check for status code 204 and treat it as a special case.

    I will explain what was happening in a bit more detail, in case anyone is interested. The program (written in Python) was doing three steps: first query the station service for the station and channel metadata as XML, second parse the metadata and extract a list of channels, third request waveform data from the dataselect service (for the intersection of a list of channels of interest and the list of available channels from the metadata). The first step was apparently succeeding (204 is a success code, so the url library was not throwing an exception, as it does for 404, for example). Then an empty string was passed to the second step, causing the XML parser to throw an exception, which was reported as an error in the log files and a failed data request, rather than a no data result. In this scenario the third step, which would have interpreted an empty channel list as 'no data', was never reached.

    Thanks again for the help.

    Matthew


    Geoscience Australia Disclaimer: This e-mail (and files transmitted with it) is intended only for the person or entity to which it is addressed. If you are not the intended recipient, then you have received this e-mail by mistake and any use, dissemination, forwarding, printing or copying of this e-mail and its file attachments is prohibited. The security of emails transmitted cannot be guaranteed; by forwarding or replying to this email, you acknowledge and accept these risks.
    -------------------------------------------------------------------------------------------------------------------------


    _______________________________________________
    webservices mailing list
    webservices<at>iris.washington.edu
    http://www.iris.washington.edu/mailman/listinfo/webservices



15:33:05 v.01697673