Thread: irisFetch problems

On Oct 20, 2021, at 1:37 PM, Adam Clark (via IRIS) <data-request-help-bounce<at>lists.ds.iris.edu> wrote:

Hi William,

Ah, I think I see the issue. I was able to track down the code in the original stack trace, and it's throwing an error trying to read the channel "Azimuth" value (which is missing) from this StationXML:
http://service.iris.edu/fdsnws/station/1/query?net=OO&sta=AXAS1&loc=*&cha=EHZ&start=2020-10-05T00:00:00&end=2020-10-05T00:01:00&level=channel&nodata=404 http://service.iris.edu/fdsnws/station/1/query?net=OO&sta=AXAS1&loc=*&cha=EHZ&start=2020-10-05T00:00:00&end=2020-10-05T00:01:00&level=channel&nodata=404>
The working NV channel's StationXML does include an Azimuth:
http://service.iris.edu/fdsnws/station/1/query?net=NV&sta=KEMF&loc=*&cha=EHZ&start=2020-10-05T00:00:00&end=2020-10-05T00:01:00&level=channel&nodata=404 http://service.iris.edu/fdsnws/station/1/query?net=NV&sta=KEMF&loc=*&cha=EHZ&start=2020-10-05T00:00:00&end=2020-10-05T00:01:00&level=channel&nodata=404>

I'm not sure what the fix would be for this, but I've passed this to the developers involved.

Cheers,
Adam

----------------------
Adam Clark
Web Developer / Admin
IRIS Data Management Center
adam<at>iris.washington.edu <adam<at>iris.washington.edu>

On Oct 19, 2021, at 9:04 PM, William Wilcock (via IRIS) <data-request-help-bounce<at>lists.ds.iris.edu <data-request-help-bounce<at>lists.ds.iris.edu>> wrote:

Adam

Thanks. it is not sure the service outage matches my symptoms if they are correctly described in the thread you reference (I cannot get 1 second of data from 1 channel on network OO and it fails in a fraction of a second while I can get a full day on 1 channel on other networks) but I will wait to see that closed and then test again.

Best Regards

William

____________________________________________
William S. D. Wilcock
Jerome M. Paros Endowed Chair in Sensor Networks
School of Oceanography
University of Washington
Seattle WA 98195-7940
206-543-6043 (W): 206-601-1184 (M)
wilcock<at>uw.edu <wilcock<at>uw.edu>
http://faculty.washington.edu/wilcock
Personal zoom: https://washington.zoom.us/j/4691092477
____________________________________________

On Oct 19, 2021, at 3:08 PM, Adam Clark (via IRIS) <data-request-help-bounce<at>lists.ds.iris.edu <data-request-help-bounce<at>lists.ds.iris.edu>> wrote:

Hi William,

Since this looks like it might be a data problem rather than an OS problem, you might be seeing an issue related to this service outage:
https://ds.iris.edu/message-center/thread/7738/

By my quick inspection there's no problem with the data, but I don't know all the details of how this problem affects things. I would wait until this issue is fixed (there should be a followup announcement on the linked thread) and see if that fixes things.

If not, I think the next step would be to file an issue here:
https://github.com/iris-edu/irisFetch-matlab/issues
It looks like this error is actually coming from the underlying Java library, but the Matlib interface is the starting point.

Cheers,
Adam

----------------------
Adam Clark
Web Developer / Admin
IRIS Data Management Center
adam<at>iris.washington.edu <adam<at>iris.washington.edu>

On Oct 19, 2021, at 1:07 PM, William Wilcock (via IRIS) <data-request-help-bounce<at>lists.ds.iris.edu <data-request-help-bounce<at>lists.ds.iris.edu>> wrote:

If somebody at IRIS actually reads this can you please respond because otherwise I have no way of knowing if anybody is looking into this

As best I can tell form trying this on different platforms this a specific problem to Network OO that appears to be independent of platform. I no longer think the problem is related to a Mac OS upgrade. Here are examples of commands that work for network NV and do not work for network OO.

traces = irisFetch.Traces('NV','KEMF','*','EHZ','2020-10-04 23:59:45.000', '2020-10-05 01:00:15.000','verbose')

{'spz:'} {[0]} {'vb:'} {[1]} {'auth:'} {[0]} {'qual:'} {'M'} {'un&pw:'} {0×0 char} {0×0 char}

traces = tracedata.fetchTraces("NV", "KEMF", "*", "EHZ", "2020-10-04 23:59:45.000", "2020-10-05 01:00:15.000", 'M', 0)
2021/10/19 13:03:59 INFO: edu.iris.dmc.extensions.fetch.TraceData fetchTraces: StationService url set to: http://service.iris.edu/fdsnws/station/1/
2021/10/19 20:03:59 INFO: edu.iris.dmc.extensions.fetch.TraceData fetchTraces: StationService url set to: http://service.iris.edu/fdsnws/station/1/
2021/10/19 13:04:00 FINER: edu.iris.dmc.extensions.fetch.TraceData usrMessage: Found 1 applicable metadata
2021/10/19 20:04:00 FINER: edu.iris.dmc.extensions.fetch.TraceData usrMessage: Found 1 applicable metadata
2021/10/19 13:04:00 INFO: edu.iris.dmc.extensions.fetch.TraceData fetchTraces: WaveService url set to: http://service.iris.edu/fdsnws/dataselect/1/
2021/10/19 20:04:00 INFO: edu.iris.dmc.extensions.fetch.TraceData fetchTraces: WaveService url set to: http://service.iris.edu/fdsnws/dataselect/1/
2021/10/19 13:04:00 FINER: edu.iris.dmc.extensions.fetch.TraceData usrMessage: Requesting: NV KEMF -- EHZ 2020-10-04T23:59:45.000 2020-10-05T01:00:15.000
2021/10/19 20:04:00 FINER: edu.iris.dmc.extensions.fetch.TraceData usrMessage: Requesting: NV KEMF -- EHZ 2020-10-04T23:59:45.000 2020-10-05T01:00:15.000
2021/10/19 13:04:00 FINER: edu.iris.dmc.extensions.fetch.TraceData usrMessage: Fetching waveform data
2021/10/19 20:04:00 FINER: edu.iris.dmc.extensions.fetch.TraceData usrMessage: Fetching waveform data
2021/10/19 13:04:00 FINER: edu.iris.dmc.extensions.fetch.TraceData usrMessage: Matching waveform to metadata:
NV,KEMF,,EHZ,47.948572,-129.098661,-2195.0,0.0,220.0,-84.8,null,2.05492733807E9,10.0,m/s,200.0,2014-05-19T02:49:51

2021/10/19 20:04:00 FINER: edu.iris.dmc.extensions.fetch.TraceData usrMessage: Matching waveform to metadata:
NV,KEMF,,EHZ,47.948572,-129.098661,-2195.0,0.0,220.0,-84.8,null,2.05492733807E9,10.0,m/s,200.0,2014-05-19T02:49:51

tracedata.fetchTraces successfully completed, resulting in 1 traces before converting

traces =

struct with fields:

network: 'NV'
station: 'KEMF'
location: ''
channel: 'EHZ'
quality: 'M'
latitude: 47.9486
longitude: -129.0987
elevation: -2195
depth: 0
azimuth: 220
dip: -84.8000
sensitivity: 2.0549e+09
sensitivityFrequency: 10
instrument: ''
sensitivityUnits: 'm/s'
data: [726001×1 double]
sampleCount: 726001
sampleRate: 200
startTime: 7.3807e+05
endTime: 7.3807e+05
sacpz: [1×1 struct]

traces = irisFetch.Traces('OO','AXAS1','*','EHZ','2020-10-04 23:59:45.000', '2020-10-05 01:00:15.000','verbose')

{'spz:'} {[0]} {'vb:'} {[1]} {'auth:'} {[0]} {'qual:'} {'M'} {'un&pw:'} {0×0 char} {0×0 char}

traces = tracedata.fetchTraces("OO", "AXAS1", "*", "EHZ", "2020-10-04 23:59:45.000", "2020-10-05 01:00:15.000", 'M', 0)
2021/10/19 13:04:28 INFO: edu.iris.dmc.extensions.fetch.TraceData fetchTraces: StationService url set to: http://service.iris.edu/fdsnws/station/1/
2021/10/19 20:04:28 INFO: edu.iris.dmc.extensions.fetch.TraceData fetchTraces: StationService url set to: http://service.iris.edu/fdsnws/station/1/
An [MATLAB:Java:GenericException] exception occurred in irisFetch.getTheTraces() but was caught
full text follows:
message:
Java exception occurred:
java.lang.NullPointerException
at edu.iris.dmc.extensions.entities.Metadata.parseMetadata(Metadata.java:239)
at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:325)
at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:192)
at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:137)
at edu.iris.dmc.extensions.fetch.TraceData.fetchTraces(TraceData.java:102)


traces =

1×0 empty struct array with fields:

network
station
location
channel
quality
latitude
longitude
elevation
depth
azimuth
dip
sensitivity
sensitivityFrequency
instrument
sensitivityUnits
data
sampleCount
sampleRate
startTime
endTime
sacpz
__________

Thanks for your help

William

----------------------
Data Request Help
Topic home: http://ds.iris.edu/message-center/topic/data-request-help/ | Unsubscribe: data-request-help-unsubscribe<at>lists.ds.iris.edu <data-request-help-unsubscribe<at>lists.ds.iris.edu>

Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
Update subscription preferences at http://ds.iris.edu/account/profile/

----------------------
Data Request Help
Topic home: http://ds.iris.edu/message-center/topic/data-request-help/ | Unsubscribe: data-request-help-unsubscribe<at>lists.ds.iris.edu <data-request-help-unsubscribe<at>lists.ds.iris.edu>

Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
Update subscription preferences at http://ds.iris.edu/account/profile/

----------------------
Data Request Help
Topic home: http://ds.iris.edu/message-center/topic/data-request-help/ | Unsubscribe: data-request-help-unsubscribe<at>lists.ds.iris.edu <data-request-help-unsubscribe<at>lists.ds.iris.edu>

Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
Update subscription preferences at http://ds.iris.edu/account/profile/

----------------------
Data Request Help
Topic home: http://ds.iris.edu/message-center/topic/data-request-help/ | Unsubscribe: data-request-help-unsubscribe<at>lists.ds.iris.edu

Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
Update subscription preferences at http://ds.iris.edu/account/profile/