[webservices] continuous data with dataselect service

Mon Jul 14 10:40:58 PDT 2014

Hi, Chad.

Maybe "block" is the wrong term, sorry if it was confusing.

I assemble a post request containing 100 lines. In general, each line
represents one channel of broadband data for one station for one day, from
00:00:00 to 23:59:59.9999.

My expectation was to get back a miniseed file containing 100 one-day
traces. Instead, contiguous traces are combined so I get some lesser number
of traces, many of which are longer than one day. I am not in principle
opposed to this behavior, but I can envision times when it may cause
problems, e.g., if I have some processing that has limits on the length of
the trace to which it can be applied. It seems to me that optimum behavior
would be a flag set by the user controlling whether you combine traces or
not.

Example: A request containing the following POST data is asking for 100
days of data. It returns 27 traces of varying length.

POST text:
quality=B
TA G06A -- BHE 2008-03-15T00:00:00 2008-03-15T23:59:59.9999
TA G06A -- BHE 2008-03-14T00:00:00 2008-03-14T23:59:59.9999
TA G06A -- BHE 2008-03-13T00:00:00 2008-03-13T23:59:59.9999
TA G06A -- BHE 2008-03-12T00:00:00 2008-03-12T23:59:59.9999
TA G06A -- BHE 2008-03-11T00:00:00 2008-03-11T23:59:59.9999
TA G06A -- BHE 2008-03-10T00:00:00 2008-03-10T23:59:59.9999
TA G06A -- BHE 2008-03-09T00:00:00 2008-03-09T23:59:59.9999
TA G06A -- BHE 2008-03-08T00:00:00 2008-03-08T23:59:59.9999
TA F10A -- BHN 2008-03-13T00:00:00 2008-03-13T23:59:59.9999
TA F10A -- BHN 2008-03-12T00:00:00 2008-03-12T23:59:59.9999
TA F10A -- BHN 2008-03-11T00:00:00 2008-03-11T23:59:59.9999
TA F10A -- BHN 2008-03-10T00:00:00 2008-03-10T23:59:59.9999
TA F10A -- BHN 2008-03-09T00:00:00 2008-03-09T23:59:59.9999
TA F10A -- BHN 2008-03-08T00:00:00 2008-03-08T23:59:59.9999
TA F10A -- BHN 2008-03-07T00:00:00 2008-03-07T23:59:59.9999
TA F10A -- BHN 2008-03-06T00:00:00 2008-03-06T23:59:59.9999
TA F10A -- BHN 2008-03-05T00:00:00 2008-03-05T23:59:59.9999
TA F10A -- BHN 2008-03-04T00:00:00 2008-03-04T23:59:59.9999
TA F10A -- BHN 2008-03-03T00:00:00 2008-03-03T23:59:59.9999
TA F10A -- BHN 2008-03-02T00:00:00 2008-03-02T23:59:59.9999
TA F10A -- BHN 2008-03-01T10:23:45 2008-03-01T23:59:59.9999
TA G06A -- BHZ 2008-03-20T00:00:00 2008-03-20T22:22:22.9999
TA G06A -- BHZ 2008-03-19T00:00:00 2008-03-19T23:59:59.9999
TA G06A -- BHZ 2008-03-18T00:00:00 2008-03-18T23:59:59.9999
TA G06A -- BHZ 2008-03-17T00:00:00 2008-03-17T23:59:59.9999
TA G06A -- BHZ 2008-03-16T00:00:00 2008-03-16T23:59:59.9999
TA G06A -- BHZ 2008-03-15T00:00:00 2008-03-15T23:59:59.9999
TA G06A -- BHZ 2008-03-14T00:00:00 2008-03-14T23:59:59.9999
TA G06A -- BHZ 2008-03-13T00:00:00 2008-03-13T23:59:59.9999
TA G06A -- BHZ 2008-03-12T00:00:00 2008-03-12T23:59:59.9999
TA G06A -- BHZ 2008-03-11T00:00:00 2008-03-11T23:59:59.9999
TA G06A -- BHZ 2008-03-10T00:00:00 2008-03-10T23:59:59.9999
TA G06A -- BHZ 2008-03-09T00:00:00 2008-03-09T23:59:59.9999
TA G06A -- BHZ 2008-03-08T00:00:00 2008-03-08T23:59:59.9999
TA G06A -- BHZ 2008-03-07T00:00:00 2008-03-07T23:59:59.9999
TA G06A -- BHZ 2008-03-06T00:00:00 2008-03-06T23:59:59.9999
TA G06A -- BHZ 2008-03-05T00:00:00 2008-03-05T23:59:59.9999
TA G06A -- BHZ 2008-03-04T00:00:00 2008-03-04T23:59:59.9999
TA G06A -- BHZ 2008-03-03T00:00:00 2008-03-03T23:59:59.9999
TA G06A -- BHZ 2008-03-02T00:00:00 2008-03-02T23:59:59.9999
TA G06A -- BHZ 2008-03-01T10:23:45 2008-03-01T23:59:59.9999
TA F18A -- BHN 2008-03-20T00:00:00 2008-03-20T22:22:22.9999
TA F18A -- BHN 2008-03-19T00:00:00 2008-03-19T23:59:59.9999
TA F18A -- BHN 2008-03-18T00:00:00 2008-03-18T23:59:59.9999
TA F18A -- BHN 2008-03-17T00:00:00 2008-03-17T23:59:59.9999
TA F18A -- BHN 2008-03-16T00:00:00 2008-03-16T23:59:59.9999
TA F18A -- BHN 2008-03-15T00:00:00 2008-03-15T23:59:59.9999
TA F18A -- BHN 2008-03-14T00:00:00 2008-03-14T23:59:59.9999
TA F18A -- BHN 2008-03-13T00:00:00 2008-03-13T23:59:59.9999
TA F18A -- BHN 2008-03-12T00:00:00 2008-03-12T23:59:59.9999
TA F18A -- BHN 2008-03-11T00:00:00 2008-03-11T23:59:59.9999
TA F18A -- BHN 2008-03-10T00:00:00 2008-03-10T23:59:59.9999
TA F18A -- BHN 2008-03-09T00:00:00 2008-03-09T23:59:59.9999
TA F18A -- BHN 2008-03-08T00:00:00 2008-03-08T23:59:59.9999
TA F18A -- BHN 2008-03-07T00:00:00 2008-03-07T23:59:59.9999
TA F18A -- BHN 2008-03-06T00:00:00 2008-03-06T23:59:59.9999
TA F18A -- BHN 2008-03-05T00:00:00 2008-03-05T23:59:59.9999
TA F18A -- BHN 2008-03-04T00:00:00 2008-03-04T23:59:59.9999
TA F18A -- BHN 2008-03-03T00:00:00 2008-03-03T23:59:59.9999
TA F18A -- BHN 2008-03-02T00:00:00 2008-03-02T23:59:59.9999
TA F18A -- BHN 2008-03-01T10:23:45 2008-03-01T23:59:59.9999
TA G06A -- BHE 2008-03-20T00:00:00 2008-03-20T22:22:22.9999
TA G06A -- BHE 2008-03-19T00:00:00 2008-03-19T23:59:59.9999
TA G06A -- BHE 2008-03-18T00:00:00 2008-03-18T23:59:59.9999
TA G06A -- BHE 2008-03-17T00:00:00 2008-03-17T23:59:59.9999
TA G06A -- BHE 2008-03-16T00:00:00 2008-03-16T23:59:59.9999
TA G06A -- BHE 2008-03-07T00:00:00 2008-03-07T23:59:59.9999
TA G06A -- BHE 2008-03-06T00:00:00 2008-03-06T23:59:59.9999
TA G06A -- BHE 2008-03-05T00:00:00 2008-03-05T23:59:59.9999
TA G06A -- BHE 2008-03-04T00:00:00 2008-03-04T23:59:59.9999
TA G06A -- BHE 2008-03-03T00:00:00 2008-03-03T23:59:59.9999
TA G06A -- BHE 2008-03-02T00:00:00 2008-03-02T23:59:59.9999
TA G06A -- BHE 2008-03-01T10:23:45 2008-03-01T23:59:59.9999
TA G16A -- BHZ 2008-03-20T00:00:00 2008-03-20T22:22:22.9999
TA G16A -- BHZ 2008-03-19T00:00:00 2008-03-19T23:59:59.9999
TA G16A -- BHZ 2008-03-18T00:00:00 2008-03-18T23:59:59.9999
TA G16A -- BHZ 2008-03-17T00:00:00 2008-03-17T23:59:59.9999
TA G16A -- BHZ 2008-03-16T00:00:00 2008-03-16T23:59:59.9999
TA G16A -- BHZ 2008-03-15T00:00:00 2008-03-15T23:59:59.9999
TA G16A -- BHZ 2008-03-14T00:00:00 2008-03-14T23:59:59.9999
TA G16A -- BHZ 2008-03-13T00:00:00 2008-03-13T23:59:59.9999
TA G16A -- BHZ 2008-03-12T00:00:00 2008-03-12T23:59:59.9999
TA G16A -- BHZ 2008-03-11T00:00:00 2008-03-11T23:59:59.9999
TA G16A -- BHZ 2008-03-10T00:00:00 2008-03-10T23:59:59.9999
TA G16A -- BHZ 2008-03-09T00:00:00 2008-03-09T23:59:59.9999
TA G16A -- BHZ 2008-03-08T00:00:00 2008-03-08T23:59:59.9999
TA G16A -- BHZ 2008-03-07T00:00:00 2008-03-07T23:59:59.9999
TA G16A -- BHZ 2008-03-06T00:00:00 2008-03-06T23:59:59.9999
TA G16A -- BHZ 2008-03-05T00:00:00 2008-03-05T23:59:59.9999
TA G16A -- BHZ 2008-03-04T00:00:00 2008-03-04T23:59:59.9999
TA G16A -- BHZ 2008-03-03T00:00:00 2008-03-03T23:59:59.9999
TA G16A -- BHZ 2008-03-02T00:00:00 2008-03-02T23:59:59.9999
TA G16A -- BHZ 2008-03-01T10:23:45 2008-03-01T23:59:59.9999
TA F10A -- BHN 2008-03-20T00:00:00 2008-03-20T22:22:22.9999
TA F10A -- BHN 2008-03-19T00:00:00 2008-03-19T23:59:59.9999
TA F10A -- BHN 2008-03-18T00:00:00 2008-03-18T23:59:59.9999
TA F10A -- BHN 2008-03-17T00:00:00 2008-03-17T23:59:59.9999
TA F10A -- BHN 2008-03-16T00:00:00 2008-03-16T23:59:59.9999
TA F10A -- BHN 2008-03-15T00:00:00 2008-03-15T23:59:59.9999
TA F10A -- BHN 2008-03-14T00:00:00 2008-03-14T23:59:59.9999

Thanks!

     -- John

On Thu, Jul 10, 2014 at 7:08 PM, Chad Trabant <chad at iris.washington.edu>
wrote:

>
> Hi John,
>
> Can you provide a bit more detail about what you mean by getting the data
> back in blocks?  In a nutshell, if the data is continuous in the archive,
> what you get should be continuous as long as you have requested continuous
> segments.
>
> In the archive, the internal blocking of the miniSEED data (referred to as
> records) are sometimes truncated at UTC day boundaries and other times
> there are records that cross the UTC day boundaries.  For the TA data in
> particular, the R quality data that arrived in real time generally does not
> include records that cross the day boundaries (it has been cut to perfectly
> fit within days).  But the Q quality data (a preferred copy) that arrives
> many months after real time does have records that cross day boundaries.
>  On top of this, by default the dataselect service will trim the data
> requested to the time window specified by the user.
>
> I think if you can clarify what you are mean by blocking and perhaps
> provide some example requests we might be able to give you a better answer.
>
> Chad
>
> On Jun 30, 2014, at 4:34 PM, John D. West <john.d.west at asu.edu> wrote:
>
> > Hi.
> >
> > I'm requesting continuous broadband data from (mostly) TA stations using
> the POST usage of the dataselect web service.
> >
> > When I request multiple successive one-day blocks in the same POST
> request, I sometimes get the data back in 1-day blocks and sometimes it is
> combined into longer multi-day blocks.
> >
> > What controls this behavior? Is it possible to force the data to be
> returned in single-day traces without resorting to sending multiple
> individual requests, one per day?
> >
> > Thanks!
> >      -- John
> > _______________________________________________
> > webservices mailing list
> > webservices at iris.washington.edu
> > http://www.iris.washington.edu/mailman/listinfo/webservices
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.iris.washington.edu/pipermail/webservices/attachments/20140714/72145da2/attachment.html>