[webservices] default in bulkdataselect

[Philip Crotwell]

HI Chad

That is really useful information, thanks.

I am finding one interesting thing. If I request the same data twice,
the first time takes about twice as long as the second. I assume there
is some caching going on somewhere in your systems.

I have done some experiments here and am finding no difference between
bulk and dataselect. The biggest difference is with request time
window, as to be expected. For 10 minutes of data I get 40 kb/s, for 1
hour 240 kb/s and for 10 hours 960 kb/s. Makes sense as these data
will be contiguous on your system and there is socket and other
overhead.

Might do some more playing, but seems a wash from the outside
perspective for at least single channel single time window requests.

thanks
Philip

On Wed, Aug 29, 2012 at 11:51 AM, Chad Trabant <chad at iris.washington.edu> wrote:
>
> Hi,
>
> Yes, the default values are as you guessed.  By default there is no limitation based on segment length, the documentation has been updated.
>
> Regarding ws-dataselect versus ws-bulkdataselect performance for large miniSEED requests:  early tests indicated there is a performance difference for the user, but I haven't tested for a while and it's dependent on a number of factors.  Also, there is a difference for the DMC internally.  In short, if you just want raw miniSEED data ws-bulkdataselect is the preferred interface.
>
> A bit more explanation:
> For the ws-dataselect requests the data are placed into an internal cache for use, for example, by other services.  The user needs to wait for the data to be extracted and cached.  For ws-bulkdataselect requests the data are not cached, but are effectively streamed back to the user from the storage system directly.  The extraction and caching can be really fast compared to the network connection to the user, so this difference is not always obvious but it would likely add up.  For large requests of raw data ws-bulkdataselect is preferred as it uses fewer resources at the DMC.
>
> Chad
>
> On Aug 29, 2012, at 5:40 AM, Philip Crotwell wrote:
>
>> Hi
>>
>> What are the default values for minimumlength and longestonly? I am
>> guessing 0 and false, but the docs don't say.
>> http://www.iris.edu/ws/bulkdataselect/
>>
>> Also, have you found a performance increase with bulkdataselect over
>> dataselect for large miniseed downloads?
>>
>> thanks
>> Philip
>> _______________________________________________
>> webservices mailing list
>> webservices at iris.washington.edu
>> http://www.iris.washington.edu/mailman/listinfo/webservices
>
>
> _______________________________________________
> webservices mailing list
> webservices at iris.washington.edu
> http://www.iris.washington.edu/mailman/listinfo/webservices