Thread: HTTP Compression for DMC webservices

Hello WS users,

The DMC has made a minor configuration change to our web service servers.
We now support HTTP compression, when requested, for the following mime types:

text/html
text/xml
text/plain
application/xml

HTTP compression is used in transferring data if
(1) the client software supports compression and includes 'gzip' in the Accept-Encoding header.
(2) the data being sent if of the correct mime-type.

A Wikipedia article describing HTTP compression can be found at:
http://en.wikipedia.org/wiki/HTTP_compression

This standard HTTP compression is generally transparent to the user, the server compresses and the client decompresses before the user ever sees the data.

For large data sets, compression can significantly reduce the amount of data being transmitted and therefore has the potential for faster data retrieval. Compression requires more processing time on both the sending (server) and receiving (client) sides of the connection. Users with slower connections to the DMC will generally see greater data transfer speeds, but due to the many variables users are encouraged to test .

The main beneficiaries will be users of ws-station and ws-event which transmit large amounts of XML and text data. The ASCII output data from ws-timeseries can also be compressed.
However, ws-timeseries has had HTTP compression enabled since it's initial release.

Common command line clients like curl and wget do not request compressed data by default. The curl and wget programs usually support compression and only need to have it enabled with a command line option, for curl use the '--compressed' option, for wget use the '--header\="Accept-Encoding: gzip,"' option.

Future versions of the DMC's Fetch scripts will enable compression support when the Perl modules are available.

The DMC's IRIS-WS Java library does not support compression, but this capability will be added in the future as an option.

regards,
IRIS DMC web services team