Thread: Big files for PDCC

Started: 2017-03-31 03:57:52
Last activity: 2017-03-31 22:18:32
Topics: PDCC
Antonio Abrego
Big files for PDCC
2017-03-31 03:57:52
Sear Sir:

I'm using PDCC to look for response files from public stations from the GNS. Among them, there is a set of stations with the network code IU; when I tried to open the IU.dataless file, which is accessible in the web, it seems that the files is too big and the software crashes. How can I overcome this problem?

Regards,

Antonio

Antonio A. Abrego M.
Ingeniero Civil
Unidad de Ingeniería Geotécnica
División de Ingeniería
Tel. 276-1989
E-Mail: aabrego<at>pancanal.com<ljethmal<at>pancanal.com>


  • Robert Casey
    Re: Big files for PDCC
    2017-03-30 21:39:44

    Hi Antonio-

    The IU dataless is indeed a bit of a challenge for PDCC to manage. Though 133 MB doesn't sound like a big file, it's very dense with information and PDCC creates objects for all of them, so it takes up a lot of memory. I was successful in loading the IU.dataless file into PDCC by running it with a larger max heap setting of 4 GB.

    java -Xmx4g -jar ./run_pdcc.jar

    I am running Java 8, and this is the recommended version to run. It does take a while to load and show you the dataless tree, about 5 minutes. Once I did this, I was able to select a station and do a Save As... RESP file, saving the BBSR station RESP to disk.

    I watched the memory usage and it creeped up to about 1.7 GB of memory at the highest point, so you would want to set the heap to be at least 2g, all depending on how much memory you have on your machine.

    You can also try setting Compressed Oops if you are running a 64-bit JVM in an attempt to reduce memory usage, but in this case, I didn't see much of a difference:

    java -XX:+UseCompressedOops -Xmx4g -jar ./run_pdcc.jar

    -Rob


    On Mar 30, 2017, at 1:59 PM, Antonio Abrego <aabrego<at>pancanal.com> wrote:

    Sear Sir:

    I’m using PDCC to look for response files from public stations from the GNS. Among them, there is a set of stations with the network code IU; when I tried to open the IU.dataless file, which is accessible in the web, it seems that the files is too big and the software crashes. How can I overcome this problem?

    Regards,

    Antonio

    Antonio A. Abrego M.
    Ingeniero Civil
    Unidad de Ingeniería Geotécnica
    División de Ingeniería
    Tel. 276-1989
    E-Mail: aabrego<at>pancanal.com <ljethmal<at>pancanal.com>


    ----------------------
    PDCC (http://ds.iris.edu/message-center/topic/pdcc/)

    Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
    Update subscription preferences at http://ds.iris.edu/account/profile/

    • Antonio Abrego
      RE: Big files for PDCC
      2017-03-31 22:18:32
      Hello Rob:

      Thank you very much for your help, however, I couldn’t open the file. The good news is that I did obtain the dataless files I wanted using the Breq_Fast request services that IRIS provides.

      Regards,

      Antonio

      Antonio A. Abrego M.
      Ingeniero Civil
      Unidad de Ingeniería Geotécnica
      División de Ingeniería
      Tel. 276-1989
      E-Mail: aabrego<at>pancanal.com<ljethmal<at>pancanal.com>

      From: pdcc-bounce<at>iris.washington.edu [pdcc-bounce<at>iris.washington.edu] On Behalf Of Rob Casey
      Sent: Thursday, March 30, 2017 4:39 PM
      To: PDCC
      Subject: Re: [pdcc] Big files for PDCC


      Hi Antonio-

      The IU dataless is indeed a bit of a challenge for PDCC to manage. Though 133 MB doesn't sound like a big file, it's very dense with information and PDCC creates objects for all of them, so it takes up a lot of memory. I was successful in loading the IU.dataless file into PDCC by running it with a larger max heap setting of 4 GB.

      java -Xmx4g -jar ./run_pdcc.jar

      I am running Java 8, and this is the recommended version to run. It does take a while to load and show you the dataless tree, about 5 minutes. Once I did this, I was able to select a station and do a Save As... RESP file, saving the BBSR station RESP to disk.

      I watched the memory usage and it creeped up to about 1.7 GB of memory at the highest point, so you would want to set the heap to be at least 2g, all depending on how much memory you have on your machine.

      You can also try setting Compressed Oops if you are running a 64-bit JVM in an attempt to reduce memory usage, but in this case, I didn't see much of a difference:

      java -XX:+UseCompressedOops -Xmx4g -jar ./run_pdcc.jar

      -Rob


      On Mar 30, 2017, at 1:59 PM, Antonio Abrego <aabrego<at>pancanal.com<aabrego<at>pancanal.com>> wrote:

      Sear Sir:

      I’m using PDCC to look for response files from public stations from the GNS. Among them, there is a set of stations with the network code IU; when I tried to open the IU.dataless file, which is accessible in the web, it seems that the files is too big and the software crashes. How can I overcome this problem?

      Regards,

      Antonio

      Antonio A. Abrego M.
      Ingeniero Civil
      Unidad de Ingeniería Geotécnica
      División de Ingeniería
      Tel. 276-1989
      E-Mail: aabrego<at>pancanal.com<ljethmal<at>pancanal.com>


      ----------------------
      PDCC (http://ds.iris.edu/message-center/topic/pdcc/)

      Sent from the IRIS Message Center (http://ds.iris.edu/message-center/)
      Update subscription preferences at http://ds.iris.edu/account/profile/


SAGE Facility has been certified as a trustworthy data repository by the CoreTrustSeal Standards and Certification Board
15:13:01 v.22510d55