Manuals: PDCC_3-0

DESCRIPTION

The Portable Data Collection Centers (PDCC) software has reached its third iteration of release. The mission of the PDCC project has been to provide network operators a portable software facility for managing seismic network data and metadata. It is also intended to provide a suite of tools for producing SEED volumes for data distribution as well as provide support for other data formats.

As a matter of history, Versions 1.0 and 2.0 of PDCC were somewhat platform-dependent, owing to the fact that some of the code distributed was written in C for the SunOS/Solaris operating system. The latest version of PDCC, currently under development, represents an effort to rethink the software design from the ground up. Written entirely in Java, and taking advantage of the flexibility of object-orientation, the new PDCC presents a wide array of features, many of them not available in previous releases.

OPEN SOURCE LICENSE AND DISCLAIMER

Copyright© 2006 Incorporated Research Institutions for Seismology

This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version.

This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

For more information, please contact the author, found at the end of this manual.

FEATURES

PDCC currently allows users to do the following:

  • Read one or more SEED volumes and edit their contents.
  • Read RESP text files and make use of the response information.
  • Save SEED information to a MySQL database for later retrieval.
  • Cut, copy, and paste Blockettes, as well as create new ones.
  • Write new SEED files.
  • Export data to SAC and RESP formats

There are many features that are being developed for future releases. This document will be updated as those features are incorporated.

REQUIREMENTS

The current requirements are:

  • Java JVM v1.4.0 or greater (Install Anywhere™ can provide the JVM for many platforms)
  • MySQL database v3.23.52 or greater (optional)

INSTALLATION

Install Anywhere™ is used to install PDCC on the user’s machine. This is a commercial package that makes it easy to install the PDCC software package on your hard disk directly from the web. It also provides downloadable Java Virtual Machines (JVM’s) when requested, so that the Java software can be run on most common computer platforms.

To download PDCC go to the IRIS DMC Software page.

Find the listing for PDCC and click on the Download link where you will be taken to the install page. A Java applet is run, which might take a moment to start up. You should then see a button indicating the recommended download for your machine. You can select that button, or one of the other download links listed below that. The program installer will be initiated at this point. Follow the instructions in the installer to put PDCC on your machine.

RUNNING PDCC

Install Anywhere™ will provide you with an icon to start PDCC. Just activate the icon to start the application.

There are a couple of alternatives to starting PDCC. Both involve using a terminal window. The first is to run the provided jar file, which is included with the installation. It’s an executable jar file that contains the latest build of PDCC at the time of distribution. Using this technique involves the least typing:

java -jar run_pdcc.jar

Your second option is for when you have modified the PDCC source code and/or need to rebuild the Java classes. In this case, you’ll want to run PDCC using the edu/ source directory in your classpath. This requires that you indicate the full class name of the starting PDCC class, as well as the complete class path. From the PDCC installation directory, you can use relative pathnames:

java -classpath jar/mysql_jdbc.jar:jar/JavaSeed.jar:jar/isis_framework.jar:
jar/isti_shape.jar:jar/oscache-2.1-mod2.jar:jar/commons-logging.jar edu.iris.dmc.pdcc.gui.PdccMain

If you run PDCC from a directory other than the installation directory, you’ll want to use absolute pathnames and include the install directory in the classpath. Your classpath would look like this if you installed PDCC in /home/pdcc:

/home/pdcc/:/home/pdcc/jar/mysql_jdbc.jar:/home/pdcc/jar/JavaSeed.jar

A third approach to starting PDCC uses a build description file. Included in the distribution is a build.xml file used by Ant, which is a utility that produces a compiled build, similar to UNIX make. You will need to modify the properties at the top off the build.xml file before using this, but if you have Ant installed you can compile, build a new run_pdcc.jar file, and run the jar file with the command:

ant run

Using any of these techniques, you will be presented with a PDCC interactive panel. If errors are encountered while running PDCC, you will see a popup window displaying an error message.

The first time you install PDCC, you will be asked to create a new configuration file. A file selector will appear for you to enter the filename of your configuration. You can manage multiple configurations as you want, and swap between them when running PDCC. More about PDCC configuration is provided in the next section.

With a configuration file available, PDCC will proceed to the main tool window. PDCC will remember the last configuration file you used at the next startup.

CONFIGURATION

PDCC features a configuration tool to make it easier for users to modify and maintain their configuration settings for PDCC. Features will continue to be added to this tool as time allows.

The first time that you run PDCC, you will be prompted to create a new configuration file. It will first prompt you for a filename, and typically you want to write this file in PDCC’s config/ directory. PDCC will then start up with default settings and present the configuration tool for you to edit. The following is a list of the parameters that you will can modify as necessary:

Parameter Description
ARGUMENTS: These are items that relate to the startup procedures of PDCC
pdccConfigPath Indicate the full pathname of this configuration file (example: /install_path/config/my_config.txt).
PDCC_SESSION: These are parameters that supply run-session information
pdccCacheFileDir Set this to the full pathname of a directory that has a lot of disk space. (default: /install_path/cache). Temporary cache files will be written here, which can get to hundreds of megabytes in size.
schemaFile Set to the full pathname of the schema configuration file (default: /install_path/schema/PDCCschema.txt).
dbLoadChildren (located down further) Set to true or false depending on whether you want to load channels and responses when importing from the database.
PDCC_DATABASE: These are parameters for a particular database connection.
dbHostName This is the hostname of the machine that the database is on, and is also what the MySQL authentication will recognize as a permitted hostname reference for connection.
dbmsName This is the name of the make of database that PDCC calls. Default is mysql.
driverClass PDCC connects to the database via JDBC using this driver class name.
dbName This is to be the name of the database used for storage of SEED information to MySQL. PDCC currently only supports a single MySQL database.
userName This is the user name used to access the PDCC database in MySQL. This name needs to be added to the user table in MySQL before PDCC can access the database (this is covered in DATABASE ACCESS section).
passWord This is the password for the user name that you have indicated.
dataFileDir (located down further) Indicates where miniSEED data files are stored when archiving SEED data to disk.

When you have finished editing changes, hit the Accept button in the tool. PDCC will adjust to your new settings without needing to be restarted. If at any time before hitting Accept, you want to revert back to the original file settings in the tool, hit the Reset button. Hitting Cancel prevents any changes from being made and returns you to PDCC.

These parameters are stored in a configuration file that can also be edited by hand as long as caution is used. While there are other parameters in this file, leave them at their default settings. Some are used by PDCC, and some are not currently used, serving as placeholders for future-supported features.

From the configuration tool, you can also load other configuration files that you have created. Hit the Load button and select the configuration file from the file dialog. You can create many different configuration files with variations in settings by changing the pdccConfigPath parameter to a new filename. The result is that the tool creates this new file and preserves the old one. You can then modify the settings for the new file and hit Accept when you are done.

READING A FILE

PDCC derives its source information from SEED files and RESP files. IRIS DMC provides three different kinds of SEED files:

  • full SEED: The standard SEED-formatted volume with station control headers, instrument responses, time span headers, and volume headers, followed by a series of waveform data records.
  • dataless SEED: A SEED-formatted volume with station control headers, instrument responses, and volume headers only. This is strictly station metadata, not including waveform data records (hence the term dataless).
  • miniSEED: A series of SEED-formatted waveform data records. Minimal station information and encoding flags are provided, but all descriptive headers are removed. This is sometimes called data-only SEED. This is commonly used for real-time data telemetry and waveform archiving.

To read a SEED file, go to the menu bar at the top of the PDCC application and select File, followed by Import, followed by SEED file (File >Import >SEED file). You will be presented with a file selection dialog box allowing you to select the file to read. Once you have selected the file to open, PDCC will begin parsing the file for its Blockette contents. If the file is not a SEED file, or is not properly formatted, an error message will be generated and the file read may be halted.

During import, PDCC will begin indexing Blockette objects from your selected file and display the volume in the navigation tree on the left.

The user can also import RESP-format text files into PDCC (File > Import > RESP file). The result will be similar to reading in a SEED volume, though RESP files have sparse station and channel information. Their main purpose is to provide instrument response information.

Whenever you want to remove a file volume from your workspace, select the volume node and activate File > Close Volume from the menu.

BLOCKETTES

The basic building block of the SEED file is the Blockette. It’s a data structure that contains a formatted set of data fields. The format and meaning of those fields depends on its type. It’s highly recommended that users of PDCC have some working knowledge of Blockettes before using this tool. The details on Blockettes and their use in the SEED format are documented in the FDSN SEED Manual available at the FDSN website .

PDCC breaks the SEED data down into a set of nested Blockette objects, arranged in a parent/child relationship. This relationship is illustrated in the tree navigation tool on the left-hand side of the PDCC panel. The topmost level is the Volume, which represents the file that you have imported. Under the Volume are the Dictionary, Station, and Data categories.

  • Dictionary: you will find one or more Dictionary blockettes, which are lookups for the detailed meaning of abbreviations and codes that may be present in the Station blockettes.
  • Station: you will find one or more Station blockettes, each of which contain one or more Channel blockettes. Each Channel contains response information in the form of a list of Response blockettes. Users not familiar with these entities and how they relate to seismic stations are encouraged to refer to the FDSN SEED Manual for details.

EDITING

PDCC allows users to edit Blockettes imported from SEED data. This is performed in the edit window found on the right hand side of the PDCC application. When you select an item in the navigation tree, you will see data fields displayed about that object in the edit window. Blockettes can be edited in this viewer window by selecting a text box and changing the value. Hit RETURN or TAB to have PDCC validate your entry. Clicking into a different text box will have the same effect.

In validating the user entry, PDCC checks that acceptable characters are entered into the field. If this is not the case, the user is prompted to change the value, indicating which characters are acceptable. If the entered value is too long, the validation process will truncate the entry to the proper length.

All changes to fields in the edit window are immediately recorded by the PDCC application. If the user has not left the current edit window, there is an option to restore the original entries before editing was performed. The Restore button on the bottom right corner triggers all of the fields in the currently viewed window to reset to their original values. The user must understand that this is not an Undo button that allows multiple reversals of individual edits. Instead, Restore resets all of the fields in the Blockette view. A caveat to this is that once the user selects another Blockette to edit, all of the field changes to the previous Blockettte are fixed and cannot be restored to a previous set of values.

DICTIONARY BLOCKETTES

There are some fields in the edit window that cannot be typed in directly, but are instead refer to other Blockettes, specifically Dictionary blockettes. Such fields will appear as buttons, containing descriptive text of the value that is referenced. The equivalent Dictionary blockette can be found under the Dictionary section of the tree navigation tool. To view this Dictionary from the Blockette you are editing, click the button and a second popup window will appear, showing you the Dictionary blockette contents. You can edit the Dictionary from this window or reference a different Dictionary blockette by selecting the combo-box at the top. To have a Dictionary field reference nothing (null), select the [No Abbreviation] option at the top of the list in the selector tool.

You can also generate new dictionary Blockettes with the selector tool. In the list of dictionary references, you will see the option [Make New Abbreviation] at the bottom of the list. By selecting this, a new dictionary of the type referenced by this Blockette field will be created, and the editor window will move to that new dictionary for you to edit. Be warned that this action will fix changes in the referencing Blockette you just left.

GROUPED FIELDS

There is also another kind of field that cannot be edited directly and shows up as a button. This is a Grouped Field. It consists of one or more rows of data columns that must be presented in a separate window. In the SEED manual, they are indicated by the REPEAT keyword. An example of this is a list of response coefficients. When you see a button that says something like Number of Poles: 3, clicking on that button will present a popup window showing the Grouped Field contents, consisting of rows and columns that you can edit directly.

In the Grouped Field popup, you can add and delete rows with the numbered buttons on the left-hand side. To add (insert) a row, click on the button for the row adjacent to where you want the new row to appear. A duplicate of the row will appear in that position. To delete a row, hold down the SHIFT key and click on the button. To append a row to the end, click on the last row’s button. If there are no rows to begin with, a button will be presented to add the first row to the group.

Because of the row and column nature of the data in a Grouped Field, two buttons are provided on the bottom left of the edit window to allow you to import plain-text row and column values from a file. The number of columns in the file need to be the same as the number of columns in the edit window. The columns should be separated by white-space. You can either Append data from the file, which adds the values to the end of the last row currently showing in the edit window, or Insert the data, which replaces any existing entries with the new entries found in the file.

CUT COPY PASTE DELETE

Four editing functions are provided to help the user in adding and deleting Blockettes from the volume. These are Cut, Copy, Paste, and Delete. You can find these functions in the Edit menu. Currently, these functions only apply to the navigation tree and not to the blockette edit panels.

After selecting on or more related items in the navigation tree, selecting Cut will remove those items (and the children) from the tree. However, until a Copy or another Cut is performed, the item (along with its children) will have a copy retained in the clipboard, which is an invisible holding space for copied objects waiting to be Pasted.

Copy adds the selected item to the clipboard, but unlike Cut, it does not remove the selected item from the navigation tree.

To place the contents of the clipboard after a selected item in the navigation tree, select Paste from the Edit menu. The objects in the clipboard will be drawn into the navigation tree just following the object you have selected, or in some cases appended as a child to the selected object.

For example if a Channel blockette were pasted onto another Channel blockette, the new entry would appear just after the selected Channel. If it were pasted onto a Station blockette, the new entry would appear at the end of the list of Channels.

If a Response Blockette is Pasted to another Channel, the response list for that channel is automatically sorted to reflect proper instrument stage ordering. This sorting is not performed on data when it is first imported from a file, due to performance concerns.

You can perform the Paste function multiple times in a row, if desired. The clipboard contents are only changed when another Cut or Copy operation is performed. If you have selected an improper location, PDCC will put up a warning for you.

Finally, the Delete operation is just like a Cut, except that no copy is made of the deleted item. It is permanently removed from the navigation tree after asking the user for confirmation. For large datasets, Delete can be faster than Cut because of the lack of copying involved.

You can select multiple items to perform these operations on, but PDCC filters out selections that don’t relate to the rest of the set. So if you select many Channels but manage to include a Station in the selection, PDCC will prompt a warning and deselect the Station. An important point to remember is that an item copied also takes its child nodes with it, so it is not necessary to select a Station and all of its Channels and response stages in order to have all of these items copied. You can simply select the Station and everything under it will be copied as well.

You cannot select multiple Paste locations, even as you are allowed to select multiple nodes in the tree. The Paste function will paste only to the first item in a list of selections.

CREATING NEW BLOCKETTES

Users can create new Stations, Channels, and other Blockettes using the File >New menu. Select the appropriate place in the navigation tree for what you are adding. For Stations, select the Station category node or the Station node you want the new one to appear after. For Channels, select the Station node or the Channel node you want the new one to appear after. For response stages, select the Channel of interest. By selecting the desired item in the menu, a new Blockette node is created with default entries. At this point, you can edit the contents and Commit the changes. For response stages a new stage number will be automatically generated.

New Dictionary objects can also be created from the menu. Select the Dictionary category node or the last Dictionary node in the tree before using the New menu.

WRITING A FILE

An important end product of the PDCC software is a formatted data file constructed from the user’s edited objects. This is termed as Exporting. There are a number of different formats supported by PDCC, and more will be available in future updates.

To write a dataless SEED file, first click on a Volume node, or an node within that Volume. Then select File > Export > Dataless SEED file from the menu. You will then be provided with a file selector tool to specify the name of the file and which directory you want it written to.

Note that you are selecting the entire Volume to go to the file. If this is not what you desire, then you should create a new Export Volume (described below), and copy the components that you want to go to the output file. Then Export that new volume to a Dataless SEED file.

The operations for other file types is quite similar, but RESP files and SAC files have different behaviors. For RESP files, you are allowed to select a directory where the RESP files will be written. Each of the RESP files are arranged by station/channel names. SAC files don’t have the ability to select an output directory at present, so the output goes to the directory where you installed PDCC.

NEW VOLUMES

Volumes are created by reading data files. You see them in the navigation tree, along with Dictionary and Station categories and their contents. You can also create new Volumes that have no initial data in them. This provides an empty space to insert just the data objects you want to represent in an Export product, such as a dataless SEED file. It can also serve as a separate safe area to perform edits to data objects that you wish to keep separate from the original source Volume.

To create a new Volume, go to the File menu and select New, and then Export Volume (File > New > Export Volume). You will be prompted for the output volume name and then a new Volume with Dictionary, Station, and Data categories will be created for you in the navigation tree.

WORKSPACES, FILES, AND CACHE

One question that a user of PDCC might ask is:

When I open a data file and start editing the contents, am I changing what’s in that file?

The answer to this is no. What you are viewing and editing in PDCC is a memory representation of the contents of that file, which is also cached to your hard drive. The simple term for this collection of objects is the Workspace. It’s a copy of the data you have extracted from a source, such as a file, and placed in memory to view, edit, and save in different forms. It is your work area, which can be used many times over to write to other output formats and maintain a catalog.

If you make changes to the data in a Volume they reside as a separate entity in memory, which is also preserved on disk so that you can exit PDCC and come back to your Workspace when you start it up again later. When PDCC is first started, it checks for any cached Workspace entries and reloads the Volumes into the navigation tree. This usually takes a minute to complete, though it can take longer if your Volumes contain a lot of data.

To remove a Volume from your Workspace and the disk cache, select File > Close Volume from the menu. This removal is permanent and cannot be recovered later unless your disk cache is backed up somewhere.

The disk cache is a directory system of encoded files that contains all of your Workspace information. This is stored in the area indicated by your pdccCacheFileDir parameter in your PDCC configuration. If you remove one or more of the Volume directories, those cached entries are lost. You can back these up or locate them elsewhere and make use of them later if you wish.

DATABASE ACCESS

PDCC manages persistent storage of data using a MySQL database. This is a widely used and freely available database that has all the functionality a PDCC user will need. In addition, any changes made by PDCC can be accessed through other MySQL database interfaces and programs, provided they follow the schema and indexing requirements.

The first step for making use of MySQL for PDCC is to have an instance of MySQL installed. This document lists the minimum recommended version of MySQL to use to ensure compatibility with PDCC. Earlier versions may work, but testing has not been performed on these earlier versions to confirm compatibility. Your installed release of PDCC may provide a copy of MySQL that is compatible with your platform. There will be a text document that comes with the MySQL distribution that details how to proceed with the installation.

Once the install is complete, you will need to ensure that the mysqld daemon is running. This is necessary for you to make use of MySQL. The details for doing this are provided with the documentation that comes with your platform’s version of MySQL.

With the daemon running, proceed to the directory where you installed MySQL. On the command line, use mysqladmin to set the root password, provided this is a new installation of the MySQL database. If the database was already installed previously, then you will need to get the root password from the MySQL administrator. To establish the root password on new installations:

bin/mysqladmin -u root password 'my-root-pass'

Please note that these instructions may have to be varied depending on the platform you are running.

We now want to create as PDCC username for MySQL. To begin our administrative session with MySQL, run the client interface as the root user.

bin/mysql -u root -p

(enter password at the prompt )

At the mysql> prompt, change to the mysql database. This is where MySQL accounts and access permissions are established.

mysql> use mysql

Create a new user called pdcc and set access privileges for this user. You can use a different username, if you wish, but you will need to enter it into your PDCC configuration. To create the new user, type:

mysql> grant all privileges on *.* to pdcc@localhost identified by 'my-pass-word';

You can optionally change the pdcc@localhost to fit your selected username and hostname for MySQL. Also, replace my-pass-word with your password of choice in the above command line. Hit RETURN to execute the creation of your new MySQL user.

For the new user permissions to take effect, you have to type the following:

mysql> flush privileges;

Let’s examine the user table, which is where user accounts are created and permissions flags are set:

mysql> select * from user;

You should see your new username in the table and almost all of the permissions turned on. For those concerned about security, please refer to the MySQL manuals to fine tune the user permissions. If everything looks fine, exit from the MySQL client and reconnect to MySQL as user pdcc (or whatever your username is set to).

mysql> exit
bin/mysql -u pdcc -p

(enter password at the prompt )

Create a database for PDCC called pdcc_3. This can also be a different name, but must be set in your PDCC configuration file.

mysql> create database pdcc_3;

Exit from the MySQL client.

mysql> exit

At this point, you should double check your PDCC configuration to see that the database settings are correctly specified. From here, you can attempt your first connection to MySQL by setting up your PDCC database with the necessary tables.

Run PDCC and then select Tools > Initialize DB to construct the database tables. Note that this will delete the contents of any existing tables in the pdcc_3 database, so please use this function with caution! While initialization takes place, a popup window will appear, showing a log of the database calls that build the PDCC schema.

Provided this process goes smoothly, you can now write your first volume to the database. It’s a process that is as simple as writing a SEED file. Select a volume in the navigation tree and in the menu select File > Export > to Database. You should see a popup window displaying SQL statements that are being passed to the database. If you get an error, then there may be something wrong with your database configuration or there is something amiss with the Volume you are attempting to write.

Reading data back from the database is quite simple. Select File > Import > from Database. Like reading a file, a Volume is created representing the contents of the PDCC database. Currently, there are no query facilities to filter and pare down what is accessed, but this feature will be available soon.

There can only be one copy of the database volume present in the navigation tree. You can perform functions in the database Volume just as you would with a file Volume. It’s a workspace copy of the database contents, so the edits you make are not immediately transferred to the database. You have to Export your changes back out.

When changes are made to the database Volume, the Volume needs to be Exported for them to be saved to the database. There are two ways to do this. The first is to select File > Export > to Database after selecting the database Volume in the navigation tree. The update queries will then be performed to save your changes. The second method, selecting File > Refresh Database, performs an update followed by a reloading of the database view to reflect the current contents of the database.

PDCC allows you to save data records as well as metadata to MySQL. However, the management of the data records is haphazard at best and future versions of PDCC will address the issues with managing a data archive using MySQL. Consider the data support for MySQL to be demonstration-level only.

The data records themselves are stored in miniSEED format in the directory indicated in your PDCC Configuration. The MySQL database catalogs the location of data records for later access but does not store details about the data records, unlike the station metadata.

NOTES

PDCC makes use of disk caching to store the bulk of the SEED data read in. This helps to stretch the object memory capacity of PDCC, which would otherwise be minimal with pure on-board memory storage. However, the capacity of PDCC to handle large datasets is still finite, and may not work well with some network operators’ data if it is on a large scale. Improvements to the scalability of PDCC are a constant concern while it is in development. If the disk cache takes up a lot of disk space on your machine, you can delete the contents of the directory when you are not running PDCC.

PDCC does not currently implement a button toolbar in the GUI, but this will be implemented in the future.

PDCC does not currently support Blockette 60 and the Blockette 40's. Some aspects in PDCC may work, but the entire read/write process has not been fleshed out yet.

An Ant build.xml file has been provided with the distribution. If you modify the XML file’s property for the installation directory, you can use this in conjunction with your Ant installation to rebuild and run PDCC. Please consult the build.xml file for the target names and dependencies.

When exporting to a SEED file or the database, only referenced dictionary Blockettes are included in the output. Dictionary elements that are not referenced by another blockette, and thus unlinked, are not included in the output. There will be a fix for this in the future to give the user the option to output all dictionaries.

The SAC export routine needs further work. Support for selecting output directories and support of SAC ASCII format will be provided in a future release.

Release date:     Modified date:

14:06:17 v.9bc884e1