EARTHWORM SETUP EXAMPLES

Example 1

Briefly, this system would function as follows: The system is started by executing the program startstop. Its job is to create the message ring(s) and start the modules. It does this by reading its configuration file (startstop.d), which lists the ring(s) to be created, the modules to be started, and the configuration files each module will read. As each module starts, it reads its configuration file, which specifies the identity of the instance of that module (in case several copies of a module are desired), the names of the message rings the module is to use for input and output, and any other module-specific parameters.

adsend is a digitizer module which controls an A/D converter, and outputs trace data messages. pick_ew is the Rex Allen picking algorithm that listens to trace data messages, and produces pick- and coda-messages. binder_ew listens to pick messages and produces trial location messages. eqproc consists of a number of sub-modules which pre-process binder_ews trial locations, and run hypo-inverse. The output of eqproc is a message containing the hypo-inverse arc output. menlo_report listens to arc messages, and passes them to remote computer(s) for further processing.

Example 2

Note that multiple message rings are used. There are two main reasons for this. One is conceptual: the configuration is easier to understand and maintain if different rings are used to carry related types of messages (and the diagram is easier to draw). The other is performance related: if the rate of message production into a single ring is too high, the computer may be unable to run all receiving modules quickly enough, with the result that some modules may miss messages of interest (note that this is a detectable error, and constitutes a system crash). Given contemporary processor speeds, this is generally no longer a problem. Missed messages do indicate that the hardware is being severely over-driven.

Internal names: Earthworm uses names to identify four types of system components. Each set is defined by a table of names and associated numeric values. In operation, the user creates the desired entries in these tables, and then uses the names in the configuration files. At run time, utilities are used to resolve the names to their respective numeric values. The four categories of names are:

• Installation Ids:These are the names given to the various institutions that operate Earthworm systems. It is the only set of names that is centrally administered. That is, Installation names and the associated numeric values must be requested from Golden. The reason for this is that as Earthworms exchange data, only the Installation Id uniquely identifies the source of a message. This table is part of a compile-time C header file, .../include/earthworm.h. All other name sets are run-time; the defining tables are kept in .../run/params/earthworm.d. This file is read during system startup, and thus can be altered without recompiling any source code. The syntax of the table is as per standard Earthworm parameter files (contributed by Carl). A # marks the start of a comment field.
• Message Rings: These are all the ring names known to the local Earthworm system. The associated numeric values must be valid shared memory key values. This table is kept in earthworm.d, and read by various utility routines at startup time. Each individual installation is free to define its own set if it so chooses. Earthworm releases, however, include the default set.
• Module names: These are all module names known to the local Earthworm system. The conventions are as with Message Ring names.
• Message types: As with Module names above.

The modules copystatus and statmgr are part of the earthworm error-processing scheme. The scheme offers two services: detection of lost heartbeats and individualized processing of various errors. Such processing includes use of pagers, email, permissible rates, limiting the number of notifications, and restarting the offending module. The scheme is optional in two ways: First, a given earthworm configuration may choose to not run statmgr. In this case, no system-wide error processing will be performed. Second, if statmgr is run, each module can decide whether to utilize the scheme. To participate, a module must do three things:

• First, submit to statmgr a file (.desc) containing processing instructions for the various errors that it can generate. This is done be creating such a file in the run/params/ directory, and making a corresponding entry in statmgrs parameter file (statmgr.d).
• Second a participating module must broadcast periodic heartbeat messages, at or above the promised rate, as stated in its .desc file. These will be received by statmgr. If statmgr does not receive a heartbeat within the advertised time period, it will initiate the actions specified in the .desc file. Such actions can include issuing notification, and terminating and restarting the offending module. This restart feature has proven to be very useful for communication modules that are subject to unusual failure modes.
• Third, a module may issue status (error) messages in the standard format. These are received by statmgr, and the actions specified in the .desc file are taken. Since the architecture allows one input and one output per module, helper modules called copystatus are used to move status and heartbeat messages from various rings to the ring to which statmgr is listening.

This configuration has three sources of trace data, all of which broadcast into a message ring named Wave_Ring:

• Four ref2_ew modules, each of which controls a communications line, receives trace data from a remote RefTek data logger, and broadcasts standard earthworm format trace data.
• A sun_demux module, which receives trace data from UofW's own digitizer, demultiplexes it, and produces standard trace data messages,
• And an import-generic module: this is the long-distance message receiver. It communicates with its companion module export_scn via a TCP socket connection. Any type of earthworm messages can be shipped in this way; in this case, several channels of trace data are being received from Menlo Park. The shipping delay is on the order of several seconds, plus transmission time.

The trace data messages must be in the standard internal Earthworm format, as described in .../src/include/trace_buf.h. Briefly, this format consists of a header followed by a variable number of data samples, represented as either 16 or 32 bit signed integers in either byte order. This format supports two channel identification schemes: The dominant scheme consists of three ASCII strings defining the 'station', 'component', and 'network' (SCN names). The second is the 'pin number'. This is an integer that can be used to identify the sensor and signal path of a channel. It addresses the case where a given sensor is acquired by two different telemetry paths. It also provides support for processing data from sensors that do not have SCN names.

Four modules listen to the messages on the Wave_Ring:

• WaveServerV: This module maintains rotating histories of selected channels, and offers a service capable of supplying requested segments of historic trace data.
• export_scn: This is the long-distance shipping module (companion to import_generic), specialized to send specific data channels (Station, Component, Network names). In this application, several selected channels of data are being sent to another installation.
• copystatus, which conducts status messages and heartbeats to the Hypo_ring. statmgr is listening to that ring, and will be able to receive and process them.
• pick_ew, as above, listens to selected trace channels, and produces pick and coda messages (along with status and heartbeat messages). These are broadcast into Pick_Ring.

• binder_ew is the event associator (conceived and written by Carl Johnson). This module listens to pick messages, and produces trial location messages as well as link messages that relate picks to trial locations (via sequence numbers).
• eqproc is the first of a series of sub-modules that perform event locations via hypo-inverse. The sub-modules are linked via operating system pipes, and as a group behave as an earthworm module. Briefly, their functions are as follows: eqproc assembles the trial locations produced by binder_ew with the associated pick and coda messages produced by pick_ew, and if the event has not been recalled by binder_ew within a time-out period, sends the assembled event parameters to eqbuf. This acts as a buffer, in case events are produced faster than the remaining sub-modules can process them. From there it passes to eqcoda, which computes coda durations from the coda measurements generated by pick_ew. It is then passed to eqverify, which applies a series of sanity checks to eliminate spurious events. From there it is passed to hypo_mgr, which is the encapsulation layer for hypo-inverse. Hypo-inverse has not been modified for Earthworm. The standard FORTRAN source code is used, permitting any future changes to hypo-inverse to be directly inserted. The result of hypo_mgr is a message containing the standard hypo-inverse arc file information. This message is broadcast into the Hypo_Ring.
• As before, copystatus is used to conduct heartbeat and status messages from the Pick_Ring to the Hypo_Ring, where statmgr can hear them.

• statmgr: As mentioned above, this module processes error messages, and takes specified actions if a module's heartbeat ceases.
• export_generic: this is the general long distance message shipper. In this case used to ship hypo-centrer messages to another institution.
• uw_report: used to send hypocentral and pick information to the UofW processing and archving system.

Return to the Earthworm Table of Contents Page.