NTT Systems
To MECMS Home Page

Creating and Running
MECM
Experiments

For more information, contact Marc Grushcow, President, NTT SYSTEMS INC.


This page describes how MECM is used in DCIEM's SUSOPS environment and how it can be adapted to the needs of other research laboratories.

Every lab engaged in cognitive psychology or similar research has its own methods for creating and running experiments. Ultimately, each one has to deal with experimental design, protocol creation, the logistics of running the experiment, handling failures, data management, and results analysis and interpretation.

DCIEM is like many labs in that research interests and client needs are increasing while available resources are decreasing. MECM, and its supporting task management system, were designed with this in mind. Its features combine to...

Experiment Design

Scientists work to define the experimental conditions, determine which tasks will be included in the protocol, and establish the number and size of subject groups. Typically an experiment consists of a number of repeated blocks and the Experiment Schedule Editor can be used to easily lay out and reconfigure the prototypical block.

The timeline on the left displays nominal time from experiment start. It adjusts automatically in response to adding and removing tasks or changing task Duration.

Research organizations tend to have their own locally developed task batteries. NTT can customize the Experiment Schedule Editor to reflect the tasks that you use.


Protocol Creation

Once the overall design has been established, the full protocol can be created. At DCIEM, this job is done completely by Research Assistants without any assistance from programming staff. Using the Editor's cut and paste features, the prototypical block is easily replicated to "fill out" the protocol. The Editor has an individual parameter sheet for each task and the Research Assistants use these to specify task parameter values that implement the experimental conditions specified by the Scientist. This, combined with cut and paste functions make counterbalancing a snap. The time line always shows the nominal start time of each task and it can be used to verify that the block duration has not been accidentally disturbed during the editing process. The resulting protocol and time line can optionally be saved as a tab delimited file and imported into a variety of applications (e.g.., Word or Excel) for review and printing.

Running The Experiment

At DCIEM experiments are started, monitored and controlled largely by Research Assistants. First they start the Console in the experiment control room. It simply opens and waits for connections from subject Workstations. When the Workstation program is started, it presents a fill-in-the-blank form. This is used to enter subject identification information (name, room location) and to identify various directories and files (including the location of the protocol file). This information is saved since much of it is the same from subject to subject between the multiple runs that comprise an experiment. When the Workstation starts, saved information is presented as defaults, thus eliminating the possibility of errors that may arise when entering long file and path names. Much of the information is predicated on DCIEM's MECM/SUSOPS environment. The form can be easily reconfigured for your local task battery and environment.

Once the Workstations start, they connect to the Console and the experimenter would see a display similar to the one here...

The two windows on the left represent Workstations for subjects peter and mike. Peter is currently held, that is, the Workstation is not going to start a new task until directed by the experimenter. Mike is currently working on a task called STMthat started at 13:23:25 and is expected to finish at 13:23:45 (in 19 seconds from "now"). The next few tasks for both are displayed in the top grids. The green VCR arrow on Mike's display shows that tasks will continue to be run automatically at his Workstation. The gray "Suspend" button offers the Experimenter the option of manually suspending the delivery of further tasks.

The large grid on the right shows Mike's full protocol file. The blue D indicator shows tasks that have been completed and the green R indicator identifies the currently running task.

The protocol file can contain scheduled HOLD commands. A HOLD automatically suspends task delivery thus creating a break. These are typically used for meal times, sample collection, electrode checks, etc. The Experimenter controls when the Workstation resumes task delivery.

As we mentioned before, the Experimenter can manually force the temporary suspension of task delivery to any or all subjects. This might be required if a subject requires attention, e.g.., medication, instruction, etc. In some cases, we might want to remove some tasks from a subject's protocol to keep them in sync with others. MECM allows the Experimenter to cancel selected upcoming tasks. As the experiment progresses it may develop that the subject is not so late as first thought. In this case, upcoming tasks that were flagged as canceled can be returned to scheduled status.

It is often necessary to document events that occur during an experiment. Examples are: giving scheduled or unscheduled medication to subjects, noting behaviour, and recording software or hardware anomalies. MECM provides an online recording facility that helps keep track of Experimenter notes in a standardized manner. New notes are entered through the following on-screen form...

The note editor allows for annotating existing notes, but will not allow direct modification of note content. The notebook can be previewed in time order. In fact, if the preview is left on the screen, it will be automatically updated as new notes are added.

The notebook is maintained as an HTML document and can be saved as such.


Handling Failures

DCIEM's SUSOPS experiments tend to run from two to four days in length. Our investment in an experiment increases with time, and maintaining state information, especially trial numbering is critical. MECM can tolerate and recover from a variety of hardware and software failures, both in Workstations and at the Console. Workstations can be restarted after failure or replacement. In this case, MECM state information is retrieved from the Console. If the Console fails, Workstations will continue their task delivery activities. When the Console is replaced or restarted, it can reconnect to running Workstations and retrieve required state information. This is done without disturbing any task that the subject may be interacting with. MECM can survive the failure of the Console or any or all of the Workstations.

Weasel Words: MECM does not retain state information for tasks that comprise the test battery. Information maintained by running tasks, or shared between tasks, or retained between task invocations must be managed and protected by those tasks. Replacement of a Workstation during an experiment will typically require transfer or reinstallation of programs and data files. Use of a file server with backup will alleviate this problem. MECM is not being advertised as a "highly reliable system" in the sense normally understood in the software engineering field. Nonetheless, we have never lost an experiment at DCIEM.


Data Management

The SUSOPS task battery is integrated into MECM in the sense that the Workstation understands the structure of SUSOPS task command lines. These consist of two groups of parameters. The first are common to all tasks in the battery and identify experiment and subject IDs, trial number, and result file name. The second group contain parameters specific to each task. These are specified when the protocol file is created. The first group are supplied by the MECM Workstation at run time based on information provided on the startup form. This strategy allows a single protocol file to be used for all subjects. Key here is the fact that the result file directory is specified on the startup form. As a result, result data can be stored in any directory and Experimenters can decide whether to pool or segregate subject results -- MECM does not impose a data storage strategy.

The Dispatcher portion of the Workstation can be easily customized to accommodate the command line structure of your task battery. Any data specified on the Workstation startup form can be integrated with command line parameters.


Results Analysis and Interpretation

MECM in no way modifies the structure or content of result files produced by the task battery during experiments. Consequently any analytical procedures in use before introducing MECM will continue to be useable.

MECM does produce additional information that can be used to interpret and document the experiment itself. First of course, is the Experimenter's log described above. NOTE: Each Workstation produces a log of all tasks that it initiates and all error output (stderr) produced by the task battery. Each line is time stamped and includes subject and experiment identification information. These files can be combined and date sorted. The result is an automatically produced log that documents which tasks were performed by which subjects during the complete experiment.


For more information, contact Marc Grushcow, President, NTT SYSTEMS INC.

Please visit our home page: NTT Systems Inc.