PCL/BenchRead paper

DREAM CLOUD LAB / RESEARCH SOFTWARE

A digital twin
for every
PCL node.

Can a scientific agent learn from one laboratory and run a reliable experiment in another? PCL-Bench turns that question into executable tests.

Read the paper ↗Download code ↓
NETWORK ATLAS
FIG. 01 / 03
SCHEMATIC VIEW
SELECT A NODE
DREAM · NorthwesternReference node for the proposed network
20PCL teams in the NSF test bed
10,000+Proposed scientific tasks
100,000+Proposed agent episodes

01  /  RESEARCH DESIGN

What travels
between labs?

In one laboratory, a binding assay may require a plate reader. In another, characterizing a material may require a potentiostat or a probe station. Both ask an agent to choose an instrument, acquire a trustworthy signal and decide what to do next.

The proposed DREAM-led effort would encode participating labs’ instrument manuals, protocols and reference records into executable environments. The atlas identifies all 20 teams using public NSF awards. The example tasks are proposals for review with each team. Cross-node participation and operating details still require confirmation.

02  /  INTERACTIVE SIMULATOR

Inside an agent trial.

Step through an illustrative plate-reader run. Inject a fault during acquisition and watch it change the recovery path.

TRACE INSPECTORFIG. 02 / 03   ·   SYNTHETIC DEMO
OPTICAL HEADPLATE
ILLUSTRATIVE OPTICAL SIGNAL

This browser illustration uses scripted events. The downloadable Python starter provides the scored evaluation tasks. Inspect a real starter trajectory (JSON) ↗

FIG. 03 / 03  ·  EXECUTABLE CONTROLS

When a read fails.

The starter runs three scripted policies on identical clean and fault-injected cases. Compare what happens when recovery or numerical analysis is removed.

Successful scripted episodes by condition
Scripted policyCleanInjected fault

Loading checked-in results…

These are deterministic synthetic software controls. Physical validation and model evaluation are future work. Download grouped results (JSON) ↗

MEASURED CHEMISTRY  /  HISTORICAL REPLAY

Ten chances to find a high-yield reaction.

Four published Suzuki–Miyaura reaction cases let us replay a search over real measurements. At each step a policy chooses one of the recorded reaction settings and sees its measured yield. These experiments come from Reizman and colleagues, distributed through Summit, independently of the PCL nodes.

Yield left on the tableMean terminal regret · percentage points · lower is better

For each ten-query campaign, regret is the best yield in that case’s measured table minus the best yield found. Repeated measurements at a setting are averaged before replay. Original experiments ↗ · Summit ↗ · Replay results (JSON) ↗

03  /  EVALUATION DESIGN

Six capabilities under test.

01 / INTERFACES

Instrument use

Read the manual, discover tool signatures and select a valid operating range.

02 / SEQUENCING

Orchestration

Order dependent actions and coordinate instruments within a resource budget.

03 / CONSTRAINTS

Safety

Detect reagent, sample and instrument limits before issuing a command.

04 / RESILIENCE

Failure recovery

Identify a recoverable fault, restore instrument state and resume the run.

05 / MEASUREMENT

Quality control

Inspect controls, detect drift and decide whether a measurement is usable.

06 / INFERENCE

Analysis & iteration

Interpret the signal and choose the next measurement with the remaining budget.

04  /  RESEARCH INDEX

Prior work.

Explore the primary sources behind the benchmark design and the specific ideas each brings to cross-laboratory evaluation.

Featured precedent. Autonomous Science Bench evaluates discovery campaigns across biology, chemistry and materials. PCL-Bench targets instrument operation, quality control and recovery within those campaigns. Explore Autonomous Science Bench ↗