ORBIT task plugins: validation outside the execution core

ORBIT puts task-specific parsing, validation, bundle construction and result summaries in TaskPlugin implementations. A shared executor handles staging, placement, launch and collection. This boundary makes a new task easier to review, but it still requires integration tests: a malformed bundle can cross the interface and fail in the shared core.

The Cost of Task-Awareness

When I first wrote the runner, I intuitively built train, eval, and collect paths. Adding a fourth task required editing the shared executor and frequently broke the first three.

This violated the Open-Closed Principle (Meyer, 1988). The executor was open for modification with every new extension. The test surface expanded combinatorially. I could no longer trust a change for one task to leave the others working.

Comparing the tasks showed where they differed: config shape, validation, and output summaries. Staging, launching, monitoring, and artifact collection stayed the same.

I used that distinction to split the code, following Parnas’s rule on modularization (Parnas, 1972): split on what varies, not on processing steps.

The Plugin Boundary

       [ Task Plugins ]
   (knows what a task is)
  +----------+ +----------+ +----------+
  | training | |   eval   | | collect  |
  +----+-----+ +----+-----+ +----+-----+
       |            |            |
       v            v            v
  +------------------------------------+
  | Control Kernel                     |
  | (registry, templates -> request)   |
  +------------------------------------+
                   | (generic bundle)
                   v
  +------------------------------------+
  | Execution Core (task-agnostic)     |
  | (placement, launch, collection)    |
  +------------------------------------+

The boundary is codified in TaskPlugin, defined in orbit/core/control/registry.py:

class TaskPlugin(Protocol):
    task_type: str
    job_kind: JobKind

    def parse_request(self, raw: dict | Any) -> Any: ...
    def validate_request(self, request: Any) -> list[str]: ...
    def build_bundle(self, *, bundle_dir: str, submission: TaskSubmission) -> JobBundle: ...
    def summarize_result(self, *, submission, bundle, status, manifest) -> TaskSummary: ...

There are no SFT or dataset references here. parse_request and validate_request handle task-specific ingestion. build_bundle maps it to a uniform JobBundle. The execution core at orbit/core/execution processes the bundle opaquely.

The TrainingPlugin in orbit/tasks/training/plugin.py enforces specific keys:

class TrainingPlugin:
    task_type = "training"
    job_kind = JobKind.TRAIN

    def validate_request(self, request: TrainingSpec) -> list[str]:
        issues: list[str] = []
        if not request.dataset_path:
            issues.append("dataset_path is required")
        if not request.output_dir:
            issues.append("output_dir is required")
        return issues

Conversely, EvaluationPlugin enforces model and environments. The core engine ignores these vocabularies completely.

I strictly enforce dependency inversion. The control kernel never imports task implementations directly. The wiring happens entirely within build_default_task_registry:

def build_default_task_registry() -> TaskRegistry:
    from orbit.tasks.collection.plugin import CollectionPlugin
    from orbit.tasks.evaluation.plugin import EvaluationPlugin
    from orbit.tasks.training.plugin import TrainingPlugin

    registry = TaskRegistry()
    registry.register(TrainingPlugin())
    registry.register(EvaluationPlugin())
    registry.register(CollectionPlugin())
    return registry

The imports are intentionally buried inside the function. There is no global registry mutated at load time. The core resolves plugins purely by string lookup.

The Abstraction Cost

This separation takes more work to debug. A run may crash in the generic core because a plugin built a malformed bundle, so the stack trace and the cause lie on different sides of the boundary.

I also have to check new fields for task-specific assumptions. If I add a generic bundle field that only makes sense for training, the core has secretly learned about the task. I rigorously prevent the execution engine from growing if statements about bundle contents.

The plugin boundary isolates task changes, while SFT, RLHF, and evaluation sweeps exercise the same executor pipeline.

Start with the control-plane overview for the submission lifecycle, then inspect the bundle contract in the next article.