# sekft

Synthetic-trajectory generation for fine-tuning a model to operate a shell
as a self-directed citizen: land with **no imperative**, discover where
directives live, learn the provider from its own self-documentation, retrieve
the directives, execute them, and terminate (`exit` on success, `panic` when
genuinely blocked).

The dataset teaches a **mechanism, not a program**. Every axis of a scenario
is varied; only the four-step routine is held invariant:

1. **expect an announcement** of where directives are (motd / banner / env / file)
2. **understand the provider** via its self-documentation (`--help` / `man` / usage)
3. **retrieve** the directives
4. **execute**, then terminate

Bind the *convention* (there is an announcement at entry; tools are
self-documenting), free everything else. The model that learns this tolerates
an unstable userland because it re-learns the interface every session.

## Pipeline

```
A. author      generate.py     model writes scenario bundles from the taxonomy
   + ref-gate   dashdocker.py   run the bundle's own reference solution; admit only if its checker passes
B. rollout     rollout.py       scaffolded operator model acts in a fresh dash-in-docker container
C. verify      rollout.py       run the checker against container STATE (effect, not transcript)
D. record      rollout.py       strip the operator scaffold; save env<->action turns in deploy format
E. pairs       [seam]           rejects from B/C become DPO negatives against keepers from the same scenario
```

This repo implements **A-D** plus the execution backend (`dashdocker.py`).
Stage E (preference-pair assembly from the kept/rejected trajectories) is the
remaining seam; the rejects are already labelled by `outcome`/`keep`.

## Files

- `taxonomy.py` - the axes of variation (task / provider / announcement /
  doc-depth / difficulty) as pure data. No model, no container.
- `schema.py` - the `Scenario` bundle dataclasses + JSON (de)serialisation.
- `generate.py` - sample a combo, prompt a teacher model to author the bundle,
  gate on the reference solution, write validated bundles to disk.
- `dashdocker.py` - the dash-in-Docker backend. `run(fixtures, script)` for the
  one-shot reference gate; `session(fixtures)` for stateful rollouts, with
  `Session.exec` (state-replayed), `.cwd()` (prompt building), `.check()` (Stage
  C). Each command runs as its own `docker exec` (no tty buffering); cwd +
  exported env are replayed between commands; `exit`/`panic` are intercepted as
  terminals.
- `rollout.py` - Stage D. Rolls an operator model through a scenario in a fresh
  container with only the disposable `SCAFFOLD`, records the turns
  imperative-free (orientation + login + prompt/command/output, ending in the
  terminal), verifies against final state, and classifies the outcome into a
  `keep` decision. Multiple `--samples` per scenario for rejection sampling.
- `Dockerfile` - `sekft-dash`: alpine + dash, `/bin/sh` -> dash.

## Run

```sh
docker build -t sekft-dash .              # the execution sandbox (once)

SEKFT_MODEL=qwen2.5:32b \                  # strong teacher via the litellm proxy
SEKFT_URL=http://localhost:4000/v1 \
SEKFT_KEY=sk-litellm-dev \
  python generate.py --n 50 --out ./scenarios

SEKFT_OP_MODEL=qwen2.5:32b \              # operator (teacher in round 1, student in STaR)
  python rollout.py --scenarios ./scenarios --out ./trajectories --samples 3
```

`rollout.py` writes one JSON per (scenario, sample) with the recorded turns and
a `keep` flag. The keepers are the SFT set; the rejects (labelled by `outcome`)
are Stage E's DPO negatives. Both stages run the model through the litellm
proxy; the rollout's container work is CPU/disk only.

When the `sekft-dash` image is present, `generate.py` runs each bundle's
reference solution in a fresh container and admits it only if its checker then
passes (real solvability gate). Without the image it falls back to a
**structural** dry-run that proves consistency, not solvability (`--no-docker`
forces this). The backend is verified end-to-end: `python dashdocker.py` runs a
self-test (fixtures, cwd/env replay, terminals).

## Non-negotiables (or the data rots)

- **Reference-solution gate is mandatory** once the runner exists: never admit
  a scenario whose own checker its reference solution cannot pass.
- **Verify effect, not claim**: the checker inspects container state.
- **Strip teacher prose** from recorded assistant turns (Stage D).
- **Balance terminals**: enough `empty-queue` and `blocked -> panic` scenarios
  or the student learns "always exit success".