Process multiple job offers in parallel via headless workers. Each worker runs the full evaluation pipeline (A-F report + PDF + tracker line) autonomously. See the Headless / Batch Mode table in AGENTS.md for the correct command per CLI.
-
Add offers to
batch-input.tsv(tab-separated:id,url,source,notes):id url source notes 1 https://jobs.example.com/role-a LinkedIn 2 https://greenhouse.io/company/role-b Greenhouse priority
-
Dry run to preview what will be processed:
./batch/batch-runner.sh --dry-run
-
Run the batch:
./batch/batch-runner.sh
-
Results are automatically merged into
data/applications.md, processed offers are reconciled out of thedata/pipeline.mdinbox, and integrity is verified withverify-pipeline.mjsat the end of the run.
| Flag | Default | Description |
|---|---|---|
--parallel N |
1 |
Number of concurrent headless workers |
--dry-run |
off | Preview pending offers without processing |
--retry-failed |
off | Only retry offers marked as failed in state |
--resume-paused |
off | Resume offers paused after a Claude session/rate limit |
--start-from N |
0 |
Skip offers with ID below N |
--limit N |
0 |
Max number of offers to process in this run (0 = no limit) |
--max-retries N |
2 |
Max retry attempts per offer before giving up |
--rate-limit-sleep N |
300 |
Maximum adaptive retry delay in seconds (0–2147483647); use 0 to pause the batch immediately |
Transient rate limits start at min(30, N) seconds, double for each retry
already consumed by the offer (including persisted retries on resume), and
saturate at N. A random integer percentage from 0 through 20 is added,
rounded down to whole seconds, then capped at N. With the default ceiling,
the first waits are 30–36s and 60–72s. Small ceilings can leave no jitter room.
The current failed attempt's log may override exponential growth with a
standalone Retry-After: <non-negative integer seconds> line (case insensitive,
surrounding whitespace and CRLF allowed). Its delay is max(min(30, N), seconds)
plus the same capped upward jitter, so it never retries before the requested
time. If several valid lines exist, the largest wins. Values exceeding N
pause the batch without consuming a retry, rather than retrying too early.
HTTP dates, fractions, signed values, JSON/embedded prose and malformed lines
are ignored; no worker log text is evaluated as code. Each attempt overwrites
its log, so a prior attempt's header cannot affect the next wait.
There is no new base-delay flag. Waits within the ceiling remain in the worker,
including in serial mode. --max-retries still bounds retries; once exhausted,
the existing failure path applies. --rate-limit-sleep 0 and session-limit
pauses retain their existing behavior. Actual waits appear in the console and
the rate_limited state note; the TSV schema is unchanged.
batch/
batch-runner.sh # Orchestrator script
batch-prompt.md # Prompt template sent to each worker
batch-input.tsv # Input offers (you create this)
batch-state.tsv # Processing state (auto-managed, resumable)
logs/ # Per-offer worker logs ({report_num}-{id}.log)
tracker-additions/ # TSV lines produced by workers
merged/ # TSVs already merged into applications.md
- batch-runner.sh reads
batch-input.tsvandbatch-state.tsvto determine which offers need processing. - For each pending offer, it assigns a report number and launches a headless worker with
batch-prompt.mdas the system prompt (placeholders like{{URL}},{{REPORT_NUM}}are resolved). - Each worker evaluates the offer, writes a report to
reports/, generates a PDF tooutput/, and writes a tracker TSV totracker-additions/. - After all workers finish, batch-runner calls
merge-tracker.mjsto merge TSVs intodata/applications.md,reconcile-pipeline.mjsto move processed offers out of thedata/pipeline.mdinbox, andverify-pipeline.mjsto check integrity.
Workers write one TSV per offer to batch/tracker-additions/. The merge script (npm run merge) handles:
- Deduplication by company + role fuzzy match and report number
- Column order conversion (TSV has status before score; applications.md has score before status)
- In-place updates when a re-evaluation scores higher than the existing entry
- Moving processed TSVs to
tracker-additions/merged/
Run npm run merge manually if you need to merge outside of a batch run.
Batch mode reads offers from batch-input.tsv, but the data/pipeline.md inbox is a separate list. Without reconciliation, an offer evaluated by a batch run stays in the pipeline "Pendientes" section and gets surfaced again on the next scan or /career-ops pipeline run -- producing duplicate reports.
reconcile-pipeline.mjs (run as npm run reconcile) closes that gap: after the tracker merge, every completed or skipped offer in batch-state.tsv whose URL is still in pipeline "Pendientes" is moved to "Procesadas" with its report link and score (entries without a report file on disk are left in place). It is idempotent -- safe to run after every batch, or manually.
batch-state.tsv tracks the status of every offer (pending, processing, completed, failed, skipped, rate_limited, paused_rate_limit). If the batch is interrupted, re-running batch-runner.sh picks up where it left off -- completed offers are skipped automatically. rate_limited is a non-completed state used while the runner waits before retrying, so interrupted rate-limited jobs are eligible on the next normal run.
paused_rate_limit is different: a worker hit a Claude session/usage limit, zero-wait pause, or a Retry-After above the configured ceiling, so the runner stopped scheduling new offers and preserved the retry count. Resume those rows explicitly after the limit resets:
./batch/batch-runner.sh --resume-pausedA PID-based lock file (batch-runner.pid) prevents concurrent batch runs. If a previous run crashed, the stale lock is detected and removed automatically.
- Your CLI in PATH (see Headless / Batch Mode table in
AGENTS.md) - Node.js >= 18, Playwright chromium installed (
npm run doctorto verify) batch-input.tsvwith at least one offer