← All projects

agentrun AI · Backend

Pick up a job where it stopped

Saving each completed step makes a long job easier to restart and inspect.

Agent jobs with spend caps and replay.

Python / Postgres

GitHub ↗
Notes & checks

A job runner with spending limits, a replay viewer, and a mocked HN extractor. Recorded test suite: 96 tests.

The problem

A long job can fail after doing useful work. Restarting everything wastes time and may repeat an external action. agentrun stores jobs and step results in Postgres. Its included draft/review/polish example and HN hiring extractor both use local mocks.

How it works

  1. DraftSave output
  2. ReviewSave output
  3. PolishSave output
The built-in mock job. Restarting can reuse completed steps.
  1. Claim a job for a short lease

    A worker claims a queued job. A lease is a temporary right to work on it. If the worker disappears and that lease expires, another worker can claim the job. A claim token and row locks guard against an old worker saving over a newer claim.

  2. Save the result of each step

    Steps pass JSON-compatible output to the next step. Before running one, the runner looks for a saved result using the job, step name, and input hash, a fingerprint of the input. Saved work can be reused after a restart.

  3. Inspect what happened

    Attempts record errors, timing, and reported usage. The replay command turns the stored job history into an HTML timeline. You can also inspect status or cancel a job from the command line.

Decisions & tradeoffs

Saving a result is only half the retry story

If a provider acts but its response is lost, the runner may call it again. Avoiding duplicate external charges depends on that provider honoring the step's idempotency key. The local mock does so.

A threshold is checked between attempts

Reported spend includes failures and timeouts, but one attempt can exceed the threshold. A timed-out function can keep running in the background. Workflow definitions also need to stay stable while saved jobs depend on them.

Try it yourself

The README walks through submitting a mock job and opening its replay. It requires Python 3.12+, uv, and a disposable Postgres database. Read that setup before running the commands.

Try the draft/review/polish job, then compare the saved steps in its timeline. Its usage and dollar amounts are pretend. The HN demo parses supplied comments in a fixed header format; it is not a live scraper or a measured model extractor.

Sources & scope

Based on the local README and runner source. Built-in workflows use mocks; provider-level retry safety is conditional.