ESC
Type to search...
S
Soli Docs

Background Jobs & Cron

In-process queues and scheduled jobs. Define a handler in app/jobs/, call .perform_later, and Soli's job engine stores, claims, runs, and retries it — on any database adapter.

How it works

Soli owns the whole lifecycle — storage, claiming, execution, retries, and cron:

  • Enqueue writes a row to the _jobs collection on your default connection (works on SolidB, PostgreSQL, MySQL, and SQLite). On a SQL connection the queue indexes itself on first enqueue (state, run_at, priority), so the claim query does not scan the table every poll tick.
  • Claim is atomic — Postgres FOR UPDATE SKIP LOCKED, MySQL a token claim, SQLite the database write lock (BEGIN IMMEDIATE), SolidB an If-Match compare-and-swap — so several app processes can share one queue without double-running a job.
  • Execute happens on a worker pool thread with its own loaded interpreter, never on a web worker, so a slow job can't delay requests.
  • Retry is exponential backoff with a lease; a job whose process dies is reclaimed once its lease expires.

Nothing calls back into your app over HTTP: there is no callback URL, no inbound route, and no shared secret to configure. Webhook.* jobs are delivered by the engine itself, with the same retry policy.

Defining a Job

Create a file under app/jobs/. Filename and class name follow the same convention as controllers and models — welcome_email_job.sl defines class WelcomeEmailJob.

class WelcomeEmailJob
  static def perform(args: Hash)
    user = User.find(args["user_id"])
    Mailer.send(user.email, "Welcome to the app")
  end
end

Every job class must define static def perform(args: Hash). That's the entry point the engine calls when the job runs.

Per-class Facade

Job classes get a set of static helpers automatically — no inheritance needed.

Method Behavior
XJob.perform_later(args, queue_or_opts?) Enqueues for a worker to run. Returns the job id.
XJob.perform_in(duration, args, queue_or_opts?) Enqueues with a relative delay ("5 minutes", "2 hours", seconds as a number).
XJob.perform_at(datetime, args, queue_or_opts?) Enqueues to run at an ISO-8601 timestamp.
XJob.perform_now(args) Runs perform inline, right now — no queue row, no worker, no database. Returns whatever the handler returns, which makes it the right tool in a test.
XJob.schedule_cron(name, expr, args?) Idempotently registers a cron entry that triggers this class.

The trailing queue_or_opts argument is either a queue-name string or an options hash { queue, priority, max_retries } — priority is an Int and higher runs first.

WelcomeEmailJob.perform_later({ "user_id": 42 })        # Default queue
WelcomeEmailJob.perform_in("5 minutes", { "user_id": 42 })
WelcomeEmailJob.perform_at("2026-05-01T08:00:00Z", { "user_id": 42 })
WelcomeEmailJob.perform_now({ "user_id": 42 })                  # Inline, no queue
WelcomeEmailJob.perform_later({ "user_id": 42 }, "mailers")     # Pick a queue
WelcomeEmailJob.perform_later({ "user_id": 42 }, { "queue": "mailers", "priority": 10 })

Job Class API

Job.enqueue(handler, args, queue_or_opts?)

Enqueue a job by handler name. Returns the job id. The trailing argument is a queue-name string or an options hash { queue, priority, max_retries } (higher priority runs first).

job_id = Job.enqueue("WelcomeEmailJob", { "user_id": 42 })
Job.enqueue("WelcomeEmailJob", { "user_id": 42 }, { "queue": "mailers", "priority": 10 })
Job.enqueue_in(handler, duration, args, queue_or_opts?)

Enqueue with a relative delay. duration accepts "5 minutes", "1 hour", "2 days", etc., or a number of seconds.

Job.enqueue_in("WelcomeEmailJob", "30 minutes", { "user_id": 42 })
Job.enqueue_at(handler, datetime, args, queue_or_opts?)

Enqueue to run at a specific ISO-8601 timestamp.

Job.enqueue_at("WelcomeEmailJob", "2026-05-01T08:00:00Z", { "user_id": 42 })
Job.cancel(job_id)

Cancel an enqueued (not yet started) job.

Job.cancel(job_id)
Job.retry(job_id)

Re-queue a failed or dead job. Keeps attempts and last_error.

Job.retry(job_id)
Job.list(queue?)

List jobs in a queue. Defaults to the configured default queue.

jobs = Job.list("mailers")
Job.queues()

Return the queues that currently hold non-terminal work.

names = Job.queues()

Webhook Class API

For work that targets a URL rather than a Soli handler class. The engine fires the HTTP request itself when the job is due, so it works on every database adapter.

Soli POSTs the payload as JSON with X-Webhook-Event: job, X-Webhook-Delivery: <job-id>, and (when a secret is configured — per-job secret, else SOLI_WEBHOOK_SECRET) X-Webhook-Signature, the lowercase hex HMAC-SHA256 of the body. Non-2xx responses count as failure and are retried with the same exponential backoff as class-target jobs.

Webhook.enqueue(url, payload, opts?)

Enqueue an HTTP POST to url with payload as the JSON body. Returns the job id.

Webhook.enqueue(
  "https://hooks.slack.com/services/T00/B00/abc",
  { "text": "Order #1234 shipped" }
)
Webhook.enqueue_in(url, duration, payload, opts?)

Enqueue with a relative delay. duration accepts "5 minutes", "1 hour", "2 days", etc., or a number of seconds.

Webhook.enqueue_in(
  "https://api.example.com/order-completed",
  "5 minutes",
  { "order_id": 1234 }
)
Webhook.enqueue_at(url, datetime, payload, opts?)

Enqueue to fire at a specific ISO-8601 timestamp.

Webhook.enqueue_at(
  "https://api.example.com/daily-summary",
  "2026-05-01T08:00:00Z",
  { "report": "daily" }
)

Options Hash

The optional last argument controls queue placement, retries, signing, and outgoing headers.

Key Type Purpose
queue String Queue name (defaults to SOLI_JOBS_DEFAULT_QUEUE).
priority Int Higher executes first.
max_retries Int Retry budget.
secret String Per-job HMAC key (overrides SOLI_WEBHOOK_SECRET).
headers Hash Extra HTTP headers attached to the outgoing request.
Webhook.enqueue(
  "https://api.partner.test/event",
  { "kind": "user.created", "user_id": user.id },
  {
    "queue": "external",
    "priority": 10,
    "secret": getenv("PARTNER_HMAC_SECRET"),
    "headers": { "Authorization": "Bearer " + getenv("PARTNER_TOKEN") }
  }
)

Webhook.cancel(job_id) and Webhook.list(queue?) operate on the same underlying _jobs collection as Job — webhook and class jobs share a queue.

Cron Class API

Cron.schedule(name, expr, handler, args?)

Idempotent upsert by name. Calling twice with the same name updates the existing entry rather than creating a duplicate.

Cron.schedule("nightly_report", Cron.daily_at("03:00"), "ReportJob", {})
Cron.schedule("warm_cache",     Cron.every("5 minutes"),  "WarmCacheJob", {})
Cron.list()

List all cron entries.

entries = Cron.list()
Cron.update(id, fields)

Update an existing cron entry. Pass a hash of fields to change.

Cron.update(cron_id, { "schedule": "0 4 * * *" })
Cron.delete(id)

Delete a cron entry by id.

Cron.delete(cron_id)

Cron Expression Helpers

Pure string builders — they produce six-field cron strings (sec min hour day-of-month month day-of-week). That leading seconds field is what distinguishes them from five-field Unix crontab lines, which are rejected rather than silently never firing. Calling a builder has no side effect; only Cron.schedule writes a schedule.

Helper Cron string
Cron.every("5 minutes") 0 */5 * * * *
Cron.every("1 hour") 0 0 * * * *
Cron.every("2 hours") 0 0 */2 * * *
Cron.every("1 day") 0 0 0 */1 * *
Cron.hourly() 0 0 * * * *
Cron.daily_at("03:00") 0 0 3 * * *
Cron.weekly_at("monday", "09:00") 0 0 9 * * Mon

You can always pass a raw cron string instead.

Cron.every refuses an interval cron cannot express, rather than emitting one that never runs: "90 seconds" (no sub-minute field), "90 minutes" (the minute field reaches 59 and 90 is not a whole number of hours), "25 hours" (the hour field reaches 23) and "40 days" (day-of-month reaches 31) are each an error naming what to use instead. Every in-range value is accepted — but note */N means “every value of this field divisible by N”, and each field restarts on its own cycle, so Cron.every("45 minutes") fires at :00 and :45 and the gap across the hour is 15 minutes. That is how cron works everywhere; pick a divisor of 60 (or of 24 for hours) for even spacing.

Declarative static cron

A class can declare a static cron field. On boot, worker 0 upserts a cron entry named after the class — the auto-derived name is the snake-case of the class (nightly_report_job).

class NightlyReportJob
  static cron: String = Cron.daily_at("03:00")

  static def perform(args: Hash)
    Report.generate()
  end
end

Re-registers idempotently on hot reload. Removing the field does not auto-delete the schedule — call Cron.delete(name) explicitly to avoid surprise data loss.

Environment Variables

Configure jobs and cron via env vars (typically in .env):

Variable Description Default
SOLI_JOBS_DEFAULT_QUEUE Queue name when none is supplied default
SOLI_JOB_WORKERS Worker threads that run job code; 0 disables the engine in this process 1
SOLI_JOBS_POLL_MS How often the poller looks for due work (milliseconds). Under --dev the poller ticks every 5000 ms unless this is set. 1000
SOLI_JOBS_LEASE_SECS Lease length; a claimed job is reclaimable this long after its last heartbeat 60
SOLI_JOBS_MAX_RETRIES Default retry budget per job 3
SOLI_JOBS_RETENTION_SECS How long completed rows are kept before pruning 604800
SOLI_WEBHOOK_SECRET Default HMAC key for outgoing Webhook.* deliveries unset

The engine starts with soli serve when the app has jobs — app/jobs/ exists, or a mailer is configured. SOLI_JOB_WORKERS=0 leaves work for soli jobs. SOLI_JOBS_CALLBACK_URL, SOLI_JOBS_SECRET, and SOLI_JOBS_DATABASE are no longer read.

Standalone Worker

soli serve can run the poller next to HTTP. To scale job capacity separately, set SOLI_JOB_WORKERS=0 on the web process and run a dedicated worker:

soli jobs                  # current directory, SOLI_JOB_WORKERS or 1 slot
soli jobs ./myapp --workers 4
soli worker ./myapp        # alias

soli jobs list --queue mailers --state failed
soli jobs retry <id>
soli jobs cancel <id>

The worker loads models, services, mailers, and app/jobs/, registers static cron, then claims and runs work until Ctrl-C / SIGTERM. No HTTP port. On SolidB, _jobs is privileged: set SOLIDB_API_KEY (or admin SOLIDB_USERNAME / SOLIDB_PASSWORD). Loopback with no credentials uses SolidB's bootstrap admin / admin. The same operations are a page at /__soli/jobs (dev-bar tools panel in --dev; in production set SOLI_JOBS_USER + SOLI_JOBS_PASSWORD, SOLI_JOBS_TOKEN, or the shared SOLI_ADMIN_* that also opens /__soli/errors and /__soli/slow_queries). A LiveView that enqueues an upload job is walked through in A Live Field Desk.

Long-Running Jobs

Nothing special is required. Every job already runs on the worker pool, off the request path, with no timeout and with retries — so a heavy handler can't delay request serving.

class MonthlyReportJob
  static def perform(args: Hash)
    # Heavy work — bulk queries, exports, PDF generation — runs on a worker
    # thread with no timeout, and is retried if it raises.
    generate_report(args["month"])
  end
end

Size the pool with SOLI_JOB_WORKERS; each worker holds its own loaded interpreter (models, services, mailers, jobs), so it costs memory as well as concurrency.

At-least-once semantics. A job that outlives its lease (longer than SOLI_JOBS_LEASE_SECS without a heartbeat — a hard-frozen process, say) can be reclaimed and run again elsewhere. Write handlers to be idempotent, and raise the lease for long jobs.

static background: Bool = true is still accepted but has no effect — it described the old opt-out from running jobs on a web worker, which is now the default for every job. It can be removed from your job classes.

Retries & Crash Recovery

  • Backoff is exponential from 5 seconds, doubling per attempt, capped at 1 hour, with a small per-job spread so a burst of sibling failures doesn't retry in lockstep.
  • Attempts increment at claim time, not at completion — so a worker that dies mid-job counts the lost attempt.
  • Leases recover crashes. A running row whose locked_until has passed becomes claimable again, so a killed process's work is picked up rather than stranded. The poller heartbeats in-flight leases each tick.
  • Exhausted budgets are buried. Once attempts passes max_retries the row becomes dead with its last error, and is kept for inspection.
  • Completed rows are pruned after SOLI_JOBS_RETENTION_SECS; failed and dead rows are not.

Because claiming is atomic, several soli serve processes can share one database and one queue safely:

# Two app processes, one Postgres, one queue — each job runs exactly once.
SOLI_DB_ADAPTER=postgres DATABASE_URL=postgres://... soli serve --port 3000
SOLI_DB_ADAPTER=postgres DATABASE_URL=postgres://... soli serve --port 3001

Common Patterns

Send an email asynchronously

class WelcomeEmailJob
  static def perform(args: Hash)
    user = User.find(args["user_id"])
    Mailer.send(user.email, "Welcome!")
  end
end
def create
  user = User.create(req["json"])
  WelcomeEmailJob.perform_later({ "user_id": user.id })
  return redirect("/users/" + str(user.id))
end

Recurring report (declarative)

class NightlyReportJob
  static cron: String = Cron.daily_at("03:00")

  static def perform(args: Hash)
    Report.generate()
    AdminMailer.send_summary()
  end
end

Fire-and-forget webhook to a third party

No handler class needed — the engine POSTs the payload to the URL when the job is due. Retries on non-2xx with exponential backoff.

def create
  order = Order.create(req["json"])
  Webhook.enqueue(
    "https://hooks.slack.com/services/T00/B00/abc",
    { "text": "Order #" + str(order.id) + " placed" },
    { "queue": "external", "max_retries": 5 }
  )
  return redirect("/orders/" + str(order.id))
end

Idempotent retries

Retries are automatic, and a lease-expiry reclaim can run a job more than once, so treat handlers as idempotent.

class ChargeJob
  static def perform(args: Hash)
    charge_id = args["charge_id"]
    if Charge.find(charge_id).processed
      return  # already done — no-op on retry
    end
    Charge.process(charge_id)
  end
end

Notes

  • Filename and class name must match. email_job.sl ↔ EmailJob. A mismatch is a startup error.
  • perform must be static — it's invoked on the class, never on an instance.
  • Job arguments round-trip through JSON; pass plain hashes / arrays / strings / numbers, not class instances.
  • A handler that isn't loaded fails the job (so it retries) rather than marking it done — a rename mid-deploy recovers once the new code is live.
  • In --dev mode, edits under app/jobs/ hot-reload the class without restarting the server.
  • Only worker 0 performs static cron auto-registration on boot, to avoid duplicate writes from multiple workers.