Asynchronous Operations: Long-Running Jobs, Polling and Webhooks

Generating a 50-page PDF or importing a million rows cannot finish inside one HTTP request: load balancers time out and connections drop. In this lesson you will learn the REST pattern for long-running work, 202 Accepted plus a job resource the client polls, and its push-based complement, signed webhooks.

The 202 Accepted Pattern

Instead of doing the work inline, the endpoint records a job, hands it to a background worker and returns 202 Accepted, meaning "received, not done yet". The Location header says where to check progress:

http

POST /reports HTTP/1.1
{ "from": "2026-01-01", "to": "2026-06-30" }

HTTP/1.1 202 Accepted
Location: /jobs/8f1c2d4e

{ "id": "8f1c2d4e", "status": "pending" }

The client then polls GET /jobs/8f1c2d4e. While the job is pending or running the server returns 200 with the current state and a Retry-After hint; clients should honour it and back off for long jobs. When the job has succeeded, the server answers 303 See Other to the finished resource; when it has failed, the body carries the error.

Implementing the Job Resource

javascript

const jobs = new Map(); // production: database table + queue (BullMQ)

app.post("/reports", (req, res) => {
  const id = crypto.randomUUID();
  jobs.set(id, { id, status: "pending", progress: 0, createdAt: new Date().toISOString() });
  queue.add("generate-report", { jobId: id, filters: req.body });
  res.status(202).set("Location", `/jobs/${id}`).json({ id, status: "pending" });
});

app.get("/jobs/:id", (req, res) => {
  const job = jobs.get(req.params.id);
  if (!job) return res.status(404).json({ error: "Job not found" });
  if (job.status === "succeeded") return res.redirect(303, `/reports/${job.resultId}`);
  if (job.status === "failed") return res.json(job);
  res.set("Retry-After", "5").json(job);
});

A worker consumes the queue, updates progress, and finally sets status to succeeded with a resultId or failed with an error. Jobs are resources like any other: scope them to the caller, support DELETE /jobs/:id for cancellation, and expire old ones.

Webhooks: Push Instead of Poll

Polling wastes requests when jobs take hours. A webhook reverses the direction: the client registers a URL (POST /webhooks with { "url": "...", "events": ["report.completed"] }) and your API POSTs to it when something happens. Each delivery carries an event envelope:

json

{ "id": "evt_01J8Z3", "type": "report.completed", "createdAt": "2026-09-27T10:15:00Z",
  "data": { "jobId": "8f1c2d4e", "reportUrl": "/reports/rep_42" } }

Because the receiver is a public URL, anyone could post fake events to it. Sign every delivery with an HMAC-SHA256 over the timestamp and raw body, using a secret shared at registration, sent as X-Signature: t=1727433600,v1=<hex>. The receiver recomputes it and compares in constant time:

javascript

app.post("/hooks/reports", express.raw({ type: "application/json" }), (req, res) => {
  const [t, v1] = req.get("X-Signature").split(",").map((p) => p.split("=")[1]);
  const expected = crypto.createHmac("sha256", process.env.HOOK_SECRET).update(`${t}.${req.body}`).digest("hex");
  const fresh = Math.abs(Date.now() / 1000 - Number(t)) < 300; // 5-minute replay window
  if (!fresh || !crypto.timingSafeEqual(Buffer.from(expected), Buffer.from(v1))) return res.status(401).end();

  enqueueForProcessing(JSON.parse(req.body)); // respond fast, work later
  res.status(200).end();
});

Note the raw body parser: signatures cover exact bytes; re-serialized JSON would differ.

Delivery Guarantees

Senders retry failed deliveries (non-2xx or timeout) with exponential backoff, so receivers get at-least-once delivery and must be idempotent: store processed event ids and skip duplicates. Senders should time out quickly, disable endpoints that keep failing, and offer an event log for catch-up.

Common Mistakes

Returning 200 or 201 from an endpoint that only queued the work; clients think it finished.
Doing heavy work inside the webhook handler before responding; the sender times out.

Key Takeaways

Return 202 Accepted with a Location header for work that outlives a single request.
Model jobs as resources with pending, running, succeeded and failed states; 303 to results.
Webhooks push events to registered URLs; sign them with HMAC and verify the raw body in constant time.
Delivery is at-least-once, so receivers respond quickly, process asynchronously and deduplicate by event id.

Next lesson: Documenting APIs with OpenAPI and Swagger UI — describe every endpoint in a machine-readable contract.