All work

Relay

A support-ticket pipeline. A customer submits a ticket, a background worker triages it with an LLM and drafts a reply, and the customer watches it happen live.

Type
Backend system with a web app
Stack
Bun, Hono, PostgreSQL, Redis, AWS SQS, Socket.io, React
Live
Live Link
Source Code
GitHub

Overview

Relay takes a support request from submission to a drafted reply without a person in the loop. A worker reads the ticket, sorts it by category, priority and sentiment, then writes a customer reply, an internal note and next steps.

The customer sees each step on their ticket page as it happens. Staff have an ops console with a live feed, alerts, and a replay button for tickets that failed.

Problem

Most help desks answer a new ticket with “we’ll get back to you” and nothing else. The customer cannot see whether anyone has read it.

An LLM can read and answer a ticket in seconds, but it is an unreliable part: providers time out, go down, and sometimes return output in the wrong shape. The problem was to put that part inside a workflow that still behaves correctly when it misbehaves.

Architecture

RelayA demo ticket through the system’s real states

Flowchart of Relay. A React app submits a ticket to the Hono API, which stores it as queued and puts a message on SQS. A worker picks it up and makes one schema-checked call per stage through the LLM gateway: triage, then draft. The gateway uses Gemini and switches to Groq, the backup model, when the primary fails or times out. A stage that fails is retried up to three times; after that the ticket needs attention, ops is alerted, and staff can replay it from the stage that failed. The ticket moves queued, triaging, drafting, completed, and each step is pushed live to the customer’s page and the ops feed.
  1. The API stores a new ticket as queued and sends a message to SQS.
  2. The worker picks it up and runs triage, then draft.
  3. Each step and outcome is written to an append-only event log and pushed live to the customer’s page and the ops feed.
  4. A stage that fails is retried up to three times. After that the ticket is marked as needing attention, ops is alerted, and staff can replay it.

Tech decisions

A queue and a worker, not work inside the request.

Submitting a ticket returns at once, and each stage can be retried on its own.

One forced, schema-checked call per stage.

Each call must return a named function schema, validated with Zod. It is not an agent loop, so cost and behavior stay predictable.

A gateway with a primary and a backup provider.

Requests go through Portkey to Gemini, with Groq as the backup after a failure or a 30 second timeout.

One function records every event.

It writes the event row and publishes to the live feed together, so the database and the screen cannot drift apart.

Socket events only invalidate cached queries.

The browser always reads ticket data from the API, so there is one source of truth.

A fake LLM mode at the provider boundary.

Validation, retries and events all still run, so the whole pipeline can be tested without a provider.

Challenges and solutions

  1. Challenge

    Output that cannot be trusted.

    A model can return malformed or incomplete output, and a provider can be slow or down.

    Solution

    Treat bad output as a failure.

    Output that fails validation is recorded as an invalid response and retried like any other failure. After three tries the ticket goes to a person, who can replay it from the stage that failed.

  2. Challenge

    Customer content in shared channels.

    Every ops user sees the ops feed, so ticket text and AI output must never reach it, even by accident in an error message.

    Solution

    Build public data field by field.

    Public views, the ops feed and alerts list their fields explicitly. Alerts carry fixed reason codes, never error text. Guard tests fail if any other file publishes to the ops room.

  3. Challenge

    Bots spending the AI budget.

    A public form that triggers paid LLM calls is an easy target.

    Solution

    Four checks in a fixed order.

    Body validation, a per-address hourly limit, Cloudflare Turnstile, then a daily cap. A refused request creates no row and costs no LLM call.

  4. Challenge

    Two runtimes.

    The app runs on Bun, but module mocking in the test runner does not work under Bun.

    Solution

    Run tests on Node, the app on Bun.

    Vitest runs on Node so mocking works. Playwright tests run against the built app.

  5. Challenge

    Migrations on a pooled database.

    The pooled connection lacks the session features that migration tools rely on.

    Solution

    Two connection strings.

    Migrations use the direct database URL. The running app uses the pooled one.

Results

Accessibility is checked automatically on every route. Every provider call is recorded with its outcome, latency and token count, and summarized in the ops console. The app ships as one Docker image and deploys only after CI passes.

test files across the API, worker, web app and end-to-end suite
120
commits since April 2026
267
minimum Lighthouse score in all four categories, on six pages
90+
Redis commands per ticket, and none while idle
10

A ticket goes from submission to a drafted reply without a staff member touching it, and the customer watches each step happen live.