When I joined Eccentric Engine to work on EchoAI, the brief was straightforward: let automotive marketers generate on-brand vehicle visuals for any background or market in seconds. What followed was one of the most architecturally interesting challenges I've worked on — because "in seconds" at scale is a very different problem from "in seconds" in a demo.
The naive approach and why it fails
The first instinct is to call the inference endpoint synchronously — user submits parameters, you POST to ComfyUI, wait for the image, return it. This works fine for one user. At ten concurrent users it starts queuing. At fifty it times out. ComfyUI is a compute-heavy process; each generation consumes significant GPU time. You cannot treat it like a database query.
Queue-first design with BullMQ
We moved to an async job model from the start. The NestJS API receives the generation request, validates parameters, enqueues a job in BullMQ backed by Redis, and immediately returns a job ID. The frontend polls for status via EventStream — a lightweight Server-Sent Events connection that pushes progress updates as the ComfyUI workflow executes. This decoupling meant we could scale the inference workers independently of the API, add priority lanes for premium users, and implement dead-letter queues for failed jobs without touching the frontend.
Prompt engineering as a typed schema
The hardest problem wasn't the queue — it was consistency. ComfyUI workflows are node graphs, and small prompt variations produce wildly different outputs. We built a typed TypeScript schema that maps vehicle attributes (make, model, trim, colour, year, market) to deterministic ComfyUI workflow parameters. Every generation is reproducible from its schema, which also made debugging dramatically easier: when a customer reported an off-brand result, we could replay the exact workflow.
Real-time UX: EventStream over WebSocket
We considered WebSockets for progress updates but chose Server-Sent Events. SSE is unidirectional, simpler to implement, works through HTTP/2 multiplexing, and doesn't require a persistent bidirectional connection. For a progress bar that updates five times over thirty seconds, it's the right tool. The NestJS EventStream module made this almost trivial to wire up.
What I'd do differently
The one thing I underestimated was observability. BullMQ has a dashboard (Bull Board) but we didn't set it up until we had our first production incident. Build observability before you need it, not after. Also: rate-limit your inference workers at the queue level, not the API level — it's much easier to reason about.