Building an Idempotent Polling Worker with QStash for AI Usage Tracking

TL;DR: Because Vercel serverless functions cannot handle long-running cron jobs reliably, Frugal uses Upstash QStash to trigger an idempotent polling worker every 5 minutes. This architecture guarantees that AI API usage is pulled accurately from providers like OpenAI and Anthropic without double-counting records during network timeouts.

What Is an Idempotent Polling Worker?

An idempotent polling worker is a background process that requests new data from an external API on a schedule, designed so that if the exact same process runs multiple times concurrently or fails halfway through, it will never create duplicate database records or corrupt the system state.

Why It Matters

When tracking API spend across hundreds of users, data integrity is paramount. If our system pulls $500 of OpenAI usage for a tenant, experiences a network timeout, and then retries 30 seconds later, a poorly designed worker might record that $500 twice. This results in false budget alerts and broken trust.

How It Works

The Vercel Problem

Vercel's hobby and pro tiers enforce strict timeout limits on serverless functions (typically 10 to 60 seconds). If you have to poll usage for 500 connected OpenAI accounts, a single API route will inevitably time out before finishing.

The QStash Solution

Instead of doing all the work in one massive cron job, we use Upstash QStash. Every 5 minutes, QStash hits our /api/poll/orchestrator route. This route quickly queries our database for all active tenant connections and immediately pushes 500 separate, lightweight messages back into a QStash queue. QStash then fans out those messages to a /api/poll/worker route, executing them in parallel.

Achieving Idempotency

To ensure we never double-count usage, every usage record fetched from OpenAI is hashed based on its timestamp, provider ID, and usage amount to create a unique fingerprint. When we insert this data into Supabase, we use a Postgres ON CONFLICT DO NOTHING clause. Even if QStash fires the same worker payload three times, the database gracefully rejects the duplicates.

Practical Steps for Implementation

1. Decouple Orchestration from Execution: Use one fast endpoint to schedule jobs, and a separate endpoint to actually perform the API polling.
2. Use a Reliable Message Queue:Configure Upstash QStash to handle retries and DLQs (Dead Letter Queues) so you don't lose data if an endpoint temporarily goes down — the same pattern we use for handling Stripe webhooks.
3. Hash Your Records: Create a deterministic unique ID for every piece of usage data you ingest.
4. Upsert, Don't Insert: Always write to your database using conflict resolution strategies to enforce idempotency at the storage layer.

Common Mistakes

The most common architectural mistake is trying to manage state within the serverless function itself. Serverless functions can be frozen, killed, and restarted dynamically. If you rely on in-memory arrays to track which users you've polled, you will lose data the moment the function scales down.

FAQ

What is Upstash QStash?

Upstash QStash is a serverless message queue and scheduling service designed specifically for HTTP-based environments like Vercel and Cloudflare Workers.

Why not use a standard Vercel Cron Job?

Vercel Cron Jobs are excellent for simple tasks, but they lack advanced retry mechanics, fan-out capabilities, and long-running execution guarantees needed for enterprise data pipelines.

What does idempotent mean in software?

Idempotent means that performing an operation multiple times has the exact same result as performing it once. In database terms, it means safe retries without creating duplicate rows.

Is polling usage data better than using a proxy?

For v1.0 applications, polling is vastly superior. It avoids adding latency to the user's application, removes the single point of failure that a proxy introduces, and is significantly cheaper to operate.

Conclusion

Building reliable background pipelines on serverless infrastructure requires embracing ephemeral compute. By shifting state management to Postgres and relying on QStash for robust delivery and retries, Frugal tracks thousands of API calls flawlessly without the operational overhead of dedicated worker instances.