Emit one usage event per completion, then rate it by model. To set up metering and billing for an API that charges per token, send input and output token counts as dimensions on one event, aggregate per customer and model, and check the balance before serving the request. Flexprice runs that check in milliseconds on the stream that produces the invoice. Counting tokens is easy. Refusing request 100,001 first is the part teams skip.

Key Takeaways

• Meter input and output tokens separately, because providers price them differently and one blended number hides your margin.

• Gate on a pre-request balance check. A wallet resolving at invoice finalization can't stop overspend, which is how Lago's OSS tier behaves.

• Emit the event when the completion finishes, since a streaming response has no token count until the stream closes.

• Flexprice checks balances in milliseconds at up to 1 million events per second, under 60ms P99.

How do you set up per-token metering for an API?

Define the billable metric first, then wire the event. Changing the metric later means re-rating historical usage.

1. Pick the metric. Input tokens, output tokens, or a weighted combination.

2. Emit one event per completion to Usage Metering, carrying customer ID, model and both token counts.

3. Aggregate per customer and model. Sum works for one rate. Weighted-sum handles input and output priced differently.

4. Map each model to a price per 1,000 tokens, so a new model is a config change, not a deploy.

5. Gate the request against an entitlement or wallet balance before calling the provider.

Skip step 5 and you've built reporting, not billing.

How do you count input and output tokens for billing?

Meter them separately, because output usually costs several times more than input. Collapsing both into one "tokens" metric means the invoice can't reflect that difference.

Two approaches work:

• Two metrics, two rates. Simplest to reason about, and the line items read plainly.

• One metric with a weighted aggregation. Multiply output tokens by a factor so one rate covers both.

Take counts from the provider's usage response, not a local tokenizer. Estimates drift from provider billing, and that lands in your margin.

How do you check balance in real time before serving a request?

Read the balance on a separate low-latency path before the provider call, and let the result decide whether the request proceeds. It has to be fast enough not to show up in your latency budget.

"Flexprice processes usage in real-time and credit checks happen in milliseconds without affecting our API performance." - Justin Benson, Co-founder.

Decide the behaviour up front:

• Block at zero balance, which protects you and frustrates the customer.

• Allow overage to an agreed cap, billed via Credits and Wallets.

• Alert at a threshold so the customer tops up first.

Flexprice is enterprise-grade, open source usage based billing infrastructure for AI and SaaS companies. It can be deployed in your own VPC, on-prem, or on Flexprice's managed cloud, so the check runs next to your inference path rather than across the public internet. We're GDPR compliant and ISO/IEC 27001 certified company-wide.

How do you handle streaming responses in token metering?

Emit the event when the stream closes, not when it opens, because the output token count doesn't exist until then. That one rule prevents most token-billing bugs.

Still needs handling:

• Client disconnects or errors mid-stream. Your provider bills you for tokens already generated, so decide whether the customer pays for a partial response.

• Long streams crossing a billing period. Attribute the event to the completion timestamp so it lands in one period.

Log the raw provider usage payload alongside your event. When a customer disputes a count, that settles it.