Why usage-based billing is an engineering problem?

Imagine you’ve just joined OpenAI as an Engineering Manager. You’re excited to work on cutting-edge AI products, but instead, your day-to-day work involves not optimizing AI models, but implementing these changes:

• “Let’s stop charging based on emails sent and instead charge based on how many conversions happened.”

• “Let’s increase the price from $1 per email to $1.5 per email.”

  • Does this apply to all users or only new sign-ups?

  • Will my existing users continue to pay the old price?

  • Should we offer a transitional discount, like $1.2 per email for old users for the next 3 months?

• “Let’s increase the number of free trial credits from 500 to 1000.”

• “Can we give ABC Corp a 90-day trial period instead of the standard 60 days?”

• “Let’s charge $1 per email for users on free credits and early adoption, then increase it to $1.5 per email once credits are over.”

Companies often assume that pricing changes are purely a business decision, but they introduce significant technical and operational complexity. It requires robust systems for tracking, measuring, and billing usage accurately. Companies making the transition often discover new challenges around data, billing infrastructure, and internal processes.

1. Data Ingestion:

   You need to reliably capture every unit of usage that you plan to bill for. This means instrumenting your product or service to emit usage events (e.g., API call logs, transaction counts, compute metrics) and ingesting those events into a billing system. The challenges include:

   • Scalable Data Pipelines: Many SaaS products generate billions of usage events per month, requiring robust data pipelines to handle this firehose of data. This often involves tools like Kafka, Pulsar, or Kinesis for event streaming, along with ClickHouse or Druid for real-time analytics.

   • Normalization and Deduplication: Data often comes from multiple sources (product itself, databases, caches, external APIs) and must be normalized to avoid double counting. This can require complex data transformation pipelines and deduplication logic to ensure accurate billing.

   • Real-Time Ingestion: You can’t afford to process usage data in batch if your customers expect real-time visibility. This means building low-latency, distributed data pipelines that can handle high throughput without dropping events.

   • Data Integrity: Preventing duplicate events and data loss is critical. According to one study, over 70% of companies cited data management issues (e.g., inconsistent formats, inability to process in real-time) as a major cause of failure in usage-billing projects.
     
     

2. Metering and Aggregation:

   Once data is ingested, it needs to be metered – counted or measured per customer according to defined metrics. This sounds straightforward, but it quickly gets complex:

   • Multi-Dimensional Metering: You might need to meter different dimensions (e.g., per workspace, per end-user, per feature). This means creating data models that can track usage at multiple levels of granularity without introducing excessive latency.

   • Retry Handling: If a user clicks the retry button on a failed API call, should that count as two billable events or just one? This requires idempotency checks and intelligent event deduplication to avoid double billing.

   • Time Alignment: Metering needs to be aligned with billing cycles, including time zone differences, pro-rata calculations, and multi-region consistency. This is particularly challenging when customers operate in different geographies.

   • Low Latency Requirements: If metering lags, customers can exceed their quotas without immediate detection, leading to billing disputes. This often means integrating your metering layer directly with real-time data pipelines.
     
     

3. Rating and Billing Computation:

   After metering usage, you need to apply pricing rules to compute charges. This involves taking the raw usage data (e.g., 12,000 API calls this month) and applying the pricing model:

   • Complex Pricing Structures: Tiers, volume discounts, overage fees, and dynamic pricing based on customer segments add significant complexity. For example, a customer on a Premium plan might have different per-unit rates than a customer on a Basic plan, requiring real-time decision logic.

   • Real-Time Rating: Some billing systems apply pricing logic in real-time as events come in, while others rely on batch processing. Real-time rating can be more responsive but requires low-latency databases and efficient in-memory computation.

   • Custom Pricing Models: Supporting customer-specific pricing, volume discounts, and special offers adds further complexity. For example, a large enterprise might have a custom pricing agreement that requires special handling in the billing system.

   • Precision in Billing: Errors here directly impact revenue and customer trust, making accuracy critical. This often involves building rigorous testing and audit pipelines to catch discrepancies before they reach the customer.
     
     

4. Automated Reports and Notifications:

   In a UBP model, providing customers with real-time insights into their usage is crucial to prevent unexpected charges and maintain trust. Key considerations include:

   • Dynamic Dashboards: Implement real-time dashboards that aggregate usage data across services, offering customers immediate visibility into their consumption patterns. This often involves building data pipelines that can handle high-cardinality metrics and real-time joins.

   • Automated Alerts: Set up threshold-based notifications to inform customers as they approach usage limits, helping them manage consumption proactively. This requires real-time event processing and intelligent alert suppression to avoid alert fatigue.

   • Spending Controls: Allow customers to define spending caps or usage limits, automatically pausing services or sending alerts when thresholds are met. This requires tight integration with your API gateways and service proxies to enforce these limits without introducing latency.
     
     

5. Managing Usage Limits and Entitlements:

   Defining and enforcing usage limits requires a robust system that can handle diverse customer plans and entitlements. Challenges include:

   • Flexible Entitlement Management: Develop systems that can assign and track entitlements per customer, accommodating various plans and custom agreements. This often involves building a dedicated entitlements microservice with fine-grained access control.

   • Real-Time Enforcement: Ensure that usage limits are enforced in real-time to prevent overages, which may involve integrating with API gateways or service meshes like Envoy or Istio.

   • Graceful Handling of Overages: Design mechanisms to handle overages gracefully, such as soft limits that notify customers or hard limits that restrict access until usage resets or upgrades occur.
     
     

6. Revenue Recognition and Financial Forecasting:

   UBP introduces variability in revenue streams, posing challenges for financial planning and compliance. Key aspects to address are:

   • Accurate Revenue Recognition: Implement systems that recognize revenue based on actual usage, aligning with accounting standards like ASC 606 or IFRS 15.

   • Forecasting Variability: Develop models that can predict revenue fluctuations due to usage patterns, aiding in budgeting and financial planning. This often involves using machine learning models to identify trends and anomalies.

   • Audit Readiness: Maintain detailed records of usage and billing to support audits and demonstrate compliance with financial regulations.

In summary, implementing usage-based pricing requires building a metering and billing pipeline that can handle high-volume data with accuracy and low latency. You need strong data engineering for ingestion, careful design of metrics and pricing logic, integration with your product to surface usage info, and rigorous testing/auditing to ensure bills are correct.

But what if I tell you that you can set up usage-based billing for your AI model within a weekend using Flexprice?

Check out this blog to learn how you can:

• Create metered features without writing custom tracking code

• Set up flexible pricing plans in minutes

• Automate real-time billing and invoicing without manual intervention

Ready to take control of your pricing? Try Flexprice today. It’s open source, free to use, and built for builders like you.