Skip to content
Shuai Zhang
中文

Independent Project · Open Source

AI Usage Infrastructure Exploration

An independent open-source exploration of token-native metering, cost attribution, and real-time consumption controls.

Problem

AI products generate consumption continuously, while revenue systems typically operate in delayed billing cycles. The difficult part is not simply counting tokens. It is maintaining accurate customer-level financial state across streaming responses, retries, failures, model-specific pricing, pricing changes, and prepaid limits.

Conceptual architecture

  1. SDK / GatewayEmit usage at the product edge
  2. Usage Event SchemaNormalize immutable usage facts
  3. Stream aggregationAggregate usage for rating and budgets
  4. Budget stateHold near-real-time remaining limits
  5. Usage & cost APIsExpose attributable cost and status
  6. Billing / warehouseDownstream commercial and analytics sinks
Conceptual architecture for an independent exploration; not a statement of customer adoption or production deployment.
  1. 01

    SDK / Gateway

    Emit usage at the product edge

    In: API calls · Out: usage events

    At-least-once delivery assumptions

  2. 02

    Usage Event Schema

    Normalize immutable usage facts

    In: raw events · Out: typed records

    Schema evolution without silent loss

  3. 03

    Stream aggregation

    Aggregate usage for rating and budgets

    In: events · Out: quantities / windows

    Replay and late-data handling

  4. 04

    Budget state

    Hold near-real-time remaining limits

    In: reserves / settlements · Out: decisions

    Reserve-and-reconcile consistency

  5. 05

    Usage & cost APIs

    Expose attributable cost and status

    In: queries · Out: customer-level views

    Read models that match ledger truth

  6. 06

    Billing / warehouse

    Downstream commercial and analytics sinks

    In: settled usage · Out: invoices / analysis

    Idempotent export boundaries

Engineering decisions

  • Event-based metering

    Immutable usage events support replay, attribution, and downstream flexibility better than mutable counters alone.

  • Idempotency and replay

    Duplicate delivery can produce duplicate financial outcomes unless deduplication is explicit.

  • Monetary precision

    Floating-point accumulation is inappropriate for financial totals; use precise decimal money math.

  • Reserve and reconcile

    Estimated pre-request cost, reserved balance, and post-response actual usage are different states and must be reconciled.

  • Lite versus streaming architecture

    Simpler HTTP/Redis paths can be enough early; stream processors become justified when volume, fan-out, and recovery requirements grow.

Non-goals

  • Not a complete invoice platform
  • Not a revenue-recognition system
  • Not a universal pricing catalog
  • Not proof of commercial adoption
  • Not an employer product or endorsed project

Throughput or latency benchmarks are withheld until methodology review is complete. No customer, adoption, production, or revenue claim is made here.

← Back to Work