An AI founder's guide to prepaid credits

Looking at OpenAI’s pricing page, you’ll notice a few things:

• It’s entirely usage-based, without any fixed fee. We call this ‘scale to zero’ pricing: you start for free and only pay more on demand as you use the API. This decreases the barrier to adoption, so an indie hacker kicking the tires on a new project can start using OpenAI on day one.
• It’s granular and flexible enough to accommodate different types of use — particularly important when there is already a large range of model capabilities. Not every query needs the same model backing the answer; some rely on larger or smaller context windows, whereas some need explicit fine-tuning.
• The pricing axis is easy to understand in the context of OpenAI’s customers. OpenAI charges for tokens that correspond to input and output sequence length. As a user of the API, you can control and predict these variables. Notably, it didn’t have to be this way: we can imagine a world in which OpenAI charged for GPU runtime in seconds which would’ve been effectively impossible for an end-user to control.

Although pay-as-you-go pricing models like these are a great fit for starter use cases, they’re oftentimes not a good fit at scale. Ultimately, they don’t offer the predictability that’s necessary for larger deal sizes, which is valued by both the vendor and the customer.

To close larger deals, most companies – including OpenAI –  elect to move towards a prepaid commitment model where customers contractually buy an upfront balance that is burned down over time. Typically, the vendor locks in the customer and their associated revenue forecast in exchange for a sizable discount on the usage. 

Considerations when implementing prepaid credits

If you’re considering choosing a prepaid model for your business, there are several decisions you’ll have to make.

The first element to establish is the currency of commitment: should customers commit a dollar amount (e.g. $50,000 of spend) or purchase a number of app-specific ‘custom credits’? Custom credit units can be powerful for two reasons:

1. Structurally, they make discounting simpler. To provide discounts for commitments, simply charge less per credit in the initial purchase. The “cost per unit of use” — i.e. per request — is priced in credits and can stay the same for all customers. Because you’re only tweaking one variable, your product catalog is easier to manage, and changing pricing in the future is a simpler task.
2. Introducing a custom credit unit can help you create different credit balances that burn down separately for different product lines. For example, OpenAI might introduce “LLM credits” and “multimodal credits” to better align pricing amongst different technologies or business units.

It’s important to think through this carefully in a fast-changing industry — it wouldn’t make much sense for OpenAI to sell “GPT-4 credits” for the next year, when the model may very well be obsolete by then. It’s best to decide this based on your product shape and optimize for what’s simplest for your users to understand. 

The next term to iron out is how credits will expire over time or optionally roll over in renewals. Customers often like prepurchase models because they’re able to negotiate long-term discounts despite seasonal use (e.g. e-commerce companies facing a large peak during the holidays). However, it’s still important for the vendor to create a contractual incentive for use by ensuring that credits cannot be held for years down the line. This is also critical from the perspective of accounting and business health; any unused credits are a deferred revenue burden, and can only be recognized as revenue when they’re used or expired. 

In many cases, customers will outpace their initial projections and their commitment will be exhausted well before the contract term. When this happens, you reach another important decision point: you can either choose to entirely block usage or to transition smoothly to a pay-as-you-go model. If you don’t block use, the rate at which extra use is charged can be a tough call. You certainly don’t want to negatively incentivize additional product usage, but you still want customers to maintain the advantage of having paid upfront.

How Orb can help

A modern billing system like Orb gives you the ability to model prepaid commitments out of the box. Here’s how it works:

• A “customer” in the Orb platform – representing your user – maintains a prepaid balance, which can be in custom credit units or a real-world currency. Orb keeps an auditable ledger of all actions that change this balance, including deductions, expirations, and credit purchases.
• The customer subscribes to a set of “prices” (e.g. compute, storage, fine-tuning, GPT-4 calls) which all have pricing information in the currency corresponding to your commitment balance. Orb allows you to represent prices like “$0.06 / 1K tokens” or “6 credits / 1k tokens” equally well.
• As your application sends usage events to Orb, the customer commitment is automatically burned down in real-time. To accomplish this, Orb calculates how that event affects the quantity that should be used (translating the event’s metadata to an updated quantity through a query) and how that quantity should translate into the commitment’s currency.
• When a customer’s existing commitment is exhausted, Orb automatically starts to accrue new usage onto the upcoming invoice, taking into account a configured overage rate.

This prepaid commitment functionality follows the core principles that underpin all aspects of the Orb platform:

• It scales reliably – Orb can keep track of real-time customer balances, even when they’re being deducted with thousands of events per second.
• It’s flexible enough to handle all metrics and pricing structures. Whether you have a simple setup (each request deducts a single credit) or a very sophisticated contract (e.g. an average metric, which potentially decreases during the period), Orb can accurately deduct the correct amount from the customer balance.
• It’s robust to real-world considerations of your business. Without any additional effort, Orb handles late-arriving event data, “backdated” actions, and amendments to usage that require replaying a series of events on the ledger. Orb also provides alerting when your customers are near usage thresholds, giving your internal teams the opportunities to motivate upsells and renewals.

Why prepaid credits fit AI companies so well

Looking to the future, AI companies will likely adopt prepaid commitments as a business strategy for three reasons. Firstly, prepaid commitments provide immediate cash inflows that increase liquidity and financial position, crucial due to high COGS. This means the companies wouldn't have to rely continuously on venture funding. Secondly, companies offering volume-based discounts for larger commitments will witness increased customer demand as these customers mature. As these customers become more adept at predicting their utilization patterns, they will gain more confidence in making commitments upfront. Thirdly, prepaid commitments will enhance the revenue stability of AI companies, particularly when dealing with customers experiencing high demand fluctuations.