Skip to main content

​
Installation

​
Overview

The @ai-billing/openai-compatible package provides middleware for tracking token usage and calculating costs when using any OpenAI-compatible provider (like xAI, Together AI, or Groq) with the Vercel AI SDK. It captures OpenAI-compatible metrics, such as internalReasoningTokens and inputCacheReadTokens, ensuring that costs are accurately reflected across different providers.

​
Usage

To use the middleware, wrap your OpenAI-compatible model using wrapLanguageModel from the ai package and pass the createOpenAICompatibleMiddleware.
1

Initialize the OpenAI Compatible provider

First, set up the provider using createOpenAICompatible. You must specify a name which will be used as the providerId in the middleware.
2

Define model pricing

Set up a price resolver to define the costs for the models you’ll be using. You can specify costs for standard prompt/completion tokens, as well as provider-specific metrics like internalReasoningTokens or inputCacheReadTokens.
3

Create the billing middleware

Initialize the OpenAI Compatible billing middleware. You need to provide a destination (such as consoleDestination), your priceResolver, and the providerId that matches the name you used when creating the provider.
4

Wrap the model

Use wrapLanguageModel from the ai package to apply the billing middleware to your OpenAI-compatible model.
5

Use the wrapped model

Finally, use the wrapped model with AI SDK functions like generateText or streamText. The billing middleware will automatically track tokens, handle provider-specific metrics, and calculate costs.