What is Usage-Based Billing?

The amount of processing power consumed while producing each response.

Additional inference computation used during response generation to improve quality or reasoning.

An inference acceleration method where a small draft model proposes tokens that a larger model verifies in parallel.

An attention pattern where each token attends only to a fixed-size window of nearby tokens to reduce compute.

The runtime phase where a trained model generates predictions or outputs.

A method that retrieves external knowledge and feeds it into generation at inference time.