СледваСледващо ръководство
Estimating LLM API Costs and Token Budgets
технически
Техническо РЪКОВОДСТВО
Max Tokens and Stop Sequences
Max tokens is the cap on how many tokens a model may generate in one response, and stop sequences are strings that end generation as soon as the model produces them.
На тази страница4 минути четене
Преглед
Together with the finish reason the API returns, they decide where an answer ends and tell you whether it ended naturally or was cut off. Getting them right prevents truncated JSON, half-finished answers and wasted spend.
Дълбоко гмуркане
A language model generates one token at a time until something tells it to stop. There are three common reasons it stops. It produces its own end-of-turn token, meaning it decided the answer is complete. It reaches the maximum number of output tokens you allowed. Or it produces text that matches one of your stop sequences. Max tokens is often confused with the context window. The context window is the total number of tokens the model can handle at once, input plus output. Max tokens only limits the output of a single call. A model with a very large context window can still produce short responses if max tokens is set low, and input plus max tokens generally has to fit inside the context window. APIs report why generation ended. OpenAI's Chat Completions API returns a finish_reason such as 'stop' for a natural end or a matched stop sequence, 'length' when the output limit was hit, and 'tool_calls' when the model wants to call a tool. Anthropic's Messages API returns a stop_reason such as 'end_turn', 'max_tokens', 'stop_sequence' or 'tool_use'. Anthropic requires max_tokens on every request. For its reasoning models, OpenAI introduced max_completion_tokens, which counts hidden reasoning tokens as well as visible output. Stop sequences are useful for structured formats: ending after a closing tag, stopping before a speaker label, or cutting off after one list item. The matched text is normally not included in the returned output, so code should not expect to see it. The most common mistake is ignoring the finish reason. A truncated response can look plausible, especially prose, while structured output simply breaks. Another misconception is that a low max tokens makes the model write concisely. It does not; the model writes as it normally would and gets cut off. Ask for brevity in the prompt and use the limit as a safety cap.
Стратегическо въздействие
Разходи и бюджет
Архитектурните решения стимулират производителността и оперативните разходи в продължение на години.
По-ясни решения
Техническото образование помага на екипите да изберат правилния стек, а не само най-новия.
Контрол на качеството
По-добрият инженерен избор намалява инцидентите, свързани с надеждността в производството.
The Future of Max Tokens and Stop Sequences
Output limits have grown as models support longer generations, and structured output features that constrain responses to a schema reduce, but do not remove, the risk of truncation, since a long enough response can still hit the cap. Reasoning models have made output budgeting more important because hidden tokens now share the same limit. Parameter names and finish reason values differ between providers and have changed over time, so it is sensible to check current documentation and wrap these details in a small layer of your own code.
Внедряване в реалния свят
A summarizer sets max tokens to 150 and some summaries end mid-sentence; checking the finish reason shows 'length' for those calls, so the app raises the limit and asks for shorter summaries in the prompt instead.
A script that extracts data as JSON fails to parse about one response in fifty; each failure had hit the output limit, so the code now retries with a higher limit when the finish reason says the output was truncated.
A developer generating one line of dialogue at a time uses the stop sequence 'User:' so the model does not continue and write the user's next line too.
A team moving to a reasoning model finds answers coming back empty because hidden reasoning used up the output limit, so they raise the limit to leave room for both reasoning and the answer.
Рискове и предпазни огради
Оптимизирането на един бенчмарк може да скрие по-широки системни слабости.
Разходите за инфраструктура и поддръжка често се подценяват.
Пропуските в сигурността и видимостта могат да нарастват, когато системите стават по-сложни.
Пътна карта за изпълнение
Определете целите за латентност, качество и разходи преди внедряването.
Бенчмарк при реалистични условия на натоварване и данни.
Мониторинг на инструмента за грешки, отклонение и въздействие върху потребителя.
Подгответе пътеките за връщане назад и реакция на инцидент преди мащабиране.
Продължете да изследвате
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Max Tokens and Stop Sequences quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Често задавани въпроси
What is Max Tokens and Stop Sequences?
Max tokens is the cap on how many tokens a model may generate in one response, and stop sequences are strings that end generation as soon as the model produces them. Together with the finish reason the API returns, they decide where an answer ends and tell you whether it ended naturally or was cut off. Getting them right prevents truncated JSON, half-finished answers and wasted spend.
What does max tokens limit?
Max tokens caps output for one call. The context window is the separate total limit covering input and output.
Your call returns a finish reason of 'length' in OpenAI's API. What happened?
'length' means generation stopped because the output limit was reached, so the response may be incomplete.
Which Anthropic stop_reason indicates the model decided its answer was complete?
'end_turn' means the model ended naturally. 'max_tokens' means it was cut off, and 'stop_sequence' means one of your strings matched.
Why does setting a low max tokens value not make answers concise?
The limit is a hard stop, not an instruction. Ask for brevity in the prompt and keep the limit as a safety cap.
Is the matched stop sequence normally included in the returned text?
Stop sequences end generation and are typically not included in the returned output, so your code should not rely on seeing them.
Продължавай да учиш
Свързани ръководства
Още ръководства, избрани за тази тема