GUIDE teknik

Max Token ak taxawal toppalante

Max tokens is the cap on how many tokens a model may generate in one response, and stop sequences are strings that end generation as soon as the model produces them.

Déglul tegtal biiIA-nettali · ~3 min

Ci xët wii4 simili jàng
  1. Résumé
  2. Plongeur bu xóot
  3. Doxal ci àdduna dëgg
  4. The Future of Max Tokens and Stop Sequences
  5. Teg ko ci jëf
  6. Weyal di banneexu
  7. Laaj yi ñuy faral di laaj

Résumé

Together with the finish reason the API returns, they decide where an answer ends and tell you whether it ended naturally or was cut off. Getting them right prevents truncated JSON, half-finished answers and wasted spend.

Plongeur bu xóot

A language model generates one token at a time until something tells it to stop. There are three common reasons it stops. It produces its own end-of-turn token, meaning it decided the answer is complete. It reaches the maximum number of output tokens you allowed. Or it produces text that matches one of your stop sequences. Max tokens is often confused with the context window. The context window is the total number of tokens the model can handle at once, input plus output. Max tokens only limits the output of a single call. A model with a very large context window can still produce short responses if max tokens is set low, and input plus max tokens generally has to fit inside the context window. APIs report why generation ended. OpenAI's Chat Completions API returns a finish_reason such as 'stop' for a natural end or a matched stop sequence, 'length' when the output limit was hit, and 'tool_calls' when the model wants to call a tool. Anthropic's Messages API returns a stop_reason such as 'end_turn', 'max_tokens', 'stop_sequence' or 'tool_use'. Anthropic requires max_tokens on every request. For its reasoning models, OpenAI introduced max_completion_tokens, which counts hidden reasoning tokens as well as visible output. Stop sequences are useful for structured formats: ending after a closing tag, stopping before a speaker label, or cutting off after one list item. The matched text is normally not included in the returned output, so code should not expect to see it. The most common mistake is ignoring the finish reason. A truncated response can look plausible, especially prose, while structured output simply breaks. Another misconception is that a low max tokens makes the model write concisely. It does not; the model writes as it normally would and gets cut off. Ask for brevity in the prompt and use the limit as a safety cap.

Doxal ci àdduna dëgg

A summarizer sets max tokens to 150 and some summaries end mid-sentence; checking the finish reason shows 'length' for those calls, so the app raises the limit and asks for shorter summaries in the prompt instead.

A script that extracts data as JSON fails to parse about one response in fifty; each failure had hit the output limit, so the code now retries with a higher limit when the finish reason says the output was truncated.

A developer generating one line of dialogue at a time uses the stop sequence 'User:' so the model does not continue and write the user's next line too.

A team moving to a reasoning model finds answers coming back empty because hidden reasoning used up the output limit, so they raise the limit to leave room for both reasoning and the answer.

The Future of Max Tokens and Stop Sequences

Output limits have grown as models support longer generations, and structured output features that constrain responses to a schema reduce, but do not remove, the risk of truncation, since a long enough response can still hit the cap. Reasoning models have made output budgeting more important because hidden tokens now share the same limit. Parameter names and finish reason values differ between providers and have changed over time, so it is sensible to check current documentation and wrap these details in a small layer of your own code.

Teg ko ci jëf

Lu tax mu am solo

  • Njëgg ak budget. Dogal yi architecture di jël dañuy indi njariñ ak njëgu liggéey bi ay at ci ginaaw.
  • dogal yu gëna leer. Njàngalem xarala yi dafay jàppale ekip yi ñu tànn li gën, te baña yam ci li gëna bees daal.
  • Xool kalite. Tanneef yu gëna baax ci wàllu ingeñër dina wàññi jafe-jafe yi ci wàllu wóor ci liggéey bi.

Moytul

  • Optimize benn benchmark mën na nëbb ñakk kattan yu gëna yaatu ci sistem bi.
  • Njëg li ñuy fay ci infrastructure yi ak ci toppatoo dañuy faral di suufeel.
  • Bu sistem yi di gëna xawa jafee xam, jafe-jafe yi am ci wàllu kaaraange ak seetlu mën nañu gëna bari.

Jéego yi ci topp

  1. Mandargal latency, kalite, ak njëg yi laata ngay jëfandikoo.
  2. Benchmark ci biir sargal ak done yu dëggu.
  3. Jumtukaay bi di saytu njuumte yi, derive bi ak njeextalu jëfandikukat bi.
  4. Waajal rollback ak yooni tontu ci jafe-jafe yi laata ngay eskale.

Weyal di banneexu

Bulletin bu amul fayda

Jàngal xibaar AI bu bes

Ñatti jaar-jaari IA yuñ firndeel bis bu nekk ci ayu-bis bi, ñu bind leen ci Àngle bu leer. Gratis ba fàww, amul yëgle.

Benn email bis bu nekk ci ayu-bis bi. Benn klik nga dindi abonemaa bi. Dunu musa jaay wala séddoo sa adres.

Nattal sa bopp

Take the Max Tokens and Stop Sequences quiz

Feedback ci saasi ci bépp tontu, ak sertifikaa buñ mëna séddoo ak ID buñ mëna saytu soo jàllee ci njàng mi.

Tambalil quiz

Jàppale njàngum IA te doo fay. AI Understanding mbootaay la buy def te Yàlla tax — amul yëgle, amul paywall, musul am. Defal ab sarax

Laaj yi ñuy faral di laaj

What is Max Tokens and Stop Sequences?

Max tokens is the cap on how many tokens a model may generate in one response, and stop sequences are strings that end generation as soon as the model produces them. Together with the finish reason the API returns, they decide where an answer ends and tell you whether it ended naturally or was cut off. Getting them right prevents truncated JSON, half-finished answers and wasted spend.

What does max tokens limit?

Max tokens caps output for one call. The context window is the separate total limit covering input and output.

Your call returns a finish reason of 'length' in OpenAI's API. What happened?

'length' means generation stopped because the output limit was reached, so the response may be incomplete.

Which Anthropic stop_reason indicates the model decided its answer was complete?

'end_turn' means the model ended naturally. 'max_tokens' means it was cut off, and 'stop_sequence' means one of your strings matched.

Why does setting a low max tokens value not make answers concise?

The limit is a hard stop, not an instruction. Ask for brevity in the prompt and keep the limit as a safety cap.

Is the matched stop sequence normally included in the returned text?

Stop sequences end generation and are typically not included in the returned output, so your code should not rely on seeing them.