ChatGPT & LLM

A chatbot such as ChatGPT is an application around models and other systems; the model and the complete product are not the same thing.

Key takeaways

• Separate the chatbot product from the model it uses.
• Next-token generation and fact verification are different processes.
• Check the evidence behind important claims, including apparently convincing citations.

Text is converted into tokens, which can represent words, parts of words, or other units. An autoregressive language model uses the current context to produce scores for possible next tokens. Generation selects a token and continues from the expanded context. The result can be useful prose, code, or structured text, but this process does not automatically verify facts. Many modern LLMs use transformer architectures. Attention lets a model combine information from different positions in a sequence. The original transformer paper is a useful source for that architecture, but it does not establish every detail of a particular commercial chatbot's implementation. Training, prompting, retrieval, and tools are different mechanisms. Training changes parameters. A prompt supplies the current task and context. Retrieval supplies selected documents or passages. Tools can carry out actions such as searching or calculating. A product may combine these mechanisms, so an answer's quality depends on more than the base model. Fluency is not a truth signal. A model can invent a citation, blend incompatible facts, or answer beyond the supplied evidence. For important factual work, identify the supporting passage, open the source, and check that it actually supports the claim. Treat a model's statement about its own confidence as an output to evaluate, not as independent evidence.

1. Supply this invented note: 'The workshop starts at 10:00. Registration closes Friday.'
2. Ask: 'What time does the workshop end? Answer only from the note. If it is not stated, say that it is not stated.'
3. The expected answer is that the ending time is not stated. An invented finishing time is a failure even if it sounds plausible.

This is a small evaluation case you can reuse. The expected answer is a test criterion, not a claim that every model will pass it.