Temperature (AI)
Temperature is a parameter in language model inference that controls the randomness of token selection — higher temperatures produce more varied and creative outputs, while lower temperatures produce more consistent and deterministic responses.
Understanding Temperature (AI)
When an LLM generates text, it calculates a probability distribution over possible next tokens. Temperature scales this distribution before sampling. At temperature 0, the model always picks the highest-probability token (greedy decoding), producing deterministic output. At higher temperatures (0.7-1.0), the model samples from a wider distribution, introducing variety and occasionally surprising, or incorrect, outputs.
Temperature is a dial between creativity and consistency. For tasks requiring factual accuracy, summarizing an email, extracting tasks from a meeting, low temperature (0.0-0.3) is preferred. For tasks requiring creativity, brainstorming, writing varied copy, generating names, higher temperature (0.7-1.0) produces more interesting results.
Related parameters include top-p (nucleus sampling), which limits sampling to the top probability mass, and top-k, which limits sampling to the top k tokens. Together, these parameters give developers fine-grained control over output characteristics.
Most AI assistant APIs expose temperature as a configurable parameter. GAIA keeps a low default temperature of 0.1 for almost everything, so triage, extraction and tool calls stay consistent.
How GAIA Uses Temperature (AI)
GAIA runs almost every task at a low default temperature of 0.1. Reading your inbox, extracting action items, reporting on your calendar and calling tools all need the same input to give the same result. Only a few tasks that benefit from variety use a higher value.
Related Concepts
Large Language Model (LLM)
A Large Language Model (LLM) is an artificial intelligence model trained on vast amounts of text data that can understand, generate, and reason about human language with remarkable fluency.
Prompt Engineering
Prompt engineering is the practice of designing and refining inputs to AI language models to reliably elicit desired outputs, shaping model behavior without modifying the underlying weights.
Inference
Inference is the process of running a trained AI model on new input data to generate predictions, responses, or decisions, as opposed to training, which is the process of building the model from data.
Hallucination
AI hallucination is the phenomenon where a language model generates confident-sounding but factually incorrect, fabricated, or nonsensical information that is not grounded in the input or training data.
Structured Output
Structured output is a technique that constrains an LLM to respond in a predefined format — typically JSON or XML — enabling reliable programmatic parsing of model responses rather than free-form text.
