Skip to content
Agent Month
Email

Glossary

AI engineering glossary

Plain-English definitions of the terms behind production AI — no fluff, written for engineers and the leaders who fund them.

Agent observability
Agent observability is the tooling that makes an agent’s behavior in production visible — tracing tool calls, prompts, costs, and failures.
Agentic coding
Agentic coding is delegating a whole software task to an AI agent that plans, edits files, runs tests, and iterates in a loop until the task is done.
Agentic development
Agentic development is the practice of delegating software tasks to AI agents that plan, edit, and verify code across a codebase, with human oversight.
AI agent
An AI agent is a system where a language model decides and takes actions through tools in a loop to accomplish a goal, rather than producing a single response.
AI code security
AI code security is the practice of controlling the risks of AI-generated code and prompts — vulnerabilities, license issues, data leakage, and prompt injection.
Batch inference (Batch API)
Batch inference submits many LLM requests as one asynchronous job in exchange for a large discount, at the cost of waiting hours instead of seconds.
Codebase readiness
Codebase readiness is how well a codebase supports AI/agentic development — measured by module boundaries, tests, types, docs, and context files.
Context engineering
Context engineering is deciding what information goes into a model’s context window, in what order, and what gets left out — the discipline that replaced prompt tweaking.
Context files (CLAUDE.md, AGENTS.md)
A context file is a checked-in document that tells every AI agent working in your repo how the codebase works and what your conventions are.
Context window
A context window is the maximum amount of text (measured in tokens) a model can consider in a single request, including both input and output.
Embeddings
Embeddings are numeric vector representations of text (or other data) that place similar meanings close together, enabling semantic search and RAG.
Fine-tuning
Fine-tuning further trains a base model on your data to adapt its behavior, format, or style for a specific task.
Function calling (tool use)
Function calling is a model capability that lets it request a structured tool call, which your code executes and returns results for.
Golden path
A golden path is the supported, paved way to do a common task — the option that is easiest precisely because it is the one you want people to take.
Guardrails
Guardrails are the automated checks around an AI system that constrain what it can output or do, independently of what the model decides.
Hallucination
A hallucination is when a model produces confident, plausible-sounding output that is factually wrong or unsupported.
LLM cost optimization
LLM cost optimization is the practice of reducing what production AI features cost — through routing, caching, batching, and right-sizing models — without losing quality.
LLM evals
LLM evals are systematic tests that measure the quality of a model’s outputs against defined criteria, so changes can be validated instead of guessed.
LLM gateway
An LLM gateway is a layer that sits between your application and model providers to handle routing, caching, fallbacks, observability, and cost control.
LLM inference cost
LLM inference cost is what it costs to actually run a model in production — priced per token on hosted APIs, or per GPU-hour when you host it yourself.
LLM-as-judge
LLM-as-judge is an evaluation technique where a language model scores another model’s output against criteria you define.
MCP server
An MCP server is a small service that exposes one system’s capabilities — a database, an API, an internal tool — as typed tools any MCP-capable AI client can call.
Model Context Protocol (MCP)
MCP is an open protocol that standardizes how applications expose tools, data, and prompts to AI models and agents.
Model distillation
Distillation trains a small, cheap model to imitate a large one on a narrow task, so you keep most of the quality at a fraction of the cost.
Model routing
Model routing sends each request to the most appropriate model — by cost, quality, or latency — instead of using one model for everything.
Prompt caching
Prompt caching reuses the model’s processing of a repeated prompt prefix, cutting cost and latency on requests that share a large, stable preamble.
Prompt compression
Prompt compression cuts the tokens sent on every request — trimming system prompts, examples, and retrieved context — without changing the output.
Prompt engineering
Prompt engineering is the practice of designing the instructions and context given to a language model to get reliable, high-quality outputs.
Prompt injection
Prompt injection is an attack where content the model reads contains instructions that hijack its behaviour — the defining security risk of tool-using agents.
Quantization
Quantization shrinks a model by storing its weights at lower numeric precision, cutting memory and cost with a small, measurable quality trade-off.
Regression testing (for LLMs)
LLM regression testing re-runs a suite of evals on every change so a prompt or model update can’t silently degrade quality.
Retrieval-augmented generation (RAG)
RAG is a technique that retrieves relevant documents at query time and adds them to the prompt so the model answers from up-to-date, specific knowledge.
Self-hosted LLM
A self-hosted LLM is an open-weight model you run on your own infrastructure, so data never leaves your environment.
Semantic caching
Semantic caching reuses a previous LLM response when a new request means the same thing as an earlier one, even if the wording differs.
Semantic search
Semantic search retrieves results by meaning rather than exact keywords, using embeddings to match intent.
Structured outputs
Structured outputs constrain a model’s response to a defined schema (such as JSON), guaranteeing parseable, valid output.
Token budget
A token budget is a ceiling on the tokens an agent may spend on a task, so a long-running loop degrades gracefully instead of billing without limit.
Tokens
Tokens are the chunks of text a language model reads and writes; pricing and context limits are measured in them, not words or characters.
Vector database
A vector database stores embeddings and finds the nearest vectors to a query efficiently, powering semantic search and RAG.
Vibe coding
Vibe coding is building software by prompting an AI in natural language and accepting what it generates, without closely reviewing the underlying code.