162 articles
Browse guides
For LLMs, latency splits into time-to-first-token, which drives perceived speed, and total generation time. Streaming attacks the first; smaller models and shorter outputs attack the second. Reasoning models trade latency for accuracy on purpose. In practice: 400ms to first word feels instant; four seconds of blank screen feels dead.
Too high a learning rate and training overshoots and diverges; too low and it crawls or gets stuck. Most real runs schedule it, starting larger and decaying over time. It is the classic example of a hyperparameter: set by you, not learned by the model. In practice: Same data, same architecture, learning rate 10× too high — the loss goes to infinity.
LLMOps shifts the concerns because you usually are not training the model. What you version, test, and monitor is prompts, retrieval, tool definitions, evals, latency, spend, and output safety. The core discipline is the same: nothing ships without a test that would catch its failure. In practice: Every prompt change runs the eval suite in CI before it reaches production.
LoRA freezes the base weights and learns a small number of extra parameters alongside them. The result is a lightweight adapter file — megabytes, not gigabytes — that can be swapped per task and trained on modest hardware. It is why fine-tuning stopped being the exclusive privilege of large labs. In practice: One base model plus five LoRA adapters serving five brand voices.
A loss function turns the gap between prediction and truth into a single number. Training is nothing more than searching for parameters that make that number small. Choosing the loss is choosing what the model will care about, which makes it a quietly consequential decision. In practice: Predicted 0.9 where the answer was 1.0 — loss is small, so the correction is small.
In machine learning you show a system many examples and it works out the rule itself by adjusting internal numbers until its outputs match the examples. Formally ML is a subfield of AI, though in practice the terms are used loosely. The key shift is that the developer supplies data and an objective rather than logic. In practice: Instead of listing every word that signals fraud, you feed a model 100,000 labelled transactions and let it find the signals. ...
Meta-prompting hands the prompt-writing task to the model itself: ask it to draft a prompt, then to find the ambiguities in it. It works because models are often better at spotting underspecification than people are at avoiding it. Pair it with real evaluation or you are just generating confident-sounding prompts. In practice: ‘Here is my prompt and three bad outputs. What is ambiguous about it?’
A mixture-of-experts model contains many specialised sub-networks and a router that picks a few per token. Total parameter count stays huge while compute per token stays modest, which decouples capacity from inference cost. It is why some very large models are surprisingly cheap to serve. In practice: A 400B-parameter model that only runs 40B worth of compute per token.
MLOps applies DevOps thinking to ML, with extra problems: data versioning, training reproducibility, drift monitoring, and retraining pipelines. The insight it encodes is that a model in a notebook is roughly 10% of the work. In practice: Automatic retraining triggered when monitored accuracy drops below a threshold.
A model is what you get at the end of training: a specific set of numbers arranged in a specific structure. It is the thing you deploy, version, and call from an API. Two models with the same architecture but different training runs are different models, which is why version numbers matter. In practice: GPT-5.6 and Claude Opus 4.8 are different models; each has its own weights, behaviour, and price. ...