33 articles
Browse guides
Learning rate, batch size, number of layers, dropout rate — these are hyperparameters. They are picked by the developer and tuned against validation data. The distinction from parameters is simple: you set hyperparameters, training sets parameters. In practice: Running the same job 20 times with different learning rates to find the one that converges best.
Inference is the production phase: the weights are frozen and the model simply maps input to output. It is where nearly all ongoing cost lives, because training happens once but inference happens on every request. Latency, price per token, and throughput are all inference concerns. In practice: Every ChatGPT message you send is an inference call; the training run finished months earlier.
A label is the target a supervised model is trying to reproduce — the category, the number, the right output. Label quality caps model quality: if annotators disagree or are careless, no amount of compute fixes it. In practice: In a photo dataset, ‘cat’ is the label; the pixels are the features.
Too high a learning rate and training overshoots and diverges; too low and it crawls or gets stuck. Most real runs schedule it, starting larger and decaying over time. It is the classic example of a hyperparameter: set by you, not learned by the model. In practice: Same data, same architecture, learning rate 10× too high — the loss goes to infinity.
A loss function turns the gap between prediction and truth into a single number. Training is nothing more than searching for parameters that make that number small. Choosing the loss is choosing what the model will care about, which makes it a quietly consequential decision. In practice: Predicted 0.9 where the answer was 1.0 — loss is small, so the correction is small.
In machine learning you show a system many examples and it works out the rule itself by adjusting internal numbers until its outputs match the examples. Formally ML is a subfield of AI, though in practice the terms are used loosely. The key shift is that the developer supplies data and an objective rather than logic. In practice: Instead of listing every word that signals fraud, you feed a model 100,000 labelled transactions and let it find the signals. ...
A model is what you get at the end of training: a specific set of numbers arranged in a specific structure. It is the thing you deploy, version, and call from an API. Two models with the same architecture but different training runs are different models, which is why version numbers matter. In practice: GPT-5.6 and Claude Opus 4.8 are different models; each has its own weights, behaviour, and price. ...
Narrow AI performs well inside its intended scope and does not transfer outside it. The label is a contrast with AGI rather than a slight — narrow systems are the ones creating real value today. A general-purpose model can look broad and still be narrow in the sense that matters: it does not act, learn, or adapt beyond its design. In practice: A model that writes excellent Python cannot drive your car, however fluent it sounds. ...
A neural network takes an input as numbers, multiplies them by learned weights, applies a non-linear function, and passes the result to the next layer. Repeat this enough times with enough data and the network can approximate remarkably complex relationships. The ’neurons’ analogy is a historical label, not a claim about biology. In practice: An image classifier is a neural network that turns 200,000 pixel values into one number per possible label. ...
An overfitted model has learned the noise as well as the signal. It scores brilliantly on data it has seen and poorly on data it has not. The usual fixes are more data, less model capacity, regularisation, and stopping training early when validation loss turns upward. In practice: 99% on training data, 62% in production — a textbook case.