33 articles
Browse guides
An activation function decides what a unit passes forward. The non-linearity is the point: stack a hundred linear layers and you still have a linear model, but insert a non-linear function between them and the network can represent curves, interactions, and structure. ReLU is the common default. In practice: ReLU: negative input becomes zero, positive input passes through unchanged.
AGI describes AI that generalises across domains the way a person does, rather than excelling at a bounded set of tasks. It is not a technical specification: labs, researchers, and regulators use materially different definitions, which is why timeline claims are so hard to compare. Treat any confident AGI prediction as an opinion with a definition hidden inside it. In practice: Today’s systems can write and code well and still fail at tasks a child handles — breadth, not skill, is the gap. ...
Artificial intelligence is an umbrella term for systems that take in data and produce useful outputs without being told, step by step, how to do it. It covers everything from a spam filter to a model that writes code. Most AI you use today is machine learning, and most of that is deep learning — so the three words often get used interchangeably even though they are nested, not identical. ...
Backpropagation runs the error backwards through the network, layer by layer, assigning blame to every weight using the chain rule. It is what makes training deep networks computationally feasible, and it is the reason deep learning works at all. In practice: The output was wrong by 0.3 — backprop decides which of the billions of weights are responsible.
Batch size trades stability against memory and speed. Large batches give smoother gradient estimates but need more GPU memory; small batches are noisier but sometimes generalise better. It interacts with learning rate, so the two are usually tuned together. In practice: Batch size 32 means 32 examples in, one weight update out.
A classification model outputs a class rather than a number: spam or not, which of five languages, which disease. Its quality is measured with precision, recall, and related metrics — accuracy alone is misleading whenever one class is rare. In practice: Sorting support tickets into billing, technical, or account.
Deep learning stacks many layers of simple mathematical units so that each layer learns a slightly more abstract representation than the one below it. Depth is what lets a model go from pixels to edges to shapes to ‘cat’ without anyone defining what an edge is. Every large language model and image generator in use today is deep learning. In practice: A vision model’s first layers detect edges; its last layers detect faces — nobody programmed either step. ...
Training normally takes many epochs, since one look at the data is rarely enough. Too few and the model underfits; too many and it starts memorising. Very large models sometimes train for less than a single epoch simply because the dataset is enormous. In practice: Ten epochs over 50,000 images means the model saw each image ten times.
A feature is a single measurable property of an example — a price, a pixel, a word count. Classical ML depended on humans engineering good features; deep learning largely learns them from raw input instead, which is a big part of why it took over. In practice: For house-price prediction: square metres, postcode, year built — each is a feature.
Gradient descent computes which direction each parameter should move to reduce loss, then takes a small step that way. Repeat millions of times and the model converges. Nearly all modern training uses a variant of it, usually Adam. In practice: Like walking downhill in fog: you cannot see the valley, but you can feel the slope under your feet.