Attention Mechanism
The operation that lets a model weigh which parts of the input matter most for each part of the output.
Attention computes, for every token, how much every other token should influence it. Self-attention does this within one sequence and is the core of the transformer. Its cost grows quadratically with sequence length, which is exactly why long context windows are expensive and why so much research targets this bottleneck.
In practice: Reading a contract clause, the model attends heavily to the defined term three pages earlier.