An attention mechanism is a Machine Learning technique that directs models to prioritize (attend to) the most relevant parts of the input data.
Attention mechanisms compute an attention matrix that maps the relative importance of each part of the input sequence.
Powers the Transformer architecture.