Courses/Deep AI Engineering/Tokens, Attention and Training

Embeddings and Attention

After this lesson you can say what an embedding is, what attention computes at each token, why that makes context the whole game, and why long contexts cost what they cost.