Building With LLMs
Intermediate · 4 concepts · 4 clips · 5 source videos. Watch the clips in order, then take the quick check to see what did not land.
The context window
Everything the model knows at answer time is in the window. Its size is not the constraint people think it is; attention across it is.
NetworkChuck on why attention degrades across a long window, and how to work within it (8:13-15:18)
Did this clip teach it?
The ideas in these clips, on Atlas
- The context window is not the constraint people think it is; attention across it isA long window degrades before it fills; curate and order what goes in rather than relying on size.
- A model never sees words, only tokens, which explains most of its odd behaviourReason about a model in tokens: that is what it reads, what fills its window and what you pay for.