Token

A token is the unit a language model actually reads and writes — roughly a short word or a fragment of a longer one. English text runs about three tokens for every four words, so a page of prose is somewhere near 700 tokens.

Tokens matter for two commercial reasons. Model providers bill per token in and per token out, so the cost of an AI feature is a direct function of how much text it moves. And every model has a context window measured in tokens, which caps how much it can consider at once.

This is why "just send it the whole knowledge base" is not a plan. Whittling the input down to the part that matters is both the cost control and the quality control.