Courses/Deep AI Engineering/Inference and Quantization

Choosing the Next Token

After this lesson you can explain what the model actually outputs, how temperature and top-p turn it into one token, why temperature zero is not a guarantee, and what a stop token is.