Glossary

Tokens

Tokens are the basic units of input and output in a language model - meaning human language transformed into sections, that are understandable by AI models