NLP/ML
Tokenisation
Breaking text into smaller units (tokens) before feeding to an LLM. Pricing for AI APIs (OpenAI, Gemini) is based on token count. Indian languages often require more tokens than English.
Use this term as a quick reference, then jump back into the part of the site where it matters.