What is an LLM and how does it work?

An LLM, or large language model, is an AI system trained on huge amounts of text to predict the next piece of a sentence. Repeating that one prediction thousands of times lets it write emails, answer questions, summarize documents and generate code. ChatGPT, Claude and Gemini all run on LLMs. They read and write well. Their facts still need checking, because they predict likely text.

Business owners ask what is an LLM once AI starts showing up in their software, their vendor proposals and their invoices. Knowing how the model works explains three things a buyer needs: what it does well, why it sometimes states wrong facts with confidence, and why it is billed by the token.

On this page

• What is an LLM in plain terms
• How does an LLM work, step by step
• How are LLMs trained
• What large means: parameters, context window and cutoff
• What an LLM does well and where it fails
• LLM vs chatbot vs AI agent
• How LLM pricing works in 2026
• Open-weight vs closed models
• How to choose an LLM for your business

What is an LLM in plain terms

Stanford HAI defines an LLM as "an AI system trained on massive amounts of text data to understand and generate human-like language." The simplest mental model is the autocomplete on your phone, scaled up by many orders of magnitude and trained on a large share of the public internet, books and code.

Phone autocomplete guesses one word from the last two or three. An LLM weighs the whole conversation, sometimes hundreds of pages, before each guess. That is why it can follow instructions, keep a consistent tone across a long proposal and answer a follow-up question about something said 20 messages earlier.

For example, say a 40-person HVAC contractor in Ohio pastes a 6-page service agreement into Claude and asks for the cancellation terms in plain English. The model reads all 6 pages as one input, finds the clause and rewrites it in three sentences. It has never seen that contract before. It works because it learned during training how contracts are usually structured and worded.

The "large" in the name refers to two things: the number of internal settings the model adjusts during training, called parameters, and the volume of text it learns from. GPT-3, released in 2020, had 175 billion parameters. Current frontier models are larger, though most vendors no longer publish exact counts.

How does an LLM work, step by step

Every reply from ChatGPT, Claude or Gemini goes through the same five steps:

1. Your text is split into tokens, small chunks of words. OpenAI's rule of thumb is that one token is about 4 characters or three quarters of an English word, so 100 tokens is about 75 words.
2. Each token becomes an embedding, a long list of numbers that places it near tokens with related meaning.
3. The transformer layers compare every token with every other token to work out which words matter to which. This step is called attention.
4. The model outputs a probability for every possible next token, for example 62 percent for "invoice," 9 percent for "bill" and so on down a list of tens of thousands.
5. The app picks one token, adds it to the text and runs the whole process again, one token at a time, until the answer is finished.

Attention is the breakthrough that made modern LLMs possible. The 2017 paper Attention Is All You Need by Ashish Vaswani and colleagues at Google introduced the transformer, a model "based solely on attention mechanisms." It set a new state of the art in English-to-French translation after 3.5 days of training on eight GPUs, and it trained faster than older designs because the work runs in parallel.

Step 5 explains two everyday quirks. Long answers take longer because each token is a separate pass. And the same question can get different answers, because the app samples from the probabilities instead of always taking the top choice. A setting called temperature controls how much variety it allows.

How are LLMs trained

Training runs in stages, from raw text prediction to a helpful assistant.

Pretraining comes first. It feeds the model trillions of tokens of text and adjusts its parameters every time a next-token guess is wrong. This stage costs the most by far and gives the model its general knowledge of language, facts and code. It ends at a training cutoff, which is why a model can be unaware of last month's news.

Size alone does not decide quality. DeepMind's Chinchilla study, led by Jordan Hoffmann, found in 2022 that "the model size and the number of training tokens should be scaled equally." Its 70-billion-parameter Chinchilla, trained on about four times as much data, outperformed the 280-billion Gopher and the 175-billion GPT-3 on a broad set of tasks.

Fine-tuning turns a text predictor into an assistant. In OpenAI's InstructGPT paper, human trainers wrote example answers, then ranked model outputs, and those rankings guided further training through reinforcement learning from human feedback. Evaluators preferred answers from the 1.3-billion-parameter InstructGPT over the 175-billion GPT-3, a model about 100 times larger. Newer reasoning models add training that rewards working through a problem step by step before answering.

What large means: parameters, context window and cutoff

Vendor spec sheets use a handful of terms that decide what a model can do for you. The table translates them.

The context and cutoff figures come from Anthropic's model overview as of October 2026. A 1-million-token window holds a full employee handbook, a year of meeting notes or a mid-size codebase in a single request. The cutoff matters more than most buyers expect: anything newer, such as this year's price list, has to be supplied in the prompt or fetched by a tool.

What an LLM does well and where it fails

LLMs are strong at language tasks where a person can check the output quickly:

• Drafting emails, proposals, product descriptions and job posts from rough notes.
• Summarizing long documents, call transcripts and ticket threads.
• Sorting and tagging inbound leads, reviews or support messages.
• Pulling fields such as dates, amounts and names out of invoices and forms.
• Writing and explaining code, the basis of vibe coding tools.
• Translating between languages while keeping a set tone.

They are weak where accuracy depends on facts the model does not hold. Without help, an LLM cannot know your current prices, your inventory or anything after its cutoff. It can make arithmetic slips on long calculations, and it can state an invented fact in the same confident tone as a true one, because it predicts what text usually looks like and has no built-in step that verifies it.

The fixes are design choices. Retrieval-augmented generation feeds the model your own documents before it answers. Tool calls let it run a calculator or query your database. A human review step catches the rest. IBM's LLM explainer lists hallucinations, bias and energy use as the main limits to manage.

LLM vs chatbot vs AI agent

These terms get used interchangeably in sales decks, and they describe different layers of the same system.

The LLM is the engine, and everything else is the vehicle built around it. The guides on how AI agents differ from chatbots and the Model Context Protocol cover the outer layers in depth.

How LLM pricing works in 2026

API providers bill per million tokens, with separate rates for the text you send (input) and the text the model writes (output). As of October 2026, Anthropic's pricing lists Claude Haiku 5.5 at $0.10 input and $0.50 output for prompts up to 100,000 tokens, Claude Sonnet 5.5 at $2 and $10, and Claude Opus 5.5 at $4 and $20. Batch jobs that can wait get 50 percent off.

As a worked example, a hypothetical 10-person property management firm in Florida summarizes 2,000 tenant emails a month. Each request sends about 850 tokens (a 400-word email plus instructions) and gets back an 80-token summary.

For text tasks like this, model fees are small enough that quality should drive the choice. Prices keep falling too. The 2025 AI Index from Stanford HAI found the cost of a model performing at GPT-3.5 level dropped from $20 per million tokens in November 2022 to $0.07 by October 2024, a more than 280-fold reduction in about 18 months.

Open-weight vs closed models

Closed models such as Claude, GPT and Gemini run only on the vendor's servers and are rented per token. Open-weight models publish their trained parameters, so a company can run them on its own hardware or cloud account.

The quality gap has narrowed. The same AI Index reported that the leading closed model beat the top open-weight model by 8.0 percent on the Chatbot Arena leaderboard in January 2024, and by only 1.7 percent in February 2025.

The trade is control against effort. Running your own model keeps data inside your infrastructure and avoids per-token fees, but you pay for GPUs, setup and maintenance. For most companies under a few hundred employees, a closed model through an API with a business data agreement is cheaper and faster to launch. Self-hosting makes sense when contracts or regulations say data cannot leave your servers.

How to choose an LLM for your business

Pick the model by the job it has to do. The comparison of ChatGPT, Claude and Gemini for business covers the main products, and these steps apply to any of them:

1. Write down the task, the input it reads and what a correct output looks like.
2. Collect 30 real examples with answers your team agrees are right.
3. Run the examples on a small, a mid-size and a large model from one or two vendors.
4. Score each output for accuracy and tone, and note the cost per 1,000 runs.
5. Pick the cheapest model that meets your accuracy bar, and keep the test set to re-check when vendors ship new versions.
6. Confirm the data terms: whether prompts are stored, for how long, and whether they are used for training. The guide on putting company data into ChatGPT lists what to check.

Start this week by writing down one task and collecting your 30 examples, because that test set answers most model questions on its own. The AI development team at Techno Mantra builds LLM features into websites, apps and CRMs on OpenAI, Claude or Gemini, and runs this kind of model test before the build starts. You own all code and accounts. Send your task through the contact page or to info@technomantraa.com, and every inquiry gets a reply within one business day.