When I first tried to understand large language models, I did not want hype. I did not want metaphors about artificial brains or claims about consciousness. I wanted a clean mental model.

I had watched a few explainer videos and understood fragments. But I could not yet explain the system in plain language without slipping into buzzwords. That was the signal I did not really understand it.What helped was breaking the field down into a handful of core concepts and asking simple, direct questions about each one.

This piece walks through those concepts in order. We will cover tokenization, embeddings, the attention mechanism, pre-training, fine-tuning, hallucination, reasoning models, multimodality, and AI agents.

If you can explain these clearly, you can follow almost any AI discussion without feeling lost. This is the foundation that made everything else click for me.

Tokenization: If LLMs are so advanced, what are they actually doing?

At their core, large language models predict the next token. Not the next idea. Not the next paragraph. The next piece of text.

When you type a prompt, the model looks at everything in its current context and calculates which token is statistically most likely to come next. It selects one, appends it to the sequence, and repeats the process. Over and over. That is how entire paragraphs are generated.

A token is not always a full word. It can be a word, part of a word, or even punctuation. For example, the word “unbelievable” might be broken into smaller chunks like “un,” “believ,” and “able.” This is not random. Prefixes and suffixes carry meaning. If the model learns what “un” does, it can apply that pattern across thousands of words without memorizing each one separately.

That was the first grounding realization for me. The model is not thinking in sentences. It is predicting structured chunks of text, one step at a time.

Everything else builds on that.

Embeddings: How does a model “understand” a word if it only sees numbers?

Computers operate on numbers. So every token must be translated into numbers before the model can process it. This translation produces a vector, which is simply a long list of numerical values. That vector is called an embedding.

You can imagine each word being placed at a location in a massive multidimensional space. Words that appear in similar contexts end up close together. Words that rarely share contexts drift far apart. If “dog” and “puppy” frequently appear in similar sentences, their embeddings move closer during training. “Cat” and “kitten” form their own nearby cluster. “Car” sits somewhere else entirely.

The model does not store dictionary definitions. It stores geometry.

One well-known illustration of this geometric structure is the relationship: King minus Man plus Woman equals something very close to Queen. This works because relationships between concepts are encoded as directions in vector space. Move in one direction, and you add a concept. Move in another, and you subtract it.

Meaning inside an LLM is not symbolic. It is spatial.

Attention Mechanism: What happens when a word has multiple meanings?

Consider the word “bank.” In one sentence it refers to a financial institution. In another, it refers to the side of a river. If every word had a single fixed embedding, the model would constantly confuse these meanings.

This is where the attention mechanism becomes central.

When processing a sentence, every token can evaluate every other token. The model calculates how much weight to assign to each surrounding word when determining meaning. In “I deposited cash at the bank,” the words “deposited” and “cash” strongly influence the interpretation. In “I sat by the river bank,” the word “river” carries more weight.

The embedding for “bank” is not static. It shifts dynamically based on context.

This is not human-like understanding. It is weighted pattern alignment across vectors. But it is powerful. Context is not an add-on. It is embedded into the core architecture.

Pre-Training: How does the model learn language in the first place?

Pre-training is the large-scale learning phase. The model is exposed to enormous volumes of raw, unstructured text. Books, articles, code, forum posts, transcripts. The data is not neatly labeled into questions and answers. It is messy. Real-world messy.

The training objective is simple. Predict the next token.

The model reads a sequence of text and tries to guess what comes next. If it guesses incorrectly, its internal parameters adjust slightly. Over billions or trillions of repetitions, it becomes very good at modeling the statistical structure of language.

But at this stage, it is still just a completion machine.

If you asked a purely pre-trained model “What is the capital of France?” it might continue a quiz format instead of answering directly. It learned patterns from web pages and textbooks, not conversational etiquette.

Pre-training builds fluency. Not helpfulness.

Fine-Tuning: What changes between a raw model and a helpful assistant?

Fine-tuning shapes behavior.

Instead of feeding the model random internet text, engineers provide structured prompt and response pairs. For example, a question followed by a clean, direct answer. The model learns that when it sees a question, the appropriate continuation is a response that satisfies the question.

Reinforcement Learning from Human Feedback adds another layer. Humans evaluate multiple model responses and rank them. The model adjusts to prefer outputs that are clearer, safer, and more useful.

If a model responds to a recipe request with a disorganized rant, that is not a failure of embeddings. It is a failure of fine-tuning.

Pre-training teaches language patterns. Fine-tuning teaches behavior and alignment. That separation clarified a lot for me.

Hallucination: Why do models sometimes make things up?

Large language models are optimized to generate the most probable next token. They are not optimized to verify truth.

If the model lacks information about a topic, it still must produce a continuation. The most statistically likely continuation might be a plausible but incorrect statement. Saying “I don’t know” is not always the highest probability pattern, depending on the training data.

So the model generates something that sounds coherent. Even if it is wrong.

Hallucination is not intentional deception. It is a side effect of optimizing for fluent completion rather than factual verification.

Understanding that removes some of the mystique. It is a structural limitation.

Reasoning Models: If these are probability machines, how do they “reason”?

Reasoning models are fine-tuned on step-by-step problem-solving traces. Instead of jumping directly to a final answer, they generate intermediate steps.

Each step becomes part of the context window. The attention mechanism references earlier steps while generating later ones. In a math problem, predicting the final answer in one leap might be unlikely. But predicting the first logical step is often very likely. Then the second step becomes likely given the first.

The model builds a path through locally probable moves.

It feels like reasoning. But technically, it is structured token prediction guided by training on logical decompositions.

The trade-off is speed. Generating many intermediate tokens requires more computation. So reasoning models are powerful, but heavier.

Multimodality: Do visual models use the same architecture?

Yes. Modern visual models often use a variant of the Transformer architecture.

An image is split into small patches. Each patch is converted into a vector, much like a token in text. Positional embeddings encode where each patch belongs in the image so the spatial structure is preserved.

When text and images are trained into a shared embedding space, the vector for a photo of a dog sits near the vector for the word “dog.” This shared space enables systems to connect language and vision.

The architecture remains largely consistent. The input modality changes.

That consistency surprised me. The same mathematical framework scales across different types of data.

AI Agents: What makes an AI system more than just a chatbot?

A standalone LLM is limited to what it learned during training. It cannot access real-time information. It cannot check today’s weather unless that information is part of its training data.

An agent wraps the model in a larger system that includes planning, memory, and tool use. If asked for the current temperature in Tokyo, the model can determine it needs external data, call a weather API, receive structured information, and then translate it into natural language.

The intelligence is not only in the model. It is in the coordination between model and tools.

This is where many current developments are heading. Orchestration. Integration. Systems built around models.

How I Actually Learned This

This entire post came from a long back-and-forth conversation with Gemini 3.1 Pro. It was not a lecture or a polished tutorial. It was questions and answers. Sometimes the model would even pause and quiz me to check whether I truly understood a concept instead of just repeating it back.

In the age of LLMs, access to knowledge is no longer the bottleneck. You can ask for simpler explanations. You can request examples. You can say “go deeper” or “explain it without math.” You can admit you are confused and keep pushing until the mental model clicks. What matters now is curiosity and the willingness to ask questions, even the ones that feel basic.

Once these pieces fell into place, AI developments stopped feeling abstract. Bigger context windows, reasoning models, multimodality, agents. They all mapped back to the same structure. The mystique faded. What remained was a system built on tokens, vectors, attention, and feedback loops.