🔬 AI Foundations

What AI Actually Is

Artificial intelligence, in 2026, means large language models and related systems that process and generate text, images, code, and more. The core idea: train a neural network on massive data, and it learns to predict useful outputs. No magic, no consciousness — just patterns at scale.

• Modern AI = large language models trained on massive datasets
• The core mechanism is predicting the next token in a sequence
• No consciousness, no understanding in the human sense — just patterns

Did you know? GPT-4, the model behind ChatGPT, has an estimated 1.8 trillion parameters — numbers that encode the patterns it learned from the internet.

---

How LLMs Work

Large Language Models (LLMs) use an architecture called the Transformer. It processes all the words in your message at once, weighs how much each word relates to every other word, and uses that to predict the best next word in a reply. Repeat millions of times — that is a conversation.

• Transformer architecture processes whole sequences at once
• Attention mechanisms weigh word relationships across the text
• Training on trillions of tokens builds general language ability

Did you know? The Transformer architecture was invented by Google researchers in 2017. Their paper was titled 'Attention Is All You Need' — one of the most cited AI papers ever.

---

Training vs Inference

Training is when a model learns — processing vast data over weeks or months using thousands of GPUs. Inference is when a trained model answers your question — fast, cheap, and running in real time. You only ever experience inference. Training happens once, offline, by the AI company.

• Training: learning from data — slow, expensive, done once by the company
• Inference: using the trained model to answer questions — fast and cheap
• You only ever interact with the inference side of AI

Did you know? Training GPT-4 is estimated to have cost over $100 million. Running one conversation costs a fraction of a cent.

---

Context Windows

A context window is the maximum amount of text an AI can consider at once — its working memory. Models in 2026 have windows of 128K to 1M tokens (roughly 100K to 800K words). Longer contexts mean better multi-step reasoning and document analysis.

• Context window = how much the model can read at once
• Measured in tokens (about ¾ of an English word each)
• Larger context = can read whole books, codebases, or reports

Did you know? In 2023 context windows were 4K tokens (about 3,000 words). By 2026 some models reached 1 million tokens — a 250× jump in two years.

---

The Model Landscape

A handful of companies dominate the frontier: OpenAI (ChatGPT/GPT-4), Anthropic (Claude), Google (Gemini), Meta (Llama — open source), Mistral (open source, European). Each has a different philosophy: commercial, safety-focused, open, or efficient. Understanding the landscape helps you choose the right tool.

• OpenAI, Anthropic, Google, Meta, and Mistral lead the field
• Each model family has different strengths and pricing
• Open-source models (Llama, Mistral) can be self-hosted for free

Did you know? Meta released Llama 3 openly in 2024, and it quickly became the most downloaded AI model ever — used in millions of apps worldwide.

---

Multimodal AI

Modern AI handles not just text but also images, audio, code, and even video. You can upload a photo and ask questions about it, speak to your assistant and get spoken replies, or have AI analyse a chart. Multimodal means AI that works across all these modes at once.

• Multimodal AI handles text, images, audio, video, and code
• You can upload a photo and ask AI what it sees
• Models like GPT-4o and Claude 3.5 are strongly multimodal

Did you know? GPT-4o ('o' for omni) can see, hear, and respond in real time — making live spoken conversations with AI possible for the first time.