Cracking the AI Jargon

Artificial Intelligence (AI) terminology can be daunting, especially for anyone trying to keep up with the rapid advancements we have been seeing.

AI, as a field, has been around for decades, with its roots tracing back to the mid-20th century. Early milestones include Alan Turing's work on machine intelligence and the development of the first AI programs in the 1950s. Although this may seem like a long time ago for humans to be working on AI, it was when mathematicians began to understand and recognise the possibilities that modern computer science would enable.

The recent surge in popularity can be attributed to significant breakthroughs in a particular branch known as Generative AI, driven by advances in algorithms, increased computational power, and the availability of large datasets. This subfield focuses on creating models that can generate new content, such as text, images, and even music, by learning patterns from existing data. Techniques such as Generative Adversarial Networks (GANs) and Transformer models like the GPT family most people are familiar with are key examples of Generative AI.

The leap in Generative AI has revolutionised various industries, making it a hot topic among professionals and enthusiasts alike. With numerous new terms and concepts emerging, staying informed can be challenging. To assist with this, here’s a detailed breakdown of essential AI terms to help you navigate this rapidly evolving landscape.

Some of these terms and concepts are significant and could warrant an entire blog post each. However, instead of diving deep into every single topic, I've summarised the minimal amount needed to help you understand the concept. This way, you can quickly grasp the basics and confidently hold a conversation on the subject.

The field doesn't sit still, either. Since I first wrote this, agentic AI, the Model Context Protocol, and reasoning models have gone from research curiosities to things you'll bump into in any AI conversation, so I've folded those in too, along with a few basic terms like prompt and context window that I'd used throughout without ever properly defining.

Disclaimer: As a software engineer, I find it much easier to communicate via code. To help keep all the definitions below on this complex topic as clear and simple as possible, I utilised numerous Generative AI models to help tune the copy for my definitions and examples. I also used several custom prompts and tools I have been working on to help check the accuracy and consistency of the information, ensuring I provide you with the most accurate explanations and examples I can.

Now that that's out of the way, let's continue.

Core AI Concepts

Artificial Intelligence (AI) 

AI refers to the simulation of human intelligence processes by machines, especially computer systems. These systems are designed to perform tasks that typically require human intelligence, such as visual perception, speech recognition, decision-making, language translation, and learning from experience. AI encompasses a variety of subfields, including Machine Learning, Neural Networks, and Natural Language Processing.

Artificial General Intelligence (AGI) 

AGI represents a form of AI that can understand, learn, and apply knowledge across a wide range of tasks at a human level of competence. Unlike current AI systems, which are designed to excel at specific tasks, AGI aims for a broader, more versatile intelligence capable of performing any intellectual task a human can. AGI systems would possess the ability to reason, solve problems, and adapt to new and unfamiliar situations with the same flexibility as humans.

Current generative AI models, such as ChatGPT, are not considered AGI. While generative AI like ChatGPT are highly sophisticated and capable within their specific domains, they do not possess the general intelligence required for AGI. Generative AI models may appear intelligent because they can understand and generate human-like text, but they operate based on pre-programmed patterns and data, making them deterministic. This means their responses are predetermined by their training data and algorithms, rather than the flexible and adaptive reasoning required in AGI.

AGI remains a theoretical goal in the field of artificial intelligence, representing a level of genuine machine intelligence that current AI technology has not yet achieved.

Generative AI 

Generative AI refers to technologies that create new content such as text, images, or music. Common examples include OpenAI's GPT models and Meta's Llama models, which produce original outputs by learning patterns from their training data. Generative AI uses different techniques, such as Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs) and Transformer-based models.

AI Model 

An AI model is a sophisticated computer program designed to learn from data and perform specific tasks by applying mathematical rules and algorithms. Think of an AI model as a recipe: just as a recipe provides instructions on how to combine ingredients to bake a cake, an AI model provides a set of instructions on how to process and interpret data to make decisions or predictions. The "ingredients" for an AI model can include numbers, words, images, or other types of data, while the "instructions" are the mathematical rules it follows.

AI models can be designed for a wide range of applications, from recognising faces in photos to predicting the weather. Each model is tailored to excel in particular tasks based on its training data and the techniques used in its development. For instance, even though models like OpenAI's GPT models and Meta's Llama both work with text, they differ in their architecture, the amount of data they have been trained on, and their specific applications. The versatility and effectiveness of AI models depend on their design and the quality of the data they are trained on.

Training 

Training is the process of teaching an AI model to perform a specific task by exposing it to a large amount of data. Similar to how a person learns a new skill through practice and repetition, an AI model improves its performance by analysing data and adjusting its internal parameters. During training, the model processes the data, identifies patterns, and learns from its mistakes to enhance its accuracy and effectiveness.

Consider training as analogous to learning a new sport. Initially, you might make numerous errors, but with continuous practice and feedback, your skills improve. Likewise, an AI model starts with basic capabilities and refines its performance through iterative training. The objective is to minimise errors and optimise the model’s ability to perform the designated task effectively. The quality of training significantly impacts the AI model's performance.

High-quality, diverse data and robust training techniques contribute to a more accurate and reliable model. The training process involves using algorithms and computational methods to adjust the model’s parameters, enabling it to make better predictions and decisions based on the learned patterns.

Fine-Tuning 

Fine-tuning is the process of taking an already-trained AI model, usually a foundation model, and continuing its training on a smaller, more specific dataset. Rather than starting from scratch, you're nudging an already-capable model to be better at a narrower task, like customer support for your product, medical terminology, or a particular writing style.

Fine-tuning sits in contrast to prompt engineering: prompt engineering changes what you ask the model, fine-tuning changes the model itself by adjusting its weights. It's more expensive and slower to iterate on than prompting, but it can produce more reliable results for a narrow, repeated task, and it means you don't have to re-explain the same instructions in every prompt.

Model Parameters 

Model parameters are the elements within an AI model that are adjusted during training through optimisation algorithms. They determine how the input data is transformed into the output, directly influencing the model's predictions and accuracy. In the context of Generative AI, parameters such as weights and biases in neural networks are adjusted to learn patterns from existing data, enabling the model to generate new content effectively.

Weights 

Weights are critical components within an AI model, particularly in neural networks, that influence how input data is transformed into output. They are adjustable parameters that determine the strength of the connections between neurons (or nodes) in different layers of the network. During the training process, weights are optimised using algorithms to minimise errors and improve the model’s predictions and accuracy.

In the context of Generative AI, weights play a crucial role in learning patterns from existing data. By adjusting the weights, the model learns the relationships and structures within the data, enabling it to generate new, coherent content. The values of the weights are continuously updated during training to better align the model’s predictions with the desired outcomes, ensuring the model can effectively perform its designated tasks.

Inference 

Inference is the process of using a trained AI model to make predictions or decisions based on new data. It represents the application phase of the model, where it provides outputs for given inputs, as opposed to the training phase where the model learns from data. In the context of Generative AI, inference involves generating new content, such as text, images, or music, based on learned patterns.

Temperature 

Temperature is a setting that controls how predictable or creative a model's output is during inference. At a low temperature, the model almost always picks the most statistically likely next token, producing consistent, safe, and sometimes repetitive answers. At a high temperature, it's more willing to pick less likely tokens, producing more varied and creative, but also less predictable and occasionally less coherent, output.

There's no universally "correct" temperature. A code generator or a legal contract reviewer usually wants a low temperature so answers stay accurate and repeatable. A tool for brainstorming product names or writing poetry benefits from a higher temperature, where a bit of unpredictability is the whole point.

Hallucinations 

In AI, hallucinations refer to instances where a model generates outputs that are not grounded in the input data or real-world information. This phenomenon can occur in various types of AI models, including generative models, where it produces results that seem plausible but are incorrect. For example, a text generation model might produce factual errors or nonsensical statements, while an image generation model might create unrealistic or distorted images. Hallucinations can be caused by issues such as insufficient training data, biases in the data, or limitations in the model architecture. Mitigation strategies include improving training data quality, fine-tuning model parameters, better and more specific prompting and incorporating additional validation mechanisms.

Bias 

AI bias occurs when an artificial intelligence system makes unfair or discriminatory decisions, often due to biased training data, algorithm issues, or human influence. For example, a facial recognition system trained mainly on photos of light-skinned individuals might perform poorly for those with darker skin tones. Bias can also stem from algorithmic design flaws that unintentionally introduce or amplify biases, and from the decisions and assumptions of the people who create and deploy AI systems. Even well-designed AI can exhibit bias if used in unintended ways or contexts.

Natural Language Processing (NLP) 

NLP enables machines to understand, interpret, and generate human language by using computational techniques. It involves tasks such as parsing, tokenisation, and semantic analysis to process and analyse text and speech. NLP is used in various applications, including chatbots, translation services, sentiment analysis, voice assistants, and text summarisation. Techniques commonly employed in NLP include machine learning, deep learning, and statistical methods to understand context, intent, and the nuances of human language. NLP plays a crucial role in enhancing human-computer interactions by making machines more adept at handling language-based tasks.

Generative Adversarial Networks (GANs) 

Generative Adversarial Networks (GANs) are a class of machine learning models used for generating new data samples by employing a two-part framework consisting of a generator and a discriminator. The generator creates synthetic data samples, such as images or text, with the goal of producing outputs that resemble real data. The discriminator, on the other hand, evaluates these samples to determine whether they are genuine or generated.

The training process involves an adversarial game between the generator and the discriminator. The generator aims to improve its ability to produce realistic data, while the discriminator strives to become better at distinguishing real data from generated samples. This competition drives both components to enhance their performance: the generator learns to create increasingly convincing samples, and the discriminator becomes more adept at detecting fakes.

GANs are widely used in various applications, including image synthesis, style transfer, and data augmentation, due to their ability to generate high-quality, diverse outputs that can closely mimic real-world data.

Variational Autoencoders (VAEs) 

Variational Autoencoders (VAEs) are a type of machine learning model that can create new data samples similar to the ones they were trained on. They consist of two main parts: an encoder and a decoder.

The encoder takes input data and compresses it into a simpler, smaller form called latent space. This compressed form captures the essential features of the data and is represented using statistical parameters like averages and variances.

The decoder takes this compressed form and turns it back into data that looks like the original input. This process allows the model to recreate and generate new examples that are similar to the training data. Once trained, VAEs can be used to generate new data. By sampling from the latent space, the decoder can create new data examples that are similar to those in the training set but are entirely new. VAEs are useful for tasks such as creating new images, cleaning up noisy data, and detecting unusual patterns due to their ability to generate diverse and high-quality samples.

Prompt 

A prompt is the instruction, question, or piece of text you give an AI model to get a response back. It can be a single sentence or several pages of background, examples, and constraints. Everything downstream, the tokens the model processes and the output it generates, starts with the prompt you write.

The quality of a prompt has a direct effect on the quality of the response. A vague prompt like "write about dogs" gives the model almost nothing to work with, so it fills the gaps with generic, average-of-the-internet content. A specific prompt, something like "write a 100 word explainer for new dog owners on why puppies need crate training, in a warm and encouraging tone", gives the model a clear target to aim for.

Prompt Engineering 

Prompt engineering is the practice of designing and refining prompts to get more accurate, useful, or consistent outputs from an AI model. It sits somewhere between writing and debugging: you're not changing the model itself, you're changing how you talk to it.

Common techniques include giving the model examples of the output you want (known as few-shot prompting), asking it to work through its reasoning step by step before answering (chain-of-thought prompting, closely related to how reasoning models work internally), and splitting a complex task into several smaller prompts rather than one giant one. Because prompt engineering doesn't require retraining or fine-tuning the underlying model, it's usually the fastest and cheapest way to improve results.

System Prompt 

A system prompt is a set of instructions given to an AI model before a conversation starts, defining its role, tone, and boundaries. Where a regular prompt is what the user types, the system prompt is set by the developer or product built on top of the model, and the user typically never sees it.

For example, a customer support chatbot might run on a system prompt along the lines of "You are a support agent for Acme Ltd. Only answer questions about Acme products. Be concise and polite. If you don't know the answer, direct the user to support@acme.com." That single instruction shapes every response the model gives for the rest of the conversation, without the user needing to repeat it.

Tokens 

Tokens are the fundamental units of text (words, subwords, or characters) that an NLP model processes.

Tokenisation is the process of breaking down text into these units, allowing the model to handle and analyse the text efficiently. For example, the sentence "AI is fascinating" can be tokenised into words: ["AI", "is", "fascinating"], or into subwords: ["A", "I", "is", "fas", "cin", "ating"].

Tokenisation is crucial for interpreting the input prompt provided to a model and generating the output response. However, not all models employ the same tokenisation techniques, making it challenging to compare models based on tokens alone. As tokenisation becomes the pseudo pricing standard for generative AI models, understanding these differences is vital when comparing models. Various models may have different tokenisation strategies, impacting both the efficiency and cost of using these models. For example, models with more granular tokenisation might handle rare words and languages better but may also require more tokens for the same text, affecting computational resources and cost.

Neural Network 

Neural networks are computational models inspired by the human brain, consisting of layers of interconnected neurons that process data and learn patterns. In the context of NLP, neural networks process sequences of tokens to understand and generate language. They capture hierarchical patterns in text through multiple layers, each extracting different levels of features from the input.

Types of neural networks commonly used in NLP include recurrent neural networks (RNNs), which handle sequential data, and transformers, which leverage self-attention mechanisms to capture long-range dependencies in text. These networks have revolutionised NLP by enabling advanced applications such as machine translation, text summarisation, and conversational agents, significantly improving the accuracy and fluency of generated language.

Neural Network Layers 

Layers of the neural networks are the building blocks of a neural network, each consisting of interconnected nodes (or neurons) that process and transform the input data. A neural network typically has three types of layers: input, hidden, and output.

• Input Layer: The first layer that receives the raw data. Each node in this layer represents a feature or variable from the input data.
• Hidden Layers: These intermediate layers process the input data through a series of transformations. The hidden layers, often more than one, perform complex computations and extract relevant features from the data. The term "hidden" refers to the fact that these layers are not directly exposed to the input or output; they function behind the scenes to uncover patterns and relationships in the data.
• Output Layer: The final layer that produces the output of the network. The number of nodes in the output layer corresponds to the number of desired outputs, such as class labels in classification tasks or continuous values in regression tasks.

Each layer’s nodes are connected by weights, which are adjusted during the training process to optimise the network's performance. The arrangement and number of layers in a neural network determine its depth and complexity, influencing the model's ability to learn and generalise from the data. In deep learning, networks with many hidden layers (deep neural networks) are used to capture intricate patterns and perform sophisticated tasks.

Transformer 

Transformers are advanced neural network architectures that process sequences of tokens in parallel, capturing long-range dependencies more effectively than previous models like recurrent neural networks (RNNs). They utilise self-attention mechanisms as layers to weigh the importance of different tokens, allowing them to understand context and relationships across an entire sequence simultaneously.

Transformers are the foundation of many state-of-the-art NLP models, including BERT (Bidirectional Encoder Representations from Transformers), GPT family of models (Generative Pre-trained Transformer), and T5 (Text-To-Text Transfer Transformer). These models have revolutionised NLP by enabling more accurate and fluent text generation, translation, summarisation, and other language tasks.

Embeddings 

An embedding is a way of representing a piece of text, image, or other data as a list of numbers (a vector) that captures its meaning. Words, sentences, or entire documents with similar meanings end up with similar embeddings, even if they don't share any of the same words. "Puppy" and "dog" end up close together in this numerical space; "puppy" and "spreadsheet" don't.

Embeddings are what let a model search by meaning rather than by keyword. This is the same underlying idea as latent space: both are ways of mapping the essence of data into a space where similar things sit close together. Embeddings are commonly generated by a dedicated embedding model and then stored for later comparison, most often in a vector database.

Vector Database 

A vector database is a database purpose-built for storing and searching embeddings rather than rows of structured data. Instead of looking for an exact match the way a traditional database does, a vector database finds the entries whose embeddings are mathematically closest to your query's embedding, which in practice means "find the documents that mean something similar to this."

Vector databases are the engine behind most real-world RAG systems: your documents get converted into embeddings and stored ahead of time, and when a user asks a question, their question is embedded too and matched against the stored vectors to find the most relevant material to feed back into the model.

Retrieval-Augmented Generation (RAG) 

RAG combines retrieval mechanisms with generative models, using retrieved documents to inform and enhance the generation of text. This approach improves the accuracy and relevance of generated responses by grounding them in real-world information.

In a RAG system, a retrieval component first searches a database or corpus to find relevant documents or pieces of information based on the input query. These retrieved documents are then fed into a generative model, which uses them to produce more accurate and contextually appropriate text. This method is particularly useful in applications like question answering, customer support, and content creation, where providing precise and relevant information is crucial. For example, in a customer support scenario, a RAG model could retrieve relevant sections from a product manual or FAQ database and use that information to generate a detailed and accurate response to a customer's query. By integrating retrieval with generation, RAG models can leverage vast amounts of existing knowledge, leading to more informed and reliable outputs.

Context 

In NLP, context refers to the surrounding text that informs the meaning of a word, phrase, or sentence. Understanding context is crucial for generating coherent and relevant text responses, as it helps models disambiguate meanings and maintain the flow of conversation.

For example, the word "bank" can mean a financial institution or the side of a river, depending on the context. In the sentence "She went to the bank to deposit money," the context clarifies that "bank" refers to a financial institution. Conversely, in "He sat by the river bank," the context indicates that "bank" refers to the side of a river.

Modern NLP models, such as transformers, use mechanisms like self-attention to effectively capture and utilise context from entire sequences of text. This capability enables them to generate more accurate and contextually appropriate responses, making them highly effective for applications like chatbots, translation services, and content generation. Additionally, the use of RAG can help provide extra context to a generative model, further enhancing the accuracy of its responses.

Context Window 

Not to be confused with the linguistic sense of context above, a context window is the maximum amount of text, measured in tokens, that a model can "see" at once: your prompt, any earlier messages in the conversation, and its own response, all combined. Once a conversation grows past that limit, the oldest parts get dropped or summarised so the model can keep going, which is why long chat sessions can sometimes seem to forget things you said earlier.

Context windows vary hugely between models, from a few thousand tokens in early GPT-3-era models to well over a million tokens in some modern frontier models. A bigger context window lets you feed a model more source material, entire codebases, long documents, whole conversation histories, but it isn't free: processing more tokens costs more and can slow down responses. This is also where RAG earns its keep, since retrieving only the relevant snippets of a large knowledge base is far more efficient than trying to stuff everything into the context window at once.

Latent Space

Latent space is like a hidden workshop inside a generative AI model where the essential features of the data it has learned are stored. This data comes from the large datasets the AI was trained on, which could include images, text, music, or other forms of information. Think of latent space as the AI's creative workspace, where it compresses and organizes complex ideas, patterns, and relationships found in this training data into simplified, abstract representations.

When you give the AI a prompt, like "create me a poem about a cat that meows at the moon," the AI dives into its latent space to find relevant features. It searches through this space for concepts related to "cats," "moon," "meowing," and "poetry," pulling together abstract representations of these ideas that it learned during training.

The AI then combines these elements in its latent space, drawing from the patterns and relationships it has stored, to generate a new, unique poem that fits your prompt. The fascinating aspect of latent space is its ability to capture the core essence of the data, allowing the AI to explore and manipulate these abstract representations to produce creative variations. By moving through different areas within this space, the AI can generate diverse outputs, each unique yet still deeply connected to the original data it learned from, enabling it to create a poem that might describe a cat serenading the moon with its meows, in a way that's both imaginative and coherent.

Advanced AI Models

Large Language Models (LLM) 

LLMs are neural networks with billions of parameters, trained on vast amounts of text data. These models are capable of generating coherent and contextually relevant text, making them powerful tools for a variety of NLP tasks.

LLMs, such as OpenAI's GPT family, Anthropic's Claude family, and Meta's Llama family, have revolutionised the field of NLP by enabling advanced applications like chatbots, language translation, text summarisation, and content creation. Their large parameter count allows them to capture intricate patterns and nuances in language, leading to high-quality text generation.

The capabilities of LLMs extend beyond simple text generation; they can also perform tasks like answering questions, completing sentences, and even editing blog posts 😁 with a high degree of relevance and coherence. By leveraging the vast amount of training data, LLMs can understand and generate human-like text, making them essential tools in modern NLP applications.

Diffusion Models 

Diffusion models are a class of generative models that create data by reversing a diffusion process. In this context, diffusion refers to a process where data is progressively transformed into noise, and the model learns to reverse this process to generate new data from noise.

Diffusion models are particularly effective in creating high-quality images and have applications in generative art and image synthesis. For example, they can generate realistic images of objects, scenes, and even artwork by starting from random noise and iteratively refining the image through the learned reverse diffusion process.

These models work by training on large datasets of images, learning how to add noise to the data, and then learning the reverse process to remove noise step by step. This results in the generation of new, high-quality images that are often indistinguishable from real ones.

Foundation Models 

Foundation models are large, pre-trained models that serve as a robust starting point for various specific tasks. These models are trained on vast datasets, capturing a wide range of knowledge and patterns that can be fine-tuned for particular applications with minimal additional training data.

Foundation models, such as OpenAI's GPT family, Google's BERT, and Meta's Llama, have revolutionised AI by providing powerful and versatile tools for a wide range of tasks in NLP and beyond. They enable applications such as text generation, sentiment analysis, translation, and summarisation by leveraging the extensive pre-training they have undergone.

The primary benefit of foundation models is that they reduce the need for extensive task-specific training data, making it easier and more efficient to develop high-performing AI systems. By fine-tuning these models on a smaller, task-specific dataset, developers can achieve excellent performance without the need for massive amounts of training. This approach accelerates development and deployment, making advanced AI capabilities more accessible and practical for various industries.

Frontier Models 

Frontier models represent the cutting edge of AI research and development, pushing the boundaries of what’s possible with current technology. These models explore novel architectures, innovative training techniques, and new applications to advance the state of AI.

Examples of frontier models include the latest reasoning-focused releases from OpenAI, Anthropic, and Google DeepMind, alongside specialised systems like DeepMind's AlphaFold for protein structure prediction. What counts as "frontier" is a moving target by definition: today's frontier model is next year's baseline, which is exactly why terms like reasoning models and agentic AI barely existed in the last version of this post and now warrant their own section below.

Agentic AI & Modern Tooling

AI Agents

An AI agent is a system built on top of an AI model that can take actions, not just generate text. Instead of only responding to a single prompt with an answer, an agent can plan a sequence of steps, call tools like a web search or a code interpreter, look at the result, and decide what to do next, looping through that cycle until the task is done.

The shift from chatbot to agent is the difference between asking "what's a good file structure for a React project?" and asking "set up a React project with this file structure" and having it actually create the files. Agentic AI has been one of the biggest changes in the field since I first wrote this post, with coding agents and general-purpose agents handling multi-step tasks that used to require a human at every decision point. The trade-off is that more autonomy means more room for the agent to go down the wrong path, so most agentic systems still include guardrails, approval steps, or a human checking in before anything irreversible happens.

Model Context Protocol (MCP)

MCP is an open standard for connecting AI models and agents to external tools and data, released by Anthropic in late 2024 and since adopted widely across the industry. Before MCP, every application that wanted an AI model to read a file, query a database, or call an API had to build a custom, one-off integration for that specific model and that specific tool.

MCP standardises that connection: a tool or data source exposes itself as an MCP server once, and any MCP-compatible model or agent can use it without custom glue code. Think of it like a USB port for AI, rather than every device needing its own proprietary cable. This is what lets an AI agent plug into your calendar, your codebase, or your project management tool through a shared protocol instead of bespoke integration work for each one.

Tool Use (Function Calling)

Tool use, also called function calling, is a model's ability to recognise when it needs external help, call a defined function or tool to get it, and use the result in its response. Rather than trying to calculate a sum, look up today's date, or produce live data purely from what it learned in training, where it might guess or hallucinate, the model calls out to a tool built specifically for that job.

For example, a model given access to a "get_weather" tool can recognise a question like "what's the weather in Melbourne right now" as something it can't reliably answer from training data alone, call the tool with "Melbourne" as the input, and use the returned data in its answer. Tool use is the foundational capability that makes AI agents possible: an agent is essentially a model with a loop and a set of tools it's allowed to call.

Reasoning Models

Reasoning models are AI models trained to work through a problem step by step before producing a final answer, rather than generating a response in one pass. This step-by-step working, often called chain-of-thought or extended thinking, happens either visibly or behind the scenes, and tends to produce more accurate answers on tasks involving logic, maths, or multi-step planning.

The trade-off is time and cost: reasoning models spend more computation working through the problem before answering, so they're typically slower and more expensive per response than a model answering directly. They're best suited to genuinely hard problems, like debugging a tricky piece of code or planning a multi-step task, rather than simple questions where the extra "thinking" adds latency without adding value.

Multimodal AI

Multimodal AI refers to models that can understand and generate more than one type of data, such as text, images, audio, and video, within the same system. Early generative AI models were largely single-purpose: a text model handled text, an image model handled images. Modern frontier models can take a photo and a question about it as input, or generate an image from a text description, within the same conversation.

This matters practically because it lets a single model handle tasks that used to require stitching several specialised models together, like describing a screenshot, transcribing a voice note, or reading a chart in a PDF and answering questions about it.

Open-Weight vs Closed Models

This describes whether an AI model's weights, the actual trained parameters, are published for anyone to download and run, or kept private and only accessible through a paid API. Meta's Llama family and Mistral's models are open-weight; OpenAI's GPT family and Anthropic's Claude models are closed.

Open-weight models can be run on your own hardware, inspected, and fine-tuned without depending on a vendor's API staying online or unchanged, which matters for privacy-sensitive or offline use cases. Closed models are usually more expensive to run at scale but tend to sit at the frontier of capability, since the company maintains full control over training, safety tuning, and ongoing improvements. Neither is strictly better; the right choice depends on whether you value control and cost, or convenience and cutting-edge performance.

Small Language Models (SLMs)

SLMs are language models with a fraction of the parameters of a full-sized LLM, often in the millions to low billions rather than hundreds of billions. They trade some general capability for being fast, cheap, and small enough to run on a phone or laptop rather than requiring a data centre.

SLMs suit narrow, well-defined tasks, like classifying support tickets or extracting fields from a form, where a much larger model would be overkill. As a rule of thumb, if a task doesn't need broad world knowledge or complex reasoning, an SLM, or a fine-tuned one, can often do the job for a fraction of the cost and latency of a frontier-scale model.