This is why LLMs are also called “generative models”, to differentiate them from models that make predictions and decisions about existing documents and images. In our cat image identification example, a generative model would start creating new cat pictures instead of deciding if there is a cat in an existing one. This is “Generative AI”.

The beauty of training for LLM’s task is that “labeled data” is everywhere around us. Pick a random web page, or even this very blog post. It alone can yield hundreds, if not thousands, of training examples. For example, if you see the string “Pick a random web “, the next word that follows could be “page”. It is an equivalent of (picture of a cat, “cat”) from our cat picture identifier from earlier. Who created this training example? Me, the author of this post. Of course, when I was writing that sentence, I did not think about it as a training example at all (and was not paid for this). And this is the beauty I was talking about – publicly available text written by humans contains billions of training examples without the need to annotate them. And because you really are talking about billions, you don’t have to worry about meticulously curating them – you just throw everything into the pot with a reasonable assumption that all possible sequences and sources are represented in your dataset, because your dataset is literally “everything”.

This gets us to the first “L” in “LLM”. It stands for “large”. These models are large in terms of the amount of data they are trained on and in terms of the number of their parameters, or the weighted variables that the model considers when creating an output. Without going into the math (and with the risk of this analogy having potential holes), a useful way to think about the “parameters” of a model is that they’re like neurons in a living creature’s brain – the more neurons, the more complex processing and memory the creature is capable of.

You must have noticed that this description of what LLMs do as text completion feels different from the experience you got used to when interacting with ChatGPT, where you ask it questions and the model converses with you. This is true. Teaching a model to participate in a conversation and follow instructions takes additional fine-tuning and training that we will not cover in this post. Still, holding a conversation is simply generating responses to what the other party is saying and therefore is a type of text completion. Since we mentioned ChatGPT, let’s also briefly say what it is. ChatGPT is an LLM that OpenAI optimized for handling conversations with humans and made available to the public.

Back to LLMs. Great – so spending a few millions of $ can get you a model that is trained to predict (or generate) the next word. What is it good for?

First of all, it is entertaining. If you ask it to write a joke about pirates in the style of William Shakespeare or a weather forecast that Ernest Hemingway would write for a local Minnesota TV station on a winter day, the model will generate text that sounds plausible and captures the style you requested (Shakespeare, Hemingway, Minnesota, etc.).

This sounds like an awesome parlor trick. Why is it useful?

LLMs’ ability to capture style enables it to “understand” (double quotes are important here, because applying human abilities to the behavior of an inanimate piece of software runs into limitations fast) what a clear, concise, and grammatically correct text looks like and to play an editor to a human writer. This already has some utility in the world of customer support where agents are tasked to communicate with customers all day, not always in a language that they are fluent in, and with the subtleties of cultures that they do not always live in.

But this is only the beginning.

LLMs Can Perform Tasks They Were Not Trained For

LLMs are able to perform tasks that they were not explicitly trained to do and some of these tasks are extremely useful.

Before we proceed, we can’t talk about LLMs without mentioning “prompts”. The text that you ask the model to complete is called a “prompt”. By crafting the right prompt, one can get a model to complete it in a way that achieves a particular task.

Here is one example from the paper Emergent Abilities of Large Language Models published by a team of researchers from Google, DeepMind and Stanford in 2022:

The authors entered this prompt to an LLM:

“Review: This movie sucks.

Sentiment: negative

Review: I love this movie.

Sentiment: “.

The model responded with the word “positive”.

This is what a sentiment classifier would say. This LLM demonstrated the ability to predict the sentiment of text without being explicitly trained to do so. Previously, this would take assembling a large set of user reviews, messages, and other examples of text, labeling them with sentiment and training a special purpose ML model that could do only sentiment classification.

How does it do this? The answer is a bit surprising. The same paper says: “there are currently few compelling explanations for why such abilities emerge in the way they do”

Having absorbed great amounts of text created by people somehow allows LLMs to produce text that not only sounds like it makes sense, but represents some real knowledge.

Prompt Engineering

Until we know the exact mechanisms behind such emerging abilities, prompt engineering (the practice of creating and testing optimal prompts for various tasks) feels a lot like SEO – you are poking a system whose rules you don’t know in order to get something useful out of it. The difference is that unlike Google, even model creators don’t know exactly how an LLM will respond. Practitioners exchange tricks that sometimes sound like magic incantations: