LLMs Versus AI Demystifying Chatbot Technology

You have likely encountered the term "large language models," or LLMs, in discussions about generative AI. However, LLMs are not exactly the same as well-known chatbots like ChatGPT, Google Gemini, Microsoft Copilot, Meta AI, and Anthropic's Claude.

These AI chatbots can deliver impressive results, but they do not understand word meanings as humans do. Instead, they function as the interface through which we interact with large language models. These underlying technologies are trained to identify how words are used and which words often appear together, enabling them to predict subsequent words, sentences, or paragraphs. Grasping how LLMs operate is fundamental to understanding how AI works. As AI becomes more prevalent in our daily online activities, this knowledge is increasingly important.

Here is what you need to know about LLMs and their connection to AI.

What is a language model?

Think of a language model as a predictor of words.

"A language model is something that tries to predict what language looks like that humans produce," stated Mark Riedl, professor in the Georgia Tech School of Interactive Computing and associate director of the Georgia Tech Machine Learning Center. "What makes something a language model is whether it can predict future words given previous words."

This forms the basis of autocomplete functions in texting, as well as AI chatbots.

What is a large language model?

A large language model processes vast quantities of words from numerous sources. These models are assessed by what are known as "parameters."

So, what exactly is a parameter?

LLMs utilize neural networks, which are machine learning models that receive an input and perform mathematical calculations to generate an output. The number of variables in these computations constitutes the parameters. A large language model can have 1 billion parameters or even more.

"We know that they're large when they produce a full paragraph of coherent fluid text," Riedl added.

How do large language models learn?

LLMs acquire knowledge through a core AI process called deep learning.

"It's a lot like when you teach a child -- you show a lot of examples," said Jason Alan Snyder, global CTO of ad agency Momentum Worldwide.

Essentially, the LLM is fed a library of content (known as training data) such as books, articles, code, and social media posts. This helps it understand word usage in various contexts and even subtle language nuances. The data collection and training methods of AI companies have become subjects of controversy and lawsuits. Publishers like The New York Times, artists, and other content owners allege that tech companies have used their copyrighted material without proper authorization.

(Disclosure: Ziff Davis, CNET's parent company, filed a lawsuit against OpenAI in April, alleging infringement on Ziff Davis copyrights in the training and operation of its AI systems.)

AI models process far more information than a human could read in a lifetime—on the order of trillions of tokens. Tokens assist AI models in breaking down and processing text. Imagine an AI model as a reader needing assistance: it breaks sentences into smaller pieces, or tokens—equivalent to about four characters in English, or roughly three-quarters of a word—to understand each part and then the overall meaning.

From this point, the LLM can analyze word connections and determine which words frequently co-occur.

"It's like building this giant map of word relationships," Snyder explained. "And then it starts to be able to do this really fun, cool thing, and it predicts what the next word is … and it compares the prediction to the actual word in the data and adjusts the internal map based on its accuracy."

This cycle of prediction and adjustment occurs billions of times, allowing the LLM to continuously refine its language understanding and improve at identifying patterns and predicting future words. It can even learn concepts and facts from the data to answer questions, generate creative text, and translate languages. However, they do not comprehend word meanings like humans do; they only recognize statistical relationships.

LLMs also enhance their responses through reinforcement learning from human feedback.

"You get a judgment or a preference from humans on which response was better given the input that it was given," said Maarten Sap, assistant professor at the Language Technologies Institute at Carnegie Mellon University. "And then you can teach the model to improve its responses."

LLMs are good at handling some tasks but not others. Alexander Sikov/iStock/Getty Images Plus

What do large language models do?

Given a sequence of input words, an LLM predicts the next word in that sequence.

For instance, consider the phrase, "I went sailing on the deep blue..."

Most people would likely guess "sea" because sailing, deep, and blue are words commonly associated with the sea. In other words, each word provides context for what should follow.

"These large language models, because they have a lot of parameters, can store a lot of patterns," Riedl noted. "They are very good at being able to pick out these clues and make really, really good guesses at what comes next."

What are the different kinds of language models?

There are several sub-categories you might have encountered, such as small, reasoning, and open-source/open-weights models. Some of these models are multimodal, meaning they are trained not just on text but also on images, video, and audio. They are all language models and perform similar functions, but there are key distinctions to be aware of.

Is there such a thing as a small language model?

Yes. Tech companies like Microsoft have introduced smaller models designed to operate "on device," not requiring the extensive computing resources of an LLM, yet still enabling users to harness generative AI power.

What are AI reasoning models?

Reasoning models are a type of LLM. These models offer a glimpse into a chatbot's thought process as it answers your questions. You might have observed this if you have used DeepSeek, a Chinese AI chatbot.

But what about open-source and open-weights models?

These are still LLMs! These models are designed for greater transparency in their operations. Open-source models allow anyone to see how the model was constructed and are typically available for customization. Open-weights models provide some insight into how the model prioritizes specific characteristics when making decisions.

What do large language models do really well?

LLMs are highly proficient at understanding the connections between words and producing text that sounds natural.

"They take an input, which can often be a set of instructions, like 'Do this for me,' or 'Tell me about this,' or 'Summarize this,' and are able to extract those patterns out of the input and produce a long string of fluid response," Riedl said.

However, they have several weaknesses.

Where do large language models struggle?

First, they are not adept at discerning truth. In fact, they sometimes fabricate information that sounds true, such as when ChatGPT cited six fictitious court cases in a legal brief or when Google's Bard (Gemini's predecessor) erroneously credited the James Webb Space Telescope with taking the first exoplanet pictures. These are known as hallucinations.

"They are extremely unreliable in the sense that they confabulate and make up things a lot," Sap stated. "They're not trained or designed by any means to spit out anything truthful."

They also encounter difficulties with queries fundamentally different from their training data, as they focus on pattern recognition and response.

A prime example is a math problem with a unique set of numbers.

"It may not be able to do that calculation correctly because it's not really solving math," Riedl explained. "It is trying to relate your math question to previous examples of math questions that it has seen before."

While excelling at word prediction, they are not proficient at predicting the future, which includes planning and decision-making.

"The idea of doing planning in the way that humans do it with … thinking about the different contingencies and alternatives and making choices, this seems to be a really hard roadblock for our current large language models right now," Riedl observed.

Finally, they struggle with current events because their training data typically has a cutoff point, and subsequent events are outside their knowledge base. Lacking the ability to distinguish factual truth from likelihood, they can confidently provide incorrect information about current events.

They also do not interact with the world as humans do.

"This makes it difficult for them to grasp the nuances and complexities of current events that often require an understanding of context, social dynamics and real-world consequences," Snyder remarked.

How are LLMs integrated with search engines?

We are witnessing the evolution of retrieval capabilities beyond model training data, including connections with search engines like Google. This allows models to conduct web searches and feed those results into the LLM, potentially enabling them to better understand queries and provide more timely responses.

"This helps our linkage models stay current and up-to-date because they can actually look at new information on the internet and bring that in," Riedl said.

This was the objective, for example, with AI-powered Bing. Instead of using search engines to enhance responses, Microsoft aimed to improve its search engine with AI, partly by better understanding user query intent and ranking results more effectively. Last November, OpenAI introduced ChatGPT Search, with access to information from some news publishers.

However, there are drawbacks. Web search could exacerbate hallucinations without adequate fact-checking. LLMs would need to learn to assess web source reliability before citing them. Google learned this lesson with the error-prone launch of its AI Overviews search results. The search company subsequently refined its AI Overviews results to reduce misleading summaries. Yet, even recent reports indicate AI Overviews cannot consistently determine what year it is.

For further information, see our experts' list of AI essentials and the best chatbots for 2025.