What Are Large Language Models? Behind the Scenes of Artificial Intelligence

Marcello Politi : 12 November 2025 22:11

In the rapidly evolving world of artificial intelligence (AI), one term has emerged with increasing importance: Language Model, especially large language models, then called Large Language Models.

You’ve likely already used tools like chatGPT, deep learning models that generate coherent, human-like text. If so, you’ve already experienced the capabilities of large language models. In this article, we’ll delve deeper into what language models are, how they work, and explore some important examples from the scientific literature.

Linguistic Models

In short, a language model is software that can predict the probability of a sequence of words . Language models are trained on large archives of text data (corpus) so that they can learn linguistic patterns.

Imagine reading a sentence: “The cat is on the __ “.

Most humans, based on their understanding of Italian, would assume the next word might be “table,” “floor,” “sofa,” or something similar. This prediction is based on our previous knowledge and experience with the language. A language model aims to do something similar, but on a much larger scale and with a vast amount of text.

Language models assign probabilities to word sequences. A well-trained model would assign a higher probability to the sequence “The cat is on the sofa” than to “The cat is in the sky,” since the latter is less common in a typical Italian text.

Evolution of Linguistic Models

Language models have been around for years, but their capabilities have improved significantly with the advent of deep learning and neural networks. These models have grown in size, capturing more nuance and complexity of human language. Let’s look at some of the most famous examples of models that have been used to process natural language.

• N-gram models : Based on traditional statistics, where N refers to a numerical parameter chosen by the data scientist. An n-gram model predicts the next word based on the n-1 preceding words. It then generates probabilities by counting the frequency of word sequences it encounters in the training text data.
• Recurrent Neural Networks (RNNs) : Introduced to overcome the limitations of n-gram models, RNNs can theoretically consider all preceding words in a text. However, in practice, they struggle to handle long-term dependencies, that is, to understand whether a word, like an adjective, refers to another word that is significantly further away in the text.
• Transformers : This architecture has revolutionized the world of natural language processing (NLP) and deep learning in general. Models such as BERT (Bidirectional Encoder Representations from Transformers) and GPT (Generative Pre-trained Transformer) fall into this category. They are able to understand the context of text and how words depend on each other. Furthermore, they are much faster to train than RNNs because training can be run in parallel. Going into more detail, a Transformer model is made up of two parts: an encoder (left), which generates a mathematical representation of the language, and a decoder (right), which generates new words given an input, as chatGPT does. BERT is a model developed using only the encoder part of the Transformer, making it excellent for creating vector representations, while GPT is based on the decoder part, which specializes in text generation.

Titanic Models

When we say “large” in “large language models,” we’re referring to the enormous number of parameters these models have, often in the billions. In fact, it seems that model performance improves as the number of parameters increases, which is why commercial models are increasingly larger.

Of course, it’s important to keep in mind that models of this size are prohibitively expensive for most businesses; training these models is estimated to cost millions of euros. Those who don’t have this financial means typically limit themselves to fine-tuning already trained models, that is, continuing to train a model like BERT on their own data so that the model becomes even better at a given area.

Obviously, these trainings also have significant environmental impacts, which is why much research is being done on how to optimize and reduce the size of models while maintaining their effectiveness. Here are some of the latest Large Language Models:

• Google’s BERT : With 340 million parameters, it was designed to understand the context of words in search queries. BERT is able to consider the full context of a word by looking at the words that precede and follow it.
• Microsoft Turing-NLG : With 17 billion parameters, this model is designed for various natural language processing tasks, including question answering and language translation.
• OpenAI’s GPT-3 : With 175 billion parameters, GPT-3 is one of the most advanced language models. It can generate coherent paragraphs of text, answer questions, write poetry, and even generate computer code.

Large language models are surpassing all expectations in the field of natural language processing. As they continue to grow and evolve, their potential applications in areas such as healthcare, finance, education, and entertainment are limitless.

Whether assisting writers in creating content, powering advanced chatbots, or helping researchers review literature, these models have paved the way for a future where humans and machines communicate seamlessly.

Marcello Politi


Lista degli articoli