“Compose a welcome email for a new employee,” “Write an article about interest rates that a 10-year-old can understand,” and “Give me 20 name ideas for a fintech start-up”: chatting with the ChatGPT conversational robot sometimes gives the impression of writing to a human, so much you can discuss anything and everything with him.

This is not the case. ChatGPT is the latest language model listed by the American company OpenAI, to which we also owe the GPT-3.5 (2022), GPT-3 (2020) and GPT-2 (2019) models. These GPT models (for “Generative Pre-trained Transformer”) were trained using AI tools from hundreds of gigabytes of texts collected, particularly in books and on the Web, from La Fontaine’s fables to Ricardo’s recipes. (he also speaks French very well).

ChatGPT is a derivative version of GPT-3.5, a model still accessible on the Web or for developers with an application programming interface (API). GPT-3.5 allows you to write a few words, which are then completed by the software, much like automatic text prediction with smartphone keyboards.

The model writes text one syllable at a time based on the likelihood of one word following another. “He is going to write based on the previous words, so he considers the context. So, for example, he will know that the next word to come is probably a verb or a noun depending on what came before”, which illustrates Waverly’s CEO, Philippe Beaudoin.

A certain amount of randomness is also inserted into the text generation, which prevents the answers from always being the same and gives them a slightly more human voice.

Models like GPT-3.5 are based on “Transformer” (the T in GPT), a model developed for language translation and made public in 2017 by Google researchers. Unfortunately, the great concepts of Transformer have since been taken over by most genre models, such as LaMDA, from Google, and even the image generators DALL-E and Midjourney.

From GPT-3.5 to ChatGPT
ChatGPT works the same way as its predecessors. Still, it has been “optimized for dialogue using reinforcement learning with human feedback (RLHF), a method that uses examples written by humans to guide the model,” explains an OpenAI spokesperson.

In other words, sample questions and answers have been provided to GPT-3.5, which has learned to converse, making it simple enough for the general public to use and more powerful for users. In addition to being able to answer questions or write in a style that is asked of it, the chatbot remembers its previous interactions and can therefore understand the context of a discussion and refine its answers if asked for clarification.