The magic chatbot - A look at future technologies

GPT-3, ChatGPT, and OpenAI are currently trending online. The latest version of ChatGPT, a chatbot created by US company OpenAI from San Francisco, is generating some media buzz as it impresses users from a wide variety of fields with its new test version. This technological milestone in AI-based text recognition could also open up new opportunities for logistics.

As the name suggests, a chatbot is a combination of “chat” and “robot”: a text- or audio-based dialogue system that allows real-time communication between people and machines based on natural language. Simple chatbots search either the web or a defined dataset for keywords entered by the user. They then use a programmed decision tree to select predefined texts or text modules from a database of possible answers so they can provide these as responses. Websites will often feature this kind of basic chatbot as a way to provide quick and easy user assistance.

Smarter chatbots use natural language processing (NLP) to determine the context for a completely new written or spoken sentence. The goal is to find suitable answers on the basis of, say, a knowledge database. NLP is based on artificial intelligence (AI) and machine learning algorithms. So far, the quality of the results from such intelligent chatbots has been of limited practical use. As tests at DACHSER showed, available algorithms from well-known cloud providers correctly recognized the context of, at best, 60 to 80 percent of the input.

The GPT-3 AI algorithm

Now, OpenAI has achieved a new dimension of context recognition with its ChatGPT software. GPT stands for “generative pre-trained transformer” and refers to a special new type of NLP model the company developed in 2018. It has now released GPT-3, the third version of the NLP model that is the foundation of ChatGPT’s success. In the meantime, the company has moved away from its original open source approach and is taking a more commercial tack.

Unlike previous chatbots, ChatGPT is able to deliver natural-sounding conversations enriched with a wealth of factual information over many rounds of text entry. This has been confirmed by multiple studies as well as by DACHSER’s initial tests. Based on a deep learning model and a large training dataset, ChatGPT is capable of handling all sorts of questions across a wide range of subject areas. Its answers are similar in pattern and structure to human speech. However, it cannot (yet) replace independent human thinking and critical probing of content and context. When it comes to mathematical skills and certain logic tasks, too, ChatGPT still has a long way to go.

But that’s not the case in semantics: each chatbot response is always built up word by word by calculating probabilities for the next word. In addition, ChatGPT can generate or correct software code. Writing messages, poems, summaries, or market analyses in different languages is basically no challenge for the ChatGPT algorithm either.

Besides its very large training dataset, a special feature of GPT-3 is the way in which the underlying AI model was trained. For ChatGPT, forms of supervised and reinforcement learning were combined and people themselves were involved in the training process (active learning). To put it simply, humans served as the trainer, assuming the roles of both questioner and respondent during the supervised learning process. In the reinforcement learning phases, the trainers devised a method for ranking the quality of the chatbot’s responses to a conversation that had already taken place. Based on these rankings, they created reward models that were then incorporated into further training iterations.

ChatGPT also includes a rule-based mechanism that aims to prevent inappropriate responses to the extent possible. The lack of this capability was previously a major criticism of the predecessor to GPT‑3 and comparable models. However, even ChatGPT still struggles to reliably exclude plagiarism, fake news, and discriminatory or sexist positions.