What is generative AI and how does it work? – The Turing Lectures with Mirella Lapata

In this section, the speaker introduces the concept of generative artificial intelligence, explaining that it involves a computer performing tasks typically done by humans, with a focus on creating new content. Examples include audio, computer code, images, text, and videos, but the speaker will focus on text. The outline includes discussing the past, present, and future of AI, beginning with its longstanding presence through tools like Google Translate.

The speaker highlights GPT-4, noting its capabilities in passing exams like the SAT and writing various forms of content. The rapid adoption of ChatGPT is shown through a graph comparing its growth to other tools, emphasizing its sophisticated technology. The explanation shifts to the underlying technology, particularly language modeling, which predicts text continuation based on context.

Continuing from language modeling, the speaker describes how predictions are made, using a context like "I want to" to illustrate potential continuations. Neural networks have replaced older methods, learning through training rather than exact counting. Key components include collecting large data corpuses and training neural networks to predict missing text, simulating language tasks for models like GPT.

The speaker explains the architecture of neural networks, using a simple model to illustrate how parameters or weights are learned. The evolution from simple architectures to complex transformer models, like those used in GPT, is highlighted. Transformers utilize layers of neural networks to handle language tasks and predict text, with training using vast data sources.

With comparisons to entity models like chunks of human text, the emphasis shifts to scaling language models, increasing parameter size to enhance capabilities. This scaling is crucial for achieving sophisticated outputs across varied tasks. The scale is depicted through a graph showing the exponential growth in parameters from early to current versions of GPT.

The process of using pre-trained models and the significance of fine-tuning them for specific tasks is examined. Language models become powerful through exposure to large text corpora and adjustments to their weights to match specific tasks. The importance of instruction-based fine-tuning to align AI performance with human expectations and standards is emphasized.

The potential and challenges of employing language models like GPT are explored through the necessity of fine-tuning. Demonstrations of human preference training for helpful, honest, and harmless AI responses are discussed. The expensive nature of such re-training with human input is acknowledged, reinforcing the alignment problem these systems present.

Through a demonstration using ChatGPT showing its limitations and strengths, the speaker highlights issues like verbosity, lack of real-time updates, and potential biases. Examples include generating creative content and providing factual information. These illustrate the challenges AI faces despite advancements in preference training.