AI Foundationsbeginner12 min · 8 steps
Large Language Models: GPT, BERT and ChatGPT
What an LLM is, why GPT predicts the next token while BERT fills in blanks, and how GPT-3, GPT-4 and ChatGPT grew out of that idea.
dataresponserequesttokenerror
Key ideaStep 1 of 8
A large language model learns language by reading
A large language model (LLM) is a neural network with billions or trillions of parameters, trained on vast amounts of text. It learns the statistical patterns of language well enough to translate, summarise, answer questions and write code.
Pre-training is self-supervised: the text itself provides the answers, so no human labels are needed. Afterwards, fine-tuning adapts the model to specific tasks. GPT-4, Claude and LLaMA are all LLMs.
What changed
- +Text corpus
- +Pre-trained LLM
← Previous lessonSequence Models: RNN, LSTM, GRU and the TransformerNext lesson →Vision Models: CNNs, Classification and Detection
Lesson 4 of 34 in AI Foundations