🤖 Machine Learning / AI
Intermediate
What is GPT and how does it differ from BERT?
Answer
GPT (Generative Pre-trained Transformer) is a decoder-only Transformer pre-trained with autoregressive language modeling — predicting the next token given all previous tokens (left-to-right). BERT is an encoder-only model using bidirectional context (sees both left and right). GPT excels at generative tasks (text completion, summarization, code generation); BERT excels at discriminative tasks (classification, NER, QA). GPT-2, GPT-3, and GPT-4 scaled this architecture massively, showing emergent capabilities with scale.
More Machine Learning / AI Questions
View all →- Intermediate What is a convolutional neural network (CNN)?
- Intermediate What is a Recurrent Neural Network (RNN)?
- Intermediate What is an LSTM and how does it solve the vanishing gradient problem?
- Intermediate What is the attention mechanism in neural networks?
- Intermediate What is the Transformer architecture?