Skip to content
Generative AI

How does BERT differ from a GPT-style model?

BERT is an encoder: bidirectional, trained by masking tokens, so every position sees both sides, which suits classification and embeddings but cannot generate. GPT is a causal decoder trained to predict the next token, which is what enables generation.

datasciencetrivia.com

Card 1 of 213. Answer: BERT is an encoder: bidirectional, trained by masking tokens, so every position sees both sides, which suits classification and embeddings but cannot generate. GPT is a causal decoder trained to predict the next token, which is what enables generation.

Created by santiviquez

About · To suggest new questions or report an error send me a dm.

All questions