What is a decision tree?

A decision tree is a tree-structured model that makes predictions by asking a sequence of binary questions about features, splitting the data at each node until reaching a leaf node with a class label or value. Splits are chosen to maximize information gain (using entropy or Gini impurity for classification, or MSE reduction for regression). Decision trees are interpretable and handle both numerical and categorical data, but are prone to overfitting on deep trees. Pruning or ensembling (Random Forests, Gradient Boosting) is used to improve generalization.