XGBoost
Library for gradient boosting of decision trees, released in 2014 by Tianqi Chen. It dominated Kaggle to the point where "try XGBoost first" became the default advice for tabular data — and largely still is.
The idea is old (Friedman, 1999): build trees sequentially, each new tree correcting the errors of the previous ones. XGBoost made it practical with regularization in the objective, second-order gradients, histogram-based split finding, built-in handling of missing values, column subsampling and efficient parallelism across cores and machines. Competitors LightGBM (Microsoft, leaf-wise growth, faster on large datasets) and CatBoost (Yandex, strong on categorical features) are effectively equivalent. On structured data these methods still beat neural networks in most comparisons.