Hello, Thank you for the nice article. I am overall very much in accordance with what you wrote. But in my experience, there is still one use case when deep neural networks are a good option even with tabular data. It is the case of extreme multi-class classification. I have been working for more than a year on a deep neural network for a recommender system. The idea is that you have a dataset with millions of observations collected in a matter of days with mostly categorical variables. You have the sequence of items the user has consumed and you must recommend the next best item for the user. Given that you have several hundred thousand possible next items, this translates into an extreme multi-class classification problem. In such a setting, tree-based methods like XGBoost won't perform as well as a neural network if properly crafted. In this case, because we are exploiting a sequence of items you can directly include an inductive bias (with the introduction of a GRU cell) to account for the fact that samples are not collected iid. This makes a huge difference. For reference, I am talking about this paper: https://arxiv.org/abs/1511.06939

I would be glad to have your opinion on this (if you have the time to answer). Cheers and thanks for the article