Anonymous
I would consider several factors:
- Amount of data
- Type of data/features
- Complexity of the problem
If the data is limited, the features are simple (not images or text) and the problem is not very complex, we can experiment with classical methods like linear regression, SVM, logistic regression, etc... If the features are complex, like text or image, or the task is very complex, deep learning has proven to be very sucessful. Depending on the amount of data available, we can either train a model from scratch if plenty of data is available, otherwise we can try to fine tune an existing model.