Anonymous
Dimensionality problem occur because there are many correlated feature causing model error and irrelevant feature make the model to overfitting. I would address by reducing the dimensionality in the datasets, that said to reduce the redundant features. I can use correlation analysis, if two features are correlated, those are redundant features. Also, I can use L1 or L2 regularization method to reduce dimension in logistic regression or linear regression problem. L1 regularization can remove irrelevant feature by shrinking coefficient to zero. L2 regularization method works by reducing the feature coefficient. Those are working by adding a penalty term in the log loss function. If we are using the tree based method like random forest, we can do recursive feature selection, because random forest have the feature importance data. It works by training the model, get the feature importance data and remove the least importance feature, recursively.
