Building machine learning models can be quite tricky. We have to choose the right architecture, tune hyperparamters and have to test several combinations for each set of different input features. Does AutoML help us to build better models?
Well, my experiences with Auto-Keras show that AutoML approaches lead to relatively good models really fast. However, the final model architectures are much more complicated than what we would come up with after a few tries. But this is about neural networks. When it comes to other machine learning algorithms, we are more or less left with feature engineering and hyperparameter tuning once the dataset “static”. How do we decide on what hyperparameter to use? Usually, we do some kind of hyperparameter optimization using some form of grid search. Again, here we have to pre-define sets of hyperparameters.
How do we choose them? It probably involves a bit more guessing than many would like to admit. Therefore, we cannot be sure that we found the optimal or just a optimal solution. This can be tricky when it comes to legal issues. Most current (consumer) product are more likely to end in a social media shitstorm but there are no severe consequences. When it comes to different applications such as medial or resource estimates for mining there is not only a log of money at stake but there might be serious legal consequences as well. Therefore, using proper mathematical optimization techniques as a basis to tune hyperparmeters and select models reduces the pressure on machine learning engineers. We can use mathematical reasoning to explain hyperparameters, feature engineering, model choice and model architectures.