Learn how to get started with AutoML with two helpful tools — Auto-Sklearn and Auto-PyTorch.
Join the DZone community and get the full member experience. Machine learning (ML) now impacts a wide swath of business, engineering, and research domains, to the extent where you’d be hard-pressed to find a niche where machine learning is totally uninvolved. Progress in ML has come on the coattails of broader trends in software and automation: Wherever human activity depends on doing repetitive tasks that can be readily described in a way that a computer can handle, it’s generally useful to write down a recipe (or program) for the task that the computer can follow. Using ML now means that for many useful tasks it’s no longer necessary to manually write a program, or even to know exactly how to solve the problem. Instead, we can approach many problems by defining a search space and a learning algorithm, and then let the machine figure it out. Modern machine learning is sometimes referred to as “software 2.0” and is a trend that has been super-charged by the effectiveness of, and ensuring research and development interest in, deep learning. There are obvious applications that make a good fit for this approach: like fitting a statistical model to data, for example; but there are more esoteric and impressive examples that were never so obvious in the days of statistical or old-school machine learning. In the last few years, we’ve seen machine learning predict protein folding better than any other method, beat expert human game players in Go, Dota II, Starcraft II, and more, and create reasonably coherent text and speech responses (though the last accomplishment can be hit or miss). Still, these projects almost always require the application of substantial world-class engineering and research talent. That’s not entirely surprising, as coaxing a computer program, even one equipped with sophisticated state-of-the-art machine learning algorithms, to accomplish completely new objectives still requires human innovation. That may change in the future, when AI researchers create new machine learning agents that crack the human expert-level AI researcher threshold. For now, while groundbreaking AI science is still difficult to automate, there’s an ever-growing volume of ML applications where a human engineer isn’t necessarily needed to optimize a model for a given task. In fact, in some tasks, leaving the task of choosing a specific model and stirring the learning hyperparameters up to human judgment might actually slow things down or lead to sub-par results. A human might do a poor job of exploring hyperparameter space, might tend to favor their preferred types of models for bad reasons, or might stop and start training runs more often than is good for effectively training the model for the task (which can also be bad for their mental state). Instead, a good ML practitioner should take advantage of all the tools at their disposal, which now include open-source off-the-shelf tools and best practices for applying ML to ML. In other words, AutoML. AutoML is a broad category of techniques and tools for applying automated search to your automated search and learning to your learning. These range from applying Bayesian optimization to the hyperparameters for a statistical learning algorithm, to neural architecture search for deep learning models. The field is quite active and diverse, with a healthy ecosystem of contests, many of which are cataloged at automl.ai. In fact, one of the most prominent AutoML packages, Auto-SciKit-Learn (Auto-Sklearn), got started as the winner of the 2014 to 2016 ChaLearn AutoML challenge. Auto-Sklearn was developed by one of the most notable research groups pursuing Automated machine learning in the pre-eminent AutoML supergroup from Germany. This collaboration is made up of labs at the University of Freiburg and the University of Hannover. Other noteworthy contributors to the field include the scientists behind Auto-WEKA, one of the first popular AutoML toolkits, and its successor Auto-WEKA 2.
