AMS: Generating AutoML Search Spaces from Weak Specifications
We consider a usage model for automated machine learning (AutoML) in which users
can influence the generated pipeline by providing a weak pipeline
specification: an unordered set of API components from which the AutoML
system draws the components it places into the generated pipeline.
Such specifications allow users to express preferences over the components that appear in the
pipeline, for example a desire for interpretable components to appear
in the pipeline. We present AMS, an approach to automatically strengthen
weak specifications to include unspecified
complementary and functionally related API components, populate the space of
hyperparameters and their values, and pair this configuration with a search
procedure to produce a strong pipeline specification: a full
description of the search space for candidate pipelines. ams uses
normalized pointwise mutual information on a code corpus to identify
complementary components, BM25 as a lexical similarity score
over the target API's documentation to identify
functionally related components, and frequency distributions in the code corpus to
extract key hyperparameters and values. We show that strengthened specifications
can produce pipelines that outperform the pipelines generated from the
initial weak specification and an expert-annotated variant, while producing pipelines that still
reflect the user preferences captured in the original weak specification.
Wed 11 NovDisplayed time zone: (UTC) Coordinated Universal Time change
17:30 - 18:00 | ML Model BuildingResearch Papers / Student Research Competition / Paper Presentations / Visions and Reflections at Virtual room 2 | ||
17:30 2mTalk | AMS: Generating AutoML Search Spaces from Weak Specifications Research Papers José Pablo Cambronero Massachusetts Institute of Technology, USA, Jürgen Cito TU Wien and MIT, Martin C. Rinard Massachusetts Institute of Technology, USA DOI | ||
17:33 1mTalk | Continuous Experimentation on Artificial Intelligence Software: A Research Agenda Visions and Reflections DOI | ||
17:35 1mTalk | DENAS: Automated Rule Generation by Knowledge Extraction from Neural Networks Research Papers Simin Chen University of Texas at Dallas, USA, Soroush Bateni University of Texas at Dallas, USA, Sampath Grandhi University of Texas at Dallas, USA, Xiaodi Li University of Texas at Dallas, USA, Cong Liu University of Texas at Dallas, USA, Wei Yang University of Texas at Dallas, USA DOI | ||
17:37 1mTalk | On Decomposing a Deep Neural Network into ModulesACM SIGSOFT Distinguished Paper Award Research Papers DOI Media Attached | ||
17:39 1mTalk | Synthesizing Correct Code for Machine Learning Programs Student Research Competition Joshua Gisi North Dakota State University, USA DOI | ||
17:41 19mTalk | Conversations on ML Model Building Paper Presentations José Pablo Cambronero Massachusetts Institute of Technology, USA, Rangeet Pan Iowa State University, USA, Simin Chen , Wei Yang University of Texas at Dallas, USA, M: John-Paul Ore North Carolina State University | ||