On Decomposing a Deep Neural Network into ModulesACM SIGSOFT Distinguished Paper Award
Deep learning is being incorporated in many modern software systems. Deep learning approaches train a deep neural network (DNN) model using training examples, and then use the DNN model for prediction. While the structure of a DNN model as layers is observable, the model is treated in its entirety as a monolithic component. To change the logic implemented by the model, e.g. to add/remove logic that recognizes inputs belonging to a certain class, or to replace the logic with an alternative, the training examples need to be changed and the DNN needs to be retrained using the new set of examples. We argue that decomposing a DNN into DNN modules— akin to decomposing a monolithic software code into modules—can bring the benefits of modularity to deep learning. In this work, we develop a methodology for decomposing DNNs for multi-class problems into DNN modules. For four canonical problems, namely MNIST, EMNIST, FMNIST, and KMNIST, we demonstrate that such decomposition enables reuse of DNN modules to create different DNNs, enables replacement of one DNN module in a DNN with another without needing to retrain. The DNN models formed by composing DNN modules are at least as good as traditional monolithic DNNs in terms of test accuracy for our problems.
Wed 11 Nov Times are displayed in time zone: (UTC) Coordinated Universal Time change
17:30 - 18:00: ML Model BuildingPaper Presentations / Research Papers / Student Research Competition / Visions and Reflections at Virtual room 2 | |||
17:30 - 17:32 Talk | AMS: Generating AutoML Search Spaces from Weak Specifications Research Papers Jose CambroneroMassachusetts Institute of Technology, USA, Jürgen CitoTU Wien and MIT, Martin RinardMassachusetts Institute of Technology, USA DOI | ||
17:33 - 17:34 Talk | Continuous Experimentation on Artificial Intelligence Software: A Research Agenda Visions and Reflections DOI | ||
17:35 - 17:36 Talk | DENAS: Automated Rule Generation by Knowledge Extraction from Neural Networks Research Papers SiminChen University of Texas at Dallas, USA, Soroush BateniUniversity of Texas at Dallas, USA, Sampath GrandhiUniversity of Texas at Dallas, USA, Xiaodi LiUniversity of Texas at Dallas, USA, Cong LiuUniversity of Texas at Dallas, USA, Wei YangUniversity of Texas at Dallas, USA DOI | ||
17:37 - 17:38 Talk | On Decomposing a Deep Neural Network into ModulesACM SIGSOFT Distinguished Paper Award Research Papers DOI Media Attached | ||
17:39 - 17:40 Talk | Synthesizing Correct Code for Machine Learning Programs Student Research Competition Joshua GisiNorth Dakota State University, USA DOI | ||
17:41 - 18:00 Talk | Conversations on ML Model Building Paper Presentations Jose CambroneroMassachusetts Institute of Technology, USA, Rangeet PanIowa State University, USA, Simin Chen, Wei YangUniversity of Texas at Dallas, USA, M: John-Paul OreNorth Carolina State University |