Estimating GPU Memory Consumption of Deep Learning Models
Deep learning (DL) has been increasingly adopted by a variety of software-intensive systems. Developers mainly use GPUs to accelerate the training, testing, and deployment of DL models. However, the GPU memory consumed by a DL model is often unknown to them before a DL job is executed. Therefore, an improper choice of neural network architecture or hyperparameters can cause the DL job to run out of the limited GPU memory and fail. Our empirical study has found that many DL job failures are due to the exhaustion of GPU memory. This leads to a horrendous waste of computing resources and a significant reduction in development productivity. In this paper, we propose DNNMem, an accurate estimation tool for GPU memory consumption of DL models. DNNMem employs an analytic estimation approach to systematically calculate the memory consumption of both the computation graph and the DL framework runtime. We have evaluated DNNMem on 5 real-world representative models with different hyperparameters under 3 mainstream frameworks (TensorFlow, PyTorch, and MXNet). Our extensive experiments show that DNNMem is effective in estimating GPU memory consumption.
Tue 10 NovDisplayed time zone: (UTC) Coordinated Universal Time change
01:00 - 01:30 | |||
01:00 2mTalk | A Comprehensive Study on Challenges in Deploying Deep Learning Based Software Research Papers Zhenpeng Chen Peking University, China, Yanbin Cao Peking University, China, Yuanqiang Liu Peking University, China, Haoyu Wang Beijing University of Posts and Telecommunications, Tao Xie Peking University, Xuanzhe Liu Peking University, China DOI Pre-print | ||
01:03 1mTalk | A First Look at the Integration of Machine Learning Models in Complex Autonomous Driving Systems: A Case Study on Apollo Industry Papers pengzi Concordia University, Canada, Jinqiu Yang Concordia University, Montreal, Canada, Tse-Hsun (Peter) Chen Concordia University, Lei Ma Kyushu University DOI | ||
01:05 1mTalk | Enhancing the Interoperability between Deep Learning Frameworks by Model Conversion Industry Papers Yu David Liu SUNY Binghamton, USA, Cheng Chen ByteDance, China, Ru Zhang Microsoft Research, Tingting Qin Microsoft Research, China, Xiang Ji Microsoft Research, China, Haoxiang Lin Microsoft Research, Mao Yang Microsoft Research DOI Pre-print | ||
01:07 1mTalk | Estimating GPU Memory Consumption of Deep Learning Models Industry Papers Yanjie Gao Microsoft Research, China, Yu David Liu SUNY Binghamton, USA, Hongyu Zhang University of Newcastle, Australia, lizhengxian Microsoft Research, China, Yonghao Zhu Microsoft Research, China, Haoxiang Lin Microsoft Research, Mao Yang Microsoft Research DOI Pre-print | ||
01:09 1mTalk | IntelliCode Compose: Code Generation using Transformer Industry Papers Alexey Svyatkovskiy Microsoft, Shao Kun Deng Microsoft Corporation, Shengyu Fu Microsoft, USA, Neel Sundaresan Microsoft Corporation DOI Pre-print | ||
01:11 1mTalk | Reducing DNN Labelling Cost using Surprise Adequacy: An Industrial Case Study for Autonomous Driving Industry Papers Jinhan Kim KAIST, Jeongil Ju Hyundai Motor Group, South Korea, Robert Feldt Chalmers University of Technology, Sweden, Shin Yoo Korea Advanced Institute of Science and Technology DOI Pre-print | ||
01:13 17m | Conversations on ML In Practice Research Papers Sidong Feng Australian National University, Australia, Tse-Hsun (Peter) Chen Concordia University, Yanbin Cao Peking University, China, Yanjie Gao Microsoft Research, China, Zhenpeng Chen Peking University, China, M: Joshua Garcia University of California, Irvine | ||