Write a Blog >>
Tue 10 Nov 2020 01:07 - 01:08 at Virtual room 1 - ML In Practice

Deep learning (DL) has been increasingly adopted by a variety of software-intensive systems. Developers mainly use GPUs to accelerate the training, testing, and deployment of DL models. However, the GPU memory consumed by a DL model is often unknown to them before the DL job executes. Therefore, an improper choice of neural architecture or hyperparameters can cause such a job to run out of the limited GPU memory and fail. Our recent empirical study has found that many DL job failures are due to the exhaustion of GPU memory. This leads to a horrendous waste of computing resources and a significant reduction in development productivity. In this paper, we propose DNNMem, an accurate estimation tool for GPU memory consumption of DL models. DNNMem employs an analytic estimation approach to systematically calculate the memory consumption of both the computation graph and the DL framework runtime. We have evaluated DNNMem on 5 real-world representative models with different hyperparameters under 3 mainstream frameworks (TensorFlow, PyTorch, and MXNet). Our extensive experiments show that DNNMem is effective in estimating GPU memory consumption.

Conference Day
Tue 10 Nov

Displayed time zone: (UTC) Coordinated Universal Time change

01:00 - 01:30
01:00
2m
Talk
A Comprehensive Study on Challenges in Deploying Deep Learning Based Software
Research Papers
Zhenpeng ChenPeking University, China, Yanbin CaoPeking University, China, Yuanqiang LiuPeking University, China, Haoyu WangBeijing University of Posts and Telecommunications, Tao XiePeking University, Xuanzhe LiuPeking University, China
DOI Pre-print
01:03
1m
Talk
A First Look at the Integration of Machine Learning Models in Complex Autonomous Driving Systems: A Case Study on Apollo
Industry Papers
pengzi Concordia University, Canada, Jinqiu YangConcordia University, Montreal, Canada, Tse-Hsun (Peter) ChenConcordia University, Lei MaKyushu University
DOI
01:05
1m
Talk
Enhancing the Interoperability between Deep Learning Frameworks by Model Conversion
Industry Papers
Yu David LiuSUNY Binghamton, USA, Cheng ChenByteDance, China, Ru ZhangMicrosoft Research, Tingting QinMicrosoft Research, China, Xiang JiMicrosoft Research, China, Haoxiang LinMicrosoft Research, Mao YangMicrosoft Research
DOI
01:07
1m
Talk
Estimating GPU Memory Consumption of Deep Learning Models
Industry Papers
Yanjie GaoMicrosoft Research, China, Yu David LiuSUNY Binghamton, USA, Hongyu ZhangUniversity of Newcastle, Australia, lizhengxian Microsoft Research, China, Yonghao ZhuMicrosoft Research, China, Haoxiang LinMicrosoft Research, Mao YangMicrosoft Research
DOI
01:09
1m
Talk
IntelliCode Compose: Code Generation using Transformer
Industry Papers
Alexey SvyatkovskiyMicrosoft, Shao Kun DengMicrosoft Corporation, Shengyu FuMicrosoft, USA, Neel SundaresanMicrosoft Corporation
DOI Pre-print
01:11
1m
Talk
Reducing DNN Labelling Cost using Surprise Adequacy: An Industrial Case Study for Autonomous Driving
Industry Papers
Jinhan KimKAIST, Jeongil JuHyundai Motor Group, South Korea, Robert FeldtChalmers University of Technology, Sweden, Shin YooKorea Advanced Institute of Science and Technology
DOI Pre-print
01:13
17m
Conversations on ML In Practice
Research Papers
Sidong FengAustralian National University, Australia, Tse-Hsun (Peter) ChenConcordia University, Yanbin CaoPeking University, China, Yanjie GaoMicrosoft Research, China, Zhenpeng ChenPeking University, China, M: Joshua GarciaUniversity of California, Irvine