Increased popularity of `intelligent' web services provides end-users with machine-learnt functionality at little effort to developers. However, these services require a decision threshold to be set which is dependent on problem-specific data. Developers lack a systematic approach for evaluating intelligent services and existing evaluation tools are predominantly targeted at data scientists for pre-development evaluation. This paper presents a workflow and supporting tool, Threshy, to help \textit{software developers} select a decision threshold suited to their problem domain. Unlike existing tools, Threshy is designed to operate in multiple workflows including pre-development, pre-release, and support. Threshy is designed for tuning the confidence scores returned by intelligent web services and does not deal with hyper-parameter optimisation used in ML models. Additionally, it considers the financial impacts of false positives. Threshold configuration files exported by Threshy can be integrated into client applications and monitoring infrastructure. Demo: \url{https://bit.ly/2YKeYhE}.
Wed 11 Nov Times are displayed in time zone: (UTC) Coordinated Universal Time change
01:00 - 01:30: Cloud / Services 1Paper Presentations / Industry Papers / Research Papers / Tool Demos at Virtual room 2 | |||
| 01:00 - 01:02 Talk | Beware the Evolving ‘Intelligent’ Web Service! An Integration Architecture Tactic to Guard AI-First Components Research Papers Alex CummaudoDeakin University, Australia, Scott BarnettDeakin University, Australia, Rajesh VasaDeakin University, Australia, John GrundyMonash University, Australia, Mohamed AbdelrazekDeakin University, Australia DOI | ||
| 01:03 - 01:04 Talk | Efficient Customer Incident Triage via Linking with System Incidents Industry Papers Jiazhen GuFudan University, China, Jiaqi WenPeking University, China, Zijian WangFudan University, China, Pu ZhaoMicrosoft Research, China, Chuan LuoMicrosoft Research, China, Yu KangMicrosoft Research, China, Yangfan ZhouFudan University, China, Li YangMicrosoft Azure, USA, Jeffrey SunMicrosoft Azure, USA, Zhangwei XuMicrosoft, China, Bo QiaoMicrosoft Research, China, Liqun LiMicrosoft Research, China, Qingwei LinMicrosoft Research, China, Dongmei ZhangMicrosoft Research, China DOI | ||
| 01:05 - 01:06 Talk | How to Mitigate the Incident? An Effective Troubleshooting Guide Recommendation Technique for Online Service Systems Industry Papers Jiajun JiangTianjin University, China, Weihai LuPeking University, China, Junjie ChenTianjin University, China, Qingwei LinMicrosoft Research, China, Pu ZhaoMicrosoft Research, China, Yu KangMicrosoft Research, China, Hongyu ZhangUniversity of Newcastle, Australia, Yingfei XiongPeking University, Feng GaoMicrosoft, China, Zhangwei XuMicrosoft, China, Yingnong DangMicrosoft, USA, Dongmei ZhangMicrosoft Research, China DOI | ||
| 01:07 - 01:08 Talk | Identifying Linked Incidents in Large-Scale Online Service Systems Research Papers Yujun ChenMicrosoft Research, China, Xian YangHong Kong Baptist University, China, Hang DongMicrosoft Research, China, Xiaoting HeChinese Academy of Sciences, China, Hongyu ZhangUniversity of Newcastle, Australia, Qingwei LinMicrosoft Research, China, Junjie ChenTianjin University, China, Pu ZhaoMicrosoft Research, China, Yu KangMicrosoft Research, China, Feng GaoMicrosoft, China, Zhangwei XuMicrosoft, China, Dongmei ZhangMicrosoft Research, China DOI | ||
| 01:09 - 01:10 Talk | Mono2Micro: An AI-Based Toolchain for Evolving Monolithic Enterprise Applications to a Microservice Architecture Tool Demos Anup K. KaliaIBM Research, USA, Jin XiaoIBM Research, USA, Chen LinIBM Research, USA, Saurabh SinhaIBM Research, John RofranoIBM Research, USA, Maja VukovicIBM Research, USA, Debasish BanerjeeIBM, n.n. DOI | ||
| 01:11 - 01:12 Talk | Threshy: Supporting Safe Usage of Intelligent Web Services Tool Demos Alex CummaudoDeakin University, Australia, Scott BarnettDeakin University, Australia, Rajesh VasaDeakin University, Australia, John GrundyMonash University, Australia DOI | ||
| 01:13 - 01:14 Talk | Towards Intelligent Incident Management: Why We Need It and How We Make It Industry Papers Zhuangbin ChenChinese University of Hong Kong, China, Yu KangMicrosoft Research, China, Liqun LiMicrosoft Research, China, Xu ZhangMicrosoft Research, China, Hongyu ZhangUniversity of Newcastle, Australia, Hui XuFudan University, China, Yangfan ZhouFudan University, China, Li YangMicrosoft Azure, USA, Jeffrey SunMicrosoft Azure, USA, Zhangwei XuMicrosoft, China, Yingnong DangMicrosoft, USA, Feng GaoMicrosoft, China, Pu ZhaoMicrosoft Research, China, Bo QiaoMicrosoft Research, China, Qingwei LinMicrosoft Research, China, Dongmei ZhangMicrosoft Research, China, Michael LyuCUHK DOI Media Attached File Attached | ||
| 01:15 - 01:30 Talk | Conversations on Cloud / Services 1 Paper Presentations Alex CummaudoDeakin University, Australia, Anup K. KaliaIBM Research, USA, Jiajun JiangTianjin University, China, Zhuangbin ChenChinese University of Hong Kong, China, M: Satish ChandraFacebook, USA | ||