Zitao Chen, Niranjhana Narayanan, Bo Fang, Guanpeng Li, Karthik Pattabiraman, and Nathan DeBardeleben, IEEE International Symposium on Software Reliability Engineering (ISSRE), 2020. (Acceptance Rate: 26%) [ PDF | Talk ] (Code)
Abstract: As machine learning (ML) has seen increasing adoption in safety-critical domains (e.g., autonomous vehicles), the reliability of ML systems has also grown in importance. While prior studies have proposed techniques to enable efficient error-resilience (e.g., selective instruction duplication), a fundamental requirement for realizing these techniques is a detailed understanding of the application’s resilience. In this work, we present TensorFI, a high-level fault injection (FI) framework for TensorFlow-based applications. TensorFI is able to inject both hardware and software faults in general TensorFlow programs. TensorFI is a configurable FI tool that is flexible, easy to use, and portable. It can be integrated into existing TensorFlow programs to assess their resilience for different fault types (e.g., faults in particular operators). We use TensorFI to evaluate the resilience of 12 ML programs, including DNNs used in the autonomous vehicle domain. The results give us insights into why some of the models are more resilient. We also present two case studies to demonstrate the usefulness of the tool. TensorFI is publicly available at https://github.com/DependableSystemsLab/TensorFI.
email: email@example.comPhone: 604-827-4245 (please email first)
Address: Rm. 4048, Fred Kaiser Building, 2332 Main Mall, Vancouver, BC V6T1Z4.
- An Empirical Study of the Impact of Single and Multiple Bit-Flip Errors in Programs
- New Wine in an Old Bottle: N-Version Programming for Machine Learning Components
- Stealthy Attacks Against Robotic Vehicles Protected by Control-based Intrusion Detection Techniques
- How Far Have We Come in Detecting Anomalies in Distributed Systems? An Empirical Study with a Statement-level Fault Injection Method
- TensorFI: A Flexible Fault Injection Framework for TensorFlow Applications
- GPU-TRIDENT: Efficient Modeling of Error Propagation in GPU Programs
- How Effective are Smart Contract Static Analysis Tools ? Evaluating Smart Contract Static Analysis Tools Using Bug Injection
- TraceSanitizer – Eliminating the Effects of Non-determinism on Error Propagation Analysis
- Improving the Accuracy of IR-Level Fault Injection