{"id":5977,"date":"2024-07-11T02:31:11","date_gmt":"2024-07-11T09:31:11","guid":{"rendered":"https:\/\/blogs.ubc.ca\/dependablesystemslab\/?page_id=5977"},"modified":"2024-07-11T09:59:58","modified_gmt":"2024-07-11T16:59:58","slug":"membership-inference-attacks-in-machine-learning-models","status":"publish","type":"page","link":"https:\/\/blogs.ubc.ca\/dependablesystemslab\/projects\/membership-inference-attacks-in-machine-learning-models\/","title":{"rendered":"Membership Inference Attacks in Machine Learning Models"},"content":{"rendered":"

Being an inherently data-driven solution, machine learning (ML) models can aggregate and process vast amounts of data, such as clinical files and financial records. With the growing application of ML-based solutions in privacy-sensitive domains such as healthcare, the potential privacy ramification of training DL models on sensitive data stands out as a notable concern.<\/p>\n

To this end, membership inference attacks (MIAs)<\/strong> represent a prominent class of privacy attacks that aim to infer whether a given data point was used to train the model.<\/p>\n

\"\"<\/p>\n

MIAs constitute a fundamental threat to data privacy, e.g., if the model is trained on a sensitive population such as patients with a rare disease, then merely divulging that some individual is part of that population may become a severe privacy risk.<\/p>\n

This project studies MIAs from both a defensive<\/strong> and adversarial<\/strong> perspective.<\/p>\n

Practical Defense against Membership Inference Attacks<\/h3>\n

We introduce HAMP, a defense technique that can achieve strong membership privacy and high accuracy. To mitigate MIAs in different forms, we observe that they can be uni\ufb01ed as they all exploit the ML model\u2019s overcon\ufb01dence in predicting training samples<\/strong> through different proxies. This motivates our design to enforce less con\ufb01dent prediction by the model<\/em>, hence forcing the model to behave similarly on the training and testing samples.<\/p>\n

\"\"<\/p>\n

We show that HAMP simultaneously achieves strong privacy protection against a range of existing MIAs (y-axis) and minimal accuracy degradation (x-axis).<\/p>\n

\"\"<\/p>\n

Zitao Chen and Karthik Pattabiraman, Overconfidence is a Dangerous Thing: Mitigating Membership Inference Attacks by Enforcing Less Confident Prediction. <\/a>Proceedings of the\u00a0Network and Distributed Systems Security Conference (NDSS), 2024.\u00a0(Acceptance Rate: 15%). Artifacts Available, Functional and Reproduced<\/strong><\/p>\n

What\u2019s next?<\/strong><\/em><\/h5>\n

Nevertheless, many existing studies assume the ML\u00a0models are trained without<\/em> being adversarially manipulated; whereas a capable adversary may compromise the models to achieve a desired outcome such as amplifying privacy leakage. This motivates the second contribution of this project.<\/p>\n

Supply chain Attack to Facilitate Stealthy and High-power Membership Inference Attacks<\/strong><\/h3>\n

Modern machine learning (ML) ecosystems offer a surging number of ML frameworks and code repositories that can greatly facilitate the development of ML models. Today, even ordinary data holders who are not ML experts can apply an off-the-shelf codebase to build high-performance ML models on their data, which are often sensitive in nature (e.g., clinical records).<\/p>\n

In this work, we consider a malicious ML provider<\/strong> who supplies model-training code to the data holders (e.g., by injecting compromised code into public repositories as shown in recent real-world incidents [example1<\/a>, example2<\/a>]), does not<\/em> have access to the training process, and has only black-box<\/em> query access to the resulting model.<\/p>\n

\"\"<\/p>\n

Under this setting, we demonstrate a new form of membership inference attack that is strictly more powerful than prior art.<\/p>\n

The core idea of our attack is to exploit the model\u2019s memorization capacity<\/strong>, and strategically have the model memorize an additional set of secret (synthetic) samples, whose outputs can be leveraged to encode the membership of the training samples.<\/p>\n

Our attack highlights four key features<\/strong>:<\/p>\n

\"\"<\/p>\n