A new competition focusing on machine learning ( ML) systems ‘ security and privacy has just been announced, and we are thrilled to announce it. Many goods and services already heavily rely on machine learning, and this trend is likely to continue. Understanding the security and privacy protections offered by cutting-edge ML algorithms is therefore essential; in fact, this is one of Microsoft’s Responsible AI Principles.
Fundamentally, ML models require training data on. Both public and non-public data can be used to generate this training data, among other sources. If ML models are trained on specialized or domain-specific data, they perform better in many domains. To protect the privacy of the data contributors or the model owner’s intellectual property, this specialized data is frequently not directly accessible to model users. Ideally, having access to an ML model should n’t make it clear which specific data sets were used to train it. Recent research on membership inference has shown that this is n’t always the case, though.
Membership inference: What is it?
A class of threats to ML models that has received a lot of attention is membership inference. The objective of obtaining access to a model is to determine whether the model’s training process was based on the provided data record. A successful membership inference attack may have grave repercussions, depending on the type of training data. A large dataset of emails and company documents, for instance, could be used to train a model for foreseeing the next word in an utterance. Any user of the model could guess candidate sentences and use it to determine whether they were used for model training, indicating that they appeared in the company’s emails or documents if it were susceptible to membership inference. A dataset of actual images from patients at a particular hospital could also serve as the model for categorizing medical images. Users of the model may be able to determine whether a particular person’s images are included in the training dataset by testing their membership inference attack, revealing that they were hospital patients.
Importantly, the attacker might not ultimately aim for membership inference. For instance, an attacker might want to use attribute inference, reconstruct records from training data, or even infer sensitive attributes about those records ( reconstruction attacks ). However, unlike membership inference, where they only need to infer one bit ( member or non-member ), the attacker in these attacks is attempting to learn more about the training data. Therefore, it is clear that a model is resistant to these other, more devastating attacks if we can demonstrate its resistance to membership inference.
What is the process of membership inference?
The scientific literature has shown a variety of membership inference attacks of varying degrees of complexity. For instance, in a straightforward scenario, the model may have overfitted its training data to the point where it produces higher confidence predictions when questioned about training records as opposed to records that it has n’t seen during training. When an attacker realizes this, they can easily query the model using the relevant records, set a confidence threshold, and determine whether outputs that are confidently above the threshold are likely training data. The attacker only needs to be able to query the model and see the results in this situation. However, because the model was deployed to edge devices, the attacker may have access to its internals, which could allow for even more complex attack strategies.
Describe MICO.
A public competition called MICO aims to gather and contrast cutting-edge methods for membership inference. Membership inference against classification models for images, text, and tabular data, as well as a unique Differential Privacy ( DP ) distinguisher category spanning all three domains, are the four distinct tasks that make up the competition. We trained 600 neural network models on various public dataset splits for each task. We offer a collection of problem points derived from the same dataset for each model. The challenge points are made up entirely of members ( i .e., the ones used to train the model ) and non-members. Participants ‘ objective is to identify which of these challenge points are members and which are not. All models are completely accessible to participants, giving them unrestricted arbitrary queries and the ability to examine the models ‘ parameters. The strongest attacker capabilities are represented by this.
There is no risk to any private or personal data because all of our models were trained on widely used public datasets. Microsoft’s open source and ethical AI guidelines have been followed when reviewing this competition.
What is my participation?
Please go to Git Hub’s main MICO competition page. Links to the four different tasks can be found there. These are hosted on the CodaLab platform, which we use to manage submissions and scorekeeping. Each task in the GitHub repository also comes with a” starting kit” notebook that shows how to download the competition data, conduct an easy membership inference attack, and submit your results to CodaLab.
Each task will receive a separate score in order to appeal to the largest audience possible. This implies that you are free to take part in as many or as few tasks as you like, regardless of how well you perform.
Scoring, accolades, and winners
- The contest will continue through January 12th, 2023 ( 23 :59 Anywhere on Earth ).
- Throughout the event, a live scoreboard based on some evaluation data will be shown. A different subset of the data will be used to determine the final scores.
- Each task’s winner will be eligible for a$ 2, 000 USD award, and the task runner-up will receive an award of$ 1,000 USD ( these awards may be modified if there are tied entries ). MSRC is the sponsor of these awards.
- The IEEE Conference on Secure and Trustworthy Machine Learning ( SaTML) 2023 will host this competition. At this conference, the winners will be given the opportunity to share their tactics.
What are the competition’s objectives?
There is a sizable body of scientific literature describing different membership inference attacks ( and defenses ), but there is currently no standard by which to compare and evaluate these various methods. Offering this benchmark dataset is one of our competition objectives. Given that our dataset contains 2,400 trained models with a combined size of more than 400 GB and an estimated training period of 600 GPU hours, this task is not trivial. We are fortunate to have the means to produce such a dataset, and we anticipate that it will be advantageous to the research community overall. We intend to make the entire dataset, along with the challenge point labels and training scripts, available for use once the competition is over.
In general, we think that public competitions like MICO play a significant part in establishing best practices and even future digital privacy standards. In many different fields, public competitions are already well-established. For instance, they are used by organizations like NIST to standardize and evaluate cryptographic algorithms. Public competitions to improve the state-of-the-art model performance on various tasks and datasets are a thriving tradition in machine learning. Similar value is seen in advancing the field of reliable machine learning through competitions. The first step in achieving this goal is having a common benchmark for evaluating attacks; the second step is to gather, contrast, and discuss the most recent methods in this area. We encourage and welcome you to take part in MICO for these reasons.
Ahmed Salem, Giovanni Cherubin, Santiago Zanella-Béguelin and Andrew Paverd from Microsoft and Ana Maria Cretu from Imperial College London are the organizers of MICO.