Survey: Secure Machine Learning

Conducted by Fraunhofer AISEC, Fraunhofer SIT, and FU Berlin

Are you dealing with machine learning (ML) in your work-related or private projects? In this case, we kindly ask you to participate in our short survey that aims to assess the awareness about implementing secure ML systems. With 15 minutes of your time, you can support a large research-community in advancing in the field of security for ML.

The survey is conducted by the Fraunhofer Institute for Applied and Integrated Security AISEC in cooperation with the Fraunhofer Institute for Secure Information Technology SIT, the National Research Center for Applied Cybersecurity ATHENE and the Freie Universität Berlin (FU).

For further questions please contact securemachinelearning@remove.this.aisec.fraunhofer.de


What is the background of the survey?

In recent years, machine learning and artificial intelligence have become major parts of most companies’ business models. They are employed for many different purposes, ranging from recommender systems that facilitate tasks for users, to internal process surveillance systems, to support the companies’ workflows. As new machine learning technologies and tools emerge in a fast fashion, their integration must generally be quickly if companies are to remain competitive. Given this performance pressure, the security of systems is generally not the first priority during the development process.

However, with the growing number of machine learning applications in use, the number of attacks on such systems is also increasing. Such attacks contain simple model exploration, e.g. if a company is providing any kind of API to their model, to sophisticated tempering of the model, including training data poisoning and introducing hidden states that can result in undesired prediction outputs. The consequences for the companies can be severe: image damage and economic loss due to erroneous prediction outputs are just some examples of the broad range of potential repercussions.

Another arising issue is that, with the introduction of the GDPR, companies are required to guarantee protection of end-user data. Since machine learning models are often built on user data, those models contain information about them. In many scenarios, it is not fully understood how individual data records influence the prediction of a model and to what extent the model is disclosing this data when being queried smartly. Hence, to safeguard end-user data, protecting machine learning models is crucial.

This survey aims to analyze the state of the art in implementation of security measures to protect ML systems in companies. We want to investigate to what extend companies are aware of the different types of attacks (both internal and external) that their models are exposed to and what types of protection are used. In addition, we want to assess to what extent the introduction of the GDPR has caused problems for the companies and how far implementation has progressed.