Common Component in Black-Boxes Is Prone to Attacks

Abstract

Neural network models are getting increasingly complex. Large models are often modular, consisting of multiple separate sharable components. The development of such components may require specific domain knowledge, intensive computation power, and large datasets. Therefore, there is a high incentive for companies to keep these components proprietary. However, when a common component is included in multiple black-box models, it could potentially provide another attack vector and weaken security. In this paper, we present a method that “extracts” the common component from black-box models, using only limited resources. With a small number of data samples, an attacker can (1) obtain accurate information about the shared component, stealing propriety information of the intellectual property, and (2) utilize this component to train new tasks or execute subsequent attacks such as model cloning, class inversion, and adversarial attacks more effectively. Comprehensive experiments demonstrate that our proposed method successfully extracts the common component through hard-label and black-box access only. Moreover, the consequent attacks are also effective against straightforward defenses that introduce noise and dummy classifiers.

A Evaluation on Time Series Audio Data and Speaker Classification

The proposed strategy is generic and can be applied on different types of data and neural network architecture. We repeat a similar set of experiments as Sect. 4 on audio data and voice classification tasks.

1.1 A.1 Dataset

This evaluation was done using LibriSpeech [29] dataset. The version we use contains 100 h of English speech from 251 unique speakers. We use 100 speakers to train the victim embedding classifiers and use 100 speakers to attack the victims. The remaining speakers are reserved for analysis.

1.2 A.2 Model Setup

We choose a SpeakerNet model [6] trained on development set of VoxCeleb2 [24] which contains 145,569 voice recordings of 5,994 speakers with data augmentation, as the victim embedder. The weights are directly obtained from GitHub repository³. For embedding classifiers, we also use simple models with only two fully connected layers and same train and test splitting as Sect. 4. Similar to our evaluation on image dataset, we also test a set of different combinations of settings for victims. During the attack, to construct the tree-like substitute, we use the ResNet34Half [13] as the trunk and shallow fully connected networks as branches.

1.3 A.3 Attack Process

We query the victim classifiers as black-boxes. We use 9,061 unlabeled voice recordings from 100 speakers which have no overlap with the training data of victims for the attack. The number of recordings we use is around 6.22% of the original dataset used to training the victim embedder and the number of speakers we use is around 1.67% of the original dataset. For all combination of settings, we train 100 epochs and save the best models.

1.4 A.4 Benchmark of Embedder Extraction

Clustering Capability of Embedder (Q1). We visualize the clustering capability of the extracted embedders using 964 voice recordings of 10 speakers from a separate testing dataset. Here we are using the embedders extracted from victims with 10 classes. The embeddings generated by the original embedder and extracted embedder are projected to 2D space using t-Distributed Stochastic Neighbor Embedding (t-SNE) [14]. We also try training from scratch using the same amount of data we used to query the victim model. However, the amount of data is too little to generate any meaningful result.

In Fig. 6, we can see that embedders extracted from multiple victim classifiers indeed have significantly better clustering capabilities. The embedder extracted from a single victim performs poorly.

Fig. 6.

Comparison of capabilities in clustering voice of 10 testing speakers. Each color represents a speaker.

Degree of Distance Preservation (Q2). Here we did a similar experiment as in Sect. 4.4 to compute ratio of pairwise distances among embeddings generated by both the extracted embedders and victim embedder. We use 964 voice recordings of 10 speakers for this experiment.

Fig. 7.

Ratio of pairwise distances among embeddings generated by victim embedder and by extracted embedders.

In Fig. 7, we plot the distribution of distance ratio for 3 embedders, each extracted from 1, 5, 10 victim classifiers of 10 classes respectively. Extracting from more victim models yields much smaller dispersion, indicating the distances are better preserved.

1.5 A.5 Performance in Attack Scenarios

Training a Composite Model for New Task (S1). We evaluate the performance of the extracted embedders when used in new voice classification tasks. Here we use the embedders we extracted from victims with 10 classes. They are the models visualized in Fig. 6(a)(b)(c). In Table 8, we can see performance of the embedder increases with the amount of victims available for extraction, and decreases with the number of classes.

Table 8. Accuracy of classification when using extracted embedders to create embedding.