NDSS 2024 - Timing Channels in Adaptive Neural Networks

Summary

The speaker, Akin, presents research on timing channels in adaptive neural networks, a joint work with Professor Tan Brenan. They explore how variations in an application's runtime can lead to timing side channel attacks, potentially leaking secret information. The study focuses on adaptive neural networks, which unlike traditional ones, process inputs differently based on their complexity, leading to varying computation times. This can result in timing profiles that correlate with sensitive attributes, such as class labels in a dataset. The research demonstrates that an attacker can generate a timing profile of the model using a manifest dataset and then train an attack model to infer sensitive attributes from timing measurements. The findings show high attack success rates, indicating a privacy vulnerability in adaptive neural networks. The study also examines the impact of hyperparameter tuning on the trade-off between accuracy, efficiency, and privacy. The results are available on GitHub for reproduction.

Takeaways

• 🕒 **Timing Channels in Adaptive Neural Networks**: The talk discusses how variations in an application's run time can leak secret information through timing side channels, specifically in the context of adaptive neural networks.
• 🔍 **Side Channel Attacks**: The presentation covers side channel attacks, including timing side channels, which exploit nonfunctional characteristics of applications to steal secret information.
• 🤖 **Adaptive Neural Networks**: Adaptive neural networks are designed to classify inputs early if the model is confident enough, thus saving computational resources.
• 📉 **Correlation Between Inference Times and Exits**: The study found a strong correlation between the inference times of the adaptive neural network and the different exits in the network.
• 🔑 **Leakage of Sensitive Attributes**: The research demonstrated that timing side channels can leak sensitive attributes, such as class labels, from the adaptive neural network.
• 📈 **Attack Success Rate**: The attack success rate metric was used to measure the accuracy of the model in inferring sensitive attributes from timing measurements.
• 📊 **Cluster-Based Analysis**: The researchers identified different timing intervals (clusters) that correlated with specific exits and had high attack success rates, indicating potential privacy violations.
• 🌐 **Experiments Over Networks**: The study included experiments conducted over the public internet and a local area network, showing that even with network latency, significant attack success rates could be achieved.
• 🛡️ **Defense Against Timing Attacks**: The paper suggests that hyperparameter tuning, such as adjusting exit thresholds in adaptive neural networks, can be a trade-off between accuracy, efficiency, and privacy.
• 📚 **Datasets and Models**: The research used different datasets like cancer and Cat 10 to train adaptive neural networks and tested the impact of these datasets on the attack success rate.
• 📉 **Impact of Network Optimizations**: The discussion acknowledged that cloud providers' optimizations, which can affect inference timing, might complicate the predictability of attack success in cloud environments.

Q & A

What is the main focus of the talk presented by Aid Akin?

The main focus of the talk is on timing channels in adaptive neural networks, specifically how variations in an application's run time can allow an attacker to leak secret information about the application.

What are side channels and how are they related to the talk?

Side channels are nonfunctional characteristics of an application, such as the time it takes to run or the power it consumes. They are related to the talk as the speaker discusses how timing side channels can lead to privacy violations in adaptive neural networks.

What is an adaptive neural network and how does it differ from a conventional neural network?

An adaptive neural network is a type of neural network that can vary the amount of computation based on the complexity of the inputs. Unlike conventional neural networks that treat all inputs the same, adaptive neural networks can differentiate between complexities and thus achieve lower computational costs and faster inference times.

How can timing side channels leak confidential information?

Timing side channels can leak confidential information by allowing an attacker to correlate the inference times of an adaptive neural network with the different exits in the network, which can reveal sensitive attributes of the input data.

What is the significance of the 'exit' in adaptive neural networks?

In adaptive neural networks, an 'exit' refers to a point at which the network can classify an input without needing to process it through the entire network. The choice of exit can indicate the complexity of the input and is used to adjust the amount of computation required, thus affecting the inference time and potentially leaking information through timing side channels.

How does the attacker generate a timing profile of the model?

The attacker generates a timing profile by using a manifest dataset, which is a dataset that belongs to the same distribution as the original dataset the model was trained with. The timing profile is a mapping of timing measurements for input samples to their sensitive attributes, such as class labels.

What are the two major questions the speaker aims to answer in the talk?

The two major questions are: 1) Is there a correlation between the inference times of the adaptive neural network and the different exits in the network? 2) Is there a correlation between any attributes in the input data to the adaptive neural network and any of the exits in the network?

What metric is used to evaluate the success of the attacker's model?

Two metrics are used: the attack success rate, which corresponds to the accuracy of the model in inferring the sensitive attribute of a user's input given their timing measurement, and the attack success rate per cluster, which corresponds to the accuracy of the attack model to infer the sensitive attribute of a user whose timing measurement falls into a particular time cluster.

How does hyperparameter tuning affect the tradeoffs between accuracy, efficiency, and privacy in adaptive neural networks?

Hyperparameter tuning, particularly the exit thresholds in early exit networks, can result in tradeoffs. Strict thresholds (conservative setting) favor accuracy over efficiency, potentially leading to higher attack success rates but reduced efficiency. Looser thresholds (relaxed setting) can improve efficiency but may reduce accuracy and the ability to exploit timing side channels for attacks.

What are the implications of unstable network delays on the effectiveness of the attack?

Unstable network delays can introduce noise into the timing measurements, potentially reducing the accuracy of the attack. However, even with delays, significant clusters with good attack success rates can still be found, suggesting that the attack strategy remains effective to some extent.

How do the results generalize when considering network latency, especially in cloud environments?

The results may generalize with some challenges due to cloud environments' optimizations and scheduling tools that can delay inference tasks. This variability can complicate timing measurements and make it harder to predict when results are returned, potentially affecting the attack's effectiveness.

Did the researchers conduct experiments with non-adaptive networks to compare the effectiveness of timing side channel attacks?

Yes, experiments were conducted with non-adaptive networks, and it was found that they did not have clusters with significantly high attack success rates, indicating that the timing side channel effect is more pronounced in adaptive neural networks.