Unable to decrypt this message - Kegan Dougal

Summary

This video discusses the challenges and technical complexities involved in end-to-end encrypted messaging, specifically focusing on Unreadable Text Detected (UTD) issues. The speaker explores scenarios where synchronization failures between devices, SDK bugs, and server-client issues cause UTDs. Examples include device lists being out of sync, partial key transfers in large group chats, and platform-specific issues like iOS multi-process coordination failures. The talk also highlights the importance of thorough testing using tools like Complement Crypto to identify and resolve UTDs, emphasizing the need for precise handling of encryption, retries, and state resolution across diverse platforms.

Takeaways

• 😀 End-to-end encryption issues can arise from sync problems between devices and servers, leading to Unencrypted Transmission Detected (UTD) errors.
• 🔑 A common issue in encryption failures is the mismatch between the public keys used for encryption and the private keys required for decryption.
• 📱 When a user logs in on a new device, their devices may not immediately synchronize with others, causing encryption failures for messages sent before synchronization.
• ⚙️ Client-side SDK issues, such as improper handling of device invitations or group chat encryption, can cause UTDs if the encryption keys aren’t correctly synchronized or sent.
• 🚦 SDK development requires careful attention to timing and retries, as partial transmission failures of encryption keys (e.g., in large groups) can lead to UTDs.
• 💻 Hardware limitations (e.g., using underpowered servers) can impact the ability to process encryption key requests, leading to failures in large group chats.
• 📲 In iOS clients, process synchronization issues between the push notification process and the main app process can corrupt internal states, requiring careful coordination.
• 🛠️ Testing frameworks like Complement Crypto are crucial in simulating different failure scenarios to identify and resolve UTD issues in end-to-end encryption systems.
• 🔄 Ensuring that encryption works reliably in various network conditions (e.g., server outages, network splits) is essential for maintaining secure communication.
• 🔍 The testing framework allows for adversarial testing and the generation of HTTP dumps to diagnose encryption issues and improve overall system robustness.
• 🤝 Coordination between SDK developers is vital for ensuring compatibility across different platforms (e.g., JavaScript, Rust) and improving the reliability of end-to-end encryption across the ecosystem.

Q & A

What causes an 'Unable to Decrypt' (UTD) error in end-to-end encrypted messaging systems?

UTD errors occur when a device is unable to decrypt a message due to mismatched encryption keys or an out-of-sync device state. This can happen when encryption keys are not properly synchronized between devices or if certain keys have not been received yet.

How does device synchronization failure lead to UTDs?

If a new device is added, such as Alice logging into a laptop, the sending device (Bob) may not be aware of the new device, causing encryption keys to be missing for that device. Without synchronization between both devices' states, decryption fails, resulting in a UTD.

Why is it important for both the client and server to be aware of device states in end-to-end encryption?

Both the client and server need to be aware of the device states to ensure that all devices can encrypt and decrypt messages. If a device isn't recognized or synced across both, messages encrypted on one device cannot be decrypted on another.

What role do SDKs play in causing UTDs, and how can they be improved?

SDKs can introduce bugs when they fail to properly manage device synchronization or membership lists. These issues often occur when the client doesn't correctly update the member list after an invitation or device login. Improving SDK development involves handling these synchronization scenarios more carefully and adding retry mechanisms.

What is the significance of 'causality violations' in the context of SDK errors?

Causality violations occur when the order of operations is incorrect, such as sending a message before a device is fully synced with the updated member list. This can cause messages not to be encrypted correctly for all participants, resulting in UTDs.

How do batching issues lead to UTDs in large group chats?

In large group chats, if keys for new devices aren't fully exchanged or if only a partial set of keys is sent due to timeouts or server issues, some devices may not be able to decrypt the message, causing UTDs.

What are some server-side issues that can cause UTDs?

Server-side issues such as timeouts, partial key exchanges, or network failures can prevent devices from receiving the necessary keys to decrypt messages. Underpowered servers or misconfigured networking setups may exacerbate these issues.

How can issues related to device state corruption be avoided in mobile clients, specifically iOS?

In iOS, device state corruption can occur when two processes (e.g., push notification and main app) are not properly synchronized. This issue can be addressed by implementing cross-process locks to ensure that the device state is handled consistently across processes.

What testing methodologies are used to detect UTDs and improve encryption reliability?

Testing methodologies include using frameworks like Compliment Crypto, which simulate real-world scenarios (e.g., network splits, server failures) to detect UTDs. This testing helps identify issues with key exchanges, device syncing, and other areas where encryption might fail.

How does the Compliment Crypto framework contribute to identifying UTDs?

Compliment Crypto inserts a man-in-the-middle proxy to inspect client-server traffic, test interactions, and block or manipulate requests to simulate potential failures. It also generates HTTP dumps for debugging, allowing developers to see where issues arise in the encryption process.