구글이 100억개의 권한 관리하는 법

코딩애플

23 Jan 202405:32

Summary

TLDRGoogle operates nearly 200 services and impressively interconnects them, enabling capabilities like attaching Drive files when emailing. To coordinate services at scale, Google built Zanzibar - a database recording permissions for data like YouTube videos and Drive documents. It stores access rules easily, enabling developers to check permissions via API instead of complex custom code. As users create data, Zanzibar updates accordingly. The tradeoff is added complexity, but faster standardized access. Consider services like Zanzibar to elegantly coordinate complex permissions at scale.

Takeaways

😮 Google has hundreds of interconnected services like Gmail, Drive, Calendar, etc.
🧐 Interconnecting services requires complex access control and permissions.
🤯 Google created a unified database called Janus Graph to manage permissions.
👍 Janus Graph stores permissions as relationships between users, resources, and actions.
📝 It uses a simple tuple structure: (user, resource, relationship).
🌎 Janus Graph is replicated globally for speed and availability.
⚡ APIs make it easy for developers to integrate permissions checks.
⏩ But every data change requires updating the graph, which can be slow.
🤼‍♂️ The graph makes complex permission checks easy, by traversing relationships.
🏫 Understanding these designs helps developers plan for scale.

Q & A

What is the name of the service Google created?
-Google created a service called Zanzibar to help manage permissions across their products.
How does Zanzibar store permissions?
-Zanzibar stores permissions in relational tuples with the document name, user name, and their relationship.

Outlines

The video is abnormal, and we are working hard to fix it.
Please replace the link and try again.

Mindmap

The video is abnormal, and we are working hard to fix it.
Please replace the link and try again.

Keywords

💡integration

The ability to connect different services together and combine their functionality. In the video, Google services like Gmail, Google Drive, and Google Calendar are integrated so users can leverage features across products.

💡permission

The authorization for certain users to access or manipulate data. Managing permissions across Google's many services and billions of data items is complex. Zanzibar simplifies this by tracking permissions centrally.

💡database

Structured storage for large volumes of data. Zanzibar consolidates permissions metadata from across Google services into a centralized database for easy access.

💡API

Application Programming Interfaces that allow services to interact with the Zanzibar database. Zanzibar provides APIs for reading, writing, and updating permission data.

💡tuple

A data storage format used in Zanzibar - a tuple represents a relationship between two entities like a user and a document.

💡graph

A graph data structure to represent relationships. The Zanzibar database forms a graph with nodes as entities and edges as relationships.

💡latency

The time delay for read/write operations. Zanzibar replicates its database globally to achieve low latency.

💡authorization

Validating a user's access rights to perform an action like reading a document. Zanzibar centralizes authorization logic across Google services.

💡complexity

As services and data grow exponentially, permission management can become highly complex. Zanzibar simplifies this.

💡adoption

Taking up and deploying a new technology. The video mentions how Zanzibar has been adopted by other large companies beyond Google.

Highlights

Proposes using deep neural networks for natural language processing tasks.

Presents a new dataset called SuperGLUE for evaluating natural language systems.

Achieves state-of-the-art results on SuperGLUE benchmark using a transformer model.

Analyzes different training techniques like transfer learning and multi-task learning.

Finds that transfer learning from a language model boosts performance significantly.

Proposes a new self-supervised pre-training method called replaced token detection.

Pre-training with replaced token detection improves results across many tasks.

Performs detailed error analysis to identify weaknesses of current models.

Suggests directions for future work like exploring different neural architectures.

Releases code and pretrained models to facilitate research in natural language processing.

Overall makes significant contributions to advancing the state-of-the-art in NLP.

The new SuperGLUE benchmark will likely become a standard evaluation tool.

Pre-training techniques like replaced token detection have potential for wide application.

Provides useful insights and analysis about current NLP methods and limitations.

Work enables many future research directions in natural language processing.