Hetu: A P2P Communication Layer With Causality Graphs
Summary
TLDRJalen Lee, an assistant professor at NUS, introduces the HE2 protocol, a novel peer-to-peer communication layer designed for large-scale decentralized applications. HE2 offers a middle ground between traditional insecure P2P networks and secure but slow blockchains, providing trusted event ordering and verifiable logical clocks to ensure consistency and security without sacrificing scalability. The protocol leverages zero-knowledge proofs and incremental verifiable computation to create a community-driven platform with a flexible consistency model, suitable for various applications like social media, gaming, and decentralized infrastructure.
Takeaways
- 🎓 Jalen Lee, an assistant professor at NUS, is discussing the HAT2 protocol at V Labs.
- 🔗 HAT2 aims to generalize distributor consistency for large-scale decentralized applications.
- 📈 There's a spectrum of consistency levels and trust in peer-to-peer infrastructures, with limited options available.
- 🚀 Traditional TCP/IP and Li P2P offer fast communication but lack security and consistency.
- 🔒 Blockchains and permissioned BFT protocols provide strong security but are slow and don't scale well.
- 🌟 HAT2 offers a middle ground with low cost, high scalability, and various consistency levels.
- 🕒 The protocol leverages a novel verifiable logical clock for trusted event ordering.
- 🔄 HAT2 is a peer-to-peer communication layer that allows for building distributed applications with different consistency guarantees.
- 🔄 The verifiable logical clock uses zero-knowledge proofs and incrementally verifiable computation to ensure trust.
- 🛠️ HAT2 stack consists of a messaging layer, a shim layer, and an application layer for community-driven development.
- 📅 The HAT2 team plans to launch a testnet in mid-year and mainnet by the end of the year or early next year.
Q & A
What is the main focus of the HAT2 protocol?
-The HAT2 protocol focuses on generalizing the notion of distributor consistency for building large-scale decentralized applications, providing a middle ground between traditional peer-to-peer networks and blockchain technology in terms of consistency and security.
What are the limitations of traditional TCP/IP stack or Li P2P in terms of security and consistency?
-Traditional TCP/IP stack or Li P2P provides fast networking and communication between peers but lacks significant security and consistency, offering only basic security features like end-to-end encryption and message integrity.
How does the HAT2 protocol address the scalability issues of blockchain technology?
-HAT2 protocol leverages a novel verifiable logical clock to provide trusted event ordering at a low cost and with high scalability, allowing for the development of distributed or decentralized applications without the scalability bottlenecks of blockchains.
What is the significance of the verifiable logical clock in the HAT2 protocol?
-The verifiable logical clock is a key component of the HAT2 protocol that allows for the establishment of causal ordering of events in a distributed system without relying on physical clocks. It ensures that the logical causality can be verified by any third party, preventing malicious users from creating false causal dependencies.
How does the HAT2 stack structure its layers?
-The HAT2 stack is structured into three layers: the messaging layer for creating and verifying proofs for logical clocks, the shim layer for building middleware like relay networks and sequencers, and the application layer where developers can propose and build various applications using the logical clocks.
What is the HAT Improvement Proposals (HIP) system?
-The HAT Improvement Proposals (HIP) system is a community-driven approach that allows anyone, not just the core development team, to propose improvements or new components for the HAT2 protocol. It is similar to the EIP system used in Ethereum.
What are some potential applications of the HAT2 protocol?
-Potential applications of the HAT2 protocol include decentralized mutual exclusion, causally and eventually consistent datastores, virtual machines, and shared sequencing layers. It can also be used for social applications, decentralized games, and other interactive applications requiring a balance between consistency and scalability.
What is the timeline for the HAT2 protocol development?
-The current proof of concept (POC) is being developed, with plans to launch a testnet in the middle of the year, followed by a mainnet release by the end of the year or early next year.
How can developers contribute to the HAT2 protocol?
-Developers can contribute by proposing improvements or new components through the HIP system. Once approved, their contributions can be merged into the HAT2 stack. They can also develop applications using the provided APIs and share their ideas with the community.
What are the challenges of maintaining a common physical clock in a distributed system?
-In a distributed system, maintaining a common physical clock is challenging due to the absence of a shared reference frame, as different nodes may have different relative motions (time dilation) and clock drifts, making it difficult to synchronize clocks accurately.
How does the HAT2 protocol ensure consistency without a common physical clock?
-The HAT2 protocol uses a logical clock based on causal ordering of events, which does not rely on physical time but instead tracks the causal dependencies between events. This allows for a consistent ordering of events without the need for a common physical clock.
Outlines
🎤 Introduction to Hat Protocol
Jalen Lee, an assistant professor at the National University of Singapore, introduces the Hat2 protocol. He discusses the challenges of building large-scale decentralized applications with existing infrastructures, which either offer high speed with minimal security or strong security with limited scalability. The Hat2 protocol aims to provide a middle ground, offering more consistency and security options for developers.
🔄 Event Ordering in Distributed Systems
The importance of event ordering in distributed systems is highlighted, with traditional methods relying on physical clocks. In decentralized systems, a common physical clock is absent, leading to the concept of causal ordering. Lamport's work on logical clocks and event ordering in distributed systems is referenced, emphasizing the need for a trusted way to order events without a physical clock.
🔒 Verifiable Logical Clock
A new form of verifiable logical clock is introduced, which allows for event ordering in a peer-to-peer environment with potential adversaries. This clock uses zero-knowledge proofs and incrementally verifiable computation to ensure that the logical clauses can be verified by any third party, preventing false logical clocks and maintaining causal dependencies.
🌐 HE2 Stack and Community Approach
The HE2 stack is described as a peer-to-peer communication layer that provides trusted event ordering. It consists of a messaging layer, a relay network, and an application layer with a simple API. The HE2 protocol is community-driven, allowing developers to propose improvements. The current proof of concept includes a verifiable logical clock, a peer-to-peer communication library, and demo applications like Z Social. The timeline for the HE2 stack includes a testnet and mainnet release in the coming year.
Mindmap
Keywords
💡Distributed Consistency
💡Decentralized Applications
💡Peer-to-Peer (P2P) Infrastructure
💡Verifiable Logical Clock
💡Zero-Knowledge Proof
💡Incrementally Verifiable Computation (IVC)
💡Causal Consistency
💡Hat2 Protocol
💡Community-Driven Approach
💡Proof of Concept (POC)
Highlights
Introduction to the hat2 protocol for building large scale decentralized applications.
Challenges in achieving consistency and security in peer-to-peer infrastructures.
The spectrum of consistency levels and trust in infrastructures, from traditional TCP/IP to blockchains.
The need for a middle ground between fast but insecure networks and slow but secure blockchains.
HE2 protocol as a peer-to-peer communication layer offering trusted event ordering.
Utilization of a novel verifiable logical clock for event ordering in a decentralized environment.
The concept of causal ordering of events in distributed systems without a common physical clock.
The development of a new logical clock that doesn't depend on physical clocks but on causal dependencies.
The application of zero-knowledge proofs for incrementally verifiable computation in building a trusted logical clock.
Examples of applications that can benefit from the verifiable logical clock, such as decentralized mutual exclusion and causally consistent datastores.
The structure of the HE2 stack, including the messaging layer, shim layer, and application layer.
The community-driven approach of the hat2 protocol, allowing developers to propose improvements.
Proof of concept development and the roadmap for the HE2 stack, including a testnet and mainnet release.
Invitation for community contribution and feedback, with team members present at the event.
Transcripts
[Music]
all right um so yeah today I'm very
excited to uh share with you guys uh our
hat to protocol um so forgive me I'm uh
I just recently got a pretty bad code so
my voice is a bit like a bit shaky uh
but hope hope everybody can hear me is
it good oh good okay cool all right so
uh first a quick introduction uh so my
name is Jalen Lee uh so I'm currently an
assistant professor at the National
University of Singapore and also I'm
representing uh at the V Labs uh to uh
to tell you more about our uh very
recent hat2 protocol uh where we try to
generalize uh this generalized notion of
distributor consistency uh for building
uh large scale decentralized
applications all right um so let me
first start with I guess all of you
heers are
uh our developers uh in the web stre
space so uh you you guys use or have
been uh developing infrastructure or use
large scale peer-to-peer
infrastructures um but so far there are
very limited uh number of options in
terms of what kind of consistency you
get uh or kind of security you get from
this infrastructure right so I here I
gave you sort of like a graphical
illustration the kind of spectrum of
consistency levels or the kind of trust
you get from this
infrastructures so on the very uh left
far end of the spectrum you get things
like traditional TCP IP stack or Li P2P
or things like that where you can build
very fast um very fast networking right
uh communication between different peers
in a uh in a peer-to-peer Network right
but the problem is that you don't really
get a lot of security or consistency
from these infrastructures right so you
kind of get uh the bare minimum kind of
security if you do end to endend
encryption right so that gives you
things like authent authentication
Integrity of your messages but you don't
get much more than that right uh so if
you build distributed applications or
decentralized applications would require
a lot more security or consistency uh
instead what you do is you choose the
far right end of the spectrum where you
use probably a blockchain use a uh
permission bft protocol or you use any
other form of infrastructure like Layer
Two rops reaking and other forms of
instructure that gives you very very
strong security right so you have a
totally ordered uh uh set of blocks
right so everybody agrees on the order
so there's Global consensus right uh so
security wise is really good uh it's a
very strong consistency uh but the
problem is it's very slow it doesn't
really scale right so for many kind of
applications it's not probably not ideal
to use this infrastructure as well right
but we can see that this is a spectrum
right so you don't only have two end
points there's a lot of ground in the
middle right so all these middle uh for
Security Options and consistency options
they're currently not a lot of uh for
you infrastructure wise for to use right
um but actually we see that there are a
lot of applications which can actually
benefit from a more uh slightly more
relaxed level consistencies or trust
models um to to scale them right so here
I just gave you a sense of uh example
application but here definitely there
are not a uh exhaustive list right so
here things like social applications
right things like communication between
different AI agents right uh like Games
social games um uh and more recent like
uh uh decentralized physical
infrastructure and so on so forth right
so all these applications right they are
so they're they're proper one of the
main properties that very interactive
right so there are a lot of interactions
between different parties in the network
a lot of communications going on right
so putting them on chain is probably not
the best option because of the
scalability bottleneck so even though
you have more scalability options these
days uh but eventually because the cost
and then there's still limited
throughput and scalability is still not
ideal for you to to to build these
applications on top on the other hand
these application do require some level
of security and trust and also
consistency right so for example if
you're building social applications you
want to have some kind of consistency so
that your followers right your tweets
right so there are there there there the
follow your followers right so these are
your your your uh your metadata changes
there there are some kind of consistency
guarantees right so so there's so at
this moment you either choose right the
uh the very uh very uh so infrastructure
that they don't give you anything or you
you choose like blockchains uh so which
are fairly Limited in terms of
scalability right uh so in this talk so
we're going to introduce HE2 uh which
does provide this kind of Middle Ground
where there are offers a lot more kind
of consistency and Security Options for
you to build distributed or
decentralized applications right so HE2
is a peer-to-peer communication layer
right so but uh so compared to existing
peer-to-peer infrastructures or
communication infrastructure uh it
additionally provides this what we call
trusted event ordering uh naturally from
this communication layer right uh and
then underlying it we are leveraging a
novel um form of verifiable logical
clock uh which we I'm going to elaborate
uh in a few minutes um so the so the so
the so the nice thing about HE2 is that
it is very low cost and then very highly
scalable um but also it gives provides
the opportunity for distributed or
decentralized application to build
various consistency levels on top of it
uh and then so it has really really wide
kind of applic applicability right so
all the applications I just show you uh
will will benefit from our communication
stack so let me just elaborate by
starting from a little bit background
right so uh so when you build any kind
of applications distributed systems
right so the order of events is a really
important aspect right so no matter what
you do you care about some kind of
ordering right either is like between
different blogs or not between a uh a
social social media post and then some
other activities right so these are all
events and you care about the ordering
so how do you order events in a system
right so the most traditional way so for
example if you come to uh this e Denver
event where you look at all the events
uh in this in this uh in on the schedule
so you clearly Mark right what is what
time is each event happening right so
that's the most traditional way of
ordering events right so my talk starts
at 1:45 and then the previous stock
start at like 1 uh 1 p.m. or 1:30 p.m.
or something right so by looking at the
different time span time stamp you can
know exactly which event happened first
so but the problem is that in a truly
distributed or decentralized system this
kind of physical clock or commonly
agreed physical clock is not present so
there are multiple reasons why you don't
have that in a decentralized
system so first from a theoretical
perspective uh if you guys uh have heard
or learned about relativity right so in
relativity right so different objects uh
which have different relative motions
they actually don't have the same common
clock right uh so this phen phenomena is
called time dilation so for example uh
here I show uh on the left is the
international uh space uh station right
so and then so because it's moving with
a relative motion with for example this
is the location of e dver right because
if if there is relative motion so the
time actually disagree with each other
right so they don't have the common
time so the other problem is that even
if you have if you don't have relative
motion among different objects because
your clocks devices they are not perfect
right so they cannot track your physical
time um very accurately right um so for
example depending on what kind of clock
technology you use if you use U small
atomic clocks right versus very uh so
sorry very sophisticated atomic clocks
versus a simple clock you can find your
computer like a crystal oscillator so
they have very different clock drifts
right so it's very hard for you have to
have perfectly synchronized clocks among
different
parties right so if we can't have like a
a common agree upon physical clock what
can we do to have uh to to order events
in a distributed system so here we look
at uh look back in the literature in
distributed systems so back in 1978 uh
so a famous distributed system
researcher uh his name is Lassie Lamport
uh who later got theer award for some of
this work including this so uh he
published a seminal work called time
clocks and The Ordering of events in
distributed system where he exactly talk
about this problem so if you don't have
physical clock in the system how can you
order events so he argued that the only
trusted way of ordering events in a
distributed system is to look at The
Logical ordering of different events or
another way of saying it is called the
Cal causal ordering of events in a
system so what is causal event ordering
in a system so here I show if you have
multiple processes in a distributed
system right so they talk to each other
through messaging right so you don't
have a clock so you don't know which
event happened first but you know for
sure that if I send the message from
process P to process Q the sending of
the event must happen before the
receiving of the message now I establish
a causal dependency between these events
and then you can use this if I I receive
a message and later send another event
uh and sorry I send another message I
know the sending of the next message
causally depending on I received the
previous message right because the
sending May cly depend on actually
receive the previous message right and
then you can do a transitive closure of
these events then you get this what they
called a happened before relationship
which captured the causal orological
dependency between different events so
that you don't require any physical
clock and then that is the true ordering
of the the system that nobody can
disagree with right so that's a pretty
cool and a pretty strong kind of uh uh
event ordering property and then uh
let's lample argue that now we can
develop a new kind of clock that doesn't
depend on physical clock but instead on
a logical clock this logical clock
essentially track this causal uh causal
dependency between different events by
looking at this causal clock sorry
logical Clause you can infer what's the
causal dependency between different
events right so so this paper and the
notion of logical clock on causal
dependency have a lot of implication a
huge implication on the development of a
lot of distributed and decentralized uh
protocols to this state um so we are uh
taking this protocol and this notion a
step forward so we are looking at how do
we build logical clock in a
decentralized environment right so a
peer-to-peer environment one of the
biggest challenges in this peer-to-peer
envir environment is the presence of
potentially malicious or adversaries
users right so pantin knows or another
way of saying it right so because
traditional logical clock depends on all
the protoc all the all the participant
in the system follow exactly the
protocol of updating The Logical clock
right so if there are any pantin node in
the system it will break the break the
entire notion of logical clock and the
caal dependency guarantees right so what
we do is we have done some uh recent re
research and and then we found or we
developed a new way a new logical clock
called verifiable logical clock uh that
ensures that the The Logical Clause pass
in the system can be verified by any
third party right so that means nobody
can can uh create a false logical clock
that breaks Lo uh causal dependencies
right and then so I mean I'm not going
to have the time to give you all the
technical background uh but a hell of a
gist is that we leverage this new in
noal application of zero knowledge proof
so this is very recent uh development in
the space uh so this new technique is
called incrementally verifiable
computation or IVC that allows us to
have multiple parties in the system by
passing these messages build a trusted
logical clock with a verifiable
proofs right and by applying this uh
this novel kind of verifiable logical
clock we're able to build a lot of
interesting applications and
infrastructure on top of it here I just
give you a some quick uh examples that
we recently developed uh but you can
certainly think of using this verifiable
logical clock for a lot of more
interesting applications right so for
example we can use uh this verifiable
logical clock to build decentralized
Mutual exclusion or the other way of
saying is like m a a lock right uh a
synchronization primitive between
different participant in the system
without centralized trust or centralized
server for log Services right and you
can build as I I hinted in the beginning
of the talk right so there are a lot of
application that I want something uh
consistency levels not that are not like
strictly consistent right not
linearizable in a more technical sense
right so here we are able to use this uh
verifiable logical clock to build
causally and eventually consistent data
stores and then uh uh things like
virtual machines right so your virtual
machine State doesn't need to be always
synchronized across each other but we
guarantee either eventually synchronized
or if there's a c dependency between the
different updates we make sure everybody
observed them in a causally consistent
order right and then we can use this
logical verifiable logical clock to
build a shared sequencer for example
right here we're able to use the partial
ordering property of this verifi logical
clock to easily scale a shared
sequencing
layer right uh and then so so far we
have built a so from this verif a
logical clock we build a HE2 stack uh so
this HE2 stack as I mentioned before is
a peer-to-peer communication layer right
um so sorry that the the the picture
might be a little bit blurry and small
uh So currently we we can structure as
three three kind of layers so the first
layer is underlying the the lowest layer
we have this messaging layer where um so
we can the messaging layer creates the
uh the proof and then generate generate
proofs and then verify uh the for the
for this logical clocks and verify this
logical clock while generating messaging
and then this layer also uh uh build
build a traditional p-to-p communic
layer things like cadamia uh DHT right
and then on top of it we have a a middle
what we call the shim layer where we
build a sort of like a a relay Network
where there's a large scale uh
peer-to-peer network with a lot of
participants messaging each other uh
with uh the underlying shared sorry
underlying a verifiable logical clock in
this each messages and on top of it
expose a simple to use API to the
application layer where the application
can use uh the The Logical clocks from
the underlying layers to build their uh
desired consistent levels and their
applications so that's the current hat2
protocol uh hat to stack uh and then we
are a very Community Driven approach
right so so we uh uh we develop a fairly
interesting and pretty novel uh protocol
called the hat to Improvement uh
proposals this is like the EIP system
right where everybody not just the our
core development team can propose um
different Improvement on top of K to
protocol or a components of the hatter
protocol right so here we gave you some
we roughly coarsely developed into three
different layers so the first layer is
called hiip a layer so this is the like
the fundamental layers of the the HE2
protocol so here so we build the
fundamental technology behind HE2 things
like logical clocks uh the peer-to-peer
Network and then uh the incrementally
verifiable verifiable computation right
and then on top of it there is the hip B
layer where we there's the middle uh
middleware that people can build things
like the relay Network I mentioned about
and also things like sequencers and then
random number generators and then there
are many developers who care more about
the upper level applications and then
they can propose uh things like
Messengers for social applications uh
decentralized games or things like
social other kind of social applications
on top of it right so we open up uh this
hip system to all the developers in the
system uh in the in the community right
everybody can propose right and then
once it's approved uh they can be merged
into uh the Hato
stack right so currently we have a proof
of concept right so if you're interested
feel free to check uh check out our uh
GitHub page right so currently we have a
few of these components currently
already built things like The verifiable
Logical clock so the underlying
peer-to-peer communication Library uh
and then we also developed some a few
applications on top of it already as a
demo right things like uh a social
application called Z social and then a
random number
generation right so um so this is our
prep proposed timeline for our uh HE2
stack and HE2 protocol uh so I don't
know if you guys can read it but we are
currently roughly here so we're uh we
develop our current propal concept POC
and then our goal is to have the test
net some sometime uh in the middle of
this year right so which is here uh and
then later this year and by the end of
this year or early next year where our
plan is to row out the main net of hat2
uh so we welcome everybody to uh
contribute uh to talk to us give us
feedback
um so uh so many of our team members are
here at East Denver so if you are
interest in our protocol or our stack
feel free to talk us uh so uh so aasha
is here in this I think somewhere here
uh that's me and then I think Sean is
also here and then we have many other
people here from the uh the hat2 team uh
and then uh also uh follow us on Twitter
uh and then we welcome all your feedback
and then yeah I think that's it and
thanks everybody
bu
Weitere ähnliche Videos ansehen
How We Did It: The First ZK Proof on Bitcoin - Edan Yago & Gadi Guy at Bitcoin Nashville #Bitcoin
What is the Internet Computer (ICP)? A beginner's explanation!
The Alephium Ecosystem Rundown
PERPETUAL PROTOCOL - Next Level In Decentralized Trading? (Layer 2, Uniswap V3)
Fluence: Cloudless DePIN Computing Platform
Verifiable Compute for AI, ML and More: Web3's New Frontier | Alison Haire at SmartCon 2023
5.0 / 5 (0 votes)