What is Event Driven Architecture (EDA)?
Summary
TLDRIn this informative talk, Whitney from IBM's Cloud team delves into event-driven architectures, highlighting their role in enabling real-time interactions, scalable microservices, and extensible data analytics. She explains the principles of the reactive manifesto, emphasizing asynchronous messaging, scalability, and resilience in system design. Whitney illustrates how events, as immutable statements of past occurrences, facilitate targeted and conversational messaging between services, leading to a responsive and maintainable system. She further explores the benefits of event logs for triggering actions, optimizing data persistence, and supporting stream processing, all of which contribute to a robust, pluggable, and history-aware system.
Takeaways
- 🌟 Event-driven architectures are designed to enable real-time user interactions, pluggable microservices, and extensible streaming and data analytics.
- 📜 The Reactive Manifesto provides a set of guidelines for building responsive, resilient, and scalable systems through asynchronous messaging.
- 🔄 Scalability in system design means the ability to expand or contract hardware usage in response to workload changes.
- 🛡️ Resilience implies the absence of single points of failure and the capacity to handle system failures gracefully.
- 📢 Asynchronous messaging is central to the Reactive Manifesto, allowing for non-blocking communication between system components.
- 📝 An event in this context is an immutable statement of fact about something that happened in the past, used for communication between services.
- 🔄 Event-driven systems allow for highly scalable architectures, as the event producer is not concerned with the number of consumers.
- 🗃️ Event logs provide a replayable history of events, enabling persistent data storage and the ability to process historical data.
- 🤖 The event backbone is the central system through which all events flow, enabling various applications to produce and consume events.
- 🚀 Event logs can trigger actions, such as serverless functions, which can be modular pieces of code associated with specific events.
- 🔧 Event-driven architectures support stream processing, allowing for real-time calculations, business rule implementation, and analytics.
- 🔌 The publish-subscribe (pub-sub) communication model is integral to event-driven systems, facilitating plug-and-play microservices that can easily integrate into the system.
Q & A
What is an event-driven architecture?
-An event-driven architecture is a system design approach that relies on the production, detection, consumption of, and reaction to events. It enables real-time interactions and is highly scalable and resilient, as services react to events without direct coupling.
What does the reactive manifesto aim to provide in system design?
-The reactive manifesto provides a set of community-driven guidelines intended to offer a cohesive approach to system design, emphasizing principles such as message-driven architecture, scalability, resilience, and responsiveness.
What are the core values of the reactive manifesto?
-The core values of the reactive manifesto include building message-driven, asynchronous systems that are scalable, resilient, and responsive, ensuring the system can handle varying workloads and avoid single points of failure.
How does event-driven architecture enable real-time user interactions?
-Event-driven architecture enables real-time user interactions by allowing systems to react instantly to events as they occur. This is achieved through the use of asynchronous messaging and the ability to trigger immediate actions in response to events.
What is the difference between messaging and eventing in system communication?
-Messaging involves direct communication between services, where the sender has knowledge of the receiver. Eventing, on the other hand, involves the production of events without concern for which service is consuming them, promoting a decoupled and scalable system design.
Why is data persistence in event-driven architectures considered to be more permanent compared to messaging?
-In event-driven architectures, data persistence can be more permanent because events are logged and can be stored indefinitely, depending on the system setup. This creates a replayable event log and a history of all events, unlike messaging where data persistence is often transient.
What is the role of the event backbone in an event-driven architecture?
-The event backbone is the central component of an event-driven architecture through which all events flow. It facilitates the distribution of events from producers to consumers, enabling a scalable and responsive system.
How can event logs be utilized to optimize and customize data persistence?
-Event logs can be used to trigger actions, update local data stores, and produce new streams of data that reflect the most current state. This allows for optimized data persistence tailored to the needs of specific services within the system.
What is the significance of the pub-sub communication model in event-driven architectures?
-The pub-sub (publish-subscribe) communication model is significant in event-driven architectures as it allows for a highly pluggable system. Services can easily produce and consume events without direct coupling, promoting extensibility and scalability.
How does event-driven architecture support the implementation of business rules and real-time analytics?
-Event-driven architecture supports the implementation of business rules and real-time analytics by providing a stream of events that can be processed in real-time. This enables systems to make immediate decisions based on current data and trends.
What is the relationship between event-driven architecture and serverless functions?
-Event-driven architecture can trigger serverless functions, which are modular pieces of code that execute in response to events. Once the function completes its task, it can send another event back into the system, creating a reactive and efficient process flow.
Outlines
🌟 Introduction to Event-Driven Architectures
Whitney from IBM's Cloud team introduces the concept of event-driven architectures, emphasizing their role in enabling real-time interactions, pluggable microservices, and scalable data analytics. She begins with an overview of system design principles from the reactive manifesto, which advocates for asynchronous messaging, scalability, and resilience in system design. The manifesto promotes a design that is responsive, maintainable, and extensible. Whitney then explains the difference between messaging and eventing, using a retail application as an example to illustrate how eventing facilitates communication between various services without a direct dependency, leading to a highly scalable system with potential for persistent data storage and event replayability.
📡 The Event Backbone and Its Capabilities
This paragraph delves into the specifics of event producers in an event-driven architecture, such as web and mobile applications, as well as IoT devices. It introduces the 'event backbone' as the central system through which all events flow. The capabilities enabled by this architecture include the triggering of actions based on events, optimization of data persistence, and the maintenance of an up-to-date inventory through local data stores. The paragraph also touches on the use of event logs for data-heavy applications like AI and the importance of a data lake for storing and accessing raw and transformed data. Additionally, it highlights the potential for stream processing using the Apache Kafka Streams API, which can facilitate real-time recommendations and analytics in applications like music streaming services.
🔌 Pub-Sub Communication and Microservices Integration
The final paragraph focuses on the pub-sub communication layer of event-driven architectures, which allows for the easy integration of microservices. It discusses the plug-and-play nature of these architectures, where new microservices can be added without disrupting existing applications. The paragraph also addresses the ability of new services to consume data from the beginning of the application's history if stream history has been preserved. This feature, along with the overall benefits of event-driven architectures, is positioned as an enhancement to the reactive manifesto's principles for system design, with eventing at its core.
Mindmap
Keywords
💡Event-driven architecture
💡Reactive manifesto
💡Asynchronous messaging
💡Scalability
💡Resilience
💡Event
💡Event log
💡Data persistence
💡Pub-sub communication
💡Serverless function as a service
💡Stream processing
Highlights
Event-driven architectures enable real-time user interactions, pluggable microservices, and extensible streaming and data analytics.
The reactive manifesto provides a cohesive approach to system design with guidelines for message-driven, scalable, and resilient systems.
Asynchronous messaging is a core principle for creating scalable and resilient system designs.
Distributed systems or Kubernetes are implied for scalable and resilient system design to avoid single points of failure.
Responsive systems are valued for their ability to maintain and update easily, as well as their extensibility.
An event is defined as an immutable statement of fact about something that happened in the past.
Messaging models in retail applications allow for direct communication between services like checkout, inventory, shipping, and contact.
Eventing allows for a system that is highly scalable, with the ability to expand to numerous services without affecting the producer.
Event logs provide a replayable stream history, offering a permanent or possibly permanent data persistence.
The event backbone is the central system through which all events run, enabling various applications to produce and consume events.
Event logs can trigger actions, such as spinning up a serverless function in response to an IoT device event.
Stream processing allows for real-time calculations and the implementation of business rules and policies.
Data lakes save all data that comes through the system, powering data-heavy applications like AI.
Pub-sub communication allows for pluggable microservices that can easily integrate into the system without disrupting other applications.
New microservices can consume from the beginning of the application's stream history, providing a complete context.
Event-driven architectures build on the reactive manifesto ideas, offering a responsive and message-driven system design.
Transcripts
What are event-driven architectures? And how can they enable real-time user interactions, pluggable
microservices, and extensible streaming and data analytics? My name is Whitney, I'm on the Cloud
team here at IBM. Before I dig in on event driven architecture, specifically I'd like to talk about
system design, more generally and specifically the reactive manifesto. The reactive manifesto
is a set of community-driven guidelines that are intended to give a cohesive approach to systems
design. So, at the core of the reactant manifesto we have, we want our system to be message driven.
And so, specifically asynchronous messaging.
And then also, we want our our system design to be scalable,
and we want it to be resilient.
So this implies distributed systems or Kubernetes. So by scalable we mean that we want the
hardware used to expand as the workload expands and contract as the workload contracts
and by resilient we mean we we don't want any single point of failure
and if the system does fail we want it to be able to tolerate tolerate that elegantly so with this
foundation in place we should be able to build a system that is responsive responsive is the value
on top of that we can expect a system that is maintainable
so it means it's easy to change easy to fix a bug or do an update and we want one that is extensible
so we should be able to expand the system easily
so now that we have the core of what we want our systems designed to look like
now let's talk about let's level set here and talk about like what is an event
so an event is a statement of fact an immutable statement of fact about something that happened
in the past so let's give that some context we'll talk about a retail application
so a retail application has a checkout service
and that checkout service is going to want to communicate with an inventory service
a shipping service and a contact service
so with the messaging model if the inventory wants to know what the checkout is doing the checkout
will send a message directly to inventory to let the inventory know a checkout happen
and another one directly to shipping and directly to contact so with messaging we have targeted
delivery um not only that but inventory can maybe send a message back to
checkout it can be a conversational delivery conversational messaging
and then finally let's talk about where that message lives before it's sent it's it's
lives on the host service or on the producing service when it's received it lives on the um
it lives on the host service until it's received by the consuming service or in the case of a
message broker it might sit on a machine in between but regardless the the data persistence is
transient
now let's compare that to eventing so if our checkout service is producing events
it's producing an event every time a checkout happens without any concern to which service
is consuming that and so with that we have a system that's highly scalable so we have um
three services here interested in the checkout but even if we expand it to a hundred services
listening to check out that wouldn't affect what the checkout service is doing in any way
not only that when we have all the checkouts in a row we come up with an event log
and it can be replayable so we have a stream history
and a history of everything that that every checkout event that has happened
and then our data persistence depending on how the cluster is set up it can be set up to be immutable
it can be there forever as long as you have the physical hardware to back it up so we have um
instead of having transient data persistent so we have uh permanent or possibly permanent
not immutable but permanent data persistence e and t um so this is the foundation of our event driven
architecture so we have a resilient a reactive responsive system design and then that message
driven element is is event messaging specifically so when we go into our proper event design the
first thing we're going to consider are our event producers so we might have a web application
that produces events and we might have a a mobile application
and let's say we also have an edge device that's producing events so uh internet of things device
and all of these are producing events into our system so
so this piece of the system here that has all the events running through it
this is called the event backbone
so what does this system enable
well first of all we have our event logs being produced by all the producing applications
so the event logs can do a few things first it can trigger an action so if we have an internet
of things device let's say our internet of things device is a robot vacuum cleaner and when we turn
on that robot vacuum cleaner it spins up a pod on the infrastructure and that pod is a function
as a service so function as a service is a modular piece of code often associated with an edge device
that doesn't exist on a server on the server until it is turned on and that's when it is spun up so
that's called serverless function as a service is sits on top of serverless infrastructure
but the point is that event triggers an action of the with the robot vacuum cleaner to spin up
bounce all around the house make everything squeaky clean and then when the function is
finished running it powers down and perhaps that'll cause our function as a service to send
another event back into the system to let it know that the the function is complete
another cool thing about the event log is it can be used to help optimize and customize data
persistence so if our web application is our maybe it's our retail application that we talked about
before and it's doing a stream of checkout events well our inventory service can be reading from
that and it's keeping track of a local data store that's keeping track of the inventory
so our inventory service will consume from that data stream from the web application it'll
modify the local data and then it'll produce again into the event backbone and so this new
stream is giving the most current inventory to any other application in the system that
wants to consume from it and so while we're talking about data another thing that happens
we can save all of the data that comes through the system so all the raw data or all of the transform
data here at the end once it goes through the system we'll save it all in a huge data lake
and this is what's going to power really data-heavy applications
like ai for example and the data lake is accessible from anywhere in the system
that any service that you let consume from it
so another thing that our event-driven architectures can enable are a stream processing
so this is built on top of the apache kafka streams api
so let's say that our mobile app is producing let's say it's a music streaming service and
every time let's say up until your in your whole life up until this point you've been listening to
only miley cyrus and britney spears but today you decide you know what i want to see what jay-z is
about and you turn jay-z on well immediately your streaming app is showing you kanye west
and lauren hill and giving you ideas about what you might want to listen to next so that stream
processing is a really powerful and what gives the gives real-time calculations like if it's a
ride share app you um that you know exactly how far away your ride is but it can also be used to
implement business rules and business policies and it can be used for real-time analytics
and then let's also talk about with our system the communications layer
so specifically it's called pub sub communication
for publish subscribe so we could have any microservice a containerized microservice
and just like we talked about before with it being extensible we can very easily plug it into we can
write a new microservice and plug it into our system and immediately have it start consuming
from any stream and producing to a new stream or even producing to a stream that already exists
and none of the other applications need to be concerned with it at all so it's
highly plugable not only that but because of the stream history we talked about before
when you plug it into the system you don't have to have it start consuming from right this moment
it can be consuming from as long as from the beginning of your application if you've been
saving your stream history for that long so as long as that stream history persists when
you plug in a new system it can consume from either the old data or can be consumed from
the present moment so with that we talked about the major benefits of event driven architecture
that sit on top of the reactive manifesto ideas for system design with eventing at its core
5.0 / 5 (0 votes)