Dapr Community Call - May 29th 2024 (#104)

Summary

In the Dapper Community call, Youron Schneider, one of Dapper's original founders, introduces the outbox pattern and payload projections in the upcoming 1.4 release. The outbox pattern allows single-transaction commits across databases and message brokers, crucial for system reliability. Schneider demonstrates the new feature enabling different payloads for database storage and message publishing, showcasing Dapper's language, cloud, and hosting agnosticism, and its ability to simplify complex distributed system challenges.

Takeaways

• 😀 The Dapper Community Call is a platform for discussing updates and developments within Dapper as a product and community.
• 🌟 The community call originally featured Ryan Winter discussing Azure IoT and Dapper pluggable components, but was replaced by a talk by Dapper's co-founder, Youron, on the outbox pattern and payload projections in Dapper 1.4.
• 🔍 Youron Schneider is a core maintainer of the Dapper project and a stream commed member from Drid, focusing on open-source and serverless container platforms.
• 📈 The outbox pattern, introduced in Dapper 1.1.2, allows committing a single transaction across a database and a message broker, solving the dual write problem in distributed systems.
• 🔧 Dapper 1.4 introduces payload projections, enabling developers to decouple the state saved in the database from the payload published to downstream subscribers.
• 🎯 The outbox pattern is highly requested by the community for its reliability and consistency in event-driven architectures.
• 🛠️ Dapper abstracts the complexities of distributed systems, allowing developers to use different databases and message brokers interchangeably.
• 📚 Youron mentioned writing a blog post to demystify the technical details of how Dapper implements the outbox pattern and payload projections.
• 💻 A live demo was given, showcasing how to use Dapper's transaction API to save state in Redis and publish a different payload to Kafka with a single HTTP call.
• 🔄 Dapper's resiliency features can be applied to both database operations and pub/sub messaging, enhancing the robustness of applications.
• 📝 The Dapper community is encouraged to try out preview features, provide feedback, and contribute to the project through documentation, blog posts, and sample code.

Q & A

What is the purpose of the Dapper Community calls?

The purpose of the Dapper Community calls is to gather everyone together to discuss new and exciting developments within the Dapper product and community, including talks, presentations, and updates.

Who was originally scheduled to talk about Azure IoT and Dapper pluggable components, and what happened?

Ryan Winter was originally scheduled to talk about Azure IoT and Dapper pluggable components, but due to a series of unfortunate events, he could not make it, and Yaron Schneider stepped in to talk about the outbox pattern and payload projections.

What is Yaron Schneider's role in the Dapper project?

Yaron Schneider is one of the original founders of the Dapper project, a core maintainer on the DAP runtime, and a stream commed member from Drid, a company he founded after working at Microsoft.

What is the outbox pattern, and why is it difficult to implement?

The outbox pattern, also known as the transaction outbox pattern, is the ability to commit a single transaction across a database and a message broker. It is difficult to implement because it requires solving the dual-write problem in distributed systems, which involves coordinating the same data or payload being saved to two separate systems transactionally.

What is the dual-write problem in the context of the outbox pattern?

The dual-write problem refers to the challenge of needing to save the same data or payload to two separate systems, such as a database and a message broker, transactionally, which is difficult due to potential failures in the systems that can leave the data inconsistent.

What is the 'anticipator pattern' mentioned by Yaron Schneider?

The 'anticipator pattern' is a term used by Yaron Schneider to describe the method used in Dapper to solve the dual-write problem. It allows Dapper to handle the complexities of committing a single transaction across different state stores and pub/sub brokers.

What is the outbox projections feature in Dapper 1.14, and how does it address a limitation of the outbox pattern?

The outbox projections feature in Dapper 1.14 allows developers to decouple the state saved in the database from the payload published to the downstream subscriber. This addresses the limitation of the outbox pattern where previously the same state had to be used for both the database and the published message.

How does Dapper handle resiliency in the context of the outbox pattern?

Dapper allows the application of resiliency policies to the database operation and the pub/sub layer independently. This means that if the entire operation fails, Dapper can retry the operation in an exponential backoff manner, and if a message cannot be delivered to the underlying subscriber, it can be sent to a dead letter.

What is the process for trying out new preview features in Dapper?

To try out new preview features in Dapper, users should look at the documentation and follow the instructions provided. Some preview features might require a flag to be turned on in the Dapper configuration, explicitly enabling the feature.

How can community members contribute to Dapper and what are the benefits?

Community members can contribute to Dapper by writing blog posts, submitting issues, creating samples, and participating in conferences or workshops. They can also earn badges for their contributions, such as presenter badges for doing presentations or writer badges for updating documentation.

Outlines

00:00

🎉 Welcome to the Dapper Community Call

The speaker warmly welcomes participants to a Dapper Community call, an event that gathers individuals to discuss updates and advancements in the Dapper product and community. The speaker expresses gratitude for the attendees and encourages live viewers to share their locations and how they engage with Dapper. There's a mention of a change in the agenda with Ryan Winter's session on Azure IoT and Dapper pluggable components being canceled, replaced by a talk by Yaron Schneider, one of Dapper's original founders, on the outbox pattern and payload projections in the upcoming 1.4 release. The speaker also provides a link for those curious about the Dapper Community and invites Yaron to introduce himself and discuss his role and contributions to the Dapper project.

05:02

🛠️ Dapper's Outbox Pattern and Payload Projections

Yaron Schneider introduces himself as a core maintainer of the Dapper project and a stream commed member from Drid, a company he founded. He outlines his experience at Microsoft, focusing on open source and serverless container platforms. Yaron then dives into the outbox pattern, a feature that commits a transaction across a database and a message broker, which is crucial for reliability and consistency but notoriously difficult to implement correctly. He explains the dual write problem in distributed systems and how Dapper's outbox pattern addresses this challenge. Yaron also teases an upcoming blog post that will demystify the technical details of Dapper's solution. The discussion then shifts to the limitations of the current outbox implementation and the introduction of payload projections in Dapper 1.4, which allows for different payloads to be saved in the database and published to downstream subscribers.

10:02

🔄 Demonstrating Outbox Projections in Dapper 1.4

In this segment, the speaker provides a live demonstration of the outbox projections feature in Dapper 1.4. He uses Visual Studio Code to show how to set up a Dapper application with an outbox-enabled state store and a pub/sub component. The speaker runs the Dapper CLI to start the application locally and then uses Postman to illustrate how to save state to a database and publish a different payload to a downstream subscriber within a single transaction. The demonstration highlights the ease with which developers can use Dapper to handle complex distributed systems operations without being tightly coupled to specific infrastructure.

15:05

📢 Community Interaction and Feature Preview

The speaker opens the floor for questions from the community and reflects on the composition of new Dapper features from existing ones, such as workflows built on actors. They discuss the upcoming distributed cron API in Dapper 1.4 and its potential to simplify scheduling of work jobs in a distributed manner. The conversation emphasizes the importance of user feedback for the stability and improvement of Dapper's preview features. The speaker also mentions the need for users to try out these features and report their experiences to help shape the project's development.

20:05

🌐 Community Contributions and Social Presence

The speaker shifts the focus to the community, highlighting the contributions of various members and the presence of Dapper across different social platforms and conferences. They mention case studies and talks by community members on using Dapper in different scenarios, such as IoT device management and real-time event processing in the gaming sector. The speaker also plugs their own recent talk at a conference and encourages community members to share their experiences and work involving Dapper on social media and through presentations.

25:06

🏆 Recognizing Community Efforts and Upcoming Events

The speaker acknowledges the efforts of the community in contributing to Dapper, offering badges for various contributions such as presenting, writing, and supporting the community. They provide QR codes for easy access to claim these badges and encourage participation in future community calls and events. The speaker also shares links to resources like the Dapper website, GitHub, Discord, and Twitter, inviting the audience to engage with the community and explore the project further.

30:07

👋 Closing Remarks and Future Community Calls

In the final segment, the speaker thanks everyone for their participation and provides information on how to stay connected with the Dapper community. They mention the bi-weekly schedule of the community calls and invite the audience to the next session. The speaker also reminds the audience to reach out with any questions and to take advantage of the resources available to learn more about Dapper and its upcoming features.

Keywords

💡Dapper

Dapper is a cloud-native, event-driven framework that simplifies building distributed systems by providing abstractions for state management, messaging, and event-driven architecture. In the video, Dapper is the central theme, discussed for its features, community, and updates. It is used as a product and community reference point throughout the script.

💡Community Call

A community call is a gathering of members within a specific community to discuss updates, share knowledge, and engage with one another. In the context of the video, the Dapper Community Call serves as a platform for discussing the latest developments within the Dapper project and its ecosystem.

💡Outbox Pattern

The outbox pattern, also known as the transaction outbox pattern, is a technique used in distributed systems to ensure that a transaction is committed across a database and a message broker atomically. The video discusses this pattern in the context of Dapper's 1.4 release, emphasizing its importance for reliability and consistency in distributed systems.

💡Payload Projections

Payload projections refer to the ability to define different payloads for database storage and message publishing within the same transaction. The video script mentions this feature as a new addition in Dapper 1.4, allowing developers to decouple the data saved in the database from the message sent to downstream subscribers.

💡Distributed Systems

Distributed systems are computer systems that consist of multiple autonomous computers that communicate and coordinate actions through message passing. The video discusses the challenges of distributed systems, such as the dual-write problem, in the context of implementing the outbox pattern with Dapper.

💡Resiliency

Resiliency in the context of distributed systems refers to the ability of a system to recover from failures and continue operating. The script mentions resiliency policies in Dapper, which can be applied to operations to handle retries and dead-letter scenarios, enhancing the robustness of applications.

💡Dapper CLI

The Dapper CLI (Command Line Interface) is a tool that allows developers to interact with Dapper from the command line, facilitating tasks such as running applications with Dapper sidecars. In the script, the CLI is used to demonstrate running a Dapper application locally.

💡State Store

A state store in Dapper is a component responsible for managing state within a distributed system. It is used in conjunction with other Dapper features, such as the outbox pattern, to ensure state consistency. The script discusses configuring a state store for use with the outbox feature in Dapper.

💡Pub/Sub

Pub/Sub, short for publish-subscribe, is a messaging pattern where the message producer (publisher) does not program the messages to be sent directly to a specific receiver (subscriber), but instead sends messages to a common channel. The video script mentions setting up a Pub/Sub component in Dapper and using it for event-driven architectures.

💡Distributed Cron

Distributed Cron is a feature in Dapper that allows scheduling of tasks in a distributed manner, ensuring that scheduled jobs are executed by only one instance of an application replica. The script mentions this feature as an upcoming addition in Dapper 1.4, highlighting its utility for simplifying task scheduling in cloud-native applications.

💡Workflows

Workflows in Dapper refer to a higher-level abstraction that combines multiple Dapper APIs to create complex event-driven architectures. The script mentions workflows as an example of how Dapper allows the composition of foundational features to build more sophisticated systems.

Highlights

Introduction to the Dapper Community call focusing on updates and discussions about the Dapper product and community.

Invitation for global Dapper users to join the live discussion and share their location and use cases in the chat.

Update on the agenda with a change in speakers, featuring Yaron Schneider, one of the original founders of Dapper, discussing the outbox pattern and payload projections.

Yaron Schneider's introduction, his role as a core maintainer of Dapper, and his involvement in the open-source community.

Explanation of the outbox pattern, its importance for transactional consistency across databases and message brokers, and the challenges it presents.

Introduction of the 'anticipator pattern' used in Dapper to solve the dual write problem in distributed systems.

Demonstration of the outbox feature in Dapper since version 1.12 and its evolution towards a stable feature in 1.14.

Announcement of the outbox projections feature in Dapper 1.14, allowing developers to decouple database state from published payloads.

Live coding demo showcasing the implementation of the outbox pattern with payload projections in Dapper.

Discussion on the composability of Dapper features, such as workflows building on actors, and the outbox pattern enhancing state store and pub/sub capabilities.

Integration of resiliency support with the new outbox feature to handle retries and dead letter scenarios in distributed transactions.

Call to action for community members to try out preview features and provide feedback to help stabilize upcoming Dapper releases.

Highlight of community contributions, including blog posts, case studies, and conference talks, showcasing various use cases of Dapper.

Introduction of community badges for contributors and participants as a way to recognize and encourage involvement in the Dapper community.

Upcoming events and community calls information, inviting members to stay engaged with the Dapper community.

Closing remarks thanking participants and providing links and resources for further exploration of Dapper.

Transcripts

coming on to one of our Dapper Community

calls this is essentially where we get

everyone together and talk about some of

the new and amazing things that are

happening both within Dapper as a

product and also with Dapper as the

community so we usually have different

talks and presentations and we share

different updates and things of that

nature so definitely appreciate you all

joining us um definitely thank you so

much for being here uh for the folks

that are watching us live I'd definitely

love to know where everyone is calling

in from you know I know we have Dapper

users from all across the world so you

know if you're watching us from Germany

or India or Australia or even in the

United States definitely go ahead use

the chat um feel free to drop um a

message in the comments and just let us

know you know where are you watching

from and what is it exactly that you do

with

that now let's move forward to with our

agenda really quickly so quick update to

the agenda originally we had Ryan winter

coming on to talk to us a little bit

about Azure iot and Dapper pluggable

components well sorry to say due for a

series of month on Unfortunate Events

that's not happening but yon is here and

we all love yon youron is one of the

original founders of the Dapper project

and he's going to be talking to us about

um the outbox pattern and payload

projections which I think is slated to

come in the release of 1.4 Dapper so

we're going to do a little bit of that

today we're going to do you know our

community Show and Tell we're g to give

you all a little bit of updates and

again hopefully we'll have a good time

talking about Dapper and again to for

folks that are curious if you want to

know a little bit more about that Dapper

Community I just added a link to the

bottom of the screen feel free to head

over to github.com Dapper Community you

can learn about these Community calls

you can learn about how you can get

involved in Dapper um the links to our

Discord Channel and all that types of

stuff is in there so again this is your

first time welcome and here's a great

place for you to get started now I'm

going to bring yourone on and I'm going

to have him just really quickly

introduce himself and you know he could

let us know a little bit about who he is

and uh what we're doing here here we go

hey youron what's going on hey happy to

be here nice to have you so obviously I

know who you are and maybe a few people

know who you are but why don't you do a

little bit of an introduction for folks

that maybe knew let them know just

exactly like who you are and you know

what do you do today yeah of course hi

everyone my name is youron Schneider I

am one of the original uh creators of

the Dapper project today um a core

maintainer on the DAP runtime um plus a

few other repositories that we have uh

on the project I'm a stream commed

member um from drid which is a company I

founded after uh spending a few years at

Microsoft where I mostly focused on open

source and building serverless container

platforms um like aure container apps

and some other services and um yeah

we're on the steering committee with

Intel Alibaba Cloud uh Microsoft and

automatics and driving the project

forward and today I'm here to talk to

you about a new feature that was highly

requested by the community coming in

depper 114 nice so you're I guess you're

G to share your screen and um again you

g to talk a little bit about the outbox

pattern and guessing you to tell us well

like what is this outbox pattern thing

and then also how does dapper actually

go through and like you know make that a

reality for developers to use in a very

language agnostic Cloud agnostic hosting

agnostic yeah of course so we've we've

actually had the the outbox feature

since um 112 um it's still a preview

feature we're hoping to take it out in

114 makes a stable feature and I'm going

to walk through uh what that is and how

D result this um pretty hard problem for

developers so I'm going to be sharing my

screen and please let me know and you

can see it um pulling up your screen

right now all right there you go take it

all right cool so we're gonna be talking

about uh a new feature that's coming in

114 called payload projections but

before that let's get on the same page

about the outbox pattern commonly known

as the transaction outbox pattern so

what is it um in short it's the ability

to exit Ute or commit a single

transaction across a database and a

message broker um and it it can be

highly useful um for reliability and

consistency purposes but it's really

really hard to get right um and the

reason is that to make it work you need

to solve a very fundamental problem uh

in in computer science and specifically

distributed Systems computer science um

which is the Dual Riot problem um and

that issue is you know basically needing

to coordinate the same same data or

payload or um uh message being saved to

two separate systems whether you know

it's a it's a file on disk or a message

broker or a database or a config store

or even something that's written in

memory transactionally um and that's a

difficult problem if you combine to that

the outbox requirements then you get an

even bigger problem um and the

requirements for the outbox pattern are

as follows if you save a state to a

database then you must guarantee that a

message is also published as a part of

that transaction so the state save in

itself might not be a transaction but

when you add the separate operation to

notify Downstream subscribers about the

state that will saved you suddenly get a

transaction and not just a single

database transaction in the outbox case

it's a single transaction between a

database and the message broker which is

extremely tricky um and then the message

must not be published if the state

wasn't saved in the database so if your

app saved the state and the state failed

you must not publish the message and um

on the same vein the state must not be

saved without a published message so

this is um very tricky in this this

diagram that I grabbed off of Google um

basically shows you what it looks like

you have the command Handler here on the

left that would probably be your

application your code that's uh wanting

to commit the the transaction wherever

it wherever it may run whatever it may

be any type of code so you're staring a

transaction against database it's being

processed transactions committed and

then the the red line here is something

that's basically you know asynchronous

and out of line with the database

operation and that's the tricky part to

get right because um in these you know

three uh different actors that are

participating any one of them can go

down the database can go down the event

bus can go down um the event bus can go

down after you save the data to the

database and before you've actually uh

saved the to the event bus and vice

versa and the application itself can you

know go down um for example if you've

saved the database and then you know the

event bus errors out and cause your

application to crash which leaves you

with state in the database and message

that isn't published um so this is

pretty difficult we might even call this

the three body problem uh now that they

think about it should actually tradem

Market well I can trade market because

Netflix probably did already well never

mind um in Dapper we have Dapper outbox

which solves this problem completely we

have um solved the Dual right problem

and I can't go in into you know very

technical details about how we did that

here because we don't have time but I am

going to write a blog post that

basically demystifies how this is being

done um you know as a sneak preview of

that blog post that's coming up um we

used something that I call the

anticipator pattern that thing I am

going to trademark probably um but using

Dapper you can actually combine any

number number of dapper supported State

stores and pubs of Brokers to achieve

this so for example if you were to write

your own outbox implementation um you

would need to solve all of these

distributed systems issues and probably

you would end up with something that's

very coupled to you know postgress SQL

or kofka um or you know some some other

combination that you're using of a

database and message broker because the

outbox logic is very very tied to the

actual infrastructure that you're using

but Apper allows you to swap in and out

dozens of different databases and

message Brokers to arrive at a

combination that works for you so with

Dapper you can basically save um a state

item to a database uh and then dapp will

make sure that that same state is

delivered to a downstream subscriber

which is a subscribing application and

um all that is pretty great but we've

had one major limitation until now and

that is that the same state you're

saving into the database is the same one

that gets published um into the

downstream subscriber and while that is

useful um also in a lot of cases it

might prove unuseful because you simply

want a different payload um and that

feedback we've heard loud and clear from

a lot of our uh community members who

are trying out this feature now and I am

happy to say that uh in 114 we're coming

out with a feature called outbox

projections which basically Works around

the limitation and in here you can see

that um we have different payloads that

are saved with the database and the pub

sub and I'm going to show you a demo of

this right now so I'm gonna go over to

code here can you see myvs

code can you hear me guess you can hear

me I'm was talking to you yeah that

looks good to me all right cool so this

is a new GS code um this is taken off of

the Dapper tutorials nothing special

here um you know 25 lines of code to

create an aveng of app with Dapper here

we're subscribing to a pub sub called

pubsub well that's not very original um

and the topic named a okay and we're

also telling Dapper Hey whenever a

message arrives on the topic a please

send it to an endpoint on my application

on a route to a route listening on the

path which is also a so here I'm just

GNA uh console log whatever comes in

pretty simple this is just Dapper stuff

um no infrastructure here uh you know

we're completely decoupled from How We

Do pubsub How We Do event driven

architectures what pubsub we're working

against and specifically from anything

outbox related um to make this work uh

we are defining the components in Dapper

because Dapper abstract set not only

from communicating with infrastructure

but from all of the distributed systems

work associated with interacting with

the apis for that infrastructure and I

have a state store here which is a r

state store I'm running this um sample

using Kafka and redis locally so the

state is going to be saved into redis

while the uh pubsub message is going to

be published into Kafka and from there

into an application that needs to be

notified whenever we're saving State

into the red database so this is a

Dapper component if you haven't seen

this before this is Dapper's way to

connect your apps with um the

infrastructure so we have the ready s r

password um and we have two specific

outbox metadata items here I'm basically

telling the state store hey um this is

outbox enabled so every time there is a

state change I want you to send uh the

state change to this pubsub and

specifically to this topic um and where

the pubsub component this is pretty

simple uh just pops up. cka telling you

know Dapper that I have a locally

running Kafka broker here on Local Host

1992 and that's it that's all I need so

since I'm running this locally I'm just

going to use the Dapper CLI to uh run

Dapper and my app so if you haven't seen

this before this is a local tool that

allows you to run the Dapper CLI and

your application in one go uh very easy

to to get started with Dapper locally so

I'm just doing Dapper run giving my

application an app ID this can really be

anything just setting up the ports and

telling Dapper um the path to find these

files so that it can connect to the cuff

CA broker and red estate store that I

have running locally and the second

command here um or the second you know

uh major parameter here is node appjs

which is going to run my application

after the side car is running so we're

going to hit this off and we can already

see that we are up and running and we

can also see this uh message here says

my app is subscribed to the following

topics so Dapper gives me feedback about

you know which topics I'm actually

subscribed to um and we're basically

good to go at this point so I'm G to

just going to go back into my um

application here and this is going to be

the code that's going to get triggered

once we save the state of the database

right so I'm I'm not going to be you

know talking this endpoint directly I'm

not even going to be doing pubsub

directly I'm actually going to go into

Postman here and show you what the

outbox projection looks like so um first

of all I'm using the Dapper transaction

API this is the same endpoint I would

use to just save state in a

transactional database in Dapper um and

this is basically my request what I'm

highlighting here let me just zoom in a

little so so we have an operation it's

an upsert and this is uh the request

data so I'm saving a key of one and this

is the value this is the value that's

going to get written in the database or

that should get written in the database

so my key is one and my value is two and

if I basically just you know publish

this then it would save an item to my

database but now and and also published

it to the end topic because we have

outbox configured um but since we want a

different payload to be sent to my

application uh we can now use

projections and you can see that my

operations is an array which means I

have a different second operation here

um which is the same as above with the

only difference being that um the value

is different so the state that's going

to be saved to the database is going to

be the value of string two while the

value that we want published into our

application is actually a Json payload

with an event ID um that I just made up

here and uh we're going to tell Dapper

uh hey look this transaction should

actually not get saved into the database

this is a projection of this operation

now how does dapper know that this is a

projection of this same uh operation

here well we have the same key right so

if you have the same key um with this

metadata item then Dapper is going to

say hey okay well this operation is not

going to be saved in the database only

this one is and we're going to be using

this value to actually send it to the

application and in here you can actually

override all of the cloud event headers

um and Fields that also get sent to the

underlying subscriber so for example if

you wanted to override the cloud event

sub field or ID or whatever you could

just put it inside of the same metadata

here and I'm also just saying that the

content type is application Json because

this uh data type is Json and this is

really all the r because we're basically

committing a single transaction across

Kafka and redus using one API call which

is post in this case of course this is

also supported using any d s Decay today

and using the grpc clients um there are

no special things you need to do in the

da s DEC to get support for this and

yeah let's see if it actually works

because this is a preview feature that's

coming up um well right now it's a

preview feature because it's in our

Master Branch but it's not in a version

so um let's run this and by the way one

last thing to mention is that if I were

to remove this U met data element here

um this request would fail which is what

we're expecting because this is an

invalid transaction because had we

removed this Dapper would have tried to

basically publish two messages not not

publish save two messages to the

database um with the same key and that's

an invalid operation and Dapper would

let us know that we're trying to do

something that is invalid all right cool

so let's do this and I'm just going to

save this uh we got response back 24

milliseconds and uh let's go back into

our code here and as you can see uh let

me just bring this closer uh our

application U log the event ID and this

is the payload that we specified in the

projection that you just saw and if we

go back here um and we get the state no

this is not the correct one and we get

the state so this is a get operation

just to see what what kind of data we

actually sa in the database um this is

going to return us the string value of

two let me just zoom in on that a little

bring this up yeah so we can see that we

saved a different uh state in the

database published a different message

to the underlying broker and it was all

done via single transaction if any of

them would have failed we wouldn't get

the state saving database we wouldn't

have gotten the event published to the

underlying subscriber and we did that

with a single HTTP call okay so I think

uh that's all for here I'm going to go

back and stop sharing in and we can take

it off for

questions I actually have a lot of

questions to be honest with you but I'm

going to give folks a chance if everyone

that's watching if you have any

questions for your own feel free to go

ahead and drop them in the chat um and I

I'm looking at the numbers I noticed a

lot of you are still coming in so if you

don't know what we're talking about if

you're just now joining the stream your

own just kind of went through preview

support for um the elbox pattern that's

going to be in Dapper um I think the

goal is to have this in preview in one

14 that should be coming up later on in

the year at some point right is that

correct correct yes okay so outbox is

already supported since 112 if you have

Dapper 112 you can already use it again

commit a single transaction between any

of the supported Dapper transactional

databases and um really any message

broker that supported that this outbox

pattern supports all message Brokers um

and yes in 114 we're going to allow you

to decouple the state that gets saved in

the database as opposed to the actual

you know Cloud event that gets published

to the underlying subscriber nice nice

you know one of the things I I I love

about some of these newer daper features

that we're getting is that it looks like

they're being composed from existing

features yeah like as you were talking

about it um and I'll bring up like

workflows as an example which Builds on

an existing feature which is you know

actors as an example right and then what

you're showing us now like some of the

out books patterns that are built up on

the state store and the cloud events

support and Pub so right and you showed

an example of using transactions and

some of these types of things right like

I love to see how like some of these

things are starting to get composed from

like I guess we can call them the

foundational bits the foundational

features of dapper to create like some

of these higher level abstractions are

you seeing like more of those types of

requests coming in from users as as

we're going definitely for sure um I

think we spent a lot of time building

all of the fundamental infrastructure

needed in Dapper to arrive at a place

where we can compose higher Lev

architectures using two or more Dapper

apis combined together um workflows is

is definitely you know combination of um

actors and and some State Source stuff

um and you know a new building block

based on an existing building block

really outbox um is kind of the same as

far as the user is concerned it you know

ties in uh two apis um but this is the

first feature I think that really ties

in two distinct um apis that you know

users can talk to independently where

workflows is a new building block outbox

is a new feature on top of an existing

building block that uses a different one

so yes we're definitely going to see

more and more of these things um coming

up uh for sure because these higher

level you know um features and

abstractions are the most useful um and

now that we have all of the Machinery

needed in Dapper to um provide for these

higher level features we're certainly

going to make more usage of that gotta I

know one other feature in Dapper that

I'm pretty sure you're Phantom 2 is the

resiliency support does this new feature

that you have does that pair well

together with like the resiliency stuff

that that's already baked in a box oh

yeah for sure so you know when you're

saving the state to to Dapper when

you're telling Dapper hey save the state

transaction which is in my demo uh it

was coupled to the outbox Pub sub where

we're basically saying hey save the

state into a database and then publish

it you can apply resiliency policy any

resiliency policy for the operation so

if the entire operation fails for

whatever reason you know PBS have went

down stat went down um Dapper can

actually retry this in an exponential

manner manner an exponential back off um

just a linear retry um you can actually

send the message even to a dead letter

um if the if the underlying subscriber

wasn't able to receive the request and

you can also put a resiliency policy on

the underlying subscriber which is

really decoupled from the database so if

depper isn't able to uh send a message

to the underlying subscriber it will

just reach right um so yes you can put

resiliency pulses both in the database

operation layer and the pub sub layer

and it'll just work out the box nice

nice um so you mentioned this is coming

out in 114 what are some of the things

that you think you need right now or

does a team need right now for this to

go from preview to to not preview to

like to to G yeah um so we've actually

gotten a lot of feedback um during 11 12

and

113 um which led to this you know major

limitation being removed now in 114 to

make it stable in general what we want

from users is to try out um preview

features that is the best thing you can

do outside of opening PRS and issues um

to help the Dapper project for example

we're coming out with a distributed cron

API

in4 um which is also going to be huge um

because it it'll allow you and this

really also nicely workflows it allows

you to um schedule Crown work jobs in in

Dapper in a distributed way so if you

have multiple replicas of an application

that will actually make sure that when

it hits um your you know application

it'll only hit one specific instance um

and and that also removes a lot of buer

plate code and Machinery that developers

need to go through but this is coming as

a preview feature we really need

feedback about that we're we're going to

be putting out a lot of content out

there about how to enable it um you know

spoiler it's going to be pretty easy

it's a configuration flag on an on a

Helm chart or a configuration um and

then we want the feedback to just flow

in and hear about your experiences

because otherwise you know we can use

our own test to um try and get a feel

for how stable things are but there is

nothing like user feedback and so far

the Dapper Community has been awesome um

we just want people to continue giving

us feedback and for more people to give

us feedback gotta so let's say I just

installed 114 right and I want to try

out this preview feature is there

anything special that I need to do or do

I just need to look at the docs and see

what the end points are install the sdks

or is there any particular type of you

know other than setting up the saate

store and The Pub St is there any

additional configuration that I need to

get that to work specifically for outbox

no um you you just look at the docs and

and you you do what they say like what

they just showed you is basically what

you need to do there were no other steps

I took before showing the demo um to get

outbox working um that being said some

preview features might explicitly need a

flag to be turned on and we have that on

the Dapper configuration crd or resource

where you basically explicitly Say Hey I

want this preview feature to be turned

on and the reason why some features have

that flug and some why don't is because

of what we call true bypass I'm not

actually sure if this is only taken you

know from the guitar playing world but

when you um when if you play guitar you

have like effects they're going into

theend amp and sometimes you just want a

clean signal going through the guitar to

the amp and so you you want a true

bypass um so some of these features if

you actually enable them without um

doing anything else they they will alter

the behavior of dapper which is why

we're disabling them by default and

we're telling people look this you know

might cause some issues and you

explicitly need to enable them other

features like the outbox pattern they

they have true bypass they don't do

anything unless you actually explicitly

use them gotcha and so like you said

with this one you don't have to do

anything different and because we did

have outbox support before like you said

in 112 this is more so just like adding

on to like some of the existing

functionality today and so it's not

going to like we're not expecting

breaking changes we're not expecting any

of the existing um Investments you made

in daa to be influenced by it but now

this is more so just like a plus one to

something else that you're able to do

correct that's very true yeah it's nice

well it doesn't look like we have any

questions for folks from folks um again

at any point in time if anyone has any

questions um I definitely recommend that

you feel free to reach out to your own

or any of us on the Discord Channel or

Twitter or on LinkedIn or on any of the

many social things that people connected

to um again just to tell everyone if you

missed the demo and if you uh came in

late then just watch the recording it's

going to be up on YouTube yeah it'll be

up on YouTube but that being said we're

not done yet because there's more stuff

to do so I'm going to bring up my screen

really quickly um and we're going to go

through the community um you know some

of the community updates that we' had so

actually I think what I will do is I'm

GNA switch over to my browser um so as

you know like Dapper has been I know

like the community and Dapper has been

so active we really appreciate all the

things that you all do from writing blog

posts to submitting issues to you know

creating these samples and things of

that nature um so you know in every

Community call we' like to highlight

just a few of our community members that

have been doing some really great stuff

so in this particular series I think

this is the first part of you know

hopefully a very lengthy series um

Vladimir here is talking about building

Cloud native apps with you know Dapper

and go um you know surprisingly I don't

see a lot of folks talking about Dapper

and go and you know Dapper is written

and go so that kind of just makes sense

to me um so it's kind of great to see

him kind of going through this um again

this is very much of an introductory

series talks a little bit about like you

know Dapper kubernetes the SARP pattern

and things of that nature uh but

definitely worth a read and definitely

looking forward to you know the next uh

the next parts of this series that U

that Vladimir's going to put

out uh next one I'm going to take a look

at Tempest if we had over to like the

cncf um there's a few different case

studies that they've released um and

this one this one was pretty interesting

because this talks about like a

manufacturing company that has

essentially they're doing a lot of iot

work and how they're using you know

daper to really help manage some of

their iot devices as you can imagine

like they're using Dapper actors and you

know they're using Pub sub and things of

that nature so definitely a good read

this is on the cncf case studies um

section of their website so make sure

you check that out and then also oops

let me scroll up a little bit um this

was an interesting one too um Dev Rico

is a a company that's in the gaming

sector I don't think they actually make

games but like they you know they're

within that space and they talk about

using Dappers um again you know using um

Pub sub and actors because they wanted

to do some realtime event processing to

be able to generate recommendations for

players which I thought was actually

really cool so this is also a good read

um if again if you're interested in

learning about like how people are using

Dappers or how you could use Dapper you

know what are some of the use case it

covers and you know what are some of the

cool things that it

does uh moving forward real quick I

actually Shameless plug I actually did a

talk on Dapper this was actually a

couple weeks ago at the

ab. NETCOM we talk a little bit about

you know microservices and Dapper and

net and you know some of the different

things that it supports as well um you

know if if you all are doing anything

with Dapper at conferences or workshops

or anything of the sort again definitely

feel free to let us know we have a show

and tell Channel inside of the G repo

and so definitely recommend that you

head over there and and share out with

us now that was these links I'll make

sure I put these links like in the

section the description section of the

video um for those of you that are

watching us outside of YouTube like

that's what I'm talking about like we're

going to put it in the YouTube uh video

description section and we'll also make

sure we share them inside of our show

and section on the Discord Channel as

well um but just a couple other things

that have been happening in the world of

dapper getting across different

conferences and whatnot you know we've

had folks been on like the cloud native

show um you know our friends from

diagrid have done you know things on you

know Dapper at scale our other community

manager Mark there at the bottom right

side you know he has a video that talks

about like how you can contribute to

Dapper um he has a whole read me and you

know Badges and all kinds of fun stuff

that that's going on there um and here's

some on to talk about like getting up to

speed with Dapper these are all videos

that are available on YouTube right now

so if you want to go and check these out

feel free to just go to the YouTube and

look these up and you know you'll be

able to see I again just some of really

cool things that folks are doing with

apptive and then across the social space

too like you know Microsoft builders

there the other day and as you can see

our friend Mark renovich on the right

hand side with the Dapper the Dapper

t-shirt off so you know shout out to

Mark and the T-shirt definitely love

that I don't have a t-shirt yet I gotta

I gotta sort that out I need to talk to

somebody to get me a Dapper t-shirt I

don't know if maybe y all know a person

that can help me do that but I need to I

need to get that figured out I need to

get a daper t-shirt um but again tons of

different conferences we have box days

that happened in Brussels um you know at

the bottom right side there you could

see like there was another webinar on um

conductor from diar conductor talking

about how you can manage Dapper in Azure

um again tons of stuff on social folks

talking about using Dapper and how it

works um here we have computer weekly um

I think these ones are coming from

LinkedIn so again if folks want to head

over to LinkedIn and check some of these

things out you can as well now that

being said this is pretty much like the

end of the show um again if if folks are

interested in just contributing to app

or being a part of the community um we

we have a couple of perks not too bad

right we have some of these presenter

badges that we have so if folks are you

know willing to come on and do a

presentation here either at a conference

or maybe even at one of these Community

calls you'll be eligible to get one of

these Dapper uh presenter Badges and

then if you help us like updating the

docs or writing blog posts or anything

of the sort um you can even get this uh

you know this writer badge as well like

that's that's super helpful um and even

just for being here everyone is ible for

getting a community supporter badge

right so just for you being here you

could you know click this little QR code

scan it with your phone um you know or

pause the recording and you know

whatever it is you need to do copy that

link at the bottom and then you'll be

able to claim this badge on hollowen as

well so definitely feel free to go ahead

and grab some of those if you know you

like collecting badges right kind of

like Pokemon like you gota gotta catch

them all kind of um but with that being

said folks and thank you so much for

being here this is the end of of

community called 104 we do these

bi-weekly so you can expect the next one

in about two weeks um if you're

interested in learning more about Dapper

again we have another QR code for you or

you could even just check out some of

these links that are here we have the

Dapper website the YouTube channel um we

have tons of quick starts that are

available in our GitHub the Discord

Channel and also we're very active on

Twitter as well um so again feel free to

reach out to your own if you have any

questions about you know 114 and the

payload projection support that he talks

about when it comes to the Outbacks

pattern and and anything else that's

going on with d so again with that being

said thank you all so much for being

here really appreciate it and uh we'll

see you next time for the next daper

Community call bye everybody