The Pattern You MUST Learn in .NET

Nick Chapsas
19 Feb 202420:47

Summary

TLDRDans cette vidéo, Nick explique le concept de 'outbox pattern' pour la messagerie asynchrone dans les applications .NET. Il montre comment implémenter ce modèle transactionnel avec MassTransit pour assurer la cohérence des données lors de la publication de messages vers des files d'attente comme AWS SQS. Il explique les avantages par rapport à d'autres approches et indique qu'il préfère une autre solution serveless qu'il détaillera dans une prochaine vidéo.

Takeaways

  • 😀 Le sujet de la vidéo est le motif de la boîte de sortie (Outbox Pattern) dans les systèmes de messagerie, un sujet très demandé.
  • 📝 Nick explique le fonctionnement, l'importance et les alternatives du motif transactionnel de la boîte de sortie dans les applications de messagerie.
  • 🛠 La vidéo utilise Amazon SQS pour les démonstrations, mais les concepts sont applicables à d'autres services de messagerie comme RabbitMQ ou Azure Service Bus.
  • 🔍 Nick présente un cas d'utilisation concret avec une API client et une base de données PostgreSQL gérée dans Docker.
  • 🚀 Le motif de la boîte de sortie est présenté comme solution aux problèmes de cohérence des données dans les opérations de base de données et de messagerie.
  • 📦 Mass Transit, une couche d'abstraction pour les systèmes de messagerie, est utilisée pour illustrer l'implémentation du motif.
  • ⚙️ La mise en œuvre du motif de la boîte de sortie inclut l'utilisation de transactions pour s'assurer que les messages sont envoyés de manière fiable et cohérente.
  • 📈 L'importance de l'idempotence dans le traitement des messages est soulignée pour éviter les actions en double dans les systèmes financiers.
  • 🎓 La vidéo mentionne également un nouveau cours sur les monolithes modulaires en .NET, soulignant l'importance de la formation continue.
  • 💡 Nick encourage les retours et les questions pour approfondir la discussion sur le motif de la boîte de sortie et son utilisation dans différentes architectures.

Q & A

  • Qu'est-ce que le modèle de boîte de sortie transactionnelle dans le contexte de la messagerie .NET?

    -Le modèle de boîte de sortie transactionnelle est un modèle de conception utilisé dans les systèmes de messagerie .NET pour garantir que les opérations de base de données et les messages envoyés à la file d'attente sont traités de manière atomique. Cela signifie que si l'opération de base de données réussit, le message correspondant est également envoyé à la file d'attente de manière fiable.

  • Pourquoi est-il important de comprendre et d'utiliser le modèle de boîte de sortie transactionnelle?

    -Comprendre et utiliser le modèle de boîte de sortie transactionnelle est crucial car il aide à maintenir la cohérence des données et l'intégrité des opérations dans des systèmes distribués. Cela garantit que les événements importants ne sont ni perdus ni dupliqués en cas de défaillance partielle du système.

  • Quel est le rôle d'Amazon SQS dans le contexte de ce modèle?

    -Amazon SQS (Simple Queue Service) est utilisé comme mécanisme de file d'attente dans ce modèle pour gérer les messages de manière asynchrone. Il permet de stocker les messages qui doivent être traités, ce qui garantit leur livraison et leur traitement ultérieur, même en cas de défaillance temporaire du système.

  • Comment Mass Transit est-il utilisé avec le modèle de boîte de sortie transactionnelle?

    -Mass Transit est utilisé comme une couche d'abstraction au-dessus des mécanismes de file d'attente comme Amazon SQS. Il facilite l'implémentation du modèle de boîte de sortie transactionnelle en fournissant un cadre pour publier et consommer des messages de manière fiable et cohérente entre les différents services et la base de données.

  • Quelles sont les alternatives au modèle de boîte de sortie transactionnelle?

    -Les alternatives au modèle de boîte de sortie transactionnelle incluent l'utilisation de transactions distribuées, de mécanismes de confirmation manuelle, ou de différentes stratégies de gestion des échecs comme les tentatives de réessai ou le traitement manuel des erreurs. Chaque alternative a ses propres avantages et inconvénients selon le contexte spécifique de l'application.

  • Quel est le défi principal abordé par le modèle de boîte de sortie transactionnelle?

    -Le défi principal abordé par le modèle de boîte de sortie transactionnelle est de garantir que les modifications de la base de données et l'envoi de messages à une file d'attente se produisent de manière atomique, évitant ainsi les problèmes de données manquantes ou en double en cas de défaillance du système.

  • Comment fonctionne la détection de messages en double dans ce modèle?

    -La détection de messages en double dans ce modèle est gérée en utilisant une fenêtre de détection et en marquant les messages traités. Cela empêche le traitement répété du même message si celui-ci est envoyé plusieurs fois à la file d'attente, un aspect crucial pour maintenir l'idempotence des opérations.

  • Pourquoi est-il important d'avoir des opérations idempotentes dans les systèmes de messagerie?

    -Les opérations idempotentes sont essentielles dans les systèmes de messagerie pour garantir que même si un message est traité plusieurs fois, il ne produit qu'un seul effet significatif. Cela est particulièrement important dans les systèmes financiers ou d'autres applications critiques où les actions en double peuvent avoir des conséquences graves.

  • En quoi consiste l'interaction entre la base de données et la file d'attente dans le modèle de boîte de sortie transactionnelle?

    -Dans le modèle de boîte de sortie transactionnelle, les modifications de la base de données et les instructions pour envoyer des messages à la file d'attente sont enregistrées dans une même transaction. Ainsi, si la transaction échoue, ni les modifications de la base de données ni l'envoi de messages ne sont effectués, garantissant la cohérence des opérations.

  • Quelle est la principale préoccupation lors de l'implémentation de ce modèle en termes de performances?

    -La principale préoccupation lors de l'implémentation du modèle de boîte de sortie transactionnelle est l'impact sur les performances, car l'ajout d'une couche de gestion de transactions peut augmenter la latence et la complexité du système. Il est crucial d'équilibrer les besoins de fiabilité et de performance selon les exigences spécifiques de l'application.

Outlines

00:00

😀 Introduction au modèle de boîte de sortie dans la messagerie .NET

Nick présente le modèle de boîte de sortie (outbox pattern), un concept crucial pour la messagerie dans les applications .NET, utilisant Amazon SQS comme exemple, mais applicable à d'autres services de messagerie grâce à MassTransit. Il explique l'importance de comprendre ce modèle pour gérer efficacement les transactions et les messages entre services. Le sponsoring d'AWS et un cours gratuit sur AWS pour les développeurs .NET sont également mentionnés, ainsi qu'une démonstration de l'utilisation d'une API client et d'une base de données PostgreSQL pour illustrer la création et la gestion des clients.

05:01

🚀 Implémentation et avantages de la boîte de sortie

Cette section détaille la mise en œuvre pratique du modèle de boîte de sortie dans une application .NET utilisant MassTransit avec Amazon SQS. Nick illustre comment publier des messages dans une file d'attente à partir d'un service de gestion des clients, en soulignant les défis liés à la garantie de la consistance des transactions entre la base de données et la file d'attente des messages. Il explique ensuite l'introduction d'une 'outbox' pour assurer des opérations atomiques dans la base de données, permettant une gestion fiable et cohérente des messages sans compromettre la transactionnalité.

10:02

🔧 Configuration technique et avantages de la boîte de sortie

Nick approfondit la configuration technique nécessaire pour implémenter le modèle de boîte de sortie, en utilisant Entity Framework Core avec MassTransit. Il guide à travers l'ajout de packages nécessaires, la configuration du DbContext, et la création de migrations pour les tables nécessaires dans la base de données. Cette approche permet d'assurer l'intégrité des transactions et la fiabilité de la messagerie en enregistrant d'abord les messages dans une table 'outbox' au sein de la même transaction que les opérations de base de données, avant de les publier dans la file d'attente, abordant ainsi les défis de cohérence des données.

15:02

📦 Démonstration pratique et résolution de problèmes avec la boîte de sortie

Nick démontre l'utilisation de la boîte de sortie dans une application en temps réel, soulignant comment les messages sont traités de manière transactionnelle et sécurisée. Il explique le fonctionnement interne, y compris la gestion des états des messages dans la base de données jusqu'à leur publication effective dans la file d'attente. La section aborde également les stratégies pour gérer les échecs de publication et assurer la livraison unique des messages, soulignant l'importance de la conception idempotente des consommateurs de messages.

20:03

🎓 Conclusion et meilleures pratiques pour l'utilisation de la boîte de sortie

En conclusion, Nick résume les avantages de l'utilisation du modèle de boîte de sortie pour gérer les transactions et la messagerie dans les applications .NET, tout en offrant un aperçu des alternatives basées sur son expérience avec le cloud et les architectures sans serveur. Il invite les spectateurs à partager leurs propres expériences et solutions alternatives, clôturant sur l'invitation à explorer plus de contenu éducatif et des cours sur le développement .NET.

Mindmap

Keywords

💡modèle

Un modèle en programmation décrit une solution réutilisable à un problème de conception logicielle courant. Dans la vidéo, il discute des modèles de boîte de sortie qui sont utilisés pour garantir la cohérence des données lors de l'envoi de messages asynchrones entre les services.

💡asynchrone

La communication asynchrone signifie que l'expéditeur d'un message ne attend pas la réponse du destinataire avant de continuer son exécution. C'est important pour les performances mais cela introduit des défis de cohérence des données.

💡transactionnel

Une transaction en bases de données garantit qu'une série d'opérations réussissent ou échouent ensemble. Le modèle de boîte de sortie transactionnelle utilise cela pour s'assurer qu'un message est bien envoyé lorsqu'une donnée est mise à jour.

💡boîte de sortie

Une boîte de sortie est une table de base de données qui stocke de façon transactionnelle les messages destinés à être envoyés de façon asynchrone aux autres services quand des données sont modifiées.

💡message

Un message dans ce contexte est une charge utile de données envoyée de façon asynchrone entre les services d'une application pour coordonner leurs actions et garder une cohérence.

💡file d'attente

Une file d'attente est une structure de données où les messages sont stockés temporairement, dans l'ordre d'arrivée, en attendant d'être traités. Ici on publie les messages de la boîte de sortie vers une file d'attente de messages.

💡transport

Un transport en messagerie définit le protocole et infrastructure utilisés pour transmettre réellement les messages, comme RabbitMQ ou Amazon SQS.

💡consommateur

Un consommateur est un composant logiciel qui reçoit et traite les messages publiés dans une file d'attente par d'autres services.

💡cohérence

La cohérence de données signifie que toutes les copies des données utilisées par les différents services d'une application sont identiques à tout moment.

💡transaction

Une transaction comme expliqué précédemment permet de garantir qu'une série d'opérations de base de données réussissent ou échouent ensemble, pour maintenir l'intégrité des données.

Highlights

Introduces transactional outbox pattern for managing side effects when mutating state

Outbox pattern queues side effects after database transaction commits

Avoids race conditions between database commits and queued messages

Implemented using EF Core to manage outbox tables transactionally

Message publishing moved after database save to ensure atomicity

Background hosted service reads outbox and publishes messages

Consumer service processes published messages

Outbox entries tracked to handle publishing failures

Idempotent message processing required in case of duplicates

Can disable delivery/cleanup services for offline processing

Implementation abstracted from transport with MassTransit

Works with any message transport like RabbitMQ or Azure Service Bus

Alternative approaches for cloud and serverless architectures

Encourages feedback on other outbox implementation options

Source code available for free implementation reference

Transcripts

play00:00

hello everybody I'm Nick and in this

play00:01

video I'm going to introduce you to the

play00:02

extremely highly requested topic of

play00:05

outbox patterns in messaging Inn net

play00:08

most specifically in this video I'm

play00:09

going to show you the transactional

play00:10

outbox pattern which is one of the most

play00:13

important patterns you need to

play00:15

understand and know how to use in

play00:17

messaging in this video I'm going to

play00:18

explain what it is how it works why we

play00:20

need it and I'm also going to talk about

play00:23

some Alternatives if you don't want to

play00:24

use that pattern what other options do

play00:26

you have in this video I'm going to be

play00:28

using Amazon sqs which which is the

play00:30

queuing mechanism in Amazon and this

play00:32

video is sponsored by AWS so massive

play00:34

thank you to them however this is not

play00:36

only applicable to sqs because we're

play00:37

going to be using mass transit to show

play00:39

all this any transport or any backing

play00:42

service we'll use whether that's rabid

play00:44

mq or Azo service bus or anything else

play00:47

will work exactly as you see it now AWS

play00:49

has also sponsored a dome train course

play00:51

so if you want to get started with AWS

play00:54

Services as a cop developer click the

play00:56

link in the description it's absolutely

play00:57

free and yours to keep forever if if you

play01:00

like of content and you want to see more

play01:01

make sure you subscribe for more

play01:02

training check out my courses on doet

play01:04

tr.com okay so let me show what I have

play01:06

here I have this customer API over here

play01:09

it has one controller we can create get

play01:11

get all update or delete a customer and

play01:15

behind the scenes this is using a

play01:17

database which I am running here in

play01:18

Docker and this is a post Christ

play01:20

database to store all the users I'm

play01:22

creating so if I go over here and I say

play01:24

run this API let me show you what I have

play01:26

and here I can see my database and as

play01:29

you can see I have a single user so if I

play01:31

go to insomnia I can say get all

play01:33

customers and I'm going to get this

play01:35

customer back over here so I'm going to

play01:37

get it I can get a customer by ID so I

play01:39

can copy this ID and I can go here and

play01:42

paste it at the end of the URL and I'm

play01:44

going to get it if it is wrong I'm going

play01:45

to get a 404 and I can also create a

play01:49

customer as you can see over here so

play01:50

let's create Nick again and this is all

play01:53

created and you can see the created

play01:54

users now I'm going to go quickly and

play01:56

delete these customers now here's what

play01:59

happens in most most applications when

play02:01

you want to create a resource or update

play02:04

or delete or basically mutate some state

play02:07

which it's mutation might be important

play02:09

for some other aspect of your business

play02:11

for example when you create a customer

play02:13

you also might want to quue an email for

play02:17

email verification for example or

play02:19

anything else any aspect of the business

play02:21

and any team of the business might need

play02:23

to know that the customer was created

play02:25

updated deleted and so on so you can

play02:27

subsequently for example delete more

play02:30

information about them for gdpr purposes

play02:32

or anonymize it well you might need to

play02:34

do many things now I could just go to my

play02:37

service over here and say okay here's

play02:39

where I create my customer as I'm adding

play02:41

them into the database and saving them I

play02:44

can also say send email I can also say I

play02:47

don't know call some API I can do many

play02:49

many things but the problem I have here

play02:52

is that this method now is doing too

play02:54

much and I get very slow and the only

play02:56

thing that's important for the customer

play02:58

is that it was added to the database and

play03:00

that is it they get the response they go

play03:02

their merry way and hopefully my

play03:04

application behind the scenes can do all

play03:06

the other things now people have tried

play03:07

naive ways for years to do this for

play03:09

example they're spinning up a background

play03:11

thread and they're putting this

play03:13

functionality in the thread but what

play03:14

happens if your application is scaled

play03:16

out what happens if this is a micros

play03:18

there's so many moving parts so a great

play03:21

way to deal with this is to use a queing

play03:23

mechanism and use asynchronous messaging

play03:26

and either publish an event where people

play03:28

can subscribe and listen to that or you

play03:29

can just send it to a specific queue and

play03:32

that queue can do something later for

play03:33

example queue that email now let's see

play03:36

how we would Implement something like

play03:37

that now before I move on I'd like to

play03:38

let you know we just launched a brand

play03:40

new course on D train called getting

play03:42

started with modular monoliths in net

play03:44

and this authored by the legend Steve

play03:46

adalis Smith I'm sure Steve has taught

play03:49

many of you already with courses on

play03:50

other platforms like prite but now he

play03:52

authored his first of many courses on

play03:55

dome train and it's all about how to get

play03:57

started with modular monoliths not only

play03:59

will he teach teach you the theory

play04:00

behind the concept and how it compares

play04:02

to microservices or traditional

play04:03

monoliths but he will also build a whole

play04:06

system in that course hands on with code

play04:09

and diagrams and examples you can follow

play04:12

along it is an amazing course and it is

play04:14

the best way to get started with modular

play04:16

monol hands down in.net now to celebrate

play04:18

the launch I'd like to offer the first

play04:20

500 of you a 20% discount so either use

play04:22

a link in the description or apply code

play04:25

modular at checkout to claim that 20%

play04:27

off it's a great opportunity to get

play04:28

started with a concept and I can't

play04:30

stress enough how much of an amazing

play04:31

author Steve is now back to the video

play04:34

I'm going to go to the program.cs and as

play04:35

you can see over here I've already added

play04:38

and you get package called mass transit

play04:40

now I've already covered mass transit in

play04:43

net in a previous video so I'm going to

play04:45

link that below if you want to see how

play04:47

to get started with mass transit this

play04:49

assumes you sort of understand what mass

play04:51

transit is but if you don't all it

play04:53

really is it's an abstraction layer over

play04:55

queing mechanisms or Pub Subs such as

play04:58

rabbit mq or maybe sqs Aus service bus

play05:01

this sort of thing so you code against a

play05:03

common abstraction and then behind the

play05:05

scenes M Transit can Implement all those

play05:08

other transports specifically so the

play05:10

only thing that changes is me saying M

play05:13

transit. Aur service bus for example to

play05:16

go from sqs to service bus and so on

play05:19

we've seen all that in the previous

play05:21

video so now that I've configured M

play05:23

Transit here and I'm saying hey use this

play05:25

sqsq over here and configure all the end

play05:27

points all I need to do to introduce the

play05:30

messaging functionality is go to my

play05:32

service and say private read only I

play05:36

publish endpoint so I'm going to use a

play05:38

publish endpoint here which is scoped by

play05:40

default in registration and I'm going to

play05:42

take it and I'm going to say oh here is

play05:44

where I want to publish a message and

play05:46

how convenient I have a few message

play05:48

contracts over here so this is the

play05:50

record representing a customer being

play05:52

created so I'm going to go here and I'm

play05:54

going to say VAR message equals new

play05:58

customer created and you can have a

play06:00

mapper if you want all I'm going to do

play06:02

is I'm going to manually map this user

play06:04

over here so ID full name email and date

play06:08

of birth and then what I'll say is await

play06:10

the publish endpoint and then publish

play06:13

that message and that is it and that

play06:15

then we'll publish it into the queue and

play06:17

just to see all that working I'm going

play06:19

to stick a breakpoint here and just say

play06:21

debug my application so I'm using AWS I

play06:24

have the toolkit over here to see all my

play06:26

messages in the sqs you can see the cues

play06:28

are automatically created because must

play06:30

Transit will do that and I'm going to go

play06:33

ahead and say create a customer so send

play06:36

that request I'm hitting my breakpoint

play06:38

I'm going here I'm saying add the user

play06:40

to the database and then I get my

play06:41

message and over here I am publishing

play06:45

that message so if I go to customer

play06:47

created the queue and I view messages

play06:50

you can see it over here it's already

play06:52

published and you can see everything

play06:54

about it over here and then after it's

play06:58

published I also saved to the database

play07:00

and thankfully it was saved successfully

play07:04

and eventually I have as you can see

play07:06

over here a consumer and if you want to

play07:08

grab this code you can use a link down

play07:10

below to get it for free but I can go to

play07:12

this worker over here which has some

play07:14

very simple configuration and it has

play07:17

consumers for every type of message in

play07:19

this case all it's really doing is it's

play07:21

logging don't do logging this way by the

play07:23

way this is not the right way to do

play07:25

logging I'm just doing it like this

play07:27

because I want to print something to the

play07:28

console to show that I consumed it so

play07:30

I'm going to run the worker and as

play07:32

you're going to see very quickly we're

play07:33

going to see that the customer created

play07:36

message has been consumed if I go ahead

play07:38

and I create a second customer you will

play07:40

see immediately that another request has

play07:43

been filed and it has been processed

play07:46

through a new message fantastic very

play07:48

easy messaging in the app now do you see

play07:51

where the problem is with this you might

play07:52

have spotted it the problem is that I'm

play07:55

publishing this message which by the way

play07:57

it could also be below the creation so

play08:00

we sort of ensure that something was

play08:03

created and only if it was created maybe

play08:06

let's say result is more than uh zero

play08:09

only then we publish the message which

play08:12

is a nice safe guard to make sure we

play08:15

don't publish a message for something

play08:16

that wasn't done but what guarantees

play08:19

that yeah okay this was added to the

play08:22

database but do we know for a fact that

play08:24

this was published to the queue we don't

play08:27

we can't yes there are ways to get

play08:30

around this with distributed

play08:31

transactions and two-phase commits but

play08:35

this is a very very hard problem in

play08:38

general in applications because you

play08:40

can't have Atomic operations between the

play08:43

database you're using and the message

play08:45

bus you're publishing towards and then

play08:47

what do you do if this failed you roll

play08:49

back and what if the roll back fail like

play08:51

this such a big problem and chicken and

play08:54

neck problem you have to solve here and

play08:57

there's no obvious easy solution but but

play08:59

the truth is there actually is it's both

play09:02

very easy to implement and pretty

play09:04

obvious you see we cannot have an atomic

play09:07

operation between a database and a

play09:09

message CU but we can have an atomic

play09:12

operation in a database you can have a

play09:14

transaction that does many things it

play09:17

adds the customer and then it adds the

play09:20

instruction to do something about the

play09:22

added customer so what instead of

play09:24

writing in a single table the customer's

play09:27

table over here we also had a second

play09:30

table called the outbox table and

play09:32

there's actually more into that behind

play09:33

the scenes but let's in theory just say

play09:35

we have the outbox table saying that hey

play09:38

a customer was created and then a

play09:41

service picks that up after it's been

play09:43

transactionally written and it has to

play09:46

push it into a queue and then that queue

play09:49

is read by a consumer and that is

play09:50

published so if I was to use a very

play09:52

scientific tool to demonstrate this I

play09:54

can go here and I can say that instead

play09:57

of writing into a single place over here

play10:01

which in this case this is the customers

play10:03

table let's also in the same transaction

play10:07

because now we are on a database like I

play10:09

said write something to the outbox then

play10:14

have something and we will see what that

play10:16

something is listen to that outbox and

play10:20

see hey do you have any messages for me

play10:23

to process and then that thing will not

play10:27

only read those messages but also it is

play10:30

going to publish them as we're going to

play10:32

see here to AQ so take that message and

play10:36

publish it now there is a bit of an

play10:39

issue here where you can have the

play10:41

situation where if this is trying to

play10:43

read from the outbox and then get the

play10:44

message and publish it what if the

play10:46

publishing fails well you can update the

play10:49

state of the outbox and say that this

play10:51

message hasn't been processed yet and

play10:53

only say that it has after you guarantee

play10:56

that it is been pushed into the message

play10:58

box the usual problem with all this is

play11:00

that in some implementations you cannot

play11:03

guarantee exactly one delivery with this

play11:06

process so you need to have IDM potency

play11:08

baked into your message processing

play11:10

meaning that if a message is pushed

play11:12

twice into the queue or consumed twice

play11:15

your application only makes one action

play11:18

not two having worked in finance dealing

play11:20

with payments this was the bread and

play11:22

butter for everything we did you don't

play11:24

want to have a message or an event

play11:25

handled twice because if you do someone

play11:27

might pay twice

play11:29

and people don't like paying twice if

play11:31

they have only paid once now you could

play11:32

definitely go and manually Implement all

play11:34

this but you don't have to because mass

play11:37

transit natively supports the outbox

play11:39

pattern the transactional outbox pattern

play11:42

on top of an inmemory one which I'm not

play11:44

going to cover so how will we introduce

play11:46

the pattern here well first what I'm

play11:48

going to do is I'm going to move this

play11:51

message publishing between the action

play11:54

I'm taking on the database the addition

play11:56

and before the save change is a sync and

play11:59

I'm going to paste it here and I'm still

play12:01

saying publish so the code I have here

play12:03

will publish into the queue nothing

play12:06

about the database as it is now I'm

play12:08

going to go to my API new packages and

play12:11

I'm going to say Mass transit. entity

play12:14

framwork core and this is natively

play12:16

supported with ENT framework you can

play12:19

just add ENT framework just for this you

play12:21

don't have to use it for your whole

play12:22

application but to simplify management

play12:25

of this whole process M Transit chose to

play12:27

use EF core which by the way is fine

play12:30

it's performance it's fast it is great

play12:32

it's net 8 we're good and once I do that

play12:35

I can go to my app DB context and I have

play12:38

to specify in the model that there are

play12:40

some tables that need to be created for

play12:42

example I'm going to say override on

play12:44

model creating and I'm going to add

play12:46

three lines first I'll say model

play12:48

builder. add inbox State entity then

play12:51

model builder. add outbox State entity

play12:55

and I'm also going to say modelbuilder

play12:56

do add outbox message entity those three

play13:00

things is what I need and once I specify

play13:02

that I need a bit more configuration on

play13:05

the admass transit method so what I'm

play13:07

going to say in there is x. add Entity

play13:11

framework outbox and I'm going to

play13:14

specify my DB context so add DB context

play13:17

some configuration over here and I'm

play13:19

going to say o do use postgress and also

play13:23

use bus outbox and I'm going to leave

play13:25

the defaults for now I will also specify

play13:29

here in this service o do query delay

play13:32

and I'm going to set that to 1 second

play13:34

I'm going to explain what that is in a

play13:36

second I'm doing this for demo purposes

play13:38

so we can have all the actions happen

play13:39

quickly once I do that I want to create

play13:42

migrations for this new feature because

play13:44

there will be tables needed and these

play13:46

tables are configured through this so

play13:48

I'll say net EF migrations add outbox

play13:52

and let's get that created here we go

play13:54

migrations created so if I go here I can

play13:56

see all the migrations and all the the

play13:58

code needed for those new tables and

play14:01

then I can say netf database update and

play14:04

once I do that we're going to get all

play14:05

those tables created if I refresh the

play14:07

tables you're going to see inbox State

play14:09

outbox message and then outbox state so

play14:12

that will now handle our outbox pattern

play14:14

and now that's it I'm not going to touch

play14:16

anything else about my code meaning that

play14:19

this publishing the dot publish that's

play14:21

supposed to push into a que won't be

play14:24

touched now let me quickly show you what

play14:26

you need to do on the consumer side of

play14:27

things first we need to add a couple of

play14:30

packages so first we need the must

play14:31

Transit entry framework core package we

play14:33

added in the main API project and then

play14:35

we'll also add the postgress um an

play14:38

framework core then we're going to

play14:39

create an app DB context it doesn't have

play14:41

to be the same as the main application

play14:43

this is just here for the outbox this

play14:46

means that there's no other DB context

play14:48

needed in here or DB set all you need is

play14:50

a traditional model creation that is it

play14:52

now back to the program.cs in the add

play14:55

mass transit call we're going to say add

play14:58

anti framework outbox appb context as in

play15:02

the other application and here all we're

play15:04

going to say is we're going to have a

play15:06

duplicate detection window to detect

play15:10

duplicate messages from the inbox so I'm

play15:12

going to have that as time span Dot from

play15:15

seconds 30 and then I'm going to say hey

play15:18

use postgress but nothing about the bus

play15:20

outbox just use postgress and that is it

play15:23

now for all this to work I also need to

play15:25

wire up of course the app DB context so

play15:28

I'm going to say add DB context as I

play15:30

would normally do app DB context and

play15:33

then configure that to use postris and

play15:36

that is it all I'm going to do now is

play15:38

I'm going to run my worker which is

play15:40

supposed to be listening to the cues and

play15:42

as you can see it is listening to all

play15:44

the cues and then I'm going to exactly

play15:47

as it is I want to remind you I did not

play15:50

touch the publishing message I'm going

play15:53

to do is I'm going to stick a breakpoint

play15:55

and run the API and let me show you what

play15:59

happens in fact I'm going to remove the

play16:01

breako for now and first I'm going to

play16:03

publish a message so I'm going to go

play16:05

straight over here into insomnia and I'm

play16:07

going to say create a customer so

play16:09

customer created a few things are

play16:11

happening here we don't really care

play16:12

about that but if I go to the worker as

play16:16

you can see we did process the worker

play16:19

because now we have the customer created

play16:22

consumer message and all the details

play16:25

that's it so nothing seems to have

play16:27

changed because it still seem like I

play16:29

just published into a que so what

play16:31

problem did I actually solve well let me

play16:34

show you I'm going to go to the customer

play16:35

service and I'm going to slap a

play16:36

breakpoint on the add a sync method

play16:38

which I want to remind you in anyti

play16:40

framework core doesn't take any action

play16:42

it just marks The Entity to be added

play16:45

when you call Save changes which again

play16:47

is transactional so let's go ahead and

play16:49

say create another customer and let's

play16:52

hit that breakpoint we marking that

play16:54

entity and then normally on this publish

play16:57

call we would publish something into the

play16:59

queue directly going to step over that I

play17:01

haven't called save changes a sync yet

play17:04

but I want to show you we have nothing

play17:07

in the queue let me just save you

play17:09

messages the queue is empty and you

play17:11

might expect that because the worker is

play17:12

running so you're assuming I publish

play17:15

something and it's been consumed by the

play17:16

worker the truth is it hasn't there is

play17:20

no message consumption because nothing

play17:23

has been published yet the moment

play17:26

however I say save changes sync then and

play17:30

only then something will happen and what

play17:32

will happen is in the database if I go

play17:34

to the outbox State you're going to see

play17:36

the state of an outbox message and all

play17:39

its details and then in the outbox

play17:41

message you're going to see a new entry

play17:43

over here which just was added with this

play17:46

save async call alongside my new

play17:49

customer which contains in the body the

play17:53

whole message serialized so now the

play17:56

moment I say run freely don't stick to

play17:59

that break point and don't stay there to

play18:02

my service my service behind the scenes

play18:05

will use a bus running in the background

play18:08

the hostage service running in the

play18:09

background and push those messages and

play18:11

as you can see it's not here anymore and

play18:13

push that into the queue and my worker

play18:16

then will pick it up and as you can see

play18:18

it has been processed that's the whole

play18:20

point of the outbox we are doing our

play18:23

action transactionally and we're adding

play18:25

an entry into the outbox and then later

play18:28

service picks it up and it pushes it

play18:30

into the message so both things happen

play18:32

automically it is implemented in a great

play18:34

way in mass transit is very simple to

play18:36

work you don't need to change your code

play18:38

this all still looks like two

play18:40

independent calls but behind the scenes

play18:43

this is all transactional now you could

play18:45

stop that worker service that picks the

play18:48

outbox and publishes it if you go over

play18:50

here and just to remind you this query

play18:52

delay is a delay that the service that

play18:54

will read the messages in that outbox

play18:57

and take them and push them will have to

play18:58

perform so we don't just Spam the

play19:00

database with many many requests per

play19:02

second basically so in this whole thing

play19:04

I could also say that disable that

play19:07

delivery service that background service

play19:09

logic because you just want to use the

play19:11

adbox and you might want to have offline

play19:13

processing so there's no que there's no

play19:15

bus working you might also want to go

play19:18

further and say disable the inbox

play19:20

cleanup service you have full control

play19:22

and you can even run this service that

play19:24

looks into the database and pushes

play19:26

messages into the queue to run out of

play19:29

this process so you split that

play19:30

responsibility to another service and

play19:32

then you have the worker that still

play19:34

consumes straight from the queue but it

play19:37

acknowledges that the Out book exists so

play19:39

it can also mutate the state as needed

play19:41

and like I said if you wanted to use any

play19:43

other provider with any of this you

play19:45

could easily just go here and say mass

play19:48

transit. rabbitmq for example remove

play19:51

Amazon sqs why that up and the only call

play19:54

that needs to change is the using

play19:56

whatever you want to use r mq or any

play19:59

other provider you want in this case sqs

play20:02

is what I've used in production

play20:03

extensively so that's the one I'm using

play20:06

and that's it your code now just

play20:07

supports the transactional outbox

play20:09

pattern again you're going to find all

play20:11

this code in the description down below

play20:12

it's all for free and if you have any

play20:14

questions please leave them in the

play20:15

comments down below now something I want

play20:17

to point out is that this is not my

play20:19

favorite way to deal with this problem I

play20:21

have a different approach having worked

play20:24

in the cloud and in serverless for an

play20:27

extensive period of time we actually

play20:29

deal with this problem differently and

play20:30

if you want to know how we deal with

play20:32

that problem leave a comment down below

play20:34

and I will make a video on that but now

play20:35

when from you what do you think about

play20:37

this and are you using something else to

play20:39

implement it maybe you're using your own

play20:41

approach leave a comment down below and

play20:42

let me know well that's all I had for

play20:44

you for video thank you very much for

play20:45

watching and as always keep coding