Don't Make this Golang Beginner Mistake!

Summary

In this video, the speaker discusses common mistakes made by Go developers, particularly in handling database connections within a Kafka consumer setup. They emphasize the importance of abstraction to avoid hard dependencies on specific database technologies. The speaker introduces the concept of a 'log persister' interface to demonstrate how to implement dependency injection, allowing for more flexible and maintainable code. They also promote their private community, 'The Total Coder,' where they provide weekly assignments and live streams to review code and offer insights.

Takeaways

• 📘 The speaker emphasizes the simplicity and readability of the Go language, but notes a common mistake made by many, especially newcomers.
• 👨‍🏫 The Total Code community is mentioned as a private space where assignments are sent out weekly for members to solve in any language, predominantly Go.
• 🎥 The speaker conducts live streams to review assignments and resumes, highlighting common issues to help members improve their coding practices.
• 🔍 The script discusses a specific assignment related to log ingestion using Kafka, where the participant made a common mistake with database connections.
• 🔗 The mistake identified was the instantiation of a new MongoDB client within a loop, which is inefficient and not a good practice for handling large volumes of data.
• 🔄 The speaker points out a structural problem in the code, where the hard dependency on a specific database client makes the code inflexible for different storage solutions.
• 🔑 The solution proposed is to use an interface, such as 'LogPersister', to abstract the database operations and allow for more flexible and maintainable code.
• 🔄 By using dependency injection with the 'LogPersister' interface, the code becomes decoupled from specific implementations, making it easier to switch or test different storage solutions.
• 🛠️ The importance of good Go practices is stressed, such as using interfaces and context as the first argument in functions, to align with common third-party library usage.
• 📚 The speaker invites viewers to join the Total Code community for more learning opportunities and to avoid common mistakes in Go programming.
• 👍 The script concludes by encouraging viewers to reflect on their coding habits, learn from the discussed mistake, and engage with the community for further improvement.

Q & A

What is the main issue the speaker identifies with the code in the script?

The main issue is that the code makes a new MongoDB client connection every time it processes a log, which is inefficient and not a good practice, especially when dealing with a high volume of logs.

What is the speaker's recommendation to avoid creating a new MongoDB client in each iteration of the loop?

The speaker suggests using an interface to abstract the database client creation, allowing for more efficient and flexible handling of different types of persistent storage.

What is the term used for the practice of creating a new database connection for each operation?

The term used is 'overkill', as it implies unnecessary complexity and inefficiency in the code.

What is the purpose of the 'total code' private community mentioned by the speaker?

The 'total code' private community is a place where the speaker sends out weekly assignments for members to solve, and then reviews those assignments in a private live stream, offering feedback and guidance.

What is the benefit of using an interface in the context of the script's code?

Using an interface allows for dependency injection and makes the code more modular and flexible, enabling the use of different types of persistent storage without modifying the core logic of the application.

What is the term for the technique where a specific implementation is replaced with a more general interface?

The term is 'abstraction', which helps to simplify the code and make it more maintainable and scalable.

What does the speaker suggest as an alternative to hard-coding the MongoDB client in the consume function?

The speaker suggests creating an interface called 'log persister' with a method 'persist log', which can then be implemented by different types of storage systems.

What is the advantage of using dependency injection in the context of the script?

Dependency injection allows for more testable and maintainable code by decoupling the specific implementation of a service from the code that uses it.

What is the 'context' argument mentioned in the script, and why is it important?

The 'context' argument is a way to pass request-scoped values across API boundaries and is important for managing request lifecycle, such as timeouts and cancellations.

What does the speaker mean by 'no-op' in the context of the script?

A 'no-op' (no operation) is a function or method that does nothing. In the context of the script, it could be used as a placeholder when no persistence is required.

What is the speaker's advice for those who want to improve their understanding of Go (Golang)?

The speaker suggests joining his Discord community, the private 'total code' community, or buying his courses to learn more about Go and improve their coding skills.

Outlines

00:00

📝 Common Mistakes in Golang Projects

In this paragraph, the speaker discusses the simplicity and readability of the Golang language, yet points out a recurring mistake made by many developers, particularly newcomers. The mistake isn't due to a lack of skill but rather a lack of awareness of best practices. The speaker introduces their private community, 'Total Code,' where they assign tasks to members and review their work in live streams. The focus is on a specific assignment related to log ingestion using Kafka, where a member made a good attempt but fell into the common trap of creating new database connections within a loop, which is inefficient. The speaker emphasizes the need for abstraction to avoid hard dependencies and suggests using interfaces to improve the code structure.

05:00

🔍 Abstracting Dependencies in Golang

The second paragraph delves into the concept of dependency injection and abstraction in Golang. The speaker critiques the hard dependency on a MongoDB client within a Kafka consumer function and suggests using an interface to abstract the persistence logic. By defining an interface with a 'persistLog' method, the speaker demonstrates how to pass this interface as a dependency to the Kafka consumer, allowing for flexibility in choosing the persistence method, such as MongoDB, Elasticsearch, or even a no-operation (no-op). This approach not only improves the code's maintainability but also its scalability, making it easier to adapt to different storage solutions or testing scenarios. The speaker encourages developers to understand and apply these principles to stand out in technical interviews and offers resources like Discord communities and courses for further learning.

Keywords

💡Goang

Goang is a misspelling or colloquial reference to the Go programming language, which is known for its simplicity and readability. In the video, the speaker mentions that many people have a good grasp of Go because it is 'very simple' and 'very easy to read and write.' The term is used to set the context of the programming language being discussed throughout the video.

💡Total Code

Total Code refers to the speaker's private community, which is a platform where members receive weekly assignments to solve in any programming language, with a majority opting for Go. The community provides a space for peer review and learning, as the speaker mentions conducting live streams to review assignments and resumes.

💡Assignment

An assignment in this context is a task or problem given to the members of the Total Code community to solve using a programming language. The video discusses an example of an assignment involving log ingestion and Kafka implementation, highlighting common mistakes made by participants.

💡Kafka

Kafka is a distributed streaming platform used for building real-time data pipelines and streaming apps. It is mentioned in the script as the technology the community member used for the log ingestion assignment, where JSON data representing a log file is produced and consumed on a Kafka queue.

💡MongoDB

MongoDB is a NoSQL database that stores data in a JSON-like format. In the video, the speaker points out that the community member's code instantiates a new MongoDB client within a loop, which is not a good practice due to the overhead of creating new connections repeatedly.

💡Interface

In Go programming, an interface is a type that specifies a method set which any other type can implement. The speaker suggests using an interface, named 'log persister' in the script, to abstract the database persistence logic, allowing for more flexible and maintainable code.

💡Dependency Injection

Dependency injection is a design pattern where an object's dependencies are provided to it, rather than the object creating or requesting its own dependencies. The speaker recommends using dependency injection by passing the 'log persister' interface to the Kafka consumer to decouple the consumer from the specific persistence implementation.

💡Structural Problem

A structural problem in code refers to an issue with the organization or architecture of the code that makes it inflexible or difficult to maintain. The video identifies a structural problem where the Kafka consumer is tightly coupled with the MongoDB client, making it hard to switch to a different persistence mechanism.

💡Persist

To persist data means to store it in a way that it can be retrieved and used later. In the context of the video, the community member's task was to consume log data and persist it efficiently in a persistent store, such as MongoDB or Elasticsearch.

💡Context

In Go, a context is used to pass request-scoped values and cancellation signals across API boundaries and between processes. The speaker emphasizes the importance of including a context as the first argument in interfaces, especially for third-party libraries that use context for managing request lifetimes.

💡No-op

A no-operation (no-op) is an empty operation that does nothing. The speaker uses the term to illustrate that the 'log persister' interface could be implemented in such a way that it does not persist anything, which might be useful during testing phases or other scenarios where persistence is not required.

Highlights

The speaker emphasizes the simplicity and readability of the Go language but notes a common mistake made by many developers.

Introduction to the Total Code private community where weekly assignments are given and reviewed.

The importance of learning from mistakes and the role of the Total Code community in facilitating this learning process.

A detailed review of a private community member's assignment, highlighting a mistake related to database connections in Go.

The inefficiency of creating a new MongoDB client within a loop for log consumption in a Kafka implementation.

The structural problem of hard dependencies in Go code, particularly with database clients.

The concept of dependency injection as a solution to hard dependencies in Go code.

The introduction of an interface, 'LogPersister', to abstract the persistence mechanism in Go code.

The benefits of using interfaces for dependency injection, allowing for more flexible and maintainable code.

A live coding example demonstrating the implementation of the 'LogPersister' interface in a Kafka consumer.

The elimination of hard dependencies by passing the 'LogPersister' interface as an argument to the Kafka consumer.

The ability to easily switch between different persistence mechanisms by simply changing the implementation of the 'LogPersister' interface.

The importance of context in Go interfaces, especially when dealing with third-party libraries.

A discussion on the common mistake of not abstracting dependencies in Go, and how to avoid it.

The speaker's invitation for viewers to join the Total Code community or Discord for more learning opportunities.

The speaker's offer of courses and live streams for those interested in learning more about Go.

A call to action for viewers to share their thoughts and experiences with the discussed concepts in the comments.

The conclusion of the video with a reminder of the value of learning from mistakes and the resources available for Go developers.

Transcripts

so after doing a lot of reviews recently

I see that a lot of people have a very

good grasp of goang because it's a very

simple language very easy to read and

very easy to write but I see people

making the same mistake over and over

and it's not basically a skill issue

it's just something that you need to

learn and if nobody tells you it you

will never learn that's why I'm here at

the total code is my private community

and each week I sent out an assignment

that you can solve right you can solve

that in any language as you want but

most people solve that in goang and by

the end of the week most likely on

Sundays what I do is I basically uh do a

private live stream for the members of

that Community where I review resumes

but I also review your assignments or

other projects so this is basically an

assignment of somebody from my private

Community he did a very good job but um

he made a mistake that I see a lot of

people making especially if you're new

to golang it's nothing to do with your

skill issues it's just something you

don't know and you need to learn that

right and that's why we are here so uh

hey if you're interested in the total

coder Community you can uh find that in

the link down below all right so let's

get started so the previous assignment

we needed to do something with loog

ingestion uh and he his implementation

was more of a Kafka implementation and

let me show you what's going on here so

we are producing some Json data which is

representing a log file and we produce

that on a Kafka queue right and of

course if you produce on a Kafka queue

we're also going to consume from that

queue and basically this is the Kafka

consumer structure which holds some data

which holds some some variables here and

the problem is hereat right so very

simple it's going to consume heat it's a

for loop it's going to read a message

it's going to basically Marshall that

into an log data because the assignment

was hey we're going to provide you some

log data and you need to basically

consume that and store that efficiently

in some kind of a persistent store you

can choose whatever that is mongodb

elastic search whatever right you could

pick whatever you want and the problem

is here right so because this is a loop

right every time we consume he's going

to actually two mistakes here each time

we going to instantiate a new mongodb

client right so we're going to make a

new mongodb connection which is not a

good practice because if you're

producing a lot of logs and you're

consuming a lot of stuff you're each

time going to create this new client

which is basically a little bit of an

Overkill right and once that client is

basically instantiated we are going to

insert that into a DB and send the A and

call it a day so from a first inspection

of this scol there's nothing really

wrong with it right it works perfectly

fine uh just well this could be more

optimized like I mentioned this client

but there is basically a little bit of a

structural problem here right because

what is going to happen if we want to

say Hey listen right now we do uh a

Mongol persistent store but what if we

want to use elastic search or we want to

use something else right uh we want to

write it to dis or we don't want to do

anything at all because we're in a test

phase or something we want to just have

a uh we just want to log it out or

something we don't want to persist it so

if we want to do that each time we

basically need to or we need to write a

new function or we're going to basically

just comment this out or Del it and make

a new implementation here right for

example elastic search or something so

you can see this basically a problem a

structureal problem actually it's a

dependency problem why well because this

client here this moongo client is

completely encapsulated into this

consume function so it's a very hard

dependency right this kka consumer

depends so hard on this client

that it cannot yeah it's just a very

hard dependency it's not an AB abstract

type right so in goang how is a very

common mistake and it's not only this is

just a Kafka example but it's basically

just the repository the DB um dependency

a lot of people tend to forget to

abstract way right so how can we solve

that well it's actually very simple

right I'm going to just do some some

pseo code here instead of basically

creating your moo client here what he

needs to do is it could make an

interface for example uh a persistor

actually it it's locked so we could do

something like a log persister and it's

going to be an interface and what could

be a good interface design here it could

be something like

persist um persist log for example and

as a good goaling engineer we're going

to give this a context context right and

then because it's an interface and

especially actually in a lot of

functions you want to have a context as

a first argument but espe specifically

for these interfaces because um yeah

a lot of these thirdparty libraries like

MB or or elastic search or whatever they

use context right so if you have an

interface that basically supports that

that's basically a big win and it's a

very very good practice to do so persist

log and we going to have I think it is

data log data something like that right

and of course an error right this could

be the log persist interface and the

only thing we need to do is say Hey

listen we're going to actually embed

well embed we're going to uh attach is

also not a good word how do you call

that we're going to basically provide

this persist you could say uh persistor

for example can I actually write this

persist is going to be a log persist

interface right and if you want to for

example new Kafka consumer it's going to

take a Kafka config but it's also going

to take for example the uh

persistor which is going to be the log

persistor right and then I'm going to

say persistor here is the persistor that

we provide as an argument which is

basically H some kind of dependency

injection right there a very big rabbit

hole we can go into this uh but that's

what I call dependency injection with

this interface provided into this Kafka

consumer structure and right now U we

basically completely abstracted the way

how our persist is going to persist here

right so we can actually

completely what's going on here we can

completely uh delete this thing and

instead

of connecting it's already connected

right it's already in entiated uh

because you're going to provide that in

the new kka consumer uh constrictor

function whatever you want to call that

and the only thing we need to do is

basically say if R is going to be uh is

the c. persist do persist log and we're

going to take in a context do we have

that yes we have one that's already a

win the context here and then we going

to put in the data which is the log data

here and then if the error is not n we

are basically going to copy some stuff

from here like this copy that paste that

in delete this and save and that's going

to be the code right of course we need

to change some other stuff uh because

yeah right now we have an extra argument

here in um new Kaa consumer but it

doesn't really matter right this is

completely out of scope so right now you

can see that if we want to consume it's

just going to call the abstract the

interface basically from the persistor

actually look at that I'm making this

small uh typo mistakes here

persister uh just like that and of

course we need to pers look at that man

the pers just like that voila now it's

perfectly fine so right now we don't

depend on a specific uh persisting type

like MB or elastic search right now we

just basically depends on the interface

and anything that has the persist log um

function or method attached to it uh

basically uh is eligible to uh be

inserted as this argument in a new

consumer can be elastic search could be

Mong be could be whatever you want could

be a no op it could be uh whatever you

want right as long as it basically uh

implements this persist log method so

and this is a very common mistake uh of

course the code will work right the code

will work like he did but like I said if

you want to do an interview assignment

and they see that you basically

understand that you need to abstract

that out

um you're going to have yeah I think

you're going to have a more better

chance than people that do not do that

right so let me know if you make the

same mistake or uh that you learned this

or whatever let me know in the comments

what you think about that and of course

hey if you want to learn more about

goang like I said join my Discord

Community or join the private Community

or buy some of my courses whatever uh

that's all up to you and I'm looking

forward to see you in one of my live

streams or next videos peace out