How I Structure New Projects In Golang
Summary
TLDRThe video discusses setting up a scalable project structure for a Go web application using Echo as the framework. It advocates for a pragmatic approach focused on simplicity, avoiding over-engineering. The proposed structure includes folders for binaries, commands, database logic, handlers, business logic packages, shared types, and utilities. It demonstrates this structure in an app called Leue - a marketplace to sell recurring revenue for upfront cash. The video argues developers should be full stack and balance tradeoffs themselves rather than follow dogmatic paradigms.
Takeaways
- 💻 The video discusses transitioning a company's project, named Laue, from Next.js and TypeScript to Go and Svelte Kit due to a significant increase in funding and project scope.
- 🔨 The presenter emphasizes there's no perfect project structure in programming; instead, it's about choosing what works best for the team and being willing to adapt.
- 💡 Encourages a result-driven development mindset over stressing about tools, packages, and organization methods.
- 🛠️ Explains the Go project structure, including the use of a 'bin' folder for binaries, a 'cmd' folder for main package entry points, and other organizational strategies.
- 📰 Highlights the flexibility of Go frameworks for the backend, stating that the choice of framework (Echo, Gin, etc.) doesn't significantly impact the end result.
- 💥 Discusses performance concerns, suggesting that for most projects, performance won't be a limiting factor until a very large user base is reached.
- 📖 Offers insights into database interaction, the use of global variables for ease of use, and the pros and cons of different approaches to handling database queries.
- 🏆 Shares a pragmatic approach to coding, advocating for simplicity and focusing on getting results rather than adhering strictly to certain programming paradigms.
- 📈 Talks about the importance of integrating business logic and data handling correctly within the project's structure to ensure scalability and maintainability.
- 📚 Advises on the significance of having a balanced view on technologies and methodologies, emphasizing that what works well for one team or project may not for another.
Q & A
What is Leue and what was the original technology stack used to build it?
-Leue is a company that the speaker created 3 years ago. The original project was built using Next.js for the front-end and TypeScript for the back-end.
Why did the speaker decide to rewrite Leue and change the technology stack?
-The scope of the project changed dramatically after the company received a large amount of funding. The speaker wanted to scale up the project, so they decided to rewrite the back-end in Go and use SpellKit for the front-end.
What is the speaker's view on choosing frameworks and architectural patterns?
-The speaker believes there is no one "perfect" choice. Developers should use what feels right for their needs instead of forcing specific paradigms. Performance differences rarely matter until an application reaches very large scale.
How does the speaker structure Go projects and why?
-The speaker uses a simple and modular structure with folders for handlers, packages, data, types, etc. This makes it easy to understand and scale up the project over time as needed.
Why does the speaker use a global database connection variable?
-For ease of use across different packages. The speaker argues this is okay if used carefully and may not matter much for testing since database access should be integration tested.
What is the purpose of the CMD folder and how does it work?
-The CMD folder contains main.go files that bootstrap different entry points into the application, like API servers. The binaries are then built into the bin folder for execution.
Where is business logic implemented in this structure?
-Most business logic is contained in the packages folder, separate from the database access in the data folder and HTTP handler functions.
How are endpoints and handlers organized?
-Handlers are organized by HTTP method into handler functions. Endpoints are grouped by concepts like buy/sell side using middleware.
How does the front-end connect to the Go back-end?
-The front-end uses SpellKit and calls API endpoints exposed from the Echo server in the main CMD API bootstrap.
What is the speaker's view on specialization between front-end and back-end development?
-The speaker argues developers should have full stack capabilities instead of identifying as a front-end or back-end specialist, since it's all just writing code.
Outlines
😊 Introducing the leue project and its transition to version 2.0
The paragraph introduces leue, a company created 3 years ago that is transitioning from a Next.js and TypeScript stack to Golang and SvelteKit. The scope and funding have increased dramatically, necessitating scaling up the backend in Golang. The video will focus on setting up a Golang project structure.
🤔 Using simple scripts over CLI packages for database management
The paragraph discusses using simple scripts to handle things like dropping databases and running migrations instead of CLI packages. These scripts are run through Make commands for convenience. A makefile is simple and commonly installed on Linux.
😉 Embracing Bun as the ORM over sqlc despite drawbacks
The paragraph analyzes the tradeoffs between using sqlc for type-safe queries versus Bun for more dynamic queries. Despite drawbacks like losing type safety, Bun is ultimately embraced for this project.
🧐 Separating code by responsibility into different packages
The paragraph explains the structure of separating code into different packages based on responsibility - data layer for db access, handlers for HTTP handling, package for business logic, types for shared types, and util for helpers. This structure scales well.
💪 Being a well-rounded developer to create full-stack applications
The paragraph advocates for being a full-stack developer comfortable on both frontend and backend instead of limiting yourself. Any language or framework has tradeoffs - focus on weighing pros and cons based on your own needs.
Mindmap
Keywords
💡project structure
💡code organization
💡framework
💡global variables
💡data layer
💡handlers
💡business logic
💡shared code
💡architectural patterns
💡simplicity
Highlights
There's no such thing as a perfect project structure, do what feels right and change it along the way
Focus on results instead of how you program things
Performance doesn't matter until you have a large user base, then it's a luxury problem
Framework doesn't matter, they all get the same results
Use simple scripts instead of complex CLI packages
Group database logic in a data folder, keep it separate
Test business logic with units tests and database interaction at integration level
Handler functions handle the HTTP requests and extract JSON
Business logic goes in the packages folder if needed
Types folder stores shared types between packages
Util folder has helper functions
Start simple then evolve architecture over time
Don't be a backend/frontend developer, be a problem solver
Weight pros and cons to see if something works for you
Project structure can scale as big as needed
Transcripts
so we're recently starting to rewrite
leue into the 2.0 version for the people
I don't know what laue is it's a company
I created three years ago and the
original project was basically written
in next GS everything in typescript
which worked very well but right now we
received a big amount of funding and the
scope is changed dramatically so it's
time to scale up and we decided to write
the back end in goaling and use spelt
kit as the front end right so uh and in
this video I'm going to show you how we
set up our gooling project structure
because it's still a question that comes
up a lot like Hey how do you structure
gooling projects and before we continue
if you like the videos that I'm
providing to you please consider
subscribing give me a thumbs up leave
some questions in the comments and jump
into the 4,000 people big Discord server
where we learn from each other each and
every single day and for the people that
want to level up I also have the
full-time GF course which is 50% off
right now for a limited amount of time
and I also have my patreon page so check
these things out if you want to learn
more about goang so before we actually
continue with the folder structure here
you can already see it on the left side
of my screen there is no such thing as
the perfect project structure or the
good project structure or something that
is good something that is wrong I think
that doesn't exist in programming uh
space I think a lot of people are
forcing paradigms to you because they
created that Paradigm and they want to
Market it so we cannot blame them right
if you make something you're probably
going to Advocate that it's going to be
the best thing because it sols a lot of
problems and maybe it will right but
whatever you do in life there's always
some it's always going to give you
benefits but it always also going to
give you some uh negative Parts also
that's something you need to keep in
mind right um so if you have the if you
have the possibility to start from
scratch you actually never know what
what's going to be the best thing you
can have something in your mind like
okay if we do it like this and and and
these are the Futures we need this is
probably something we're going to
utilize in the future maybe we should do
it this way most of the time it's going
to change dramatically over time right
it's going it's never going to stick
like you like you originally planned it
right so that's something you need to
take in consideration and that's why I
think everybody needs to have a more
result driven development mindset which
is basically instead of being confused
about oh the tools and packages and ways
to organize stuff I think you just need
to go and Chase your good feeling what
feels right what feels sus what feels
good you know what I mean and change it
along the way I think and just focus on
the results instead of on um the way you
program stuff right all right cool uh so
basically how do we do these things well
it's very simple uh we have um goang
project here
and first of all we have this bin folder
here and this bin folder is basically
just uh a folder where we are dumping in
our binaries right uh and these binaries
they are coming from the CMD folder
right you can see there is no main. go
in the rout right that's because we
using this pattern it's it's a very
common thing where you have these um
separate packages right and each of
these packages they have a main. go file
here you can see that right and in this
main.go we basically bootstrap um or API
which is an
echo project right uh it's using the
echo framework so what framework do you
need to use actually guys it does not
really matter right you can use gen
fiber Echo gorilla you can use the
standard Library it doesn't really
matter right people say yeah you don't
need to use that or you need to use this
it doesn't matter it really doesn't
matter they all do the same things and
they will all get you the same result
right that's that's that's the reality
right uh and you could say performance
performance doesn't matter right again
people are going to be very very
confused and angry about the comment I
say like performance doesn't matter like
if per maybe performance will matter at
a certain point of time and if
performance is a
bottleneck then I'm going to say
congratulations to you because you
actually makes made something that has
some traction that that's a big traction
because before you go to be in a
performance bottleneck you're going to
have a very very very very large user
base and that's a luxury problem and
that's going to be solved at the time
it's going to be there right uh which in
your case most of the time will never be
right so performance is not something uh
especially in goling with echoin you
will you cannot go wrong it it it has a
good performance don't worry about that
guys don't worry about about about these
paradigms uh about that fugazi right so
we have this main thing which basically
boots up um our API server and all that
stuff right then we're going to have
some other things so what I like to do
um which is a little bit weird a lot of
people use some kind of um CLI package I
think I don't know maybe Viper or
something I have no clue I never use
these things and with the CLI package
you can write all these commands and
then you can execute them based on the
user input but what I like to do is very
lazy and very simple is just make um for
example this drop thingy here right
which basically drops all our accounts
and why do I have this because um in our
migration sometimes you can do make down
for example which will um migrate will
will
will use all the down migration you know
what I mean delete these tables but
sometimes you're going to end up in a
dirty State and then it it basically is
going to refuse to uh drop your tables
right make down is not going to work
it's going to say yo your database is in
a dirty state of course in production
that's a good thing that that happens
right uh but in development it's
sometimes it's nasty because you have in
a dirty State and you need to force do
that stuff uh you need to force drop and
sometimes you need to manually delete
some tables and it's just a pain and
yeah so I made this very dirty simple
script it basically just deletes the
whole shebang even the
migrations uh and then we can just do a
makeup again
um and it's all good right that's this
drop and the same thing with migrate you
can see that basic um with this other
thing here for for example the seat it
basically go it's a dirty script that
seeds some stuff into the database um
just for yeah testing purposes right if
you want to test your front end uh you
just dump some some some seats uh into
into your database um so you can see
this all very simple scripts and the way
we actually trigger them is in our make
file you can see uh for example make
down it just runs these main. go files
right this the same thing here with up
with uh drop and all that stuff it it
uses these these uh separate programs
right very simple um you could use
something like a CLI package like I said
what's good what's bad it doesn't really
matter this was just easy peasy without
a lot of squeezy you know what I mean uh
so that's that so the next step here uh
like you let let me show up this make
file right so I'm using a make file
people say you should drop that in in
favor for other
stuff like I said man if make file works
for me it works for me right maybe it
doesn't work for you maybe you want to
use something else that's perfectly fine
right um it all depends what you want to
use it doesn't really matter right uh
it's
fine all right because you have these
goal tools I don't know go make or
something you have these these cool
stuff U where you can write goang things
and but like I said a make file
everybody knows it it's simple it's most
of the time already installed on Linux
things so it's fine right uh so the next
step is a very important step is our DB
I'm going to uh start with this DB
because this is a little bit of
controversial right so the DB package
has just one file and what this does is
basically creates
um it it basically uh bootstraps or
database right and as you can see we are
using bun here but we and it's a global
variable so there's a lot of things we
need to discuss here right um so I
started out we started out with using
sqlc right sqlc is something that
generates um goang codes and structures
models based on U your schema your SQL
schema uh and you can write your SQL
queries and it basically generates code
for that which is actually very very
good right
but need to drink some coffee for this
one but the problem with sqlc is it's
it's getting very tedious something we
had a ton of queries already and I
thought man this is just so nasty all
these queries and then we have a schema
for our migrations we have a schema for
sqlc and you need to sync these things
and then some people are working uh like
me and Teddy we working on that it's a
two people project right now and we are
doing work for for 16 uh but the thing
is that it's just it gives conflict it
was was
nasty for current operations I'll really
like it but we need to have some
more dynamically stuff going on some
more genetic stuff and I used BN for a
long time um in in other projects and I
really like it right of course something
that's the downside like I said for
every positive thing also comes
something that's some negative stuff and
the cons for bun is that your queries
are not going to be type safe right if
you make a mistake
that's bad for you nobody's going to
it's going to it's going to give you an
error or something or or user that not
being found uh but hey it is what it is
right like I said pros and cons what do
you want it's it's a a weight scale that
needs to be in Balan for you right not
for somebody else on the internet it
needs to be in balanced for you and for
your team and is everybody thinks that
the scale is in balanc and it's fine
right everybody know the consequences
here um but also everybody can use the
positive side Ides on that right so
that's what we do and it's a global
variable why is it a global variable
just for the ye of use and of course
this is very this could be dangerous but
like I said a global at the right time
at the right place is is is okay it's
it's it's no big of a deal because we
are using this data
folder and this data folder is most of
the time is going to be used models for
a lot of people but I think models is so
weird because I think models they belong
on the catwalk you know what I mean um
it's I think everything that involves
storing data retrieving data structures
which is just
data I think it it can be in a data
folder it makes a lot of sense right so
everything uh that's
involving data
structures queries for inserting or
retrieving or updating whatever all
database related logic is housing into
this data folder right just like models
most of the time
and what this does is basically it uses
it uses queries um it the only thing it
does is basically interacting with your
database right that's what it's doing
you got have a query you're going to do
some joints or some some separate
queries you're going to aggregate that
data and return it that's basically what
this data folder is this data package is
going to do right and nothing else and
nothing nothing less and nothing
more for tests it's very important
because the global variables uh can be a
problem in your tests but I don't like
to test the data or your database into
the unit form right I don't want to use
unit tests for data I think interacting
with your database should be on the
integration Port again this is something
that can be discussed it has its pros
and it has its cons it really doesn't
matter right um I think if you test your
business logic based on on unit tests
that's pretty good and if you use your
data everything that needs to be
interacting with your database if you do
that on an integration
level it doesn't really matter that your
database is
global the only thing you need to do is
instantiate the correct database right
if you have if you want to do some test
you use a test database um and if you
yeah whatever right and if you deploy
the production you're basically going to
init initialize your database with the
production credentials right so that's
the data thing right the next step we
have is very simple is our handlers and
our handlers are just plain functions
right there are no uh you can see for
example here uh some middleware where we
validate gbt we have um some error stuff
for centralized error
handling some lock middleware it's very
simple uses the S from gooling the new
stuff pretty amazing let me show you
something here can I show some something
diff uh something interesting uh for
example handle get User it's just very
simple handle get user we always prefix
this with handle and then we use get and
that's not for example not for getting
the user it's the get request right
handle get user handle post user handle
update user right um just a convenience
method is that the way to go I have no
clue it works for me
uh you're you're like I said you're the
ball pros of your painting you can do
whatever you want right of course if you
join a team um that already has these
kind of things set out laid out yeah you
need to follow them right but if you can
choose from the start you can do
whatever you want right as long as it
makes sense for you um so what do we do
with these hand handlers well uh like I
said before we have the CMD API Maino
file and then we have these very simple
um uh yeah like like these these these
route handlers right this these routing
stuff so for example uh what we're going
to do here because in leue we have the
concept of sside accounts and buy side
accounts because there's a double-sided
marketplace where you're going to sell
or you're going you're going going to
auction your uh recurring Revenue in
return for upfront cash from
institutional investors right so we have
sell site and buy side and of course the
sell side they cannot they can only do
specific certain actions and the same
for the buy side so what we do here is
we make this a group thingy which
basically is most of the time in every
single web framework you're going to
have something like that where you're
going to group your handlers with some
um how do you call it with some
middleware right and uh it basically
some some authentication authorization
middleware right he's going to see oh
yeah this user is this a cide user yes
then he can do these things if it's a
buy side user then you cannot do these
things right then uh you are basically
not authorized to access these things
right so that's what we do here handlers
and then we're going to say handle
cellsite demo metrics in this case we
don't follow this stuff it's probably my
bad handlers handle post funding request
handlers handle get funding request you
know what I mean very simple um no big
of a deal here right so that's basically
our handlers here then we have package
um which is
something where more of the business
logic houses because most of the time
you're going to have something like a
transport right you're going to have
your Json transport your Json HTTP
Handler so what it's going to do is it's
going to get some Json data it's going
to uh Marshall that or decode that into
a structure right and then you're going
to have that structure and then you're
going to decide do we need to do some
business business logic onto that data
or can we basically just use our data uh
package and just store that user into a
database right or store a funding
request into a database some some simple
crit doesn't really need to have some um
business logic because it's very simple
it's just receiving data decoding data
validating data and storing data that
doesn't actually is some kind of a
business logic in my opinion but if
there is some business Logic for example
you need to fetch some transaction from
the the bank you need to aggregate them
do some calculations and all that stuff
uh maybe some third party things are
going on some third party API calls or
need to be um executed then we put
everything inside of a
package folder here and in this case
it's going to be salt and then client y
yada y you can see the project is very
very early in the stage we already have
some project running but that's the next
GS this is still in development right so
that's going into package uh that's
basically a goaling em to to put it in
in package you could use it lip you can
you can call it whatever you want but I
think package is something that is very
common inside of the goang community
right then we have some types here uh
and types is very important
because you could put these types into
Data but these types are basically
shared you can use you can make you can
you can call this common is is is I can
I see that a lot of times in other
projects in other repositories on getup
people call it common people call it
shared in this case just types because U
the only thing we have is some kind of
enums quote quote I need to be very
careful for the YouTube police right
because they are actually no enims but
they act like enims right um constants I
would say all the constants are here all
the types are here that are shared
between uh packages right that's in
types um and you could basically split
that out right you could make it a a
file for each type but then you're going
to have a file with three lines of
sometimes that's who cares right you
just dump everything into the types at
go and if your types are getting too big
then you can start splitting things out
and group Things based on on domain or
something right but that's for later on
that's a problem we don't have yet right
and then of course the UIL right util
utils shared whatever you're going to
call this is a very simple stuff where
we going to have uh like nothing inside
right is Pro app M and all that stuff
some simple helper functions that are
going to help you with uh simple simple
things in your application right crypto
is
this not quite sure what this is
probably T did some stuff right anyway I
think it's for encrypting um some kind
of API tokens we don't want to uh every
API token from from a third party
library for our user we don't want to
expose that uh with plain text we want
to basically encrypt that stuff into the
database so if it's get hacked they are
the hacker right not the client
uh that's basically it of course get
ignore go mod some make read me that's
all that stuff that's basically it guys
and with this project structure you can
scale it as as far as you want right you
because every data structure you're
going to have or every data logic you're
going to have is going to be in the data
every new Handler is going to be in the
handlers every new business logic you
can put that into package and every new
type you put it into type and you can
scale it as big as you want is this
going to change over time most likely it
will right it's like I said it's
something that works for me for a very
long time this this structure it's very
simple um don't make it too complex with
all that clean architecture and
hexagonal architecture and they will all
have his benefits don't get me wrong
like these are basically invented by
very smart people but I always say start
very simple right start simple make some
uh diesel driven stuff and I'm going to
show you how it looks right so you can
see this is basically the sells side
dashboard right um which again another
question is like people say how do you
make this beautiful wise well guys it's
very simple uh don't be a single-sided
developer very important people want to
be a backend developer people want to be
a front-end developer I don't think
there is something such as a backend and
a front end developer in my opinion you
are a programmer or you're not you are
an engineer or you're not you can solve
problems or you're not because by the
end of the day the only thing you do is
write code right you're pressing buttons
If it's JavaScript typescript goang
python rust it doesn't really matter you
can write it in every every single
language there are some benefits and
there are some cons there are some pros
there are some Cons with everything in
life and the only thing that matters is
if your weight scale of pros and cons is
inbalance or may basically if your Pros
are a little bit
um uh if if your Pros weigh a little bit
more actually weit a little bit less
right if the balance is a little bit
more in the favor of the pros instead of
the Conant it's a win for you right but
that does not mean that my balance is
the same my weight balance is going to
be balanced the same as yours or my
neighbors or whatever right or other
produ um engineering teams it's all
subject to the people are using it right
that's something you need to uh take in
consideration um that's basically the
thing of course we are we are doing some
stuff it's it's it's still in progress
here we we we have some look at this
it's it's we have some stuff but we're
still uh working on that right that's
basically it and it's calling basically
spell kit in the back end and all that
stuff maybe I'm going to make it another
video because we're already 20 minutes
in and I'm not even done it's crazy guys
I really want to provide 10minute videos
but I can because the things I need to
say uh take some time right you know
what I mean it takes some time so that's
basically on B it if you have more
questions feel free to dump it in the
comments like I said uh is not for
everyone right people going to going to
dislike it people going to like it it is
what it is it's the balance again right
it's again the weight balance as long as
the people like it or in favor of the
people that dislike it it's all good for
me Isn't it so thank everyone for
watching this video and I'm looking
forward to see you in the next video or
live stream peace stay safe and have fun
Voir Plus de Vidéos Connexes
Belajar Laravel 11 | 3. Struktur Folder
Node.js + Express - Tutorial - What is Express? And why should we use it?
Nestjs backend rest api - Ecommerce project. Introduction & setup.
03. Folder Structure - Laravel 11 tutorial for beginners
React tutorial for beginners ⚛️
Create ANYTHING With Replit AI Agents
5.0 / 5 (0 votes)