5 advanced coding projects to expose you to complex concepts

Summary

The video script discusses the most effective way to learn new technical skills through hands-on projects. It emphasizes that while tutorials and documentation can provide foundational knowledge, it is through actual project implementation that one truly internalizes and masters new skills. The speaker shares five advanced project ideas suitable for experienced coders looking to expand their expertise. These include an image manipulation project involving flood fill algorithms, a Docker project to containerize a full-stack application, building an OAuth authorization server with specific focus on PKCE and OIDC compliance, implementing a Sudoku solver with potential concurrency for efficiency, and finally, a data engineering project that involves creating an ETL pipeline to extract, transform, and visualize Twitter data. The projects are designed to challenge and enhance the viewer's understanding of complex topics beyond basic coding skills.

Takeaways

• 📈 **Project-Based Learning**: The best way to learn a new technical skill is by doing a project, which allows you to internalize new skills through practical application.
• 🎨 **Image Manipulation**: For advanced learners, a project could involve image manipulation, such as creating a program to perform a flood fill on an image, replacing the most prevalent color with another.
• 🐳 **Docker and Containerization**: Building a full-stack web application using Docker can expose you to complex topics in containerization and deployment strategies.
• 🔐 **OAuth and Security**: Constructing an authorization server using OAuth, including implementing the Pixie (PKCE) authorization flow and ensuring OIDC compliance, is a complex project that dives into security protocols.
• 🧩 **Sudoku Solver**: Implementing a Sudoku solver can challenge your algorithmic skills and involve solving for multiple solutions or detecting unsolvable boards.
• 🌐 **Data Engineering with APIs**: Creating a pipeline to extract, transform, and load data from an API, like Twitter, and then visualizing it, is a project that covers a range of data engineering skills.
• 📚 **Documentation and Research**: For advanced topics, it's important to read the relevant documentation and research the technologies involved before starting a project.
• 🛠️ **Algorithmic Complexity**: When undertaking projects, pay attention to the algorithmic complexity to ensure efficient solutions.
• 📁 **File Manipulation**: Understanding how to manipulate files, such as working with JPEG images, is a key aspect of certain projects.
• 🤓 **Concurrency**: Enhancing projects with concurrency can improve performance, such as processing multiple images or handling various Twitter feeds simultaneously.
• 📈 **Bonus Challenges**: Pushing projects further by adding bonus challenges, like implementing JSON Web Tokens (JWT) for user authentication, can deepen your understanding of complex topics.

Q & A

What is the best way to learn a new technical skill according to the speaker?

The best way to learn a new technical skill, as per the speaker, is by doing a project. This involves building something with the new technology to internalize the skills being learned.

What is the first project idea mentioned in the script for learning advanced topics?

The first project idea is to create a program that performs image manipulation, specifically a flood fill. This involves selecting one color in an image and replacing it with a different color throughout the image.

What is the significance of using an algorithm with low algorithmic complexity in the image manipulation project?

Using an algorithm with low algorithmic complexity is important because it helps to perform the image manipulation task more effectively and efficiently, especially when dealing with large images or a batch of images.

Which image format is suggested to start with for the image manipulation project?

The speaker suggests starting with the JPEG image format for the image manipulation project.

What is the second project idea presented in the script?

The second project idea is about Docker and involves spinning up the entire stack of an application, specifically a web application, which includes a database, cache layer, web server, back-end API, and a JavaScript front end, all deployed together using Docker.

Why is Docker considered a complex and beneficial project to learn?

Docker is considered a complex and beneficial project because it is widely used across engineering teams, and learning about containerization and deployment can provide valuable insights into modern application development and deployment practices.

What is the third project idea that the speaker suggests for advanced learners?

The third project idea is to build an OAuth authorization server that adheres to the OAuth specification, specifically implementing various flows and learning about Pixie (Proof Key for Code Exchange) and OIDC (OpenID Connect) compliance.

What is the importance of implementing JWT in the OAuth 2.0 project?

JWT (JSON Web Tokens) is important in the OAuth 2.0 project because it is used to securely transmit information between parties as part of an authorization process. It allows for the passing of user information back to the final application in a secure manner.

What is the fourth project idea mentioned in the script?

The fourth project idea is to implement a Sudoku solver, which is a program that can fill out and solve a 9x9 Sudoku grid given a partially completed puzzle.

What are some additional considerations for the Sudoku solver project?

Additional considerations for the Sudoku solver project include using concurrency to solve the puzzle faster, detecting multiple solutions, and determining if there is no valid solution for a given Sudoku board.

What is the fifth and final project idea presented in the video script?

The fifth project idea is to pull data from the internet, specifically tweets from Twitter, and orchestrate this process as an Airflow task. This involves using the Twitter API, scheduling the data pull, and performing an extract, transform, load (ETL) pipeline to clean and visualize the data.

Why is Apache Airflow mentioned in the context of the fifth project idea?

Apache Airflow is mentioned because it is a platform used to programmatically author, schedule, and monitor workflows. It is suitable for the fifth project idea as it allows for the orchestration and scheduling of tasks related to pulling and processing tweets from Twitter.

Outlines

00:00

🚀 Advanced Coding Projects for Skill Enhancement

The paragraph introduces the concept that engaging in projects is the most effective way to learn new technical skills. It emphasizes the importance of hands-on experience rather than just tutorials or documentation. The speaker proposes five advanced projects suitable for those who already have coding experience and are looking to delve into more complex topics. The projects are aimed at internalizing new skills and understanding advanced concepts through practical application.

05:01

🖼️ Image Manipulation and File Handling

The first project idea involves using a coding language to manipulate images by performing a flood fill operation. This involves selecting the most prevalent color in an image and replacing it with another color. The project also touches on file manipulation and the intricacies of image formats like JPEG. An advanced challenge is to modify the program to work concurrently on a batch of images, performing multiple flood fills for different colors.

10:04

🐳 Dockerizing a Full Stack Application

The second project focuses on using Docker to containerize a full stack web application, which includes a database, caching layer, web server, back-end API, and a JavaScript front end. The goal is to deploy all components together using Docker and to understand the concepts and terms associated with Dockerization. This project aims to provide a deep understanding of containerization and deployment, which are widely used in engineering teams.

🔐 Building an OAuth 2.0 Authorization Server

The third project is about creating an OAuth 2.0 authorization server from scratch or using a library to facilitate user authorization in an application. The speaker highlights the importance of implementing various OAuth flows, specifically Pixie (Proof Key for Code Exchange - PKCE), and ensuring OIDC (OpenID Connect) compliance for modern web applications. The project also introduces JSON Web Tokens (JWT) for securely transmitting user information, offering a comprehensive learning experience in security protocols.

🧩 Implementing a Sudoku Solver with Algorithmic Complexity

Project four challenges the developer to implement a Sudoku solver that can fill out and solve a 9x9 grid given a partial solution. The project aims to test the developer's ability to handle algorithmic complexity and edge cases, such as detecting multiple solutions or identifying an unsolvable board. Bonus points are given for using concurrency to solve the puzzle faster, making this a fun and intellectually stimulating project.

📊 Extracting and Visualizing Data from Twitter

The final project involves pulling data from the internet, specifically tweets from Twitter, and processing them through an extract, transform, load (ETL) pipeline, possibly orchestrated with Apache Airflow. The developer is tasked with authorizing access to the Twitter API, pulling tweets based on schedules or commands, and storing the raw data in a chosen data store. The project extends to cleaning and transforming the data into a digestible format for analysis and visualization, offering insights into data engineering and API interaction.

Keywords

💡Projects

Projects are the central theme of the video, emphasizing that they are the best way to learn a new technical skill. The video suggests that engaging in hands-on projects allows for the internalization of new skills, as opposed to merely reading documentation or watching tutorials. Projects provide practical application and a deeper understanding of the technology being learned.

💡Image Manipulation

Image manipulation refers to the process of altering images using coding languages. In the context of the video, the first project idea involves opening an image and performing a flood fill, which is a technique to replace a selected color with another throughout the image. This concept is used to illustrate how coding can interact with files and is a practical application of learning a new coding language.

💡Docker

Docker is a platform for developing, shipping, and running applications in containers. In the video, it is mentioned as a key technology for a project that involves spinning up an entire stack of a web application. Docker is significant for its role in modern software development, enabling developers to containerize applications for easier deployment and scalability.

💡OAuth

OAuth is an open standard for authorization that is used to grant applications limited access to user accounts on an HTTP service. The video suggests building an authorization server using OAuth, specifically implementing the Pixie (PKCE) authorization flow and ensuring OIDC compliance. This keyword is important as it relates to the security and authentication aspects of modern web applications.

💡Sudoku Solver

A Sudoku solver is a program that can solve a Sudoku puzzle, which is a 9x9 grid with a set of numbers already filled in. The video challenges viewers to implement a Sudoku solver that can fill out the grid completely, with additional complexity in handling concurrency and determining if there are multiple or no valid solutions. This project is a fun and engaging way to explore algorithmic complexity.

💡Data Pipeline

A data pipeline is a series of processes through which data is acquired, transformed, and then made available for consumption. In the context of the video, the final project involves creating an ETL (Extract, Transform, Load) pipeline to pull data from Twitter using the Twitter API, clean and transform it, and then visualize it. This keyword is central to the theme of data engineering and the practical application of data processing techniques.

💡Algorithmic Complexity

Algorithmic complexity refers to the efficiency of an algorithm, often expressed in terms of time and space complexity. The video touches on this concept in the context of image manipulation and Sudoku solving, where efficient algorithms are crucial for processing data quickly and effectively. Understanding algorithmic complexity is key to optimizing the performance of software projects.

💡File Manipulation

File manipulation involves the ability to create, modify, and handle files using a programming language. The video discusses this in the context of image manipulation, where understanding how to manipulate files is necessary to replace colors in an image format like JPEG. This concept is fundamental to learning how to work with various types of data in coding projects.

💡Concurrent Processing

Concurrent processing refers to the ability of a computer program to handle multiple tasks simultaneously. The video suggests using concurrent processing to improve the performance of the image flood fill and potentially the Sudoku solver. This concept is significant as it relates to the efficiency and speed of processing large data sets or complex tasks.

💡API

An API, or Application Programming Interface, is a set of protocols and tools for building software applications. The video mentions using APIs, specifically the Twitter API, to pull data from Twitter. Understanding how to work with APIs is essential for accessing and utilizing external services and data sources in software development.

💡JWT (JSON Web Tokens)

JWT is a compact, URL-safe means of representing claims to be transferred between two parties. In the video, it is mentioned as a technology to be used for passing user information back to the application in the context of OAuth 2. JWTs are important for secure communication and are widely used in modern web applications for stateless authentication.

Highlights

Projects are the best way to learn any new technical skill.

Beginners can find numerous project ideas, but it's challenging for senior developers to find complex projects.

The first project involves image manipulation through a flood fill algorithm.

Focus on algorithmic complexity and file manipulation, especially with JPEG format.

Bonus challenge: Modify the flood fill program to work concurrently on a batch of images.

The second project is about using Docker to containerize and deploy a full stack web application.

Learn about Docker, containerization, and deployment through building a web application stack.

Project three involves building an OAuth authorization server with different flows, including PKCE.

Ensure the OAuth 2 implementation is OIDC compliant and use JWT for passing user information.

The fourth project is implementing a Sudoku solver with potential use of concurrency for efficiency.

The Sudoku solver should be able to detect multiple solutions or the lack of a valid solution.

The fifth and final project is an ETL pipeline to pull data from the internet, such as tweets from Twitter.

Use Apache Airflow to orchestrate the task and consider using the Twitter API for data extraction.

Store the raw tweet data in a chosen data store and perform data cleaning for better analysis.

The projects are designed to push understanding beyond beginner levels and expose learners to complex topics.

Each project comes with its unique set of challenges and opportunities for learning advanced concepts.

The presenter encourages learners to enjoy the process and look forward to the next video.

Transcripts

hey everybody welcome back to the

channel today I want to talk a little

bit about the best way that I know to

learn a new technical skill

projects are the best way to learn any

new technical skill you can learn a

little bit about from tutorials or from

reading the docs or something like that

but really the way you're going to

internalize the new skills that you're

learning is by doing a project actually

building something with the new

technology that you're trying to learn

fortunately for beginners there's a ton

of project ideas and prompts out there

for new coders people who are new to the

industry but how do you learn those much

more advanced topics

it's more difficult to find

complex projects things that would

really apply to a senior developer

somebody who already knows how to code

but is trying to learn a new interesting

technology so today I want to share with

you five Advanced topics Advanced

projects that can expose you to some

much more complex topics these are not

beginner projects these are projects for

people who already know how to code and

want to learn more so let's get into it

um the first project that I want to

bring up is a program that does a little

bit of image manipulation so you might

be surprised but basically any

um

coding language that you choose has some

type of way that they can interact with

files and that means that you can edit

images in your coding language of choice

so this first project idea is to open an

image and perform what's called a flood

fill basically select one color in the

image and replace it for a different

color in that same image so find the

most prevalent color that's in that

image what what color appears the most

and then replace that color everywhere

in the image with another color now

there are algorithms that will help you

do this more effectively than others so

it's worth paying attention to the

algorithmic complexity when you're doing

this project that's one piece of this

project the other piece is file

manipulation it'll get you a little bit

more into the complex topics of file

manipulation how jpeg actually works to

replace colors

um I would say jpeg is probably the best

image format to start with here for this

thing and then bonus points like if you

want to take this project even further

make that same program that performs the

flood fill work concurrently on a whole

batch of images so if you have many

images perform multiple flood fills for

a different color

so there's your first project

number two

is about

um

Docker and spinning up the entire stack

of an application specifically what I

have in mind is a web application but

you can do it for any type of

application as long as you're learning

about containerization Docker and how

you might deploy those as well so for my

project that first comes to mind is a

web application it should have a

database

cash layer something like redis an nginx

or Apache web server that's actually

serving the things

a a back-end API web server and a

JavaScript front end that should all get

deployed together in a Docker file and a

Docker compose dot yaml so if you've

never played around with Docker some of

those terms may be new and you may not

heard of any of them but go read the

docker docs and then try to build

something a full stack application

within Docker containerize an entire

application so there's a complex project

and Docker is so huge it's so widely

used across all engineering teams right

now that you'll learn a lot from that

project so that's your second project

project number three number three is use

a library or or you can build it from

scratch but a library is going to make

this a lot easier to build an oauth to

to authorization Services an

authorization server a way that somebody

can authorize to your application using

the oauth specification

specifically you should Implement all

the various flows and start learning

about how oauth Works to exchange

authentication and you know different

tokens

but specifically I think the most

important one to learn about for modern

web applications is Pixie authorization

which stands for proof key for code

exchange so pkce and people end up

pronouncing that as pixie but pkce

authorization should be like one of your

goals for this application this

authorization server you're building and

make sure that it is oidc compliance so

there's another term that you may not

have heard yet but go read the oauth 2

docs specifically read about Pixi pkce

and and oidc to learn a lot more about

the security that happens when you

authorize with a server with a with a

resource and and

for kind of bonus points or or actually

just

a full implementation of oauth 2 with

the Pixy flow oidc compliant use Json

web tokens to pass the ID token back the

the user information back to the final

application so JWT usually pronounced

jot is another important technology to

know about and that's another Advanced

topic that you're going to get exposed

to by doing this project so there's a

ton there Pixi oidc JWT

and oauth 2 generally you're going to

learn a ton from doing that project okay

so that's project number three project

number four here's another idea is to

this one's kind of a fun one Implement a

Sudoku solver so if I give you a Sudoku

puzzle which is a 9x9 grid with a couple

numbers in it you'll be able to fill out

and solve the Sudoku puzzle if you don't

know what a Sudoku is you'll have to go

do a little bit of Googling to figure

that out but a Sudoku solver should be

able to fill the Grid in entirely

um bonus points if you can use some type

of concurrency or somehow break the

problem down a little bit

to be able to solve it faster so there's

some algorithmic complexity here there

also is

um

like a complexity around whether or not

the board can actually be solved so

there are cases in Sudoku where you

might have a board with a few numbers

and there is more than one solution

um so your application should be able to

detect that should be able to tell what

the multiple Solutions are your

application should also be able to tell

if there is no valid solution for a

given Sudoku board

so go check all of those things out see

if you can create a Sudoku solver

that solves the problem as quickly as

possible to complete the Matrix but then

also check does the board have more than

one solution and does the board have no

valid solution so check those edge cases

as well so that's project number four

Sudoku solver I mean you could do that

on the command line or anywhere you want

project number five this is the final

project I'll cover in this video

is

um pulling

some type of data from some Source on

the internet my idea is to pull tweets

from Twitter

so let's think about this though in

terms of an extract transform load data

pipeline so let's write it and

orchestrate it as an airflow task if you

don't know about Apache airflow you're

gonna have to go Google that and figure

out the the details there read the docs

for airflow how can you authorize to the

Twitter API and pull tweets

either using a schedule like once an

hour or once a day or maybe you just run

it as a command on the command line and

then try to pull those tweets

concurrently for various Twitter handles

like maybe you're gonna track certain or

handles or hashtags or whatever you'd

like so you could follow certain people

or you could follow certain topics you

could pull all the tweets that have

happened in the past hour that have come

from a given

handle or a given hashtag and then save

those raw tweet data to some type of

data store so you'll have to pick which

type of data store is going to work best

for your application

and then bonus points if you can somehow

map reduce that data into a more

digestible format

clean up the data when you store it in

your data store so do an extract

transform load a whole Pipeline and then

visualize it with some sort of analysis

at the end of the at the end of the

pipeline so this is a data engineering

task but you're going to learn a lot

about not just the Twitter API but also

about

how to extract data and then visual

clean it up and visualize it

so those are my five project ideas and

they should give you plenty of complex

topics that go beyond just beginner

project ideas they're going to really

push your understanding so hope you

enjoy see you in the next one