15 futuristic databases you’ve never heard of
Summary
TLDRThe video script explores the evolution of databases, highlighting the shift from traditional SQL systems like MySQL and Postgres to modern, innovative databases. It reviews 15 emerging databases, including PlanetScale, CockroachDB, Neon, and others, focusing on their unique features such as serverless scalability, graph-based data models, and AI integration. The video emphasizes how these new technologies aim to solve the limitations of older systems, while also addressing challenges like cloud scaling, branching, and real-time analytics. It's a deep dive into the future of data management for developers.
Takeaways
- 🚀 The video discusses the need for speed and innovation in the tech industry, often symbolized by the phrase 'move fast and break things'.
- 🗄️ Relational Database Management Systems (RDBMS) like SQL have been foundational in computing, with Oracle and MySQL leading in their respective sectors.
- 🌐 The script introduces several new databases designed to handle modern challenges, including scalability and cloud-native operations.
- 🌌 PlanetScale is highlighted as a serverless platform for MySQL, leveraging Vitess for horizontal scaling, and offering a managed service with a user-friendly dashboard.
- 🧘♂️ Yugabyte offers a serverless database for PostgreSQL, aiming for infinite scalability and supporting multi-cloud strategies to avoid vendor lock-in.
- 🌟 Neon, currently in technical preview, is a serverless PostgreSQL option that can scale to zero and is written in Rust, offering a polished admin interface.
- 🔄 Dolt, inspired by Git, allows for database branching and merging, making it easier to experiment with data changes without disrupting the main database.
- 🦀 CockroachDB is a 'new SQL' database compatible with PostgreSQL, designed for horizontal scaling in the cloud, with a free tier and pay-as-you-go pricing.
- 💡 Cloudflare's D1 is a SQL Lite database for edge computing, allowing developers to write stored procedures in JavaScript for transaction management.
- 📊 Zeta is a serverless relational database with a spreadsheet-like interface, offering full-text search and branching capabilities, similar to Dolt.
- 📈 Eightbase is a relational database with a GraphQL API and a low-code tool for building front-end applications, aiming to simplify full-stack development.
Q & A
What is the concept behind 'move fast and break things' in the tech industry?
-'Move fast and break things' is a philosophy encouraging rapid development and innovation, even if it involves taking risks or making mistakes along the way. It emphasizes speed over perfection, especially in early stages of development.
What is the relational database management system (RDBMS), and why is it important?
-The relational database management system (RDBMS) organizes data in a structured way using rows and columns, forming relationships between different datasets. It's one of the most successful ideas in computing, still forming the backbone of many modern data systems like Oracle and MySQL.
Why is SQL considered outdated by some despite its widespread use?
-SQL, developed in the 1970s, is considered outdated by some due to newer, more scalable database solutions and modern demands for handling big data. However, it remains critical as it powers many of today’s enterprise and web applications.
What problem did YouTube face with MySQL, and how was it solved?
-YouTube faced scaling challenges with MySQL due to its massive traffic. Instead of switching to a NoSQL database, YouTube developed a library called Vitas, which allowed MySQL to scale horizontally, addressing the issue without changing the database system.
What are serverless databases, and how do platforms like PlanetScale and Neon use them?
-Serverless databases are databases that automatically scale and manage infrastructure without user intervention. PlanetScale offers a MySQL-compatible serverless platform, and Neon provides a serverless PostgreSQL solution, with features like scaling to zero and advanced management dashboards.
What is 'branching' in databases, and why is it useful?
-Branching in databases allows users to create experimental branches from the main database, similar to version control systems like Git. This enables developers to test changes without affecting the main database and merge successful changes later.
What is 'NewSQL' and how does CockroachDB fit into this category?
-NewSQL refers to a class of databases that offer the scalability of NoSQL while maintaining the ACID properties and relational features of traditional SQL. CockroachDB is an example of NewSQL, built to scale horizontally while remaining compatible with PostgreSQL.
How does Cloudflare D1 differ from traditional SQL databases?
-Cloudflare D1 is a lightweight, serverless SQL database running on the edge, designed for faster, localized queries. It offers stored procedures in JavaScript and can scale for web applications, though it lacks native transactions typical in SQL databases.
What are some advantages of using Zeta and 8base for database management?
-Zeta provides a serverless relational database with a spreadsheet-like interface and full-text search capabilities. 8base, on the other hand, offers a relational database with a built-in GraphQL API and low-code tools for quickly building front-end applications.
How does SurrealDB combine features from different types of databases?
-SurrealDB is a multi-model database that combines relational, document, and graph database features, offering flexibility in data modeling. It allows for SQL-like queries while supporting more complex, graph-based relationships, simplifying data interactions.
Outlines
🚀 Tech Success and the Evolution of Databases
This paragraph begins by humorously recounting a mishap in a tech job where moving fast led to a significant mistake—deleting a production database. The narrative transitions to discussing relational databases, specifically SQL, and their continued dominance in the tech world. SQL, developed in the 1970s, powers many modern platforms like YouTube and Facebook. The rise of new, cutting-edge databases that aim to 'move fast and break things' is highlighted, showcasing how innovation is reshaping traditional database management. The paragraph sets the stage for a deep dive into new databases attempting to improve or replace SQL.
🌐 PlanetScale and MySQL Scalability
This section introduces PlanetScale, a serverless platform based on MySQL and the open-source Vitas library. It explores how YouTube scaled MySQL using Vitas to handle millions of simultaneous connections, allowing for horizontal scaling. PlanetScale offers this technology as a fully managed service with an accessible dashboard and SDKs for integration. The paragraph also compares PlanetScale's pricing favorably to other cloud services like AWS and Google Cloud. It briefly mentions competing services like YogaByte and Neon, which aim to scale Postgres with features like multi-cloud hosting and serverless architecture.
🛠️ Dolt, CockroachDB, and Other New SQL Databases
Here, the focus shifts to Dolt, a database that combines MySQL with Git-like features, allowing users to branch, experiment, and merge changes easily. CockroachDB is also introduced, a 'New SQL' database built by ex-Google engineers for horizontal cloud scalability. Other alternatives such as FoundationDB, TitaniumDB, and Cloudflare's D1 are mentioned. D1, a lightweight SQL Lite database that runs on the edge, is particularly noted for allowing stored procedures to be written in JavaScript, simplifying transactions. The paragraph underscores the variety of new SQL databases designed to tackle modern scalability challenges.
📊 Zeta and Low-Code Database Solutions
The spotlight now turns to Zeta, another serverless relational database that integrates Postgres and Elasticsearch. Zeta stands out for treating data like a spreadsheet, offering developer-friendly tools with built-in full-text search capabilities. The paragraph also introduces 8base, a relational database that provides a GraphQL API and low-code tools for quickly building front-end apps. These tools position Zeta and 8base as developer-friendly alternatives to services like Airtable or Notion, emphasizing the growing trend of merging relational databases with ease-of-use and powerful search functionalities.
🌟 EdgeDB and the Post-SQL Era
This paragraph discusses EdgeDB, a graph-relational database powered by Postgres. Unlike traditional relational databases that rely on columns and rows, EdgeDB allows developers to define types (similar to programming languages) that naturally link to other types. This approach eliminates the need for complex joins and simplifies querying. It positions EdgeDB as more human-friendly and intuitive, describing it as an ORM-like solution for Postgres. The paragraph also briefly mentions SurrealDB, a graph-relational database with similar goals, highlighting its ease of use and scalability with Rust-powered performance.
🔄 Multi-Model Databases: SurrealDB and Fauna
This section expands on SurrealDB, which combines relational, document, and graph database elements to create an easy-to-use API. It offers horizontal scalability and ACID transaction support, positioning it as a flexible solution for complex data models. Fauna is another multi-model database created by ex-Twitter engineers, described as a document database with native joins. This feature makes it a strong choice for relational data management while maintaining the simplicity of document databases. The paragraph emphasizes Fauna's closed-source nature but highlights its ease of use and powerful capabilities.
⚡ Real-Time Databases: Memgraph, KeyDB, and Meilisearch
The focus here is on Memgraph, a real-time graph database similar to Neo4j but optimized for performance with C++ and memory-first architecture. Memgraph is ideal for real-time analytics platforms. KeyDB, a faster alternative to Redis, is also introduced, emphasizing its role as an in-memory cache for speeding up data retrieval in high-performance applications like Snap. Meilisearch, an open-source Rust-based full-text search engine, rounds out the section. It's noted for its ease of implementation, allowing developers to fine-tune search parameters and integrate search features quickly into web applications.
🤖 AI and Machine Learning Databases: MindsDB
The final paragraph introduces MindsDB, a database that integrates machine learning models directly into the database. Its 'AI Tables' allow users to train machine learning models using SQL-like commands and then make predictions from the data. This innovation simplifies the machine learning lifecycle by embedding model training and inference within the database itself. MindsDB is positioned as a forward-looking tool that could pave the way for more databases with built-in AI capabilities, offering a glimpse into the future of data management and machine learning.
Mindmap
Keywords
💡Relational Database Management System (RDBMS)
💡SQL
💡Serverless
💡NoSQL
💡Horizontal Scaling
💡PostgreSQL
💡Dolt
💡GraphQL
💡Graph Database
💡Multi-Model Database
💡AI Tables
Highlights
The necessity to move fast and innovate in tech, as illustrated by the speaker's anecdote about their experience with a production database.
Relational database management systems (RDBMS) are a cornerstone of modern computing, with SQL remaining foundational despite being from the 1970s.
Oracle's dominance in the enterprise sector and MySQL's widespread use, particularly by platforms like YouTube and Facebook.
Introduction to PlanetScale, a serverless platform for MySQL that scales horizontally using the Vitess library.
YugaByte's open-source solution for PostgreSQL, aiming for infinite scalability in the cloud while maintaining native features.
Neon's serverless approach to PostgreSQL, featuring the ability to scale down to zero and a polished admin dashboard.
Dolt's innovative database branching inspired by Git, allowing for experimentation without disrupting the main database.
CockroachDB, developed by ex-Google employees, is a newSQL database designed for cloud scalability and compatibility with PostgreSQL.
FoundationDB and TitaniumDB as alternatives in the newSQL class, offering different approaches to scalability and performance.
Cloudflare's D1, a SQL Lite database for edge computing, allowing for data creation and querying on the edge.
Zeta, a serverless relational database with a unique spreadsheet-like approach to data management.
Eightbase's combination of a relational database with a GraphQL API and a low-code tool for building front-end applications.
EdgeDB, a graph-relational database that simplifies data relationships and queries by eliminating the need for joins.
SurrealDB's unique approach to data modeling, combining the strengths of SQL and graph databases with Rust's performance.
FaunaDB's multi-model capabilities, offering the ease of use of a document database with the power of native joins.
Memgraph, a real-time graph database designed for high-performance data pipelines, as an alternative to Neo4j.
KeyDB, a high-performance in-memory database acquired by Snap, offering over 1 million operations per second.
MeiliSearch, an open-source alternative to Elasticsearch, providing customizable full-text search capabilities.
MindsDB's integration of machine learning frameworks, allowing for predictive models to be embedded directly into the database.
Transcripts
if you want to be successful in Tech you
need to move fast and break things
that's exactly what I said to my boss
before he fired me for deleting a
production database I should fire
the relational database management
system is one of the most successful
ideas in the history of computing SQL is
1970s Tech that still forms the backbone
of the modern Big Data World Oracle
dominates Enterprise while MySQL is the
primary database for platforms like
YouTube and Facebook but SQL is by no
means perfect in 2023 there's a whole
new wave of futuristic Cutting Edge
avant-garde game changing databases
looking to move fast and break things
some of them build on top of existing
relational patterns While others bring
entirely new ideas to the table that
you've never seen before no pun intended
over the next few minutes we'll look at
15 crazy ass new databases and talk
about the problems they're trying to
solve first on the list we have Planet
scale which is not a new database but
rather a serverless platform for MySQL
like I mentioned before YouTube uses
MySQL however back in 2010 when it was
getting popular it was extremely
difficult to scale at least for an app
with millions of simultaneous
connections instead of switching to a
nosql document database they developed a
library called Vitas which makes some
trade-offs to scale MySQL horizontally
and this is the technology that planet
scales based on it provides a fully
managed database that can scale with
vtas and makes it available from a nice
dashboard with sdks to go along with it
and it also integrates nicely with tools
like Prisma it has a free tier and good
pricing on the page here that they're
likely losing money on by comparison a
basic database on Heroku is 50 bucks and
it's unreasonably expensive on AWS and
Google Cloud Planet scale is not the
only company with this serverless
database idea though yoga byte has its
own open source solution for postgres
its goal is to scale infinitely in the
cloud but attempt to support all the
native features in postgres which is not
easy to do in a cloud native environment
you can actually use a multi-cloud
strategy to host your database on
multiple clouds at the same time
eliminating the dreaded vendor lock-in
issue but there's another option for
scaling postgres called neon it's
currently in technical preview but looks
extremely promising it's serverless and
can scale down to zero and has a highly
polished admin dashboard to go along
with it and most importantly it's
written in Rust which is the only thing
that matters if you want to look cool in
front of your friends it also supports
branching which I think was inspired by
another database called dolt this
database would be like give MySQL and
git had a baby most importantly it
allows you to create branches from the
main database where you can experiment
with new data without breaking things it
makes it easy to analyze the diff
between changes and when everything
looks good the changes can be merged
back into the main branch and it's done
with the same conventions found in git
that virtually every developer is
familiar with dolt has a hosted option
but it is quite expensive they've raised
21 million dollars and because I'm
extremely entitled I demand they burn
more of this money to give us a free
tier I just paid eight dollars for a
dozen eggs and can't afford to put
another fifty dollars on my mom's credit
card now another sequel flavor I want to
talk about is cockroachdb it's been
around for quite a while and was
developed by ex Google employees and is
often referred to as new SQL it's
compatible with postgres but was built
from the ground up with go to scale
horizontally in the cloud it has a nice
free tier pretty similar to Planet scale
then becomes pay as you go pricing after
that it's yet another great option but
you might also want to consider
Foundation DB or titanium DB as well
which are also in this new SQL class but
in some cases you might prefer a
lightweight alternative like cloud flare
D1 this is a brand new product that's
still in beta and basically it's a SQL
Lite database that runs on the edge
cloudflare already has its own key value
store for workers as well as R2 for
object storage but D1 allows you to
create data on the edge that can be
queried one trade-off though is that
native transactions will not work
however they're taking a very cool
approach to overcome that limitation by
allowing developers to write stored
procedures with JavaScript which are
just Snippets of code that run directly
in your database which can be used to
create transactions or other reusable
queries this approach has a ton of
potential because historically store
procedures are a pain to work with but
if it's in JavaScript even I could do it
pretty cool but it's not production
ready yet if you want something that's
ready to go today check out Zeta yet
another serverless relational database
that's based on postgres and
elasticsearch under the hood but what
makes it special is that it treats your
data like a spreadsheet it feels kind of
like a developer friendly alternative to
airtable or notion and what's really
awesome is that it has full text search
built in without needing to duplicate
your data in some other service like
algolia it has sdks for typescript in
Python and has a schema editor that
makes it very easy to create and
visualize relationships between tables
and it can also do branching like we
looked at in adult DB for pricing it has
a solid free tier then it scales up in a
linear way based on usage but another
tool that takes things one step further
is eight base it's also a relational
database that provides a graphql API for
your data out of the box making it
similar to tools like hasura what makes
it unique though is that it also comes
with a low code tool for quickly
building front-end applications that
connect to your back end it's like a
backend as a service that can also build
your front end but that's enough talk
about relational databases now we enter
the post SQL ERA with Edge DB it's a
graph relational database that's powered
by postgres under the hood when it comes
to relationships humans don't naturally
think of columns and rows but rather
objects connected to each other like a
graph instead of defining tables like
you would in SQL you define types like
you would in most programming languages
these types link to other types which is
awesome because it eliminates the need
for joins and simplifies the way you
make queries overall all it almost feels
like an orm for postgres that allows you
to think about relationships in a far
more human friendly way currently it's
open source and can be self-hosted but
there's a wait list for their cloud
service now another relational graph
like database is surreal DB it's written
in Rust and I have a full tutorial on my
second Channel Beyond fireship it's
designed to support acid transactions
while scaling horizontally but what I
really like about it is how easy and
flexible it makes data modeling for the
most part it feels like SQL but also
uses arrows to connect nodes and edges
like you would in Cipher with a graph
database like neo4j it takes the best
elements of relational document and
graph databases and combines them into
one easy to use API currently it can
only be self-hosted but they do have a
cloud service coming soon the
multi-model idea is nothing new though
another database pushing the limits is
fauna it was created by X Twitter
Engineers who set out to build the
perfect database it's extremely easy to
use like a document database similar to
mongodb but unlike a document database
it supports features like native joins
which is the number one missing feature
in document databases and that means
fauna is more well suited for complex
relational data like social graphs to
interact with it it has its own custom
query language called fql but also comes
with a graphql API it is a closed Source
product so you can't self-host it but
its fully managed service has a nice
free tier another new database you've
likely never heard of is memograph it's
a drop in replacement for neo4j which is
the gold standard for graph databases
but memgraph is focused on building
real-time data pipelines like neo4j it
can be queried with Cipher but it's
written in C plus with a memory first
architecture that claims to achieve
better performance overall it looks like
it would be a great choice for building
something like a real-time analytics
platform next up we'll look at Key DB
which is a faster alternative to the
already extremely fast redis it was
recently acquired by snap a company that
needs to fetch data as fast as humanly
possible it can handle over 1 million
operations per second on a single node
which sounds like a lot to me but it's
not something you would use as your
primary database rather it's mostly used
as an in-memory cache to duplicate high
priority data from another database
making it load faster for the end user
it also has a library called mod.js
allowing you to extend the database with
your own custom commands and
functionality with JavaScript another
specialty database to look into is melee
search the full text search engine space
is dominated by tools like elasticsearch
and algolia a melee search is an open
source rust-based alternative that makes
it easy to implement an awesome search
feature into your app it's not a primary
database but instead a place where you
would duplicate data that needs to be
searchable in complex ways like you can
fine-tune different parameters to
control which attributes have a higher
ranking in the search in addition it has
a bunch of client libraries like instant
melee search that can quickly Implement
a search feature in a front-end web
application and finally that brings us
to mindsdb a database that integrates
popular machine learning Frameworks it
created a concept called AI tables that
embed predictive models directly into
the database at first glance it feels
like SQL except you can use it to write
statements that train machine learning
models when the model is done training
you can then use select to make
predictions from it so basically it
provides an into integrated way to
handle the ml Ops life cycle and it can
also integrate with your existing
machine learning models or those from
third parties like hugging face or open
AI in the not so distant future I
believe in betting AI directly into
databases will be a huge thing and
mindsdb gives you an idea of how that
could look and now you have 15 different
futuristic databases to choose from for
your next failed side project good luck
with that if I missed anything let me
know in the comments and if you want to
learn more about awesome serverless
databases like firestore and superbase
support my work as a pro member on
fireship iO thanks for watching and I
will see you in the next one
Посмотреть больше похожих видео
Learn What is Database | Types of Database | DBMS
Choosing a Database for Systems Design: All you need to know in one video
DAY 10 | PHP AND MYSQL | VI SEM | B.CA | ACCESSING MYSQL | L1
What is a Database?
SQL vs NoSQL in 2024 Make the Right Choice (Difference Explained)
How Tech is Revolutionizing Environmental Science! Crazy GeoMapping, Data, & More!
5.0 / 5 (0 votes)