15 futuristic databases you’ve never heard of

Summary

このビデオスクリプトでは、テック業界で成功するための速い動きと、物事を壊すことの重要性について語られています。その中で、1970年代の技術であるSQLがいかに現代のビッグデータの世界の基盤を形成しているか、そしてOracleやMySQLなどがどのように企業やプラットフォームを支配しているかを解説します。さらに、PlanetScaleやYugaByte、Neonなど、既存の関係データベースのパターンを基にしたものから、全く新しいアイデアを提供する15の革新的なデータベースについて紹介し、それぞれが解決しようとしている問題に焦点を当てています。このスクリプトは、最新のデータベース技術の進化と、それがどのようにテック業界に新たな可能性をもたらしているかを探求しています。

Takeaways

• 🤯 PlanetScale offers a serverless MySQL platform
• 🌟 Yugabyte and Neon provide serverless Postgres options
• 📝 DoltDB integrates database with Git version control
• 🦞 CockroachDB is a distributed SQL database built on Go
• ☁️ Cloudflare D1 runs SQL database on the edge
• 💪 Zeta combines Postgres and Elasticsearch
• 🔗 EightBase offers GraphQL API and low-code frontends
• 🕸 EdgeDB is a graph-relational database
• 🌠 Fauna is a multi-model database with joins and GraphQL
• 🤖 MindsDB integrates machine learning models into SQL

Q & A

PlanetScaleはどのような技術に基づいていますか？

PlanetScaleは、MySQLを水平方向にスケーリングするためのライブラリであるVitessに基づいています。

YugaByteの目標は何ですか？

YugaByteの目標は、クラウドで無限にスケールすることを目指しながら、Postgresのネイティブ機能を全てサポートすることです。

Neonデータベースの特徴は何ですか？

Neonはサーバーレスで、ゼロまでスケールダウンでき、高度に磨かれた管理ダッシュボードを持ち、Rustで書かれています。

Doltデータベースのユニークな機能は何ですか？

Doltは、MySQLとGitの機能を組み合わせたデータベースで、データのブランチ作成や変更のマージが可能です。

CockroachDBの開発者はどのような背景を持っていますか？

CockroachDBは、元Google社員によって開発されました。

Cloudflare D1の特異なアプローチは何ですか？

Cloudflare D1は、データベース内で直接実行されるJavaScriptで書かれたストアドプロシージャを使用して、トランザクションの制限を克服しています。

Zetaデータベースが特別な理由は何ですか？

Zetaは、データをスプレッドシートのように扱い、フルテキスト検索を内蔵しているため、データを他のサービスに複製する必要がありません。

Edge DBはどのようにしてデータベースのクエリを簡素化しますか？

Edge DBは、オブジェクト間の関係をグラフのように扱い、結合の必要性をなくして、クエリを簡素化します。

Faunaデータベースの独特な機能は何ですか？

Faunaは、ドキュメントデータベースの使いやすさと、複雑な関係データに適したネイティブジョインのサポートを組み合わせています。

MindsDBはどのようにしてデータベースを革新していますか？

MindsDBは、AIテーブルを使用して機械学習モデルをデータベースに直接組み込み、クエリを使用してモデルの訓練と予測を行うことができます。

Outlines

00:00

Introducing innovative new databases

The paragraph discusses several new databases that aim to improve on traditional SQL databases by scaling better, having nicer interfaces, integrating machine learning models, etc.

05:02

Graph databases for more flexible data modeling

The paragraph highlights graph databases like EdgeDB and SurrealDB that allow more intuitive data modeling based on connections between data types rather than strict tables and joins.

Keywords

💡リレーショナルデータベース

従来のSQLベースのデータベース。 YouTubeやFacebookなどのプラットフォームの中核を支えているが、スケーラビリティに課題がある。PlanetscaleやDoltなどの新しいクラウドネイティブのデータベースがこれに代わろうとしている。

💡スケーラビリティ

データベースが大規模なトラフィックに対応できる性能。YouTubeがVitessを開発した背景にある主要な課題の1つ。サーバーレスアーキテクチャや水平スケーリングなどのアプローチで解決しようとしている。

💡Dolt

Gitのブランチ・マージ機能を取り入れたリレーショナルデータベース。データの変更履歴を容易に追跡、解析できる。

💡クラウドネイティブ

パブリッククラウドを前提としたアプリケーション設計・開発手法。スケーラビリティや信頼性を高めることに焦点。PlanetscaleやServerless Postgresなどが該当。

Highlights

Planetscale provides a fully managed MySQL database that can scale horizontally

YugabyteDB is an open source database that can scale PostgreSQL infinitely in the cloud

NeonDB is a serverless PostgreSQL database written in Rust that can scale to zero

DoltDB combines MySQL and Git to allow branching experimental database changes

CockroachDB is a distributed SQL database built to scale horizontally in the cloud

Cloudflare D1 is a lightweight SQL database for the edge that runs JavaScript transactions

ZetaDB is like a developer-friendly Airtable, combining PostgreSQL and Elasticsearch

EdgeDB is a graph-relational hybrid database for PostgreSQL with intuitive data modeling

SurrealDB combines SQL, document, and graph models with horizontal scaling written in Rust

FaunaDB supports joins and transactions like SQL with flexible schema like MongoDB

Memgraph is a graph database alternative to Neo4j focused on real-time analytics

KeyDB is an ultrafast Redis-compatible in-memory database for caching

MeiliSearch is an open source search engine written in Rust as an alternative to Elasticsearch

MindsDB integrates machine learning workflows directly into a database via AI Tables

The database landscape is rapidly evolving with many new and innovative options

Transcripts

if you want to be successful in Tech you

need to move fast and break things

that's exactly what I said to my boss

before he fired me for deleting a

production database I should fire

the relational database management

system is one of the most successful

ideas in the history of computing SQL is

1970s Tech that still forms the backbone

of the modern Big Data World Oracle

dominates Enterprise while MySQL is the

primary database for platforms like

YouTube and Facebook but SQL is by no

means perfect in 2023 there's a whole

new wave of futuristic Cutting Edge

avant-garde game changing databases

looking to move fast and break things

some of them build on top of existing

relational patterns While others bring

entirely new ideas to the table that

you've never seen before no pun intended

over the next few minutes we'll look at

15 crazy ass new databases and talk

about the problems they're trying to

solve first on the list we have Planet

scale which is not a new database but

rather a serverless platform for MySQL

like I mentioned before YouTube uses

MySQL however back in 2010 when it was

getting popular it was extremely

difficult to scale at least for an app

with millions of simultaneous

connections instead of switching to a

nosql document database they developed a

library called Vitas which makes some

trade-offs to scale MySQL horizontally

and this is the technology that planet

scales based on it provides a fully

managed database that can scale with

vtas and makes it available from a nice

dashboard with sdks to go along with it

and it also integrates nicely with tools

like Prisma it has a free tier and good

pricing on the page here that they're

likely losing money on by comparison a

basic database on Heroku is 50 bucks and

it's unreasonably expensive on AWS and

Google Cloud Planet scale is not the

only company with this serverless

database idea though yoga byte has its

own open source solution for postgres

its goal is to scale infinitely in the

cloud but attempt to support all the

native features in postgres which is not

easy to do in a cloud native environment

you can actually use a multi-cloud

strategy to host your database on

multiple clouds at the same time

eliminating the dreaded vendor lock-in

issue but there's another option for

scaling postgres called neon it's

currently in technical preview but looks

extremely promising it's serverless and

can scale down to zero and has a highly

polished admin dashboard to go along

with it and most importantly it's

written in Rust which is the only thing

that matters if you want to look cool in

front of your friends it also supports

branching which I think was inspired by

another database called dolt this

database would be like give MySQL and

git had a baby most importantly it

allows you to create branches from the

main database where you can experiment

with new data without breaking things it

makes it easy to analyze the diff

between changes and when everything

looks good the changes can be merged

back into the main branch and it's done

with the same conventions found in git

that virtually every developer is

familiar with dolt has a hosted option

but it is quite expensive they've raised

21 million dollars and because I'm

extremely entitled I demand they burn

more of this money to give us a free

tier I just paid eight dollars for a

dozen eggs and can't afford to put

another fifty dollars on my mom's credit

card now another sequel flavor I want to

talk about is cockroachdb it's been

around for quite a while and was

developed by ex Google employees and is

often referred to as new SQL it's

compatible with postgres but was built

from the ground up with go to scale

horizontally in the cloud it has a nice

free tier pretty similar to Planet scale

then becomes pay as you go pricing after

that it's yet another great option but

you might also want to consider

Foundation DB or titanium DB as well

which are also in this new SQL class but

in some cases you might prefer a

lightweight alternative like cloud flare

D1 this is a brand new product that's

still in beta and basically it's a SQL

Lite database that runs on the edge

cloudflare already has its own key value

store for workers as well as R2 for

object storage but D1 allows you to

create data on the edge that can be

queried one trade-off though is that

native transactions will not work

however they're taking a very cool

approach to overcome that limitation by

allowing developers to write stored

procedures with JavaScript which are

just Snippets of code that run directly

in your database which can be used to

create transactions or other reusable

queries this approach has a ton of

potential because historically store

procedures are a pain to work with but

if it's in JavaScript even I could do it

pretty cool but it's not production

ready yet if you want something that's

ready to go today check out Zeta yet

another serverless relational database

that's based on postgres and

elasticsearch under the hood but what

makes it special is that it treats your

data like a spreadsheet it feels kind of

like a developer friendly alternative to

airtable or notion and what's really

awesome is that it has full text search

built in without needing to duplicate

your data in some other service like

algolia it has sdks for typescript in

Python and has a schema editor that

makes it very easy to create and

visualize relationships between tables

and it can also do branching like we

looked at in adult DB for pricing it has

a solid free tier then it scales up in a

linear way based on usage but another

tool that takes things one step further

is eight base it's also a relational

database that provides a graphql API for

your data out of the box making it

similar to tools like hasura what makes

it unique though is that it also comes

with a low code tool for quickly

building front-end applications that

connect to your back end it's like a

backend as a service that can also build

your front end but that's enough talk

about relational databases now we enter

the post SQL ERA with Edge DB it's a

graph relational database that's powered

by postgres under the hood when it comes

to relationships humans don't naturally

think of columns and rows but rather

objects connected to each other like a

graph instead of defining tables like

you would in SQL you define types like

you would in most programming languages

these types link to other types which is

awesome because it eliminates the need

for joins and simplifies the way you

make queries overall all it almost feels

like an orm for postgres that allows you

to think about relationships in a far

more human friendly way currently it's

open source and can be self-hosted but

there's a wait list for their cloud

service now another relational graph

like database is surreal DB it's written

in Rust and I have a full tutorial on my

second Channel Beyond fireship it's

designed to support acid transactions

while scaling horizontally but what I

really like about it is how easy and

flexible it makes data modeling for the

most part it feels like SQL but also

uses arrows to connect nodes and edges

like you would in Cipher with a graph

database like neo4j it takes the best

elements of relational document and

graph databases and combines them into

one easy to use API currently it can

only be self-hosted but they do have a

cloud service coming soon the

multi-model idea is nothing new though

another database pushing the limits is

fauna it was created by X Twitter

Engineers who set out to build the

perfect database it's extremely easy to

use like a document database similar to

mongodb but unlike a document database

it supports features like native joins

which is the number one missing feature

in document databases and that means

fauna is more well suited for complex

relational data like social graphs to

interact with it it has its own custom

query language called fql but also comes

with a graphql API it is a closed Source

product so you can't self-host it but

its fully managed service has a nice

free tier another new database you've

likely never heard of is memograph it's

a drop in replacement for neo4j which is

the gold standard for graph databases

but memgraph is focused on building

real-time data pipelines like neo4j it

can be queried with Cipher but it's

written in C plus with a memory first

architecture that claims to achieve

better performance overall it looks like

it would be a great choice for building

something like a real-time analytics

platform next up we'll look at Key DB

which is a faster alternative to the

already extremely fast redis it was

recently acquired by snap a company that

needs to fetch data as fast as humanly

possible it can handle over 1 million

operations per second on a single node

which sounds like a lot to me but it's

not something you would use as your

primary database rather it's mostly used

as an in-memory cache to duplicate high

priority data from another database

making it load faster for the end user

it also has a library called mod.js

allowing you to extend the database with

your own custom commands and

functionality with JavaScript another

specialty database to look into is melee

search the full text search engine space

is dominated by tools like elasticsearch

and algolia a melee search is an open

source rust-based alternative that makes

it easy to implement an awesome search

feature into your app it's not a primary

database but instead a place where you

would duplicate data that needs to be

searchable in complex ways like you can

fine-tune different parameters to

control which attributes have a higher

ranking in the search in addition it has

a bunch of client libraries like instant

melee search that can quickly Implement

a search feature in a front-end web

application and finally that brings us

to mindsdb a database that integrates

popular machine learning Frameworks it

created a concept called AI tables that

embed predictive models directly into

the database at first glance it feels

like SQL except you can use it to write

statements that train machine learning

models when the model is done training

you can then use select to make

predictions from it so basically it

provides an into integrated way to

handle the ml Ops life cycle and it can

also integrate with your existing

machine learning models or those from

third parties like hugging face or open

AI in the not so distant future I

believe in betting AI directly into

databases will be a huge thing and

mindsdb gives you an idea of how that

could look and now you have 15 different

futuristic databases to choose from for

your next failed side project good luck

with that if I missed anything let me

know in the comments and if you want to

learn more about awesome serverless

databases like firestore and superbase

support my work as a pro member on

fireship iO thanks for watching and I

will see you in the next one