What is Azure Cosmos DB? | Azure Cosmos DB Essentials Season 1

Summary

Welcome to the Azure Cosmos DB essentials series with Mustafa, the technical product marketing manager. This series covers Cosmos DB concepts for developers of all levels. Season one explores the basics, starting with an introduction to Azure Cosmos DB, a NoSQL database designed for horizontal scaling and high availability. It discusses the importance of partitioning and replication for performance and fault tolerance. Cosmos DB's schema-less nature allows for flexible data models, supporting SQL API, graph data model via Gremlin API, and APIs for MongoDB and Cassandra. The series promises to showcase real-world use cases in upcoming episodes.

Takeaways

• 🌐 **Introduction to Azure Cosmos DB**: Mustafa introduces Azure Cosmos DB as a NoSQL database service.
• 📈 **Big Data Challenges**: Discusses the evolution from gigabytes to petabytes and the need for quick data access.
• 🔄 **Horizontal Scaling**: Explains how Cosmos DB scales horizontally to handle large data volumes and high throughput.
• 🔑 **Partitioning**: Highlights the importance of partitioning in improving database latency and throughput.
• 🌐 **Replication**: Describes how replication within and across regions enhances fault tolerance and availability.
• 📚 **Schema-less**: Emphasizes Cosmos DB's schema-less nature allowing for flexible data structures.
• 💾 **Data Models**: Cosmos DB supports multiple data models including document, key-value, graph, and wide-column store.
• 🔍 **SQL API**: Cosmos DB offers a SQL API for document and key-value data models.
• 🌐 **Global Distribution**: Cosmos DB can be geo-replicated across multiple Azure regions for higher availability.
• 🔄 **Consistency**: The series will cover the concept of consistency in a distributed database in later episodes.

Q & A

What is the main focus of the Azure Cosmos DB essentials series?

The series focuses on discussing Cosmos DB concepts, tips, and tricks for both beginners and expert level Azure Cosmos DB developers.

Who is the presenter of the Azure Cosmos DB essentials series?

Mustafa, the technical product marketing manager for Azure Cosmos DB.

What will be covered in season one of the series?

Season one will cover the basics, starting with an introduction to Azure Cosmos DB, diving into use cases, discussing consistency in a distributed database, and ending with hybrid transactional and analytical processing and synapse link.

What is Azure Cosmos DB?

Azure Cosmos DB is a NoSQL database service that offers horizontal scaling, replication, partitioning, and a flexible database schema.

How does Azure Cosmos DB handle the demands of big data?

Azure Cosmos DB handles big data by scaling horizontally, which allows it to uncap the throughput and volume of data in the database, serving requests concurrently across multiple nodes.

What are the two kinds of partitions in Azure Cosmos DB?

There are two kinds of partitions in Azure Cosmos DB: logical partitions, which are logical groups across items referenced through a partition key, and physical partitions, which are actual units of storage physically located in the Azure region.

Why is the partition key important in Azure Cosmos DB?

The partition key is important because it tells the database engine where to look for data, which is a crucial part of optimizing the database for performance gains.

What are the two types of replication in Cosmos DB?

There are two types of replication in Cosmos DB: replication within a region, which improves fault tolerance, and geo-replication outside of a region, which increases availability.

How does Cosmos DB's schema-less nature benefit developers?

Cosmos DB's schema-less nature allows developers to work with structurally variable data without issues, making it easier to adapt and evolve applications as use cases change.

What data models does Cosmos DB support?

Cosmos DB supports multiple data models including document, key-value, graph, and it provides APIs for SQL, MongoDB, and Cassandra to cater to different developer preferences and use cases.

How does the graph data model work in Cosmos DB?

The graph data model in Cosmos DB uses the Gremlin API, which models complex systems with many-to-many relationships using edges and vertices, and traversal to navigate these relationships.

What are the benefits of using Cosmos DB for modern applications?

Cosmos DB offers benefits such as high performance, availability, flexible schema, and support for multiple data models, making it suitable for modern applications that require high volumes of variable data without compromising on performance or availability.

Outlines

00:00

🌐 Introduction to Azure Cosmos DB

The video script introduces the Azure Cosmos DB essentials series with Mustafa, the technical product marketing manager for Azure Cosmos DB. The series aims to cover concepts, tips, and tricks for developers of all levels. Season one will focus on the basics, starting with an introduction to Azure Cosmos DB, its capabilities, and use cases. The script discusses the evolution of data demands and how Azure Cosmos DB, a NoSQL database, addresses these with horizontal scaling, replication, partitioning, and flexible database schema. It highlights the importance of choosing the right partition key for optimizing database performance and the two types of replication within and outside of Azure regions for fault tolerance and availability. The schema-less nature of Cosmos DB is also emphasized, allowing for flexible data structures and easy adaptation to changing data needs.

05:02

🔍 Exploring Data Models in Cosmos DB

The second paragraph of the script delves into the various data models supported by Cosmos DB, which are designed to cater to different types of applications and data structures. It mentions the SQL API that allows for fast and cost-effective point reads when the document ID and partition key are known, making it an optimization tool for applications. The Gremlin API for the graph data model is introduced, which uses edges and vertices to model complex systems with many-to-many relationships. The script also discusses the MongoDB API and Cassandra API, which are designed for developers familiar with these open-source databases, allowing them to use their existing tools, SDKs, and drivers. The MongoDB API leverages document storage, while the Cassandra API is a wide-column store optimized for analytics. The paragraph concludes by summarizing the benefits of Cosmos DB, including its horizontal scaling, replication for availability, and flexible schema for ease of use.

Keywords

💡Azure Cosmos DB

Azure Cosmos DB is a globally distributed, multi-model database service provided by Microsoft Azure. It is designed to allow developers to work with various data types, including document, key-value, wide-column, and graph, through a series of APIs. The service is highlighted in the video as a solution for big data demands, emphasizing its ability to scale horizontally and handle large data volumes with high availability and performance.

💡Horizontal Scaling

Horizontal scaling, also known as scaling out, refers to the process of adding more machines or nodes to a system to increase its capacity or performance. In the context of the video, horizontal scaling is a key feature of Azure Cosmos DB that allows it to handle more data and traffic by distributing the workload across multiple nodes, as opposed to vertical scaling, which involves increasing the capacity of a single machine.

💡Partitioning

Partitioning in databases is the process of dividing a database into distinct, smaller parts called partitions. The video explains that Azure Cosmos DB uses both logical and physical partitions. Logical partitions are groups of items within a dataset referenced by a partition key, while physical partitions are actual units of storage in the cloud. Partitioning is crucial for improving database latency and throughput by distributing data across nodes.

💡Replication

Replication in the context of databases means creating copies of data to ensure redundancy and improve fault tolerance. The video describes two types of replication in Azure Cosmos DB: within a region, where data is replicated four times, and geo-replication, which extends data replication across different Azure regions. This feature enhances the database's availability and durability.

💡Consistency

Consistency in distributed databases refers to the ability to ensure that data is the same across all nodes at any given time. Although not explicitly detailed in the provided script, consistency is a concept that is important when discussing distributed databases like Azure Cosmos DB, where ensuring that all users see the same data at the same time can be challenging due to replication and partitioning.

💡NoSQL

NoSQL is an abbreviation for 'Not only SQL' and refers to a category of database management systems that do not adhere to the traditional relational database structure. The video emphasizes NoSQL databases like Azure Cosmos DB for their ability to scale horizontally and handle non-relational data structures, which is particularly useful for modern applications dealing with large and varied data sets.

💡Schema-less

Schema-less databases do not enforce a strict structure on the data that is stored. In the video, it is mentioned that Cosmos DB is schema-less, allowing different documents in the same collection to have different structures. This flexibility is beneficial for developers as it accommodates varying data types and structures without the need for predefined schemas.

💡Document Model

The document model is one of the data models supported by Azure Cosmos DB, which treats data as a set of documents. Documents are typically represented in JSON or BSON format and can vary in structure. The video mentions that Cosmos DB's SQL API supports document storage, allowing for flexible data modeling and efficient storage of semi-structured data.

💡Key-Value Store

A key-value store is a type of database that pairs every piece of data with a unique key, making it easy to retrieve the data quickly. The video script highlights that Cosmos DB supports key-value pairs and can perform fast point reads when the document ID and partition key are known, which is beneficial for applications requiring quick data access.

💡Graph Data Model

The graph data model is another data model supported by Azure Cosmos DB, accessible via the Gremlin API. It is used to represent complex systems with many-to-many relationships, using nodes (vertices) and edges to model data. The video script uses the analogy of a web or network to describe graphs, emphasizing their utility in modeling interconnected data.

💡Hybrid Transactional and Analytical Processing (HTAP)

HTAP refers to the ability of a database to perform both transactional processing (OLTP) and analytical processing (OLAP) simultaneously on the same data set. Although not directly mentioned in the script, HTAP is a concept that could be related to the video's theme, as it is a feature that Cosmos DB supports, allowing for real-time analytics on operational data.

Highlights

Introduction to Azure Cosmos DB essentials series by Mustafa, the technical product marketing manager.

Season one focuses on the basics of Cosmos DB, including use cases, consistency, and hybrid transactional and analytical processing.

Cosmos DB is a NoSQL database designed for horizontal scaling, replication, partitioning, and flexible database schema.

The evolution from simple applications to big data demands faster and more frequent data retrieval.

Azure Cosmos DB addresses big data demands with horizontal scaling capabilities.

Horizontal scaling is achieved through partitioning and replication, differentiating Cosmos DB from relational databases.

There are two types of partitions in Cosmos DB: logical and physical.

The partition key is crucial for optimizing database performance by directing the database engine to the correct data location.

Partitioning data across nodes improves database latency and throughput.

Replication within a region enhances fault tolerance, while geo-replication across regions increases availability.

Cosmos DB is schema-less, allowing for flexible and variable data structures.

Flexible schema and horizontal scaling make NoSQL databases like Cosmos DB attractive for modern operations.

Cosmos DB supports multiple data models, including SQL API for document and key-value support.

The graph data model in Cosmos DB uses Gremlin API to model complex systems with many-to-many relationships.

Cosmos DB provides APIs for MongoDB and Cassandra, allowing developers to use familiar tools and drivers.

The MongoDB API in Cosmos DB leverages documents, while the Cassandra API is a wide-column store optimized for analytics.

The first episode concludes by emphasizing Cosmos DB's horizontal scaling, NoSQL features, and ease of working with variable data.

Transcripts

(smooth music)

- Welcome to the Azure Cosmos DB essentials series.

My name is Mustafa and I'm the technical product

marketing manager for Azure Cosmos DB.

Now, throughout this series,

we'll discuss Cosmos DB concepts,

tips, tricks for both beginners

and expert level Azure Cosmos DB developers.

Season one will focus on the basics.

We'll start with this episode, What is Azure Cosmos DB,

then we'll dive into some use cases,

talk about the concept of consistency

in a distributed database,

and then the last episode of the season we'll cover

hybrid transactional and analytical processing

and synapse link.

So in this episode, let's start from the beginning.

I'll introduce you to Cosmos DB our NoSQL database.

We'll discuss at a high level horizontal scaling,

replication, partitioning, and database schema.

And then we'll dive into some of the data models

that Cosmos DB supports.

So, let's get started.

Applications used to be relatively simple.

We would have an app, an API layer

and a slow drip of data dropping into our database.

Now occasionally, we may have needed to surface that data

and look up device information in a catalog,

or look up an appointment on a calendar.

But often the data sat in the database

until it was called on however many days, months,

or even years later.

In the age of big data, where data sets have now grown

from a few gigabytes to tens of gigabytes and even petabytes

for some of our most demanding applications.

Data is pouring into our databases

and applications are demanding that data be surfaced quickly

and frequently to fulfill user expectations

of a personalized and fast application experience.

Now, Azure Cosmos DB has boomed as the answer

to the big data demands of cloud native applications.

Because non-relational or NoSQL databases

like Azure Cosmos DB scale horizontally

rather than vertically, we can essentially uncap

the throughput and volume of data in our database.

Rather than upgrading hardware on a single node

to serve requests faster, Cosmos DB distributes that work

across multiple node so requests can be served concurrently.

Scaling out verse scaling up is the biggest difference

between relational databases and non-relational databases

like Cosmos DB.

Horizontal scaling or scaling out

is the consequence of two techniques:

partitioning and replication.

Now there are two kinds of partitions in Azure Cosmos DB:

logical and physical.

A physical partition is an actual unit of storage

that physically sits in the Azure region that we dictate.

This is a piece of hardware holding our data

somewhere in the cloud.

Logical partitions are logical groups across items

within our data set, and we reference these groups

through something called a partition key.

The partition key is important

because it tells our database engine

where to look for our data.

Choosing the right partition key

is a simple but important part of optimizing our database,

and when done correctly,

we can expect to see performance gains proportional

to the additional number of nodes serving our requests.

Put simply, partitioning our data across nodes

improves database latency and throughput.

Now like with partitions,

there are also two kinds of replication in Cosmos DB:

replication within a region

and geo-replication outside of a region.

Now within a region, our data is replicated four times

as a redundancy measure, improving fault tolerance.

Outside of a region, our data is geo-replicated

into any additional Azure regions we select

resulting in higher availability.

Replication within a single region

increases fault tolerance.

Geo-replication into additional Azure regions

increases availability.

Now these two techniques, replication and partitioning,

are in large part what makes horizontal scaling possible.

Now what makes NoSQL attractive to modern op devs,

performance and availability gains aside,

are the relaxed constraints

around keeping our data relational.

Cosmos DB is schema-less meaning structure is not enforced

on any data entering the database.

Two documents in the same collection

can have completely different structure without any issue.

And as a result, developers can land

structurally variable data into the database

and later build, change or add functionality

as a use for that data evolves.

This combination of flexible schema and horizontal scaling

are what makes NoSQL different.

Cosmos DB makes it easy to ingest

and resurface high volumes of variable data

without compromising on performance or availability.

In addition to flexible schema,

Cosmos also supports multiple data models.

Our team has built Cosmos to be the hub

for all NOSQL data types.

Cosmos has a SQL API with document

and key value support built in.

Documents and key value pairs go hand in hand

in most NoSQL databases because documents

are just key value pairs in which the key is a document ID

and the value is a JSON object that we just call a document.

Now to reap the performance benefits

of a true key value store, with a SQL API,

we can do fast and cost-effective point reads

as long as we know the document ID and partition key

for the item we want to read.

This is a great optimization tool for applications.

The next data model, the graph data model,

is accessible via the Gremlin API,

which uses edges and vertices as well as well as traversal

to model complex systems with many to many relationships.

Think of a graph as a web or a network of nodes.

The API is for Mongo DB and Cassandra respectively,

are created so that developers

familiar with both open-source databases

can work with the tools, SDKs and drivers

that they're already used to.

API for Mongo DB also leverages documents,

and Cassandra API is a wide-column store,

which orients data efficiently in columns instead of rows

to optimize for analytics.

That's it for our first episode.

Cosmos DB is our horizontally scaling, NoSQL database.

Partitioning makes it fast, replication makes it available

and flexible schema makes it simple to work with.

Thanks for joining us, and tune into our next episode

on use cases to see some examples of how Cosmos DB

is being used in real applications.

(smooth music)