What is Object Storage?

Summary

This script delves into the concept of object storage, ideal for low-cost, low-performance storage needs, particularly for internet workloads like web applications and content delivery. It explains the fundamental components of an object, including its unique ID, data, metadata, and attributes, and how they are organized into scalable 'buckets.' The script also highlights the benefits of object storage, such as global replication for improved latency in video streaming, efficient file sharing for global collaboration, and secure hosting for web assets and regulatory data.

Takeaways

• 💾 Object storage is designed for low-cost, low-performance storage suitable for Internet workloads, including web applications and hosting.
• 🗂️ The term 'object' in object storage refers to any kind of file, which can be stored without file restrictions, but requires certain characteristics.
• 🔑 Every object in object storage must have a unique identifier (ID) for retrieval, along with the actual data, metadata, and attributes.
• 📁 Metadata is crucial for object storage as it provides information about the file, such as creation date, file type, and size, enabling search and indexing.
• 🛡️ Attributes in object storage define permissions and access rights, such as who can override, download, or delete the object.
• 🗑️ Objects are stored in 'buckets' which can scale to hold billions of objects and are virtual constructs in the storage system.
• 🔄 The replication of objects across multiple physical devices ensures data integrity and security, providing redundancy in case of hardware failure or network issues.
• 📈 Object storage pricing and metering are unique and can vary based on performance needs, with different tiers for 'cold' and 'cold-ish' storage.
• 🌐 Object storage is ideal for global use cases like video streaming, where replication across different geographical locations can reduce latency.
• 🤝 It supports file sharing and collaboration across the globe, allowing for version control and easy access for distributed teams.
• 🏛️ Suitable for regulatory data and digital archiving, object storage provides a secure and cost-effective solution for long-term data retention.

Q & A

What is the primary purpose of object storage?

The primary purpose of object storage is to provide a low-cost, low-performance storage solution designed for Internet workloads, such as web applications and hosting, as well as a secure and durable place for data that needs to be retained for long periods due to regulatory or legal reasons.

What are the four essential components of an object in object storage?

The four essential components of an object in object storage are: 1) A unique identifier (ID) for the object, 2) The actual data content of the object, 3) Metadata that provides information about the object, and 4) Attributes that define permissions and other properties of the object.

How does object storage handle data replication for data integrity and security?

Object storage replicates the data across multiple physically separated devices. This ensures that even if one device fails or there is a network outage, the data remains accessible from the other replicated locations, thus maintaining data integrity and security.

What is a 'bucket' in the context of object storage?

In object storage, a 'bucket' is a virtual container where objects are stored. Buckets can scale to hold billions of objects and are a fundamental part of organizing data in object storage systems.

How does object storage pricing and metering work?

Object storage pricing and metering are based on factors such as the number of 'gets' (data retrievals), 'puts' (data uploads), and the amount of storage used. Different providers may have unique metrics for billing, but performance level and data access frequency are key considerations that affect cost.

What are the different storage tiers mentioned in the script, and how do they differ in cost and use case?

The script mentions 'cold times 10' as the coldest and least expensive tier, suitable for data archived for long periods. 'Cold' is the next tier, slightly more expensive, for data accessed occasionally. 'Cold-ish' is for more frequently accessed data, like website assets, and is more costly than the colder tiers.

How can object storage be used for video streaming?

Object storage can be used for video streaming by replicating video files globally across different locations. This reduces latency and improves the streaming experience for users by hosting the content closer to them geographically.

What benefits does object storage offer for file sharing and collaboration?

Object storage facilitates file sharing and collaboration by allowing multiple versions of a file to be uploaded and stored with metadata tracking changes. This enables secure, low-cost, and efficient global collaboration on documents, spreadsheets, and other files.

How is object storage useful for web hosting and digital archiving?

Object storage is useful for web hosting as it can host website assets at a lower cost and with high availability. For digital archiving, it provides a secure and durable place to store and access large volumes of data that are not frequently accessed, such as digital photographs of manuscripts.

What is the role of APIs in interacting with object storage?

APIs are the primary method of interaction with object storage. Users perform operations such as uploading, downloading, and managing objects through API calls, which allows for automation and integration with other systems.

How can object storage help with regulatory data and cold storage requirements?

Object storage can store large volumes of data securely and durably, making it suitable for regulatory data that must be retained for long periods. The ability to replicate data across devices ensures that the data is protected against hardware failures or other issues.

Outlines

00:00

🗃️ Object Storage Basics and Components

The first paragraph introduces the concept of object storage, which is a storage tier designed for low-performance, low-cost storage suitable for Internet workloads such as web applications and hosting. It discusses the need for a secure and long-term storage solution for regulatory and legal reasons. The paragraph delves into the definition of an 'object' in object storage, explaining that it can be any kind of file without restrictions. It then outlines the four essential components of an object: a unique identifier (ID), the data itself, metadata which includes details about the file such as creation date and file type, and attributes that pertain to access and permissions. The paragraph concludes with an explanation of how objects are stored in 'buckets', which can scale to hold billions of objects and are replicated across different physical devices for data integrity and security.

05:01

📊 Object Storage Pricing, Performance, and Use Cases

The second paragraph explores the complexities of object storage, particularly its pricing and metering structures, which can vary among providers. It emphasizes the importance of response time and performance, noting that higher performance options are more expensive. The paragraph categorizes data storage into different tiers based on access frequency, from the 'coldest of cold tiers' for rarely accessed data to 'cold-ish' for more frequently accessed data like website assets. It also highlights the benefits of object storage, such as its API-driven nature, which enables front-end customization. The paragraph discusses various use cases for object storage, including video streaming with global replication to reduce latency, file sharing for global collaboration, and web hosting for regulatory data and digital archiving projects, all while maintaining security and cost-effectiveness.

10:01

📞 Closing Remarks and Further Resources

The final paragraph serves as a closing statement, thanking the audience for their time and inviting them to ask questions in the comments section. It encourages viewers to like, subscribe, and provide feedback for more content like the current video. Additionally, it promotes IBM Cloud Labs, which are free, browser-based interactive Kubernetes labs, and provides a call to action for viewers to grow their skills and earn badges. The paragraph concludes with a reminder for viewers to check the provided links for more information on IBM Cloud Labs.

Keywords

💡Object Storage

Object storage is a type of storage architecture that manages data as objects, as opposed to other storage architectures like file systems which manage data as a hierarchy of files and directories. It is designed for scalability, flexibility, and is often used for internet-scale applications. In the video, object storage is described as a relatively low-cost, low-performance storage solution ideal for internet workloads, such as web applications and hosting, as well as for archiving large volumes of data.

💡Internet Workloads

Internet workloads refer to the demands and processes associated with running applications and services over the internet. This includes activities like web browsing, online transactions, and data transfers. The video discusses how object storage is tailored to serve these types of workloads, emphasizing its use for web applications, website hosting, and content delivery across the internet.

💡Metadata

Metadata in the context of object storage is data that describes other data. It includes information such as who created the file, when it was created, its purpose, file type, and size. The video explains that metadata is crucial for searching, indexing, and retrieving objects within object storage, as it provides the necessary details to manage and understand the stored data.

💡Attributes

Attributes in object storage are properties or characteristics of an object that define its behavior or access permissions. The video mentions that attributes might include information like whether certain users are allowed to override, download, or delete an object. These attributes are distinct from metadata, focusing more on the object's permissions and less on the data itself.

💡Bucket

In object storage, a bucket is a basic container that holds objects. The video describes buckets as scalable and capable of holding billions of objects. They are virtual constructs that can be used to organize and manage the storage of data objects, with the ability to replicate objects across different physical devices for data integrity and security.

💡Replication

Replication in object storage refers to the process of making copies of data and storing them in different physical locations. The purpose, as explained in the video, is to ensure data integrity and security. If one physical device fails, the replicated copies on other devices ensure that the data remains accessible.

💡API

API stands for Application Programming Interface, which is a set of rules and protocols for building and interacting with software applications. In the context of the video, object storage is interacted with via an API, allowing users to manage objects programmatically without direct interaction with the storage buckets.

💡Cold Storage

Cold storage in the context of data storage refers to the practice of storing data that is infrequently accessed, often for long-term retention. The video discusses how object storage can be used for cold storage, particularly for data that needs to be retained for regulatory or legal reasons, and how it can be cost-effective for such use cases.

💡Responsiveness

Responsiveness in the video refers to the speed and performance of data retrieval from object storage. It is a factor that affects the cost of using object storage services, with higher performance levels incurring higher costs. The video explains that the responsiveness needed depends on how often the data is accessed.

💡Versioning

Versioning in object storage is the process of keeping multiple versions of an object, allowing users to track changes and revert to previous states if necessary. The video uses the example of file sharing and collaboration across the globe, where different versions of a file are created and stored with updated metadata, facilitating easy collaboration and version control.

💡Digital Archiving

Digital archiving is the process of preserving digital materials, such as documents, photographs, and manuscripts, in a digital format. The video mentions object storage as an ideal solution for digital archiving projects, particularly when working with institutions like universities or libraries, due to its scalability, security, and cost-effectiveness for storing and accessing large volumes of data.

Highlights

Object storage is designed for low-performance, low-cost storage suitable for Internet workloads.

Internet workloads include web applications, website hosting, and content delivery across the Internet.

Object storage serves as a modern alternative to tape storage for preserving large volumes of data.

An object in object storage can be any kind of file without restrictions.

Every object requires a unique identifier for retrieval.

Objects must contain actual data, which can be any file type.

Metadata is crucial for understanding the file's details and enabling search and indexing.

Attributes define permissions and access controls for the object.

Objects are stored in 'buckets' which can scale to hold billions of objects.

Buckets are virtual constructs that can be replicated across multiple physical devices for data integrity and security.

Object storage is accessed via an API, not through direct interaction with the storage buckets.

Different pricing tiers exist based on performance needs and access frequency.

Cold storage is the most cost-effective for data that is infrequently accessed.

Object storage is ideal for hosting assets for websites due to its scalability and cost-efficiency.

Replication of objects can be done globally to reduce latency for users in different locations.

Object storage supports versioning for collaborative file sharing across the globe.

It is cost-effective for file sharing compared to traditional methods like email or physical media.

Object storage is suitable for web hosting, regulatory data storage, and digital archiving projects.

IBM Cloud Labs offers free, browser-based interactive Kubernetes labs for skill enhancement.

Transcripts

If we think about object storage,

which is kind of a new-ish tier,

"object storage",

the idea is for relatively

low performance storage at a

relatively low cost

that is designed to serve the needs

of the Internet workload.

And so by Internet workloads, I'm

talking about Web applications.

I'm talking about website and

website hosting and delivering

content across the Internet,

but also a way to find

a new home for all of that data

that we used to store on tape.

Right. We have to have a way to

take big volumes of data,

write that down, put it somewhere

that is safe, that is secure,

and where we can hold on to it for a

long time for regulatory reasons,

legal reasons and other

requirements that make it

necessary for us to keep that data

for a long amount of time.

So let's start off with,

"what is an object?" and

why do we call it "object storage"?

So, the concept

in object storage is that you have

an object, right.

Now, that object

can be any kind

of file.

There aren't really any file

restrictions or anything like that.

There are some characteristics that

are very important.

But as a general rule, any

kind of file can be an object.

So, we'll throw

an object in there now, there

are 4 essential components

to that object that make

it usable in the computing sense.

The first one is for every

object, we have to have an ID,

we have to have some sort of a

unique identifier that lets

us know what this

object is when it comes time to

retrieve it. That's the first piece.

The second piece is, obviously,

you've got to have some data.

What's the point of having a file

and all of this information about it

if you don't have the data itself?

So this could be an Excel file

and could be a video file.

It could be an audio file, any kind

of file. But you've got to have the

actual data itself.

The third piece that you

have to have is

metadata.

So what is metadata?

Metadata is everything that

you need to know about

this file and about the

data itself.

Who created it?

When was it created?

What is it used for?

What is the file type?

How large is it?

The metadata is what makes it

possible to go out and find

it, search it, index it

so that you can bring it up and use

it whenever you need to.

Because, again, all that's being

displayed out right now is the ID.

The ID is not terribly useful.

And then the last thing that you

have to have are your attributes.

Now, attributes are related to

metadata. They aren't exactly the

same thing, but they're close.

Attributes can be, "are certain

users are allowed to override

it?", "are certain users are allowed

to download it?", "are certain users

are allowed to delete it?".

So the attributes are about the

object itself rather than about

the data.

But again, those are those 4

essential components.

This is what you use

when you actually create something.

So you've got your object here,

right.

And then what do you do with it?

Well, in the world of object

storage, which is unstructured,

you take all of your objects and you

drop them into a bucket.

So let's do this on this

side. So, imagine

I have a bucket, and

my bucket can have a handle, why

not?

And so into my bucket, I'm

going to put all of these objects

and now this is the neat part: these

buckets can be as big as you

need them to be.

They can scale to hold billions

of objects.

Now, it's not always a good idea to

put billions of objects in it, but

they can, in fact, scale to that

size.

And then what's really neat about

object storage - because remember,

we want to be able to use this for

cold storage, too, right, we need to

put things in it that can never be

deleted - is whenever you,

the user, you're going to interact

with this thing via an API.

Right. So you're not ever going to

interact with the bucket

individually here.

We'll throw you in.

There you are.

You're using object storage.

It's great.

So you're going to take your

objects. You're going to drop them

into this bucket. Now, on the back

end, the magic of object

storage is your object

here. Call it "O".

It is going to replicate out of this

bucket, the bucket is a virtual

construct, but we're going to take

it and we are going to say, all

right, well, I have

3 physically

separated devices.

And I'm going to take a copy of your

object and I'm going to replicate

it out into all 3 of them,

and my purpose in replicating

them out into all 3 of those

locations is data integrity,

data security.

That way, if the physical device

here, right, each of these buckets

is going to reside on a physical

device. So we'll call this P1,

P2, and P3.

So we're going to replicate it out,

your object is going to be on 3

physical, different devices, so if

there's a hardware problem or a

network outage or whatever in P1,

that's OK. You've still got copies

in P2 and P3, you can still get

the data that you need.

Now, when we get into

the details of object storage, all

of a sudden you start saying, man,

there's a lot of complexity in this.

Right? So not only do I have

the object buckets, but the way

that it's priced and metered and

build is kind of unique.

And people are asking me about how

many "gets" and how many "puts" and

how much storage am I taking up.

And the metering and billing around

object storage, everyone that sells

it has their own different metrics

on it. But here's the important

parts to consider.

The first is what

is your responsiveness or your

response time?

Right. So from a price perspective,

the higher performance that you want

and this is true for all cloud

storage, but the more performance

you want, the more it's

going to cost.

If you have data that you

have packaged up and it's data that

for 20 years your company has been

taking and writing it to tape,

sending that tape to vaults and

storing it in vaults where it will

probably never be seen again.

That can go to the very coldest of

cold tiers, right.

And that is going to be the least

expensive.

And so we're going to call that

"cold times 10".

That is the coldest of cold tiers.

Up from that, where you need

to get to it every so often, but you

don't really need to get to it much,

that's going to be a two dollar sign

one, right? We're going to charge a

little bit more for that.

We're just going to call that

"cold".

Maybe this is data that you need to

get to once every six months,

something like that.

But maybe you're earning a website

and you are using these object

storage buckets to host all of the

assets for your website.

Obviously that is going to cost

a little bit more, right, because

you are sending data out on a

regular basis.

So that's going to get our three

dollar signs.

And I'm going to call this

one called "cold-ish".

It's not really warm because

the concept of a warm here doesn't

really exist an object storage, but

it's cold-ish.

And so this is

videos, it is pictures,

it is physical files,

and what makes object storage really

neat is on the back end through

the magic - because remember, you,

you're interacting via an API,

you're not interacting with these

things directly - you

can do all kinds of nifty

front-end things for it.

And that's the really neat part of

object storage and that's the

benefits of it.

So let's talk about some of them,

right?

Hosting videos for streaming over

the Internet.

Perfect.

Perfect use case for object storage,

because like I said before, we can

replicate these things out.

Right. So for your video streaming,

rather than just replicating it into

three buckets that are all in the

same place, let's replicate it

out globally. Let's replicate it out

so that we are hosting those videos

not just in their primary location,

but in three different locations

in seven different countries around

the world, decreasing the latency

for your users so that their

streaming experience is better.

Let's talk about another good use

case: file

sharing. So let's imagine

a company where you're collaborating

on spreadsheets and PowerPoint, and

things like that, and you've got

collaborators that work all

over the world.

Well, for versioning

purposes, you can have

different versions that overwrite

the previous version as long as

those attributes allow for it.

And so if I've got my

friend Eric, who is working

in London, can create a file,

I can get a copy of that file,

make my changes to it, upload it

back in. And I've got version two.

That's the metadata.

And then I can have a colleague

in Singapore, Sam, who downloads

that file, make some more changes to

it, uploads it back in version

three, again, incrementing the

metadata.

But that allows for this Cross Globe

collaboration on a single file type,

and it makes everybody's lives

easier. But we're doing it at a very

low cost, much less expensive than

trying to send files

around via email because emails have

to get stored in a much more

expensive storage tier, much

less expensive than FedExing thumb

drives around.

And again, being able to be done

in a secure manner because we have

control over who can see and access

these objects.

Other Web hosting, obviously, is

going to be important. We talked

about regulatory data before, the

regulatory data and cold storage.

Let's say that you involved in a

digital archiving project

and you're working with a university

or a library and you're taking

digital photographs of manuscript

documents. And you want to host

those out for everyone in the world

to be able to see.

Object storage, perfect place to

store that kind of data - because

again, it's not getting accessed

very often.

It is write

once and then read many,

many, many times and you can host

it in a secure manner.

So hopefully you've enjoyed this.

This has been a good overview of

object storage.

As always, if you have any

questions, please feel free to leave

them in the comments. I'll do my

best to answer, or some of my

colleagues at IBM will.

And if you have any other questions,

just let us know.

Thank you so much for your time

today. If you have any questions,

please drop us a line below.

If you want to see more videos like

this in the future, please do like

and subscribe and let us know.

And don't forget, you can grow your

skills and earn a badge with IBM

Cloud Labs, which are free

browser-based interactive Kubernetes

labs that you can find more

information on by looking below.

Thanks again.