What is Object Storage?
Summary
TLDRThis script delves into the concept of object storage, ideal for low-cost, low-performance storage needs, particularly for internet workloads like web applications and content delivery. It explains the fundamental components of an object, including its unique ID, data, metadata, and attributes, and how they are organized into scalable 'buckets.' The script also highlights the benefits of object storage, such as global replication for improved latency in video streaming, efficient file sharing for global collaboration, and secure hosting for web assets and regulatory data.
Takeaways
- đŸ Object storage is designed for low-cost, low-performance storage suitable for Internet workloads, including web applications and hosting.
- đïž The term 'object' in object storage refers to any kind of file, which can be stored without file restrictions, but requires certain characteristics.
- đ Every object in object storage must have a unique identifier (ID) for retrieval, along with the actual data, metadata, and attributes.
- đ Metadata is crucial for object storage as it provides information about the file, such as creation date, file type, and size, enabling search and indexing.
- đĄïž Attributes in object storage define permissions and access rights, such as who can override, download, or delete the object.
- đïž Objects are stored in 'buckets' which can scale to hold billions of objects and are virtual constructs in the storage system.
- đ The replication of objects across multiple physical devices ensures data integrity and security, providing redundancy in case of hardware failure or network issues.
- đ Object storage pricing and metering are unique and can vary based on performance needs, with different tiers for 'cold' and 'cold-ish' storage.
- đ Object storage is ideal for global use cases like video streaming, where replication across different geographical locations can reduce latency.
- đ€ It supports file sharing and collaboration across the globe, allowing for version control and easy access for distributed teams.
- đïž Suitable for regulatory data and digital archiving, object storage provides a secure and cost-effective solution for long-term data retention.
Q & A
What is the primary purpose of object storage?
-The primary purpose of object storage is to provide a low-cost, low-performance storage solution designed for Internet workloads, such as web applications and hosting, as well as a secure and durable place for data that needs to be retained for long periods due to regulatory or legal reasons.
What are the four essential components of an object in object storage?
-The four essential components of an object in object storage are: 1) A unique identifier (ID) for the object, 2) The actual data content of the object, 3) Metadata that provides information about the object, and 4) Attributes that define permissions and other properties of the object.
How does object storage handle data replication for data integrity and security?
-Object storage replicates the data across multiple physically separated devices. This ensures that even if one device fails or there is a network outage, the data remains accessible from the other replicated locations, thus maintaining data integrity and security.
What is a 'bucket' in the context of object storage?
-In object storage, a 'bucket' is a virtual container where objects are stored. Buckets can scale to hold billions of objects and are a fundamental part of organizing data in object storage systems.
How does object storage pricing and metering work?
-Object storage pricing and metering are based on factors such as the number of 'gets' (data retrievals), 'puts' (data uploads), and the amount of storage used. Different providers may have unique metrics for billing, but performance level and data access frequency are key considerations that affect cost.
What are the different storage tiers mentioned in the script, and how do they differ in cost and use case?
-The script mentions 'cold times 10' as the coldest and least expensive tier, suitable for data archived for long periods. 'Cold' is the next tier, slightly more expensive, for data accessed occasionally. 'Cold-ish' is for more frequently accessed data, like website assets, and is more costly than the colder tiers.
How can object storage be used for video streaming?
-Object storage can be used for video streaming by replicating video files globally across different locations. This reduces latency and improves the streaming experience for users by hosting the content closer to them geographically.
What benefits does object storage offer for file sharing and collaboration?
-Object storage facilitates file sharing and collaboration by allowing multiple versions of a file to be uploaded and stored with metadata tracking changes. This enables secure, low-cost, and efficient global collaboration on documents, spreadsheets, and other files.
How is object storage useful for web hosting and digital archiving?
-Object storage is useful for web hosting as it can host website assets at a lower cost and with high availability. For digital archiving, it provides a secure and durable place to store and access large volumes of data that are not frequently accessed, such as digital photographs of manuscripts.
What is the role of APIs in interacting with object storage?
-APIs are the primary method of interaction with object storage. Users perform operations such as uploading, downloading, and managing objects through API calls, which allows for automation and integration with other systems.
How can object storage help with regulatory data and cold storage requirements?
-Object storage can store large volumes of data securely and durably, making it suitable for regulatory data that must be retained for long periods. The ability to replicate data across devices ensures that the data is protected against hardware failures or other issues.
Outlines
đïž Object Storage Basics and Components
The first paragraph introduces the concept of object storage, which is a storage tier designed for low-performance, low-cost storage suitable for Internet workloads such as web applications and hosting. It discusses the need for a secure and long-term storage solution for regulatory and legal reasons. The paragraph delves into the definition of an 'object' in object storage, explaining that it can be any kind of file without restrictions. It then outlines the four essential components of an object: a unique identifier (ID), the data itself, metadata which includes details about the file such as creation date and file type, and attributes that pertain to access and permissions. The paragraph concludes with an explanation of how objects are stored in 'buckets', which can scale to hold billions of objects and are replicated across different physical devices for data integrity and security.
đ Object Storage Pricing, Performance, and Use Cases
The second paragraph explores the complexities of object storage, particularly its pricing and metering structures, which can vary among providers. It emphasizes the importance of response time and performance, noting that higher performance options are more expensive. The paragraph categorizes data storage into different tiers based on access frequency, from the 'coldest of cold tiers' for rarely accessed data to 'cold-ish' for more frequently accessed data like website assets. It also highlights the benefits of object storage, such as its API-driven nature, which enables front-end customization. The paragraph discusses various use cases for object storage, including video streaming with global replication to reduce latency, file sharing for global collaboration, and web hosting for regulatory data and digital archiving projects, all while maintaining security and cost-effectiveness.
đ Closing Remarks and Further Resources
The final paragraph serves as a closing statement, thanking the audience for their time and inviting them to ask questions in the comments section. It encourages viewers to like, subscribe, and provide feedback for more content like the current video. Additionally, it promotes IBM Cloud Labs, which are free, browser-based interactive Kubernetes labs, and provides a call to action for viewers to grow their skills and earn badges. The paragraph concludes with a reminder for viewers to check the provided links for more information on IBM Cloud Labs.
Mindmap
Keywords
đĄObject Storage
đĄInternet Workloads
đĄMetadata
đĄAttributes
đĄBucket
đĄReplication
đĄAPI
đĄCold Storage
đĄResponsiveness
đĄVersioning
đĄDigital Archiving
Highlights
Object storage is designed for low-performance, low-cost storage suitable for Internet workloads.
Internet workloads include web applications, website hosting, and content delivery across the Internet.
Object storage serves as a modern alternative to tape storage for preserving large volumes of data.
An object in object storage can be any kind of file without restrictions.
Every object requires a unique identifier for retrieval.
Objects must contain actual data, which can be any file type.
Metadata is crucial for understanding the file's details and enabling search and indexing.
Attributes define permissions and access controls for the object.
Objects are stored in 'buckets' which can scale to hold billions of objects.
Buckets are virtual constructs that can be replicated across multiple physical devices for data integrity and security.
Object storage is accessed via an API, not through direct interaction with the storage buckets.
Different pricing tiers exist based on performance needs and access frequency.
Cold storage is the most cost-effective for data that is infrequently accessed.
Object storage is ideal for hosting assets for websites due to its scalability and cost-efficiency.
Replication of objects can be done globally to reduce latency for users in different locations.
Object storage supports versioning for collaborative file sharing across the globe.
It is cost-effective for file sharing compared to traditional methods like email or physical media.
Object storage is suitable for web hosting, regulatory data storage, and digital archiving projects.
IBM Cloud Labs offers free, browser-based interactive Kubernetes labs for skill enhancement.
Transcripts
If we think about object storage,
which is kind of a new-ish tier,
"object storage",
the idea is for relatively
low performance storage at a
relatively low cost
that is designed to serve the needs
of the Internet workload.
And so by Internet workloads, I'm
talking about Web applications.
I'm talking about website and
website hosting and delivering
content across the Internet,
but also a way to find
a new home for all of that data
that we used to store on tape.
Right. We have to have a way to
take big volumes of data,
write that down, put it somewhere
that is safe, that is secure,
and where we can hold on to it for a
long time for regulatory reasons,
legal reasons and other
requirements that make it
necessary for us to keep that data
for a long amount of time.
So let's start off with,
"what is an object?" and
why do we call it "object storage"?
So, the concept
in object storage is that you have
an object, right.
Now, that object
can be any kind
of file.
There aren't really any file
restrictions or anything like that.
There are some characteristics that
are very important.
But as a general rule, any
kind of file can be an object.
So, we'll throw
an object in there now, there
are 4 essential components
to that object that make
it usable in the computing sense.
The first one is for every
object, we have to have an ID,
we have to have some sort of a
unique identifier that lets
us know what this
object is when it comes time to
retrieve it. That's the first piece.
The second piece is, obviously,
you've got to have some data.
What's the point of having a file
and all of this information about it
if you don't have the data itself?
So this could be an Excel file
and could be a video file.
It could be an audio file, any kind
of file. But you've got to have the
actual data itself.
The third piece that you
have to have is
metadata.
So what is metadata?
Metadata is everything that
you need to know about
this file and about the
data itself.
Who created it?
When was it created?
What is it used for?
What is the file type?
How large is it?
The metadata is what makes it
possible to go out and find
it, search it, index it
so that you can bring it up and use
it whenever you need to.
Because, again, all that's being
displayed out right now is the ID.
The ID is not terribly useful.
And then the last thing that you
have to have are your attributes.
Now, attributes are related to
metadata. They aren't exactly the
same thing, but they're close.
Attributes can be, "are certain
users are allowed to override
it?", "are certain users are allowed
to download it?", "are certain users
are allowed to delete it?".
So the attributes are about the
object itself rather than about
the data.
But again, those are those 4
essential components.
This is what you use
when you actually create something.
So you've got your object here,
right.
And then what do you do with it?
Well, in the world of object
storage, which is unstructured,
you take all of your objects and you
drop them into a bucket.
So let's do this on this
side. So, imagine
I have a bucket, and
my bucket can have a handle, why
not?
And so into my bucket, I'm
going to put all of these objects
and now this is the neat part: these
buckets can be as big as you
need them to be.
They can scale to hold billions
of objects.
Now, it's not always a good idea to
put billions of objects in it, but
they can, in fact, scale to that
size.
And then what's really neat about
object storage - because remember,
we want to be able to use this for
cold storage, too, right, we need to
put things in it that can never be
deleted - is whenever you,
the user, you're going to interact
with this thing via an API.
Right. So you're not ever going to
interact with the bucket
individually here.
We'll throw you in.
There you are.
You're using object storage.
It's great.
So you're going to take your
objects. You're going to drop them
into this bucket. Now, on the back
end, the magic of object
storage is your object
here. Call it "O".
It is going to replicate out of this
bucket, the bucket is a virtual
construct, but we're going to take
it and we are going to say, all
right, well, I have
3 physically
separated devices.
And I'm going to take a copy of your
object and I'm going to replicate
it out into all 3 of them,
and my purpose in replicating
them out into all 3 of those
locations is data integrity,
data security.
That way, if the physical device
here, right, each of these buckets
is going to reside on a physical
device. So we'll call this P1,
P2, and P3.
So we're going to replicate it out,
your object is going to be on 3
physical, different devices, so if
there's a hardware problem or a
network outage or whatever in P1,
that's OK. You've still got copies
in P2 and P3, you can still get
the data that you need.
Now, when we get into
the details of object storage, all
of a sudden you start saying, man,
there's a lot of complexity in this.
Right? So not only do I have
the object buckets, but the way
that it's priced and metered and
build is kind of unique.
And people are asking me about how
many "gets" and how many "puts" and
how much storage am I taking up.
And the metering and billing around
object storage, everyone that sells
it has their own different metrics
on it. But here's the important
parts to consider.
The first is what
is your responsiveness or your
response time?
Right. So from a price perspective,
the higher performance that you want
and this is true for all cloud
storage, but the more performance
you want, the more it's
going to cost.
If you have data that you
have packaged up and it's data that
for 20 years your company has been
taking and writing it to tape,
sending that tape to vaults and
storing it in vaults where it will
probably never be seen again.
That can go to the very coldest of
cold tiers, right.
And that is going to be the least
expensive.
And so we're going to call that
"cold times 10".
That is the coldest of cold tiers.
Up from that, where you need
to get to it every so often, but you
don't really need to get to it much,
that's going to be a two dollar sign
one, right? We're going to charge a
little bit more for that.
We're just going to call that
"cold".
Maybe this is data that you need to
get to once every six months,
something like that.
But maybe you're earning a website
and you are using these object
storage buckets to host all of the
assets for your website.
Obviously that is going to cost
a little bit more, right, because
you are sending data out on a
regular basis.
So that's going to get our three
dollar signs.
And I'm going to call this
one called "cold-ish".
It's not really warm because
the concept of a warm here doesn't
really exist an object storage, but
it's cold-ish.
And so this is
videos, it is pictures,
it is physical files,
and what makes object storage really
neat is on the back end through
the magic - because remember, you,
you're interacting via an API,
you're not interacting with these
things directly - you
can do all kinds of nifty
front-end things for it.
And that's the really neat part of
object storage and that's the
benefits of it.
So let's talk about some of them,
right?
Hosting videos for streaming over
the Internet.
Perfect.
Perfect use case for object storage,
because like I said before, we can
replicate these things out.
Right. So for your video streaming,
rather than just replicating it into
three buckets that are all in the
same place, let's replicate it
out globally. Let's replicate it out
so that we are hosting those videos
not just in their primary location,
but in three different locations
in seven different countries around
the world, decreasing the latency
for your users so that their
streaming experience is better.
Let's talk about another good use
case: file
sharing. So let's imagine
a company where you're collaborating
on spreadsheets and PowerPoint, and
things like that, and you've got
collaborators that work all
over the world.
Well, for versioning
purposes, you can have
different versions that overwrite
the previous version as long as
those attributes allow for it.
And so if I've got my
friend Eric, who is working
in London, can create a file,
I can get a copy of that file,
make my changes to it, upload it
back in. And I've got version two.
That's the metadata.
And then I can have a colleague
in Singapore, Sam, who downloads
that file, make some more changes to
it, uploads it back in version
three, again, incrementing the
metadata.
But that allows for this Cross Globe
collaboration on a single file type,
and it makes everybody's lives
easier. But we're doing it at a very
low cost, much less expensive than
trying to send files
around via email because emails have
to get stored in a much more
expensive storage tier, much
less expensive than FedExing thumb
drives around.
And again, being able to be done
in a secure manner because we have
control over who can see and access
these objects.
Other Web hosting, obviously, is
going to be important. We talked
about regulatory data before, the
regulatory data and cold storage.
Let's say that you involved in a
digital archiving project
and you're working with a university
or a library and you're taking
digital photographs of manuscript
documents. And you want to host
those out for everyone in the world
to be able to see.
Object storage, perfect place to
store that kind of data - because
again, it's not getting accessed
very often.
It is write
once and then read many,
many, many times and you can host
it in a secure manner.
So hopefully you've enjoyed this.
This has been a good overview of
object storage.
As always, if you have any
questions, please feel free to leave
them in the comments. I'll do my
best to answer, or some of my
colleagues at IBM will.
And if you have any other questions,
just let us know.
Thank you so much for your time
today. If you have any questions,
please drop us a line below.
If you want to see more videos like
this in the future, please do like
and subscribe and let us know.
And don't forget, you can grow your
skills and earn a badge with IBM
Cloud Labs, which are free
browser-based interactive Kubernetes
labs that you can find more
information on by looking below.
Thanks again.
5.0 / 5 (0 votes)