Anyone who has ever had to distribute massive file content has experienced first-hand that quickly finding and moving that data can be a challenge. With the rise of media files, HD videos, engineering files, or even medical images, more and more organizations are struggling to manage and archive an ever rising flood of data.
In addition, it seems that the wide-spread use of mobile devices such as smartphones and tablets in the workplace has just added to this phenomenon. Social networks have placed heavy demand on businesses to have their tech savvy employees active on social media, using their mobile devices. They are now creating, storing and sharing huge amounts of pictures and streaming videos anytime, anyplace.
So, how to best surpass the key storage challenges that organizations face today? We have only one thing to say: Object Storage, Here We Come!
Object storage based solutions are receiving a lot of attention lately, and rightfully so. Its inherent ease of integration and low cost have made it an ideal solution for specific use cases. Where we used to see a file- or block-based storage system, we now increasingly see an object storage system, especially when talking about backups or any long-term data archives.
Additionally, its ease of integration with modern web and mobile applications is creating an environment where IT professionals from a wide variety of industries are seeing the benefits of object storage. Not to mention, it is quite cost effective and can even be used as the foundation for a very affordable alternative to traditional Content Delivery Networks (CDN). Object storage can scale nearly indefinitely and, therefore, it's relatively inexpensive, scalable and self-healing solution for storage of massive amounts of unstructured data and an ideal variation of storage that works for cloud storage.
"Custom metadata allows building rich, self-contained file objects that can be stored away in the object store, enabling building massive unstructured data stores with reduced administration overhead."
Terri McClure, a senior analyst at Enterprise Strategy Group in Milford, Massachusetts
So, How Does Object Storage Work?
What we like about it, is that it transcends hierarchy. To see exactly how different it is from traditional storage systems, let’s move in closer and take a look at the main attributes of object storage.
Objects. Rather than managing blocks or files, a pure object storage system manages objects. Objects are addressed via a unique identifier, similar to how files are addressed via file path in file-based storage systems. Objects are stored in a flat address space, which eliminates the complexity and scalability challenges of the hierarchical file systems used by file-based storage.
The compromise of not having to support in-place updates makes object storage a good fit as distributed storage and for distributed access.
Metadata. Objects consist of metadata, which provides contextual information about the data in the object, and the payload or actual data. This metadata in pure object storage systems can be enriched with any number of custom attributes. To do that with a file-based system (as metadata is limited to file attributes) separate apps are needed (with a database) to handle any additional information related to files. With custom metadata you can store all information related to a file (object) in the object itself.
Object Storage Use Case
For object storage, you don’t have to have a single, or aggregated, namespace governing all the data. Instead, you have a loose federation of individual data elements that control their own fate. Read more on how it performs compared to traditional data storage:
Fixed objects. Pure object storage represents a repository of fixed content; that means objects can be created, deleted and read, but they can't be updated in place. Objects are updated by creating new object versions, instead. As a result, the challenges of locking and multiuser access (flaws of file-based systems) simply don't exist with object storage. Object storage acomplishes redundancy and high availability by storing copies of the same object on multiple nodes. When an object is created, it's created on one node and subsequently copied to one or more additional nodes, depending on the policies in place. The nodes can be within the same data center or geographically dispersed. For traditional storage systems, keeping copied (replicated) files and blocks in-sync across multiple instances is a tremendous challenge.
Protocol support. Traditional block- and file-based protocols work well within the data center where performance is good and latency isn't an issue. But they're not appropriate for geographically distributed access and the cloud where latency is unpredictable. Furthermore, object storage is usually accessed through a REST API over HTTP. Commands sent over HTTP to object storage are simple: put to create an object, get to read an object, delete to purge an object and list to list objects.
Application support and integration. The lack of traditional data storage protocol support and reliance on a REST API requires integration efforts for object storage to become accessible. Besides custom application integration, some commercial applications, especially for backup and archiving, have added object storage integration support.
Cloud features. Have you ever wondered how all those photos on Facebook are stored, songs on Spotify or shared files in Dropbox? Well, cloud storage and web applications are key targets of object storage. This means that features related to a shared access over the Internet come to play. Having a multi-tenant, securely segregated users' data is vital so that the object storage product can be used beyond the enterprise and adopt wider use in the cloud.
"Today, object storage is good for post-process-type data as found in the media, entertainment and health care industries, as well as for archival," said Jeff Lundberg, senior product marketing manager at Hitachi Data Systems (HDS). "But as performance increases and features mature, it can not only support cloud storage but will enable distributed IT environments."
Due to the technologies object storage employs and how they work to create a system/service capable of easily locating and accessing individual objects, the associated repositories of fixed content are nearly tailor made for archiving and backup scenarios. While using an object storage service on its own definitely has its benefits, using it in conjunction with any of the third-party S3 compatible tools makes it simple.
If you like this post, please share it. And if you want to see what Object Storage can do for you, click on the banner below.