One problem holding back general purpose cloud storage has been the lack of a “standard” way to get data in and out of the cloud. Most cloud storage providers supply a REST interface, an object file interface or other proprietary ways to use their facilities. The problem with this is that they all require some form of code development on the part of the cloud storage customer in order to make use of these interfaces.
It would be much easier if cloud storage could just talk iSCSI, FCoE, FC, NFS, CIFS, FTP, etc. access protocols. Then any data center could use the cloud with a NIC/HBA/CNA and just configure the cloud storage as a bunch of LUNs or file systems/mount points/shares. Probably FCoE or FC might be difficult to use due to timeouts or other QoS (quality of service) issues but iSCSI and the file level protocols should be able to support cloud storage access without such concerns.
So which cloud storage support these protocols today? Nirvanix supports CloudNAS used to access their facilities via NFS, CIFS and FTP, ParaScale supports NFS and FTP, while Amazon S3 and Rackspace CloudFiles do not seem to support any of these interfaces. There are probably other general purpose cloud storage providers I am missing here but these will suffice for now. Wouldn’t it be better if some independent vendor supplied one way to talk to all of these storage environments.
How can gateways help?
For one example, Nasuni recently emerged from stealth mode, releasing a beta version of a cloud storage gateway that supports file access to a number of providers. Currently, Nasuni supports CIFS file protocol as a front end for Amazon S3, IronMountain ASP, Nirvanix, and coming soon Rackspace CloudFile.
However, Nasuni is more than just a file protocol converter for cloud storage. It also supplies a data cache, file snapshot services, data compression/encryption, and other cloud storage management tools. Specifically,
- Cloud data cache – their gateway maintains a disk cache of frequently accessed data that can be accessed directly without having to go out to the cloud storage. File data is chunked by the gateway and flushed out of cache to the backend provider. How such a disk cache is maintained coherently across multiple gateway nodes was not discussed.
- File snapshot services – their gateway supports a point-in-time copy of file date used for backup and other purposes. The snapshot is created on a time schedule and provides an incremental backup of cloud file data. Presumably these snapshot chunks are also stored in the cloud.
- Data compression/encryption services – their gateway compresses file chunks and then encrypts it before sending them to the cloud. Encryption keys can optionally be maintained by the customer or be automatically maintained by the gateway
- Cloud storage management services – the gateway configures the cloud storage services needed to define volumes, monitors cloud and network performance and provides a single bill for all cloud storage used by the customer.
By chunking the files and caching them, data read from the cloud should be accessible much faster than normal cloud file access. Also by providing a form of snapshot, cloud data should be easier to backup and subsequently restore. Although Nasuni’s website didn’t provide much information on the snapshot service, such capabilities have been around for a long time and found very useful in other storage systems.
Nasuni is provided as a software only solution. Once installed and activated on your server hardware, it’s billed for as a service and ultimately is charged for on top of any cloud storage you use. You sign up for supported cloud storage providers through Nasuni’s service portal.
How well all this works is open for discussion. We have discussed caching appliances before both from EMC and others. Two issues have emerged from our discussions, how well caching coherence is maintained across nodes is non-trivial and the economics of a caching appliance are subject to some debate. However, cloud gateways are more than just caching appliances and as a way of advancing cloud storage adoption, such gateways can only help.
Full disclosure: I currently do no business with Nasuni.
Ray — Full disclosure: I work at Nasuni.
This is a great overview of the potential for these gateways that go beyond being just a protocol translation layers from CIFS/NFS to HTTP. I think that we are going to see both file and block level approached to the problem. Our approach at Nasuni is files. The biggest benefit is being able to roll back changes for a directory or file without having to rely on a separate backup application.
For more information on how our snapshots work checkout https://www.nasuni.com/product/technology/#how-na…
Zetta.net is an On-Demand Storage Service that talks CIFS, NFS, webDAV, sFTP – etc….and no special API required to write to Zetta. Zetta's storage service provides a full file system in the cloud and snapshots – all standard features embedded in the Zetta in a simple cost of $0.25GB per month (1TB minimum). Replication is also available in June 2010.
Ray, great article. I work at SMEStorage.com where we provide a multi-tenant hosted SaaS Cloud Gateway. We enable the aggregation of over 10 Storage Clouds and layer on top what we term as a Data Access Platform (DAP). This platform provides Data Access Clients for Windows, Mac, Linux, iPhone, iPad, BlackBerry and Android. We also make the Gateway platform available as an in-house appliance and the clients available with source for corporate clients. Our API is open, and you can use the API to do CRUD functions on many different clouds from a single API as well as providing higher level services such as email backup and sync.