Amazon S3 – Very Cheap Storage in the Sky

When I was building the Vermont Telecom Authority’s broadbandwiki web page, I had a problem: where do I store the information that users provide? Obviously, the data has to be somewhere that is accessible from anywhere on the web; it has to be reasonably secure against accidental or malicious destruction; and there has to be enough space and Internet connectivity to that space available for an unknown amount of use. The answer turned out be a Amazon’s Simple Storage Service (S3).

Because Amazon is sharing its buying power with you, the storage is very, very cheap. My bill for the couple of weeks I was testing on it (granted, small volumes) was $.04 – and I suspect there is some rounding up to the nearest penny in that.

With a little more packaging by Amazon and a lot more promotion, there could be a whole wave of Internet innovation enabled by the availability of S3. Note to developers: I did not use any application server; all the communication is directly between the web pages of the application and S3. There’s no PHP; no LAMP stack, no use of Amazon EC2. The web pages themselves are hosted on S3. More in a subsequent howto post.

When you use S3 your data is stored on Amazon’s servers. In fact it’s replicated (copied) to a number of different Amazon servers to make it both more secure and more accessible. You pay for the amount of data you actually store and the amount of access there is to that data. You DO NOT reserve any amount of storage or access in advance so your costs are not affected by the fact that you don’t know in advance the quantity of resources you’re going to need or what your usage is going to be. If you should be so lucky as to have a huge spike of interest, it’s unlikely that Amazon’s servers will notice the difference.

One alternative to using Amazon would have been to buy a server, put it under my desk, and make it accessible through my Internet access. That would have been at least a thousand dollars up front, more as more capacity is needed, and would not have provided either redundancy or reliability. Moreover, I would have had to do some additional programming to get my server to talk to the browser application.

More realistic would have been to have my server hosted at any one of a number of hosting services or to rent access to a server at a hosting service. Still, I would have had to make a minimum volume commitment and would be restricted to using the amount of capacity – both storage and bandwidth – that I reserved. Relatively easy to set up (I hear) but almost certainly more expensive and more limited than Amazon. This is, however, the way most new Web applications are launched.

LocalReplay.com, a company in which I’ve invested, has its very professional and very large data store including video clips hosted on S3. They’ve been as happy with it as I am. They’ve experience less outages than any other startup service I’ve been involved with. Moreover, when traffic is low between sports seasons, their costs go down. When a tourney gets huge attention, costs are up some but the capacity is there.

In the future, I’ll want to know why any web application company I advise or invest in DOESN’T use S3 (or a competitor should they emerge). If providing storage is NOT a key talent of a company and a key differentiator, it’s hard to see why it would want to compete with Amazon. No Amazon brand at all needs to show through, just quality and price. There’s even a mechanism for having Amazon bill your customers on your behalf if that’s appropriate.

Even companies which are not in the web business but use the web as part of their business should think about S3 as an alternative to both self-hosting and traditional hosting services. Here are the rates for storage including access in the US (Europe slightly higher).

Storage
$0.15 per GB-Month of storage used

Data Transfer
$0.10 per GB - all data transfer in

$0.18 per GB - first 10 TB / month data transfer out
$0.16 per GB - next 40 TB / month data transfer out
$0.13 per GB - data transfer out / month over 50 TB

Requests
$0.01 per 1,000 PUT or LIST requests
$0.01 per 10,000 GET and all other requests*
* No charge for delete requests