Why I Started Using Amazon S3
I have been tracking Amazon S3 ever since it was announced back in March. S3 is an innovative, developer-oriented Simple Storage Service with extremely low prices - 15 cents per gig stored, 20 cents per gig transferred monthly. Jeremy Zawodny has written a comprehensive piece on how cost effective S3 is compared to maintaining his own backup server. But it wasn’t until Jeremy’s follow-up post regarding tools for utilizing S3 that I became much more interested.
The Possibilities
Amazon created S3 with the developer in mind. That means there is a capable REST-based API for people to use the service regardless of the particular application at hand. That means you can establish fast server to server connections such as this handy article that shows how it is possible to backup your Flickr photos to S3.
However, I was more intrigued with a small Windows/Mac/Linux application called JungleDisk. It lets you connect to your S3 account as a networked drive on your computer (WebDAV). JungleDisk adds a bit of user-friendliness to the essentially UI-less S3. Forget paying Apple’s ridiculous .Mac prices and setup JungleDisk+S3. It’s the perfect solution for backing up important files (it’s so cheap, why not backup everything?) and sharing files amongst computers. Just install JungleDisk on each computer and provide it your Amazon S3 access keys. I’m so impressed with this combination that I would consider ditching my external backup FireWire drive if I had a faster Internet connection (Comcast is a little stingy on the upload).
To date, many people have taken note of S3. Web applications harnessing S3’s affordability and features are starting to appear, such as Filicio.us. Someone has developed a useful PHP class for S3 while others have discovered how to integrate S3 file uploads with AJAX.
What Have We Established Here?
- Amazon S3 is cheap.
- It’s scalable - use as much or as little storage and bandwidth as you want. Don’t worry when your S3-based web application gains 20,000 users overnight.
- There are numerous ways of tapping into S3; be it via some custom python script or JungleDisk’s WebDAV interface. If you have an idea, S3 can mold around how you plan to bring your idea into reality.
- Anyone can use S3 - while it’s got the features for developers, it is just as potent for the casual or unexperienced user.
- Your data is safe. S3 is an Amazon service. Amazon’s been around the proverbial block several times and they know what they’re doing.
- S3 is more reliable than your homebuilt server or cheap web hosting account at some company with a fancy name.
You are a genius! And to think I was about to pickup another external (portable) hardrive. I’ve signed up thanks to this share of information.
thanks for the info, I would like to see plugins developed for wordpress / textpattern that let you tightly integrate this file storage solution into a personal website, like mine
That’s just brilliant. I heard about S3 a while back but didn’t know how it could be used for personal backups. I just set up an account and JungleDisk in about 10 minutes. Thanks for the tip!
No need for name calling Stammy, lol!!! We know you love Media Temple…
Only an idiot would compare A3 with .Mac.
That’s like comparing a Ferrari with a Dodge Sprinter. They’re built to do two different things for people with different needs. You should learn more about .mac before knocking it. It’s NOT for everyone but it’s a great for all its features much like if you have 200 crates of lettuce, you don’t buy a Ferrari.
I’m not comparing A3 with .Mac. IIRC, Audi’s A3 is nothing like Amazon’s S3.
Damn it Paul… stop telling me about things I can spend my money on! ;-)
(Goes and signs up for A3… *Sigh*)
Very nice!
Didn’t know about this service until now! I am going to sign up ASAP, although i already have .mac the speed is killing me. How is S3, speed wise? (Compared to .mac)
nice read Paul. I haven’t gotten into the “online” server movement and use my own hosting account for most of it, not really for backup but more for transfers.
I am intrigued with the concepts of API’s and tapping into Flickr etc or server to server connections.
BTW, theres an extra space in the “this handy article” link. (backing-u%20p-flickr) so it hits a 404
http://www.postneo.com/2006/03/22/backing-up-flickr-photos-with-amazon-s3
and this S3 is far superior to Amazon’s :P
Audi S3 seen at the recent Paris Auto Show
http://oneighturbo.com/?p=475
How is S3, speed wise? Who will answer?
Speedwise? I’ve shoved 2.5 gb to s3 in about 10 minutes or so from colo. That might be throttled on my end, it might not, but at any rate, it’s fast enough for what I’m doing.
Sadly its really, (and I mean REALLY), slow here in the UK. After waiting 4 hours for a 1.8GB zip to upload… and it only geting to 20% I gave up. Shame it was looking promising too.
@Adam - for upload, are you sure it’s not your connection that is slow?
There are at least three VERY easy to use interfaces out there that I know of:
Jungle Disk:
* On one S3 account, create different sub-accounts by using a different bucket name for each.
* Mirror and automatic backup functionality is coming in the next version.
* Easy setup, faily easy usage. REALLY easy once I write up the steps :)
sync2s3:
* Easy setup and usage
filicio:
* Incredibly easy to use
* Online file access (currently causes slightly less secure file transfers)
* Online file sharing can be turned on or off
I don’t think that this service is as cheap as you say for personal use. If you compare S3 with a medium quality 250 GB hard drive like this http://www.newegg.com/Product/Product.asp?Item=N82E16822144701, which is sold for 78$, and do some math you will find that on Amazon’s S3 storing an equivalent amount of data will costs (250×0,15)+(250×0,2)= 87,5 only for filling on the server your 250 GB backup folder and for keeping the data only one month, leaving out the cost of getting something back to your pc(or mac of course ;)).
Obviously it strongly depends on your needs, if you would access your data from anywhere this is a good solution.
P.S. I’m speaking of personal use, I’m referring to your external hard drive consideration.
Paul, (and other readers),
what do you think is the number one easiest way to use S3. i signed up but im having a hard time figuring it out.
Adrian, for the absolute beginner I don’t think you can go wrong with Jungle Disk.
http://jungledisk.com/
While I was initially floored by all of this, my needs exceed the feasability of the service.
I need to backup 150GB of music and 180-200GB of files at least once a week. The cost for transfers alone well exceed the cost of 2 hard drives. and that means i could buy 2 new hard drives every month for that price.
It *is* very cool, though. I’m extremely happy that a company the calibre of Amazon is creating services like this. Thanks for the writeup!
Ah yeah you need a lot then. I keep my music on the iMac and have it sync’d on an external hard drive but for everything else - personal files, website docs, school docs, random files - I keep them on S3.. but they are under 2gigs at the moment.
Thanks for the tip. I have followed your example and now have several computers (both linux and windows) connected to the same S3 account through Jungledisk. Works great!
Jungle disk encryption might be an issue if they disappeared some time in the future. There really is no need for it considering the transfer is secure and the bucket is private. If one was really paranoid about there data you could always place things in an encrypted disc image first and then transfer.
My interest in Amazon s3 peaked when Strongspace recently increased their pricing with the ‘bonus’ of a connector account and low volume web hosting. Things I don’t need. S3 is no where as easy to use unfortunately and I think the investment of your time in making this work in your personal workflow negates the advantages.
Bingo disk might be a better alternative.
I went to a very interesting talk on this subject on Monday with Jeff Barr of Amazon. My only issue with this amazing service, is they don’t have Datacentre’s in the UK.
Does anyone know of similar service in the UK.
Jamie
I’ve been backing up my server data to S3 for over a year now using JS3tream. If your a Linux/Unix user, and prefer to use tar, then this tool is the way to go. http://js3tream.sourceforge.net