Amazon S3

Automatic WordPress Backup using Amazon S3

I mentioned the Automatic WordPress Backup plugin briefly in my post about Standing Cloud. I've now installed and configured it, and aside from a few UI issues, it's great.

You'll need an Amazon Web Services / S3 account to get started. Grab your access key and security key from the security credentials part of your account. Download / install the backup plugin (here it is on wordpress.org - search inside WP for "automatic wordpress backup" and it should come up).

You choose what to backup. You really do want to check all the boxes -- your database, all your files, plugins, themes, uploaded content, etc. On the site in which I have it installed, each backup is about 100MB.

Backups are kept for a month and then deleted, and monthly backups are kept for a year. At 100MB / backup, that's about 3GB (plus a bit more for keeping those monthly backups). At Amazon S3's pricing of 15¢ / GB, that's.... under 50¢ / month to have your site backed up every single day.

You can restore directly from within the interface as well, so this is a pretty turn key system. The UI is all in one page and a little clunky, and I did make some comments on what could be improved:

  • In general, I would split the one big page into several tabs. A backups tab, a restore tab, and so on.
  • if you add a progress indicator, or in general display a floating message that says “backup in progress”, then people will understand that a backup is happening.
  • I would also do an immediate check against the access key / secret key and put in a green checkmark to indicate that it has connected correctly – with the secret key hidden, I’m always unsure if I’ve cut / pasted it correctly.

Of course, what about VaultPress, the commercial service? Well, in beta, they're charging $15 / month per blog at the Basic level, $40 / month for Premium. Or, um, 30x - 80x more expensive than doing it yourself with this plugin and Amazon S3. Their FAQ says that they are also doing security monitoring -- aka figuring out when your blog is hacked. And yes, most people are going to have to spend some time figuring out the Amazon interface and so on. But I really can't see how they can keep pricing at this level with this kind of free plugin available.

Private code repo hosting with Beanstalk and RepositoryHosting.com

For many years, I've settled on Unfuddle as my hosted tool of choice for development-focused project management and version control repo hosting. It's a great tool if you want all in one development ticketing / bug tracking (which is what I mean when I say "dev focused" PM) plus your code repository all in one place.

However, I've been moving away from using Unfuddle for this type of project management, or been called into other projects where an existing PM tool is already in place -- most notable, either Basecamp or Open Atrium, with the Drupal-based Atrium being something I'm very keen on supporting (interested in seeing more hosted tools integrated with Open Atrium? contact me).

In cases like this, I really ONLY need a (private) hosted code repository*. I saw an article recently comparing various options for private DVCS hosting (DVCS = distributed version control systems like Git or Mercurial) which nicely complimented some of the research/experimentation that I've been doing.

The review states its goals very clearly: lowest cost-per-repository, with storage used as a secondary consideration. Everyone will have their own needs to rank providers across.

For me, I'm interested in a mix between features and pricing, especially using it on a consulting basis with lots of clients, and different kinds of clients (from those that don't use version control at all, to those that have in house developers already).

RepositoryHosting.com is clearly the lowest price-per-repository: it's $6 / month for unlimited repos with a bundled 2GB of storage, plus $1 per additional GB of storage. No "cost per project" means I can generate a project per client / idea / whatever and not have to worry about it. There is bundled ticketing in the form of Trac, and it's likely OK / has evolved since I got sick of it when we used it at Bryght -- I'm not giving points in this review for PM tools in any case. The WebDAV shares is a very cool feature, especially for integrating clients, designers, and other folks into the mix that will find it hard to adopt the version control system directly.

On the features side of things, my pick goes to Beanstalk. It starts with a free plan so you can check it out right away with 1 private repo. The first paid plan is $15 per month, and I'm fine with their 10 repository limit, but I'm always annoyed at limited user counts. Regardless, it is a super clean interface that people will find very easy to use and it integrates with a ton of different tools (Twitter, Basecamp, Harvest, etc.). But the main reason it is a choice that I'm going to be using more and more is that it offers direct FTP / SFTP deployment. I think this makes it a perfect tool for design-centric shops -- they can continue to use FTP-based hosting services and workflows, while starting to adopt best practices version control.

Bottom line: Unfuddle is a great all in one solution and one that I recommend to clients over and over again if they have no PM or development management tools in place, but for just repo hosting, you should consider RepositoryHosting.com and Beanstalk.

Web apps should let me "Bring my own storage"

I've written up Dropbox vs. Jungle Disk before, and then followed up with the concept of side loading - having software assets (purchased or otherwise) be moved from one online location to a cloud storage location owned by you.

The guys at Jungle Disk are looking to add features to their service, asking people to vote between public/private file sharing, multi-computer sync, and network drive offline access. Since this is exactly what I use Dropbox for (I use Jungle Disk for backup and long term archive), I really would love to see them add all three.

Dropbox still only has two downsides for me. First, it's reselling Amazon S3 storage to me at a premium, and I resent that. Secondly, because of it's sync model, I have to have enough room for all my files on every computer I enable it. I'm storing all my files in the cloud so I don't *have* to have unlimited storage. Like ZumoDrive does, there should be some smart caching to keep some stuff local, and make all of it available when needed.

I can't really blame Dropbox - they're doing the same thing countless other web app companies are doing. Amazon S3 is one of the biggest, cheapest, and most reliable options out there. Joyent's BingoDisk, Rackspace's CloudFS, and Nirvanix are three examples of somewhat equivalent offerings with slightly differing price points, so let's assume those are included in the concept of reselling cloud storage. But here's the thing I realized, and posted on Twitter the other day:

Lots of online apps differentiate pricing based on storage. I'd much rather they didn't offer storage, and let me plugin S3 / Dropbox / etc.

 

Dropbox vs. JungleDisk

I wrote a bit about Jungle Disk in passing. I am using it for personal archive and backup. It's been working great, and I decided to try out the Workgroup edition: you add additional accounts and can set permissions on different buckets / folders for each person / account. At $2 / account / month for the workgroup functionality, it's quite good.

Except, you have to get people to install and setup Jungle Disk (the download link for Workgroup is a bit hidden). And ... it's not Dropbox. I tried it for a bit, and it works as advertised, but you a) have to keep paying on a monthly basis and b) you have to do a fair bit of handholding and account management.

Then I tried Dropbox today. Easy. Amazing. Amazingly easy. And it does shared files, too. Share a folder, add some email addresses to invite people, and you've got synced folders / documents on multiple computers. The public stuff is actually easier ... there is a default folder called Public, and files in there you can right click on and get a publicly accessible link directly to.

Update: CAUTION! -- I didn't realize this, but according to Michael Tsai, Dropbox doesn't support resource forks on Mac OS X -- "If you use Dropbox, resource forks disappear, packages turn into folders and can no longer be double-clicked, etc. ". What this means is that some files will have issues. Basic files like Word docs and binaries shouldn't run into issues, but for applications, potentially Keynote files and others, your files may not work correctly any more.

Currently, there is a 2GB storage limit to the accounts (free). This also sits on Amazon S3, although on their account, not yours like Jungle Disk. Dropbox is offering a paid upgrade to 50GB of space for $9.99 / month, or $99 / year. Hmmm....2GB still seems enough for now...

I'll stick with the Jungle Disk Desktop edition for my backups and long term archives. I've paid the $20 for the Desktop edition and I can backup and store as much as I want on my own Amazon S3 account.

For multi user sharing of documents, Dropbox is just so much simpler. The low end pricing is cheaper than Jungle Disk (free!) while the high end of 50GB is cheaper with Jungle Disk (0.15/GB/month with S3 x 50GB = $7.50).

I think we're going to continue to see great innovation in better ways to share / sync / collaborate on files, in part driven by cheap, reliable, API-driven storage options like S3. Epd.io is a local Vancouver startup to keep an eye on...

Omnibus: BBQ, Jungle Disk, WordPress etc.

Well, contrary to the lack of posting here, I actually have a ton of stuff tumbling around my head right now. And so, an Omnibus post that covers a couple of different items.

I've been heads down busy and haven't been attending (or organizing!) any social media type events lately. I did get out Wednesday night to attend the Freshbooks / Redwerks BBQ. Look, there's me holding a puppy (photo by Ianiv)! It was a beautiful sunny evening and the Redwerks rooftop patio is awesome. I ended up manning the grill, my secret ploy to meet everyone (at least, everyone that was hungry). It was nice to meet some new people and catch up with a bunch of regulars.

I'm trying Jungle Disk for my personal backup. In short, it's a cross platform app that both serves as a kind of iDisk as well as some simple backup operations, except that your data is actually stored on Amazon's S3 service. You pay a one time license for the application (and you can install it on as many computers as you want), and you pay as you go for storage. And can get your files from any machine.

I'm currently backing up my Documents folder to a Backup area, and then I also have a second "bucket" (that's actually Amazon tech talk, but it makes sense) that is a true archive -- I copy old stuff there and delete if off my local disk. I'm still debating whether it's worth it for me to put my entire iTunes collection online -- it would solve being able to get my music from anywhere, and it would cost about $12 / month (for 60GB). Not sure what the calculation is for streaming that music some of the time? And yes, this is like MP3 Tunes music locker.

So that's my use, but Jungle Disk *also* launched the WorkGroup edition -- which is the same thing, but lets multiple users in a company use it from a single Amazon account, with things like their own storage space as well as granular user permissions. So you can have a Finance folder that only senior management can access. And if you don't have senior management, then just think about how great it would be to have a small business shared file system that you can access from any computer, anywhere. That's $2/month per employee, which I think is a good price.

WordPress! I've been mucking about in WordPress core and theme code. Once was with Rachael's site, which I upgraded using the FTP dance. I really hate not having command line access.... The second was for the Bootup Labs Blog, which I moved off of WordPress.com so we could add some more plugins and do stuff like have a feed for every category / tag. Except, when I went digging around, it seems that the main feed is the only one that is ever injected into the link rel header. So, here's my feature request if you're interested in the gory details: http://trac.wordpress.org/ticket/7190 -- heck, I might even submit a patch :P