27
Jan
10

Cloudified

With the recent advancement of HD content, storage requirements are becoming aggressive and require higher networking bandwidth than ever before.  Disks are getting larger and cheaper but most modern consumer computer devices don’t have enough internal storage to handle all of our requirements when it comes to photos, music, video, documents, etc.; and, to make matters worse, on a daily basis I use three to four different computers, all of which require backing up in case anything should go wrong.

Right now reliable, safe, long-term data storage means having multiple, redundant disks on the network – which can be a considerable cost.  Our home requires somewhere in the vicinity of 2-4TB of storage to handle everything comfortably, with some room for growth.  Whilst this is not a lot of storage in today’s terms, any future-proof device would still be quite large, noisy, hot and have a low spouse acceptance factor.

So, what’s the answer?

Cloud computing and storage has always appealed to me and have reached the point where they’re just as good, if not better, than local alternatives.

The home network will always require some local storage.  Home broadband hasn’t yet reached the level where everything can be streamed from the Internet as an on-demand service – not in the UK at any rate (and certainly not in the US or Australia).  So, a local 1-2TB, low-power NAS for transient video data is still required and, because of the nature of the content, backups aren’t required.

Music is roughly in the same camp.  Since most of the music is synchronised with a portable music player, in the event of an disaster, a simple re-sync should be sufficient.

Photos are a tricky call and one that I’ve had to take quite a firm stance on.  Typically, when taking a bunch of photo only a few are ever really any good and worth showing to people.  What do we do with the rest of them?  They usually languish in a folder somewhere getting pushed from computer to computer, medium to medium without ever being accessed again.  So, I say, get rid of ‘em.  The rest should be put into the cloud for sharing with the rest of humanity.  Flickr, PhotoBucket, Picasa… take your pick.  Let them worry about the storage and bandwidth.

Documents are fairly easy.  There are plenty of services out there that will do simple, cloud-based file storage sitting on top of Amazon’s S3 or Rackspace.  Again, take your pick and get them off your local disk.

If you take this approach it becomes fairly obvious that your local PC, laptop or tablet (hello, iPad!) becomes a commodity device as it simply consists of an OS and a bunch of applications that allow you to access the cloud and your content.  Nothing of consequence is ever stored locally.  It also means that you become free and untethered to a particular machine or environment.

So, what happens if the cloud disappears?

First of all, all of The Cloud would have to disappear at once, which is unlikely.  Secondly, you can always backup your data.  There are services that offer backups of your cloud-based content.  Backupify are an example of such an organisation.  Their restore and export procedures are not yet complete but their experts will help you out should the worst happen.

As a geek I also tend to have a slightly above-average set of requirements when it comes to the Internet; all of the above as well as running a blog, a micro-blog, having several domains, source code repositories, etc. all of which have to be hosted somewhere.  Previously the sensible option was to host these myself.   However, recently I’ve come to the conclusion that out-of-the-box software (such as WordPress & Google Mail) do a pretty good job as they come and no longer require me to install, configure, upgrade and maintain software — leaving me with more time to do other things.

Not only have I save myself some effort but I’ve also saved quite a bit of money, as a majority of the cost before was running a server.  The cloud now provides a plethora of services for free (usually subsidised by advertising) that have relatively low-cost upgrade paths also providing scalability, should you need it.

The following table is a simple side-by-side cost comparison for the content that I was hosting myself previously versus the cost, today, of utilising cloud technologies:

Function Hosted solution Yearly cost Cloud Solution Yearly cost
Server 1and1.co.uk £720 N/A £0
Email Exim, Dovecot, Horde & Imp £0 Google Apps Mail £0
Calendar Horde & Kronolith £0 Google Apps Calendar £0
Contacts Horde & Turba £0 Google Apps Contacts £0
Docs and spreadsheets Microsoft Office £160
(£320 over 2 years)
Google Apps Docs £0
Blog WordPress software £0 WordPress hosting £7 domain mapping
£16 custom CSS
Online filesystem SSH FS £0 Dropbox £0
Photos Gallery 2 £0 Flickr Pro £16
Encryption SSL certificates £35 (included) £0
Backups Custom script £0 Backupify £0
Total £915 £39

Not bad, eh?

I still, however, have some concerns about turning all my data over to the cloud.  Will these companies disappear one day, without a trace – taking my data with them?  Possibly.  Will the data be backed up somewhere?  Maybe. Will I be able to access it?  Probably not.

These questions are unanswerable right now and only time will tell – but I really can’t think of a better way of doing it, today, without having a vast, “expensive” storage array at home.  Hopefully, someday soon, there will be a breakthrough in storage technology and I can store my ever-growing digital life safely and locally, somewhere that I trust.  But, if that does happen, you can be sure that content generators will find a way to use it — resulting in an arms race again, pushing the limits of storage technology ever forward.

I do, however, feel lighter for having made the move.

03
Jan
10

Built-in Wi-Fi scanner in Mac OS X

After having another wireless network stomp all over mine, it was time to look for a less crowded channel. With Snow Leopard a lot of the third-party scanner tools don’t seem to work. Apparently, Apple modified the Airport Extreme drivers and removed the API.

However, for a simple scan, the one built into Mac OS X is sufficient.

/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport

To perform a scan, use the ‘-s’ option.

A GUI wrapper can be found here.

07
Dec
09

Quote

The world is my country, science my religion.
Christiaan Huygens

06
Dec
09

Bulk uploading email into Gmail

Recently I joined the hoards of people who use Gmail as their primary mail service.  It has all the bells and whistles and provides virtually unlimited space for storing email.  I won’t go into all the cool features but let’s just say, it rocks.

The question is, how do we get all our old email into our shiny new Gmail account?  The answer is fairly straight forward but my method requires a bit of know-how and a *NIX mail client.

First, the overview: Basically I used Alpine (the mail client formally known as ‘Pine’) and Gmail’s IMAP abilities to bulk save the messages from the local folders to Gmail.

Primarily I was concerned with my ‘Sent’ items from the past decade but this will work with any folder(s):

  1. Enable IMAP in Gmail.
  2. Install Alpine on your *NIX workstation or server.
  3. Configure Alpine to see to your existing email folders.
  4. Configure Alpine to talk via IMAP to Gmail:
    • Inbox: {imap.gmail.com/ssl/user=username@gmail.com}inbox
    • Collection List:
      • Server: imap.gmail.com/ssl/user=username@gmail.com
  5. Go to the folder you wish to bulk upload to Gmail.
  6. Select all messages (shortcut keys: ‘;’ followed by ‘A’).
  7. Save all messages (shortcut keys: ‘A’ followed by ‘S’).
  8. Select the destination folder (shortcut keys: CTRL-T)
  9. Choose your Gmail collection and then the destination folder.
  10. Hit Enter to bulk save the messages into Gmail.
  11. Rinse and repeat for any other folders.

Using this method, I was able to bulk upload approx. 10,000 messages from the past decade of my Sent messages in approx. 1 hour (this will depend on your bandwidth, of course!).

04
Dec
09

Enabling SNMP in Mac OS X 10.6 (Snow Leopard)

Under Snow Leopard there is a slight change to the way services are enabled.

-w       Overrides the Disabled key and sets it to false. In previous versions, this
         option would modify the configuration file. Now the state of the Disabled key
         is stored elsewhere on-disk.

So, to enable the SNMP daemon correctly:

$ sudo launchctl load -w /System/Library/LaunchDaemons/org.net-snmp.snmpd.plist



Twitter

Bookmarks