Tuesday, October 18, 2011

paranoid about your cloud provider? why not raid1 mirror two cloud services?

Usual disclaimer, this is nuts, don't try this at home, your mileage will vary, things will no doubt fail #exercisetoreader etc. ;-)

Summary: A RAID mirror of box.net and dropbox.com - just for fun!

Following on from yesterday's post, I was sitting on the redline this morning thinking about using a RAID mirror to take two discrete cloud storage providers to provide an n+1 file system. Turns out that you can! This is just a proof of concept, no more no less, no one was injured in the making of this motion picture!

So here we go, first follow the fuse system commands from yesterday to set up your box.net account. Then grab and set up dropbox.com like this:

jcuff@shuttle:~$ wget https://www.dropbox.com/download?dl=packages/nautilus-dropbox_0.6.9_i386.deb
jcuff@shuttle:~$ sudo dpkg -i ./download\?dl\=packages%2Fnautilus-dropbox_0.6.9_i386.deb 
jcuff@shuttle:~$ dropbox status
Dropbox isn't running!
jcuff@shuttle:~$ dropbox start -i
Starting Dropbox...

and set up a file, just like we did yesterday:
jcuff@shuttle:~/Dropbox$ dd if=/dev/zero of=test.dat bs=1024k count=10
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.0464433 s, 226 MB/s

Now you can simply set up your loop back interfaces, remember to use the fuse interface we described yesterday for box.net, webdav (davfs) will not mount the loopback and will fail with an I/O error, but this one passes muster. Basically you are setting up two block devices on the remote cloud offerings.
jcuff@shuttle:~/Dropbox$ sudo losetup /dev/loop3 ~/Dropbox/test.dat
jcuff@shuttle:~/Dropbox$ sudo losetup /dev/loop4 /tmp/boxed/test.dat

jcuff@shuttle:~$ sudo losetup -a
/dev/loop3: [0900]:18352720 (/home/jcuff/Dropbox/test.dat)
/dev/loop4: [0017]:6 (/tmp/boxed/test.dat)

Ok now let us assemble the RAID1 and mount it:
root@shuttle:/home/jcuff# mdadm --create /dev/md2 --level=1 --raid-devices=2 --auto=part /dev/loop[34]
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md2 started.

root@shuttle:/home/jcuff# cat /proc/mdstat 
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
md2 : active raid1 loop4[1] loop3[0]
      10228 blocks super 1.2 [2/2] [UU]

root@shuttle:/home/jcuff# mkfs.ext4 /dev/md2
mke2fs 1.41.14 (22-Dec-2010)

root@shuttle:/home/jcuff# mount /dev/md2/ /mnt/cloudraid/

root@shuttle:/home/jcuff# df -H /mnt/cloudraid/
Filesystem             Size   Used  Avail Use% Mounted on
/dev/md2                11M   1.2M   8.5M  12% /mnt/cloudraid

Does it work?
root@shuttle:/home/jcuff# echo hellojames > /mnt/cloudraid/hellothere

root@shuttle:/home/jcuff# strings /dev/md2 
lost+found
hellothere
lost+found
hellothere
hellojames
Again, please don't try this at home ;-)



[any opinions here are all mine, and have absolutely nothing to do with my employer]
(c) 2011 James Cuff