Tuesday, November 23, 2010

hey you get on my tiny micro aws cloud ;-)

So, it took me a while to get with this whole cloud thing.  The hype seemed bigger than the reality for HPC at least, so time to dig in and see what's occurring.  Turns out it's pretty sweet for a cheap guy (did I tell you that I have a tiny budget?).  

Time to take up the offer for a t1.micro instance over at the amazon free tier for a whirl.  Micro is 2 cents an hour, less than the 8.5 cent "small" offering.  However they let you run up an instance for a month for no charge, and they even bundle some extra gadgets to make it all work out with a hint of extra awesome.  Our great colleagues on the interwebs helped me make sure that you can also spin up a 10.10 Ubuntu Maverick release in under 10GB of elastic block storage - I did mine in slightly over 5GB - hurray!.

Many, many thanks to Scott Moser with his epic link over here for <15GB Ubuntu 10.10 installs!

My install went totally spiffy with Scott's help.  Top tip set up a setup.sh, or add this to your .bashrc to have your private and cert files set up automatically for ec2 commands:

jcuff@shuttle:~/amazon$ cat setup.sh 

export EC2_PRIVATE_KEY=/home/jcuff/amazon/private.pem
export EC2_CERT=/home/jcuff/amazon/cert.pem

Oh and you need to (if you have not before):

jcuff@shuttle:~/amazon$ apt-get install ec2-api-tools

Then simply follow Scott's awesomeness:

jcuff@shuttle:~/amazon$ ec2-run-instances --region us-east-1 --instance-type t1.micro --key wickedpsyched ami-548c783d

And set up the iid:

jcuff@shuttle:~/amazon$ iid=`ec2-describe-instances | grep INSTANCE | awk '{print $2}'`

We are good to roll with two more bash env variables:

jcuff@shuttle:~/amazon$ zone=$(ec2-describe-instances $iid | awk '-F\t' '$2 == iid { print $12 }' iid=${iid} )

jcuff@shuttle:~/amazon$ host=$(ec2-describe-instances $iid | awk '-F\t' '$2 == iid { print $4 }' iid=${iid} )

Then you can quickly create a volume (here's the 5GB money shot I needed):

jcuff@shuttle:~/amazon$ ec2-create-volume --size 5 --availability-zone ${zone}

And attach it (once you have grabbed the output:

jcuff@shuttle:~/amazon$ ec2-attach-volume --instance ${iid} --device /dev/sdh ${vol}

Scott then plays some magic, I've recreated here to image the new /dev/sdh drive inside the aws image:

ubuntu@ip-10-112-99-219:~$ sudo chown ubuntu:ubuntu /mnt
ubuntu@ip-10-112-99-219:~$ cd /mnt
ubuntu@ip-10-112-99-219:/mnt$ wget http://uec-images.ubuntu.com/releases/10.10/release/ubuntu-10.10-server-uec-i386.tar.gz
ubuntu@ip-10-112-99-219:/mnt$ mkdir src target
ubuntu@ip-10-112-99-219:/mnt$ sudo mount -o loop,ro maverick-server-uec-i386.img /mnt/src/
ubuntu@ip-10-112-99-219:/mnt$ sudo mkfs.ext4 -m 0 -L uec-rootfs /dev/sdh

Then some swift rsync action between the source and the target:

ubuntu@ip-10-112-99-219:/mnt$ sudo rsync -aXHAS /mnt/src/ /mnt/target/

And we are pretty much nearly there.  Then Scott shows us how to register the AMI:

jcuff@shuttle:~/amazon$ rel=maverick ; region=us-east-1 ; arch=i386
jcuff@shuttle:~/amazon$ qurl=http://uec-images.ubuntu.com/query/${rel}/server/released.current.txt
jcuff@shuttle:~/amazon$ aki=$(curl --silent "${qurl}" | awk '-F\t' '$5 == "ebs" && $6 == arch && $7 == region { print $9 }' arch=$arch region=$region )
jcuff@shuttle:~/amazon$ ec2-register --snapshot ${snap} --architecture=i386 --kernel=${aki} --name "my-ubuntu-${rel}" --description "my-ubuntu-${rel}"
IMAGE ami-e750a78e

And remove the old sdh drive:

jcuff@shuttle:~/amazon$ ec2-detach-volume ${vol}
jcuff@shuttle:~/amazon$ ec2-delete-volume ${vol}

Finally run it up again:

jcuff@shuttle:~/amazon$ ec2-run-instances --instance-type t1.micro --key wickedpsyched  ${ami}

And do some needed updates:

ubuntu@ip-10-205-13-84:~$ sudo apt-get update && sudo apt-get dist-upgrade


and done!

Total cost 4 cents on top of all the free stuff!

[any opinions here are all mine, and have absolutely nothing to do with my employer]
(c) 2011 James Cuff