Wednesday, January 12, 2011

clouding a dataverse on an aws t1.micro

My Research Computing team have recently been collaborating with Prof. Gary King and his researchers at IQSS to stand up a dataverse on our large KVM/GFS 72 core 300TB cluster. Team RC did a great job getting dataverse up and running. Non trivial installation, there are awesome installation instructions over at thedata.org/book/installers-guide so installation was easy and fast.

I decided it might also be fun to push this "to the cloud!" :-)


A couple of tiny EC2 fixes were needed to get this to work on the t1.micro. First make sure you have the following CentOS 5.5 AMI booted:
root@shuttle:/home/jcuff/amazon# ec2-describe-instances | grep INSTANCE | awk '{print $3}'
ami-92dd2afb

Other than a few yum installs:
yum install R-DVN-2.9.0-4.x86_64.rpm
yum install postgresql
yum install postgresql-server
yum install java-1.6.0-openjdk.x86_64
yum install java-1.6.0-openjdk-devel.x86_64
yum install unzip
yum install lsof
yum install subversion

and a few wgets:
wget http://dvn.iq.harvard.edu/dist/R/DVN-2.0/R-DVN-2.9.0-4.x86_64.rpm
wget http://download.java.net/javaee5/v2.1.1_branch/promoted/Linux/glassfish-installer-v2.1.1-b31g-linux-ml.jar
wget http://sourceforge.net/projects/dvn/files/dvn/2.1.2/
DVN-EAR_v2_1_2.ear/download
wget http://sourceforge.net/projects/dvn/files/dvn/2.1.2/
referenceData_v2_1_2.sql/download
wget http://sourceforge.net/projects/dvn/files/dvn/2.1.2/
README_v2_1_2.txt/download
wget http://sourceforge.net/projects/dvn/files/dvn/2.1.2/
config_v2_1_2.zip/download
wget http://sourceforge.net/projects/dvn/files/dvn/2.1.2/
buildupdate_v2_1_1_v2_1_2.sql/download

you are basically good to go. You will need to do the swap trick I've mentioned before http://blog.jcuff.net/2010/12/me-waves-clouds.html, if you are on the free tier m1.tiny or you will see this when importing the EAR:
java invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0

Call Trace:
[] out_of_memory+0x6d/0x1c4
[] __alloc_pages+0x23e/0x2c7
[] __do_page_cache_readahead+0xf4/0x258
[] __wait_on_bit_lock+0x5b/0x66
[] __lock_page+0x5e/0x64
[] filemap_nopage+0x140/0x332
[] __handle_mm_fault+0x442/0x114a
[] do_page_fault+0xf23/0x128d
[] error_exit+0x0/0x79

Other than that, it's all as per the documentation. You can then access your new "verse" via a simple ssh tunnel to your instance:
isrv:~ jcuff$ ssh -L8080:localhost:8080 \
-L4848:localhost:4848 \
-L8181:localhost:8181 \
-L8888:localhost:80 root@aws



QED!




[any opinions here are all mine, and have absolutely nothing to do with my employer]
(c) 2011 James Cuff