NSCEE Research Cloud -- Compute

From NSIwiki

Getting Started Using the UNLV Research Compute Cloud

  • In order to use the UNLV Research cloud, you must be affiliated with UNLV (i.e. faculty, staff or student w/ faculty approval). Then send an email requesting to use the cloud. Please include a description containing the amount of resources (including compute, network, and storage), a brief description of your research and any special requirements that you need. Make sure you include your contact information (especially a valid email address).

    Once your request has been processed, you will receive an attachment containing the credentials by return email. The credentials are contained in a zip file with a name like <namelab>-admin-509.zip, where <namelab> is a name identifying your cloud account. Note: you don't use passwords to manage your cloud resources, you must use the enclosed credentials.

  • The UNLV Research cloud uses the Eucalyptus 3.x version of cloud management software to control your cloud resources. It is a good idea to read the online document located at: http://www.eucalyptus.com. Particularly read the "Euca2ools Standalone Installation" and the "Euca2ools Guide". Euca2ools are the commands that you issue on the command line to control your cloud instances (an instance is the cloud computer system assigned to you to process your data). If you search for eucalyptus information on the web, make sure it is for version 3.x and not an earlier version.
  • The research cloud uses the Linux operating system (either Centos or Ubuntu) for your instances. In order to use your cloud instances effectively, you must be able to perform system administration duties on a Linux system.

    The rest of this document assumes that you are using either a Centos or Ubuntu system. For Microsoft Windows users, we suggest installing a Linux virtual machine (i.e. using VirtualBox or VMWare) to manage you cloud instances. If you use an environment other that Centos or Unbuntu, the amount of support that NSCEE can provide may be limited.

  • Install the cloud management tools. In order to use the cloud, you must install the "euca2ools" package. This package contains the commands that will allow you to create, start, stop, etc. your cloud instances.


    sudo apt-get install euca2ools

    Centos: (as root)

    # Configure the EPEL & Euca2ools package repositories:
    yum install http://downloads.eucalyptus.com/software/eucalyptus/3.4/centos/6/x86_64/epel-release-6.noarch.rpm
    yum install http://downloads.eucalyptus.com/software/euca2ools/3.0/centos/6/x86_64/euca2ools-release-3.0.noarch.rpm
    # Install Euca2ools:
    yum install euca2ools
  • Using the credential file attachement you received in step 1 above):

    # place a copy of the zip file in the CREDENTIALS directory
    # i.e. cp ~/Mail/inbox/<namelab>-admin-x509.zip .
    unzip <namelab>-admin-x509.zip

    This will unpack your cloud credentials so you can use them. The following files will be unpacked:

    cloud-cert.pem, <namelab>.private, euca2-admin-c2ba9090-cert.pem, euca2-admin-c2ba9090-pk.pem, eucarc, iamrc, & jssecacerts.

  • In order to easily use the eucalyptus tools, you must "source" the eucarc file to setup necessary envirnment variables. To do this in bash, add the following lines to ~/.bashrc:
    # add the research cloud environment variables:
    . ~/CREDENTIALS/eucarc
  • To make sure that your credentials are loaded, run a euca2ools command:
    euca-describe-images -a

    The output should end with something like:

    IMAGE	emi-ACD043F7	132511203270/nscee-centos	132511203270	available	private		x86_64 machine				ebs
    BLOCKDEVICEMAPPING	/dev/sda	snap-46213902	15
    IMAGE	eki-03933D29	centos6/vmlinuz-kexec.manifest.xml	508052726197	available	public		x86_64	kernel				instance-store
    IMAGE	eri-19083D17	centos6/initramfs-kexec.manifest.xml	508052726197	available	public		x86_64	ramdisk				instance-store
    IMAGE	emi-E635326E	centos6/ks-centos6-201310281433.img.manifest.xml	508052726197	available	public		x86_64	machine	eki-03933D29	eri-19083D17		instance-store

    If you don't get similar output, make sure that you have sourced the eucarc file.

At this point you have installed the cloud tools and are able to issue commands. You are ready to start using the cloud.

Starting Up An Instance

Before you can start an instance, you must setup two different security items associated with it: a security group and a key-pair. If you don't do both of these steps, you will not be able to reliably connect to your instance over the network.

  • Create a new keypair to use for communicating with your instance:
    euca-create-keypair <namelab> > <namelab>.private
  • Create/Modify a security group to control the network access to your instance. All connections to your instance must have a rule allowing it to use the network. By default, these rules are empty. Normally, you will need at least a rule allowing SSH. You will also need a rule allowing web access if you are running a website on your instance.

    Use the "euca-describe-group <groupname>" command to see what access rules are defined:

    euca-describe-group default
    GROUP	sg-92823DD4   604304557083     default	default group

    If you get output similar to the above with no additional information, you have no rules defined. Without any rules, you will not be able to access your instance. To add a rule (to allow ssh access):

    euca-authorize -P tcp -p 22 -s default
    GROUP	default
    PERMISSION	default	ALLOWS	tcp	22	22	FROM	CIDR

    To add a rule to allow http and https access, use something like:

    euca-authorize -P tcp -p 80 -s default
    euca-authorize -P tcp -p 443 -s default
  • To start a running instance use the euca-run-instances command: (emi-E635326E is the default starting Centos that you can then customize, demo is the keypair to use -- yours will be namelab or whatever the name that you specified in the above euca-create-keypair command above.)
    euca-run-instances emi-E635326E -k demo
    RESERVATION	r-C98344DB	132511203270	default
    INSTANCE	i-3CD53F66	emi-E635326E	pending demo
    0 m1.small	2014-02-04T18:36:27.590Z	CLUSTER01	eki-03933D29	eri-19083D17
    monitoring-disabled		instance-store		

    Wait a short amount of time (a few seconds) for your instance to be created and booted. Then check on its status (i-3CD53F66 was assigned above):

    euca-describe-instances i-3CD53F66
    RESERVATION	r-C98344DB	132511203270	default
    INSTANCE	i-3CD53F66	emi-E635326E	running	demo	0		m1.small	2014-02-04T18:36:27.590Z	CLUSTER01	eki-03933D29	eri-19083D17		monitoring-disabled			instance-store		

    Your instance is should now be running and have a public ip number assigned to it ( in this example). Default accounts on cloud instances do not use password, they use identity (key) files. In the following example, the identity file demo.private is used, you should use something like: $HOME/CREDENTIALS/<namelab>.private. You can now login and start using it:

    ssh -i demo.private ec2-user@
    [ec2-user@ip-172-31-255-50 ~]$ sudo su
    [root@ip-172-31-255-50 ec2-user]# df
    Filesystem           1K-blocks      Used Available Use% Mounted on
    /dev/vda1              1290112    554552    722464  44% /
    tmpfs                   122204         0    122204   0% /dev/shm
    /dev/vda2              3354360     71424   3112544   3% /mnt
    [root@ip-172-31-255-50 ec2-user]# 

    The default instance has a single account, ec2-user, that you use to login using the keypair found in you credential file. Do not change this. You can add additional accounts, but please do not change the ec2-user and root accounts or sshd configurations.

  • Now you can start customizing your instance, a good starting point is to make sure you are up-to-date on patches:
    [root@ip-172-31-255-50 ec2-user]# yum update

Note: The initial OS image loaded in an instance, contains a limited amount of storage. To add more, consult the documentation on selecting instance types and how to create & attach volumes. You are limited to about 150Gb of volume storage per instance. If you need more storage than allowed by the above, contact NSCEE about setting up a remote filesystem for your use.