Specification for the storage and cataloging of STAR virtual machine images

 

About this document:

This documentation describes where and how STAR virtual hard disk images are stored. Through standardization we can allow images to be:

• reusable by others
• cataloged, so the total available inventory if images is known
• locatable quickly
• somewhat self describing

 

What is a virtual machine image:

An image is a virtual hard disk, basically a large file on your "real" hard disk, whose contents are presented to the virtual machines (VM) as if it were a complete hard disk. STAR has a repository of virtual machine images which have operating systems and the STAR software stack pre-installed. Businesses now provide large computing facilities with many nodes running virtual machines that can be rented by the hour. One can upload a virtual machine image and essentially rubber stamp as many nodes of a particular configuration as desired. So they are not exactly identical, for example you will be wanting different names for each node (rcas1,rcas2,rcas3,...), a special step called contextualization is used to slightly customize each image.

A common type of image format many people are familiar with is the .ISO file. For example one can dump a block device like a hard disk or optical drive to one of these files. (could be a little irrelevant)

Example:

dd if=/dev/cdrom of=/home/bobfox/myCDimage.iso 

One can then mount the file as if it where any other hard disk. In most newer desktop Linux distributions in the default Gnome desktop one can just click on an iso file and the mounted drive icon will appear on your desktop. Most virtual machines (VMware, VirtualBox) support mounting of ISO files as if they where CD-ROMS attached to a real system.

What does STAR want to do with this technology:

The STAR software stack is not yet as easy to install as “just keep clicking next”. We can make most of these complexities go away by providing virtual machine images to tier-2(+) STAR sites with the software stack pre-installed and ready to run out of the box.

STAR's dreams of running on the grid have languished for a long time because of the non-homogeneous nature of GRID host sites. STAR can not recompile and certify a customized version of its software stack for each site even if the resources are free and available. However most of the differences between sites offering computing resource can be leveled out if they all run virtual machines. This means STAR can guaranty that its software will run on different sites and that the output file produced are as valid as if they had been run on the local BNL Tear 1 site via the usual stringent quality assurance processes STAR employs.

It all sounds too good to be true. That is because it is. There are many different virtual machine softwares all with different formats of virtual machine images.

Some common virtual machine softwares are:

• VirtualBox
• VMware
• KVM
• XEN

 

Virtual machine images:

These different virtual machine softwares require different image formats. Some can be easily converted, however most can not. For example Xen images don't contain any files in it /boot partition because it uses its own built in kernel. On the other hand VirtualBox uses the original operating system kernel. So converting from XEN to VirtualBox images is not really possible. Many packages will offer to convert the file system structure, but that doesn't mean the image will be able to boot.

The type of image you need is not up to you. It is determined by the virtual machine software of the host site. For example a XEN virtual machine will require a XEN image. Support for image formats other then the native format of the VM is very limited and in most cases non-existent and or unreliable.

Key parameters of images are:

• Virtual Machine Format
• Storage space

 

Why is storage space on the list

When you create an image file, its size needs to be specified, which represents a fixed geometry of the virtual disk. It is not possible to change the size of the virtual hard disk later. If you have a fixed-size image of e.g. 10 GB, an image file of roughly the same size will be created on your host system. There are also dynamically expanding images these will initially be small and not occupy much space for unused virtual disk sectors, but the image file will grow every time a disk sector is written to for the first time to some max size which can not be made bigger once the full size is reached.

 

When you run your job at some remote site you will need to write the data somewhere. There are mapped network drives and such, but the most convenient space to write is in the image it's self. So the total data of the node can not be bigger then the max size of the image. We can express it like this:

 

( [OS] + [swap space] + [STAR software] + [your data] ) < ( [Image max size] ) 

 

Now that we have gotten the parameters out of the way that are specific to virtual machines there are some parameters specific to the software installed on the image.

 

Key parameters of the software installed on the image are:

• STAR libraries installed
• Operating System
• Kernel Version
• Instruction set architecture
• Instruction set architecture word size (32bit / 64bit)

 

One image may hold many different library versions, all of the other parameters may only have one value at a time.  The data08 volume is devoted to grid work. It is the location of STARs image repository.

1. Note: Even though more then one star software library version may reside in an image this is hard to do because of the size of the image. So we are assuming only one library per image.
2. Note:In the case that an image is more then one file an additional directory will be needed, the directory will take the file naming convention.

The path to the repository and scheme used to derive the name of the image is below:

 

/star/data08/OSG/APP/vm/[VM]/[Operating System]_[Instruction set architecture]_[Instruction set architecture word size]_[STAR lib version]_[maxsize]_[addition detail].[extension]

Place Holder	Definitions
[VM]	The name of the VM software (xen, kvm, virtualBox, ....)
[Operating System]	The Operating System installed (sl4, sl5.3, ubuntu9.10, fedora12, centOS5, ...)
[Instruction set architecture]	x86, SPARC, ARM, Alpha, PowerPC, AVR
[Instruction set architecture word size]	8, 16, 32, 64, 128, 512
[STAR lib version]	The STAR library version (example: sl05a)
[maxsize]	The maxsize to which the image will grow (example: 2GB)
[addition detail]	Any additional detail we may want to add
[extension]	The file extension

 

 

Examples:

/star/data08/OSG/APP/vm/xen/sl5.3_x86_32_sl05c_8GB_ec2.img
/star/data08/OSG/APP/vm/virtualBox/ubuntu9.10_x86_32_sl05c_10GB_ec2.img

In addition there will be a text file with the .checksum extension holding an MD5 checksum hash of the image.

Example of making the hash:

[rcas6016] xen/> ls
sl4_x86_32_sl08e_8GB_ec2.img
sl4_x86_32_sl08e_8GB_ec2.txt
[rcas6016] xen/> md5sum sl4_x86_32_sl08e_8GB_ec2.img > sl4_x86_32_sl08e_8GB_ec2.checksum
/star/data08/OSG/APP/vm/virtualBox/ubuntu9.10_x8

In addition there will be a text file with the .txt extension giving more detail about the contents of the image.

Examples:

/star/data08/OSG/APP/vm/xen/sl5.3_x86_32_sl05c_8GB_ec2.txt
/star/data08/OSG/APP/vm/virtualBox/ubuntu9.10_x86_32_sl05c_10GB_ec2.txt

 

Recommendations for security and standardization (but not yet implemented):

 

Users will need to be able to login without editing the /etc/shadow file.

 

1)There will be a account “root” on all images with a password.

 

Note: In the case of Ubuntu just make the first account root.

 

2) There should be a hard password that changes regularly. This can be done by scripting the below command to run every time the image is started (by putting in /etc/rc.local for example):

 

date '+%y%m SEED=someLongString' | md5sum | base64 | sed 's|\(...........\).*|\1|' | passwd --stdin username

Then the command below can be placed in the text description of the image. The user instantiating the image can run the command to get the password. Example:

[rcas6007] ~/> date '+%y%m SEED=someLongString' | md5sum | sed 's|\(...........\).*|\1|' | \
base64 | sed 's|\(...........\).*|\1|' 
YjNkNTE5YWJ

If the command is compromised the seed will need to be changed.  

 

Alternatively:

There could also be a repository of ssh public keys available via a network connection which images can pull in.

 

3) There will be an account “star” on all images. This account will have the STAR environment (startup scripts). This is the account under which the actual jobs are run.