Tuesday, July 3, 2012

Convert EBS Backed AMI into Instace Store Backed

We had following requirements
  1. Have test environment all on t1.micro instances. The problem with t1.micro at that time was it was available for EBS backed instances only.
  2. We did not want to use EBS backed images for Cassandra or production machines as some standard tests complain about EBS' IO performance. Refer this.
  3. Even in production, we had a mix of t1.micro and m1.large machines.
This forced the following design
  1. Unified images -- all images will have all software. We start just the required ones on any instance.
  2. Two types of AMIs -- we had to have identical EBS AMI and Instance Store AMI.
The obvious solution was to make the updates on both the AMIs separately. It was painfully frustrating. Good for us that we could make an EBS backed AMI into Instance store backed. It's pretty simple.

  1. You have Amazon EC2 AMI Tool and Amazon EC2 API Tool -- installed and configured.
  2. Proper Access Key Id, Secret Access Key, X.509 Certificate, and Private Key. Refer.
Procedure It's same as creating a normal Instance Store backed AMI.

Upload the certificate and private key file to a location that does not go into image file. One such place is /mnt, and make sure they have read-only permission.
[root@domU-12-34-56-AA-AA-78 ~]# chmod 400 /mnt/*pem
[root@domU-12-34-56-AA-AA-78 ~]# ls -l /mnt
total 24
-r-------- 1 root root   916 Jul  3 09:05 cert-xxxxxxxx.pem
-r-------- 1 root root   926 Jul  3 09:05 pk-xxxxxxxx.pem
Next, bundle the instance using ec2-bundle-vol command

[root@domU-12-34-56-AA-AA-78 ~]# ec2-bundle-vol -d /mnt -k /mnt/pk-xxxxxxxx.pem -c /mnt/cert-xxxxxxxx.pem -u 123456789012 -r x86_64 -p naishe_ami
Copying / into the image file /mnt/naishe_ami...
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.001876 seconds, 559 MB/s
mke2fs 1.39 (29-May-2006)
Bundling image file...
 Splitting /mnt/naishe_ami.tar.gz.enc...
Created naishe_ami.part.000
Created naishe_ami.part.001
Created naishe_ami.part.002
Created naishe_ami.part.140
Generating digests for each part...
Creating bundle manifest...
ec2-bundle-vol complete.

Upload to your S3 bucket, using ec2-upload-bundle

[root@domU-12-34-56-AA-AA-78 ~]#  ec2-upload-bundle -b amibucket/naishe_ami -m /mnt/naishe_ami.manifest.xml -a 123456789012 -s th3SecRETkey10nGGGs7rinG
Uploading bundled image parts to the S3 bucket brtctx00 ...
Uploaded naishe_ami.part.000
Uploaded naishe_ami.part.001
Uploaded naishe_ami.part.002
Uploaded naishe_ami.part.140
Uploading manifest ...
Uploaded manifest.
Bundle upload completed.

Finally, register your newly bundled AMI using ec2-register command

[root@domU-12-34-56-AA-AA-78 ~]# ec2-register -C /mnt/cert-xxxxxxxx.pem -K /mnt/pk-xxxxxxxx.pem  amibucket/naishe_ami/naishe_ami.manifest.xml -n naishe_ami_20120703
IMAGE    ami-flac07ka

1 comment: