Document the process of re-spinning an image

* Document the process of respinning jcloud images in README.txt
* Create /scripts/respin-jcloud-images.sh
* Create /vagrant/lib/respin-functions.sh
* Automate the process of fetching latest base images
* Automate the process of creating new "LF upload" snapshots
* Automate process of creating jcloud image
- Boot fresh VM
- run bootstrap.sh
- run system_reseal.sh
- snapshot re-sealed system
* Unable to automate updating jcloud image regex for staging nodes
* Correct shasum to sha1sum
* Add cloud-initramfs-* to PACKAGES list
* Install cloud-init during system reseal

Change-Id: Ie70d5f1fdee0e9f06810da6f37bfc710e3e06b8e
Signed-off-by: C.J. Collier <cjcollier@linuxfoundation.org>

3 files changed, 142 insertions, 3 deletions

diff --git a/vagrant/basebuild/README.txt b/vagrant/basebuild/README.txt
new file mode 100644
index 000000000..adeb0bb81
--- /dev/null
+++ b/vagrant/basebuild/README.txt
@@ -0,0 +1,136 @@
+
+Standard Operating Procedures:
+
+=== Establish Cloudstack Credentials ===
+
+To get the correct configuration for the environment you will need
+openstack credentials.  Those used for this project are managed by
+vexxhost.  Establish credentials through appropriate channels.  Once
+credentials are acquired, visit this page:
+
+https://secure.vexxhost.com/console/#/account/credentials
+
+The environment variables you need will be at the bottom of the page,
+formatted as such:
+
+ export OS_TENANT_NAME="00000000-0000-0000-0000-000000000000"
+ export OS_USERNAME="00000000-0000-0000-0000-000000000000"
+ export OS_PASSWORD="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
+ export OS_AUTH_URL="https://auth.vexxhost.net/v2.0/"
+ export OS_REGION_NAME="ca-ymq-1"
+
+Save these lines to /tmp/openstack-credentials.sh
+
+=== Establish a python-virtual environment ===
+
+For Debian-based systems, be sure that you install these packages:
+* libpython-dev
+* python-virtualenv
+
+ CPPROJECT=fdio
+
+ export PVENAME=openstack-${CPPROJECT}
+ export PVEPATH=/usr/src/git/lf/git.lf.org/cjcollier/python-virtual
+ export PVERC=${PVEPATH}/${PVENAME}/bin/activate
+ mkdir -p ${PVEPATH}
+ cd ${PVEPATH}
+ virtualenv ${PVENAME}
+ cat /tmp/openstack-credentials.sh >> ${PVERC}
+ source  ${PVERC}
+ pip install --upgrade pip setuptools
+ pip install python-{cinder,glance,keystone,neutron,nova,openstack}client
+
+=== Install your public key into the cloudstack environment ===
+
+https://secure.vexxhost.com/console/v2.html#/compute/keypairs
+
+This will install your yubikey-backed ssh key:
+
+  ssh-add -L | grep cardno | nova keypair-add --pub-key - $USER
+
+=== Establish the GUID of our network ===
+
+Via the web interface: https://secure.vexxhost.com/console/#/networking/networks
+
+Command line:
+
+ NETID=$(nova network-list | awk '/${CPPROJECT}/ {print $2}')
+ echo "NETID=${NETID}" >> ${PVERC}
+
+=== Install Vagrant, vagrant-openstack-provider ===
+
+On Debian, you must uninstall the system vagrant and instead install the
+upstream package:
+
+  # https://releases.hashicorp.com/vagrant/1.8.1/vagrant_1.8.1_i686.deb
+  wget https://releases.hashicorp.com/vagrant/1.8.1/vagrant_1.8.1_${RH_ARCH}.deb
+  sudo dpkg -i vagrant_1.8.1_${RH_ARCH}.deb
+  vagrant plugin install vagrant-openstack-provider
+
+=== Configure openstack 'dummy' box ===
+
+CLI:
+  vagrant box add dummy https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box
+  wget -P ~/.vagrant.d/boxes/dummy/0/openstack/ https://raw.githubusercontent.com/cjac/scripts-n-things/master/vagrant/Vagrantfile
+  vi ~/.vagrant.d/boxes/dummy/0/openstack/Vagrantfile
+
+=== Check out the ci-management branch from fdio gerrit ===
+
+CLI:
+
+ FDIO_GDIR=/usr/src/git/lf/gerrit.fd.io
+ mkdir -p ${FDIO_GDIR}
+ cd ${FDIO_GDIR}
+ git clone ssh://gerrit.fd.io:29418/ci-management
+
+=== Acquire base images ===
+
+Cloud image links by platform:
+
+CentOS: http://cloud.centos.org/centos/
+Ubuntu: https://cloud-images.ubuntu.com/
+Debian: http://cdimage.debian.org/cdimage/openstack/
+Arch:   http://linuximages.de/openstack/arch/
+Gentoo: http://linuximages.de/openstack/gentoo/
+
+CLI:
+
+ source ${FDIO_GDIR}/vagrant/lib/respin-functions.sh
+
+ download_deb_image 'Ubuntu' '14.04' 'amd64'
+ download_deb_image 'Ubuntu' '16.04' 'amd64'
+ download_rh_image 'CentOS' '7' 'x86_64'
+
+=== Upload base images ===
+
+CLI:
+
+ source ${FDIO_GDIR}/vagrant/lib/respin-functions.sh
+
+ create_deb_image 'Ubuntu' '14.04' 'amd64'
+ create_deb_image 'Ubuntu' '16.04' 'amd64'
+ create_rh_image 'CentOS' '7' 'x86_64'
+
+
+== Re-spin jcloud images ==
+
+CLI:
+
+  export VAGRANT_DEFAULT_PROVIDER=openstack
+  export SERVER_NAME=kf7bmp-vagrant
+  export RESEAL=1
+  cd $FDIO_GDIR/ci-management/vagrant/basebuild
+
+  source ${FDIO_GDIR}/vagrant/lib/respin-functions.sh
+  respin_deb_image 'Ubuntu' '14.04' 'amd64'
+  respin_deb_image 'Ubuntu' '16.04' 'amd64'
+  respin_rh_image 'CentOS' '7' 'x86_64'
+
+
+=== Verify images ===
+
+* Update *-staging images at https://jenkins.fd.io/configure
+* Submit a probe patch to vpp (as DO NOT MERGE) and comment 'verify-images' to check the new images against vpp
+* Submit a probe patch to honeycomb (as DO NOT MERGE) and comment 'verify-images' to check the new images against honeycomb
+* If and only if both probe patches pass verify-images, update remaining images at https://jenkins.fd.io/configure
+* Abandon the vpp and honeycomb probe patches
diff --git a/vagrant/basebuild/Vagrantfile b/vagrant/basebuild/Vagrantfile
index ccca7c239..3aab38250 100644
--- a/vagrant/basebuild/Vagrantfile
+++ b/vagrant/basebuild/Vagrantfile
@@ -14,6 +14,10 @@ Vagrant.configure(2) do |config|
       override.ssh.username = 'baduser'

     end

 

+    if ENV['SERVER_NAME']

+      os.server_name = ENV['SERVER_NAME']

+    end

+

     case ENV['IMAGE']

       when /.*ubuntu.*/i

         override.ssh.username = 'ubuntu'

diff --git a/vagrant/basebuild/bootstrap.sh b/vagrant/basebuild/bootstrap.sh
index 6e0735c86..4aea50d65 100644
--- a/vagrant/basebuild/bootstrap.sh
+++ b/vagrant/basebuild/bootstrap.sh
@@ -9,8 +9,8 @@ set -e
 do_setup
 
 # record the bootstrap.sh checksum
-shasum $0 > /etc/bootstrap.sha
-shasum /vagrant/lib/bootstrap-functions.sh > /etc/bootstrap-functions.sha
+sha1sum $0 > /etc/bootstrap.sha
+sha1sum /vagrant/lib/bootstrap-functions.sh > /etc/bootstrap-functions.sha
 
 echo "---> Attempting to detect OS"
 # OS selector
@@ -26,7 +26,6 @@ then
     echo "---> Debian type system detected"
     export DEBIAN_FRONTEND=noninteractive
 
-    deb_enable_serial_console
     deb_aptconf_batchconf
     deb_sync_minor
     deb_correct_shell