The code for this post is in the following repositories:
As we turn to clould frameworl like Hadoop, Spark, Mesos or Storm, it becomes necessary to test not only the code but also its deployment. In order to do this in a reproductible manner, we have started using ansible roles, and to test these we use dummy clusters.
To create these we first need a basic centos image (We use centos in our deployment cluster.) Quickly problems arose from using different providers and providers versions with Vagrant. In order to solve them, we started creating our own boxes using packer. This allows everyone to create a box containing the VirtualBox guest modules corresponding to his/her VirtualBox version. The code is in the first repository aborsu/packer-centos7.
Once this image is launched, a common Vagrantfile is used to launch a cluster. The vagrant file allows to easily modify the quantity and settings of the cluster nodes. Also by changing the vm.network setting from private to public, we can create a shared test cluster in our local network. aborsu/test-cluster.
Finally the aborsu/ansible-roles repository contains our roles and a test inventory file. The same repo is actually used with a different inventory file to deploy on the cluster.
s