Update Jan 2019: Now using CredHub instead of ‘–vars-store’ (which will be deprecated in CF 3)
Even if you are developing a service or application that will ultimately be deployed to a private Cloud Foundry instance, having a local CF instance for development work is still an ideal development workflow.
There is a local CF deployment called PCF Dev that you can run on a local VirtualBox but Pivotal stopped releasing Linux versions, and another downside is that does not include any BOSH components.
In this article, I’ll show how to deploy Cloud Foundry on top of a locally installed version of BOSH on VirtualBox for Ubuntu.
Prerequisite
As a prerequisite, install the BOSH CLI v2 and BOSH Lite unto a VirtualBox VM as described in my article here. Hopefully you can allocate 4vCPU and 8G RAM to the VM as I recommended, otherwise the installation will be slow.
When this is complete, you should have an alias named ‘vbox’ pointing at 192.168.50.6, and be logged into the BOSH Director using the ‘admin’ credentials.
Cloud Foundry deployment
Get CF project from github
Retrieve the Cloud Foundry project from github.
sudo apt-get install git curl -y git clone https://github.com/cloudfoundry/cf-deployment.git cd cf-deployment
Verify if any deployments or VMs are already deployed on BOSH Lite. If you have the ‘webapp-warden’ sample project or ZooKeeper still deployed from my previous article, go ahead and delete it to free up resources.
Upload stemcells to Blobstore
Cloud Foundry needs a version of the BOSH Lite Warden stemcell in order to build the solution. To upload this stemcell into the Blobstore, run the following commands.
# stemcell type export STEMCELL_TYPE=$(bosh int cf-deployment.yml --path /stemcells/alias=default/os) # stemcell version export STEMCELL_VERSION=$(bosh int cf-deployment.yml --path /stemcells/alias=default/version) # put stemcell into blobstore bosh upload-stemcell https://bosh.io/d/stemcells/bosh-warden-boshlite-$STEMCELL_TYPE-go_agent?v=$STEMCELL_VERSION
Update cloud config
Set the infrastructure/network configuration cloud config for BOSH Lite.
# cloud config bosh update-cloud-config iaas-support/bosh-lite/cloud-config.yml
Update runtime config
If you used my instructions for setting up BOSH, the runtime config will already be set. But if not, go back into your BOSH installation directory and set the native DNS runtime config.
# is the runtime config already set? If so, you can skip the next step bosh runtime-config # runtime config bosh update-runtime-config bosh-deployment/runtime-configs/dns.yml --name dns
Set CredHub env vars
For CF deployments, CredHub is the new way of holding secrets instead of using “–vars-store” (which will be deprecated in CF 3).
If you used my instructions for deploying BOSH, there is a convenience script that can be invoked like this:
source ./credhub.sh
OR you can run the same commands manually:
export CREDHUB_SERVER=https://192.168.50.6:8844 export CREDHUB_CLIENT=credhub-admin export CREDHUB_SECRET=$(bosh interpolate ./creds.yml --path=/credhub_admin_client_secret) export CREDHUB_CA_CERT="$(bosh interpolate ./creds.yml --path=/credhub_tls/ca )"$'\n'"$(bosh interpolate ./creds.yml --path=/uaa_ssl/ca)"
You also need to install the CredHub CLI so that you can access your secrets from the server located in the BOSH Lite VM.
# get release wget https://github.com/cloudfoundry-incubator/credhub-cli/releases/download/2.2.0/credhub-linux-2.2.0.tgz # extract tar xvfz cred*.tgz # put in path and executable chmod ugo+r+x credhub sudo chown root:root credhub sudo mv credhub /usr/local/bin/. # validate credhub --version
Start CF Build
You are now ready to start the CF deployment using BOSH. We are specifying a system domain of ‘bosh-lite.com’ which is specially crafted to send traffic locally and described in more detail further down.
bosh -d cf deploy cf-deployment.yml \ -o operations/bosh-lite.yml \ -v system_domain=bosh-lite.com \ -o operations/scale-to-one-az.yml \ -o operations/use-compiled-releases.yml \ --no-redact
The “scale-to-one-az.yml” and “use-compiled-releases.yml” are optional. We use them to keep the CF jobs limited to a single instance (minimize resource usage), and to use compiled releases in order to minimize deployment time.
If you get errors about unknown VM types, make sure you updated the cloud config as shown earlier. If errors occur, you can troubleshoot and rerun the same deployment command.
When this is done, check the BOSH deployments and vms, which are now populated with CF releases and VMs for the various CF components (diego, router, database, etc.)
$ bosh -d cf deployments ... cf binary-buildpack/1.0.28 bosh-warden-boshlite-ubuntu-xenial-go_agent/170.16 ... $ bosh -d cf vms ... adapater/... api/... cc-worker/... credhub/... database/... diego-api/... diego-cell/... doppler/... log-api/... nats/... router/f..e1 running z1 10.244.0.34... scheduler/... singleton-blobstore/... tcp-router/... uaa/... ...
Notice that the “router” VM has an internal IP of 10.244.0.34. This coincides with our use of the “bosh-lite.com” domain.
Explanation of bosh-lite.com
The name “bosh-lite.com” has a public DNS resolution to 10.244.0.34. Based on the custom route we created during BOSH Lite installation, all addresses which look like 10.244.0.0/16 get routed to our BOSH Lite host at 192.168.50.6 which then knows how send the traffic to its private containers in 10.244.0.0.
No matter what the DNS configuration for you host, ultimately you need to be able to get an IP resolution of 10.244.0.34 on “bosh-lite.com”.
# should resolve to 10.244.0.34 nslookup bosh-lite.com nslookup api.bosh-lite.com # should resolve to 10.244.0.34 dig bosh-lite.com ANY dig api.bosh-lite.com
There are other specialized domains like sslip.io that offer similar functionality. If working offline, using dnsmasq as I described in this article can also get you the proper DNS resolution (append “address=/.bosh-lite.com/10.244.0.34” to /etc/dnsmasq.conf and restart service).
Cloud Foundry Validation
In order to validate the CF installation, we need to install the CF CLI tool, use that to login to CF, then do a test deployment of an application.
Install CF CLI tool
First you must install the CF CLI which is a console based client application that communicates with the CF Cloud Controller.
# get key, add repo source wget -q -O - https://packages.cloudfoundry.org/debian/cli.cloudfoundry.org.key | sudo apt-key add - echo "deb http://packages.cloudfoundry.org/debian stable main" | sudo tee /etc/apt/sources.list.d/cloudfoundry-cli.list # install package sudo apt-get update sudo apt-get install cf-cli curl -y
Then verify the CLI version, which should look similar to below:
$ cf --version cf version 6.41.0+dd4c76cdd.2018-11-28
Login to Cloud Foundry
Now login to Cloud Foundry using the CLI. First, verify that the CredHub CLI can reach its servers and has an entry for the CF admin secret.
$ credhub find -n cf_admin_password credentials: - name: /bosh-lite/cf/cf_admin_password version_created_at: "2019-01-13T17:50:46Z"
Then issue the CF login command like:
cf login -a https://api.bosh-lite.com --skip-ssl-validation -u admin -p $(credhub get -n /bosh-lite/cf/cf_admin_password -q)
This is in contrast to the command we would have ran in older versions without CredHub, where the cf_admin_password would have been fetched using this command:
bosh interpolate deployment-vars.yml --path /cf_admin_password
This login should succeed, but you will get a message saying there is no space targeted, which is expected. Create an organization named “cloudfoundry” and a space named “development”.
# create org cf create-org cloudfoundry cf target -o cloudfoundry # create space cf create-space development cf target -o cloudfoundry -s development
Simple CF deployment
To test our CF instance, there is a staticfile buildpack which makes it very easy to deploy static web pages as a CF application.
# create content mkdir simple && cd $_ touch Staticfile echo "<h1>This is simple</h1>" > index.html # push app $ cf push simple -m 64M # view apps $ cf apps .. simple started 1/1 64M 1G simple.bosh-lite.com ... $ cf app simple ... buildpack: staticfile ... #0 running 2018-03-08T22:27:00Z ... # view logs $ cf logs simple --recent
Now retrieve the content from http://simple.bosh-lite.com
$ curl http://`cf app simple | grep routes | awk {'print $2'}`
<h1>This is simple</h1>
Remember that any subdomain under “bosh-lite.com” resolves to 10.244.0.34, which is the internal address of the router in the BOSH Lite VM.
Teardown
If you want to delete the ‘simple’ application:
cf delete simple
If you needed to delete the entire CF deployment:
bosh -d cf delete-deployment
SSH into Container
If you need to ssh directly into a container, such as instance #0 of ‘simple’:
cf ssh simple -i 0
BOSH Lite VM lifecycle
If the BOSH Lite guest VM is rebooted, or the host server is restarted, the system won’t persist your CF deployment or containers. You need to save the state of the VM and only then stop if you want your state to remain intact.
$ vboxmanage controlvm $(bosh int state.json --path /current_vm_cid) savestate $ vboxmanage startvm $(bosh int state.json --path /current_vm_cid) --type headless
Credit to Cristopher Banck for these commands.
REFERENCES
CF Deployment Guide to BOSH Lite
Article that goes through director, then CF deployment
Older instructions for CF deployment locally
older CF deployment instructions
env vars made accessible to cf applications
public DNS for private IP resolution, e.g. api.10-244-0-34.sslip.io
CloudFoundry slideshare, deploy BOSH2 AWS/GCP, bbl, CF
CF description of operations files in standard deployment