Running Docker containers on ESXi standalone hosts with the vSphere Integrated Container engine

The vSphere Integrated Containers Engine (VIC Engine) is a container runtime for vSphere, allowing developers familiar with Docker to develop in containers and deploy them alongside traditional VM-based workloads on vSphere clusters, and allowing for these workloads to be managed through the vSphere UI in a way familiar to existing vSphere admins. Full support of vSphere Integrated Containers requires the vSphere Enterprise Plus license and an official VMware release of vSphere Integrated Containers.

But I was just curious if it is possible to use Docker containers to consolidate some of my own workloads on my ESXi standalone host. I’m now using full blown Virtual Machines running multiple services. With the container approach I can create segmentation and isolation and threat every service as an individual application.

We need to download and extract VIC for managing the Virtual Container Host (VCH) and installing Docker for container management.

Now first we’re deploying a VCH:

    1. Get fingerprint of ESXi host
      [root@esxi:~] openssl x509 -in /etc/vmware/ssl/rui.crt -fingerprint -sha1 -noout
      SHA1 Fingerprint=#fingerprint#
    2. Change firewall of ESXi host
      vic-machine-windows update firewall -target #user#:#password#@#host# -allow -thumbprint=#fingerprint#
      Dec 24 2017 09:14:38.107+01:00 INFO ### Updating Firewall ####
      Dec 24 2017 09:14:38.508+01:00 INFO Validating target
      Dec 24 2017 09:14:38.538+01:00 INFO Validating compute resource
      Dec 24 2017 09:14:38.538+01:00 INFO
      Dec 24 2017 09:14:38.542+01:00 WARN ### WARNING ###
      Dec 24 2017 09:14:38.544+01:00 WARN This command modifies the host firewall on the target machine or cluster
      Dec 24 2017 09:14:38.546+01:00 WARN The ruleset "vSPC" will be enabled
      Dec 24 2017 09:14:38.548+01:00 WARN This allows all outbound TCP traffic from the target
      Dec 24 2017 09:14:38.550+01:00 WARN To undo this modification use --deny
      Dec 24 2017 09:14:38.551+01:00 INFO
      Dec 24 2017 09:14:38.583+01:00 INFO Ruleset "vSPC" enabled on host "HostSystem:ha-host @ /ha-datacenter/host/#host#.#domain#.#tld#/#host#.#domain#.#tld#"
      Dec 24 2017 09:14:38.583+01:00 INFO
      Dec 24 2017 09:14:38.589+01:00 INFO Firewall changes complete
      Dec 24 2017 09:14:38.601+01:00 INFO Command completed successfully
    3. Deploy VCH host
      vic-machine-windows.exe create -target #user#:#password#@#host# -name vch --ops-user #user# --ops-password #password# -tls-cname vch -image-store #datastore#/vch-images -volume-store #datastore#/vic-volumes:default -bridge-network bridge-pg -public-network "VM Network" -public-network-gateway #gateway# -public-network-ip #ip#/24 -dns-server #dns1# -dns-server #dns2# --endpoint-memory 3072 --no-tlsverify -thumbprint=#fingerprint#
      Dec 24 2017 09:41:45.024+01:00 INFO ### Installing VCH ####
      Dec 24 2017 09:41:45.028+01:00 INFO vSphere password for root:
      Dec 24 2017 09:41:48.876+01:00 INFO Loaded server certificate vch\server-cert.pem
      Dec 24 2017 09:41:48.876+01:00 WARN Configuring without TLS verify - certificate-based authentication disabled
      Dec 24 2017 09:41:49.297+01:00 INFO Validating supplied configuration
      Dec 24 2017 09:41:49.549+01:00 INFO Configuring static IP for additional networks using port group "VM Network"
      Dec 24 2017 09:41:49.686+01:00 INFO Firewall status: ENABLED on "/ha-datacenter/host/#fqdn#/#fqdn#"
      Dec 24 2017 09:41:49.702+01:00 INFO Firewall configuration OK on hosts:
      Dec 24 2017 09:41:49.702+01:00 INFO "/ha-datacenter/host/#fqdn#/#fqdn#"
      Dec 24 2017 09:41:49.737+01:00 INFO License check OK
      Dec 24 2017 09:41:49.737+01:00 INFO DRS check SKIPPED - target is standalone host
      Dec 24 2017 09:41:49.817+01:00 INFO
      Dec 24 2017 09:41:50.245+01:00 INFO Creating Resource Pool "vch"
      Dec 24 2017 09:41:50.258+01:00 INFO Creating VirtualSwitch
      Dec 24 2017 09:41:50.381+01:00 INFO Creating Portgroup
      Dec 24 2017 09:41:50.471+01:00 INFO Creating appliance on target
      Dec 24 2017 09:41:50.506+01:00 INFO Network role "public" is sharing NIC with "management"
      Dec 24 2017 09:41:50.506+01:00 INFO Network role "client" is sharing NIC with "management"
      Dec 24 2017 09:41:50.761+01:00 INFO Creating directory [ESXI] vic-volumes
      Dec 24 2017 09:41:50.778+01:00 INFO Datastore path is [ESXI] vic-volumes
      Dec 24 2017 09:41:51.183+01:00 INFO Uploading images for container
      Dec 24 2017 09:41:51.183+01:00 INFO "appliance.iso"
      Dec 24 2017 09:41:51.184+01:00 INFO "bootstrap.iso"
      Dec 24 2017 09:42:06.318+01:00 INFO Waiting for IP information
      Dec 24 2017 09:42:18.140+01:00 INFO Waiting for major appliance components to launch
      Dec 24 2017 09:42:18.514+01:00 INFO Obtained IP address for client interface: "#host#"
      Dec 24 2017 09:42:18.514+01:00 INFO Checking VCH connectivity with vSphere target
      Dec 24 2017 09:42:18.916+01:00 INFO vSphere API Test: https://#host# vSphere API target responds as expected
      Dec 24 2017 09:42:28.049+01:00 INFO Initialization of appliance successful
      Dec 24 2017 09:42:28.049+01:00 INFO
      Dec 24 2017 09:42:28.054+01:00 INFO VCH Admin Portal:
      Dec 24 2017 09:42:28.058+01:00 INFO https://#host#:2378
      Dec 24 2017 09:42:28.059+01:00 INFO
      Dec 24 2017 09:42:28.061+01:00 INFO Published ports can be reached at:
      Dec 24 2017 09:42:28.063+01:00 INFO #host#
      Dec 24 2017 09:42:28.069+01:00 INFO
      Dec 24 2017 09:42:28.071+01:00 INFO Docker environment variables:
      Dec 24 2017 09:42:28.074+01:00 INFO DOCKER_HOST=#host#:2376
      Dec 24 2017 09:42:28.083+01:00 INFO
      Dec 24 2017 09:42:28.084+01:00 INFO Environment saved in vch/vch.env
      Dec 24 2017 09:42:28.085+01:00 INFO
      Dec 24 2017 09:42:28.088+01:00 INFO Connect to docker:
      Dec 24 2017 09:42:28.090+01:00 INFO docker -H #host#:2376 --tls info
      Dec 24 2017 09:42:28.092+01:00 INFO Installer completed successfully
    4. Show container host information
      set DOCKER_HOST=tcp://#host#:2376
      docker --tls info
      Containers: X
       Running: X
       Paused: 0
       Stopped: 0
      Images: 5
      Server Version: v1.3.0-15556-473375a
      Storage Driver: vSphere Integrated Containers v1.3.0-15556-473375a Backend Engine
      VolumeStores: default
      vSphere Integrated Containers v1.3.0-15556-473375a Backend Engine: RUNNING
       VCH CPU limit: 4864 MHz
       VCH memory limit: 4.511 GiB
       VCH CPU usage: 614 MHz
       VCH memory usage: 5.445 GiB
       VMware Product: VMware ESXi
       VMware OS: vmnix-x86
       VMware OS version: 6.5.0
       Registry Whitelist Mode: disabled.  All registry access allowed.
      Plugins:
       Volume: vsphere
       Network: bridge
       Log:
      Swarm: inactive
      Operating System: vmnix-x86
      OSType: vmnix-x86
      Architecture: x86_64
      CPUs: 4864
      Total Memory: 4.511GiB
      ID: vSphere Integrated Containers
      Docker Root Dir:
      Debug Mode (client): false
      Debug Mode (server): false
      Registry: registry.hub.docker.com
      Experimental: false
      Live Restore Enabled: false
    5. Start first ‘management’ container
      docker --tls run -d -p 8282:8282 --name admiral vmware/admiral
      Unable to find image 'vmware/admiral:latest' locally
      latest: Pulling from vmware/admiral
      1c0b69d98c5b: Pull complete
      a3ed95caeb02: Pull complete
      f1bf54e3bee2: Pull complete
      a636bec27aa0: Pull complete
      9cf592e78ba2: Pull complete
      827165f1c6de: Pull complete
      3addb704a0c6: Pull complete
      2ca7dc8e087d: Pull complete
      e14e9eff31ca: Pull complete
      2626a5abb3b1: Pull complete
      f2c95f6064e6: Pull complete
      Digest: sha256:82474001628fb5043caceb1c3c5a1c4a9b8246a84eddbf95756d46c125c51966
      Status: Downloaded newer image for vmware/admiral:latest
      390c91691cf551452a4aec72cadc9a420b7e5294a54624167b9ed298e067c043
    6. Browse to http://#host#:8282/ and add the VCH host under Clusters and view the logging of the VCH host at http://#host#:2376/.

You can make the DOCKER_HOST variable persistent on MACHINE level during reboots:

PS > [Environment]::SetEnvironmentVariable("DOCKER_HOST", "tcp://#host#:2376", "Machine")
PS > Get-ChildItem Env:DOCKER_HOST
Name Value
---- -----
DOCKER_HOST tcp://#host#:2376