Fast development workflow with Docker and Kubernetes
Keeping development environments in sync is a constant pain. Containerizing your development environment enables your service to run in the exact same environment everywhere: from your laptop to production (for more details on the benefits of a container native development workflow, see this post by Matt Butcher.)
Telepresence, in conjunction with a containerized development environment, gives the developer a fast development workflow in developing a multi-container application on Kubernetes. Telepresence lets you run a Docker container locally, while proxying it to your Kubernetes cluster.
In this HOWTO, we'll walk through how to use Telepresence with a containerized Docker environment to build a fast development workflow.
Install Telepresence with Homebrew/apt/dnf
You will need the following available on your machine:
kubectlcommand line tool (here's the installation instructions).
- Access to your Kubernetes cluster, with local credentials on your machine. You can test this by running
kubectl get pod- if this works you're all set.
On OS X you can install Telepresence by running the following:
brew cask install osxfuse brew install datawire/blackbird/telepresence
Ubuntu 16.04 or later
Run the following to install Telepresence:
curl -s https://packagecloud.io/install/repositories/datawireio/telepresence/script.deb.sh | sudo bash sudo apt install --no-install-recommends telepresence
If you are running another Debian-based distribution that has Python 3.5 installable as
python3, you may be able to use the Ubuntu 16.04 (Xenial) packages. The following works on Linux Mint 18.2 (Sonya) and Debian 9 (Stretch) by forcing the PackageCloud installer to access Xenial packages.
curl -sO https://packagecloud.io/install/repositories/datawireio/telepresence/script.deb.sh sudo env os=ubuntu dist=xenial bash script.deb.sh sudo apt install --no-install-recommends telepresence rm script.deb.sh
A similar approach may work on Debian-based distributions with Python 3.6 by using the Ubuntu 17.10 (Artful) packages.
Fedora 26 or later
Run the following:
curl -s https://packagecloud.io/install/repositories/datawireio/telepresence/script.rpm.sh | sudo bash sudo dnf install telepresence
If you are running a Fedora-based distribution that has Python 3.6 installable as
python3, you may be able to use Fedora packages. See the Ubuntu section above for information on how to invoke the PackageCloud installer script to force OS and distribution.
Until we have a correct and working AUR package, please install from source. See issue #135 for the latest information.
See the Windows support documentation.
Install from source
On systems with Python 3.5 or newer, install into
/usr/local/bin by running:
sudo env PREFIX=/usr/local ./install.sh
Install the software from the list of dependencies to finish.
Install into arbitrary locations by setting other environment variables before calling the install script. See the install script for more information. After installation you can safely delete the source code.
Don't see your favorite platform? Let us know and we'll try to add it. Also try installing from source.
We'll start with a quick example. Start by running a service in the cluster:
$ kubectl run hello-world --image=datawire/hello-world --port 8000 --expose [...] service/hello-world created deployment.apps/hello-world created $ kubectl get service hello-world NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE hello-world ClusterIP 10.111.122.25 <none> 8000/TCP 45s
It may take a minute or two for the pod running the server to be up and running, depending on how fast your cluster is.
You can now run a Docker container using Telepresence that can access that service, even though the process is local but the service is running in the Kubernetes cluster:
$ telepresence --docker-run --rm -it pstauffer/curl curl http://hello-world:8000/ [...] T: Setup complete. Launching your container. Hello, world! T: Your process has exited. [...]
(This will not work if the hello-world pod hasn't started yet... If so, try again.)
Setting up a development environment in Docker
So how would we use Telepresence to do actual development of the hello-world service? We'll set up a local Dockerized development environment for hello-world. Clone the hello-world repo:
$ git clone https://github.com/datawire/hello-world Cloning into 'hello-world'... [...] $ cd hello-world
In the repository is a
Dockerfile that builds a runtime environment for the hello-world service.
Build the runtime environment and tag it
$ docker build -t hello-dev . Sending build context to Docker daemon 24.58kB Step 1/7 : FROM python:3-alpine ---> a93594ce93e7 [...] ---> 7d692d619894 Successfully built 7d692d619894 Successfully tagged hello-dev:latest
We'll use Telepresence to swap the hello-world deployment with the local Docker image. Behind the scenes, Telepresence invokes
docker run, so it supports any arguments you can pass to
docker run. In this case, we're going to also mount our local directory to
/usr/src/app in your Docker container. Make sure your current working directory is the
hello-world directory, since we're going to mount that directly into the container.
$ telepresence --swap-deployment hello-world --docker-run --rm -it -v $(pwd):/usr/src/app hello-dev T: Volumes are rooted at $TELEPRESENCE_ROOT. See https://telepresence.io/howto/volumes.html for details. T: Starting network proxy to cluster by swapping out Deployment hello-world with a proxy T: Forwarding remote port 8000 to local port 8000. T: Setup complete. Launching your container. * Serving Flask app "server" (lazy loading) [...]
We can test this out. In another terminal, we'll start a pod remotely on the Kubernetes cluster.
$ kubectl run curler -it --rm --image=pstauffer/curl --restart=Never -- sh If you don't see a command prompt, try pressing enter. / # curl http://hello-world:8000 Hello, world! / #
Let's change the message in
server.py. At a shell prompt in the
hello-world directory, modify the file using
$ sed -i.bak -e s/Hello/Greetings/ server.py [no output]
or just use your editor to change the file. The change we have made is very simple:
$ git diff diff --git a/server.py b/server.py index 04f15e2..7fffeb1 100644 --- a/server.py +++ b/server.py @@ -1,7 +1,7 @@ from flask import Flask PORT = 8000 -MESSAGE = "Hello, world!\n" +MESSAGE = "Greetings, world!\n" app = Flask(__name__)
curl command from your remote pod:
/ # curl http://hello-world:8000 Greetings, world! / #
And notice how the code has changed, live. Congratulations! You've now:
- Routed the hello-world service to the Docker container running locally
- Configured your Docker service to pick up changes from your local filesystem
- Made a live code edit and seen it immediately reflected in production
How it works
Telepresence will start a new proxy container, and then call
docker run with whatever arguments you pass to
--docker-run to start a container that will have its networking proxied. All networking is proxied:
- Outgoing to Kubernetes.
- Outgoing to cloud resources outside the cluster
- Incoming connections from the cluster to ports specified with
Volumes and environment variables from the remote
Deployment are also available in the container.
Cleaning up and next step
- Quit your remote pod shell (
exit) to clean up that pod.
- Press Ctrl-C at your Telepresence terminal. Telepresence will swap the deployment back to its original state.
- In a real development situation, you would commit your development work and let CI do its thing. Or build and deploy your changes however you normally would.
Install Telepresence with Homebrew/apt/dnf
If this tutorial was useful to you, we'd appreciate it if you'd share it online.