Fast development workflow with Docker and Kubernetes

Keeping development environments in sync is a constant pain. Containerizing your development environment enables your service to run in the exact same environment everywhere, from your laptop to production (for more details on the benefits of a container native development workflow, see this post by Matt Butcher.)

Telepresence, in conjunction with a containerized development environment, gives the developer a fast development workflow in developing a multi-container application on Kubernetes. Telepresence lets you run a Docker container locally while proxying it to your Kubernetes cluster.

In this HOWTO, we'll walk through how to use Telepresence with a containerized Docker environment to build a fast development workflow.

Install Telepresence with Homebrew/apt/dnf

You will need the following available on your machine:

  • kubectl command line tool (here's the installation instructions).
  • Access to your Kubernetes cluster, with local credentials on your machine. You can test this by running kubectl get pod - if this works you're all set.

OS X

On OS X you can install Telepresence by running the following:

shell
brew install --cask osxfuse
brew install datawire/blackbird/telepresence-legacy

Ubuntu 16.04 or later

Run the following to install Telepresence:

shell
curl -s https://packagecloud.io/install/repositories/datawireio/telepresence/script.deb.sh | sudo bash
sudo apt install --no-install-recommends telepresence

If you are running another Debian-based distribution that has Python 3.5 installable as python3, you may be able to use the Ubuntu 16.04 (Xenial) packages. The following works on Linux Mint 18.2 (Sonya) and Debian 9 (Stretch) by forcing the PackageCloud installer to access Xenial packages.

shell
curl -sO https://packagecloud.io/install/repositories/datawireio/telepresence/script.deb.sh
sudo env os=ubuntu dist=xenial bash script.deb.sh
sudo apt install --no-install-recommends telepresence
rm script.deb.sh

A similar approach may work on Debian-based distributions with Python 3.6 by using the Ubuntu 17.10 (Artful) packages.

Fedora 26 or later

Run the following:

shell
curl -s https://packagecloud.io/install/repositories/datawireio/telepresence/script.rpm.sh | sudo bash
sudo dnf install telepresence

If you are running a Fedora-based distribution that has Python 3.6 installable as python3, you may be able to use Fedora packages. See the Ubuntu section above for information on how to invoke the PackageCloud installer script to force OS and distribution.

Arch Linux

Until we have a correct and working AUR package, please install from source. See issue #135 for the latest information.

Windows

See the Windows support documentation.

Install from source

On systems with Python 3.5 or newer, install into /usr/local/share/telepresence and /usr/local/bin by running:

shell
sudo env PREFIX=/usr/local ./install.sh

Install the software from the list of dependencies to finish.

Install into arbitrary locations by setting other environment variables before calling the install script. See the install script for more information. After installation you can safely delete the source code.

Other platforms

Don't see your favorite platform? Let us know and we'll try to add it. Also try installing from source.

Quick example

We'll start with a quick example. Apply this manifest to create a deployment and service both named hello-world, exposed on port 8000. Then confirm that the deployment becomes ready:

Terminal
$ kubectl apply -f https://raw.githubusercontent.com/telepresenceio/telepresence/master/docs/tutorials/hello-world.yaml
deployment.apps/hello-world created
service/hello-world created
$ kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/hello-world 1/1 1 1 6s

It may take a minute or two for the pod running the server to be up and running, depending on how fast your cluster is.

You can now run a Docker container using Telepresence that can access that service, even though the process is local but the service is running in the Kubernetes cluster:

Terminal
$ telepresence --docker-run --rm -it pstauffer/curl curl http://hello-world:8000/
[...]
T: Setup complete. Launching your container.
Hello, world!
T: Your process has exited.
[...]

Setting up a development environment in Docker

So how would we use Telepresence to do actual development of the hello-world service? We'll set up a local Dockerized development environment for hello-world. Clone the hello-world repo:

Terminal
$ git clone https://github.com/datawire/hello-world
Cloning into 'hello-world'...
[...]
$ cd hello-world

In the repository is a Dockerfile that builds a runtime environment for the hello-world service.

Build the runtime environment and tag it hello-dev:

Terminal
$ docker build -t hello-dev .
Sending build context to Docker daemon 24.58kB
Step 1/7 : FROM python:3-alpine
---> a93594ce93e7
[...]
---> 7d692d619894
Successfully built 7d692d619894
Successfully tagged hello-dev:latest

We'll use Telepresence to swap the hello-world deployment with the local Docker image. Behind the scenes, Telepresence invokes docker run, so it supports any arguments you can pass to docker run. In this case, we're going to also mount our local directory to /usr/src/app in your Docker container. Make sure your current working directory is the hello-world directory, since we're going to mount that directly into the container.

Terminal
$ telepresence --swap-deployment hello-world --docker-run --rm -it -v $(pwd):/usr/src/app hello-dev
T: Volumes are rooted at $TELEPRESENCE_ROOT. See https://telepresence.io/howto/volumes.html for details.
T: Starting network proxy to cluster by swapping out Deployment hello-world with a proxy
T: Forwarding remote port 8000 to local port 8000.
T: Setup complete. Launching your container.
* Serving Flask app "server" (lazy loading)
[...]

We can test this out. In another terminal, we'll start a pod remotely on the Kubernetes cluster.

Terminal
$ kubectl run curler -it --rm --image=pstauffer/curl --restart=Never -- sh
If you don't see a command prompt, try pressing enter.
/ # curl http://hello-world:8000
Hello, world!
/ #

Let's change the message in server.py. At a shell prompt in the hello-world directory, modify the file using sed:

Terminal
$ sed -i.bak -e s/Hello/Greetings/ server.py
[no output]

or just use your editor to change the file. The change we have made is very simple:

Terminal
$ git diff
diff --git a/server.py b/server.py
index 04f15e2..7fffeb1 100644
--- a/server.py
+++ b/server.py
@@ -1,7 +1,7 @@
from flask import Flask
PORT = 8000
-MESSAGE = "Hello, world!\n"
+MESSAGE = "Greetings, world!\n"
app = Flask(__name__)

Rerun the curl command from your remote pod:

Terminal
/ # curl http://hello-world:8000
Greetings, world!
/ #

Notice how the output has updated in realtime. Congratulations! You've now:

  • Routed the hello-world service to the Docker container running locally
  • Configured your Docker service to pick up changes from your local filesystem
  • Made a live code edit and seen it immediately reflected in production

How it works

Telepresence will start a new proxy container and then call docker run with whatever arguments you pass to --docker-run to start a container that will have its networking proxied. All networking is proxied:

  • Outgoing to Kubernetes.
  • Outgoing to cloud resources outside the cluster
  • Incoming connections from the cluster to ports specified with --expose.

Volumes and environment variables from the remote Deployment are also available in the container.

Cleaning up and next step

  • Quit your remote pod shell (exit) to clean up that pod.
  • Press Ctrl-C at your Telepresence terminal. Telepresence will swap the deployment back to its original state.
  • In a real development situation, you would commit your development work and let CI do its thing. Or build and deploy your changes however you normally would.
Install Telepresence with Homebrew/apt/dnf

You will need the following available on your machine:

  • kubectl command line tool (here's the installation instructions).
  • Access to your Kubernetes cluster, with local credentials on your machine. You can test this by running kubectl get pod - if this works you're all set.

OS X

On OS X you can install Telepresence by running the following:

shell
brew install --cask osxfuse
brew install datawire/blackbird/telepresence-legacy

Ubuntu 16.04 or later

Run the following to install Telepresence:

shell
curl -s https://packagecloud.io/install/repositories/datawireio/telepresence/script.deb.sh | sudo bash
sudo apt install --no-install-recommends telepresence

If you are running another Debian-based distribution that has Python 3.5 installable as python3, you may be able to use the Ubuntu 16.04 (Xenial) packages. The following works on Linux Mint 18.2 (Sonya) and Debian 9 (Stretch) by forcing the PackageCloud installer to access Xenial packages.

shell
curl -sO https://packagecloud.io/install/repositories/datawireio/telepresence/script.deb.sh
sudo env os=ubuntu dist=xenial bash script.deb.sh
sudo apt install --no-install-recommends telepresence
rm script.deb.sh

A similar approach may work on Debian-based distributions with Python 3.6 by using the Ubuntu 17.10 (Artful) packages.

Fedora 26 or later

Run the following:

shell
curl -s https://packagecloud.io/install/repositories/datawireio/telepresence/script.rpm.sh | sudo bash
sudo dnf install telepresence

If you are running a Fedora-based distribution that has Python 3.6 installable as python3, you may be able to use Fedora packages. See the Ubuntu section above for information on how to invoke the PackageCloud installer script to force OS and distribution.

Arch Linux

Until we have a correct and working AUR package, please install from source. See issue #135 for the latest information.

Windows

See the Windows support documentation.

Install from source

On systems with Python 3.5 or newer, install into /usr/local/share/telepresence and /usr/local/bin by running:

shell
sudo env PREFIX=/usr/local ./install.sh

Install the software from the list of dependencies to finish.

Install into arbitrary locations by setting other environment variables before calling the install script. See the install script for more information. After installation you can safely delete the source code.

Other platforms

Don't see your favorite platform? Let us know and we'll try to add it. Also try installing from source.

Still have questions? Ask in our Slack chatroom or file an issue on GitHub.