Install and run with Docker

Run Deephaven from Docker

Deephaven can be run from pre-built Docker images and requires only Docker. This guide will teach you how to run Deephaven from Docker, choose a deployment, and customize it for your applications.

Note

Docker isn't the only way to run Deephaven. Users who wish to use Python without Docker should install Deephaven with pip. Developers interested in tinkering with and modifying source code should build Deephaven from source. Users who wish to run from build artifacts without Docker should install and run the Deephaven production application.

Supported operating systems

Deephaven is only supported on:

Prerequisites

Running Deephaven with Docker requires docker >= 20.10.8.

Docker Compose is recomended for customized deployments, especially those that require multiple containers.

The simplest possible installation

The following shell command downloads and runs the server image:

docker run --name deephaven -p 10000:10000 ghcr.io/deephaven/server:latest

Replace server with any other deployment to change the pre-built image that gets downloaded and run.

Warning

docker run creates a new container from an image every time it's called. To reuse a container created from docker run, use docker start.

Note

This default configuration uses a pre-shared key to authenticate users. If not explicitly set, a randomly generated key gets printed to the Docker logs. See set a pre-shared key for how to set your own key.

Choose a deployment

Deephaven offers several pre-built Docker images that can be downloaded and used to run Deephaven:

  • server: Python
  • server-slim: Groovy
  • server-nltk: Python with NLTK
  • server-pytorch: Python with PyTorch
  • server-sklearn: Python with SciKit-Learn
  • server-tensorflow: Python with TensorFlow

Deephaven also has many pre-built docker-compose.yml files that build the images above with some additional features, including example data and Redpanda. They are located here.

Note

The Python packages in the images above can be installed manually like any other Python package. For more information, see installing Python packages in Deephaven.

Image versions

The latest version is used in the examples below. This corresponds to the most recent release number (e.g. 0.33.3, 0.33.0, etc.). While it's recommended to stay up-to-date with recent releases, Deephaven has many releases that can be used if desired. Versions can be any of the following:

  • latest (default): The most recent release.
  • A specific release tag: 0.32.0, 0.33.3, etc.
  • edge: An image published nightly and contains unreleased features.

Modify the deployment

The Deephaven deployment can be modified through Docker alone or with Docker Compose. The subsections below present ways to modify the deployment using both. Deephaven recommends the use of Docker Compose when creating custom deployments. See Key benefits of Docker Compose for more information on why.

Modifying the deployment with Docker should be done with docker create. docker run will always create a new container from an image. To run a pre-existing container, use docker start.

Modifying the deployment with Docker Compose requires updating the docker-compose.yml file used to build the container. The examples below will modify the following docker-compose.yaml file.

docker-compose.yml
services:
  deephaven:
    image: ghcr.io/deephaven/server:${VERSION:-latest}
    ports:
      - '${DEEPHAVEN_PORT:-10000}:10000'
    volumes:
      - ./data:/data
    environment:
      - START_OPTS=-Xmx4g

Set a pre-shared key

Deephaven, by default uses a pre-shared key to authenticate users looking to access Deephaven. If the key isn't set, Deephaven uses a randomly generated key that gets printed to the Docker logs.

The following deployment set the pre-shared key to YOUR_PASSWORD_HERE.

docker create --name deephaven -p 10000:10000 --env START_OPTS=-Dauthentication.psk=YOUR_PASSWORD_HERE ghcr.io/deephaven/server:latest

services:
  deephaven:
    image: ghcr.io/deephaven/server:${VERSION:-latest}
    ports:
      - '${DEEPHAVEN_PORT:-10000}:10000'
    volumes:
      - ./data:/data
    environment:
      - START_OPTS=-Xmx4g -Dauthentication.psk=YOUR_PASSWORD_HERE

Disable authentication

Anonymous authentication allows anyone to access a Deephaven instance. The following deployment enables anonymous authentication.

docker create --name deephaven -p 10000:10000 --env START_OPTS=-DAuthHandlers=io.deephaven.auth.AnonymousAuthenticationHandler ghcr.io/deephaven/server:latest

services:
  deephaven:
    image: ghcr.io/deephaven/server:${VERSION:-latest}
    ports:
      - '${DEEPHAVEN_PORT:-10000}:10000'
    volumes:
      - ./data:/data
    environment:
      - START_OPTS=-Xmx4g -DAuthHandlers=io.deephaven.auth.AnonymousAuthenticationHandler

Add more memory

The following deployment tell the server to allocate 8GB of heap memory instead of the default of 4GB.

docker create --name deephaven -p 10000:10000 --env START_OPTS=-Xmx8g ghcr.io/deephaven/server:latest

services:
  deephaven:
    image: ghcr.io/deephaven/server:${VERSION:-latest}
    ports:
      - '${DEEPHAVEN_PORT:-10000}:10000'
    volumes:
      - ./data:/data
    environment:
      - START_OPTS=-Xmx8g

Change the port

The following deployment tell Deephaven to expose port 9999 for the user to connect to via their web browser.

docker create --name deephaven -p 9999:10000 ghcr.io/deephaven/server:latest

services:
  deephaven:
    image: ghcr.io/deephaven/server:${VERSION:-latest}
    ports:
      - '${DEEPHAVEN_PORT:-9999}:10000'
    volumes:
      - ./data:/data
    environment:
      - START_OPTS=-Xmx4g

Add a second volume

Deephaven, by default, comes with a single data volume. The following deployment mounts a second specialty volume:

docker create --name deephaven -p 10000:10000 -v ./specialty:/specialty ghcr.io/deephaven/server:latest

services:
  deephaven:
    image: ghcr.io/deephaven/server:${VERSION:-latest}
    ports:
      - '${DEEPHAVEN_PORT:-10000}:10000'
    volumes:
      - ./data:/data
      - ./specialty:/specialty
    environment:
      - START_OPTS=-Xmx4g

Add a second image

Docker Compose specializes in running multi-container applications. In fact, Deephaven used to run in four separate containers before being reduced to one. Adding a second image to a Docker application should always be done with Docker Compose. The following YAML file runs Deephaven with Redpanda.

services:
  deephaven:
    image: ghcr.io/deephaven/server:${VERSION:-latest}
    ports:
      - '${DEEPHAVEN_PORT:-10000}:10000'
    volumes:
      - ./data:/data
    environment:
      - START_OPTS=-Xmx4g

  redpanda:
    command:
      - redpanda
      - start
      - --kafka-addr internal://0.0.0.0:9092,external://0.0.0.0:19092
      - --advertise-kafka-addr internal://redpanda:9092,external://localhost:19092
      - --schema-registry-addr internal://0.0.0.0:8081,external://0.0.0.0:18081
      - --smp 1
      - --memory 1G
      - --mode dev-container
    image: docker.redpanda.com/redpandadata/redpanda:v23.2.18
    ports:
      - 8081:8081
      - 18081:18081
      - 9092:9092
      - 19092:19092

Build a custom image

Custom Docker deployments often require things that cannot be done in a YAML file or Docker command. For instance, it is not possible to install a Python package this way. Such deployments typically extend a Docker image using both a Dockerfile and a docker-compose.yml file.

The following subsections build a custom Deephaven application through Docker with several Python packages installed that do not ship with official Deephaven Docker images.

Note

This example uses a flat directory structure - all files are placed in the same directory.

Dockerfile

A Dockerfile dictates which Docker images to build containers from and what else distinguishes these containers from their standard counterparts.

The following Dockerfile takes the latest Deephaven server image and installs the Python packages defined in requirements.txt into the container created from it.

FROM ghcr.io/deephaven/server:latest
COPY requirements.txt /requirements.txt
RUN pip install -r /requirements.txt && rm /requirements.txt

docker-compose.yml

Docker Compose can be told to build a Docker image from a local Dockerfile. The following YAML file builds a Docker container from a Dockerfile named Dockerfile in the same directory.

services:
  deephaven:
    build: .
    ports:
      - '${DEEPHAVEN_PORT:-10000}:10000'
    volumes:
      - ./data:/data
    environment:
      - START_OPTS=-Xmx4g

Start the application

To start a Deephaven application built from a docker-compose.yml file, run:

docker compose up --build

The --build flag tells Docker to build the services specified by the docker-compose.yml file. The Dockerfile defines the custom installation process of the service.

Note

If you've previously run docker compose up, add --pull to the command above to ensure you have the latest version of the Docker images.

Run Deephaven IDE

Once Deephaven is running, you can launch a Deephaven IDE in your web browser. The Deephaven IDE allows you to interactively analyze data and develop new analytics.

  • If Deephaven is running locally, navigate to http://localhost:10000/ide/.
  • If Deephaven is running remotely, navigate to http://<hostname>:10000/ide/, where <hostname> is the address of the machine Deephaven is running on.

img

Manage example data

The Deephaven examples repository contains data sets to help learn how to use Deephaven. Deephaven's documentation uses these data sets extensively, and they are needed to run some examples.

If you have chosen a deployment with example data, the example data sets will be downloaded to data/examples within your Deephaven folder, which translates to /data/examples within the Deephaven Docker container. See Docker data volumes for more information on how files get mounted in Docker.

What to do next?

New to Deephaven?

A crash course in Deephaven that covers the basics of working with tables and queries.