Install¶

This page will help you install Crane on your cluster nodes. Refer to the deploy guide to actually have Crane running.

Docker¶

Machines that directly participate as nodes in the Crane cluster require Docker and the NVIDIA container runtime. Especially, the Docker Engine version must support Docker swarm. The minimum required version of Docker Engine is 20.10.6.

Docker: https://docs.docker.com/get-docker/

NVIDIA Container Runtime: https://github.com/NVIDIA/nvidia-container-runtime

Crane Components¶

Crane has four components:

• Crane Core: The core logic of Crane: cluster manager, node manager, gateway, and state DB. Each component runs in a Docker container.
• Crane Admin Daemon: A daemon process named crane-admind runs on each node in the cluster. The daemon can initialize a new Crane cluster, join/leave an existing Crane cluster, and promote/demote a node's role in the cluster (leader or follower).
• Crane CLI: The Admin and User command-line tools. The Admin CLI is paired with the Crane Admin Daemon, while the User CLI communicates with the Crane gateway to query and set the cluster/job/user state.
• Crane Bootstrap: This is a simple command-line tool to start the crane-admind process on the current node.

Crane can be installed as a system package, via pip, or from source.

Crane Core, Admin Daemon, and Bootstrap¶

Install crane components

$ sudo add-apt-repository ppa:friendliai/crane
$ sudo apt-get update
$ sudo apt-get install crane
---> *

This will install boot.crane on the node, and configure it as a systemd service. When crane-admind first starts, it pulls the latest Crane Docker images from Docker Hub.

Crane CLI¶

First, create and activate a virtual environment with python 3.7 or above. We'll use pyenv in this guide. You may want to update your PATH environment variable or initialize venv providers in your shell rc file based on your choice.

# Create virtual environment
$ curl https://pyenv.run | bash
$ pyenv install 3.7.6
---> 100%
$ pyenv virtualenv 3.7.6 crane
$ pyenv activate crane

Then, install Crane components with pip install. DOCSTODO

>>> (crane) $ pip install crane-cli
---> 100%

Install from Source¶

First, create virtual environment as in crane-cli

Then, clone the Crane Github repository, and install crane.

>>> (crane) $ git clone https://github.com/friendliai/crane.git
>>> (crane) $ make install-all
---> *

# install crane bootstrap and docker images
>>> (crane) $ make install-node
# install crane cli
>>> (crane) $ make install-cli
# install crane development & testing tools
>>> (crane) $ make install-dev