This page will help you install Crane on your cluster nodes. Refer to the deploy guide to actually have Crane running.
Machines that directly participate as nodes in the Crane cluster require Docker and the NVIDIA container runtime. Especially, the Docker Engine version must support Docker swarm. The minimum required version of Docker Engine is 20.10.6.
NVIDIA Container Runtime: https://github.com/NVIDIA/nvidia-container-runtime
Crane has four components:
- Crane Core: The core logic of Crane: cluster manager, node manager, gateway, and state DB. Each component runs in a Docker container.
- Crane Admin Daemon: A daemon process named
crane-admindruns on each node in the cluster. The daemon can initialize a new Crane cluster, join/leave an existing Crane cluster, and promote/demote a node's role in the cluster (leader or follower).
- Crane CLI: The Admin and User command-line tools. The Admin CLI is paired with the Crane Admin Daemon, while the User CLI communicates with the Crane gateway to query and set the cluster/job/user state.
- Crane Bootstrap: This is a simple command-line tool to start the
crane-admindprocess on the current node.
Crane can be installed as a system package, via pip, or from source.
As of now, we recommend that you install Crane from source. System packages or pip distributions may lag behind because Crane is under active development.
Crane Core, Admin Daemon, and Bootstrap¶
Currently, system package installs are only supported for Ubuntu bionic.
Install crane components
$ sudo add-apt-repository ppa:friendliai/crane $ sudo apt-get update $ sudo apt-get install crane ---> *
This will install
boot.crane on the node, and configure it as a systemd service. When
crane-admind first starts, it pulls the latest Crane Docker images from Docker Hub.
First, create and activate a virtual environment with python 3.7 or above.
We'll use pyenv in this guide.
You may want to update your
PATH environment variable or initialize venv providers in your shell rc file based on your choice.
# Create virtual environment $ curl https://pyenv.run | bash $ pyenv install 3.7.6 ---> 100% $ pyenv virtualenv 3.7.6 crane $ pyenv activate crane
Then, install Crane components with
pip install. DOCSTODO
>>> (crane) $ pip install crane-cli ---> 100%
Install from Source¶
First, create virtual environment as in crane-cli
Then, clone the Crane Github repository, and install crane.
>>> (crane) $ git clone https://github.com/friendliai/crane.git >>> (crane) $ make install-all ---> *
Custom installation commands. You can install crane for development and testing via
# install crane bootstrap and docker images >>> (crane) $ make install-node # install crane cli >>> (crane) $ make install-cli # install crane development & testing tools >>> (crane) $ make install-dev