Development

Prerequisites

Start by cloning the project repository:

git clone https://github.com/run-ai/genv.git

Setup

Local

Open a terminal in the project directory.

Warning

We support bash terminals only at the moment. Query your shell using echo $SHELL and run bash if it is not your shell, or use a Docker setup instead.

Install the Python package in editable mode with the following command:

pip install -e .[dev]

Then run the following commands to configure your terminal:

eval "$(genv shell --init)"

You can test it by running:

$ genv
usage: genv [-h] SUBCOMMAND ...

Query and control Genv on this machine or in a cluster
...

To uninstall the package, use the following command:

pip uninstall -y genv

Docker

You can also use a container for the development. This is useful if you are using macOS as Genv is developed for Linux systems and some things are not available in macOS (e.g. /proc filesystem).

First, build the Docker image genv:devel with the following command:

docker build -t genv:devel -f devel/Dockerfile devel

Now, run a development container using the following command from the root directory of the project:

docker run -it --rm --name genv \
    -v $PWD:/root/genv \
    -w /root/genv \
    genv:devel

Note

Pass -v /var/tmp:/var/tmp --pid host if you want to share the state with the host machine or with other containers and -v $HOME/.ssh:/root/.ssh if you want to use remote features as well.

To open another terminal inside the container use:

docker exec -it genv bash

CPU-Only Setup

Some Genv features rely on executing nvidia-smi commands. Those commands will not work if you are developing on a machine without GPUs.

Here is what you will probably see if you will try running nvidia-smi:

$ nvidia-smi
bash: nvidia-smi: command not found

In such cases, it is recommended to use the mock shim of nvidia-smi. Set up your shell with the following command:

export PATH=$PATH:$PWD/devel/shims

Note

If you are using a Docker development setup, your shell should already be set.

Now, execute nvidia-smi once again. This time it should work and you should see an nvidia-smi-like output printed to the screen.

Mock Devices

You can control the mock devices by executing a command similar to the following and specifying the supported GENV_MOCK_* environment variables:

GENV_MOCK_DEVICE_COUNT=4 GENV_MOCK_DEVICE_TOTAL_MEMORY=8g genv devices --reset

Remote Features

If you are working on remote features, you might want to test them on a few remote machines.

However, many times you will not have as many GPU machines as you would like, or SSH access to them. You might also want to work on remote features using only your single CPU machine.

For this case, you can use the genv:sshd Docker image that acts as a remote machine over SSH.

First, build the Docker image genv:devel as it is the base image of genv:sshd and then build the Docker image genv:sshd with the following command:

docker build -t genv:sshd -f devel/sshd.Dockerfile devel

Now, run a container using the following command from the root directory of the project:

docker run -d --rm \
    --name genv-server-1 \
    -p 2221:22 \
    -v $PWD:/root/genv \
            genv:sshd

This command runs a container in the background that is named genv-server-1 and accepts SSH connections on port 2221.

You can rerun this command as many times as you want to simulate more remote machines. Make sure to change the host port each time and also rename the container (or have the container unnamed by omitting the flag --name entirely).

Note

By default, Genv is installed in such containers. If you want to skip the installation, for example when testing remote monitoring on machines without Genv, add -e GENV_INSTALL=0 to the docker run command.

You can open a terminal in such a container using a command similar to the following:

docker exec -it genv-server-1 bash

To terminate such a container, use a command similar to the following:

docker kill genv-server-1

Then, because of how remote features work, you will have to edit the SSH configuration on the host machine to allow simple SSH commands that genv remote uses.

Edit the SSH configuration file by running the following command on the host machine:

vim ~/.ssh/config

Add the following configuration for each of the containers. Make sure to set the correct port for every container:

Host genv-server-1
    Port 2221
    Hostname 127.0.0.1
    User root

Then, test the SSH connectivity using the command:

ssh genv-server-1

Warning

You might need to approve the SSH key of the container on the first time. Type yes if you see a message similar to Are you sure you want to continue connecting (yes/no)?.

Note

You can also control the mock devices by running over SSH what is described here.

After setting up all containers, test your setup with a command similar to the following:

genv remote -H genv-server-1,genv-server-2 devices

Docs

Setup

docker build -t genv:docs -f devel/docs.Dockerfile docs

Build

docker run -it --rm -v $PWD/docs:/docs genv:docs

Python Package

Build

python setup.py sdist bdist_wheel

Note

You might need to upgrade wheel using pip install wheel==0.31.0

Publish to PyPI

python -m twine upload dist/*

Note

You might need to pip install twine

Conda Installation

Setup

Install Conda-build:

conda install conda-build

Clone the Conda feedstock project repository and enter to its directory:

git clone https://github.com/conda-forge/genv-feedstock.git
cd genv-feedstock

Build

conda build .

Note

You can clean the build with conda build purge-all

Install

conda install --use-local genv