Launcher for hypha services
Project description
Work In Progress
Features
- CLI for downloading model from s3 and pulling docker image of triton server
- Support different container engines
- Docker
- Apptainer
- Support different compute environments
- Local
- Slurm
Installation
pip install hypha-launcher
Usage
$ hypha-launcher
NAME
hypha-launcher
SYNOPSIS
hypha-launcher - GROUP | COMMAND | VALUE
GROUPS
GROUP is one of the following:
container_engine
Container engine abstraction. Provides a common interface to container engines, such as docker, apptainer, podman, etc.
COMMANDS
COMMAND is one of the following:
download_models_from_s3
Download models from S3
pull_image
run_launcher_server
Start a launcher server, run in the login node of HPC.
run_worker
Run a worker server, run in the compute node of HPC.
VALUES
VALUE is one of the following:
debug
store_dir
Download model from s3
$ hypha-launcher --store-dir="./triton_store" - download_models_from_s3 bioengine-model-runner.* --n_parallel=5
Pull docker image of triton server
$ hypha-launcher --store-dir="./triton_store" - pull_image
Start launcher
On local machine:
$ hypha-launcher --store-dir="./triton_store/" - run_launcher_server
On HPC(Slurm):
$ hypha-launcher --store-dir="./triton_store/" - run_launcher_server --slurm-settings='{"account": "your-account", "gpus_per_node": "V100:1", "time": "01:00:00"}'
TODO
- Download model from s3
- Pull docker image of triton server
- Run triton server
- Register service on hypha
- Conmmunicate with triton server
- Test on HPC
- Job management(Auto stop and restart)
- Load balancing
- Documentation
Development
Install the package in editable mode with the following command:
pip install -e .
pip install -r requirements-dev.txt
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
hypha-launcher-0.1.0.tar.gz
(13.5 kB
view hashes)
Built Distribution
Close
Hashes for hypha_launcher-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e714c63d4e96f70149ec168fcd19724bedf1346341b1bd88cdba59376d3e1d48 |
|
MD5 | d0a7d927d4f1be1c428a55a7c0fece38 |
|
BLAKE2b-256 | b050655d725a945b6788574112da49a76012fcb5bb381d90ba632c8f54f49d62 |