You can install the package via pip once it's published:
pip install scystream-sdkOne of the central concepts of scystream are the so-called Compute Blocks.
A Compute Block describes an independent programm, that acts as some kind of worker which will be scheduled using the scystream-core application. This worker executes a task (e.g. a NLP task, a crawling task).
This SDK aims to provide helper functions and all other requirements you need to implement a custom Compute Block on your own.
Each worker can have multiple entrypoints, each aiming to solve one task. These entrypoints can be configured from the outside using the Settings. These are basically ENV-Variables, which will be parsed & validated using pydantic.
Find the whole Documentation here!
- Create a venv and use it
python3 -m venv .venv
source .venv/bin/activate- Install the package within the venv
Note
This will also install all the install_requirements from the setup.py
pip install -e .- Install the dev dependencies
pip install -e ".[dev]"- Develop!
To run all the tests run the following command:
python3 -m unittest discover -s tests