Merge pull request #163 from PySpur-Dev/feat/package

Feat/package

.env.example

@@ -14,18 +14,23 @@ VERSION=latest
 # GitHub repository (username/repo-name)
 GITHUB_REPOSITORY=pyspur-dev/pyspur
 
+
+# ======================
+# Application Configuration
+# ======================
+
+# Application Host Configuration
+# This is the host that the application will be running on
+# By default, the application will be running on
+
+PYSPUR_HOST=0.0.0.0
+PYSPUR_PORT=6080
+
+
 # Backend Configuration
-BACKEND_PORT=8000
-BACKEND_HOST=0.0.0.0
 DEBUG=False
 
 # Frontend Configuration
-FRONTEND_PORT=3000
-FRONTEND_HOST=0.0.0.0
-
-# Application Port Configuration
-# This is the port that will be exposed to access the PySpur application
-PYSPUR_PORT=6080
 
 # ======================
 # Database Settings

.github/workflows/release.yml

@@ -2,7 +2,7 @@ name: Release
 
 on:
   release:
-    types: [published]
+    types: [created]
 
 env:
   REGISTRY: ghcr.io
@@ -35,6 +35,11 @@ jobs:
           git commit -m "chore: update version to ${{ steps.get-version.outputs.version }}"
           git push
 
+      - name: Update release tag
+        run: |
+          git tag -f ${{ github.event.release.tag_name }}
+          git push --force origin ${{ github.event.release.tag_name }}
+
   build-and-push-docker:
     needs: update-version
     runs-on: ubuntu-latest

Dockerfile.backend

@@ -4,10 +4,13 @@ RUN apt-get update && apt-get install -y \
     gcc \
     curl \
     && rm -rf /var/lib/apt/lists/*
-
+RUN pip install uv
 WORKDIR /pyspur/backend
 COPY backend/pyproject.toml .
-RUN pip install .
+RUN uv pip compile pyproject.toml > requirements.txt && \
+    uv pip install --system --no-cache-dir -r requirements.txt && \
+    rm requirements.txt
+
 
 # Development stage
 FROM base AS development
@@ -30,4 +33,5 @@ COPY backend/ .
 RUN mkdir -p /pyspur/backend/pyspur/static
 RUN rm -rf /pyspur/backend/pyspur/static/*
 COPY --from=frontend-builder /pyspur/frontend/out/ /pyspur/backend/pyspur/static/
+COPY .env.example /pyspur/backend/pyspur/templates/.env.example
 # Production-specific instructions here 

README.md

@@ -81,66 +81,142 @@ https://github.com/user-attachments/assets/5bef7a16-ef9f-4650-b385-4ea70fa54c8a
 
 # ⚡ Quick start
 
-You can launch PySpur using pre-built docker images in the following steps:
+## Option A: Using `pyspur` Python Package (Experimental)
 
-1. **Clone the repository:**
+This is the quickest way to get started. Python 3.12 or higher is required.
+
+1. **Install PySpur:**
     ```sh
-    git clone https://github.com/PySpur-com/pyspur.git
-    cd pyspur
+    pip install pyspur
     ```
 
-2. **Create a .env file:**
-
-    Create a `.env` file at the root of the project. You may use `.env.example` as a starting point.
+2. **Initialize a new project:**
     ```sh
-    cp .env.example .env
+    pyspur init my-project
+    cd my-project
     ```
-    **Please go through the .env file and change configs wherver necessary**
-    **If you plan to use third party model providers, please add their API keys in the .env file in this step**.
-
-3. **Start the docker services:**
+    This will create a new directory with a `.env` file.
 
+3. **Start the server:**
     ```sh
-    docker compose -f ./docker-compose.prod.yml up --build -d
+    pyspur serve --sqlite
     ```
+    By default, this will start PySpur app at `http://localhost:6080` using a sqlite database.
+    We recommend you configure a postgres instance URL in the `.env` file to get a more stable experience.
 
-    This will start a local instance of PySpur that will store spurs and other state information in a postgres database. A local postgres service is used by default. Override `POSTGRES_*` variables in the `.env` file to use an external postgres database.
+4. **[Optional] Customize Your Deployment:**
+    You can customize your PySpur deployment in two ways:
+
+    a. **Through the app** (Recommended):
+       - Navigate to the API Keys tab in the app
+       - Add your API keys for various providers (OpenAI, Anthropic, etc.)
+       - Changes take effect immediately
 
-4. **Access the portal:**
+    b. **Manual Configuration**:
+       - Edit the `.env` file in your project directory
+       - It is recommended to configure a postgres database in .env for more reliability
+       - Restart the app with `pyspur serve`. Add `--sqlite` if you are not using postgres
 
-    Go to `http://localhost:6080/` in your browser.
+## Option B: Using Docker (Recommended)
+
+This is the recommended way for production deployments:
+
+1. **Install Docker:**
+    First, install Docker by following the official installation guide for your operating system:
+    - [Docker for Linux](https://docs.docker.com/engine/install/)
+    - [Docker Desktop for Mac](https://docs.docker.com/desktop/install/mac-install/)
+
+2. **Create a PySpur Project:**
+    Once Docker is installed, create a new PySpur project with:
+    ```sh
+    curl -fsSL https://raw.githubusercontent.com/PySpur-com/pyspur/main/start_pyspur_docker.sh | bash -s pyspur-project
+    ```
+    This will:
+    - Start a new PySpur project in a new directory called `pyspur-project`
+    - Set up the necessary configuration files
+    - Start PySpur app automatically backed by a local postgres docker instance
+
+3. **Access PySpur:**
+    Go to `http://localhost:6080` in your browser.
+
+4. **[Optional] Customize Your Deployment:**
+    You can customize your PySpur deployment in two ways:
+
+    a. **Through the app** (Recommended):
+       - Navigate to the API Keys tab in the app
+       - Add your API keys for various providers (OpenAI, Anthropic, etc.)
+       - Changes take effect immediately
+
+    b. **Manual Configuration**:
+       - Edit the `.env` file in your project directory
+       - Restart the services with:
+         ```sh
+         docker compose up -d
+         ```
+
+That's it! Click on "New Spur" to create a workflow, or start with one of the stock templates.
 
+# 🛠️ PySpur Development Setup
+#### [ Instructions for development on Unix-like systems. Development on Windows/PC not supported ]
 
-Set up is completed. Click on "New Spur" to create a workflow, or start with one of the stock templates.
+For development, follow these steps:
 
+1. **Clone the repository:**
+    ```sh
+    git clone https://github.com/PySpur-com/pyspur.git
+    cd pyspur
+    ```
 
-5. **[Optional] Manage your LLM provider keys from the app:**
+2. **Launch using docker-compose.dev.yml:**
+    ```sh
+    docker compose -f docker-compose.dev.yml up --build -d
+    ```
+    This will start a local instance of PySpur with hot-reloading enabled for development.
 
-   Once PySpur app is running you can manage your LLM provider keys through the portal:
+3. **Customize your setup:**
+    Edit the `.env` file to configure your environment. By default, PySpur uses a local PostgreSQL database. To use an external database, modify the `POSTGRES_*` variables in `.env`.
 
-   <img width="1913" alt="image" src="https://github.com/user-attachments/assets/32fe79f1-f518-4df5-859c-1d1c0fc0570e" />
+# 🦙 Using PySpur with Ollama (Local Models)
 
-   Select API keys tab
+PySpur can work with local models served using Ollama.
 
-   <img width="441" alt="image" src="https://github.com/user-attachments/assets/cccc7e27-c10b-4f3a-b818-3b65c55f4170" />
+Steps to configure PySpur to work with Ollama running on the same host.
 
-   Enter your provider's key and click save (save button will appear after you add/modify a key)
+### 1. Configure Ollama
+To ensure Ollama API is reachable from PySpur, we need to start the Ollama service with environment variable `OLLAMA_HOST=0.0.0.0` . This allows requests coming from PySpur docker's bridge network to get through to Ollama.
+An easy way to do this is to launch the ollama service with the following command:
+```sh
+OLLAMA_HOST="0.0.0.0" ollama serve
+```
 
-   <img width="451" alt="image" src="https://github.com/user-attachments/assets/e35ba2bb-4c60-4b13-9a8d-cc47cac45375" />
+### 2. Update the PySpur .env file
+Next up we need to update the `OLLAMA_BASE_URL` environment value in the `.env` file.
+If your Ollama port is 11434 (the default port), then the entry in `.env` file should look like this:
+```sh
+OLLAMA_BASE_URL=http://host.docker.internal:11434
+```
+(Please make sure that there is no trailing slash in the end!)
 
+In PySpur's set up, `host.docker.internal` refers to the host machine where both PySpur and Ollama are running.
 
-# 🛠️ PySpur Development Setup
-#### [ Instructions for development on Unix-like systems. Development on Windows/PC not tested ]
+### 3. Launch the PySpur app
+Follow the usual steps to launch the PySpur app, starting with the command:
+```sh
+docker compose -f docker-compose.prod.yml up --build -d
+```
 
-The steps for dev setup are same as above, except for step 3: we launch the app in the dev mode instead
+If you wish to do PySpur development with ollama please run the following command instead of above:
+```sh
+docker compose -f docker-compose.yml up --build -d
+```
 
-3. **Start the docker services:**
 
-    ```sh
-    docker compose up --build -d
-    ```
+### 4. Using Ollama models in the app
+You will be able to select Ollama models [`ollama/llama3.2`, `ollama/llama3`, ...] from the sidebar for LLM nodes.
+Please make sure the model you select is explicitly downloaded in ollama. That is, you will need to manually manage these models via ollama. To download a model you can simply run `ollama pull <model-name>`.
 
-    This will start a local instance of PySpur that will store spurs and other state information in a postgres database. A local postgres service is used by default. Override `POSTGRES_*` variables in the `.env` file to use an external postgres database.
+## Note on supported models
+PySpur only works with models that support structured-output and json mode. Most newer models should be good, but it would still be good to confirm this from Ollama documentation for the model you wish to use.
 
 # ⭐ Support us
 
@@ -171,4 +247,4 @@ You can support us in our work by leaving a star! Thank you!
 - [ ] Generate Spurs via AI
 
 Your feedback will be massively appreciated.
-Please [tell us](mailto:founders@pyspur.dev?subject=Feature%20Request&body=I%20want%20this%20feature%3Ai) which features on that list you like to see next or request entirely new ones.
+Please [tell us](mailto:founders@pyspur.dev?subject=Feature%20Request&body=I%20want%20this%20feature%3Ai) which features on that list you like to see next or request entirely new ones.

backend/pyproject.toml

@@ -66,12 +66,16 @@ dependencies = [
     "httpx[http2]==0.27.2",
     "sendgrid==6.11.0",
     "resend==2.6.0",
+    "typer[all]==0.9.0",
 ]
 
 [project.urls]
 Repository = "https://github.com/pyspur-dev/pyspur"
 Documentation = "https://docs.pyspur.dev"
 
+[project.scripts]
+pyspur = "pyspur.cli:main"
+
 [project.optional-dependencies]
 dev = [
     "pytest>=7.0",
@@ -85,6 +89,11 @@ dev = [
 [tool.hatch.build.targets.wheel]
 universal = false
 packages = ["pyspur"]
+zip-safe = false
+
+[tool.hatch.build.targets.wheel.force-include]
+"pyspur/templates" = "pyspur/templates/"
+"pyspur/static" = "pyspur/static/"
 
 [tool.ruff]
 line-length = 100
@@ -94,10 +103,14 @@ target-version = "py312"
 select = ["E", "F", "I", "N", "W", "B", "C", "D"]
 ignore = [
     "D100",  # Missing docstring in public module
+    "D101",  # Missing docstring in public class
+    "D102",  # Missing docstring in public method
+    "D103",  # Missing docstring in public function
     "D104",  # Missing docstring in public package
+    "D105",  # Missing docstring in magic method
     "D107",  # Missing docstring in __init__
-    "D203",  # Ignore 'incorrect-blank-line-before-class' in favor of D211
-    "D213",  # Ignore 'multi-line-summary-second-line' in favor of D212
+    "I001",  # Import block is un-sorted or un-formatted
+    "E402",  # Module level import not at top of file
 ]
 
 [tool.black]

backend/pyspur/api/api_app.py

@@ -0,0 +1,41 @@
+from fastapi import FastAPI
+
+from ..nodes.registry import NodeRegistry
+
+NodeRegistry.discover_nodes()
+
+from ..integrations.google.auth import router as google_auth_router
+from .dataset_management import router as dataset_management_router
+from .evals_management import router as evals_management_router
+from .file_management import router as file_management_router
+from .key_management import router as key_management_router
+from .node_management import router as node_management_router
+from .openai_compatible_api import router as openai_compatible_api_router
+from .output_file_management import router as output_file_management_router
+from .rag_management import router as rag_management_router
+from .run_management import router as run_management_router
+from .template_management import router as template_management_router
+from .workflow_management import router as workflow_management_router
+from .workflow_run import router as workflow_run_router
+
+# Create a sub-application for API routes
+api_app = FastAPI(
+    docs_url="/docs",
+    redoc_url="/redoc",
+    title="PySpur API",
+    version="1.0.0",
+)
+
+api_app.include_router(node_management_router, prefix="/node")
+api_app.include_router(workflow_management_router, prefix="/wf")
+api_app.include_router(workflow_run_router, prefix="/wf")
+api_app.include_router(dataset_management_router, prefix="/ds")
+api_app.include_router(run_management_router, prefix="/run")
+api_app.include_router(output_file_management_router, prefix="/of")
+api_app.include_router(key_management_router, prefix="/env-mgmt")
+api_app.include_router(template_management_router, prefix="/templates")
+api_app.include_router(openai_compatible_api_router, prefix="/api")
+api_app.include_router(evals_management_router, prefix="/evals")
+api_app.include_router(google_auth_router, prefix="/google")
+api_app.include_router(rag_management_router, prefix="/rag")
+api_app.include_router(file_management_router, prefix="/files")