Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/package #163

Merged
merged 37 commits into from
Feb 18, 2025
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
2b51d29
refactor: move templates inside pyspur
srijanpatel Feb 16, 2025
58be77a
fix: change release trigger type to 'created' and update release tag …
srijanpatel Feb 16, 2025
563d970
Merge remote-tracking branch 'origin/main' into feat/package
srijanpatel Feb 16, 2025
6b0243f
feat: include .env.example in Dockerfile and update pyproject.toml to…
srijanpatel Feb 16, 2025
8965762
feat: add CLI initialization command and .env template handling
srijanpatel Feb 16, 2025
a214fd5
feat: update .env.example and docker-compose to standardize applicati…
srijanpatel Feb 16, 2025
45d85ae
feat: add environment loading and migration handling to CLI server co…
srijanpatel Feb 16, 2025
95e40c2
refactor: organise CLI code
srijanpatel Feb 16, 2025
5d2fae4
docs: update README with quick start guide for PySpur using Python pa…
srijanpatel Feb 16, 2025
b65caea
rename docker-compose.yml to docker-compose.dev.yml
srijanpatel Feb 16, 2025
8ef4319
rename docker-compose.prod.yml to docker-compose.yml
srijanpatel Feb 16, 2025
5f1f7de
docs: update README for improved setup instructions and add start scr…
srijanpatel Feb 17, 2025
b0aa9c8
feat: add SQLite support for local development and update README inst…
srijanpatel Feb 17, 2025
4534cd2
Merge remote-tracking branch 'origin/main' into feat/package
srijanpatel Feb 17, 2025
96ba2b8
remove env loading from database.py
srijanpatel Feb 17, 2025
1a22e1b
refactor: import engine and URL from database module
srijanpatel Feb 17, 2025
e3bfe31
fix: update initialization message to reflect server start command
srijanpatel Feb 17, 2025
18817e1
refactor: replace local templates directory with resource files for t…
srijanpatel Feb 17, 2025
d676bc0
feat: add data directory creation during project initialization
srijanpatel Feb 17, 2025
b9da870
fix: improve error handling by preserving original exception context
srijanpatel Feb 17, 2025
7f1bec3
refactor: remove redundant EVALS_DIR definition in workflow_run.py
srijanpatel Feb 17, 2025
ad22b81
feat: include static files in wheel build for pyspur
srijanpatel Feb 17, 2025
6cc542a
chore: suppress linter warnings temporarily
srijanpatel Feb 17, 2025
b56c707
chore: move API app to its own file
srijanpatel Feb 17, 2025
a4be229
feat: serve static files from a tempdir to handle package scenario
srijanpatel Feb 17, 2025
e2d8738
feat: refactor template file handling and migration script retrieval …
srijanpatel Feb 17, 2025
ff99631
fix: set zip-safe to false for wheel build to ensure template accessi…
srijanpatel Feb 17, 2025
95a5a93
feat: we're a uv company now
srijanpatel Feb 17, 2025
472d57a
fix: update wheel build configuration to correctly include template a…
srijanpatel Feb 18, 2025
9e31fb1
Merge remote-tracking branch 'origin/main' into feat/package
srijanpatel Feb 18, 2025
635127f
fix: add SA compiler for sqlite column comments
srijanpatel Feb 18, 2025
1213fab
Merge remote-tracking branch 'origin/main' into feat/package
srijanpatel Feb 18, 2025
18fe59c
refactor: update PROJECT_ROOT to use Path.cwd() and simplify file pat…
srijanpatel Feb 18, 2025
f417090
feat: enhance run_migrations to support SQLite database creation and …
srijanpatel Feb 18, 2025
897c52d
feat: add PROJECT_ROOT to .env during initialization
srijanpatel Feb 18, 2025
48a82e4
feat: update Google integration to use PROJECT_ROOT for credential fi…
srijanpatel Feb 18, 2025
db4b592
Merge remote-tracking branch 'origin/main' into feat/package
srijanpatel Feb 18, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 13 additions & 8 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -14,18 +14,23 @@ VERSION=latest
# GitHub repository (username/repo-name)
GITHUB_REPOSITORY=pyspur-dev/pyspur


# ======================
# Application Configuration
# ======================

# Application Host Configuration
# This is the host that the application will be running on
# By default, the application will be running on

PYSPUR_HOST=0.0.0.0
PYSPUR_PORT=6080


# Backend Configuration
BACKEND_PORT=8000
BACKEND_HOST=0.0.0.0
DEBUG=False

# Frontend Configuration
FRONTEND_PORT=3000
FRONTEND_HOST=0.0.0.0

# Application Port Configuration
# This is the port that will be exposed to access the PySpur application
PYSPUR_PORT=6080

# ======================
# Database Settings
Expand Down
7 changes: 6 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: Release

on:
release:
types: [published]
types: [created]

env:
REGISTRY: ghcr.io
Expand Down Expand Up @@ -35,6 +35,11 @@ jobs:
git commit -m "chore: update version to ${{ steps.get-version.outputs.version }}"
git push

- name: Update release tag
run: |
git tag -f ${{ github.event.release.tag_name }}
git push --force origin ${{ github.event.release.tag_name }}

build-and-push-docker:
needs: update-version
runs-on: ubuntu-latest
Expand Down
8 changes: 6 additions & 2 deletions Dockerfile.backend
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,13 @@ RUN apt-get update && apt-get install -y \
gcc \
curl \
&& rm -rf /var/lib/apt/lists/*

RUN pip install uv
WORKDIR /pyspur/backend
COPY backend/pyproject.toml .
RUN pip install .
RUN uv pip compile pyproject.toml > requirements.txt && \
uv pip install --system --no-cache-dir -r requirements.txt && \
rm requirements.txt


# Development stage
FROM base AS development
Expand All @@ -30,4 +33,5 @@ COPY backend/ .
RUN mkdir -p /pyspur/backend/pyspur/static
RUN rm -rf /pyspur/backend/pyspur/static/*
COPY --from=frontend-builder /pyspur/frontend/out/ /pyspur/backend/pyspur/static/
COPY .env.example /pyspur/backend/pyspur/templates/.env.example
# Production-specific instructions here
110 changes: 71 additions & 39 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,67 +80,99 @@ https://github.com/user-attachments/assets/5bef7a16-ef9f-4650-b385-4ea70fa54c8a

# ⚡ Quick start

You can launch PySpur using pre-built docker images in the following steps:
## Option A: Using `pyspur` Python Package

1. **Clone the repository:**
This is the quickest way to get started. Python 3.12 or higher is required.

1. **Install PySpur:**
```sh
git clone https://github.com/PySpur-com/pyspur.git
cd pyspur
pip install pyspur
```

2. **Create a .env file:**

Create a `.env` file at the root of the project. You may use `.env.example` as a starting point.
2. **Initialize a new project:**
```sh
cp .env.example .env
pyspur init my-project
cd my-project
```
**Please go through the .env file and change configs wherver necessary**
**If you plan to use third party model providers, please add their API keys in the .env file in this step**.

3. **Start the docker services:**
This will create a new directory with a `.env` file.

3. **Start the server:**
```sh
docker compose -f ./docker-compose.prod.yml up --build -d
pyspur serve --sqlite
```
By default, this will start PySpur app at `http://localhost:6080` using a sqlite database.

This will start a local instance of PySpur that will store spurs and other state information in a postgres database. A local postgres service is used by default. Override `POSTGRES_*` variables in the `.env` file to use an external postgres database.

4. **Access the portal:**

Go to `http://localhost:6080/` in your browser.


Set up is completed. Click on "New Spur" to create a workflow, or start with one of the stock templates.

4. **[Optional] Customize Your Deployment:**
You can customize your PySpur deployment in two ways:

a. **Through the app** (Recommended):
- Navigate to the API Keys tab in the app
- Add your API keys for various providers (OpenAI, Anthropic, etc.)
- Changes take effect immediately

5. **[Optional] Manage your LLM provider keys from the app:**
b. **Manual Configuration**:
- Edit the `.env` file in your project directory
- It is recommended to configure a postgres database in .env for more reliability
- Restart the app with `pyspur serve`. Add `--sqlite` if you are not using postgres

Once PySpur app is running you can manage your LLM provider keys through the portal:
## Option B: Using Docker (Recommended)

<img width="1913" alt="image" src="https://github.com/user-attachments/assets/32fe79f1-f518-4df5-859c-1d1c0fc0570e" />
This is the recommended way for production deployments:

Select API keys tab

<img width="441" alt="image" src="https://github.com/user-attachments/assets/cccc7e27-c10b-4f3a-b818-3b65c55f4170" />

Enter your provider's key and click save (save button will appear after you add/modify a key)

<img width="451" alt="image" src="https://github.com/user-attachments/assets/e35ba2bb-4c60-4b13-9a8d-cc47cac45375" />
1. **Install Docker:**
First, install Docker by following the official installation guide for your operating system:
- [Docker for Linux](https://docs.docker.com/engine/install/)
- [Docker Desktop for Mac](https://docs.docker.com/desktop/install/mac-install/)

2. **Create a PySpur Project:**
Once Docker is installed, create a new PySpur project with:
```sh
curl -fsSL https://raw.githubusercontent.com/PySpur-com/pyspur/main/start_pyspur_docker.sh | bash -s pyspur-project
```
This will:
- Start a new PySpur project in a new directory called `pyspur-project`
- Set up the necessary configuration files
- Start PySpur app automatically backed by a local postgres docker instance

3. **Access PySpur:**
Go to `http://localhost:6080` in your browser.

4. **[Optional] Customize Your Deployment:**
You can customize your PySpur deployment in two ways:

a. **Through the app** (Recommended):
- Navigate to the API Keys tab in the app
- Add your API keys for various providers (OpenAI, Anthropic, etc.)
- Changes take effect immediately

b. **Manual Configuration**:
- Edit the `.env` file in your project directory
- Restart the services with:
```sh
docker compose up -d
```

That's it! Click on "New Spur" to create a workflow, or start with one of the stock templates.

# 🛠️ PySpur Development Setup
#### [ Instructions for development on Unix-like systems. Development on Windows/PC not tested ]

The steps for dev setup are same as above, except for step 3: we launch the app in the dev mode instead
#### [ Instructions for development on Unix-like systems. Development on Windows/PC not supported ]

3. **Start the docker services:**
For development, follow these steps:

1. **Clone the repository:**
```sh
docker compose up --build -d
git clone https://github.com/PySpur-com/pyspur.git
cd pyspur
```

This will start a local instance of PySpur that will store spurs and other state information in a postgres database. A local postgres service is used by default. Override `POSTGRES_*` variables in the `.env` file to use an external postgres database.
2. **Launch using docker-compose.dev.yml:**
```sh
docker compose -f docker-compose.dev.yml up --build -d
```
This will start a local instance of PySpur with hot-reloading enabled for development.

3. **Customize your setup:**
Edit the `.env` file to configure your environment. By default, PySpur uses a local PostgreSQL database. To use an external database, modify the `POSTGRES_*` variables in `.env`.

# 🦙 Using PySpur with Ollama (Local Models)

Expand Down Expand Up @@ -213,4 +245,4 @@ You can support us in our work by leaving a star! Thank you!
- [ ] Generate Spurs via AI

Your feedback will be massively appreciated.
Please [tell us](mailto:founders@pyspur.dev?subject=Feature%20Request&body=I%20want%20this%20feature%3Ai) which features on that list you like to see next or request entirely new ones.
Please [tell us](mailto:founders@pyspur.dev?subject=Feature%20Request&body=I%20want%20this%20feature%3Ai) which features on that list you like to see next or request entirely new ones.
17 changes: 15 additions & 2 deletions backend/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -66,12 +66,16 @@ dependencies = [
"httpx[http2]==0.27.2",
"sendgrid==6.11.0",
"resend==2.6.0",
"typer[all]==0.9.0",
]

[project.urls]
Repository = "https://github.com/pyspur-dev/pyspur"
Documentation = "https://docs.pyspur.dev"

[project.scripts]
pyspur = "pyspur.cli:main"

[project.optional-dependencies]
dev = [
"pytest>=7.0",
Expand All @@ -85,6 +89,11 @@ dev = [
[tool.hatch.build.targets.wheel]
universal = false
packages = ["pyspur"]
zip-safe = false

[tool.hatch.build.targets.wheel.force-include]
"pyspur/templates/*" = "pyspur/templates/"
"pyspur/static/*" = "pyspur/static/"

[tool.ruff]
line-length = 100
Expand All @@ -94,10 +103,14 @@ target-version = "py312"
select = ["E", "F", "I", "N", "W", "B", "C", "D"]
ignore = [
"D100", # Missing docstring in public module
"D101", # Missing docstring in public class
"D102", # Missing docstring in public method
"D103", # Missing docstring in public function
"D104", # Missing docstring in public package
"D105", # Missing docstring in magic method
"D107", # Missing docstring in __init__
"D203", # Ignore 'incorrect-blank-line-before-class' in favor of D211
"D213", # Ignore 'multi-line-summary-second-line' in favor of D212
"I001", # Import block is un-sorted or un-formatted
"E402", # Module level import not at top of file
]

[tool.black]
Expand Down
41 changes: 41 additions & 0 deletions backend/pyspur/api/api_app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
from fastapi import FastAPI

from ..nodes.registry import NodeRegistry

NodeRegistry.discover_nodes()

from ..integrations.google.auth import router as google_auth_router
from .dataset_management import router as dataset_management_router
from .evals_management import router as evals_management_router
from .file_management import router as file_management_router
from .key_management import router as key_management_router
from .node_management import router as node_management_router
from .openai_compatible_api import router as openai_compatible_api_router
from .output_file_management import router as output_file_management_router
from .rag_management import router as rag_management_router
from .run_management import router as run_management_router
from .template_management import router as template_management_router
from .workflow_management import router as workflow_management_router
from .workflow_run import router as workflow_run_router

# Create a sub-application for API routes
api_app = FastAPI(
docs_url="/docs",
redoc_url="/redoc",
title="PySpur API",
version="1.0.0",
)

api_app.include_router(node_management_router, prefix="/node")
api_app.include_router(workflow_management_router, prefix="/wf")
api_app.include_router(workflow_run_router, prefix="/wf")
api_app.include_router(dataset_management_router, prefix="/ds")
api_app.include_router(run_management_router, prefix="/run")
api_app.include_router(output_file_management_router, prefix="/of")
api_app.include_router(key_management_router, prefix="/env-mgmt")
api_app.include_router(template_management_router, prefix="/templates")
api_app.include_router(openai_compatible_api_router, prefix="/api")
api_app.include_router(evals_management_router, prefix="/evals")
api_app.include_router(google_auth_router, prefix="/google")
api_app.include_router(rag_management_router, prefix="/rag")
api_app.include_router(file_management_router, prefix="/files")
Loading