D3v (#1398)

[emrgnt-cmplxty]

• Feature/add zerox parser (Feature/add zerox parser #1396)

• Add KG tests (Add KG tests #1351)

• cli tests

• add sdk tests

• typo fix

• change workflow ordering

• add collection integration tests (add collection integration tests #1352)

• bump pkg

• remove workflows

• fix sdk test port

• fix delete collection return check

• Fix document info serialization (Fix document info serialization for Hatchet orchestration #1353)

• Update integration-test-workflow-debian.yml

• pre-commit

• slightly modify

• up

• up

• smaller file

• up

• typo, change order

• up

• up

• change order

---

• add graphrag docs (add graphrag docs #1362)

• add documentation

• up

• Update js/sdk/src/models.tsx

• pre-commit

---

• Concurrent index creation, allow -1 for paginated entries (Concurrent index creation, allow -1 for paginated entries on documents overview #1363)

• update webdev-template for current next.js and r2r-js sdk (update webdev-template for current next.js and r2r-js sdk #1218)

• Feature/extend integration tests rebased (Feature/extend integration tests rebased #1361)

• cleanups

• add back overzealous edits

• extend workflows

• fix full setup

• simplify cli

• add ymls

• rename to light

• try again

• start light

• add cli tests

• fix

• fix

• testing..

• trying complete matrix testflow

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• up

• up

• up

• All actions

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• try offic pgvec formula

• sudo make

• sudo make

• push and pray

• push and pray

• add new actions

• add new actions

• docker push & pray

• inspect manifests during launch

• inspect manifests during launch

• inspect manifests during launch

• inspect manifests during launch

• setup docker

• setup docker

• fix default

• fix default

• Feature/rebase to r2r vars (Feature/rebase to r2r vars #1364)

• cleanups

• add back overzealous edits

• extend workflows

• fix full setup

• simplify cli

• add ymls

• rename to light

• try again

• start light

• add cli tests

• fix

• fix

• testing..

• trying complete matrix testflow

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• up

• up

• up

• All actions

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• try offic pgvec formula

• sudo make

• sudo make

• push and pray

• push and pray

• add new actions

• add new actions

• docker push & pray

• inspect manifests during launch

• inspect manifests during launch

• inspect manifests during launch

• inspect manifests during launch

• setup docker

• setup docker

• fix default

• fix default

• make changes

• update the windows workflow

• update the windows workflow

• remove extra workflows for now

• bump pkg

• push and pray

• revive full workflow

• revive full workflow

• revive full workflow

• revive full workflow

• revive full workflow

• revive full workflow

• revive full workflow

• revive full workflow

• revive tests

• revive tests

• revive tests

• revive tests

• update tests

• fix typos (fix typos #1366)

• update tests

• up

• up

• up

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• Add ingestion concurrency limit (Add ingestion concurrency limit #1367)

• up

• up

• up

---

• tweaks and fixes

• Fix Ollama Tool Calling (Fix Ollama Tool Calling #1372)

• Update graphrag.mdx

• Fix Ollama tool calling

---

• Clean up Docker Compose (Clean up Docker Compose #1368)

• Fix hatchet, dockerfile

• Update compose

• point to correct docker image

• Fix bug in deletion, better validation error handling (Fix bug in deletion, better validation error handling #1374)

• Update graphrag.mdx

• Fix bug in deletion, better validation error handling

---

• vec index creation endpoint (vec index creation endpoint #1373)

• Update graphrag.mdx

• upload files

• create vector index endpoint

• add to fastapi background task

• pre-commit

• move logging

• add api spec, support for all vecs

• pre-commit

• add workflow

• Modify KG Endpoints and update API spec (Modify KG Endpoints and update API spec #1369)

• Update graphrag.mdx

• modify API endpoints and update documentation

• Update ingestion_router.py

• try different docker setup (try different docker setup #1371)

• try different docker setup

• action

• add login

• add full

• update action

• cleanup upload script

• cleanup upload script

• tweak action

• tweak action

• tweak action

• tweak action

• tweak action

• tweak action

• Nolan/ingest chunks js (Nolan/ingest chunks js #1375)

• Update graphrag.mdx

• Clean up ingest chunks, add to JS SDK

• Update JS docs

---

• up (Change vector index name #1376)

• Bump JS package (Bump JS Package #1378)

• add conversation

• checkin progress

• checkin progress

• Fix Create Graph (Fix Create Graph #1379)

• up

• up

• modify assertion

• up

• up

• increase entity limit

• changing aristotle back to v2

• pre-commit

• typos

• add test_ingest_sample_file_2_sdk

• Update server.py

• checkin progress

• up

• update

• Graphrag docs (Graphrag docs #1382)

• add docs and refine code

• add python SDK documentation

• up

• update

• checkin

• up

• cleanup

• working sync logging

• test conversation history

• fix runner tests, rename CHUNKS to chunks

• adding zerox parser

---

• Nolan/update hatchet (Nolan/update hatchet #1397)

• Move Hatchet to latest

• Update js package-lock

---

• Add hatchet logging (Add hatchet logging #1391)

• Add KG tests (Add KG tests #1351)

• cli tests

• add sdk tests

• typo fix

• change workflow ordering

• add collection integration tests (add collection integration tests #1352)

• bump pkg

• remove workflows

• fix sdk test port

• fix delete collection return check

• Fix document info serialization (Fix document info serialization for Hatchet orchestration #1353)

• Update integration-test-workflow-debian.yml

• pre-commit

• slightly modify

• up

• up

• smaller file

• up

• typo, change order

• up

• up

• change order

---

• add graphrag docs (add graphrag docs #1362)

• add documentation

• up

• Update js/sdk/src/models.tsx

• pre-commit

---

• Concurrent index creation, allow -1 for paginated entries (Concurrent index creation, allow -1 for paginated entries on documents overview #1363)

• update webdev-template for current next.js and r2r-js sdk (update webdev-template for current next.js and r2r-js sdk #1218)

• Feature/extend integration tests rebased (Feature/extend integration tests rebased #1361)

• cleanups

• add back overzealous edits

• extend workflows

• fix full setup

• simplify cli

• add ymls

• rename to light

• try again

• start light

• add cli tests

• fix

• fix

• testing..

• trying complete matrix testflow

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• up

• up

• up

• All actions

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• try offic pgvec formula

• sudo make

• sudo make

• push and pray

• push and pray

• add new actions

• add new actions

• docker push & pray

• inspect manifests during launch

• inspect manifests during launch

• inspect manifests during launch

• inspect manifests during launch

• setup docker

• setup docker

• fix default

• fix default

• Feature/rebase to r2r vars (Feature/rebase to r2r vars #1364)

• cleanups

• add back overzealous edits

• extend workflows

• fix full setup

• simplify cli

• add ymls

• rename to light

• try again

• start light

• add cli tests

• fix

• fix

• testing..

• trying complete matrix testflow

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• cleanup matrix logic

• up

• up

• up

• All actions

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• rename to runner

• try offic pgvec formula

• sudo make

• sudo make

• push and pray

• push and pray

• add new actions

• add new actions

• docker push & pray

• inspect manifests during launch

• inspect manifests during launch

• inspect manifests during launch

• inspect manifests during launch

• setup docker

• setup docker

• fix default

• fix default

• make changes

• update the windows workflow

• update the windows workflow

• remove extra workflows for now

• bump pkg

• push and pray

• revive full workflow

• revive full workflow

• revive full workflow

• revive full workflow

• revive full workflow

• revive full workflow

• revive full workflow

• revive full workflow

• revive tests

• revive tests

• revive tests

• revive tests

• update tests

• fix typos (fix typos #1366)

• update tests

• up

• up

• up

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• bump max connections

• Add ingestion concurrency limit (Add ingestion concurrency limit #1367)

• up

• up

• up

---

• tweaks and fixes

• Fix Ollama Tool Calling (Fix Ollama Tool Calling #1372)

• Update graphrag.mdx

• Fix Ollama tool calling

---

• Clean up Docker Compose (Clean up Docker Compose #1368)

• Fix hatchet, dockerfile

• Update compose

• point to correct docker image

• Fix bug in deletion, better validation error handling (Fix bug in deletion, better validation error handling #1374)

• Update graphrag.mdx

• Fix bug in deletion, better validation error handling

---

• vec index creation endpoint (vec index creation endpoint #1373)

• Update graphrag.mdx

• upload files

• create vector index endpoint

• add to fastapi background task

• pre-commit

• move logging

• add api spec, support for all vecs

• pre-commit

• add workflow

• Modify KG Endpoints and update API spec (Modify KG Endpoints and update API spec #1369)

• Update graphrag.mdx

• modify API endpoints and update documentation

• Update ingestion_router.py

• try different docker setup (try different docker setup #1371)

• try different docker setup

• action

• add login

• add full

• update action

• cleanup upload script

• cleanup upload script

• tweak action

• tweak action

• tweak action

• tweak action

• tweak action

• tweak action

• Nolan/ingest chunks js (Nolan/ingest chunks js #1375)

• Update graphrag.mdx

• Clean up ingest chunks, add to JS SDK

• Update JS docs

---

• up (Change vector index name #1376)

• Bump JS package (Bump JS Package #1378)

• Fix Create Graph (Fix Create Graph #1379)

• up

• up

• modify assertion

• up

• up

• increase entity limit

• changing aristotle back to v2

• pre-commit

• typos

• add test_ingest_sample_file_2_sdk

• Update server.py

• add docs and refine code

• add python SDK documentation

• up

• add logs

• clean

• rm vq

• rm conflicts

• pre-commit

• up

• add logging

• update logs

• up

• up

• Update kg_service.py

---

• feat: Add delete_node_via_document_id method to KGProvider (Feature/ delete node via document #1387)

---

---

Important

Add Zerox PDF parser, update to Python 3.11, enhance CLI documentation, and introduce comprehensive integration tests.

• Features:
  • Add Zerox PDF parser in pdf_parser.py for advanced PDF parsing.
  • Introduce run_ingest_with_zerox.py script for ingesting documents using Zerox.
  • Add run-script-zerox-tests GitHub action for testing Zerox ingestion.
• Python Version:
  • Update Python version to 3.11 in Dockerfile, Dockerfile.unstructured, and several GitHub actions.
• Documentation:
  • Add CLI documentation for ingestion, graph management, and server management in docs/documentation/cli/.
• Testing:
  • Add integration tests for Zerox ingestion in runner_scripts.py and runner_sdk.py.
  • Update existing tests to accommodate new features and Python version changes.
• Configuration:
  • Update r2r.toml and pyproject.toml to reflect new parser and Python version.
• Miscellaneous:
  • Add HatchetLogger for improved logging in base_utils.py.
  • Modify vector quantization settings in vector.py and postgres.py.

^{This description was created by for fc659aa. It will automatically update as commits are pushed.}

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
yc_demo	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 15, 2024 2:16am
yc-demo	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 15, 2024 2:16am

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
recommendation_platform	⬜️ Ignored (Inspect)			Oct 15, 2024 2:16am

[ellipsis-dev]

👍 Looks good to me! Reviewed everything up to fc659aa in 1 minute and 31 seconds

More details

• Looked at 5144 lines of code in 96 files
• Skipped 2 files when reviewing.
• Skipped posting 1 drafted comments based on config settings.

1. services/unstructured/Dockerfile.unstructured:1

• Draft comment:
  Consider removing unnecessary packages like libmagic1, pandoc, and libreoffice if they are not required for your application. This can help reduce the image size and potential security vulnerabilities.
• Reason this comment was not posted:
  Confidence changes required: 50%
  The Dockerfile is installing unnecessary packages for the application, which can increase the image size and potentially introduce security vulnerabilities.

Workflow ID: wflow_epnd76fQses1Z454

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.