Setup everything for sotopia in one script

.github/workflows/tests.yml

@@ -17,19 +17,27 @@ on:
 
 jobs:
   Pytest:
-    runs-on: ubuntu-latest
     strategy:
       max-parallel: 5
+      matrix:
+        os: [ubuntu-latest, macos-13]
+
+    runs-on: ${{ matrix.os }}
 
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
     - name: Set up Python 3.11
       uses: actions/setup-python@v4
       with:
         python-version: 3.11.2
+    - name: Set up Docker
+      if: runner.os == 'ubuntu-latest'
+      uses: docker-practice/actions-setup-docker@master
+      timeout-minutes: 12
+    - name: Install Poetry
+      uses: abatilo/actions-poetry@v2
     - name: Install dependencies
       run: |
-        curl -sSL https://install.python-poetry.org | python3
         poetry lock
         poetry install --with test -E chat
     - name: Test with pytest

.gitignore

@@ -161,3 +161,5 @@ backup/*
 node_modules/*
 docs/.next/*
 docs/node_modules/*
+
+redis-data/*

README.md

@@ -82,7 +82,7 @@ For some experiments, TogetherAI key is required to run the code. Please set the
 conda env config vars set TOGETHER_API_KEY=your_key
 ```
 
-A redis-stack server is required to run the code. Please follow the [instruction](https://redis.io/docs/stack/get-started/install/docker/) to start a redis-stack server or use an existing server. You can also check [Q&A](/docs/all_the_issues.md) to initiate the redis server with the Sotopia data.
+A redis-stack server is required to run the code. Please follow the [instruction](https://redis.io/docs/stack/get-started/install/docker/) to start a redis-stack server or use an existing server. You can also check [Q&A](/docs/troubleshooting.md) to initiate the redis server with the Sotopia data.
 
 The `REDIS_OM_URL` need to be set before loading and saving agents:
 ```bash
@@ -156,7 +156,7 @@ To run a large batch of environments, you can change the `ENV_IDS` parameter in
 ## Getting access to your simulation
 After running experiments, you can go to the `examples/redis_stats.ipynb` notebook to check the existing episodes (Episode Log section), as well as calculate the performance.
 
-For the original Sotopia simulation in our paper's experiments, you can find how to get them in the [Q&A](/docs/all_the_issues.md) section in the `./docs` folder.
+For the original Sotopia simulation in our paper's experiments, you can find how to get them in the [Q&A](/docs/troubleshooting.md) section in the `./docs` folder.
 
 ## Adding new characters and environments
 You can use the following function with the `**kwargs` being the properties of the `AgentProfile` class. This is the same for the scenarios/environments.

docs/pages/_meta.json

@@ -7,6 +7,14 @@
         "title": "Documentation",
         "type": "menu",
         "items": {
+            "agents": {
+                "title": "Agents",
+                "href": "/agents"
+            },
+            "environments": {
+                "title": "Environments",
+                "href": "/environments"
+            },
             "examples": {
                 "title": "Examples",
                 "href": "/examples"

docs/pages/index.mdx

@@ -107,7 +107,7 @@ Redis stack is a required dependency for using Sotopia. There are two ways to se
       <AccordionItem value="item-1">
         <AccordionTrigger>Docker is my thing.</AccordionTrigger>
         <AccordionContent>
-        Please follow the [instruction](https://redis.io/docs/stack/get-started/install/docker/) to start a redis-stack server or use an existing server. You can also check [Q&A](/docs/all_the_issues.md) to initiate the redis server with the Sotopia data.
+        Please follow the [instruction](https://redis.io/docs/stack/get-started/install/docker/) to start a redis-stack server or use an existing server. You can also check [Q&A](/docs/troubleshooting.md) to initiate the redis server with the Sotopia data.
 
         The `REDIS_OM_URL` need to be set before loading and saving agents:
         ```bash

docs/pages/all_the_issues.md → docs/pages/troubleshooting.md

@@ -1,4 +1,4 @@
-# Q&A
+# Troubleshooting
 ## Missing episodes
 
 Large batch size may cause some episodes to be skipped. This is due to the fact that the server may not be able to handle the load. Try reducing the batch size. But you can also use the script in `examples/fix_missing_episodes.py` to fix the missing episodes.

pyproject.toml

@@ -15,7 +15,6 @@ langchain = "~0.2.5"
 rich = "^13.6.0"
 PettingZoo = "1.24.3"
 redis-om = "^0.2.1"
-types-tqdm = "*"
 gin-config = "^0.5.0"
 absl-py = "^2.0.0"
 together = "^0.2.4"
@@ -42,6 +41,8 @@ pandas = { version = "^2.1.1", optional = true }
 pre-commit = "*"
 nbmake = "*"
 types-setuptools = "*"
+types-requests = "*"
+types-tqdm = "*"
 ruff = "*"
 
 [tool.poetry.group.test.dependencies]
@@ -79,4 +80,4 @@ testpaths = ["tests"]
 python_files = "test_*.py"
 
 [tool.poetry.scripts]
-sotopia_benchmark = "sotopia.benchmark:app"
+sotopia = "sotopia.cli:app"

sotopia/cli/__init__.py

@@ -0,0 +1,5 @@
+from .app import app
+from .install import install
+from .benchmark import benchmark
+
+__all__ = ["app", "install", "benchmark"]

sotopia/cli/app.py

@@ -0,0 +1,3 @@
+from typer import Typer
+
+app = Typer()

sotopia/cli/benchmark/__init__.py

@@ -0,0 +1,3 @@
+from .benchmark import benchmark
+
+__all__ = ["benchmark"]

sotopia/benchmark/cli.py → sotopia/cli/benchmark/benchmark.py

@@ -34,8 +34,7 @@
 
 import typer
 from pathlib import Path
-
-app = typer.Typer()
+from ..app import app
 
 
 def check_existing_episodes(
@@ -264,7 +263,7 @@ def _set_up_logs(
 
 
 @app.command()
-def cli(
+def benchmark(
     model: str = typer.Option(..., help="The language model you want to benchmark."),
     partner_model: str = typer.Option(
         "together_ai/meta-llama/Llama-3-70b-chat-hf",
@@ -297,7 +296,3 @@ def cli(
         verbose=False,
         push_to_db=True,
     )
-
-
-if __name__ == "__main__":
-    app()

sotopia/cli/install/__init__.py

@@ -0,0 +1,3 @@
+from .install import install
+
+__all__ = ["install"]