diff --git a/.env.template b/.env.template index 28658393..4e173aab 100644 --- a/.env.template +++ b/.env.template @@ -1,26 +1,23 @@ -ENV= development +isDevelopmentMode=enabled +ENV=development OPENAI_API_KEY= -OPENAI_MODEL_REASONING= -# POSTGRES_SERVER=postgresql://postgres:mysecretpassword@host.docker.internal:5432/momentum #for use with wsgl POSTGRES_SERVER=postgresql://postgres:mysecretpassword@localhost:5432/momentum -MONGO_URI= mongodb://127.0.0.1:27017 -MONGODB_DB_NAME= momentum -NEO4J_URI= bolt://127.0.0.1:7687 -NEO4J_USERNAME= neo4j -NEO4J_PASSWORD= mysecretpassword +NEO4J_URI=bolt://127.0.0.1:7687 +NEO4J_USERNAME=neo4j +NEO4J_PASSWORD=mysecretpassword REDISHOST=127.0.0.1 REDISPORT=6379 BROKER_URL=redis://127.0.0.1:6379/0 CELERY_QUEUE_NAME=dev +defaultUsername=defaultuser +PROJECT_PATH=projects +# following are for production mode PORTKEY_API_KEY= GCP_PROJECT= -defaultUsername= defaultuser FIREBASE_SERVICE_ACCOUNT= KNOWLEDGE_GRAPH_URL= GITHUB_APP_ID= GITHUB_PRIVATE_KEY= -PROJECT_PATH=projects -KNOWLEDGE_GRAPH_PROJECT_ID= TRANSACTION_EMAILS_ENABLED= EMAIL_FROM_ADDRESS= RESEND_API_KEY= diff --git a/app/main.py b/app/main.py index 01adcce2..c223c9b2 100644 --- a/app/main.py +++ b/app/main.py @@ -92,7 +92,6 @@ def initialize_database(self): def check_and_set_env_vars(self): required_env_vars = [ "OPENAI_API_KEY", - "OPENAI_MODEL_REASONING", ] for env_var in required_env_vars: if env_var not in os.environ: diff --git a/readme.md b/readme.md index 05fa697f..a2e7f21b 100644 --- a/readme.md +++ b/readme.md @@ -5,73 +5,67 @@


+

+ +
+ App | Documentation | API Reference | Chat with ๐Ÿฅง Repo +
+

+ Apache 2.0 + + + GitHub Repo stars +
Join our Discord +
+ + tweet + +

+ Prompt-To-Agent: Create custom engineering agents for your code

-

-
- Get started! -
- Documentation -
- - -Potpie parses and understands your codebase by building a knowledge graph out of your codeโ€™s components. -It provides pre-built agents that are expert on your codebase to perform common engineering tasks for you, and also provides the platform for you to build your own custom agents. +Potpie is an open-source platform that creates AI agents specialized in your codebase, enabling automated code analysis, testing, and development tasks. By building a comprehensive knowledge graph of your code, Potpie's agents can understand complex relationships and assist with everything from debugging to feature development.

- - Potpie Dashboard - -

- ---- -## Table of Contents +Screenshot 2025-01-09 at 2 18 18โ€ฏPM -- [What are Codebase Agents?](#why-agents) -- [Our Prebuilt Agents](#prebuilt-agents) -- [Tooling](#potpies-tooling-system) -- [Getting Started](#getting-started) -- [Use Cases](#use-cases) -- [Custom Agents](#custom-agents-upgrade) -- [Make Potpie Your Own](#make-potpie-your-own) -- [Contributing](#contributing) -- [License](#license) -- [Contributors](#-thanks-to-all-contributors) - - -## What are Codebase Agents? +

-AI agents are autonomous tools that have the ability to reason, take decisions and perform actions on their own. They are provided with 'tools' that they can use to perform tasks. Agents are iterative in nature and build on top of the results of the previous iteration in order to perform any task assigned to them. +## ๐Ÿ“š Table of Contents +- [๐Ÿฅง Why Potpie?](#why-potpie) +- [๐Ÿค– Our Prebuilt Agents](#prebuilt-agents) +- [๐Ÿ› ๏ธ Tooling](#potpies-tooling-system) +- [๐Ÿš€ Getting Started](#getting-started) +- [๐Ÿ’ก Use Cases](#use-cases) +- [๐Ÿ› ๏ธ Custom Agents](#custom-agents-upgrade) +- [๐ŸŽจ Make Potpie Your Own](#make-potpie-your-own) +- [๐Ÿค Contributing](#contributing) +- [๐Ÿ“œ License](#license) +- [๐Ÿ’ช Contributors](#-thanks-to-all-contributors) -Software development is a similarly iterative process and agents can be used to automate and optimize key aspects of software development. -Things that developers do daily, like debugging, can be broken down into a series of iterative steps that can be automated by agents. -For example, debugging can be broken down into: -1. Understanding the stacktrace -2. Understanding the code around the stacktrace -3. Coming up with a hypothesis -4. Testing the hypothesis -5. Repeating the above steps until the bug is fixed -In order to perform these steps, an agent would need to understand the codebase, the code around the stacktrace, the flow of the code, the project structure etc. +## ๐Ÿฅง Why Potpie? +- ๐Ÿง  **Deep Code Understanding**: Built-in knowledge graph captures relationships between code components +- ๐Ÿค– **Pre-built & Custom Agents**: Ready-to-use agents for common tasks + build your own +- ๐Ÿ”„ **Seamless Integration**: Works with your existing development workflow +- ๐Ÿ“ˆ **Flexible**: Handles codebases of any size or language -Potpie parses your codebase and builds a graph tracking relationships between functions, files, classes, etc. We generate inferences for each node and embed and store it in the graph. This can be used to curate the correct context by performing a similarity search based on users query. The graph can also be queried to understand the code flow, it can be queried to understand the project structure etc. -This allows Potpie's agents to understand the codebase and reason about the code. -## Potpie's Prebuilt Agents +## ๐Ÿค– Potpie's Prebuilt Agents Potpie offers a suite of specialized codebase agents for automating and optimizing key aspects of software development: @@ -81,213 +75,176 @@ Potpie offers a suite of specialized codebase agents for automating and optimizi - **Integration Test Agent**: Generates integration test plans and code for flows to ensure components work together properly. - **Unit Test Agent**: Automatically creates unit test plan and code for individual functions to enhance test coverage. - **LLD Agent**: Creates a low level design for implementing a new feature by providing functional requirements to this agent. +- **Code Generation Agent**: Generates code for new features, refactors existing code, and suggests optimizations. -Potpie's agents leverage tools that interact with your codebase's knowledge graph stored in neo4j. These tools look up project structure, fetch code from github, fetch code flow from graph etc +## ๐Ÿ› ๏ธ Potpie's Tooling System -### Potpie's Tooling System +Potpie provides a set of tools that agents can use to interact with the knowledge graph and the underlying infrastructure: -Potpie provides a set of tools that agents can use to interact with the knowledge graph and the underlying infrastructure. These tools are vital for creating custom agents and for performing highly contextual tasks with precision. - -#### Available Tools: - **get_code_from_probable_node_name**: Retrieves code snippets based on a probable node name. - **get_code_from_node_id**: Fetches code associated with a specific node ID. - **get_code_from_multiple_node_ids**: Retrieves code snippets for multiple node IDs simultaneously. -- **ask_knowledge_graph_queries**: Executes vector similarity searches to obtain relevant information from the knowledge graph. -- **get_nodes_from_tags**: Retrieves nodes tagged with specific keywords from the knowledge graph. -- **get_code_graph_from_node_id/name**: Fetches code graph structures for a specific node ID or name. +- **ask_knowledge_graph_queries**: Executes vector similarity searches to obtain relevant information. +- **get_nodes_from_tags**: Retrieves nodes tagged with specific keywords. +- **get_code_graph_from_node_id/name**: Fetches code graph structures for a specific node. - **change_detection**: Detects changes in the current branch compared to the default branch. - **get_code_file_structure**: Retrieves the file structure of the codebase. - -These tools are the foundation for the custom agents you create, allowing them to intelligently access and manipulate your codebase efficiently. - ---- -## Getting Started - -Refer to the [Getting Started Guide](./GETTING_STARTED.md) for detailed instructions on setting up Potpie and making your first agent work for you! - -Once you have set up Potpie, you can get started with the following steps: - -## Step 1: Logging in to get a bearer token -```bash -curl -X 'POST' \ - 'http://localhost:8001/api/v1/login' \ - -H 'accept: application/json' \ - -H 'Content-Type: application/json' \ - -d '{ - "email": "string", - "password": "string" -}' -``` -## Step 2: Submit a Parsing Request -Replace the repo name and branch name with the repo you want to talk to. -```bash -curl -X 'POST' \ - 'http://localhost:8001/api/v1/parse' \ - -H 'accept: application/json' \ - -H 'Content-Type: application/json' \ - -d '{ - "repo_name": "owner/repo-name", - "branch_name": "branch-name" -}' -``` -## Step 3: Check Parsing Status -Use the project id generated from previous request. -```bash -curl -X 'GET' \ - 'http://localhost:8001/api/v1/parsing-status/project-id' \ - -H 'accept: application/json' -``` -## Step 4: List Available Agents -```bash -curl -X 'GET' \ - 'http://localhost:8001/api/v1/list-available-agents/?list_system_agents=true' \ - -H 'accept: application/json' -``` -## Step 5: Create a Conversation -```bash -curl -X 'POST' \ - 'http://localhost:8001/api/v1/conversations/' \ - -H 'accept: application/json' \ - -H 'Content-Type: application/json' \ - -d '{ - "user_id": "your_user_id", - "title": "Conversation Title", - "status": "active", - "project_ids": [ - "project_id" - ], - "agent_ids": [ - "agent_id" - ] -}' -``` -## Step 6: Send Messages in a Conversation - -This API returns a stream response for the -```bash -curl -X 'POST' \ - 'http://localhost:8001/api/v1/conversations/1234/message/' \ - -H 'accept: application/json' \ - -H 'Content-Type: application/json' \ - -d '{ - "content": "Your message content here", - "node_ids": [ - { - "node_id": "node_identifier", - "name": "node_name" - } - ] -}' -``` -## Step 7: Get all the messages of a conversation -```bash -curl -X 'GET' \ - 'http://localhost:8001/api/v1/conversations/conversation-id/ - messages/?start=0&limit=10' \ - -H 'accept: application/json' -``` - ---- - -## Use Cases - -- **Onboarding**: For developers new to a codebase, the codebase QnA agent can help them understand the codebase and get up to speed quickly. Ask it how to setup a new project, how to run the tests etc -We tried to onboard ourselves with Potpie to the [**AgentOps**](https://github.com/AgentOps-AI/AgentOps) codebase and it worked like a charm : Video [here](https://youtu.be/_mPixNDn2r8). - -- **Codebase Understanding**: Answer questions about any library you're trying to integrate, explain functions, features, and architecture. Ask it how a feature works, what does a function do, how to change a function etc. -We used the Q&A agent to understand the underlying working of a feature of the [**CrewAI**](https://github.com/CrewAIInc/CrewAI) codebase that was not documented in official docs : Video [here](https://www.linkedin.com/posts/dhirenmathur_what-do-you-do-when-youre-stuck-and-even-activity-7256704603977613312-8X8G). - -- **Low level design**: Before writing that first line of code, it is important to know which files and functions need to be changed. This agent takes your functional requirements as an input and then generates a low level design for the feature. The output will consist of which files need to be changed, what all functions need to be added etc. -We fed an open issue from the [**Portkey-AI/Gateway**](https://github.com/Portkey-AI/Gateway) project to this agent to generate a low level design for it: Video [here](https://www.linkedin.com/posts/dhirenmathur_potpie-ai-agents-vs-llms-i-am-extremely-activity-7255607456448286720-roOC). - -- **Reviewing code changes**: Every commit to the codebase has the potential to be a breaking change. Use this agent to understand the functional impact of the changes in the codebase. It compares the code changes with the default branch and computes the blast radius of the changes. -Here we analyse a PR from the [**mem0ai/mem0**](https://github.com/mem0ai/mem0) codebase and understand its blast radius : Video [here](https://www.linkedin.com/posts/dhirenmathur_prod-is-down-three-words-every-activity-7257007131613122560-o4A7). - -- **Debugging**: Debugging is an iterative process that usually follows a set of well known steps. This agent emulates those steps and can be used to debug issues in the codebase. It takes a stacktrace as an input and then generates a list of steps that can be used to debug the issue. - -- **Unit and Integration testing**: Use the Unit Test Agent to generate unit test plans and code for individual functions to enhance test coverage, similarly use the Integration Test Agent to generate integration test plans and code for flows to ensure components work together properly. These agents are highly contextual and will use the codebase graph to gather context for generating the tests. - ---- - -## Custom Agents [Upgrade] - -Potpie doesnโ€™t stop at pre-built agents. With **Custom Agents**, developers can design personalized tools that handle repeatable tasks with precision. Whether it's generating boilerplate code, identifying security vulnerabilities, or suggesting optimizations, Potpieโ€™s custom agents are flexible and built to adapt to your unique project requirements. - -### Custom Agents for Advanced Workflows - -Potpieโ€™s cloud platform supports **Custom Agents**, enabling you to create agents that automate specific, repeatable tasks tailored to your project's unique requirements. - -#### Key Components of Custom Agents -- **System Instructions**: Guidelines that define the agent's task, its goal, and the expected output. -- **Agent Information**: Metadata such as the agentโ€™s role, goal, and task context. -- **Tasks**: The individual steps the agent will take to complete its job. -- **Tools**: Functions that allow the agent to perform its tasks, such as querying the knowledge graph or retrieving code snippets. - - -#### Example Use Cases: -- Automating code optimization and offering improvement suggestions. -- Identifying and reporting security vulnerabilities in the codebase. -- Automatically generating unit tests based on existing code logic. - ---- - -## Make Potpie Your Own +## ๐Ÿš€ Getting Started + +### Prerequisites +- Docker installed and running +- OpenAI API key +- Git installed (for repository access) + +### Setup Steps + +1. **Prepare Your Environment** + - Create a `.env` file based on the `.env.template` + - Add the following required configurations: + ```bash + isDevelopmentMode=enabled + ENV=development + OPENAI_API_KEY= + ``` + +2. **Start Potpie** + ```bash + chmod +x start.sh + ./start.sh + ``` + +3. **Authentication Setup** (Skip this step in development mode) + ```bash + curl -X POST 'http://localhost:8001/api/v1/login' \ + -H 'Content-Type: application/json' \ + -d '{ + "email": "your-email", + "password": "your-password" + }' + # Save the bearer token from the response for subsequent requests + ``` + +4. **Initialize Repository Parsing** + ```bash + # For development mode: + curl -X POST 'http://localhost:8001/api/v1/parse' \ + -H 'Content-Type: application/json' \ + -d '{ + "repo_path": "path/to/local/repo", + "branch_name": "main" + }' + + # For production mode: + curl -X POST 'http://localhost:8001/api/v1/parse' \ + -H 'Content-Type: application/json' \ + -d '{ + "repo_name": "owner/repo-name", + "branch_name": "main" + }' + # Save the project_id from the response + ``` + +5. **Monitor Parsing Status** + ```bash + curl -X GET 'http://localhost:8001/api/v1/parsing-status/your-project-id' + # Wait until parsing is complete + ``` + +6. **View Available Agents** + ```bash + curl -X GET 'http://localhost:8001/api/v1/list-available-agents/?list_system_agents=true' + # Note down the agent_id you want to use + ``` + +7. **Create a Conversation** + ```bash + curl -X POST 'http://localhost:8001/api/v1/conversations/' \ + -H 'Content-Type: application/json' \ + -d '{ + "user_id": "your_user_id", + "title": "My First Conversation", + "status": "active", + "project_ids": ["your-project-id"], + "agent_ids": ["chosen-agent-id"] + }' + # Save the conversation_id from the response + ``` + +8. **Start Interacting with Your Agent** + ```bash + curl -X POST 'http://localhost:8001/api/v1/conversations/your-conversation-id/message/' \ + -H 'Content-Type: application/json' \ + -d '{ + "content": "Your question or request here" + }' + ``` + +9. **View Conversation History** (Optional) + ```bash + curl -X GET 'http://localhost:8001/api/v1/conversations/your-conversation-id/messages/?start=0&limit=10' + ``` + +## ๐Ÿ’ก Use Cases + +- **Onboarding**: For developers new to a codebase, the codebase QnA agent helps them understand the codebase and get up to speed quickly. Ask it how to setup a new project, how to run the tests etc +>We tried to onboard ourselves with Potpie to the [**AgentOps**](https://github.com/AgentOps-AI/AgentOps) codebase and it worked like a charm : Video [here](https://youtu.be/_mPixNDn2r8). + +- **Codebase Understanding**: Answer questions about any library you're integrating, explain functions, features, and architecture. +>We used the Q&A agent to understand the underlying working of a feature of the [**CrewAI**](https://github.com/CrewAIInc/CrewAI) codebase that was not documented in official docs : Video [here](https://www.linkedin.com/posts/dhirenmathur_what-do-you-do-when-youre-stuck-and-even-activity-7256704603977613312-8X8G). + +- **Low Level Design**: Get detailed implementation plans for new features or improvements before writing code. +>We fed an open issue from the [**Portkey-AI/Gateway**](https://github.com/Portkey-AI/Gateway) project to this agent to generate a low level design for it: Video [here](https://www.linkedin.com/posts/dhirenmathur_potpie-ai-agents-vs-llms-i-am-extremely-activity-7255607456448286720-roOC). + +- **Reviewing Code Changes**: Understand the functional impact of changes and compute the blast radius of modifications. +>Here we analyse a PR from the [**mem0ai/mem0**](https://github.com/mem0ai/mem0) codebase and understand its blast radius : Video [here](https://www.linkedin.com/posts/dhirenmathur_prod-is-down-three-words-every-activity-7257007131613122560-o4A7). + +- **Debugging**: Get step-by-step debugging guidance based on stacktraces and codebase context. + +- **Testing**: Generate contextually aware unit and integration test plans and test code that understand your codebase's structure and purpose. + +## ๐Ÿ› ๏ธ Custom Agents [Upgrade โœจ](https://potpie.ai/pricing) + +With Custom Agents, you can design personalized tools that handle repeatable tasks with precision. Key components include: +- **System Instructions**: Define the agent's task, goal, and expected output +- **Agent Information**: Metadata about the agent's role and context +- **Tasks**: Individual steps for job completion +- **Tools**: Functions for querying the knowledge graph or retrieving code + +## ๐ŸŽจ Make Potpie Your Own Potpie is designed to be flexible and customizable. Here are key areas to personalize your own deployment: ### 1. System Prompts Configuration - -Modify the system prompts to align with your organization's tone and terminology. - -**Edit Prompt Text**: In `app/modules/intelligence/prompts/system_prompt_setup.py`, update the `system_prompts` lists to change the text for each agent. +Modify prompts in `app/modules/intelligence/prompts/system_prompt_setup.py` ### 2. Add New Agents -**Add New Agents**: Create new agents by referring existing agents in the `app/modules/intelligence/agents/chat_agents` and `app/modules/intelligence/agents/agentic_tools` directory. +Create new agents in `app/modules/intelligence/agents/chat_agents` and `app/modules/intelligence/agents/agentic_tools` ### 3. Agent Behavior Customization - -Adjust existing agent behaviors to suit your operational needs. - -**Modify Guidelines**: Change the guidelines within each agent's prompt to emphasize specific aspects of your codebase. You can do this by editing the prompts in the crewai agents in the `app/modules/intelligence/agents` directory. - +Modify guidelines within each agent's prompt in the `app/modules/intelligence/agents` directory ### 4. Tool Integration +Edit or add tools in the `app/modules/intelligence/tools` directory -Customize which tools are available to each agent based on your requirements. - -**Edit existing tools**: Edit tools for your usecase by refactoring the existing tools in the `app/modules/intelligence/tools` directory. - -**Add New Tools**: Add new tools by referring existing tools in the `app/modules/intelligence/tools` directory. - - -By customizing system prompts, agent behaviors, and tool integrations you can tailor Potpie to effectively meet your organization's unique needs and enhance your software development processes. - - ---- - -## Contributing - -We welcome contributions from the community. Contributions can be of the form: -1. Documentation : Help improve our docs! If you fixed a problem, chances are others faced it too. -2. Code : Help us make improvements to existing features and build new features for Potpie. -3. Tests : Help us make Potpie resilient by contributing tests. +## ๐Ÿค Contributing -To contribute: -1. Fork the repository. -2. Create a new branch (`git checkout -b feature-branch`). -3. Stage your changes (`git add `), then commit them (`git commit -m 'Add new feature'`). -4. Push to the branch (`git push origin feature-branch`). -5. Open a Pull Request. +We welcome contributions! To contribute: +1. Fork the repository +2. Create a new branch (`git checkout -b feature-branch`) +3. Make your changes +4. Commit (`git commit -m 'Add new feature'`) +5. Push to the branch (`git push origin feature-branch`) +6. Open a Pull Request -Refer to the [Contributing Guide](./contributing.md) for more details. +See [Contributing Guide](./contributing.md) for more details. -## License +## ๐Ÿ“œ License This project is licensed under the Apache 2.0 License - see the [LICENSE](LICENSE) file for details. ## ๐Ÿ’ช Thanks To All Contributors -Thanks a lot for spending your time helping build Potpie. Keep rocking ๐Ÿฅ‚ +Thanks for spending your time helping build Potpie. Keep rocking ๐Ÿฅ‚ Contributors