Skip to content

Commit

Permalink
feat: add new knowledge contribution
Browse files Browse the repository at this point in the history
  • Loading branch information
lynchee-owo committed Apr 11, 2024
1 parent d78de1c commit e22cfbd
Show file tree
Hide file tree
Showing 2 changed files with 53 additions and 0 deletions.
51 changes: 51 additions & 0 deletions CONTRIBUTING_KNOWLEDGE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Contributing to Prior Knowledge Augmentation

WebWand's Prior Knowledge Augmentation system is designed to enhance the tool's web navigation and task execution capabilities by leveraging a shared knowledge base. Contributions to this system help make WebWand smarter and more capable.

## What Kind of Knowledge Are We Looking For?

We seek knowledge that:
- Enhances the understanding of specific web pages or actions, making task execution more reliable.
- Includes insights into website layouts, common patterns, and user interfaces that are not immediately obvious.
- Provides rules or annotations that help the AI better interpret the purpose of elements on a page.

For example, if a website has two buttons with the same name but different functionalities, it's crucial to describe in notes how to distinguish between them.

## How to Add and Test New Knowledge

We offer two convenient ways to add and test new knowledge in real-time:
- Via Form: Within the WebWand UI settings, navigate to the "Custom Knowledge Base" and select "Add Host Knowledge with Form" to input new knowledge using a user-friendly form.
- Via JSON: If you prefer to work directly with JSON, choose "Add Host Knowledge with JSON" to enter your custom knowledge.

You can test the new knowledge by running several tasks on the relevant web pages to ensure WebWand behaves as expected.

Once you've tested various knowledge inputs and are satisfied with the new knowledge's performance, you can then copy that knowledge into the db.json file.

1. Locate the `db.json` file in the `src/helpers/knowledge` directory of the WebWand repository.
2. Add your knowledge in the JSON format, following the existing structure. `annotationRules` is optional.
```json
{
"example.com": {
"rules": [
{
"regexes": ["some-page-regex"],
"knowledge": {
"notes": ["Your insights or notes about this page or action"],
"annotationRules": [
{
"selector": "CSS selector",
"allowInvisible": true,
"allowCovered": true,
"allowAriaHidden": true
}
]
}
}
]
}
}
3. Please ensure your contributions are clear and concise, with regexes and selectors accurately defined.

## Submitting Your Contribution

Please check out the [Contribution Guide](CONTRIBUTING.md). Share your testing process and results in your pull request to help reviewers understand the impact of your contribution. Specifically, describe how the new knowledge help WebWand acheive something it previously cannot perform correctly.
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,8 @@ When loading the extension, you will need to load the `dist` folder created by t

Interested in contributing to WebWand? We'd love your help! Check out our [Contribution Guide](CONTRIBUTING.md) for guidelines on how to contribute, report bugs, suggest enhancements, and more.

We also encourage everyone in the community to add new knowledge to the "Prior Knowledge Augmentation" system to make WebWand even smarter. For detailed instructions on what kind of knowledge we're looking for and how to test and submit it, please see our [Contributing Knowledge Guide](CONTRIBUTING-KNOWLEDGE.md).

## Credits

- WebWand's image annotation method was inspired by Microsoft's [UFO paper](https://arxiv.org/abs/2402.07939).
Expand Down

0 comments on commit e22cfbd

Please sign in to comment.