From aca53f57bff3b867d616fee0799eb3bc063f81c1 Mon Sep 17 00:00:00 2001 From: Richard Abrich Date: Mon, 11 Dec 2023 22:16:17 -0600 Subject: [PATCH 1/3] docs: README --- README.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index f9f808a19..9a20063b1 100644 --- a/README.md +++ b/README.md @@ -154,7 +154,13 @@ This will open a scrollable window that looks something like this: image -With `config.VISUALIZE_RUN_NATIVELY == False`, this will open your browser. It will look something like this: +For an alternative visualization, run: + +``` +python -m openadapt.deprecated.visualize +``` + +This will open up a tab in your browser that looks something like this: ![image](https://github.com/OpenAdaptAI/OpenAdapt/assets/774615/5d7253b7-ae12-477c-94a3-b388e4f37587) From ef4c9010e7b19c5f8174eeae4aca416952cb1c63 Mon Sep 17 00:00:00 2001 From: Richard Abrich Date: Mon, 11 Dec 2023 23:18:31 -0600 Subject: [PATCH 2/3] docs(README): Transformers -> LLMs Transformers -> LMMs --- README.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 9a20063b1..594759dbf 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,8 @@ [Join us on Discord](https://discord.gg/yF527cQbDG) -# OpenAdapt: AI-First Process Automation with Transformers +# OpenAdapt: AI-First Process Automation with Large Multimodal Models (LMMs). + +OpenAdapt is the *Open* source software *Adapt*er between Large Multimodal Models (LMMs) and traditional desktop and web Graphical User Interfaces (GUIs). ### Enormous volumes of mental labor are wasted on repetitive GUI workflows. @@ -14,7 +16,7 @@ ([Slides](https://t.ly/7RGr)) Welcome to OpenAdapt! This Python library implements AI-First Process Automation -with the power of Transformers by: +with the power of Large Multimodal Modals (LMMs) by: - Recording screenshots and associated user input - Aggregating and visualizing user input and recordings for development @@ -24,7 +26,7 @@ with the power of Transformers by: The goal is similar to that of [Robotic Process Automation](https://en.wikipedia.org/wiki/Robotic_process_automation), -except that we use transformers instead of conventional RPA tools. +except that we use Large Multimodal Models instead of conventional RPA tools. The direction is adjacent to [Adept.ai](https://adept.ai/), with some key differences: 1. OpenAdapt is model agnostic From ec5875503b529f8be6aee77606902608e5eb0d90 Mon Sep 17 00:00:00 2001 From: Richard Abrich Date: Tue, 12 Dec 2023 01:14:53 -0600 Subject: [PATCH 3/3] docs(README) --- README.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 594759dbf..30b5ee8ba 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ # OpenAdapt: AI-First Process Automation with Large Multimodal Models (LMMs). -OpenAdapt is the *Open* source software *Adapt*er between Large Multimodal Models (LMMs) and traditional desktop and web Graphical User Interfaces (GUIs). +**OpenAdapt** is the **open** source software **adapt**er between Large Multimodal Models (LMMs) and traditional desktop and web Graphical User Interfaces (GUIs). ### Enormous volumes of mental labor are wasted on repetitive GUI workflows. @@ -12,7 +12,6 @@ OpenAdapt is the *Open* source software *Adapt*er between Large Multimodal Model image - ([Slides](https://t.ly/7RGr)) Welcome to OpenAdapt! This Python library implements AI-First Process Automation @@ -22,7 +21,8 @@ with the power of Large Multimodal Modals (LMMs) by: - Aggregating and visualizing user input and recordings for development - Converting screenshots and user input into tokenized format - Generating synthetic input via transformer model completions -- Replaying synthetic input to complete tasks +- Generating task trees by analyzing recordings (work-in-progress) +- Replaying synthetic input to complete tasks (work-in-progress) The goal is similar to that of [Robotic Process Automation](https://en.wikipedia.org/wiki/Robotic_process_automation), @@ -83,7 +83,6 @@ The direction is adjacent to [Adept.ai](https://adept.ai/), with some key differ
-
### Manual Setup