diff --git a/.gitignore b/.gitignore index 88bacb4..ca150e4 100644 --- a/.gitignore +++ b/.gitignore @@ -34,3 +34,4 @@ downloads/ .clj-kondo/ resources/config.edn *.log +files diff --git a/README.md b/README.md index d6ddbd4..8603c4c 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,12 @@ -# Simmie +# Simmis -Simmie is a conversational AI framework that tries to blend in to human conversations as much as possible while doing useful background research and thinking ahead. It is trying to simulate what you need to know or decide next. The goal is to have both a principled organizational framework to structure problem solving and processes for humans and machines (i.e. a unified computational framework for intelligence) and a la carte abstractions that can be picked up from something as trivial as a group chat on a popular chat platform such as Telegram, WhatsApp, Slack, ... A concrete first example of this is a notion/wiki-like note taking system that is used by the conversational AI both as a memory and as a shared interface with humans. +Simmis is an interactive AGI system that tries to blend in to human environments as much as possible while doing useful background research and thinking ahead. It is trying to simulate what you need to know or decide next. The goal is to have both a principled organizational framework to structure problem solving and processes for humans and machines (i.e. a unified computational framework for intelligence) and a la carte abstractions that can be picked up from something as trivial as a group chat on a popular chat platform such as Telegram, WhatsApp, Slack, ... A concrete first example of this is a notion/wiki-like note taking system that is used by the conversational AI both as a memory and as a shared interface with humans. At the moment Telegram is supported including automatic voice transcription. The system is also intended to support payments, e.g. through the Telegram API, and general resource awareness to guide its planning and computation. This code base is currently exploring design decisions and used in private betas with the Telegram bot interface. If you want to participate join our [Telegram group](https://t.me/+ANfiXz-khkpiOTg0) or open an issue on this repository. # Design -The framework is based on communicating sequential processes (CSP) through [superv.async](https://github.com/replikativ/superv.async) and aims at decoupled asynchronous binding between parallel processes in [runtimes](./src/ie/simm/runtimes/) and high-level [language](./src/ie/simm/languages/) abstractions that send messages to these runtimes. The message flow follows [kabel](https://github.com/replikativ/kabel) which is a [ring](https://github.com/ring-clojure/ring)-like, continuous, bidirectional processing framework that allows separation of concerns between different middlewares. Middlewares can initiate events at any time and create multiple "return" messages. The system is therefore *not* purely reactive and goes beyond the standard lambda calculus employed in ring. A more functional perspective could be provided by FRP through [missionary](https://github.com/leonoel/missionary). +The framework is based on communicating sequential processes (CSP) through [superv.async](https://github.com/replikativ/superv.async) and aims at decoupled asynchronous binding between parallel processes in [runtimes](./src/is/simm/runtimes/) and high-level [language](./src/is/simm/languages/) abstractions that send messages to these runtimes. The message flow follows [kabel](https://github.com/replikativ/kabel) which is a [ring](https://github.com/ring-clojure/ring)-like, continuous, bidirectional processing framework that allows separation of concerns between different middlewares. Middlewares can initiate events at any time and create multiple "return" messages. The system is therefore *not* purely reactive and goes beyond the standard lambda calculus employed in ring. A more functional perspective could be provided by FRP through [missionary](https://github.com/leonoel/missionary). The goal is to build a compositional framework of building blocks that can be easily integrated into JVM based backends. The backend uses HTMX by default to render implementations easy for backend developers, but a React frontend might make sense at some point. [libython-clj](https://github.com/clj-python/libpython-clj) is used for all data science, machine learning and generative AI related APIs and in general is useful for effects that can be invoked through stateless remote procedure calls (RPCs), while all state is managed by Clojure and [Datahike](https://github.com/replikativ/datahike) to keep a persistent, git-like memory model that is ideal for simulation environments. Future versions could leverage the GraalVM with [graalpython](https://github.com/oracle/graalpython) and node.js exposure to render the Clojure code easier to integrate into popular backends. The Clojure code itself including Datahike also is fairly portable and could be moved to frontends, e.g. browser, Android or maybe iOS through GraalVM native compilation. @@ -16,7 +16,7 @@ Make sure to define your credentials in `resources/config.edn`. Prompts are stor ## Usage -Just start a REPL and explore the [assistance](./src/ie/simm/runtimes/assistance.clj) namespace. +Just start a REPL and explore the [assistance](./src/is/simm/runtimes/assistance.clj) namespace. ## License diff --git a/build.clj b/build.clj index 03eee1d..5bc9125 100644 --- a/build.clj +++ b/build.clj @@ -2,9 +2,9 @@ (:refer-clojure :exclude [test]) (:require [clojure.tools.build.api :as b])) -(def lib 'ie.simm/simmie) +(def lib 'is.simm/simmie) (def version "0.1.0-SNAPSHOT") -(def main 'ie.simm.simmie) +(def main 'is.simm.simmie) (def class-dir "target/classes") (defn test "Run all the tests." [opts] diff --git a/deps.edn b/deps.edn index 57acac9..b3e687b 100644 --- a/deps.edn +++ b/deps.edn @@ -1,6 +1,6 @@ {:paths ["src" "resources"] :deps {org.clojure/clojure {:mvn/version "1.11.1"} - io.replikativ/datahike {:mvn/version "0.6.1584"} + io.replikativ/datahike {:mvn/version "0.6.1592"} clj-python/libpython-clj {:mvn/version "2.025"} morse/morse {:mvn/version "0.4.3"} io.replikativ/kabel {:mvn/version "0.2.2"} @@ -18,12 +18,14 @@ ;; exploratory missionary/missionary {:mvn/version "b.34"} + io.github.jbellis/jvector {:mvn/version "3.0.2"} pangloss/pattern {:git/url "https://github.com/pangloss/pattern" - :sha "affc7f3ac907f5b98de6638574a741e4693f1648"} + :sha #_"affc7f3ac907f5b98de6638574a741e4693f1648" + "93fb43e3223bbcfe08c4e37414709021d8a99604"} anglican/anglican {:mvn/version "1.1.0"}} - :jvm-opts ["-Xmx1g"] + :jvm-opts ["-Xmx1g" "--add-modules jdk.incubator.vector"] :aliases - {:run {:main-opts ["-m" "ie.simm.simmie" "--middleware" "[cider.nrepl/cider-middleware]"]} + {:run {:main-opts ["-m" "is.simm.simmis" "--middleware" "[cider.nrepl/cider-middleware]"]} :build {:deps {io.github.clojure/tools.build {:mvn/version "0.9.6"}} :ns-default build} diff --git a/doc/intro.md b/doc/intro.md index 93dbdd5..08303a3 100644 --- a/doc/intro.md +++ b/doc/intro.md @@ -1,3 +1,3 @@ -# Introduction to ie.simm/simmie +# Introduction to is.simm/simmis TODO: write [great documentation](http://jacobian.org/writing/what-to-write/) diff --git a/resources/public/index.html b/resources/public/index.html index 67c10d9..a246950 100644 --- a/resources/public/index.html +++ b/resources/public/index.html @@ -3,7 +3,7 @@ - Simmie + Simmis @@ -13,14 +13,14 @@
- Simmie logo + Simmis logo @@ -32,10 +32,34 @@
-
+

Hey!

-

Simmie is a conversational AI system that combines different forms of intelligence to help you pursue your goals or just have fun.
To access the bot on Telegram join https://t.me/+ANfiXz-khkpiOTg0

+

Simmis is an interactive, open-source, self-organizing system that coordinates the different forms of intelligence in your life (human and otherwise) to help you pursue your goals or just have fun.
To access the bot on Telegram join https://t.me/+ANfiXz-khkpiOTg0

+
+
+
+
+
+
+

Features

+

Here are some of the features that Simmis offers:

+
Screensharing and recording
+
Chatbot
+
Auto journaling
+
Goal setting
+
Calendar (TODO)
+
Task management (TODO)
+
Time tracking (TODO)
+
And more...
+
+
+
+
+
+
+

About

+

Simmis is a project by Christian Weilbach. It is a work in progress and is constantly being improved. If you have any feedback or suggestions, please let me know.

diff --git a/resources/public/simmie.png b/resources/public/simmis.png similarity index 100% rename from resources/public/simmie.png rename to resources/public/simmis.png diff --git a/src/ie/simm/languages/chat.clj b/src/ie/simm/languages/chat.clj deleted file mode 100644 index 7010a43..0000000 --- a/src/ie/simm/languages/chat.clj +++ /dev/null @@ -1,16 +0,0 @@ -(ns ie.simm.languages.chat - "General chat interface language. It should be a common denominator for all chat interfaces." - (:require [ie.simm.languages.dispatch :refer [create-upstream-msg-handler]])) - - -(let [handler (create-upstream-msg-handler ::send-text)] - (defn send-text! [chat-id msg] - (handler chat-id msg))) - -(let [handler (create-upstream-msg-handler ::send-photo)] - (defn send-photo! [chat-id msg] - (handler chat-id msg))) - -(let [handler (create-upstream-msg-handler ::send-document)] - (defn send-document! [chat-id msg] - (handler chat-id msg))) \ No newline at end of file diff --git a/src/ie/simm/processes/build_accounting.clj b/src/ie/simm/processes/build_accounting.clj deleted file mode 100644 index 6883c40..0000000 --- a/src/ie/simm/processes/build_accounting.clj +++ /dev/null @@ -1,119 +0,0 @@ -(ns ie.simm.processes.build-accounting - (:require [clojure.core.async :as async :refer [timeout]] - [superv.async :refer [go-try go-loop-try "accounting system" - brave/search-brave - brave/extract-url - etaoin/extract-body) - -(etaoin/extract-body "https://en.wikipedia.org/wiki/Accounting") - - -(defn interleave-with-delimiters - ([coll] (interleave-with-delimiters coll "=========================================\n")) - ([coll delimiter] - (str/join "\n" (interleave coll (repeat delimiter))))) - - -(defn feedback [question] - (println question) - (read-line)) - -(defn expert [type question] - (openai/chat "gpt-4o" (interleave-with-delimiters [(format "You are an %s." type) question]))) - -(defn critic [type question answer] - (Integer/parseInt - (openai/chat "gpt-4o" (interleave-with-delimiters [(format "You are a critic for a %s." type) - "We gave the expert this question:" question - "The expert replied with this answer:" answer - "Rate the expert's answer on a scale from 1 to 10. Only reply with a number."])))) - - -;; a building process loop with a fixed point, it is given an environmental context -;; in every loop we first gather and refine requirements -;; we derive subgoals to currently pursue -;; we define tests and progress metrics -;; then we do an iteration given the system so far -;; and measure the progress -;; we then check if we have reached a fixed point - -subgoals (expert expert-role (str "You have this goal: " goal "\nWith these requirements: " requirements "\nAnd these subgoals derived so far:" subgoals "\nUpdate the subgoals if reasonable and enumerate them as a comma separated list: ")) - tests (expert expert-role (str "You have this goal: " goal "\nWith these requirements: " requirements "\nAnd these subgoals: " subgoals "\nAnd these tests derived so far: " tests "\n Update the tests if reasonable and return them as Clojure code string that can be passed to eval: ")) - progress-metrics (expert expert-role (str "You have this goal: " goal "\nWith these requirements: " requirements "\nAnd these subgoals: " subgoals "\nAnd these tests: " tests "\nAnd these progress metrics derived so far: " progress-metrics "\n Update the progress metrics if reasonable and return them only as a single Clojure code string that can be passed to eval directly: ")) - iteration (expert expert-role (str "You have this goal: " goal "\nWith these requirements: " requirements "\nAnd these subgoals: " subgoals "\nAnd these tests: " tests "\nAnd these progress metrics: " progress-metrics "\nAnd this iteration derived so far: " iteration "\n Update the iteration if reasonable and return it as a single Clojure code string that can be passed to eval: ")) - progress (expert expert-role (str "You have this goal: " goal "\nWith these requirements: " requirements "\nAnd these subgoals: " subgoals "\nAnd these tests: " tests "\nAnd these progress metrics: " progress-metrics "\nAnd this iteration: " iteration "\nAnd this progress derived so far: " progress "\n Update the progress metric if reasonable and return it as Clojure code. The progress metric function needs to return a real number: ")) - - -(defn build [ctx goal] - (let [expert-role (openai/chat "gpt-4o" - (interleave-with-delimiters - ["Given this context:" ctx - "What expert would you hire to pursue this goal:" goal - "Answer a descriptive job title."]))] - (loop [requirements "" - subgoals "" - tests "" - progress-metrics "" - iteration "" - progress "" ] - (let [requirements (expert expert-role (interleave-with-delimiters ["You have this goal:" goal - "With these requirements derived so far:" requirements - "What are the requirements?"])) - subgoals (expert expert-role (interleave-with-delimiters ["You have this goal:" goal - "With these requirements:" requirements - "And these subgoals derived so far:" subgoals - "Update the subgoals if reasonable and enumerate them as a comma separated list: "])) - tests (expert expert-role (interleave-with-delimiters ["You have this goal:" goal - "With these requirements:" requirements - "And these subgoals:" subgoals - "And these tests derived so far:" tests - " Update the tests if reasonable and return them as Clojure code string that can be passed to eval: "])) - progress-metrics (expert expert-role (interleave-with-delimiters ["You have this goal:" goal - "With these requirements:" requirements - "And these subgoals:" subgoals - "And these tests:" tests - "And these progress metrics derived so far:" progress-metrics - " Update the progress metrics if reasonable and return them only as a single Clojure code string that can be passed to eval directly: "])) - iteration (expert expert-role (interleave-with-delimiters ["You have this goal:" goal - "With these requirements:" requirements - "And these subgoals:" subgoals - "And these tests:" tests - "And these progress metrics:" progress-metrics - "And this iteration derived so far:" iteration - " Update the iteration if reasonable and return it as a single Clojure code string that can be passed to eval: "])) - progress (expert expert-role (interleave-with-delimiters ["You have this goal:" goal - "With these requirements:" requirements - "And these subgoals:" subgoals - "And these tests:" tests - "And these progress metrics:" progress-metrics - "And this iteration:" iteration - "And this progress derived so far:" progress - " Update the progress metric if reasonable and return it as Clojure code. The progress metric function needs to return a real number: "]))] - (println "Requirements: " requirements) - (println "Subgoals: " subgoals) - (println "Tests: " tests) - (println "Progress Metrics: " progress-metrics) - (println "Iteration: " iteration) - (println "Progress: " progress) - [expert-role requirements subgoals tests progress-metrics iteration progress])))) - - -(comment - - (def build-result (build "exploring games" "Build a Snake game in Clojure without any external libraries. You have clojure.core, core.async and Datahike available.")) - - (let [[expert-role requirements subgoals tests progress-metrics iteration progress] build-result] - (println iteration)) - - ) diff --git a/src/ie/simm/processes/datahike_unify.clj b/src/ie/simm/processes/datahike_unify.clj deleted file mode 100644 index 0d0f3f4..0000000 --- a/src/ie/simm/processes/datahike_unify.clj +++ /dev/null @@ -1,272 +0,0 @@ -(ns ie.simm.processes.datahike-unify - (:require [datahike.api :as d])) - -;; Schema Definition -(def schema - [{:db/ident :type/const - :db/valueType :db.type/string - :db/cardinality :db.cardinality/one} - {:db/ident :type/type-app - :db/valueType :db.type/ref - :db/cardinality :db.cardinality/one} - {:db/ident :type/args - :db/valueType :db.type/ref - :db/cardinality :db.cardinality/many} - {:db/ident :type/unbound - :db/valueType :db.type/long - :db/cardinality :db.cardinality/one} - {:db/ident :type/link - :db/valueType :db.type/ref - :db/cardinality :db.cardinality/one} - {:db/ident :type/generic - :db/valueType :db.type/long - :db/cardinality :db.cardinality/one} - ;; Schema for Function Types - {:db/ident :type/fn - :db/valueType :db.type/boolean - :db/cardinality :db.cardinality/one} - {:db/ident :fn/params - :db/valueType :db.type/ref - :db/cardinality :db.cardinality/many} - {:db/ident :fn/ret-type - :db/valueType :db.type/ref - :db/cardinality :db.cardinality/one}]) - -;; Initialize Datomic connection -(defn initialize-db [cfg] - (let [cfg (d/create-database cfg) - conn (d/connect cfg)] - (d/transact conn {:tx-data schema}) - conn)) - -;; Define the unification rules -(def unify-rules - '[;; Resolve any links for a type recursively - [(resolve-link ?x ?final) - [(not [?x :type/link _])] ;; No link means ?x is the final type ;; TODO this syntax does not match Datomic - [(identity ?x) ?final]] ;; The type is itself - - [(resolve-link ?x ?final) - [?x :type/link ?y] ;; If there is a link, follow it - (resolve-link ?y ?final)] ;; Recurse until we find the final type - - ;; Ensure constants are equal - [(unify-const ?t ?o) - [?t :type/const ?const] ;; Both must have the same constant value - [?o :type/const ?const]] - - ;; Unify type applications (main type and args) - [(unify-type-app ?t ?o) - [?t :type/type-app ?main] ;; Both must share the same main type - [?o :type/type-app ?main] ;; Use the same logic variable ?main to unify - - [?t :type/args ?t-args] - [?o :type/args ?o-args] - (coll-equal ?t-args ?o-args)] ;; Recursively unify arguments - - ;; Unify function types (params and return types) - [(unify-fn ?t ?o) - [?t :fn/params ?params] ;; Unify the parameters - [?o :fn/params ?params] ;; Same logic variable ?params unifies them - - [?t :fn/ret-type ?ret] ;; Unify the return type - [?o :fn/ret-type ?ret]] ;; Same logic variable ?ret unifies them - - ;; Recursively compare collections - [(coll-equal ?a ?b) - [(= (empty? ?a) (empty? ?b))]] - - [(coll-equal ?a ?b) - [(first ?a) ?a-head] - [(rest ?a) ?a-tail] - [(first ?b) ?a-head] - [(rest ?b) ?b-tail] - (coll-equal ?a-tail ?b-tail)] ;; Recursively unify tails - ]) - -;; Unification query that leverages the above rules -(defn unify-query [db t o] - (d/q '[:find ?t-final ?o-final - :in $ % ?t ?o - :where - ;; Resolve any links first - (resolve-link ?t ?t-final) - (resolve-link ?o ?o-final) - - (or - ;; Try to unify constants - (unify-const ?t-final ?o-final) - - ;; Try to unify type applications - (unify-type-app ?t-final ?o-final) - - ;; Try to unify function types - (unify-fn ?t-final ?o-final))] - db unify-rules t o)) - -;; Handle unbound types by linking them in Datomic -(defn handle-unbound! [conn t-final o-final] - (let [db (d/db conn) - ;; Check if either t or o is unbound - t-unbound (first (d/q '[:find ?id - :in $ ?type - :where - [?type :type/unbound ?id]] db t-final)) - o-unbound (first (d/q '[:find ?id - :in $ ?type - :where - [?type :type/unbound ?id]] db o-final))] - (cond - ;; If t is unbound, link it to o - t-unbound (d/transact conn [{:db/id t-final :type/link o-final}]) - - ;; If o is unbound, link it to t - o-unbound (d/transact conn [{:db/id o-final :type/link t-final}])))) - -;; Main unification function -(defn unify! [conn t o] - (let [db (d/db conn) - result (first (unify-query db t o))] - (if result - (let [[t-final o-final] result] - ;; Handle unbound variables (linking them) - (handle-unbound! conn t-final o-final) - (println "Unification successful for:" t-final o-final)) - (throw (ex-info "Unification failed" {:type :unification-failed :result result}))))) - -;; TODO replace with squuid from datahike -(defn squuid [] - (.getTime (java.util.Date.))) - -;; Helper to create a new unbound variable type -(defn new-var! [conn] - (let [new-id (d/tempid :db.part/user)] - (-> - (d/transact conn [{:db/id new-id - :type/unbound (squuid)}]) - :tempids - (get new-id)))) - -;; Helper to create a constant type -(defn new-const! [conn value] - (let [new-id (d/tempid :db.part/user)] - (-> (d/transact conn [{:db/id new-id - :type/const value}]) - :tempids - (get new-id)))) - -;; Helper to create a function type -(defn new-fn-type! [conn params ret-type] - (let [fn-id (d/tempid :db.part/user)] - (-> - (d/transact conn [{:db/id fn-id - :type/fn true - :fn/params params - :fn/ret-type ret-type}]) - :tempids - (get fn-id)))) - -;; Helper to create a function application type -(defn new-type-app! [conn main args] - (let [type-id (d/tempid :db.part/user)] - (-> - (d/transact conn [{:db/id type-id - :type/type-app main - :type/args args}]) - :tempids - (get type-id)))) - -(defn instantiate-fn-type! [db n-params] - (let [params (repeatedly n-params #(new-var! db)) - ret-type (new-var! db)] - (new-fn-type! db params ret-type))) - - -(defn match-fn-type! [conn n-params type] - (let [db (d/db conn)] - (if-let [[params ret-type] (first (d/q '[:find ?params ?ret-type - :in $ ?fn-type - :where - [?fn-type :fn/params ?params] - [?fn-type :fn/ret-type ?ret-type]] - db type))] - ;; If the type is already a function, check parameter counts and return type - (do (assert (= (count params) n-params) "Unexpected number of arguments") - [params ret-type]) - - ;; If unbound, instantiate a new function type with n-params arguments - (if-let [unbound (first (d/q '[:find ?unbound - :in $ ?type - :where - [?type :type/unbound ?unbound]] - db type))] - (instantiate-fn-type! conn n-params) - (throw (Exception. (str "Type is not a function: " type))))))) - -(defn infer! [conn env expr] - (match expr - ;; Variable lookup - [:var v-name] - (if-let [t (env v-name)] - (instantiate conn t) ;; Ensure instantiation of type variables - (throw (Exception. (str "Variable " v-name " not found")))) - - ;; Function creation (lambda) - [:fun params body] - (let [param-types (mapv (fn [_] (new-var! conn)) params) - fn-env (merge env (zipmap params param-types)) - ret-type (infer! conn fn-env body)] - (new-fn-type! conn param-types ret-type)) - - ;; Function application - [:call fnn args] - (let [[param-types ret-type] (match-fn-type! conn (count args) (infer! conn env fnn))] - (doseq [[p a] (map vector param-types args)] - (unify! conn p (infer! conn env a))) ;; Unify each argument with the corresponding parameter - ret-type))) - - - -(comment -;; Initialize the in-memory database - (def conn (initialize-db {})) - -;; Create a few constants and variables - (def const1 (new-const! conn "int")) - - (def const2 (new-const! conn "int")) - - (def const3 (new-const! conn "string")) - - (def var1 (new-var! conn)) - - (def var2 (new-var! conn)) - -;; Unifying two identical constants should succeed - (unify! conn const1 const2) ;; This should succeed without issues - - (unify-query (d/db conn) const1 const2) - -;; Trying to unify different constants should fail - (try - (unify! conn const1 const3) - (catch Exception e (println e))) ;; This should throw an exception - - ;; add a function type taking var1 in and returning const1 - (def fn1 (new-fn-type! conn [const2] const1)) - - (def app1 (new-type-app! conn fn1 [var2])) - - (unify! conn var2 const2) - - (unify-query (d/db conn) var2 const2) - - ;; Define a function type: (Int -> Bool) - (def fn-id (new-fn-type! conn [(new-var! conn)] (new-var! conn))) - - ;; Example of a function call with arguments - (infer! conn {} [:call fn-id [const1]]) - - - - ) \ No newline at end of file diff --git a/src/ie/simm/processes/related_work.clj b/src/ie/simm/processes/related_work.clj deleted file mode 100644 index 805ac34..0000000 --- a/src/ie/simm/processes/related_work.clj +++ /dev/null @@ -1,222 +0,0 @@ -(ns ie.simm.related-work - (:require [datahike.api :as d] - [superv.async :refer [S go-try > (brave/search-brave query) :web :results (map :url) (filter #(.startsWith % "https://arxiv.org/pdf"))) - ;; arxiv-html (when (seq urls) (etaoin/extract-body (first urls))) - rand-path (str "/tmp/" (java.util.UUID/randomUUID) ".pdf") - _ (io/copy (:body (http/get (.replace (first urls) ".pdf" "") {:as :stream}) ) (io/file rand-path)) - text (text/extract rand-path) - references (when text - (mapv (fn [[author title]] {:author author :title title}) - (-> (openai/chat "gpt-4o-mini" - (format "Do not reply, only extract the first author family name and title for each paper from the reference section of the following paper as a JSON vector of string tuple vectors: %s" - text)) - (.replace "```json" "") - (.replace "```" "") - read-string))) - ;; extract github urls with regex - github-urls (->> text - (re-seq #"https://github.com/[^ ]+") - (map #(.replace % "/\n" "/")) - (map #(re-find #"https://github.com/[^/]+/[^/\.\n]+" %)))] - {:query query - :urls urls - :text text - :references references - :github-urls github-urls}) - (catch Exception e - (timbre/error e) - (Thread/sleep 1000) ;; hack to deal with brave rate limiting - nil))) - -(comment - - (def gsdm-arxiv (extract-arxiv-paper "Graphically Structured Diffusion Model")) - - (openai/chat "gpt-4o" (format "Given the following github links, provide a regular expression that extracts them while avoiding the extracted PDF clutter at the end. Be aware that there are new lines sometimes interspersed, try to extract the correct links also in these cases. \n========\n%s\n========\n" - (str/join "\n" (filter identity (map :github-urls references))))) - - (:github-urls gsdm-arxiv) - - (def references (mapv (fn [{:keys [author title]}] - (extract-arxiv-paper (str author " " title))) - (:references gsdm-arxiv))) - - (map :github-urls references) - - - (openai/chat "gpt-4o-mini" - (format "Given the following related work \n========\n%s\n========\n Describe how the following paper is situated in this context: \n========\n%s\n========\n" - (str/join "\n\n\n" (for [r references - :let [text (:text r)] - :when text - :let [text (subs text 0 (min 10000 (count text)))]] - text)) - (:text gsdm-arxiv))) - - (openai/chat "gpt-4o-mini" - (format "Given the following paper \n========\n%s\n========\n Which top 5 references do you want to look up to evaluate the novelty of the paper. Do not reply, return a JSON list of title strings only. Reference titles: \n========\n%s\n========\n" - (:text gsdm-arxiv) - (str/join "\n" (map :title (:references gsdm-arxiv))))) - - (:references gsdm-arxiv) - - - (http/get (format "https://libgen.is/search.php?req=%s&column=title" (.replace "Machine Learning - an Algorithmic Perspective" " " "+"))) - - - (io/copy (:body (http/get "https://download.library.lol/main/263000/be9884dfad35cf3c240b774e8810413b/%28BBC%29%20Susie%20Conklin%2C%20Sue%20Birtwistle%20-%20The%20Making%20of%20Pride%20and%20Prejudice-Penguin%20%28Non-Classics%29%20%282003%29.pdf" {:as :stream})) (io/file "/tmp/test.pdf")) - - - - ) - - -(defn search-libgen [text] - (let [search-page (http/get (format "https://libgen.is/search.php?req=%s" (.replace text " " "+"))) - soup (hickory/as-hickory (hickory/parse (:body search-page))) - information-table (nth (->> soup (select/select (select/tag :table))) 2) - urls (for [row (select/select (select/tag :tr) information-table) - td (select/select (select/tag :td) row) - a (select/select (select/tag :a) td) - :when (= (:content a) ["[1]"])] - (-> a :attrs :href))] - urls)) - -(defn extract-libgen [url] - (let [page (http/get url) - soup (hickory/as-hickory (hickory/parse (:body page))) - title (->> soup (select/select (select/tag :h1)) first :content) - metadata (->> soup (select/select (select/tag :p)) (map (comp first :content))) - download-link (->> soup (select/select (select/tag :a)) first :attrs :href)] - {:title title - :metadata (vec (filter string? metadata)) - :download-link download-link})) - -(comment - (def libgen-urls (search-libgen "norvig artificial intelligence")) - - - (def extracted (mapv extract-libgen libgen-urls)) - - (def extracted-pdfs (filter #(re-find #"pdf" (:download-link %)) extracted)) - - (future - (let [rand-path (str "/tmp/" (java.util.UUID/randomUUID) ".pdf")] - (io/copy (:body (http/get (:download-link (first extracted-pdfs)) {:as :stream})) (io/file rand-path)) - (prn "downloaded " rand-path))) - - ;; extract text from pdf - (def book-text (text/extract "/tmp/test.pdf")) - - (count book-text) - - (openai/chat "gpt-4o" (format "What does the following book say about the relationship between P and NP and how this relates to inference algorithms? Reply with pointers to the specific sections and quotes where suitable.\n======\n%s\n======\n" - (subs book-text 0 400000))) - - (def important-algorithm-reply - (openai/chat "gpt-4o-mini" (format "What is the most important algorithm in the following book? Please try to rewrite it in Clojure and return a single, self-contained, complete namespace with test invocations in a comment block.\n======\n%s\n======\n" - (subs book-text 0 400000)))) - - (println important-algorithm-reply) - - (spit "/tmp/gpt-4o-reply.clj" important-algorithm-reply) - - - ;; extract a clojure markdown block from the reply - (def clojure-block (re-find #"```clojure\n(.|\n)*```" important-algorithm-reply)) - - (re-find #"```clojure\n(.|\n)*```" "```clojure\n(defn foo []\n (println \"Hello, World!\"))\n```") - - (require '[clojure.edn :as edn]) - - (def clojure-src (first (.split (second (.split important-algorithm-reply "```clojure\n")) "```"))) - - (import '[java.io StringReader PushbackReader]) - - (defn read-forms-from-string [s] - (let [reader (PushbackReader. (StringReader. s))] - (loop [forms []] - (let [form (read reader false nil)] - (if (nil? form) - forms - (recur (conj forms form))))))) - - -(doseq [form (read-forms-from-string clojure-src)] - (println form) - (eval form)) - - - - - ) - - - - -(defn semantic-scholar [title] - (e/with-firefox-headless driver - (e/go driver (format #_"https://scholar.google.com/scholar?q=%s" "https://www.semanticscholar.org/search?q=%s&sort=relevance" title)) - (e/set-window-size driver {:width 1920 :height 1080}) - (e/wait 5) - #_(e/get-element-text driver {:tag :body}) - (e/query-all driver {:css "div.cl-paper-row"}) - #_(e/wait-visible driver {:tag :div :class "cl-paper-row"}) - #_(e/get-element-text driver {:tag :div :class "cl-paper-row"}))) - -(comment - (semantic-scholar "Attention+is+All+You+Need") - -(json/read-value - (:body - (http/post "https://api.semanticscholar.org/graph/v1/paper/batch" {:body (json/write-value-as-string {:ids ["649def34f8be52c8b66281af98ae884c09aef38b" "ARXIV:2106.15928" "Attention is all you need"] - :fields "referenceCount,citationCount,title"})}))) - -;; bulk search in clojure -(http/post "https://api.semanticscholar.org/graph/v1/paper/search/bulk" {:body (json/write-value-as-string {:query "covid vaccination" - :fields "paperId,title,year,authors,fieldsOfStudy" - :limit 10 - :publicationTypes "JournalArticle,Review" - :fieldsOfStudy "Medicine,Public Health" - :year "2020-2023"})}) - -(http/get "https://api.semanticscholar.org/graph/v1/paper/search?query=halloween&limit=3") - -"http://export.arxiv.org/api/query?search_query=cat:cs.AI+AND+abs:\"identifiability\"&start=0&sortBy=submittedDate&max_results=10" - -(parse-url (format "http://export.arxiv.org/api/query?search_query=%s&max_results=10" "attention+is+all+you+need") - #_(str (- (.getTime (java.util.Date.)) - (* 1000 60 60 24)))) - - -(-> - (parse-url (format "http://export.arxiv.org/api/query?search_query=abs:\"identifiability\"+AND+abs:\"causal\"&start=0&sortBy=submittedDate&max_results=10" - 0 - #_(str (- (.getTime (java.util.Date.)) - (* 1000 60 60 24))))) - :feed :entries) - - - ) diff --git a/src/ie/simm/processes/thesis.clj b/src/ie/simm/processes/thesis.clj deleted file mode 100644 index dad7884..0000000 --- a/src/ie/simm/processes/thesis.clj +++ /dev/null @@ -1,88 +0,0 @@ -(ns ie.simm.processes.thesis - (:require [clojure.core.async :as async :refer [timeout]] - [superv.async :refer [go-try go-loop-try = (count text) (* 4 (window-sizes model))) - (do - (warn "Text too long for " model ": " (count text) (window-sizes model)) - (throw (ex-info "Sorry, the text is too long for this model. Please try a shorter text." {:type ::text-too-long :model model :text-start (subs text 0 100) :count (count text)}))) - (let [res (create :model model :messages [{:role "system" :content text}])] - (py.- (py.- (first (py.- res choices)) message) content)))) - -(defn image-gen [model text] - (let [res ((py.- (py.- client images) generate) :model model :prompt text)] - (py.- (first (py.- res data)) url))) - -(comment - (image-gen "dall-e-3" "a dog playing in a small house") - - ) - -(defn stt [model input-path] - (let [audio-file ((py.- (py.- (py.- client audio) transcriptions) create) :model model :file ((py/path->py-obj "builtins.open") input-path "rb"))] - (py.- audio-file text))) - -(defn whisper-1 [input-path] - (stt "whisper-1" input-path)) - -(require-python '[pathlib :refer [Path]]) - -(defn tts-1 [text] - (let [res ((py.- (py.- (py.- client audio) speech) create) :model "tts-1" :voice "alloy" :input text) - rand-path (str "/tmp/" (java.util.UUID/randomUUID) ".mp3")] - ((py.- res stream_to_file) (Path rand-path)) - rand-path)) - -(defn openai [[S peer [in out]]] - (let [p (pub in (fn [{:keys [type]}] - (or ({:ie.simm.languages.gen-ai/cheap-llm ::gpt-4o-mini - :ie.simm.languages.gen-ai/reasoner-llm ::gpt-4o - :ie.simm.languages.gen-ai/stt-basic ::whisper-1 - :ie.simm.languages.gen-ai/image-gen ::dall-e-3} type) - :unrelated))) - gpt-4o-mini (chan) - _ (sub p ::gpt-4o-mini gpt-4o-mini) - - gpt-4o (chan) - _ (sub p ::gpt-4o gpt-4o) - - whisper-1 (chan) - _ (sub p ::whisper-1 whisper-1) - - dall-e-3 (chan) - _ (sub p ::dall-e-3 dall-e-3) - - next-in (chan) - _ (sub p :unrelated next-in)] - ;; TODO use async http requests for parallelism - ;; TODO factor dedicated translator to LLM language - (go-loop-try S [{[m] :args :as s} (> (ns-publics 'is.simm.languages.chat) + keys + #_(filter #(re-find #"^:" %)) + (map (comp meta resolve)) + #_(map :spec))) + + (require '[is.simm.runtimes.openai :as openai]) + + (-> + (openai/chat "gpt-4o-mini" (format "You are provided the following Clojure functions.\n===== functions =====\n%s\n==========\nYou need to send a reply to the user greeting them and then send them `report342.pdf`. Reply without any context with pure clojure code invoking the function." + (pr-str chat-vars))) + (.split "```clojure") + second + (.split "```") + first + read-string) + + (require '[sci.core :as sci]) + + ;; get source + (-> #'send-text! + meta + :file + slurp + println) + +;; clojure.core function to retrieve source code of function, example for send-text!: + (clojure.repl/source send-text!) + + @#'send-text! + + send-text! + + + + ) \ No newline at end of file diff --git a/src/ie/simm/languages/dispatch.clj b/src/is/simm/languages/dispatch.clj similarity index 94% rename from src/ie/simm/languages/dispatch.clj rename to src/is/simm/languages/dispatch.clj index 5307097..63aaaa4 100644 --- a/src/ie/simm/languages/dispatch.clj +++ b/src/is/simm/languages/dispatch.clj @@ -1,6 +1,6 @@ -(ns ie.simm.languages.dispatch +(ns is.simm.languages.dispatch "Mapping function dispatch to internal message dispatch to downstream or upstream runtimes." - (:require [ie.simm.languages.bindings :refer [*chans*]] + (:require [is.simm.languages.bindings :refer [*chans*]] [taoensso.timbre :refer [debug]] [clojure.core.async :refer [close! pub sub chan]] [hasch.core :refer [uuid]] diff --git a/src/ie/simm/languages/gen_ai.clj b/src/is/simm/languages/gen_ai.clj similarity index 85% rename from src/ie/simm/languages/gen_ai.clj rename to src/is/simm/languages/gen_ai.clj index 34fe8e7..770e54a 100644 --- a/src/ie/simm/languages/gen_ai.clj +++ b/src/is/simm/languages/gen_ai.clj @@ -1,9 +1,9 @@ -(ns ie.simm.languages.gen-ai +(ns is.simm.languages.gen-ai "Providing high-level generative AI functions for the runtime. The syntax is async (go-routine) functions that invoke lower level runtimes/downstreams with a piece of derived syntax (IR) and handle the replies transparently." - (:require [ie.simm.languages.dispatch :refer [create-downstream-msg-handler]])) + (:require [is.simm.languages.dispatch :refer [create-downstream-msg-handler]])) (let [handler (create-downstream-msg-handler ::cheap-llm)] @@ -24,7 +24,7 @@ (comment - (require '[ie.simm.runtimes.openai :refer [openai]]) + (require '[is.simm.runtimes.openai :refer [openai]]) ;; TODO needs to use pub-sub (let [in (chan) diff --git a/src/ie/simm/languages/notes.clj b/src/is/simm/languages/notes.clj similarity index 72% rename from src/ie/simm/languages/notes.clj rename to src/is/simm/languages/notes.clj index f0d5299..1f50a27 100644 --- a/src/ie/simm/languages/notes.clj +++ b/src/is/simm/languages/notes.clj @@ -1,6 +1,6 @@ -(ns ie.simm.languages.notes +(ns is.simm.languages.notes "General chat interface language. It should be a common denominator for all chat interfaces." - (:require [ie.simm.languages.dispatch :refer [create-downstream-msg-handler]])) + (:require [is.simm.languages.dispatch :refer [create-downstream-msg-handler]])) (let [handler (create-downstream-msg-handler ::related-notes)] diff --git a/src/ie/simm/languages/web_search.clj b/src/is/simm/languages/web_search.clj similarity index 67% rename from src/ie/simm/languages/web_search.clj rename to src/is/simm/languages/web_search.clj index 8f259b8..98e6ab1 100644 --- a/src/ie/simm/languages/web_search.clj +++ b/src/is/simm/languages/web_search.clj @@ -1,5 +1,5 @@ -(ns ie.simm.languages.web-search - (:require [ie.simm.languages.dispatch :refer [create-downstream-msg-handler]])) +(ns is.simm.languages.web-search + (:require [is.simm.languages.dispatch :refer [create-downstream-msg-handler]])) (let [handler (create-downstream-msg-handler ::search)] (defn search [terms] @@ -9,7 +9,7 @@ (comment ;; TODO factor into test - (require '[ie.simm.runtimes.brave :refer [brave]]) + (require '[is.simm.runtimes.brave :refer [brave]]) (let [in (chan) out (chan)] diff --git a/src/is/simm/parse.clj b/src/is/simm/parse.clj new file mode 100644 index 0000000..c1a8c99 --- /dev/null +++ b/src/is/simm/parse.clj @@ -0,0 +1,12 @@ +(ns is.simm.parse + (:require [clojure.data.json :as json])) + +(defn parse-json [json-str] + (let [markdown-body (first (.split (second (.split json-str "```json\n")) "```"))] + (when-not (empty? markdown-body) + (json/read-str markdown-body :key-fn keyword)))) + +(comment + (parse-json "```json\n{\"body\":\"# Hello, world!\"}\n```") + + ) \ No newline at end of file diff --git a/src/ie/simm/peer.clj b/src/is/simm/peer.clj similarity index 85% rename from src/ie/simm/peer.clj rename to src/is/simm/peer.clj index 5ff5c26..8addb4b 100644 --- a/src/ie/simm/peer.clj +++ b/src/is/simm/peer.clj @@ -1,4 +1,4 @@ -(ns ie.simm.peer) +(ns is.simm.peer) (defn add-routes! [peer middleware routes] (swap! peer assoc-in [:http :routes middleware] routes)) \ No newline at end of file diff --git a/src/ie/simm/prompts.clj b/src/is/simm/prompts.clj similarity index 98% rename from src/ie/simm/prompts.clj rename to src/is/simm/prompts.clj index 2ccecdf..8974b81 100644 --- a/src/ie/simm/prompts.clj +++ b/src/is/simm/prompts.clj @@ -1,4 +1,4 @@ -(ns ie.simm.prompts +(ns is.simm.prompts (:require [clojure.java.io :as io])) (def assistance (slurp (io/resource "prompts/assistance.txt"))) diff --git a/src/ie/simm/runtimes/assistance.clj b/src/is/simm/runtimes/assistance.clj similarity index 93% rename from src/ie/simm/runtimes/assistance.clj rename to src/is/simm/runtimes/assistance.clj index ccc3fe3..6ef1f93 100644 --- a/src/ie/simm/runtimes/assistance.clj +++ b/src/is/simm/runtimes/assistance.clj @@ -1,19 +1,19 @@ -(ns ie.simm.runtimes.assistance +(ns is.simm.runtimes.assistance "Active conversation mode that tries to extract factual and procedural knowledge from the user for later use in reasoning and recall. Properties: stateful, persistent, durable" - (:require [ie.simm.languages.bindings :as lb] - [ie.simm.languages.gen-ai :refer [cheap-llm cheap-llm stt-basic image-gen]] - [ie.simm.languages.web-search :refer [search]] - [ie.simm.languages.browser :refer [extract-body]] - [ie.simm.languages.notes :refer [related-notes]] - [ie.simm.languages.chat :refer [send-text! send-photo! send-document!]] - [ie.simm.prompts :as pr] - [ie.simm.db :refer [ensure-conn conversation extract-links msg->txs window-size]] - [ie.simm.peer :as peer] - [ie.simm.http :refer [response]] - [ie.simm.website :refer [md-render default-chrome base-url]] - [ie.simm.runtimes.notes :refer [zip-notes]] ;; TODO move + (:require [is.simm.languages.bindings :as lb] + [is.simm.languages.gen-ai :refer [cheap-llm cheap-llm stt-basic image-gen]] + [is.simm.languages.web-search :refer [search]] + [is.simm.languages.browser :refer [extract-body]] + [is.simm.languages.notes :refer [related-notes]] + [is.simm.languages.chat :refer [send-text! send-photo! send-document!]] + [is.simm.prompts :as pr] + [is.simm.db :refer [ensure-conn conversation extract-links msg->txs window-size]] + [is.simm.peer :as peer] + [is.simm.http :refer [response]] + [is.simm.website :refer [md-render default-chrome base-url]] + [is.simm.runtimes.notes :refer [zip-notes]] ;; TODO move [superv.async :refer [? put? go-for] :as sasync] [clojure.core.async :refer [chan pub sub mult tap timeout] :as async] [taoensso.timbre :refer [debug info warn error]] @@ -94,7 +94,7 @@ ;; subscriptions for this runtime context msg-ch (chan 1000) - _ (sub pi :ie.simm.runtimes.telegram/message msg-ch) + _ (sub pi :is.simm.runtimes.telegram/message msg-ch) ;; do the same in reverse for outputs from below prev-out (chan) @@ -134,6 +134,7 @@ @conn (:id chat)) _ (debug "message count" msg-count)] (when (or (= (:type chat) "private") + (.contains text "@simmis") (.contains text "@simmie")) (let [;; 3. retrieve summaries for active links {:keys [active-links summaries]} (txs window-size]] - [ie.simm.peer :as peer] - [ie.simm.website :refer [md-render default-chrome base-url]] - [ie.simm.http :refer [response]] - [ie.simm.config :refer [config]] + (:require [is.simm.languages.bindings :as lb] + [is.simm.languages.gen-ai :refer [cheap-llm cheap-llm]] + [is.simm.prompts :as pr] + [is.simm.db :refer [ensure-conn conversation extract-links msg->txs window-size]] + [is.simm.peer :as peer] + [is.simm.website :refer [md-render default-chrome base-url]] + [is.simm.http :refer [response]] + [is.simm.config :refer [config]] [superv.async :refer [? put? go-for] :as sasync] [clojure.core.async :refer [chan pub sub mult tap timeout] :as async] [taoensso.timbre :refer [debug info warn error]] @@ -220,10 +220,10 @@ ;; subscriptions for this runtime context msg-ch (chan 1000) - _ (sub pi :ie.simm.runtimes.telegram/message msg-ch) + _ (sub pi :is.simm.runtimes.telegram/message msg-ch) related-ch (chan 1000) - _ (sub pi :ie.simm.languages.notes/related-notes related-ch) + _ (sub pi :is.simm.languages.notes/related-notes related-ch) ;; do the same in reverse for outputs from below prev-out (chan) @@ -261,7 +261,7 @@ @conn (concat active-links (extract-links conv))) _ (debug "related notes reply" active-links summaries)] (put? S out (assoc m - :type :ie.simm.languages.notes/related-notes-reply + :type :is.simm.languages.notes/related-notes-reply :response {:active-links active-links :summaries summaries}))) (recur ( cf + (.thenApply (reify Function + (apply [_ response] + (put! res (-> response + :body + json/read-str + (get "choices") + first + (get "message") + (get "content")))))) + (.exceptionally (reify Function + (apply [_ e] + (put! res (ex-info "Error in OpenAI chat." {:type :error-in-openai :error e})))))) + res)) + + +(defn text-chat [model text] + (let [res (chan)] + (if (>= (count text) (* 4 (window-sizes model))) + (do (warn "Text too long for " model ": " (count text) (window-sizes model)) + (put! res (ex-info "Sorry, the text is too long for this model. Please try a shorter text." + {:type ::text-too-long :model model :text-start (subs text 0 100) :count (count text)})) + res) + (chat model [{"type" "text" "text" text}])))) + +(comment + + (async/= (count text) (* 4 (window-sizes model))) + (do + (warn "Text too long for " model ": " (count text) (window-sizes model)) + (throw (ex-info "Sorry, the text is too long for this model. Please try a shorter text." {:type ::text-too-long :model model :text-start (subs text 0 100) :count (count text)}))) + (let [res (create :model model :messages [{:role "system" :content text}])] + (py.- (py.- (first (py.- res choices)) message) content)))) + + +(defn image-gen [model text] + (let [res ((py.- (py.- client images) generate) :model model :prompt text)] + (py.- (first (py.- res data)) url))) + +(comment + (image-gen "dall-e-3" "a dog playing in a small house") + + ) + +(defn stt [model input-path] + (let [audio-file ((py.- (py.- (py.- client audio) transcriptions) create) :model model :file ((py/path->py-obj "builtins.open") input-path "rb"))] + (py.- audio-file text))) + +(defn whisper-1 [input-path] + (stt "whisper-1" input-path)) + +(require-python '[pathlib :refer [Path]]) + +(defn tts-1 [text] + (let [res ((py.- (py.- (py.- client audio) speech) create) :model "tts-1" :voice "alloy" :input text) + rand-path (str "/tmp/" (java.util.UUID/randomUUID) ".mp3")] + ((py.- res stream_to_file) (Path rand-path)) + rand-path)) + +(defn openai [[S peer [in out]]] + (let [p (pub in (fn [{:keys [type]}] + (or ({:is.simm.languages.gen-ai/cheap-llm ::gpt-4o-mini + :is.simm.languages.gen-ai/reasoner-llm ::gpt-4o + :is.simm.languages.gen-ai/stt-basic ::whisper-1 + :is.simm.languages.gen-ai/image-gen ::dall-e-3} type) + :unrelated))) + gpt-4o-mini (chan) + _ (sub p ::gpt-4o-mini gpt-4o-mini) + + gpt-4o (chan) + _ (sub p ::gpt-4o gpt-4o) + + whisper-1 (chan) + _ (sub p ::whisper-1 whisper-1) + + dall-e-3 (chan) + _ (sub p ::dall-e-3 dall-e-3) + + next-in (chan) + _ (sub p :unrelated next-in)] + ;; TODO use async http requests for parallelism + ;; TODO factor dedicated translator to LLM language + (go-loop-try S [{[m] :args :as s} (? go-loop-try]])) diff --git a/src/is/simm/runtimes/rustdesk.clj b/src/is/simm/runtimes/rustdesk.clj new file mode 100644 index 0000000..0e70a1e --- /dev/null +++ b/src/is/simm/runtimes/rustdesk.clj @@ -0,0 +1,414 @@ +(ns is.simm.runtimes.rustdesk + "This runtime provides obligations from screensharing. + + Languages: screen + It is a source runtime that does not discharge any additional outputs and does not handle other inputs." + (:require [is.simm.peer :as peer] + [is.simm.http :refer [response]] + [is.simm.website :refer [md-render default-chrome base-url]] + [is.simm.runtimes.openai :refer [text-chat chat]] + [is.simm.parse :refer [parse-json]] + [clojure.core.async :refer [timeout put! chan pub sub close! take! poll!] :as async] + [superv.async :refer [go-try S ! "Puts given value on given channel, returns a task completing with true when put is accepted, of false if port was closed." + [c x] (doto (m/dfv) (->> (async/put! c x)))) + +(defn > (async/take! c)))) + +(def base-screenshot-folder "/home/ubuntu/simmis/files/screenshots") ;; Base folder for screenshots + +;; Function to list all screenshots in the screenshot-folder +(defn all-screenshots [screenshot-folder] + (->> (file-seq (io/file screenshot-folder)) + (filter #(re-matches #"frames_.*\.jpg" (.getName %))) + (map #(.getPath %)) + set)) + +(defn encode-image [image-path] + (with-open [input-stream (io/input-stream image-path)] + (let [image-bytes (.readAllBytes input-stream)] + (.encodeToString (Base64/getEncoder) image-bytes)))) + +(defn frame-iterator + "Iterates over all screenshots in the screenshot-folder and updates every 60 seconds. Properly handles backpressure." + [] + (m/ap + (loop [screenshots #{} + to-emit #{} + first-run? true] + (if (empty? to-emit) + ;; if there is nothing to emit, we need to check for new screenshots + (let [new-screenshots (all-screenshots base-screenshot-folder) + [_ delta _] (diff screenshots new-screenshots) + delta (set delta)] + (when-not first-run? + (m/? (m/sleep (* 60 1000)))) + (recur new-screenshots delta false)) + ;; sequentially emit the screenshots + (let [screenshot (first to-emit)] + (m/amb screenshot (recur screenshots (disj to-emit screenshot) false))))))) + +(def schema [{:db/ident :screen/id + :db/valueType :db.type/string + :db/cardinality :db.cardinality/one} + {:db/ident :screenshot/path + :db/valueType :db.type/string + :db/unique :db.unique/identity + :db/cardinality :db.cardinality/one} + {:db/ident :screenshot/transcript + :db/valueType :db.type/string + :db/cardinality :db.cardinality/one} + {:db/ident :screenshot/created + :db/valueType :db.type/instant + :db/cardinality :db.cardinality/one}]) + +(defn extract-timestamp [s] + (let [m (re-matches #".+frames_(\d+)\.jpg" s)] + (java.util.Date. (* 1000 (Long/parseLong (m 1)))))) + +(defn extract-screen [s] + (let [m (re-matches #".+/(\d+)/frames_.+\.jpg" s)] + (m 1))) + +(defn ensure-conn [peer screen-id] + (or (get-in @peer [:conn screen-id]) + (let [path (str "databases/screens/" screen-id) + _ (io/make-parents path) + cfg {:store {:backend :file :scope "simm.is/screens" :path path} + :allow-unsafe-config true} + conn + (try + (let [cfg (d/create-database cfg) + conn (d/connect cfg)] + (d/transact conn schema) + conn) + (catch Exception _ + (d/connect cfg)))] + (swap! peer assoc-in [:conn screen-id] conn) + conn))) + +(def transcripts (read-string (slurp "/home/ubuntu/simmis/transcriptions.edn"))) + +(def screenshot-prompt "You are a note taker for a screen transcription tool. Describe all the visual information as precisely as possible such that it can be retained even if the image is lost. Describe the screen hierarchically. If a browser is visible, write down the URL that is open. Write down *all* text you can see in the respective context. Summarize your observations in the end and try to infer what the user is doing on a high-level.") + +(defn add-screenshot! [conn file-name] + (m/sp + (let [created (extract-timestamp file-name) + screen (extract-screen file-name) + db @conn] + (when-not (d/entity db [:screenshot/path file-name]) + (debug "adding screenshot" file-name) + (try + (d/transact! conn [{:screenshot/path file-name + :screen/id screen + :screenshot/created created + :screenshot/transcript (or (transcripts file-name) + (m/? (> (all-screenshots "/home/ubuntu/simmis/files/screenshots/") + sort + (take-last 100) + (map (fn [s] + (prn "processing" s) + [s (openai/chat "gpt-4o" + [{:type "text" + :text "You are a note taker for a screen transcription tool. Describe all the visual information as precisely as possible such that it can be retained even if the image is lost. Describe the screen hierarchically. If a browser is visible, write down the URL that is open. Write down *all* text you can see in the respective context. Summarize your observations in the end and try to infer what the user is doing on a high-level."} + {:type "image_url" :image_url {:url (str "data:image/jpeg;base64," (encode-image s))}}])])) + (into {}))) + + (> (d/q '[:find ?c ?t + :where + [?i :screenshot/created ?c] + [?i :screenshot/transcript ?t]] @my-conn) + (sort-by first) + reverse + (map (fn [[c t]] [c (remove-headline (last (.split t "\n\n")))]))) + ) + +(defn remove-headline [s] + (let [lines (.split s "\n")] + (.replace + (if (or (.startsWith (first lines) "#") + (.startsWith (first lines) "*")) + (apply str (rest lines)) + (apply str lines)) + "- " ""))) + +(def question-prompt "You are taking to the user with biography:\n======\n%s\n======\nThe user has the following goal:\n======\n%s\n======\nYou are given the following last screenshot descriptions (60 secs apart).\n======\n%s\n======\nAnswer the following question:\n======\n%s\n======\n") + +(def related-work-prompt "You are taking to the user with biography:\n======\n%s\n======\nThe user has the following goal:\n======\n%s\n======\nYou are given the following last screenshot descriptions (60 secs apart).\n======\n%s\n======\nDescribe possible related work searches in a single sentence search query objective for the user to chose. Reply as a JSON array of strings.") + +(def biography "The user is a seasoned Clojure developer who has build distributed databases and has a PhD in machine learning.") + +(def goal "The goal is to build an AI assistant that watches the screen through a remote desktop interface and makes predictions about what the user is going to need or do next. It also makes suggestions to streamline the workflow.") + +(defn ask-question [peer {{:keys [screen-id]} :path-params {:strs [question goal biography]} :params}] + (let [conn (ensure-conn peer screen-id) + _ (prn "Asking question: " question goal) + recent-screens (->> (d/q '[:find ?c ?t + :where + [?i :screenshot/created ?c] + [?i :screenshot/transcript ?t]] @conn) + (sort-by first) + reverse + (map second) + (take 100) + (apply str)) + #_(prn "Recent screens: " recent-screens) + reply-ch (chat "gpt-4o-mini" [{:type "text" :text (format question-prompt biography goal recent-screens question)}]) + reply (> (d/q '[:find ?t :where [?n :note/title ?t]] @conn) + ;; (map first) + ;; sort) + ] + (response + (default-chrome title + [:div {:class "container"} + [:nav {:class "breadcrumb" :aria-label "breadcrumbs"} + [:ul {} + [:li [:span #_{:href "/#"} [:span {:class "icon is-small"} [:i {:class "bx bx-circle"}]] [:span "Systems"]]] + [:li.is-active + [:a {:href (str "/screens/" screen-id)} + [:span {:class "icon is-small"} [:i {:class "bx bx-chat"}]] + [:span title]]]]] + [:div {:class "container"} + [:h2 {:class "title is-2 is-spaced" :id "bots"} + [:a {:class "" :href "bots"} [:i {:class "bx bx-bot"}]] + [:span {:class ""} "Screen assistant"]] + [:div {:class "content"} + [:p (format "There are %d screenshots from this screen." screenshot-count)]] + [:div {:class "container" :id "assistant"} + [:form {:class "box"} + [:div.field + [:label.label "Biography"] + [:div.control + [:textarea {:class "textarea assistant" :rows 5 :placeholder "Biography" :id "biography" :name "biography"} biography]]] + [:div.field + [:label.label "Goal"] + [:div.control + [:textarea {:class "textarea assistant" :rows 5 :placeholder "Your goal." :id "goal" :name "goal"} goal]]] + [:div.field + [:label.label "Question"] + [:div.control + [:input {:class "input is-primary assistant" :type "input" :placeholder "Ask a question..." :id "question" :name "question"}]]] + [:button {:class "button is-primary" :hx-post (str "/screen/" screen-id "/ask") :hx-target "#assistant" :hx-trigger "click" :hx-swap "outerHTML" :hx-include ".assistant" :hx-on "click: this.disabled = true"} "Ask"] + [:div {:class "box" :style "margin-top: 10px;"} + [:div.field + [:label.label "Reply"] + [:div {:class "content" :id "reply"} "-"]]]]] + [:div {:class "container" :id "screenshots" :style "margin-top: 10px;"} + [:div.box + [:div.content + [:h4 {:class "title is-4 is-spaced"} + [:a {:class "" :href "screenshots"} "# "] + [:span {:class ""} "Recent screenshots"]] + (let [screenshots (->> (d/q '[:find ?c ?t ?p + :where + [?i :screenshot/created ?c] + [?i :screenshot/transcript ?t] + [?i :screenshot/path ?p]] @conn) + (sort-by first) + reverse + (take 100) + (map (fn [[c t p]] [c + t #_(remove-headline (last (.split t "\n\n"))) + (.replace (.replace p "/home/ubuntu/simmie" "") + "/home/ubuntu/simmis" "")])))] + (if (seq screenshots) + [:ul + (map (fn [i [c t p]] + [:li {:class "highlight"} #_[:a {:href (str "/screen/" screen-id "/screenshots/" c)} c] + [:p {} (md-render t)] + [:img {:src p :width "100%"}]]) + (range) + screenshots)] + "No screenshots."))]]]]])))) + +(defn rustdesk + ([[S peer [in out]]] + (let [routes [["/screen/:screen-id" {:get (partial #'screen-overview peer)}] + ["/screen/:screen-id/ask" {:post (partial #'ask-question peer)}]]] + + (def dispose! + ((main peer) #(prn ::success %) #(prn ::error %))) + + (def my-peer peer) + (peer/add-routes! peer :screen routes) + [S peer [in out]]))) diff --git a/src/ie/simm/runtimes/telegram.clj b/src/is/simm/runtimes/telegram.clj similarity index 90% rename from src/ie/simm/runtimes/telegram.clj rename to src/is/simm/runtimes/telegram.clj index 5282f59..a55fd3c 100644 --- a/src/ie/simm/runtimes/telegram.clj +++ b/src/is/simm/runtimes/telegram.clj @@ -1,9 +1,9 @@ -(ns ie.simm.runtimes.telegram +(ns is.simm.runtimes.telegram "This telegram runtime provides obligations from a Telegram bot. Languages: chat It is a source runtime that does not discharge any additional outputs and does not handle other inputs." - (:require [ie.simm.config :refer [config]] + (:require [is.simm.config :refer [config]] [morse.handlers :as h] [morse.polling :as p] [clojure.core.async :refer [put! chan pub sub close! take!]] @@ -57,6 +57,9 @@ _ (def channel (p/start (:telegram-bot-token config) bot-api))] #(p/stop channel))) +(defn get-webhook-info [token] + (http/get (str "https://api.telegram.org/bot" token "/getWebhookInfo"))) + (defn telegram ([[S peer [in out]]] (telegram server [S peer [in out]])) @@ -70,9 +73,9 @@ prev-out (chan) po (pub prev-out (fn [{:keys [type]}] - (or ({:ie.simm.languages.chat/send-text ::send-text - :ie.simm.languages.chat/send-photo ::send-photo - :ie.simm.languages.chat/send-document ::send-document} type) + (or ({:is.simm.languages.chat/send-text ::send-text + :is.simm.languages.chat/send-photo ::send-photo + :is.simm.languages.chat/send-document ::send-document} type) :unrelated))) send-text (chan) _ (sub po ::send-text send-text) @@ -92,7 +95,7 @@ (when m (debug "sending telegram message:" chat-id msg) (put? S next-in (assoc m - :type :ie.simm.languages.chat/send-text-reply + :type :is.simm.languages.chat/send-text-reply :response (try (t/send-text (:telegram-bot-token config) chat-id msg) ;; TODO bug in slingshot forces us to catch Throwable for HTTP errors, ;; https://github.com/scgilardi/slingshot/issues/60 @@ -106,7 +109,7 @@ (when m (debug "sending telegram photo:" chat-id url) (put? S next-in (assoc m - :type :ie.simm.languages.chat/send-photo-reply + :type :is.simm.languages.chat/send-photo-reply :response (try (t/send-photo (:telegram-bot-token config) chat-id url) (catch Throwable e @@ -118,7 +121,7 @@ (when m (debug "sending telegram document:" chat-id url) (put? S next-in (assoc m - :type :ie.simm.languages.chat/send-document-reply + :type :is.simm.languages.chat/send-document-reply :response (try (t/send-document (:telegram-bot-token config) chat-id url) (catch Throwable e @@ -130,8 +133,8 @@ (comment - (require '[ie.simm.languages.chat :refer [send-text!]] - '[ie.simm.languages.bindings :refer [*chans*]]) + (require '[is.simm.languages.chat :refer [send-text!]] + '[is.simm.languages.bindings :refer [*chans*]]) (let [in (chan) out (chan) diff --git a/src/ie/simm/runtimes/text_extractor.clj b/src/is/simm/runtimes/text_extractor.clj similarity index 59% rename from src/ie/simm/runtimes/text_extractor.clj rename to src/is/simm/runtimes/text_extractor.clj index 1e32ece..30ee92a 100644 --- a/src/ie/simm/runtimes/text_extractor.clj +++ b/src/is/simm/runtimes/text_extractor.clj @@ -1,29 +1,92 @@ -(ns ie.simm.runtimes.text-extractor +(ns is.simm.runtimes.text-extractor "Extract texts from incoming message and augment the text. Properties: stateless" - (:require [ie.simm.languages.bindings :as lb] - [ie.simm.languages.gen-ai :refer [cheap-llm stt-basic]] - [ie.simm.languages.chat :refer [send-text!]] - [ie.simm.languages.browser :refer [extract-body]] - [ie.simm.prompts :as pr] + (:require [is.simm.languages.bindings :as lb] + [is.simm.languages.gen-ai :refer [cheap-llm stt-basic]] + [is.simm.languages.chat :refer [send-text!]] + [is.simm.languages.browser :refer [extract-body]] + [is.simm.prompts :as pr] [superv.async :refer [? put? go-for] :as sasync] [clojure.core.async :refer [chan pub sub mult tap timeout] :as async] [taoensso.timbre :refer [debug info warn error]] [hasch.core :refer [uuid]] [clojure.string :as str] - [libpython-clj2.require :refer [require-python]] - [libpython-clj2.python :refer [py. py.. py.-] :as py])) + #_[libpython-clj2.require :refer [require-python]] + #_[libpython-clj2.python :refer [py. py.. py.-] :as py] + + [babashka.http-client :as http] + [clojure.data.json :as json] + [clojure.string :as str] + )) + +(def watch-url "https://www.youtube.com/watch?v=") + +(defn- raise-http-errors [response video-id] + (let [status (:status response)] + (if (= 200 status) + (:body response) + (throw (ex-info (str "YouTube request failed for video: " video-id) + {:status status + :video-id video-id}))))) + +(defn fetch-video-html [client video-id] + (let [response @(http/get (str watch-url video-id) {:headers {"Accept-Language" "en-US"} :client client :async true})] + (raise-http-errors response video-id))) + +(defn extract-captions-json [html video-id] + (if-not (re-find #"\"captions\":" html) + (throw (ex-info "Transcripts are not found for this video" {:video-id video-id}))) + + (let [captions-data (-> (str/split html #"\"captions\":") + second + (str/split #"\"videoDetails") + first + clojure.string/trim + (json/read-str :key-fn keyword))] + (if-let [captions-json (get captions-data :playerCaptionsTracklistRenderer)] + (if (contains? captions-json :captionTracks) + captions-json + (throw (ex-info "No transcript available for this video" {:video-id video-id}))) + (throw (ex-info "Transcripts are disabled for this video" {:video-id video-id}))))) + +(defn fetch-transcript [client video-id] + (let [html (fetch-video-html client video-id) + captions-json (extract-captions-json html video-id)] + (for [caption (:captionTracks captions-json)] + {:language (:languageCode caption) + :url (:baseUrl caption)}))) + +(defn fetch-transcript-data [client transcript-url] + (let [response @(http/get transcript-url {:headers {"Accept-Language" "en-US"} :client client :async true})] + (->> (json/read-str (:body (raise-http-errors response transcript-url)) + :key-fn keyword) + :events + (map #(select-keys % [:segs :tOffsetMs :dDurationMs]))))) + +(def client (http/client (assoc-in http/default-client-opts [:proxy] {:host "localhost" :port 8118}))) + +;; Example usage with async: -(require-python '[youtube_transcript_api :refer [YouTubeTranscriptApi]]) (defn youtube-transcript [video-id] + (-> (fetch-transcript client video-id) + first + :url + (http/get {:client client}) + :body + (str/replace #"<[^>]*>" " "))) + + +#_(require-python '[youtube_transcript_api :refer [YouTubeTranscriptApi]]) + +#_(defn youtube-transcript [video-id] ;; " ".join([t['text'] for t in transcript]) (let [transcript (py. YouTubeTranscriptApi get_transcript video-id)] (str/join " " (map :text transcript)))) (comment - (youtube-transcript "20TAkcy3aBY") + (fetch-transcript client "20TAkcy3aBY") (youtube-transcript "wkH1dpr-p_4") @@ -70,7 +133,7 @@ pub-subs (chan) _ (tap mi pub-subs) p (pub pub-subs (fn [{:keys [type]}] - (or ({:ie.simm.runtimes.telegram/message ::message} type) + (or ({:is.simm.runtimes.telegram/message ::message} type) :unrelated))) msg-ch (chan 1000) next-in (chan 1000) diff --git a/src/ie/simm/simmie.clj b/src/is/simm/simmis.clj similarity index 92% rename from src/ie/simm/simmie.clj rename to src/is/simm/simmis.clj index 82d7a43..41427c3 100644 --- a/src/ie/simm/simmie.clj +++ b/src/is/simm/simmis.clj @@ -1,11 +1,11 @@ -(ns ie.simm.simmie +(ns is.simm.simmis (:require [taoensso.timbre :as log] [taoensso.timbre.appenders.core :as appenders] [superv.async :refer [S go-try nor < would work for this, because they depend on getting a value. So that does not work.\"\n - **Later:** \"Or m/batch should be added that would work similar to m/sample but would be designed to work with discrete processes, and could do it.\"\n - \"I hope I am not talking nonsense here... I took 2 days to think about this, and was reading the docs, the cookbook and missionary code over and over again.\"\n - **Reply Timestamp:** 1 reply Today at 12:28 AM\n\n#### 5. Message Thread Panel:\n- **User:** whilo\n - **Time:** 11:57 AM\n - **Message:** \n - \"Ok, thanks! I really love your work. I am also aiming at removing the backend-frontend boundary costs with replikativ...\"\n - \"Ultimately I want to integrate my PhD work on compiling Clojure to diffusion models for probabilistic inference...\"\n - Links: `https://github.com/whilo/simmie`, `https://github.com/plai-group/daphne`\n - **GitHub Repos:**\n - whilo/simmie\n - **Stars:** 2\n - plai-group/daphne\n - **Stars:** 37\n - **Language:** Clojure\n - Discussion on datalake and differential data flow.\n - Additional remarks about the Swedish government's tests on swapping in datahike for Datomic.\n\n#### 6. Additional Elements:\n- **Unread messages notification:** \"More unread\"\n- **Search Bar:** Placeholder text \"Search clojurians\"\n- **Sidebar Options:**\n - Home\n - Activity\n - Later\n - More\n\n### Summary and High-Level Inference:\nThe Slack workspace for \"clojurians\" is focused on discussions related to Clojure, a programming language. The user is viewing the #missionary channel with discussions centered around Clojure's operational processes and potential improvements. The thread includes technical conversations about software repositories related to Clojure projects, specifically concerning missionary, a Clojure library. The discussion involves technical considerations on implementing asynchronous processes and integrating distributed systems, with references to specific GitHub repositories and their uses in academic and professional settings. The overall activity is highly technical, likely aimed at developing or troubleshooting complex software projects in Clojure.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024426.jpg" "### Visual Information Description\n\n#### Application: Visual Studio Code\n\n**Top Bar:**\n- Title: `rustdesk.clj - simmie [SSH: telegram] - Visual Studio Code`\n- Menu: File, Edit, Selection, View, Go, Run, Terminal, Help\n\n**Main Layout:**\n- **Left Panel:**\n - Header: CALVA INSPECTOR\n - Contains multiple lines of text, primarily file paths and IDs such as:\n - `#object[clojure.core.async.impl.channels.ManyToManyChannel 0x782967a8 \"clojure.core.async.impl.channels.ManyToManyChannel@782967a8\"]`\n - `#file ... /simm.runtimes.rustdesk/kernel`\n - Repeated patterns of file names and object references like `#file ...` and `#object[...]`\n - Symbols like `nil` and `(#function ...)` are present.\n \n- **Center Panel (Editor):**\n - Filename: `simmie.clj`\n - Code structure with syntax highlighting:\n - Lists imports such as `[clojure.data.json :refer [write-json]]`\n - Definitions using `def` for encoding images, file paths, functions, etc.\n - Functions like `defn encode-image` dealing with image manipulation.\n - Conditional constructs with `let`, `recur`, etc.\n - `main` function containing `try` and `catch` blocks.\n - Time extraction using Java utilities `java.util.Date`.\n \n- **Right Panel (Output/Terminal):**\n - Files with paths like `/home/ubuntu/screenshots/454345782/frames_1728909561.jpg`\n - Status messages repeating text `new screenshot` followed by a file path.\n - Line numbers on the left, indicating tracking of new screenshots.\n \n**Bottom Bar:**\n- Status information about version control and file properties:\n - Branch indicator: `main*`\n - Git status: `0`\n - Characters encoding: `UTF-8`\n - File type: Clojure\n - Line and character positions: `Ln 59, Col 1`\n - Active extensions display: `Calva: Joyride`\n\n### Summary and Inference\n\nThe user is working within Visual Studio Code, specifically on a Clojure project, indicated by file names and syntax. The current file `simmie.clj` is involved with processing screenshots—potentially encoding images and organizing the image files based on timestamp and other factors. \n\nThe left panel shows inspection or debugging output, likely Calva related (a Clojure and ClojureScript interactive programming environment for VS Code). The right panel lists operational logging or console output, repeatedly logging the creation of new screenshots, suggesting an active monitoring or batch processing task. \n\nOverall, the user seems to be developing or debugging a Clojure application that handles screenshots, perhaps for an automation tool or screenshot management system. The use of SSH indicates remote development or server management tasks.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024528.jpg" "### Screen Overview\n\n#### Application: \n- Visual Studio Code (VS Code) is open.\n\n#### Editor Layout:\n- Three-pane layout.\n\n---\n\n### Left Pane: Explorer\n\n#### File Structure:\n- **Folder:** `SIMMLIE [SSH TELEGRAM]`\n - Subfolders: `.calva`, `.clojure`, `.nrepl`, `.vscode`, `assets`, `classes`, `code`, `downloads`, `jokes`, `models`, `neoars`, `process`, `public`, `resources`, `samples`, `src`, and others.\n - **Active Folder:** `examples`\n - **Subfolder:** `runtimes`\n - Files: `base32.clj`, `clojure.clj`, `clojurescript.clj`, `constant.clj`, `decoder.clj`, `encoder.clj`, `report.clj`, `reporter.clj`, `routstock.clj`, `telegram.clj`, `text_extractor.clj`\n - Note: Files such as `telegram.clj`, `routstock.clj`, and `encoder.clj` have modifications (indicated by an \"M\" icon).\n\n---\n\n### Center Pane: Editor\n\n- **File:** `simmlie.clj`\n - Code sections include functions and variable definitions.\n - Includes imports from different libraries and methods.\n - Notable functions or keywords:\n - `process-frames`, `extract-timestamp`, `test-screenshots`\n - Usage of libraries such as `clojure.string`, `clojure.core.async`, `spy.core`\n - A comment about a path `/home/ubuntu/screenshots/454345782/frames_...`\n \n---\n\n### Right Pane: Terminal/Output\n\n- **File:** `output.calva-repl`\n - Lines output image paths such as:\n - `/home/ubuntu/screenshots/454345782/frames_...jpg`\n - Clojure code snippets and shell commands are present (`nil`, `SIMM.runtimes.rustdesk`)\n - Example text query: \"You are a note taker for a screen transcription tool.\"\n\n---\n\n### Miscellaneous\n\n#### Status Bar:\n- Information present: Source control information (branch `main`, file changes), Clojure-specific environment info, mode (`NORMAL`), line, column, spaces, language mode, etc.\n\n---\n\n### Summary and Inference\n\nThe user is working within Visual Studio Code on a project titled `SIMMLIE [SSH TELEGRAM]` with a focus on dealing with image screenshots and possibly a transcription tool, indicated by the presence of image processing functions. The output panel suggests that the user is testing or verifying paths of stored screenshots. The user is utilizing Clojure for scripting and organizing functions. The context indicates active development or debugging of features related to processing image data within a larger application framework.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024457.jpg" "### Visual Structure:\n\n#### Application:\n- **Application**: Visual Studio Code\n- **Environment**: Remote connection (SSH) to a server named \"telegram\"\n- **Active Files**: Multiple tabs, including:\n - `simmie.clj`\n - `nudatex.clj`\n - `async.clj`\n - `default_schema.edn`\n - `tower.clj`\n - `peer.clj`\n - `openai.clj`\n\n#### Layout:\n- **Split View**: \n - **Left Panel**: File Explorer\n - Directory path: `~/.output-window/`\n - Project folder: `simmie`\n - Subdirectories and files under `src` include `languages`, `processes`, `telegram`, `twitter`, etc.\n - Highlighted file: `examples.clj` under `telegram`.\n\n - **Middle Panel**: Code Editor\n - File: `runtimes.clj`\n - Contains Clojure code with function definitions and variables.\n - Comments: Indicated with `;;`\n - Functions: `extract-timestamp`, `process-frames`, `put!`, `def test-screenshots`, `encode-to-based64`, `require`, `def last-screenshot`, `map count`, etc.\n - Sample Code Snippet:\n ```clojure\n (def test-query\n (openai/chat \"gpt-4q\"\n (vec (concat \"{\\nYou are a note taker for a screen transcription tool. Describe all the visual information ... }\\n\")))\n )\n ```\n\n - **Right Panel**: Search/Output Results\n - Output text includes snippets related to application use, function snippets, and process management summary.\n - Keywords and highlighted terms: `Application`, `Slack`, `workspace`, `core-type`, `data-science`, `process management`, etc.\n - List small code snippets and file references.\n\n#### Interface:\n- **Status Bar**:\n - Indicators for line and column numbers\n - Git Branch information: `main`\n - Language mode: Clojure\n\n- **Sidebar Tools**:\n - Explorer\n - Source Control\n - Extensions\n\n- **Menu Bar**:\n - Options include File, Edit, Selection, View, Go, Run, Terminal, Help.\n\n### Summary and Inference:\nThe user is working on a Clojure-based project, specifically focusing on a file called `runtimes.clj`. The visible code and comments suggest the project involves processing image files and screenshots, possibly for automation or data parsing tasks. The user seems to be integrating functions related to screenshot handling, OpenAI API communication, and IDE-based development. The right panel indicates context for development tools and their configurations, which might relate to project setup or management. This setup is typical for a development environment involving automation or data processing tasks with connections to external services like OpenAI.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729027293.jpg" "# Visual Description of the Screen\n\n## Application Window\n- **Application**: Visual Studio Code\n- **Title Bar**: \n - Title: `rustdesk.clj - simmie [SSH: telegram] - Visual Studio Code`\n \n## Layout\n- **Split View**: Three main vertical panels\n\n### Left Panel\n- **Section**: Explorer\n- **Directory Structure**:\n - simmie [SSH: TELEGRAM]\n - .zshrc\n - .arcadies\n - .cache\n - ...\n - languages (expanded)\n - runtimes\n - `basic.c9` (selected)\n - elixir.clj\n - deno.clj\n \n### Center Panel\n- **Open File**: `basic.c9`\n- **Code Content**:\n - Comments and code snippets are visible.\n - Code includes Clojure syntax with functions such as `encode-to-base64`, `require`, and `def defvar`.\n - A function `test-query` is defined with subfunctions and parameters `[chat gpt-4]`.\n - Text related to chatbot processing is present.\n \n### Right Panel\n- **Output/Terminal**: `output.calva-repl`\n- **Content**:\n - Logs and error messages from a REPL session are visible.\n - Items like error traces, evaluation commands, and session details are listed.\n - Certain lines indicate processes like encoding, error handling, and logging.\n \n## Status Bar\n- **Information**:\n - `Ln 126, Col 1` (Cursor position)\n - `Spaces: 2` (Indentation setting)\n - `UTF-8` (Encoding)\n - `Clojure` (Language mode)\n - Additional indicators for Git branch and file status.\n\n## Code Elements:\n- Functions, logic control structures, variables, and paths are visible.\n- Code hints at file manipulation and interaction with AI chatbots (e.g., `openai/chat`).\n\n## Summary and Inference\nThe user is working in Visual Studio Code, possibly on a Clojure project. The focus is on developing scripts that involve language runtimes and handling files, potentially for AI processing or interaction. The presence of various scripts and file paths suggests development work related to automation tasks, data encoding/decoding, and chatbot integration. The terminal logs indicate active debugging processes, error handling, and code execution.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729009893.jpg" "**Screen Description:**\n\n**Overall Structure:**\n- The screen displays a dark-themed webpage within a web browser. The URL visible in the address bar is \"github.com/xyrobbes/vfbdbt\".\n\n**Browser Elements:**\n- The browser has a visible tab bar at the top with numerous open tabs.\n- There is a URL address bar showing the address of the current page.\n- Standard browser navigation icons (back, forward, reload), and various extensions or bookmarks are visible.\n\n**Content Hierarchy:**\n\n1. **Header:**\n - Title: \"README\"\n\n2. **Main Content:**\n - An image is displayed predominantly near the top of the page.\n - Image text (overlay or caption): \"Using AI to mutate the canvas\"\n - Included elements: Visual representation of a digital interface with multiple data tables or grids.\n - There are futuristic graphics in the background, featuring rows of rabbits.\n\n - Caption below the image: \"User Space Kits and Plug-ins\"\n\n3. **Section: Getting Started:**\n - Subheader: \"Getting Started\"\n - Bulleted instructions or notes:\n - \"Give it a try - I'm still working on proper documentation (for example, the entire Clover DSL is not yet documented), please open GH issues, discussions or reach out to me on Twitter if you have any questions!\"\n - \"Go to http://localhost:8888\"\n - \"Still working on legit documentation, but for now:\"\n - Additional bullet point explanations regarding folders and file paths:\n - \"The /assets/ folder - this is for user content like images, files, etc. Can be referenced as http://localhost:8888/assets/* - very useful for anything that the backend needs to read/write to that the frontend can also access.\"\n - \"The /defs/ folder - this is where all config EDN files are\"\n - \"The /connections/ folder - this is where your SQL db connections are. They are each an EDN map with JDBC connection info that gets use by a Hikari connection pool (so there can be Hikari specific settings)\"\n - \"The /screens/ folder - this is where screen or 'deck' definitions live (in the middle of a re-naming of things) a screen can be opened on boot by going to http://localhost:8888/#/[my-screen-name] - or by using the\"\n\n4. **Additional Interface Elements:**\n - There might be some navigation or editing icons along a toolbar at the top of the webpage section, albeit barely visible.\n\n**Summary and High-Level Inference:**\n- The user is viewing a GitHub repository page, likely related to a software project involving AI and visual canvas manipulation.\n- The \"Getting Started\" section indicates a focus on setting up or instructing users on how to begin using the software. It includes instructional content on accessing local server resources and organizing project files.\n- There seems to be an emphasis on visual representation and user interface customization, likely using AI features as suggested by the imagery and text captions.\n- Overall, the user is engaged in reviewing or contributing to project documentation for a development project hosted on GitHub.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729009901.jpg" "### Visual Information Description:\n\n#### Application Window:\n- **Title Bar:**\n - Title: `output.calva-repl - simmie [SSH: telegram] - Visual Studio Code`\n\n- **Menu Bar:**\n - Options: `File`, `Edit`, `Selection`, `View`, `Go`, `Run`, `Terminal`, `Help`\n\n#### Side Panel:\n- **Icons:**\n - Search, Source Control, Extensions, Debug, Account\n\n#### Main Panel:\n- **Tabs Open:**\n - `simmie.clj`, `rudisteck84_work.clj`, `default_schema.adm`, `tower.clj`, `openssl`, `original_cm`\n\n- **Editor Section (Left):**\n - Language: Clojure\n - Code Snippets: \n ```clojure\n (output-files screenshots-folder)\n ;(file-seq (io/file screenshots-folder))\n (file-seq-matches #\"frames/\\.*\\.jpg$\" getName)\n (map .getPath)\n ```\n ```clojure\n (def process-frames [session term-ch]\n (let [screenshots-folder (user :screenshots-folder session)]\n (go-loop [ss (process-frames term-ch)]\n (let [ new-screenshots (all-screenshots screenshots-folder)\n [ old-screenshots new-screenshots]= new-screenshots]\n (go-loop [ss screenshots]\n (let [ details]\n (or ( ie > simmiie**\n - Folders: runtimes, processes, rustdesk.\n - Files: `extractor.clj`, `clojure.clj`, `peer.clj` (selected), `telegram.clj`, `tweet.clj`, etc.\n\n### Main Editor (Central)\n- **File: rustdesk.clj**\n - Logical blocks of code in Clojure language.\n - Functions like `process-frames`, `main`, `dispose`, and `extract-timestamp`.\n - Code features including `let` bindings, `mapv`, `reduce`, `defn`, and various Clojure data structures (vectors, maps).\n\n### Editor (Right Panel)\n- **File: output.cljs.repl**\n - Contains a REPL output with multiple entries logging file paths and technical logs concerning executing processes.\n - Many lines stating `new screenshot` with file paths like `/home/ubuntu/screenshots/...`.\n\n### Status Bar (Bottom)\n- Information about the editing context such as LN: 55, Col: 1, Spaces: 2, UTF: 8, LF, CLJ, Normal mode, Clojure.\n\n### Observations and Inferences\n- The user is working on a Clojure project in Visual Studio Code.\n- They are dealing with file management and processing, possibly related to screenshots or image frames based on file path logs.\n- The `process-frames` and `extract-timestamp` functions suggest work involving image processing or managing image file metadata.\n- Use of REPL indicates testing or running Clojure code interactively.\n- User is managing multiple files related to a project indicating it involves multiple components or features, potentially for a larger application or setup involving Clojure.\n\nIn summary, the user is actively engaged in a development session focused on handling image file processing, utilizing Clojure functions to automate or manage certain processes within a large Clojure-based codebase.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024455.jpg" "**Visual Information Description:**\n\n1. **Application and Environment:**\n - The application being used is Visual Studio Code.\n - The screen is divided into several sections: Explorer, text editor panes, and a sidebar for additional information.\n\n2. **Explorer Pane:**\n - Located on the left side.\n - Lists folders and files in a hierarchical format.\n - Main directory: \"simmie\"\n - Subdirectories: \n - \"src\"\n - \"simm\"\n - Files and folders: \"languages.clj\", \"runtimes\", \"bot\", \"clean.clj\", \"extractor.clj\", \"reporter.clj\", \"telegram.clj\", \"text_extractor.clj\", \"http.clj\", \"peer.clj\", \"primitives.clj\", \"test.clj\", and \"twitter.clj\".\n - \"classes\", \"docs\", \"download\", \"resources\", \"public\", etc.\n - Additional files: \".gitignore\", \"README.md\", \"LICENSE\", etc.\n\n3. **Editor Panes:**\n - **Left Editor Pane:**\n - Displays code: \n - Language: Clojure.\n - Functions like `encode-to-based`.\n - Code for processing screenshots:\n ```clojure\n (defn extract-timestamp [filename])\n (def test-screenshot [])\n (encode-to-based [data])\n ```\n - Includes comments for further clarification.\n\n - **Right Editor Pane:**\n - Displays a long text, which appears to document or describe features and functionalities.\n - Contains code snippets and references to various programming functions.\n\n4. **Open Tabs:**\n - Tabs open at the top, including: \n - \"simmie.clj\", \"telegram.clj\", \"default_schema.edn\", \"deps.edn\", \"output.edn\", \"README.md\".\n\n5. **Status Bar:**\n - Information shown: \n - The file \"runtimes.clj\" is being edited.\n - Line and column number: 100:14.\n - Other settings like \"Spaces: 2,\" \"UTF-8,\" etc.\n\n6. **Additional Context:**\n - The session is identified as \"SSH: telegram.\"\n - Various lines of potentially related code or documentation in the background, indicating comprehensive development tasks.\n\n**Summary and Inferences:**\n\n- The user is working within a Clojure project focused on features like screenshot processing and integration with services like Telegram.\n- There's active development or documentation related to software functionalities. The project might involve backend or automation tasks given the file names and structure.\n- The Visual Studio Code setup and various files suggest a comprehensive software development environment, likely for Clojure-based applications, possibly involving multiple modules or microservices.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024437.jpg" "**Visual Hierarchy and Description:**\n\n1. **Top Bar:**\n - Main Menu: \n - File\n - Edit\n - Selection\n - View\n - Go\n - Run\n - Terminal\n - Help\n\n - Open File Tabs:\n - simmie.clj\n - nuttedsk.clj\n - nuttedsk.ck.edn\n - output.calva-repl\n - default_schema.edn\n - tower.clj\n - peer.clj\n - openai.clj\n - bot.c\n - test.clj\n\n2. **Left Sidebar:**\n - Title: EXPLORER\n - Directory Structure:\n - .clj-kondo\n - .git\n - deps.edn\n - src\n - simmie\n - runtimes\n - rustdesk.clj (highlighted)\n - telegram.clj\n - text_extractor.clj\n - .github\n - charts\n - classes\n - deps\n - downloads (icon for download)\n - docs\n - nrepl\n - nodes\n - notebooks\n - public\n - resources\n - scripts\n - example_config.edn\n - .gitignore\n - build.clj\n - CODE-OF-CONDUCT.md\n - deops.edn\n - LICENSE\n - README.md\n - server.log\n\n3. **Center Panel:**\n - Open File: rustdesk.clj\n - Clojure Code Snippets:\n ```clojure\n (def dispose! \n (main prn! :success #(! prn! :error)))\n ```\n - Functions for extracting timestamps, schemas, handling errors, and reading images.\n - Code comments and annotations.\n - Use of 'def' for defining functions and variables.\n - Use of various libraries and namespaces like java.util.Date.\n\n4. **Right Panel:**\n - Open File: output.calva-repl\n - Clojure REPL Output:\n - Listing screenshots from specific directories.\n - Occurrence of errors and cancellation messages.\n - Various `cli` commands and outputs.\n - Execution of processes, showing success and error cases.\n\n5. **Bottom Bar:**\n - Displaying Git branch and status (master*).\n - Language mode (Clojure).\n - Line, column, and space information (Ln 84, Col 32).\n - File type and encoding info.\n\n**Summary and High-Level Inference:**\n\nThe user is working in Visual Studio Code, focused on Clojure development, specifically editing and testing a Clojure script (`rustdesk.clj`). This script seems to involve processing and handling screenshot files, possibly with timestamps or schemas. The REPL output on the right indicates active testing or debugging, as errors and command outputs are being analyzed. The environment is set up for a project involving multiple languages and tools, reflecting an organized workflow in software development, specifically in scripting automation or data processing through Clojure.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729027308.jpg" "**Visual Information Description:**\n\n**1. Application:**\n - Visual Studio Code is open.\n\n**2. Window Structure:**\n - Top bar with menu items: File, Edit, Selection, View, Go, Run, Terminal, Help.\n - A status bar at the bottom showing `SSH: telegram ✗ main`, `⊕`, `2,434`, `744`, `0`, `16`, `⚙️ Clojure`, `☁️`, `⊝`, `eopmit`.\n\n**3. Content Layout:**\n - Split into three main vertical panels.\n\n**4. Left Panel:**\n - Explorer tab active.\n - Folder structure visible:\n - simmie\n - .db\n - output-window\n - .clojure\n - deps.edn\n - .shadow-cljs\n - compile.js\n - resources\n - languages\n - runtimes (expanded)\n - abstractos.clj\n - base.clj\n - etc…\n - runtime.clj (selected file)\n - .gitignore\n - LICENSE\n - README.md\n - server.log\n\n**5. Middle Panel:**\n - Editing area showing `runtime.clj` file code.\n - The code is in Clojure language.\n - Key structures visible:\n - Multi-line `comment` block.\n - `def`, `map`, `let`, `into` Clojure functions used.\n - Variables such as `term`, `test-query`, `transcriptions`.\n - Use of external tool `openai/chat` in code.\n\n**6. Right Panel:**\n - Terminal window open.\n - Displays a list of files all named as frames with `.jpg` extension, located at `/home/ubuntu/simmie/files/screenshots/`.\n - Processing log with messages displaying \"processing\".\n - Command prompt at the bottom, last command run: `clojure -e simn.runtimes.rustdesk/transcriptions`.\n\n**7. Tabs Open:**\n - Multiple file tabs open at the top, names include: `simmie.clj`, `rustdesk.clj`, `example_config.edn`, `peer.clj`, `output.clj.html`.\n\n**Summary and Inference:**\nThe user is working on a Clojure programming project using Visual Studio Code connected through SSH. The focus is on the file `runtime.clj`, which involves scripts related to screenshot processing and possibly interacting with the OpenAI chat API. The terminal is actively logging processes related to image files stored on the system. The user's actions suggest they are debugging or developing code for a tool that processes and interacts with screenshots, possibly involving automated transcription or image analysis functionalities.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024518.jpg" "### Visual Description Hierarchy\n\n1. **Application: Visual Studio Code (VS Code)**\n - **Top Toolbar:**\n - Menu Options: File, Edit, Selection, View, Go, Run, Terminal, Help.\n - **Tabs Open:**\n - `simmi.utils.clj`\n - `simmie.clj`\n - `nudxtest.clj`\n - `default_schema.edn`\n - `tower.clj`\n - `perci.clj`\n - `openai.clj`\n - **Status Bar:**\n - Branch: main\n - Editor status: Ln 59, Col 1\n - Spaces: 2\n - UTF-8\n - LF\n - Clojure language mode\n - REPL status\n - Connection Status: Already at newest change.\n\n2. **Side Panel:**\n - **Clojure Inspector:**\n - Various listings in a hierarchical format, prefixed with `#`, indicating comments or meta information such as file paths and namespaces.\n - Lines contain entries like: \n - `#object[clojure.core.async.impl.channels.ManyToManyChannel ...]`\n - `#file is.simmi.runtimes.rustdesk/`\n - Related files and functions are listed, such as `/process-frames/`, `/frame-files/`.\n\n3. **Main Code Editing Panel:**\n - **File: `simmie.clj`:**\n - **Namespace Declaration:**\n - Various imports and usage declarations for libraries like `clojure.java.io`, `clojure.data.json`, etc.\n - **Function Definitions:**\n - `defn base-screenshot-folder`\n - `defn all-screenshots`\n - `defn encode-image`\n - `defn process-frames`\n - Main function with real-time operations described.\n - **Code:**\n - Code snippets detailing file handling and image processing.\n - Use of loops, delta calculations, and timestamp parsing.\n - **Comments:**\n - Present to describe functions and operations.\n\n4. **Right Panel:**\n - **Output or Log Display:**\n - Continuous logging of events such as new screenshots being saved.\n - Log entries: \"new screenshot /home/ubuntu/screenshots/...jpg\"\n - Timestamped entries indicating the process flow and debug information.\n\n### Summary and High-Level Inference\n\nThe user is coding and debugging a Clojure application in Visual Studio Code. The main tasks seem to involve image and file processing, specifically handling screenshots in the `/home/ubuntu/screenshots/` directory. The user is working with functions to manage screenshots, including listing, encoding, and processing image files. The side and right panels show logs and outputs from running this code, indicating active development and debugging efforts. This work might be related to automating screenshot management for a project, possibly for a Rustdesk-related activity or a server-side process under development.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024527.jpg" "The screen displays the Slack application in a web browser. The URL currently open is `https://app.slack.com/client/T03RZCGRP/CL8SMB9PF?cdn_fallback=2`.\n\nHere's the hierarchical breakdown of the screen contents:\n\n### Browser Interface\n- **URL**: `https://app.slack.com/client/T03RZCGRP/CL8SMB9PF?cdn_fallback=2`\n- **Bookmarks Bar**: Numerous icons are present, indicating saved bookmarks or extensions.\n\n### Slack Interface\n\n#### Left Sidebar\n- **Workspace Name**: \"clojurians\"\n- **Drafts & sent**: An icon for drafts with \"8\" indicated.\n- **Home**: An icon with \"Home\" label, \"Activity\", \"Later\", \"More\" options.\n- **Channels**: \n - `#admin-announcements`\n - `#anglican`\n - `#asami`\n - `#babashka`\n - `#clara`\n - `#clj-kondo`\n - `#cljs-dev`\n - `#clojure`\n - `#clojure-bay-area`\n - `#clojureuml`\n - `#clojurescript`\n - `#closh`\n - `#clr`\n - `#consultantsdirectory`\n - `#core-typed`\n - `#cortex`\n - `#data-science`\n - `#datahike`\n - `#datalog`\n - `#datascript`\n - `#datomic`\n - `#declarative-dataflow`\n - `#events`\n - `#graalvm`\n - `#holy-lambda`\n - `#hyperfiddle`\n - `#jank`\n - `#klipse`\n - `#local-first-clojure`\n - `#metaprob`\n - `#missionary` (highlighted)\n - `#mxnet`\n - `#offtopic`\n\n#### Main Content Area\n- **Channel**: `#missionary`\n- **Messages**\n - **awb99 (Today at 11:56 AM)**\n - \"The operator that I believe is missing could be m/flow-or ...\"\n - \"1 reply Today at 12:28 AM\"\n - **whilo (Yesterday at 7:10 PM)**\n - \"Hey! I really like FRP and am happy that missionary ...\"\n - \"8 replies Last reply Today at 12:02 PM\"\n - **Absolute Negativity (Today at 6:55 AM)**\n - \"I just ran into this too. *ignored-metadata* didn’t work ...\"\n - Code snippet: \n ```clojure\n (defn foo [jdbc select]\n ;; etc.\n ```\n - \"View newer replies\"\n- **Input Field**\n - \"Message #missionary\"\n - Icons for formatting, attachments, and more options.\n\n#### Right Sidebar (Thread)\n- **whilo (Today at 11:57 AM)**\n - A lengthy message about technical topics, describing integration of Clojure and missionary for AI and scaling purposes.\n - Mentions:\n - HTTPS Links: `https://github.com/whilo/simmie`, `https://github.com/plai-group/daphne`\n - Star ratings for repositories:\n - simmie: Stars 2\n - daphne: Stars 37\n - Text includes additional context about using datahike and swapping data for Datomic optimizations.\n\n### Summary\nThe user is in a Slack channel titled `#missionary` within the \"clojurians\" workspace. The conversation involves advanced discussions about the Clojure programming language, involving functional reactive programming (FRP), integration with AI, data flow, and repositories on GitHub. Participants are discussing technical enhancements and sharing code snippets and resources.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729027299.jpg" "# Visual Description\n\n## Application Window\n- **Application**: Visual Studio Code\n- **Title Bar**: Displays \"rustdesk.clj - simmie [SSH: telegram] - Visual Studio Code\"\n- **Split View**: The window is divided into three main panels.\n\n### Left Panel (File Explorer)\n- **Section**: EXPLORER\n- **Repository**: simmie [SSH: telegram]\n- **Highlighted Path**: `... > src > ie > sim > runtimes`\n- Files and Folders:\n - `.output`\n - `src`\n - `ie`\n - `sim`\n - `runtimes`\n - `extract.clj`\n - `report.clj`\n - `telegram.clj`\n - `texts.clj`\n - `twitter.clj`\n - etc.\n - `project.clj`\n - `.gitignore`\n - `README.md`\n\n### Middle Panel (Editor)\n- **File Open**: `simmie.clj`\n- **Editing Mode**: Normal\n- **Lines of Code**: 80-140 visible\n- **Code Content**:\n - Usage of Clojure language with syntax highlighting.\n - Code sections include:\n - `def` statements\n - Comments: Extract and process functions\n - Example use of `encode-to-based64` function.\n - `let` bindings and vector operations.\n - Function declarations with sample data manipulation.\n\n### Right Panel (Output/Debug Console)\n- **Section**: OUTPUT\n- Fields that are visible:\n - Details of a generated screenshot saved as \"jpeg\".\n - Stack trace or processing log that includes:\n - Use of Clojure functions.\n - Error messages related to interruption and execution.\n - Log messages prefixed with `nREPL middleware` responses.\n - Example Commands:\n - Calls to functions, indicating actions like \"processing\" and \"transcriptions\".\n\n## Bottom Bar\n- **Indicators**:\n - Git branch: `main`\n - Line: 120 | Column: 16\n - Spaces: 2\n - Statuses for coding and debugging tools.\n\n# Summary\nThe user is working within Visual Studio Code, primarily engaged in writing and debugging code for a project named \"simmie,\" using the Clojure language. The file being actively edited appears to be dealing with a task related to processing image data or handling screenshots, likely for a transcription tool or similar application. The right panel indicates that the user is viewing logs related to the execution and testing of their code. The context suggests ongoing software development, possibly involving automated data processing or similar tasks, accessed over an SSH connection labeled \"telegram.\"", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024439.jpg" "### Visual Information Description\n\n#### Application\n- **Application Name:** Visual Studio Code\n- **Window Title:** `rustdesk.clj - simmie [SSH: telegram] - Visual Studio Code`\n\n#### Layout\n- **Three Main Sections:**\n 1. **Explorer Panel (Left Sidebar):**\n - **Header:** EXPLORER\n - **Project Name:** `simmie [SSH: TELEGRAM]`\n - **File Tree:**\n - `/`: \n - `cljs/`\n - `output_window.cljs`\n - `.clojure/`\n - `output.calva-repl`\n - `.vscode/`\n - `.idea/`\n - `src/`\n - `calva/`\n - `languages/`\n - Various `.clj` files (e.g., `runtime.clj`, `base.clj`, etc.)\n - `report.clj`\n - `report2.clj`\n - `telegram.clj` (highlighted)\n - `text_extractor.clj`\n - `input.clj`\n - `output.clj`\n - `peer.clj`\n - `twtweet.clj`\n - Other files (e.g., `project.clj`, `deps.edn`, `LICENSE`, etc.)\n \n 2. **Main Editor (Center Panel):**\n - **File:** `simmie.clj`\n - **Content:** Contains Clojure code that appears to define functions and logic related to processing frames and screenshots. It seems to involve:\n - Functions like `extract-timestamp`, `process-frames`, and others.\n - Command `peer/add-routes!`.\n\n 3. **Output/Log Panel (Right Panel):**\n - **File:** `output.calva-repl`\n - **Content:** Log messages, primarily \"new screenshot\" followed by file paths, such as:\n - `/home/ubuntu/simmie/files/screenshots/454354782/frames/... .jpg`\n - File path appears to be consistent with varying image filenames.\n\n#### Status Bar (Bottom)\n- **Information:**\n - `Ln 75, Col 17 Spaces: 2 UTF-8 Clojure LSP`\n - Connection indication to `telegram` with 5 items queued.\n\n### Summary and Inferences\n- **High-Level Activity:**\n - The user is programming using Clojure within Visual Studio Code.\n - They are connected via SSH to a remote environment named \"telegram\".\n - The focus is on developing and testing a system related to processing and managing screenshots, as indicated by the function names and output log messages.\n- **Purpose & Functionality:**\n - The system appears to involve handling screenshots, likely involving timestamp extraction and frame management.\n - Multiple `.clj` files suggest modular code possibly related to different functionalities like running, reporting, telegram integration, and text extraction.\n- **Development Phase:**\n - The presence of active logs and a detailed file structure suggests the user is in a testing or debugging phase.\n\nOverall, the user is deeply engaged in software development concerning image or screenshot processing, utilizing Clojure and integrating with remote systems for development or continuous integration tasks.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024522.jpg" "# Image Transcription\n\n## Screen Overview\nThe screen displays a Visual Studio Code editor on a dark-themed background. The interface is divided into three main sections:\n\n1. **File Explorer (Left Panel)**\n2. **Code Editor (Middle Panel)**\n3. **Output Window (Right Panel)**\n\n### Detailed Description\n\n#### Visual Studio Code Header\n- **Window Title**: \"rustdesk.cli - simmie [SSH: telegram] - Visual Studio Code\"\n- **Menu Bar Options**: File, Edit, Selection, View, Go, Run, Terminal, Help\n\n#### File Explorer (Left Panel)\n- **Project Name**: simmie (SSH: telegram)\n- **Directories and Files Displayed**:\n - Hidden files: `.bashrc`, `.gitignore`, etc.\n - Folders:\n - `examples`\n - `resources`\n - `resources/public`\n - `src`\n - Files (some with modification indicators):\n - `default_schema.edn`\n - `example_config.edn`\n - `simmie.clj` (currently selected)\n - `report.clj` (modified)\n - `telegram.clj`\n - `text_extractor.clj`\n - `config.clj`\n - `simmie.clj`\n - `.gitignore`\n - `README.md`\n - `server.log`\n\n#### Code Editor (Middle Panel)\n- **File Opened**: `simmie.clj`\n- **Code Snippets**:\n - Functions defined: `extract-timestamp`, `process-frames`, `put!`, `test-screenshots`, etc.\n - Macro usage: `defn`, `let`, `require`\n - Key variables: `delta`, `current`, `term-ch`, `longParseLong`\n - Comments inline: \n - `; comment`\n - Details about processing screenshots and encoding them.\n - Output code snippet involving JSON encoding:\n ```clojure\n {:type \"image/ur\", :image_ur (str \"data:image/jpeg;base64,\" sp)}\n ```\n\n#### Output Window (Right Panel)\n- **Tab Open**: `output.calva-repl`\n- **Contents**: Sequential console output logs\n - Repetitive logs stating: `\"new screenshot\" \"/home/ubuntu/screenshots/454345782/frames_1728996749.jpg\"`\n - Logs are numbered sequentially from `1521` to `1599`.\n\n### Summary and Inference\n\n- **Work Context**: The user is working on a Clojure project called \"simmie\" using Visual Studio Code. The project involves processing and handling images, specifically screenshots, as indicated by function names and log outputs.\n- **Specific Task**: The code involves functions for timestamp extraction, screenshot processing, and possibly encoding images in base64 format for further use.\n- **Output Analysis**: The right panel logs demonstrate output indicating the storage or processing of image files, likely showing completed steps or stages in the processing sequence.\n \nThe user appears to be working on developing or debugging a feature related to screenshot handling in a Clojure project.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024436.jpg" "### Overview\nThe screen displays the Visual Studio Code (VS Code) environment. It is configured in a dark theme and covers various aspects of file management, coding, and terminal output.\n\n### Top Menu Bar\n- **Options**: File, Edit, Selection, View, Go, Run, Terminal, Help.\n\n### Explorer Panel (Left Side)\n- **Path/File Structure**:\n - Several collapsed directories such as `.clj`, `apache`, `classes`, `downloads`, `resources`, etc.\n - Highlighted file: `rustdesk.clj` located under `simmie > runtimes`.\n\n### Main Editor Panel (Center)\n- **File Opened**: `rustdesk.clj` within `src > simmie > runtimes`.\n- **Code Context**:\n - Primary language: Clojure.\n - Functions include `main`, `timestamp`, and `dispose`.\n - Involves operations with timestamps, file paths, and processing screenshots.\n - Example code snippets:\n ```clojure\n (defn main...)\n (let [timestamp (re-matches #\"...jpg\" ...)]\n (extract-timestamp \"/home/ubuntu/screenshots/...jpg\")\n ```\n\n### Additional Tabs Open\n- `simmie.clj`\n- `default_schema.edn`\n- `output.calva-repl`\n- Additional files related to project schemas and outputs.\n\n### Terminal Panel/Output Panel (Right Side)\n- **Visible Output**: Listings of image file paths such as:\n - `\"new screenshot\" \"/home/ubuntu/screenshots/454345782/frames_17299047371.jpg\"`\n - A sequence from similar lines indicating multiple screenshot paths.\n- **Highlighted Commands**:\n ```clojure\n (clj ie.simm.runtimes.rustdesk)\n nil\n ```\n - Suggests usage of a REPL for interactive coding/testing.\n\n### Bottom Status Bar\n- **Information Displayed**:\n - `Ln 64, Col 1`: Current cursor position.\n - `Spaces: 2`: Tab settings.\n - `UTF-8`: File encoding.\n - `Clojure`: Programming language mode.\n - `Normal`: Editing mode.\n\n### Summary\nThe user is likely developing or maintaining a Clojure project using VS Code. The specific focus is on a file named `rustdesk.clj` related to managing or processing screenshot data. The terminal and REPL outputs indicate active testing or logging of functions related to this data. The user might be debugging or developing functionality related to timestamp extraction and file management in Clojure.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024443.jpg" "### Visual Description of Screen in Visual Studio Code\n\n#### General Layout\n- **Application:** Visual Studio Code\n- **Tabs Open:** \n - `simnie.clj - simnie (SSH: telegram)`\n - `rustdesk.clj - simnie (SSH: telegram)`\n - `default_schema.edn`\n - `peer.clj`\n - `openai.clj`\n - `output.calva-repl`\n\n#### Sidebar (Left)\n- **Sections:**\n - **EXPLORER**\n - Presented as a directory tree structure.\n - Folders:\n - `.clj-kondo`\n - `.output`\n - `simnie` (expanded):\n - Folders: `languages`, `resources`\n - Files (selected and open): Many files with `.clj` extension such as `runtimes`, `extract.clj`, `telegram.clj`, `report.clj` (highlighted), etc.\n - Miscellaneous files like `README.md`, `LICENSE`, `deps.edn`, etc. \n\n#### Main Editor Area (Center)\n- **File Open:** `simnie.clj`\n- **Code Snippet:**\n - Clojure code.\n - Contains function definitions and operations such as `defn`, `let`, `reduce`, `doseq`.\n - Example functions: `process-frames`, `defschema`, `extract-timestamp`, `term-ch`.\n - Code structure includes indentation and brackets, typical of Clojure syntax.\n - Comment and code sections highlighted in different colors for differentiation.\n\n#### Secondary Editor Panel (Right)\n- **File Open:** `output.calva-repl`\n- **Content:**\n - Appears to be system output or logs.\n - Repeated message format: `\"new screenshot\" \"/home/ubuntu/simnie/files/screenshots/454354782/frames_172094058XX.jpg\"`\n - Incremental numbers indicating sequential file naming.\n\n#### Footer\n- **Information:**\n - `ln 74, col 1` indicates the cursor position.\n - Language mode: `Clojure`.\n - File setup status, e.g., `Spaces: 2`, `UTF-8`.\n - Git status indicator - changes in files (`M` for modified), clean status, etc.\n\n### Summary and High-Level Inference\nThe user is developing or analyzing a Clojure-based project in Visual Studio Code, specifically working on files related to the `simnie` project. The main focus seems to be on Clojure functions, potentially related to processing or interacting with image files based on the log outputs in the adjacent panel. The repetitive log entries suggest the user is processing multiple images or screenshots, possibly automating tasks related to image analysis or capture. The environment is set up for remote development using SSH on a server named \"telegram,\" indicated in the file paths and SSH designation. This reflects an active development or debugging session involving analyzing screenshots and adjusting code accordingly.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024466.jpg" "### Visual Hierarchy and Information:\n\n**Application:**\n- **Software:** Visual Studio Code\n- **Layout:** SSH session to `telegram`\n- **Windows:** \n - **Three Main Panels:** \n - Left (File Explorer)\n - Center (Code Editor)\n - Right (Terminal and Output)\n\n---\n\n**Top Panel (Tabs Bar):**\n- Active Tabs: \n - `simme.clj`\n - `default_schema.edn`\n - `peer.clj`\n - `calva-repl`\n - Others visible tabs (partially): \n - `output-window`\n - `def...` (unfinished)\n\n---\n\n**Left Panel (File Explorer):**\n- **Project Directory:**\n - **SIMME (SSH TELEGRAM)**\n - Directories:\n - `.vscode`\n - `.calva`\n - `arcadia`\n - `classes`\n - `Class`\n - `Downloads`\n - `notes`\n - `resources`\n - `runtimes`\n - `save`\n - `screenshots`\n - `public`\n - `src`\n - Files:\n - `.gitignore`\n - `README.md`\n - `LICENSE`\n - `server.loj`\n - `build.clj`\n - `CHANGELOG.md`\n - `deps.edn`\n - **Content in `src` Directory:**\n - **Subdirectory: languages**\n - **Subdirectory: runtimes**\n - Files:\n - `base.clj`\n - `control.clj`\n - `estonia.clj`\n - `mass.clj`\n - `repeater.clj`\n - `reporter.clj`\n - `spider.clj`\n - `telegram.clj`\n - `tweets.clj`\n - `text_extractor.clj`\n - `.clj` files with percentage indicators (possibly showing changes/conflicts)\n\n---\n\n**Center Panel (Code Editor):**\n- **File: simme.clj**\n- **Code Examples:**\n ```clojure\n (ns ...\n (:require [clojure.core.async :refer [timeout pub ...]])\n (:import ...))\n ```\n\n ```clojure\n (defn ...\n (let [{:keys [x]} ...]\n ...))\n ```\n\n ```clojure\n (defn dispose!\n [m]\n (prn m))\n (prn \"new screenshot\" ...)\n ```\n\n---\n\n**Right Panel (Terminal and Output):**\n- **Output Text:** Repeated logs of \"new screenshot\" messages\n - File paths: \n - `/home/ubuntu/simme/files/screenshots/454354782/frames...`\n- **Advanced Terminal (REPL and Output):** \n - Paths and code strings visible\n - Highlighted lines of references to various `.clj` files and functions\n\n---\n\n### Summary and Inference:\nThe user is working in Visual Studio Code through an SSH connection to a remote environment named `telegram`. The project appears to involve Clojure development, possibly focused on handling screenshots. The left panel shows project structure, the center panel includes code scripts, and the right panel demonstrates terminal outputs, primarily logging the creation of new screenshots.\n\nHigh-Level Activity:\nThe user is likely developing or maintaining a Clojure-based application that processes or manages screenshots, perhaps related to a project named SIMME. Frequent updates, directory navigation, and code editing suggest active development or debugging of the codebase. The terminal output signifies a focus on real-time processing, likely involving image capturing or data processing functions.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024454.jpg" "Title Bar:\n- \"rustdesk.clj - simmie - Visual Studio Code\"\n\nMenu Bar:\n- File, Edit, Selection, View, Go, Run, Terminal, Help\n\nTabs:\n1. `simmie.clj`\n2. `runtimes.clj`\n3. `network.clj`\n4. `default_schema.edn`\n5. `towork.cjs`\n6. `peer.clj`\n7. `openai.clj`\n\nExplorer Panel (left side):\n- Current directory \"simmie (SSH: telegram)\"\n- Subdirectories and files:\n - `daily`\n - `object-odyssey`\n - `src`\n - `simmie`\n - `languages`\n - `runtimes`\n - `base.clj`\n - `extractor.clj`\n - `io.clj`\n - `reporter.clj`\n - `rustdesk.clj` (highlighted)\n - `telegram.clj`\n - `text_extractor.clj`\n - Others: `http.clj`, `network.clj`, `peer.clj`, etc.\n - `resources`, `public`, `bb.edn`, etc.\n\nMain Coding Area (center):\n- Code in `runtimes.clj`:\n ```clj\n (def test-openai\n (let [openaid (chat \"gpt-4da\")\n _ (vec (concat [\"You are a note taker for a screen transcription tool. You get the following recent\"\n \" last test-screenshots\"]\n (for [...])))))\n ```\n\nOutput Panel (right side):\n- Displays logs or outputs with content like file paths, and function calls.\n- Sections of text are marked with relevant topics or identifiers, e.g., \"Application,\" \"Missionary,\" \"cljs,\" etc.\n\nCode Overview (right scroll bar):\n- Code map showing highlights and locations of code changes and errors.\n\nSummary and Inference:\nThe screen shows a Visual Studio Code setup with several tabs, mostly associated with a project involving Clojure (a Lisp-like programming language). The user is likely working on a development project related to screen transcription, involving handling runtime files. The code panel shows work with Clojure functions for processing data, possibly using the OpenAI API. Based on the directories and files, the project may involve libraries or modules for various operations like network, text extraction, and interaction with external tools (e.g., Telegram, RustDesk, OpenAI). The user might be debugging or developing new features, as there are code logs and a structured setup indicating a development workflow.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024461.jpg" "## Screen Overview\n\n### Application\n- **Software**: Visual Studio Code\n- **Project**: `rustdesk.clj`\n- **Connection**: SSH to a remote machine (`[SSH: telegram]`)\n\n### Layout\nThe screen is divided into three main panels: \n\n1. **Left Panel (Explorer)**\n - **Title**: Explorer\n - **Project Tree**:\n - **Folders and Files**:\n - `.output.window`\n - `.output.calva-repl`\n - `.lsp`\n - `_local`\n - `apache`\n - `classes`\n - `classes2`\n - `classes3`\n - `downloads`\n - `ideas`\n - `resources`\n - `simmie`\n - `languages`\n - `runtimes`\n - Highlighted file: `rustdesk.clj`\n - `telegrams.clj`\n - `text_extractor.clj`\n - `static`\n - `tmp.dl`\n - `db.clj`\n - `peer.clj`\n - `prompts.clj`\n - `tweets.clj`\n - `twitter.clj`\n - **Other Files**:\n - `.gitignore`\n - `build.clj`\n - `deps.edn`\n - `LICENSE`\n - `README.md`\n - `server.log`\n\n2. **Middle Panel (Code Editor)**\n - **File**: `rustdesk.clj`\n - **Code Snippets**:\n - Functions and definitions, including:\n ```clojure\n (def test-screenshots \n (sort (all-screenshots \"/home/ubuntu/simmie/files/screenshots/\")))\n (def last-test-screenshots \n (take-last 10 test-screenshots))\n ```\n\n - **Active Function**: \n ```clojure\n (def test-openai\n (let [openai/chat \"gpt-4.0\"\n vec (concat [{:type \"text\"\n :text \"You are a note taker for a screen transcription tool. Describe all the visual information as precisely as possible such that it can be retained even if the image is lost.\"} \n last-test-screenshots]\n ...)])\n ```\n\n3. **Right Panel (Output/Terminal)**\n - **Section**: `.output.calva-repl`\n - **Content**:\n - Includes summary notes, system logs, error messages, and outputs related to code execution.\n\n - **Highlighted Content**:\n - Time stamps, image URLs, and descriptions\n - Commands and function invocations like:\n ```clojure\n ie.simm.runtimes.rustdesk/last-screenshots\n ie.simm.runtimes.rustdesk/test-query\n ```\n\n### General Observations\n- **Theme**: Dark theme with syntax highlighting for code clarity.\n- **Language**: Clojure\n- **Enabled Features**: Calva (for Clojure development)\n- **Active Task**: Code development, likely around image processing or backend service interaction with `OpenAI`.\n\n### High-Level Inference\n\nThe user is actively developing a Clojure-based application or script involving image processing and interaction with OpenAI. The use of multiple SSH connections suggests the user is working on tasks that involve remote server resources. The detailed notes in the REPL output imply an iterative testing process, possibly focused on enhancing or debugging functionalities specific to `rustdesk.clj`.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024460.jpg" "# Visual Studio Code Environment\n\n## Application Header\n- **Name**: Visual Studio Code\n- **Project**: 'rustdesk.clj - simmie [SSH: telegram]'\n- **Status Bar**: Displays current branch `main`, Git changes `224↑ 213↓ 16`, text editing options (e.g., Spaces: 2, UTF-8), file type detected as `Clojure`, and additional information `LN 100, COL 18` indicating cursor position.\n\n## Main Layout\n\n### Panel Layout\n- **Top Panel**: \n - **Tabs**: Multiple files open including:\n - `simmie.clj`\n - `output.calva-repl`\n - `default_schema.edn`\n - `peer.clj`\n - `operator.clj`\n - `openai.clj`\n \n### Three Panels\n1. **Left Panel (File Explorer)**\n - **Directory**: Project root labeled 'simmie (see telegram)'\n - **Subdirectories/Files**:\n - `.dir-locals.el`, `archives`, `backend`, `.calva-output-window`, etc.\n - **Expanded Folders**:\n - `src`: Contains files such as `simmie.clj` (highlighted), `telegran.m`, `peer.clj`, `openai.clj`, etc.\n - Different icons denote status (`M` for modified, `U` for untracked).\n\n2. **Center Panel (Code Editor)**\n - **File Open**: `simmie.clj`\n - **Highlighted Code**: Clojure code syntax with functions and definitions:\n - `(def test-screenshots ...)`, `(def test-query ...)`, `(require ...)`\n - Encapsulation of functions like `encode-to-based`, `require`, `sub`.\n - Comments denoted by `;;`.\n\n3. **Right Panel (Side Output/Documentation)**\n - **File Open**: `output.calva-repl`\n - Includes session logs and output from running code:\n - Discussion and debugging information.\n - Screenshots outputs such as `'new screenshot: /home/ubuntu/simmie/files/screenshots/...'`\n\n## Side Bar and Footer\n- **Explorer Tab**: Active\n- **Minimap**: Provides an overview of the code in the central editor.\n\n## High-Level Summary\n- The user is working within a Visual Studio Code environment, editing Clojure scripts in a structured project setup.\n- They are managing files within the `simmie` directory, possibly a backend/front-end project involving screenshot processing and session management.\n- Output logs and session summaries are visible in the `output.calva-repl` file, indicating active testing and iteration of code.\n- It's apparent that the user is utilizing version control with Git operations and SSH access to a project labeled `telegram`.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729027303.jpg" "# Screen Hierarchical Breakdown\n\n## Top Bar\n- **Application**: Visual Studio Code\n- **File Path**: `rustdesk.clj - simmie [SSH: telegram]`\n\n## Sidebar (Left Panel)\n- **Sections**:\n - **EXPLORER**:\n - Root Directory: `simmie [SSH: TELEGRAM]`\n - Subdirectories:\n - `.ssh`\n - `src`\n - Subdirectories:\n - `simmie`\n - `output-window`\n - `clojurebot-repl`\n - `app.clj`\n - `arcadia`\n - `cb`\n - `classes`\n - `docs`\n - `downloads`\n - `resources`\n - `promptproj`\n - `public`\n - `scp`\n - `tests`\n - `output`\n - `default_schema.edn`\n - `example_config.edn`\n - `processes`\n - Files:\n - `runtimes`\n - `custom.clj`\n - `bot.clj`\n - `citation.clj`\n - `extract.clj`\n - `report.clj`\n - `report2.clj`\n - `telegram.clj`\n - `text_extractor.clj`\n - Other Files:\n - `.gitignore`\n - `.github`\n - `build.clj`\n - `README.md`\n - `LICENSE`\n - `deps.edn`\n - `server.log`\n\n## Main Editor (Center Panel)\n- **File**: `simmie.clj` (Code file)\n- **Content**:\n - **Comments** and **Code Statements**:\n - `(comment` block indicating screenshot processing paths and details.\n - **Function Definitions**:\n - `def sc`\n - `def test-query`\n - `def transcriptions`\n - Code involving:\n - `all-screenshots`\n - `image-url`\n - `extract-timestamp`\n - **Function Calls** and **Library Requirements**:\n - `require`\n - `encode-to-base64`\n - `map`\n - `clojure.core`\n\n## Right Panel\n- **File**: `output.clj`\n- **Content**:\n - **Error Logs** and **Stack Traces**\n - **Text References and Comments**:\n - Descriptions and processing indications within log entries.\n - **Special Notations**:\n - Execution errors\n - Line numbers indicating possible error points\n\n## Status Bar (Bottom)\n- **Editor Settings and Status**:\n - **Git Branch**: main\n - **Editor Mode**: Normal\n - **Space/Indentation Settings**: Line 120, Column 71, Spaces: 2\n - **Language**: Clojure\n - **Other Indicators**: Git branch, changes, and layout settings\n\n# Summary\nThe user is working in Visual Studio Code primarily with a Clojure project connected over SSH named \"simmie\". The activities involve handling file paths, extracting information, and performing operations related to image processing or session management. The content suggests dealing with screenshot processing, transcriptions, and potentially debugging errors (as seen in the right panel with stack traces). The main focus is on the file `simmie.clj`, with various tasks like encoding, mapping, and querying within this session.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024432.jpg" "### Screen Description Hierarchy\n\n#### Application\n- Visual Studio Code\n\n#### Main Sections\n1. **Explorer Panel (Left)**\n - Title: \"EXPLORER\"\n - Folder: `SIMNIE [SSH TELEGRAM]`\n - File Tree:\n - `window`\n - `output-calva-repl`\n - `deps.edn`\n - `LICENSE`\n - `README.md`\n - `project.clj`\n - `server.log`\n - `src/simn`\n - `runtimes`\n - `rustdesk.clj`\n - (other `.clj` files listed indicating edits or modifications, marked by 'M' or '%')\n - Outline and Timeline sections are mentioned but not expanded.\n\n2. **Editor Panels (Center and Right)**\n - **Center Panel**: File `rustdesk.clj`\n - Various Clojure code snippets for processing image sessions, timestamps, and coordinates.\n - Functions like `process`, `main`, `dispose`, and `extract-timestamp`.\n - Comments and code suggest functionality for screenshots and timestamp extraction.\n \n - **Right Panel**: File `output.calva-repl`\n - Continuous log output listing new screenshots.\n - Sample log entries:\n - \"1524 new screenshot '/home/ubuntu/screenshots/454345782/frames_1728984268.jpg'\"\n - Repeated pattern of log entries indicating the processing of images.\n\n#### Bottom Bar\n- Status Information\n - Branch: `SSH: telegram main`\n - Line: `Ln 53, Col 100`\n - Spaces: `2`\n - UTF: `UTF-8`\n - Clojure version and memory usage details.\n\n#### Top Bar\n- Tabs Open\n - `simnnie.clj - simnie [SSH telegram]`\n - Additional files: `rustdesk.clj`, `default_schema.edn`, `tower.clj`, etc.\n\n### Summary and High-level Inference\n\nThe user is working in Visual Studio Code on a Clojure-based project named \"simnie\" accessed via SSH labeled \"telegram.\" The left panel's file explorer shows project structure and source files, with emphasis on the `runtimes` folder containing modified Clojure code files. The center editor displays a Clojure script for the `rustdesk.clj` file, detailing functions for processing images and extracting timestamps from screenshot files. The right panel displays a REPL-like output log showing the processing of sequential screenshots, likely for analysis or documentation. Overall, the user is developing or debugging a script involved in handling and analyzing large sets of screenshot data, possibly remotely or in a shared environment.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024464.jpg" "### Visual Hierarchical Description\n\n**Application: Visual Studio Code**\n\n- **Title Bar:**\n - Contains the text: `rustdesk.clj ⨯ simmie.clj ⨯ default_schema.edn ... - Visual Studio Code`\n - Indicates SSH connection to a server labeled `[SSH: telegram]`.\n\n- **Menu Bar:**\n - Options: File, Edit, Selection, View, Go, Run, Terminal, Help.\n\n- **Main Window Layout:**\n - **Three main panels:**\n 1. **Left Panel (File Explorer):**\n - Displays a project directory named `simmie`.\n - Contains a hierarchical file structure:\n - Folders:\n - `.clj-kondo`, `.github`, `.repl`, `archive`, `backup`, `classes`, `docs`, `downloads`, `eseercs`, `flowclards`, `modules`, `notes`, `resources`, `promptrc`, `public`, `repos`, `simmie`, `tasks`, and more.\n - Files in the selected project: `default_schema.edn`, `example_config.edn`, with subfolders like `languages`, `telegram.clj`, `export.clj`, `text_extractor.clj`.\n - Highlighted file: `nutask.clj` under `languages`.\n\n 2. **Center Panel (Code Editor):**\n - File being edited: `simmie.clj`.\n - Code in focus:\n ```clojure\n (def test-query (openai/chat \"gpt-4\"\n {:messages\n [{:role \"system\" :content \"You are a note taker for a screen transcription tool. Describe all the visual information as precisely as possible such that it can be retained even if the image is lost. Describe the screen hierarchically. If a browser is visible, write down the URL that is open. Write down all text you can see in the respective context. Summarize your observations in the end and try to infer what the user is doing on a high-level.\"}]})\n ```\n\n 3. **Right Panel (Output/Terminal):**\n - Displays the result of executed scripts or logs.\n - Text includes:\n - Logs indicating \"new screenshot\" events with paths such as `/home/ubuntu/simmie/files/screenshots/454354782/frames_1729024426.jpg`.\n\n- **Bottom Bar:**\n - Shows file state and cursor position.\n - Information includes:\n - `Ln 104, Col 18`, `Spaces: 2`, `UTF-8`, `LF`, Language mode: `Clojure`.\n\n### Summary and Inference\n\nThe user is working on a Clojure project within Visual Studio Code. The code in the center panel suggests the user is defining and experimenting with an OpenAI GPT-4 model query. They are likely testing text-based AI functionalities. The logs in the right panel reveal the user might be running a script that processes screenshots, possibly related to testing or generating outputs using the code being edited. The focus on Clojure code and interaction with AI models in this development environment suggests tasks involving automation or data processing are being carried out. The user is probably in an active development and testing phase, dealing with file management, coding custom functions, and verifying outputs in real-time.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024429.jpg" "## Visual Information Description\n\n**Application: Visual Studio Code**\n\n### Layout\n- **Top Bar**\n - Contains menu items: `File`, `Edit`, `Selection`, `View`, `Go`, `Run`, `Terminal`, `Help`.\n - Open tabs: \n - `simmie.clj - simmie`\n - `rustdesk.clj - simmie`\n - `default_schema.edn.clj`\n - `tower.clj`\n - `awe.clj`\n - `openaicl.clj`\n \n- **Left Sidebar: Explorer**\n - Title: `EXPLORER`\n - Current project labeled: `SIMMIE [SEE TELEGRAM]`\n - Directory structure:\n - `.clojure`\n - `.clj-kondo`\n - `.dirs`\n - `.husky`\n - `.vscode`\n - `dazn`\n - `docs`\n - `downloads`\n - `resources`\n - `public`\n - `src`\n - `simmie`\n - Folders and files including:\n - `handlers`\n - `languages`\n - `assistance.clj`\n - `availability.clj`\n - `contact.clj`\n - `notes.clj`\n - `report.clj`\n - `reporter.clj`\n - Partially visible files denoted by ellipses and their sizes or modification times.\n - `test`\n - Additional files including `build.clj`, `CHANGELOG.md`, `LICENSE`, `README.md`, `server.log`\n\n### Editor Pane (Center)\n- **Current Open File: `simme.clj`**\n - Contains Clojure code with syntax highlighting.\n - Functions and variables highlighted, examples include:\n - `encodeToString`, `process-frames`\n - `main`, `extract-timestamp`\n - Code comments are included.\n - Indicated lines for different sections.\n\n### Right Pane: Terminal or Output\n- Label: `output.calva-repl`\n- Content:\n - Series indicating 'new screenshot' commands or outputs:\n - e.g., `\"new screenshot=\"/home/ubuntu/screenshots/45434782/frames_1729876871.jpg\"`\n - Numerous repeated lines with similar paths indicating logging or debugging of file output.\n\n### Status Bar (Bottom)\n- Information displayed:\n - SSH connection status: `[SSH: telegram]`\n - Editing modes and settings: `Ln 63, Col 97`, `Spaces: 2`, `UTF-8`, `Clojure`, `CP`, `l33`, `point`, `z`.\n\n## Summary\nThe user is working in Visual Studio Code connected via SSH to a remote host referred to as \"telegram\". They are developing or maintaining a project named \"SIMMIE\" with files primarily written in Clojure. The main task appears to be handling or processing image files, suggested by references to encoding and timestamp extraction from image filenames. The presence of continuous log updates in the terminal/output pane hints at either debugging or monitoring processes related to screenshot handling.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729009897.jpg" "**Screen Description**\n\n1. **Environment Overview**\n - The screen displays the Visual Studio Code application, indicated by the window title: `output.calva-repl - simmie [SSH: telegram] - Visual Studio Code`.\n - The application is in dark mode with a typical code editor layout.\n\n2. **Top Bar**\n - Menu options are visible: `File`, `Edit`, `Selection`, `View`, `Go`, `Run`, `Terminal`, `Help`.\n\n3. **Sidebar (Left)**\n - Contains icons representing common functions, including:\n - File explorer\n - Search\n - Source control\n - Run and debug\n - Extensions\n\n4. **Editor Area**\n - Split into three main sections:\n - **File Tabs**: Several files are open, with one active tab: `output.calva-repl`.\n - **Left Editor Pane**:\n - Shows code written in Clojure.\n - Includes function definitions and operations involving images and timestamps.\n - Functions like `defn`, `let`, `when`, and `comment` are visible.\n - Example of some code structure:\n ```clojure\n (defn extract-timestamp []\n (let [m (re-matches #\"\\frames_(\\d+)\\.jpg\" s)]\n (java.util.Date. 1000 (Long/parseLong m))))\n ```\n\n - **Right Editor Pane**:\n - Primarily displays a log with repeated entries, e.g.,:\n ```\n \"processing\" \"/home/ubuntu/screenshots/434345782/frames_1728961561.jpg\"\n ```\n\n5. **Bottom Bar**\n - Information about the file: `Ln 747, Col 1 | Spaces: 2 | UTF-8 | Clojure`.\n\n6. **Status Bar**\n - Includes information like:\n - Notifications (`bell` icon)\n - Git branch\n - Sync status\n - Errors or warnings\n - Language mode: `Clojure`\n\n**Summary of Observations**\n\n- The user is operating within Visual Studio Code, working on a project that involves processing image files (`frames_xxx.jpg`) likely residing in a specific directory (`/home/ubuntu/screenshots`).\n- The code involves defining functions to extract information, possibly from filenames, and process these images.\n- The frequent use of timestamps and paths in the log suggests the output is part of an automated image processing workflow.\n- The user seems to be editing or developing a Clojure-based application, potentially related to managing or analyzing screenshots.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024435.jpg" "## Visual Information Description\n\n### General Layout\n- **Application:** Visual Studio Code\n- **Activity:**\n - File editing and directory exploration in progress.\n - Multiple files are open in separate tabs.\n\n### Tabs Opened\n- **Tabs:**\n - simmie.clj\n - nutask.clj\n - default_schema.edn\n - peers.clj\n - openai.clj\n - rustdesk.clj\n\n### Sidebar\n- **Explorer Pane:**\n - **Structure:**\n - Directories: `.clj`, `output`, `apache`, `flow`, `node`, `public`, `resources`, etc.\n - Highlighted file: `telegram.clj`\n- **Text:**\n - `SIMMIE [SEE TELEGRAM]`\n - Various .clj files listed with some displaying modified status.\n\n### Main Editor (Middle Section)\n- **File Opened:** simmie.clj\n- **Code Snippet:**\n - Functions and code related to processing frames, timestamps, etc.\n - Highlighted text: process frame code, timestamp extraction logic.\n - Active cursor with a suggestion popup showing keywords like `def`, `defn`, `defonce`, etc.\n\n### Right Pane\n- **File:**\n - Output logs (output.calva-repl)\n- **Text Content:**\n - Sequence of “new screenshot” log messages with paths to images.\n - Highlighted line: `\"/home/ubuntu/screenshots/454345782/frames_172990158.jpg\"`\n\n### Themes and Tools\n- **Color Theme:** Dark mode with syntax highlighting.\n- **Operational Indicators:**\n - Bottom toolbar showing file details: Line (Ln), Column (Col), Spaces, Encoding, Programming language (Clojure).\n\n## Summary and High-Level Inference\nThe user is working within Visual Studio Code, primarily coding in the Clojure language. They are developing or debugging a program named “simmie”, which processes frame images and manages timestamps. The sidebar indicates exploration or modification of multiple source files. The explorer pane suggests that the project has components dealing with outputs, schemas, and integrations with tools like `telegram`. The right pane displays log outputs suggesting functionality related to screenshot processing or management is being executed or tested. \n\nOverall, the user is focused on software development and debugging tasks, likely involving real-time image or frame data processing.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729027304.jpg" "### Visual Information Description\n\n#### Main Window\n- **Application:** Visual Studio Code\n - **Theme:** Dark mode with syntax highlighting\n - **Layout:** Split into three main columns\n\n#### Left Column: Explorer\n- **Folders/Files Tree:**\n - Root: `simnie [SSH: TELEGRAM]`\n - Structure includes:\n - `.calva`, `.github`, `.vscode`, `example_config.edn`\n - Folders: `src`, `resources`, `prmprq`, `public`, `output.cljc`\n - Specific files: `default_schema.edn`, `nrepl.edn`, `deps.edn`, `LICENSE`, `README.md`\n - **Selected Directory:** `src > sim > runtimes`\n - **Highlighted Files:**\n - `telegrams.clj`, `extractions.clj`, `nssance.clj`, etc.\n - **Indicators:** \n - Red dots indicating changes on files: `telegrams.clj`, `report.clj`, `extractor.clj`\n\n#### Middle Column: Code Editor\n- **File in Focus:** `telegrams.clj`\n- **Code Content:**\n - Functions written in Clojure with specific syntax such as `defn`, `map`, `require`\n - Code comments and commands for processing image screenshots and data extraction\n - Hover tooltip showing the function signature: `clojure.core/concat`\n\n#### Right Column: Output/Logs\n- **Files:**\n - `output.clj`\n- **Debugging Information:**\n - Logging data showing file paths such as `/home/ubuntu/simnie/files/screenshots/454354782/...`\n - Console outputs displaying executed commands and errors\n - Compilation details with file locations and line indicators like `(core.clj:87)`\n\n#### Top Bar\n- **Menu Options:** File, Edit, Selection, View, Go, Run, Terminal, Help\n- **Open Tabs:**\n - `telegrams.clj`, `extractions.clj`, `default_schema.edn`, `telegrams.clj`, `output.cljc`\n\n### Observations Summary\nThe user is working in a development environment, specifically Visual Studio Code, dealing with Clojure scripts primarily related to image or data processing operations. They are examining code and reviewing execution logs, possibly debugging or developing scripts. The active session is connected via SSH to a remote server called \"TELEGRAM\" for accessing and editing code files remotely. There is an emphasis on file management, script modification, and monitoring of error outputs in the console.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024529.jpg" "### Screen Overview\n\n#### Application:\n- Visual Studio Code\n\n#### Title Bar:\n- File name: `rustdesk.clj - simmie [SSH: telegram]`\n- Application name: Visual Studio Code\n\n#### Panel Layout:\n1. **Left Sidebar - Explorer**\n - Directory and Files:\n - Folders: `.vscode`, `.calva`, `.output`, `.calva-repl`, `src`, `apache`, `classes`, `downloads`, `flow`, `resources`, `public`, `.keep`, `project.clj`\n - Files under `languages` and `runtimes`: `base.clj`, `ci.clj`, `ciatom.clj`, `extractor.clj`, `reporter.clj`, `telegram.clj`, `ext.extractor.clj`, etc.\n - Additional files: `.gitignore`, `build.clj`, `CHANGELOG.md`, `deps.edn`, `LICENSE`, `README.md`, `server.log`\n - Outline: Not visible\n - Project Tree: Visible\n\n2. **Middle Panel - Editor**\n - File: `simmie.clj`\n - Programming Language: Clojure\n - Code Snippets:\n ```clojure\n .encodeToString (.getEncoder (java.util.Base64/getEncoder) image-bytes))\n (defn process-frames [session term-ch]\n (let [screenshots-folder-str base-screenshot-folder ^String/IO!]\n (go-loop-try [screenshots screenshots (atom [])\n [new-screenshots all-screenshots screenshots-folder]\n (called (all new all)\n (meta (let delete (defn value (let into)\n (do (.delete frame-file)))\n (Other util (term-ch\n (def main\n (let [kc (reduce `k [async `frame-file] `screen-term-ch!) nil] ex)\n (def kc (fn prm `orpm `(pm (cm screen-shot-frame? ]\n [defn dispose-or (Choose a success s. error. ny:\n [defn extract-timestamp\n [let (k = (let [re .matches ~ frames [^d+^d+^)} s]\n (java.util.Date. * 1000 (Long/parseLong g)))\n (put tern-ch :done)\n ```\n - Autocomplete Suggestions:\n - Options: `defn`, `def`, `def-inline`, `defonce`, `defmulti`, `defn-`, `defn+', `deftype`, `defprotocol`, `defrecord`, `deftest`, `deftask`, `defstruct`\n\n3. **Right Panel - Terminal/Output**\n - Filename: `output.calvarepl`\n - Contents: List of strings indicating \"new screenshot\" files in a directory, e.g., `\"new screenshot\" \"/home/ubuntu/screenshots/454343782/frames_1729924426.jpg\"`\n\n### Summary\nThe user is working on a Clojure project in Visual Studio Code, possibly a screenshot processing tool, as indicated by the function names and terminal output. The setup suggests a development environment, with code completion tools active and a focus on manipulating image data files, indicated by Base64 encoding operations and file path listings in the output panel.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024524.jpg" "## Visual Information Description\n\n### Application Window: Visual Studio Code\n\n#### Window Title:\n- \"rustdesk.clj - simmie [SSH: telegram] - Visual Studio Code\"\n\n#### Top Menu Bar:\n- Options include: File, Edit, Selection, View, Go, Run, Terminal, Help\n\n#### Left Sidebar: Explorer Pane\n- **Header:** “EXPLORER”\n- **Project Folder:** \"simmie [SSH: TELEGRAM]\"\n\n##### Folder Structure:\n- `.clojure`\n - `config.edn`\n- `arcadia`\n- `bin`\n- `classes`\n- `doc`\n- `downloads`\n- `log`\n- `resources`\n- `public`\n- `src`\n - `languages`\n - `assistance.clj`\n - `simmie`\n - `processes`\n - `brand.clj`\n - `note.clj`\n - `report.clj`\n - `reporter.clj`\n - `telegram.clj`\n - `text_extractor.clj`\n - `config.clj`\n - `http.clj`\n - `openai.clj`\n - `simmie.clj` (selected)\n - `towvecs.clj`\n - `web`\n- `.gitignore`\n- `build.clj`\n- `CHANGELOG.md`\n- `deps.edn`\n- `LICENSE`\n- `README.md`\n- `server.log`\n\n#### Main Workspace: Split View\n- **Left Pane (Editor):** \n - File: `simmie.clj`\n - **Programming Language:** Clojure\n - **Code Content:** Functions and logics for image processing, extracting timestamps, and handling Telegram messages or sessions. The code involves:\n - Enclosures and functions like `encodeToString`, `process-screens`, `extract-timestamp`.\n - Path operations like `\"/home/ubuntu/screenshots/434543762/frames_1727984268.jpg\"`\n - Function definitions: `defn main`, `defn extract-timestamp`, `defn process-frames`\n - Comments regarding functionality such as extracting timestamps or processing frames.\n\n- **Right Pane (Terminal/Output):** \n - File: `output.calva-repl`\n - **Content:** \n - List of image paths: `\"new screenshot\"` followed by paths like `\"/home/ubuntu/screenshots/434543762/frames_1729076521.jpg\"`\n - Section showing usage of a Clojure REPL with namespace indicators `clj.simn.runtimes.rustdesk`.\n\n#### Bottom Status Bar:\n- **Information Displayed:** \n - Branch: `main`\n - Line/Column: `Ln 63, Col 109`\n - Spaces: `2`\n - Encoding: `UTF-8`\n - File Type: `Clojure`\n - Git indicators and other status icons\n\n### Summary:\nThe screen shows a developer working in Visual Studio Code over an SSH connection, editing a Clojure project named \"simmie.\" The left pane contains code concerning image processing and integration with Telegram. The right pane displays the output/repl log with file paths to screenshots processed by the code. The user's activities involve coding and debugging processes related to handling images and possibly interacting with a Telegram bot or application as part of a broader Clojure project.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729009902.jpg" "**Visual Hierarchy Description:**\n\n**Application Window:**\n- **Title Bar:**\n - \"output:calva-repl - simmie (SSH: telegram) - Visual Studio Code\"\n- **Menu Bar:**\n - Options: File, Edit, Selection, View, Go, Run, Terminal, Help\n- **Sidebar on the Left:**\n - Icons: Explorer, Search, Source Control, Debug, Extensions\n- **Editor Area:**\n - **Left Pane:**\n - **Title:** \"CALVA INSPECTOR\"\n - Text content:\n ```\n robin@clojure.core.async.impl.channels.ManyToManyChannel 0x8350ce63\n clojure.core.async.impl.channels.ManyToManyChannel\n #1:s ...n.runtimes.rustdesk/bernh (ctrl-clickable)\n nil\n nil\n ```\n - **Middle Pane:**\n - **Filename Tab:** \"simmie.clj\"\n - **File Content:**\n - Code sample below:\n ```clojure\n (defn all-screenshots [screenshots-folder]\n (file-seq (cl file screenshots-folder)\n (filter #(re-matches #\"frames_\\.*jpg$\" (.getName %))\n map #(.getPath %)))\n\n (defn encode-image-input!\n [^input-stream input-stream ^input-stream image-path]\n (let [image-bytes (read-all-bytes input-stream)\n image-bytes]\n .encodeToString (java.util.Base64/getEncoder) image-bytes)))\n\n (defn process-frames [session term-ch]\n (let [screenshots-folder (->session.all-screenshots set session)\n screenshots (filter empty:all-screenshots screenshots-folder)\n (letfn (let (details)\n {\n :encoded))\n ((((enc-input input-stream image-path)\n 1000 100 count [])\n \n recur re-enc-input!))))\n\n ;; Remaining code content\n ```\n- **Right Pane:**\n - **Filename Tab:** \"output.calva-repl\"\n - **File Content:**\n - Log information:\n ```\n \"processing\" \"/home/ubuntu/screenshots/453457362/frames_1728970689.jpg\"\n ...\n \"processing\" \"/home/ubuntu/screenshots/453457362/frames_1728970913.jpg\"\n ```\n\n**Editor Status Bar:**\n- Branch: \"main\"\n- Details: Indicators for number of edits, line/column, file type (Clojure)\n\n**Summary and Inference:**\n\nThe user is working in Visual Studio Code, editing and executing Clojure code remotely via SSH. The code appears to relate to processing image files, encoding them, and handling them in some type of system or application. The right pane shows log entries indicating that images are being processed, suggesting an active debugging or development workflow, likely focusing on automating or processing image frames.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024449.jpg" "The screenshot depicts a Visual Studio Code interface with several panes and elements. Below is a hierarchical description of the contents:\n\n### 1. **Application Title Bar**\n - **Title:** `rustdesk.cli - simmie [SSH: telegram] - Visual Studio Code`\n\n### 2. **Menu Bar**\n - **Options:** File, Edit, Selection, View, Go, Run, Terminal, Help\n\n### 3. **Sidebar (Explorer)**\n - **Open Folders and Files:**\n - `SIMMIE [SSH: TELEGRAM]`\n - `output.window`\n - `clojure.calva-repl`\n - `src`\n - `simm`\n - `languages`\n - `processes`\n - `runtimes`\n - File highlighted: `extractor.clj`\n - Other files: `database.clj`, `btc.clj`, `operator.clj`, etc.\n - `telegram`\n - `telegram.clj` (highlighted)\n - `text_extractor.clj`\n - `resources`\n - Files: `.keep`, `.gitignore`, `build.clj`, `CHANGELOG.md`, `deps.edn`, `LICENSE`, `README.md`, `server.log`\n\n### 4. **Main Editing Area**\n - **File Open:** `extractor.clj`\n - **Code Snippets:**\n ```clojure\n (def test-query \"\")\n ```\n\n### 5. **Bottom Bar**\n - **Information:**\n - `main*`, `⬡`, `2.24.4`, `⬢`, `213.0`, `16`, `REPL`, `Clojure`, `NORMAL`\n\n### 6. **Integrated Terminal Area (Visible on the Right)**\n - **File Open:** `output.calva-repl`\n - **Recent Commands and Outputs:**\n - Filenames listed: `/home/ubuntu/simmie/files/screenshots/...`\n - Command example: `(e! sim.runtimes.rustdesk/last-screenshots)`\n\n### Summary\n\nThe screenshot represents a Visual Studio Code interface captured in the context of software development. The user is working on a Clojure project concerning screenshot processing and managing runtime data. They are interacting with multiple files, primarily focusing on an extraction-related script (`extractor.clj`). Syntax highlighting is used for various Clojure keywords, and the integrated terminal provides access to runtime commands and outputs. The activity suggests a workflow related to code editing, testing, and debugging functions/modules.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024526.jpg" "The screen displays a Visual Studio Code window with multiple files and a terminal open. Here's a detailed hierarchical breakdown:\n\n### Top Bar\n- The window title is: `rustdesk.clj - simmie [SSH: telegram] - Visual Studio Code`.\n\n### Sidebar (Left Panel)\n- **Explorer Section**\n - Root directory: `simmie [SSH TELEGRAM]`\n - Subdirectories/files:\n - `.calva/output-window`\n - `.output-calva-repl`\n - `.lsp`\n - `_capture`\n - `.clj-kondo`\n - `arcadia`\n - `classes`\n - `doc`\n - `flows`\n - `downloads`\n - `zeus`\n - `resources`\n - `promptup`\n - `public`\n - `spec`\n - `src`\n - `languages`\n - `processes`\n - `runtimes.clj` (open in editor)\n - `bitcoin.clj`\n - `monerogetter.clj`\n - `doge.clj`\n - `report.clj`\n - `twitter.clj`\n - `telegram.clj`\n - `renf_extractor.clj`\n - `test`\n - `rustdesk.clj` (highlighted)\n - `btcl.clj`\n - `eth.clj`\n - `peepr.clj`\n - `prmtup.clj`\n - `tweets.clj`\n - `simmie.clj`\n - `.bashrc`\n - `.gitignore`\n - `build.clj`\n - `CHANGELOG.md`\n - `deps.edn`\n - `LICENSE`\n - `README.md`\n - `server.log`\n\n### Main Editing Area\n- **File: `runtimes.clj`**\n - Lines of Clojure code, including function definitions and comments.\n - Code structure includes function definitions, variable assignments, and comments.\n - Significant codified function: `(defn process-frames ... )`\n - Inline comments and docstrings elaborate the purpose of certain code sections.\n - Code involves file system operations, string manipulation, and uses libraries like `super.async`.\n\n### Right Panel\n- **Output/Terminal Section**\n - Filename: `output.calva-repl`\n - Displays repeated log entries like: `\"new screenshot\" \"/home/ubuntu/screenshots/45434752/frames_1729805782.jpg\"`\n - Command prompt for executing Clojure commands.\n - Display of recent screenshots outputs.\n\n### Bottom Bar\n- Status bar showing:\n - File information: `Ln 63, Col 109 | Spaces: 2 | UTF-8 | Clojure | normal`\n - Git branch/status indicator: `master`\n - Language mode: `Clojure`\n\n### Summary\nThe user is currently engaged in software development using Clojure within Visual Studio Code, focusing on a file called `runtimes.clj`. The project appears related to screenshot processing or management, possibly involving automation or image recognition tasks. The terminal logs show a script or application generating screenshot file paths, suggesting active testing or debugging of the code.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729027307.jpg" "**Application**: Visual Studio Code\n\n**Title Bar**: \n- Title: `rustdesk.clj - simmie [SSH: telegram]`\n\n**Open Tabs**: \n- `rustdesk.clj`\n- `released_work.clj`\n- `missionary.clj`\n- `default_schema.edn`\n- `tower.clj`\n- `peer.clj`\n- `prompts.clj`\n- `openai.clj`\n- `simmie.clj`\n\n### Interface Layout\n\n**Explorer Panel (Left Sidebar)**:\n- Root folder: `simmie [SSH TELEGRAM]`\n- Folders: \n - `.clj_kondo`\n - `classes`\n - `doc`\n - `downloads`\n - `resources`\n - `promptpkg`\n - `public`\n - `src`\n - Expanded: `simmie`\n - Files: \n - `runtimes`\n - `abstractions.clj`\n - `ci.clj`\n - `elocation.clj`\n - `process.clj`\n - `telegram.clj`\n - `text_extractor.clj`\n - `http.clj`\n - `mod.clj`\n - `peer.clj`\n - `prompt.clj`\n - `tweets.clj`\n - Additional files:\n - `.gitignore`\n - `README.md`\n - `server.log`\n\n**Main Editor Panel (Center-Left)**:\n- Open file: `simmie.clj` with Clojure syntax highlighting.\n- Shows code snippet with comment and defined functions:\n - Function `def test_query`\n - Code for screenshot processing and testing using OpenAI API\n\n**Output/Console Panel (Center-Right)**:\n- Displays logs and outputs with line numbers.\n- Includes process details and log of executing code.\n\n### Specific Code Snippets:\n\n- **Code Functions**:\n - Extract timestamps from file paths\n - Use of `(openai/chat \"gpt-4.0\" ...)` to interact with OpenAI API\n - Logs processing workflows for screenshots\n\n- **Visible Text in Console**:\n - \"Processing videos to frames\"\n - \"You are a note taker for a screen transcription tool\"\n\n- **Log Messages**: \n - Outputs repetitive logs indicating function use and results.\n - Error trace and path logs to file directories with line numbers.\n\n**Summary and High-Level Inference**:\n- The user is working on a Clojure project using Visual Studio Code.\n- Code editing primarily involves handling screenshots and interacting with an API.\n- The console shows live outputs from the executed code, indicating active debugging and testing of Clojure scripts.\n- Project appears to focus on multimedia handling and automation tasks.\n\nThis analysis provides a comprehensive understanding of the current environment and user activity.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729027290.jpg" "**Visual Information Description**\n\n**Application interface: Visual Studio Code**\n\n- **Top Bar:**\n - Menu Options: File, Edit, Selection, View, Go, Run, Terminal, Help\n - Open Tabs: `rustdesk.clj`, `simmie.clj`, `simmie.clj`, `default_schema.edn`, `tower.clj`, `peer.clj`, `openai.clj`, `output.calva-repl`\n - SSH Information: `simmie [SSH: telegram] - Visual Studio Code`\n\n- **Main Layout:**\n - **Left Panel (Explorer)**\n - Project folders/files with a hierarchical tree structure\n - Main directory: `simmie [SSH: telegram]`\n - Subfolders/File Names:\n - `.calva`, `classes`, `doc`, `download`, `eaves`, `log`, `node`, `resources`\n - Files:\n - `input`, `output`, `public`, `src`\n - Within `src`: folders or files such as `languages`, `runtimes`\n - Specific files in `runtimes`: `atc.clj`, `botanic.clj`, `citation.clj`, `exit.clj`, `reporter.clj`, `rustdesk.clj`, `telegram.clj`, `text_extractor.clj`\n - Other visible files: `.gitignore`, `build.clj`, `CHANGELOG.md`, `deps.edn`, `LICENSE`, `README.md`, `server.log`\n - Highlighted/decorated files: with modified or staged changes.\n\n - **Center Panel**\n - Open File: `rustdesk.clj` \n - Code Structure: \n - Functions such as `defn`, `let`, `process-frames`, `main`, `apr`, `schema`, etc.\n - Color-coded syntax formatting (e.g., `defn`, `let`, `if`, etc.)\n - Comments in code using `;`\n - Code lines referencing file paths such as `/home/ubuntu/...`\n\n - **Right Panel**\n - Open File: `output.calva-repl`\n - Terminal output style text\n - File Paths: \n - Repeated entries like `/home/ubuntu/simmie/files/screenshots/454345782/frames/...`\n - Log-type messages: \"Error on openai chat\", error stacks, timestamps, session evaluations.\n\n**Summarized Observations:**\n\nThe user is working within Visual Studio Code over SSH connected to a `telegram` server, most likely developing or maintaining a software project named `simmie`. The left panel shows the project's folder and file structure, providing quick access to different source files. The center panel is active with coding work on a file named `rustdesk.clj`, written in Clojure, focusing on functions that appear to manage data processes related to files and screenshots. The right panel shows a REPL (Read-Eval-Print Loop) output, documenting log data and error messages, indicative of testing or runtime sessions. Overall, the activity suggests programming and debugging tasks.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024452.jpg" "# Screen Description:\n\n## Browser and URL:\n- **Browser:** Web browser window is open.\n- **URL:** community.openai.com/t/how-the-max-tokens-are-considered/135314\n\n## Top Bar:\n- **Icons:** Numerous app shortcuts and system icons are displayed at the top of the screen.\n- **Time/Date/Connectivity Indicators:** Various system icons related to connectivity and settings.\n\n## Main Section:\n- **Header:**\n - **Logo:** OpenAI Logo is displayed.\n - **Menu Links:**\n - Topics\n - Resources:\n - Documentation\n - API reference\n - Categories:\n - Announcements\n - API\n - Prompting\n - Documentation\n - Plugins / actions builders\n - Tags:\n - chatgpt\n - gpt-4\n - gpt-3\n - log-user\n - assistant-api\n - All tags\n\n- **Main Content Area:**\n - **Title:** How the max tokens are considered\n - **Tags:** API, api, token\n \n- **Discussion Thread:**\n - **Green Box:** Welcome to the OpenAI Developer Forum!\n - What to know before posting a new question:\n 1. Search the forum for similar topics - the question might have been discussed before.\n 2. If the question relates account issues (e.g., billing and login issues), please contact us through our Help Center.\n 3. Please be kind and helpful in conversations!\n \n - **Post by deb23 (Aug 2023):**\n - Text: Hi, probably mine is a stupid question but I have this doubt: each model has a maximum number of tokens (for example the gpt-3.5-turbo model has a maximum number of tokens 4,096 tokens) but are they considered for each prompt (prompt + answer) or in the total code? ...\n\n - **Solution Section:**\n - **Solved by Foxalabs in post #6**\n - Text: For prompt and reply A to influence prompt and reply B ...\n\n - **Responses:**\n - **By Foxalabs (Aug 2023, Leader):**\n - Text: Not a stupid question at all, it confuses many. 4096 tokens is your entire world ...\n - Reactions: 3 thumbs up\n - **By udm17 (Aug 2023):**\n - Text: 4096 tokens is your entire world ... A morbid way to put it but yes, on point with this statement\n\n## Side Panel:\n- **Timeline for posts:** \n - Visible timeframe: Aug 2023 to Feb 6 \n - Current position: 1/10, Jul 2023 selected\n\n## Sign In/Log In Section:\n- **Buttons:** \n - Sign Up\n - Log In\n- **Search Icon:** Magnifying glass icon\n\n# Summary:\nThe current activity involves browsing a forum discussion on the OpenAI Developer Forum, focusing on how the maximum token limits are applied in AI models. The user is likely interested in understanding token usage, particularly for prompt and response models like GPT-3.5-turbo. They are reviewing a specific question and its responses, including an accepted solution by a forum leader. This activity seems related to AI development or research concerns.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024428.jpg" "### Visual Information Description\n\n#### Application Header\n- **Application**: Visual Studio Code\n- **Current File**: `rustdesk.clj - simmie [SSH: telegram]`\n- **Open Tabs**: \n - `simmie.clj`\n - `nutask.clj`\n - `output.calva:repl `\n - `default_schema.edn`\n - `tower.clj`\n - `percsj`\n - `stumpanic.clj`\n\n#### Sidebar (Left)\n- **Explorer Section**: Displays a project directory with multiple files and folders.\n - **Folders and Files**:\n - `.db`\n - `.m2`\n - `.vscode`\n - `.zachine` and many others.\n - **Highlighted Folder**: `src`\n - Contains:\n - `clojure/calva/repl`\n - `assistance.clj`\n - `routines.clj`\n - `simmie.clj`\n - `report.clj` \n - `telegrams.clj` (highlighted with a red icon)\n - Others\n\n#### Central Pane\n- **Current File Content**: `simmie.clj`\n - **Code Snippet**:\n - Function definitions and operations primarily written in Clojure.\n - Contains operations related to timestamps, screenshots, and data processing.\n - Functions like `extract-timestamp`, `dispose`, `process-frames`, and custom comments.\n - Example part of the code:\n ```clojure\n (def test-query\n (ql `openai/chat `gpt-4.0\n {:messages\n [{:role \"system\" :content \"You are a note taker ...\"}\n {:role \"user\" :content\n (str \"data:image/jgp;base64,\" sp)}]}))}\n ```\n\n#### Right Pane\n- **File Content**: `output.calva:repl`\n - Appears to be a log or output consoled from a REPL (Read-Eval-Print Loop) session showing a list of screenshots.\n - Example output:\n - `\"new screenshot\" \"/home/ubuntu/screenshots/454345782/frames_1728947783.jpg\"`\n - Lines are sequentially numbered.\n\n#### Status Bar (Bottom)\n- **Information Displayed**:\n - **Branch**: `main`\n - **Language**: `Clojure`\n - **Formatting Info**: `Spaces 2`\n - **Mode**: `NORMAL`\n\n### Summary and Inferred User Activities\nThe user is working in Visual Studio Code, likely on development tasks related to a project involving Clojure. They appear to be handling image processing or timestamp operations within the filesystem structure. The user is probably debugging or developing functionality for handling screenshots. The presence of SSH suggests remote work or code related to Telegram. The user might be integrating with OpenAI's GPT models based on the code content.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729027300.jpg" "# Visual Information Description\n\n## Application Overview\n- **Application:** Visual Studio Code\n- **Title Bar:**\n - Text: \"rustdesk.clj - simmie [SSH: telegram] - Visual Studio Code\"\n\n## Tabs\n- **Open Files:**\n - \"simmie.clj\"\n - \"nutdesk.clj\"\n - \"default_schema.edn\"\n - \"towers.clj\"\n - \"peer.clj\"\n - \"openai.clj\"\n\n## Main Editor (Left Side)\n- **Editor:**\n - **File:** \"simmie.clj\" from directory `/src/`\n - **Language:** Clojure\n - **Highlighted Line:** 97\n - **Code Context:**\n ```clojure\n (def test-query\n @(q/one! chat \"gpt-4q\"\n (vec (concat [\n \"You are a note taker for a screen transcription tool. Describe all the visual ...\n ]))\n ```\n\n- **File Explorer:**\n - **Structure:**\n - **Folders:**\n - `.clj-kondo`\n - `.output`\n - `src` (expanded)\n - `simmie`\n - `languages`\n - `runtimes`\n - `nutdesk.clj`\n - `base.clj`\n - `citea.clj`\n - `edit.clj`\n - `parse.clj`\n - `report.clj`\n - `simmie.clj` (current)\n - `telegram.clj`\n - `text_extractor.clj`\n - **Files:** Several listed with version control indicators (e.g., modified, untracked).\n\n## Side Panel (Right Side)\n- **Terminal Output:**\n - **File Path:** `/output/output.clj.eval`\n - Output includes logs indicating Clojure operations and evaluations.\n - Error messages and stack traces present:\n - Example: \"Execution error (InterruptedException)\".\n\n## Status Bar\n- **Details:**\n - Language Mode: \"Clojure\"\n - Git: Branch \"main\"\n - Document settings: Spaces (2), UTF-8, Line ending (LF)\n\n# Summary of User Activity\nThe user is working in Visual Studio Code within a Clojure project. They have multiple files open relating to different functionalities (e.g., runtimes, text extraction, and API usage) in a project named \"simmie\". The primary focus seems to be on a Clojure script (\"simmie.clj\") that includes data processing and string concatenation functions, possibly for a screen transcription tool. The user is connected remotely via SSH to a \"telegram\" server, as suggested by the title bar. They are actively compiling or running Clojure code, with console feedback providing debugging information and error traces, indicating development or testing phases.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729027301.jpg" "**Visual Hierarchical Description:**\n\n1. **Application Window: Visual Studio Code**\n\n - **Title Bar:** \n - Title: `rustdesk.clj - simnie (SSH: telegram) - Visual Studio Code`.\n\n - **Menu Bar:**\n - Options: \"File\", \"Edit\", \"Selection\", \"View\", \"Go\", \"Run\", \"Terminal\", \"Help\".\n\n - **Explorer Panel:**\n - Directory Path: `simmie/src/simn/runtimes`.\n - Files and folders listed:\n - `db.clj`\n - `extractor.clj`\n - `utils.clj`\n - `openaicl.clj`\n - `telegram.clj` (selected)\n - `simmie.clj` \n - `tweet.clj`\n - `build.clj`\n - Others including configuration and documentation files.\n - Status Indicators (some files have status indicators like modifications).\n\n - **Main Editor Panel (Left Split):**\n - File: `simmie.clj`\n - Content with syntax highlighting focuses on Clojure code:\n - Code snippets include functions, variables, and comments. \n - Example lines: \n - `(def test-query ...)`\n - `(def transcriptions ....)`\n - Highlighted green strings include comments and function definitions.\n - Function calls, variable assignments, and other operations are present.\n\n - **Secondary Editor Panel (Right Split):**\n - File: `output.clj` (specifically `output.clj#evaleruje` in the alert box).\n - Shows execution logs of code related to Clojure evaluation.\n - Highlighted sections with various statuses like \"evaluation was interrupted\".\n - Lines display a sequence of logging text illustrating process flows and results.\n\n - **Terminal Panel (Partially Visible at Bottom):**\n - Displays a Clojure REPL, showing an active session with execution results and stack traces.\n - Messages include error handling and debug output such as `Execution error (InterruptedException)`.\n\n2. **Status Bar:**\n - Shows details like `Ln 120, Col 21`.\n - Current Git branch and other quick access icons are visible.\n\n**Summary and Inference:**\n\nThe user is working in Visual Studio Code on a project that involves multiple Clojure scripts under a remote SSH session labeled \"telegram\". They are actively editing and managing files related to runtimes, processing, and potential API interactions (e.g., Telegram, OpenAI). The right editor panel suggests they are reviewing output or debugging using a log of recently executed expressions, potentially focusing on error resolution or process verification in the context of Clojure scripting.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729009895.jpg" "The screen displays an interface of Visual Studio Code (VS Code), a popular code editor, with multiple files open and a focus on a Clojure script. Below is a hierarchical description of the visual information:\n\n### Application\n- **Application Name:** Visual Studio Code\n- **Top Bar (Tabs with open files):**\n - Tabs open include:\n - `simmie.clj`\n - `nudetext.clj`\n - `default-schema.edn`\n - `tower.clj`\n - `cava-repl.clj`\n - `openai.clj`\n - `output.cavaluerged.clj`\n\n### Left Side Panel\n- **Explorer Activity Bar:**\n - Icons representing different activities (explorer, search, SCM, run, etc.)\n - File explorer showing \"CALVA: INSPECTOR\"\n - Some list items labeled \"nil\"\n\n### Main Area\n- **Left Editor Window:**\n - Open file: `simmie.clj`\n - Some excerpts from the code visible:\n - `(defn encode-image-input ...)`\n - `(defn extract-timestamp ...)`\n - Comments in the code using `;`\n\n- **Right Editor Window:**\n - Open file: `output.cavaluerged.clj`\n - Contains comments and documentation:\n - \"This file is configured as the output destination for all REPL output.\"\n - Errors or notes regarding `calva`\n - Information about connecting sessions and settings.\n\n### Bottom Status Bar\n- **Information:**\n - \"SSH: telegram\"\n - Details about current line: \"Ln 40, Col 1 | Spaces: 2 | UTF-8 | LF | Clojure | print |\"\n\n### Summary\nThe user is working in Visual Studio Code, focusing on Clojure development. They have multiple files open and are likely editing or reviewing code, possibly involving image processing or output configuration with `calva`, a Clojure environment tool. The SSH connection tag \"telegram\" suggests they might be working on a remote server.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729027289.jpg" "**Visual Information Hierarchy:**\n\n1. **Application Window:**\n - The application running is Visual Studio Code.\n - The theme is dark with different syntax highlighting for code.\n\n2. **Top Menu Bar:**\n - Options available: File, Edit, Selection, View, Go, Run, Terminal, Help.\n\n3. **Explorer Pane (Left):**\n - Label: EXPLORER\n - Project name: simmie [SSH: TELEGRAM]\n - File structure:\n - .git\n - Clara\n - deps.edn\n - documentation\n - languages\n - core.clj\n - extract.clj\n - nrepl.clj\n - report.clj\n - src\n - simmie\n - runtimes\n - openai.clj (Marked with changes)\n - rustdesk.clj (Marked as active, open tab)\n \n4. **Editor Pane (Center):**\n - Open file: simmie > runtimes > rustdesk.clj\n - Line numbers: Indicated on the left.\n - contains code using Clojure language\n - Code Snippet:\n ```clojure\n (def test-screenshots)\n (encode-to-base64 ...)\n (require '[...openai :as openai])\n (defn transcriptions ...)\n (def test-query (openai/chat \"gpt-4\" ...))\n ```\n\n5. **Output Pane (Right):**\n - Open file: output.ciawrepl\n - Contains terminal output lines and file paths:\n - Lines referencing file paths such as “/home/ubuntu/simmie/files/screenshots/...” with sequentially numbered frames.\n - An error message referencing Java stack traces and Clojure core files.\n\n6. **Status Bar (Bottom):**\n - Displays: \"Ln 124, Col 56\", \"Spaces: 2\", \"UTF-8\", \"Clojure\", \"NORMAL.\"\n\n**Tabs Open (Top):**\n- rustdesk.clj (active tab)\n- simmie.clj\n- default_schema.edn\n\n**Summary and High-Level Inference:**\nThe user appears to be developing or debugging a Clojure application related to handling screenshots and possibly integrating OpenAI’s API within Visual Studio Code. The presence of terminal output suggests active testing or running of the application, likely involving screen transcription as referenced in the comments. The user is dealing with files and functions related to image processing and OpenAI’s chat capabilities. There are code changes detected in the rustdesk.clj file, indicating ongoing development.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729027305.jpg" "**Application:** Visual Studio Code\n\n**Title Bar:**\n- Title: `rustdesk.clj - simmie [SSH: telegram] - Visual Studio Code`\n\n**Open Tabs:**\n1. rustdesk.clj\n2. simmie.clj\n3. runtimes.clj\n4. example_config.edn\n5. telegram.clj\n6. reporter.clj\n\n**Interface:**\n- *Explorer Panel* (Left):\n - Project: simmie [SSH: telegram]\n - Directory items including files and folders, such as:\n - .output_window\n - .clojure-deps\n - languages/\n - telegram.clj\n - runtimes/\n - babel\n - core.clj\n\n- *Main Editor Panel* (Center):\n - Code of `simmie.clj` is open with lines involving `(def test-query`, `(zen.zcl/extract-timestamp`, and Clojure functions.\n - Comments and highlighted code snippets are visible.\n - Code includes functions for image processing, specifying commands, and encoding screenshots to base64.\n\n- *Output/Console Panel* (Right):\n - Repetitive logs and outputs from commands, indicating evaluations and print schemas.\n - Logs contain text like \"cider.nrepl.middleware.interruptible_eval\", \"processing\", and directory paths.\n - Errors and debugging information are present.\n\n**Bottom Bar:**\n- Information visible includes:\n - Branch: `main`\n - Statuses: 2 ⬇️ 4 ⬆️\n - Language: Clojure\n - Tab Size: 2\n\n**Summary and High-Level Inference:**\n\nThe user is working on a Clojure project in Visual Studio Code opened through SSH. The main task involves handling image processing, including encoding and managing screenshots. Code snippets indicate multimedia handling scripts in Clojure, including file operations and directory mappings. The console panel outputs messages related to code execution, errors, and debugging information.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729009903.jpg" "## Visual Description of Screen\n\n### Application\n- **Title Bar**: \n - Text: `output:calva-repl - simmie (SSH: telegram) - Visual Studio Code`\n \n### Layout\n- **Main Split View**: \n - Two main sections, left and right split view.\n\n### Left Section\n- **File Explorer and Tabs**:\n - **File Tabs**: \n - `multimethods.clj`, `runtests.clj`, `output.calva/repl`, `output.calva/repl.window`\n- **Code Content**: \n - Multiple functions and expressions written in Clojure, including:\n - Function Definitions:\n - `defn encode-image`\n - `defn extract-timestamp`\n - `def term-ch`\n - Defining `term-ch` as a `chan`.\n - Logic Related to Image Processing:\n - Paths to screenshots, usage of `java.util.Date`.\n - Looping and conditional logic.\n - Example Code:\n - `let`, `if`, `fn`, `def`, `require`, `spit`, `doseq`.\n\n### Right Section\n- **Terminal/REPL Output**:\n - Lines showing processing of image files:\n - Repeated lines:\n - `processing \"/home/ubuntu/screenshots/454345782/frames_1728978860.jpg\"`\n - Continued with different frame numbers.\n - Text area populated with logs or outputs.\n\n### Bottom Bar\n- **Status Bar Components**:\n - Branch: `main`\n - Details: \n - `Git`, `Ln 745, Col 1`, `Space: 2`, `UTF-8`, `LF`, `Clj-kondo`, `points: 1`\n - Indications: `⦿`\n\n## Summarized Observations and High-Level Inference\nThe user is working within Visual Studio Code on a Clojure project related to processing image files, as indicated by multiple references to encoding images, handling timestamps, and managing directories for screenshots. The right panel likely shows the output from a REPL or terminal where a script or process is running that processes images, based on the consistent log messages. The bottom bar provides additional information about the coding environment, including line and column numbers, indicating active coding and debugging. The use of SSH in the title bar suggests remote access to a server or another machine for development.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729027287.jpg" "### Visual Hierarchical Description\n\n#### Application Window\n- **Application**: Visual Studio Code\n- **Connection**: SSH to \"telegram\"\n- **Project**: rustdesk.clj - simmie\n\n#### Top Bar\n- **Menu Options**: File, Edit, Selection, View, Go, Run, Terminal, Help\n- **Open Files**: \n - simmie.clj\n - nutdesk.clj\n - default_schema.edn\n - core.clj\n - openai.clj\n - more files (indicated by additional tabs)\n\n#### Left Panel\n- **Explorer View**: \n - Folder: SIMMIE (SSH TELEGRAM)\n - Structure:\n - .api\n - .clojure (collapsed)\n - .nrepl-port\n - .vscode (collapsed)\n - apache\n - classes\n - doc\n - downloads\n - releases\n - resources\n - src\n - simm\n - runtimes (open)\n - assistance.clj 91\n - base.clj 96\n - classic.clj 95\n - example.clj 91\n - operator.clj 91\n - ruddesk.clj 91\n - telegram.clj \n - text_extractor.clj\n - ...\n - .gitignore\n - build.clj\n - CHANGELOG.md\n - deps.edn\n - LICENSE\n - README.md\n - server.log (selected)\n\n#### Middle Panel\n- **File**: simmie.clj\n- **Code Snippet**:\n ```clojure\n ;;custom directives omitted for brevity\n\n (defn test-query\n (let [opterm (sh \"opt-4\" \n (vec (concat [\"-i\" type \"text\"\n ;;instructions omitted for brevity\n ])))\n ```\n- **Highlighted Lines**:\n- Functions and defs like `encode-screenshot`, `last-screenshots` \n\n#### Right Panel\n- **File**: output.calva-repl\n- **Console Output**: Log of file paths and operations\n - File Paths: /home/ubuntu/simmie/files/screenshots/4543547872/...\n - Log Messages: \n - \"new screenshot...\"\n - \"Trace...\"\n- **Highlighted Errors/Warnings** in Yellow:\n ```shell\n clojure.lang.Compiler.eval (Compiler.java:457)\n ```\n\n#### Bottom Bar\n- **Status Bar Information**:\n - Branch: main\n - Changes: 94\n - Issues: 0.235\n - Files: 94\n - Extensions (indicative): UTF-8, LF, Clojure, clj \n\n### Summary and Inference\nThe user is working in Visual Studio Code, editing Clojure files within a project titled \"rustdesk.clj - simmie.\" They have multiple code files open and are connected remotely via SSH to a server or device named \"telegram.\" The focus is on writing and possibly debugging code related to some kind of image processing or automation task, evidenced by functions dealing with screenshots and timestamps. The right panel displays logging output from a REPL session, indicating active development and debugging.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729009892.jpg" "### Visual Description\n\n**Browser URL:**\n- URL: `https://github.com/ryrobes/rvbbit`\n\n**Browser Tabs:**\n- Multiple tabs with various icons, indicating several open pages or applications.\n\n**Repository Overview:**\n- Repository Owner: ryrobes\n- Main Repository Section:\n - Branch: `master`\n - 1 Branch\n - 3 Tags\n - Last commit ID: `985d428`\n - \"Go to file\" button and dropdowns for \"Add file\" and \"Code\"\n - Total Commits: 267\n\n**File List:**\n- **Folders:**\n - `.github/`: \"funding yml\" - last month\n - `backend/`: \"PRE-HURRICANE COMMIT!\" - last week\n - `docs/`: \"- Added value Reactor REST endpoints.\" - last week\n - `frontend/`: \"PRE-HURRICANE COMMIT!\" - last week\n- **Files:**\n - `.gitignore`: \"PRE-HURRICANE COMMIT!\" - last week\n - `Dockerfile`: \"fun with docker\"\n - `README.md`: \"- Added value Reactor REST endpoints.\"\n - `compile_all.sh`: \"PRE-HURRICANE COMMIT!\"\n - `loc_history.csv`: \"PRE-HURRICANE COMMIT!\"\n - `write-cloc-history.sh`: \"more compile-all changes + various nits\" - 2 months ago\n - `zprint-config.sh`: \"over the deep end\" - 4 months ago\n - `zprint-flows.sh`: \"over the deep end\" - 4 months ago\n - `zprint-screens.sh`: \"over the deep end\" - 4 months ago\n\n**Right Sidebar:**\n- **About:**\n - Description: \"Reactive Data Board & Visual Flow Platform\"\n - Readme, Activity\n - 246 stars, 8 watching, 11 forks\n - \"Report repository\"\n\n- **Releases:**\n - 3 Tags\n\n- **Sponsor This Project:**\n - User: ryrobes (Ryan Robitaille)\n - \"Sponsor button\"\n - \"Learn more about GitHub Sponsors\"\n\n- **Packages:**\n - 1 Package: rvbbit\n\n- **Languages:**\n - Language Breakdown by bar graph and percentages:\n - Clojure: 91.6%\n - CSS: 6.4%\n - HTML: 1.6%\n - Other: 0.4%\n\n**README Section:**\n- **Title:**\n - \"RVBBIT DYNAMIC DATASCAPES\"\n- **Subtitle:** \n - \"Reactive Data Board & Visual Flow Platform\"\n- **Content:**\n - **Composability:**\n - \"The art of creating complex systems from simple, reusable parts.\"\n - **Feedback loop:** *[Additional content not displayed]*\n\n### Summary and Inference\n\nThe user is viewing a GitHub repository named \"rvbbit\" owned by a user named ryrobes. The repository is focused on a project described as a \"Reactive Data Board & Visual Flow Platform.\" The repository includes various folders and files related to software development, particularly involving Clojure, as indicated by the language breakdown.\n\nThe commit messages indicate a recent set of changes, marked with \"PRE-HURRICANE COMMIT!\" and others related to REST endpoints. The user is exploring project details, potential sponsor options, and examining code and documentation files within the project.\n\nOverall, the user appears to be reviewing or managing a software project on GitHub that involves managing complex systems through composability, potentially preparing for significant updates or a release, as suggested by the \"PRE-HURRICANE\" commits.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024462.jpg" "### Visual Information Description\n\n#### Application Details:\n- **Application Name:** Visual Studio Code\n- **Appearance:** Dark theme\n- **Toolbar Present:** File, Edit, Selection, View, Go, Run, Terminal, Help.\n\n#### Explorer Panel (Left Side):\n- **Open Folder:** simmie (Visible files and directories)\n - **Directories:** \n - `.calva`, `.output-window`, `.shadow-cljs`, `apache`, `baseline`, `cache`, `classes`, `docs`, `downloads`, `images`, `languages`, `runtimes`, `src`, `target`, `telegrams`, `tests`, `tmp`, `zutils`\n - **Files:** \n - `.gitignore`, `deps.edn`, `LICENSE`, `README.md`, `build.clj`, `server.log`\n - **Visible File in `src/simmie/languages/`:** `opencai.clj` (Highlighted)\n\n#### Editor Pane (Middle):\n- **File Open:** `simmie.clj` from `src/simmie` \n- **Code Content:**\n - **Namespaces Used:** `simmie`, `languages`, `db`\n - **Code Constructs:**\n - Use of `def`, `fn`, `let`, `map`, `require`, `comment`, etc.\n - **Functions:**\n - `extract-timestamp`, `test-screenshots`, `encode-to-base64`\n - **Variables:** \n - `all-screenshots`, `term-ch`\n - **Sample Code Snippet:**\n ```clojure\n (def test-screenshots (sort (all-screenshots \"/home/ubuntu/simmie/files/screenshots/\")))\n (def last-screenshots (map encode-image (take-last 10 test-screenshots)))\n ```\n\n#### Side Bar (Right Pane):\n- **Plugin/Extension:**\n - Possibly using `output/calva-repl` due to naming\n- **Text Content:**\n - **URLs:** \n - `https://github.com/…`\n - **Non-code Text Snippet:** \n - Discussion about using tools, metadata issues, FRP, messaging library integration\n - **Actions:** \n - Messages like \"new screenshot\" related operations on text lines\n - **References:** \n - Mention of intents, extraction functions, discussions on libraries (Clojure, OpenAI)\n\n### Observations and High-Level Inference:\n\n- **File Structure and Coding Environment:**\n - The environment is set up for Clojure development, potentially for a project or application named “simmie”.\n - The user is organizing and manipulating screenshots, suggesting work related to image processing or data extraction.\n\n- **Development Context:**\n - The user appears to be involved with backend functionalities, possibly interacting with APIs or handling real-time data processing.\n - Tasks seem to include development setup, code testing, and possibly integrating libraries.\n\n- **Task Insights:**\n - The user is likely debugging or reviewing the code for image handling, focusing on capturing, processing, or analyzing screenshots.\n\nThe user is working within a coding environment, handling processes related to screenshots, possibly linked with data extraction or visualization tasks. They are engaged in coding tasks requiring use of Clojure and integration with larger software systems or libraries.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024424.jpg" "### Screen Overview\n\n#### Application Window\n- **Application**: Visual Studio Code\n- **Title Bar**: `rustdesk.clj - simmie (SSH: telegram) - Visual Studio Code`\n\n#### Browser Tabs\n- No URL is visible since this is not a browser window.\n\n#### Layout\n- **Three Pane Layout**:\n - **Left Pane**: Stacktrace/Navigator\n - **Middle Pane**: Code Editor\n - **Right Pane**: Output/Terminal\n\n### Left Pane: Stacktrace/Navigator\n- **Header**: CALVA INSPECTOR\n- List of text entries, mostly related to Clojure core async implementation.\n - Examples include: \n - `#object[clojure.core.async.impl...`\n - `#file ...rustdesk/bench.clj`\n - `#file ...rustdesk/process_frames.clj`\n- Contains nested and hierarchical positioning with indentation.\n\n### Middle Pane: Code Editor\n- The main area displays code, focused on Clojure code.\n- Sections of code are color-coded for syntax highlighting.\n- **Code Elements**:\n - **Imports**:\n - `clojure.java.io`\n - `java.util.Date`\n - **Functions**:\n - `defn call-screenshots`\n - `defn encode-image`\n - `defn process-frames`\n - `def main`\n - **Variables & Data Structures**:\n - `image-bytes`\n - `term-ch`\n - `file-ex`\n - Commented with semicolons `;`\n - Code focuses on image processing tasks, particularly file handling of screenshots.\n\n### Right Pane: Output/Terminal\n- Displays a series of repetitive output indicating new screenshots being handled.\n - Format: `\"new screenshot\" \"/home/ubuntu/screenshots/...jpg\"`\n- Line numbers correspond to the entries.\n- Persistent output indicative of logging or terminal operations.\n\n### Status Bar\n- Indicators for line and column position, file type, and version control status.\n - Example: `Ln 59, Col 1 | Spaces: 2 | UTF-8 | Clojure | Git |`\n\n### Summary and Inference\n- **Purpose**: The user is working on a Clojure project within Visual Studio Code, specifically focused on handling and processing screenshots.\n- **Activities**: \n - Writing and debugging Clojure code for image processing.\n - Monitoring program output in relation to screenshots, likely generated or processed by the written script.\n- **Environment Configuration**: Connected to a remote environment via SSH, as indicated by `telegram` in the title bar.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024430.jpg" "## Visual Hierarchy Description\n\n### Application Window\n- **Application**: Visual Studio Code\n- **Title Bar**: \"rustdesk.clj - simmie [SSH: telegram] - Visual Studio Code\"\n\n### URL\n- No URL is visible as this is a code editor, not a web browser.\n\n### Main Interface\n1. **Sidebar (Explorer) on the left**\n - **EXPLORE Section**\n - **Folder Name**: \"SIMMIE [SEE TELEGRAM]\"\n - **Directories and Files**:\n - `.calva`, `output-window`, `output-calva-repl`, `.clojure`, `.lsp`, `.m2`, `public`, `resources`, `src`, `target`, etc.\n - **Highlighted Files** (some marked with red dots indicating errors or changes):\n - `assistance.clj`\n - `brand.clj`\n - `named.clj`\n - `report.clj`\n - `telegram.clj`\n - `text_extractor.clj`\n - `tower.clj`\n - **Bold/Italic files indicating opened or modified files**:\n - `rustdesk.clj`\n - `telegram.clj`\n\n2. **Central Editing Area (Main Editor)**\n - **File Open**: `rustdesk.clj`\n - **Code Content**:\n - Code relates to image and screenshot processing in Clojure.\n - Functions for processing frames and screenshots, timestamp extraction, and queries.\n - Methods such as `encodeToString`, `process-frames`, `extract-timestamp`, `delete-file`.\n - Code includes comments and function definitions.\n\n3. **Right Side Panel**\n - **File Open**: `output.calva-repl`\n - **Text Content**: Repeated lines of text displaying \"new screenshot\" along with file paths.\n - Example: `\"new screenshot\" \"/home/ubuntu/screenshots/454345782/frames_1728904269.jpg\"`\n\n### Bottom Bar\n- **Status Information**:\n - Branch: `main`\n - Version Control: Sync status\n - Errors/Warnings count\n - File information: `Ln 63, Col 109`, `Spaces: 2`, `UTF-8`, `LF`\n\n### Summary and User Activity Inference\n- **Current Activity**: The user is editing and debugging Clojure code in Visual Studio Code. The code appears to be related to screenshot management and processing, possibly for an application involving image handling or data extraction. The presence of SSH in the title suggests remote work or development on another machine.\n- **Tasks Involved**: Code analysis and debugging, managing project files, reviewing output logs for \"new screenshots,\" and possibly incorporating OpenAI's API based on the query example seen in the code.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024459.jpg" "### Hierarchical Description of Visual Information\n\n#### Application\n- **Type:** Visual Studio Code\n- **Mode:** Split Editor View\n- **Theme:** Dark\n\n#### Top Bar\n- **File Name:** `rustdesk.clj - simmie [SSH: telegram] - Visual Studio Code`\n- **Tabs Open:** \n - `dimmed.clj`\n - `simmie.clj`\n - `rustdesk.clj`\n - `twocrl.clj`\n - `default_schema.edn`\n - `peer.clj`\n - `openai.clj`\n\n#### Left Panel\n- **Section:** Explorer\n- **Directory Hierarchy:**\n - **Main Directory:** `simmie [SSH: TELEGRAM]`\n - **Folders and Subdirectories:**\n - `output`\n - `resources`\n - `src`\n - `sum`\n - `runtimes`\n - `abstractions.clj`\n - `eszte.clj`\n - `ess.clj`\n - `openai.clj`\n - `rustdesk.clj` (active)\n - `operator.clj`\n - `testdesk1.clj`\n - `testdesk2.clj`\n - `twitter.clj`\n - `telegram.clj`\n - `extractor.clj`\n - `tests`\n - `public`\n - `example_config.edn`\n - **Files:**\n - `.gitignore`\n - `LICENSE`\n - `README.md`\n - `server.java`\n\n#### Middle Editor Panel\n- **File:** `rustdesk.clj`\n- **Content Example:**\n ```clojure\n (ns simm.runtimes.rustdesk\n (:require\n [clojure.core.async :refer [!!]])\n )\n \n (def test-query [<-- openai/chat \"gpt-4\"\n (vec (concat [{:type \"text\"\n :text \"You are a note taker for a screen transcription tool...\" }]\n (map (fn [x] [:data/image! x]) last-screenshots)\n ))])\n ```\n\n#### Right Editor Panel\n- **File:** `output.calva-repl`\n- **Content:** Appears to be a REPL or log output with code execution results and comments\n- **Example Output:**\n ```shell\n ;; => new Screenshot: \"/home/ubuntu/simmie/files/screenshots/4543547852/frames_1729920436.jpg\"\n ;; => ilesm.runtimes.rustdesk/last-screenshots\n ;; => ilesm.runtimes.rustdesk/test-query\n ```\n\n#### Bottom Bar\n- **File Navigation:** `Line 100, Col 20 | Spaces: 2 | UTF-8 | Clojure`\n- **Notification:** `No problems | Synchronized to: Telegram`\n\n### Summary and High-Level Inferences\n- The user is working within Visual Studio Code, primarily engaging with Clojure code files. The directories suggest a focus on runtime functionalities for various services such as `openai`, `rustdesk`, and `telegram`.\n- The left panel shows file navigation focused on Clojure namespaces, particularly within the `runtimes` directory.\n- The middle editor panel displays active development on the `rustdesk.clj` file, with functions involve asynchronous programming and interaction with OpenAI's services.\n- The right panel indicates execution output or logging from the REPL, which might be validating the functionality of the code or debugging.\n- It seems the user is working on backend/frontend interaction, likely involving image processing or timestamp extraction based on folder names and code comments.\n- Overall, this session involves interaction with external APIs, the application of Clojure libraries, and script testing via a terminal session to `Telegram`.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024517.jpg" "## Hierarchical Screen Description\n\n### Main Application\n- **Application Name**: Visual Studio Code\n- **File Name**: `rustdesk.clj`\n- **Workspace Information**: Connected via SSH to \"telegram\"\n\n### Layout\n- **Top Bar Tabs**: Multiple files opened including `simmie.clj`, `numnex.clj`, `rustdesk.clj`, `default_schema.edn`, `tower.clj`, `perci.clj`, `openai.clj`\n \n### Left Panel\n- **Explorer/Navigation Panel**: Contains a list of files, methods, or objects under the heading \"Clojure Inspector\"\n - Includes a list of `ManyToManyChannel` objects with identifiers (e.g., `0x879280a7`, `0x196b1d07`)\n - File paths related to namespaces such as `si.simm.runtimes.nusitex/frame`, `si.simm.runtimes.rustdesk/bkem`, etc.\n \n### Middle Panel\n- **Code Editor**\n - Programming Language: Presumably Clojure based on the file extension and syntax\n - Code Context: Defining functions and processing screenshots\n - Function definitions include `all-screenshots`, `encode-image`, `process-frames`\n - Code operations involve file path manipulations and image processing\n - Usage of libraries like `clojure.data.json`, `clojure.java.io`, `java.util.Date`\n\n### Right Panel\n- **Output or Log Panel**\n - Displays a continuous list indicating the creation of new screenshots, with file paths:\n - Example paths: `/home/ubuntu/screenshots/454354782/frames_1728949761.jpg`\n - Each entry is prefixed by \"new screenshot\" followed by the file path\n\n### Bottom Bar\n- **Status Bar Information**\n - Branch: `main`\n - Code Metrics: 2964 characters, 1110 words\n - Programming Language: Clojure\n - Additional Info: Line 59, Column 2, UTF-8 encoding, LF line feed\n\n## Summary and High-Level Inference\nThe user is working in Visual Studio Code, primarily editing a Clojure file named `rustdesk.clj`. The focus of their work appears to involve manipulating screenshots, likely automating tasks involving image processing and organization. They are using several functions and a structured workspace with multiple script files open, indicating they are possibly developing or maintaining a part of a larger codebase. The presence of log outputs in the right panel suggests active debugging or monitoring of the script's operations.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024467.jpg" "Sure, here's a comprehensive description of the screen:\n\n### General Layout\n- **Application:** Visual Studio Code\n- **Open Tabs:** \n - `simmie.clj`\n - `nutdesk.clj`\n - `default_schema.edn`\n - `peer.clj`\n - `output.calva-repl`\n- **Color Theme:** Dark theme\n- **View Split:** Two main panels with a vertical divider.\n\n### Top Section\n- **Top Panel (Menu Bar):** \n - Options: File, Edit, Selection, View, Go, Run, Terminal, Help\n\n### Left Panel (File Explorer)\n- **Project:** `rustdesk.clj`\n- **Directory Structure:**\n - `output-window`\n - `.clojure`\n - `src`\n - `language.clj`\n - `runtimes`\n - Selected file: `telegram.clj`\n - Other Files:\n - `.gitignore`\n - `build.clj`\n - `CHANGELOG.md`\n - `LICENSE`\n - `README.md`\n - `server.log`\n- **Highlighted Item:** `telegram.clj`\n\n### Middle Panel (Code Editor)\n- **Open File:** `simmie.clj` (with unsaved changes)\n- **Code Content:**\n - Contains functions and operations related to screenshots, encoding files, disposing sessions, timestamping, etc.\n - `defschema` usage, `db` operations, `alt-screenshots`, `test-query`\n - Frequent words: `openai`, `def`, `process-frame`, `async`, `session`.\n- **Code Examples:**\n ```clojure\n (def term-ch (chan))\n (process-frame \"454354782\" term-ch)\n (encode-to-base64 (read-file-as-byte-array))\n (def test-query \"opt-4o\")\n ```\n\n### Right Panel (Terminal/Output)\n- **Open File:** `output.calva-repl`\n- **Output Details:**\n - Logs from the REPL, showing operations like screenshot encoding, dates, file paths, and function reference details.\n - Display of code execution results and stack traces.\n- **Frequent File Paths:**\n - `/home/ubuntu/simmie/files/screenshots`\n- **Most Recent Logs:**\n ```plaintext\n New screenshot: /home/ubuntu/simmie/files/screenshots/454354782/frames_1729824493.jpg\n ```\n\n### Bottom Bar\n- **Status Information:**\n - `Ln 117, Col 2 Spaces: 2 UTF-8 LF`\n - Language details and extensions in use, like Calva.\n\n### Observations and Inferences\n- **User Activity Context:**\n - The user appears engaged in both collaborative discussions and technical development.\n - Likely using Clojure as the primary coding language.\n - Files and operations suggest backend/frontend interaction.\n- **Inferred Goal:**\n - Modifying or maintaining a tool or script related to screenshot processing.\n - Possibly involved in open-source or research projects.\n\nThe image shows in detail the configuration and work setup of a developer involved in backend/frontend operations, primarily focused on tasks like encoding screenshots and processing files using screenshots and other related functions.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729009891.jpg" "### Screen Hierarchy Description\n\n**Application:** Slack\n\n**Browser:** No web browser visible, Slack is a standalone app\n\n**Workspace:** clojurians\n\n---\n\n#### Sidebar (Left Panel)\n\n- **Sections:**\n - \"DMs\"\n - \"Activity\"\n - \"Later\"\n\n- **Channels:**\n - Pinned/unread:\n - `#clojurelm` (highlighted)\n - Other channels:\n - `#admin-announcements`\n - `#anglican`\n - `#asaml`\n - `#babashka`\n - `#clara`\n - `#clj-kondo`\n - `#cljs-dev`\n - `#clojure`\n - `#clojure-bay-area`\n - `#clojurescript`\n - `#closh`\n - `#cir`\n - `#consultantsdirectory`\n - `#core-typed`\n - `#cortex`\n - `#data-science`\n - `#datahike`\n - `#datalog`\n - `#datascript`\n - `#datomic`\n - `#declarative-dataflow`\n - `#events`\n - `#graalvm`\n - `#holy-lambda`\n - `#hyperfiddle`\n - `#jank`\n - `#klipse`\n - `#local-first-clojure`\n - `#metaprob`\n - `#missionary`\n - `#mxnet`\n - `#off-topic`\n - \"More unread\"\n\n#### Main Panel (Center)\n\n- **Channel:** `#clojurelm`\n\n- **Header:**\n - \"Messages\" | \"Canvas\" | \"Files\" | \"Bookmarks\"\n\n- **Messages:**\n - Older message by John:\n - Date: Thursday, September 26th\n - Time: 1:07 PM\n - Content:\n - Discussion on Model2Vec and AI tool for distilling small models\n - Link to article: `https://www.marktechpost.com/2024/09/25/minish-lab-releases-model2vec-an-ai-tool-for-distilling-small-super-fast-models-from-any-sentence-transformer/`\n - Article summary:\n - Minish Lab's Model2Vec tool description\n - Benefits and goals in NLP tasks\n - Est. reading time: 5 minutes\n - Written by Asif Razzaq\n\n - **Reactions and Replies:**\n - 33 replies\n - 21 reactions (last reply 24 days ago)\n\n#### Thread Panel (Right)\n\n- **Thread Messages:**\n - Discussion participants: phronmophobic, John\n - Recent content discusses LORA, quantization, and smaller models.\n - Topics cover model sizes, future predictions for coding assistants, combining chips with LLM, etc.\n - Mention of future AI capabilities in runtimes and exception handling in code.\n\n#### Summary and User Activity:\n\nThe user is likely involved in a Slack workspace focused on programming, specifically related to Clojure. They are engaged in discussions about AI model optimization and future technology projections within a selected channel (`#clojurelm`). The main topic in both the main and thread panels revolves around Model2Vec and its potential in developing efficient AI models, which is discussed in articles and user feedback.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729027291.jpg" "### Screen Description\n\n**Application:** Visual Studio Code\n\n**Top Bar:**\n- Currently opened file: `rustdesk.clj - simmie [SSH: telegram]`\n- Menu options: `File`, `Edit`, `Selection`, `View`, `Go`, `Run`, `Terminal`, `Help`\n\n**Editor Tabs:**\n- Open files include: `simmie.clj`, `output.calva-repl`, `nudgedesk.clj`, `default.schema.edn`, `tower.clj`, `peer.clj`, `openai.clj`\n\n**Left Panel - Explorer:**\n- Header: `EXPLORER`\n- Directory: `simmie [SSH: TELEGRAM]`\n- Various folders and files are displayed with a hierarchical structure:\n - `.calva`, `output-window`\n - `docker`\n - `edn`\n - `node`\n - `project.clj`\n - `src`\n - `simmie`\n - `runtimes`\n - Files like `base.clj`, `etc.clj`, `telegram.clj`\n - File `rustdesk.clj` is highlighted\n - `.gitignore`\n - `README.md`\n\n**Main Editor (Middle Panel):**\n- File Open: `rustdesk.clj`\n- Syntax highlighting suggests the file is code.\n- Contains Clojure code, with functions and definitions such as:\n - `frame-file`, `process-frames`, `main`, `aprm`\n - `defn` for defining functions\n - Associations with various `db` operations like `db.type/ref`, `db.cardinality/one`\n - Comments explaining code or intentions\n - Example paths like `/home/ubuntu/...`\n\n**Right Panel - Output:**\n- File: `output.calva-repl`\n- Appears to display console or REPL output.\n- Contains file paths (e.g., `/home/ubuntu/simmie/files/screenshots/...`)\n- Error messages or stack traces:\n - Mentions of `clojure.core`, `clojure.lang.Compiler`\n - Functions such as `invokeStatic`, `eval`, `apply`\n- Highlighted lines focus on runtime errors and stack traces\n\n### Summary and Inference\n\nThis Visual Studio Code setup is configured for coding and compiling Clojure projects, with SSH access to a remote machine (\"telegram\"). The user appears to be working on a Clojure project involving file and screenshot processing. They are actively coding by modifying files (`rustdesk.clj`) and monitoring output and errors on the REPL pane (`output.calva-repl`). Error messages suggest a troubleshooting or debugging phase, potentially resolving runtime issues. The use of SSH indicates remote development or testing.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729009894.jpg" "### Screen Description\n\n#### Browser\n- **URL Bar**: \n - URL: `https://github.com/yrro/flowmaps`\n- **Tabs**: Various icons are visible, suggesting multiple open tabs.\n\n#### Content Area\n- **Header**:\n - \"README\" and \"EPL-2.0 License\" are visible, indicating a code repository page.\n \n- **Main Section**:\n - **Title**: Large text reads, \"Rebooting flow-based programming in Clojure by enabling effortless orchestration of core.async pipelines & intricate application flows.\"\n - **Bullet Points**:\n - \"Craft flows in a simple bullshit-free map structure, watch them come to life in a time-traveling 'Rabbit' canvas UI.\"\n - \"Debug, visualize, and experiment in real-time, helping to ensure observability and understanding anywhere you need async chains.\"\n - **Navigation Links** (listed with indentation showing hierarchy):\n - What is Flow Based Programming?\n - Getting started (Start Here!)\n - Basics\n - “Curiouser and curiouser!”\n - Conditional Paths\n - Optional block \"views\"\n - Static value inputs\n - Optional block \"speak\" / text-to-speech\n - Grid data / \"rowsets\"\n - Multi-input Blocks (i.e. input \"ports\")\n - Sub-flows!\n - pre-when? and :post-when?\n - Pre-run “starter” values for functions\n - channel REST access / pre-configured endpoints with :joints (WIP)\n - Push only\n - Push and fetch (wait)\n - pre-configured named endpoints - :joints\n - Hiding sensitive values from the UI (API Keys, passwords, etc)\n - Optional block \"canvas\" metadata\n - Features\n - The Core \"flow runner\"\n\n### Summary and Inference\nThe user is viewing a GitHub page for a project named \"flowmaps\" focused on flow-based programming using Clojure. This page provides documentation and possibly instructions or an overview of the project's features and structures. The information suggests the user might be reviewing or considering using this tool for developing or understanding asynchronous programming flows.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024444.jpg" "## Hierarchical Screen Description\n\n### Browser Information\n- **URL**: `https://app.slack.com/client/T0R3CGP7R/missionary`\n\n### Application Interface\n- **Application**: Slack\n- **Workspace**: clojurians\n\n### Left Sidebar\n- **Sections**:\n - **Channels**:\n - missionary\n - mxnet\n - off-topic\n - other-languages\n - pattern\n - releases\n - remote-jobs\n - remote-meetup\n - shadow-cljs\n - slack-help\n - spectrum\n - uncomplicate\n - visual-tools\n - xtdb\n - Add channels\n - **Direct messages**:\n - alekc2\n - awb99\n - Coby Tamayo\n - Diana Fernanda Belle\n - grounded_sage\n - jaged\n - littlj Josu San Martín\n - Jonás Östlund\n - Josu San Martín\n - pat\n - silian\n - timo\n - viesti\n - whilo you\n - Add coworkers\n - **Apps**:\n - Slackbot\n - Add apps\n\n### Main Panel\n- **View**: Channel - #missionary\n- **Tabs**:\n - **Messages** (selected)\n - Canvas\n - Files\n - Bookmarks\n\n- **Message Display**:\n - **awb99** at 11:56 AM\n - Text: \"The operator that I believe is missing could be m/flow-or if let flow return the value from the flow, or a fixed value.\"\n - Reply: \"I hope I am not talking nonsense here... I took 2 days to think about this, and was reading the docs, the cookbook and missionary code over and over again.\"\n - Timestamp: Yesterday 12:28 AM\n\n - **whilo** at 7:10 PM\n - Text: \"Hey! I really like FRP and am happy that missionary makes so much progress...\"\n - Replies: 6 replies last Friday at 12:02 PM\n \n - **Absolute Negativity** at 6:55 AM\n - Reply to a thread: \"First time I notice, missionary doesn't always work with honeysql...\"\n - Code snippet:\n ```clojure\n (defn foo [jdbc select]\n (jdbc/execute! jdbc\n (sql/find?\n :product :products\n :select [(vary-meta select-clause (constantly nil))])))\n ```\n - Text: \"ignored-metadata didn't work in my case, not sure why, ended up doing (vary-meta select-clause (constantly nil)).\"\n - New replies and options\n\n### Right Panel\n- **Thread**:\n - **whilo** at 11:57 AM\n - Text: Detailed project description including GitHub links.\n - Mentioned projects:\n - **simmie**: Stars: 2, Language: Clojure\n - **daphne**: Stars: 37, Language: Clojure\n - Closing statement regarding discussions about mission projects.\n\n### Observations\n- **User Activity**: \n - Engaging in technical discussions about Clojure programming and related projects in a Slack workspace.\n - Participating in threads about projects and sharing code snippets.\n - Collaborating on software development ideas related to the `missionary` library and other Clojure-based initiatives.\n\n- **Additional Information**:\n - Users are discussing technical implementations and sharing code and resources via GitHub.\n - There is an interest in improving software functionality and integrating new features.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024441.jpg" "### Hierarchical Visual Description of the Screenshot\n\n#### Application\n- **Application**: Visual Studio Code\n\n#### Top Navigation Bar\n- **Text**: `File`, `Edit`, `Selection`, `View`, `Go`, `Run`, `Terminal`, `Help`\n\n#### Title Bar\n- **Title**: `rustdesk.clj - simmie [SSH: telegram] - Visual Studio Code`\n\n#### Main Window\n1. **Left Pane (Explorer)**\n - **Section**: Explorer\n - **Files and Directories Tree**:\n - `calva/`, `output.window/`\n - `.clojure/`, `.nrepl.log`, `.nrepl.port`, `README.md`, etc.\n - `src/`\n - `simmie/`\n - `languages/`, `processes/`, `runtimes/`\n - Highlighted file: `rustdesk.clj`\n - `telegram.nii`, `test_extractor.clj`, etc.\n - **Other Sections**:\n - Timeline, Outline, Project Tree\n\n2. **Middle Pane (Code Editor)**\n - **Filename**: `simmie.clj`\n - **Code Content**: \n - Function definitions such as `main`, `extract-timestamp`, `schema`\n - Requires directives, comments\n - Example:\n ```clojure\n (ns simmie)\n (defn process-frames [session term-ch])\n ;; rest of the code\n ```\n - Highlighted line number: 54\n - Status details at the bottom: `Ln 74, Col 1 Spaces: 2 UTF-8 Clojure`\n\n3. **Right Pane (Terminal/Output)**\n - **Tab**: `output.calva/repl`\n - **Content**: \n - Repeated lines starting with `\"new screenshot\"` followed by file paths, e.g., \n ```\n \"new screenshot\" \"/home/ubuntu/simmie/files/screenshots/454354782/frames_1728907909.jpg\"\n ```\n - Scroll bar indicates a large list\n\n#### URL/Browser\n- **No URL or Browser** visible. The application is focused on code editing and terminal outputs.\n\n### Summary and Inference\nThe user is working in Visual Studio Code with a focus on Clojure code (file: `simmie.clj`) which seems to involve processing screenshot files on a system. The code likely deals with manipulation or management of image files. The output pane shows a log or continuous printout of new screenshot file paths, suggesting automated screenshot handling is part of the current task. The user is likely developing or testing code related to image processing or system monitoring.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024446.jpg" "## Hierarchical Screen Description:\n\n### URL:\n- `app.slack.com/client/T03RZCPGFP/R03R2MD9F6H?cdn_fallback=2`\n\n### Main Interface:\n- Purple sidebar on the left with Slack workspace information\n- Main chat window in the center\n- Thread panel on the right\n\n### Sidebar (Left):\n- Workspace name: *clojurians*\n- Channels listed:\n - `clojure`\n - `clojure-bay-area`\n - `clojurellm`\n - `clojurescript`\n - `closh`\n - `clr`\n - `consultantsdirectory`\n - `core-typed`\n - `cortex`\n - `data-science`\n - `datahike`\n - `datalog`\n - `datascript`\n - Highlighted channel: `#datomic`\n - (and many more)\n\n### Main Chat Window (Center):\n- Channel: `#datomic`\n- Button: `+ More unread`\n- Messages:\n - **Tobias Sjögren**: \n - Date: Friday, October 4th\n - \"Does this assert that Francis is a person?: 1234 :person/name \"Francis\"\"\n - **Tobias Sjögren**:\n - Date: Saturday, October 5th\n - \"What could be some typical examples of assertions for the same entity where different namespaces are used for its attributes?\"\n - **Black**:\n - Date: Monday, October 7th\n - Question about backing up `postgres db` for `datomic transactor` as storage\n - **pez**:\n - Date: October 7th\n - Shared resource link: `https://www.learndatalogtoday.org/chapter/3`\n - Discusses sorting movie titles by year of release, example with code snippet provided\n - **samedhi**:\n - Date: Wednesday, October 9th\n - Seeks guidance on a \"consistent pain point\" related to `nil` values in datoms\n\n### Thread Panel (Right):\n- **Tobias Sjögren**:\n - Date: October 3rd\n - Question about replicating RDF's `rdfs:Range` constraint in Datomic\n- **favila** (response):\n - Explains that restricting range to specific types involves \"gatekeeping functions\"\n - Mentions \"attribute predicates,\" \"transaction functions,\" and \"entity specs and predicates\"\n - Notes that \"violations are 'unsatisfiable' after the fact, not inexpressible before\"\n\n### Observations Summary:\n- The user is in a Slack workspace named \"clojurians\" and is focused on the `#datomic` channel.\n- The discussions are technical, revolving around Datomic and data handling, with a particular focus on attributes, namespaces, and constraints.\n- Questions and explanations involve data assertions, potential errors with `nil` values, and replication of RDF constraints.\n- A collaborative environment with active threads and resource sharing, indicating community support and learning.\n\n### High-level User Activity:\nThe user is participating in a Slack community focused on discussing and solving issues related to Datomic. They are engaged in technical discussions, seeking advice, sharing resources, and responding to queries on data management and programming constraints within Datomic.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729009898.jpg" "### Visual Information Description\n\n#### Application Window\n- **Software**: Visual Studio Code\n- **Window Title**: `output.calva-repl - simmie (SSH: telegram) - Visual Studio Code`\n \n#### Main Layout\n- **Divided View**: The screen is divided into two main sections—left (code editor) and right (output panel).\n\n#### Left Section (Code Editor)\n- **Tabs Open**:\n - `simmie.clj`\n - `output_window.clj`\n - `nudgedesk_v4.clj`\n - `default_schema.clj`\n - `tower.clj`\n - `openai.clj`\n - `apollo.clj`\n - `marshal.clj`\n\n- **File Displayed**: Appears to be `simmie.clj`\n- **Code Language**: Clojure\n- **Code Content**:\n - Functions defined such as `defn`, `process-frames`, `extract-timestamp`.\n - Comments start with `;`.\n - Various paths like `/home/ubuntu/screenshots/...`\n - Variables like `term-ch`, `all-screenshots`, `new-screenshots`.\n - Functionality for processing screenshots, encoding images, and handling timestamps.\n\n- **Visible Code Elements**:\n - Functions definitions.\n - Conditional and looping constructs.\n - Use of libraries such as `clojure.core.async`.\n\n#### Right Section (Output Panel)\n- **File Displayed**: `output.calva-repl`\n- **Content**: \n - Repeated lines displaying:\n - `\"processing\" \"/home/ubuntu/screenshots/434347582/frames_1728961561.jpg\"`\n - Appears to show processed items, likely log or execution output.\n\n#### Interface Details\n- **Menu Options** (at the top): File, Edit, Selection, View, Go, Run, Terminal, Help.\n\n### Summary and Inference\n\nThe user is working within Visual Studio Code, focusing on a Clojure project involving image processing. The code editor shows a script (`simmie.clj`) that processes screenshots by extracting frames, timestamps, and encoding them. The output panel (`output.calva-repl`) displays a log of processed frames, suggesting real-time or recent execution of the script.\n\nThe user is likely developing or debugging an application that automatically processes images/screenshots, perhaps for archiving or analysis tasks, in a remote environment accessed via SSH (indicated by \"SSH: telegram\" in the window title).", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024434.jpg" "### Visual Description\n\n#### Application Window\n- **Application:** Visual Studio Code\n- **Title:** `rustdesk.clj - simmie [SSH: telegram] - Visual Studio Code`\n \n#### Main Interface Layout\n- **Split Window:**\n - **Left Pane (Explorer):**\n - **Header:** `EXPLORER`\n - **Project Root Folder:** `simmie [SSH: TELEGRAM]`\n - **File Structure:**\n - `.repl`\n - `default.nrepl.edn`\n - `.vscode`\n - `launch.json`\n - `.gitignore`\n - Directories: `assets`, `classes`, `deps`, `downloadsd`, `logs`, `nbeces`, `resources`, `src`, `public`, `target`\n - Specific Directory Focused: `src`\n - Subdirectory: `simmie`\n - `process.clj`, `runtime.clj`, `extract.clj`, `iotacicl.clj`, `reporter.clj`, `rustdesk.clj`, etc.\n - **Highlighted File:** `rustdesk.clj`\n\n - **Middle Pane:**\n - **Opened File:** `rustdesk.clj`\n - **Content Overview:**\n - Functions defined: `frame-file`, `process-frames`, `main`, `dispose`, `extract-timestamp`, etc.\n - Contains Clojure code with functions and implementations focusing on image processing and screenshot management.\n\n - **Right Pane:**\n - **Opened File:** `output.calva-repl`\n - **Content:**\n - Listing of screenshots from a directory path, e.g., `/home/ubuntu/screenshots/...`\n - Log of actions and evaluations, primarily involving interaction with runtime and processing scripts.\n \n#### Status Bar\n- **Position and Information:**\n - Line: `64`, Column: `1`, Spaces: `2`\n - Encoding/Language: `UTF-8`, Clojure programming mode.\n - Status: `NORMAL` operational mode.\n\n### Summary and Inference\nThe user is working within Visual Studio Code, actively developing and debugging in a Clojure environment on a project named `simmie` connected via SSH to a host named \"telegram\". The focus is on a file called `rustdesk.clj`, part of likely a larger system dealing with image processing, specifically connected to managing and handling screenshots. The logs show significant output from executed scripts possibly related to testing or monitoring output. The user's activities seem to include code development, and running tests in a REPL environment, likely fine-tuning functions that handle files and directories in an automated or procedural fashion involving screenshots.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024440.jpg" "**Visual Information Breakdown:**\n\n1. **Application and UI:**\n - The image shows a Visual Studio Code (VSCode) environment.\n - The active tabs in the editor are: `simwire.clj`, `rustdesk.clj`, `default_schema.edn`, `operator.clj`, `peer.clj`, `openai.clj`.\n - The status bar at the bottom indicates the branch `main` is being worked on in a Git repository.\n - Other status indicators: Lines, Columns, Spaces, Language Mode (Clojure), and other tools/extensions.\n\n2. **Explorer Pane (Left Side):**\n - Project folder named `simwire (SSH: TELEGRAM)`.\n - File structure includes folders and files such as `.cljc` files, Clojure source files (`*.clj`), and configuration files (`*.edn`).\n - Selected file in the explorer is `telegram.clj`.\n\n3. **Editor Pane (Middle):**\n - Source code is written in Clojure.\n - Functions and macros like `defschema`, `extract-timestamp`, `main`, and commented sections for different purposes.\n - Code logic includes database schema definitions, processing images, and screen capture handling.\n\n4. **Output Pane (Right Side):**\n - Lists numerous log entries with the format: `new screenshot \"/home/ubuntu/simwire/files/screenshots/...`.\n - Each line records a timestamped screenshot file path.\n\n5. **Visible Text:**\n - Code includes function definitions and schema declarations.\n - Comments inside the code: `; require 'supervy.async :refer :all`, `; require 'clojure.core.async :refer [go]`.\n\n6. **Summary of Activity:**\n - The user is working on a Clojure-based project named `simwire` that involves real-time operations, possibly related to processing screenshots.\n - The project involves database schemas, asynchronous processes, and some form of interaction or data processing with the Telegram service.\n - The output pane indicates continuous logging of screenshot captures, possibly for debugging or monitoring purposes. \n\nThis indicates the user is engaged in software development, specifically in handling real-time data or media processing, likely in an asynchronous or event-driven architecture.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024448.jpg" "# Screen Structure and Content\n\n## Application\n- **Application:** Visual Studio Code\n- **Title Bar:** \"rustdesk.clj - simmie [SSH: telegram] - Visual Studio Code\"\n\n## Layout\n- **Split View:**\n - **Left Pane:** Explorer panel\n - **Middle Pane:** Editor panel\n - **Right Pane:** Output panel\n\n## Explorer Panel (Left)\n- **File Structure:**\n - Root Directory: `simmie`\n - Subdirectories and files are listed hierarchically, with some expanded and others collapsed.\n - **Highlighted File:** `rustdesk.clj`\n\n## Editor Panel (Middle)\n- **File Opened:** `rustdesk.clj`\n- **Visible Code Snippet:**\n - Code appears to be written in a Lisp-like syntax, possibly Clojure.\n - Various functions and variables are defined such as:\n - `extract-timestamp`\n - `term-ch`\n - `test-screenshot`\n - `encode-to-based64`\n - Comment text includes:\n - `\"you are a note taker for a screen transcription tool.\"`\n- **Code Snippets:**\n - Functions for processing screenshots.\n - Usage of channels and asynchronous operations (`go` and `>\n (all-screenshots \"/home/ubuntu/simmiev/files/screenshots/\")\n (take 5)\n (map (fn [_] \"processing\"))\n (map (fn [s] (openai/chat \"gpt-4-0\"\n {:type \"text\"\n :vec (concat (\"You are a note taker for a screen transcription tool. Describe all the visual information...\" ...)\n ...\n ))\n ```\n\n#### Right Pane (Terminal/Output):\n- **Output/Debug Panel (Right Pane):**\n - **Text Output:**\n - Repeatedly lists:\n - `\"processing\" \"/home/ubuntu/simmiev/files/screenshots/454345782/frames_1729024464.jpg\"`\n - `\"processing\" \"/home/ubuntu/simmiev/files/screenshots/454345782/frames_1729024468.jpg\"`\n - ... (numerous similar entries following the same pattern)\n - Command and status messages related to a process/script being run:\n - `clojure` and file path `... transcriptions.clj`\n\n#### Status Bar (Bottom):\n- Language mode, line and column numbers, spaces, clojure parameters, etc.\n - **Details:** `Ln 129, Col 1 Spaces: 2 UTF-8 LF Clojure ... `\n\n### Summary and High-Level Inference:\n- The user is working on a Clojure project in Visual Studio Code, accessed remotely via SSH connection to a host labeled \"telegram.\"\n- They are working with a file named `rustdesk.clj` and are likely developing or testing a script involving screenshots processing or transcription.\n- The script likely uses some form of AI model interaction (`openai/chat gpt-4-0`) to describe visual information in screenshots.\n- The right pane shows output logs that indicate ongoing processing tasks, which suggests the script is actively running or being debugged. The tasks include loading images and transcribing their content.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729027302.jpg" "## Screen Description\n\n### Application Title Bar\n- **Application Name**: Visual Studio Code\n- **Current File**: `rustdesk.clj - simmie [SSH: telegram]`\n- **Application State**: `Visual Studio Code - [SSH connection with a host named telegram]`\n\n### Menu Bar\n- **Menu Options**: \n - File\n - Edit\n - Selection\n - View\n - Go\n - Run\n - Terminal\n - Help\n\n### Sidebar (Explorer Panel)\n- **Open Folders/Files**:\n - Current path: `~/deft`\n - `output-`: Directory\n - `example_clojure_repl`: Directory\n - `.vscode`: Directory\n - `api.clj`: Clojure file\n - `classes`: Directory\n - `docs`: Directory\n - `flowcharts`: Directory\n - `public`: Directory\n - `resources`: Directory\n - `resources`: Directory\n - `scripts`: Directory\n - `simmie`: Directory\n- **Focused File**: `simmie`\n - Subdirectories:\n - `languages`\n - `assistance.clj`\n - `backup.clj`\n - `citation.clj`\n - `eliza.clj`\n - `report.clj`\n - Highlighted: `report.clj`\n - `telegram.clj`\n - `text_extractor.clj`\n - `.gitignore`\n - `build.clj`\n - `LICENSE`\n - `README.md`\n\n### Editor Panel\n- **File in Focus**: `simmie.clj`\n- **Highlighted Code Snippet**:\n ```clojure\n (comment\n (extract-timestamp \"/home/ubuntu/simmie/files/screenshots/454345782/frames_1729027469.jpg\")\n\n (def test-screenshots\n (all-screenshots \"/home/ubuntu/simmie/files/screenshots/454347852/frames_1729024246.jpg\")\n )\n )\n\n (def term (-> chan))\n\n (process-frames \"/454347352\") => term-ch\n \n (put-term ch..)\n \n (def test-cover\n @(openai/chat “gpt-4-0”\n (vec (concat\n [\n \"You are a note taker for a screen transcription tool. Describe alt the visual information as precisely as possible such that it can be retained even if the image is lost. Describe the screen hierarchically. If a browser is visible, write down the URL that is open. Write down *all* text you can see in the respective context. Summarize your observations in the end and try to infer what the user is doing on a high-level.\"\n ]\n (image_url “image/jpg” {:url “sdata:image/jpeg;base64, “ +> 98{@sp\"}\n )\n )\n )\n ```\n\n### Terminal/Output Panel\n- **Terminal Path**: `/home/ubuntu/simmie/files/screenshots/`\n- **Logs/Errors**:\n - Messages related to Clojure code execution:\n - Errors, evaluation, session execution details with the path and operations.\n - Example: Execution error - concurrent lock acquisition failure in Java environment.\n\n### Key Observations & High-Level Summary\n- The user is developing a Clojure code snippet within the Visual Studio Code using an SSH connection to a host named \"telegram\".\n- The highlighted code snippet involves a function that is processing image file paths, possibly related to transcription or analysis of image data.\n- The sidebar showcases a typical directory structure for project management, with separate files and subdirectories for different functionalities, like language support and reports.\n- The terminal output window shows detailed logs which might correspond to running tasks or scripts, indicating some encountered errors in concurrent Java operations.\n- Overall, this setup is tailored for a software development environment focused on scripting and automation using Clojure, potentially for processing image data or related tasks.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024465.jpg" "**Screen Hierarchical Description:**\n\n**Application Window:**\n- **Application:** Visual Studio Code\n- **Project/Folder Name:** simmie [SSH: TELEGRAM]\n- **Visible Tabs:**\n - simmie.clj\n - htop.clj\n - nudges.clj\n - default_schema.edn\n - tweetex.clj\n - peer.clj\n - opencv.clj\n - output.clj\n\n**Left Panel: (Explorer)**\n- **Folders and Files in Main Directory:**\n - .vscode\n - output-window\n - calva-output\n - default-repl\n - calva-reply\n - .clj-kondo\n - .lsp\n - .shadow-cljs\n - .calva\n - public\n - doc\n - languages\n - src\n - target\n - deps.edn\n - LICENSE\n - README.md\n - server.log\n\n**Specific Folder/File in src:**\n- simmie\n- runtimes\n - abstinence.clj\n - ataxia.clj\n - escapade.clj\n - indolence.clj\n - nudge.clj\n - reactor.clj\n - rustdesk.clj (highlighted)\n - telegram.clj\n - text_extractor.clj\n- Other files: htop.clj, tweetex.clj, peer.clj, opencv.clj\n\n**Middle Panel: (Code Editor)**\n- **File Opened:** simmie.clj\n- **Code Snippets:**\n - Various functions and definitions\n - Code with comments on screenshot processing and encoding\n - Function names such as `process-frames`, `test-screenshots`, and `test-query`\n - Comments and code regarding image processing\n\n**Right Panel: (Output/Terminal)**\n- **Output Panel Content:**\n - Text related to image paths \n - \"new screenshot\" with file paths\n - Commentary and log-like content\n\n**Bottom Bar: (Status Bar)**\n- **Git Branch:** main\n- **Line/Character Position:** Ln 107, Col 1\n- **Spaces:** 2\n- **Edits:** 2843\n- **Errors:** 2\n- **Warnings:** 0\n- **Debug Console and Terminal Options**\n\n**Summary and Inference:**\nThe user is working in Visual Studio Code, managing a project called \"simmie\" accessed via SSH. The focus is on scripting in Clojure, with emphasis on image processing and interaction functionalities. Specific code seen involves manipulating screenshots, potentially for an application that integrates with Telegram. The open files and code snippets suggest the user is debugging or developing features related to screenshot analysis or extraction. The project seems to involve advanced programming concepts and libraries, possibly for open-source development.", "/home/ubuntu/simmie/files/screenshots/454345782/frames_1729024447.jpg" "The image shows a Visual Studio Code interface configured in a dark theme. Below is a hierarchical description of the elements present in the interface:\n\n### Application Window\n- **Title Bar:**\n - Displays `rustdesk-cli - simmie [SSH: telegram] - Visual Studio Code`.\n\n### Editor Layout\n- **Split Screen:**\n - **Left Panel:**\n - **Explorer Sidebar:**\n - Lists directories and files. Some directories such as `src`, `languages`, and `public` are visible. \n - Files like `rustdesk.clj`, `operator.clj`, `telegram.clj` are visible with some denoting edits (`M`), perhaps representing modifications, and others marked with warnings or errors (`9` warnings/errors).\n - **Open File:**\n - Filename: `src -> simm -> runtimes -> rustdesk.clj`.\n - Lines of code are visible with line numbers on the left. The specific code shown is as follows:\n ```plaintext\n 110 \" screen transcription tool. You get the following recent images of the screen of a user. Each image is taken 60 seconds apart. De\"\n 111 (str \"data:image/jpeg;base64,\"\n ```\n - **Right Panel:**\n - **Open File:**\n - Filename: `output.calva-repl`.\n - Contains numerous repetitive text lines such as:\n ```plaintext\n 2436 \"new screenshot\"/home/ubuntu/simmie/files/screenshots/454351782/frames_1722967897.jpg\n ```\n - The list continues with similar formatting, showing the file path and different image identifiers.\n\n### Tabs\n- Multiple open tabs including `simmie.clj`, `output.calva-repl`, and others like `tower.clj`, `default_schema.edn`.\n\n### Status Bar\n- Indicates the following:\n - Branch: `main`\n - File details: `Ln 111, Col 199 Spaces: 2 UTF-8 LF Clojure`\n - Sync, bell, running processes or notifications, git branch status.\n\n### Summary\nThe user is editing Clojure code in Visual Studio Code, likely related to a project dealing with image processing or screen capture automation. Specifically, a tool is processing or managing screenshots on a remote server, possibly for a transcription service or analysis task. The setup and messages suggest the environment is being accessed via SSH on a server named \"telegram.\" The presence of many image paths in the output file implies that the system might be processing or logging multiple images continuously."} \ No newline at end of file