Skip to content

FAQ

Jump to bottom
Rob Royce edited this page Aug 13, 2024 · 8 revisions

Table of Contents

General ROSA Questions

Question: What are the benefits of using AI Agents like ROSA?
Answer: AI agents like ROSA can help humans interact with robots more effectively by providing natural language understanding and generation capabilities. This can improve the user experience and make it easier for non-experts to interact with robots. Additionally, ROSA can be used to automate many of the tedious tasks and procedures needed to effectively develop, diagnose, debug, and troubleshoot complex robotic systems.

Question: What’s the major advantage of using a tool like this over just copy-pasting some files into or asking such questions to, say, ChatGPT? Is it just easier since it can talk to the running ROS system directly or do the custom plugins provide something you couldn’t achieve ‘manually’?

Answer: ROSA provides several advantages over traditional chat services like ChatGPT.

First, the agent can select from a diverse set of tools to address a query. It can utilize multiple tools simultaneously and even execute them in parallel. The responses are then synthesized into a final result, or the agent can continue employing additional tools if the query hasn’t been fully resolved. This is enabled by its “reasoning - action - observation” loop.

You can also integrate custom tools for tasks beyond information retrieval. For instance, we’ve developed a custom ROSA-Spot agent that can sit, stand, and even walk around. We’ve essentially equipped ROSA with the same control capabilities you’d have using an Xbox controller, with safety measures in place to mitigate any risks of unwanted behavior. You can instruct it to “stand up, walk forward and to the left about 3 meters, then sit down,” and it will execute the command. (short demo video attached to this post)

Question: How can I get started with ROSA?
Answer: To get started with ROSA, see the Developer Documentation for instructions on installing ROSA and creating @tool functions. You can also refer to the Custom Agents guide for information on adapting ROSA for your robot.

Question: Will robot-specific ROSA agents become available in the future?
Answer: Yes! We are working on creating robot-specific ROSA agents that are tailored to the capabilities and constraints of specific robots. In particular, we are focusing on agents for the robots listed below. Stay tuned for updates on these agents and their availability to the public.

Question: What are ROSA's limitations?
Answer: ROSA has various limitations that can be grouped into the following categories:

  • Model Limitations:
    • ROSA may not work with all LLM models. See the Supported Models page for more information.
    • The chosen model may not understand certain ROS commands or may provide incorrect responses.
    • Each model has a context window limited to a certain number of tokens. If the context window is exceeded, the model may not provide accurate responses or may fail to respond at all. See the Supported Models page for more information.
  • Physical Capabilities:
    • The core ROSA agent is limited to text-based interactions and cannot interact with the physical world. To enable physical interactions, you will need to create custom tools and prompts.
  • Tool Limitations:
    • The tools available to ROSA may not cover all possible ROS commands.
    • The tools may not be able to handle all possible variations of ROS commands.
    • Some tools might require additional parameters or context to function correctly.

Question: How can I adapt ROSA for my ROS-based robot?
Answer: See the Custom Agents and Developer Documentation guides for information on adapting ROSA for your robot.

AI Model Questions

Question: Which LLM models are supported for use with ROSA?
Answer: ROSA currently works with OpenAI and AzureOpenAI chat models.

Question: Which model do you recommend using?
Answer: We recommend using GPT-4o due to its balance of low-cost and strong reasoning abilities.

Safety and Guardrails

Question: How can I ensure that ROSA does not perform unsafe actions?
Answer: You can add safety checks to your tools to ensure that ROSA does not perform unsafe actions. For example, you can check the validity of the parameters passed to the tool or add constraints to limit the actions that ROSA can take. See the Custom Agents guide for more information on adding safety checks to your tools.

Question: How can I prevent ROSA from using certain topics, nodes, services, logs, etc. that match a certain pattern?
Answer: ROSA implements a global retrieval filter in the form of a blacklist. This blacklist is automatically passed into all tools registered for use with ROSA and is used by those tools to omit anything that matches the blacklist. Simply pass in a list of strings to the blacklist parameter when creating a new instance of the ROSA class.

Question: How can I ensure that ROSA does not perform actions that are outside the robot's capabilities?
Answer: You can add constraints to your tools to ensure that ROSA does not perform actions that are outside the robot's capabilities. For example, you can check the robot's battery level or the availability of certain hardware components before executing a tool. See the Custom Agents and Developer Documentation guides for more information on adding constraints to your tools.

ROSA: Robot Operating System Agent

Copyright (c) 2024. Jet Propulsion Laboratory. All rights reserved.

Clone this wiki locally