Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dev docs, community section #147

Merged
merged 2 commits into from
Aug 19, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 16 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,39 +143,33 @@ Try different commands, see how the robot responds, and use this experience to u

Follow this guide: [husarion-rosbot-xl-demo](./docs/demos.md)

# Planned demos
## Planned demos

- [agriculture demo 🌾](https://github.com/RobotecAI/rai-agriculture-demo)
- [rosbot xl demo demo 🤖](https://github.com/RobotecAI/rai-rosbot-xl-demo)
- [manipulation demo 🦾](https://github.com/RobotecAI/rai-manipulation-demo)
- Quadruped inspection demo
| Application | Robot | Description | Link |
| ------------------------------------------ | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------- |
| Mission and obstacle reasoning in orchards | Autonomous tractor | In a beautiful scene of a virtual orchard, RAI goes beyond obstacle detection to analyze best course of action for a given unexpected situation. | [🌾 demo](https://github.com/RobotecAI/rai-agriculture-demo) |
| Manipulation tasks with natural language | Robot Arm (Franka Panda) | Complete flexible manipulation tasks thanks to RAI and OpenVLA | [🦾 demo](https://github.com/RobotecAI/rai-manipulation-demo) |
| Quadruped inspection demo | A robot dog (ANYbotics ANYmal) | Perform inspection in a warehouse environment, find and report anomalies | link TBD |

## Manipulation demo
# Community

We are actively working on integrating RAI with Vision-Language-Action (VLA) models to further extend the generalization capabilities of RAI. The upcoming manipulation demo will showcase a practical use case where a single prompt can be used to instruct a robotic arm to perform a series of specific tasks. For more details about our plans and ongoing work related to the integration and fine-tuning of VLA models, please refer to [manipulation.md](docs/manipulation.md).
## RAI Q&A

# Further documentation
Please take a look at [Q&A](https://github.com/RobotecAI/rai/discussions/categories/q-a).

- See some [beta demos](./docs/demos.md)
- See Multimodal Messages definition: [multimodal messages](docs/multimodal_messages.md)
- Browse available ROS 2 packages: [ros packages](docs/ros_-_packages.md)
## Developer Resources

# ROSCon 2024
See our [Developer Guide](docs/developer_guide).
adamdbrw marked this conversation as resolved.
Show resolved Hide resolved

RAI will be released before [ROSCon 2024](https://roscon.ros.org/2024/)!
## Contributing

<p align="center">
<img width="400" src="./docs/imgs/sponsor.png" />
</p>
You are welcome to contribute to RAI! Please see our [Contribution Guide](CONTRIBUTING.md).

## RAI Talk
## RAI release and talk

RAI will be presented as a talk at [ROSCon 2024](https://roscon.ros.org/2024/), make sure to participate!
RAI will be released on **October 15th**, right before [ROSCon 2024](https://roscon.ros.org/2024/).
If you are going to the conference, come join us at RAI talk on October 23rd.

<p align="center">
<img width="400" src="./docs/imgs/talk.png" />
</p>

---

For any inquires about RAI feel free to contact us on [LinkedIn](https://www.linkedin.com/company/robotec-ai) or visit us on [Robotec.ai](https://robotec.ai/)!
10 changes: 10 additions & 0 deletions docs/developer_guide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Developer Guide

Developer Guide is coming soon, here are some resources:

- How to [configure RAI for your robot](create_robots_whami.md).
adamdbrw marked this conversation as resolved.
Show resolved Hide resolved
- [Beta demos](demos.md).
- [Multimodal Messages](multimodal_messages.md) definition.
- Available ROS 2 packages: [ros packages](ros_-_packages.md).
adamdbrw marked this conversation as resolved.
Show resolved Hide resolved
- [Human-Robot Interface](human_robot_interface.md) through voice and text.
- [Manipulation](manipulation.md) with OpenVLA.
4 changes: 2 additions & 2 deletions docs/manipulation.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

## Overview

The upcoming manipulation demo will showcase the currently developed functionalities of RAI enhanced with Vision-Language-Action (VLA) models. You will be able to instruct a robotic arm to perform complex tasks, such as organizing scattered toys, with a single command prompt. It is also meant to showcase how a more generalist agent cooperates with a specialized model.
Vision-Language-Action (VLA) models represent a novel approach to controlling robots, offering significant potential to automate and generalize robotic operations. We are working on integrating this approach with RAI to further extend RAI's versatility.

VLA models represent a novel approach to controlling robots, offering significant potential to automate and generalize robotic operations. We are working on integrating this approach with RAI to further extend RAI's versatility.
The upcoming manipulation demo will showcase a practical use case where a single prompt can be used to instruct a robotic arm to perform a series of specific tasks. It is also meant to showcase how a more generalist agent cooperates with a specialized model.

## Currently planned and developed pipeline

Expand Down