-
Notifications
You must be signed in to change notification settings - Fork 190
vrx_2022 phase3_challenge
The process and content of the Phase 3 will be very similar to the Phase 2: Dress Rehearsal.
Please review our Phase 2 Retrospective and Lessons Learned and the Phase 2 Results for notes and feedback from the Phase 2 Dress Rehearsal. (These will be posted after Phase 2 has been completed).
For the purposes of continued development, teams can expect that the submission process will be the same as Phase 2. We will only introduce changes to incorporate lessons learned from Phase 2.
The VRX Tasks will be be the same as provided in the VRX task description document (to be released in September 2021). Just as in Phase 2, team performance for these tasks will be assessed by running the team software on multiple trials of each task, where each trial instantiates a specific task scenario consistent with the general task description.
To ensure that your system can adapt to previously-unseen scenarios, teams should test their system at least against all released sample tasks.
To help you prepare, we will provide updated testing examples for VRX 2022. These will be released in September 2021. In the meantime, you may find it helpful to look at Phase 2 Task Testing from VRX 2019, which includes example worlds for some of the tasks previously used in VRX 2019. These example worlds are representative of the task configurations and environmental conditions specified in the task documentation.
For the evaluation of your Phase 3 submissions, the solutions will be run through new configurations that are not released prior to the Phase 3 deadline, to exercise the solutions ability to deal with new configurations and conditions.
Some features of docker require extra command line arguments. If your team would like to specify an extra command line argument when executing your submitted docker image, please let the VRX Technical Team know as soon as possible by creating a new issue in the issue tracker. We will deal with these requests on a case by case basis. We will make every effort to be as flexible as we can with the team solutions while providing a fair competition for all.
We do not expect many teams to want or need the flexibility to add additional command line arguments, but in cases were it can allow teams to innovate without giving them a competitive advantage, we want to work to make that happen.
We expect to receive three files from each competitor prior to this event:
-
dockerhub_image.txt: Contains the name of the image to be pulled from DockerHub. -
thruster_config.yaml: Defines the WAM-V thruster configuration. -
sensor_config.yaml: Defines the WAM-V sensor configuration.
We highly recommend that each team test their solution and the docker image by following the Testing Your Submission wiki. This will allow teams to reproduce the evaluation and assessment system prior to submission.
- If you're using a private DockerHub repository, please grant access to the
virtualrobotxDockerHub user, which will be used by the organization to access your solution. - Follow this tutorial for instructions about how to create your own Docker image.
- Check out this tutorial to learn how to verify your submission.
- See the submission process section in the wiki for instructions about how to submit these files.
Once you submit the pull request, the VRX technical team will do two things before merging (accepting) the submission:
- Check that the WAM-V thruster and sensor configuration complies with the configuration constraints described in the VRX Technical Guide
- Check that the DockerHub image is accessible by the virtualrobotx DockerHub user.
Once those two requirements are met, the pull request will be merged and your submission is considered ready.
| Date | Description |
|---|---|
| 2022, April, date/time TBA | Deadline for submitting your solution |
| 2022, April, date/time TBA | Deadline for submitting a correction |
| 2022, April, date/time TBA | Results published |
Note: We'll allocate time in the schedule (date TBA) for fixing any problems related with the content of the submission files (e.g.: A noncompliant sensor or thruster configuration, a typo pointing to the DockerHub image, a problem with DockerHub permissions not letting Open Robotics to download your solution image). Teams are not allowed to modify the DockerHub image after the deadline for submitting solutions.
As a rule, late submissions will not be accepted. All teams must submit the required elements by the solution deadline. The technical team does reserve the right to accept late submissions if the submission was delayed due to unforeseen problems with the submission process. Acceptance of any submissions after the solution deadline will be at the discretion of the VRX Technical Team.