Save Proto Log #3225

Mr-Anyone · 2024-06-11T13:04:13Z

The following implements the following features.

the ability to load and save proto to different files
the ability to load a custom default proto on startup that would be send to unix_full_system
the ability to reset proto

Which essentially resolves #3195

Description

All of the feature is implemented in a widget. You could play around with it. Here is a screenshot:

Testing Done

Save protos
Checked to see if the proto is actually loaded in the unix_full_system
checked to see if the reset button actually worked as intended

Resolved Issues

resolves #3195

Length Justification and Key Files to Review

N/A

Review Checklist

Not done. This is only a draft PR!

Function & Class comments: All function definitions (usually in the .h file) should have a javadoc style comment at the start of them. For examples, see the functions defined in thunderbots/software/geom. Similarly, all classes should have an associated Javadoc comment explaining the purpose of the class.
Remove all commented out code
Remove extra print statements: for example, those just used for testing
Resolve all TODO's: All TODO (or similar) statements should either be completed or associated with a github issue

nimazareian

Had some initial feedback. Don't forget to remove the debugging logs once the PR is ready

src/software/thunderscope/common/proto_configuration_widget.py

itsarune · 2024-06-15T20:15:34Z

src/software/backend/unix_simulator_backend.cpp

+ LOG(DEBUG) << "I've got new: " << request.ai_config().ai_control_config().run_ai();
+ LOG(DEBUG)
+ << "I've got new: "
+ << request.ai_config().dribble_tactic_config().lose_ball_possession_threshold();


remove this later

itsarune · 2024-06-15T20:15:55Z

src/software/thunderscope/common/proto_configuration_widget.py

+ # where we are saving the default configuration file
+ DEFAULT_SAVE_DIRECTORY = "/tmp/tbotspython/thunderbots_configurations_proto"


put this in thunderscope.constants

itsarune · 2024-06-15T20:16:49Z

src/software/thunderscope/common/proto_configuration_widget.py

 :param proto_to_configure: The protobuf we would like to generate
- a parameter tree for. This should be 
+ a parameter tree for. This should be
 populated with the default values


it doesn't seem like you're passing proto_to_configure anymore

indeed. this is loaded by the file instead of passed by the parameter instead.

itsarune · 2024-06-15T20:17:07Z

src/software/thunderscope/common/proto_configuration_widget.py

 def __init__(
- self, proto_to_configure, on_change_callback, search_filter_threshold=60,
+ self, on_change_callback, is_yellow, search_filter_threshold=60,
 ):
 """Create a parameter widget given a protobuf

 NOTE: This class handles the ParameterRangeOptions

 :param proto_to_configure: The protobuf we would like to generate


is_yellow isn't documented

src/software/thunderscope/common/proto_configuration_widget.py

itsarune · 2024-06-15T20:21:08Z

src/software/thunderscope/common/proto_configuration_widget.py

+ """
+
+ # refreshing widgets after the parameters is called
+ logging.info("I am updating widget?")


remove before merging

itsarune · 2024-06-15T20:21:52Z

src/software/thunderscope/common/proto_configuration_widget.py

+ logging.info("Something is changing?")
+ logging.info(changes)
+


delete before merging

itsarune · 2024-06-15T20:22:01Z

src/software/thunderscope/common/proto_configuration_widget.py

@@ -106,6 +299,7 @@ def __handle_parameter_changed(self, param, changes):
 except (TypeError, NameError):
 exec(f"self.proto_to_configure.{child_name} = data")

+ logging.info(f"child name: {child_name} data: {data}")


delete before merging

itsarune · 2024-06-15T20:26:44Z

src/software/thunderscope/common/proto_configuration_widget.py

+ save_button = QPushButton("Save Parameters")
+ load_proto_button = QPushButton("Load Parameters")


ideally save and load would pop up a dialog box

itsarune

awesome work! I'm excited for this to be merged in

Co-authored-by: itsarune <42703774+itsarune@users.noreply.github.com>

…ware into save_proto_log

nimazareian · 2024-07-05T07:31:24Z

@Mr-Anyone If this PR is ready, you should make it "ready for review" so it's not a draft anymore.

nimazareian · 2024-07-05T07:32:10Z

src/software/backend/unix_simulator_backend.cpp

+ LOG(DEBUG) << "I've got new: " << request.ai_config().ai_control_config().run_ai();
+ LOG(DEBUG)
+ << "I've got new: "
+ << request.ai_config().dribble_tactic_config().lose_ball_possession_threshold();


nimazareian · 2024-07-05T07:32:25Z

src/software/thunderscope/common/proto_configuration_widget.py

+ # where we are saving the default configuration file
+ DEFAULT_SAVE_DIRECTORY = "/tmp/tbotspython/thunderbots_configurations_proto"


src/software/thunderscope/common/proto_configuration_widget.py

nimazareian · 2024-07-06T06:26:47Z

src/software/thunderscope/constants.py

+
+class ProtoConfigurationConstant:
+ DEFAULT_SAVE_DIRECTORY = "/tmp/tbotspython/thunderbots_configurations_proto"
+ DEFAULT_SAVE_PATH = DEFAULT_SAVE_DIRECTORY + "/default_conrfiguration.proto"


Suggested change

DEFAULT_SAVE_PATH = DEFAULT_SAVE_DIRECTORY + "/default_conrfiguration.proto"

DEFAULT_SAVE_PATH = DEFAULT_SAVE_DIRECTORY + "/default_configuration.proto"

bump. file name is misspelled

src/software/thunderscope/common/proto_configuration_widget.py

nimazareian

Could you also pull master and run formatting? That should get CI running I think.

itsarune

I noticed the same issue that nima observed

Mr-Anyone · 2024-07-08T06:40:14Z

I believe that the load button is not working. This is because I loaded everything but forgot to send the proto to unix_full_system.

I've fixed.

Build doesn't seem to be working. I am not quite sure if it is related to my changes.

…ware into save_proto_log

nimazareian

Just tried the recent changes and saving, loading, and resetting seems to be working well!

src/software/thunderscope/common/proto_configuration_widget.py

nimazareian · 2024-07-09T17:09:41Z

src/software/thunderscope/constants.py

+
+class ProtoConfigurationConstant:
+ DEFAULT_SAVE_DIRECTORY = "/tmp/tbotspython/thunderbots_configurations_proto"
+ DEFAULT_SAVE_PATH = DEFAULT_SAVE_DIRECTORY + "/default_conrfiguration.proto"


bump. file name is misspelled

nimazareian · 2024-07-09T17:12:37Z

src/software/thunderscope/constants.py

@@ -335,6 +335,25 @@ class TrailValues:
 DEFAULT_TRAIL_LENGTH = 20
 DEFAULT_TRAIL_SAMPLING_RATE = 0

+
 class ProtoConfigurationConstant:
 DEFAULT_SAVE_DIRECTORY = "/tmp/tbotspython/thunderbots_configurations_proto"


Actually I just realized that you're saving to /tmp which is not persisted between computer restart. You should be saving in /opt/tbotspython/thunderbots_configurations_proto instead

that was intentional. since there is permission error if I were to put this in /opt.

I think you would have to chown -R the directory in setup.sh if you were going to do that.

How come we can save the layouts to /opt then?

Software/src/software/thunderscope/thunderscope.py

Lines 157 to 169 in 207a688

with shelve.open(filename, "c") as shelf:

for key, val in self.tab_dock_map.items():

try:

shelf[key] = val.saveState()

except AttributeError:

pass

with shelve.open(LAST_OPENED_LAYOUT_PATH, "c") as shelf:

for key, val in self.tab_dock_map.items():

try:

shelf[key] = val.saveState()

except AttributeError:

pass

If it works on your end, then I've no idea. It doesn't seem to work on my end. Here is the traceback.

===== 2024.07.10 10:07:27 ===== Traceback (most recent call last): File "/home/vhe/.cache/bazel/_bazel_vhe/917dd48f905a75cc113dac63db2f3aa5/execroot/__main__/bazel-out /k8-fastbuild/bin/software/thunderscope/thunderscope_main.runfiles/__main__/software/thunderscope/thun derscope.py", line 140, in save_layout pathlib.Path(SAVED_LAYOUT_PATH).mkdir(exist_ok=True) File "/usr/lib/python3.8/pathlib.py", line 1288, in mkdir self._accessor.mkdir(self, mode) PermissionError: [Errno 13] Permission denied: '/opt/tbotspython/saved_tscope_layout' ===== 2024.07.10 10:07:27 ===== Traceback (most recent call last): File "/home/vhe/.cache/bazel/_bazel_vhe/917dd48f905a75cc113dac63db2f3aa5/execroot/__main__/bazel-out /k8-fastbuild/bin/software/thunderscope/thunderscope_main.runfiles/__main__/software/thunderscope/thun derscope.py", line 140, in save_layout pathlib.Path(SAVED_LAYOUT_PATH).mkdir(exist_ok=True) File "/usr/lib/python3.8/pathlib.py", line 1288, in mkdir self._accessor.mkdir(self, mode) PermissionError: [Errno 13] Permission denied: '/opt/tbotspython/saved_tscope_layout'

I have pressed ctrl+s.

Could you show me your ls -l in the /opt directory.

For me, it shows the following:

drwxr-xr-x 7 vhe vhe 4096 Feb 1 1980 gradle-8.8 drwxr-xr-x 10 vhe vhe 4096 Jun 18 18:38 idea drwxr-xr-x 10 vhe vhe 4096 Jun 26 22:23 meld-3.22.2 drwxr-xr-x 3 root root 4096 Oct 7 2023 microsoft drwxr-xr-x 2 vhe vhe 4096 Jul 2 18:04 shadowsocks drwxr-xr-x 5 root root 4096 Jan 28 05:26 tbotspython drwxrwxr-x 9 vhe vhe 4096 Jun 19 21:32 tomcat drwxr-xr-x 6 root root 4096 Jun 15 17:11 wine-stable

the folder need root permission I think as a result. I think for you, it may be nima instead of root for the tbotspython directory. Do you still want me to change it?

Mine seems to be owned by my user and not root.

~/thunderbots/Software/src (nima/fullsystem_proto_logger*) » ls -l /opt nima@nima-MS-7C87 total 36 drwxr-xr-x 4 root root 4096 Aug 16 2022 cisco drwxr-xr-x 8 root root 4096 Apr 30 2021 clion-2019.3.5 drwxr-xr-x 8 root root 4096 Jan 8 2022 clion-2021.2.3 drwxr-xr-x 7 root root 4096 Nov 26 2023 docker-desktop drwxr-xr-x 3 root root 4096 Apr 30 2021 google drwxr-xr-x 2 root root 4096 Jul 16 2021 jetbrains-toolbox-1.21.9547 drwxr-xr-x 3 root root 4096 Dec 9 2023 microsoft drwxrwxrwx 10 root root 4096 Apr 30 2021 STM32CubeMX_6.2.0 drwxr-xr-x 8 nima nima 4096 May 22 20:42 tbotspython

I just tried it and I can save layouts (ctrl + S in thunderscope), but relaunching Thunderscope with it doesn't show anything from fullsystem anymore, though the layout is automatically loaded.

I guess if needed, we could update setup_software.sh as well.

I looked into it and we already do chown -R in setup_software.sh. For what ever reason that did not work for you, or maybe when you were testing with upgrading to Python 3.10 you manually recreated the /opt/tbotspython python virtual env directory.

Software/environment_setup/setup_software.sh

Line 138 in 00372df

sudo chown -R $USER:$USER /opt/tbotspython

I think so. That could really be the case. Anyhow, I think I would just manually chown my tbotspython directory for now.

Okay. Could you update DEFAULT_SAVE_DIRECTORY to use /opt then?

yeah, I will push an update now.

Co-authored-by: Nima Zareian <nimazareian80@gmail.com>

itsarune

Awesome work vincent!

src/software/thunderscope/constants.py

Co-authored-by: itsarune <42703774+itsarune@users.noreply.github.com>

Mr-Anyone added 5 commits May 31, 2024 11:57

added widgets and features to create saving feature

265aede

added comments and other stuff

abfc5a7

fixed reset button not sending protobufs

e7c9e1a

some more updates

5bd7a34

added new features

fc37ed0

Mr-Anyone requested review from nimazareian and itsarune as code owners June 11, 2024 13:04

Mr-Anyone marked this pull request as draft June 11, 2024 13:04

[pre-commit.ci lite] apply automatic fixes

3df4233

nimazareian reviewed Jun 14, 2024

View reviewed changes

itsarune requested review from sauravbanna, a team, Bvasilchikov and jameschuang2002 and removed request for a team June 15, 2024 20:15

itsarune reviewed Jun 15, 2024

View reviewed changes

src/software/thunderscope/common/proto_configuration_widget.py Show resolved Hide resolved

itsarune reviewed Jun 15, 2024

View reviewed changes

src/software/thunderscope/common/proto_configuration_widget.py Outdated Show resolved Hide resolved

itsarune reviewed Jun 15, 2024

View reviewed changes

Mr-Anyone and others added 3 commits July 2, 2024 17:43

Update src/software/thunderscope/common/proto_configuration_widget.py

62adb15

Co-authored-by: itsarune <42703774+itsarune@users.noreply.github.com>

address some concerns

8a429f7

Merge branch 'save_proto_log' of github.com:Mr-Anyone/Thunderbot_Soft…

ddf1aa1

…ware into save_proto_log

Mr-Anyone requested a review from nimazareian July 2, 2024 09:48

nimazareian mentioned this pull request Jul 3, 2024

Add feature to bind to specific network interfaces via Thunderscope #3235

Open

4 tasks

nimazareian reviewed Jul 5, 2024

View reviewed changes

made some changes

34e55ad

Mr-Anyone marked this pull request as ready for review July 5, 2024 10:44

made some actual changes

b203cba

Mr-Anyone requested a review from nimazareian July 6, 2024 04:22

nimazareian reviewed Jul 6, 2024

View reviewed changes

Mr-Anyone and others added 3 commits July 7, 2024 14:27

merged upstream

5139d87

added type hinting

5967cf6

[pre-commit.ci lite] apply automatic fixes

c23f534

itsarune reviewed Jul 7, 2024

View reviewed changes

fixed load button

e2a38ea

Mr-Anyone and others added 2 commits July 8, 2024 14:41

Merge branch 'save_proto_log' of github.com:Mr-Anyone/Thunderbot_Soft…

6684bbc

…ware into save_proto_log

[pre-commit.ci lite] apply automatic fixes

7f57395

nimazareian reviewed Jul 9, 2024

View reviewed changes

nimazareian added the Robocup-2024 High priority for Rbocup 2024 label Jul 9, 2024

Mr-Anyone and others added 2 commits July 10, 2024 10:19

Update src/software/thunderscope/common/proto_configuration_widget.py

df08db9

Co-authored-by: Nima Zareian <nimazareian80@gmail.com>

change filenam

b99cf83

itsarune previously approved these changes Jul 10, 2024

View reviewed changes

itsarune reviewed Jul 10, 2024

View reviewed changes

src/software/thunderscope/constants.py Outdated Show resolved Hide resolved

Update src/software/thunderscope/constants.py

709c06e

Co-authored-by: itsarune <42703774+itsarune@users.noreply.github.com>

Mr-Anyone dismissed itsarune’s stale review via 709c06e July 10, 2024 06:40

Mr-Anyone added 2 commits July 10, 2024 15:03

update constant.py

cc4c820

moved from /tmp to /opt

f6bbfa4

nimazareian approved these changes Jul 10, 2024

View reviewed changes

williamckha approved these changes Jul 10, 2024

View reviewed changes

Mr-Anyone merged commit dea10f8 into UBC-Thunderbots:master Jul 10, 2024
6 checks passed

		# where we are saving the default configuration file
		DEFAULT_SAVE_DIRECTORY = "/tmp/tbotspython/thunderbots_configurations_proto"

		save_button = QPushButton("Save Parameters")
		load_proto_button = QPushButton("Load Parameters")

	DEFAULT_SAVE_PATH = DEFAULT_SAVE_DIRECTORY + "/default_conrfiguration.proto"
	DEFAULT_SAVE_PATH = DEFAULT_SAVE_DIRECTORY + "/default_configuration.proto"

	with shelve.open(filename, "c") as shelf:
	for key, val in self.tab_dock_map.items():
	try:
	shelf[key] = val.saveState()
	except AttributeError:
	pass

	with shelve.open(LAST_OPENED_LAYOUT_PATH, "c") as shelf:
	for key, val in self.tab_dock_map.items():
	try:
	shelf[key] = val.saveState()
	except AttributeError:
	pass

Save Proto Log #3225

Save Proto Log #3225

Conversation

Mr-Anyone commented Jun 11, 2024

Description

Testing Done

Resolved Issues

Length Justification and Key Files to Review

Review Checklist

nimazareian left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

itsarune left a comment

Choose a reason for hiding this comment

nimazareian commented Jul 5, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nimazareian left a comment

Choose a reason for hiding this comment

itsarune left a comment

Choose a reason for hiding this comment

Mr-Anyone commented Jul 8, 2024 • edited Loading

nimazareian left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mr-Anyone Jul 10, 2024 • edited Loading

Choose a reason for hiding this comment

Mr-Anyone Jul 10, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nimazareian Jul 10, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

itsarune left a comment

Choose a reason for hiding this comment

Mr-Anyone commented Jul 8, 2024 •

edited

Loading

Mr-Anyone Jul 10, 2024 •

edited

Loading

Mr-Anyone Jul 10, 2024 •

edited

Loading

nimazareian Jul 10, 2024 •

edited

Loading