Initialize params via yaml from command line #488

sloretz · 2018-06-01T23:00:16Z

Blocks #477
Requires #485
Requires #486
Requires ros2/rcl#253

This initializes the parameters on a node from yaml files. Since there are multiple sources of initial values passed to a node (multiple yaml files on the command line, multiple yaml files on node specific CLI arugments, explicit initial_values passed to constructor) this PR merges all the sources such that constructor initial_values overwrites constructor arguments which overwrites global command line arguments. Within CLI arguments Yaml files specified later overwrite ones specified earlier.

i.e. this works

ros2 run demo_nodes_cpp talker __params:=generic-default-values.yaml __params:=more-specific-values.yaml

~~The only commit is 2cb890e. This PR needs to be rebased when #485 and #486 are merged. Also this PR needs tests.~~

sloretz · 2018-06-06T01:02:36Z

Keeping In progress label while CI runs

Linux
Linux-aarch64
macOS
Windows

sloretz · 2018-06-06T01:39:48Z

Optimistically moving to in review

wjwwood

Looks fine in general, but there are a few things that need to be addressed, like memory leaking.

wjwwood · 2018-06-06T08:34:39Z

rclcpp/src/rclcpp/node_interfaces/node_parameters.cpp

+    throw std::runtime_error("Need valid node handle in NodeParameters");
+  }
+  const rcl_node_options_t * options = rcl_node_get_options(node);
+  if (NULL == options) {


I'd recommend using nullptr in C++.

Oops bfb497c

wjwwood · 2018-06-06T08:36:04Z

rclcpp/src/rclcpp/node_interfaces/node_parameters.cpp

+  auto get_yaml_paths = [&yaml_paths, &options](const rcl_arguments_t * args) {
+      int num_yaml_files = rcl_arguments_get_param_files_count(args);
+      if (num_yaml_files > 0) {
+        char ** param_files;


This is never deallocated?

Fixed in 934fa77

wjwwood · 2018-06-06T08:39:46Z

rclcpp/src/rclcpp/node_interfaces/node_parameters.cpp

+    combined_name = node_namespace + node_name;
+  } else {
+    combined_name = node_namespace + '/' + node_name;
+  }


Should probably be a function for this in rcl, I wouldn't do it now, but an issue would be nice.

Ticketed in ros2/rcl#255

wjwwood · 2018-06-06T08:40:22Z

rclcpp/src/rclcpp/node_interfaces/node_parameters.cpp

+
+  std::map<std::string, rclcpp::Parameter> parameters;
+
+  // TODO(sloretz) rcl too parse yaml when circular dependency is solved


"use rcl to parse..."?

Grammar fix in 25716b1

wjwwood · 2018-06-06T08:46:55Z

rclcpp/src/rclcpp/node_interfaces/node_parameters.cpp

+      throw std::runtime_error("Failed to parse parameters " + yaml_path);
+    }
+
+    rclcpp::ParameterMap initial_map = rclcpp::parameter_map_from(yaml_params);


Does this fini the yaml_params? If not then it needs to be, if so then... eew lol

It does not. Added call to ...fini() in d765738

wjwwood · 2018-06-06T08:50:36Z

rclcpp/src/rclcpp/node_interfaces/node_parameters.cpp

+  for (const std::string & yaml_path : yaml_paths) {
+    rcl_params_t * yaml_params = rcl_yaml_node_struct_init(options->allocator);
+    if (NULL == yaml_params) {
+      throw std::runtime_error("Failed to initialize yaml params struct");


Should be able to narrow this down to a better exception, like std::bad_alloc. Looking at them now, the yaml parsing API in yaml leaves something to be desired in terms of error reporting and namespacing.

Changed this exception to bad_alloc in ef427a7

sloretz · 2018-06-06T17:03:33Z

CI as of ef427a7

Linux
Linux-aarch64
macOS
Windows

wjwwood

lgtm, with one suggestion on how to make it even better.

wjwwood · 2018-06-06T21:10:21Z

rclcpp/src/rclcpp/node_interfaces/node_parameters.cpp

+          rclcpp::exceptions::throw_from_rcl_error(ret);
+        }
+        for (int i = 0; i < num_yaml_files; ++i) {
+          yaml_paths.emplace_back(param_files[i]);


What happens if this throws? Maybe a "scope exit" kind of thing to clean up the whole param_files all at once would be better.

cleanup on scope exit in b9a7e74

wjwwood · 2018-06-06T21:10:31Z

rclcpp/src/rclcpp/node_interfaces/node_parameters.cpp

+    combined_name = node_namespace + node_name;
+  } else {
+    combined_name = node_namespace + '/' + node_name;
+  }


sloretz · 2018-06-07T22:52:05Z

Full CI

Linux
Linux-aarch64
macOS
Windows

sloretz · 2018-06-08T21:58:00Z

It looks like ros2/rmw_fastrtps#205 is the cause of the test failure on the OSX build. I'll re-write the test to avoid destroying clients as a workaround.

sloretz · 2018-06-08T22:12:26Z

CI with workaround to not destroy client while spinning

Linux
Linux-aarch64
macOS
Windows

dhood · 2018-09-14T00:00:00Z

rclcpp/src/rclcpp/node_interfaces/node_parameters.cpp

+
+  std::map<std::string, rclcpp::Parameter> parameters;
+
+  // TODO(sloretz) use rcl to parse yaml when circular dependency is solved


@sloretz do you happen to remember the context around this todo? I came across it today and was confused since it seems like yaml parsing is done in rcl already, so it must be referring to something else.

since it seems like yaml parsing is done in rcl already

This is actually done in rcl_yaml_param_parser and not rcl.
That package is currently depending on rcl to be able to use rcl types, so rcl cannot depenc on rcl_yaml_param_parser to avoid circular dependency.
ros2/rcl#252

oh I see! Thanks, I thought from the rcl_ prefix that they were in rcl, but of course that's not the case 😄 I'll reference that ticket in the todo.

Signed-off-by: Erik Boasson <eb@ilities.com>

…ros2#488) * Remove exception throw in dtor Update logging errors to avoid duplication Signed-off-by: Devin Bonnie <dbbonnie@amazon.com> * Cleanup reset logic Add reset documentation Signed-off-by: Devin Bonnie <dbbonnie@amazon.com> * Fix cpplint error Signed-off-by: Devin Bonnie <dbbonnie@amazon.com> * Fix docstring Signed-off-by: Devin Bonnie <dbbonnie@amazon.com>

sloretz added the in progress Actively being worked on (Kanban column) label Jun 1, 2018

sloretz added this to the bouncy milestone Jun 1, 2018

sloretz self-assigned this Jun 1, 2018

sloretz changed the title ~~[WIP] Init params via yaml~~ Initialize params via yaml from command line Jun 1, 2018

sloretz mentioned this pull request Jun 5, 2018

Test initializing parameters from command line ros2/system_tests#274

Merged

mikaelarguedas mentioned this pull request Jun 5, 2018

Support passing yaml parameter files via commandline ros2/rcl#253

Merged

sloretz mentioned this pull request Jun 5, 2018

[WIP] Initialize parameters #477

Closed

10 tasks

sloretz added 2 commits June 5, 2018 15:48

Initialize params from yaml files

945a70e

Style fixes

f067f5f

sloretz force-pushed the init_params_via_yaml branch from 2cb890e to f067f5f Compare June 5, 2018 23:02

sloretz added in review Waiting for review (Kanban column) and removed in progress Actively being worked on (Kanban column) labels Jun 6, 2018

typo

818b7ad

wjwwood requested changes Jun 6, 2018

View reviewed changes

sloretz added 5 commits June 6, 2018 09:02

NULL -> nullptr

bfb497c

Deallocate param_files

934fa77

rcl too -> use rcl to

25716b1

Finalize c yaml struct

d765738

runtime_error -> bad_alloc

ef427a7

wjwwood approved these changes Jun 6, 2018

View reviewed changes

mikaelarguedas added in progress Actively being worked on (Kanban column) and removed in review Waiting for review (Kanban column) labels Jun 7, 2018

Exception safe cleanup

b9a7e74

sloretz added in review Waiting for review (Kanban column) and removed in progress Actively being worked on (Kanban column) labels Jun 7, 2018

sloretz mentioned this pull request Jun 8, 2018

Nullify pointer to ClientListener in CustomClientInfo in destructor ros2/rmw_fastrtps#205

Closed

sloretz mentioned this pull request Jun 8, 2018

Read only parameters #495

Merged

wjwwood approved these changes Jun 9, 2018

View reviewed changes

sloretz merged commit 4886b24 into master Jun 9, 2018

sloretz deleted the init_params_via_yaml branch June 9, 2018 00:24

dhood reviewed Sep 14, 2018

View reviewed changes

dhood mentioned this pull request Sep 14, 2018

[YAML parser] Be able to use the parser in rcl without circular dependency ros2/rcl#252

Closed

nnmm pushed a commit to ApexAI/rclcpp that referenced this pull request Jul 9, 2022

Allow get_node_names to return result in any order (ros2#488)

2431ed6

Signed-off-by: Erik Boasson <eb@ilities.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initialize params via yaml from command line #488

Initialize params via yaml from command line #488

sloretz commented Jun 1, 2018 •

edited

Loading

sloretz commented Jun 6, 2018

sloretz commented Jun 6, 2018

wjwwood left a comment

wjwwood Jun 6, 2018

sloretz Jun 6, 2018

wjwwood Jun 6, 2018

sloretz Jun 6, 2018

wjwwood Jun 6, 2018

sloretz Jun 6, 2018

wjwwood Jun 6, 2018

wjwwood Jun 6, 2018

sloretz Jun 6, 2018

wjwwood Jun 6, 2018

sloretz Jun 6, 2018

wjwwood Jun 6, 2018

sloretz Jun 6, 2018

sloretz commented Jun 6, 2018

wjwwood left a comment

wjwwood Jun 6, 2018

sloretz Jun 7, 2018

wjwwood Jun 6, 2018

sloretz commented Jun 7, 2018

sloretz commented Jun 8, 2018

sloretz commented Jun 8, 2018

dhood Sep 14, 2018

mikaelarguedas Sep 14, 2018

dhood Sep 14, 2018


		std::map<std::string, rclcpp::Parameter> parameters;

		// TODO(sloretz) rcl too parse yaml when circular dependency is solved


		std::map<std::string, rclcpp::Parameter> parameters;

		// TODO(sloretz) use rcl to parse yaml when circular dependency is solved

Initialize params via yaml from command line #488

Initialize params via yaml from command line #488

Conversation

sloretz commented Jun 1, 2018 • edited Loading

sloretz commented Jun 6, 2018

sloretz commented Jun 6, 2018

wjwwood left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sloretz commented Jun 6, 2018

wjwwood left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sloretz commented Jun 7, 2018

sloretz commented Jun 8, 2018

sloretz commented Jun 8, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sloretz commented Jun 1, 2018 •

edited

Loading