-
Notifications
You must be signed in to change notification settings - Fork 594
Use argparse + make chmod not fail in all cases #1493
Conversation
This makes me so happy, I was just thinking about this broken window! :) I'll review soon. Would you please accept the CLA: |
…with unit test mocking. Fixed bug with jvm opts too
for binary in chmod_x_binaries: | ||
stat_result = os.stat(binary)[stat.ST_MODE] | ||
if not stat_result & stat.S_IXOTH: | ||
binaries_to_chmod.append(binary) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
stat seems like a good approach, provided it works on all distros.
There's no need to do all chmods in one concatenated command, so better to break it up to make the code more readable. No need to build up arrays and concated strings, we can just do this:
- chmod log dir
- for each binary
- if chmod needed
- do chmod
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Works for me, I'll make that change. Stat appears to work on everything major, but I'll do some additional digging and testing on some VMs to be sure.
self.tmaster_binary = args[7] | ||
self.stmgr_binary = args[8] | ||
self.metricsmgr_classpath = args[9] | ||
self.shard = int(parsed_args.shard) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's make this an int type in argparse and remove the cast
self.component_rammap =\ | ||
map(lambda x: {x.split(':')[0]: int(x.split(':')[1])}, args[16].split(',')) | ||
map(lambda x: {x.split(':')[0]: | ||
int(x.split(':')[1])}, parsed_args.component_rammap.split(',')) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can args.component_rammap do this and just return a map?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand this comment. Can you explain?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was wondering if this logic could be encapsulated into the argparse logic via custom actions. I'm not certain it could, and maybe it's not worth the effort, so feel free to disregard.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree. If @dmarchand thinks it's worth the effort, feel free to refer to https://docs.python.org/dev/library/argparse.html#action
thanks @dmarchand - this is a great PR, we have been wanting to do this. Great to see you picked this up. |
parser.add_argument("scheduler_port") | ||
parser.add_argument("python_instance_binary") | ||
|
||
return parser.parse_args(args[1:]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since you removed this part, can you use parse_known_args
as in heron/tools/explorer/src/python/main.py#L172, and check if there is any unknown args?
|
||
def run_command_or_exit(self, command): | ||
if self._run_blocking_process(command, True, self.shell_env) != 0: | ||
Log.info("Failed to run command: %s. Exiting" % command) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Log.error
|
||
if unknown_args: | ||
Log.error('Unknown argument: %s' % unknown_args[0]) | ||
parser.print_help() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since we didn't write any help message for any argument, it doesn't really matter if we'll print help message here...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In that case argparse is supposed to dump the list of required arguments: https://docs.python.org/3/library/argparse.html#usage
It's probably not the best possible help message and we should likely take some time at some point to document the individual arguments, but this will at least clue the developer in as to which argument they added doesn't belong.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for pointing out. Agreed.
👍 |
1 similar comment
👍 |
This PR has two components to it: