#1400 Helper scripts run custom JAR

[AdrianOlosutean]

Closes #1400

[Zejnilovic]

Generally, I like the solution and it will work. but then this doesn't seem like a plain environment script.

How about introducing SPARK_JOBS_JAR_OVERRIDE or something that is then used in run_standardization.sh like export JAR=${SPARK_JOBS_JAR_OVERRIDE:=$SPARK_JOBS_JAR}

Basically using your solution but in a lower place.

[AdrianOlosutean]

So then it would be modified in more places? How about something like export SPARK_JOBS_JAR=${SPARK_JOBS_JAR:=$SPARK_JOBS_JAR_OVERRIDE} ?

[Zejnilovic]

I don't want evaluation and user overriding in ENV script. Yes, it would be in more places. 2 places vs 1. run_standardization.sh and run_conformance.sh. Or you can try and do it in run_enceladus.sh.

Now that I am looking into run_enceladus.sh I see

    --jar)
    JAR="$2"
    shift 2 # past argument and value
    ;;

What does it do? Does it do the same?

[AdrianOlosutean]

Yes, so I think it's about modifying run_standardization.sh and run_conformance.sh to include this override.

[Zejnilovic]

Did you already find out what the --jar does?

[Zejnilovic]

for it to be an override you would have to do

Suggested change

	export JAR=${SPARK_JOBS_JAR:=$SPARK_JOBS_JAR_OVERRIDE}
	export JAR=${SPARK_JOBS_JAR_OVERRIDE:=$SPARK_JOBS_JAR}

[yruslan]

--jar allows you to override the jar location from the command line. There is also --class, but there is a bug in the script and it doest it wrong. You can fix --class if you want.

[AdrianOlosutean]

How about the user having to specify SPARK_JOBS_JAR_OVERRIDE so it is a bit more explicit?

[yruslan]

Left a comment: #1400 (comment)

[Zejnilovic]

Clicked the button update branch so I can approve

[benedeki]

I would move this to run_enceladus.sh so it's actually present in the code only once.

The separation is due to historical reasons, when Standardization and Conformance were in different jar files.

[Zejnilovic]

This ended up being an easy task ... 😄

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities (and 0 Security Hotspots to review)
0 Code Smells

No Coverage information
No Duplication information