Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 3 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,15 +52,13 @@ Open `bazel-diff-example.sh` to see how this is implemented. This is purely an e

* Next we checkout the initial revision, then we run `generate-hashes` with the file output of `modified-filepaths` and write that JSON to a file. Now we have our final hashmap representation for the Bazel graph.

* We run `bazel-diff` on the starting and final JSON hash filepaths to get our affected set of targets. This impacted set of targets is written to a file

* Finally we run `impacted-tests` with the filepath to the list of impacted targets. This returns us the impacted test targets.
* We run `bazel-diff` on the starting and final JSON hash filepaths to get our impacted set of targets. This impacted set of targets is written to a file. You can also pass the `-t` flag to only return tests

## CLI Interface

`bazel-diff` Command
~~~
Usage: bazel-diff [-hV] -b=<bazelPath> [-fh=<finalHashesJSONPath>]
Usage: bazel-diff [-htV] -b=<bazelPath> [-fh=<finalHashesJSONPath>]
[-o=<outputPath>] [-sh=<startingHashesJSONPath>]
-w=<workspacePath> [COMMAND]
Writes to a file the impacted targets between two Bazel graph JSON files
Expand All @@ -76,28 +74,12 @@ Writes to a file the impacted targets between two Bazel graph JSON files
-sh, --startingHashes=<startingHashesJSONPath>
The path to the JSON file of target hashes for the initial
revision. Run 'generate-hashes' to get this value.
-t, --tests Return only targets of kind 'test')
-V, --version Print version information and exit.
-w, --workspacePath=<workspacePath>
Path to Bazel workspace directory.
~~~

`impacted-tests` Command
~~~
Usage: bazel-diff impacted-tests [-hV] -b=<bazelPath> -w=<workspacePath>
<impactedBazelTargetsPath> <outputPath>
Write to a file the impacted test targets for the list of Bazel targets in the
provided file
<impactedBazelTargetsPath>
The filepath to a newline separated list of Bazel targets
<outputPath> The filepath to write the impacted test targets to
-b, --bazelPath=<bazelPath>
Path to Bazel binary
-h, --help Show this help message and exit.
-V, --version Print version information and exit.
-w, --workspacePath=<workspacePath>
Path to Bazel workspace directory.
~~~

`modified-filepaths` Command
~~~
Usage: bazel-diff modified-filepaths [-hV] -b=<bazelPath> -w=<workspacePath>
Expand Down
6 changes: 3 additions & 3 deletions bazel-diff-example.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ final_revision=$4

modified_filepaths_output="/tmp/modified_filepaths.txt"
starting_hashes_json="/tmp/starting_hashes.json"
final_hashes_json="/tmp/final_hashes_json.json"
final_hashes_json="/tmp/final_hashes.json"
impacted_targets_path="/tmp/impacted_targets.txt"
impacted_test_targets_path="/tmp/impacted_test_targets.txt"

Expand All @@ -28,7 +28,7 @@ git -C $workspace_path checkout $previous_revision --quiet
echo "Generating Hashes for Revision '$previous_revision'"
$bazel_path run :bazel-diff -- generate-hashes -w $workspace_path -b $bazel_path $starting_hashes_json

git -C $workspace_path checkout - --quiet
git -C $workspace_path checkout - --quiet

echo "Generating Hashes for Revision '$final_revision'"
$bazel_path run :bazel-diff -- generate-hashes -w $workspace_path -b $bazel_path -m $modified_filepaths_output $final_hashes_json
Expand All @@ -37,7 +37,7 @@ echo "Determining Impacted Targets"
$bazel_path run :bazel-diff -- -sh $starting_hashes_json -fh $final_hashes_json -w $workspace_path -b $bazel_path -o $impacted_targets_path

echo "Determining Impacted Test Targets"
$bazel_path run :bazel-diff -- impacted-tests -w $workspace_path -b $bazel_path $impacted_targets_path $impacted_test_targets_path
$bazel_path run :bazel-diff -- -sh $starting_hashes_json -fh $final_hashes_json -w $workspace_path -b $bazel_path -o $impacted_test_targets_path -t

IFS=$'\n' read -d '' -r -a impacted_targets < $impacted_targets_path
formatted_impacted_targets=$(IFS=$'\n'; echo "${impacted_targets[*]}")
Expand Down
15 changes: 9 additions & 6 deletions integration/integration_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -28,16 +28,19 @@ ruby ./integration/update_final_hashes.rb

$bazel_path run :bazel-diff -- -sh $starting_hashes_json -fh $final_hashes_json -w $workspace_path -b $bazel_path -o $impacted_targets_path

$bazel_path run :bazel-diff -- impacted-tests -w $workspace_path -b $bazel_path $impacted_targets_path $impacted_test_targets_path
$bazel_path run :bazel-diff -- -sh $starting_hashes_json -fh $final_hashes_json -w $workspace_path -b $bazel_path -o $impacted_test_targets_path -t

IFS=$'\n' read -d '' -r -a impacted_targets < $impacted_targets_path
target="//src/main/java/com/integration:StringGenerator.java"
if containsElement $target "${impacted_targets[@]}";
target1="//test/java/com/integration:bazel-diff-integration-test-lib"
target2="//src/main/java/com/integration:bazel-diff-integration-lib"
target3="//test/java/com/integration:bazel-diff-integration-tests"
if containsElement $target1 "${impacted_targets[@]}" && \
containsElement $target2 "${impacted_targets[@]}" && \
containsElement $target3 "${impacted_targets[@]}"
then
echo "Correct first impacted target"
echo "Correct impacted targets"
else
echo "Impacted Targets: ${impacted_targets[@]}"
echo "Incorrect first impacted target: ${target}"
echo "Incorrect impacted targets: ${impacted_targets[@]}"
exit 1
fi

Expand Down
22 changes: 19 additions & 3 deletions src/main/java/com/bazel-diff/BazelClient.java
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@

interface BazelClient {
List<BazelTarget> queryAllTargets() throws IOException;
Set<String> queryForImpactedTestTargets(Set<String> impactedTargets) throws IOException;
Set<String> queryForImpactedTargets(Set<String> impactedTargets) throws IOException;
Set<String> queryForTestTargets(Set<String> targets) throws IOException;
Set<BazelSourceFileTarget> convertFilepathsToSourceTargets(Set<Path> filepaths) throws IOException, NoSuchAlgorithmException;
}

Expand All @@ -35,11 +36,11 @@ public List<BazelTarget> queryAllTargets() throws IOException {
}

@Override
public Set<String> queryForImpactedTestTargets(Set<String> impactedTargets) throws IOException {
public Set<String> queryForImpactedTargets(Set<String> impactedTargets) throws IOException {
Set<String> impactedTestTargets = new HashSet<>();
for (List<String> partition : Iterables.partition(impactedTargets, 100)) {
String targetQuery = partition.stream().collect(Collectors.joining(" + "));
List<Build.Target> targets = performBazelQuery(String.format("kind(test, rdeps(//..., %s))", targetQuery));
List<Build.Target> targets = performBazelQuery(String.format("rdeps(//..., %s)", targetQuery));
for (Build.Target target : targets) {
if (target.hasRule()) {
impactedTestTargets.add(target.getRule().getName());
Expand All @@ -49,6 +50,21 @@ public Set<String> queryForImpactedTestTargets(Set<String> impactedTargets) thro
return impactedTestTargets;
}

@Override
public Set<String> queryForTestTargets(Set<String> targets) throws IOException {
Set<String> impactedTestTargets = new HashSet<>();
for (List<String> partition : Iterables.partition(targets, 100)) {
String targetQuery = partition.stream().collect(Collectors.joining(" + "));
List<Build.Target> testTargets = performBazelQuery(String.format("kind(test, %s)", targetQuery));
for (Build.Target target : testTargets) {
if (target.hasRule()) {
impactedTestTargets.add(target.getRule().getName());
}
}
}
return impactedTestTargets;
}

@Override
public Set<BazelSourceFileTarget> convertFilepathsToSourceTargets(Set<Path> filepaths) throws IOException, NoSuchAlgorithmException {
Set<BazelSourceFileTarget> sourceTargets = new HashSet<>();
Expand Down
8 changes: 4 additions & 4 deletions src/main/java/com/bazel-diff/TargetHashingClient.java
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

interface TargetHashingClient {
Map<String, String> hashAllBazelTargets(Set<Path> modifiedFilepaths) throws IOException, NoSuchAlgorithmException;
Set<String> getImpactedTargets(Map<String, String> startHashes, Map<String, String> endHashes);
Set<String> getImpactedTargets(Map<String, String> startHashes, Map<String, String> endHashes) throws IOException;
Set<String> getImpactedTestTargets(Map<String, String> startHashes, Map<String, String> endHashes) throws IOException;
}

Expand Down Expand Up @@ -50,21 +50,21 @@ public Map<String, String> hashAllBazelTargets(Set<Path> modifiedFilepaths) thro
}

@Override
public Set<String> getImpactedTargets(Map<String, String> startHashes, Map<String, String> endHashes) {
public Set<String> getImpactedTargets(Map<String, String> startHashes, Map<String, String> endHashes) throws IOException {
Set<String> impactedTargets = new HashSet<>();
for ( Map.Entry<String,String> entry : endHashes.entrySet()) {
String startHashValue = startHashes.get(entry.getKey());
if (startHashValue == null || !startHashValue.equals(entry.getValue())) {
impactedTargets.add(entry.getKey());
}
}
return impactedTargets;
return bazelClient.queryForImpactedTargets(impactedTargets);
}

@Override
public Set<String> getImpactedTestTargets(Map<String, String> startHashes, Map<String, String> endHashes) throws IOException {
Set<String> impactedTargets = getImpactedTargets(startHashes, endHashes);
return bazelClient.queryForImpactedTestTargets(impactedTargets);
return bazelClient.queryForTestTargets(impactedTargets);
}

private MessageDigest createDigestForTarget(
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/com/bazel-diff/VersionProvider.java
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
class VersionProvider implements IVersionProvider {
public String[] getVersion() throws Exception {
return new String[] {
"1.0.2"
"1.1.0"
};
}
}
68 changes: 14 additions & 54 deletions src/main/java/com/bazel-diff/main.java
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ public Integer call() {
Set<String> modifiedFilepaths = gitClient
.getModifiedFilepaths(startingGitRevision, endingGitRevision)
.stream()
.map( path -> path.toString())
.map(Path::toString)
.collect(Collectors.toSet());
FileWriter myWriter = new FileWriter(outputPath);
myWriter.write(String.join(System.lineSeparator(), modifiedFilepaths));
Expand Down Expand Up @@ -115,58 +115,10 @@ public Integer call() {
}
}

@Command(
name = "impacted-tests",
description = "Write to a file the impacted test targets for the list of Bazel targets in the provided file",
mixinStandardHelpOptions = true,
versionProvider = VersionProvider.class
)
class ImpactedTests implements Callable<Integer> {

@ParentCommand
private BazelDiff parent;

@Parameters(index = "0", description = "The filepath to a newline separated list of Bazel targets")
File impactedBazelTargetsPath;

@Parameters(index = "1", description = "The filepath to write the impacted test targets to")
File outputPath;

@Override
public Integer call() {
FileReader fileReader = null;
try {
fileReader = new FileReader(impactedBazelTargetsPath);
} catch (FileNotFoundException e) {
System.out.println("Unable to read impactedBazelTargetsPath! Exiting");
return ExitCode.USAGE;
}
BufferedReader bufferedReader = new BufferedReader(fileReader);
BazelClient bazelClient = new BazelClientImpl(parent.workspacePath, parent.bazelPath);
Set<String> impactedTargets = bufferedReader.lines().collect(Collectors.toSet());
Set<String> testTargets = null;
try {
testTargets = bazelClient.queryForImpactedTestTargets(impactedTargets);
} catch (IOException e) {
System.out.println("Unable to query rdeps tests of impacted targets");
return ExitCode.SOFTWARE;
}
try {
FileWriter myWriter = new FileWriter(outputPath);
myWriter.write(testTargets.stream().collect(Collectors.joining(System.lineSeparator())));
myWriter.close();
} catch (IOException e) {
System.out.println("Unable to write to outputPath!");
return ExitCode.USAGE;
}
return ExitCode.OK;
}
}

@Command(
name = "bazel-diff",
description = "Writes to a file the impacted targets between two Bazel graph JSON files",
subcommands = { GenerateHashes.class, FetchModifiedFilepaths.class, ImpactedTests.class },
subcommands = { GenerateHashes.class, FetchModifiedFilepaths.class },
mixinStandardHelpOptions = true,
versionProvider = VersionProvider.class
)
Expand All @@ -187,7 +139,10 @@ class BazelDiff implements Callable<Integer> {
@Option(names = {"-o", "--output"}, scope = ScopeType.LOCAL, description = "Filepath to write the impacted Bazel targets to, newline separated")
File outputPath;

public Integer call() {
@Option(names = {"-t", "--tests"}, scope = ScopeType.LOCAL, description = "Return only targets of kind 'test')")
boolean testTargets;

public Integer call() throws IOException {
if (startingHashesJSONPath == null || !startingHashesJSONPath.canRead()) {
System.out.println("startingHashesJSONPath does not exist! Exiting");
return ExitCode.USAGE;
Expand All @@ -209,8 +164,8 @@ public Integer call() {
return ExitCode.USAGE;
}
Gson gson = new Gson();
FileReader startingFileReader = null;
FileReader finalFileReader = null;
FileReader startingFileReader;
FileReader finalFileReader;
try {
startingFileReader = new FileReader(startingHashesJSONPath);
} catch (FileNotFoundException e) {
Expand All @@ -226,7 +181,12 @@ public Integer call() {
Map<String, String > gsonHash = new HashMap<>();
Map<String, String> startingHashes = gson.fromJson(startingFileReader, gsonHash.getClass());
Map<String, String> finalHashes = gson.fromJson(finalFileReader, gsonHash.getClass());
Set<String> impactedTargets = hashingClient.getImpactedTargets(startingHashes, finalHashes);
Set<String> impactedTargets;
if (testTargets) {
impactedTargets = hashingClient.getImpactedTestTargets(startingHashes, finalHashes);
} else {
impactedTargets = hashingClient.getImpactedTargets(startingHashes, finalHashes);
}
try {
FileWriter myWriter = new FileWriter(outputPath);
myWriter.write(impactedTargets.stream().collect(Collectors.joining(System.lineSeparator())));
Expand Down
7 changes: 5 additions & 2 deletions test/java/com/bazel-diff/TargetHashingClientImplTests.java
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,10 @@ public void hashAllBazelTargets_sourceTargets_modifiedSources() throws IOExcepti
}

@Test
public void getImpactedTargets() {
public void getImpactedTargets() throws IOException {
when(bazelClientMock.queryForImpactedTargets(anySet())).thenReturn(
new HashSet<>(Arrays.asList("rule1", "rule3"))
);
TargetHashingClientImpl client = new TargetHashingClientImpl(bazelClientMock);
Map<String, String> hash1 = new HashMap<>();
hash1.put("rule1", "rule1hash");
Expand All @@ -130,7 +133,7 @@ public void getImpactedTargets() {

@Test
public void getImpactedTestTargets() throws IOException {
when(bazelClientMock.queryForImpactedTestTargets(anySet())).thenReturn(
when(bazelClientMock.queryForTestTargets(anySet())).thenReturn(
new HashSet<>(Arrays.asList("rule1test", "rule3test", "someothertest"))
);
TargetHashingClientImpl client = new TargetHashingClientImpl(bazelClientMock);
Expand Down