Fix for rust and swift coverages #6517

catenacyber · 2021-09-24T11:02:09Z

Replaces #6449

Introduces lvm-cov-rel, a tool to convert profraw files from version 5 and 7/clang-13 to version 7/clang-14

cc @jonathanmetzman @inferno-chromium
Tested with

python3 infra/helper.py build_image --no-pull base-builder
python3 infra/helper.py build_image --no-pull base-builder-rust
python3 infra/helper.py build_image --no-pull base-runner
export PROJECT=suricata
python3 infra/helper.py build_image --no-pull $PROJECT
python3 infra/helper.py build_fuzzers --sanitizer coverage $PROJECT
python3 infra/helper.py coverage --no-corpus-download $PROJECT

with projects suricata, image-rs and swift-nio

jonathanmetzman · 2021-09-24T16:41:19Z

infra/base-images/base-runner/coverage

@@ -99,7 +99,12 @@ function run_fuzz_target {
    return 0
  fi

-  llvm-profdata merge -j=1 -sparse $profraw_file_mask -o $profdata_file
+  llvm-profdata merge -j=1 -sparse $profraw_file_mask -o $profdata_file || (
+    # on error, try to first run llvm-cov-rel to fix profraw file


This is pretty hacky to me. Why can't you just do the latter when using rust or swift?
Or would doing the latter for every language work.

Please begin with capital and end with period.

Yes, or we can do it in every case, as it leaves clang-14 profraw files identical

jonathanmetzman

Thanks for this. This is better than #6449 but I think could still be improved.

jonathanmetzman · 2021-09-24T17:01:05Z

infra/base-images/base-builder/cargo

@@ -35,7 +35,11 @@ then
    # go into fuzz directory if not already the case
    cd fuzz || true
    fuzz_src_abspath=`pwd`
-    export RUSTFLAGS="$RUSTFLAGS --remap-path-prefix fuzz_targets=$fuzz_src_abspath/fuzz_targets"
+    # default directory is fuzz_targets, but some projects like image-rs use fuzzers


Please begin with capital and end with period.

jonathanmetzman · 2021-09-24T17:03:02Z

infra/base-images/base-runner/gocoverage/llvm-cov-rel/llvm-profraw-relative.go

@@ -0,0 +1,180 @@
+package main


In the future I would prefer code for OSS-Fuzz be written in python.
I still need to figure out how to remove the go toolchain from the runner image because we didn't stick to this policy. All of the other code in OSS-Fuzz is python or bash (also an unfortunate choice), lets try to do python only from now on.

So, do you want me to rewrite this in python ?

Not necessary for this PR, but in future, I'd prefer it if we had as little go code as possible yes.

This PR LGTM, but if it won't take to long, could you please convert this script to Python?
Thanks!

jonathanmetzman · 2021-09-24T17:04:06Z

infra/base-images/base-builder/cargo

+    # default directory is fuzz_targets, but some projects like image-rs use fuzzers
+    while read i; do
+        export RUSTFLAGS="$RUSTFLAGS --remap-path-prefix $i=$fuzz_src_abspath/$i"
+        # bash while syntax so that we modify RUSTFLAGS in main shell instead of a subshell


Please begin with capital and end with period.

jonathanmetzman · 2021-09-24T17:04:09Z

infra/base-images/base-runner/coverage

@@ -99,7 +99,12 @@ function run_fuzz_target {
    return 0
  fi

-  llvm-profdata merge -j=1 -sparse $profraw_file_mask -o $profdata_file
+  llvm-profdata merge -j=1 -sparse $profraw_file_mask -o $profdata_file || (
+    # on error, try to first run llvm-cov-rel to fix profraw file


Please begin with capital and end with period.

jonathanmetzman · 2021-09-24T17:04:29Z

infra/base-images/base-runner/gocoverage/llvm-cov-rel/llvm-profraw-relative.go

+		}
+
+		relativizeAddress(data, offset+16, dataref, sectPrfCnts, sectPrfData)
+		// cf CountersDelta -= sizeof(*SrcData); in __llvm_profile_merge_from_buffer


Why not delete this?

This comment explains the addition afterwards

jonathanmetzman · 2021-09-24T17:04:32Z

infra/base-images/base-runner/gocoverage/llvm-cov-rel/llvm-profraw-relative.go

+		log.Fatalf("needs exactly three arguments : binary, profraw, output")
+	}
+
+	// first find llvm profile sections addresses in the elf


Please begin with capital and end with period.

jonathanmetzman · 2021-09-24T17:04:39Z

infra/base-images/base-runner/gocoverage/llvm-cov-rel/llvm-profraw-relative.go

+			sectPrfCnts = f.Sections[i].Addr
+		} else if f.Sections[i].Name == "__llvm_prf_data" {
+			sectPrfData = f.Sections[i].Addr
+			// maybe rather sectPrfCntsEnd as f.Sections[i].Addr + f.Sections[i].Size for __llvm_prf_cnts


Please begin with capital and end with period.

jonathanmetzman · 2021-09-24T17:04:41Z

infra/base-images/base-runner/gocoverage/llvm-cov-rel/llvm-profraw-relative.go

+		log.Fatalf("Elf has not __llvm_prf_cnts and __llvm_prf_data sections")
+	}
+
+	// process profraw file


Please begin with capital and end with period.

jonathanmetzman · 2021-09-24T17:04:44Z

infra/base-images/base-runner/gocoverage/llvm-cov-rel/llvm-profraw-relative.go

+		log.Fatalf("failed to process file: %v", err)
+	}
+
+	// write output file


Please begin with capital and end with period.

Cf google#6268 Latest clang-14 and clang-13 used by rust or swift have a slightly different profraw file format llvm-cov-rel is tool that will update the profraw file produced by clang-13 to one readable by clang-14 llvm-cov tools

So that projects not using default fuzz_targets subdir get the good remap, and hence the good coverage report

catenacyber · 2021-09-24T18:39:12Z

@jonathanmetzman I pushed a new version with the right syntax for comments

jonathanmetzman

LGTM

jonathanmetzman · 2021-09-27T15:11:24Z

infra/base-images/base-runner/gocoverage/llvm-cov-rel/llvm-profraw-relative.go

@@ -0,0 +1,180 @@
+package main


This PR LGTM, but if it won't take to long, could you please convert this script to Python?
Thanks!

catenacyber · 2021-09-28T10:05:04Z

Looks like base-runner dit not get updated yet so today's coverage builds still fail

I will wait to see how it goes before trying python

catenacyber · 2021-09-29T11:24:51Z

Still no updated version of base-runner ...
Is something going wrong ?

catenacyber · 2021-09-30T13:38:08Z

ping @jonathanmetzman why do the docker images not updated ?
When I run python infra/helper.py pull_images I always get

latest: Pulling from oss-fuzz-base/base-runner
Digest: sha256:41c2805f5f035993f7582796ed954b7acd9c14873a9d73860cd542cdeba1c1c6

catenacyber · 2021-10-03T18:57:00Z

Getting some fixes :-)
All swift projects, and C++ projects with rust are ok

But none of the pure rust projects are working yet... because of a new change in LLVM 14 cf #6551

jonathanmetzman reviewed Sep 24, 2021

View reviewed changes

catenacyber mentioned this pull request Sep 24, 2021

Coverage for rust projects with clang 13 #6449

Closed

jonathanmetzman reviewed Sep 24, 2021

View reviewed changes

catenacyber force-pushed the llvm-cov-rel branch from 1d17253 to ad39e43 Compare September 24, 2021 18:37

catenacyber added 3 commits September 24, 2021 20:38

coverage: introduces llvm-cov-rel

e317bd2

Cf google#6268 Latest clang-14 and clang-13 used by rust or swift have a slightly different profraw file format llvm-cov-rel is tool that will update the profraw file produced by clang-13 to one readable by clang-14 llvm-cov tools

Suricata as a rust project

15953ac

rust coverage: remaps every rust subdirectory in fuzz

5d27ce5

So that projects not using default fuzz_targets subdir get the good remap, and hence the good coverage report

catenacyber force-pushed the llvm-cov-rel branch from ad39e43 to 5d27ce5 Compare September 24, 2021 18:38

jonathanmetzman approved these changes Sep 27, 2021

View reviewed changes

jonathanmetzman merged commit 5c386a4 into google:master Sep 27, 2021

catenacyber mentioned this pull request Oct 3, 2021

rust coverage: fix binary ids padding #6551

Merged

catenacyber mentioned this pull request Oct 6, 2021

profraw: move from golang to python #6565

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix for rust and swift coverages #6517

Fix for rust and swift coverages #6517

catenacyber commented Sep 24, 2021

jonathanmetzman Sep 24, 2021

jonathanmetzman Sep 24, 2021

catenacyber Sep 24, 2021

jonathanmetzman left a comment

jonathanmetzman Sep 24, 2021

catenacyber Sep 24, 2021

jonathanmetzman Sep 24, 2021

catenacyber Sep 24, 2021

jonathanmetzman Sep 24, 2021

jonathanmetzman Sep 27, 2021

jonathanmetzman Sep 24, 2021

catenacyber Sep 24, 2021

jonathanmetzman Sep 24, 2021

jonathanmetzman Sep 24, 2021

catenacyber Sep 24, 2021

jonathanmetzman Sep 24, 2021

catenacyber Sep 24, 2021

jonathanmetzman Sep 24, 2021

catenacyber Sep 24, 2021

jonathanmetzman Sep 24, 2021

catenacyber Sep 24, 2021

jonathanmetzman Sep 24, 2021

catenacyber Sep 24, 2021

catenacyber commented Sep 24, 2021

jonathanmetzman left a comment

jonathanmetzman Sep 27, 2021

catenacyber commented Sep 28, 2021

catenacyber commented Sep 29, 2021

catenacyber commented Sep 30, 2021

catenacyber commented Oct 3, 2021

Fix for rust and swift coverages #6517

Fix for rust and swift coverages #6517

Conversation

catenacyber commented Sep 24, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonathanmetzman left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

catenacyber commented Sep 24, 2021

jonathanmetzman left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

catenacyber commented Sep 28, 2021

catenacyber commented Sep 29, 2021

catenacyber commented Sep 30, 2021

catenacyber commented Oct 3, 2021