Possibly better handling of MSAN reports

[LebedevRI]

I may be completely wrong here, but i'm under the impression that currently the reports
from memory sanitizer are deduplicated by the "Crash State", and then reported, correct?

Consider the following pseudo-code:

// both functions do some different stuff with memory
// but both of them do *not* init all of the memory
void f0(char* mem); 
void f1(char* mem);

int main(int argc, char* argv[]) {
  char buf[32];

  if(argc == 1)
   f0(&buf[0]);
  else
   f1(&buf[0]);

  __msan_check_mem_is_initialized(&buf[0], sizeof(buf)); // ta-da!
}

There are two distinct problems, but currently i believe oss-fuzz would only report one,
and only after it is fixed, report the second-one.

[Dor1s]

Why are there two distinct problems? It looks like buf is not initialized, that's it. What else?

[LebedevRI]

This is just a dumb example to show the problem :)
The f0 / f1 are supposed to fully populate the buffer.

In the real code the buf is some image, which definitively should not be pre-initialized,
and the functions absolutely should correctly fully fill it.

This is a real problem i'm having (well, finding/fixing) in the rawspeed

[Dor1s]

But how do you want to deduplicate those cases if the use-of-uninitialized-value happens outside of f0 / f1?

[oliverchang]

I'm not sure if there's anything we can do here to catch this case as distinct reports -- it seems to me that this is not feasible.

I'm going to say this is WAI, but happy to discuss more if you have any suggestions though.

[LebedevRI]

By comparing the sets of Features each of the crash testcase triggers.
But i agree that there are likely cases where it would erroneously detect the same problem as multiple.

[LebedevRI]

Another idea. Please compare these two issues:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3531
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3497

The Stacktraces specified in "Uninitialized value was created by a heap allocation" differ.
So perhaps accounting for that while deduplicating could work?

That wouldn't help with the case i presented in the first comment here, obviously.

[Dor1s]

Do you want these two bugs be filed as a single one? I think it would be so, but the second bug got fixed before the first got filed, this is why those are reporter separately. //cc @inferno-chromium

[LebedevRI]

Do you want these two bugs be filed as a single one?

No, exactly the opposite.

the second bug got fixed before the first got filed

That is exactly the problem.

So far, there is exactly one of these bugs open at any point in time.
I fix the open issue, it is auto-closed, and the new issue is reported after that.
Thus i think they are being deduplicated, hiding different issues.

Thus i'm bringing up the issue, that deduplication is not ideal.

[LebedevRI]

Any thoughts on this? Or would taking the allocation trace into account result in too many duplicates too?

[Dor1s]

That sounds like a potential improvement. Not a priority right now, but let's keep the issue open. Thanks!

[LebedevRI]

And the pattern continues...
I don't suppose the code that is used for that report deduplication is public and open-source, so i can provide a patch? :)

[Dor1s]

Yeah, the code is not open source...

[inferno-chromium]

@LebedevRI - code is open source and patch can be provided there if possible.