Sanitization and Fuzzing

[hmelder]

It would be great to run address and ub sanitisation during testing in CI.

[rfm]

What sort of thing did you have in mind?

…

On 5 Nov 2024 at 00:39, Hugo Melder ***@***.***> wrote: It would be great to run address and ub sanitisation during testing in CI. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[hmelder]

What sort of thing did you have in mind?

Address sanitization greatly helps to reduce memory errors and time spend debugging them. Running the unit tests with CI is a good start as we already have decent code coverage.

However, I've just noticed that we leak a lot of memory that we don't cleanup before program exit, which results in ASAN being triggered in nearly every ObjC program compiled with GNUstep. There is an ignore list we can use.

Here is a patch to get ASAN working for projects using GNUstep Make: gnustep/tools-make#47.

[rfm]

I updated that adding en environment variable setting GS_WITH_ASAN=1 to turn it on.
With that it's easy to build both the libraries and run their tests with asan.
Of course, most tests then fail because asan aborts them at the end because the testcases are quite happy to leak objects ... I'm working on fixing that (and improving cleanup at exit and other changes to prevent normally harmless leaks etc in the base library) in the atexit branch.

[hmelder]

I'm working on fixing that (and improving cleanup at exit and other changes to prevent normally harmless leaks etc in the base library) in the atexit branch.

Thank you very much. I can help you with that.

[rfm]

I spent about 8 days getting most of the base library tests to pass. That's mostly(say 95%) fixing leaks in the testcases (previously there was no incentive to avoid leaking memory in a short-running test), but also quite a few issues in the base library.

The remaining issues are with some of the code I'm least familiar with:
Some of the NSURLConnection tests leaking; I'm planning to look into those next.
Leaks in the NSURLSession tests: perhaps you could look at that?
Leaks in some of the NSXMLNode etc tests: probably Greg would be best to look into those (I think they are libxml2 bugs or errors in the way we interact with libxml2 memory management).

[rfm]

`I spent all yesterday tracking down and fixing the issues that were causing leaks in NSURLConnection tests.
As part of that I improved the new -trackOwnership method to report tracked instances still in existence at process exit and fixed subtle errors in that code related to situations where we instantiate instances of subclasses of a tracked class and/or track those subclass instances. Tracking the lifecycle of an individual object should now be reliable and easy.

[rfm]

Changed GS_WITH_ASAN to be GNUSTEP_WITH_ASAN
Added --asan option to gnustep-tests and made it report tests with an exitcode of 23 as leaked memory.
Added lsan.txt file in Tests/base directory with suppression information for the gnu runtime
Added a load of changes to get most tests to pass when we build with gcc and the gnu runtime setting:
GNUSTEP_WITH_ASAN=1
LSAN_OPTIONS=exitcode=23:suppressions=/home/richard/git/libs-base/Tests/base/lsan.txt

Testcases for NSURLSession and the NSXMLNode classes still fail.
I will take a look at the libxml2 integration next, but will probably need help from Greg, and can't get round to it immediately anyway as I'm away on Monday and Tuesday,