exim: Use har-reader

[kingthorin]

Overview

• CHANGELOG > Added change note.
• build file > Updated/added dependencies.
• HarImporter > Re-worked to use har-reader lib.
• HarUtils > Added to to facilitate use of the new lib. Contains a number of methods that still need to be re-worked for writing.
• MenuImportHar > Re-worked and simplified to take advantage of the new utils, importer, and lib.
• HarImporterUnitTest > Tweaked to accommodate HarImporter changes.

Related Issues

• Fixes HAR import fails silently zaproxy#8500

Checklist

• Update help
• Update changelog
• Run ./gradlew spotlessApply for code formatting
• Write tests
• Check code coverage
• Sign-off commits
• Squash commits
• Use a descriptive title

[kingthorin]

Removed the inner classes, still need to tweak the HTTP Version handling.

[kingthorin]

I've adjusted the version handling now. I believe this now has all the previously discussed changes.

[thc202]

I'd do a final review later in the day.

[kingthorin]

I think I got all the comments, just need to add tests. Please lemme know if I've missed something.

[kingthorin]

I started to add tests. There were a number of challenges. I'll add some comments to cover those.

The tests will fail, they succeed individually but there seems to be some sort of concurrency or cleanup issue when run together.

[thc202]

#5250 (comment)

The logger should be set up for each test not all of them, also better to call Configurator.reconfigure(getClass().getResource("/log4j2-test.properties").toURI()); to reset the configuration to previous state. (To be clear that does not fix the test failures, there are still some tests that need further changes, like setting up some more mocking and the messages.)

[thc202]

I was not clear with #5250 (comment) I meant that the logger should be initialised in a BeforeEach rather than BeforeAll, sorry.

[kingthorin]

Okay I've updated a bunch of stuff. Things are a bit better. The log handling was ripped off from GraphQL tests. I had to put your reconfigure suggestion in each method because AfterAll is static and didn't like me using getClass (and I couldn't figure out an alternative to getClass).

Using mockMessages didn't seem to fix the console barf, the key is there so I don't get the issue....

Can you think of another place we've done mocking to handle persisting messages? I think I recall talking about this for another PR but I can't figure out what it was.

[thc202]

The log reset should be done after each test.

Let me look.

SpiderTaskUnitTest

[kingthorin]

Getting closer... 🤷‍♂️

[kingthorin]

Some of those comments might be ordered poorly, I didn't realize I hadn't finished my comments/review on Friday

[kingthorin]

I dropped the header pre-processing.

The examples I had only impacted responses. Not sure if I should handle requests similarly?

[thc202]

I'm good if done only to responses.

[kingthorin]

I think/hope this is ready for review. I moved the remainder of the core HarUtils code here and commented it out. I'll work on writing/export in another PR.

[thc202]

I don't think we should be copying commented code that applies to the older library, why not just apply the changes when needed instead?

[kingthorin]

Okay, I can just add a TODO if preferred

[thc202]

Just a TODO is better IMO.

[kingthorin]

Done & done

[kingthorin]

Done

[thc202]

Thank you!