add bqm

[noraj]

Description

add https://github.com/Acceis/bqm

Related issues

N/A

Point of attention

maybe it could be used to update https://github.com/ThePorgs/Exegol-images/blob/main/sources/bloodhound/customqueries.json

[ShutdownRepo]

Like #102 and #103, switching to dev branch. Please take notice for future pull request 😉

[noraj]

I'm terribly sorry, I missed to read https://github.com/ThePorgs/Exegol-images/blob/main/CONTRIBUTING.md

[ShutdownRepo]

One change required and PR will be ready for testing. One comment though, Exegol already has a set of custom queries. It would be nice to include its dataset in those queried by bqm.
Also, this tool will need some further manual testing. I'm wondering if this aggregator takes the categories into consideration when deduplicating and merging custom queries.

[noraj]

One comment though, Exegol already has a set of custom queries. It would be nice to include its dataset in those queried by bqm.

Yeah that was the point of attention I brought

maybe it could be used to update https://github.com/ThePorgs/Exegol-images/blob/main/sources/bloodhound/customqueries.json

If the sub-set provided by bqm is larger than the actual one if could replace it unless there are queries that are not in bqm.

I'm wondering if this aggregator takes the categories into consideration when deduplicating and merging custom queries.

Yeah I use it and categories are preserved.

[ShutdownRepo]

Ok nice 👌
Beware though, exegol custom queries include some homemade queries that probably won't be in the datasets bqm is fetching.
Also, any reason why https://github.com/noraj/bqm would be used instead of origin https://github.com/Acceis/bqm ? I also notice your fork is a few commits behind origin, hence my interrogation

[noraj]

Also, any reason why https://github.com/noraj/bqm would be used instead of origin https://github.com/Acceis/bqm ? I also notice your fork is a few commits behind origin, hence my interrogation

An error from my side. The gem comes from acceis repo, I just pasted the wrong URL in the issue.

[ShutdownRepo]

Also, any reason why https://github.com/noraj/bqm would be used instead of origin https://github.com/Acceis/bqm ? I also notice your fork is a few commits behind origin, hence my interrogation

An error from my side. The gem comes from acceis repo, I just pasted the wrong URL in the issue.

No biggie.
Once the pipeline tests pass successfully, we (you if possible) will need to make some further testing to make sure the tool integrates swiftly with Exegol, not removing queries 😉

[noraj]

Once the pipeline tests pass successfully, we (you if possible) will need to make some further testing to make sure the tool integrates swiftly with Exegol, not removing queries wink

The tool doesn't overwrite anyfile, the user has to provide an output path eg. bqm --output-path ~/.config/bloodhound/customqueries.json so it's the user choice to overwrite the existing queries or to save the output file anywhere. I can guarantee 0% conflcit. No default location is written to if no option is provided either.

[ShutdownRepo]

Once the pipeline tests pass successfully, we (you if possible) will need to make some further testing to make sure the tool integrates swiftly with Exegol, not removing queries wink

The tool doesn't overwrite anyfile, the user has to provide an output path eg. bqm --output-path ~/.config/bloodhound/customqueries.json so it's the user choice to overwrite the existing queries or to save the output file anywhere. I can guarantee 0% conflcit. No default location is written to if no option is provided either.

Yes, but if I'm correct, BloodHound only loads custom queries from ~/.config/bloodhound/customqueries.json. And that's what's in the history command. Meaning the user will overwrite the existing custom queries in most cases. We need to make sure the already existing queries are taken into account, probably by adding Exegol's custom queries to bqm's datasets, or by having an Exegol fork of bqm or whatever different solution.

[noraj]

Yes, but if I'm correct, BloodHound only loads custom queries from ~/.config/bloodhound/customqueries.json. And that's what's in the history command. Meaning the user will overwrite the existing custom queries in most cases. We need to make sure the already existing queries are taken into account, probably by adding Exegol's custom queries to bqm's datasets, or by having an Exegol fork of bqm or whatever different solution.

Right, I can add https://github.com/ThePorgs/Exegol-images/blob/main/sources/bloodhound/customqueries.json to the data set of bqm. Acceis/bqm#14

[ShutdownRepo]

Also, or in addition to, I think it would be nice to have bqm check if the file indicated in the path already exist, and use it’s content as part of the dataset. This would allow, whether it’s Exegol or somethings else, to always sync with what’s already there locally

[noraj]

Also, or in addition to, I think it would be nice to have bqm check if the file indicated in the path already exist, and use it’s content as part of the dataset. This would allow, whether it’s Exegol or somethings else, to always sync with what’s already there locally

Nice idea Acceis/bqm#15

[noraj]

@ShutdownRepo https://github.com/Acceis/bqm/releases/tag/v1.3.0 should satisfy the requirements for this PR.

[ShutdownRepo]

@ShutdownRepo https://github.com/Acceis/bqm/releases/tag/v1.3.0 should satisfy the requirements for this PR.

That was fast!! Can you resolve the conflicts? We'll then proceed to manual checks before merging

[ShutdownRepo]

Also, what happens if twp exact same queries exist in different categories? Both are kept I suppose?

[noraj]

Also, what happens if twp exact same queries exist in different categories? Both are kept I suppose?

A query is considered duplicate if it has the same name and same queryList. It was primarily to avoid copy/paste because some data set are some custom queries + merged ones from other datasets. But duplicates can exist if the same query has different names.

[noraj]

@ShutdownRepo Have you been able to test it?

[ShutdownRepo]

Didn't find the time to do the tests manually. Sorry for the delay, it's been a few crazy weeks. Let's merge and iterate later on if needed. Thank you again for the addition and edits on bqm by taking our discussion into account.