Error generating report: FATA could not connect to database err="constraint failed: FOREIGN KEY constraint failed (787)"

[NotoriousRebel]

Describe the bug

Here is how I ran gowitness initially: gowitness scan file -f urls.txt --chrome-path /snap/bin/chromium --threads 2 --write-db --write-jsonl --write-jsonl-file gowitness_results.json --delay 5 -P screenshots_path

When attempting to generate the report I tried two different ways:

First attempt: with JSONL file

gowitness report generate --json-file gowitness_results.json --zip-name gowitness_report_zip.zip --screenshot-path screenshots_path

After running here is the output:

2024/09/25 16:14:54 /root/go/pkg/mod/github.com/glebarez/sqlite@v1.11.0/migrator.go:423 constraint failed: FOREIGN KEY constraint failed (787)
[23.336ms] [rows:0] DROP TABLE `tls`
2024/09/25 16:14:54 FATA could not connect to database err="constraint failed: FOREIGN KEY constraint failed (787)"

With the way the flag is defined ( --json-file string The location of a JSON Lines results file (e.g., ./gowitness.jsonl). This flag takes precedence over --db-uri) it makes it seems the DB is not needed but that doesn't seem to be the case.

Second Attempt with DB

gowitness report generate --db-uri gowitness.sqlite3 --zip-name gowitness_report_zip.zip --screenshot-path screenshots_path

2024/09/25 16:15:28 /root/go/pkg/mod/github.com/glebarez/sqlite@v1.11.0/migrator.go:423 constraint failed: FOREIGN KEY constraint failed (787)
[8.887ms] [rows:0] DROP TABLE `tls`
2024/09/25 16:15:28 FATA could not connect to database err="constraint failed: FOREIGN KEY constraint failed (787)"

I did also check the DB and the TLS table is populated with results, although I am not sure why it is attempting to be dropped.

Version Information:

• OS: Ubuntu 22.04 LTS
• gowitness: 3.0.3

[leonjza]

Sorry haven't looked at this one yet, but will get to it soonest. Thanks for reporting.

[leonjza]

I think this is fixed now because of #240.

Here is how I ran gowitness initially: gowitness scan file -f urls.txt --chrome-path /snap/bin/chromium --threads 2 --write-db --write-jsonl --write-jsonl-file gowitness_results.json --delay 5 -P screenshots_path

The -P flag here needs to be -s. I'm surprised this did not error for you? From the --help output of gowitness scan:

  -s, --screenshot-path string     Path to store screenshots (default "./screenshots")

To test since c7eace5, I've run with the following invocation:

❯ go run main.go scan file -f ~/scratch/top1mlessporn100.txt --chrome-path "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" --threads 2 --write-db --write-jsonl --write-jsonl-file gowitness_results.json --delay 5 -s screenshots_path
2024/10/02 13:37:37 INFO result 🤖 target=http://google.com:80 status-code=200 title=Google have-screenshot=true
2024/10/02 13:37:37 INFO result 🤖 target=http://google.com:443 status-code=200 title=Google have-screenshot=true
2024/10/02 13:37:45 INFO result 🤖 target=https://google.com:443 status-code=200 title=Google have-screenshot=true
2024/10/02 13:37:50 INFO result 🤖 target=http://microsoft.com:80 status-code=200 title="Microsoft – AI, Cloud, Productivity, Computing, Gaming & Apps" have-screenshot=true
2024/10/02 13:37:54 INFO result 🤖 target=http://microsoft.com:443 status-code=200 title="Microsoft – AI, Cloud, Productivity, Computing, Gaming & Apps" have-screenshot=true
2024/10/02 13:38:02 INFO result 🤖 target=https://microsoft.com:443 status-code=200 title="Microsoft – AI, Cloud, Productivity, Computing, Gaming & Apps" have-screenshot=true
2024/10/02 13:38:08 INFO result 🤖 target=http://amazonaws.com:80 status-code=200 title="Cloud Computing Services - Amazon Web Services (AWS)" have-screenshot=true
2024/10/02 13:38:13 INFO result 🤖 target=http://facebook.com:80 status-code=200 title="Facebook – log in or sign up" have-screenshot=true
2024/10/02 13:38:15 INFO result 🤖 target=http://facebook.com:443 status-code=200 title="Facebook – log in or sign up" have-screenshot=true
2024/10/02 13:38:20 INFO result 🤖 target=https://facebook.com:443 status-code=200 title="Facebook – log in or sign up" have-screenshot=true

...

Generating the report seems to work okay.

❯ go run main.go report generate --json-file gowitness_results.json --zip-name gowitness_report_zip.zip --screenshot-path screenshots_path
2024/10/02 13:39:46 INFO generating HTML report for results count=21
2024/10/02 13:39:46 INFO report zip file generated successfully path=gowitness_report_zip.zip

❯ ls -lah gowitness_report_zip.zip
-rw-r--r--@ 1 user  staff   2.7M Oct  2 13:39 gowitness_report_zip.zip