Problem with crash detection in the fuzzstati0n/fuzzgoat project?

[ihadnsgpsr]

Hi,

I failed to launch a fuzzer in a way that allows me to detect crashes in the fuzzgoat project.

https://github.com/fuzzstati0n/fuzzgoat

This C program has been deliberately backdoored with several memory corruption bugs to test the efficacy of fuzzers and other analysis tools.

For some reason manul has failed to detect them despite fuzzing for hours, there is a 99% chance that I failed to set it up properly.

Do I need to build the target in a different way compared to standard AFL build with e.g. afl-gcc?
Am I doing something wrong?

Steps needed to reproduce steps: 

1. Download and build fuzzgoat

git clone https://github.com/fuzzstati0n/fuzzgoat.git
cd /home/user/fuzzgoat
make

Here is the fuzzgoat Makefile:

CC=afl-gcc
DEPS=main.c fuzzgoat.c
ASAN=-fsanitize=address
CFLAGS=-I.
LIBS=-lm

all: $(DEPS)
	$(CC) -o fuzzgoat $(CFLAGS) $^ $(LIBS)
	$(CC) $(ASAN) -o fuzzgoat_ASAN $(CFLAGS) $^ $(LIBS)

afl: fuzzgoat
	afl-fuzz -i in -o out ./fuzzgoat @@

.PHONY: clean

clean:
	rm ./fuzzgoat ./fuzzgoat_ASAN

2. Download and build manul

pip3 install psutil
git clone https://github.com/mxmssh/manulcd manul
mkdir in
mkdir out
echo "AAAAAA" > in/test

3. Launch manul against the target

python3 manul.py -i in -o out -n 4 "/home/user/fuzzgoat/fuzzgoat @@"

[ihadnsgpsr]

Ok, let's say I "fixed it".

I simply changed the input seed.

default manul test file contained --> " AAAAAA "
default fuzzgoat test file contained --> " {"":"'} "

I tried to launch AFL with both test strings against fuzzgoat, but only with the first one using manul.
Manul detected crashes using the second string as an input.

In my defense... both inputs generated crashes immediately using AFL, and I haven't thought that it might be the problem.

[mxmssh]

Seems like this issue related to #54