suggested: make test displays (expected; actual) values for failed tests

[guberathome]

It would greatly help debugging if a failed test case (e.g. in make test) would return the offending value.

Here's an example from the current MSYS2-Cygwin environment
(fixing RESIZE-FILE on this platform is a separate topic).

OOTB we get:

cd ../../fth && ../platforms/unix/pforth_standalone -q t_file.fth
...
INCORRECT RESULT: T{ FID2 @ FILE-SIZE -> 37. 0 }T
INCORRECT RESULT: T{ CBUF BUF 100 FID2 @ READ-FILE -> 37 0 }T
INCORRECT RESULT: T{ PAD 38 BUF 38 S= -> FALSE }T
...
  91 passed,    3 failed.

The suggested change would give us:

cd ../../fth && ../platforms/unix/pforth_standalone -q t_file.fth
...
INCORRECT RESULT (expected=37, got=87): T{ FID2 @ FILE-SIZE -> 37. 0 }T
INCORRECT RESULT (expected=37, got=87): T{ CBUF BUF 100 FID2 @ READ-FILE -> 37 0 }T
INCORRECT RESULT (expected=0, got=-1): T{ PAD 38 BUF 38 S= -> FALSE }T
...
  91 passed,    3 failed.

I do have a few concerns about int>str, concat and concat+free though:

• Are similar words already defined elsewhere perhaps?
• Is this the right place to define them (instead of e.g. t_strings.fth or strings.fth)?
  For now I preferred NOT to create a dependancy to other forth code files.

Feedback is welcome.

[philburk]

Thanks! I think printing the EXPECTED and ACTUAL will make debugging much easier.

[philburk]

There may be a memory leak here because ERROR does not free the memory allocated by CONCAT.
Rather that trying to insert the new message into the ERROR message, let's just add this after the call to ERROR.

Suggested change

	s" , got=" concat+free R> int>str concat+free
	." (expected=" r> .
	." , got=" r> .
	." )"

Then we don't need the INT>STR or CONCAT tools.

[guberathome]

Agreed, not having to deal with memory (de)allocation would be a definite bonus.

However, having the values at the start of each line makes it easier to read for me.
Could we perhaps directly output the values BEFORE calling error?

            if
                2drop
            else
                -1 test-passed +!
                1 test-failed +!
                ." INCORRECT RESULT (got=" (.) type
                ." , expected=" (.) type
                s" ) from "
                error
                LEAVE
            THEN

This would give as output:

cd ../../fth && ../platforms/unix/pforth_standalone -q t_file.fth
...
INCORRECT RESULT (got=87, expected=37) from T{ FID2 @ FILE-SIZE -> 37. 0 }T
INCORRECT RESULT (got=87, expected=37) from T{ CBUF BUF 100 FID2 @ READ-FILE -> 37 0 }T
INCORRECT RESULT (got=-1, expected=0) from T{ PAD 38 BUF 38 S= -> FALSE }T
...
  91 passed,    3 failed.

[philburk]

I agree that the actual/expected looks better where you put it.
But I don't like passing a funky ") from" to the ERROR word.
This was inherited from the test framework. We can change it but I am planning to update to a more official
version of the test framework. So we will have to carry over these changes.

[guberathome]

Done.

Output is now:

(got=87, expected=37) INCORRECT RESULT: T{ FID2 @ FILE-SIZE -> 37. 0 }T
(got=87, expected=37) INCORRECT RESULT: T{ CBUF BUF 100 FID2 @ READ-FILE -> 37 0 }T
(got=-1, expected=0) INCORRECT RESULT: T{ PAD 38 BUF 38 S= -> FALSE }T

[philburk]

Nice. A small simple change that helps debugging.