How to run a test-kit from terminal?

[ppKrauss]

Before to say that tidy-html5 (terminal) have a bug, I need to check my installation, so any standard HTML corpus and standard assert commands can be used.

There are a test-kit? How to run it at UBUNTU terminal?

[balthisar]

@ppKrauss, you can clone to testing repository: https://github.com/htacg/tidy-html5-tests

You can use run-tests.sh on Ubuntu. Check your environment so the script knows where to find your Tidy.

[ppKrauss]

Hi @balthisar, thanks!

1. git clone https://github.com/htacg/tidy-html5-tests.git ok
2. cd tidy-html5-tests.git ok
3. sh tools-sh/run-tests.sh ERROR:
   run-tests.sh: Can NOT locate ../../tidy-html5/build/cmake/tidy! *** FIX ME ***.

I also tryed cd tools-sh and sh run-tests.sh but same error...
But checking more run-tests.sh I not see what I need. Need to test my tidy, not to install a new tidy neither to check compiler, directives, etc.

tidy --version
HTML Tidy for Linux version 5.4.0

[balthisar]

Like I said, check your environment so the script knows where to find your Tidy. Or you can modify run-tests.sh so that TMPTIDY points to the Tidy you want to test.

[geoffmcl]

@ppKrauss, as @balthisar advised, you need to know the full path to the tidy you want to run and test...

Since you seem to have just typed $ tidy -v, then this seems to be a tidy available in your PATH, so you should be able to get its full path by $ whereis tidy...

If you want to test a tidy not installed, and/or not in your PATH then you need to supply the relative, or fully qualified path to the tidy you want to use... that can be where ever you built tidy...

Now once you have the path to the tidy to be used, you can either modify the run-tests.sh, putting your path in TMPTIDY="/path/to/tidy", or directly run $ ./testall.sh /path/to/tidy, but then you need to run the second diff ... part manually...

Note all testing must be run in the tools-sh folder... all the setup is relative to that...

Further you are indicating you want to test tidy version 5.4.0, so to get the correct test cases for that version, you will need to backup the repo to around that version...

To get a suitable commit hash to back up to you can run $ git log -p cases/_version.txt > ../tempv.log... doing that it seems you will need commit 3123900, that is $ git checkout 3123900

I just did that in windows, git status = HEAD detached at 3123900, running Tidy Version: HTML Tidy for Windows version 5.4.0, and got a clean run...

Hope you can do the same in Ubuntu...

Yes, the process is a bit complicated!, and we hope someone can find the time to improve the testing documentation, and the runtime testing situation...

It will always be a little difficult to test older tidy versions, in that you must know how to find a suitable commit to back up the repo, which requires git knowledge, and until PR #662 is merged, there is a known test case failure for current 5.7.0 tidy in next...

HTH...

[geoffmcl]

@ppKrauss using my explanation here add a README/RUNTESTS.md document which may be helpful...

[geoffmcl]

@ppKrauss and as an idea of how to make it more easy to back-up the repo to test a specific older version of tidy, created a test 5.4.0 branch...

$ git checkout tests/5.4.0

Maybe this could be done on each needed/required tests change... this is for sure better than finding a suitable commit in a git log... but we will see if that takes on...

That would allow the test scripts to be modified to test for a specific branch before running the tests... say adding a [-v 5.4.0], default next, to the alltest.bat or unix script... but again work to be done...

And because the tidy source versions, and test repo versions do not run in sync, and usually do not need to!, in that there can be many source changes that do not require a tests repo change, this would still be a user choice, with that full understanding in mind...

It remains, there is some heavy work to be done in the tests repo to make is easy to run the tests, anytime, with any version... help needed... thanks...

[geoffmcl]

@ppKrauss have just done several merges, including PR #662, and just done running HTML Tidy for Linux version 5.7.3, in Ubuntu, built from the next branch, using the next tests branch, testbase version 5.5.84 and it PASSES both phases of the tests...

Hope you, and others, have the same good results, in this new year, 2018...