Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add many valid and invalid test JSON files #919

Merged
merged 2 commits into from
Jul 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,40 @@
# Major changes to the IOCCC entry toolkit

## Release 1.2.5 2024-07-28

We add a lot of good and bad JSON files to the test JSON directories
(`test_ioccc/test_JSON/good` and `test_ioccc/test_JSON/bad` respectively) that
come from the [JSONTestSuite](https://github.com/nst/JSONTestSuite) repo with
all tests succeeding. We give our gratitude to the maintainers of the repo! For
a quick note on why these results are significant, besides the fact that it is
more validation that our parser is in good shape, is from the maintainers of the
repo. They write about the files:

These files contain weird structures and characters that parsers may understand
differently, e.g.:

huge numbers
dictionaries with similar keys
NULL characters
escaped invalid strings

Again we give a **HUGE THANK YOU** to the
[JSONTestSuite](https://github.com/nst/JSONTestSuite) repo maintainers!

The jparse/README.md file has been updated to move the history section to the
very bottom and to discuss (briefly) the testing procedure that Landon and I
(Cody) have implemented.

The `make test`, `make prep` and `make slow_prep` rules now show the time of
starting and time of ending.

Also, since this was not mentioned, we recently added (with thanks to
@SirWumpus) a workflow that runs `make slow_prep` on push and pull request so we
can more easily find problems (in case someone forgets to run the rule prior to
commit or if there is a case where the local environment of the committer makes
the test pass even though it should not, which has happened before).


## Release 1.2.4 2024-07-14

Add constants for `test_formed_timestamp_usec()` in `entry_util.c`.
Expand Down
10 changes: 8 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -565,6 +565,7 @@ reset_min_timestamp: soup/Makefile
# rule instead.
#
prep: test_ioccc/prep.sh
${S} echo "${OUR_NAME}: make $@ starting at: `date`"
${Q} ${RM} -f ${TMP_BUILD_LOG}
${Q} ./test_ioccc/prep.sh -m${MAKE} -l "${TMP_BUILD_LOG}"; \
EXIT_CODE="$$?"; \
Expand All @@ -576,15 +577,18 @@ prep: test_ioccc/prep.sh
echo; \
echo "make $@: see ${BUILD_LOG} for details"; \
if [[ $$EXIT_CODE -ne 0 ]]; then \
${S} echo "${OUR_NAME}: make $@ ending at: `date`"; \
exit "$$EXIT_CODE"; \
else \
${S} echo "${OUR_NAME}: make $@ ending at: `date`"; \
echo "All Done!!! All Done!!! -- Jessica Noll, Age 2"; \
fi

# a slower version of prep that does not write to a log file so one can see the
# full details.
#
slow_prep: test_ioccc/prep.sh
${S} echo "${OUR_NAME}: make $@ starting at: `date`"
${Q} ${RM} -f ${TMP_BUILD_LOG}
${Q} ./test_ioccc/prep.sh -m${MAKE}; \
EXIT_CODE="$$?"; \
Expand All @@ -595,8 +599,10 @@ slow_prep: test_ioccc/prep.sh
echo; \
echo "make $@: see ${BUILD_LOG} for details"; \
if [[ $$EXIT_CODE -ne 0 ]]; then \
${S} echo "${OUR_NAME}: make $@ ending at: `date`"; \
exit "$$EXIT_CODE"; \
else \
${S} echo "${OUR_NAME}: make $@ ending at: `date`"; \
echo "All Done!!! All Done!!! -- Jessica Noll, Age 2"; \
fi

Expand Down Expand Up @@ -940,7 +946,7 @@ all_tags:
#
test:
${S} echo
${S} echo "${OUR_NAME}: make $@ starting"
${S} echo "${OUR_NAME}: make $@ starting at: `date`"
${S} echo
${E} ${MAKE} ${MAKE_CD_Q} -C dbg $@
${E} ${MAKE} ${MAKE_CD_Q} -C dyn_array $@
Expand All @@ -949,7 +955,7 @@ test:
${E} ${MAKE} ${MAKE_CD_Q} -C test_ioccc $@
${E} ${RM} -f jparse/test_jparse/pr_jparse_test
${S} echo
${S} echo "${OUR_NAME}: make $@ ending"
${S} echo "${OUR_NAME}: make $@ ending at: `date`"
${S} echo "All done!!! All done!! -- Jessica Noll, Age 2."

# run test-chkentry on test_JSON files
Expand Down
43 changes: 30 additions & 13 deletions jparse/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,17 +109,6 @@ process at the same time is not tested so even though it should be okay there
might be some issues that have yet to be discovered.


# History

For more detailed history that goes beyond this humble document we
recommend you check `jparse(1)` and the `chkentry(1)` man page in the
`mkiocccentry` repo as well as its `CHANGES.md` and `README.md` files. If you
want to go further than that you can read the GitHub git log as well as reading
the source code. If you do read the source code we **STRONGLY** recommend you
read the `jparse.l` and `jparse.y` files and **NOT** the bison or flex generated
code! This is because the generated code might give you nightmares and cause
other horrible symptoms. :-) See `sorry.tm.ca.h` for more details.

# Examples

Parse the JSON string `{ "test_mode" : false }`:
Expand Down Expand Up @@ -215,17 +204,45 @@ man ./man/man1/jstrdecode.1

NOTE: After doing a `make all`, this tool may be found as: `./jstrdecode`.

# Our testing suite:

In the [jparse/test_jparse](jparse/test_jparse/README.md) subdirectory we have a
test suite script [jparse_test.sh](jparse/test_jparse/jparse_test.sh) which runs
a battery of tests on both valid and invalid JSON files, both as strings and as
files, and if valid JSON files do **NOT** pass as valid **OR** if invalid JSON
files **DO** pass as valid then it is an error.

We have used our own files (with some Easter eggs included due to a shared
interest between Landon and Cody :-) ) as well as from the
[JSONTestSuite](https://github.com/nst/JSONTestSuite) repo (and with **MUCH
GRATITUDE** to the maintainers: **THANK YOU**!) and all is good. If for some
reason the parser were to be modified, in error or otherwise, and the test fails
then we know there is a problem. As the GitHub repo has workflows to make sure
that this does not happen it should never be added to the repo.

## Other `jparse` tools:

## Other `jparse` tools (WARNING: these are **VERY INCOMPLETE and will be heavily modified**):

We also provide a number of tools that are, at least in what they will do when
completed, in [json_util_README.md](json_util_README.md). But please note the
warning there that says:

<hr>
Please be advised that the tools `jfmt`, `jval` and `jnamval` are **VERY
As above, please be advised that the tools `jfmt`, `jval` and `jnamval` are **VERY
INCOMPLETE** and **WILL BE HEAVILY MODIFIED**. It is also highly likely that
they will be almost **entirely rewritten** as things changed as they were first
worked on. Almost everything will be redone at this point. When this is done
this notice will be removed.
<hr>

# History

For more detailed history that goes beyond this humble document we
recommend you check `jparse(1)` and the `chkentry(1)` man page in the
`mkiocccentry` repo as well as its `CHANGES.md` and `README.md` files. If you
want to go further than that you can read the GitHub git log as well as reading
the source code. If you do read the source code we **STRONGLY** recommend you
read the `jparse.l` and `jparse.y` files and **NOT** the bison or flex generated
code! This is because the generated code might give you nightmares and cause
other horrible symptoms. :-) See [sorry.tm.ca.h](sorry.tm.ca.h) for more details
on this.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[1 true]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[a�]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
["": 1]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[""],
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[,1]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_array_double_comma.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[1,,2]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
["x",,]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_array_extra_close.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
["x"]]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_array_extra_comma.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
["",]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_array_incomplete.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
["x"
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[x
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[3[4]]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_array_invalid_utf8.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[�]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[1:2]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_array_just_comma.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[,]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_array_just_minus.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[-]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[ , ""]
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
["a",
4
,1,
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[1,]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[1,,]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[" a"\f]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_array_star_inside.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[*]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_array_unclosed.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[""
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[1,
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[1,
1
,1
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[{}
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_incomplete_false.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[fals]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_incomplete_null.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[nul]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_incomplete_true.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[tru]
Binary file not shown.
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_++.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[++1234]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_+1.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[+1]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_+Inf.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[+Inf]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_-01.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[-01]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_-1.0..json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[-1.0.]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_-2..json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[-2.]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_-NaN.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[-NaN]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_.-1.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[.-1]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_.2e-3.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[.2e-3]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_0.1.2.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[0.1.2]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_0.3e+.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[0.3e+]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_0.3e.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[0.3e]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_0.e1.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[0.e1]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[0E+]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_0_capital_E.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[0E]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_0e+.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[0e+]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_0e.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[0e]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_1.0e+.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[1.0e+]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_1.0e-.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[1.0e-]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_1.0e.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[1.0e]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_1_000.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[1 000.0]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_1eE2.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[1eE2]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_2.e+3.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[2.e+3]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_2.e-3.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[2.e-3]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_2.e3.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[2.e3]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_9.e+.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[9.e+]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_Inf.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[Inf]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_NaN.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[NaN]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[1]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_expression.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[1+2]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_hex_1_digit.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[0x1]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[0x42]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_infinity.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[Infinity]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_invalid+-.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[0e+-1]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[-123.123foo]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[123�]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[1e1�]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[0�]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[-Infinity]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[-foo]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[- 1]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[-012]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[-.123]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[-1x]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[1ea]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[1e�]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[1.]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[.123]
1 change: 1 addition & 0 deletions jparse/test_jparse/test_JSON/bad/n_number_with_alpha.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[1.2a-3]
Loading