Some testcases fail and one never finishes #1032

yurivict · 2018-03-31T20:34:04Z

      Start 10: test-cbor_all
10/78 Test #10: test-cbor_all .......................***Exception: Child aborted  9.98 sec
      Start 56: test-msgpack_all
56/78 Test #56: test-msgpack_all ....................***Exception: Child aborted  9.99 sec
      Start 74: test-ubjson_all
74/78 Test #74: test-ubjson_all .....................***Exception: Child aborted  2.96 sec
      Start 78: test-unicode_all
/// {never finishes}

OS: FreeBSD-11.1 amd64
Version 3.1.2 built from port, while trying to update it.

The text was updated successfully, but these errors were encountered:

nlohmann · 2018-03-31T20:50:17Z

Which compiler are you using? What are the error messages?

yurivict · 2018-03-31T21:06:11Z

$ c++ --version
FreeBSD clang version 5.0.1 (tags/RELEASE_501/final 320880) (based on LLVM 5.0.1)

Some cases that failed before individually don't print errors for some reason.

$ ./work/.build/test/test-cbor 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test-cbor is a Catch v1.9.7 host application.
Run with -? for options

-------------------------------------------------------------------------------
single CBOR roundtrip
  sample.json
-------------------------------------------------------------------------------
/usr/ports/devel/nlohmann-json/work/json-3.1.2/test/src/unit-cbor.cpp:1376
...............................................................................

/usr/ports/devel/nlohmann-json/work/json-3.1.2/test/src/unit-cbor.cpp:1376: FAILED:
due to unexpected exception with message:
  [json.exception.parse_error.101] parse error at 1: syntax error - unexpected
  end of input; expected '[', '{', or a literal

===============================================================================
test cases:       5 |       4 passed | 1 failed
assertions: 1246600 | 1246599 passed | 1 failed

$ ./work/.build/test/test-msgpack 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test-msgpack is a Catch v1.9.7 host application.
Run with -? for options

-------------------------------------------------------------------------------
single MessagePack roundtrip
  sample.json
-------------------------------------------------------------------------------
/usr/ports/devel/nlohmann-json/work/json-3.1.2/test/src/unit-msgpack.cpp:1135
...............................................................................

/usr/ports/devel/nlohmann-json/work/json-3.1.2/test/src/unit-msgpack.cpp:1135: FAILED:
due to unexpected exception with message:
  [json.exception.parse_error.101] parse error at 1: syntax error - unexpected
  end of input; expected '[', '{', or a literal

===============================================================================
test cases:       2 |       1 passed | 1 failed
assertions: 1049596 | 1049595 passed | 1 failed


$ ./work/.build/test/test-ubjson
===============================================================================
All tests passed (591507 assertions in 3 test cases)

$ ./work/.build/test/test-unicode 
===============================================================================
All tests passed (351 assertions in 1 test case)

yurivict · 2018-03-31T21:58:29Z

I got these errors in the FreeBSD port with this patch applied: https://bz-attachments.freebsd.org/attachment.cgi?id=192018

nlohmann · 2018-04-01T14:06:00Z

The error message reads like an input file could not be found. Can you please make sure you ran the tests as described in https://github.com/nlohmann/json#execute-unit-tests ?

Viech · 2018-04-04T14:22:16Z

For me on Arch Linux on an Intel i7-3520M all tests pass but the last one just took an enormous amount of time:

78/78 Test #78: test-unicode_all ....................   Passed  114.18 sec

Maybe you need to wait a littler longer for it to complete?

nlohmann · 2018-04-04T18:30:49Z

Indeed the Unicode tests take the longest time. It's best to execute them with ctest -VV to see some progress output.

yurivict · 2018-04-04T18:43:26Z

I do out-of-source build, that is located in /usr/ports/devel/nlohmann-json/work/.build.
To run tests, I do:

cd /usr/ports/devel/nlohmann-json/work/.build &&
cmake -DJSON_BuildTests:BOOL=ON /source/path &&
gmake all &&
gmake test

Is this not the right way to run tests?

nlohmann · 2018-04-04T18:44:42Z

I'm not a Cmake expert - I am certain this works: https://github.com/nlohmann/json#execute-unit-tests

This is how we do it on Travis and AppVeyor.

yurivict · 2018-04-04T18:51:00Z

Exact same sequence of commands fail on 2 testcases:

56/78 Test #56: test-msgpack_all ....................***Exception: Child aborted  9.98 sec
56/78 Test #56: test-msgpack_all ....................***Exception: Child aborted  9.98 sec
Assertion failed: (m_parent), function moveToParent, file /usr/ports/devel/nlohmann-json/work/json-3.1.2/test/thirdparty/catch/catch.hpp, line 6222.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test-msgpack is a Catch v1.9.7 host application.
Run with -? for options

-------------------------------------------------------------------------------
MessagePack roundtrips
  input from msgpack-python
  output to output adapters
  std::vector<uint8_t>
  output to output adapters
-------------------------------------------------------------------------------
/usr/ports/devel/nlohmann-json/work/json-3.1.2/test/src/unit-msgpack.cpp:1377
...............................................................................

/usr/ports/devel/nlohmann-json/work/json-3.1.2/test/src/unit-msgpack.cpp:1377: FAILED:
  {Unknown expression after the reported line}
with expansion:

due to a fatal error condition:
  filename := "test/data/json_nlohmann_tests/all_unicode.json"
  filename := "test/data/json.org/1.json"
  SIGABRT - Abort (abnormal termination) signal

Assertion failed: (m_parent), function moveToParent, file /usr/ports/devel/nlohmann-json/work/json-3.1.2/test/thirdparty/catch/catch.hpp, line 6222.
      Start 74: test-ubjson_all
74/78 Test #74: test-ubjson_all .....................***Exception: Child aborted  3.00 sec
Assertion failed: (m_parent), function moveToParent, file /usr/ports/devel/nlohmann-json/work/json-3.1.2/test/thirdparty/catch/catch.hpp, line 6222.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test-ubjson is a Catch v1.9.7 host application.
Run with -? for options

-------------------------------------------------------------------------------
UBJSON roundtrips
  input from self-generated UBJSON files
  output to output adapters
  std::vector<uint8_t>
  output to output adapters
-------------------------------------------------------------------------------
/usr/ports/devel/nlohmann-json/work/json-3.1.2/test/src/unit-ubjson.cpp:2203
...............................................................................

/usr/ports/devel/nlohmann-json/work/json-3.1.2/test/src/unit-ubjson.cpp:2203: FAILED:
  {Unknown expression after the reported line}
with expansion:

due to a fatal error condition:
  filename := "test/data/json_nlohmann_tests/all_unicode.json"
  filename := "test/data/json.org/1.json"
  SIGABRT - Abort (abnormal termination) signal

Assertion failed: (m_parent), function moveToParent, file /usr/ports/devel/nlohmann-json/work/json-3.1.2/test/thirdparty/catch/catch.hpp, line 6222.

These might be FreeBSD-specific, not sure.

nlohmann · 2018-04-04T18:55:30Z

In all cases, this is an assertion inside the Catch code. This is not a library issue, but should be reported at https://github.com/catchorg/Catch2 - maybe they know what this failing assertion means.

nlohmann · 2018-04-05T05:08:34Z

Thanks for reporting!

nlohmann · 2018-04-05T15:50:25Z

I close this issue as this seems to be an issue in the intersection between Catch and FreeBSD.

sections, see also catchorg/Catch2#816 (comment) As a result, when built with gcc, loop iterations were skipped. When built with clang, the test aborted with an assertion in catch.hpp line 6222. This also addresses the issues discussed here: nlohmann#1032 (comment) and here: catchorg/Catch2#1241 Please note that this introduces new problems, as some of the unit tests fail now - the library stores keys in lexographical order, while the cbor/msgpack/ubjson examples store them in original order.

grembo · 2018-07-29T08:52:27Z

@nlohmann This is a bug in the unit test, that leads to aborts when built with clang, while it skips tests without crashing on gcc. I'll open a pull request.

yurivict · 2019-08-08T17:02:10Z

test-unicode_all still never finishes in version 3.7.0 on FreeBSD 12 amd64 (clang8).

nlohmann · 2019-08-08T18:29:22Z

This test performs millions of assertions. It will finish, it just takes a lot of time. Von meinem iPhone gesendet

…

Am 08.08.2019 um 19:02 schrieb ***@***.*** ***@***.***>: test-unicode_all still never finishes in version 3.7.0 on FreeBSD 12 amd64 (clang8). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

grembo · 2019-09-09T22:18:02Z

test-unicode_all still never finishes in version 3.7.0 on FreeBSD 12 amd64 (clang8).

@yurivict I can confirm @nlohmann's clarification:

[root@vm /usr/ports/devel/nlohmann-json]# time CXX=clang++80 make test
...
Label Time Summary:
all        = 978.95 sec*proc (41 tests)
default    =   4.36 sec*proc (41 tests)

Total Test time (real) = 990.70 sec

real    19m8.467s
user    15m31.744s
sys     5m58.867s
[root@vm /usr/ports/devel/nlohmann-json]#

nlohmann added the state: needs more info the author of the issue needs to provide more details label Mar 31, 2018

nlohmann removed the state: needs more info the author of the issue needs to provide more details label Apr 4, 2018

yurivict mentioned this issue Apr 4, 2018

Assertion fails during nlohmann/json tests catchorg/Catch2#1241

Closed

nlohmann closed this as completed Apr 5, 2018

grembo mentioned this issue Jul 29, 2018

Fix unit tests that were silently skipped or crashed (depending on the compiler) #1176

Merged

4 tasks

nickaein mentioned this issue Dec 29, 2019

Split test-unicode to avoid timeouts #1884

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Some testcases fail and one never finishes #1032

Some testcases fail and one never finishes #1032

yurivict commented Mar 31, 2018

nlohmann commented Mar 31, 2018

yurivict commented Mar 31, 2018

yurivict commented Mar 31, 2018

nlohmann commented Apr 1, 2018

Viech commented Apr 4, 2018 •

edited

Loading

nlohmann commented Apr 4, 2018

yurivict commented Apr 4, 2018

nlohmann commented Apr 4, 2018

yurivict commented Apr 4, 2018

nlohmann commented Apr 4, 2018

nlohmann commented Apr 5, 2018

nlohmann commented Apr 5, 2018

grembo commented Jul 29, 2018

yurivict commented Aug 8, 2019

nlohmann commented Aug 8, 2019 via email

grembo commented Sep 9, 2019

Some testcases fail and one never finishes #1032

Some testcases fail and one never finishes #1032

Comments

yurivict commented Mar 31, 2018

nlohmann commented Mar 31, 2018

yurivict commented Mar 31, 2018

yurivict commented Mar 31, 2018

nlohmann commented Apr 1, 2018

Viech commented Apr 4, 2018 • edited Loading

nlohmann commented Apr 4, 2018

yurivict commented Apr 4, 2018

nlohmann commented Apr 4, 2018

yurivict commented Apr 4, 2018

nlohmann commented Apr 4, 2018

nlohmann commented Apr 5, 2018

nlohmann commented Apr 5, 2018

grembo commented Jul 29, 2018

yurivict commented Aug 8, 2019

nlohmann commented Aug 8, 2019 via email

grembo commented Sep 9, 2019

Viech commented Apr 4, 2018 •

edited

Loading