Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unstable TestRDocGeneratorJsonIndex#test_generate test #1048

Open
junaruga opened this issue Oct 30, 2023 · 5 comments
Open

Unstable TestRDocGeneratorJsonIndex#test_generate test #1048

junaruga opened this issue Oct 30, 2023 · 5 comments

Comments

@junaruga
Copy link
Member

I am seeing the following failure that happens randomly on Travis Ubuntu jammy ppc64le environment in ruby/ruby. The latest failure is at the relatively recent commit ruby/ruby@1d51e4c .

https://app.travis-ci.com/github/ruby/ruby/jobs/612435804#L2936

  1) Failure:
TestRDocGeneratorJsonIndex#test_generate [/home/travis/build/ruby/ruby/test/rdoc/test_rdoc_generator_json_index.rb:108]:
.js files should be the same timestamp of original

The test is built and executed with the following steps.

+ export NPROC="$(nproc)"
+ export JOBS=-j$((1+${NPROC}))
+ export RUBY_TESTOPTS="$JOBS -q --tty=no"
+ export optflags=-O1
+ export debugflags=-ggdb3
+ export CC=gcc-11
+ gcc-11 --version
gcc-11 (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ ../tool/travis_wait.sh $SETARCH make -s test-all RUBYOPT="-w"
...
Run options: 
  --seed=2200
  "--ruby=./miniruby -I../lib -I. -I.ext/common  ../tool/runruby.rb --extout=.ext  -- --disable-gems"
  --excludes-dir=../test/.excludes
  --name=!/memory_leak/
  -j3
  -q
  --tty=no
@junaruga
Copy link
Member Author

Another error log after removing the optflags=-O1 in .travis.yml. So, the optflags=-O1 is not criteria to make this issue happen.
https://app.travis-ci.com/github/ruby/ruby/jobs/612440144

@junaruga
Copy link
Member Author

I was able to capture the failing test, and compared values.

https://app.travis-ci.com/github/ruby/ruby/jobs/612446128#L2936

  1) Failure:
TestRDocGeneratorJsonIndex#test_generate [/home/travis/build/ruby/ruby/test/rdoc/test_rdoc_generator_json_index.rb:108]:
.js files should be the same timestamp of original.
<"2023-10-30 17:17:00.869180353 +0000"> expected but was
<"2023-10-30 17:17:00 +0000">.

@junaruga
Copy link
Member Author

I just tested the following script on RubyCI ppc64le server.

$ cat test.rb 
require 'pathname'

f = Pathname('/tmp/a.txt')
p f.mtime.inspect
$ touch /tmp/a.txt
$ ruby test.rb 
"2023-10-30 18:30:57.305517101 +0000"

@junaruga
Copy link
Member Author

junaruga commented Nov 1, 2023

I was able to capture the failing test, and compared values.

https://app.travis-ci.com/github/ruby/ruby/jobs/612446128#L2936

  1) Failure:
TestRDocGeneratorJsonIndex#test_generate [/home/travis/build/ruby/ruby/test/rdoc/test_rdoc_generator_json_index.rb:108]:
.js files should be the same timestamp of original.
<"2023-10-30 17:17:00.869180353 +0000"> expected but was
<"2023-10-30 17:17:00 +0000">.

Here is just another failing case with the values.
https://app.travis-ci.com/github/ruby/ruby/jobs/612583543#L2936

  1) Failure:
TestRDocGeneratorJsonIndex#test_generate [/home/travis/build/ruby/ruby/test/rdoc/test_rdoc_generator_json_index.rb:108]:
.js files should be the same timestamp of original.
<"2023-11-01 15:38:55.065902574 +0000"> expected but was
<"2023-11-01 15:38:55 +0000">.

@junaruga
Copy link
Member Author

junaruga commented Nov 3, 2023

I tried to reproduce this issue on RubyCI ppc64le Ubuntu jammy server for the Ruby ruby/ruby@ad4f973. However, I was not able to reproduce the issue.

test_repeat.sh

#!/bin/bash

set -eux

for i in $(seq 100); do
  echo "=== ${i}"
  RUBY_TESTOPTS="-j8 -q" make test-all TESTS="test/rdoc/test_rdoc_generator_json_index.rb"
done
$ nproc
8

$ nohup /path/to/test_repeat.sh >& test.log &

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant