Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

inconsistent (and sometimes weird) testing issues in CI #5

Open
paciorek opened this issue Jul 30, 2024 · 2 comments
Open

inconsistent (and sometimes weird) testing issues in CI #5

paciorek opened this issue Jul 30, 2024 · 2 comments

Comments

@paciorek
Copy link
Contributor

paciorek commented Jul 30, 2024

In CI, but not locally we are seeing:

  • Windows: with a PR-based run, we get a vague exit code at start of test-laplace. test-AGHQ is fine.
  • Windows: with a push-based run, we get an issue with too many nodes.
  • MacOS: we have tolerance issues and at least one NaN
  • Linux release: all fine
  • Linux devel: failure in installing pkgdown or some such setup step, unrelated to nimble.

Fortunately I can reproduce the MacOS tolerance issues on my M2 Mac. Given Daniel reported he could not reproduce them, I wonder if the difference is he is using an Intel or M1 Mac.

So for a next step, I can dig deeper into the tolerance issues by running on my Mac.

@paciorek
Copy link
Contributor Author

paciorek commented Aug 1, 2024

The MacOS numerical issues are all in the "nested random effects" test case and all appear to be from the optimization not converging to the true MLE (and disconcertingly reporting optimization success). I think this is another manifestation of our ongoing issues with Laplace optimization.

If I slightly tweak the starting values (from (2,2,2,2) to (2.01,2,2,2)), on my M2 Mac, the failures go away.

I am planning to look deeper into the general Laplace optimizer issues, and this will be a good test case to consider then.

For now so that we get testing to pass I am going to tweak the starting values for this test case to the values given above.
Fingers crossed that will address the MacOS failure. Not sure what will happen with Windows.

@paciorek
Copy link
Contributor Author

paciorek commented Aug 1, 2024

See PR #7 for more discussin of the Windows issue with the number of nodes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant