Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runnableExamples: automatically test on all backends and report which work #360

Open
timotheecour opened this issue Nov 12, 2020 · 2 comments

Comments

@timotheecour
Copy link
Owner

refs: nim-lang#15903 (comment)

@ringabout
Copy link
Collaborator

But runnableExamples is mainly for test. If JS backend pass but C backend doesn't pass, should CI report this?(add flags may solve it)
Another advantage is increasing the testing time.

@timotheecour
Copy link
Owner Author

timotheecour commented Nov 24, 2020

the goal would be for docgen to report for each symbol whether it's supported on js,c,cpp,backend, without having to annotate those symbols with "proc bar is not available on VM or js backend" (or doing so at module level, which is not granular).

note that runnableExamples should primarily be used for documentation (that's guaranteed to stay in sync), and that independently of that tests should be in place to make sure tests pass on intended backends for that proc.

Another advantage is increasing the testing time.

yes; it would likely 2x or 3x the time to run the runnableExamples. but note that it hardly is a bottleneck now that runnableExamples are joined (see benchmark nim-lang#9300 (comment) or the relative times of github CI doc pipeline vs azure non-doc test pipelines)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants