Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow user to supply a c++ header file #464

Merged
merged 5 commits into from
Oct 11, 2021
Merged

Allow user to supply a c++ header file #464

merged 5 commits into from
Oct 11, 2021

Conversation

WardBrian
Copy link
Member

Submission Checklist

  • Run unit tests
  • Declare copyright holder and open-source license: see below

Summary

Adds the ability to specify a .hpp file for external function definitions. Closes #463.

This requires storing some .hpp files in the test data, which overcomplicated the removal routine. I've switched to a global fixture which uses git clean -Xf data/ to remove all files in the git ignore in the test/data folder. This is only run once per session at the moment, at the end of all testing.

Copyright and Licensing

Please list the copyright holder for the work you are submitting (this will be you or your assignee, such as a university or company):
Simons Foundation

By submitting this pull request, the copyright holder is agreeing to license the submitted work under the following licenses:

@codecov-commenter
Copy link

codecov-commenter commented Oct 4, 2021

Codecov Report

Merging #464 (d90af13) into develop (acab458) will increase coverage by 0.10%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #464      +/-   ##
===========================================
+ Coverage    77.95%   78.06%   +0.10%     
===========================================
  Files           30       30              
  Lines         9168     9231      +63     
===========================================
+ Hits          7147     7206      +59     
- Misses        2021     2025       +4     
Impacted Files Coverage Δ
cmdstanpy/cmdstanpy/model.py 85.86% <0.00%> (-0.15%) ⬇️
...runner/work/cmdstanpy/cmdstanpy/cmdstanpy/model.py 85.86% <0.00%> (-0.15%) ⬇️
cmdstanpy/cmdstanpy/compiler_opts.py 93.51% <0.00%> (+0.18%) ⬆️
a/cmdstanpy/cmdstanpy/cmdstanpy/compiler_opts.py 93.51% <0.00%> (+0.18%) ⬆️
...ork/cmdstanpy/cmdstanpy/cmdstanpy/compiler_opts.py 93.51% <0.00%> (+0.18%) ⬆️
a/cmdstanpy/cmdstanpy/cmdstanpy/model.py 88.17% <0.00%> (+0.35%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update acab458...d90af13. Read the comment docs.

@WardBrian
Copy link
Member Author

@mitzimorris this is now failing with 2.28 and wasn’t before, need to investigate tomorrow but very odd

@mitzimorris
Copy link
Member

merges #461, this should clear up spurious failures

Copy link
Member

@mitzimorris mitzimorris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good and appreciate the global cleanup.

question for @rok-cesnovar and @jgabry - add similar functionality to CmdStanR?

cmdstanpy/compiler_opts.py Show resolved Hide resolved
@rok-cesnovar
Copy link
Member

rok-cesnovar commented Oct 8, 2021

question for @rok-cesnovar and @jgabry - add similar functionality to CmdStanR?

CmdStanR supports this already via the cpp_options, stanc_options:

library(cmdstanr)

stan_file <- "bernoulli_external.stan"

mod <- cmdstan_model(
  stan_file,
  cpp_options = list(USER_HEADER="/home/rok/Desktop/testing/external/make_odds.hpp"),
  stanc_options = list("allow-undefined")
)
data_list <- list(N = 10, y = c(0,1,0,0,0,0,0,0,0,1))
fit <- mod$sample(data = data_list)

The only "problem" we currently have is one needs to specify the full path to the .hpp file. And we need to make a vignette. Users keep asking on how to do this. Though the namespace thing in the external file is annoying -> stan-dev/stanc3#712

@mitzimorris
Copy link
Member

@WardBrian - should CmdStanPy do the same? do we need to introduce add'l argument user_header ?

question for @rok-cesnovar and @jgabry - add similar functionality to CmdStanR?

CmdStanR supports this already via the cpp_options, stanc_options:

@WardBrian
Copy link
Member Author

Having it separate is nice because it lets us allow relative paths and do some checking on those paths from within python. I didn't even think to try it the other way, but if we want we can just take the example notebook I wrote and adapt it to that style

@mitzimorris
Copy link
Member

it lets us allow relative paths and do some checking on those paths from within python.

that seems like a good enough reason to me. happy to add this argument.

@WardBrian
Copy link
Member Author

WardBrian commented Oct 11, 2021

I checked, and it does indeed work in the example I wrote up to replace

model_external.compile(user_header='make_odds.hpp')

with

model_external.compile(cpp_options={'USER_HEADER':'/home/brian/Dev/py/cmdstanpy/docsrc/examples/make_odds.hpp'}, stanc_options={'allow_undefined':True})

While I do think it is a good idea to have this as a separate argument, if only so we can do some path resolution, I'm going to rewrite it slightly so it uses self._cpp_options under the hood.

@mitzimorris
Copy link
Member

mitzimorris commented Oct 11, 2021

did you check in file test/data/return_one.hpp. ?

update: nevermind, my local cleanup script clobbered the .hpp file.

Copy link
Member

@mitzimorris mitzimorris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, approved, please fix the example notebook before merging.

Comment on lines +81 to +92
"We can provide a definition in a C++ header file by using the `user_header` argument to either the CmdStanModel constructor or the `compile` method. \n",
"\n",
"This will enables the `allow_undefined` flag automatically."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"model_external.compile(user_header='make_odds.hpp')"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update this to cpp_opts

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what you mean. The user interface is still to use model_header, this just gets turned into part of the CPP options during validation

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you're right - user error on my part - testing in wrong environment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FR] Support "USER_HEADER" CmdStan flag
4 participants