More Symbol Exports and C-API Example

[tknopp]

This is work in progress:

• Export more symbols in julia.h
• Add a convenience inline function "jl_apply_array_type"
• Add an example how to use the C API (see Document jlapi #3111 and https://github.com/tknopp/julia/blob/docuCAPI/doc/manual/embedding.rst)

On linux, this requires #4955 to be merged.

I am pretty excited how easy the C API is to use.

[tknopp]

Added some convenience functions to julia.h

• jl_call, jl_call0, and jl_call3 (jl_call1 and jl_call2 were already there)
• jl_get_function
• jl_add_new_closure
• jl_get_exception_str

[tknopp]

These are already exported in my "moreExport" branch. But maybe they are not
visible as the expmap has to be changed (see #4955
#4955 )

Jake Bolewski notifications@github.com hat am 2. Dezember 2013 um 15:50
geschrieben:

@tknopp https://github.com/tknopp can you export jl_set_global,
jl_set_const? These are needed to pass variables to Julia in the python ->
Julia bindings I have been working on, unless there is another way I'm not
seeing.

—
Reply to this email directly or view it on GitHub
#4997 (comment) .

[JeffBezanson]

Most of this is great. But, we don't really want to expose the JL_CALLABLE mechanism; in fact this might be removed entirely at some point. C functions should be called from julia via ccall, which has many advantages. Similarly jl_add_new_closure is not really necessary, and we don't want to encourage adding functions that way.

[tknopp]

@JeffBezanson: While it is certainly possible to call C functions only using ccall, I still think that it is great that we have the alternative way to define C functions and register them as Julia callable:

• ccall only works if you have a shared library. JL_CALLABLE does not have this restriction. In fact, I do not see how I can achieve what I have done in embedding.c without having to create a dedicated shared library for the callable C function
• The scenario where this is of interest is, if you use Julia as an embedded scripting language. Think of an audio player that is scriptable with Julia. Then you have to expose parts of your C-API to Julia (play, next track ...) and this is the main use case I thought of
• jl_add_new_closure is of course only there to make adding JL_CALLABLE functions convenient. If you don't want that, this is not necessary.

Is there some performance problem when using the JL_CALLABLE mechanism?

[tknopp]

@JeffBezanson: I see that my argument with ccall being limited to shared libraries is not the strongest one in practice. I have changed the example to use a ccall. This makes the example from my perspective a little bit more complex (two compilation units, how to define the path to the shared library) but it is ok.

If you approve this I will update the embedding docu that Tim has already merged to master.

[JeffBezanson]

Yes, it has several problems. With JL_CALLABLE you have to handle type checking and boxing/unboxing manually, while the compiler would like to be able to eliminate such operations. There is also no way to determine the julia return type of these functions. This API was designed for a very limited set of primitive functions whose type information is encoded manually in the compiler.

You can compile an executable so that it exports dynamic symbols that can be looked up by ccall. This is how most of the calls into the runtime system are done.

[tknopp]

Do you mean it is possible to directly make a ccall to an executable?

[tknopp]

Ok I see that it is possible and so simple and great. I will update the example.

[tknopp]

I have squashed the commits

[tknopp]

fixed jl_call and removed the WIP to indicate that this is finished

[tknopp]

rebased

[ihnorton]

Bump - probably needs another rebase? Would be nice to get this targeted for 0.3 if it's feasible... (I think we are starting to see more people who have an inverted/backend migration path)

[tknopp]

Yes, furthermore the embedding docu that is already in master has to be removed when this does not go in.

I had rebased this after the static compilation got in.

[jakebolewski]

Yes this would be great for 0.3. It makes the interop story so much better.

[tknopp]

@ihnorton @JeffBezanson I have rebased this.

[StefanKarpinski]

What, if anything, remains to be done before merging this? cc: @JeffBezanson

[tknopp]

I have rebased this recently. So I am happy once this gets in.

Probably the example should be automatically build. Currently its not.

[JeffBezanson]

Here the result of jl_box_long is not GC rooted while the new tuple is allocated.

[tknopp]

Can this be tagged 0.3 please? The docu is already in since some time and this needs either be in for 0.3 or the docu has to removed. I am a little sorry that this has already confused some people and kind of feel responsible for this.

[StefanKarpinski]

Yes, we should really get this into 0.3. We've now advertised a bunch of nice C APIs that we haven't yet integrated.