Backend-specific source files

[FroMage]

We need to figure out how to have certain files only compiled for certain backends. It could be as simple as saying .java files for the JVM backend and .js files for the JS backend, but we may also want different Ceylon source files for different backends, as glue or something.

[tombentley]

One way to do it would be to have the source source directory hold pure ceylon, and have separate source-java (which could contain .ceylon and .java source files) and source-js (which could contain .ceylon and .js source files) source directories which are only included when compiling for those backends.

[FroMage]

I side with @tombentley here, except I suppose a more correct name would be ceylon-jvm. It can be implemented trivially. And people need it now.

[quintesse]

But do we want to add --src-jvm and --src-js options to all the commands (and the respective API functions everywhere we have things like getSrcDir())? Being able to separately specify those folders doesn't seem like a really necessary option.
Maybe it would be a better to use sub folders and be simply able to deduce them, _jvm and _js for example.

[FroMage]

@quintesse if we do backend-specific in the same source folder using subfolders like you propose then stuff will be in different packages, like com.foo and com.foo._jvm and you won't be able to import them without breaking one backend. Unless imports are also backend-specific.

I really think putting them in separate source folder is cleaner.

I don't think we need --src-jvm because if you define --src it overrides both sources and sources-jvm defaults.

[tombentley]

One aspect of using different directories would be that eclipse would show multiple source folders. That's annoying a lot of the time (I certainly find it annoying that half of ceylon.language is in runtime, and the rest of it is in src), and means you have to form the union of all the compilation units in your head.

[FroMage]

How many modules do you expect with backend-specific stuff, aside from the SDK and a few others?

[luolong]

How many modules do you expect with backend-specific stuff, aside from the SDK and a few others?

For JVM backend -- quite a few, if we're lucky :)

[tombentley]

The truth is that none of us can know that. What we do know is that the easier it is to write modules for multiple backends the more portable modules will be written.

[FroMage]

It's already trivial to write JVM-specific modules because the JVM compiler includes java files by default. It's only marginally harder to write multi-backend modules.

[quintesse]

then stuff will be in different packages, like com.foo and com.foo._jvm

Well the idea was to ignore that folder and treat the code in it as if it were in a directory one level higher. But I agree it might not be the best option either.

But if right now we are able to write:

- sources
  - foo
    - bar.java
    - baz.ceylon

and make it work, then it would be a shame that to make it work for multiple backends we have to do:

- jvm-sources
  - foo
    - baz.ceylon
- js-sources
  - foo
    - baz.ceylon
- sources
  - foo
    - bar.java
    - bar.js

because suddenly we have different Ceylon implementations and are forced to maintain them in different folders away from all the other files.

[quintesse]

Now hopefully with native we can prevent having to write different Ceylon implementations in most cases and we can still go for the first solution and need this only for very specific cases where native won't work.

Another thing I can think of (that doesn't seem very nice either) is:

- sources
  - foo
    - bar.java
    - bar.js
    - baz.jvm.ceylon
    - baz.js.ceylon

But at least all the files would be together.

[luolong]

Nice, I would like that.

To me the most comon story for native annotation is to implement the tiny bit of the glue code needed for interop with the native platform.

And this would serve my purpose perfectly!

[FroMage]

Tako: it would more look like:

- jvm-sources
  - foo
    - bar.java
    - baz.ceylon
- js-sources
  - foo
    - bar.js
    - baz.ceylon
- sources
  - foo
    - barUser.ceylon

And then again you only need the two baz.ceylon if you need a Ceylon glue. If you write JS and Java code that follows the Ceylon compilation mapping (like we do for all native stuff in the language module) you don't need that glue.

[FroMage]

native is orthogonal to this. Its only purpose is to give you a pretend Ceylon schema, which ATM is ignored.

[quintesse]

Stef, the only thing you changed is the location of the native files, which shouldn't matter. You still need to move the files away to a different folder than the code that's actually using it (in your case barUser.ceylon)
You're basically using a project structure with "overlays". And I'm wondering how we're going to handle that in the IDE for example (you'll suddenly have 2 of everything defined in those files).

[FroMage]

Well, I find it quite nice that I don't see JS files when I'm working on the Ceylon parts.

[luolong]

well, I think we are mashing together here 2 separate use cases for "native" code n ceylon project.

I am mostly interested in the case where I need to provide native implementation of a platform specific feature, while you are probably just talking about a module that wants to compile to both, JVM and JS backend, so that you could write both server side and client side code in Ceylon.

In my case, the Foo.js source file is actually a partial implementation of the Foo.ceylon class, in which case I really want it to be as close to the .ceylon file as possible. Specially, if I manage to keep the platform specific implementation as simple and lightweight as possible.

[FroMage]

@gavinking we're still waiting for your take on this.

[loicrouchon]

I don't really see why there would be specific ceylon sources for a particular backend.

If this is to avoid to include too many sources in a compilation to java or to js, I would rather use several modules:

• 1 common module
• 1 module per backend
  For example for a client-server application we would have common (compiled to both java / js), ceylon server (compiled to java only) and ceylon client (compiled to js only)

But maybe I'm missing something

Do you have use cases / examples of were this would be needed?

[FroMage]

Yes, when you write a Ceylon implementation class that interacts with native stuff. Interacting with a Java API in one case, and with dynamic JS stuff in the other case.

[quintesse]

But like @loicrouchon syas, in that case you could use 2 different modules (something we want to implement anyway). So we'd be talking about the case where we want to have 2 different Ceylon implementations, but they are so trivial that we don't want to turn them into separate modules.

So is it worth it to add these backend-specific source folders? (Remember that we'd still be able to have a single Ceylon file interfacing with 2 different native implementations using native)

[FroMage]

Well, it allows you to write trivial implementations that talk to different backend-specific stuff while writing only Ceylon and not a line of Java or JS. That appeals to me a lot.

[quintesse]

You mean without using any native implementation at all, just different behaviour depending on the backend? Hmmm okay, but that seems like something that you could trivially handle within a single source folder with a simple if-statement, which is something that at least won't confuse Eclipse.

[FroMage]

No you can't because you'd have to import backend-specific stuff and when compiling the code each backend compiler would not be able to typecheck the other backend-specific part.

[quintesse]

Import backend specific stuff? Meaning different module.ceylon files? Doesn't that just shout "backend specific modules" to you? Just imagine how that will fuck with the IDE, having to handle alternative module files and imports and such, I'm not even sure how to visualize that. We don't have any of those problems with backend-specific modules. Or am I seeing things to bleak?

[gavinking]

If you have a module that depends on backend-specific modules, it definitely needs backend-specific code to interact with those backend-specific modules.

But have you guys considered just using if for that?

if (process.isJava) {
    //use java-specific module
}
else if (process.isJavaScript) {
    //use javascript-specific module
}

Of course we could do something more typesafe than that, if you think it matters.

But this splitting stuff across multiple source dirs thing feels overengineered to me...

[quintesse]

Yes, you can do it like this for example (directly from my test module):

test.ceylon:

shared void run() {
    testNative();
}

native("jvm") void testNative() {
    myprintJvm(NativeCode().test());
}

native("js") void testNative() {
    dynamic {
        myprintJs(nativeCode());
    }
}

and then have:

NativeCode.java:

public class NativeCode {
    public String test() {
        return "This is a native Java class";
    }
}

and:

nativecode.js:

function nativeCode() {
    return "This is a native JavaScript function";
}

(you'll need to have the latest code though, I just fixed a bug related to this)

[FroMage]

Oh, so you can. How does the IDE behave in this case? I expect it not to see nativeCode from js since the JS model loader is not plugged in the IDE, but will it see the NativeCode class in Ceylon code? Will it not allow you to call it even outside native methods?

[quintesse]

The IDE hasn't changed behaviour for what was already there.
So you can just call NativeCode outside of native methods, you'll just not be able to enable the JS backend because you'll get errors.
The same with using dynamic and trying to enable the JVM backend.
But you can enable both backends and then put those calls/dynamic in native methods and everything should work.
I say "should" because right now the IDE does have a problem still with JVM code. I'm looking into that.

[quintesse]

Ok, the IDE problem is now fixed.