Create modules in Java #9422

eunomie · 2025-01-21T11:04:40Z

This PR allows to create Dagger modules using Java.

$ dagger init --sdk=java my-java-module

$ tree my-java-module
my-java-module
├── dagger.json
├── pom.xml
└── src
    └── main
        └── java
            └── io
                └── dagger
                    └── sample
                        └── module
                            ├── MyJavaModule.java
                            └── package-info.java

8 directories, 4 files

This will create the module with a default template. This template module is based on the same than Go and others, with the container-echo and grep-dir functions.

A default pom.xml is also generated.

Once done, it's possible to list the functions or call them:

$ dagger functions -m my-java-module

Name             Description
container-echo   Returns a container that echoes whatever string argument is provided
grep-dir         Returns lines that match a pattern in the files of the provided Directory

$ dagger call -q -m my-java-module container-echo "hello dagger" stdout

hello dagger

The java module code is using annotations to know what to expose and call:

@Module
@Object
@Function

The entrypoint code will be generated at build time. In that way it's close to the Go modules (not a generic endpoint at runtime only).

The default pom.xml set the main class to the generated entrypoint one automatically using the maven-shade plugin.

By default only the src directory and the pom.xml are necessary. But to keep a nice Developer Experience and help IDEs to complete code, all the Dagger SDK dependencies (the Dagger java sdk plus all the generated files for the different types and the entrypoint) can be generated and made available under the target/generated-sources directory:

dagger: the generated files based on Dagger introspection JSON
entrypoint: the generated entrypoint based on user's code
sdk: the sdk/java/dagger-java-sdk source code (this is not really generated sources but a way to make it available easily)

generated files

$ tree target my-java-module/target └── generated-sources ├── dagger │ └── io │ └── dagger │ └── client │ ├── BuildArg.java │ ├── CacheSharingMode.java │ ├── CacheVolume.java │ ├── CacheVolumeID.java │ ├── Client.java │ ├── Container.java │ ├── ContainerID.java │ ├── CurrentModule.java │ ├── CurrentModuleID.java │ ├── Directory.java │ ├── DirectoryID.java │ ├── EnumTypeDef.java │ ├── EnumTypeDefID.java │ ├── EnumValueTypeDef.java │ ├── EnumValueTypeDefID.java │ ├── EnvVariable.java │ ├── EnvVariableID.java │ ├── Error.java │ ├── ErrorID.java │ ├── FieldTypeDef.java │ ├── FieldTypeDefID.java │ ├── File.java │ ├── FileID.java │ ├── Function.java │ ├── FunctionArg.java │ ├── FunctionArgID.java │ ├── FunctionCall.java │ ├── FunctionCallArgValue.java │ ├── FunctionCallArgValueID.java │ ├── FunctionCallID.java │ ├── FunctionID.java │ ├── GeneratedCode.java │ ├── GeneratedCodeID.java │ ├── GitModuleSource.java │ ├── GitModuleSourceID.java │ ├── GitRef.java │ ├── GitRefID.java │ ├── GitRepository.java │ ├── GitRepositoryID.java │ ├── ImageLayerCompression.java │ ├── ImageMediaTypes.java │ ├── InputTypeDef.java │ ├── InputTypeDefID.java │ ├── InterfaceTypeDef.java │ ├── InterfaceTypeDefID.java │ ├── JSON.java │ ├── Label.java │ ├── LabelID.java │ ├── ListTypeDef.java │ ├── ListTypeDefID.java │ ├── LocalModuleSource.java │ ├── LocalModuleSourceID.java │ ├── Module.java │ ├── ModuleDependency.java │ ├── ModuleDependencyID.java │ ├── ModuleID.java │ ├── ModuleSource.java │ ├── ModuleSourceID.java │ ├── ModuleSourceKind.java │ ├── ModuleSourceView.java │ ├── ModuleSourceViewID.java │ ├── NetworkProtocol.java │ ├── ObjectTypeDef.java │ ├── ObjectTypeDefID.java │ ├── PipelineLabel.java │ ├── Platform.java │ ├── Port.java │ ├── PortForward.java │ ├── PortID.java │ ├── ReturnType.java │ ├── ScalarTypeDef.java │ ├── ScalarTypeDefID.java │ ├── Secret.java │ ├── SecretID.java │ ├── Service.java │ ├── ServiceID.java │ ├── Socket.java │ ├── SocketID.java │ ├── SourceMap.java │ ├── SourceMapID.java │ ├── Terminal.java │ ├── TerminalID.java │ ├── TypeDef.java │ ├── TypeDefID.java │ ├── TypeDefKind.java │ ├── Version.java │ └── Void.java ├── entrypoint │ └── io │ └── dagger │ └── gen │ └── entrypoint │ └── Entrypoint.java └── sdk └── io └── dagger ├── client │ ├── Arguments.java │ ├── Convert.java │ ├── Dagger.java │ ├── DaggerQueryException.java │ ├── IDAble.java │ ├── InputValue.java │ ├── PrivateVisibilityStrategy.java │ ├── QueryBuilder.java │ ├── QueryPart.java │ ├── Scalar.java │ └── engineconn │ ├── CLIDownloader.java │ ├── CLIRunner.java │ ├── ConnectParams.java │ ├── Connection.java │ ├── FileFetcher.java │ └── Provisioning.java └── module ├── Base.java ├── annotation │ ├── Function.java │ ├── Module.java │ └── Object.java └── info ├── FunctionInfo.java ├── ModuleInfo.java ├── ObjectInfo.java └── ParameterInfo.java

19 directories, 112 files

Those files are not intended to be added to git. To re-generate them: dagger develop

[!WARNING]
Due to some issues between Docker Desktop, Java and MacBook Pro m4 (it looks like other machines are not impacted) in case of arm64 platform a specific flag is added to Java/Maven: -XX:UseSVE=0.
See for instance corretto/corretto-21#85

...-processor/src/main/java/io/dagger/annotation/processor/DaggerModuleAnnotationProcessor.java

sdk/java/dagger-java-sdk/src/main/java/io/dagger/module/Base.java

...-processor/src/main/java/io/dagger/annotation/processor/DaggerModuleAnnotationProcessor.java

sdk/java/dagger-java-sdk/src/main/java/io/dagger/client/Convert.java

eunomie · 2025-01-22T15:34:30Z

I improved the primitive types support, they are both working for returning types and arguments.
I also included your (@jcsirot ) improvements regarding javadoc parsing.

...-processor/src/main/java/io/dagger/annotation/processor/DaggerModuleAnnotationProcessor.java

.dagger/build/builder.go

jedevc

I think this looks like a good start to me, just left some general integration points as comments.

Really not qualified to comment on any of the Java code - will trust you and @jcsirot on this 🎉

Would also like @helderco to take a look if possible, since he's more familiar with the process of how we've integrated with the other SDKs 🎉

sdk/java/runtime/main.go

jedevc · 2025-01-28T15:42:30Z

sdk/java/runtime/main.go

+		// - dagger-codegen-maven-plugin: this plugin will be used to generate the SDK code, from the introspection file,
+		//   this means including the ability to call other projects (not part of the main dagger SDK)
+		//   - this plugin is only used to build the SDK, the user module doesn't need it
+		// - dagger-java-annotation-processor: this will read dagger specific annotations (@Module, @Object, @Function)
+		//   and generate the entrypoint to register the module and invoke the functions


I like this separation - I think this is actually the way we'd like to get all the SDKs to work eventually (see #7707).

...dagger-java-annotation-processor/src/test/resources/io/dagger/gen/entrypoint/entrypoint.java

sdk/java/README.md

core/schema/sdk.go

sdk/java/dagger-codegen-maven-plugin/src/main/resources/schemas/schema-0.15.1.json

...-processor/src/main/java/io/dagger/annotation/processor/DaggerModuleAnnotationProcessor.java

sdk/java/runtime/main.go

sdk/java/runtime/template/src/main/java/io/dagger/sample/module/DaggerModule.java

sdk/java/runtime/main.go

sdk/java/runtime/template/pom.xml

sdk/java/runtime/main.go

core/schema/sdk.go

Like assert from golangci-lint module Signed-off-by: Yves Brissaud <gh@lgtd.io>