Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: bootstrap API reference docs generation #119

Merged
merged 12 commits into from
May 4, 2021
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,21 @@

## [Unreleased]

## [0.2.0-M1] - 05/07/2021

**WARNING: Beta releases may contain bugs and no guarantee is made about API stability. They are not recommended for production use!**

### Services new in this release

* S3
* NOTE: S3 is a complicated service, this initial release **DOES NOT** have complete support for all S3 features.
Comment on lines +11 to +12
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Can we note which features this release does not support? Or if it's easier, which features it does support?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we can articulate it well yet but we can try. Let's leave review of the CHANGELOG to another PR though. I started mucking with it here when I was testing some things. We'll have a "bump versions" PR before M1.


## Changes

* (feat): add conversions to/from `java.time.Instant` and SDK `Instant` (https://github.com/awslabs/smithy-kotlin/issues/271)
* (fix): generate per/service base exception types (https://github.com/awslabs/smithy-kotlin/issues/233)


## [0.1.0-M0] - 03/19/2021

**WARNING: Beta releases may contain bugs and no guarantee is made about API stability. They are not recommended for production use!**
Expand Down
39 changes: 28 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ See the [Getting Started Guide](docs/GettingStarted.md)
Generated sources are not checked into the repository, you first have to generate the clients before you can build them.


```
```sh
./gradlew :codegen:sdk:bootstrap
```

Expand All @@ -28,7 +28,7 @@ NOTE: To re-run codegen for the same set of services multiple times add the `--r
After generating the services you care about they are available to build:

e.g.
```
```sh
./gradlew :services:lambda:build
```

Expand All @@ -37,6 +37,32 @@ Where the task follows the pattern: `:services:SERVICE:build`

To see list of all projects run `./gradlew projects`

##### Generating a single service
See the local.properties definition above to specify this in a config file.

```
./gradlew -Paws.services=lambda :codegen:sdk:bootstrap
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: ```sh


##### Testing Locally
Testing generated services generally requires publishing artifacts (e.g. client-runtime) of `smithy-kotlin`, `aws-crt-kotlin`, and `aws-sdk-kotin` to maven local.

#### Generating API Documentation

API documentation is generated using [Dokka](http://kotlin.github.io/dokka) which is the official documentation tool maintained by JetBrains for documenting Kotlin code.

Unlike Java, Kotlin uses it's own [KDoc](https://kotlinlang.org/docs/kotlin-doc.html) format.


To generate API reference documentation for the AWS Kotlin SDK:


```sh
./gradlew --no-daemon --no-parallel dokkaHtmlMultiModule
```

This will output HTML formatted documentation to `build/dokka/htmlMultiModule`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion

Mention python3 -m http.server here for people wanting to view local content. I looked for a Kotlin equivalent but there is no one liner I could find.



### Build properties

Expand All @@ -55,12 +81,3 @@ aws.services=lambda
```


##### Generating a single service
See the local.properties definition above to specify this in a config file.

```
./gradlew -Paws.services=lambda :codegen:sdk:bootstrap
```

##### Testing Locally
Testing generated services generally requires publishing artifacts (e.g. client-runtime) of `smithy-kotlin`, `aws-crt-kotlin`, and `aws-sdk-kotin` to maven local.
61 changes: 60 additions & 1 deletion build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,73 @@
*/
plugins {
kotlin("jvm") version "1.4.31" apply false
id("org.jetbrains.dokka")
}

dependencies {
dokkaPlugin(project(":dokka-aws"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question

with the name is it your intent that this would be shared outside of the Kotlin SDK?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope just needed a name. I don't expect it to be published.

}

allprojects {
repositories {
mavenLocal()
mavenCentral()
// for dokka
maven("https://maven.pkg.jetbrains.space/public/p/kotlinx-html/maven") {
content {
includeGroup("org.jetbrains.kotlinx")
}
}
}

tasks.withType<org.jetbrains.dokka.gradle.DokkaTaskPartial>().configureEach {
// each module can include their own top-level module documentation
// see https://kotlinlang.org/docs/kotlin-doc.html#module-and-package-documentation
if (project.file("API.md").exists()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question

Do you see this mainly being used to describe APIs? Perhaps something like docs.md or supplemental-info.md would be more descriptive?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do, or rather only included in the generated API reference docs. This is because it has to follow a particular format for KDoc.

dokkaSourceSets.configureEach {
includes.from(project.file("API.md"))
}
}
}

tasks.withType<org.jetbrains.dokka.gradle.AbstractDokkaTask>().configureEach {
val sdkVersion: String by project
moduleVersion.set(sdkVersion)

val year = java.time.LocalDate.now().year
val pluginConfigMap = mapOf(
"org.jetbrains.dokka.base.DokkaBase" to """
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment

Maybe this JSON would be more maintainable as a file rather than embedded here? I see you resolve file paths though, maybe it wouldn't work with relative paths?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there was a lot of JSON then maybe but this is literally the only 3 things you can configure with this JSON.

{
"customStyleSheets": ["${rootProject.file("docs/api/css/logo-styles.css")}"],
"customAssets": [
"${rootProject.file("docs/api/assets/logo-icon.svg")}",
"${rootProject.file("docs/api/assets/aws_logo_white_59x35.png")}"
],
"footerMessage": "© $year, Amazon Web Services, Inc. or its affiliates. All rights reserved."
}
"""
)
pluginsMapConfiguration.set(pluginConfigMap)
}
}

// configure the root multimodule docs
tasks.dokkaHtmlMultiModule {
moduleName.set("AWS Kotlin SDK")

includes.from(
// NOTE: these get concatenated
rootProject.file("docs/api/README.md"),
rootProject.file("docs/GettingStarted.md")
)

val excludeFromDocumentation = listOf(
project(":client-runtime:testing"),
project(":client-runtime:crt-util")
)
removeChildTasks(excludeFromDocumentation)
}

val ktlint: Configuration by configurations.creating
val ktlintVersion: String by project
dependencies {
Expand All @@ -22,7 +80,8 @@ dependencies {
val lintPaths = listOf(
"codegen/smithy-aws-kotlin-codegen/**/*.kt",
"client-runtime/**/*.kt",
"examples/**/*.kt"
"examples/**/*.kt",
"dokka-aws/**/*.kt"
)

tasks.register<JavaExec>("ktlint") {
Expand Down
6 changes: 5 additions & 1 deletion client-runtime/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ description = "AWS client runtime support for generated service clients"

plugins {
kotlin("multiplatform")
id("org.jetbrains.dokka") version "1.4.20"
id("org.jetbrains.dokka")
jacoco
}

Expand Down Expand Up @@ -82,6 +82,10 @@ subprojects {
}

apply(from = rootProject.file("gradle/publish.gradle"))

dependencies {
dokkaPlugin(project(":dokka-aws"))
}
}

// FIXME - resolves build deadlock with aws-client-rt when using composite builds
Expand Down
12 changes: 4 additions & 8 deletions docs/GettingStarted.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# Beta Release Quickstart
## Beta Release Quickstart

Beta releases of the AWS Kotlin SDK are published as a complete maven local repository with all associated dependencies.


1. Download the [latest release](https://github.com/awslabs/aws-sdk-kotlin/releases) from Github
1. Download the [latest release](https://github.com/awslabs/aws-sdk-kotlin/releases) from GitHub

2. Unzip the repository somewhere on your local machine

```sh
> unzip aws-sdk-kotlin-0.1.0-M0.zip
> unzip aws-sdk-kotlin-0.2.0-M1.zip
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice you've already written the M1 quick start guide!

```

There should be a folder named `aws-sdk-kotlin-repo`
Expand Down Expand Up @@ -49,11 +49,9 @@ repositories {

4. Add services to your project

Services available for testing: `cognitoidentityprovider`, `dynamodb`, `kms`, `lambda`, `polly`, `secretsmanager`, `translate`

```kt

val awsKotlinSdkVersion = "0.1.0-M0"
val awsKotlinSdkVersion = "0.2.0-M1"
// OR put it in gradle.properties
// val awsKotlinSdkVersion: String by project

Expand All @@ -62,8 +60,6 @@ dependencies {
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.4.3")

// The following line adds a dependency on the dynamodb client.
// Services available in the M0 release:
// cognitoidentityprovider, dynamodb, kms, lambda, polly, secretsmanager, translate
implementation("aws.sdk.kotlin:dynamodb:$awsKotlinSdkVersion")

// The following will cause SDK logs to emit to the console:
Expand Down
3 changes: 3 additions & 0 deletions docs/api/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# AWS SDK for Kotlin API Reference

Welcome to the API documentation
Binary file added docs/api/assets/aws_logo_white_59x35.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/api/assets/favicon.ico
Binary file not shown.
15 changes: 15 additions & 0 deletions docs/api/assets/logo-icon.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
25 changes: 25 additions & 0 deletions docs/api/css/logo-styles.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
/*
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: Apache-2.0.
*/

#logo {
background-image: url('../images/aws_logo_white_59x35.png');
background-color: #232f3e;
color: #333;
background-size: 59px 35px;
}

body {
background-color: #f2f3f3;
}

/*
* override markdown samples which appear to have broken and/or minimal style
*/
.sample-container {
padding: 16px;
border-radius: 6px;
background-color: rgb(280, 280, 280);
margin: 12px 0 12px 0;
}
19 changes: 19 additions & 0 deletions dokka-aws/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
/*
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: Apache-2.0.
*/
plugins {
kotlin("jvm")
}

description = "Custom Dokka plugin for AWS Kotlin SDK API docs"

dependencies {
val dokkaVersion: String by project
compileOnly("org.jetbrains.dokka:dokka-base:$dokkaVersion")
compileOnly("org.jetbrains.dokka:dokka-core:$dokkaVersion")
}

tasks.withType<org.jetbrains.kotlin.gradle.tasks.KotlinCompile> {
kotlinOptions.jvmTarget = "1.8"
}
Comment on lines +17 to +19
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Can the JVM target version be read from the parent project?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be made into a variable but it honestly doesn't matter, this is more dictated by what works with the DokkaPlugin than choices we make for our own code we publish.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh good point. I mistakenly thought these should be unified.

Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
/*
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: Apache-2.0.
*/

package aws.sdk.kotlin.dokka

import aws.sdk.kotlin.dokka.transformers.FilterInternalApis
import org.jetbrains.dokka.base.DokkaBase
import org.jetbrains.dokka.plugability.DokkaPlugin

/**
* Dokka plugin for customizing the AWS Kotlin SDK generated API docs
*/
class AwsDokkaPlugin : DokkaPlugin() {
init {
println("AwsDokkaPlugin loaded!")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question

do you still need this or was part of getting things working initially?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both? So I was going to remove it but I had so many issues figuring out if it was actually being applied I think it's still useful (it gets printed out per/module when loaded).

}

val dokkaBase by lazy { plugin<DokkaBase>() }

val filterInternalApis by extending {
dokkaBase.preMergeDocumentableTransformer providing ::FilterInternalApis
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
/*
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: Apache-2.0.
*/

package aws.sdk.kotlin.dokka.transformers

import org.jetbrains.dokka.base.transformers.documentables.SuppressedByConditionDocumentableFilterTransformer
import org.jetbrains.dokka.model.*
import org.jetbrains.dokka.model.properties.WithExtraProperties
import org.jetbrains.dokka.plugability.DokkaContext

/**
* Filters out anything annotated with `InternalSdkApi`
*/
class FilterInternalApis(context: DokkaContext) : SuppressedByConditionDocumentableFilterTransformer(context) {
override fun shouldBeSuppressed(d: Documentable): Boolean {
val isInternal = when (d) {
is DClass -> d.isInternalSdk()
is DObject -> d.isInternalSdk()
is DTypeAlias -> d.isInternalSdk()
is DFunction -> d.isInternalSdk()
is DProperty -> d.isInternalSdk()
is DEnum -> d.isInternalSdk()
is DEnumEntry -> d.isInternalSdk()
is DTypeParameter -> d.isInternalSdk()
else -> false
}
Comment on lines +18 to +28
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Why separate when branches for each subclass? Do we not want to check for the internal annotation on some subclasses?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well the annotation itself only applies to certain types but also not every subclass of Documentable inherits the properties necessary to check if it has an annotation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that the annotation itself is visible in docs. Is there a reason to keep it?

http://localhost:8000/client-runtime/aws-client-rt/aws-client-rt/aws.sdk.kotlin.runtime/-internal-sdk-api/index.html

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh good point we can suppress it. I don't think it matters either way I was just trying to get most of the things people shouldn't be using or care about out of the docs.

if (isInternal) println("suppressing ${d.dri}")
return isInternal
}
}

fun <T> T.isInternalSdk() where T : WithExtraProperties<out Documentable> =
internalAnnotation != null

val <T> T.internalAnnotation where T : WithExtraProperties<out Documentable>
get() = extra[Annotations]?.let { annotations ->
annotations.directAnnotations.values.flatten().firstOrNull {
it.dri.toString() == "aws.sdk.kotlin.runtime/InternalSdkApi///PointingToDeclaration/"
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
aws.sdk.kotlin.dokka.AwsDokkaPlugin
5 changes: 5 additions & 0 deletions gradle.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
kotlin.code.style=official

# gradle
# FIXME - see https://github.com/Kotlin/dokka/issues/1405
org.gradle.jvmargs=-Xmx4g -XX:MaxMetaspaceSize=1G

# sdk
sdkVersion=0.2.0-SNAPSHOT

Expand All @@ -12,6 +16,7 @@ smithyKotlinVersion=0.2.0-SNAPSHOT
# kotlin
kotlinVersion=1.4.31
kotlin.native.ignoreDisabledTargets=true
dokkaVersion=1.4.32

# kotlin JVM
kotlinJVMTargetVersion=1.8
Expand Down
Loading