Configure writerside to include tables #324

koperagen · 2023-03-24T17:38:24Z

We now have a custom component for embedding tables. This feature is available on a custom branch of writerside, but still should reach our CI/CD before KotlinConf #233

Still need to figure out some details like displaying before / after, schema and maybe giving some visual clues to highlight what the operation does.

koperagen · 2023-04-13T18:12:49Z

I'll add documentation later as the process of adding tables to documentation should be refined. But the general idea is that you annotate a function and the compiler plugin adds "also { }" call after each "interesting" expression. That enables us to process samples (compose HTML outputs) without manually rewriting all samples using samplesTest and copySamplesOutputs. Created HTML are copied to docs/snippets, where writerside picks them up and embeds as iframes.
It does not affect a normal test task.
If a sample needs fine tuning, i think most straightforward way is to disable compiler transformation and create HTML output manually.

Jolanrensen

Very cool! Makes it easy to see intermediate dataframe states in documentation. However, we do need to consider the extra maintainability of another compiler plugin. Especially when we need to move to k2 eventually. It's difficult to understand most of the testing of the plugin as well as the code itself. But that's the case with most compiler plugins. That said, I do think it's worth it :)

docs/StardustDocs/cfg/resize-iframes.js

Jolanrensen · 2023-04-24T11:11:52Z

docs/StardustDocs/topics/convert.md

@@ -45,6 +46,7 @@ df.convert { name.firstName and name.lastName }.to { it.length() }
 df.convert { weight }.toFloat()
 ```

+<dataFrame src="org.jetbrains.kotlinx.dataframe.samples.api.Modify.convertTo.html"/>


Which of the convert functions is rendered here? or all?

core/src/test/kotlin/org/jetbrains/kotlinx/dataframe/explainer/PluginCallback.kt

Jolanrensen · 2023-04-24T11:17:32Z

core/src/test/kotlin/org/jetbrains/kotlinx/dataframe/explainer/PluginCallback.kt

+                0 -> {
+                    val table = convertToHTML(expression.df)
+                    val description = table.copy(
+                        body = """


same here with @language

plugins/expressions-converter/src/org/jetbrains/kotlinx/dataframe/ExplainerIrTransformer.kt

plugins/expressions-converter/build.gradle.kts

core/build.gradle.kts

…nts compared to the IDEA

this is a verbose variant of manual generation, should be improved at some point

Jolanrensen · 2023-04-26T13:24:21Z

settings.gradle.kts

@@ -32,3 +33,4 @@ pluginManagement {
    }
 }

+


another newline? haha

Jolanrensen · 2023-04-26T13:27:08Z

core/src/test/kotlin/org/jetbrains/kotlinx/dataframe/explainer/PluginCallbackProxy.kt

+        return data
+    }
+
+    var action: PluginCallback = PluginCallback { source, name, df, id, receiverId, containingClassFqName, containingFunName, statementIndex ->


Jolanrensen · 2023-04-26T13:29:31Z

plugins/expressions-converter/src/org/jetbrains/kotlinx/dataframe/ExplainerIrTransformer.kt

+                }
+            }
+            val alsoLambdaExpression = IrFunctionExpressionImpl(
+                -1,


missed some named arguments. A good linter could probably help with these... We really need to fix that soon :)

I don't think it's for a linter to decide :D I would add them, but generally, in my opinion, it's up for a debate at review

Jolanrensen · 2023-04-26T13:30:32Z

plugins/expressions-converter/src/org/jetbrains/kotlinx/dataframe/ExplainerIrTransformer.kt

+                type = pluginContext.irBuiltIns.functionN(2)
+                    .typeWith(listOf(expression.type, pluginContext.irBuiltIns.unitType)),
+                alsoLambda,
+                IrStatementOrigin.LAMBDA


same with trailing comma's. I think currently I'm the one adding them and you're the one leaving them out haha.

I believe this constructor is quite a stable API and won't require changing parameters order :p

You never know, especially with k2 around the corner. We had those incompatibility issues with korro because in 1.8.20 they suddenly added another parameter. Naming them at least helps a bit :) (Also for reviewing)

koperagen · 2023-04-26T14:37:19Z

@Jolanrensen Thank you!

koperagen changed the title ~~Configure writerside to include tables #233~~ Configure writerside to include tables Mar 24, 2023

koperagen force-pushed the documentation-tables branch 2 times, most recently from 8ace975 to 61a56ab Compare April 6, 2023 16:04

koperagen had a problem deploying to github-pages April 6, 2023 18:59 — with GitHub Actions Failure

koperagen temporarily deployed to github-pages April 6, 2023 19:03 — with GitHub Actions Inactive

koperagen temporarily deployed to github-pages April 6, 2023 19:07 — with GitHub Actions Inactive

koperagen force-pushed the documentation-tables branch 4 times, most recently from 95dade0 to e04e0cd Compare April 6, 2023 19:50

koperagen temporarily deployed to github-pages April 6, 2023 19:52 — with GitHub Actions Inactive

koperagen temporarily deployed to github-pages April 10, 2023 20:12 — with GitHub Actions Inactive

koperagen force-pushed the documentation-tables branch from 467cff3 to d1e4f88 Compare April 11, 2023 11:34

koperagen temporarily deployed to github-pages April 11, 2023 13:32 — with GitHub Actions Inactive

koperagen force-pushed the documentation-tables branch from d6ff368 to ccc19e3 Compare April 11, 2023 14:19

koperagen temporarily deployed to github-pages April 11, 2023 14:22 — with GitHub Actions Inactive

koperagen temporarily deployed to github-pages April 11, 2023 14:30 — with GitHub Actions Inactive

koperagen force-pushed the documentation-tables branch from ccc19e3 to 014fa67 Compare April 11, 2023 14:34

koperagen temporarily deployed to github-pages April 11, 2023 14:36 — with GitHub Actions Inactive

koperagen temporarily deployed to github-pages April 12, 2023 15:18 — with GitHub Actions Inactive

koperagen marked this pull request as ready for review April 13, 2023 17:35

koperagen force-pushed the documentation-tables branch from c92dc2b to 0fd4d4f Compare April 13, 2023 17:41

Jolanrensen self-requested a review April 17, 2023 10:20

Jolanrensen requested changes Apr 24, 2023

View reviewed changes

koperagen requested a review from Jolanrensen April 26, 2023 12:29

koperagen added 4 commits April 26, 2023 15:34

prototype explainer

cd0762e

disable import-ordering because it seems to have additional requireme…

149348e

…nts compared to the IDEA

process dataframe-like objects in call chains

6effa86

Configure writerside to include tables

3fef07e

koperagen added 24 commits April 26, 2023 15:34

cleanup js

6b18e39

update samples and generated sources

00642ed

fix escaping

f26a8c1

format code

64ae5b6

revert html escape changes

66ee93a

fix "split" step visualization

f28bd30

disable example that looks bad for now

66bcb51

fix escaping for embedding into iframe srcdoc

97229bc

disable remaining logging

e2921ef

fix linter and control output writing with env variable

6488b9d

remove irrelevant code

843a96a

update test

5179987

clean up

8775465

remove unused code / potential features

7f3ba71

stable "shuffle" test

fe6716e

this is a verbose variant of manual generation, should be improved at some point

rollback from korro dev for now

d8c4bc1

Add language references to DataFrameHtmlData

2ee3b8c

remove outdated code

179f687

remove unused code

791072c

fix some formatting issues

67cdf05

add linter

3be39fe

remove unused code

241bcd4

rename test task

ed8e69c

rework plugin contract

0485832

koperagen force-pushed the documentation-tables branch from 5838988 to 0485832 Compare April 26, 2023 12:37

koperagen mentioned this pull request Apr 26, 2023

Gradle plugin references fix #365

Merged

Jolanrensen approved these changes Apr 26, 2023

View reviewed changes

fix formatting

a19dd96

koperagen merged commit 6208e47 into master Apr 26, 2023

Configure writerside to include tables #324

Configure writerside to include tables #324

Uh oh!

Conversation

koperagen commented Mar 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

koperagen commented Apr 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Jolanrensen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

koperagen Apr 26, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

koperagen commented Apr 26, 2023

Uh oh!

Uh oh!

koperagen commented Mar 24, 2023 •

edited

Loading

koperagen commented Apr 13, 2023 •

edited

Loading

koperagen Apr 26, 2023 •

edited

Loading