Enhance painless documentation #22720

gmoskovicz · 2017-01-20T16:06:34Z

We state that painless is similar than Groovy, but to be fair most of the code that you paste from groovy will need to be modified in order to work with this new language.

One simple example is the following that works in Groovy but not in Painless:

variable = list.sort { a, b -> a.value == b.value ? 0 : a.value < b.value ? -1 : 1 }

Now, this could be translated into the following:

variable = list.sort( (a, b) -> a.value == b.value ? 0 : a.value < b.value ? -1 : 1 );

Also, looks like joda.time.DateTime cannot be used, and there is no clear explanation how to deal with dates. See #22162.

As you can see, it's similar, but still you usually need to modify a big part of the code. Unfortunately this type of similarities are not documented, nor you can understand in an easy way how to translate groovy or how to develop with painless. My proposal is to:

Clearly explain in the documentation that while it's similar to groovy, there are a lot of differences which means that you will need to translate most of your code into painless.
Enhance the documentation with sections like "how to translate groovy to painless".
Add more sample docs around painless.
Add documentation on how the language works, what things we support, and what things we don't support.

We shouldn't expect to have a large documentation, but at least to have the general concepts. Just to see how other languages are explained:

C#: https://msdn.microsoft.com/en-us/library/67ef8sbd.aspx
Java: https://docs.oracle.com/javase/specs/jls/se8/html/index.html
Groovy http://groovy-lang.org/documentation.html (note the "differences with Java section")

The text was updated successfully, but these errors were encountered:

jasontedor · 2017-01-20T17:02:19Z

We state that painless is similar than Groovy, but to be fair most of the code that you paste from groovy will need to be modified in order to work with this new language.

To be fair, when developers hear that two languages are similar, they do not have an expectation that they can paste code from one language into another. For example, developers freely admit that C# and Java are similar (in fact, very similar), but no developer comfortable with one of those languages expects they can paste code back and forth between the two (sometimes you can paste a snippet, the vast majority of the time you can not).

This does not negate the point that we should always work to improve our documentation, especially for a feature as nascent as Painless is, but I do think it's worth correcting the expectation that developers have when they say and hear that two languages are similar.

jdconrad · 2017-01-20T17:32:35Z

Agree 100% the documentation needs improvements. I'm currently working on making some, but there's a lot to cover and it's going to take some time to get there. I also see how comparing this language to Groovy could be confusing given the lack of documentation here.

nik9000 · 2017-01-20T17:41:47Z

I'm still working on prerequisites for an API reference.... I think I have one review left before I can revive my work on the reference. That'll help. Some.

nik9000 · 2017-01-25T20:24:33Z

The API reference I was talking about: #22775

Adds "Appending B. Painless API Reference", a reference of all classes and methods available from Painless. Removes links to java packages because they contain methods that we don't expose and don't contain methods that we do expose (the ones in Augmentation). Instead this generates a list of every class and every exposed method using the same type information available to the interpreter/compiler/whatever-we-call-it. From there you can jump to the relevant docs. Right now you build all the asciidoc files by running ``` gradle generatePainlessApi ``` These files are expected to be committed because we build the docs without running `gradle`. Also changes the output of `Debug.explain` so that it is easy to search for the class in the generated reference documentation. You can also run it in an IDE safely if you pass the path to the directory in which to generate the docs as the first parameter. It'll blow away the entire directory an recreate it from scratch so be careful. And then you can build the docs by running something like: ``` ../docs/build_docs.pl --out ../built_docs/ --doc docs/reference/index.asciidoc --open ``` That is, if you have checked out https://github.com/elastic/docs in `../docs`. Wait a minute or two and your browser will pop open in with all of Elasticsearch's reference documentation. If you go to `http://localhost:8000/painless-api-reference.html` you can see this list. Or you can get there by following the links to `Modules` and `Scripting` and `Painless` and then clicking the link in the paragraphs below titled `Appendix B. Painless API Reference`. I like having these in asciidoc because we can deep link to them from the rest of the guide with constructs like `<<painless-api-reference-Object-hashCode-0>>` and `<<painless-api-reference->>` and we get link checking. Then the only brittle link maintenance bit is the link generation for javadoc. Which sucks. But I think it is important that we link to the methods directly so they are easy to find. Relates to #22720

PhaedrusTheGreek · 2017-01-31T18:08:33Z

My findings from working with painless a bit:

You can't declare global scope variables in painless, like you can in groovy. If you need to share variables, pass them in functions. (not documented)
All functions must be declared at the top of the script (documented)
In compile errors, Character number is thrown, not Line number. In vim you can do :goto <character> (not documented, existing issue)
in .vimrc , add this for syntax highlighting: au BufRead,BufNewFile *.painless set filetype=groovy (helpful hints?)
You can't use groovy shorthand for loop iteration (list.each {}), you must use the Java style syntax: for (Object it : list) { } (documented)

Attached is an example that will traverse each JSON document in order to fix invalid fields names. Can be attached to an ingest pipeline:

trim_dots.painless

jdconrad · 2017-01-31T18:10:18Z

@PhaedrusTheGreek Thank you for the feedback here! Appreciate it.

dakrone · 2017-01-31T18:15:03Z

I wouldn't want Emacs to be left out, so:

(add-to-list 'auto-mode-alist '("\\.painless$" . groovy-mode))

(requires groovy-mode to already be installed of course)

rjernst · 2017-01-31T18:28:50Z

in .vimrc , add this for syntax highlighting: au BufRead,BufNewFile *.painless set filetype=groovy
I wouldn't want Emacs to be left out

I would not do this. If we want syntax highlighting, then we should create our own filetype plugins for vim/emacs. Otherwise you will think eg a closure is valid, because of syntax highlighting, but painless does not have closures. Likewise, it would not properly format lambdas, since groovy does not support them.

nik9000 · 2017-01-31T18:29:30Z

You can't use groovy shorthand for loop iteration (list.each {}), you must use the Java style syntax: for (Object it : list) { } (documented)

I believe you can do it java style like list.stream.forEach(e -> {code}). Painless doesn't support groovy's funky rules around parameters.

dakrone · 2017-01-31T18:39:05Z

we should create our own filetype plugins for vim/emacs

Sure, I'll add this to my TODO list

jdconrad · 2017-01-31T21:23:26Z

@dakrone If/when you find the time to do this, maybe you can just pull from the existing grammar files to know what valid syntax is.

nik9000 · 2017-02-07T14:14:08Z

maybe you can just pull from the existing grammar files to know what valid syntax is.

Don't forget the semicolon hacks!

Painless uses Ruby-like method dispatch (reciever type, method name, and arity) rather than Java-like (reciever type, method name, and argument compile time types) or Groovy-like method dispatch (receiver type, method name, and argument run time types). We do this for mostly good reasons but we never documented it. Relates to elastic#22720

Painless uses Ruby-like method dispatch (reciever type, method name, and arity) rather than Java-like (reciever type, method name, and argument compile time types) or Groovy-like method dispatch (receiver type, method name, and argument run time types). We do this for mostly good reasons but we never documented it. Relates to #22720

Cinerar · 2017-02-10T22:14:16Z

@jdconrad Please find some time to answer StackOverFlow questions, right now it is very hard learning curve.

jdconrad · 2017-02-10T22:22:31Z

@Cinerar I'd be happy to. Do you have any particular questions or links to questions that you would like to me to prioritize?

gmoskovicz · 2017-02-10T22:24:56Z

@Cinerar it is more appropriate to open a new issue in http://discuss.elastic.co/ if you have a specific question. I am sure that @jdconrad or any other Engineer will be more than happy to help 😃 .

Cinerar · 2017-02-10T22:30:55Z

@jdconrad There are not so much of them right now.
http://stackoverflow.com/search?q=%5Belasticsearch%5D+painless

I'm interested in http://stackoverflow.com/questions/42009778/elasticsearch-painless-script-how-to-iterate-in-an-array-of-nested-objects

Thank you.

PhaedrusTheGreek · 2017-06-06T20:02:09Z

A few things

Semicolons are required, unlike groovy
In script file, as noted here, params are accessible via params.<your_var> instead of in the base scope.

jdconrad · 2019-04-26T22:43:56Z

Closing this in favor of (#23777)

gmoskovicz added the >docs General docs changes label Jan 20, 2017

clintongormley added the :Plugin Lang Painless label Jan 20, 2017

clintongormley assigned debadair and jdconrad Jan 20, 2017

nik9000 mentioned this issue Feb 7, 2017

Explain painless's method dispatch #23021

Merged

gmoskovicz mentioned this issue Mar 28, 2017

Painless and Working with Dates #23775

Closed

clintongormley added the :Core/Infra/Scripting Scripting abstractions, Painless, and Mustache label Feb 14, 2018

clintongormley removed the :Plugin Lang Painless label Feb 14, 2018

jdconrad closed this as completed Apr 26, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enhance painless documentation #22720

Enhance painless documentation #22720

gmoskovicz commented Jan 20, 2017

jasontedor commented Jan 20, 2017 •

edited

Loading

jdconrad commented Jan 20, 2017

nik9000 commented Jan 20, 2017

nik9000 commented Jan 25, 2017

PhaedrusTheGreek commented Jan 31, 2017 •

edited

Loading

jdconrad commented Jan 31, 2017

dakrone commented Jan 31, 2017

rjernst commented Jan 31, 2017

nik9000 commented Jan 31, 2017

dakrone commented Jan 31, 2017

jdconrad commented Jan 31, 2017

nik9000 commented Feb 7, 2017

Cinerar commented Feb 10, 2017

jdconrad commented Feb 10, 2017

gmoskovicz commented Feb 10, 2017

Cinerar commented Feb 10, 2017

PhaedrusTheGreek commented Jun 6, 2017

jdconrad commented Apr 26, 2019

Enhance painless documentation #22720

Enhance painless documentation #22720

Comments

gmoskovicz commented Jan 20, 2017

jasontedor commented Jan 20, 2017 • edited Loading

jdconrad commented Jan 20, 2017

nik9000 commented Jan 20, 2017

nik9000 commented Jan 25, 2017

PhaedrusTheGreek commented Jan 31, 2017 • edited Loading

jdconrad commented Jan 31, 2017

dakrone commented Jan 31, 2017

rjernst commented Jan 31, 2017

nik9000 commented Jan 31, 2017

dakrone commented Jan 31, 2017

jdconrad commented Jan 31, 2017

nik9000 commented Feb 7, 2017

Cinerar commented Feb 10, 2017

jdconrad commented Feb 10, 2017

gmoskovicz commented Feb 10, 2017

Cinerar commented Feb 10, 2017

PhaedrusTheGreek commented Jun 6, 2017

jdconrad commented Apr 26, 2019

jasontedor commented Jan 20, 2017 •

edited

Loading

PhaedrusTheGreek commented Jan 31, 2017 •

edited

Loading