Document custom derive. #38770

steveklabnik · 2017-01-02T01:37:21Z

These are some bare-bones documentation for custom derive, needed
to stabilize "macros 1.1",
#35900

The book chapter is based off of a blog post by @cbreeden,
https://cbreeden.github.io/Macros11/

Normally, we have a policy of not mentioning external crates in
documentation. However, given that syn/quote are basically neccesary
for properly using macros 1.1, I feel that not including them here
would make the documentation very bad. So the rules should be bent
in this instance.

So far, this PR includes only docs; @alexcrichton said in #35900 that he'd be okay with landing them before stabilization; I don't mind either way.

rust-highfive · 2017-01-02T01:37:25Z

@steveklabnik: no appropriate reviewer found, use r? to override

steveklabnik · 2017-01-02T01:37:43Z

/cc @rust-lang/docs @rust-lang/lang

est31 · 2017-01-02T01:41:52Z

Would it make sense to document it in the reference as well? Derive is documented there too.

steveklabnik · 2017-01-02T01:47:07Z

Would it make sense to document it in the reference as well? Derive is documented there too.

Ah ha! I knew I forgot something. Yes.

killercup

Nice. Added a few inline comments with suggestions :)

killercup · 2017-01-02T06:05:31Z

src/doc/book/procedural-macros.md

Shouldn't that be src/main.rs?

killercup · 2017-01-02T06:06:55Z

src/doc/book/procedural-macros.md

Maybe add a sentence about naming convention? I've seen a bunch of crates that call these <name>_derive.

killercup · 2017-01-02T06:08:02Z

src/doc/book/procedural-macros.md

Link "syn" and "quote" to their repositories or crates.io pages?

killercup · 2017-01-02T06:08:39Z

src/doc/book/procedural-macros.md

People who don't know nom will be confused by this.

killercup · 2017-01-02T06:10:27Z

src/doc/book/procedural-macros.md

Lowercase "rust", same a few times below

killercup · 2017-01-02T06:12:04Z

src/doc/book/procedural-macros.md

Double space after sentence here and a few times below; I don't think the rest of the book does this.

killercup · 2017-01-02T06:12:52Z

src/doc/book/procedural-macros.md

Put "String" in back ticks?

killercup · 2017-01-02T06:14:03Z

src/doc/book/procedural-macros.md

Hm, maybe "serializing rust code" → "generating Rust code"?

killercup · 2017-01-02T06:17:17Z

src/doc/book/procedural-macros.md

Error handling probably deserves a short paragraph as well, even if it's just "panicking the proc macro code will show it as a compiler error pointing to the derive".

withoutboats · 2017-01-02T06:25:48Z

How do you feel about making HelloWorld into a trait? Implementing traits is intended as the normal use case for derive, and it seems odd to document it with a non-trait impl (though of course that does work).

steveklabnik · 2017-01-02T16:59:41Z

Okay! I think I've fixed up everything, @withoutboats @killercup . Thanks for the review 😄

Still have to update the reference, should push that in a minute.

killercup · 2017-01-02T17:03:04Z

src/doc/book/procedural-macros.md

FYI this path uses backslashes, the rest of the book probably not

frewsxcv · 2017-01-02T17:41:51Z

src/doc/reference.md

It's a little weird to me that ## Macros is a subheading within # Macros, but I see why you did it

yeah i would like to re-do this and other parts of the reference but i'm trying to be minimally invasive here

frewsxcv · 2017-01-02T17:44:35Z

src/doc/reference.md

Earlier in this section, "derive" is styled as

`derive`

with backticks, so might be good to be consistent.

frewsxcv · 2017-01-02T17:46:02Z

src/doc/reference.md

s/t will/it will/

It looks like crate types are styled with backticks, so also:

s/proc-macro/`proc-macro`/

frewsxcv · 2017-01-02T17:48:08Z

src/doc/reference.md

s/rustc-macro/`rustc-macro`/

nikomatsakis

This seemed like a decent summary. The one thing I would say is that I would have expected a bit more up-front material discussing the 'basic model' for derive (string in, string out, you do some parsing and serialization in the model), but that's because I tend towards more of a "top-down" style, I think, whereas this is kind of "bottom-up". (i.e., here is the code to write, let's explain why you wrote it) In any case, all the important stuff seems to be there, and it reads pretty smoothly.

I left some nits.

nikomatsakis · 2017-01-02T19:49:42Z

src/doc/book/procedural-macros.md

typo: procedural

nikomatsakis · 2017-01-02T19:54:20Z

src/doc/book/procedural-macros.md

Nit: s/repition/repetition/

nikomatsakis · 2017-01-02T19:55:50Z

src/doc/reference.md

Nit: s/Procedrual/Procedural/

nikomatsakis · 2017-01-02T19:56:27Z

src/doc/book/procedural-macros.md

Nit: s/Procedrual/Procedural/

i am... really bad at this spelling apparently, haha

nikomatsakis · 2017-01-02T19:58:44Z

src/doc/book/procedural-macros.md

This stream of consciousness bit felt a little over the top to me, but it's fine. =)

steveklabnik · 2017-01-02T20:32:25Z

The one thing I would say is that I would have expected a bit more up-front material discussing the 'basic model' for derive

I'm not totally opposed to this; I was thinking about making it future proof. That is, right now it's string in string out because it's a hack. It's really TokenStream -> String -> process -> String -> TokenStream, and ideally those string bits are going away. But I guess it will be a long time...

chris-morgan · 2017-01-03T03:28:11Z

src/doc/book/SUMMARY.md

I see we’re continuing the tradition of inconsistent capitalisation—nay, taking it to a new level, with mixed title and sentence case all in one line!

Yeah "Derive" felt like something that deserved a capital here, maybe not, idk

F001 · 2017-01-03T03:45:08Z

Thanks for writing the doc. I can get the basic concept of proc-macro now, but still haunted by some questions.

Is it possible to write this macro without any third party dependency? What if we create it from scratch? What kind of benefit we can get from syn&quote crate?
Why does TokenStream type exist, if the only purpose of it is to be converted to a String? It is weird to convert a structured type to a String, and then parse. I would like to see more explanation about what kind of API does the proc-macro feature provide for us, not the API of a third party lib.

withoutboats · 2017-01-03T06:19:07Z

src/doc/reference.md

We should probably continue to document compiler plugins in the reference for the time being; they still exist & will for some time, and of course some project are heavily reliant on them.

This isn't removing documentation; it's re-ordering it. I thought putting macros by example first was a better way of doing things.

It looks like it went from Compiler plugins, Macros to Macros, Procedural Macros, shouldn't it be Macros, Procedural Macros, Compiler plugins?

My understanding is that procedural macros are compiler plugins. That is, today "compiler plugins" are "using libsyntax to extend the compiler" but that's never going to be stable, only the new interface we're calling "procedural macros"

The names of all of this stuff has been very hard.

That's correct, but we've implemented (and now stabilized) the new procedural macros interface only for controlling derive macros. Users who want to define custom attributes or function style procedural macros (without using a hack) are still using the never-to-be-stabilized compiler plugin interface.

Until we've fully implemented procedural macros, compiler plugins will still be used by these users and so we shouldn't drop whatever docs we have on it (even if we don't focus any attention on improving it).

I didn't actually remove any of those docs generally, just the small description here. Let me fix it.

parir · 2017-01-03T12:57:54Z

src/doc/book/procedural-macros.md

s/crated/created/

steveklabnik · 2017-01-03T15:54:30Z

@F001

Is it possible to write this macro without any third party dependency? What if we create it from scratch? What kind of benefit we can get from syn&quote crate?

Sure, but it's much, much, much harder. The benefits are that syn and quote do parsing and generation for you.

Why does TokenStream type exist, if the only purpose of it is to be converted to a String? It is weird to convert a structured type to a String, and then parse.

This is the difference between macros 1.1 and macros 2.0, in a nutshell.

I would like to see more explanation about what kind of API does the proc-macro feature provide for us, not the API of a third party lib.

This is the entire API. That's it. There's nothing else to explain. It is extremely bare-bones.

steveklabnik · 2017-01-03T15:59:00Z

Okay, I believe that I've addressed everyone's comments now, both on GitHub and in the text. I've squashed, since this is going to need to be backported, and want to make that easy.

nikomatsakis · 2017-01-03T18:16:48Z

Let's do it.

@bors r+

bors · 2017-01-03T18:16:50Z

📌 Commit eb2c83f has been approved by nikomatsakis

nikomatsakis · 2017-01-03T18:16:53Z

@bors rollup

steveklabnik · 2017-01-03T22:17:06Z

@bors: r=nikomatsakis rollup (#38770 (comment))

bors · 2017-01-03T22:17:08Z

📌 Commit c0efdbf has been approved by nikomatsakis

@cbreeden

…ikomatsakis Document custom derive. These are some bare-bones documentation for custom derive, needed to stabilize "macros 1.1", rust-lang#35900 The book chapter is based off of a blog post by @cbreeden, https://cbreeden.github.io/Macros11/ Normally, we have a policy of not mentioning external crates in documentation. However, given that syn/quote are basically neccesary for properly using macros 1.1, I feel that not including them here would make the documentation very bad. So the rules should be bent in this instance. So far, this PR includes only docs; @alexcrichton said in rust-lang#35900 that he'd be okay with landing them before stabilization; I don't mind either way.

alexcrichton · 2017-01-04T22:01:44Z

@bors: rollup- p=1

Let's land this quickly so we can backport

bors · 2017-01-04T22:08:06Z

🔒 Merge conflict

bors · 2017-01-04T22:11:06Z

☔ The latest upstream changes (presumably #38783) made this pull request unmergeable. Please resolve the merge conflicts.

@cbreeden

These are some bare-bones documentation for custom derive, needed to stabilize "macros 1.1", rust-lang#35900 The book chapter is based off of a blog post by @cbreeden, https://cbreeden.github.io/Macros11/ Normally, we have a policy of not mentioning external crates in documentation. However, given that syn/quote are basically neccesary for properly using macros 1.1, I feel that not including them here would make the documentation very bad. So the rules should be bent in this instance.

steveklabnik · 2017-01-04T22:35:06Z

@bors: r=alexcrichton p=1

bors · 2017-01-04T22:35:07Z

📌 Commit 3075c1f has been approved by alexcrichton

sgrif · 2017-01-04T22:48:20Z

src/doc/book/procedural-macros.md

+moment, procedural macros need to be in their own crate. Eventually, this
+restriction may be lifted, but for now, it's required. As such, there's a
+convention; for a crate named `foo`, a custom derive procedural macro is called
+`foo-derive`. Let's start a new crate called `hello-world-derive` inside our


I think the convention is foo_derive not foo-derive.

package names are supposed to prefer - over _.

bors · 2017-01-04T23:04:55Z

⌛ Testing commit 3075c1f with merge 8b9bc29...

bors · 2017-01-04T23:08:51Z

💔 Test failed - status-travis

steveklabnik · 2017-01-04T23:21:10Z

@bors: retry

(travis is spurious)

steveklabnik · 2017-01-04T23:23:45Z

@bors: force

bors · 2017-01-05T00:11:18Z

⌛ Testing commit 3075c1f with merge 5d994d8...

@cbreeden

Document custom derive. These are some bare-bones documentation for custom derive, needed to stabilize "macros 1.1", #35900 The book chapter is based off of a blog post by @cbreeden, https://cbreeden.github.io/Macros11/ Normally, we have a policy of not mentioning external crates in documentation. However, given that syn/quote are basically neccesary for properly using macros 1.1, I feel that not including them here would make the documentation very bad. So the rules should be bent in this instance. So far, this PR includes only docs; @alexcrichton said in #35900 that he'd be okay with landing them before stabilization; I don't mind either way.

bors · 2017-01-05T02:10:38Z

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing 5d994d8 to master...

killercup reviewed Jan 2, 2017

View reviewed changes

killercup approved these changes Jan 2, 2017

View reviewed changes

src/doc/book/procedural-macros.md Outdated

Copy link

Contributor

killercup Jan 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI this path uses backslashes, the rest of the book probably not

frewsxcv reviewed Jan 2, 2017

View reviewed changes

nikomatsakis approved these changes Jan 2, 2017

View reviewed changes

alexcrichton added the beta-nominated Nominated for backporting to the compiler in the beta channel. label Jan 2, 2017

chris-morgan reviewed Jan 3, 2017

View reviewed changes

withoutboats reviewed Jan 3, 2017

View reviewed changes

parir reviewed Jan 3, 2017

View reviewed changes

src/doc/book/procedural-macros.md Outdated

Copy link

Contributor

parir Jan 3, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/crated/created/

steveklabnik force-pushed the doc-custom-derive branch from eb2c83f to 16f76c4 Compare January 3, 2017 15:58

nikomatsakis added the beta-accepted Accepted for backporting to the compiler in the beta channel. label Jan 3, 2017

steveklabnik force-pushed the doc-custom-derive branch from 16f76c4 to c0efdbf Compare January 3, 2017 22:16

GuillaumeGomez mentioned this pull request Jan 4, 2017

Rollup of 9 pull requests #38826

Closed

steveklabnik force-pushed the doc-custom-derive branch from c0efdbf to 3075c1f Compare January 4, 2017 22:34

sgrif reviewed Jan 4, 2017

View reviewed changes

bors merged commit 3075c1f into rust-lang:master Jan 5, 2017

nikomatsakis mentioned this pull request Jan 6, 2017

Beta backports #38872

Merged

17 tasks

alexcrichton removed the beta-nominated Nominated for backporting to the compiler in the beta channel. label Jan 6, 2017

Document custom derive. #38770

Document custom derive. #38770

Uh oh!

Conversation

steveklabnik commented Jan 2, 2017

Uh oh!

rust-highfive commented Jan 2, 2017

Uh oh!

steveklabnik commented Jan 2, 2017

Uh oh!

est31 commented Jan 2, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steveklabnik commented Jan 2, 2017

Uh oh!

killercup left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

killercup Jan 2, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

withoutboats commented Jan 2, 2017

Uh oh!

steveklabnik commented Jan 2, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nikomatsakis left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

steveklabnik commented Jan 2, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

F001 commented Jan 3, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

est31 commented Jan 2, 2017 •

edited

Loading

killercup Jan 2, 2017 •

edited

Loading

F001 commented Jan 3, 2017 •

edited

Loading

steveklabnik Jan 3, 2017 •

edited

Loading