Why add new tokens instead of purely decorating existing structures?

[bitwiseman]

Just wondering if you could comment on why you chose to add new tokens to handle comments and whitespace, instead of purely decorating existing tokens or nodes with additional information?

By adding new tokens you make the output of rocambole incompatible with tools that consume esprima output. You also make processing of the output more complex.

For example, most formatting rules are based on what node a token is inside and sometimes what token came before them. On js-beautify, we found that comments and whitespace are secondary infomation to rules - keeping them as a sideband of information makes rules simpler while still letting the cases where they matter function.

Another example is the UglifyJs token structure. It adds the comment tokens as decoration for regular tokens.

[millermedeiros]

my idea was mainly because in some cases line breaks should be kept/removed based on the comments.. and grabbing the Comments/LineBreaks inside a node is simpler if it is already inside the tokens list.

if you search inside esformatter code, you will see that I use isComment in many places same thing for LineBreak tokens.

it's also way easier to rebuild the JS string if the tokens already have line breaks and comments. Also easier to identify if previous/next/current token is a LineBreak or Comment.

[millermedeiros]

But you are right.. I could have just added the new properties and kept the tokens array with the original data and have the LineBreak and Comments just on the linked list. Maybe on a future version.

[bitwiseman]

Cool thanks, you might want to put this on the project wiki. It a valid design choice, probably not valid for the readme abut worth having documented as something other than an issue.