SDK update + fix failing AST transformation for a valid JS file #57

bzz · 2019-02-12T18:32:43Z

Update SDK to latest version and verifies that it fixes #56 and #61

Update: it does not,

full SDK update scope is in SDK update + fix failing AST transformation for a valid JS file #57 (comment)
fix for Parsing fails on valid JS file: extra + parenthesized/parenStart #56 in 70ce5c1
map JSXIdentifier to uast:Identifier from UASTv2 has no tokens for JSX Nodes #58 in f20563e
skip mapping function comments 175bec4

bzz · 2019-02-12T18:39:08Z

CI is failing with latest SDK, presumably due to bblfsh/sdk#356, logs

bzz · 2019-02-13T11:05:54Z

There are 788 log lines mentioning failing checks, with 7 uniq failures

wget https://api.travis-ci.org/v3/job/492309486/log.txt ci-unused-fields-log.txt
grep -c "check: " ci-unused-fields-log.txt
788
grep "check: " ci-unused-fields-log.txt | sort | uniq | wc -l
7

Those are

check: unused field(s) on node BlockStatement: trailingComments
check: unused field(s) on node Identifier: [optional typeAnnotation]
check: unused field(s) on node Identifier: leadingComments
check: unused field(s) on node Identifier: typeAnnotation
check: unused field(s) on node ImportDefaultSpecifier: leadingComments
check: unused field(s) on node StringLiteral: [leadingComments trailingComments]
check: key "specifiers": key "Node": unused field(s) on node uast:Identifier: @pos

bzz · 2019-02-25T11:59:06Z

grep -c "check:" ci-unused-fields-log-2.txt
798
grep "check:" ci-unused-fields-log-2.txt| sort | uniq | wc -l
9

on the latest master it's 9 failing checks now

going to address them in this PR.

bzz · 2019-02-27T15:19:49Z

Keeping the comment above updated with the current status

bzz · 2019-02-27T20:54:04Z

All import issues resolved, 3 more extra from #56 added.

Signed-off-by: Alexander Bezzubov <bzz@apache.org>

@pos

Fix for: check: key "specifiers": key "Node": unused field(s) on node uast:Identifier: @pos Signed-off-by: Alexander Bezzubov <bzz@apache.org>

Signed-off-by: Alexander Bezzubov <bzz@apache.org>

Flow: right now only function return type annotations are supported. Signed-off-by: Alexander Bezzubov <bzz@apache.org>

Signed-off-by: Alexander Bezzubov <bzz@apache.org>

bzz · 2019-03-01T16:33:10Z

Few more fixed and sever new showed up, #57 (comment) scope updated.

To check current status:

$ go run vendor/gopkg.in/bblfsh/sdk.v2/cmd/bblfsh-sdk/main.go test 2>&1 | tee log.txt
$ curl -s log.txt | grep "check:" | sort | uniq | wc -l
7

Skip mapping 'optional' field on Indetifier for Flow "Optional Prameters" https://flow.org/en/docs/types/functions/#toc-optional-parameters `function acceptsNumberMaybe (value?: number) {}` Signed-off-by: Alexander Bezzubov <bzz@apache.org>

Signed-off-by: Alexander Bezzubov <bzz@apache.org>

bzz · 2019-03-01T18:08:48Z

Just 4 to go :)

Skip mapping Flow type annotation for function's - arguments - return types in Sematic UAST Signed-off-by: Alexander Bezzubov <bzz@apache.org>

bzz · 2019-03-02T18:52:00Z

and CI is green now! 🎉

Technically, SDK update is finished BUT 3 errors above are a fix for #56 and would have very similar changes so going to take care of them here as well.

Parenthisized literals result in this information saved in 'extra' field. Signed-off-by: Alexander Bezzubov <bzz@apache.org>

Instead of dropping comments of ImportNamespaceSpecifier as implemented in bblfsh#60, we just skip both, leading and trailing comments now. This avoids loosing a Comment node in UAST. Signed-off-by: Alexander Bezzubov <bzz@apache.org>

bzz · 2019-03-03T09:18:37Z

Done, scope an description updated, everything is ready for review 🚀

driver/normalizer/normalizer.go

Signed-off-by: Alexander Bezzubov <bzz@apache.org>

bzz · 2019-03-05T14:24:21Z

Thank you for the prompt review, @dennwc @juanjux !

All feedback addressed, scope in PR description updated and only last 3 commits need to be reviewed again.

This was referenced Feb 14, 2019

Non semantically mapped properties on semantic objects bblfsh/sdk#361

Open

Set timeout for parse request bblfsh/scala-client#78

Closed

bzz mentioned this pull request Feb 22, 2019

Parsing fails on JSX + Flow file #59

Closed

bzz self-assigned this Feb 22, 2019

bzz mentioned this pull request Feb 22, 2019

Two identical parse requests can return different results #54

Closed

bzz force-pushed the fix-56 branch from eddcc64 to 54623d2 Compare February 25, 2019 11:50

bzz force-pushed the fix-56 branch 4 times, most recently from 7dd666c to c6b0eb7 Compare February 27, 2019 15:16

bzz force-pushed the fix-56 branch 2 times, most recently from a8efdb3 to 998ff2c Compare February 27, 2019 20:49

bzz added 8 commits March 1, 2019 17:12

bump sdk to 2.14.x + dep version update

afefaf7

Signed-off-by: Alexander Bezzubov <bzz@apache.org>

sem: don't ignore position for Import>specifiers>Node

748c74c

Fix for: check: key "specifiers": key "Node": unused field(s) on node uast:Identifier: @pos Signed-off-by: Alexander Bezzubov <bzz@apache.org>

sem: drop ImportDefaultSpecifier>leadingComments

aa2dcb9

Signed-off-by: Alexander Bezzubov <bzz@apache.org>

sem: drop BlockStatement>trailingComments

8a68fd3

Signed-off-by: Alexander Bezzubov <bzz@apache.org>

sem: drop StringLiteral>[leadingComments trailingComments]

a7c32cf

Signed-off-by: Alexander Bezzubov <bzz@apache.org>

sem: drop Flow variable type annotations

097c630

Flow: right now only function return type annotations are supported. Signed-off-by: Alexander Bezzubov <bzz@apache.org>

sem: drop un-used empty array ImportDeclaration>specifiers

b2ce4a1

Signed-off-by: Alexander Bezzubov <bzz@apache.org>

sem: drop ImportDeclaration>[leadingComments trailingComments]

aece00e

Signed-off-by: Alexander Bezzubov <bzz@apache.org>

bzz force-pushed the fix-56 branch from 998ff2c to 097c630 Compare March 1, 2019 16:22

bzz added 2 commits March 1, 2019 17:46

sem: don't map Flow Identifier>optional flag

d51414d

Skip mapping 'optional' field on Indetifier for Flow "Optional Prameters" https://flow.org/en/docs/types/functions/#toc-optional-parameters `function acceptsNumberMaybe (value?: number) {}` Signed-off-by: Alexander Bezzubov <bzz@apache.org>

sem: don't map Identifier>[leadingComments, trailingComments]

c20f8d5

Signed-off-by: Alexander Bezzubov <bzz@apache.org>

sem: don't map FunctionDeclaration>[predicate returnType typeParameters]

ec04b0c

Skip mapping Flow type annotation for function's - arguments - return types in Sematic UAST Signed-off-by: Alexander Bezzubov <bzz@apache.org>

bzz added 2 commits March 3, 2019 01:28

sem: don't map parenthesis of literals; fix bblfsh#56

70ce5c1

Parenthisized literals result in this information saved in 'extra' field. Signed-off-by: Alexander Bezzubov <bzz@apache.org>

sem: re-work ImportNamespaceSpecifier comments

0be5d4a

Instead of dropping comments of ImportNamespaceSpecifier as implemented in bblfsh#60, we just skip both, leading and trailing comments now. This avoids loosing a Comment node in UAST. Signed-off-by: Alexander Bezzubov <bzz@apache.org>

bzz changed the title ~~WIP fix failing AST transformation for valid JS file~~ SDK update + fix failing AST transformation for a valid JS file Mar 3, 2019

bzz requested review from dennwc and creachadair March 3, 2019 00:45

bzz requested a review from juanjux March 5, 2019 11:15

juanjux reviewed Mar 5, 2019

View reviewed changes

driver/normalizer/normalizer.go Outdated Show resolved Hide resolved

juanjux approved these changes Mar 5, 2019

View reviewed changes

bzz mentioned this pull request Mar 5, 2019

Parsing fails on valid JS file: extra + parenthesized/parenStart #56

Closed

dennwc approved these changes Mar 5, 2019

View reviewed changes

bzz mentioned this pull request Mar 5, 2019

Better type annotations #63

Open

bzz force-pushed the fix-56 branch from ffb4980 to 5ac9d9d Compare March 5, 2019 14:06

bzz added 2 commits March 5, 2019 15:07

sem: map JSXIdentifier -> uast:Identifier

f20563e

Signed-off-by: Alexander Bezzubov <bzz@apache.org>

sem: skip mapping function's comments

175bec4

Signed-off-by: Alexander Bezzubov <bzz@apache.org>

bzz force-pushed the fix-56 branch from 5ac9d9d to 175bec4 Compare March 5, 2019 14:07

sem: add comments to clarify mapping skip cases

a836d36

Signed-off-by: Alexander Bezzubov <bzz@apache.org>

dennwc approved these changes Mar 5, 2019

View reviewed changes

bzz merged commit 6263505 into bblfsh:master Mar 5, 2019

bzz deleted the fix-56 branch March 5, 2019 17:23

bzz mentioned this pull request Mar 5, 2019

panic: slice bounds out of range in uast/transformer #61

Closed

bzz mentioned this pull request Mar 12, 2019

Error message check: key "extra": field was not used: parenthesized for v2.6.0 #67

Closed

bzz added this to the v2.7.0 milestone Mar 15, 2019

bzz mentioned this pull request Mar 20, 2019

Better comment handling #74

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SDK update + fix failing AST transformation for a valid JS file #57

SDK update + fix failing AST transformation for a valid JS file #57

bzz commented Feb 12, 2019 •

edited

Loading

bzz commented Feb 12, 2019

bzz commented Feb 13, 2019 •

edited

Loading

bzz commented Feb 25, 2019 •

edited

Loading

bzz commented Feb 27, 2019

bzz commented Feb 27, 2019 •

edited

Loading

bzz commented Mar 1, 2019

bzz commented Mar 1, 2019

bzz commented Mar 2, 2019

bzz commented Mar 3, 2019

bzz commented Mar 5, 2019

SDK update + fix failing AST transformation for a valid JS file #57

SDK update + fix failing AST transformation for a valid JS file #57

Conversation

bzz commented Feb 12, 2019 • edited Loading

bzz commented Feb 12, 2019

bzz commented Feb 13, 2019 • edited Loading

bzz commented Feb 25, 2019 • edited Loading

bzz commented Feb 27, 2019

bzz commented Feb 27, 2019 • edited Loading

bzz commented Mar 1, 2019

bzz commented Mar 1, 2019

bzz commented Mar 2, 2019

bzz commented Mar 3, 2019

bzz commented Mar 5, 2019

bzz commented Feb 12, 2019 •

edited

Loading

bzz commented Feb 13, 2019 •

edited

Loading

bzz commented Feb 25, 2019 •

edited

Loading

bzz commented Feb 27, 2019 •

edited

Loading