Skip to content
This repository has been archived by the owner on Jul 8, 2019. It is now read-only.

Exception while linting #69

Closed
charpour opened this issue Nov 18, 2016 · 16 comments
Closed

Exception while linting #69

charpour opened this issue Nov 18, 2016 · 16 comments

Comments

@charpour
Copy link

I'm getting the following exception while sonar is linting TS code:

ERROR: Error during SonarQube Scanner execution
com.google.gson.JsonSyntaxException: com.google.gson.stream.MalformedJsonException: Unterminated string at line 1 column 118932 path $[0][0].name
at com.google.gson.Gson.fromJson(Gson.java:820)
at com.google.gson.Gson.fromJson(Gson.java:770)
at com.google.gson.Gson.fromJson(Gson.java:719)
at com.google.gson.Gson.fromJson(Gson.java:691)
at com.pablissimo.sonar.TsLintParserImpl.parse(TsLintParserImpl.java:12)
at com.pablissimo.sonar.TsLintSensor.execute(TsLintSensor.java:101)
at org.sonar.scanner.sensor.SensorWrapper.analyse(SensorWrapper.java:53)
at org.sonar.scanner.phases.SensorsExecutor.executeSensor(SensorsExecutor.java:57)
at org.sonar.scanner.phases.SensorsExecutor.execute(SensorsExecutor.java:49)
at org.sonar.scanner.phases.AbstractPhaseExecutor.execute(AbstractPhaseExecutor.java:78)
at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:184)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:142)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:127)
at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:241)
at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:236)
at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:234)
at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:234)
at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:226)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:142)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:127)
at org.sonar.scanner.task.ScanTask.execute(ScanTask.java:47)
at org.sonar.scanner.task.TaskContainer.doAfterStart(TaskContainer.java:86)
at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:142)
at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:127)
at org.sonar.scanner.bootstrap.GlobalContainer.executeTask(GlobalContainer.java:115)
at org.sonar.batch.bootstrapper.Batch.executeTask(Batch.java:118)
at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
at com.sun.proxy.$Proxy0.execute(Unknown Source)
at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:244)
at org.sonarsource.scanner.api.EmbeddedScanner.runAnalysis(EmbeddedScanner.java:154)
at org.sonarsource.scanner.cli.Main.runAnalysis(Main.java:110)
at org.sonarsource.scanner.cli.Main.execute(Main.java:72)
at org.sonarsource.scanner.cli.Main.main(Main.java:60)
Caused by: com.google.gson.stream.MalformedJsonException: Unterminated string at line 1 column 118932 path $[0][0].name
at com.google.gson.stream.JsonReader.syntaxError(JsonReader.java:1572)
at com.google.gson.stream.JsonReader.nextQuotedValue(JsonReader.java:1029)
at com.google.gson.stream.JsonReader.nextString(JsonReader.java:825)
at com.google.gson.internal.bind.TypeAdapters$13.read(TypeAdapters.java:358)
at com.google.gson.internal.bind.TypeAdapters$13.read(TypeAdapters.java:346)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:95)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:183)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:40)
at com.google.gson.internal.bind.ArrayTypeAdapter.read(ArrayTypeAdapter.java:72)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:40)
at com.google.gson.internal.bind.ArrayTypeAdapter.read(ArrayTypeAdapter.java:72)
at com.google.gson.Gson.fromJson(Gson.java:805)
... 37 more

Any ideas what the problem might be?

I'm using SonarQube 6.0 and the latest released version of the plugin (0.94)
@Pablissimo
Copy link
Owner

Not fun!

  • What version of tslint are you running? (tslint -v on the command line to find out)
  • What OS are you running?
  • Can you run the sonar-runner with the -X -e arguments and attach/email me the log please? It'll give you the exact command line that was used to invoke tslint, which you can then run yourself - the output of that would also be handy if you can?

@charpour
Copy link
Author

charpour commented Nov 18, 2016 •
edited
Loading

I'll try to debug first and then I'll let you know of the outcome. Looks like that even though the tslint command generates a correct JSON, not the whole thing is passed to the parser.

ts lint version: 3.15.1
OS: debian linux

-X -e didn't provide any additional useful information

@charpour
Copy link
Author

out of curiosity, in order to debug the plugin, where do you configure the debug JVM flags? the scanner itself or one of the sonar server processes (web, ce, search)?

Thanks

@charpour
Copy link
Author

charpour commented Nov 18, 2016 •
edited
Loading

I've noticed that out of the 349066 characters of output that the actual tslint command produces, only 117603 make it to the stream when consumed via the CommandExecutor, for some reason.

Thus, to reproduce you can create a file that has a lot of lint errors.

@Pablissimo
Copy link
Owner

Cracking, thanks - should be easy enough to repro from that investigation!

@Pablissimo
Copy link
Owner

Seems to be Linux-specific, and am wondering if it's a bug in the Sonar API's CommandExecutor - on Ubuntu 14.04 LTS the command executes fine but only generates a maximum of 79,278 characters of output and just seems to bin the rest silently, oddly. Whole thing's weird.

Am curious to see if a different mechanism for executing the command might help. Shall have a nose.

@charpour
Copy link
Author

Quite weird indeed.
perhaps apache commons exec could help

@charpour
Copy link
Author

As a workaround to the problem on linux, '-o /dev/stdout' argument has to be passed to the tslint command. For some reason, when it outputs to stdout some data gets discarded. If it's a file, everything works as expected. Some flushing node issue perhaps?

@charpour
Copy link
Author

Finally.. found the issue. It's in tslint:

outputStream.write(lintResult.output, () => {
        if (lintResult.failureCount > 0) {
            process.exit(argv.force ? 0 : 2);
        }
    });

process.exit is the culprit. It terminates the process before all of the data is flushed.

@Pablissimo
Copy link
Owner

Nice debugging - hooray for cross-platform development I guess! I notice that there are Node issues around this specific case (e.g. nodejs/node#2972, nodejs/node#6456). Seems the guidance on the node side is not to use process.exit unless you just want to tear the whole thing down even without draining output streams.

The behaviour also seems to vary by Node version, excitingly. Wondering if it's safer to just skip streams entirely and always on both platforms output to a temporary file and re-read it (slower, and generates a little IO but I wouldn't have thought noticeable).

I'll try a patch along the lines you're suggesting - detect Linux, be explicit about the output stream. I'm already special-casing Windows to get quoted paths to work properly so that ship's sailed

@charpour
Copy link
Author

I'd think that writing the output to an intermediate file on all platforms (i.e. using the -o argument of tslint) and consuming it from there is the better approach, as personally I'm not fond of using OS based branches in the code. It'd be harder to remove it in the future if for some reason you decide to do so.

@Pablissimo
Copy link
Owner

Got a local build of this running against a temporary file that gets re-read, seems performant enough and the issue goes away on Ubuntu 14.04 LTS. Got to check Windows yet, and fix a few tests but seems to be a go-er.

Pablissimo added a commit that referenced this issue Nov 19, 2016
@Pablissimo
Fix issue #69 - long output from TsLint no longer truncated on Linux
ebe1754
@Pablissimo
Copy link
Owner

Ok, CI build for the issue69 branch finished and should hopefully be an improvement for you (appears to continue to work fine on Windows and the couple of Linux variants I've got access to). You can either pull and build or give the binary a bash:

https://s3-us-west-2.amazonaws.com/com.pablissimo.github.sonartsplugin/builds/issue69/128/target/sonar-typescript-plugin-0.95-SNAPSHOT.jar

Will make it a proper release once you confirm it's done the job!

@charpour
Copy link
Author

charpour commented Nov 20, 2016 •
edited
Loading

I've tried 0.95 against tslint@4.0.0, but the run failed with the following error:

01:55:45 INFO: Sensor Linting sensor for TypeScript files
01:55:47 INFO: ------------------------------------------------------------------------
01:55:47 INFO: EXECUTION FAILURE
01:55:47 INFO: ------------------------------------------------------------------------
01:55:47 INFO: Total time: 1:32.425s
01:55:47 INFO: Final Memory: 59M/383M
01:55:47 INFO: ------------------------------------------------------------------------
01:55:47 ERROR: Error during SonarQube Scanner execution
01:55:47 ERROR: 54 is not a valid line for pointer. File [moduleKey=<some_ts_file>, basedir=<some_directory>] has 24 line(s)

However, it works great against tslint@3.15.1. I'll stick to that version for now.

Thanks for the quick fix!

@Pablissimo
Copy link
Owner

Never rains but it pours! Thanks for feedback, will raise a new issue for that then!

-------- Original message --------
From: charpour notifications@github.com
Date: 20/11/2016 12:18 (GMT+00:00)
To: Pablissimo/SonarTsPlugin SonarTsPlugin@noreply.github.com
Cc: Pablissimo turnacre@hotmail.com, Comment comment@noreply.github.com
Subject: Re: [Pablissimo/SonarTsPlugin] Exception while linting (#69)

I've tried 0.95 against tslint@4.0.0, but the run failed with the following error:

01:55:45 INFO: Sensor Linting sensor for TypeScript files
01:55:47 INFO: ------------------------------------------------------------------------
01:55:47 INFO: EXECUTION FAILURE
01:55:47 INFO: ------------------------------------------------------------------------
01:55:47 INFO: Total time: 1:32.425s
01:55:47 INFO: Final Memory: 59M/383M
01:55:47 INFO: ------------------------------------------------------------------------
01:55:47 ERROR: Error during SonarQube Scanner execution
01:55:47 ERROR: 54 is not a valid line for pointer. File [moduleKey=<some_ts_file>, basedir=<some_directory>] has 24 line(s)

However, it works great against tslint@3.15.1mailto:tslint@3.15.1. I'll stick to that version for mow.

Thanks for the quick fix!

You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com//issues/69#issuecomment-261774808, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABZvgB3cTp2v-tPPDatOFapUXir59jzNks5rADqOgaJpZM4K2bzk.

@Pablissimo Pablissimo mentioned this issue Dec 6, 2016
Merged
Pablissimo added a commit that referenced this issue Dec 6, 2016
@Pablissimo
tslint 4.0.0+ support (#71)
35918a9 
Adds support for tslint 4.0.0's new (fixed!) output format, and definitions for more rules introduced in the past while. Fixes long tslint output on Linux being truncated (merged issue69 branch).

Fixes #67, #69, #70
@Pablissimo
Copy link
Owner

Fixed in the above-linked release but also in 0.96, since 0.95 never made it to the Releases page.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Pablissimo @charpour