Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Json format return message for lintserver() #194

Merged
merged 20 commits into from
Mar 7, 2017
Merged

Json format return message for lintserver() #194

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
4242143
convertmsgtojson() added
TeroFrondelius Feb 19, 2017
6ca5e53
Adding also LintMessage style to convertmsgtojson()
TeroFrondelius Feb 19, 2017
0c68068
Adding tests
TeroFrondelius Feb 20, 2017
ab52c79
Changing lintserver() to return single json
TeroFrondelius Feb 20, 2017
8351265
Fixes according to @TotalVerb review
TeroFrondelius Feb 21, 2017
f4440a6
Updating lintserver() receive side to accept JSON
TeroFrondelius Feb 21, 2017
1ccc0e9
Fine tuning after testing with linter-julia
TeroFrondelius Feb 21, 2017
39c50f5
Updates based on @TotalVerb comments
TeroFrondelius Feb 22, 2017
a81d069
Adding the feature to turn off the codes and some fine tuning of the …
TeroFrondelius Feb 22, 2017
c5ce694
Adding the missing test to previous commit
TeroFrondelius Feb 22, 2017
ddefddb
Writing the new server protocol documentation to features.md
TeroFrondelius Feb 25, 2017
343f842
More text at features.md
TeroFrondelius Feb 25, 2017
0222c9d
changes according to review
TeroFrondelius Feb 27, 2017
8e268fb
Fixing the standard-linter-v1 message bug when show_code = true
TeroFrondelius Mar 4, 2017
91dd2f0
Documentation update based on @TotalVerb review
TeroFrondelius Mar 4, 2017
175ab5c
Documentation update based on @TotalVerb review, fixing the example
TeroFrondelius Mar 4, 2017
f814b29
Adding the test for the bug: "show_code"=true
TeroFrondelius Mar 5, 2017
b4593be
tets/server.jl updated to use JSON.print style
TeroFrondelius Mar 5, 2017
78dfe6c
lintdir() doc string rewording to remove asterisk sign
TeroFrondelius Mar 5, 2017
38ccf65
resolving merge conflict
TeroFrondelius Mar 6, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 98 additions & 20 deletions src/Lint.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ module Lint

using Base.Meta
using Compat
using JSON

export LintMessage, LintContext, LintStack
export lintfile, lintstr, lintpkg, lintserver, @lintpragma
Expand Down Expand Up @@ -340,7 +341,7 @@ function lintinclude(ctx::LintContext, file::AbstractString)
end

"""
Lint all *.jl files at a given directory.
Lint all \*.jl files at a given directory.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you're escaping the markdown here, but I don't think this works. Need two backslashes, that is \\*, because \* is the same as * in a string.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I forgot it here. This is atom-language-julia bug of syntax highlighting. I just use the backslash to show the rest of the file highlighting properly.

Will ignore LintContext file and already included files.
"""
function lintdir{T<:AbstractString}(dir::T, ctx::LintContext=LintContext())
Expand All @@ -355,34 +356,110 @@ function lintdir{T<:AbstractString}(dir::T, ctx::LintContext=LintContext())
ctx.messages
end

function readandwritethestream(conn)
# println("Connection accepted")
# Get file, code length and code
file = strip(readline(conn))
# println("file: ", file)
code_len = parse(Int, strip(readline(conn)))
# println("Code bytes: ", code_len)
code = Compat.UTF8String(read(conn, code_len))
# println("Code received")
# Do the linting
msgs = lintfile(file, code)
# Write response to socket
for i in msgs
write(conn, string(i))
function convertmsgtojson(msgs, style)
if style == "lint-message"
return JSON.json(msgs)
end
output = []
for msg in msgs
evar = msg.variable
txt = msg.message
file = msg.file
linenumber = msg.line
# Atom index starts from zero thus minus one
errorrange = Array[[linenumber-1, 0], [linenumber-1, 80]]
code = string(msg.code)
if code[1] == 'I'
etype = "info"
etypenumber = 3
elseif code[1] == 'W'
etype = "warning"
etypenumber = 2
else
etype = "error"
etypenumber = 1
end

if style == "standard-linter-v1"
push!(output, Dict("type" => etype,
"text" => "$code $evar $txt",
"range" => errorrange,
"filePath" => file))
elseif style == "vscode"
push!(output, Dict("severity" => etypenumber,
"message" => "$evar $txt",
"range" => errorrange,
"filePath" => file,
"code" => code,
"source" => "Lint.jl"))
elseif style == "standard-linter-v2"
push!(output, Dict("severity" => etype,
"location" => Dict("file" => file,
"position" => errorrange),
"excerpt" => code,
"description" => "$evar $txt"))

end
end
return JSON.json(output)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I think it might be better to just return the output here, and handle the vector of dicts in the other function. Then we can avoid allocating an intermediate string and print directly to the stream, which might be faster. It also becomes easier to test this function if we want to add unit tests for more styles in the future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this idea. I will make the changes.

end


function filtermsgs(msgs,dict_data)
if haskey(dict_data,"ignore_warnings")
if dict_data["ignore_warnings"]
msgs = filter(i -> !iswarning(i), msgs)
end
end
if haskey(dict_data,"ignore_info")
if dict_data["ignore_info"]
msgs = filter(i -> !isinfo(i), msgs)
end
end
if haskey(dict_data,"ignore_codes")
msgs = filter(i -> !(string(i.code) in dict_data["ignore_codes"]), msgs)
end
return msgs
end


function readandwritethestream(conn,style)
if style == "original_behaviour"
# println("Connection accepted")
# Get file, code length and code
file = strip(readline(conn))
# println("file: ", file)
code_len = parse(Int, strip(readline(conn)))
# println("Code bytes: ", code_len)
code = Compat.UTF8String(read(conn, code_len))
# println("Code received")
# Do the linting
msgs = lintfile(file, code)
# Write response to socket
for i in msgs
write(conn, string(i))
write(conn, "\n")
end
# Blank line to indicate end of messages
write(conn, "\n")
else
json_data = readline(conn)
dict_data = JSON.parse(json_data)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe you can parse the connection directly; i.e. JSON.parse(conn) instead of JSON.parse(readline(conn)). I recommend that over readline so the client can send newlines in their JSON if they prefer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I really appreciate your effort to review and improve my code.

msgs = lintfile(dict_data["file"], dict_data["code_str"])
msgs = filtermsgs(msgs,dict_data)
out = convertmsgtojson(msgs,style)
write(conn,out)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, we can do JSON.print(conn, out), if out hasn't already been converted to JSON (see comment above).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this idea. I will make the changes.

end
# Blank line to indicate end of messages
write(conn, "\n")
end

function lintserver(port)
function lintserver(port,style="original_behaviour")
server = listen(port)
try
println("Server running on port $port ...")
println("Server running on port/pipe $port ...")
while true
conn = accept(server)
@async try
readandwritethestream(conn)
readandwritethestream(conn,style)
catch err
println(STDERR, "connection ended with error $err")
finally
Expand All @@ -396,6 +473,7 @@ function lintserver(port)
end
end


# precompile hints
include("precompile.jl")

Expand Down
178 changes: 147 additions & 31 deletions test/server.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,33 @@ port = conn[1]
server = @async lintserver(port)
sleep(1) #let server start

function lintbyserver(socket,str)
println(socket, "none")
println(socket, sizeof(str)) # bytes of code
println(socket, str) # code
end

function readfromserver_old(socket)
response = ""
line = ""
while line != "\n"
response *= line
line = readline(socket)
end
return response
end

function readfromserver_new(socket)
response = ""
line = ""
while isopen(socket)
response *= line
line = readline(socket)
end
return response
end


@testset "lintserver() tests" begin
conn = connect(port)
write(conn, "empty\n")
Expand All @@ -24,41 +51,130 @@ sleep(1) #let server start
end

@testset "Testing the lintserver addition" begin
function lintbyserver(socket)
str = """
test = "Hello" + "World"
"""
println(socket, "none")
println(socket, sizeof(str)) # bytes of code
println(socket, str) # code
end

str = """
test = "Hello" + "World"
"""
socket = connect(port)
lintbyserver(socket)
response = ""
line = ""
while line != "\n"
response *= line
line = readline(socket)
end

lintbyserver(socket,str)
response = readfromserver_old(socket)
@test response == "none:1 E422 : string uses * to concatenate\n"

socket = connect(port)
lintbyserver(socket)
res = ""
line = ""
while isopen(socket)
res *= line
line = readline(socket)
end

lintbyserver(socket,str)
res = readfromserver_new(socket)
@test res == "none:1 E422 : string uses * to concatenate\n\n"
end

# This isn't working on the nightly build. Ideally we explicitly stop the server process (as
# it loops forever). It seems to get stopped when the tests end, so it's not necessary.
#
#try # close the server
# Base.throwto(server, InterruptException())
#end
@testset "Testing lintserver() with named pipe and JSON format" begin
if is_windows()
pipe = "\\\\.\\pipe\\testsocket"
pipe2 = "\\\\.\\pipe\\testsocket2"
pipe3 = "\\\\.\\pipe\\testsocket3"
pipe4 = "\\\\.\\pipe\\testsocket4"
else
pipe = tempname()
pipe2 = tempname()
pipe3 = tempname()
pipe4 = tempname()
end
server_LintMessage = @async lintserver(pipe,"lint-message")
sleep(1)
socket = connect(pipe)
json_input = JSON.json(Dict("file" => "none", "code_str" => "something"))
write(socket, json_input * "\n")
json_output = readline(socket)
results_array = JSON.parse(strip(json_output))
@test results_array[1]["line"] == 1
@test results_array[1]["message"] == "use of undeclared symbol"
@test results_array[1]["file"] == "none"
@test results_array[1]["code"] == "E321"

server_slv1 = @async lintserver(pipe2,"standard-linter-v1")
sleep(1)
socket = connect(pipe2)
write(socket, json_input * "\n")
json_output = readline(socket)
results_array = JSON.parse(strip(json_output))
@test results_array[1]["text"] == "E321 something use of undeclared symbol"
@test results_array[1]["filePath"] == "none"
@test results_array[1]["range"] == Array[[0, 0], [0, 80]]
@test results_array[1]["type"] == "error"


socket = connect(pipe2)
json_input2 = JSON.json(Dict("file" => "none",
"code_str" => "pi=3"))
write(socket, json_input2 * "\n")
json_output = readline(socket)
results_array = JSON.parse(strip(json_output))
@test results_array[1]["text"] == "W351 pi redefining mathematical constant"
@test results_array[1]["filePath"] == "none"
@test results_array[1]["range"] == Array[[0, 0], [0, 80]]
@test results_array[1]["type"] == "warning"

socket = connect(pipe2)
json_input3 = JSON.json(Dict("file" => "none",
"code_str" => "function a(b)\nend"))
write(socket, json_input3 * "\n")
json_output = readline(socket)
results_array = JSON.parse(strip(json_output))
@test results_array[1]["text"] == "I382 b argument declared but not used"
@test results_array[1]["filePath"] == "none"
@test results_array[1]["range"] == Array[[0, 0], [0, 80]]
@test results_array[1]["type"] == "info"


server_vscode = @async lintserver(pipe3,"vscode")
sleep(1)
socket = connect(pipe3)
write(socket, json_input * "\n")
json_output = readline(socket)
results_array = JSON.parse(strip(json_output))
@test results_array[1]["message"] == "something use of undeclared symbol"
@test results_array[1]["filePath"] == "none"
@test results_array[1]["range"] == Array[[0, 0], [0, 80]]
@test results_array[1]["code"] == "E321"
@test results_array[1]["severity"] == 1
@test results_array[1]["source"] == "Lint.jl"


server_slv2 = @async lintserver(pipe4,"standard-linter-v2")
sleep(1)
socket = connect(pipe4)
write(socket, json_input * "\n")
json_output = readline(socket)
results_array = JSON.parse(strip(json_output))
@test results_array[1]["description"] == "something use of undeclared symbol"
@test results_array[1]["location"]["file"] == "none"
@test results_array[1]["location"]["position"] == Array[[0, 0], [0, 80]]
@test results_array[1]["severity"] == "error"
@test results_array[1]["excerpt"] == "E321"


json_input = JSON.json(Dict("file" => "none",
"code_str" => "function a(b)\nend",
"ignore_info" => true))
socket = connect(pipe)
write(socket, json_input * "\n")
json_output = readline(socket)
results_array = JSON.parse(strip(json_output))
@test isempty(results_array)

json_input = JSON.json(Dict("file" => "none",
"code_str" => "pi = 1",
"ignore_warnings" => true))
socket = connect(pipe)
write(socket, json_input * "\n")
json_output = readline(socket)
results_array = JSON.parse(strip(json_output))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't need to strip json, here and elsewhere.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I forgot this one.

@test isempty(results_array)

json_input = JSON.json(Dict("file" => "none",
"code_str" => "pi = 1\nfunction a(b)\nend",
"ignore_codes" => ["I382","W351"]))
socket = connect(pipe)
write(socket, json_input * "\n")
json_output = readline(socket)
results_array = JSON.parse(strip(json_output))
@test isempty(results_array)
end