Skip to content
This repository has been archived by the owner on Aug 7, 2023. It is now read-only.

Commit

Permalink
Use -sourcepath to improve compilation speed (#99)
Browse files Browse the repository at this point in the history 
There are multiple problems with the previous approach of
scanning the current project and including each and every
file to linter javac.

  * It takes a long time since every source java file in
    the current project is recompiled with every save.
  * The command line may become "too long" (see #38)
  * It consumes a lot of computer resources (CPU, memory etc)

These problems increase with the number of source files.

Using sourcepath will let javac find the required sourcefiles
by itself, greatly reducing the number of source files it needs
to recompile. This is one of the things we need to make in order
to resolve #98.

It is still not optimal. Javac does not include any caching of the
files it compiles, so saving the same file twice (without editing it)
will recompile it twice as well as any java file referenced by it.

In this commit I also:
  * Added ordering to the config variables (I really wanted
    the sourcepath option to be next to the classpath option).
  * Moved all `if (@verboseLogging)` to the `@_log` method
  * Some spacing and cleanup
  • Loading branch information
@noseglid
noseglid authored and Florian Breisch committed Jun 8, 2016
1 parent 570bb8e commit 03ae1a9
Show file tree
Hide file tree
Showing 2 changed files with 79 additions and 143 deletions.
198 changes: 61 additions & 137 deletions lib/init.coffee
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ helpers = null
voucher = null
fs = null

cpConfigFileName = '.classpath'

module.exports =
activate: (state) ->
# state-object as preparation for user-notifications
Expand Down Expand Up @@ -53,6 +51,11 @@ module.exports =
(newValue) =>
@classpathFilename = newValue.trim()
)
@subscriptions.add(
atom.config.observe 'linter-javac.sourcepathFilename',
(newValue) =>
@sourcepathFilename = newValue.trim()
)
@subscriptions.add(
atom.config.observe 'linter-javac.javacArgsFilename',
(newValue) =>
Expand All @@ -78,131 +81,80 @@ module.exports =
helpers = require 'atom-linter'
voucher = require 'voucher'
fs = require 'fs'
if @verboseLogging
@_log 'requiring modules finished.'
@_log 'requiring modules finished.'

if @verboseLogging
@_log 'providing linter, examining javac-callability.'
@_log 'providing linter, examining javac-callability.'

grammarScopes: ['source.java']
scope: 'project'
lintOnFly: false # Only lint on save

lint: (textEditor) =>
filePath = textEditor.getPath()
wd = path.dirname filePath
searchDir = @getProjectRootDir() || path.dirname filePath

# Classpath
cp = ''

if @verboseLogging
@_log 'starting to lint.'
# Sourcepath - this is a fair default for many Java projects
sp = 'src/main/java/'

@_log 'starting to lint.'

# Find project config file if it exists.
cpConfig = @findClasspathConfig(wd)
cpConfig = @findConfigFile(wd, @classpathFilename)
if cpConfig?
# Use the location of the config file as the working directory
wd = cpConfig.cfgDir
wd = cpConfig.dir
# Use configured classpath
cp = cpConfig.cfgCp
cp = cpConfig.content
# Use config file location to import correct files
searchDir = wd

spConfig = @findConfigFile(wd, @sourcepathFilename)
if spConfig?
sp = spConfig.content

# Add extra classpath if provided
cp += path.delimiter + @classpath if @classpath

# Add environment variable if it exists
cp += path.delimiter + process.env.CLASSPATH if process.env.CLASSPATH

if @verboseLogging
@_log 'start searching java-files with "',
searchDir,
'" as search-directory.'
@_log 'start searching java-files with "', searchDir, '" as search-directory.'

lstats = fs.lstatSync searchDir

atom.project.repositoryForDirectory(
new Directory(searchDir, lstats.isSymbolicLink())
)
.then (repo) =>
@getFilesEndingWith searchDir,
'.java', repo?.isPathIgnored.bind(repo)
.then (files) =>
# Arguments to javac
args = ['-Xlint:all']
args = args.concat(['-cp', cp]) if cp

# add additional options to the args-array
if @additionalOptions.length > 0
args = args.concat @additionalOptions
if @verboseLogging
@_log 'adding',
@additionalOptions.length,
'additional javac-options.'

if @verboseLogging
@_log 'collected the following arguments:', args.join(' ')

# add javac argsfile if filename has been configured
if @javacArgsFilename
args.push('@' + @javacArgsFilename)
if @verboseLogging
@_log 'adding', @javacArgsFilename, 'as argsfile.'

args.push.apply(args, files)
if @verboseLogging
@_log 'adding',
files.length,
'files to the javac-arguments (from "',
files[0],
'" to "',
files[files.length - 1]
'").'

# TODO: remove this quick fix
# count the size of expected execution-command
# see issue #58 for further details
cliLimit = if _os.platform() == 'win32' then 7900 else 130000
expectedCmdSize = @javaExecutablePath.length
sliceIndex = 0
for arg in args
expectedCmdSize++ # add prepending space
if (typeof arg) == 'string'
expectedCmdSize += arg.length
else
expectedCmdSize += arg.toString().length
if expectedCmdSize < cliLimit
sliceIndex++

if sliceIndex < (args.length - 1)
# coffeelint: disable=max_line_length
console.warn """
linter-javac: The lint-command is presumed to break the limit of #{cliLimit} characters on the #{_os.platform()}-platform.
Dropping #{args.length - sliceIndex} source files, as a result javac may not resolve all dependencies.
"""
# coffeelint: enable=max_line_length
args = args.slice(0, sliceIndex) # cut args down
args.push(filePath) # ensure actual file is part


if @verboseLogging
@_log 'calling javac with',
args.length,
'arguments by invoking "', @javaExecutablePath,
'". The approximated command length is',
args.join(' ').length,
'characters long, the last argument is:',
args[args.length - 1]

# Execute javac
helpers.exec(@javaExecutablePath, args, {
stream: 'stderr',
cwd: wd,
allowEmptyStderr: true
})
.then (val) =>
if @verboseLogging
@_log 'parsing:\n', val
@parse(val, textEditor)
# Arguments to javac
args = ['-Xlint:all']
args = args.concat(['-cp', cp]) if cp
args = args.concat(['-sourcepath', sp]) if sp

# add additional options to the args-array
if @additionalOptions.length > 0
args = args.concat @additionalOptions
@_log 'adding', @additionalOptions.length, 'additional javac-options.'

@_log 'collected the following arguments:', args.join(' ')

# add javac argsfile if filename has been configured
if @javacArgsFilename
args.push('@' + @javacArgsFilename)
@_log 'adding', @javacArgsFilename, 'as argsfile.'

# Append the file to actually lint
args.push(filePath)

# Execute javac
helpers.exec(@javaExecutablePath, args, {
stream: 'stderr',
cwd: wd,
allowEmptyStderr: true
})
.then (val) =>
@_log 'parsing:\n', val
@parse(val, textEditor)

parse: (javacOutput, textEditor) ->
languageCode = @_detectLanguageCode javacOutput
Expand Down Expand Up @@ -231,8 +183,7 @@ module.exports =
lastIndex = messages.length - 1
messages[lastIndex].range[0][1] = column
messages[lastIndex].range[1][1] = column + 1
if @verboseLogging
@_log 'returning', messages.length, 'linter-messages.'
@_log 'returning', messages.length, 'linter-messages.'

return messages

Expand All @@ -253,59 +204,32 @@ module.exports =
# TODO: The following fails if there's a symlink in the path
return textEditor.getPath().substr(0, realpath.length) == realpath

getFilesEndingWith: (startPath, endsWith, ignoreFn) ->
foundFiles = []
folderFiles = []
voucher fs.readdir, startPath
.then (files) ->
folderFiles = files
Promise.all files.map (f) ->
filename = path.join startPath, f
voucher fs.lstat, filename
.then (fileStats) =>
mapped = fileStats.map (stats, i) =>
filename = path.join startPath, folderFiles[i]
if ignoreFn?(filename)
return undefined
else if stats.isDirectory()
return @getFilesEndingWith filename, endsWith, ignoreFn
else if filename.endsWith(endsWith)
return [ filename ]

Promise.all(mapped.filter(Boolean))

.then (fileArrays) ->
[].concat.apply([], fileArrays)

findClasspathConfig: (d) ->
# Search for the .classpath file starting in the given directory
findConfigFile: (directory, filename) ->
# Search for the config file starting in the given directory
# and searching parent directories until it is found, or we go outside the
# project base directory.
while atom.project.contains(d) or (d in atom.project.getPaths())
while atom.project.contains(directory) or (directory in atom.project.getPaths())
try
file = path.join d, @classpathFilename
file = path.join directory, filename
result =
cfgCp: fs.readFileSync(file, { encoding: 'utf-8' })
cfgDir: d
result.cfgCp = result.cfgCp.trim()
content: fs.readFileSync(file, { encoding: 'utf-8' }).trim()
dir: directory
return result
catch e
d = path.dirname(d)
directory = path.dirname(directory)

return null

_detectLanguageCode: (javacOutput) ->
if @verboseLogging
@_log 'detecting languages'
@_log 'detecting languages'
for language, pattern of @patterns
if javacOutput.match(pattern.detector)
if @verboseLogging
@_log 'detected the following language-code:', language
@_log 'detected the following language-code:', language
return language

return false

_log: (msgs...) ->
if (msgs.length > 0)
if (@verboseLogging && msgs.length > 0)
javacPrefix = 'linter-javac: '
console.log javacPrefix + msgs.join(' ')
24 changes: 18 additions & 6 deletions package.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,32 +27,44 @@
"javacExecutablePath": {
"type": "string",
"default": "javac",
"description": "Path to the javac executable. This setting will be used to call the java-compiler. The entered value should be immediately callable on commandline. Example: `C:\\Program Files\\Java\\jdk1.6.0_16\\bin\\javac.exe`. Keep in mind that placeholders like `~` do **not** work. If your [path-variable](https://en.wikipedia.org/wiki/PATH_%28variable%29) is set properly it should not be necessary to change the default."
"description": "Path to the javac executable. This setting will be used to call the java-compiler. The entered value should be immediately callable on commandline. Example: `C:\\Program Files\\Java\\jdk1.6.0_16\\bin\\javac.exe`. Keep in mind that placeholders like `~` do **not** work. If your [path-variable](https://en.wikipedia.org/wiki/PATH_%28variable%29) is set properly it should not be necessary to change the default.",
"order": 1
},
"additionalClasspaths": {
"type": "string",
"default": "",
"description": "Additional classpaths to be used (for the `-cp`-option) when calling javac, separate multiple paths using the right path-delimiter for your os (`:`/`;`). Be aware that existing classpath-definitions from the environment variable \"CLASSPATH\" will be merged into the argument, as well as the content of your optional [`.classpath`-files](https://atom.io/packages/linter-javac). Example: `/path1:/path2` will become `javac -cp :/path1:/path2`. Keep in mind that placeholders like `~` do **not** work."
"description": "Additional classpaths to be used (for the `-cp`-option) when calling javac, separate multiple paths using the right path-delimiter for your os (`:`/`;`). Be aware that existing classpath-definitions from the environment variable \"CLASSPATH\" will be merged into the argument, as well as the content of your optional [`.classpath`-files](https://atom.io/packages/linter-javac). Example: `/path1:/path2` will become `javac -cp :/path1:/path2`. Keep in mind that placeholders like `~` do **not** work.",
"order": 2
},
"additionalJavacOptions": {
"type": "string",
"default": "",
"description": "Your additional options will be inserted between the javac-command and the sourcefiles. Example: `-d /root/class-cache` will become `javac -Xlint:all -d /root/class-cache .../Test.java` take a look to the [javac-docs](http://docs.oracle.com/javase/8/docs/technotes/tools/unix/javac.html) for further information on valid options. Keep in mind that placeholders like `~` do **not** work."
"description": "Your additional options will be inserted between the javac-command and the sourcefiles. Example: `-d /root/class-cache` will become `javac -Xlint:all -d /root/class-cache .../Test.java` take a look to the [javac-docs](http://docs.oracle.com/javase/8/docs/technotes/tools/unix/javac.html) for further information on valid options. Keep in mind that placeholders like `~` do **not** work.",
"order": 3
},
"classpathFilename": {
"type": "string",
"default": ".classpath",
"description": "You can change the default .classpath filename. This is a useful option if You e.g. bump into conflicts with Eclipse users."
"description": "You can change the default .classpath filename. This is a useful option if You e.g. bump into conflicts with Eclipse users.",
"order": 4
},
"sourcepathFilename": {
"type": "string",
"default": ".sourcepath",
"description": "This file should point out all the locations where you have sources. That way javac can find other source files you're referring to in your code (which are likely not on the classpath).",
"order": 5
},
"javacArgsFilename": {
"type": "string",
"default": "",
"description": "Optionally you can define filename for a [javac argsfile](https://docs.oracle.com/javase/8/docs/technotes/tools/windows/javac.html#BHCCFGCD) that is located alongside with the .classpath file in the same directory. Contents of the argfile are passed to javac as arguments."
"description": "Optionally you can define filename for a [javac argsfile](https://docs.oracle.com/javase/8/docs/technotes/tools/windows/javac.html#BHCCFGCD) that is located alongside with the .classpath file in the same directory. Contents of the argfile are passed to javac as arguments.",
"order": 6
},
"verboseLogging": {
"type": "boolean",
"default": false,
"description": "If enabled linter-javac will log its internal states to the console. This option is intended for debugging purposes **only** - enable this **only temporarily**, if the linter is not working as expected. You can access the output from the dev-console (alt+cmd+i), you may file the log-messages to a proper issue in order to help us improving this package. Do not forget to disable this option afterwards - since this may lead to performance issues."
"description": "If enabled linter-javac will log its internal states to the console. This option is intended for debugging purposes **only** - enable this **only temporarily**, if the linter is not working as expected. You can access the output from the dev-console (alt+cmd+i), you may file the log-messages to a proper issue in order to help us improving this package. Do not forget to disable this option afterwards - since this may lead to performance issues.",
"order": 7
}
}
}

0 comments on commit 03ae1a9

Please sign in to comment.