Add shell parser #2598
Merged
Add shell parser #2598
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… support for it in Javascript, Perl, Python, Ruby and Bash lexers.
If '$' is directly followed by '#' the variable ends right there. This is especially important at the end of a string, as the closing quote would be lexed as part of the variable instead, but was wrong in every other case as well. Fixed by simply removing the '#?' and letting the following '.' eat it.
2. extract base session lexer ShellSessionBaseLexer.
Add a lexer for the Test Anything Protocol (TAP).
…422) Add FishShellLexer
Improve tests. Use latest pygmars Use newer parameter-expansion2 (temp advanced patched version) Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
And more and regenerate lexing tests using this new function Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
4 tasks
|
Note: the reason why there are so many commits is that I forked Pygments Bashlexer as a based, keeping the commit history by using |
- Use improved parameter-expansion - Parse bash arrays - Simplify lexing and grammar for better readability - Add new "needed_variables" argument to parsing top-level function to limit the set of variables and errors returned to these named in this set. - Improve tracing and variable resolution logic. Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
In license detection, we now accept extra expression symbols to use for license expression detection. This enables the support of custom SPDX-like license symbols used in some package repos and extra non-SPDX symbols thta are also commonly used. - Add a new function "match_spdx_lid.get_spdx_expression()" - Add new argument "expression_symbols" to all needed functions and methods. - Update doc strings accordingly Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This rule is mostly bsd-new and seldom bsd-simplified. We now account for this with reporting a bsd-new of gpl and a lowered relevance Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
These are exceptions to Copyleft licenses that strictly apply to OpenSSL and do not turn the licenses in a some Copyleft Limited license. Therefore I am making these Copyleft instead of Copyleft Limited. Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
- Parse APKBUILDs including shell parameter expansion using a shell parser and some minimal fixes to ensure we can resolve all key package metadata. (in particular sources) - Refine installed packages data processing to align this with APKBUILD parsing - Add new declared_license detection that can correctly detect licenses on all Alpine 3.13 and 3.14 packages. It is composed of expression syntax fixes, flexible expression parsing and non-SPDX symbol substitution Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This was passing with previsou license updates Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
In the refactoring of 64ad114 a test was not carried over to ensure we can handle empty license expression such as "SPDX-License-Identifier: " and not attempt to return an empty match (with an empty Rule which would fail) if this happens. Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
- Apply minor cleanups and formatting - Refactor EnhancedDebianCopyright to use consistently the method debian_copyright.paragraphs() to select parapgraphs. Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Use proper quotes rather than backticks Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
These rules were detected incorrectly and theyu are closely related to other simpler rules. Bumping the coverage helped fix the issues Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This test nee to be updated sometimes when we add new words in the license index. This is such a case Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This test nee to be updated sometimes when we add new words in the license index. This is such a case. Added a comment to clarify why this is happening. Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
This is better than representation Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
Bump pygmars Remove setuptools, wheel and pip pins Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
|
Things are looking greenish... So I am merging this now! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is need to parse static package manifest written in bash/sheel such as APKBUILD/PKGBUILD/ebuild/autotools and more (possibly some bitbake too?)
Tasks
Run tests locally to check for errors.