Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrong function name parsed when method escapes first argument with ~ #90

Closed
PekaryGergelyR opened this issue Apr 3, 2019 · 1 comment

Comments

@PekaryGergelyR
Copy link
Contributor

The following class:

classdef example < handle
    methods
        function output = someMethod1(obj, argument)
            % Docstring
        end
        function output = someMethod2(~, argument)
            % Docstring
        end
    end
end

Will be tokenized as:

[(Token.Keyword, 'classdef')
 (Token.Text, ' ')
 (Token.Name, 'example')
 (Token.Text, ' ')
 (Token.Operator, '<')
 (Token.Text, ' ')
 (Token.Name, 'handle')
 (Token.Text, '\n')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Keyword, 'methods')
 (Token.Text, '\n')
 (Token.Keyword, '        function')
 (Token.Text.Whitespace, ' ')
 (Token.Text, 'output ')
 (Token.Punctuation, '=')
 (Token.Text.Whitespace, ' ')
 (Token.Name.Function, 'someMethod1')   <---------------- Good method name
 (Token.Punctuation, '(')
 (Token.Text, 'obj, argument')
 (Token.Punctuation, ')')
 (Token.Text.Whitespace, '\n            ')
 (Token.Comment, '% Docstring')
 (Token.Text, '\n')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Keyword, 'end')
 (Token.Text, '\n')
 (Token.Keyword, '        function')
 (Token.Text.Whitespace, ' ')
 (Token.Text, 'output ')
 (Token.Punctuation, '=')
 (Token.Text.Whitespace, ' ')
 (Token.Name.Function, 'someMethod2()') <---------------- Bad method name with ()
 (Token.Punctuation, '(')
 (Token.Text, '~, argument')
 (Token.Punctuation, ')')
 (Token.Text.Whitespace, '\n            ')
 (Token.Comment, '% Docstring')
 (Token.Text, '\n')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Keyword, 'end')
 (Token.Text, '\n')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Text, ' ')
 (Token.Keyword, 'end')
 (Token.Text, '\n')
 (Token.Keyword, 'end')
 (Token.Text, '\n')]

This happens to functions as well, however, this issue is not so common there probably.

I'd suggest the following changes in MatFunction class:

    def __init__(self, name, modname, tokens):
        if name is not None:
            name = re.sub('\(\)$', '', name)
        super(MatFunction, self).__init__(name)

and

# =====================================================================
            # function name
            func_name = tks.pop()
            temp_name = re.sub('\(\)$', '', func_name[1])
            if (Token.Name.Function, temp_name) != (Token.Name.Function, self.name):  # @UndefinedVariable
                if isinstance(self, MatMethod):
                    self.name = temp_name
                else:
                    msg = '[sphinxcontrib-matlabdomain] Unexpected function name: "%s".' % name
                    msg += ' Expected "{}" in module "{}".'.format(name, modname)
                    logger.warning(msg)

The first code should fix functions and the second the class methods.

@PekaryGergelyR PekaryGergelyR changed the title Wrong function named parsed when method escapes first argument with ~ Wrong function name parsed when method escapes first argument with ~ Apr 3, 2019
@joeced joeced closed this as completed in b9e921d Apr 3, 2019
@joeced
Copy link
Collaborator

joeced commented Apr 3, 2019

Thanks for the bug-report. The detail of the reports are really appreciated!

apozharski added a commit that referenced this issue Sep 18, 2024
* [WIP] Work on writing a parser using the textmate grammar backend.   (#253)

* Actually not only enums, quick and dirty start to using textmate parser

* a little more boilerplate

* property validator parsing

* working function parsing without docstrings yet

* start enum work

* some enum parsing

* working parsing for enumeration comments

* add handling for block comments to enums

* backport enum docstring parsing to properties

* remove vestigial file

* minor fixes + black

* Hack for object hierarchy

* Initial hack to get enumerations to work

* better classdef parsing including changes to MATLAB-language-grammar prs #86, #88, and #90

* parse function docstring

* extract function parser

* initial work

* finish integrating mat_textmate_parser with mat_types

* [skip-ci] some minor changes

* initial work on a tree sitter based parser

* nearly finished with tree-sitter implementation

* everything but events working

* working events

* exit early if query returns for block with no elements

* integrating tree-sitter parser into mat_types

* fixing default value parsing

* some test fixes and requires tree-sitter

* rm textmate parser on this branch

* also install tree-sitter

* bump required tree-sitter

* tree-sitter version bump

* dealing with tree-sitter version diffs to maintain py 3.8 compatibility

* ML_LANG versions

* a better attributes query

* Ci: Test on Sphinx 8 / Dev. (#259)

* CI: Testing for latest Sphinx (8.0)

* CI: Fix helper class version checking.

* Fixing nearly all tests in test_parse_mfile

* fix a _lot_ of autodoc

* temporarily point to tree-sitter-matlab branch on apozharski fork

* fix old property syntax + update tests

* fixing test_autodoc and test_matlabify, only comment and line continuation issues remain

* fixing the last of the tests

* remove dead code

* address PR comments made by @joeced

* fix

* lint

* fix typo in __all__

* remove Pygments dependencies

* py-tree-sitter v0.23.0 has a breaking change for Query.match()

* remove duplicat entry in yaml

* fix spec printing

* minor fixes for arg block parsing and a test

* fix bug regarding output block argument parsing and add test

* remove print and fix test_matlabify

* remove textmate from dev-reqs

* fix typo

---------

Co-authored-by: Jørgen Cederberg <jorgen@cederberg.be>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants