Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Iterative Parsing (for issue #35) #76

Merged
merged 23 commits into from
Jul 18, 2014

Conversation

thebusytypist
Copy link
Contributor

  • Implementation Overview
    This pull request provides an iterative(non-recursive) implementation of JSON parsing.
    It tries to resolve the issue Research on using custom stack instead of call-stack in Reader #35 by:

    • saving parser state in an internal stack rather than using function local stack. In this way the complexity of function stack size is constant.
    • providing a stack size limit to the parsing function.
  • Implementation Details
    This pull request adds a new flag kParseIterativeFlag to ParseFlag.
    Parsing work with this flag will be directed to GenericReader::IterativeParse, which is the entry point of this iterative parser.
    Then the parser will traverse the given token stream and make transitions on a state machine.
    The state transition table is manually encoded in a static array, which is in function GenericReader::Predict.
    And the action code for every transition is in function GenericReader::Transit.

  • Performance Report
    Performance tests have been taken among:

    • kParseDefaultFlags (recursive parser, without in-situ)
    • kParseIterativeFlag (iterative parser, without in-situ)
    • kParseInsituFlag (recursive parser, with in-situ)
    • kParseIterativeFlag | kParseInsituFlag (iterative parser, with in-situ)

    I expected the parsers with kParseIterativeFlag are either slower or quicker than their counterparts.
    But the iterative parser without in-situ flag runs exceptionally slower than other parsers, while the iterative parser with in-situ being the most quick one.
    @pah pointed out this may be compiler-specific in Research on using custom stack instead of call-stack in Reader #35 (comment).

@pah
Copy link
Contributor

pah commented Jul 17, 2014

The build currently fails (due to -Werror) with the warning:

../../include/rapidjson/reader.h:1233:29: error: ‘rapidjson::GenericReader<rapidjson::UTF8<>, rapidjson::UTF8<> >::stack_’ will be initialized after [-Werror=reorder]
../../include/rapidjson/reader.h:1232:15: error: ‘const size_t rapidjson::GenericReader<rapidjson::UTF8<>, rapidjson::UTF8<> >::kStackSizeLimit’ [-Werror=reorder]
../../include/rapidjson/reader.h:280:2: error: when initialized here [-Werror=reorder]

You should move the kStackSizeLimit member after the stack_ member to fix it. Secondly, following the current naming conventions, the member should probably be named stackSizeLimit_ instead.

miloyip added a commit that referenced this pull request Jul 18, 2014
@miloyip miloyip merged commit 19a2279 into Tencent:master Jul 18, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants