Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Non-blocking parser reports incorrect locations when fed with non-zero offset #531

Closed
dnault opened this issue Apr 4, 2019 · 1 comment

Comments

@dnault
Copy link

dnault commented Apr 4, 2019

When feeding a non-blocking parser, the input array offset leaks into the offsets reported by getCurrentLocation() and getTokenLocation().

For example, feeding with an offset of 7 yields tokens whose reported locations are 7 greater than they should be. Likewise the current location reported by the parser is 7 greater than the correct location.

It's not possible for a user to work around this issue by subtracting 7 from the reported locations, because the token location may have been established by an earlier feeding with a different offset.

Jackson version: 2.9.8

Unit test:

import com.fasterxml.jackson.core.JsonFactory;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonToken;
import com.fasterxml.jackson.core.async.ByteArrayFeeder;
import org.junit.Test;

import static java.nio.charset.StandardCharsets.UTF_8;
import static org.junit.Assert.assertEquals;

public class FeedingOffsetTest {

  @Test
  public void inputOffsetShouldNotAffectLocations() throws Exception {
    JsonFactory jsonFactory = new JsonFactory();
    JsonParser parser = jsonFactory.createNonBlockingByteArrayParser();
    ByteArrayFeeder feeder = (ByteArrayFeeder) parser.getNonBlockingInputFeeder();

    byte[] input = "[[[".getBytes(UTF_8);

    feeder.feedInput(input, 2, 3);
    assertEquals(JsonToken.START_ARRAY, parser.nextToken());
    assertEquals(1, parser.getCurrentLocation().getByteOffset()); // ACTUAL = 3
    assertEquals(1, parser.getTokenLocation().getByteOffset());   // ACTUAL = 3

    feeder.feedInput(input, 0, 1);
    assertEquals(JsonToken.START_ARRAY, parser.nextToken());
    assertEquals(2, parser.getCurrentLocation().getByteOffset());
    assertEquals(2, parser.getTokenLocation().getByteOffset());
  }
}
@cowtowncoder
Copy link
Member

Thank you for reporting this problem.

@cowtowncoder cowtowncoder added this to the 2.10 milestone Apr 6, 2019
@cowtowncoder cowtowncoder removed this from the 2.10 milestone Apr 6, 2019
cowtowncoder added a commit that referenced this issue Apr 6, 2019
finaglehelper pushed a commit to twitter/util that referenced this issue Jul 25, 2019
Problem
In order to make util cross building with scala 2.13, we need to move to Jackson
2.9.9 for 2.13 support.

Solution
Upgrade it.

Result
As a bonus, Jackson fixes a bug
FasterXML/jackson-core#531 when feeding async json
parser, remove the `slicedOffset` as now parser records the right position.

Differential Revision: https://phabricator.twitter.biz/D345969
finaglehelper pushed a commit to twitter/finagle that referenced this issue Jul 25, 2019
Problem
In order to make util cross building with scala 2.13, we need to move to Jackson
2.9.9 for 2.13 support.

Solution
Upgrade it.

Result
As a bonus, Jackson fixes a bug
FasterXML/jackson-core#531 when feeding async json
parser, remove the `slicedOffset` as now parser records the right position.

Differential Revision: https://phabricator.twitter.biz/D345969
finaglehelper pushed a commit to twitter/twitter-server that referenced this issue Jul 25, 2019
Problem
In order to make util cross building with scala 2.13, we need to move to Jackson
2.9.9 for 2.13 support.

Solution
Upgrade it.

Result
As a bonus, Jackson fixes a bug
FasterXML/jackson-core#531 when feeding async json
parser, remove the `slicedOffset` as now parser records the right position.

Differential Revision: https://phabricator.twitter.biz/D345969
finaglehelper pushed a commit to twitter/finatra that referenced this issue Jul 25, 2019
Problem
In order to make util cross building with scala 2.13, we need to move to Jackson
2.9.9 for 2.13 support.

Solution
Upgrade it.

Result
As a bonus, Jackson fixes a bug
FasterXML/jackson-core#531 when feeding async json
parser, remove the `slicedOffset` as now parser records the right position.

Differential Revision: https://phabricator.twitter.biz/D345969
alanbato pushed a commit to twitter/finagle that referenced this issue Jul 29, 2019
Problem
In order to make util cross building with scala 2.13, we need to move to Jackson
2.9.9 for 2.13 support.

Solution
Upgrade it.

Result
As a bonus, Jackson fixes a bug
FasterXML/jackson-core#531 when feeding async json
parser, remove the `slicedOffset` as now parser records the right position.

GitOrigin-RevId: d63afbf0bdede0777cf869b83a9620e76cb3c6a2
alanbato pushed a commit to twitter/finagle that referenced this issue Jul 29, 2019
Problem
In order to make util cross building with scala 2.13, we need to move to Jackson
2.9.9 for 2.13 support.

Solution
Upgrade it.

Result
As a bonus, Jackson fixes a bug
FasterXML/jackson-core#531 when feeding async json
parser, remove the `slicedOffset` as now parser records the right position.

GitOrigin-RevId: d63afbf0bdede0777cf869b83a9620e76cb3c6a2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants