Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xdr: Add a custom marshaller for claim predicate timestamp #3183

Merged

Conversation

tamirms
Copy link
Contributor

@tamirms tamirms commented Nov 2, 2020

PR Checklist

PR Structure

  • This PR has reasonably narrow scope (if not, break it down into smaller PRs).
  • This PR avoids mixing refactoring changes with feature changes (split into two PRs
    otherwise).
  • This PR's title starts with name of package that is most changed in the PR, ex.
    services/friendbot, or all or doc if the changes are broad or impact many
    packages.

Thoroughness

  • This PR adds tests for the most critical parts of the new functionality or fixes.
  • I've updated any docs (developer docs, .md
    files, etc... affected by this change). Take a look in the docs folder for a given service,
    like this one.

Release planning

  • I've updated the relevant CHANGELOG (here for Horizon) if
    needed with deprecations, added features, breaking changes, and DB schema changes.
  • I've decided if this PR requires a new major/minor version according to
    semver, or if it's mainly a patch change. The PR is targeted at the next
    release branch if it's not a patch change.

What

Add a custom marshaller for representing the AbsBefore timestamp.

Why

Currently, claimable balances which have a AbsBefore timestamp beyond year 9999 causes Horizon ingestion to crash. The crash is due to the fact that the time.Parse() implementation does not support parsing strings that far into the future.

Known limitations

  • Using a regexp to get around the fact that time.Parse() doesn't accept time.Time instances with years outside the range of 0000..9999 range is a bit hacky
  • Perhaps it would be cleaner to represent AbsBefore as a unix timestamp integer. However, doing such a change now would break the API and it would require changes downstream in the SDK implementations.

@cla-bot cla-bot bot added the cla: yes label Nov 2, 2020
@tamirms tamirms requested a review from a team November 2, 2020 20:00
@tamirms tamirms changed the base branch from master to release-horizon-v1.11.0 November 2, 2020 20:05
xdr/json.go Outdated
if string(b) == "null" {
return nil
}
if b[0] != '"' || b[len(b)-1] != '"' {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should check len(b), otherwise it will panic for len(b) < 2.

xdr/json.go Outdated Show resolved Hide resolved
xdr/json.go Outdated Show resolved Hide resolved
xdr/json.go Outdated Show resolved Hide resolved
xdr/json.go Outdated
Comment on lines 29 to 53
if string(b) == "null" {
return nil
}
if b[0] != '"' || b[len(b)-1] != '"' {
return fmt.Errorf("%s does not begin and end with double quotes", b)
}
trimmed := string(b[1 : len(b)-1])
m := reISO8601.FindStringSubmatch(trimmed)

if len(m) != 4 {
return fmt.Errorf("UnmarshalJSON: cannot parse %s", trimmed)
}
// No need to check for errors since the regexp guarantees the matches
// are valid integers
year, _ := strconv.Atoi(m[1])
month, _ := strconv.Atoi(m[2])
day, _ := strconv.Atoi(m[3])

ts, err := time.Parse(time.RFC3339, "2006-01-02"+trimmed[len(m[0]):])
if err != nil {
return errors.Wrap(err, "Could not extract time")
}

t.Time = time.Date(year, time.Month(month), day, ts.Hour(), ts.Minute(), ts.Second(), ts.Nanosecond(), ts.Location())
return nil
Copy link
Member

@leighmcculloch leighmcculloch Nov 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 I think we're doing more lifting than we need to do with the parsing we're doing here. Checking null, and looking for quotes is all work that the json package will do for us. We also don't need to parse out the full date with a regex manually given that the time package will do that for us. I think we should leverage json.Unmarshal if we can, so that we reduce our chance of not handling edge cases as much as possible.

This is one approach:

Suggested change
if string(b) == "null" {
return nil
}
if b[0] != '"' || b[len(b)-1] != '"' {
return fmt.Errorf("%s does not begin and end with double quotes", b)
}
trimmed := string(b[1 : len(b)-1])
m := reISO8601.FindStringSubmatch(trimmed)
if len(m) != 4 {
return fmt.Errorf("UnmarshalJSON: cannot parse %s", trimmed)
}
// No need to check for errors since the regexp guarantees the matches
// are valid integers
year, _ := strconv.Atoi(m[1])
month, _ := strconv.Atoi(m[2])
day, _ := strconv.Atoi(m[3])
ts, err := time.Parse(time.RFC3339, "2006-01-02"+trimmed[len(m[0]):])
if err != nil {
return errors.Wrap(err, "Could not extract time")
}
t.Time = time.Date(year, time.Month(month), day, ts.Hour(), ts.Minute(), ts.Second(), ts.Nanosecond(), ts.Location())
return nil
var s *string
err := json.Unmarshal(data, &s)
if err != nil {
return err
}
if s == nil {
return nil
}
parts := strings.SplitN(*s, "-", 2)
year, err := strconv.ParseInt(parts[0], 10, 64)
if err != nil {
return err
}
rest, err := time.Parse(time.RFC3339, "0000-"+parts[1])
if err != nil {
return err
}
t.Time = rest.AddDate(int(year), 0, 0)
return nil

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unfortunately splitting on the first "-" doesn't work on timestamps which have a negative year

@tamirms tamirms merged commit aa35ab6 into stellar:release-horizon-v1.11.0 Nov 2, 2020
@tamirms tamirms deleted the custom-timestamp-marshaller branch November 2, 2020 21:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants