Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow underscores in integers with bases #190

Merged

Conversation

irevoire
Copy link
Contributor

@irevoire irevoire commented Oct 2, 2023

Hey, it would be nice to allow underscores in numbers with bases.

@sharkdp
Copy link
Owner

sharkdp commented Oct 3, 2023

Thank you very much for your contribution.

This sounds like a good idea, yes 👍 Can we please add some tests? I think we should disallow things like 0x_0 or 0x0_. We should definitely disallow 0x_ which currently leads to a "overflow in number literal" error.

@irevoire
Copy link
Contributor Author

irevoire commented Oct 5, 2023

Hey @sharkdp, it's pretty much done, I added a lot of tests for every possible case I could think of.
To ease the addition and maintenance of new tests, I would like to introduce insta to the codebase, are you okay with this change?
We use it extensively at work, and honestly, we just won so much time since we switched on it 😅
Writing and updating tests gets so much faster.

numbat/src/tokenizer.rs Outdated Show resolved Hide resolved
numbat/src/tokenizer.rs Outdated Show resolved Hide resolved
numbat/src/tokenizer.rs Outdated Show resolved Hide resolved
Copy link
Owner

@sharkdp sharkdp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the updates. I have just a few more comments.

To ease the addition and maintenance of new tests, I would like to introduce insta to the codebase, are you okay with this change?

I am okay with this, but then we should probably go all in and also replace the other tokenizer and parser unit tests with insta (not in this PR)? For the parser unit tests, this could actually be really nice and replace a lot of hand-coded ASTs. We might also be able to get rid of my custom-made ReplaceSpans/replace_spans logic by using instas redactions feature on the span fields — which would be really nice.

@irevoire
Copy link
Contributor Author

Hey @sharkdp!
I finally got the time to take a second look at this.

On my computer the execution of your test suites goes from 23s to 2s so I guess it's a success somehow.
But I’m really not sure if I did the right thing, please take a look at 0a60f80b.
I had to change some of your type to requires that they're Sync + I derived Clone on a lot of type without knowing if it was actually correct.

@irevoire irevoire force-pushed the handle-underscore-in-integer-with-base branch from 0a60f80 to 7c1df9f Compare October 16, 2023 09:17
@irevoire
Copy link
Contributor Author

Rebased on main

Copy link
Owner

@sharkdp sharkdp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much!

On my computer the execution of your test suites goes from 23s to 2s so I guess it's a success somehow.

Awesome!

But I’m really not sure if I did the right thing, please take a look at 0a60f80b.
I had to change some of your type to requires that they're Sync + I derived Clone on a lot of type without knowing if it was actually correct.

Looks good to me 👍

@sharkdp sharkdp merged commit 1033cf4 into sharkdp:master Oct 18, 2023
@sharkdp
Copy link
Owner

sharkdp commented Oct 18, 2023

Changes are online on https://numbat.dev

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants