Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mark parsing higher order tokens easier. (perhaps implement Compare for &[T] where T:Compare ?) #637

Open
2 tasks done
rbtcollins opened this issue Dec 21, 2024 · 1 comment
Labels
C-enhancement Category: Raise on the bar on expectations
Milestone

Comments

@rbtcollins
Copy link
Contributor

Please complete the following tasks

winnow version

0.6.20

Describe your use case

I'm doing a small codecraft project with a language interpreter, and the design has multiple layers of parsing - string to tokens, and tokens to ast, etc.

It all works fine, but it is not very ergonomic:

To match [Token:X, Token:Y] I seem to need to do:

(
    one_of(|t| matches!(t, Token::X(_))).map(|t| match t {
                Token::X(s) => s.to_string(),
                _ => unreachable!(),
            }),
    one_of(|t| matches!(t, Token::Y)),
)

The ergonomics vs parsing strings are very vivid - there I could do

(any, "Y") or use literal() etc.

I cannot implement Compare, which literal depends on, as it hits the foreign type rule. For one_of, ContainsToken is sufficient.


pub fn test(tokens: Vec<Token>) -> anyhow::Result<()> {
    use winnow::token::literal;
    let mut p = literal(&[Token::Var][..]);
    p.parse(&tokens[..])
        .map(|_| ())
        .map_err(|e: ParseError<_, ContextError>| anyhow!("failed {e:?}"))
}

pub fn test2(tokens: Vec<Token>) -> anyhow::Result<()> {
    use winnow::token::one_of;
    let mut p = one_of(&[Token::Var][..]);
    p.parse(&tokens[..])
        .map(|_| ())
        .map_err(|e: ParseError<_, ContextError>| anyhow!("failed {e:?}"))
}

What I would like to be able to write things like

(Token::If, Token::LeftParen, expr, Token::RightParen, statement).parse_next(i)

Describe the solution you'd like

See the use case

Alternatives, if applicable

No response

Additional Context

No response

@rbtcollins rbtcollins added the C-enhancement Category: Raise on the bar on expectations label Dec 21, 2024
@epage
Copy link
Collaborator

epage commented Dec 21, 2024

FYI we are exploring improving lex+parsing in #591.

@epage epage added this to the 0.7.0 milestone Jan 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-enhancement Category: Raise on the bar on expectations
Projects
None yet
Development

No branches or pull requests

2 participants