Wanting to use Parlot to replace a hand-written shunting-yard parser, would like feedback

[CyberBotX]

Hi there. I figured this was better as a discussion since it isn't an issue with Parlot per-se.

So I have a math/dice expression parser that was written by me in C++ that I was converting to C# to use within a Discord bot, and in the process, I decided to have a look at other parser generators and parser combinators as potential replacements. I found that Parlot is faster and uses less memory than my own solution, so I'm currently considering replacing my parser with Parlot. But there were a few things I wanted to see if they were possibly with Parlot or not.

The main thing to point out is that my parser took input that had no whitespace, so I settled on using Parlot's Literals instead of Terms.

Firstly, in my parser, I would preprocess the input, so that a lone % becomes 1d100, any dX (where X is a number) that is found without a number before it gets an implicit 1 placed before it (so if I did something like d6+1 it would be treated like 1d6+1 and if I did 1+d6 it would be treated like 1+1d6, as examples), and it would add in *s to handle implicit multiplication. The last one isn't that important to me, but the other two, I'm wondering if Parlot could handle this on its own or if I'd still need to preprocess the input or not. I'd rather not have to preprocess the input, because that implies knowing what tokens are what ahead-of-time (so I'd need to know what characters are operators, what characters are numbers, what characters are constants, etc.). Technically, it also had to convert what it assumed were unary minus signs into another character (I used _) so that I could handle it properly in the shunting-yard algorithm later, but I find I don't need to do that with Parlot due to how it handles the expressions.

The next thing my parser did was check the "fixed" expression. It basically checked for function calls being invalid (basically missing the open parenthesis), checking to make sure commas and operators had valid characters before and after them, and making sure that parentheses were placed in proper spots, and it would report on exactly where that happened, as well as if an unexpected character was encountered. The best I've been able to do with Parlot is to let me know when I hadn't closed an open parenthesis, when my primary expression (which is a number, a function call, a grouped expression or a constant) hasn't been satisfied and when I get to the end of the line unexpectedly (like if I did 1+2) for instance). I'm wondering if that is enough to cover the problems. I also technically put an ElseError on my unary expression, but I don't think it would ever actually get hit?

After that I tokenize my input by injecting spaces between everything so that the shunting-yard algorithm knows what to do, but obviously I don't need that since Parlot handles that.

I use Parlot to create class instances that inherit ultimately from a base class, and each class instance has an Evaluate() method, with the ones for numbers and constants giving out a concrete value and all others evaluating a nested expression. This works quite well in comparison to taking the postfix notation received from the shunting-yard algorithm and parsing that, plus with Parlot, I am more able to define how my function calls work, so I can specifically say that some functions only allow for one argument and some allow for 2 arguments and some allow for 2+ arguments.

Mainly, I'd like input to see if what I've done is good for replacing my previous parser. For reference, my original parser is located primarily starting here: https://github.com/CyberBotX/DiceServ/blob/7b260e254e011113f03cd46b1168f2d5e44a9825/diceserv.cpp#L673 and below is what I am planning to do with Parlot.

// Some of this comes from the Parlot library's fluent example, some of it comes from another project, NCalc, but stripped down a lot

/*
 * Grammar:
 * additive       => multiplicative ( ( "-" | "+" ) multiplicative )* ;
 * multiplicative => exponential ( ( "/" | "*" | "%" ) exponential )* ;
 * exponential    => dice ( "^" dice )* ;
 * dice           => unary ( "d" unary )* ;
 * unary          => ( "-" ) primary ;
 *
 * primary        => NUMBER
 *                  | function
 *                  | "(" expression ")" 
 *                  | CONSTANT ;
 *
 * function       => Identifier "(" arguments ")" ;
 * arguments      => expression ( "," expression )* ;
 */

// The Deferred helper creates a parser that can be referenced by others before it is defined
var expression = Parsers.Deferred<Expression>();

var number = Parsers.Literals.Number<double>(NumberOptions.Float).Then<Expression>(static d => new NumberExpression(d));

var constant = Parsers.OneOf(
	Parsers.Literals.Text("e", true).Then<Expression>(new EExpression()),
	Parsers.Literals.Text("pi", true).Then<Expression>(new PiExpression()),
	Parsers.Literals.Text("tau", true).Then<Expression>(new TauExpression())
);

var comma = Parsers.Literals.Char(',');
var d = Parsers.Literals.Char('d').Or(Parsers.Literals.Char('D'));
var caret = Parsers.Literals.Char('^');
var percent = Parsers.Literals.Char('%');
var slash = Parsers.Literals.Char('/');
var asterisk = Parsers.Literals.Char('*');
var minus = Parsers.Literals.Char('-');
var plus = Parsers.Literals.Char('+');
var lParen = Parsers.Literals.Char('(');
var rParen = Parsers.Literals.Char(')');

// "(" expression ")"
var groupExpression = Parsers.Between(lParen, expression, rParen.ElseError("Parenthesis not closed"));

var oneArgFunction = Parsers.OneOf(
	Parsers.Literals.Text("abs", true),
	Parsers.Literals.Text("acosh", true),
	Parsers.Literals.Text("acos", true),
	Parsers.Literals.Text("asinh", true),
	Parsers.Literals.Text("asin", true),
	Parsers.Literals.Text("atanh", true),
	Parsers.Literals.Text("atan", true),
	Parsers.Literals.Text("cbrt", true),
	Parsers.Literals.Text("ceil", true),
	Parsers.Literals.Text("cosh", true),
	Parsers.Literals.Text("cos", true),
	Parsers.Literals.Text("deg", true),
	Parsers.Literals.Text("exp", true),
	Parsers.Literals.Text("fac", true),
	Parsers.Literals.Text("floor", true),
	Parsers.Literals.Text("ilogb", true),
	Parsers.Literals.Text("log10", true),
	Parsers.Literals.Text("log2", true),
	Parsers.Literals.Text("log", true),
	Parsers.Literals.Text("rad", true),
	Parsers.Literals.Text("round", true),
	Parsers.Literals.Text("sinh", true),
	Parsers.Literals.Text("sin", true),
	Parsers.Literals.Text("sqrt", true),
	Parsers.Literals.Text("tanh", true),
	Parsers.Literals.Text("tan", true),
	Parsers.Literals.Text("trunc", true)
).And(Parsers.Between(lParen, expression, rParen)).Then<Expression>(static x => x.Item1 switch
{
	"abs" => new AbsExpression(x.Item2),
	"acos" => new AcosExpression(x.Item2),
	"acosh" => new AcoshExpression(x.Item2),
	"asin" => new AsinExpression(x.Item2),
	"asinh" => new AsinhExpression(x.Item2),
	"atan" => new AtanExpression(x.Item2),
	"atanh" => new AtanhExpression(x.Item2),
	"cbrt" => new CbrtExpression(x.Item2),
	"ceil" => new CeilExpression(x.Item2),
	"cos" => new CosExpression(x.Item2),
	"cosh" => new CoshExpression(x.Item2),
	"deg" => new DegExpression(x.Item2),
	"exp" => new ExpExpression(x.Item2),
	"fac" => new FacExpression(x.Item2),
	"floor" => new FloorExpression(x.Item2),
	"ilogb" => new ILogBExpression(x.Item2),
	"log" => new LogExpression(x.Item2),
	"log10" => new Log10Expression(x.Item2),
	"log2" => new Log2Expression(x.Item2),
	"rad" => new RadExpression(x.Item2),
	"round" => new RoundExpression(x.Item2),
	"sin" => new SinExpression(x.Item2),
	"sinh" => new SinhExpression(x.Item2),
	"sqrt" => new SqrtExpression(x.Item2),
	"tan" => new TanExpression(x.Item2),
	"tanh" => new TanhExpression(x.Item2),
	"trunc" => new TruncExpression(x.Item2),
	_ => null!
});

var twoArgFunction = Parsers.OneOf(
	Parsers.Literals.Text("atan2", true),
	Parsers.Literals.Text("rand", true)
).And(Parsers.Between(lParen, expression.AndSkip(comma).And(expression), rParen.ElseError("Parenthesis not closed"))).
	Then<Expression>(static x => x.Item1 switch
	{
		"atan2" => new Atan2Expression(x.Item2.Item1, x.Item2.Item2),
		"rand" => new RandExpression(x.Item2.Item1, x.Item2.Item2),
		_ => null!
	});

var threeArgFunction = Parsers.OneOf(
	Parsers.Literals.Text("clamp", true)
).And(Parsers.Between(lParen, expression.AndSkip(comma).And(expression).AndSkip(comma).And(expression), rParen.ElseError("Parenthesis not closed"))).
	Then<Expression>(static x => x.Item1 switch
	{
		"clamp" => new ClampExpression(x.Item2.Item1, x.Item2.Item2, x.Item2.Item3),
		_ => null!
	});

var variableArgFunction = Parsers.OneOf(
	Parsers.Literals.Text("max", true),
	Parsers.Literals.Text("min", true)
).And(Parsers.Between(lParen, expression.AndSkip(comma).And(Parsers.Separated(comma, expression)), rParen.ElseError("Parenthesis not closed"))).
	Then<Expression>(static x => x.Item1 switch
	{
		"max" => new MaxExpression([x.Item2.Item1, .. x.Item2.Item2]),
		"min" => new MinExpression([x.Item2.Item1, .. x.Item2.Item2]),
		_ => null!
	});

// primary => NUMBER | function | groupExpression | CONSTANT ;
var primary =
	Parsers.OneOf(number, variableArgFunction, threeArgFunction, twoArgFunction, oneArgFunction, groupExpression, constant).ElseError("Expected a number, function call, constant or parenthesis");

// ( "-" ) primary | primary;
var unary = primary.Unary((minus, static x => new NegateExpression(x))).ElseError("Expected expression after unary '-'");

// dice => unary ( "d" unary )* ;
var dice = unary.LeftAssociative((d, static (a, b) => new DiceExpression(a, b)));

// exponential => dice ( "^" dice )* ;
var power = dice.RightAssociative((caret, static (a, b) => new ExponentExpression(a, b)));

// multiplicative => exponential ( ( "/" | "*" | "%" ) exponential )* ;
var multiplicative = power.LeftAssociative(
	(percent, static (a, b) => new ModulusExpression(a, b)),
	(slash, static (a, b) => new DivisionExpression(a, b)),
	(asterisk, static (a, b) => new MultiplicationExpression(a, b))
);

// additive => multiplicative ( ( "-" | "+" ) multiplicative )* ;
var additive = multiplicative.LeftAssociative(
	(plus, static (a, b) => new AdditionExpression(a, b)),
	(minus, static (a, b) => new SubtractionExpression(a, b))
);

expression.Parser = additive;

var lambda = expression.Eof().ElseError("Unexpected end of line");