- #5, ec2233e - Avoid panic on spelling inconsistencies errors.
- #7, bff52b5 - Fix the calculation of inverse bracket expressions.
- 12bfeb8 - Refactor the UCD file parsers.
- 2359623 - Fix key of
generalCategoryCodePoints
map. Use the abbreviationcn
of the general category valueunassigned
as a key ofgeneralCategoryCodePoints
map. - bedf0c1, 6ebbc8f, e9af227, 301d02d - Support
White_Space
,Lowercase
,Uppercase
,Alphabetic
, andScript
properties. (Meet RL1.2 of UTS #18 partially) - 10d0c5d - Make character properties available in an inverse expression (Make [^\p{...}] available).
- 5ebc2f4 - Move all UCD-related processes to ucd package.
- cb9d92f - Make contributory properties unavailable except internal use.
- f0870a4 - Remove default value's code points of
General_Category
. - 847bcc7 - Move UTF8-related processes to utf8 package.
- 19b68a5, e3195d8, d595194, 4321811 - Use
CPTree
andbyteTree
instead of AST.CPTree
represents characters in code points, whilebyteTree
represents characters in UTF-8 byte sequences. In the past, when we excluded a part of a character range, like[^...]
, we needed to subtract byte sequences from each other. This process is complicated. Therefore, we simplified the UTF-8 related processing by calculating the character range in code points and then converting it to UTF-8 byte sequences. - 46d49df - Make character properties unavailable in bracket expressions.
- 3ec662c - Add tests of compiler/parser package.
- a630029 - Remove
--lex-spec
option frommaleeni compile
command.
- fe865a8 - Generate constant values representing mode IDs, mode names, kind IDs, and kind names.
- 7be1d27 - Add
name
field to the lexical specification. maleeni uses thename
field to generate a source file name of the lexer. For instance, when the name is my_lex, the source file of the lexer is named my_lex_lexer. - cf4f533 - Keep the order of AST nodes constant. This change is intended to output the same transition table for the same inputs.
- 9f3a334 - Remove
--debug
option from compile command. - a8ed73f - Disallow upper cases in an identifier.
- 12658e0 - Format the source code of a lexer maleeni-go generates.
- 60a5089 - Remove the
ModeName
andKindName
fields from thedriver.Token
struct.
- 6332aaf - Remove
--debug
option frommaleeni lex
command. - f691b5c, 96a555a - Add
maleeni-go
command to generate a lexer that recognizes a specific lexical specification.
- 7e169f8 - Support passive mode transition.
- a30fe0c - Add a function
spec.EscapePattern
to escape the special characters that appear in patterns.
- maleeni v0.1.0, this is the first release, supports the following features.
- Definitions of lexical specifications by regular expressions
- Alternation:
|
- Dot expression:
.
- Bracket expressions:
[...]
,[^...]
- Code point expressions:
\u{...}
- Character property expressions:
\p{...}
- Repetition:
*
,+
,?
- Grouping:
(...)
- Fragment:
\f{...}
- Alternation:
- Mode transition
- Definitions of lexical specifications by regular expressions