feature: Binary representation support for numbers

[felpafel]

Did you check the tree-sitter docs?

• I have read all the tree-sitter docs if it relates to using the parser

Is your feature request related to a problem? Please describe.

LuaJIT 2.x support number declarations using binary representation, but Lua 5.x doesn't. So, local a = 0b101 is valid only in LuaJIT.

In the current state this parser outputs the following tree for local a = 0b101:

Using :InspectTree in Neovim.

(chunk ; [0, 0] - [1, 0]
  local_declaration: (variable_declaration ; [0, 0] - [0, 11]
    (assignment_statement ; [0, 6] - [0, 11]
      (variable_list ; [0, 6] - [0, 7]
        name: (identifier)) ; [0, 6] - [0, 7]
      (expression_list ; [0, 10] - [0, 11]
        value: (number)))) ; [0, 10] - [0, 11]
  (ERROR ; [0, 11] - [0, 15]
    (identifier))) ; [0, 11] - [0, 15]

Describe the solution you'd like

Handle binary representation in the same way hexadecimal representation is implemented.

A possible implementation: (This is only a suggestion)

// grammar.js - Number:
const bin_digits = /[01]+/;
const bin_literal = seq(
  choice('0b', '0B'),
  choice(
    seq(bin_digits, /U?LL/i),
    seq(
      seq(optional(bin_digits), bin_digits),
      optional(choice('i', 'I'))
    )
  )
);

return token(choice(decimal_literal, hex_literal, bin_literal));

Valid representations:

0b101     -- 5
0B101     -- 5
-0b101    -- -5
0b101LL   -- 5LL
0b101ULL  -- 5ULL
0b101i    -- 0+5i
0b101I    -- 0+5i

Describe alternatives you've considered

No response

Additional context

I think this feature fall under the same category of complex number representation. local a = 5i is valid only in LuaJIT and is implemented by this parser.

Lua 5.4:

Lua 5.4.2  Copyright (C) 1994-2020 Lua.org, PUC-Rio
> a = 0b101
stdin:1: malformed number near '0b101'
> a = 5i
stdin:1: malformed number near '5i'

LuaJIT 2.1.1720049189:

LuaJIT 2.1.1720049189 -- Copyright (C) 2005-2023 Mike Pall. https://luajit.org/
JIT: ON SSE3 SSE4.1 BMI2 fold cse dce fwd dse narrow loop abc sink fuse
> a = 0b101
> print(a)
5
> a = 5i
> print(a)
0+5i

Tree for local a = 5i:

(chunk ; [0, 0] - [1, 0]
  local_declaration: (variable_declaration ; [0, 0] - [0, 12]
    (assignment_statement ; [0, 6] - [0, 12]
      (variable_list ; [0, 6] - [0, 7]
        name: (identifier)) ; [0, 6] - [0, 7]
      (expression_list ; [0, 10] - [0, 12]
        value: (number))))) ; [0, 10] - [0, 12]

All LuaJIT code shown is also valid inside Neovim.

NVIM v0.10.2
Build type: Release
LuaJIT 2.1.1713484068

edit: Added implementation suggestion.