Code

New lexer 2 — Electric Boogaloo#557

Code

Merged

ISSOtm merged 59 commits intogbdev:mastergbdev/rgbds:masterfrom ISSOtm:new-lexer-electric-boogalooISSOtm/rgbds:new-lexer-electric-boogalooCopy head branch name to clipboardOct 4, 2020

+3,275-2,837

Commits

Commits on Oct 4, 2020

• Implement infrastructure around new lexer

  Show description for 6dc4ce6
  ISSOtm

  committed
  6dc4ce6

  Copy full SHA for 6dc4ce6

• Implement more functionality

  Show description for 71f8871
  ISSOtm

  committed
  71f8871

  Copy full SHA for 71f8871

• Implement almost all functionality

  Show description for 4c9a929
  ISSOtm

  committed
  4c9a929

  Copy full SHA for 4c9a929

• Fix PC's name not being passed to parser

  ISSOtm

  committed
  e56c6cc

  Copy full SHA for e56c6cc

• Fix mmap read offset not being initialized

  ISSOtm

  committed
  2ec1001

  Copy full SHA for 2ec1001

• Add EQUS expansion

  ISSOtm

  committed
  5ad7a93

  Copy full SHA for 5ad7a93

• Add string expansion reporting

  Show description for 61b2fd9
  ISSOtm

  committed
  61b2fd9

  Copy full SHA for 61b2fd9

• Fix nested expansions being incorrectly handled

  Show description for fed252b
  ISSOtm

  committed
  fed252b

  Copy full SHA for fed252b

• Fix incorrect freeing of expansions

  Show description for 149db9a
  ISSOtm

  committed
  149db9a

  Copy full SHA for 149db9a

• Fix string expansion reporting

  Show description for 38bda7e
  ISSOtm

  committed
  38bda7e

  Copy full SHA for 38bda7e

• Fix diagnostic formatting

  Show description for 7c895f8
  ISSOtm

  committed
  7c895f8

  Copy full SHA for 7c895f8

• Add test for built-in file symbol

  Show description for e11f250
  ISSOtm

  committed
  e11f250

  Copy full SHA for e11f250

• Implement macro args

  Show description for 6e805cd
  ISSOtm

  committed
  6e805cd

  Copy full SHA for 6e805cd

• Re-implement block copy to avoid expanding macro args

  Show description for 81a77a9
  ISSOtm

  committed
  81a77a9

  Copy full SHA for 81a77a9

• Fix crash when no macro args are being used

  ISSOtm

  committed
  adcaf4c

  Copy full SHA for adcaf4c

• Fix expansion reporting being incorrect

  ISSOtm

  committed
  df75fd2

  Copy full SHA for df75fd2

• Fix many lexer bugs

  Show description for cd747d8
  ISSOtm

  committed
  cd747d8

  Copy full SHA for cd747d8

• Add debug tracing code to lexer

  Show description for d9ecaab
  ISSOtm

  committed
  d9ecaab

  Copy full SHA for d9ecaab

• Fix RAW lexer length underflow

  Show description for b27b821
  ISSOtm

  committed
  b27b821

  Copy full SHA for b27b821

• Fix lexer capture sometimes not being reset

  ISSOtm

  committed
  cf99216

  Copy full SHA for cf99216

• Reinstate macro arg scan distance

  Show description for 9081fea
  ISSOtm

  committed
  9081fea

  Copy full SHA for 9081fea

• Enable catching invalid macro arg 0

  ISSOtm

  committed
  08867b3

  Copy full SHA for 08867b3

• Fix numeric constant overflow checks

  ISSOtm

  committed
  3f5f9bc

  Copy full SHA for 3f5f9bc

• Support line continuations in main scope

  ISSOtm

  committed
  e4f2fad

  Copy full SHA for e4f2fad

• Fix line-continuation-macro test

  ISSOtm

  committed
  62ecdce

  Copy full SHA for 62ecdce

• Implement __FILE__ symbol

  Show description for fd02ffb
  ISSOtm

  committed
  fd02ffb

  Copy full SHA for fd02ffb

• Apply error reporting changes to tests

  ISSOtm

  committed
  baeb180

  Copy full SHA for baeb180

• Fix file name reporting

  Show description for ae77893
  ISSOtm

  committed
  ae77893

  Copy full SHA for ae77893

• Support missing register tokens

  Show description for 8d18b39
  ISSOtm

  committed
  8d18b39

  Copy full SHA for 8d18b39

• Fix files being unmapped when still referenced by macros

  ISSOtm

  committed
  35396e6

  Copy full SHA for 35396e6

• Fix incorrect error reporting of INCLUDEd files

  ISSOtm

  committed
  4d1333e

  Copy full SHA for 4d1333e

• Fix macro args not being restored when exiting macros

  ISSOtm

  committed
  d641972

  Copy full SHA for d641972

• Fix _NARG crashing outside of macros

  Show description for b83b982
  ISSOtm

  committed
  b83b982

  Copy full SHA for b83b982

• Add line+col trace info to lexer

  ISSOtm

  committed
  aa76603

  Copy full SHA for aa76603

• Fix else working incorrectly from macros

  Show description for f9b48c0
  ISSOtm

  committed
  f9b48c0

  Copy full SHA for f9b48c0

• Fix "REPT 0" not being a no-op

  ISSOtm

  committed
  b7b03ee

  Copy full SHA for b7b03ee

• Implement opt b and opt g

  ISSOtm

  committed
  ece6853

  Copy full SHA for ece6853

• Prevent expanding macro args in comments

  Show description for f7b7a97
  ISSOtm

  committed
  f7b7a97

  Copy full SHA for f7b7a97

• Fix readFractionalPart never shifting characters

  ISSOtm

  committed
  615f107

  Copy full SHA for 615f107

• Fix INCLUDE ignoring -MG

  ISSOtm

  committed
  e33c2ad

  Copy full SHA for e33c2ad

• Handle comments in line continuations

  ISSOtm

  committed
  9e3d7a5

  Copy full SHA for 9e3d7a5

• Use common function to discard comments in macro args

  ISSOtm

  committed
  ac011fe

  Copy full SHA for ac011fe

• Fix C2x use of static_assert

  ISSOtm

  committed
  71a0a42

  Copy full SHA for 71a0a42

• Fix possible capture buffer size overflow

  Show description for 542b5d1
  ISSOtm

  committed
  542b5d1

  Copy full SHA for 542b5d1

• Add newlines to all test output

  Show description for b65ea64
  ISSOtm

  committed
  b65ea64

  Copy full SHA for b65ea64

• Fix fixed-point constants not working correctly

  Show description for c952dd8
  ISSOtm

  committed
  c952dd8

  Copy full SHA for c952dd8

• Move isWhitespace to a place where it makes more sense

  ISSOtm

  committed
  dbef51b

  Copy full SHA for dbef51b

• Remove unnecessarily nested symbol data union

  ISSOtm

  committed
  7381d7b

  Copy full SHA for 7381d7b

• Harmonize printing distance

  ISSOtm

  committed
  b224cab

  Copy full SHA for b224cab

• Fix range-dependent dead code in recursion depth check

  ISSOtm

  committed
  96cb5e1

  Copy full SHA for 96cb5e1

• Shim around mmap on Windows

  ISSOtm

  committed
  82469ac

  Copy full SHA for 82469ac

• Fix possible uninitialized read on Windows

  ISSOtm

  committed
  1385235

  Copy full SHA for 1385235

• Move some MSVC-specific defines to platform.h

  ISSOtm

  committed
  8e7afb0

  Copy full SHA for 8e7afb0

• Mark not unmapping macro-containing files as okay

  Show description for 930080f
  ISSOtm

  committed
  930080f

  Copy full SHA for 930080f

• Implement compact file stacks in object files

  Show description for 5a65188
  ISSOtm

  committed
  5a65188

  Copy full SHA for 5a65188

• Change assertion condition in __FILE__ buf dumping

  Show description for ee9e45b
  ISSOtm

  committed
  ee9e45b

  Copy full SHA for ee9e45b

• Handle \\r better

  Show description for 423a7c4
  ISSOtm

  committed
  423a7c4

  Copy full SHA for 423a7c4

• Fix incomplete duplication of REPT nodes

  Show description for c246942
  ISSOtm

  committed
  c246942

  Copy full SHA for c246942

• Fix critical oversight in lexer buffer refilling

  Show description for 2eca43c
  ISSOtm

  committed
  2eca43c

  Copy full SHA for 2eca43c