[Feature] Replace Antlr4 TypeScript Parser with Rust Parser to optimise parsing using wasm-bindgen and antlr4rust

[Luna-Klatzer]

Is there an existing proposal for this?

• I have searched the existing issues

This feature does not exist in the latest version

• I am using the latest version

Proposal

Currently, the performance of the Kipper compiler can be compared to the TSC compiler, though with relatively few features, as Kipper is still in built-up and early development. Though considering the early development stage, there could be a possible advantage of adopting a Rust + WASM-based Antlr4 Lexer and Parser, instead of TypeScript ones, to significantly improve speed

This could be useful, as the current TypeScript Parser takes up almost 90-95% of the compilation time, and optimisations are rather hard to do, as the lexer and parser are generated by antlr4ts. To that, Kipper will slowly get more and more complex, and as more features are added the Antlr4 Parser could be a serious bottleneck to the performance, as it slows down even simple compilations.

A workaround for that bottleneck is already in use on the playground of the Kipper website, where the website will when it loads warm up the compiler and make the Parser and Lexer cache commonly used items. This significantly improves performance but is a work-around that only works for environments, where the Kipper Compiler is loaded before the compilation happens, as the user usually takes a few seconds before they will try to run their program on the site.

As such, migrating to WASM or similar would help get rid of that issue. To that since wasm-bindgen supports importing Rust functions and data into JavaScript and TypeScript environments, the Kipper Compiler could use Rust functions from the Antlr4 Parser and Lexer and generate the parse tree without using any JavaScript or TypeScript. The fetched data could afterwards simply be put into JS/TS classes, which shouldn't cost too much performance and allow the Compiler to generate its abstract syntax tree like it usually would. That also wouldn't change the internal Compiler structure too much and allow the use of the already existing code base.

Though, since all of that is theoretical and likely hasn't been done before, this would require a lot of testing and experiments to see if that would even work and be stable. To that, it would require a Rust-based Antlr4 Parser and Lexer using antlr4rust, which is a package that is still experimental and in development in the Antlr4 project.

Exact behaviour / changes you want

• Create a new Rust-based Antlr4 Parser and Lexer generation workflow using antlr4rust.
• Set up wasm-bindgen compilations and poly-fills for the Rust Parser and Lexer. This is going to be the hardest step and importing the lexer and parser may not work.
• Integrate the new WASM-based Parser and Lexer into the existing @kipper/core package and use its functions, classes and data to create a parse tree in a JS object format.
• Integrate the generated parse tree into the existing AST-generation workflow and ensure that the data, context instances etc. work with the existing structure.