Language features driven by something other than Textmate-style grammar


As far as I know, at present, language grammars are based on the old Textmate format and then handled by First Mate.

Can any provide me with tips on getting started with how to integrate a different system?

In general, the process would involve passing the contents of the current file to its relevant language parser and receiving a tokenized version in return.

The part I’d like help with is how to then use those tokens to provide a feature such as syntax highlighting, for example.


I’d start by looking at the GrammarRegistry and Grammar classes. Beyond that, I suspect it’ll be a lot of digging and refactoring.


Thanks @leedohm

I found this issue on the First Mate repo which describes the need for asynchronous tokenization and links to some work by @basarat on creating a dynamic grammar.