How to debug tree-sitter grammars and syntax highlighting

I decided to dip my toes into creating a tree-sitter parser to be used in conjunction with a new language package for Atom. The parser is very simple, and I got it to work perfectly; all the tests succeed etc., and tree-sitter parse <various test files> work great too.

I generated a package and added a small .cson file. In debug mode, I can see the syntax-- classes being added with the value of the scopeName field, but nothing within the scopes definitions seems to be taking. I opened up the language package files in debug mode, and even opened up the debug tools to try to place breakpoints and step through the parser being loaded but couldn’t find where that was happening.

How do I debug the parser?

Here’s a couple links to my code:
Parser: https://github.com/jakofranko/tree-sitter-tablatal
Language Package: https://github.com/jakofranko/language-tablatal

  • Parser debugging is done on the Tree-sitter repo side (with the corpus tests). There are options to increase verbosity, such as printing each step it takes as it parses (useful to identify unintended zero width loops).

  • Your package works as expected for me.

If you are wondering why comments aren’t working, all you’ve done so far is declare ~ to be the comment character Atom uses when toggling comments. To get syntax highlighting, you need to add them to the parser on the tree-sitter-tablatal side, and give them the appropriate scope on the language-tablatal side (like any other rule).

Other than that, it looks fine.

1 Like

Thanks so much for your help! Hmmm, I must be missing something then. How did you install that package?

I deleted the symlink from ~/.atom/packages, and then did a fresh install of the package (git clone, npm i, apm link .) and am seeing the same thing I described above. What tool are you using to display the syntax tree at cursor?

That was a nice tip to see the scope at cursor, I was able to find that in the command palette after seeing your screenshot…

Here’s my screenshot:
image

To follow up, I think it might have been an issue with an older version of Atom. Now I seem to be dealing with this issue: https://github.com/atom/atom/issues/17029

Yeah, I’ll try and make a PR for that soon.

You shouldn’t remove the type: 'tree-sitter' property though; it probably is important.

Great! I applied your patch to my init.coffee and that did the trick. Thanks @Aerijo!