Help setting up tree-sitter grammar development


#1

I’m excited to start translating my existing regex grammar to a tree-sitter version, but I’ve hit a road block with activating the syntax package in Atom. Please note I have no experience building/setting up dependencies. The closest I’ve come to that is running update package dependencies for developing normal packages.

Steps I followed:

  • I downloaded the mac build linked by maxbrunsfeld in this pull

  • Opening it, went to Settings -> Core and enabled tree sitter parsers. Reloading, I could see the core ones were being applied (specifically JavaScript).

  • Dowloaded tree-sitter-javascript and ran apm link on it (this was to see how to set up a package, so I used this one to guarantee everything else was properly configured).

  • Got following error message:

The module '/Users/benjamingray/github/tree-sitter-javascript/build/Release/tree_sitter_javascript_binding.node'
was compiled against a different Node.js version using
NODE_MODULE_VERSION 59. This version of Node.js requires
NODE_MODULE_VERSION 54. Please try re-compiling or re-installing
the module (for instance, using `npm rebuild` or`npm install`).
  • Ran npm rebuild command from the packages root folder & restart Atom.

  • Got following console warning:

/private/var/folders/1x/bw45w2n90pd8b3v6tfydvqxr0000gn/T/AppTranslocation/DF198AAD-796A-41C9-B06B-2…:766 Failed to require the main module of 'tree-sitter-javascript' because it requires one or more incompatible native modules (tree-sitter, tree-sitter-cli).
Run `apm rebuild` in the package directory and restart Atom to resolve.
  • Followed the Incompatible Packages prompt to rebuild all problematic packages & reloaded Atom.

  • Got original error again.

Again, I have no idea what I’m doing. Any workflow I can follow to start developing my own tree sitter grammar would be appreciated.


#2

This post might help shed some light on the error message. Based on the author’s results, my conjecture is that the original error is because the version of Node contained in the version of Electron contained in the version of Atom you’re running doesn’t match the one used to develop that package. However, you downloaded the build shared by @maxbrunsfeld in the linked issue, which seems to me like it should solve that problem.


#3

OK, I’ve made (some) progress:

  • When I got the first error message, I ran the following command suggested here with --target=1.7.10 (the result in atom -v)
HOME=~/.electron-gyp node-gyp rebuild --target=1.7.10 --arch=x64 --dist-url=https://atom.io/download/electron
  • It didn’t complain of an error, and console.log statements in the index.js file show it was successfully required.

  • I made a minor change: single line comment pattern now starts with /// (from //)

  • I ran the command tree-sitter generate

  • I ran the command at the start.

  • After reloading, the console showed the log statements I had set up. Requiring was successful.

  • Attempting to test the change, comments would still be started by //, as opposed to the /// I changed it to.

  • Repeated the entire process with a new package, using the arithmetic grammar on the tree-sitter-cli repo. I copied the index.js file from the JS package, changing the occurrences of javascript to arithmetic, and added nan as a dependency.

  • Again, log statements show the require was successful, but the grammar does not appear in the language selection window.

  • Running console.log on the required file, it is a Language object with the sole property name (for both JavaScript and arithmetic), and a __proto__.


#4

I got a reply on the PR itself, and was able to start developing a tree-sitter grammar. If anyone needs more guidance, I’ll do my best to remember how I did it (no promises).


#5

For anyone reading this, here’s a guide I’ve started that walks through the setup of Tree-sitter grammars, and (eventually) how to load them in an Atom package