Atom grammars should be composable


Currently there seems to be an issue with “subgrammars” f.ex. which has to redefine every file type with added jinja syntax (It doesn’t support a lot of files, for example .groovy.j2). Same thing with ERB files etc.

Now it would be really cool to be able to say in the Jinja grammar Compose with first extension and then foo.groovy.j2 would get a composed Syntax of Groovy - Jinja2 or bar.html.erb would be HTML - ERB.

The way it would work is that the file get’s both grammars applied to it foo.groovy.j2 first applies Groovy grammar and the Jinja2 grammar and baz.j2.groovy would first apply Jinj2 and then Groovy grammar. That way composing grammars have a way define how overrides work if necessary.
The grammar “API” could even have a cannot override flag for parts of grammars, but that sounds only slightly useful.


Some thoughts:

  • As I understand it, a grammar rule that appears later in the document has less priority. For a templating language, you would want to match for its tags first, before the base language. bar.html.erb would have to access ERB first, then concatenate the HTML rules onto that.
  • Because rules get matched in order, there’s no overriding by later rules and there’s no need for an !important analog.
  • Will this immediately make sense to someone who hasn’t been told how it works? Does it match how frameworks are naming these files? It might make it slightly easier to develop a template language package, but if it makes things more confusing for the user, that’s not a gain. From the user’s perspective, Jinja just works. From the dev’s perspective, it has seven nearly identical and very small files that basically serve as a routing table. Using an existing one as a template, it would take about thirty seconds to make a new one and expand the package to cover a different file type.
  • Would grammars be composable through the grammar selector?


The reason I started thinking about this was that language packages don’t necessarily get updated with the combo you need. My PR to a language package has been hanging for 3 months.
And to be fair, the actual grammar files are “composable” in a sense that you can just include a scope and everything just works.

  1. You are correct, so it’s just the opposite of how I thought about it :slight_smile:
  2. Yup
  3. Well, it should be designed so that it would make sense, the framework naming thing is probably the reason I’ve even stumbled upon this :smiley: So for me that’s the primary motivation to get to work. This shouldn’t even appear that much different for the user, maybe the syntax selection screen would need to be redesigned a bit
  4. With some redesigning, maybe? :stuck_out_tongue:

Thinking about the syntax selector screen, it would need some kind of simple understandable way of either specifying which grammar to use for which extension or just show single grammars and the one composed grammar for the current extensions


There isn’t much that can be done about people who don’t maintain their packages except to fork the packages.

With the existing system, the onus for composition and understanding how the grammar system works is placed on the package developer. My main reservation about your idea is that it shifts that burden to the user and may introduce situations that are outside of the developer’s control.