How to split grammars up into smaller files in a subfolder of grammars?


#1

Hi,

I was wondering how would I split a grammar file (i.e., in the grammars/ folder of my forked language-shellscript package) into several smaller, more manageable cson files. Specifically I would like to split the repository: field in my shell-unix-bash.cson file into these smaller files. My current repository field is of the general format (if you want the exact, full field see here):

repository:
    key1:
      patterns: ...
    key2:
      patterns: ...
    key3:
      patterns: ...

and I would like to split this field into CSONs which specify each individual key (i.e., key1, key2, key3, etc.).

I have noticed that @burodepeper’s language-markdown package has managed something like this, and while I tried to copy their code my knowledge of CoffeeScript isn’t enough (my attempt at following in their footsteps left my package unable to highlight shell scripts at all). See, from what I can tell, they use the file lib/main.coffee to merge all these CSON files. This is the specific segment of code that does this, I think:

  compileGrammar: ->
    if atom.inDevMode()
      input = '../grammars/repositories/markdown.cson'
      output = '../grammars/language-markdown.json'
      repositoryDirectories = ['blocks', 'flavors', 'inlines']
      filepath = path.join(__dirname, input)
      grammar = CSON.readFileSync(filepath)

      for directoryName in repositoryDirectories
        directory = new Directory(path.join(__dirname, '../grammars/repositories/'+directoryName))
        entries = directory.getEntriesSync()
        for entry in entries
          {key, patterns} = CSON.readFileSync(entry.path)
          if key and patterns
            grammar.repository[key] =
              patterns: patterns

      # Compile and add fenced-code-blocks to repository
      grammar.repository['fenced-code-blocks'] =
        patterns: @_compileFencedCodeGrammar()

      # Write {grammar} to {filepath},
      # and reload window when complete
      filepath = path.join(__dirname, output)
      CSON.writeFileSync filepath, grammar, do ->
        atom.commands.dispatch 'body', 'window:reload'

I tried creating my own lib/main.coffee file, copying this segment of code and some parts at the start of this file, and including this file as my package.json’s main file, but unfortunately this simplistic approach failed me. I think part of the reason is because the language-markdown package has folders within the grammars/repositories folder, while mine merely has a grammars/repositories folder with all the split CSONs found in this folder.

So that you can see what I was doing I have committed the files to this repository branch.

Thanks for your time,
Brenton


#2

Hi Brenton,

From a quick glance at your code, here are a few pointers that might help.

First of all, you’ll need a command to trigger Atom to compile the grammar. This doesn’t happen automatically. Right now, your main.coffee doesn’t do anything, I think. See: https://github.com/burodepeper/language-markdown/blob/master/lib/main.coffee#L62-63

Secondly, your compileGrammar method looks fine, except that your output on line 9 has to end in .json. The CSON parses will automatically write the output as JSON this way. I can’t recall what the exact reasoning behind this was (I believe something with escaped characters), but writing the grammar as JSON (CSON becomes JSON anyway, so I think this step actually speeds things up slightly) fixes this. It’s possibly that you don’t need to write directly to JSON, but it’s a safe step to do anyway.

Lastly, your lib/main.js might be confusing Atom. No need to keep it if you’re also using a lib/main.coffee.

Let me know if you need any more help. If the above doesn’t work, I might have to actually delve into it. This is all just from a quick glance.


#3

So I can’t use a CSON for my main grammar file (i.e., shell-unix-bash.cson)? lib/main.js was my attempt at debugging what was going on. I’ll remove it, but I was really desperate for how to debug this problem.


#4

So I can’t use a CSON for my main grammar file (i.e., shell-unix-bash.cson)?

Possibly you can, but there’s no difference in outputting to JSON. CSON is converted to JSON, in the same way that CoffeeScript is converted to Javascript. To Atom, they’re the same thing.

lib/main.js was my attempt at debugging what was going on. I’ll remove it, but I was really desperate for how to debug this problem.

You can do the same debugging from lib/main.coffee. I think the best thing you can do is read up on how Atom works (see the docs) to understand how and why language-markdown works in the way it does. For instance, this little bit reloads Atom immediately after you’ve compiled the grammar, which makes it impossible to read anything you’ve written to console.log() making it rather impossible to debug anything at all.


#5

@burodepeper I have tried to do this. But it’s failed quite miserably. See my testing branch for my attempts to date (https://github.com/fusion809/language-shellscript/tree/testing). I really haven’t the foggiest what I need to get it to work. My knowledge of CoffeeScript is pretty minimal.


#6

I’m quite swamped at the moment, but I’ll see if I can free some time this week to have a look at it.


#7

I’ve submitted a PR. See: https://github.com/fusion809/language-shellscript/pull/2