Weird grammar highlighting behavior with `include`


#1

Hello.

I am trying to make a custom grammar for a JavaScript extension that I’m making, but it is very frustrating, as everything behaves weirdly.

scopeName: 'source.simp'
name: 'Simp'
fileTypes: [
    'simp'
]
patterns: [
    # {
    #     include: 'source.js'
    # }
    {
        match: '\\s*func(?= +\\w*\\()'
        name: 'storage.type.function.simp'
    }
    {
        match: '(\\w+): *(\\d+)\\.\\.(\\d+)'
        name: 'constant.numeric.loop.forwards.simp'
    }
    {
        match: '(\\w+): *(\\d+)\\.\\.<(\\d+)'
        name: 'constant.numeric.loop.backwards.simp'
    }
    {
        match: '\\s*ret(?=( +\\w+))|ret\\.'
        name: 'keyword.control.return.simp'
    }
    {
        match: '->'
        name: 'keyword.operator.arrow.simp'
    }
]

This is the script that I’m using to test my grammar:

var test = 0

let hellonum = 0

func hello() {
    ret hellonum
}

for (i: 0..100) {

}

for (i: 0..<100) {

}

func call(funct) {
    funct();
}

() -> {}

hello()

I know, there are syntax errors, but this is just a highlighting test. Anyway, if you uncomment the include in CSON, syntax highlighting for keywords like ret, x: min..max, is broken and it works just like in plain old JS. Can anybody tell me what am I doing wrong?


#2

Because you’re including source.js at the beginning, Atom is matching everything in the file against source.js and nothing is left except for things that don’t match any valid JS syntax. Include the big grammar at the end so that your patterns have first dibs on potential matches.


#3

I already tried, with no effect :confused:


#4

Does this do what you want?

scopeName: 'source.simp'
name: 'Simp'
fileTypes: [
  'simp'
]

injections:
  "source.simp": {
    patterns: [
      {
        match: '\\s*func(?= +\\w*\\()'
        name: 'storage.type.function.simp'
      }
      {
        match: '(\\w+): *(\\d+)\\.\\.(\\d+)'
        name: 'constant.numeric.loop.forwards.simp'
      }
      {
        match: '(\\w+): *(\\d+)\\.\\.<(\\d+)'
        name: 'constant.numeric.loop.backwards.simp'
      }
      {
        match: '\\s*ret(?=( +\\w+))|ret\\.'
        name: 'keyword.control.return.simp'
      }
      {
        match: '->'
        name: 'keyword.operator.arrow.simp'
      }
    ]
}

patterns: [
  {
    include: 'source.js'
  }
]


#5

Thank you, it worked!


#6

@Aerijo So, what you provided worked, but as soon as I started adding more patterns they didn’t work and behaved just like in the beginning. Of course the patterns work when I remove the include.

scopeName: 'source.simp'
name: 'Simp'
fileTypes: [
    'simp'
]

injections: {
    "source.simp": {
        patterns: [
            {
                match: '\\s*func(?= +\\w*\\()'
                name: 'storage.type.function.simp'
            },
            {
                match: '(\\w+): *(\\d+)\\.\\.(\\d+)'
                name: 'constant.numeric.loop.forwards.simp'
            },
            {
                match: '(\\w+): *(\\d+)\\.\\.<(\\d+)'
                name: 'constant.numeric.loop.backwards.simp'
            },
            {
                match: '\\s*ret(?=( +\\w+))|ret\\.'
                name: 'keyword.control.return.simp'
            },
            {
                match: '->'
                name: 'keyword.operator.arrow.simp'
            },
            {
                match: '\\? *(\\w+)-(\\w+)'
                name: 'support.function.random.range.simp'
            },
            {
                match: '\\? *(\\w+)%'
                name: 'support.function.random.chance.simp'
            }
        ]
    }
}

patterns: [
    {
        include: 'source.js'
    }
]


#7

OK. Updated with priority prefix. And I also changed literal spaces to \\s because I feel that’s easier to read (especially because the (?x) exists, which makes literal whitespace ignored)

scopeName: 'source.simp'
name: 'Simp'
fileTypes: [
    'simp'
]

injections: {
    "L:source.simp": {
        patterns: [
            {
                match: '\\s*func(?=\\s+\\w*\\()'
                name: 'storage.type.function.simp'
            },
            {
                match: '(\\w+):\\s*(\\d+)\\.\\.(\\d+)'
                name: 'constant.numeric.loop.forwards.simp'
            },
            {
                match: '(\\w+):\\s*(\\d+)\\.\\.<(\\d+)'
                name: 'constant.numeric.loop.backwards.simp'
            },
            {
                match: '\\s*ret(?=(\\s+\\w+))|ret\\.'
                name: 'keyword.control.return.simp'
            },
            {
                match: '\\->'
                name: 'keyword.operator.arrow.simp'
            },
            {
                match: '\\?\\s*(\\w+)\\-(\\w+)'
                name: 'support.function.random.range.simp'
            },
            {
                match: '\\?\\s*(\\w+)%'
                name: 'support.function.random.chance.simp'
            }
        ]
    }
}

patterns: [
    {
        include: 'source.js'
    }
]

Note: I didn’t know anything about this “prefix” stuff either. I stumbled across it when reading the source code behind how Atom applies grammars, and had to ask about it in a question