Retokenizing certain lines in a text editor in response to edits


Is there a way to tell a TextEditor that certain lines need to be retokenized?

Here’s my particular case. I wrote a package (language-csound—I’d link to it but I’m a new user so I’m limited to two links) to support Csound. Csound requires variable names to have one- or two-letter prefixes that identify their types. Here’s a Csound snippet:

instr Sawtooth
  aSignal vco2 0.5 * 0dbfs, 440
  outc aSignal

The prefix a of aSignal identifies aSignal as an audio variable, and in the Atom package the a is scoped as storage.type.csound to give it a different color. Here is the same example with syntax highlighting showing what I mean.

Things start to get tricky when you highlight what Csound calls opcodes; these are similar to functions. Here’s another snippet:

opcode anOscillator, a, kk
  kAmplitude, kFrequency xin
  aSignal vco2 kAmplitude, kFrequency
  xout aSignal
instr Oscillator
  outc anOscillator(0dbfs, 110)

And here is a version with better syntax highlighting. The a at the start of anOscillator shouldn’t be scoped as storage.type.csound. Instead, all of anOscillator should be scoped as something else ( in my case) everywhere in the buffer.

I’m able to handle this tokenization with subclasses of Grammar and Pattern, but I have to deal with the name of the opcode changing from anOscillator to something else.

Lines containing anOscillator need to be retokenized when the name of the opcode is changed, but I’m not sure how to tell a TextEditor to retokenize only particular lines. Instead, I’m telling the TextEditor to retokenize the entire buffer. This is not only inefficient, but also makes comments in the Csound file seem to flicker when they briefly change to plain text. Is it possible to tell a TextEditor to retokenize only particular lines?


For posterity’s sake, it’s possible to do this with a (private, I think) method of TokenizedBuffer. In CoffeeScript, if editor is a TextEditor, this can be done in one line:

editor.displayBuffer.tokenizedBuffer.invalidateRow row for row in [0...editor.getLineCount()] when !editor.isBufferRowCommented row