Is it possible to programmatically generate syntax highliting, without using a grammar?


#1

I am building a programming language. Because of this, I am already writing a lexer/parser but more importantly a type checker. Writing again a grammar for syntax highlighting would be time consuming, but more importantly limited compared to the information a quick call to my compiler would be able to provide.

Is it possible to use code (in this case a call to an external program) to do syntax highlighting instead of the grammar DSL?


#2

Hello.

I have no clue. Perhaps @Wliu could say more.
Why not use the built-in system though?

See if you can learn anything from:

Cheers.


#3

Both to not duplicate work and to have more information to use. Using my compiler I would be able to highlight initialized and unitialized variables, stack vs heap allocated objects and so on. I assume that by writing different languages themes I would also be able to turn on different semantic highlighting modes depending on what I’m doing, ie. debugging (init/uninit, heap/stack) or general coding (lexer and parser pass, without running the typechecker).

Unfortunately I don’t know {java,coffee}script, so I can’t grok that link easily. I would want to try to write a little wrapper that calls my compiler and gets all the different tokens, if that’s supported.


#4

I cannot see that you can do what you ask without digging in to some type of Javascript based language. What languages can you code in?

There are other text editors you can try instead for your needs. I would consider Notepad++. But again a choice would depend on what you know and what you are willing to teach yourself. Perhaps also see: https://alternativeto.net/software/notepad-plus-plus/.

Notepad++ is very easy to configure a new grammar for colouring. Just not so friendly when the standards are not good enough.

:innocent: Or explain your aim here and see if you can motivate someone to partner with you.
{Slim to none chance but worth the ‘risk’}

Ping me some time and tell me what you decided to do.
Good luck!


#5

There is a possibility that tree-sitter is what you’re looking for. However, I do not know if/when tree-sitter will be integrated into Atom.


#6

I do intend to program a thin wrapper to integrate my compiler in atom with javascript, and I have used browser javascript for some very small things (<1000sloc music experiments), but I don’t think I’m proficient enough to dive in atom’s code. As long as someone points me to some relevant api docs and I can ask for help when I inevitably get stuck, I think I’m going to be fine.

I can code in python and the compiler I’m writing is in C. Again, I can get a javascript app going, but in a style that would have been considered horrible 5 years ago :stuck_out_tongue:

Sorry that wasn’t very clear!

That looks pretty cool, but I think it would require lots of changes in the compiler itself. I can’t add this kind of complexity right now, unfortunately.


#7

Notepad++ has the worst syntax highlighting format I’ve ever encountered… and I’ve seen many! :face_vomiting:

It’s unreliable and unflexible.


#8

@idleberg:

(sarcasm := true)
:cup_with_straw: …here is some water for that ill feeling.

(sarcasm := false)
You will have no argument from me on the point you make.
I have used it before on some limited and strict format syntax language - it was quick to set and worked well enough to be usable.


You are the perfect person to ask this question for.

Please - any of those that are worth mentioning?
Let us ignore getting the perfect fit and think about:

Thank you for sharing your insights.


#9

I’m not sure I understand. I was replying to the question of what languages I coded in. I meant that I can hack some javascript together since I know a couple other languages.


#10

Sorry, the previous post was meant exclusively for our visiting expert (@idleberg ).
…and I mean ‘expert’ sincerely.

Making a grammar work in Atom does not rely on real programming. It is more on configuration. Though saying so, I have to also say that the structuring and constructing the regular expressions are tricky.

Have a look at one of my own queries and evaluate if there is hope for you there.

Regards.

EDIT 1: now with link :blush:


#11

That’s a great start, thanks! Lot’s of helpful stuff in there. I’m starting to work with that right now. If anyone has any idea on how to hook my compiler directly I’m still interested, though!


#12

For most of the program languages, people have made packages (add-ons) that uses an emulated terminal. Think for example ho you would compile C code when using the OS terminal.

Most notably is script LINK. script has a look-up table that contains the command you would use for a program language. For example for the C language:
https://github.com/rgbkrk/atom-script/blob/master/lib/grammars.coffee#L82

It would be possible for you to fork script and include your compiler in there.

There is a bit of a catch though ->
Python console windows?

I hope you find something helpful from the above notes.
Go well.
- Dan Padric


#13

Let’s keep the talk about Atom


#14

@idleberg: Fair enough.
Please could I convince you to share your insights on Atom related materials?


Perhaps in particular on:

I hope on your positive reply.


#15

That’s very helpful and I will have to install one of those, but I mean hook up my compiler’s abstract syntax tree to atom’s syntax highlighting, so I can take advantage of the typechecker’s information.