Proposal: Real ES6 JS with Coffeescript syntax


#1

This is a proposal. I’m looking for comments, and eventually project help. Buried this this proposal is an Atom package that is part of this effort.

Background

ES6 is starting to offer many CS features, like classes and fat arrows, but ES6 is stuck with the awful noisy C syntax. But the differences are getting so small that this ancient syntax is virtually the only difference.

Proposal

I am proposing a translator tool JSW (Javascript with Significant Whitespace) that supports an alternative syntax for Javascript with the advantages of both ES6 and CS.

I have tried twice before to start this development. See here and here. Both were not practical to implement. I think I have finally cracked the problem with a third try, See the JSW repo here.

Eventually an Atom package will allow you to open a real JS ES6 file, edit in the Atom editor as JSW, and save back again as the same JS. You usually won’t even need a JSW file format. This solves the problem in contributing to JS projects using coffeescript syntax.

The readme …

JSW

An alternate version of Javascript syntax that uses significant whitespace

JSW (Javascript with Significant Whitespace) is a syntax for Javascript, and especially ES6, that allows editing real JS with Coffeescript-like syntax.

The utility in this repository is a bi-directional translator between JS and JSW.

Unlike Coffeescript, JSW does not provide a new language with differing semantics. It is a thin skin over Javascript that only changes the syntax. This JSW utility is a syntax translator, not a compiler or transpiler.

JSW is a great way for Coffeescript users to migrate to real Javascript with ES6.

Sample JSW with translated Javascript

# JSW
-> func1 arg1, arg2                 
  var hash1 = key1: val1            
              key2: val2            
              key3:                 
  block                             
    let x = y                       
    func2 x                         
    if q and z                      
      func1 "This is text spread    
             over two lines."       
// Javascript                                    
function func1 (arg1, arg2) {
  var hash1 = {
    key1: val1,
    key2: val2,
    key3: key3
  }
  {
    let x = y;
    func2(x);
    if (q && z) {
      func1("This is text spread " +
            "over two lines.");
    }
  }
}

JSW Overall Features

  • JSW syntax is much less noisy than the JS syntax, just like Coffeescript. No more unnecessary parens and braces. No typing of function.

  • Some syntax features, like simple line continuations and blocks, are an improvement over Coffeescript.

  • The translation of JSW to/from JS is one-to-one and reversible. Changing JS to JSW and back preserves the original text. This means that JS commits to GIT will have minimal lines changed, only where JSW changed.

  • The JSW utility in this repository includes translation to/from JS and JSW. Both directions are equally supported.

  • The JS produced by translating JSW is easily predictable. It consists of simple short mappings like function to ->. Writing JSW and debugging with the translated JS takes no mental effort.

  • JSW takes advantage of new ES6 features. The syntax -> is a replacement for function while ES6 provides => with no change in JSW. The combination of the two matches coffescript.

  • Coffeescript highlighting works pretty will with JSW (see sample above)

  • An Atom editor extension, coffee-js-translator, is planned that allows opening normal JS, editing in JSW, and saving in the original JS. This allows working with others who don’t even know about JSW. One does not even need to have JSW files.

JSW features not in Coffeescript

  • Most of ES6
  • Simple continuations that can break anywhere, including strings
  • Blocks
  • No hashes require braces

Coffeescript features not in JSW

  • Some ES6 replacements not as complete (e.g. Classes)
  • All statements are expressions
  • Ranges
  • this reference in params and object declrations
  • Matching ES6 features often have different syntax (e.g. interpolation)
  • Combined comparison (0 < x < 1)

JSW to JS Text Mapping

When JS is converted to JSW, a compact set of metadata is added to the bottom of JSW as a comment. This is very similar to how Coffeescript provides a source map. If support for exact JS text matching is not required this can be disabled. In general it can be ignored and the Atom package JSW will hide that metadata when editing.

Status

You are looking at it. No coding has started. Even this readme is a work-in-progress.

License

Copyright by Mark Hahn using the MIT license.


Dreaming of dynamic CoffeeScripted JavaScript
Atom moving to JS?
#2

Especially since the release of ES6, I’ve also been thinking of sticking with plain JavaScript. This is an interesting compromise, though, for people who hate semicolons. Also, multi-line strings and so on are nice too.


#3

Aye Aye Captain!
After being accustomed to the CoffeeScript syntax with whitespace that makes the code so neat, it damages my eyes to see plain JS/ES6.

ES6 sure has awesome features that CS lacks and thus a new language must be born!

Code faster, neater and better with JSW.


#4

I’ve put a lot of mental effort into this. It turns out that any two syntaxes that share one AST can implement the awesome one-to-one reversible translation. And I’ve designed the metadata needed to ensure the translations preserve the text well enough to not trigger unwanted git diffs.

This gives a really good way to design the new syntax. I can define the new syntax as exactly the maximum amount of coffeescript features that can be overlaid on JS/ES6 without changing the AST.

Things like function mapping to -> clearly don’t modify the AST while for x in [1..3] to for (x = _i = 1; _i <= 3; x = ++_i) clearly does modify it. I’m pretty confident that the significant whitespace indentation will also use the same AST. It is just a difference in how blocks are rendered.

So I can now start the real implementation. My last two ideas never got to this point.


#5

BTW: I’ve run into an interesting question that affects how small my metadata can be compressed.

If I convert JS to an AST (e.g. with Acorn) and then generate JS code from that AST (e.g. with escodegen) I get a prettified version of the code that doesn’t match the original. The questions is: do the original and final JS only differ in whitespace?

Acorn and escodegen have options to preserve parens. So if you don’t use the paren-preservation option the changes will be more than whitespace. Is that the only counter-example? I am going to do tests sending arbitrary JS through the round-trip and checking the results. As I said, this only affects the compression of the metadata and does not impact the overall project.

Edit: The answer to this question is that you definitely cannot send javascript on a round-trip to/from an AST and preserve the text. However there is a new type of concrete syntax tree, CST, that was designed for this exact problem. It can do this and everything an AST can. This CST tool will reduce the overall effort for this project tremendously.


#6

BTW again, if you think about it there is nothing special about using Coffeescript-like syntax. Anyone can use any syntax they want as long as it preserves the AST. This makes the dream of editing in the syntax of our choice a reality.

I will make everything in the code that is particular to my choice of a coffeescript syntax in a separate plugin to make it easy to change syntaxes. The plugin will consist of forked versions of Acorn and escodegen. If the syntax changes are small then the code changed in the forks will be small and it should be relatively easy to keep the forks up-to-date with the main branches.


#7

Personally, I’d rather see CS itself move forward a bit faster. That’s a lot easier said than done, though. Creating a full language is hard, and maybe you don’t really have to?

You might also want to look at LiveScript as a way to do this as well. If you look at how they got ES6 through Babel working, it can give us a good example for how to get that to work, and it supports a whole lot of crazy stuff, including things that hook up to ES6 features.


#8

I have used nothing but CS for many years (4?) and still like it a lot. However, there has always been a problem collaborating with devs who don’t want to code in CS. And when I help a JS project I have to write in JS. So there is an impedance mismatch. Also, as you referred to, CS is behind and probably always will be.

IMHO, Livescript and other extensions of CS are bloated. They are moving further away from JS while I’m trying to get very close. It’s kind of an opposite approach.

I may be optimistic to think I can get the advantages of both worlds but I’m trying. It isn’t going to be easy.


#9

I don’t think of JSW as a new full language. Technically it is only lexically different than JS. The abstract syntax is identical. And even then I’m only changing a small percentage of the JS lexicon. I’ll be measuring it in terms of what percentage of the parser (babel) and generator (escodegen) code lines change.


#10

I understand what you’re trying to do, and have wondered how cool it would be for something like this to actually exist without being a transpiler.

Just have one slightly off topic remark to do: looking at those two simple examples, I can’t help but notice that a similar conversion might be accomplished via a syntax theme instead. One that would dim out {}();, and maybe even dim out function and or plus a non-dimmed overlay of -> and and.

Edit:

As an example, putting this in styles.less

atom-text-editor::shadow .source.js {
  .punctuation, .delimiter, .brace {
    opacity: 0.1;
  }
}

Will do make the example

look like this


#11

Definitely an interesting approach! But it would still require the braces to be written in the code. Not the end of the world, but it’s just so nice not having to think about them :slight_smile:


#12

It is amazing how much that looks like coffee.

There could be a toggle key-binding to enable this view when reading the code and disable it when writing. The package could be named quiet. It might work on many languages.

Edit: That also gives me the idea that maybe JSW should do only this and the function thing. It would be easier for users coming from JS. I had said earlier that the JSW definition would be to put in everything I could from coffee while keeping the same AST. That might be the wrong approach.


#13

You might be on to something there Mark. I tried leaving that css on yesterday and definitely felt it was great for reading, not so much for writing code.

The keybinding for toggling might not even be necessary in most cases - it could just check if the file had been modified (via whatever mechanism shows the little modified dot on the tabs). I poked around for a bit but couldn’t find a css class being applied to the text editor when it has been modified though.


#14

That would have to be added in a package.


#15

Would this translator tool be the same thing as a transpiler, or would there be any differences? Would the idea be that you could debug in the transpiled language using a .map file, the same way CoffeeScript does it?

Or instead of a map file, could the transpiled code match each line of the source code?.. so if there’s an error on main.js:36, then you would look at main.jsw:36. Would there be any value in doing that? Would that make it easier for anybody to integrate this tool into their workflow?


#16

Like a transpiler, a utility must be run to switch between JSW and JS. But the JSW utility can also go in the reverse direction with no loss of formatting. No compiler/transpiler can do this. So when you are using Atom you can toggle between the two with a keypress.

The theory is that because the JS and JSW have the same semantics you will have no problem debugging in JS. The debugger will match what you see in the editor when you are in the JS mode. The line numbers will be the same.

JSW code is just another “view” of JS code, much like the example @filipesilva showed in the post above where it was literally the same code with the noise characters hidden. I can’t say for sure this will be useful. I myself may find it’s not worth the trouble to use.

I hope this answered your questions. I know this idea is bizarre and may even be unique. I’d love to find an analogy to something we are familiar with.


#17

I think you hit an interesting point. It’s like a code visualizer and/or formatter. I wonder if this could be 2 existing features put together: auto-complete and code-folding… So it auto-codes, then immediately folds the code that it auto-coded!

Let’s look at this in the example of adding/removing braces:

Auto-complete tools will automatically add the braces for you when you write a specific line of code that would need a code block under it. One could extend the functionality of this feature to enable the braces to automatically be removed when you remove the last line of code inside of them and the parent line of code.

Code-folding is commonly used to make consecutive lines of code disappear, so in your IDE you will see line 35 then line 42 right under it because 6 lines of code exist but aren’t being displayed. What if this were used to fold only one line of code at a time, so you’d have a “for” loop on line 35, then right under it, on line 37 you would have your indented code.


#18

I am hoping that features like auto-complete and code-folding will work with standard coffee settings. During development I’ve set the grammar for JSW to be Coffee and it is working.


#19

Very cool.
I briefly introduced your proposal in Japan.

Thanks :wink:


#20

Thanks.

Unlike a lot of my proposals I’m seriously working on this. I’ve forked the Uglify parser/generator. I’m about to demonstrate going to/from jsw code without disturbing the js formatting, although I’ve only implemented the skinny arrow -> so far.

This round-trip feature is critical for the usefulness and the part that is hardest to implement. I have a complex mapping implemented that keeps track of the js format. It is very much like a source-map for debugging.