Layered Configuration


#1

I’ve been thinking about this ever since the discussions raged about not having per-syntax or per-project settings. The Config class already has support for two “layers” of settings, defaults and the user’s settings. We just need to extend it to handle a couple more. I wanted to discuss it here first so that people could give me ideas before I tried to implement it.

The layers that I’ve seen suggested in one form or another are the following (in order from highest to lowest precedence):

  1. Project-specific settings
  2. Syntax-specific settings
  3. User settings
  4. OS defaults
  5. Editor defaults

Layers #3 and #5 we already have. It was suggested to add #4 and I think it is a good idea. The thing that layers #3-#5 have in common is that they’re not conditional. Layers #1 and #2 change depending on what file or folder we’re talking about. (Though I want to get rid of the idea of syntaxes keying off of file extensions and change to keying off of grammar names … that’s an idea for another time.) So my thinking was that the signature of the configuration methods would change slightly …

class Config
  get: (keyPath, editor, project) ->
    # ... snip ...

If editor or project weren’t supplied, then everything would work as it does currently (with the added benefit of having OS defaults available) … so no breaking change. If they were supplied, then Config would add the syntax- and project-specific settings to the setting resolution scheme and get the value.

The reason why I chose the Editor object rather than the file path is because perhaps the user has set the grammar to something different this session? I think we would want the configuration retrieved to reflect the grammar selected. For similar reasons I chose the Project object, though this means probably that sub-paths under the project wouldn’t be able to have their own settings that override the project settings.

What does everyone think?


#2

Seems nice to me, but I’d rather have an option object instead:

class Config
  get: (keyPath, {editor, project}={}) ->
    # ...

At least it doesn’t force an order for arguments, all configs aren’t related to editors or grammars and calling atom.config.get('some.path', null, atom.project) doesn’t seems like the prettiest way to do it.

I also wonder if the project argument is really required. As is, every calls that omit the project argument won’t be able to get access to the project specific settings, it puts the burden to figure out which settings can be overridden at the project level onto the user and I feel that it should be completely transparent from a user perspective.


#3

I like the concept, and is something I really feel that Atom needs. That being said, I’m against changing the signature of Config::get(keyPath). A plugin author should neither need to, or be allowed to, specify where configuration values are supposed to come from. This should be entirely controlled by the user.

In addition, I strongly suggest that the overrides you suggest at the project level, be driven by directories; the closest ancestral config file’s value to the file being edited, should be used. E.g. if I’m editing ./project/some/path/file.hs, and there are config files located at each of those subdirectories (including the root), then the overrides should flow through from the root towards file.hs, and the closest overrides should be returned when retrieved via Config::get(keyPath). That way, even a project with e.g. a git submodule that uses a different code style from the main project, could have its own config values for indentation etc.

I believe this is the simplest, yet most flexible and common pattern for solving this kind of scenario.


#4

Except that every packages and configs aren’t related to the active editor and its grammar. What if I set a config for the tree-view so that it’ll display on the right for coffeescript files and on the left for everything else ? Does it make sense ?


#5

My initial thought is, if the user wants to override that per directory, then why shouldn’t she be allowed to?

Now, if we really don’t want to support this, then we should rather introduce a differentiation between different types of configs; editor config and package config. The former being anything related to editing of files (grammar based indentation rules, tab style, etc) and the latter being anything related to the functionality of the package itself. That could be things like whether to place the tree view on the left or the right side of the GUI, whether to trim whitespace from otherwise empty lines etc.
This differentiation should be done via different instance methods on Config, or better yet, two config classes?

Perhaps the existing Config::get(keyPath) would still be used for package specific configuration (unaffected by grammar) to maximize backwards compatibility. And a new EditorConfig::get(keyPath) that would be used for editor/grammar settings? With a name like that, it ties in beautifully with http://editorconfig.org/ as well.


#6

True, maybe the tradeoff can be to have the per grammar/project layers enabled by default, but packages writers could choose to exclude one of this layers in the get call. It’s just a matter of reversing the logic:

  • atom.config.get(keyPath) returns the version affected by all settings layers
  • atom.config.get(keyPath, excludeGrammar: true) ignores grammar overrides

#7

Sure, but personally I’d prefer the EditorConfig class. Supporting the editor config concept makes more sense than anything else for grammar specific config anyway, because it’s cross-editor.

There’s already an Atom package for it, but it lacks support for the entire EditorConfig spec, most likely due to missing support in Core. Although in my opinion it might make more sense for this to be a core feature of Atom. But at the very least we’d need to extend Core enough (if required) for the package to be capable of supporting the full spec.

I can have a look at extending the package with further support for the spec over the weekend, and I’ll make a list of what changes need to be done to Core (if any).

/cc @sindresorhus


#8

I like your idea for the signature better, consider it adopted.

As to the project thing, I suppose one can get to the project instance from the editor instance … so specifying both may be moot. I’ll look into that.


#9

Well, I disagree that it is the simplest … it is more flexible yes. But leaving that aside for now …

I dislike the idea of having two configuration classes. The whole point of having a layered configuration is so that the client code doesn’t have to worry where the actual setting comes from. Otherwise we can just make it that Config can get its settings from anywhere and make the client code instantiate a bunch of them and put all the burden on the client code.

I have issues with the implementation of EditorConfig, actually, especially with regard to compatibility with Atom’s configuration system:

  1. As I have stated before, the INI format it uses is less flexible than Atom’s CSON infinitely nestable objects or Sublime Text’s JSON config … or a theoretical YAML config … or … well, you get the picture
  2. I also disagree with having two places to specify file masks:
    1. In the grammar
    2. In all the .editorconfigs

Specifically with regard to #2, if I want to make it so that Foofile now is interpreted as Ruby I have to add that to the grammar (which requires a pull request to the grammar author … something else I want to fix) and I have to edit potentially every … single … .editorconfig file … ever. Whereas if the project-specific settings files simply say “Ruby”, then I only have to change the file mask for Ruby files in one place.

If you have ideas @thomasjo or @sindresorhus on how to make my layered config design pluggable to make later extension of it to support EditorConfig easier (for those that want to use it), I’m more than willing to go to great pains to enable that. I just don’t want to lock Atom into supporting a project-specific config system that I believe is flawed.

Because really … that’s what the Atom philosophy is about right? Giving people more choices?


#10

I think you might be misunderstanding the point of EditorConfig (but I also might have misunderstood what you truly want to achieve); the purpose of the .editorconfig files is that a single file will work across every single editor and IDE that supports EditorConfig (to date most do so via plugins). I fully agree that the INI format is less flexible and what not, but that’s what the EditorConfig folks chose, sadly.

In addition, EditorConfig is only concerned with how to format code (tab style, tab width, line endings etc), and there’s nothing wrong with adding custom stuff, although it will likely be unsupported. There’s no concept of grammar in the spec. You merely override config values based on GLOB patterns, which are typically of the form *.rb, *.py and what not.

A project typically only ever has one .editorconfig file, so there’s no need to edit every single file ever (paraphrasing). I don’t really see the problem you are seeing. But perhaps you’re after something more than what it is trying to solve? I want a file that tells everyone how the code in my repo should be formatted, and ideally that file should be interpreted by the editor (or IDE) and adhered to. This is not something that is user-specific, it’s truly project-specific and is always checked in as part of the source code.

I think we might be misunderstanding each other. Perhaps we can each come up with a simple, minimal example of exactly what we’re trying to achieve? A simple text based project tree outline, with some mockup config files?


#11

I’ll try to find time to mock something up in the next few days.

:+1:


#12

The problem I see is that I have 15 Ruby projects, each of them with their own .editorconfig file. If I want Foofile to follow the rules that I’ve set up for Ruby code in all of my projects I have two choices:

  • Make the change in my global .editorconfig file which only makes that change for me
  • Make the change in all 15 project-specific .editorconfig files so that everyone gets the benefit of the change

I realize that the “every single file ever” scenario is worst case, but I feel it is unnecessary if we raise the abstraction one level from file globs to syntax names.

But I’m sure EditorConfig works for a lot of people … it obviously has its staunch supporters :laughing: I just feel like there is a better way for some of these things. And maybe this discussion is exactly the kind of cross-pollination that will improve both projects.


#13

So I have finally gotten back to this concept and rather than trying to keep up, in my spare time, with the pace of the Atom team, I decided that I can test all of my ideas in a package. Then when things are fully fleshed out, I can see about merging those ideas into Atom proper via a pull request. (This approach also makes testing easier. No wonder Atom is mostly just a big collection of packages :grinning:)

I’ve created a repository called layered-config where I’ll be showing my work, if people are curious. I also intend to write one or more blog posts about it including my thoughts and intermediate ideas (though the priority on this is way lower than just getting the thing done … so there’s that :laughing:). If people see gotchas or flaws in my design, feel free to write up an Issue or even Pull Request on the repository … just know that there are a lot of ideas in my head that are not reflected in the repository at any given time … so what you’re thinking may already be covered, just not apparent.


#14

I’ve gotten the code and the README to where I think it is really ready for feedback and ideas. The basic idea is for Atom to create a series of configuration wrapper objects for various purposes. When one queries the outermost wrapper, they are getting the culmination of the following layered settings:

  1. Default
  2. User-configured
  3. Syntax-specific
  4. Project-specific
  5. Project-specific Syntax-specific

I’d love to hear what people think!

cc @nathansobo, @ProbablyCorey, @benogle, @kevinsawicki


#15

Given that the concept of multiple projects in the same instance of Atom, is probably going to be coming at some point in the future, do you see this approach fit in with that?


#16

Yes, the underlying approach is sound and it doesn’t matter how complicated the list of layers is, just so long as you can identify the set of configurations that you want to participate. On the other hand … the concept of multiple root folders in a project does not necessarily equate to multiple sets of project settings. As a matter of fact, you have to store the project definition somewhere and storing the project’s settings in the same place is an obvious solution.


#17

So this is what I was wondering about. Mainly because it makes the definition of what a “project” is, kind of fuzzy. Or at least I don’t fully understand what it might be, once people have the ability to open a number of independent folders at the same time.
Are they then all part of the same “project” as far as your project settings layer is concerned, with nested sub-project layers?


#18

That’s something that will have to be decided when this change is made. Personally, the concept of a “project” is “all the stuff that I might need to access for whatever it is I’m working on”. This would match with the idea of a project having multiple “root” folders with a “project root” that doesn’t necessarily live anywhere in the file system. But this is all kind of separate from the configuration idea.


#19

I think it’s okay if we allow the user to shoot themselves in the foot. Let every config option be tweakable in every layer, and maybe some options don’t make sense in some layers, but so what. Also, if you want the package author to say which option makes sense in which layer, then you will be less flexible about changing the layers later.


#20

Sorry for bumping but is this something that’s happening?