Package not triggered in spec / no onDidStopChanging event


#1

Hello !

So, my package is triggered after an onDidStopChanging event. Which does work fine.

However, I would like to add some tests to make sure everything runs smoothly when this event is triggered. But, I have never been able to do so.

I intended to first, just test that after a modification file is not in isModified state anymore, but my package doesn’t seem to be triggered, and the file never gets saved.

I have never been able to catch the onDidStopChanging call when trying stuff with specs, my package is activated and enabled, I’m really lost here.

Here’s the last code

    it("Should call automatically save file after a change is made", () => {

        atom.config.set('autosave-onchange.enabled', true);

        let buffer = editor.getBuffer();

        editor.insertText( 'Stuff' );
        buffer.emitter.emit('did-stop-changing');

        waitsFor(() => buffer.onDidStopChanging.callCount > 0 );
        runs(() => expect( true ).toBe( false ) )

    });

returns a timeout error
( This one just tries to catch the onDidChange event for now )

I’m probably forgetting something, but would love to know what.

Thank you package pros :smiley:

( solutions provided in this thread doesn’t seem to work )


#2

There could be any number of things wrong here. A couple that come to mind:

  1. Does your package check the autosave-onchange.enabled value after activation?
  2. Did you mock onDidStopChanging? And if so, does the mock know to call your handler? Or does the mock properly get called when emit('did-stop-changing') gets called?

Also, you’re running into a problem that I call testing code that isn’t yours. Your test here is designed to test that the event system works, which you already know it does based on your manual tests and the lack of bugs reported against Atom on it’s event system :wink: What you should be testing is if your code does the right thing when the event occurs.

A completely made up example:

export default class AwesomeSaucePackage {
  activate () {
    this.subscriptions = atom.workspace.observeTextEditors((editor) => {
      let buffer = editor.getBuffer()
      let stopChangingSubscription = buffer.onDidStopChanging(() => {
        this.sauceItUp(editor)
      })

      editor.onDidDestroy(() => {
        stopChangingSubscription.dispose()
      })
    })
  }

  deactivate () {
    this.subscriptions.dispose()
  }

  sauceItUp (editor) {
    // Do something awesome with the sauce
  }
}

And a chunk of test code:

// Some `beforeEach` code above sets up `package` and `editor`
it('should do the awesome with the sauce', () => {
  package.sauceItUp(editor)

  expect(isItSauced(editor)).toBe(true)
})

If your event handlers are always one-line calls to specific functions in your package, then you can test your package there and not worry about testing if Atom’s event handler system works.

You can do end-to-end testing if you like, but trust me when I tell you that end-to-end tests are amazingly fragile by nature and you’ll spend far more time in trying to get them to work and maintaining them than they’re worth unless you have a dedicated test team backing you up.


#3

Thank you leedohm for that very thourough answer !

You’re so right, I ended up trying to test the trigering of onDidStopChanging because I couldn’t figure out why my test wasn’t passing.
But then I completely lost focus and ended up doing just as you said. Trying to test code that isn’t mine :smile:

1 - yop, I found that out looking through some packages :blush:

2 - I did mock onDidStop changing, but I never managed to get it called after emitting ‘did-stop-changing event’. Still haven’t figured ou why.

I do have a question though,
this test does rely on code that isn’t mine ( as you said, it’s up to atom to trigger the call ), shouldn’t I test that my plugin does what I expect it to do ?

What I mean by that is that, at the end of the day, I want to test that { when stuff happens in editor }, { new state gets saved }.
I do test my other functions, but if I forgot to place the call inside onDidStopChanging or something, all my tests pass… but my plugin kinda fails at life.

Do you see what I mean ? (not sure I’m being clear)


#4

I understand what you’re saying, yes. And yes, you do want to test that your package doesn’t fail at life. But that can be achieved by one manual test once in a blue moon. Take a look at the tests in my tabs-to-spaces package. Here is the breakdown:

  • 2 tests to ensure that I add and remove the package’s commands at the proper times (and even these aren’t really necessary, but I leave them in because meh)
  • 27 functional (unit) tests

I don’t test that the commands call the right functions because I wrote those once ages ago, tested them manually and never touched them again :grinning: The thing to keep in mind is that you only need to test what you might break. Or perhaps more accurately, you should test what is most likely to break most often and things that are very unlikely to break you should test least often.

If you test:

  1. That you add a handler to onDidStopChanging
  2. That the function called in that handler does the right thing

Leaving the part that the handler calls the correct function untested is really not a very big risk. (And testers are the lawyers of the software world, their job is to assess and mitigate risk :wink:) It is an even lower risk if you test this manually even once in a blue moon.

All of the time that:

  • You have spent trying to get this test working
  • You have spent writing up your problems getting this test working
  • I have spent replying :wink:
  • You will spend over time maintaining a brittle test

completely outweighs the benefit that this single test will provide, especially when a simple manual test that takes at most two minutes to execute will provide the same benefit. (Possibly even more because the automated version still won’t guarantee that you wrote the test correctly.)

You may also want to check out this post on the Google Testing Blog that describes the (in my opinion optimal) test pyramid. It does say that you should have 10% end-to-end tests … but in something as small as most Atom packages that have a single developer, I would say that 10% is overkill.


#5

Hello leedohm,

I don’t have much to say after that since I’m completely won over, It actually made me question my code way outside the package itself :slight_smile:

Again, thank you so much for your answer, and your link to the google testing blog totally made my day (neeerd :D)


#6

You’re very welcome :grinning: I’m glad that I could help. Quality Assurance (or as one of my mentors would say, Quality Visualization) is one of my passions in software development.