Spectron Tests Unreliable on Travis


#1

Hi All,

I’m wondering if I could get some feedback from those who have setup Spectron tests to run on Travis CI (or anything else).

Have you noticed that they are unreliable? I have several tests setup, and they seem to fail somewhat randomly. On one run, one test might fail and others pass. The next run, that test passes, but another fails. The tests never fail on my machine, however.

Feel free to look through my tests, too! This is one that often fails: https://github.com/atdrago/negative/blob/develop/test/tests/menu/edit/undo.js

These tests are mainly designed to test menu item functionality. I’ve described my strategy here: https://github.com/electron/spectron/issues/21#issuecomment-219057770

Would love anyone’s thoughts.

Thank you!
Adam