Grunt return 0 even if a protractorjs test fail

[manuelmazzuola]

Grunt returns 0 even if a protractorjs test fail and consequently jenkins return a false positive.
Instead when a karmajs test fail grunt returns with a fail status code as I expect.
Is this a grunt or a protractor issue ?

Running "protractor:run" (protractor) task

------------------------------------
PID: 12690 (capability: phantomjs #1)
------------------------------------

Starting selenium standalone server...
Selenium standalone server started at http://10.11.12.3:51446/wd/hub
F

Failures:

  1) E2E Scenario should redirect to login page
   Message:
     Expected false to be truthy.
   Stacktrace:
     Error: Failed expectation
    at null.
...
...
...
==== async task ====
asynchronous test function

Finished in 1.695 seconds
1 test, 2 assertions, 2 failures

Shutting down selenium standalone server.
[launcher] Runner Process Exited With Error Code: 1
>> 
>> Test failed but keep the grunt process alive.

Done, without errors.

[vladikoff]

Which plugin are you using? If it is this one https://github.com/teerapap/grunt-protractor-runner/blob/master/tasks/protractor_runner.js#L112 then it probably needs to adjust how it deals with errors.

[jarl-dk]

It seems that it is a configuration issue, see comments in teerapap/grunt-protractor-runner#63

[AndersDJohnson]

I still have a need to run other grunt tasks after a failed test - e.g. to shutdown my local server and disconnect from Sauce Connect. Does Grunt have an API for deferring failed exit status codes? teerapap/grunt-protractor-runner#63

[chitwant82]

@adjohnson916 : I use keepAlive in protractor conf and in the case of test failures grunt exits with code 0, however on the CI server (Jenkins in my case) I use text finder plugin to search in console output for specific pattern like # spec, 0 failures. If this is not seen mark build fail. You can also write a shell script as a post build action to get console output via curl and parse for specific string and mark the build pass/fail accordingly

[rohitsakala]

@chitwant82 any other simple solution ? rather than checking for patterns ?

[ArmorDarks]

You can try to use process.on('exit') in Gruntfile, check for code and do a cleanup before exit.