Find a good pattern for waiting for Jasmine Reporters

[juliemr]

Right now, jasmine does not wait for anything asynchronous that may have been started in a reporter on specDone or suiteDone. If the asynchronous events get added to the webdriver control flow properly they will still happen before the process shuts down, but it's hard to rely on this and we should have a more general solution.

[hco]

+1

[eddywashere]

@juliemr are there any patterns for waiting, ex: in specDone?

[eddywashere]

For now I ended up using:

afterAll(function(done){
    process.nextTick(done);
});

[tokunbo]

@eddywashere

[ksheedlo]

@tokunbo @eddywashere The problem with that hack is that it looks like a noop. If I saw that in a code base I was working on without any surrounding context, I would rip it out.

[sheltonial]

+1

[iljapavlovs]

+1

[ocombe]

Thanks @eddywashere you saved my day, this error is such a pain ...

[kimwangit]

+1. @juliemr is there any progress about this bug?

[sjelin]

No progress. For what it's worth, I once ran into this problem and used teardown function of a plugin to keep the process alive until everything is finished, but it's not a great solution (e.g. everything will finish but not necessarily in the order you'd like). Donno if that helps.

[kimwangit]

@sjelin thanks for your update, could you help give some code sample to explain how to use teardown to keep the process alive? I want to see if this workaround is suitable for my e2e application.

[sjelin]

Sure thing. Basically, have some plugin file, let's call it waitPlugin.js:

var q = require('q');

var deferred = q.defer();

exports.resolve = function() {
  deferred.resolve.apply(deferred, arguments);
};

exports.teardown = function() {
  return deferred.promise;
};

Then, in your config file, you include your plugin:

exports.config = {
  ...
  plugins = [{
    path: 'waitPlugin.js'
  }]
  ...
};

Now Protractor will know to wait until the promise returned by waitPlugin.teardown() is resolved. So in your tests, when everything's done, you resolve the promise from the plugin:

var waitPlugin = require('./waitPlugin.js');

...

  waitPlugin.resolve();

...

Because of the way node caches require(), calling waitPlugin.resolve() in one file will resolve the result of waitPlugin.teardown() from another.

Does that make sense? Like I said, it's a bit of a hack.

[tsaikd]

I patch jasmine to support kriskowal/q on specDone()
tsaikd/jasmine@d539eb2

// conf.js
var Q = require("q");
exports.config = {
    // some config ...
    onPrepare: function() {
        jasmine.getEnv()
            .addReporter({
                specDone: function() {
                    var deferred = Q.defer();
                    setTimeout(function() {
                        deferred.resolve();
                    }, 1000);
                    return deferred.promise;
                }
            });
    }
};

I think it's ugly, so I don't send PR to jasmine.
Maybe this patch will help someone.

[guiprav]

@tsaikd, what's ugly about it? I wish the patch was available from upstream :(

[tsaikd]

tsaikd/jasmine@d539eb2 will make some tests failed
I am not sure how to fix the test because of the spec description :(

spec/core/integration/EnvSpec.js#L1461
an async spec that is actually synchronous

I found jasmine/jasmine#965 is also a Promises PR, but not yet merged.

[joelfogue]

Any update on this issue; I'm running into same problem after running last scenario's step in Cucumber "Error: This driver instance does not have a valid session ID (did you call WebDriver.quit()?) and may no longer be used."
Is this a bug in Protractor? Don't want to use noops as suggested earlier; a proper fix would be greatly appreciated.

[sjelin]

Still no update. Realistically, the frameworks need to provide the updates.

[guiprav]

@juliemr, how is this ranking in your backlog? It's causing me some real pain and I can't find satisfactory solutions. I couldn't get the afterAll hack to work either, so I'm at a loss.

[sjelin]

It's unlikely to happen soon because it's dependent on the jasmine team. I assume the plugin hack also didn't work for you?

[guiprav]

Yeah... On that note, am I the only one who thinks Jasmine was a seriously poor choice for Protractor? (not suggesting Mocha would have been any different).

[sjelin]

A lot of people like jasmine. But regardless, you can use any framework if you write a provider. Cucumber support is maintained by a third party for instance: https://github.com/mattfritz/protractor-cucumber-framework

[guiprav]

A lot of people like the Jasmine API. I don't mind it, but Protractor had to patch Jasmine to make it all work, and clearly it hasn't patched it enough. That's what I meant: Jasmine was never meant to be used in these ways; that's why I think it may have been a really bad choice.

Jasmine expects many things to be synchronous (describe calls, it calls, reporter callbacks), and Protractor needs to jump over hoops to make those work asynchronously. And since end-to-end testing falls outside Jasmine's core purposes, we're stuck in this situation where it's unlikely that we'll get the upstream features we need anytime soon.

(And yes, you can use any other framework you like, but Jasmine is the default, and I didn't expect to find so many problems with the default, official choice).

[sjelin]

Well, within Google and within the Angular team jasmine is very popular. We actually do have first party support for mocha (my personal favorite), but it's never been as popular as jasmine. We have to do what our users want. To be fair, jasmine had a head start, but still.

Anyway, I'll try to finish #1944 by the end of the week, which might help provide you with an easier way to wait for jasmine reporters

[fgather]

Thank you, sjelin looks like #1944 will solve all our problems. Is there a release date yet?

Edit: by the way your waitPlugin.js works for me, I resolved the promise in the onComplete() function. Thx!

[sjelin]

No timeline on when #1944 will be released. Glad waitPlugin.js worked for you!

[kummerer94]

@fgather Since version 3.1.0 onComplete() can return a promise. Still, I can't seem to find a working fix for the problem which still occurs to me. I am using the protractor-jasmine2-screenshot-reporter (version 0.3.0).

Can anyone suggest a workaround?

[juliemr]

Can we make the wait plugin easily available/default in some way? Assigning @sjelin since you've got background here.

[mvndaai]

This is a hack, but I was just using the reporter because I wanted to send results to Testrail. Since specDone is not asynchronous, I use specStarted to leak my results then use the afterEach to send them.

This is my simple reporter:

resultLeaker = {
  suiteStarted: function(result){ jasmine.results = {suite:result}; },
  specStarted: function(result){ jasmine.results.spec = result; }
};
jasmine.getEnv().addReporter(resultLeaker);

Then in your afterEach you can wait for promises:

afterEach(function(done){
      browser.sleep(0).then(function(){
        console.log("jasmine.results.spec.fullName:", jasmine.results.spec.fullName);
        console.log("jasmine.results.spec.failedExpectations.length:", jasmine.results.spec.failedExpectations.length);
        done();
      });
 });

[Kayuaga]

Hi everyone . I found the way how to deal with jasmine's report functions =). Hope that isn't the hardest one. I work at the reporter that uses rest API for sending test data to the external server. And faced problem that jasmine doesn't wait for async functions. That have been resolved in the next way.
Our reporter has a common JS client , which API provides for test frameworks reporters. So this client has field that stores array of the promises and method that returns this array.

getPromiseFinishAllItems(launchTempId) {
// this.map is an array of promises
        const launchObj = this.map[launchTempId];
        return Promise.all( launchObj.childrens.map((itemId) => { return this.map[itemId].promiseFinish; }));
    }

After I create main API , i start implement it with jasmine's report functions. All this functions send informations about promises to the JS client. For handling commutation between this module i created a launch file which create instance of the JS client , that has been sent to the jasmine reporter, so it has the same instances. At this launch file I created functions that returns all promises stored at the JS client that has been send from the jasmine reporter

class ReportportalAgent {
    constructor(conf) {
 . . .
        this.reporterConf = Object.assign({
            client: this.client,
            tempLaunchId: this.tempLaunchId,
            attachPicturesToLogs: true
        }, conf);
    }

// this method must be used at the conf function   //jasmine.getEnv().addReporter(agent.getJasmineReporter());
    getJasmineReporter() {
        return new JasmineReportportalReporter(this.reporterConf);
    }

 /*
     * This method is used for frameworks as Jasmine and other. There is problems when
     * it doesn't wait for promise resolve and stop the process. So it better to call
     * this method at the spec's function as @afterAll() and manually resolve this promise.
     *
     * @return a promise
     */
    getAllClientPromises(launchTempId){
        return this.client.getPromiseFinishAllItems(launchTempId)
    }

. . .
}

module.exports = ReportportalAgent;

And all this stuff must be used at the protractor's or jasmine's config file . Method that returns all promises must be called at afterAll or afterEach sections with done() callback. For me that works great with single and multithreading launches.

const ReportportalAgent = require('../../lib/reportportal-agent.js');
let agent;

exports.config = {
    specs: ['testAngularPage.js', 'testGithubPage.js'],
    onPrepare(){
            agent = new ReportportalAgent({
            token: "00000000-0000-0000-0000-000000000000",
            endpoint: "http://your-instance.com:8080/api/v1",
            launch: "LAUNCH_NAME",
            project: "PROJECT_NAME",
            attachPicturesToLogs: false,
        });
       afterAll((done) => agent.getAllClientPromises(agent.tempLaunchId).then(()=> done()));
        jasmine.getEnv().addReporter(agent.getJasmineReporter());
    },
  
};

I hope that would help anybody =)

[yjaaidi]

Hello,

As you can see in our fork of protractor-beautiful-reporter https://github.com/wishtack/protractor-beautiful-reporter/blob/master/index.js we are using the following pattern:

class Reporter {

    _asyncFlow: Promise<any>;
    
    jasmineStarted() {

        /* Wait for async tasks triggered by `specDone`. */
        beforeEach(async () => {
            
            await this._asyncFlow;
            this._asyncFlow = null;
            
        });
        
    }
    
    specDone(result) {
        
        this._asyncFlow = this._asyncSpecDone(result);
        
    }
    
    async _asyncSpecDone(result) {
        
        // @todo: Do your async stuff here depending on `result.status`, take screenshots etc...
        // await takeScreenshot();
        
    }

}