TestNG, Multiple duplicate listener warnings on implementing multiple listener interfaces

[bipo1980]

TestNG Version

6.11

Also refer - Link to Stackoverflow Comments

Expected behavior

If i have a single Listener class that implements multiple listeners eg: IAnnotationTransformer, IInvokedMethodListener, ITestListener and IReporter etc...
Then the warning indicating multiple listeners must not be displayed if this the act of implementing multiple TestNG listener interface is of no consequence.

Actual behavior

My Test run successfully BUT i get a warning "[TestNG] [WARN] Ignoring duplicate listener : com.NGtest.RetryListener". This warning is shown for every method call i.e. Listener gets called multiple times per test method invocation, and hence the IGNORED duplicate warning.

Is the issue reproductible on runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

Test case sample

Listener Class:

import java.util.List;
import org.testng.IInvokedMethod;
import org.testng.IInvokedMethodListener;
import org.testng.IReporter;
import org.testng.ISuite;
import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;
import org.testng.xml.XmlSuite;

public class RetryListener implements IAnnotationTransformer, IInvokedMethodListener, ITestListener, IReporter {
	@Override
	public void transform(ITestAnnotation annotation, Class testClass, Constructor testConstructor, Method testMethod) { }

	@Override
	public void beforeInvocation(IInvokedMethod arg0, ITestResult arg1) { }

	@Override
	public void afterInvocation(IInvokedMethod arg0, ITestResult iTestResult) { }
	
	@Override
	public void onFinish(ITestContext context) { }

	@Override
	public void onStart(ITestContext arg0) { }

	@Override
	public void onTestFailedButWithinSuccessPercentage(ITestResult arg0) {}

	@Override
	public void onTestFailure(ITestResult arg0) { }

	@Override
	public void onTestSkipped(ITestResult arg0) { }

	@Override
	public void onTestStart(ITestResult arg0) { }

	@Override
	public void onTestSuccess(ITestResult arg0) { }

	@Override
	public void generateReport(List<XmlSuite> arg0, List<ISuite> arg1, String arg2) { }
}

Test Class:

import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class RetryProvider {
    @Test (dataProvider = "datasource")
    public void Test1(int code, String type, ITestContext context){
        System.out.println("Test1(): " + code + " : " + type);
    }

    @DataProvider(name = "datasource")
    public Object[][] getData(){
        Object[][] data = new Object[3][2];
        data[0][0] = 1;
        data[0][1] = "apple";
        data[1][0] = 2;
        data[1][1] = "mango";
        data[2][0] = 1;
        data[2][1] = "guava";
        return data;
    }       
}

Test Xml:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="NGBug Suite">
	<listeners><listener class-name="com.NGtest.RetryListener" /></listeners>
	<test name="NGBug Test">
		<classes><class name="com.NGtest.RetryProvider" /></classes>
	</test> <!-- Test -->
</suite> <!-- Suite -->

Test Result:

[TestNG] [WARN] Ignoring duplicate listener : com.NGtest.RetryListener
[TestNG] [WARN] Ignoring duplicate listener : com.NGtest.RetryListener
[TestNG] [WARN] Ignoring duplicate listener : com.NGtest.RetryListener
Test1(): 1 : apple
Test1(): 2 : mango
Test1(): 3 : guava

[MartinKanters]

I have seen this issue as well today, while debugging issues with pitest versions 1.1.5 and up.
I will make a ticket there as well, but it is related to TestNG, since the pitests are failing because of the error "Ignoring duplicate listener : org.pitest.testng.TestNGAdapter".

As mentioned in the StackOverflow comment by Robby Cornelissen it probably boils down to this commit which is in TestNG versions 6.10 and 6.11.

I've verified that TestNG 6.10 and 6.11 in combination with pitest versions 1.1.5 and upwards are breaking.

Project showcase of the conflicting versions: https://github.com/MartinKanters/pitest-testng-issues

[krmahadevan]

@MariusVolkhart - You are right about that commit. TestNG was changed to ensure that duplicate listener instances aren't wired in. Can you please elaborate as to what is the issue that you are facing ?

[krmahadevan]

@bipo1980

This warning is shown for every method call i.e. Listener gets called multiple times per test method invocation, and hence the IGNORED duplicate warning.

That's not how the behavior was expected to be. It should be showing this warning ONCE per duplicate instance being injected.
Can you please help share a full fledged sample that I can execute to recreate this problem ?

[MartinKanters]

@krmahadevan Hm, it seems that the issues are not really related after all. I failed to notice that this issue was about adding one class implementing multiple listeners. It seems that pitest is reusing the same TestNG instance for their mutation tests. The owner of pitest is working on it on their side.

[bipo1980]

@krmahadevan Done. updated the original sample with the complete code. Hope this will suffice.

[krmahadevan]

@bipo1980 - This looks like a problem only when the suite xml is executed from within eclipse (using the eclipse TestNG plugin). I have not been able to recreate the problem using either IntelliJ (or) when executing via a command line using Maven.

ping @missedone - Can you please help take a look at this one ?

[missedone]

@krmahadevan, RetryListener in the example implements IReporter, when launch the suite in Eclipse Plugin, it was added multi times.

1. AbstractRemoteTestNG#run() -> TestNG#run() -> initializeConfiguration() -> add all suite listeners, see code here
2. TestNG#run() -> runSuites() -> runSuitesLocally() -> createSuiteRunners() -> createSuiteRunner() -> add all suite reporters, see code here

so the fix might be set a flag (e.g. cfgInitialized=true) in initializeConfiguration() then check the value in createSuiteRunner() to avoid duplicate