Speed up suite html reporter

[psychon]

As the second commit explains, this speeds up a random test suite (the reason that I am working on this at all) from

   [testng] [TestNG] Time taken by org.testng.reporters.SuiteHTMLReporter@4f2410ac: 6644 ms

to

   [testng] [TestNG] Time taken by org.testng.reporters.SuiteHTMLReporter@78e03bb5: 275 ms

I tested this with the following patch (note: I did not get an OOM error by just using a StringBuffer, as proposed in #567, so this doesn't say much; however, the StringBuffer had a size of 42563696 = 40.6 MiB which is a lot, I guess). I watched the test suite's memory usage in top and could not see any difference with this PR compared to master or compared to my StringBuffer patch (always peaks over 1.5 GiB).

diff --git a/src/test/java/test/factory/FactoryTest.java b/src/test/java/test/factory/FactoryTest.java
index f44f3e9..45a4e28 100644
--- a/src/test/java/test/factory/FactoryTest.java
+++ b/src/test/java/test/factory/FactoryTest.java
@@ -6,21 +6,19 @@ import org.testng.Assert;
 import org.testng.annotations.AfterSuite;
 import org.testng.annotations.Factory;
 import org.testng.annotations.Parameters;
+import org.testng.internal.Utils;

 public class FactoryTest {
   static boolean m_invoked = false;

-  @Parameters({ "factory-param" })
   @Factory
-  public Object[] createObjects(String param) {
-    Assert.assertEquals(param, "FactoryParam");
-    assertFalse(m_invoked, "Should only be invoked once");
+  public Object[] createObjects() {
     m_invoked = true;

-    return new Object[] {
-        new FactoryTest2(42),
-        new FactoryTest2(43)
-    };
+    Object[] objs = new Object[2000];
+    for (int i = 0; i < objs.length; i++)
+       objs[i] = new FactoryTest2(i);
+    return objs;
   }

   @AfterSuite
diff --git a/src/test/java/test/factory/VerifyFactoryTest.java b/src/test/java/test/factory/VerifyFactoryTest.java
index 3ec6d56..9e6d244 100644
--- a/src/test/java/test/factory/VerifyFactoryTest.java
+++ b/src/test/java/test/factory/VerifyFactoryTest.java
@@ -12,8 +12,8 @@ public class VerifyFactoryTest {
       : "Didn't find 42";
     assert null != numbers.get(43)
       : "Didn't find 43";
-    assert 2 == numbers.size()
-      : "Expected 2 numbers, found " + (numbers.size());
+    assert 2000 == numbers.size()
+      : "Expected 2000 numbers, found " + (numbers.size());
   }


diff --git a/src/test/java/test/retryAnalyzer/RetryAnalyzerTest.java b/src/test/java/test/retryAnalyzer/RetryAnalyzerTest.java
index ee3fca6..e9c0d98 100644
--- a/src/test/java/test/retryAnalyzer/RetryAnalyzerTest.java
+++ b/src/test/java/test/retryAnalyzer/RetryAnalyzerTest.java
@@ -34,4 +34,12 @@ public class RetryAnalyzerTest extends SimpleBaseTest {
         List<ITestResult> skipped = tla.getSkippedTests();
         assertEquals(skipped.size(), InvocationCountTest.invocations.size() - fsp.size());
     }
+
+    @Test
+    public void foo() {
+       TestNG tng = new TestNG();
+       tng.setTestClasses(new Class[] { test.factory.FactoryTest.class});
+
+       tng.run();
+    }
 }

[juherr]

TestNG is using Java7, so you can use the try-with-resources statement.

[psychon]

Thanks for the info, I just copied this from Utils.java. Anyway, done in e353be2 (which I will rebase into a9c69dd).

[juherr]

@psychon Do you think you can add a test, extracted from the patch you provided?
And I think you can make another one (or just add a assertion in the previous test) that checks TestNG doesn't take too long time to generate the report?

[psychon]

@juherr What kind of test? The patch that I have in my previous comment doesn't have any behavior that can be tested for. As I said, I don't get any OutOfMemoryErrors and yet the above patch changes the test suite's peak memory usage from about 800 MiB to at least 1.5 GiB (I just looked at a system monitor for this).

Also I don't know about "too long". Someone might just run this over a slow USB harddisk which makes everything slow.

Sorry, I'm no good at this....