-
Notifications
You must be signed in to change notification settings - Fork 8
/
TODO.txt
516 lines (488 loc) · 34.4 KB
/
TODO.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
!! - memory-mapped files don't work on macOS: communicate by writing/tailing normal files
- upgrade to Java 21
- commit the generated event files to Git
- stop using jumi-actors-generator
- convert the events to record classes
https://openjdk.org/jeps/395 (Java 16)
- seal the classes: SuiteEvent, RunEvent etc.
https://openjdk.org/jeps/409 (Java 17)
- use pattern matching switch/case instead of visitor pattern
https://openjdk.org/jeps/441 (Java 21)
- remove the old visitor-based event interfaces
- rewatch "A Classfile API for the JDK" and think if our current event model is good enough
https://www.youtube.com/watch?v=pcg-E_qyMOI
- is the current normalized event format ok?
- normalize only when serializing, and use denormalized events in memory?
- do we need helpers for easier consumption?
OK - refactor BuildTest & SuiteConfiguration
OK - extract reusable components from BuildTest to an external library
OK - jumi: extract reusable components from TestEnvironment
OK - jumi: extract reusable components from BuildTest
OK - use improved parameterized runner; remove hack in fi.jumi.test.util.JarFileUtils#checkAllClasses
OK - create new project, groupId: fi.luontola.buildtest
OK - create a stripped history of /end-to-end-tests/src/test/java/ that contains only:
- BuildTest, TestEnvironment (and any new classes extracted from them)
- AsmMatchers, AsmUtils, CompositeMatcher, JarFileUtils, XmlUtils
OK - merge the stripped history to the new project, reorder packages
OK - tests for non-trivial helpers: JAR, Maven
OK - downgrade to Java 6
OK - update the BuildTest in jumi and jumi-actors to use that library (at first in a feature branch, because Maven Central Sync will take time to activate)
OK - rename SuiteConfiguration.classPath to classpath
OK - deprecate current methods
OK - create new methods
OK - create a process for removing the deprecated methods after a particular number of days or releases
OK - BuildTest: maintain a list of all deprecated APIs and a date/version when they were deprecated, and when to remove them
OK - release the buildtest library
OK - merge feature branch, use the released buildtest library
!! - communicate using memory-mapped files
http://en.wikipedia.org/wiki/Memory-mapped_file
http://javarevisited.blogspot.fi/2012/01/memorymapped-file-and-io-in-java.html
http://stackoverflow.com/questions/5943365/java-tcp-ip-socket-latency-stuck-at-50-s-microseconds-used-for-java-ipc
http://stackoverflow.com/questions/6096485/how-to-dynamically-expand-a-memory-mapped-file
https://github.com/peter-lawrey/Java-Chronicle (see how IPC is done here)
* benefits:
- more secure than sockets (uses file system permissions)
- should be faster (benchmark latency and throughput)
- automatically saved to file for later use
- no need to close socket connections
- no need for a networking library
* disadvantages:
- how to manage multiple writers? one file per writer?
- probably requires polling to get informed about updates
* need to find out how to do synchronization
http://stackoverflow.com/questions/4743380/controlling-read-writes-to-memory-mapped-files-windows
- create a util like Fences? http://hg.openjdk.java.net/lambda/lambda/jdk/rev/6482b24ddd4c
- lazySet/StoreStore barrier good for throughput in single-writer scenarios (as long as doing writes continually): http://psy-lob-saw.blogspot.fi/2012/12/atomiclazyset-is-performance-win-for.html
- beware of word tearing, MMFs guarantee nothing: http://psy-lob-saw.blogspot.fi/2013/02/alignment-concurrency-and-torture-x86.html
- list of possible optimizations: http://psy-lob-saw.blogspot.fi/2013/07/single-producer-single-consumer-queue.html
- MappedByteBuffer slower than ByteBuffer: http://mentablog.soliveirajr.com/2012/12/asynchronous-logging-versus-memory-mapped-files/
OK - spike communicating over memory-mapped files
OK - create IpcBuffer which wraps multiple MappedByteBuffers; combine the good parts of:
Mina 3:
+ lookup using singly linked list -> let's use a doubly linked list with O(1) in both directions, and field "current" instead of "head & tail"
- accesses primitives using 1-byte operations
+ only 1 public and 2 private classes
- class is final, cannot customize easily
- has bugs and low coverage (just reported https://issues.apache.org/jira/browse/DIRMINA-949 and https://issues.apache.org/jira/browse/DIRMINA-950)
- no putString
- doesn't add ByteBuffers automatically
Netty 4:
- lookup using binary search
+ accesses primitives using smart divide-and-conquer
- tons of classes, do many more things than is necessary (e.g. detects Unsafe and Java version; see PlatformDependent) and increase class loader overhead
- has low coverage
- no putString
- doesn't add ByteBuffers automatically
+ naming: get/set for absolute, read/write for relative
OK - absolute get/set, relative read/write
OK - byte
OK - generic scaffolding for random data; on failure show the seed that was used
OK - short (currently not needed by our listeners)
OK - char
OK - int
OK - long (currently not needed by our listeners)
OK - use Integer.SIZE etc. in absolute tests
OK - TestableRandom: on failure, print all generated values and events (including "reset"), maybe also allow logging custom messages
OK - make it based on multiple ByteBuffers
OK - create AllocatedByteBufferSequence
OK - interface ByteBufferSequence.get(int index):ByteBuffer for dynamically creating the MMFs (mockable in tests)
OK - same index returns same backing data
OK - we can assume sequential access to new indeces; IpcBuffer's position depends on every preceding segment's size
OK - expand buffer automatically
OK - maintain our own position
OK - convert positions to buffer's relative positions
OK - test well traversing forwards and backwards
OK - use lambdas to remove duplication in all get/set methods
OK - escape analysis did not remove the allocations; undo the lambda refactoring
OK - test well the splitting of data between buffers
OK - generic tests with initial buffer size varying from 0 to sizeInBytes
SKIP - consider using parameterized tests for the above
OK - also check that there is no gap in buffers; it should be possible to dumbly concatenate them all and read the same data
OK - create MMF based MappedByteBufferSequence
OK - extract contract test
OK - use MMFs
OK - multiple instances/processes with same path should access same data
OK - doesn't need to cache the mapped buffers, but may re-map if called multiple times with same index
OK - IpcBuffer: avoid looking up adjacent buffers repeatedly (opening a MMF is relatively slow)
OK - new FileSegmenter(initialPath, initialSize, maxSize).pathOf(index) & sizeOf(index)
OK - when MMF already exists, use its file size (i.e. map whole file) instead of whatever FileSegmenter tells by default
OK - retry if opening the file fails at first
OK - see about making fi.jumi.core.ipc.MappedByteBufferSequence#tryMapFile thread safe
OK - hand-written serialization of SuiteListener and CommandListener events (rename to LauncherListener?)
protocol draft:
header: "Jumi" (4 bytes), protocol-version (int), interface (string), interface-version (int), events...
events: status (byte), event-type (byte), event-data
status: empty (0), exists (1), end-of-file (2)
* stuff that depends on interface & interface-version: event-type, event-data
OK - roundtrip tests
OK - string operations (only relative read/write, because strings are variable size)
OK - fully test serializing StackTrace
OK - refactor: create StackTrace.Builder, use it also in StackTrace.copyOf, keep the constructor private
OK - make StackTrace.copyOf aware of StackTrace, to not rewrap it? else it might obstruct getExceptionClass (test: chain two copyOf() calls)
SKIP - test equality using Apache Commons EqualsBuilder; Mockito might already provide a matcher which uses it
OK - test deep structural equality using a custom matcher; the existing tools weren't enough
OK - test: simple case
OK - test: multiple chained causes
OK - test: multiple suppressions
OK - make deepEqualTo compare Collections using iteration
OK - refactor: unify cause and suppressed
OK - test serializing strings
OK - empty string
OK - random string of random length
OK - all non-printable characters, i.e. 00-1F
http://web.itu.edu.tr/~sgunduz/courses/mikroisl/ascii.html
OK - java.lang.Throwable#getMessage may be null
OK - add test for StackTrace serialization
OK - create writeNullableString or change writeString? use length of -1 as marker, or write a single byte marker?
OK - use a fixed-size buffer for the round-trip tests, to avoid PIT timeouting (removing a writeString() causes it to try read huge strings)
OK - spike whether that solved the timeout by mutating manually
OK - rename StubByteBufferSequence to FixedByteBufferSequence
OK - make FixedByteBufferSequence pass the contract
OK - test that accessing unavailable indexes throws IllegalArgumentException with clear message
OK - use FixedByteBufferSequence with one big enough buffer in the serializer's tests
OK - there are still a couple of PIT timeouts in writeStackTrace
OK - use FixedByteBufferSequence for StackTrace tests or add totalCapacityLimit to AllocatedByteBufferSequence
OK - handle all (4) failure conditions when reading data that has wrong header fields
OK - concurrency tests
OK - test: concurrent producer-consumer pair and some 100 events
OK - make header work: write protocol-version after the rest of the header (i.e. same way as status) to avoid race conditions?
OK - make events work: between writing an event and the event's status there must be a happens-before relationship
OK - refactor: separate the protocol and SuiteListener specific stuff
OK - remove duplication of writing the status: take event objects as input (implement MessageSender), so that we can surround them with the status writes
OK - extract deserialization of events to its own method (excluding status reads)
OK - move the event reading and writing to its own class (SuiteListenerEncoding)
OK - String read/write to its own class
OK - extract Encoding interface
OK - accessors for interface name and version
OK - make SuiteListenerEncoding#deserialize non-static
OK - rename source/target IpcBuffers to just buffer, keep it in a instance field
OK - rename SuiteEventSerializer to IpcProtocol
OK - split SuiteEventSerializerTest to match the new classes
OK - rename packages: ipc -> ipc.buffer, serialization -> ipc
OK - refactor: support deserializing one event at a time
OK - create a poll method (no sleeps nor loops) that returns NONE, SOME or END based on whether the stream had events
OK - move the wait loop to a utility method
OK - refactor: create a wait loop timer utility that uses progressively longer sleeps
OK - make sure that all the memory barrers are in place
OK - public APIs for read-only & write-only access
OK - make IpcProtocol implement restricted interfaces IpcReader and IpcWriter
OK - create factory methods IpcChannel.reader/writer(Path)
OK - add a read-only flag to MappedByteBufferSequence (or factory methods readWrite/readOnly), in which case will open/map files read-only
OK - the writer should be the first to touch a new segment, so that the writer would determine its size
OK - concurrency test: writer uses 1 byte segments, reader 2 byte segments; afterwards all segments should be 1 byte (can we do it also for the first segment?)
OK - append a null byte before writing the previous event's status?
SKIP - should read-only MappedByteBufferSequence not be able to create new segments?
OK - refactor fi.jumi.core.util.Resilient#tryRepeatedly to use only IOException, no generic exceptions
SKIP - IpcBuffer: test how it handles it if some segments are of size 0 (e.g. if there was a concurrency race in creating the file)
-> MappedByteBufferSequence will throw an exception
OK - TestId: create a method for getting the path as int[]
OK - make fi.jumi.api.drivers.TestId#getPath public
OK - write tests directly for getPath
OK - refactor SuiteListenerEncoding#writeTestId to use it
!! - communicate suite results and commands over MMFs instead of a socket
OK - tell the other side that where to put the MMFs
OK - DaemonConfiguration: daemon specific --daemon-dir
OK - DirBasedSteward: create a new daemon dir (JUMI_HOME/daemons/*)
OK - ProcessStartingDaemonSummoner: get the daemon dir from Steward, tell it to daemon
OK - remove the setDaemonDir spike code ProcessStartingDaemonSummoner#connectToDaemon
OK - make the daemon poll receive commands from daemon dir
OK - create CommandListenerEncoding
OK - example usage & serializing all events
OK - remove duplication: CommandListenerEncoding & SuiteListenerEncoding
OK - remove duplication: CommandListenerEncodingTest & SuiteListenerEncodingTest
OK - new packages:
ipc.encoding (*Encoding)
ipc.channel/protocol (IpcChannel, IpcProtocol, IpcReader, IpcWriter, PollResult, *WaitStrategy)
ipc.api (CommandListener)
ipc.watcher/monitor/endpoint/coordinator/supervisor or just the top-level ipc package
OK - make the daemon monitor the daemon dir for new command MMFs (daemon-dir/commands/*), create IpcCommandMonitor as replacement for DaemonNetworkEndpoint
OK - util for noticing new files
http://docs.oracle.com/javase/tutorial/essential/io/notification.html
OK - read commands from every new command file (daemon-dir/commands/*/request)
OK - read commands until IPC channel closed (though typically only one command per file)
OK - something to manage the daemon dir, e.g. creating command dirs
OK - write result to a corresponding file (daemon-dir/commands/*/reply)
OK - include the suite results file path in the reply (daemon-dir/suites/*/suite)
OK - write suite results to a MMF
OK - write suite results header before sending the reply
OK - close and forget the command MMFs after use
OK - close the suite results MMF after the suite is finished
OK - start it in Main, alongside DaemonNetworkEndpoint (using a different SuiteFactory instance?)
!! - switch to communicating over MMFs
- rename ProcessStartingDaemonSummoner to NetworkedProcessStartingDaemonSummoner
- rename RemoteSuiteLauncher to NetworkedRemoteSuiteLauncher
- create a new (non-actor) ProcessStartingDaemonSummoner which creates the daemon dir, starts the daemon process, and returns a CommandListener to it
- create a new RemoteSuiteLauncher
- send the runTests command
- handle timeout if onSuiteStarted doesn't happen
- handle timeout if process dies
- create a heartbeat MMF on the launcher side (daemon-dir/heartbeat), includes currentTimeMillis (in binary)
- daemon updates the heartbeat every 100 ms or so; create daemon thread in main
- monitor that heartbeat on the launcher side, give an internal error if the process dies in the middle of a suite
- use the new MMF-based classes instead of the old networked classes
- remove NetworkedRemoteSuiteLauncher
- remove NetworkedProcessStartingDaemonSummoner
- remove DaemonNetworkEndpoint
- remove fi.jumi.core.network and all that depends on it
- enable fi.jumi.test.DaemonProcessTest#classes_showing_up_in_actor_logs_have_custom_toString_methods
- possibly subclass SuiteListenerToEvent (IPC writer?) and give it a custom toString
- after implementing, consider refactoring the following places:
fi.jumi.core.TestRunCoordinator#addSuiteListener
fi.jumi.daemon.DaemonNetworkEndpoint#onConnected
fi.jumi.launcher.remote.RemoteSuiteLauncher#onDisconnected
- DirBasedSteward: remove old daemon dirs after they are no more needed (keep last X dirs, at least Y days)
- refactor: do not convert to a Path at SuiteEventSerializer#readTestFile and then back to String in TestFile#fromPath, but expose a raw String constructor or factory?
- save v1 events to file for testing backward compatibility
- save the current example usage
- save all the variations of exceptions (hand-written stack trace elements) and test them for deep equality
- ensure platform compatibility of MMF communication (detect and degrade gracefully)
- lack of file change events
- DirectoryObserverTest: on OS X 10.9.4 it takes 10 seconds for the WatchService to notice changes; file system format: Mac OS Extended (Journaled, Encrypted)
* no native support for OS X
http://stackoverflow.com/questions/9588737/is-java-7-watchservice-slow-for-anyone-else
FAIL - SensitivityWatchEventModifier.HIGH only lowers the delay to 2 seconds, which is too much
http://stackoverflow.com/a/18362404
- use a faster poller (50-100ms) as a failsafe?
- custom support for Mac?
! https://code.google.com/p/barbarywatchservice/
http://en.wikipedia.org/wiki/FSEvents
https://github.com/zch/NativeFSEvents
http://lists.apple.com/archives/java-dev/2009/May/msg00238.html
http://stackoverflow.com/questions/1515730/is-there-a-command-like-watch-or-inotifywait-on-the-mac
- inability to map files into memory
- on VirtualBox it's not possible to map files that reside on a directory shared from Windows; network drives probably won't support mapping files either
- fall back to using RandomAccessFile/FileChannel and FileLock
- write whole events to file under a file lock
- buffer modifications to internal byte buffers
- flush the buffer regularly but not after every small message
- keep track of which buffers are dirty and need flushing
- send the actor asynchronously a flush event after writing an event, so that it would be flushed only after all currently queued events have been written to the buffer; noop if nothing to flush
- CI: run all tests in restricted environments to tests each of the above mentioned degradations
OK - BuildTest.all_classes_must_be_annotated_with_JSR305_concurrent_annotations does not report all classes that are missing annotations, but only the first one that is found
OK - Eclipse's JUnit integration adds to classpath entries that are not valid file paths on Windows
C:\Users\ORFJackal\Desktop\test\gid5\aid5\target\test-classes;C:\Users\ORFJackal\Desktop\test\gid5\aid5\target\classes;C:\Users\ORFJackal\.m2\repository\junit\junit\4.11\junit-4.11.jar;C:\Users\ORFJackal\.m2\repository\org\hamcrest\hamcrest-core\1.3\hamcrest-core-1.3.jar;C:\Users\ORFJackal\.m2\repository\fi\jumi\jumi-launcher\0.5.390\jumi-launcher-0.5.390.jar;C:\Users\ORFJackal\.m2\repository\fi\jumi\jumi-core\0.5.390\jumi-core-0.5.390.jar;C:\Users\ORFJackal\.m2\repository\fi\jumi\jumi-actors\0.1.196\jumi-actors-0.1.196.jar;C:\Users\ORFJackal\.m2\repository\fi\jumi\jumi-api\0.5.390\jumi-api-0.5.390.jar;/C:/eclipse/configuration/org.eclipse.osgi/bundles/200/1/.cp/;/C:/eclipse/configuration/org.eclipse.osgi/bundles/199/1/.cp/
- detect when running on Windows and a path starts with "/C:/" (regex "/[A-Z]:/"), drop the first "/"
- detect with os.name?
http://stackoverflow.com/questions/2357758/what-does-system-getpropertyos-name-return-in-latest-windows-oss
http://www.mkyong.com/java/how-to-detect-os-in-java-systemgetpropertyosname/
http://www.javaneverdie.com/java/java-os-name-property-values/
- detect by checking the FileSystem's roots?
FileSystems.getDefault().getRootDirectories() - check if first of them starts with "/"
http://docs.oracle.com/javase/7/docs/api/java/nio/file/FileSystems.html#getDefault()
http://docs.oracle.com/javase/7/docs/api/java/nio/file/FileSystem.html#getRootDirectories()
- optimize PIT timings; too many TIMED_OUT mutations, makes the build slow
FAIL - tried to lower timeoutConstant; didn't help on the slow CI server, and only marginal improvement on fast desktop (total build time from 3:30 to 3:00)
- add a sanity check that will detect infinite loops and will throw an exception
- investigate individual TIMED_OUT mutations; in what kind of an infinite loop do they get stuck? can we detect it and break?
- safety threshold to readNullableString and everywhere else where an array length is read?
- write daemon output to log file
- use ProcessBuilder.redirectOutput to write the log file (else the process would hang when the launcher JVM exits)
- write log to JUMI_HOME/daemons/*/console.log
- default JUMI_HOME to ~/.jumi
- update e2e tests to read the output from the log file
- get rid of fi.jumi.launcher.remote.ProcessStartingDaemonSummoner#copyInBackground
- don't create background threads in the launcher automatically (the tests should create them explicitly, if any)
- also remove the hack in fi.jumi.test.ReleasingResourcesTest#launcher_stops_the_threads_it_started
- update ErrorHandlingTest#gives_an_error_if_starting_the_daemon_process_failed
- show the path to the daemon log file in the error message
- consider showing the log file path for all internal errors? or leave it for later, for another mechanism to tell stats about the process?
- end-to-end tests: persistent daemon process
* should drive better design on launcher side
- connect to existing process
- cancel idleTimeout if somebody connects again (unit test only?)
- if daemon dies unexpectedly (System.exit/halt, JVM crash etc.), report it and restart next time
http://vanillajava.blogspot.com/2011/12/test-complete-failure-of-jvm.html
- how to show unfinished test runs in the TextUI? (both JVM crash unfinishing all tests, and buggy testing framework unfinishing some tests)
- if daemon disconnects due to network problem, reconnect and resume where we were left? (should not happen with MMFs?)
- should not exit the JVM (due to idle timeout) while tests are running
- have a second look at whether the launcher-side design improved
- make it reliable
- release
- tidy up release notes
- bump version
- release
- measure speedup, but don't yet publish (wait for class loader caching before doing a full comparison)
- run tests in a loop on a weakly ordered, multi-core CPU, to look for cache coherency issues
- where to find a multi-core ARM for a test drive? and are the newer ARMs anyhow cache coherent?
http://www.calxeda.com/trystack/
http://trystack.org/
RPI 2 is a quad core ARMv7: http://www.theverge.com/2015/2/2/7954617/raspberry-pi-2-announced-on-sale
- are there Itaniums (IA64) on any cloud for rent?
- Azul's Vega should have a relaxed memory model; ask for co-operation, also to ensure scalability?
"Azul hands out 'academic' accounts all the time." https://groups.google.com/d/msg/clojure/XHqWLMcsH-c/ZYeyMP_w66cJ
- JVM/ClassLoader caching: reuse loaded vendor libraries and JVM after previous test runs
- JVM/ClassLoader preloading: load vendor libraries in a new JVM before user runs tests, thrown away the JVM after one test run - should be more reliable than class loader caching
- release
- tidy up release notes
- bump version
- release
- create a performance comparison (e.g. with Dimdwarf's tests, or some publicly known open source project with fast tests; Joda-Time?)
- promote widely
- plugin system for custom DriverFinders (keep it minimal, nearly walking skeleton level, until we have more plugins)
- hook into fi.jumi.core.drivers.DriverFinderFactory
- move JUnit compatibility into its own module
- create jumi-junit.jar
- extract it automatically to ~/.jumi/plugins, the same way as the daemon jar
- tell to daemon as system property what plugins to load: e.g. -Djumi.plugin.path=C:/plugin.jar;C:/another.jar (separated by path.separator)
- load each plugin in a separete plugin classloader, with access to test classloader and to jumi-core
- at first make the test classloader the parent of the plugin classloader (it'll work as long as we don't hide jumi-core from the test classloader, after which the plugin classloader might need two parents or use context classloader)
- refactor JUnitCompatibilityDriverFinder et al to avoid reflection
- refactor tests that use JUnitCompatibilityDriver.class.getName() instead of instanceOf checks (in DriverFinderFactoryTest)
- investigate maven-shade-plugin; can we now turn on minimizing? it used to remove the JUnitCompatibilityDriver from jumi-daemon.jar
- experiment:
- how much is daemon jar minimized? if it minimizes only a little, it might not be worth it
- otherwise: write a test that the daemon jar contains everything from the jumi-core
- ignoring tests
- ignoring without running a test
- ignoring after test has started
- JUnit @Ignore
- update the test in fi.jumi.test.JUnitCompatibilityTest
- implement fi.jumi.core.junit.JUnitRunListenerAdapter#testIgnored (JUnitCompatibilityDriverTest)
- JUnit assumptions
- update the test in fi.jumi.test.JUnitCompatibilityTest
- implement fi.jumi.core.junit.JUnitRunListenerAdapter#testAssumptionFailure (JUnitCompatibilityDriverTest)
- release
- tidy up release notes
- bump version
- release
- promote
- make option 2 explicit by adding a test for it; check for "nihao: ??" and "umlauts: ???" on US-ASCII
> - when default charset is ISO-8859-1 (fi.jumi.test.StandardOutputTest#compensates_for_the_default_charset_of_the_daemon_process), Unicode chars are lost already when they are captured (nihao becomes ??); should we do something about it?
> (1) We could have an internal PrintStream which uses UTF-8/16 and capture losslessly from it, after which it's routed to a PrintStream which uses the actual encoding (might interfere with writing binary data to stdout).
> (2) Or then keep it as is. That would be the way that System.out works normally. Principle of least surprise?
- misc refactoring (choose some, defer others)
- SuiteMother: twoInterleavedRuns seems to be missing onTestFound for the root test
- EventBuilder: nextRunId could use the RunIdSequence
- upgrade to Java 8
- check that most developers have upgraded to Java 8, or wait for Java 7 end-of-life (April 2015, as estimated as of May 2014)
- search the code for "Java 8" TODO comments and upgrade those to use Java 8 APIs
- look for other places that could benefit
- for loops -> Stream
- StringBuilder usage -> StringJoiner
- update documentation of Jumi and dependent projects to reflect the new minimum requirement
- PIT shows incorrect coverage for TestId; stopped working near commit cccff66d4541a0811509a3dadf41084abca278a4, after upgrading to JDK 8
- prepend TestFile as context information to the message in fi.jumi.core.runs.RunEventNormalizer#onInternalError?
- create a custom taglet (as independent project) for cross-references from public API's javadocs to tests: e.g. {@ref com.example.FooTest#someTestName}
http://docs.oracle.com/javase/6/docs/technotes/guides/javadoc/taglet/overview.html
http://docs.oracle.com/javase/6/docs/jdk/api/javadoc/taglet/com/sun/tools/doclets/Taglet.html
http://docs.oracle.com/javase/7/docs/jdk/api/javadoc/doclet/com/sun/javadoc/Tag.html
http://www.oracle.com/technetwork/java/javase/documentation/index-137483.html
http://maven.apache.org/plugins/maven-javadoc-plugin/examples/taglet-configuration.html
- DaemonConfiguration: refactor the code for parsing command line parameters (try to minimize shotgun surgery, e.g. commit 1ea1b44a186ef00b32b2ae0ad05f88387e422fcb)
fi.jumi.core.config.DaemonConfigurationBuilder#parseProgramArgs
fi.jumi.core.config.DaemonConfiguration#toProgramArgs
fi.jumi.core.config.DaemonConfiguration#JUMI_HOME and other constants
- thread-safety-checker: when an inner class is missing the annotation, use the annotation of the enclosing class or default to @NotThreadSafe when the enclosing class has any concurrency annotation
- replace SuiteRunnerIntegrationHelper with TestBench?
- consider implementing SynchronizedPrintStream using something else than CGLIB (ASM? Tapestry's Plastic?) to make the JAR smaller
- check whether we still have many ASM versions we have on classpath (caliper depends on ASM 3), block "asm:*" artifacts with maven-enforcer-plugin?
- make JUnit print a thread dump when a test times out due to @Test(timeout)
- improve JUnit's test timeouts to show a stack trace of the timed out test, to help debugging sporadic timeouts in RunningTestsTest
- evaluate whether some of the concurrent tests could be made simpler by using MultithreadedTC
http://code.google.com/p/multithreadedtc/
- BuildTest: make sure that jumi-daemon.jar contains everything from jumi-api.jar no matter what
- test the test by adding an unused class to jumi-api
- add includes if necessary: http://maven.apache.org/plugins/maven-shade-plugin/examples/includes-excludes.html
- fix Maven's pomPropertiesFile element, get rid of the current workaround of a separate build.properties
http://jira.codehaus.org/browse/MSHARED-154
http://stackoverflow.com/questions/10102818/custom-values-in-maven-pom-properties-file
<archive> element: http://maven.apache.org/plugins/maven-jar-plugin/jar-mojo.html
<pomPropertiesFile> element: http://maven.apache.org/shared/maven-archiver/index.html
- there might be unnecessary eventizers in fi.jumi.daemon.Main and /jumi-core/pom.xml
- create EventizerUsageStatistics (extends EventizerProvider), print usage statistics when program exits
- organize the project so that it's possible to run unit tests separate from end-to-end tests
- put end-to-end tests in their own package, test.fi.jumi?
- or use JUnit categories, maybe with a custom runner?
- or use ClasspathSuite?
- evaluate JarJar or ProGuard, maybe it could minimize the jars better than the shade plugin by removing also unused methods
http://sonatype.github.com/jarjar-maven-plugin/
http://pyx4me.com/pyx4me-maven-plugins/proguard-maven-plugin/
https://github.com/wvengen/proguard-maven-plugin (uses newer version of ProGuard)
1. run shade plugin to embed all dependencies and create the dependency-reduced POM
2. run proguard to minimize the JAR at method level and preverify (in and out files must be different?)
3. replace the main artifact
http://mojo.codehaus.org/build-helper-maven-plugin/attach-artifact-mojo.html
- consider logging with MinLog http://code.google.com/p/minlog/
or create our own clone, since it's just one file (and )
- actors: extract AbstractMessageLogger from PrintStreamMessageLogger
- create logger specific implementation of AbstractMessageLogger
- customize MinLog's output to use microsecond accuracy, similar to PrintStreamMessageLogger
- consider using http://mojo.codehaus.org/flatten-maven-plugin/
- minor release
- tidy up release notes
- release
- promote
- deployment pipeline
jumi:
build
build-release
analyze
coverage-report
jumi-integration:
test
integration-test
jumi-publish:
publish [M]
push-staging
promote-staging
smoke-test
promote-central
announce [M]
update-www
send-emails
- create jumi-integration and add it to the pipeline
- integration tests for Specsy
- platform for running the tests
- run an untrusted go-agent in a restricted environment
- add it to the pipeline
- push-button releasing the staging repository
- run smoke tests before promoting
- if smoke tests pass, promote automatically from OSSRH to Central
- consider migrating to nexus-staging-maven-plugin, https://github.com/sonatype/nexus-maven-plugins/tree/master/nexus-staging-maven-plugin
http://www.sonatype.com/books/nexus-book/reference/staging-sect-managing-plugin.html
https://repository.sonatype.org/content/sites/maven-sites/nexus-maven-plugin/
- automate updating the web site
- update the version number of latest release to the web site
- update javadocs to the web site
- web site improvements
- site for jumi-actors-maven-plugin
http://www.vineetmanohar.com/2009/04/how-to-auto-generate-maven-plugin-documentation/
http://stackoverflow.com/questions/2912665/maven-plugin-site
http://maven.apache.org/guides/mini/guide-site.html
- aggregated javadoc of all modules?
http://maven.apache.org/plugins/maven-javadoc-plugin/aggregate-mojo.html
- improving TextUI
(- if multiple readers are needed: Streamer class for receiving events from UI thread)
- different colors for out (default), err (red), metadata e.g. test names (blue), binary output (yellow BG)?
- JCons (uses JNI on Windows, so maybe support only Unix)
http://jcons.sourceforge.net/
http://sourceforge.net/projects/jcons/
- could also JLine do it? (uses a native binary)
http://jline.sourceforge.net/
https://github.com/jline
- misc links
http://www.codeguru.com/forum/showthread.php?t=504096
http://blogs.atlassian.com/2009/10/colour_your_console_ansicolor/
http://en.wikipedia.org/wiki/ANSI_escape_code
http://www.velocityreviews.com/forums/t143949-text-color-in-windows-console.html
http://academic.evergreen.edu/projects/biophysics/technotes/program/ansi_esc.htm
http://www.tldp.org/LDP/abs/html/colorizing.html
http://stackoverflow.com/questions/7445658/how-to-detect-if-the-console-does-support-ansi-escape-codes-in-python
- ANSI escape codes work in IDEA?
http://youtrack.jetbrains.net/issue/IDEA-68679
- write performance tests
- test: suite with a hundred test classes, each with a hundred tests, some of which also do printing; measure total time of running the suite
- experiment with different queue implementations, for example ConcurrentLinkedQueue
- try following the single writer principle: http://mechanical-sympathy.blogspot.com/2011/09/single-writer-principle.html
- for example: separate queue for each test worker (implies parallelism = locally sequential, except for slow tests?, use a work stealing executor?) and one thread which loops through them and writes to the coordinator actor queue
- "Error creating shaded jar", probably an issue with NTFS's pending delete and the shade plugin
- make a bug request, maybe also a patch
http://mail-archives.apache.org/mod_mbox/maven-dev/200808.mbox/%3C835d522e0808281121g47eedd04u3c056998569be860@mail.gmail.com%3E
- investigate warnings on build:
- happens when jumi-core-0.1.0-SNAPSHOT.jar (or daemon JAR) exists from the previous build
[WARNING] We have a duplicate net/orfjackal/jumi/core/INTERNAL/org/apache/commons/io/input/TailerListener.class in C:\DEVEL\Jumi\jumi\jumi-core\target\jumi-core-0.1.0-SNAPSHOT.jar
[WARNING] We have a duplicate net/orfjackal/jumi/core/INTERNAL/org/apache/commons/io/input/TailerListenerAdapter.class in C:\DEVEL\Jumi\jumi\jumi-core\target\jumi-core-0.1.0-SNAPSHOT.jar
[WARNING] We have a duplicate net/orfjackal/jumi/core/INTERNAL/org/apache/commons/io/input/TeeInputStream.class in C:\DEVEL\Jumi\jumi\jumi-core\target\jumi-core-0.1.0-SNAPSHOT.jar
[WARNING] We have a duplicate net/orfjackal/jumi/core/INTERNAL/org/apache/commons/io/input/XmlStreamReader.class in C:\DEVEL\Jumi\jumi\jumi-core\target\jumi-core-0.1.0-SNAPSHOT.jar
- Pronouncing Jumi: ['jumi]
http://en.wikipedia.org/wiki/International_Phonetic_Alphabet