[tests] Intermittent NoSuchElementException

[eclipse-ocl-bot]

| --- | --- |
| Bugzilla Link | 417663 |
| Status | RESOLVED FIXED |
| Importance | P3 normal |
| Reported | Sep 20, 2013 02:23 EDT |
| Modified | May 28, 2014 02:17 EDT |
| Reporter | Ed Willink |

Description

Since adding the AS2XMIidVisitors Hudson is failing about 25% of the time with a NSEE. Interactively it also happens but perhaps 5% of the time. (Perhaps the same rate, just Hudson runs multiple test suites.)

Build #890

java.util.NoSuchElementException
at org.eclipse.emf.common.util.AbstractEList$EIterator.doNext(AbstractEList.java:713)
at org.eclipse.emf.common.util.AbstractEList$EIterator.next(AbstractEList.java:692)
at org.eclipse.emf.ecore.util.EContentsEList$FeatureIteratorImpl.hasNext(EContentsEList.java:480)
at org.eclipse.emf.ecore.util.EContentsEList$FeatureIteratorImpl.next(EContentsEList.java:559)
at org.eclipse.emf.common.util.AbstractTreeIterator.next(AbstractTreeIterator.java:133)
at org.eclipse.ocl.examples.pivot.utilities.AS2XMIid.assignIds(AS2XMIid.java:91)
at org.eclipse.ocl.examples.pivot.utilities.AS2XMIid.assignIds(AS2XMIid.java:127)
at org.eclipse.ocl.examples.pivot.utilities.PivotSaveImpl.init(PivotSaveImpl.java:48)
at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.save(XMLSaveImpl.java:244)
at org.eclipse.emf.ecore.xmi.impl.XMLResourceImpl.doSave(XMLResourceImpl.java:365)
at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(ResourceImpl.java:1430)
at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(ResourceImpl.java:999)
at org.eclipse.ocl.examples.test.xtext.RoundTripTests.doRoundTripFromEcore(RoundTripTests.java:196)
at org.eclipse.ocl.examples.test.xtext.RoundTripTests.doRoundTripFromEcore(RoundTripTests.java:175)
at org.eclipse.ocl.examples.test.xtext.RoundTripTests.doRoundTripFromEcore(RoundTripTests.java:170)
at org.eclipse.ocl.examples.test.xtext.RoundTripTests.testXMLNamespaceRoundTrip(RoundTripTests.java:545)

Build #887

java.util.NoSuchElementException
at org.eclipse.emf.common.util.AbstractEList$EIterator.doNext(AbstractEList.java:713)
at org.eclipse.emf.common.util.AbstractEList$EIterator.next(AbstractEList.java:692)
at org.eclipse.emf.ecore.util.EContentsEList$FeatureIteratorImpl.hasNext(EContentsEList.java:480)
at org.eclipse.emf.ecore.util.EContentsEList$FeatureIteratorImpl.next(EContentsEList.java:559)
at org.eclipse.emf.common.util.AbstractTreeIterator.next(AbstractTreeIterator.java:133)
at org.eclipse.ocl.examples.pivot.utilities.AS2XMIid.assignIds(AS2XMIid.java:91)
at org.eclipse.ocl.examples.pivot.utilities.AS2XMIid.assignIds(AS2XMIid.java:127)
at org.eclipse.ocl.examples.pivot.utilities.PivotSaveImpl.init(PivotSaveImpl.java:48)
at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.save(XMLSaveImpl.java:244)
at org.eclipse.emf.ecore.xmi.impl.XMLResourceImpl.doSave(XMLResourceImpl.java:365)
at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(ResourceImpl.java:1430)
at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(ResourceImpl.java:999)
at org.eclipse.ocl.examples.test.xtext.RoundTripTests.doRoundTripFromEcore(RoundTripTests.java:196)
at org.eclipse.ocl.examples.test.xtext.RoundTripTests.doRoundTripFromEcore(RoundTripTests.java:175)
at org.eclipse.ocl.examples.test.xtext.RoundTripTests.doRoundTripFromEcore(RoundTripTests.java:170)
at org.eclipse.ocl.examples.test.xtext.RoundTripTests.testOCLCSTRoundTrip(RoundTripTests.java:506)

Same problem, but in a different test, and the AS2XMIid.assignIds TreeIteration has already been simplified to eliminate any EObject modification (setID) during the loop.

[eclipse-ocl-bot]

By Ed Willink on Sep 20, 2013 07:04

commit 0d82f2f

Instrumentation added creating a list of WeakReference on a pre-pass that can be compared in a post-pass to detect any domain corruption, and which can identify the previous object if an NSEE occurs.

However in 7 Hudson branch tests runs we get 5 successes and two stale NFS failures. No sign of the intermittency.

The instrumentation adds perhaps 5-10% to the *.oclas save time, so let's push it master and wait for the intermittency to reoccur during the normal course of events.

[eclipse-ocl-bot]

By Ed Willink on Sep 22, 2013 09:21

Problem has occurred under instrumentation but with no useful debug output since the problem occurs in the first tree-iteration that is very close to pure readonly.

java.util.NoSuchElementException
at org.eclipse.emf.common.util.AbstractEList$EIterator.doNext(AbstractEList.java:713)
at org.eclipse.emf.common.util.AbstractEList$EIterator.next(AbstractEList.java:692)
at org.eclipse.emf.ecore.util.EContentsEList$FeatureIteratorImpl.hasNext(EContentsEList.java:480)
at org.eclipse.emf.ecore.util.EContentsEList$FeatureIteratorImpl.next(EContentsEList.java:559)
at org.eclipse.emf.common.util.AbstractTreeIterator.next(AbstractTreeIterator.java:133)
at org.eclipse.ocl.examples.pivot.utilities.AS2XMIid.assignIds(AS2XMIid.java:93)
at org.eclipse.ocl.examples.pivot.utilities.AS2XMIid.assignIds(AS2XMIid.java:197)
at org.eclipse.ocl.examples.pivot.utilities.PivotSaveImpl.init(PivotSaveImpl.java:48)
at org.eclipse.emf.ecore.xmi.impl.XMLSaveImpl.save(XMLSaveImpl.java:244)

AS2XMIid.java:93 is tit.next() in the following loop.

List<WeakReference> debugOldEObjects = new ArrayList<WeakReference>();
for (TreeIterator tit = asResource.getAllContents(); tit.hasNext(); ) {
EObject eObject = tit.next();
assert eObject != null;
debugOldEObjects.add(new WeakReference(eObject));
}

This would appear to be possible only if something is creating/removing an EObject during the traversal.

[eclipse-ocl-bot]

By Ed Willink on Oct 11, 2013 13:08

With extra instrumentation, there is a concurrent thread called "EMF Reference Cleaner", but instrumentation fails because the domain grows at the end and gives an IOOBE in the instrumentation. Imprive it.

[eclipse-ocl-bot]

By Ed Willink on Oct 15, 2013 10:38

Progress: instrumentation reveals that the domain change involved addition of a SetTypeImpl at the end of the domain.

This would be consistent with a new specialization added to the orphanage, perhaps the consequence of a concurrent validation thread.

So instrument CollectionTypeImpl to log the thread on which creation occurs concurrently to XMI id assignment.

[eclipse-ocl-bot]

By Ed Willink on Oct 17, 2013 02:27

(In reply to Ed Willink from comment #4)

So instrument CollectionTypeImpl to log the thread on which creation occurs
concurrently to XMI id assignment.

Once again a SequenceTypeImpl has been added to the Resource, but the assert bomb in CollectionTypeImpl did not go off. So presumably the specialization had already been created but was only added concurrently.

Odd. Confirms that the spurious creation was not in the current thread, but leaves limited opportunity for other threads to mess up either.

Improve instrumentation to diagnose all rather than first extra object.

Add Bomb to CollectionTypeImpl.eBasicSetContainer in the hope that it goes off.

[eclipse-ocl-bot]

By Ed Willink on Oct 17, 2013 10:01

With instrumentation of all new objects, on Hudson we got:

210 New object : SetTypeImpl@44b8c770
210 New object : TemplateBindingImpl@624075c
211 New object : TemplateParameterSubstitutionImpl@275f2c2a

(ignoring an instrumentation bug: 210,211,212)

three new object arose corresponding to one template specialisation.

But no bomb on constuction or setContainer. Not possible. -ea must be missing. But -ea is in the launch config that Hudson re-uses.

[eclipse-ocl-bot]

By Ed Willink on Nov 11, 2013 16:09

Happened on the new HIPP Hudson.

Once again three new objects:

724 New object : SetTypeImpl@7b23858e
724 New object : TemplateBindingImpl@7b36290a
725 New object : TemplateParameterSubstitutionImpl@20c08eb8

clearly created during AS2XMIid.assignIds despite assert bombs in CollectionTypeImpl constructor and CollectionTypeImpl.eBasicSetContainer

[eclipse-ocl-bot]

By Ed Willink on Dec 20, 2013 11:30

Just happened again; this time on a QVTd build and this time the addition is more interesting. An UnspecifiedType. Perhaps an already existing type is being added and so bypassing the earlier bombs.

MetaModelManager.addOrphanClass invokes setPackage as a direct set containment. So add a bomb there too.

[eclipse-ocl-bot]

By Ed Willink on Jan 06, 2014 11:35

Happened four times in a row for a QVTd build on Hudson, but not reproducible locally.

Closer examination reveals that Orphange is using a WeakEList to hide a WeakHashMap and this is maintained by setPackage.

WeakEList is not synchronized; shouldn't be a problem.

However if a WeakEList has a list for a map and an entry is garbage collected, the list has a stale length; definitely could be a problem.

Add synchronization and nulling of cache at every change.

Nastier. The iteration support has size() accessing the map which the iterator accesses the list, so if any garbage collection occurs map and list differ; the list is longer, the NSEE occurs and the bombs do not fire.

Looks like an explanation.

Provide a custom iterator that caches the list.

Implementing isEmpty directly to the map, avoids the minor efficiency of size() creating a list and then iterator() creating another. If the worst comes to the worst the isEmpty returns false, garbage collection empties the list and the iteration iterates over the empty list.

Initial appearance of this bug also makes sense since it is assignIds that does the late iteration over a list that might have changed.

Hopefully fixed by commit f74cc33

Pushed to master for M5.

Leaving open in case intermittency persists.

[eclipse-ocl-bot]

By Ed Willink on May 28, 2014 02:17

(In reply to Ed Willink from comment #9)

Pushed to master for M5.

Leaving open in case intermittency persists.

Not seen in M6 M7 RC1 RC2