Support programmatic starting and stopping of transactions in the TestContext framework [SPR-5079]

[spring-projects-issues]

Den Orlov opened SPR-5079 and commented

Background

I have integration tests that:

1. put some test data into database using Hibernate
2. wait some time, so DB stored procedures triggered by scheduler will process my data
3. retrieve from db some data and check its correctness, again using Hibernate

When I used Spring's "JUnit 3.8 legacy support" all of these three steps were organized in one test method using the protected endTransaction() and startNewTransaction() methods in AbstractTransactionalSpringContextTests between steps #1 and #2.

---

Status Quo

Now I am migrating my code to use the Spring TestContext Framework (TCF), but it doesn't provide support for programmatically starting or stopping the test-managed transaction.

---

Deliverables

The term test-manged means a transaction managed by the TCF, either declaratively or programmatically.

See also "Design Considerations" below.

1. Introduce a mechanism in the TCF that supports the following features:
   1. end the current test-managed transaction
   2. start a new test-managed transaction
   3. configure the current test-managed transaction to be committed once the test completes
   4. configure the current test-managed transaction to be rolled back once the test completes
2. Refactor TransactionalTestExecutionListener to use this new mechanism internally.
3. Consider introducing a JUnit TestRule that simplifies the programming model (e.g., by delegating to the proposed TestTransaction façade).

---

Design Considerations

Introduce a TestTransaction class that acts as a façade to the functionality previously available in TransactionalTestExecutionListener. For example, TestTransaction could define methods that could provide the following API:

• TestTransaction.start()
• TestTransaction.end()
• TestTransaction.rollback(boolean)

Options for Interacting with the Test Transaction

Dependency Injection of TestTransaction

Ideally, we would like to be able to have the TestTransaction injected into our test instance. However, since the DependencyInjectionTestExecutionListener must come before the TransactionalTestExecutionListener in the chain of listeners, dependency injection would only be possible by introducing yet another "transactional" TestExecutionListener that creates a proxy bean or bean of type ObjectFactory in the ApplicationContext. Such a proxy bean would serve as a placeholder for dependency injection into the test instance, and the TransactionalTestExecutionListener could later set a value either directly in the proxy/ObjectFactory or via a ThreadLocal.

But we would like to avoid having two (2) TestExecutionListener implementations for transactional support. Plus, the placeholder-bean approach could present more problems than it solves.

Dependency Injection of TestContext

A second option would be to inject the TestContext into test instances (see #12947) and provide access to the TestTransaction as an attribute (note that TestContext implements AttributeAccessor), but this would open up use of the TestContext within test classes (i.e., no longer limited to the TestExecutionListener API). In addition, developers would be forced to navigate the TestContext to obtain the TestTransaction, thus making the programming model less intuitive.

Purely ThreadLocal Approach

The final option is to follow a purely ThreadLocal-based approach with TestTransaction encapsulating the details and providing static methods instead of instance methods.

---

Related Resources

• Diff for TransactionalTestExecutionListener. This commit introduced caching of the TransactionContext keyed by Method and removed the previous support for the transactionFlaggedForCommit functionality.
  • TransactionalTestExecutionListener uses cached TransactionManager (TM), even after ApplicationContext that created the TM has been marked 'dirty' and closed. [SPR-4638] #9315
  • Transactional JUnit test with timeout does not create a transaction [SPR-4541] #9218

---

Affects: 2.5.6

Issue Links:

• Document programmatic test transaction management support [SPR-11941] #16558 Document programmatic test transaction management support ("is depended on by")
• Provide support in TestContext framework equivalent to AbstractTransactionalSpringContextTests [SPR-5879] #10548 Provide support in TestContext framework equivalent to AbstractTransactionalSpringContextTests ("is duplicated by")
• TransactionalTestExecutionListener uses cached TransactionManager (TM), even after ApplicationContext that created the TM has been marked 'dirty' and closed. [SPR-4638] #9315 TransactionalTestExecutionListener uses cached TransactionManager (TM), even after ApplicationContext that created the TM has been marked 'dirty' and closed.
• Introduce autowiring support for the current TestContext [SPR-8299] #12947 Introduce autowiring support for the current TestContext
• Transactional JUnit test with timeout does not create a transaction [SPR-4541] #9218 Transactional JUnit test with timeout does not create a transaction

Referenced from: commits bdceaa4, f667e43, 90f0d14

30 votes, 22 watchers

[spring-projects-issues]

Jorg Heymans commented

see also http://forum.springsource.org/showthread.php?t=61949

please add setComplete() and endTransaction() back in the test context framework, they are incredibly useful for example to test correct lazy loading behaviour of DTO's

[spring-projects-issues]

Edwin Dhondt commented

Is there any means to get this in a 2.5.x release so we don't have to do a major upgrade ?
Thanks,
EDH

[spring-projects-issues]

Sam Brannen commented

Hi Edwin,

No, this functionality will not be backported to 2.5.x. Current development is only performed on the 3.x branch.

For 2.5.x your only option would be to implement your own custom TransactionalTestExecutionListener which makes programmatic access to transaction management available to your test instance.

Regards,

Sam

[spring-projects-issues]

Edwin Dhondt commented

Sam,

In which 3.x milestone build will this support be exactly available or is it already available ?

Back to Spring 2.5.6.
How would I go about testing the following with Spring 2.5.6 and the "new" Test Context framework ?

• Run testmethod:
• Before executing the testmethod: load database.
• Execute testmethod within a txn: inserting and updating db rows. with changes committed when testmethod returns.
• ???: Verifying whether the inserts/updates have been correctly persisted to the db, by executing a query against the db (in a different txn that the one in which the testmethod ran) ?
• After executing the testmethod: empty the database deleting all rows.

Please advice.

Thanks,
EDH

[spring-projects-issues]

Sam Brannen commented

Edwin,

How would I go about testing the following with Spring 2.5.6 and the "new" Test Context framework ?

• Run testmethod:
• Before executing the testmethod: load database.
• Execute testmethod within a txn: inserting and updating db rows. with changes committed when testmethod returns.
• ???: Verifying whether the inserts/updates have been correctly persisted to the db, by executing a query against the db (in a different txn that the one in which the testmethod ran) ?
• After executing the testmethod: empty the database deleting all rows.

Based on your described use case, I assume that you do not care about corrupting (i.e., permanently changing) the state of the database. So if my assumption is correct, you can...

• load the DB in an @BeforeTransaction method
• annotate your test class or test method with @Transactional
• annotate your test method with @Rollback(false)
• verify committed state in an @AfterTransaction method
• in the same @AfterTransaction method, empty the DB

Hope this helps,

Sam

[spring-projects-issues]

Jim commented

There is a whole category of tests that require programmatic control of transactions (potentially even multiple transactions).
These are, of course, more integration tests than unit tests, but they are no less vital for that.
In my situation there is no risk of damaging the database because the tests will be running against an in-memory Derby database, not the real MySQL database.
The fact that JPA queries ignore the local cache makes programmatic transactions even more necessary (so you can persist an object and then find it).

Sam's solution is very inconvenient because it requires a separate class for each test case, given that the AfterTransaction is class-wide.

[spring-projects-issues]

Sam Brannen commented

Jim,

Sam's solution is very inconvenient because it requires a separate class for each test case, given that the AfterTransaction is class-wide.

The steps I outlined are not intended to be a solution per se but rather a work-around until we have added the desired programmatic support to the framework.

Regards,

Sam

[spring-projects-issues]

Jim commented

Understood, no offence intended.
I've written by own replacement for the transactional listener (starting by copying the Spring one) that talks to the test base class, gives it access to the listener and the test context and thus provides a stopStartTransaction method (simply by copying the code from beforeTestMethod and afterTestMethod (without calling the before transaction and after transaction methods).
The code is horribly tied together and rather messy but it works for providing the thing I need most (commit the current transaction and start a new one - I also had to add in a "setCommit" method to not commit).
If you want I'm happy to make it available, but it's probably only useful for giving you a laugh at how limited it is.
I'd be interested in knowing whether you think a general solution should call after transaction methods if a transaction is ended early.

[spring-projects-issues]

Julian commented

For those coming to this ticket and interested in a workaround give my code below a try. It is less than ideal since it uses reflection and unlocks private methods, but it works. Pay attention to the semantics of calling these methods as documented in the my javadoc comments. This can be placed in your test class or a parent class. I have TestNG annotations on the start and end transaction methods, but these can be easily replaced with JUnit @Ignore.

    private static <T> T _GetField(Object target, String fieldName){

		Assert.notNull(target, "Target object must not be null");
		final Field field = ReflectionUtils.findField(target.getClass(), fieldName);
		if (field == null) {
			throw new IllegalArgumentException("Could not find field [" + fieldName + "] on target [" + target + "]");
		}
		ReflectionUtils.makeAccessible(field);
        return (T)ReflectionUtils.getField(field, target);
    }

    private TransactionalTestExecutionListener _GetTransactionalTestExecutionListener(){
		final TestContextManager testContextManager = _GetField(this, "testContextManager");
        for(TestExecutionListener tel: testContextManager.getTestExecutionListeners()){
            if(tel instanceof TransactionalTestExecutionListener){
                return (TransactionalTestExecutionListener)tel;
            }
        };

        throw new RuntimeException("TransactionalTestExecutionListener not found!");
    }

    private TestContext _GetTestContext(){
        final TestContextManager testContextManager = _GetField(this, "testContextManager");
        final TestContext testContext = _GetField(testContextManager,"testContext");
        return testContext;
    }

    /**
     * This is a helper method to allow us to programmatically
     * stop a running transaction.  Calling this method will
     * also trigger any methods marked as @AfterTransaction to fire.
     *
     * @see org.springframework.test.context.transaction.TransactionalTestExecutionListener#afterTestMethod(TestContext)
     */
    @Test(enabled = false)
    public void endTransaction() {
        final TransactionalTestExecutionListener tel = _GetTransactionalTestExecutionListener();
        try {
            tel.afterTestMethod(_GetTestContext());
        } catch (Exception e) {
            e.printStackTrace();
            throw new RuntimeException(e);
        }
    }

    /**
     * This is a helper method to allow us to programmatically
     * start a running transaction.  Calling this method will
     * also trigger any methods marked as @BeforeTransaction to fire.
     *
     * @see org.springframework.test.context.transaction.TransactionalTestExecutionListener#beforeTestMethod(TestContext)
     */
    @Test(enabled = false)
    public void startTransaction() {
        final TransactionalTestExecutionListener tel = _GetTransactionalTestExecutionListener();
        try{
            tel.beforeTestMethod(_GetTestContext());
        } catch (Exception e) {
            e.printStackTrace();
            throw new RuntimeException(e);
        }
    }

[spring-projects-issues]

Julian commented

One update, the e.printStackTrace calls aren't needed and can be removed from my workaround.

[spring-projects-issues]

daniel carter commented

In my case i setup some test data for an integration test. Each piece of data represents an event, and the code under test starts a new transaction to process each event. As the test data is uncommited, the code under test deadlocks when it tries to mark the event as processed.

My 'workaround' for this issue was to simply call commit on the active connection after creating the test data.

DataSourceUtils.getConnection(appContext.getBean(DataSource.class)).commit();

[spring-projects-issues]

Val Blant commented

Hmm. 6 years since this problem was reported, and still no official solution?

In any case, here's how to get the endTransaction() and startNewTransaction() methods back.

This solution is based on Julian's post.

 
/**
 * There is a regression in Spring 4, which removes the APIs for manual transaction control
 * in tests. So calls like endTransaction() and startNewTransaction() are no longer available.
 * 
 * Since this functionality is still required by some of our integration tests, this custom
 * Junit 4 Runner is necessary to store a reference to the current 
 * <code>org.springframework.test.context.TestContextManager</code>. This gives us the ability 
 * to manipulate transactions directly, thus restoring the lost functionality.
 * 
 * https://jira.springsource.org/browse/SPR-5079
 * 
 * @author Val Blant
 */
public class SpringTestContextManagerAwareClassRunner extends SpringJUnit4ClassRunner {

	public SpringTestContextManagerAwareClassRunner(Class<?> clazz) throws InitializationError {
		super(clazz);
	}
	
	@Override
	protected Object createTest() throws Exception {
		Object testInstance = super.createTest();
		if ( testInstance instanceof AbstractTestCase ) {
			((AbstractTestCase)testInstance).setTestContextManager( getTestContextManager() );
		}
		
		return testInstance;
	}
	

}

@RunWith(SpringTestContextManagerAwareClassRunner.class)
public abstract class AbstractTestCase extends AbstractTransactionalJUnit4SpringContextTests {
	private TestContextManager testContextManager;
	public void setTestContextManager(TestContextManager testContextManager) {
		this.testContextManager = testContextManager;
	}

	/**
	 * This is a helper method to allow us to programmatically stop a running
	 * transaction. Calling this method will also cause any methods marked as @AfterTransaction
	 * to fire.
	 * 
	 * @see org.springframework.test.context.transaction.TransactionalTestExecutionListener#afterTestMethod(TestContext)
	 */
	protected void endTransaction() {
		final TransactionalTestExecutionListener tel = _GetTransactionalTestExecutionListener();
		try {
			tel.afterTestMethod(_GetTestContext());
		} catch (Exception e) {
			e.printStackTrace();
			throw new RuntimeException(e);
		}
	}

	/**
	 * This is a helper method to allow us to programmatically start a running
	 * transaction. Calling this method will also cause any methods marked as @BeforeTransaction
	 * to fire.
	 * 
	 * @see org.springframework.test.context.transaction.TransactionalTestExecutionListener#beforeTestMethod(TestContext)
	 */
	protected void startNewTransaction() {
		final TransactionalTestExecutionListener tel = _GetTransactionalTestExecutionListener();
		try {
			tel.beforeTestMethod(_GetTestContext());
		} catch (Exception e) {
			e.printStackTrace();
			throw new RuntimeException(e);
		}
	}
	
	@SuppressWarnings("unchecked")
	private static <T> T _GetField(Object target, String fieldName) {
		assertNotNull(target);
		final Field field = ReflectionUtils.findField(target.getClass(), fieldName);
		if (field == null) {
			throw new IllegalArgumentException("Could not find field [" + fieldName + "] on target [" + target + "]");
		}
		ReflectionUtils.makeAccessible(field);
		return (T) ReflectionUtils.getField(field, target);
	}

	private TransactionalTestExecutionListener _GetTransactionalTestExecutionListener() {
		for (TestExecutionListener tel : testContextManager.getTestExecutionListeners()) {
			if (tel instanceof TransactionalTestExecutionListener) {
				return (TransactionalTestExecutionListener) tel;
			}
		}

		throw new RuntimeException("TransactionalTestExecutionListener not found!");
	}

	private TestContext _GetTestContext() {
		final TestContext testContext = _GetField(testContextManager, "testContext");
		return testContext;
	}

}

[spring-projects-issues]

Dennis Kieselhorst commented

Thanks for the sample. As I'm using @BeforeTransaction and @AfterTransaction these methods get also called each time I do endTransaction/ startNewTransaction. I think I need something like @BeforeFirstTransaction and @AfterLastTransaction. Hopefully there will be a solution in 4.1.

[spring-projects-issues]

Sam Brannen commented

FYI: the current work on this feature can be seen on my #9753 branch on GitHub.

Feedback is welcome (and encouraged)! :)

Cheers,

Sam

[spring-projects-issues]

Sam Brannen commented

Completed as described in the comments for GitHub commit f667e43:

Introduce programmatic tx mgmt in the TCF

Historically, Spring's JUnit 3.8 TestCase class hierarchy supported
programmatic transaction management of "test-managed transactions" via
the protected endTransaction() and startNewTransaction() methods in
AbstractTransactionalSpringContextTests.

The Spring TestContext Framework (TCF) was introduced in Spring 2.5 to
supersede the legacy JUnit 3.8 support classes; however, prior to this
commit the TCF has not provided support for programmatically starting
or stopping the test-managed transaction.

This commit introduces a TestTransaction class in the TCF that provides
static utility methods for programmatically interacting with
test-managed transactions. Specifically, the following features are
supported by TestTransaction and its collaborators.

• End the current test-managed transaction.

• Start a new test-managed transaction, using the default rollback
  semantics configured via @TransactionConfiguration and @Rollback.

• Flag the current test-managed transaction to be committed.

• Flag the current test-managed transaction to be rolled back.

Implementation Details:

• TransactionContext is now a top-level, package private class.

• The existing test transaction management logic has been extracted
  from TransactionalTestExecutionListener into TransactionContext.

• The current TransactionContext is stored in a
  NamedInheritableThreadLocal that is managed by
  TransactionContextHolder.

• TestTransaction defines the end-user API, interacting with the
  TransactionContextHolder behind the scenes.

• TransactionalTestExecutionListener now delegates to
  TransactionContext completely for starting and ending transactions.

[spring-projects-issues]

Dennis Kieselhorst commented

Just tried it out, this solution is easy to use and works fine for me. Thanks!

[spring-projects-issues]

Sam Brannen commented

Dennis Kieselhorst, I'm very glad to hear that. :)

Thanks for trying it out and reporting back!