Regression Testing

As the number of ways this plugin can be used grows, it is important to devise a way of doing better regression testing. This is where most of my effort has gone (when I can spend time on this project). The problem is that I can't simply alter the way any particular component works anymore. There are users that require certain behaviors and may not appreciate having a favorite feature or default setting suddenly change on them. There has also been several occasions where we've found that certain database drivers interact with the Hibernate + GORM layer in just a peculiar way that causes problems.

Most users of this plugin are using MySQL, Oracle, or PostgreSQL databases. These three are tested in a final round of testing where I build a custom project donated to me by an Audit Logging user. This project features most of the common problems that folks have encountered. The problem here is I've never automated this process. I have to build the plugin, install it, then run the child-project's tests. This is bad. Why?

Primarily, it's bad because the feedback loop for changes is really really long and cumbersome. The longer the feedback loop, the more likely you are to lose the thread of the narrative you are working on. So the more likely it is that in a project where I get to work on it an hour or two at a time once or twice a month ... it means I'm completely going to lose the thread of... what was I talking about again?

So to alleviate this problem I've been studying the grails-core project and other projects to see how to better test this plugin. The answer seems to be having the plugin testing harness generate a mock project. As I work in this space I plan on breaking out this work in a way that other plugin projects can use. A kind of meta-plugin testing framework if you will... some of that code is already here on github now.

I will build up a full testing system on the donated hudson server from our friends at CloudBees and I'll keep you posted here. As always, I welcome help, direction, constructive criticism, and general commentary. If you are using this project and it is helpful to you please make sure to make it known.

So far I hear about how people use this plugin when I'm at conferences and I have a filter that is supposed to tell me anytime someone posts code with "static auditable = " in it to let me survey how much use the plugin is silently getting. I'm flattered that so many people find my work useful and this keeps me motivated to work more on the project. Thank you for your patience and thank you for your time.