Error: Failed to initialize Spring Context

[padawahn]

Expected Behavior

It is expected that the plugins install fine without error, or at least known errors.

Current Behavior

Error when installing webhook rest plugin.

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'webHookTeamCityRestApiZipPluginFixer' defined in Byte array resource [plugin: tcWebHooks-rest-api#tcwebhooks-rest-api-1.1-alpha13.203.278-jar-with-dependencies.jar!/META-INF/build-server-plugin-RESTListener.xml]: Invocation of init method failed; nested exception is java.lang.IllegalArgumentException: Illegal character in opaque part at index 11: jar:file:d:\TeamCity\bin\..\webapps\ROOT\WEB-INF\plugins\rest-api.zip at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1628) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543) at jetbrains.buildServer.plugins.spring.SpringPluginLoader.pluginClassesLoaded(SpringPluginLoader.java:102) at sun.reflect.GeneratedMethodAccessor70.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at jetbrains.buildServer.util.EventDispatcher$3.run(EventDispatcher.java:126) at jetbrains.buildServer.util.NamedThreadFactory.executeWithNewThreadName(NamedThreadFactory.java:71) at jetbrains.buildServer.util.EventDispatcher.dispatch(EventDispatcher.java:120) at jetbrains.buildServer.util.EventDispatcher$2.invoke(EventDispatcher.java:70) at com.sun.proxy.$Proxy25.pluginClassesLoaded(Unknown Source) at jetbrains.buildServer.plugins.PluginManagerImpl$2.visitPlugin(PluginManagerImpl.java:140) at jetbrains.buildServer.plugins.PluginsCollection.foreachLoadedPluginsAsync(PluginsCollection.java:277) at jetbrains.buildServer.plugins.PluginManagerImpl.firePluginClassesLoaded(PluginManagerImpl.java:138) at jetbrains.buildServer.plugins.PluginManagerImpl.loadPlugins(PluginManagerImpl.java:85) at jetbrains.buildServer.web.plugins.PluginManagerConfigurator.initializePlugins(PluginManagerConfigurator.java:15) at jetbrains.buildServer.web.impl.BuildServerConfigurator.loadConfiguration(BuildServerConfigurator.java:56) at java.util.concurrent.CopyOnWriteArrayList.forEach(CopyOnWriteArrayList.java:891) at jetbrains.buildServer.serverSide.impl.BuildServerLifecycleProcessor.doStartup(BuildServerLifecycleProcessor.java:13) at jetbrains.buildServer.maintenance.TeamCityDispatcherServlet$WebApplicationCreatorAndDestroyer.createApplication(TeamCityDispatcherServlet.java:22) at jetbrains.buildServer.maintenance.StartupProcessor.doApplicationStarting(StartupProcessor.java:736) at jetbrains.buildServer.maintenance.StartupProcessor.access$700(StartupProcessor.java:499) at jetbrains.buildServer.maintenance.StartupProcessor$3.call(StartupProcessor.java:14) at jetbrains.buildServer.util.NamedThreadFactory.executeWithNewThreadName(NamedThreadFactory.java:86) at jetbrains.buildServer.maintenance.StartupProcessor.processConcreteStage(StartupProcessor.java:806) at jetbrains.buildServer.maintenance.StartupProcessor.processConcreteStageSafe(StartupProcessor.java:311) at jetbrains.buildServer.maintenance.StartupProcessor.processTeamCityLifecycle(StartupProcessor.java:11) at jetbrains.buildServer.maintenance.StartupProcessor.access$000(StartupProcessor.java:93) at jetbrains.buildServer.maintenance.StartupProcessor$1.run(StartupProcessor.java:2) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.IllegalArgumentException: Illegal character in opaque part at index 11: jar:file:d:\TeamCity\bin\..\webapps\ROOT\WEB-INF\plugins\rest-api.zip at java.net.URI.create(URI.java:852) at webhook.teamcity.server.WebHookTeamCityRestApiZipPluginFixer.doesRestApiZipFileContainJaxJars(WebHookTeamCityRestApiZipPluginFixer.java:197) at webhook.teamcity.server.WebHookTeamCityRestApiZipPluginFixer.findRestApiZipPlugins(WebHookTeamCityRestApiZipPluginFixer.java:71) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1758) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1695) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1624) ... 37 more Caused by: java.net.URISyntaxException: Illegal character in opaque part at index 11: jar:file:d:\TeamCity\bin\..\webapps\ROOT\WEB-INF\plugins\rest-api.zip at java.net.URI$Parser.fail(URI.java:2848) at java.net.URI$Parser.checkChars(URI.java:3021) at java.net.URI$Parser.parse(URI.java:3058) at java.net.URI.<init>(URI.java:588) at java.net.URI.create(URI.java:850) ... 46 more

Steps to Reproduce (for bugs)

1. Fresh Windows Install of Teamcity 2017.2.1 (build 50732)
2. Upload Plugins tcWebHooks & Rest Plugin zips
3. Restart Teamcity Server
4. Notice Error for the Rest Plugin

Your Environment

• tcWebHooks Version: 1.1-alpha13.203.278
• TeamCity Version: 2017.2.1 (build 50732)
• TeamCity server Operating System: Windows 10
• Are you using a WebHook Template?: no

Example Configuration (xml)

[netwolfuk]

Hi @padawahn
This is the same error as #84 fixes. There is a build here for testing (just log in as guest), I just haven't had a chance to release it yet. Could you please try this and let me know if it resolves the issue. I don't have a windows machine to test with.

[netwolfuk]

FYI. There are three things to test:

1. Does the plugin load correctly
2. Does fixing the jar work correctly.
3. Does the plugin load correctly after restarting TeamCity.

The second item is achieved by clicking the "fix" link in the TeamCity administration section on the Webhooks Rest API tab (bottom left).

[padawahn]

@netwolfuk
the new version installs fine, but does not let me fix the jar files.
I thought it might be a read/write flag issue and removed the read only flag on the files, but that didn't help either.

When I click the fix button i get following message:

The file you asked to clean does not appear to be in error. No cleaning was attemtped

[netwolfuk]

Thanks for testing it. Hmm that's odd.
Is there anything useful in the teamcity-server.log file?

[padawahn]

no sadly - no warnings or errors related.

[netwolfuk]

Ok. Thanks for looking. I'll look at spinning up something in AWS to do some testing. I'm sorry this isn't as straight forward as I hoped. Thanks for working through it with me so far.

[padawahn]

np - let me know if there is anything I can do!

[netwolfuk]

Thanks. Hopefully I'll have a new version to test within a few days.

[netwolfuk]

Btw. Unless you want to edit templates, the rest API is not required. Webhook functionality does not require the API.

[padawahn]

Well technically I want to edit headers - at least that's what I think is the problem why I can't post to a discord webhook.

[netwolfuk]

Ah. Well, you would've been disappointed. Can you raise a new issue with requirements on what header(s) you need to set?
And whether the content needs to be dynamically generated.

[padawahn]

well technically discord supports slack like json format.
discord is quite similar to slack - I don't know much on why it doesn't work, all I know is when i do it through hurl.it (manually) it works with the template you use in the slack "Slack.com Compact Notification (JSON)" payload.

Added new issue #86 for this

[netwolfuk]

I have half fixed this issue on windows.
The actual problem with the "fix" link was that the file path was not escaped in javascript when it was sent to the ajax endpoint to initiate the clean.

So, names like /foo/bar/api.zip are fine, but \foo\bar\api.zip are not, because javascript thinks the \ is an escape and it gets all kinds of messed up.

So that bug is fixed and removing the jars from the API zip file now works. However, removing the jars from TeamCity's "unpacked" folder does not work because Windows has the files locked.

Interestingly, they are also not cleared out (re-unpacked) on a teamcity restart. I don't know how I can work around that.

I'll have a look at issue #86 and then do a release.

[padawahn]

So I have tested this too.

The error I get is:
java.nio.file.FileSystemException: d:\TeamCity\bin\..\webapps\ROOT\WEB-INF\plugins\.unpacked\rest-api\server\jaxb-api-2.2.5.jar: The process cannot access the file because it is being used by another process.

Which seems to be the windows file lock you are talking about.
Is there a manual way to force the clearing?
Also how do I know if the zip has been updated?

Both "Fix" buttons cause the same error for me.

[padawahn]

Maybe a temporary workaround could be that the tool would list all files that need to be deleted/updated and the owner of teamcity would be able to do that manually during a restart cycle.

[netwolfuk]

Yes. Great idea. I will list then all explicitly and what their fix status is.

The fix link attempts to fix all items, so a UI change to indicate that would be an improvement.

Currently it stops processing when it encounters an error. That was a change I made yesterday so that I could return the error, but in hindsight that was not a great idea.

[netwolfuk]

I have improved the UX a bit and now show more information. A new build should kick off shortly and then be available for download.
I've not had a chance to test it on windows yet. If you get a chance, I'd be grateful.

[padawahn]

sure let me know when the build is ready for testing.

[netwolfuk]

Oops. String.join is java 8, so the build failed. Sorry. I'll sort that out when I get back to the computer. I really should configure my ide for java 7.

[netwolfuk]

A new build is available. I won't get to test it. Busy at our place.

And I've fixed my ide to build with Java 7. Hopefully no more of these embarrassing compiler issues.

[netwolfuk]

I just did a quick test in AWS:

• Windows 2016
• TeamCity 2017.2.1
• TeamCity Server and Agent services running as SYSTEM user

When I ran the new "Fix...." link, it indicated that the files were in use.
The table refreshes and the ZIP shows as cleaned and in need of a restart, but that the unpacked jars required manual intervention.

Restarting TeamCity (from the TeamCity diagnostics area) did not clear the unpacked folder, and neither did restarting the windows service for TeamCity Server.
I had to stop the TeamCity service, remove the two jars and then restart the service.

The two jars are slightly easier to spot, because they have the timestamp of the previous unpack, not when the service is restarted.

[netwolfuk]

BTW, the new screen looks like the below, to be more explicit about what needs to be fixed.

You will probably see something more like this... (but not exactly as it will say "Manual intervention required")

Unfortunately, I did not take a screenshot of the result in windows on AWS.

[padawahn]

Good news. On Win 10, System user & TC as service everything works just fine now.
Was able to edit templates and tested them with Discord.
I had to do the manual delete for the unpacked files jaxb-impl-2.5.5.jar & jaxb-api-2.5.5.jar.
So I turned of teamcity (stopped the service) manually delete the files and restarted.

Cheers!

[netwolfuk]

Terrific. Thanks for the feedback. I will check that it all still works correctly on Linux and in Docker and then do a release.

[netwolfuk]

What did you think of the template edit process? Was it straightforward enough?

Did you override an existing template or copy it to create a new one?

I have created a new github repository for sharing templates. I need to write up some docs and test that it all works, but the plan is to allow people to submit pull requests to add new templates.

It's possible to export a template as XML, I just need to validate that the import process works correctly.

[padawahn]

It was fine - I had no problem cloning the template and editing some wordings for my purpose.
Since it was working pretty well from the Slack config, there was really little to do.

[netwolfuk]

Great. I was wondering about providing a test button, to execute a test webhook POST before saving the template. Would that be useful?

[padawahn]

a test is always nice, especially since else you have to actually do a build to test it :)

[netwolfuk]

This bug is now closed as it is fixed in alpha14.

The testing stuff (#79) will be deferred for either 1.1 RC, or the 1.2 release cycle. It's currently allocated against the 1.2 milestone.