Memoized implementation for PathMatchingResourcePatternResolver [SPR-17344]

[spring-projects-issues]

Rahul Shinde opened SPR-17344 and commented

For applications that get bundled as an uber jar, while benchmarking the startup timings, it was found that there is always a fixed cost associated with every lookup that is performed by PathMatchingResourcePatternResolver (PMRPR) while component scanning.

This is a result of opening up the jar file and traversing the entries each time when PMRPR is used to find the resources.

With 10K+ beans and large number of packages for component-scanning, this adds up pretty quick. 

After adding a Memoized implementation of the PathMatchingResourcePatternResolver, the refresh timings dropped around 2-4 minutes depending on the hardware spec.

The Memoized implementation builds a cache when a given jar is looked up the very first time. For subsequent lookups, it simply does a binary search over the cached entries. 

This memoized implementation is attached to the ApplicationContext by overriding getResourcePatternResolver(..)

It would be good to have this as an option for applications that want to use a memoized version of PMRPR.

 

 

---

Affects: 5.0.9, 5.1 GA

Referenced from: pull request #1981

[spring-projects-issues]

Stéphane Nicoll commented

That's interesting. Have you considered using the components index on this application?

[spring-projects-issues]

Rahul Shinde commented

The index approach though I agree would have helped doesn't work for us at the moment.

We have use-cases where @ComponentScan uses filters on the beans that need to be discovered that reside in other modules. The spring-context-indexer I think gathers all the beans in the module and not necessarily the ones specified by @ComponentScan.

[spring-projects-issues]

Stéphane Nicoll commented

It does work regardless of the modules you have. You just need to make sure to apply the indexer for every module that defines components.

[spring-projects-issues]

Rahul Shinde commented

It is not really a limitation of the indexer but how each of the modules gets built and what information they have at build time.
Since the source of @ComponentScan resides in another module, the indexer for the module that has the components wont even know about it and would build up the full list as their build lifecycle is managed independently by separate teams.
At runtime however, this gets resolved correctly when all the modules come together and only components matching the criteria get discovered.
We also happen to auto-generate components that currently cannot take advantage of the indexer (limitation of our incremental build process).

[spring-projects-issues]

Stéphane Nicoll commented

That's the wrong place to discuss this but it doesn't matter where @ComponentScan is. As for generating components at runtime, this should be possible if you use an AP and they are ordered properly. I'd be interested to know more offline to see if there is something we can improve in the indexer for your use cases: snicoll at pivotal dot io.

[spring-projects-issues]

Rahul Shinde commented

Let me run and see if this is reproducible with the latest version. Will contact you offline. Thanks

[spring-projects-issues]

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

[spring-projects-issues]

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.