Review startup times for a large, legacy Spring Application #6
Comments
|
This is issue I raised around lazy loading all beans when in running in a unit test profile. We have found it pretty useful in our large app when running integration tests. |
|
@dsyer I have some new information on the startup footprint for our application. Please let me know if you have any questions or are in need of more information. Not sure if it makes sense to open up a new issue in Spring Boot repo for the rebinder? Thanks! TL;DR;
The details: My approach with each use case was brute force, I just cloned each library and added some well placed timing code: Mybatis: org.mybatis.spring.mapper.MapperFactoryBean.checkDaoConfig(): protected void checkDaoConfig() {
super.checkDaoConfig();
notNull(this.mapperInterface, "Property 'mapperInterface' is required");
Configuration configuration = getSqlSession().getConfiguration();
if (this.addToConfig && !configuration.hasMapper(this.mapperInterface)) {
try {
long start = System.currentTimeMillis();
configuration.addMapper(this.mapperInterface);
logger.info("Loaded Mapper '" + this.mapperInterface + "' in " + (System.currentTimeMillis() - start) + "ms.");
} catch (Exception e) {
logger.error("Error while adding the mapper '" + this.mapperInterface + "' to configuration.", e);
throw new IllegalArgumentException(e);
} finally {
ErrorContext.instance().reset();
}
}
}PostConstruct: I have opened up a suggested issue to include timing logic around the calls to each post construct method. https://jira.spring.io/browse/SPR-16287 ConfigurationPropertiesRebinder.rebind: public boolean rebind(String name) {
if (!this.beans.getBeanNames().contains(name)) {
return false;
}
if (this.applicationContext != null) {
try {
Object bean = this.applicationContext.getBean(name);
if (AopUtils.isCglibProxy(bean)) {
bean = getTargetObject(bean);
}
long start = System.currentTimeMillis();
this.binder.postProcessBeforeInitialization(bean, name);
this.applicationContext.getAutowireCapableBeanFactory().initializeBean(bean, name);
logger.debug("Rebind on Bean [" + name + "] took " + (System.currentTimeMillis() - start) + "ms");
return true;
}
catch (RuntimeException e) {
this.errors.put(name, e);
throw e;
}
}
return false;
}For reference, looking at the /env actuator endpoint, there are just over a 1000 properties across 13 property sources. Here is the logging I am seeing in our legacy application, again, just timing the call around rebind(String beanName): |
|
That's good data. Maybe time for some optimization in spring-cloud-context. It might be worth trying Spring Boot 2.0 (even though there is a startup time issue there since 2.0.0.M3) because property binding is a lot faster now. I also noticed that a few of the expensive beans in the list above were yours, so maybe they are still doing unnecessary stuff in |
|
I tried commenting out that line in @Override
public void afterSingletonsInstantiated() {
// After all beans are initialized send a pre-emptive EnvironmentChangeEvent
// so that anything that needs to rebind gets a chance now (especially for
// beans in the parent context)
this.context
.publishEvent(new EnvironmentChangeEvent(Collections.<String>emptySet()));
}and I don't see anything really bad happening. You could try doing that and see if it gets you back your 5 seconds. I'll have a think about why it was there originally. |
|
Commenting out the publishEvent does indeed fix the issue, I am down to 27 seconds (and I have still not fixed our PostConstruct issues). Assuming there are no side effects from that, that is awesome! As for your first question, most of the beans being rebound are straight Java beans (properties, getters/setter). The most expensive one, "mainDataSource" is just a jdbc DataSource and the bean definition looks like this: @Bean(name = "mainDataSource", destroyMethod="")
@Primary
@ConfigurationProperties(prefix = "datasource.main")
public DataSource driverManagerMainDataSource() {
return DataSourceBuilder.create().build();
}One of the more expensive beans that is ours: @Component
@ConfigurationProperties(prefix = "amazon.marketplace")
public class AmazonMarketplaceSettings {
private int marketplaceId;
private String merchantId;
private String mwsMarketplaceId;
private String reportsOutputPath;
private String submissionResultsOutputPath;
private String awsAccessKeyId;
private String awsSecretAccessKey;
private String applicationName;
private String applicationVersion;
private String configServiceUrl;
private String merchantToken;
private Map<Integer, AmazonAPICredential> storeCredentialMap;
private Map<String, String> serviceUrlMap;
public int getMarketplaceId() {
return marketplaceId;
}
public void setMarketplaceId(int marketplaceId) {
this.marketplaceId = marketplaceId;
}
public void setMerchantId(String merchantId) {
this.merchantId = merchantId;
}
public String getMerchantId() {
return merchantId;
}
public void setMwsMarketplaceId(String mwsMarketplaceId) {
this.mwsMarketplaceId = mwsMarketplaceId;
}
public String getMwsMarketplaceId() {
return mwsMarketplaceId;
}
public String getReportsOutputPath() {
return reportsOutputPath;
}
public void setReportsOutputPath(String reportsOutputPath) {
this.reportsOutputPath = reportsOutputPath;
}
public String getSubmissionResultsOutputPath() {
return submissionResultsOutputPath;
}
public void setSubmissionResultsOutputPath(
String submissionResultsOutputPath) {
this.submissionResultsOutputPath = submissionResultsOutputPath;
}
public String getAwsAccessKeyId() {
return awsAccessKeyId;
}
public void setAwsAccessKeyId(String awsAccessKeyId) {
this.awsAccessKeyId = awsAccessKeyId;
}
public String getAwsSecretAccessKey() {
return awsSecretAccessKey;
}
public void setAwsSecretAccessKey(String awsSecretAccessKey) {
this.awsSecretAccessKey = awsSecretAccessKey;
}
public String getApplicationName() {
return applicationName;
}
public void setApplicationName(String applicationName) {
this.applicationName = applicationName;
}
public String getApplicationVersion() {
return applicationVersion;
}
public void setApplicationVersion(String applicationVersion) {
this.applicationVersion = applicationVersion;
}
public String getConfigServiceUrl() {
return configServiceUrl;
}
public void setConfigServiceUrl(String configServiceUrl) {
this.configServiceUrl = configServiceUrl;
}
public String getMerchantToken() {
return merchantToken;
}
public void setMerchantToken(String merchantToken) {
this.merchantToken = merchantToken;
}
public Map<Integer, AmazonAPICredential> getStoreCredentialMap() {
return storeCredentialMap;
}
public void setStoreCredentialMap(Map<Integer, AmazonAPICredential> storeCredentialMap) {
this.storeCredentialMap = storeCredentialMap;
}
public Map<String, String> getServiceUrlMap() {
return serviceUrlMap;
}
public void setServiceUrlMap(Map<String, String> serviceUrlMap) {
this.serviceUrlMap = serviceUrlMap;
}
}And the nested public class AmazonAPICredential implements Serializable {
private static final long serialVersionUID = 1L;
private Integer storeId;
private String sellerId;
private String accessKey;
private String secretKey;
public Integer getStoreId() {
return storeId;
}
public void setStoreId(Integer storeId) {
this.storeId = storeId;
}
public String getSellerId() {
return sellerId;
}
public void setSellerId(String sellerId) {
this.sellerId = sellerId;
}
public String getAccessKey() {
return accessKey;
}
public void setAccessKey(String accessKey) {
this.accessKey = accessKey;
}
public String getSecretKey() {
return secretKey;
}
public void setSecretKey(String secretKey) {
this.secretKey = secretKey;
}
} |
|
That Unrelated: you mentioned you had some success with a BPP that turns application components into lazy loading proxies, to speed up startup in integration tests. Do you have a version of that you can share? |
|
We do not have any custom property sources in play, just the ones from the config server, the classpath, and system/environment variables. As for the BPP, I submitted this issue: spring-projects/spring-boot#9685. There is a copy of that post-processor in the issue details. |
|
FYI, I did a local build of 1.3.1-SNAPSHOT: 1.3.0-RELEASE : 35 seconds Your changes around rebinding the parent contexts made a big difference in our application, thank you. |
|
Thanks for the feedback. I made a little utility library to print initialization timing data: https://github.com/dsyer/spring-boot-aspectj/tree/50f9061254e093b5c23273adf4d0117b71ee5186/timing. You can get it from repo.spring.io/snapshots ( |
|
Results: I cloned your aspectj repo and made some changes to the timing interceptor, which at least for me made it a little easier to parse the timing results. I am using a Thread Local to keep track of nested "initialization" and then indenting my messages, if you are interested in a PR, I can put one together. This is what the interceptor looks like with my changes, it's not pretty but works. I am getting what appears to be redundant, nested timing...I couldnt figure out a good way to prune out those "extras". @Aspect
public class TimingInterceptor {
private static Log logger = LogFactory.getLog(TimingInterceptor.class);
private StopWatch bind = new StopWatch("bind");
private StopWatch init = new StopWatch("init");
private ThreadLocal<Integer> level = new ThreadLocal<>();
@Around("execution(private * org.springframework.boot.context.properties.ConfigurationPropertiesBindingPostProcessor.postProcessBeforeInitialization(Object, String, ..)) && args(bean,..)")
public Object bind(ProceedingJoinPoint joinPoint, Object bean) throws Throwable {
bind.start();
Object result = joinPoint.proceed();
bind.stop();
long time = bind.getLastTaskTimeMillis();
if (time > 0) {
logMessage("Bind + " + time + "ms : " + bean.getClass().getName());
}
return result;
}
@Around("execution(* org.springframework.beans.factory.config.BeanPostProcessor+.*(Object, String)) && args(bean,..)")
public Object post(ProceedingJoinPoint joinPoint, Object bean) throws Throwable {
long t0 = System.currentTimeMillis();
Object result = joinPoint.proceed();
long t1 = System.currentTimeMillis();
if ((t1 - t0) > 0) {
logMessage("Post + " + (t1 - t0) + "ms : " + joinPoint.getSignature().getDeclaringType().getSimpleName()
+ "." + joinPoint.getSignature().getName() + ","
+ bean.getClass().getName());
}
return result;
}
@Around("execution(* org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory+.initializeBean(String, Object, ..)) && args(name,bean,..)")
public Object init(ProceedingJoinPoint joinPoint, String name, Object bean)
throws Throwable {
String task = init.currentTaskName();
if (task != null) {
init.stop();
}
init.start(name);
int count = level.get() == null?0:level.get();
count++;
level.set(count);
logMessage("Init Start : " + bean.getClass().getName());
long t0 = System.currentTimeMillis();
Object result = joinPoint.proceed();
long t1 = System.currentTimeMillis();
init.stop();
if (task != null) {
init.start(task);
}
logMessage("Init End : " + (t1 - t0) + "ms : " + bean.getClass().getName());
level.set((count - 1));
return result;
}
private void logMessage(String message) {
Integer index = level.get();
if (index == null || index == 0) {
logger.info(message);
} else {
char[] indent = new char[index];
Arrays.fill(indent, ' ');
logger.info(new String(indent) + message);
}
}
@EventListener
public void started(ContextRefreshedEvent event) {
logger.info("Total bind: " + bind.getTotalTimeMillis());
logger.info("Total init: " + init.getTotalTimeMillis());
}
}And I have put the entire log up on google drive here: |
Please feel free to DM me on twitter @tkvangorder, I am at SpringOne until Thursday afternoon (although, I could extend some time afterwards if necessary. I know your schedule is likely a lot more impacted than me. If it doesnt work out in your schedule, I totally understand.
The text was updated successfully, but these errors were encountered: