Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Sync the Metadata storage type into ApplicationConfig #4858

Merged
merged 71 commits into from
Aug 16, 2019
Merged
Changes from 1 commit
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
f51b394
Polish apache/dubbo#4542 : [Enhancement] Adapt the Java standard Even…
mercyblitz Jul 11, 2019
9e58caa
Polish apache/dubbo#4541 : [Feature] Add local File System DynamicCon…
mercyblitz Jul 15, 2019
9252be1
Polish apache#4541 : Bugfix
mercyblitz Jul 15, 2019
08089b6
Polish apache/dubbo#4541 : Optimization
mercyblitz Jul 15, 2019
cdd9338
Polish apache/dubbo#4541 : Add the compatibility for PollingWatchServ…
mercyblitz Jul 16, 2019
f0408f7
Polish apache/dubbo#4541 : Add delay publish without ThreadPoolExecutor
mercyblitz Jul 16, 2019
5f354a0
Polish apache/dubbo#4541 : Refactor the extension name
mercyblitz Jul 17, 2019
8152a75
Polish apache/dubbo#4541 : Add remove ops
mercyblitz Jul 17, 2019
df1a258
Polish apache/dubbo#4541 : Add testable constructor
mercyblitz Jul 17, 2019
52c0205
Polish apache/dubbo#4541 : Add getConfigGroups method
mercyblitz Jul 17, 2019
293a70d
Polish apache/dubbo#4610 : [Refactor] Refactor the bootstrap module
mercyblitz Jul 20, 2019
3218a1f
Polish apache/dubbo#4541 : Fix the nulling URL issue
mercyblitz Jul 20, 2019
d5c78ac
Polish apache/dubbo#4622 : [Refactor] Refactor ConfigManager
mercyblitz Jul 25, 2019
1d54a1d
Polish apache/dubbo#4622 : [Refactor] Refactor ConfigManager
mercyblitz Jul 25, 2019
5071a04
Polish apache/dubbo#4622 : Support multiple configcenters
mercyblitz Jul 26, 2019
ccf614c
Polish apache/dubbo#4671 : ServiceNameMapping will not map the group,…
mercyblitz Jul 26, 2019
87e88e2
update referenceCount log (#4683)
haiyang1985 Jul 29, 2019
5d299af
Polish /apache/dubbo#4687 : Remove the duplicated test code in dubbo-…
mercyblitz Jul 29, 2019
25ec31e
#4685 修改代码if判断false问题 if (hasException == false)修改成if (!hasException…
smipo Jul 30, 2019
c3132b8
Fixed Service annotation method parameters are not in effect (#4598)
Leishunyu Jul 31, 2019
f490df8
keep demo simple, and switch to use zookeeper as registry center (#4705)
beiwei30 Jul 31, 2019
c8dbd7d
@Reference auto-wires the instance of generic interface #4594 (#4677)
CodingSinger Jul 31, 2019
65a0140
try to shorten maven output to make travis build pass (#4710)
beiwei30 Aug 1, 2019
5b38331
use CountDownLatch to check zk registry if establish connection (#4589)
tswstarplanet Aug 1, 2019
a646874
Minor change
mercyblitz Aug 1, 2019
b8e601a
Merge remote-tracking branch 'upstream/master' into dubbo-cloud-native
mercyblitz Aug 2, 2019
0644fe1
Rename the extension name of WritableMetadataService
mercyblitz Aug 5, 2019
9f405d7
Polish apache/dubbo#4759 : [Refactor] Change the signature of methods…
mercyblitz Aug 7, 2019
a344121
Merge remote-tracking branch 'upstream/master' into dubbo-cloud-native
mercyblitz Aug 7, 2019
8c7f37a
Polish apache/dubbo#3984 : Add the implementation of Page<ServiceInst…
mercyblitz Aug 7, 2019
2a88323
Code merge
mercyblitz Aug 7, 2019
240c456
Code merge
mercyblitz Aug 8, 2019
391d053
Fix the cases
mercyblitz Aug 8, 2019
3e57a0a
Merge remote-tracking branch 'upstream/cloud-native' into dubbo-cloud…
mercyblitz Aug 8, 2019
83fc425
Merge remote-tracking branch 'upstream/cloud-native' into dubbo-cloud…
mercyblitz Aug 8, 2019
a147829
Refactor ConfigManager
mercyblitz Aug 12, 2019
eea35ce
Merge remote-tracking branch 'upstream/cloud-native' into dubbo-cloud…
mercyblitz Aug 12, 2019
e307e98
Refactor ConfigManager
mercyblitz Aug 12, 2019
f84a33d
Merge remote-tracking branch 'upstream/cloud-native' into dubbo-cloud…
mercyblitz Aug 12, 2019
ad3e638
Resolve the issues on ConfigManager
mercyblitz Aug 12, 2019
19f1474
Resolve conflict
mercyblitz Aug 12, 2019
07cf7b1
Refactor and add test-cases for ConfigManager
mercyblitz Aug 13, 2019
b4bf6ba
Merge branch 'cloud-native' into dubbo-cloud-native
mercyblitz Aug 13, 2019
a7088ef
Merge remote-tracking branch 'upstream/cloud-native' into dubbo-cloud…
mercyblitz Aug 13, 2019
df9ff69
Merge remote-tracking branch 'origin/dubbo-cloud-native' into dubbo-c…
mercyblitz Aug 13, 2019
51d2be4
Polish apache/dubbo#4774 : [Feature] Dubbo Cloud Native - To Support …
mercyblitz Aug 13, 2019
c284dad
Polish apache/dubbo#4808 : [Feature] Add the registered/unregistered …
mercyblitz Aug 13, 2019
3d84c35
Polish apache/dubbo#4807 : [Feature] Add the callback mechanism Shutd…
mercyblitz Aug 13, 2019
21bcfaa
Polish apache/dubbo#4813 : [Feature] add Prioritized implementation f…
mercyblitz Aug 13, 2019
63a315c
Polish apache/dubbo#4815 : [Feature] Add the ServiceLoader for Dubbo'…
mercyblitz Aug 14, 2019
741321c
Polish apache/dubbo#4815 : [Feature] Add the ServiceLoader for Dubbo'…
mercyblitz Aug 14, 2019
70f9de2
Merge remote-tracking branch 'upstream/cloud-native' into dubbo-cloud…
mercyblitz Aug 14, 2019
16c77fc
Merge remote-tracking branch 'upstream/cloud-native' into dubbo-cloud…
mercyblitz Aug 14, 2019
d448ab6
Merge remote-tracking branch 'upstream/cloud-native' into dubbo-cloud…
mercyblitz Aug 14, 2019
1ac1ad7
Merge remote-tracking branch 'upstream/cloud-native' into dubbo-cloud…
mercyblitz Aug 15, 2019
f67d769
Polish apache/dubbo#4813 : [Feature] add Prioritized implementation f…
mercyblitz Aug 15, 2019
b286e1d
Polish apache/dubbo#4807 : Add sort implementation
mercyblitz Aug 15, 2019
9f6e79e
Merge remote-tracking branch 'upstream/cloud-native' into dubbo-cloud…
mercyblitz Aug 15, 2019
0bfda83
Refactor
mercyblitz Aug 15, 2019
b5da5da
Refactor
mercyblitz Aug 16, 2019
8a7f52b
Polish apache/dubbo#4845 : [Feature] Enhance the Event-Publishing fea…
mercyblitz Aug 16, 2019
4a43d0c
Merge branch 'cloud-native' into dubbo-cloud-native
mercyblitz Aug 16, 2019
f50eed8
Merge remote-tracking branch 'upstream/cloud-native' into dubbo-cloud…
mercyblitz Aug 16, 2019
c70f55b
Merge remote-tracking branch 'origin/dubbo-cloud-native' into dubbo-c…
mercyblitz Aug 16, 2019
176d56b
Merge remote-tracking branch 'upstream/cloud-native' into dubbo-cloud…
mercyblitz Aug 16, 2019
49de00e
Merge remote-tracking branch 'upstream/cloud-native' into dubbo-cloud…
mercyblitz Aug 16, 2019
147c477
Merge remote-tracking branch 'upstream/cloud-native' into dubbo-cloud…
mercyblitz Aug 16, 2019
3fb793a
Merge remote-tracking branch 'upstream/cloud-native' into dubbo-cloud…
mercyblitz Aug 16, 2019
39fca05
Polish apache/dubbo#4854 : [Feature] MetadataService supports the Dub…
mercyblitz Aug 16, 2019
1f749cf
Merge remote-tracking branch 'upstream/cloud-native' into dubbo-cloud…
mercyblitz Aug 16, 2019
4ef7e30
Polish apache/dubbo#4857 : [Enhancement] Sync the Metadata storage ty…
mercyblitz Aug 16, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Polish #4541 : Add testable constructor
mercyblitz committed Jul 17, 2019
commit df1a2587e18caf5c4c27c5f04ba988ac6981ac8a
Original file line number Diff line number Diff line change
@@ -137,7 +137,7 @@ public class FileSystemDynamicConfiguration implements DynamicConfiguration {
/**
* the delay to action in seconds. If null, execute indirectly
*/
private static final Integer delayAction;
private static final Integer delay;

/**
* The thread pool for {@link WatchEvent WatchEvents} loop
@@ -152,16 +152,14 @@ public class FileSystemDynamicConfiguration implements DynamicConfiguration {
watchService = newWatchService();
basedPoolingWatchService = detectPoolingBasedWatchService(watchService);
modifiers = initWatchEventModifiers();
delayAction = initDelayAction(modifiers);
delay = initDelay(modifiers);
watchEventsLoopThreadPool = newWatchEventsLoopThreadPool();
}

/**
* The Root Directory for config center
*/
private final File directory;

private final String threadPoolPrefixName;
private final File rootDirectory;

private final String encoding;

@@ -183,10 +181,16 @@ public class FileSystemDynamicConfiguration implements DynamicConfiguration {
private final Map<File, List<ConfigurationListener>> listenersRepository;

public FileSystemDynamicConfiguration(URL url) {
this.directory = initDirectory(url);
this.threadPoolPrefixName = getThreadPoolPrefixName(url);
this.encoding = getEncoding(url);
this.workersThreadPool = initWorkersThreadPool(url);
this(initDirectory(url), getEncoding(url), getThreadPoolPrefixName(url), getThreadPoolSize(url));
}

public FileSystemDynamicConfiguration(File rootDirectory, String encoding,
String threadPoolPrefixName,
int threadPoolSize
) {
this.rootDirectory = rootDirectory;
this.encoding = encoding;
this.workersThreadPool = initWorkersThreadPool(threadPoolPrefixName, threadPoolSize);
this.processingDirectories = initProcessingDirectories();
this.listenersRepository = new LinkedHashMap<>();
}
@@ -195,18 +199,14 @@ private Set<File> initProcessingDirectories() {
return isBasedPoolingWatchService() ? new LinkedHashSet<>() : emptySet();
}

private static String getThreadPoolPrefixName(URL url) {
return url.getParameter(THREAD_POOL_PREFIX_PARAM_NAME, DEFAULT_THREAD_POOL_PREFIX);
}

@Override
public void addListener(String key, String group, ConfigurationListener listener) {
doInListener(key, group, (configFilePath, listeners) -> {

if (listeners.isEmpty()) { // If no element, it indicates watchService was registered before
ThrowableConsumer.execute(configFilePath, configFile -> {
FileUtils.forceMkdirParent(configFile);
// A directory to be watched
// A rootDirectory to be watched
File configDirectory = configFile.getParentFile();
if (configDirectory != null) {
// Register the configDirectory
@@ -230,7 +230,7 @@ public void removeListener(String key, String group, ConfigurationListener liste

protected File configDirectory(String group) {
String actualGroup = isBlank(group) ? DEFAULT_GROUP : group;
return new File(directory, actualGroup);
return new File(rootDirectory, actualGroup);
}

protected File configFile(String key, String group) {
@@ -307,7 +307,7 @@ private void signalConfigDirectory(File configDirectory) {
// notify configDirectory
notifyProcessingDirectory(configDirectory);
if (logger.isDebugEnabled()) {
logger.debug(format("The config directory[%s] is signalled...", configDirectory.getName()));
logger.debug(format("The config rootDirectory[%s] is signalled...", configDirectory.getName()));
}
}
}
@@ -383,9 +383,7 @@ public String removeConfig(String key, String group) {
});
}

private ThreadPoolExecutor initWorkersThreadPool(URL url) {
int size = url.getParameter(THREAD_POOL_SIZE_PARAM_NAME, DEFAULT_THREAD_POOL_SIZE);
String prefix = url.getParameter(THREAD_POOL_PREFIX_PARAM_NAME, DEFAULT_THREAD_POOL_PREFIX);
private ThreadPoolExecutor initWorkersThreadPool(String prefix, int size) {
return (ThreadPoolExecutor) newFixedThreadPool(size, new NamedThreadFactory(prefix));
}

@@ -405,7 +403,7 @@ protected <V> V delay(String key, String group, ThrowableFunction<File, V> funct
File configDirectory = configFile.getParentFile();
executeMutually(configDirectory, () -> {
if (hasListeners(configFile) && isProcessing(configDirectory)) {
Integer delay = getDelayAction();
Integer delay = getDelay();
if (delay != null) {
// wait for delay in seconds
long timeout = SECONDS.toMillis(delay);
@@ -439,9 +437,9 @@ private boolean hasListeners(File configFile) {
}

/**
* Is processing on {@link #configDirectory(String) config directory}
* Is processing on {@link #configDirectory(String) config rootDirectory}
*
* @param configDirectory {@link #configDirectory(String) config directory}
* @param configDirectory {@link #configDirectory(String) config rootDirectory}
* @return if processing , return <code>true</code>, or <code>false</code>
*/
private boolean isProcessing(File configDirectory) {
@@ -487,8 +485,8 @@ private <V> V execute(Callable<V> task, long timeout) {
return value;
}

protected File getDirectory() {
return directory;
protected File getRootDirectory() {
return rootDirectory;
}

protected ThreadPoolExecutor getWorkersThreadPool() {
@@ -499,8 +497,8 @@ protected String getEncoding() {
return encoding;
}

protected Integer getDelayAction() {
return delayAction;
protected Integer getDelay() {
return delay;
}

/**
@@ -515,6 +513,10 @@ protected static boolean isBasedPoolingWatchService() {
return basedPoolingWatchService;
}

private static String getThreadPoolPrefixName(URL url) {
return url.getParameter(THREAD_POOL_PREFIX_PARAM_NAME, DEFAULT_THREAD_POOL_PREFIX);
}

protected static ThreadPoolExecutor getWatchEventsLoopThreadPool() {
return watchEventsLoopThreadPool;
}
@@ -537,7 +539,7 @@ private static <T> T[] of(T... values) {
return values;
}

private static Integer initDelayAction(WatchEvent.Modifier[] modifiers) {
private static Integer initDelay(WatchEvent.Modifier[] modifiers) {
return Stream.of(modifiers)
.filter(modifier -> modifier instanceof SensitivityWatchEventModifier)
.map(SensitivityWatchEventModifier.class::cast)
@@ -587,7 +589,7 @@ private static File initDirectory(URL url) {
String directoryPath = url.getParameter(CONFIG_CENTER_DIR_PARAM_NAME, DEFAULT_CONFIG_CENTER_DIR_PATH);
File rootDirectory = new File(url.getParameter(CONFIG_CENTER_DIR_PARAM_NAME, DEFAULT_CONFIG_CENTER_DIR_PATH));
if (!rootDirectory.exists() && !rootDirectory.mkdirs()) {
throw new IllegalStateException(format("Dubbo config center directory[%s] can't be created!",
throw new IllegalStateException(format("Dubbo config center rootDirectory[%s] can't be created!",
directoryPath));
}
return rootDirectory;
@@ -597,6 +599,10 @@ private static String getEncoding(URL url) {
return url.getParameter(CONFIG_CENTER_ENCODING_PARAM_NAME, DEFAULT_CONFIG_CENTER_ENCODING);
}

private static int getThreadPoolSize(URL url) {
return url.getParameter(THREAD_POOL_SIZE_PARAM_NAME, DEFAULT_THREAD_POOL_SIZE);
}

private static ThreadPoolExecutor newWatchEventsLoopThreadPool() {
return new ThreadPoolExecutor(THREAD_POOL_SIZE, THREAD_POOL_SIZE,
0L, MILLISECONDS,
Original file line number Diff line number Diff line change
@@ -52,7 +52,7 @@ public void init() {
String classPath = getClassPath();
URL url = valueOf("dubbo://127.0.0.1:20880").addParameter(CONFIG_CENTER_DIR_PARAM_NAME, classPath + File.separator + "config-center");
configuration = new FileSystemDynamicConfiguration(url);
deleteQuietly(configuration.getDirectory());
deleteQuietly(configuration.getRootDirectory());
}

private String getClassPath() {
@@ -62,7 +62,7 @@ private String getClassPath() {
@Test
public void testInit() {

assertEquals(new File(getClassPath(), "config-center"), configuration.getDirectory());
assertEquals(new File(getClassPath(), "config-center"), configuration.getRootDirectory());
assertEquals("UTF-8", configuration.getEncoding());
assertEquals(ThreadPoolExecutor.class, configuration.getWorkersThreadPool().getClass());
assertEquals(1, (configuration.getWorkersThreadPool()).getCorePoolSize());
@@ -72,9 +72,9 @@ public void testInit() {
assertEquals(1, (configuration.getWatchEventsLoopThreadPool()).getMaximumPoolSize());

if (configuration.isBasedPoolingWatchService()) {
assertEquals(2, configuration.getDelayAction());
assertEquals(2, configuration.getDelay());
} else {
assertNull(configuration.getDelayAction());
assertNull(configuration.getDelay());
}
}

@@ -96,7 +96,7 @@ public void testPublishAndRemoveConfig() throws InterruptedException {
});
assertTrue(configuration.publishConfig(KEY, CONTENT));
assertEquals(CONTENT, configuration.removeConfig(KEY));
Thread.sleep(configuration.getDelayAction() * 1000);
Thread.sleep(configuration.getDelay() * 1000);
}

@Test