Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consumer cannot get the list of providers when use ZK as registry(sometimes it happens). #1422

Closed
chshyin opened this issue Mar 5, 2018 · 3 comments
Labels
type/bug Bugs to being fixed

Comments

@chshyin
Copy link

chshyin commented Mar 5, 2018

the bug is:when consumer and provider started at the same time(or it happens in runtime,when cosumer and provider can't connect to zk,and then network recover),sometimes the consumer will lost the provider list,causing the RpcException。restart consumer or provider will recover it。
code is here:
doSubscribe method id ZookeeperRegistry.java(169-182lines)

                List<URL> urls = new ArrayList<URL>();
                for (String path : toCategoriesPath(url)) {
                    ConcurrentMap<NotifyListener, ChildListener> listeners = zkListeners.get(url);
                    if (listeners == null) {
                        zkListeners.putIfAbsent(url, new ConcurrentHashMap<NotifyListener, ChildListener>());
                        listeners = zkListeners.get(url);
                    }
                    ChildListener zkListener = listeners.get(listener);
                    if (zkListener == null) {
                        listeners.putIfAbsent(listener, new ChildListener() {
                            // A. it will invoke in zk thread,when zk node changed
                            public void childChanged(String parentPath, List<String> currentChilds) {
                                ZookeeperRegistry.this.notify(url, listener, toUrlsWithEmpty(url, parentPath, currentChilds));
                            }
                        });
                        zkListener = listeners.get(listener);
                    }
                    zkClient.create(path, false);
                    List<String> children = zkClient.addChildListener(path, zkListener);
                    if (children != null) {
                        urls.addAll(toUrlsWithEmpty(url, path, children));
                    }
                }
                // B.it will invoke in main thread,or failoverRegister thread
                notify(url, listener, urls);

notity(url,listener,urls) method will invoke at:
A:it will invoke in zk thread,when zk node changed
B:it will invoke in main thread,or failoverRegister thread

The cause of the problem:

  1. when consumer start,before main thread invoke B, thread completes its running slice on the CPU,and wait the next CPU time.at this time(1. provider not registed in zk,the urls is empty list。 2. it registed the zk watch event,will listener to zk node change event.)
    2.provider started,registed in zk, the zk node is changed
    3.reveive the zk node change event,invoke A in the zk thread。(A will invoke in zk thread,and B will invoke in another thread)
    4.At the end of the A method,the consumer obtain the real provider list.
    5.main thread obtain the CPU time,invoke B method,and this time urls is empty list。and update the consumer's provider list to empty。it cause the problem

the problem happens A invoke before B,If you can control the method execution order(B invoke before A),the bug will fix。

@chickenlj chickenlj changed the title dubbo使用zk注册中心时服务提供者列表丢失BUG Consumer cannot get the list of providers when use ZK as registry(sometimes it happens). Mar 6, 2018
@chickenlj
Copy link
Contributor

chickenlj commented Mar 6, 2018

Hi chshyin,

Please consider using English, Dubbo is a global community, and all the issues may be viewed and discussed by people all over the world.

Meanwhile, you can send any question to dev@dubbo.apache.org to discuss questions like this, we highly recommend you subscribe to dev@dubbo.apache.org to receive any discussions and news of dubbo. And we will make sure that every email will receive a reply.

Github issues should be a pure place for tracking determined features, enhancements or bugs.

@chickenlj chickenlj added type/bug Bugs to being fixed type/suspicious labels Mar 7, 2018
@nzomkxia
Copy link
Member

can you give me a procedure on how to reproduce this issue?

@chickenlj
Copy link
Contributor

Please provide more details. Reopen if necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Bugs to being fixed
Projects
None yet
Development

No branches or pull requests

3 participants