Question Involving one topic being spread across NSQds

[spacehash]

I have a question involving an NSQ topology that I can't find an answer for anywhere.

My use case is a distributed network with about 10 servers. Each server will have an instance of NSQ, and one "master" will have an instance of NSQLookup and NSQAdmin. Data will be entered into topics by web crawlers in Docker containers spun up by Kubernetes.

My question is, due to how I intend for Kubernetes to work, it is undefined which server will have which crawler - and one server can be running two different crawlers at the same time (different crawlers will input information into different topics).

How will this work? When I want to request data from one topic, how will NSQLookup decide which NSQ instance to pull from, given the topic is spread across multiple NSQ instances?

I apologize if this is a trivial question, or if it has been already asked.

[spacehash]

update - i've referenced #1078 and #980, but still stuck.

[ploxiln]

You can ask general questions on the "google group" mailing list: https://groups.google.com/forum/#!forum/nsq-users

A topic can exist on multiple nsqd, and this is common. When a consumer process uses nsqlookupd to find a topic, it gets all nsqd which have that topic, and connects to them all. The consumer process continues to poll nsqlookupd (by default every 30 seconds I think) and if the topic appears on a new nsqd the consumer process will soon find out.

[ploxiln]

see https://nsq.io/overview/design.html#eliminating-spofs