Implement startup plugin as framework for service discovery

[johann8384]

This would allow users to integrate with their own internal service discovery mechanism of choice. It could be used to announce OpenTSDB nodes as well as discover HBase clusters to communicate with.

Related to #688

Manolama said:

What kind of mods would you make to the config with the plugin? Change HBase clusters,
that kinda stuff? If so the best place would probably be in TSDMain.java where the config is loaded,
then you could use the config to load a plugin, tweak the config, then instantiate the pipeline and
TSDB.

My Response:

My initial thought was simply a plugin that got notified during the early startup phase
and was given a copy of the current configuration.

Being able to modify the configuration and hand it back to OpenTSDB (after OpenTSDB
parses it, but before it used it) was an after-thought that I thought might lead to more
possibilities in the future.

The main thing I wanted was just to allow the service to announce itself, indicate what
ports it listened on, whether it was ro/rw, and what zkQuorum it was talking to. I figured
others might "abuse" it in the future to go out and register with a service discovery and
then possibly inject things like a zkquorum into the configuration dynamically. You could
also use it for things like coordinating listening ports or something.

[johann8384]

What about this as the interface:

// This file is part of OpenTSDB.
// Copyright (C) 2013  The OpenTSDB Authors.
//
// This program is free software: you can redistribute it and/or modify it
// under the terms of the GNU Lesser General Public License as published by
// the Free Software Foundation, either version 2.1 of the License, or (at your
// option) any later version.  This program is distributed in the hope that it
// will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
// of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Lesser
// General Public License for more details.  You should have received a copy
// of the GNU Lesser General Public License along with this program.  If not,
// see <http://www.gnu.org/licenses/>.
package net.opentsdb.tools;

import com.stumbleupon.async.Deferred;

import net.opentsdb.utils.Config;
import net.opentsdb.stats.StatsCollector;

/**
 * The StartupPlugin allows users to interact with the OpenTSDB configuration
 * as soon as it is completely parsed, just before OpenTSDB begins to use it.
 * <p>
 * <b>Note:</b> Implementations must have a parameterless constructor. The
 * {@link #initialize(TSDB)} method will be called immediately after the plugin is
 * instantiated and before any other methods are called.
 * @since 2.3
 */
public abstract class StartupPlugin {

  /**
   * Called by TSDB to initialize the plugin
   * Implementations are responsible for setting up any IO they need as well
   * as starting any required background threads.
   * <b>Note:</b> Implementations should throw exceptions if they can't start
   * up properly. The TSD will then shutdown so the operator can fix the
   * problem. Please use IllegalArgumentException for configuration issues.
   * @param tsdb The parent TSDB object
   * @throws IllegalArgumentException if required configuration parameters are
   * missing
   * @throws Exception if something else goes wrong
   */
  public abstract Config initialize(final Config config) throws IllegalArgumentException, Exception;

  /**
   * Called to gracefully shutdown the plugin. Implementations should close
   * any IO they have open
   * @return A deferred object that indicates the completion of the request.
   * The {@link Object} has not special meaning and can be {@code null}
   * (think of it as {@code Deferred<Void>}).
   */
  public abstract Deferred<Object> shutdown();

  /**
   * Should return the version of this plugin in the format:
   * MAJOR.MINOR.MAINT, e.g. "2.0.1". The MAJOR version should match the major
   * version of OpenTSDB the plugin is meant to work with.
   * @return A version string used to log the loaded version
   */
  public abstract String version();

  /**
   * Called by the TSD when a request for statistics collection has come in. The
   * implementation may provide one or more statistics. If no statistics are
   * available for the implementation, simply stub the method.
   * @param collector The collector used for emitting statistics
   */
  public abstract void collectStats(final StatsCollector collector);

}