[homematic] Fix for two (re)connection problems (openhab#9692)

* Replace deprecated constructors
* Removed no longer existing settings from the documentation. They were already marked as deprecated since several versions.
* Refactored communication with the HM gateway
- simplified coding for the communication with the gateway
- buffer size for communication is now configurable to avoid problems
with too small buffers
- Previous solution for openhab#6963 was not sufficient. Should be finally done
with these changes
* Retrieving the duty cycle is sufficient to check connection
- ping requests could therefore be safely removed problems with the automatic reconnection were solved.
* Changed to explicit list of Exception

Fixes openhab#8808

Signed-off-by: Martin Herbst <develop@mherbst.de>

bundles/org.openhab.binding.homematic/README.md

@@ -133,24 +133,12 @@ Callback network address of the system runtime, default is auto-discovery
 - **bindAddress**
 The address the XML-/BINRPC server binds to, default is value of "callbackHost"
 
-- **callbackPort** (DEPRECATED, use "binCallbackPort" resp. "xmlCallbackPort")
-Callback port of the binding's server, default is 9125 and counts up for each additional bridge
-
 - **xmlCallbackPort**
 Callback port of the binding's XML-RPC server, default is 9125 and counts up for each additional bridge
 
 - **binCallbackPort**
 Callback port of the binding's BIN-RPC server, default is 9126 and counts up for each additional bridge
 
-- **aliveInterval** (DEPRECATED, not necessary anymore)
-The interval in seconds to check if the communication with the Homematic gateway is still alive. If no message receives from the Homematic gateway, the RPC server restarts (default = 300)
-
-- **reconnectInterval** (DEPRECATED, not necessary anymore)
-The interval in seconds to force a reconnect to the Homematic gateway, disables "aliveInterval"! (0 = disabled, default = disabled).
-If you have no sensors which sends messages in regular intervals and/or you have low communication, the "aliveInterval" may restart the connection to the Homematic gateway to often.
-The "reconnectInterval" disables the "aliveInterval" and reconnects after a fixed period of time.
-Think in hours when configuring (one hour = 3600)
-
 - **timeout**
 The timeout in seconds for connections to a Homematic gateway (default = 15)
 
@@ -183,6 +171,11 @@ If set to true, devices are automatically unpaired from the gateway when their c
 If set to true, devices are automatically factory reset when their corresponding things are removed.
 Due to the factory reset, the device will also be unpaired from the gateway, even if "unpairOnDeletion" is set to false! (default = false)
 
+- **bufferSize** 
+If a large number of devices are connected to the gateway, the default buffersize of 2048 kB may be too small for communication with the gateway. 
+In this case, e.g. the discovery fails. 
+With this setting the buffer size can be adjusted. The value is specified in kB.
+
 The syntax for a bridge is:
 
 ```java
@@ -676,6 +669,11 @@ Var_1.sendCommand(RefreshType.REFRESH)
 **Note:** adding new and removing deleted variables from the GATEWAY-EXTRAS thing is currently not supported.
 You have to delete the thing, start a scan and add it again.
 
+**`openhab.log` contains an exception with message: `Buffering capacity 2097152 exceeded` resp. discovery detects no devices**
+
+In case of problems in the discovery or if above mentioned error message appears in `openhab.log`, the size for the transmission buffer for the communication with the gateway is too small.
+The problem can be solved by increasing the `bufferSize` value in the bridge configuration.
+
 ### Debugging and Tracing
 
 If you want to see what's going on in the binding, switch the log level to DEBUG in the Karaf console

bundles/org.openhab.binding.homematic/src/main/java/org/openhab/binding/homematic/internal/common/HomematicConfig.java

@@ -58,6 +58,7 @@ public class HomematicConfig {
  private long discoveryTimeToLive = -1;
  private boolean unpairOnDeletion = false;
  private boolean factoryResetOnDeletion = false;
+ private int bufferSize = 2048;
 
  private HmGatewayInfo gatewayInfo;
 
@@ -381,6 +382,13 @@ public String getEncoding() {
  }
  }
 
+ /**
+ * Returns the buffers size used for the communication with the Homematic gateway.
+ */
+ public int getBufferSize() {
+ return bufferSize;
+ }
+
  /**
  * Returns true, if the configured gatewayType is CCU.
  */

bundles/org.openhab.binding.homematic/src/main/java/org/openhab/binding/homematic/internal/communicator/AbstractHomematicGateway.java

@@ -730,9 +730,6 @@ public void eventReceived(HmDatapointInfo dpInfo, Object newValue) {
  logger.debug("Echo event detected, ignoring '{}'", dpInfo);
  } else {
  try {
- if (connectionTrackerThread != null && dpInfo.isPong() && id.equals(newValue)) {
- connectionTrackerThread.pongReceived();
- }
  if (initialized) {
  final HmDatapoint dp = getDatapoint(dpInfo);
  HmDatapointConfig config = gatewayAdapter.getDatapointConfig(dp);
@@ -900,32 +897,17 @@ private int translateFlags(boolean reset, boolean force, boolean defer) {
  */
  private class ConnectionTrackerThread implements Runnable {
  private boolean connectionLost;
- private boolean ping;
- private boolean pong;
 
  @Override
  public void run() {
  try {
- if (ping && !pong) {
- handleInvalidConnection("No Pong received!");
- }
-
- pong = false;
- if (config.getGatewayInfo().isCCU1()) {
- // the CCU1 does not support the ping command, we need a workaround
- getRpcClient(getDefaultInterface()).listBidcosInterfaces(getDefaultInterface());
- // if there is no exception, connection is valid
- pongReceived();
- } else {
- getRpcClient(getDefaultInterface()).ping(getDefaultInterface(), id);
- }
- ping = true;
-
- try {
- updateDutyCycleRatio();
- } catch (IOException e) {
- logger.debug("Could not read the duty cycle ratio: {}", e.getMessage());
+ ListBidcosInterfacesParser parser = getRpcClient(getDefaultInterface())
+ .listBidcosInterfaces(getDefaultInterface());
+ Integer dutyCycleRatio = parser.getDutyCycleRatio();
+ if (dutyCycleRatio != null) {
+ gatewayAdapter.onDutyCycleRatioUpdate(dutyCycleRatio);
  }
+ connectionConfirmed();
  } catch (IOException ex) {
  try {
  handleInvalidConnection("IOException " + ex.getMessage());
@@ -935,21 +917,6 @@ public void run() {
  }
  }
 
- public void pongReceived() {
- pong = true;
- connectionConfirmed();
- }
-
- private void updateDutyCycleRatio() throws IOException {
- ListBidcosInterfacesParser parser = getRpcClient(getDefaultInterface())
- .listBidcosInterfaces(getDefaultInterface());
- Integer dutyCycleRatio = parser.getDutyCycleRatio();
-
- if (dutyCycleRatio != null) {
- gatewayAdapter.onDutyCycleRatioUpdate(dutyCycleRatio);
- }
- }
-
  private void connectionConfirmed() {
  if (connectionLost) {
  connectionLost = false;
@@ -959,7 +926,6 @@ private void connectionConfirmed() {
  }
 
  private void handleInvalidConnection(String cause) throws IOException {
- ping = false;
  if (!connectionLost) {
  connectionLost = true;
  logger.warn("Connection lost on gateway '{}', cause: \"{}\"", id, cause);

bundles/org.openhab.binding.homematic/src/main/java/org/openhab/binding/homematic/internal/communicator/client/RpcClient.java

@@ -88,7 +88,7 @@ public void init(HmInterface hmInterface, String clientId) throws IOException {
  request.addArg(getRpcCallbackUrl());
  request.addArg(clientId);
  if (config.getGatewayInfo().isHomegear()) {
- request.addArg(new Integer(0x22));
+ request.addArg(Integer.valueOf(0x22));
  }
  sendMessage(config.getRpcPort(hmInterface), request);
  }

bundles/org.openhab.binding.homematic/src/main/java/org/openhab/binding/homematic/internal/communicator/client/XmlRpcClient.java

@@ -13,23 +13,19 @@
 package org.openhab.binding.homematic.internal.communicator.client;
 
 import java.io.ByteArrayInputStream;
-import java.io.ByteArrayOutputStream;
 import java.io.IOException;
-import java.io.InputStream;
-import java.io.UnsupportedEncodingException;
 import java.util.concurrent.ExecutionException;
 import java.util.concurrent.TimeUnit;
 import java.util.concurrent.TimeoutException;
 
 import javax.xml.parsers.ParserConfigurationException;
 
 import org.eclipse.jetty.client.HttpClient;
+import org.eclipse.jetty.client.api.ContentResponse;
 import org.eclipse.jetty.client.api.Request;
-import org.eclipse.jetty.client.api.Response;
 import org.eclipse.jetty.client.util.BytesContentProvider;
-import org.eclipse.jetty.client.util.InputStreamResponseListener;
+import org.eclipse.jetty.client.util.FutureResponseListener;
 import org.eclipse.jetty.http.HttpHeader;
-import org.eclipse.jetty.http.HttpStatus;
 import org.openhab.binding.homematic.internal.common.HomematicConfig;
 import org.openhab.binding.homematic.internal.communicator.message.RpcRequest;
 import org.openhab.binding.homematic.internal.communicator.message.XmlRpcRequest;
@@ -111,39 +107,18 @@ private byte[] send(int port, RpcRequest<String> request) throws IOException {
  }
  Request req = httpClient.POST(url).content(content).timeout(config.getTimeout(), TimeUnit.SECONDS)
  .header(HttpHeader.CONTENT_TYPE, "text/xml;charset=" + config.getEncoding());
- try {
- ret = req.send().getContent();
- } catch (IllegalArgumentException e) { // Returned buffer too large
- logger.info("Blocking XmlRpcRequest failed: {}, trying non-blocking request", e.getMessage());
- InputStreamResponseListener respListener = new InputStreamResponseListener();
- req.send(respListener);
- Response resp = respListener.get(config.getTimeout(), TimeUnit.SECONDS);
- ByteArrayOutputStream respData = new ByteArrayOutputStream(RESP_BUFFER_SIZE);
-
- int httpStatus = resp.getStatus();
- if (httpStatus == HttpStatus.OK_200) {
- byte[] recvBuffer = new byte[RESP_BUFFER_SIZE];
- try (InputStream input = respListener.getInputStream()) {
- while (true) {
- int read = input.read(recvBuffer);
- if (read == -1) {
- break;
- }
- respData.write(recvBuffer, 0, read);
- }
- ret = respData.toByteArray();
- }
- } else {
- logger.warn("Non-blocking XmlRpcRequest failed, status code: {} / request: {}", httpStatus,
- request);
- resp.abort(new IOException());
- }
+ FutureResponseListener listener = new FutureResponseListener(req, config.getBufferSize() * 1024);
+ req.send(listener);
+ ContentResponse response = listener.get(config.getTimeout(), TimeUnit.SECONDS);
+ ret = response.getContent();
+ if (ret == null || ret.length == 0) {
+ throw new IOException("Received no data from the Homematic gateway");
  }
  if (logger.isTraceEnabled()) {
  String result = new String(ret, config.getEncoding());
  logger.trace("Client XmlRpcResponse (port {}):\n{}", port, result);
  }
- } catch (UnsupportedEncodingException | ExecutionException | TimeoutException | InterruptedException e) {
+ } catch (InterruptedException | ExecutionException | TimeoutException | IllegalArgumentException e) {
  throw new IOException(e);
  }
  return ret;

bundles/org.openhab.binding.homematic/src/main/java/org/openhab/binding/homematic/internal/communicator/message/BinRpcMessage.java

@@ -221,7 +221,7 @@ private Object readRpcValue() throws IOException {
  int type = readInt();
  switch (type) {
  case 1:
- return new Integer(readInt());
+ return Integer.valueOf(readInt());
  case 2:
  return binRpcData[offset++] != 0 ? Boolean.TRUE : Boolean.FALSE;
  case 3:
@@ -235,7 +235,7 @@ private Object readRpcValue() throws IOException {
  return new Date(readInt() * 1000);
  case 0xD1:
  // Int64
- return new Long(readInt64());
+ return Long.valueOf(readInt64());
  case 0x100:
  // Array
  int numElements = readInt();

bundles/org.openhab.binding.homematic/src/main/java/org/openhab/binding/homematic/internal/communicator/message/XmlRpcResponse.java

@@ -112,10 +112,10 @@ public void endElement(String uri, String localName, String qName) throws SAXExc
  break;
  case "int":
  case "i4":
- data.add(new Integer(currentValue));
+ data.add(Integer.valueOf(currentValue));
  break;
  case "double":
- data.add(new Double(currentValue));
+ data.add(Double.valueOf(currentValue));
  break;
  case "string":
  case "name":

bundles/org.openhab.binding.homematic/src/main/resources/OH-INF/thing/bridge.xml

@@ -120,7 +120,12 @@
  <advanced>true</advanced>
  <default>false</default>
  </parameter>
-
+ <parameter name="bufferSize" type="integer" min="0">
+ <label>Buffer Size</label>
+ <description>Size of the response buffer retrieved from the gateway (default 2048 kB)</description>
+ <default>2048</default>
+ <advanced>true</advanced>
+ </parameter>
  </config-description>
  </bridge-type>