[homematic] Fix for two (re)connection problems (#9692)

* Replace deprecated constructors
* Removed no longer existing settings from the documentation. They were already marked as deprecated since several versions.
* Refactored communication with the HM gateway
- simplified coding for the communication with the gateway
- buffer size for communication is now configurable to avoid problems
with too small buffers
- Previous solution for #6963 was not sufficient. Should be finally done
with these changes
* Retrieving the duty cycle is sufficient to check connection
- ping requests could therefore be safely removed problems with the automatic reconnection were solved.
* Changed to explicit list of Exception

Fixes #8808

Signed-off-by: Martin Herbst <develop@mherbst.de>

bundles/org.openhab.binding.homematic/README.md

@@ -133,24 +133,12 @@ Callback network address of the system runtime, default is auto-discovery
 - **bindAddress**
 The address the XML-/BINRPC server binds to, default is value of "callbackHost"
 
-- **callbackPort** (DEPRECATED, use "binCallbackPort" resp. "xmlCallbackPort")
-Callback port of the binding's server, default is 9125 and counts up for each additional bridge
-
 - **xmlCallbackPort**
 Callback port of the binding's XML-RPC server, default is 9125 and counts up for each additional bridge
 
 - **binCallbackPort**
 Callback port of the binding's BIN-RPC server, default is 9126 and counts up for each additional bridge
 
-- **aliveInterval** (DEPRECATED, not necessary anymore)
-The interval in seconds to check if the communication with the Homematic gateway is still alive. If no message receives from the Homematic gateway, the RPC server restarts (default = 300)
-
-- **reconnectInterval** (DEPRECATED, not necessary anymore)
-The interval in seconds to force a reconnect to the Homematic gateway, disables "aliveInterval"! (0 = disabled, default = disabled).
-If you have no sensors which sends messages in regular intervals and/or you have low communication, the "aliveInterval" may restart the connection to the Homematic gateway to often.
-The "reconnectInterval" disables the "aliveInterval" and reconnects after a fixed period of time.
-Think in hours when configuring (one hour = 3600)
-
 - **timeout**
 The timeout in seconds for connections to a Homematic gateway (default = 15)
 
@@ -183,6 +171,11 @@ If set to true, devices are automatically unpaired from the gateway when their c
 If set to true, devices are automatically factory reset when their corresponding things are removed.
 Due to the factory reset, the device will also be unpaired from the gateway, even if "unpairOnDeletion" is set to false! (default = false)
 
+- **bufferSize** 
+If a large number of devices are connected to the gateway, the default buffersize of 2048 kB may be too small for communication with the gateway. 
+In this case, e.g. the discovery fails. 
+With this setting the buffer size can be adjusted. The value is specified in kB.
+
 The syntax for a bridge is:
 
 ```java
@@ -676,6 +669,11 @@ Var_1.sendCommand(RefreshType.REFRESH)
 **Note:** adding new and removing deleted variables from the GATEWAY-EXTRAS thing is currently not supported.
 You have to delete the thing, start a scan and add it again.
 
+**`openhab.log` contains an exception with message: `Buffering capacity 2097152 exceeded` resp. discovery detects no devices**
+
+In case of problems in the discovery or if above mentioned error message appears in `openhab.log`, the size for the transmission buffer for the communication with the gateway is too small.
+The problem can be solved by increasing the `bufferSize` value in the bridge configuration.
+
 ### Debugging and Tracing
 
 If you want to see what's going on in the binding, switch the log level to DEBUG in the Karaf console

bundles/org.openhab.binding.homematic/src/main/java/org/openhab/binding/homematic/internal/common/HomematicConfig.java

@@ -58,6 +58,7 @@ public class HomematicConfig {
     private long discoveryTimeToLive = -1;
     private boolean unpairOnDeletion = false;
     private boolean factoryResetOnDeletion = false;
+    private int bufferSize = 2048;
 
     private HmGatewayInfo gatewayInfo;
 
@@ -381,6 +382,13 @@ public class HomematicConfig {
         }
     }
 
+    /**
+     * Returns the buffers size used for the communication with the Homematic gateway.
+     */
+    public int getBufferSize() {
+        return bufferSize;
+    }
+
     /**
      * Returns true, if the configured gatewayType is CCU.
      */

bundles/org.openhab.binding.homematic/src/main/java/org/openhab/binding/homematic/internal/communicator/AbstractHomematicGateway.java

@@ -730,9 +730,6 @@ public abstract class AbstractHomematicGateway implements RpcEventListener, Home
             logger.debug("Echo event detected, ignoring '{}'", dpInfo);
         } else {
             try {
-                if (connectionTrackerThread != null && dpInfo.isPong() && id.equals(newValue)) {
-                    connectionTrackerThread.pongReceived();
-                }
                 if (initialized) {
                     final HmDatapoint dp = getDatapoint(dpInfo);
                     HmDatapointConfig config = gatewayAdapter.getDatapointConfig(dp);
@@ -900,32 +897,17 @@ public abstract class AbstractHomematicGateway implements RpcEventListener, Home
      */
     private class ConnectionTrackerThread implements Runnable {
         private boolean connectionLost;
-        private boolean ping;
-        private boolean pong;
 
         @Override
         public void run() {
             try {
-                if (ping && !pong) {
+                ListBidcosInterfacesParser parser = getRpcClient(getDefaultInterface())
-                    handleInvalidConnection("No Pong received!");
+                        .listBidcosInterfaces(getDefaultInterface());
-                }
+                Integer dutyCycleRatio = parser.getDutyCycleRatio();
-
+                if (dutyCycleRatio != null) {
-                pong = false;
+                    gatewayAdapter.onDutyCycleRatioUpdate(dutyCycleRatio);
-                if (config.getGatewayInfo().isCCU1()) {
-                    // the CCU1 does not support the ping command, we need a workaround
-                    getRpcClient(getDefaultInterface()).listBidcosInterfaces(getDefaultInterface());
-                    // if there is no exception, connection is valid
-                    pongReceived();
-                } else {
-                    getRpcClient(getDefaultInterface()).ping(getDefaultInterface(), id);
-                }
-                ping = true;
-
-                try {
-                    updateDutyCycleRatio();
-                } catch (IOException e) {
-                    logger.debug("Could not read the duty cycle ratio: {}", e.getMessage());
                 }
+                connectionConfirmed();
             } catch (IOException ex) {
                 try {
                     handleInvalidConnection("IOException " + ex.getMessage());
@@ -935,21 +917,6 @@ public abstract class AbstractHomematicGateway implements RpcEventListener, Home
             }
         }
 
-        public void pongReceived() {
-            pong = true;
-            connectionConfirmed();
-        }
-
-        private void updateDutyCycleRatio() throws IOException {
-            ListBidcosInterfacesParser parser = getRpcClient(getDefaultInterface())
-                    .listBidcosInterfaces(getDefaultInterface());
-            Integer dutyCycleRatio = parser.getDutyCycleRatio();
-
-            if (dutyCycleRatio != null) {
-                gatewayAdapter.onDutyCycleRatioUpdate(dutyCycleRatio);
-            }
-        }
-
         private void connectionConfirmed() {
             if (connectionLost) {
                 connectionLost = false;
@@ -959,7 +926,6 @@ public abstract class AbstractHomematicGateway implements RpcEventListener, Home
         }
 
         private void handleInvalidConnection(String cause) throws IOException {
-            ping = false;
             if (!connectionLost) {
                 connectionLost = true;
                 logger.warn("Connection lost on gateway '{}', cause: \"{}\"", id, cause);

bundles/org.openhab.binding.homematic/src/main/java/org/openhab/binding/homematic/internal/communicator/client/RpcClient.java

@@ -88,7 +88,7 @@ public abstract class RpcClient<T> {
         request.addArg(getRpcCallbackUrl());
         request.addArg(clientId);
         if (config.getGatewayInfo().isHomegear()) {
-            request.addArg(new Integer(0x22));
+            request.addArg(Integer.valueOf(0x22));
         }
         sendMessage(config.getRpcPort(hmInterface), request);
     }

bundles/org.openhab.binding.homematic/src/main/java/org/openhab/binding/homematic/internal/communicator/client/XmlRpcClient.java

@@ -13,10 +13,7 @@
 package org.openhab.binding.homematic.internal.communicator.client;
 
 import java.io.ByteArrayInputStream;
-import java.io.ByteArrayOutputStream;
 import java.io.IOException;
-import java.io.InputStream;
-import java.io.UnsupportedEncodingException;
 import java.util.concurrent.ExecutionException;
 import java.util.concurrent.TimeUnit;
 import java.util.concurrent.TimeoutException;
@@ -24,12 +21,11 @@ import java.util.concurrent.TimeoutException;
 import javax.xml.parsers.ParserConfigurationException;
 
 import org.eclipse.jetty.client.HttpClient;
+import org.eclipse.jetty.client.api.ContentResponse;
 import org.eclipse.jetty.client.api.Request;
-import org.eclipse.jetty.client.api.Response;
 import org.eclipse.jetty.client.util.BytesContentProvider;
-import org.eclipse.jetty.client.util.InputStreamResponseListener;
+import org.eclipse.jetty.client.util.FutureResponseListener;
 import org.eclipse.jetty.http.HttpHeader;
-import org.eclipse.jetty.http.HttpStatus;
 import org.openhab.binding.homematic.internal.common.HomematicConfig;
 import org.openhab.binding.homematic.internal.communicator.message.RpcRequest;
 import org.openhab.binding.homematic.internal.communicator.message.XmlRpcRequest;
@@ -111,39 +107,18 @@ public class XmlRpcClient extends RpcClient<String> {
             }
             Request req = httpClient.POST(url).content(content).timeout(config.getTimeout(), TimeUnit.SECONDS)
                     .header(HttpHeader.CONTENT_TYPE, "text/xml;charset=" + config.getEncoding());
-            try {
+            FutureResponseListener listener = new FutureResponseListener(req, config.getBufferSize() * 1024);
-                ret = req.send().getContent();
+            req.send(listener);
-            } catch (IllegalArgumentException e) { // Returned buffer too large
+            ContentResponse response = listener.get(config.getTimeout(), TimeUnit.SECONDS);
-                logger.info("Blocking XmlRpcRequest failed: {}, trying non-blocking request", e.getMessage());
+            ret = response.getContent();
-                InputStreamResponseListener respListener = new InputStreamResponseListener();
+            if (ret == null || ret.length == 0) {
-                req.send(respListener);
+                throw new IOException("Received no data from the Homematic gateway");
-                Response resp = respListener.get(config.getTimeout(), TimeUnit.SECONDS);
-                ByteArrayOutputStream respData = new ByteArrayOutputStream(RESP_BUFFER_SIZE);
-
-                int httpStatus = resp.getStatus();
-                if (httpStatus == HttpStatus.OK_200) {
-                    byte[] recvBuffer = new byte[RESP_BUFFER_SIZE];
-                    try (InputStream input = respListener.getInputStream()) {
-                        while (true) {
-                            int read = input.read(recvBuffer);
-                            if (read == -1) {
-                                break;
-                            }
-                            respData.write(recvBuffer, 0, read);
-                        }
-                        ret = respData.toByteArray();
-                    }
-                } else {
-                    logger.warn("Non-blocking XmlRpcRequest failed, status code: {} / request: {}", httpStatus,
-                            request);
-                    resp.abort(new IOException());
-                }
             }
             if (logger.isTraceEnabled()) {
                 String result = new String(ret, config.getEncoding());
                 logger.trace("Client XmlRpcResponse (port {}):\n{}", port, result);
             }
-        } catch (UnsupportedEncodingException | ExecutionException | TimeoutException | InterruptedException e) {
+        } catch (InterruptedException | ExecutionException | TimeoutException | IllegalArgumentException e) {
             throw new IOException(e);
         }
         return ret;

bundles/org.openhab.binding.homematic/src/main/java/org/openhab/binding/homematic/internal/communicator/message/BinRpcMessage.java

@@ -221,7 +221,7 @@ public class BinRpcMessage implements RpcRequest<byte[]>, RpcResponse {
         int type = readInt();
         switch (type) {
             case 1:
-                return new Integer(readInt());
+                return Integer.valueOf(readInt());
             case 2:
                 return binRpcData[offset++] != 0 ? Boolean.TRUE : Boolean.FALSE;
             case 3:
@@ -235,7 +235,7 @@ public class BinRpcMessage implements RpcRequest<byte[]>, RpcResponse {
                 return new Date(readInt() * 1000);
             case 0xD1:
                 // Int64
-                return new Long(readInt64());
+                return Long.valueOf(readInt64());
             case 0x100:
                 // Array
                 int numElements = readInt();

bundles/org.openhab.binding.homematic/src/main/java/org/openhab/binding/homematic/internal/communicator/message/XmlRpcResponse.java

@@ -112,10 +112,10 @@ public class XmlRpcResponse implements RpcResponse {
                     break;
                 case "int":
                 case "i4":
-                    data.add(new Integer(currentValue));
+                    data.add(Integer.valueOf(currentValue));
                     break;
                 case "double":
-                    data.add(new Double(currentValue));
+                    data.add(Double.valueOf(currentValue));
                     break;
                 case "string":
                 case "name":

bundles/org.openhab.binding.homematic/src/main/resources/OH-INF/thing/bridge.xml

@@ -120,7 +120,12 @@
 				<advanced>true</advanced>
 				<default>false</default>
 			</parameter>
-
+			<parameter name="bufferSize" type="integer" min="0">
+				<label>Buffer Size</label>
+				<description>Size of the response buffer retrieved from the gateway (default 2048 kB)</description>
+				<default>2048</default>
+				<advanced>true</advanced>
+			</parameter>
 		</config-description>
 	</bridge-type>