Troubleshooting MQTT

[papa]

        Needing to Troubleshoot MQTT

As I mentioned elsewhere, my test machine with Windows 10 & openHAB2.5.1 blew up & I had to reinstall Windows, openHAB, Arduino IDE, etc.

For initial installs, I continue to use PaperUI to start configuring openHAB & then add text configuration files of items, rules & sitemaps.

Other than it's not using the latest version, I believe this thread on installing openHAB & MQTT is still relevant. What that approach produced was still workable when I upgraded to openHAB2.5.0 stable & openHAB2.5.1 stable.

As this thread describes, I'm now installing/updating openHAB using very helpful scripts.

When I finished the above install, openHAB looked pretty good, but I had not tried MQTT yet. Then I discovered MQTT was not working, a MQTT device would connect to WiFi, but NOT to the MQTT broker. The following thread describes how I fixed various things to get MQTT communicating.

Some fixes relate to Windows 10 & I'll label them as such. Others relate to all operating systems.

        Next, Fixing Offline Embedded MQTT Broker #1

[papa]

        Fixing Offline Embedded MQTT Broker #1

At one point I noticed that the MQTT system/embedded broker was offline. Usually stopping & restarting openHAB fixes this, but not this time.

Oh, I realized, with the new Windows install, I did not open MQTT ports in Windows Defender Firewall.

So go to Control Panel\All Control Panel Items\Windows Defender Firewall & click "Advanced Settings."

    In the new resulting window, click on "Inbound Rules" then on "New Rule"

    Work thru the New Inbound Rule Wizard:
Rule Type: Port [Next]
   Protocol and Ports: TCP         Specific local ports: 1883 [Next]
Action: Allow the connection [Next]
Profile: UNcheck at least the box beside "Public" (Don't expose this to the world ! )
    Leave checked Domain or Private or both  [Next]
Name: MQTT [Finish]

Do the wizard again with the same information as just above except...
    local port: 8883 & Name: MQTT_2

Restart Windows & openHAB & see results: In my case, the above needed doing, but the embedded MQTT broker was still not back online.

        Next, Fixing Offline Embedded MQTT Broker #2

[papa]

    Fixing Offline Embedded MQTT Broker #2

I remembered that with an offline MQTT broker, Windows may be missing the JAVA_HOME environment variable.

Activate cmd.exe & in the console enter the command              where java
        For me that displayed    C:\Program Files\Zulu\zulu-8\bin\java.exe

For the JAVA_HOME value, we delete the path's last 2 sections & get C:\Program Files\Zulu\zulu-8

   Work out your equivalent path, one level above the \bin folder.

    Go to Control Panel\All Control Panel Items\System
At the end of the list on the left, click "Advanced System Settings"

    Near the lower right, click on [Environment variables...]
Scroll thru the bottom list of System variables, looking for JAVA_HOME & its value.
If JAVA_HOME is present, see if its value matches your equivalent of C:\Program Files\Zulu\zulu-8

        In my case, this is two folders higher than the folder holding the java.exe file.

    If the JAVA_HOME value is present but wrong, select it, click [edit], correct it, & [OK]

If the JAVA_HOME value is missing, click the lowest [New], complete variable name & variable value, & [OK] [OK]

Restart Windows & openHAB & see results: In my case, I needed to do the above, but the embedded MQTT broker was still OFFLINE.

        Next, Fixing Offline Embedded MQTT Broker #3

[papa]

    Fixing Offline Embedded MQTT Broker #3

I kept researching what else might get the MQTT broker ONLINE. I came across this site.

        For all operating systems:

In openHAB's PaperUI/Configuration/Things, click [trash can icon] to delete MQTT Broker.

    [We may also need to go to PaperUI/Add-ons/Misc to uninstall the broker.]

Then stop openHAB & delete the mqttembedded.bin file from Linux's /var/lib/openhab2/ folder or Windows' openHAB's /userdata folder (or papa: rename mqttembedded.bin)

    Restart openHAB.

Then in PaperUI/Add-ons/Misc, install again MQTT Broker Moquette.

    In PaperUI/Inbox, accept the MQTT Broker & click [Add Thing]

Restart openHAB & see results: In my case, the MQTT broker was ONLINE (yay), but my MQTT device was connected to WiFi, but NOT to the MQTT broker.

        Next, Clear the Cache

[papa]

        Clear the Cache (For All Operating Systems)

When asked about openHAB troubles, openHAB gurus often suggest two fixes: 1) Stop & restart openHAB.  2) Clear the Cache.

Stuff openHAB saves may not reflect recent changes until we clear the cache & let openHAB fill it again:

Stop openHAB.  Go to openHAB's /userdata folder & delete its subfolders: /cache & /tmp
    For Linux apt-get install: the userdata folder is at /var/lib/openhab

Restart openHAB & see results: (This restart takes longer.) Often this helps. However ...

In my case, the MQTT broker was ONLINE, but my MQTT device was connected to WiFi, but NOT to the MQTT broker.

        Next, Final Fix for My MQTT Troubles

[papa]

    Another Clue about My MQTT Troubles
        A Windows Feature ??

Much of the above probably added something to my working through MQTT troubles, but here are the last clues that brought results.

From time to time in troubleshooting MQTT, I entered the following command in a Windows cmd.exe window:
        netstat -a

Along the way from that command, I'd see that MQTT's main port (1883) was LISTENING, & I'd be encouraged, but my MQTT device still did not connect to the broker.

After a while, I noticed port 1883 was listening on the local host computer, but not listening on port 1883 at the MQTT broker's IP address. Hmmm, what could cause that?

I wondered, "Is my network is private as my Windows Firewall rule for MQTT's port requires?"

        Next, Final Fix for My MQTT Troubles

[papa]

        Final Fix for My MQTT Troubles
           A Windows Feature ??

Now we check our connected network & make sure it is private because we made a protective Firewall rule (see above) only allowing MQTT port 1883 connections thru the private network.  We probably need to be logged in with a Windows user account that has administrator privileges.

At one end of the Windows task bar see the notification area (by default, at screen's lower right). For the first step, do one of these: a) find the network icon (tiny monitor with cable for an Ethernet Network or radio waves for WiFi Network), right click the network icon, & left click "Open Network & Internet Settings" or b) At the far end of the notification area, see the Notifications icon (a squarish caption bubble), left click on that icon, left click on [Network] (you may need to click [All Settings] to see [Network])

Next, click on "Change connection properties." Under "Network Profile," if Public is selected (mine was), click the circle to the left of Private. Now we can close settings windows & our connected network should be private.

Indeed cmd.exe netstat -a shows port 1883 is listening on the ip address:port 1883. Moreover, when I press the reset button on my MQTT device once or twice, the device connects to both WiFi AND the MQTT broker.

Perhaps some of the above will help you with MQTT & other troubleshooting.

[papa]

        More Troubleshooting of MQTT Embedded Broker

On my Windows 10 system, sometimes I need to stop & restart openHAB. Likewise when Windows updates, it restarts itself & I have it set to restart openHAB. An issue with recent openHAB versions (including 2.5.3) is when openHAB is restarted, the embedded broker (AND all the related Things!) may be offline.

As noted in this post above, this may be helped by stopping OH, stop openHAB & deleting the mqttembedded.bin file from Linux's /var/lib/openhab2/ folder or Windows' openHAB's /userdata folder (or papa: rename mqttembedded.bin), & restarting OH. At best, this is an annoying nuisance. When Windows restarts, this may take our MQTT nodes offline until we fix it in the host computer.

Fix 1: Since I run OH with start.bat, a few new lines in start.bat can delete mqttembedded.bin (MQTT broker persistence), wait 5 seconds, & then start OH proper:

echo Launching the openHAB runtime...

echo Deleting mqttembedded.bin

del C:\openhab2.5.3\userdata\mqttembedded.bin

rem delay 5 secs for delete to work
PING localhost -n 6 >NUL

setlocal [file continues]

From this OH community post, possible fix 2 (I have not tried it yet):

Go to PaperUI\Configuration\Services\MQTT\  In MQTT Embedded Broker, click "CONFIGURE"

Delete mqttembedded.bin from the "Persistence file" blank. Save. Restart openHAB.

[papa]

        More Troubleshooting of MQTT Embedded Broker
                        (Windows)

Sometimes I install openHAB on Windows 10 as a service that runs automatically when Windows restarts. Other times I've put a link to my modified version of openHAB's start.bat (see post just above) in Window's Start menu folder. Using the Start menu folder reboots openHAB somewhat later than installing openHAB as a Windows service, but generally works. However, sometimes the Start menu method does NOT work because Windows updates are installed AND Windows offers another screen waiting for us to further configure Windows.

On one hand, installing openHAB as a Windows service avoids the Windows update problem. However, the openHAB service does not delete the mqttembedded.bin so that keeps the MQTT embedded from coming online. The following is my work around.

First, see here, configure openHAB as a Windows service AFTER openHAB is installed on Windows. Next run services.msc & scroll down the list to openHAB. Right click "openHAB"  Left click "properties"  In the drop down "startup type" list, choose "Automatic (Delayed Start)."  This will start openHAB when Windows reboots, but will do it somewhat later.

    In a folder where you can find it create the ~del mqtt persist.bat file with the following contents:

@echo off

echo Deleting mqttembedded.bin

del C:\openhab2.5.10\userdata\mqttembedded.bin

rem delay 5 secs for delete to work
PING localhost -n 6 >NUL

Schedule ~del mqtt persist.bat (giving its full path) as a task to run at system startup & choose the options:
    Run whether user logged on or not
        Run with highest privileges (you will need your Windows login password)

Through the above, when Windows starts, mqttembedded.bin persistence is deleted BEFORE the openHAB service's delayed start. This allows the MQTT broker & related things to come online & not stall.