Migrating to OpenHAB 3 (with MQTT) from OH 2.5.x

[papa]

        MQTT Preliminaries

You must first Install & Configure Mosquitto (MQTT communication) as shown in two posts above starting here.

In the browser, go to localhost:8080/settings/things/    At the lower right, click the blue plus (+) sign to Add a Thing.  Add MQTT Broker.

Click on the MQTT Broker listed to reach its configuration. You may fill the optional Location field, perhaps with Network. Importantly, fill the Broker Hostname/IP field with the IP address of your openHAB/Mosquitto computer.  Click Create Thing & hope to see a green Online button.

        More Time-Consuming Prelimaries

You must have MQTT devices for openHAB/Mosquitto to communicate with.  This forum provides two options:

        (See this thread for an overview of the two device options.)

1) RFM69 Gateway & Node Network  (See here for instructions)    AND/OR

       See this thread where RFM69 instructions are being updated for OH 3.

2) ESPxxxx WiFi Nodes   (See this board for possibilities.)

        Next: Another MQTT Preliminary

[papa]

        Another MQTT Preliminary

    In the browser, go to localhost:8080/settings/

Near the bottom of the right hand column of settings, click System MQTT Broker

        Make sure of the settings in three fields:
Broker host [IP address of your openHAB/Mosquitto computer]

Secure Connection?   [off]             Hostname Validated?   [off]

If you changed any fields' contents, Click Save at the upper right

        Next: Create a Thing for an RFM69 Node

[papa]

            Create a Thing for an RFM69 Node

The following assumes how I built & programmed 1) a RFM69 Gateway & 2) a RFM69 Node (Node05)  (See this thread,)

        This is how I created a Node05 Thing that will help communicate with the Node.

At the bottom right of localhost:8080/settings/things/   I clicked the + icon in a blue disc (Add Thing)

Click MQTT Binding.     Click Generic MQTT Thing

Leave these fields as is:   Unique ID & Identifier

For Label, I changed "Generic MQTT Thing" to Node05

For Location, I put BaseFR for my Basement Family Room.
        Use your location name or leave blank.

Click the Bridge field & select MQTT Broker.    Click [Create Thing]

Hope to see a [green Online button].

        Next: Create Channels for an RFM69 Node

[papa]

        Create Channels for an RFM69 Node Named Node05

                Introduction

With proper setup, computourist-style nodes exchange types of data & commands with openHAB/Mosquitto. In the computourist-style programming, the "types" are called devices. In openHAB, the types are called channels. For each node, we can create channels that match the devices that are or might be activated on the node.

Via the default node sketch, Node 05 publishes devices 2 & 4. In this post, #define HT programs Node 05 to publish devices 48 & 49. So in the Node05 Thing, we could create (state reporting) channels 2 (RSSI, radio signal strength), 4 (node voltage), 48 (temperature), & 49 (humidity). To show how, I will also create channel 16 (a channel that can report a ACTOR state AND can receive a command). Channel 16 is not usable until hardware & software changes to Node 05.

[papa]

            Create Node05 Channel for Device 2 (RSSI)

Logged in to your openHAB configuration, go to localhost:8080/settings/things/

In the listed Things, click Node05. Then click the [channels tab] near the top.

Near the bottom, click Add Channel.

        In the Add Channel Window, fill the fields with configs:

Channel identifier Node05Channel02
    ^^ arbitrary ? as long as we match it in a linked Item (see below)

Label Node05_RSSI
    ^^ arbitrary ?, make it useful for yourself, but not too long

Channel type Text Value
   ^^ choose type compatible with the device/channel data type

MQTT State Topic home/rfm_gw/nb/node05/dev02
MQTT Command Topic [leave blank]

        Click [Create] to save your configs

            Next: Create Item to Link to Node05's Channel 02

[papa]

        Create Item to Link to Node05's Channel 02

In openHAB's items folder (Linux   /etc/openhab/items), have an .items file to hold the Node05 Items. Mine is called Basement.items, but the name before the .items tag is up to you.

        The RSSI (channel 02) Item takes the form:
String Node05RSSI "Node05rssi (db)[%s]" <network> (Nodes) { channel="Node05channelUID" }

Go to localhost:8080/settings/things/
    Click on the Listed Node05 Thing        Click on the [channels] tab
After the Node05_RSSI channel is a small stacked pages icon.
    Click on that icon to copy the channel UID

       Into the Item, paste that channel UID in place of Node05channelUID color-coded above.

For example, my channel's UID is
       mqtt:topic:c08f02ca8d:b3a7c9ed64:Node05Channel02

        So my Item (your channel UID for the Item will be different) is
String Node05RSSI "Node05rssi (db)[%s]" <network> (Nodes) { channel="mqtt:topic:c08f02ca8d:b3a7c9ed64:Node05Channel02" }

       The above is somewhat involved, but the rest of Node05 channel links are almost the same, except the last 2 characters.

For example, my Item for Node05 voltage (device 04) ends with the following channel link:
     { channel="mqtt:topic:c08f02ca8d:b3a7c9ed64:Node05Channel04" }

            Next: Display Node05's RSSI Data on HABPanel
   

[papa]

        Display Node05's RSSI Data on HABPanel

Adapting from two posts starting here, you can display Node05's RSSI data on a HABPanel Dashboard of your choice.

Above, to handle Node05's RSSI data, we configured a Text Value Channel, a String type Item, & used a Dummy type widget to display on HABPanel. That can be used for other data. Below I illustrate other types: Switch & Number.

        Next: Configure a Switch Type of Channel

[papa]

        Configure a Switch Type of Channel

If you built & configured Node05 as a full DHT end node, it publishes the (on/off) status of the ACTOR (node device 16). OpenHAB can also command the ACTOR to toggle On or Off.

            Create Node05 Channel for Device 16 (ACTOR)

Logged in to your openHAB configuration, go to localhost:8080/settings/things/

In the listed Things, click Node05. Then click the [channels tab] near the top.

Near the bottom, click Add Channel.

        In the Add Channel Window, fill the fields with configs:

Channel identifier Node05Channel16
    ^^ arbitrary ? as long as we match it in a linked Item (see below)

Label Node05_Switch
    ^^ arbitrary ?, make it useful for yourself, but not too long

Channel type On/Off Switch
   ^^ choose type compatible with the device/channel data type

MQTT State Topic home/rfm_gw/nb/node05/dev16
MQTT Command Topic home/rfm_gw/sb/node05/dev16

        Click [Create] to save your configs

            Next: Create Item to Link to Node05's Channel 16

[papa]

        Create Item to Link to Node05's Channel 16

In an .items file, create an item entry based on the following. However, replace my channel UID with your channel UID determining it like you did in this post above.

Switch Act_Node05 "Node05switch" { channel="mqtt:topic:c08f02ca8d:b3a7c9ed64:Node05Channel16" }

To display on HABPanel, use a Switch widget.

        Next: Configure a Number Type of Channel

[papa]

            Configure a Number Type of Channel

If you built & configured Node05 as a full DHT end node, it publishes the status of DHT temperature (node device 48).

            Create Node05 Channel for Device 48 (Temperature)

Logged in to your openHAB configuration, go to localhost:8080/settings/things/

In the listed Things, click Node05. Then click the [channels tab] near the top.

Near the bottom, click Add Channel.

        In the Add Channel Window, fill the fields with configs:

Channel identifier Node05Channel48
    ^^ arbitrary ? as long as we match it in a linked Item (see below)

Label Node05_DHTtemp
    ^^ arbitrary ?, make it useful for yourself, but not too long

Channel type Number Value
   ^^ choose type compatible with the device/channel data type

MQTT State Topic home/rfm_gw/nb/node05/dev48
MQTT Command Topic [Leave blank]

        Click [Create] to save your configs

            Next: Create Node05 Channel for Device 49 (Humidity)

[papa]

        Create Node05 Channel for Device 49 (Humidity)

A Node05 with DHT sensor also publishes air humidity (per cent). Configure its channel like the Temperature channel.

        In the Add Channel Window, fill the fields with configs:

Channel identifier Node05Channel49
    ^^ arbitrary ? as long as we match it in a linked Item (see below)

Label Node05_DHThumidity
    ^^ arbitrary ?, make it useful for yourself, but not too long

Channel type Number Value
   ^^ choose type compatible with the device/channel data type

MQTT State Topic home/rfm_gw/nb/node05/dev49
MQTT Command Topic [Leave blank]

        Click [Create] to save your configs

            Next: Create Items to Link to Node05's Channel 48 & 49

[papa]

        Create Items to Link to Node05's Channel 48 & 49

In an .items file, create items based on the following. However, replace my channel UID with your channel UID determining it like you did in this post above.

Number Temperature_Node05 "Temp_BaseFR05 [%.1f °F]" <temperature> { channel="mqtt:topic:c08f02ca8d:b3a7c9ed64:Node05Channel48" }

Number Humidity_Node05 "Humidity_BaseFR05 [%d %%]" <humidity> { channel="mqtt:topic:c08f02ca8d:b3a7c9ed64:Node05Channel49" }

To display on HABPanel, use Dummy widgets.

        Next: Migrating HABPanel Configs from OH 2.5.x to OH 3
                Manual Intervention Needed ! ! !

[papa]

        Migrating HABPanel Configs from OH 2.5.x to OH 3
                Manual Intervention # 1 ! ! !

Earlier in this thread, I said that my OH 2.5.x computer crashed. I had a backup .zip file, but my extensive HABPanel config did not restore to OH 3. Off & on, I looked for how to accomplish the HABPanel migration. Today I learned that the easiest HABPanel migration requires a working OH 2. I found this here:

HABPanel's storage back end had to change [for OH 3] & is not compatible [with OH 2.5.x]. Go to the panel settings on a [running] OH2 instance, click on “Edit the local panel configuration (experts only)” under “Local storage”. Then copy the code or export it with the button. Go to the same screen on your [running] OH3 instance, and paste or import. Then on the previous screen, save it under a new name with “Save the current configuration to a new panel configuration”. Repeat for each configuration you want to migrate.

            I also found these hints here:

In [running] OH2, select the Radio Button next to Local Storage in the HabPanel SETTINGS Page. Then below the Local Storage item it says “Edit the local panel configuration (experts only)”. Click on that & be taken to a page that contains the HabPanel definitions. In the upper Left are Export/Import Buttons. Use them create a file to export HabPanel configs (from running OH 2) & import (into running OH 3). [See illustration below.]

       << Members, click on pic for larger view.

            For OH 3, you can go to
http://localhost:8080/habpanel/index.html#/settings/localconfig

    Disclosure: Not having a working OH 2.5.x, I could not try the above. However, it looks like it should work.

        Next: HABPanel, OH 2.5.x to OH 3, Manual Intervention #2

[papa]

                HABPanel, OH 2.5.x to OH 3, Manual Intervention #2
            Recreate OH 3 HABPanel Config from OH 2.5.x Backup.zip

Perhaps like me you have no working OH 2.5.x to export HABPanel to OH 3. What then? The following presupposes a Backup.zip from OH 2.5.x

My OH 2.5.x Backup.zip has the file /userdata/config/org/openhab/habpanel.config. I suggest you copy your file where you can safely see & edit it.

Search through the habpanel.config file for "panel_name" (you may have more than one). A few characters after each "panel_name" see a panel's name in two forms (which may be the same). The first form gives a go to reference & the second is a display name. I suggest you insert a two blank lines before each "panel_name" to section off each panel's configs.

In the following snips from my habpanel.config, I use the ellipsis (...) to represent multiple forward slashes (\) & other unused characters. Here's the start of one panels in my habpanel.config:

"panel_name\":\ \"PCken\"..."MotoKen\"...

PCken is a go to reference & MotoKen is a display name for the panel.

In OH 3's HABPanel, I can use this ^^ information to restore a panel I had in OH 2.5.x

        Next: Recreate a Panel's Dashboards

[papa]

        Recreate OH 3 HABPanel Config from OH 2.5.x Backup.zip
                        Recreate a Panel's Dashboards

As said in the post just above, this identifies a panel:

"panel_name\":\ \"PCken\"..."MotoKen\"...

                After that section ^^ , see "dashboards\"

Then see a series of run-on sections that start with "id\":\ \"xxxxxx\"..."name\":\ \"xxxxxx\"
As with panel name, the first xxxxxx stands for a go to reference & the second stands for the dashboard's display name. An example from my habpanel.config:

... dashboards\"..."id\":\ \"Cooker\"..."name\":\ \"Cooker\"

In OH 3's HABPanel, I can use this ^^ information to restore a dashboard I had in OH 2.5.x

        Next: Recreate a Dashboard's Widget's

[papa]

        Recreate OH 3 HABPanel Config from OH 2.5.x Backup.zip
                    Recreate a Dashboard's Widget's

So far above we've seen a section of my OH 2.5.x habpanel.config file that says (... represents unused characters):

"panel_name\":\ \"PCken\"..."MotoKen\"..."dashboards\"..."id\":\ \"Cooker\"..."name\":\ \"Cooker\"...

After that, see "widgets" & the listed configs for the widgets of the Cooker dashboard. My example:

"widgets\":\ ..."name\":\ \"Cookr18rssi\"..."sizeX\":\ 2,\..."sizeY\":\ 1..."item\":\ \"Node18RSSI\"..."type\":\ \"dummy\"..."row\":\ 1..."col\":\ 4... etc. through the Cooker dashboard widgets.

The first widget named (Cookr18rssi) is 2 units horizontal & 1 unit vertical. That widget displays data from the (properly created & channel-linked) Item (Node18RSSI). The widget type is (dummy). The widget is in row 1 (second from the dashboard's top) & column 4 (5th from the left). The next Cooker widgets follow in the same pattern.

In OH 3's HABPanel, I can use this ^^ information to restore a dashboard's widgets that I had in OH 2.5.x

        Next: Migrating OH 2.5.x Text File Rules to OH 3

[papa]

        Migrating OH 2.5.x Text File Rules to OH 3

Early in this thread, I wondered if OH 2 to OH 3 migration might take significant work to update rules' text file configuration.

At this point, I've activated only a few rules which seem to run with no problems. I will continue to test a few rules at a time.

Starting here, the official openHAB docs give extensive documentation & examples on text file configuration for rules.  If I get rule errors, I will check the logs & official rule documentation for hints on correcting my text file configurations.

    Next: Evaluating Migration from OH 2.5.x to OH 3

[papa]

        Migrating to OpenHAB 3, Looking Back

I've done most of the migrating & will now reflect on the experience.

Running the OH 3 restore script using a backup.zip from OH 2.5.x did not work for me, either with Windows or Linux. So I did a fresh install of OH 3 & then manually used OH 2.5.x configs from backup.zip, especially org.eclipse.smarthome.core.thing.Thing.json, habpanel.config, habpanel-config.json, .items, .rules. /transform folder configs

I had to install Java 11 instead of Java 8.

Installing Things & channels took much clicking & field filling. At least I had that info from OH 2.5.x's backup.zip to use manually

OH 3 handles Thing/channel UIDs differently so I had to edit Items files accordingly. I could often copy UIDs between channels of the same Thing.

On OH 3, my .rules files from OH 2.5.x seem to run with no problems.

Compared to OH 2.5.x PaperUI configuring, OH 3 Dashoard's configuring worked much the same except using somewhat different browser addresses.

I hope upgrading within OH 3 will now be easier because the backup & restore scripts should work better.

[papa]

            Once Again I Remind You       
          Regularly Backup Your OH Work

When I upgraded OH or had to rebuild OH, my backups either made a smooth, quick transition or saved from much reinventing.

    In the Linux Terminal app, execute          

sudo $OPENHAB_RUNTIME/bin/backup

The backup script stores your OH configuration in a time-stamped zip file which is saved in /var/lib/openhab/backups for automatic installs & openhab/backups for manual installs.

For safety & upgrading convenience, do backups regularly.

Not only do backups on your OH computer's storage. ALSO (in case your OH computer crashes like mine) backup your backups to a thumb drive or external hard drive

        Next: OpenHAB Command Line Functions (Linux)

[papa]

        OpenHAB Command Line Functions (Linux)

            Usage: openhab-cli command [options]

            Possible commands:
backup [--full] [filename] -- Stores the current configuration of openHAB.
clean-cache -- Cleans the openHAB temporary folders.
console -- Opens the openHAB console.
info -- Displays distribution information.
reset-ownership -- Gives openHAB control of its own directories.
restore filename -- Restores the openHAB configuration from a backup.
showlogs -- Displays the log messages of openHAB.
start [--debug] -- Starts openHAB in the terminal.
status -- Checks to see if openHAB is running.
stop -- Stops any running instance of openHAB

This list has useful access to certain openHAB related commands to execute in the Linux Terminal.

For example, "openhab-cli showlogs" (in real time) displays the new posts to openHAB's /var/log/openhab/events.log & /var/log/openhab/openhab.log.  events.log displays updates in item states. openhab.log displays initializations, loadings of items & rules files, warnings, & errors. "[Ctrl] c" will stop the display & return to a terminal prompt.]

The logs display is one way to monitor openHAB & catch errors. Multiple errors & warnings can be confusing to watch so opening the logs files works better then.

        Ahead: Another Entry into OpenHAB Command Functions

[papa]

        Another Entry into OpenHAB Command Functions
                (Linux Terminal)

At the Terminal prompt, enter:   openhab-cli console

Logging in as openhab
Password:                     [blind enter: habopen]

Use '<tab>' for a list of available commands  [303 !!]
and '[cmd] --help' for help on a specific command.
To exit, use '<ctrl-d>' or 'logout'

Enter: "log:tail" for an interleaved, color-coded display of the real-time contents of  events.log & openhab.log. To stop the rolling logs display, enter [Ctrl] c

This way to monitor openHAB items states, warnings, & errors is easier to read.

        Next: Make OpenHAB Automatically Start When the Linux Host Boots

[papa]

        Make OpenHAB Automatically Start When the Linux Host Boots

For Systems based on systemd (e.g. Debian 8, Ubuntu 15.x, Raspbian Jessie and newer):
         (commands for other systems near the above link)

sudo systemctl daemon-reload
sudo systemctl enable openhab.service

[papa]

            Save / Backup Your HABPanel Configuration in a .json file

Have a file to reproduce settings you created for HABPanel.  It can be used to restore those settings to your current machine or to transfer to another machine.

Go to http://localhost:8080/habpanel/#/

At the top left of the HABPanel screen, click on the "stacked pages" icon.  Then at the bottom, click on the gear icon.

   << Click on Edit local panel configuration.

     << Click on the Export button.

This will give you the opportunity to save the file habpanel-config.json. I recommend that you add a date to the file name. Save to a location you can find.

To bring this HABPanel config into your current machine or another, use the Import button & navigate to where you stored the .json file.