Moteino via Serial to MQTT (RPi using simple gateway)

[steve]

papa: As I see it, this thread is about the "Holy Grail" of a Raspberry Pi Gateway or eliminating the Arduino Gateway.  See below for links to earlier & later discussions.

Hello!  I've been trawling the forum, and all over looking for someone who has already solved this problem.

I have some Moteinos. And think they are great. I also have a small openHAB server (running on a RPi) controlling a few lights and simple things. I would like to connect the Moteinos to openHAB in the simplest way. With two way traffic between openHAB and the Moteinos.

I have MQTT installed on the openHAB server (it's easy, and MQTT will also talk relatively easily to ESP8266 based nodes, if I ever get round to playing with those). I have a Moteino connected to the RPi GPIO header. The radio traffic from sensors is translated to serial in the Moteino with a simple gateway sketch. I can see this works on the RPi by monitoring the serial connection with minicom.

So I think a neat solution (and should be fairly straight forward!) would be to just read the serial input and send it to MQTT (and read other topics from MQTT and send to serial). I'm sure it can be done with a python script / daemon, or C program, but I can't seem to manage it. In fact I am sure it is possible for someone with a python background and time: I had a one way script for reading serial and writing to MQTT working. I was a fool and didn't back up the RPi sd card and you can guess what happened next... Corrupt SD card. So I'm back at square one - months later :-(

Can anyone please help with the script for the bridge? I imagine others will also be interested. If I'm the only one, then I'm probably on the wrong track, so hopefully someone will give me a gentle shove in the right direction...

I know Felix has spent quite a lot of effort on his gateway (which I have installed at one point). I can't see how to integrate this with an openHAB install, on the same box. Ie Moteino <> Gateway <> MQTT <> openHab. I don't want to have two servers (even if they are as low power as RPis) running for this (simple - oh it should be so simple!) translation task.

Lots of people use an arduino with RFM69 board (Moteino) with an ethernet shield to bridge the wireless to MQTT. While this will probably work, I don't see why I can't get the Moteino <> GPIO <> serial <> MQTT bridge working.

I also see there is a solution (https://github.com/computourist/RFM69-MQTT-client) that reads and writes via GPIO to a RFM69W and then on to MQTT (but not necessarily a RFM69HW). This seems quite active, and would be neat, but my Moteinos are RFM69HW, and the spare radio I have is HW...

Another (https://github.com/bbx10/nanohab) claims to work for Moteinos <> ESP8266 <> WiFi <> openHAB. I can't tell if this working or under development.
If someone knows if this can be made to work, I'd be happy to try this out!

This solution (https://github.com/Gussoh/serial-mqtt-bridge) seems to work in Java (but I haven't had a chance to work out what it is doing - does it need to be that complex to move data back and forth?)

Help?

[greginkansas]

I'm sure Papa will be along soon. But a few thoughts 1 RFm69 traffic is very time sensitive with a full OS you might miss some, putting this on a Arduino makes sense. 2 It has been for me good to split the system in two you can see with a MQTT client what is not working. 3 The RFM69HW and the W are the same they work together the HW just has more power.

[papa]

Steve, welcome to the DIY Home Automation forum. Much more than confirm greg's comment about RFM69W vs RFM69HW, what you ask is beyond me. Regarding RFM69HW vs RFM69W, I vaguely remembered (see links listed in a later post) that the H higher power version might require more power than a RPi GPIO can provide & may get erratic. I.E. the RFM69W might work better though with limited range.

From your "trawling the forum," you can probably tell I (& I believe computourist) am happy with & committed to the RFM Gateway <> RFM End Node approach with some computer (Windows or Linux, including RPi) hosting OpenHAB / MQTT. From what I've read here, I've seen people get encouraged or hopeful about a Pi Gateway system & then report a speed bump or grow quiet.  (PS computourist also likes the ESP wifi nodes not needing a gateway.)

Since I recently scanned all the DIY Home Automation threads & categorized them, I can offer the following links to RPi Gateway discussions. Maybe having these links in one place will help you be able to benefit from any useful nuggets on the forum. Perhaps you can personal message some of the writers to compare notes. Acekrystal seems to have gotten farther than anyone.

If you have further questions or make progress, we will be glad for you to publish it here. Best wishes.

    Here are the links, with more recently activity near the top:
Links involving Raspberry Pi as Gateway & OpenHAB / MQTT server

Raspberry Pi <-> RFM69 (Sensor Node Gateway)  << oops, added this big one on 3/29/2017

abouillot PiGateway interfacing

piGateway returns SIGSEGV, Segmentation fault

piGateway returns *** stack smashing detected ***

<solved> Getting -0.000000 readings</solved>

Need help for great opportunity ( >5 houses )

Raspberry Pi as Gateway

RFM69 Python Library for Raspberry Pi with Duplex Gateway

node blocked at radio.sendWithRetry(GATEWAYID.... line

[Question] GatewayRpi + RFM69 - abouillot

        As mentioned above I can say little about python, but I can search.  Did you see the following?

RFM2Pi gateway which mentions a python gateway script though in 2013

How about this Google Search?

    Especially this lowpower.com listing.

[ragoth]

Dear Papa, as regards to RFM69-W and RFM69-HW, the difference is in their RF out put power only and not in the control input provided by GPIOs I believe. Pls clarify.

[papa]

computourist, & others I'm sure, are more knowledgeable about power consumption matters. I was remembering some posts which I finally found & will list links below. Seems logical that if output power is higher that it takes more power to achieve that higher output. Also in my understanding, power draw on one pin (& its possible capability) is one concern. Another concern is that total draw on the whole electronic device can reduce what a pin can provide. In my experience, erratic things can happen when power draws hit an Arduino at the same time. I assume the same is true for a Pi. (which is running an operating system, OpenHAB, maybe a browser & other things) Off hand, I don't know if later models of Pis are more robust.

Here's two forum links about RFM69's power draw related to the Raspberry Pi.

computourist - RFM69HW power needs

gandalph

Here's some other links. In the first, see the graph near "current consumption of the RFM69 modules while sending with different power settings."

Figuring out power level settings of RFM69 (H)W modules

RFM69W and RFM69HW current consumption
        ^^ lowpower.com confirms the above, but current draw can be reduced by using Automatic Transmission Control.

[steve]

Thanks Papa, there's a lot to read there, and a few listings that I missed. I also noticed that there are several posts of partial success and then silence, although the last in your list (one on lowpower.com) seems promising. Thanks for taking the time to put this list together. I'll keep looking.

[papa]

You're welcome, Steve. Best wishes on finding solutions & success OR reasons things won't work. If you discover more either way, please report back here. Forum readers seem to revisit this RPi Gateway topic from time to time.

[steve]

I've had a go at writing the gateway. It serves my purpose (for now). It seems to work ok for me. I don't imagine too many others will be interested, but since I was asking for help, and got a few good pointers on the way, I have put the code on github. github.com/stevogray/SerialMQTTGateway/blob/master/SerialGateway.py

[papa]

Thanks for reporting back, Steve & letting us know what you got working & providing it. So far I'm content with Arduino compatibles as Gateways. However, your efforts may benefit others out there.

[papa]

I'm posting this message from radu so it might be included with previous related discussions & perhaps bring attention leading to success. In the next post, I'll respond from my perspective.

Aug 25, 2017 7:32:41 GMT -5 radudorurus said:
Hello.

I just registered.
I do not have a home automation system exactly but I made myself a heating
system automation for the wood stove + heat buffer + domestic water
heater. I am gathering information for making a sensor network in the house.
I have read an instructable on internet and ended on this forum.
Usually I start searching/reading the threads for the info I wish...
I do not know what is the status of the home automation project you developed.

Right now I think the gateway is still an Arduino which sends information over LAN. (I have read in one post)
Let's make the communication chain shorter if possible. Less pieces in the chain means more reliability.
If the LAN server fails there will be no DHCP so that wiznet5100 and OpenHAB computer will not communicate.
Sure we can assign fixed IP addresses and avoid this.
I am thinking to eliminate the necessity of the LAN component from sensor network.
Sometimes there are power outages (electrical company) or power failures (because of lightning strike, I got this 2 times). These events causes the LAN to go down if there is no UPS for the router/switch.
The sensors can be powered by batteries in the outage period, the Raspberry (OpenHAB host computer) or other SBC (I have an Odroid C2) also, but there will be no LAN between the Gateway and OpenHAB machine.


Can we get rid of this and attach the RFM69HW directly to the computer which has the OpenHAB installed?
The homeautomation will work if the router/server/switch fails. At least the logger and the automated tasks (if exist) does not depend of the LAN.
I know OpenHAB needs network/internet in order to be accessed from remote computer/phone.

If it is a windows/linux/mac PC, there are several usb gateways for SPI/I2C/serial/uart.
If it is a Raspberry Pi something similar, it has SPI on its GPIO port.

What do you think?
Is it possible to make the OpenHAB+network_sensor a LAN independent system?
Or is it worth to consider this variant since a OpenHAB without network connection has very limited use (scheduled actions, logger...).

Kindly, Radu

[papa]

Radu, thanks for questions & proposals related to using a Raspberry Pi as both OpenHAB-MQTT computer AND RFM69 Gateway.

Given your situation, I can see how you desire Home Automation communication that can work when your local network is down.

I encourage you to read the above posts in this thread & check out the links I provided. I believe what you are asking has come up off and on for at least two years. Forum members report some success & then hit snags or we don't hear from them again.

I agree that what you ask would be a great option, but I don't have documentation to achieve it other than the hints above. I believe one problem was that the Raspberry Pi pins did not always provide enough power for the RFM69 radio (at least the "H" higher power version) to communicate reliably.

Certainly this approach is beyond my programming ability (I'm truly a tinker & better, I believe, at adapting & documenting.) Besides, in my case, my power in a small U.S. city is very reliable. Also for me my network connected system of computer, Arduino gateway-nodes, & ESP82xx WiFi nodes works quite reliably.

Perhaps drawing attention to the above information will help you or someone else to make progress on your approach.

Forum members & guests, has anyone made progress on home automation using a Raspberry Pi Gateway?

[radudorurus]

Hi after doing some research on google, I reached back here!
homeautomation.proboards.com/thread/7/raspberry-rfm69-sensor-node-gateway

[papa]

Best wishes, Radu, on getting the Raspberry Pi Gateway to work. If you are successful, please post details on this forum so others can benefit.

Again, forum members & guests, has anyone made progress on home automation using a Raspberry Pi Gateway (A Raspberry Pi hosting OpenHAB-MQTT AND communicating with nodes via RFM69 radios)?

I did some more searching online.  This search for example.  In this search's results, this jeelabs series of posts, RFM69 on Raspberry Pi looks interesting.

    Also in that search, this entry from April 2017 also seems interesting.  That links to a MySensors page, Building a Raspberry Pi Gateway, created a year ago and updated two months ago.

One could perhaps use the above approaches as is.  (If so, please seek help at those sites, not here.)

These approaches could perhaps be adapted to this forum's approach.  If so, I would welcome the working details being posted here.