A Beginner Makes Progress

[flyyboyy]

Hello, Papa - thanks for the details throughout this forum. . . I somehow stumbled my way here (I think it was from a link in Tsai's uber -able). I had been spending most time in sensors.org, though they lean very strongly toward the NRFs and I purposely had previously chosen 915Mhz RFMs instead of 2.4Ghz for a number of reasons. In the end, I was never able to get the RFMs to work with their sketches. Still hunting for RFM implementation, I found Tsai, then here -- Looks promising!

Here's what I have: RPi3B running openHABian 2, with MQTT installed and working. For the GW, I'm using a Pro Mini 3v3 (8Mhz), a W5100 mini, and an RFM69HW on 915. (I had originally planned on a 3v3 Uno clone and W5100 shield, though I could never get the Uno to be recognized by Windows; I have a replacement on order, has not yet arrived.) I may/may not go back to the Uno in the end, but spent yesterday trying to get the GW working with the parts that I already have on hand. I've set up reserved static IP addresses for both the Pi server and the planned GW within the router.

I followed along on your thread/280/rfm69-gateway-assemble-program page (among others), wired the modules, putting the RFM's NSS on D8, installed needed libraries, adjusted the code in the sketch RFM_MQTT_GW_25.1_pub1 for my hardware and added the cli() and sei() snippets in the W5100.h library.

Powered it up, loaded the sketch, waited anxiously for the Serial window to come alive with good news. . . and ***nothing***. Back to where I was with sensor.org. Troubleshooting ensued. Removed the radio, tried again, and had output to the serial window -- wow, progress. Though, in the end, had a constant string of "no link". I had good IP addresses on both the Pi and the W5100, could ping them, they show up in the router's device table.

I verified the code adjustments, poked around the forum looking for "no link" troubleshooting tips, and kinda gave up late last night. Other that having the RFM disconnected (should not need it at this point, right?), I'm just trying to get the GW to talk to the MQTT server on the Pi.

Still no joy this morning. I'm missing something somewhere, just unable to find the gremlin. Hints?

Thanks -- Lee

[papa]

Welcome, flyyboyy, to the joys & usual startup frustrations of DIY Home Automation. Using RFM69 radios enables a lot, but getting it going can be a challenge.

You did a good description of what you have tried. Let's see where we can go from there.

flyyboyy: "I have RPi3B running openHABian 2, with MQTT installed and working."

papa: Sounds like you have the MQTT binding installed in OpenHAB. BUT on the openahabian Pi, did you do the optional setup of Mosquitto, the MQTT broker service? Without Mosquitto service running on the Pi, the Gateway cannot link to the Pi nor to openhab.

Question: What 3.3 volt Uno clone did you try? Some have USB chip sets that are challenging to work with. The Buono Uno I recommend (for which I get no compensation) does well with connecting to Windows USB & provides decent power for the current-hungry RFM69HW.

Speaking of current-hungry RFM69HW, I don't believe the ProMini necessarily supplies enough 3.3 volt current to keep the RFM69 radio stable.

Again I use a Buono Uno switched to 3.3 volts & that's how I adapted the Gateway schematic I posted. Member computourist developed the basic schematic & coding for the Gateway & node. Computourist used the pro Mini, but added a power supply for the RFM69 radio. If you don't have that, the pro mini may be rebooting & preventing the linking. See his schematic of a pro mini gateway with the power supply in the upper left corner.

flyyboyy: "Other than having the RFM disconnected (should not need it at this point, right?)"

papa: I need to check the Gateway code about that.

[flyyboyy]

Thanks so much for the quick reply. . .

I rechecked mosquitto (actually reinstalled it to make sure: ssh > sudo openhabian-config > [select 20 | Optional Components > [select 23 | Mosquitto]; completed (re-)installation.

Still no joy from GW.

Have been wondering: are mqtt login credentials required from the GW? (There's so much that I don't know about MQTT/mosquitto yet). Have looked through the sketch but see no code other than ENCRYPTKEY for the radio.

Didn't think about the current needed by the RFM -- it would be drawn from the 3.3v bus that the Pro Mini does, not supplied by the Arduino itself, right?. Actually, wasn't at this point too concerned since I have a more immediate hurdle, though I was hoping to run them low power by commenting out the .setHighPower() line. Planning on running most battery-operated nodes with endurance in mind; still a topic for later.

So, still looking for solving the GW-MQTT connection issue.

Thanks again!

[greginkansas]

I have used some pro minis and they are ok. You can check the specs on the regulator to be sure.

[flyyboyy]

OK, managed to get them talking now - burned a new openHABian image and started over.

So, on to the next chapter, gotta get radio working; fortunately I didn't pull ALL my hair out yet, still have some for next hurdle.

Thanks, and TIA for future help

[papa]

flyyboyy, I'm glad you are keeping a sense of humor about this & that you are making some progress.

flyyboyy: "are mqtt login credentials required from the GW?"

papa: no

[flyyboyy]

Yup, never saw any code about it but was looking for ANYTHING that I might have missed and might be the cause. Siwwy wabbit!

I brought out my big brute bench supply, set it for 3.3v now have radio wired in, can get consistent mqtt connections. Set debug for radio but see nothing on the terminal. Looking at the code, I assume I wont until I have a node sending it stuff (?), so I guess that's the next project.

Thanks

[flyyboyy]

Built the barebones node, adjusted the sketch as needed, fired her up, and guess what???
No connection. . . Rechecked wiring, rechecked code adjustments, just not seeing the cause - looks like the same gremlin is back.

At this point, it could be either xcvr, right? or is there a way to narrow the conx issue to one or the other? Gonna save this for tmrw, try again with fresher mind

[papa]

flyyboyy: "Set debug for radio but see nothing on the terminal. Looking at the code, I assume I wont until I have a node sending it stuff (?)"

papa: That is correct.
=============================

Getting the first reliable communication among mosquitto service, gateway, & node is often maddening & as you say, "hair-pulling," but rewarding when it works.

flyyboyy, you said, you "added the cli() and sei() snippets in the W5100.h library."   In this post, did you see this other change to make in the Arduino library files:

"Added 1/1/2017: From what I see, for w5100.h changes to be used, also edit Ethernet.h, at your Windows (or comparable Linux) path of "C:\Program Files (x86)\Arduino\libraries\Ethernet\src

Line 5 of Ethernet.h is //#include "w5100.h" which means do NOT include w5100.h. So delete "//" at the start of line 5 to activate the include. Save the changed Ethernet.h file. P.S. pato found that for some IDE versions, to avoid an error, we may need to edit the Ethernet.h line to this:

#include "utility\w5100.h"

Changes to the TWO libraries are important to reliable communication between gateway & node.
===============================================

Please post here the IP address of the Pi that is running the mosquitto message service. Also post the customization changes you made in the gateway & node sketches. Maybe those will give some clues.

[flyyboyy]

May 31, 2018 13:00:54 GMT papa said:

[snip]

"Added 1/1/2017: From what I see, for w5100.h changes to be used, also edit Ethernet.h, at your Windows (or comparable Linux) path of "C:\Program Files (x86)\Arduino\libraries\Ethernet\src

Line 5 of Ethernet.h is //#include "w5100.h" which means do NOT include w5100.h. So delete "//" at the start of line 5 to activate the include. Save the changed Ethernet.h file. P.S. pato found that for some IDE versions, to avoid an error, we may need to edit the Ethernet.h line to this:

#include "utility\w5100.h"

Changes to the TWO libraries are important to reliable communication between gateway & node.
===============================================

Please post here the IP address of the Pi that is running the mosquitto message service. Also post the customization changes you made in the gateway & node sketches. Maybe those will give some clues.

Yes, did that early on (same time as the cli() and sei()) and also found that the sketch would not compile without including the "utility\"; found it necessary.

Have been reading others' experiences with the RFMs, now thinking that the antenna might be an issue. I'm working with the node and gateway only inches apart and assumed that a random length ant (using duponts; obviously not 1/4 wave and certainly not optimal) on each would at least let each communicate at such a short distance. Others' experiences might indicate that a high VSWR from the mismatched antenna might cause the radio to simply "shut off" the xmtr. So, I'm gonna remove the pin and dupont and solder a true 1/4 wave to the circuit board pad. Stay tuned. . .

[flyyboyy]

Node is phoning home now. Thought it may have been antenna mismatch (definitely, but don't think that was the problem). Started over with two new RFMs (just in case I screwed one or both of them), hard wired them, (soldered to each board rather than pins & duponts) - still no joy. Rechecked code adjustments, still couldn't find problem. Went back to both sketches, returned the ENCRYPTKEY back to all xs, and now it's gold.

In the end, I think the issue was not the key (they really DID match, but I wanted to remove one more possible cause nevertheless) - the breadboards have from the start periodically proven kinda flaky. I think that once I get everything on a real board and hard-wired, most gremlins will go away.

On to adding sensors to the node. Think I'm gonna start with DHT (I have 22s, not 11s), but haven't looked if defines need to be changed, or any other code. I want to start learning how they show up in openhabian.


HAPPY DAYS!

[papa]

flyyboyy, good to hear that you are making progress.

flyboyy: "On to adding sensors to the node. Think I'm gonna start with DHT (I have 22s, not 11s), but haven't looked if defines need to be changed, or any other code"

papa: When looking for information, I encourage you to use the search box near the upper right (above "Last Post") when you are in a board. This search gave several possibilities including these two threads with DHT22 in their name & conversation about using them:

homeautomation.proboards.com/thread/8/dht-sensors-various-mhz-speeds

homeautomation.proboards.com/thread/60/dht22-humidity-temperature-sensor

Near the search box, you can click on the magnifying glass icon to refine your search.

[flyyboyy]

OK, thanks for the input on Gateways & end nodes in this post. I had installed Gateway 25.1 last spring, but had) to focus time/attention elsewhere til recently. I had built the GW on perf protoboard since I didn't think I'd draw up a board for a single use. I'M using the HW radio and a 3.3V Pro Mini with a Mini W5100 also set for 3.3V. I built a small on-board PS (used a 317 since that's what I had on hand) to tame and regulate the 5V down to 3.3. Both modules are mounted vertically, facing each other, with the serial terminals on the bottom of the board so that I could attach the FTDI when needed.

25.1 has been solid, though never was able to devote time to actual end nodes (multi-function "bare bones" was built and used for setup and testing). Now I'm back in gear for making it do something, thinking out end node design and construction.

So here's a couple more Qs:

1 - I'd like to design a "universal" PCB that would have open pads connected to I/O pins to be used for wiring external components for the nodes I'm interested in mostly sensors: door/window, temp/humidity, gas and flame, photo/light, leak, PIR. . . You know, one PCB fits all.

Relay control for LOTSA stuff!

I've casually looked through the choose_nodes sketches for devoted pins and it seems that the following are purposed as:

    D3 - PIR; Hall effect xducers; also D3 - change state for mail

    D4 - DHT

    D5 - switch contacts; relay control

    D6 - switch contacts; relay control

    D7 - switch contacts; relay control

    D8 - relay control, servo swing

    D9 - relay control

Are there other uses for these pins (or other pins that I missed)? Likely! Gotta study the code more.

2- Any analog pins used? Haven't noticed any so far. If so, for what???

3 - What other magic can this box do that I'm not seeing here? Servos for what???

Other projects for the future, like sprinkler controllers, timers and control for pool equipment, LED "mood lighting, etc as time will allow.

_____

Also trying to make sense of the needed openHAB files. Have general understanding, but haven't found enough examples to know what the .items, .sitemaps, etc files are supposed to look like. Talk about a steep learning curve (at least for this old guy)!

Thanks for your help

(It's talking to MQTT: I'm beside myself)

[papa]

You are welcome, flyyboyy. Thanks for sharing what you have done & what you hope to do.  It's rewarding to hear your good start & your excitement for more.  I'd love to hear more on what you try & what you learn.

flyboyy: ""universal" PCB that would have open pads"

papa: As I've mentioned, I use the Buono Uno compatible (switched to 3.3 volts). For the Gateway, I add W5100 board. For most end nodes I use a prototype board that has like a mini solderless breadboard near its center. I hack the prototype board to add more 3.3 volt connections & add more sockets as needed for digital or analog connections. I use dupont cables to connect to the sockets on the W5100 board & hacked prototype board. If the Buono Uno stops taking programming, I move the top board to a new one. These function as nearly universal boards for me.

Start here to see about a custom board brump has been working on.

flyyboyy: "choose_nodes sketch's devoted pins?"

papa: Yes, several pins (digital & analog) are "assigned" as the sketch stands now. Start looking near the end of the latest choose_nodes sketch (found here), like about line 3471 for "assigned" pins relating to each device.

However, for the most part you are NOT locked in to using those pin assignments. Only two pins can wake a sleeping Arduino, D2 & D3, & D2 is reserved for the RFM69 radio. So if you want to wake a sleeping Arduino (as for the mailbox projects) or some other interrupt-driven function, D3 must be reserved for that. Otherwise, you can pretty much assign pins as you want, as long as your assignments don't interfere with each other.  To see where the assignments are made (& could be changed), search in the sketch for the #define name of the desired node option.  Pin assignments are usually found in the sketch's initial variable section or void setup() section. (Of course, node wiring would change as well)

        papa's responses continue in the next post

[papa]

flyyboyy: "What other magic?"

papa: For many possible RFM69 projects, see here. When you want relay control, but don't need sensors on the same board, I encourage you to look at the ESP8266 WiFi projects (no Gateway needed).

flyyboyy: "trying to make sense of the needed openHAB files"

papa: Look in this list for my projects. They usually include not only suggested parts, how to build & program the node (& expected Serial Monitor results), but also related OpenHAB config entries. Often with the OpenHAB configs, you only need to change the NODEID to the one you want to use. Also start here for a tutorial I wrote on interpreting OpenHAB text config entries.

PS I'm considering a new sketch function that would output sample OpenHAB config entries based on the node options chosen. One could copy them from the Serial Monitor & paste them into an OpenHAB .item or .sitemap file Do you think that would be helpful for those starting with this DIY Home Automation?