Newbie after some advice for initial build

[lhw455]

Hi all, newbie here, I wanted to some advice to make sure I'm buying the right stuff



I figured I'd start with the garage door open/closed sensor as this is something that I often do i.e. leave the garage door open!



So the initial parts list:



1) For the sensor, Ultrasonic distance sensor and BUONO UNO R3 ( Arduino compatible ) and prototype shield and an RFM69HW

2) For the wireless gateway, a BUONO UNO R3 ( Arduino compatible ) and a RFM69HW

3) For the etherne gateway, a BUONO UNO R3 ( Arduino compatible ) and Wiznet 5100 ethernet shield





Also, some 22 gauge wire? I assume I'll need some solder and a soldering iron?



Also, per hackaday.io/post/4337, the original author mentions that you can use the "Arduino clone with built in RFM69HW module" from

lowpowerlab.com/shop/index.php?_route_=moteino-r4 - if I go that route, does it mean that I need one Moteino for the gateway and one for the ethernet?

If so, what options do I need on that page i.e. 433 HW or 868 HW? Do I need any of the other options?



Notes/questions:



1) I have a Rpi2 all setup and ready to go

2) Can I combine the Ardunio ethernet gateway/wireless gateway?

3) I read on this forum that the Arduino's can be eliminated and instead used in conjunction with the Rpi, is this currently a viable solution?

Thanks

[lhw455]

Oh, and perhaps I'm being dumb, but how do I power each of the units?

[graham]

Some info on power:
homeautomation.proboards.com/thread/35/question-powering-buono-uno-r3

Where to start:
homeautomation.proboards.com/thread/50/get-started

Welcome to the forums
I'm only just starting as well and I am pretty sure that: 2) is a yes and 3) is a not working 100% so more no than yes.

[lhw455]

Thanks, went on an ordering spree, just bought 3 BUONO UNO R3's, I figured if I can use 1 instead of 2 for the gateway/ethernet, then I can just use the other for a node. On power, is there a battery option? That thread mentions AC adaptors, ideally I want them to run on some sort of battery setup.

[papa]

Greetings Ihw455 & welcome to the DIY Home Automation Party.

When you said, "I figured I'd start with the garage door open/closed sensor," it sounds like you might have read my "Success ... post on this forum at homeautomation.proboards.com/thread/62/success-home-automation-arduinos-rfm69s

Starting with hardware, I'm amid the effort of detailed documentation of what I summarized there. I aim to post it here a section at a time. I've gotten a little beyond newbie, but remember how much I did need to know & still do.

In the meantime, some responses to your statements & questions:

Yes, you'll need tools like soldering iron with small tip, solder, 22 gauge wire (some different colors help) & a tool to strip it's insulation, a Helping Hand is helpful for soldering, etc.

You say "I have a Rpi2 all setup and ready to go." I assume that means you have installed OpenHAB & what it needs on the Rpi. For this (like the following suggestions) to work, you'll need to create / edit OpenHAB configuration files (especially Items, Sitemap, & Rules)

I cannot help with specific Rpi questions. As my "Success..." post said, I run OpenHAB on Windows 7.

Power: I use cheap 9 volt 1amp adapters from China that plug into the Arduino's power connector. My Buono Unos came with a usb cable for programming, but for power, one can plug things with a usb port: a computer, a spare phone charger, ports on some power strips.

Batteries & battery backup: Not sure about with Rpi, but with Arduinos & Windows on a laptop, I don't need battery backup. Windows startup also reboots OpenHAB & with power restored, Arduinos start right up.

Battery power could be handy when wanting to put a node somewhere without an AC outlet or where one wants to avoid stringing wires. Eric Tsai's original Uber Home Automation pictured a battery powered node built on strip board. It's a version of a Bare Bones Arduino. Especially for a newbie, I recommend using something better than a strip board. Also using the DHT end node below (which tests & reports voltage), you can eliminate the voltage testing parts (the two resistors & two capacitors just right of the battery connectors). Go to www.instructables.com/id/Uber-Home-Automation-w-Arduino-Pi/?ALLSTEPS & search for "stripboard wiring diagram" To minimize power usage, these have no usb connection to program them. One needs to use an Arduino with a programming sketch & suitable parts & wiring. As I said in "Success..." I adapted Eric's strip board version to make a battery powered DHT end node. I hope to document that eventually.

Using Moteinos: I'm not sure. I saw vague references to them along the way, seems like they worked for some people. I've no experience with them. A quick look at the low power listing shows 250 milliamps at 3.3 volts vs 300 milliamps for Buono so one might reach current limits sooner. Supposed to be battery friendly. Looks like a Moteino with 915MHZ HW transceiver (H means higher power, supposed to communicate farther) costs near $20 plus $15 for one FTDI programming device. Different construction process: No need to solder 8 wires to transceiver & plug or solder other end of 7 wires, but header pins to solder which would plug in to a solderless breadboard. Bullet point about shields so maybe that could provide a more finished product.

1) The wireless/ethernet gateway is consolidated on one Arduino with a W5100 Ethernet Shield on top. Download & extract github.com/computourist/RFM69-MQTT-client/archive/master.zip Read & refer to "[path to where you extracted master.zip]\RFM69-MQTT-client-master\Gateway_2.1\RFM_MQTT_gateway_description_V21.pdf In the folder [your path]\RFM69-MQTT-client-master\Gateway_2.2, see the Gateway schematic & Arduino sketch. One can assemble the schematic on the Ethernet Shield. Read thru & edit sketch lines that need your specific info & choices.

2) You can build two regular nodes with your other parts. I recommend you start with [your path]\RFM69-MQTT-client-master\DHT end node That node uses a DHT11 Temperature & Humidity Sensor (eBay sells them) & reports to the Gateway. It also needs an LED & a resistor (100 or 220 ohms work) One can toggle the LED on/off with the sketch button on the shield. Also more awesomely OpenHAB (on your computer or phone) can toggle LED with a User Interface switch on a browser or app screen or schedule the LED to go on & off at certain times. Hardware wise, this can get you started with bidirectional wireless communication: node sending sensor information to OpenHAB, OpenHAB sending output commands to a node.

    ^^ Note about LED resistors: computourist used 1 kilo ohm resistor with LEDs. I prefer 100ohm which lets the LED be somewhat brighter without burning out. For dimmer LED & to reserve more current to other node parts, use higher than 100 ohms. May 19, 2016, computourist said: With 100 ohms, current will be around 15 mAmps, well within specs.

[papa]

PS CompuTourist's schematics use Arduino ProMinis. You can use Buonos or other 3.3 volt Arduinos by using the same labelled connections as on the Pro Minis.

Couple more thoughts on using Moteinos vs Buonos: In one sense it's convenience at a higher $ cost vs more higher time & trouble costs. One Moteino advantage could be it's compact size for stuffing in tight places. Maybe more hideable for controlling light fixtures or outlets.

Using the CompuTourist Gateway2.2 you only need one Moteino.  I use the 915MHz transceivers with the HCW naming which work very well throughout my home & I tested it fine a little ways out in my yard.

[lhw455]

OK, so I've finally received all the bits that I ordered, I have some more questions now!

1) With reference to the gateway schematic, I see 3 components, the Buono, the Ethernet W5100 and the wireless module - I purchased an ethernet shield too (Adafruit Protoshield R3) which came dissembled, how do I connect this all together? Do I even need the shield?
2) On the schematic again, there are little black plastic "things" which the wires go into (from the Ethernet module to the Buono, for example), does the 22 gauge wire I bought just go right into these (no soldering required, I see some small bulges in the pics on www.instructables.com/id/Uber-Home-Automation-w-Arduino-Pi/?ALLSTEPS)?
3) The schematic itself, how do I trace the wires through? For example, in the 2.1 gateway, mosi on the W5100 goes to (what I think) is mosi on the Arduino, but also from the RFM69? Is there a way to "read" the schematic?

I think that's all for the moment!

Thanks

[computourist]

Hello lhw455,

I'll try to answer your questions:

1- The Protoshield is not an ethernet shield, it is used to build your own custom project. You could use it to mount your RFM69 module. To connect it all together you just stack the modules. The connectors will provide the interconnections you need, like gnd, Vcc and also Miso Mosi etc.

2- If you do not use the protoshield, you'll have to solder some wires to the RFM69 module. These wires go to the connectors on the Arduino module.

3- as already stated: when you stack the modules the connectors will provide connection for most signals. The RFM module has to be connected using wire or the protoshield. The SPI bus signals go to Arduino, W5100 and RFM69.

Success !

[lhw455]

Thanks, seems I'm confused then!

Can I confirm: to build the 2.1 gateway schematic I don't need a protoshield? In the article (http://www.instructables.com/id/Uber-Home-Automation-w-Arduino-Pi/?ALLSTEPS) there is talk of a protoshield being required for the nodes? Do I need one for a node then (I'm building the temp/humidity one first).

Also, when you say "These wires go to the connectors on the Arduino module", you mean the black things that stick out of the Buono? Do I simply strip a small piece of the plastic from the 22 gauge wire and insert the wire into there?

Another way to state the above: if I'm a beginner:

1) Can I build the gateway 2.1 using only the Buono, the ethernet W5100 and the RFM module (obviously adding wires/soldering to the mix)?
2) Can i build a temp/humidity using only a Buono, the RFM module and the temp/humidity sensor?

Sorry for the newb questions. I think what I might do is lay all the parts out, connect what I think is the correct way to connect it all together and then take some pics, might be easier!

Thanks

[papa]

You ask, 1) Can I build the gateway 2.1 using only the Buono, the ethernet W5100 and the RFM module (obviously adding wires/soldering to the mix)?
2) Can i build a temp/humidity using only a Buono, the RFM module and the temp/humidity sensor?

1) Yes. Solder the needed 22 gauge wires to the RFM module & strip the other wire ends to fit into the right buono sockets for each Arduino connection. For the signal LEDs, solder a 100 ohm or somewhat greater resistor to one end of the LED. Plug that end into the designated Arduino digital socket & the other LED lead into a Arduino ground socket on the other side.  (Resistor & LED leads are small gauge & fit loose in sockets.  You can "bulk them up" by spreading some solder along each end. Experiment with amount until lead solidly enters & stays in socket.)

    ^^ Note about LED resistors: computourist used 1 kilo ohm resistor with LEDs. I prefer 100ohm which lets the LED be somewhat brighter without burning out. For dimmer LED & to reserve more current to other node parts, use higher than 100 ohms. May 19, 2016, computourist said: With 100 ohms, current will be around 15 mAmps, well within specs.

2) Mostly yes, if you leave off the push button & LED. Solder the needed wires to the RFM module & strip the other wire ends to fit into the right buono sockets (except 3.3v, see more below) for each Arduino connection. (Caution: For the Gateway node, the RFM69's NSS wire goes to the Arduino digital pin D8.  For the DHT Temp/humidity node (& other end nodes), the NSS wire connects to D10 instead. The Buono by itself will not give you enough Arduino sockets to also connect all 3 pins of the DHT temp/humidity sensor. You'll need create a way to connect Buono's 3.3v to both RFM69's 3.3v AND the Vcc connection of the sensor. (I've used a break away header with 3 female sockets on top & all 3 bottom male pins connected together. Jump wire from Buono's 3.3v socket to one break away female socket. RFM69's 3.3v wire into another female socket & Sensor's vcc connected to the 3rd female socket.) Off hand I think that would do it if you leave off the push button & LED.

Maybe the above will get you started. If needed, I can later post a picture of something similar I made.

[lhw455]

Sept 18, 2015 0:26:16 GMT papa said:

You ask, 1) Can I build the gateway 2.1 using only the Buono, the ethernet W5100 and the RFM module (obviously adding wires/soldering to the mix)?
2) Can i build a temp/humidity using only a Buono, the RFM module and the temp/humidity sensor?

1) Yes. Solder the needed 22 gauge wires to the RFM module & strip the other wire ends to fit into the right buono sockets for each Arduino connection. For the signal LEDs, solder a 100 ohm or somewhat greater resistor to one end of the LED. Plug that end into the designated Arduino digital socket & the other LED lead into a Arduino ground socket on the other side.  (Resistor & LED leads are small gauge & fit loose in sockets.  You can "bulk them up" by spreading some solder along each end. Experiment with amount until lead solidly enters & stays in socket.)

2) Mostly yes, if you leave off the push button & LED. Solder the needed wires to the RFM module & strip the other wire ends to fit into the right buono sockets (except 3.3v, see more below) for each Arduino connection. (Caution: For the Gateway node, the RFM69's NSS wire goes to the Arduino digital pin D8.  For the DHT Temp/humidity node (& other end nodes), the NSS wire connects to D10 instead. The Buono by itself will not give you enough Arduino sockets to also connect all 3 pins of the DHT temp/humidity sensor. You'll need create a way to connect Buono's 3.3v to both RFM69's 3.3v AND the Vcc connection of the sensor. (I've used a break away header with 3 female sockets on top & all 3 bottom male pins connected together. Jump wire from Buono's 3.3v socket to one break away female socket. RFM69's 3.3v wire into another female socket & Sensor's vcc connected to the 3rd female socket.) Off hand I think that would do it if you leave off the push button & LED.

Maybe the above will get you started. If needed, I can later post a picture of something similar I made.

Thanks, that's starting to make sense. Some pictures would be appreciated if you get the time, I'm going try and do a "dry-run" this weekend and I'll try post some pics (i.e. before soldering)

[papa]

Here's a picture where I used a break away header to expand access to certain Buono Uno connections.  This piece has 9 sockets on top & 9 pins on the bottom.  For this application, I soldered the bottom pins together (3 at a time using 3 small lengths of wires.  Pins 1-3 are thus connected together, likewise 4-6, 7-9.  The middle 3 sockets can then connect to Buono Uno's 3.3 volt socket, DHT11 sensor's vcc (power) pin, & something else.

[papa]

Here's a pic of the Gateway before I shortened the wires some. The only soldering was wires to the RFM69 wireless transceiver & attaching resistors to the LEDs. Everything else is a wire inserted in a socket.



Here's a pic of the start of an end node (not the gateway & not the garage door opener) showing short wires soldered to the RFM69 wireless transceiver & inserted into Buono Uno (arduino) sockets.  The antenna is curled here, not straight, which seems to work fine.  According to my research, a good antenna for 915mHZ RFM69 is about 3 inches, which works throughout my 3 level home & into the yard.



You can see more of my detailed posts at homeautomation.proboards.com/thread/62/success-home-automation-arduinos-rfm69s.

[lhw455]

Thanks Papa, I finally had some time to start trying to put this all together!

I've started with the gateway, and have essentially tried to copy it from your picture above. However, I am referencing the gateway schema from github.com/computourist/RFM69-MQTT-client/archive/master.zip and that shows an Ardunio, which I *think* is not the right schematic to follow if I want to recreate what you've built in the gateway picture above?

So my question is, where can I find the schematic for your gateway above - I see it has resistors and LED's - is it similar to this one - homeautomation.proboards.com/attachment/download/14 ?

Sorry for all the newbie questions, as you can see I'm no electronic engineer!

[papa]

As I wrote on Sept. 18, 12:55pm in my progressive post at homeautomation.proboards.com/thread/62/success-home-automation-arduinos-rfm69s ...
"For the Gateway, from github.com/computourist/RFM69-MQTT-client, I basically followed the schematic & Arduino sketch for Gateway 2.2, EXCEPT I used a Buono Uno switched to 3.3v mode instead of a ProMini."

The Gateway pictured above is a Buono Uno Arduino compatible with a W5100 Arduino Ethernet shield plugged on top.  Into the Ethernet shield's schematic-indicated Arduino sockets, I plugged the indicated wires soldered to the RFM69 transceiver mini board.  The schematic has 2 LEDs, each needing a resistor (with 100 ohms or 220 ohms, the LEDs shine pretty bright without burning).  One LED's (plugged into the Ethernet shield's D7 & a Gnd) steady shine indicates the gateway is connected to the MQTT service on the computer.  The other LED's (plugged into Ethernet shield's D9 & a Gnd) light flashes to indicate radio signals being sent.

   The Gateway schematic has some similarities to the adapted End Node schematic you indicated.  However, the Gateway has an Ethernet Shield plugged on top the Buono & has the indicated LEDs. (It would work without LEDs, but with them, it's easier to see that it is working.)  Also notice that on the Gateway, the RFM69's NSS wire plugs into D8, while it plugs into D10 on the end node.  The main similarities between Gateway & End node are that both use Buono's or some 3.3 volt Arduino compatible AND all RFM69 wires EXCEPT NSS plug into the same Arduino locations.

Here's a Gateway schematic I adapted from CompuTourist's.  I'll add it to my "Success ..." posts

    

[lhw455]

Thank you, some clarifications on the diagram

1) Where there is a crossover, represented by the little dot, I need 2 wires. For example, GND on the RFM69 to the D9 PWN on the ethernet shield a need a wire and also from D9 PWN on the ethernet shield to GND on the ethernet shield, with a resistor and LED in between?

2) The resistors, these seem to be 100 ohm, looking on the net there are several variations (tolerance, etc), which ones do I need for the gateway
3) Same question for the LED's, what's the exact one I need to purchase?

I do thank you and computourist (and everyone else) for the help, as you can probably see I'm a complete novice to HA and also electronics. Thanks once again.

[papa]

It's ok, lhw455, we all start somewhere. I've learned a lot about electronics, but still feel I have big holes in what I know. That just gives us more areas to explore & have fun. It's good you admit your level & ask questions to help focus the documentation posted here & may help others who need to know similar things. Now to your good questions ...

1) Some things about schematics that I had to learn: a) Some things in a schematic may not mean anything, but may result from how the drawing was created.  To my knowledge, the "dots" on the lines in CompuTourist's schematic (which I edited) do not mean anything.  b) Schematics try to represent a circuit's electrical connections in the simplest way possible, not necessarily components' physical locations or the only way to achieve those connections.

What's important about the lines in the schematic & the labels at each end & along the way is they tell WHAT connections must be made, but not necessarily HOW (You & I work out the best way with the components we have). In a simplistic way, the Gateway schematic says the RFM69 transceiver's GND solder hole & one end on each of both LEDs need a connection to the Arduino compatible's GND. Fortunately, the Gateway's Arduino & the Ethernet Shield plugged into its top have THREE GND sockets AND the RFM69HC transceiver has a GND solder hole on each side.  For positioning sake, I solder a wire the GND hole on the RFM69 side that also has MISO, MOSI, etc. Then after I strip that wire's other end, I can plug it in the Shield's GND socket next to the D13 socket.  Two more GND sockets are next to the 5v socket on the Ethernet Shield's analog side. Each of those two Arduino GND socket can receive one lead of a LED & resistor soldered together.

There are other ways to connect multiple components to the same Arduino connection (For example, in my ongoing Success ... posts, see where I talk about making a 3.3 volt friendly prototype shield to be plugged into the top of an Arduino compatible.  There I create a junction for multiple components using a section of break away female sockets soldered to a shield's solder strip.)  This is not needed for the Gateway since the Arduino & Ethernet Shield have all sockets needed for 3 GND connections.

2) You just need some low level resistance to protect the LEDs, like 100ohm or 220ohm. I prefer 100ohm which lets the LED be somewhat brighter without burning out. Tolerance should not matter. Buy cheap & enough to go with other LEDs in the future.

3) Likewise you just need a couple of LEDs.  I preferred having 2 colors (I picked yellow & green)  Again buy cheap & enough to have for other projects.

A good rule of thumb: Responsible circuit descriptions say when a component "quality" is important, like how for this project one must use Arduino compatibles that are (or switch to) primarily 3.3 volt operation. Otherwise it should be safe to use adequate parts available at a good price.  If one can, it's also good to ask.

P.S. Another (confusing ?) thing about schematics is the GND label & the GND symbol (not in the Gateway schematic, but shown below). Here GND (ground) does not mean connected to a stake in the ground. It means connect to a GND connection in the circuit supplying the power, in this case a GND socket on the Arduino. An Arduino's 3v (or 5v) or digital outputs "send" power & then connections to Arduino GNDs complete the circuit's circle for the current to flow as desired.
     (http://rsandas.com/images/Ground.gif)

[lhw455]

OK, thanks!

Update: I've done things (I'm slow, sorry!)

imgur.com/7ZFdWAq

and

imgur.com/8DGFaE8


Before I get soldering, I had a question or 2, or gosh like 4 or 5: in the schematic, there is an NC, which shows green but doesn't go anywhere - do I need anything connected here (papa, your gateway RFM dosen't have this connector?) - same question for the D10 PWN on the ethernet shield?

On the GND on the RFM, papa, you said 2 GND's one on each side - mine has 2 on one side and 1 on the other, can I confirm it doesn't matter which one I use, as you mention above?

And to confirm the schematic - D7 on the ethernet to GND on the other side of the ethernet, with resister and LED in between, it doesn't matter on which side I position the LED/resistor or should I have the resistor first in line on the D7 side and then the LED, as per the diag?

Also, D9 PWM on the ethernet to GND on the RFM and also to any GND on the ethernet (with resistor and LED in between)?

Finally, I am assuming on my RFM the ANA = the antenna (ANT on yours papa?)

[papa]

Good to be cautious about soldering, lhw455, at least until a circuit gets more familiar to us. I try to practice a rewording of a longtime carpenter expression: Check (schematic / instructions) twice (or more) & solder once. De-Soldering is such a time-sucking hassle.

I believe NC means no connection & certainly neither the Gateway nor the end nodes need that connection.  (My RFM69 does not even have it.) So nothing connects to NC.  For just the Gateway, do NOT connect D10 on the Ethernet Shield. The Ethernet Shield itself uses that Arduino connection. That's why (different from an end node) the Gateway's RFM69 uses D8 for NSS.

Generally speaking if a circuit's instructions do not specify which ground to use, you can use any GND socket on an Arduino or anything connected to an Arduino GND socket. To make things convenient for yourself, you might choose an Arduino GND connection & a GND connection on something like a RFM69 transceiver that are on the same side as each other.

Yes, you are correct about the Gateway's D7 & D9 connections. Since the resistors are to limit current flowing thru LEDs, logic would suggest to put resistors between D7 (likewise D9) & LED. I've seen some say that it does not matter which side of LED to put resistor, but to be safe, I follow "logic" & put resistor between the power from D7/D9 & the LED.

Thanks for the pics of your RFM69. From what I've seen, these transceiver mini-boards can vary in footprint & connection labels. (Your RFM69 connections are located differently than mine.) The key thing is to match the NAME of the RFM69 connection to the schematic connection & it should work.  Yes, apparently, on your RFM69, ANA is the antenna connection (mine is labelled ANT).

PS From your pics, I notice that your RFM69 is an "H" version ("high power"), so in the Arduino program sketches for Gateway & end nodes, you can use the customization for that version (which might be the default in CompuTourist's sketch & is so chosen in my sketch version).

I wish you well on your assembly & hope to hear how it goes for you.

[lhw455]

Making progress!

The soldering was quite hard, I'm not sure I did a great job and I don't even know if this gateway will work, but pics!

i.imgur.com/8aydi4F.jpg

i.imgur.com/7CgbX3S.jpg

i.imgur.com/1DGwMaR.jpg

i.imgur.com/kcig5CL.jpg

I still need to heat the heatshrink, but wanted to test everything first. So the next step is uploading the .ino file to the gateway, is that correct? I've downloaded the arduino IDE, is that the best way to do it?

Thanks, super excited!

[papa]

Good to hear your progress, lhw455. You're welcome. Forgive me if I explain things you already know (because I don't always know or remember what you know). Ask more questions if I assume too much.

I looked at your pics. Seems like you've not soldered together two RFM69 solder holes. Like I describe in the Success... posts, you can use the continuity test feature of a digital multimeter to make sure the right things are connected & the wrong things are not. For sure, be careful to use the 3.3 volt socket & that the ground wires go in the right sockets. Other mistakes don't necessarily risk damage.

Yes, the RFM69 solder holes are very close together. It gets easier with practice. My Success... posts include suggestions about using Help Hand tools & other ways to stabilize components to help with soldering such things. As I've mentioned along the way, if you want, your wires & leads can be shorter, making a more compact unit, but right now, you're getting your feet wet & experiencing the spaces you have to work with.

Yes, using the upload feature of the Arduino IDE, while the Arduino connects to your computer via USB, you will need to send the Gateway sketch program (ending in .ino) to the Arduino. I assume you've looked through the sketch & chosen the options that apply to your setup. (That is, commenting out what does not apply, uncommenting what does apply.) Before uploading a sketch, I find it helpful to use the verify feature & let it tell me if it finds problems to fix in the sketch.

Speaking of options, I recommend that for test purposes, you turn on the debug mode in the Gateway sketch. Just realize that this makes the sketch big enough that (after uploading) the Arduino IDE will warn you that the remaining memory for operation is limited. The Gateway will still work.

With the programmed Gateway's Arduino / Ethernet shield connected to your ethernet network & to your computer via USB, with Arduino IDE serial monitor running at same baud as set in the Gateway sketch, with the sketch using debug mode, serial monitor will show if Gateway is connected to your home network. With all that in this paragraph, plus MQTT running on your Raspberry Pi, you want the serial monitor to show MQTT connection & the Gateway's MQTT LED to light up. You won't see more good messages on the serial monitor nor a flashing Gateway radio signal LED until an end node is assembled, programmed & connected.

[lhw455]

Thanks Papa, much appreciated. I tried a "compile" in the IDE and it had said that it needed rfm69.h, which it couldn't find - Googling says I need to download this file, I suspect I need to add it to the folder where the .ino file is?

I haven't got to the point of uncommenting anything, although I did see some options in there...will tackle that tonight if I get time.

Most importantly, on your statement of " not soldered together two RFM69 solder holes", this I didn't know about - what do I need to do here (can't see anything on the sketch)?

Thanks!

[papa]

These notes are in the Gateway program sketch, RFM_MQTT_GW_22.ino
(I corrected some errors / changes.) They give essential things to do:

// ==> Note:
// - Interrupts are disabled during ethernet transactions in w5100.h (ethernet library)
// (See harizanov.com/2012/04/rfm12b-and-arduino-ethernet-with-wiznet5100-chip/)
// ... ...
//
// RFM69 Library by Felix Rusu - [e-mail]
// Get the zip file of RFM69 library at: github.com/LowPowerLab/RFM69

The RFM69 Library provides the RFM69.h library file that the IDE is expecting.
      Download the library zip file at github.com/LowPowerLab/RFM69/archive/master.zip
& extract where you can find it.
Then you must put that collection of files where the IDE can find it:
It will be your path that looks something like
C:\Users\[your user name]\Documents\Arduino\libraries\RFM69
    The collection of files which include RFM69.h must be in that libraries folder named RFM69
         (You'll probably need to rename a folder from RFM69-master to RFM69.)

The other note (changing the w5100.h file) will make your wireless transmissions more reliable. I need to leave for a while shortly, but I'll describe that later.  You won't need that until you also have an end node built & programmed.

Sorry, my comment about "not soldering together two RFM69 solder holes" is a POSITIVE comment meaning that the pics looked like you did not solder together the wrong things even though they are close together.

[papa]

lhw455, you raise good questions that stymie newbies.  I know the things you raise certainly held me up until I found answers.  I'm adding what I've written here to my Success... posts.

  Concerning the Gateway sketch notes about the RFM69 library and w5100.h (a library file),
Arduino sketches "include" additional programming code from libraries, which are standardized code to help us use particularly add-on devices like RFM69 transceivers & Ethernet Shields.  They are text files which may be opened, viewed, & CAREFULLY edited with a text editing application.

Library files are stored in TWO places on a computer. 1) As mentioned above about the RFM69 library, in the ...Documents\Arduino\libraries folder 2) With the Arduino IDE files at your path like "C:\Program Files (x86)\Arduino\libraries".
             ---------------------------------------------------
    Now, lhw455, regarding the change in the w5100.h file that is noted in the Gateway sketch,

Summarizing & narrowing down the harizanov.com ... info for our purposes:

            in the New release duplex gateway thread
      CompuTourist posted the following on Feb 19, 2015 at 10:45am

        Yes, replace: [Papa: find these lines in w5100.h]

#else
inline static void initSS() { DDRB |= _BV(2); };
inline static void setSS() { PORTB &= ~_BV(2); };
inline static void resetSS() { PORTB |= _BV(2); };
#endif

        with: [Papa: actually ADD what I put in bold face below]

#else
inline static void initSS() { DDRB |= _BV(2); };
inline static void setSS() { cli(); PORTB &= ~_BV(2); };
inline static void resetSS() { PORTB |= _BV(2); sei(); };
#endif
        -------------------------------------------------

According to my limited understanding, the (bold faced) addition above cli() turns off interrupts briefly & sei() turns them back on.  This is supposed to improve transmissions between RFM69 transceivers, preventing drop outs, & I found that true.

      Maybe you or another reader wonder "Where & how do I make those changes in w5100.h?"

w5100.h is a library file needed for the Arduino Ethernet Shield that is part of the Gateway.

w5100.h comes with the Arduino IDE & is at your path version of "C:\Program Files (x86)\Arduino\libraries\Ethernet\src\utility\w5100.h"

Hint: In order to edit & re-save w5100, you'll probably need to shut down the Arduino IDE & possibly reboot your computer because the Arduino IDE / Gateway sketch "lock" the file.

Once you find the w5100.h file, open it with a text editor (like notepad), make the 2 additions noted above (in bold face for visibility, not in your file) & re-save w5100.h.  As noted above, these changes are not needed until you also have an end node going, but probably wise to make the changes soon before forgotten.

[lhw455]

Got it uploaded - took me awhile to figure out that I needed to upload the .ino without the W5100 attached. However, I switch it all on and .... nothing. Light on the W5100 shows power, but nothing else, even when I plug in the ethernet cable.

Neither of the LED comes on either, and attempting to get the serial console to show any output is fruitless. I'll have to find my multimeter as I can only suspect that one of the connections is a bit flaky.

Thanks all so far, especially papa!

[papa]

Hmmm, troubleshooting is a hassle but it's a good skill to learn & will often be useful later.  Hang in there.  It took me a long time to work through all the troubleshooting & start to see things working.

You said you got the Gateway sketch uploaded to the Arduino so I assume you located & installed the PubSubClient library the sketch also wants to include & the Arduino IDE did not protest about not finding that library, right?

Yes, good to check your connections with a gentle shaking of each soldered wire to see if it's loose & with a multimeter.  Also recheck where all the wires are plugged in the sockets.  Are they securely in the right socket indicated by the schematic?

The LED plus resistor wires are a little small to be tight in the sockets & might not always be in good connection.  You can add a little solder to each end to make the connections a little tighter.  That said, the Gateway's radio signal LED may not flash until you have an end node wirelessly connected to the Gateway. The Gateway's MQTT LED
should light steady when via the ethernet cable & home network, the Gateway connects to the MQTT service on the computer.

You definitely need the ethernet cable connecting the Gateway's Arduino Ethernet Shield to your home network. Did you activate debug mode in the Gateway sketch, connect the Gateway to your computer's usb, open up the Arduino IDE's serial monitor with same baud rate as set in the Gateway sketch? If & when you do that, what messages show on the serial monitor?  Say more about results on the serial monitor, even "fruitless" ones.

Are you sure the MQTT service is running on your "computer," Raspberry Pi, I believe?  I find it helps to have your router give that computer a static/permanent IP address rather than one the router's DHCP sets & changes occasionally.

Are you sure you've set the needed options in the Gateway Sketch, especially for now, the network related ones?  
      Those are in these Gateway Sketch lines:

// Ethernet settings
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xBA, 0xBE }; // MAC address for ethernet [shield]
byte mqtt_server[] = { 192, 168, xx, xx}; // MQTT broker address
byte ip[] = { 192, 168, xx , xx }; // Gateway address (if DHCP fails)

I believe the line byte mac[]... is ok as is, at least it was for me.  The sketch seems to assign that MAC address to the ethernet shield.  After my Gateway connects to my home network, the router's DHCP list says client name (for Ethernet Shield) is WIZnetEFBABE & MAC address is DE-AD-BE-EF-BA-BE

In the line byte mqtt_server[]... change the 2 sets of "xx" so the address matches the IP address of the computer running the MQTT service.  This is VERY, VERY important.  If wrong, the Gateway Ethernet Shield cannot find & link to the MQTT service needed to pass all the messages.  I know because I made this mistake for a while.

In the line byte ip[]... change the 2 sets of "xx" so the ip address is a valid & available address in your home network.  It will use the same first 3 numbers as the MQTT server IP address. You can probably make the last number somewhat beyond the mqtt server’s IP address.  That is, if the MQTT computer's IP address is 192.168.10.103, you can probably use something like byte ip[] = { 192, 168, 10 , 133 }

It can help to log in to your home router & see the DHCP list of MAC & IP addresses assigned (or not) to different devices, especially your MQTT computer & the Gateway's Arduino Ethernet Shield.  That will help you know what is connected & not & how they are addressed.

I found that to get all the components (MQTT computer, Gateway, & home network router) talking to each other, I had to restart the Gateway & router more than once.  You might try that as well as the other troubleshooting above.

[lhw455]

Thanks papa, I must buy you a beer (or beverage of your choice) if we ever cross paths in this great big world!

You said you got the Gateway sketch uploaded to the Arduino so I assume you located & installed the PubSubClient library the sketch also wants to include & the Arduino IDE did not protest about not finding that library, right?

Correct

Yes, good to check your connections with a gentle shaking of each soldered wire to see if it's loose & with a multimeter.  Also recheck where all the wires are plugged in the sockets.  Are they securely in the right socket indicated by the schematic?

I checked this, one the connections appears to have little solder, so I might try . All are in the correct socket, but I wanted to ask one thing: in the schematic, the radio LED/resistor goes to both GND on the Arduino/ethernet GND and the GND on the RFM69, mine is just plugged into a GND on the Arduino, is that OK?

The LED plus resistor wires are a little small to be tight in the sockets & might not always be in good connection.  You can add a little solder to each end to make the connections a little tighter.  That said, the Gateway's radio signal LED may not flash until you have an end node wirelessly connected to the Gateway. The Gateway's MQTT LED
should light steady when via the ethernet cable & home network, the Gateway connects to the MQTT service on the computer.

I discovered this early on and soldering a little cable to the end of each side, but you advice is good, I want to make everything a little more secure in the sockets, so I may put some solder on each of the ends

You definitely need the ethernet cable connecting the Gateway's Arduino Ethernet Shield to your home network. Did you activate debug mode in the Gateway sketch, connect the Gateway to your computer's usb, open up the Arduino IDE's serial monitor with same baud rate as set in the Gateway sketch? If & when you do that, what messages show on the serial monitor?  Say more about results on the serial monitor, even "fruitless" ones.

Ethernet cable is connected, debug in enbabled in the .ino file, baud rate set - by fruitless, I mean nothing, nilch, nada shows, no output in the window whatsoever. I'll have more time to troubleshoot on Sunday though, last night was a bit of a hurry.

Are you sure the MQTT service is running on your "computer," Raspberry Pi, I believe?  I find it helps to have your router give that computer a static/permanent IP address rather than one the router's DHCP sets & changes occasionally.

The pi is a static IP, I'll double check that the MQTT service is running, as I simply installed it via the apt-get commands

Are you sure you've set the needed options in the Gateway Sketch, especially for now, the network related ones?  
      Those are in these Gateway Sketch lines:

// Ethernet settings
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xBA, 0xBE }; // MAC address for ethernet [shield]
byte mqtt_server[] = { 192, 168, xx, xx}; // MQTT broker address
byte ip[] = { 192, 168, xx , xx }; // Gateway address (if DHCP fails)


I believe the line byte mac[]... is ok as is, at least it was for me.  The sketch seems to assign that MAC address to the ethernet shield.  After my Gateway connects to my home network, the router's DHCP list says client name (for Ethernet Shield) is WIZnetEFBABE & MAC address is DE-AD-BE-EF-BA-BE

In the line byte mqtt_server[]... change the 2 sets of "xx" so the address matches the IP address of the computer running the MQTT service.  This is VERY, VERY important.  If wrong, the Gateway Ethernet Shield cannot find & link to the MQTT service needed to pass all the messages.  I know because I made this mistake for a while.

In the line byte ip[]... change the 2 sets of "xx" so the ip address is a valid & available address in your home network.  It will use the same first 3 numbers as the MQTT server IP address. You can probably make the last number somewhat beyond the mqtt server’s IP address.  That is, if the MQTT computer's IP address is 192.168.10.103, you can probably use something like byte ip[] = { 192, 168, 10 , 133 }

It can help to log in to your home router & see the DHCP list of MAC & IP addresses assigned (or not) to different devices, especially your MQTT computer & the Gateway's Arduino Ethernet Shield.  That will help you know what is connected & not & how they are addressed.

I found that to get all the components (MQTT computer, Gateway, & home network router) talking to each other, I had to restart the Gateway & router more than once.  You might try that as well as the other troubleshooting above.

I did all the above, but one thing I did do wrong which I need to change, I thought the "byte ip" is the IP of the gateway (for my network 192.168.1.1 - but I'll change this to a free IP and try.

Of all the things I've done so far, I'm most comfortable with the networking side, but I'll check the above.

Thanks once again

[papa]

It's nice to have my offerings appreciated & good to have a techie pen pal. Sharing a craft beer would be great, maybe we can do that at a distance. I'll warn that I will be more busy for the weekend starting Friday night so not sure how much I'll be in touch then.

As I've said, I know little about Raspberry Pi & Linux (more on the Windows side) so I'll leave that to you & others. Good to know that you're more comfortable with networking.

You ask "in the schematic, the radio LED/resistor goes to both GND on the Arduino/ethernet GND and the GND on the RFM69, mine is just plugged into a GND on the Arduino, is that OK?" Yes, it's OK. As I was suggesting earlier, the schematic just expresses that RFM69's GND & the Radio LED must be SOMEHOW connected to an Arduino GND. That's what's important to gain from the schematic.

I've looked carefully through the text & pictures above. In the pictures, it seems you have wires soldered to the right locations & then the wires' other ends are in the right sockets (though, of course, I cannot tell the quality of the soldering or how long you stripped the wires to put in sockets.)

Let's see what else might be checked. Forgive me if I suggest things that are obvious & old hat to you.

You've had the Bunono Uno Arduino compatible always switched to 3.3 volts, right?

Does your ethernet cable go DIRECTLY from the Gateway's Ethernet Shield to the router that assigns IP addresses in your network?  I had some trouble when my cable went thru another device after the router.

I mention again that it can help to restart your router & the Arduino / Ethernet Shield (by pressing the RESET button).  After you do this, do you see the Ethernet Shield (Wiznet...) in your router's DHCP list.

In the Gateway sketch, you have the line "#define DEBUG" & NOT "// #define DEBUG" , right?   Because the two forward slashes ("//") make the rest of a line a "comment" (not programming code) & disables debug mode. If you need to correct that & upload back to the Arduino, do you then see anything on the Arduino IDE serial monitor (Ctrl+Shift+M or from the Tools menu).

I tend to take it for granted now, but it can be a little ticklish getting the Arduino & the serial monitor communicating.  I find this usually works:  Unplug Arduino from usb, close ALL Arduino sketches, open one Arduino sketch (like for the Gateway), plug the arduino into the computer's usb, wait a little, then open the serial monitor, & make sure it is set to 115200 baud.  Now what do you see in the serial monitor window on the computer?

When Gateway components are plugged together & powered, look at the tiny LEDs soldered on to the Ethernet Shield & the Buono Uno Arduino compatible underneath.  What do you see happening with those tiny LEDs?  Which LEDs light steadily & which LEDs flash?  What color are they?

If you've done what I & you said (including activating debug mode in the sketch), you should get some result on the serial monitor, even just initial start up & error messages like "Gateway Software Version..." "Failed to configure Ethernet using DHCP" "connecting..." "connection failed..."

        If you still see no results on the serial monitor ...
Maybe you can reduce Gateway components to a minimum & test that. Try this: From the Ethernet Shield on top, unplug the LEDs/resistors and unplug all the RFM69 wires. That should leave the Buono Uno (3.3 volt mode), Ethernet Shield plugged on top, USB cable just connected to the Arduino (& not the computer yet) & NO ethernet connection yet to the Ethernet Shield.  Follow the sequence above for starting a fresh Arduino IDE & connecting the Arduino to the computer's usb, & opening the serial monitor.  What do you now see in the serial monitor window?  Something like "Gateway Software Version..." "Failed to configure Ethernet using DHCP" ?  If you get like that start up & error message, then connect the Ethernet Shield to your home network. Now what do you see in the serial monitor?
      If serial monitor reports a MQTT connection, you can unplug from usb, plug in the MQTT LED/resistor, replug to usb, & MQTT LED should light.  Then in similar way, you can plug in the RFM69 module.  Maybe with this progressing assembly you can isolate a component that's not working.

[papa]

PS It's probably a good idea to re-read through what I've written because I go back several times & reword & add things.

This project is not for the faint-hearted, many necessary interlinking aspects, but it is SO SO COOL when it works.

Troubleshooting with you, lhw455, has taken some time but it's helping me remember & document what I've learned from failures & successes.

[lhw455]

Suspect it's the W5100, when disconneted the R3 works fine (I see output in the serial monitor), but not when either the W5100 with and without the RFM and LED/resistors attached. I've contacted the ebay seller.