Newbie after some advice for initial build

[papa]

Thanks camblonie. Yes, also may be easier to see others' mistakes than our own.

[papa]

Trouserhams, in my Success... posts, take a look at the Oct 8, 2015, 6:40pm Post where I give very detailed documentation of building & programming a Gateway using a Buono Uno. You might find something useful in there.

I run (& document) MQTT & OpenHAB on Windows, but Gateway & nodes should work the same.

When you get to the DHT End node, you might use my posts starting on Oct. 10.

Earlier posts on reading schematics, soldering, troubleshooting, etc might also be helpful. I've tried to provide info that I, as a newbie had to scare up before I could move forward.

[lhw455]

Oct 23, 2015 23:32:42 GMT trouserhams said:

Oct 14, 2015 17:53:14 GMT lhw455 said:

Follow this guide: jpmens.net/2013/09/01/installing-mosquitto-on-a-raspberry-pi/

Except, when you get to this part:


curl -O http://repo.mosquitto.org/debian/mosquitto-repo.gpg.key
sudo apt-key add mosquitto-repo.gpg.key
rm mosquitto-repo.gpg.key
cd /etc/apt/sources.list.d/
sudo curl -O http://repo.mosquitto.org/debian/mosquitto-repo.list
sudo apt-get update


Instead follow this:


curl -O http://repo.mosquitto.org/debian/mosquitto-repo.gpg.key
sudo apt-key add mosquitto-repo.gpg.key
rm mosquitto-repo.gpg.key
cd /etc/apt/sources.list.d/
sudo curl -O http://repo.mosquitto.org/debian/mosquitto-wheezy.list
sudo apt-get update




Hi lhw455, I seem to be a few months behind you, but am also attempting this from a newbie starting point.

Currently I am unable to connect when plugging the gateway into the router, I have even given up completely with one set of pro mini hardware and started again with a buono and ethernet shield to the same outcome. (I have gradually stripped components off it to see if the RF69 chip was damaged, and now jsut have the arduino and ethernet shield in place!)

I have installed mosquitto and tested it works on the raspberry pi by communicating via two terminal windows as follows;

        Start the Mosquitto Broker Open the terminal and type mosquitto_sub -h 127.0.0.1 -t topic



        Open another terminal and type mosquitto_pub -h 127.0.0.1 -t topic -m "Hello"



        Now you can switch to the previous terminal and there you can able to see the "Hello" Message.One terminal acts as publisher and another one         subscriber.

However, in order to get this working I literally only typed in what you have above, and didn't follow the rest of the guide. I just wanted to check whether this was correct?

Otherwise I am guessing I must have an issue with my router or something along those lines? very frustrating seeming as it sounds like it should be virtually plug and play!!!





Hi there, welcome!

No, you don't need to run the rest, that's only if you need encrypted communications - which for me since it's on the local network, I don't need, which I think would be the same for you.

To confirm, it sounds like you have, but I didn't include the snippet to actually install the packages, you ran the following, after the code snippet I posted above?

sudo apt-get install mosquitto mosquitto-clients python-mosquitto


Couple things I would check if that's all OK above:

1) Make sure the ethernet gateway is getting an IP, use a scanning tool to check the IP range (I used one called Angry IP scanner, get the older 2.21 version) 
2) Make sure you've got the right mqtt server variable (which is the Raspberry Pi in yours and mines cases)
3) Set a IP for the gateway in case DHCP isn't working

2 and 3 are done in the gateway sketch.

[lhw455]

Oct 23, 2015 21:05:06 GMT papa said:

When you say yes, I get Node V2.1, communicating at 868mhz, followed by the repeating "no connection"
        I believe you identified a problem.

                Because your pics show RFM69's at 915MHz.

Looks like you need to make about lines 70-71 in DHT sketch to look like this:

//#define FREQUENCY RF69_868MHZ
#define FREQUENCY RF69_915MHZ

first line is disabled with "//" comment indicator
   second line is active

Also make sure the corresponding lines in the Gateway sketch are correct.

Make those changes, upload, & see how it works.

Thanks papa, that was intentional on my part, as I'm in the UK 915mhz is a restricted frequency (which I only found out after ordering the 915mhz RFM69's) - I later read that the 915mhz modules can simply be programmed as 868mhz modules (which is the unrestricted frequency) and they would work fine. Is this correct?

[papa]

According to Felix, the RFM69 guy, you are correct, by software settings, RFM69s can be used at 915MHz or 868MHz.

So I guess you'll have to do other troubleshooting.

[xpokor02]

Hi,
If I understand it well than in theory you could change frequency, but each version(frequency) has different "crystal reference oscillator" and they are not precise so to make bigger change in frequency you would have to measure real frequency and adjust each RFM69's configuration differently so they are all communicating on same frequency.
At least that is what I understand from ukhas.net/wiki/ideas:rfm69_frequency_trimming .
Thanks

[trouserhams]

Oct 24, 2015 10:17:03 GMT lhw455 said:

Couple things I would check if that's all OK above:

1) Make sure the ethernet gateway is getting an IP, use a scanning tool to check the IP range (I used one called Angry IP scanner, get the older 2.21 version) 
2) Make sure you've got the right mqtt server variable (which is the Raspberry Pi in yours and mines cases)
3) Set a IP for the gateway in case DHCP isn't working

2 and 3 are done in the gateway sketch.

Thank you for your response!

To clarify, step 2 you mention - is this just setting the IP address of the raspberry Pi as the MQTT variable, or is there something else in the sketch I am missing? I have done this since the start and still don't get anything through.

I have also been using angry IP scanner to try and diagnose what's going on. The gateway does seem to get an IP address assigned in the DHCP.

The last attempt was with just the arduino and ethernet shield running as per Papa's helpful debugging post, however I get nowhere once the ethernet cable is plugged in! (can it be a firewall issue?)

earlier I also noticed that my Pi had two IP addressed assigned to Ethernet due to dhcp leasing one. I managed to resolve this, but it didn't make any difference.


This is all very frustrating!

I'm starting from scratch again and have just completely wiped the raspberry Pi to start again. I have already checked my w5100 is functioning with the simple Arduino provided 'webserver'.

Completely at a loose end if this complete restart of everything - arduino program, all the libraries etc doesn't work!

[lhw455]

Oct 24, 2015 11:57:17 GMT trouserhams said:

Oct 24, 2015 10:17:03 GMT lhw455 said:

Couple things I would check if that's all OK above:

1) Make sure the ethernet gateway is getting an IP, use a scanning tool to check the IP range (I used one called Angry IP scanner, get the older 2.21 version) 
2) Make sure you've got the right mqtt server variable (which is the Raspberry Pi in yours and mines cases)
3) Set a IP for the gateway in case DHCP isn't working

2 and 3 are done in the gateway sketch.

Thank you for your response!

To clarify, step 2 you mention - is this just setting the IP address of the raspberry Pi as the MQTT variable, or is there something else in the sketch I am missing? I have done this since the start and still don't get anything through.

I have also been using angry IP scanner to try and diagnose what's going on. The gateway does seem to get an IP address assigned in the DHCP.

The last attempt was with just the arduino and ethernet shield running as per Papa's helpful debugging post, however I get nowhere once the ethernet cable is plugged in! (can it be a firewall issue?)

earlier I also noticed that my Pi had two IP addressed assigned to Ethernet due to dhcp leasing one. I managed to resolve this, but it didn't make any difference.


This is all very frustrating!

I'm starting from scratch again and have just completely wiped the raspberry Pi to start again. I have already checked my w5100 is functioning with the simple Arduino provided 'webserver'.

Completely at a loose end if this complete restart of everything - arduino program, all the libraries etc doesn't work!

Yes, the byte mqtt_server[] should be set to the IP of the raspberry pi running Mosquitto. Only other thing I can think of is that the Moquitto server is bound to the wrong interface? When you ping both IP's now do you get a response from both or only the static one you've setup? I don't believe Raspbian has a firewall set as standard, but that could definitely be something to check. Also, very basic, but try another port and network cable, sometimes it's the littlest things!

[camblonie]

Trouser you can't be far off. Save an image when you get the RPi setup again. Rebuilding the RPi takes forever and is probably not the issue if you test the mqtt and it works. Are you running Openhab from RPi? What does your gateway serial monitor tell you with debug on?

[lhw455]

Oct 24, 2015 21:55:50 GMT camblonie said:

Trouser you can't be far off. Save an image when you get the RPi setup again. Rebuilding the RPi takes forever and is probably not the issue if you test the mqtt and it works. Are you running Openhab from RPi? What does your gateway serial monitor tell you with debug on?

The bolded part is a good point, I'd check that (in case you weren't aware, look for the debug line in the gateway .ino and uncomment it out).

[trouserhams]

Hi All,

Thank you for your help, I now have this working. Frustratingly I believe it was because I downloaded the PubSubClient library through the Arduino library manager rather than the latest version from the GitHub source.

Realistically this was the only thing I did differently when I rebuilt everything and it worked straight away. Frustratingly in order to get this far I took apart my nice neat gateway as I thought that must have been the issue, and now have a gangly prototype buono uno board with wires hanging all over it set up at the moment. I may keep this and build the RFM69 into the space between the pins on the Ethernet shield as then I can buy a ready made case for it similar to my RPi and not mill one from billet as I was going to!

I did have the debug uncommented, however part of the reason I switched from my Pro Mini gateway to the Buono is that there was not enough memory due to the boot loader to fit the sketch with full debug on it!

Its amazing how happy you can be to have two LED's light up ...

I'm working on the OpenHab side of things today - aiming to get the DHT node reading onto the iPad interface. I'll then rebuild my DHT node into a relay node to get the basic relay control set up for my later goal of a full smart heating control system. I'm currently waiting on a further 15 RFM69HW's as I'm having signal quality issues with the RFM69W.

[lhw455]

Good to hear!

[lhw455]

Ok, so I'm a bit stuck - I've bought a multimeter, and determined that it seems there was a cross over on 2 of the wires from my RFM69 on the DHT end node - so I purchased another RFM69, however I didn't check the listing very well and ended up getting RFM69H (as opposed to RFM69HW, which is what I have on the gateway) - I'm still getting "no connetion" on the end node.

Any ideas? I'm going to do the same continuity tests on the new RFM69H end node tomorrow - does anyone know if the RFM69H and the RFM68HW "talk" to each other?

[papa]

If I remember correctly, those two RFM69 models should communicate as long as they are at the same frequency & you connect the RFM69 labeled holes to their proper Arduino connections. I believe the two RFM69 versions function the same, but may have a different "footprint," namely the connections may be at somewhat different locations around the edge. Certainly both have the "H" high power designations & would use that parameter in the sketches.

Good thought to continuity check your RFM69 & soldering.  You say you have "crossover," I assume between RFM69 locations that are supposed to be different.  Is it caused by solder between them?  If so, can you use desoldering braid to remove enough solder to fix that?

[lhw455]

Thanks papa, I'm going to double check my new RFM69H to make sure the holes are in the same place and I've also done a better job of soldering. I've also ordered some soldering braid to see if I can get the other RFM69HW working properly.

Yes, cross-over, what I mean (in noobie speak) is that when I did a continuity test on each of the RFM wires to the bottom of the relevant Arduino pins, those are all OK. However, when I do it between each of the Arduino pins (on the bottom) where I expec there to be no connection, I found a connection between 2 of the pins, so I guess something is wrong on the RFM side.

Gonna desolder them and try again.

Also, just an update on the gateway, both LED's are working now (well I think the radio one is, but I'm not getting a connection so I can't tell) but I've interchanged the LED's between the MQTT broker and both light up as expected.

[lhw455]

OK, so I've checked and re-checked, I can't find anything physically wrong with the continuity of the new end node - I've also confirmed the holes are the same. A question, something I discovered when testing continuity on the gateway, it appears that when I put the continuity tester on the GND and SCK, for example, it shows a connection (this is when the whole gateway is connected up, but powered off) - is this normal?

[papa]

With my working Gateway (& also a working DHT end node), assembled, but not powered, I did a continuity check on the RFM69 connections. GND was not connected to anything other than other GNDs. So your situation does not sound right.

If for the RFM69 GND, one intends to use the Arduino GND on the digital side, then RFM 69's SCK plugs into D13 right next to the GND. Is it possible you plugged RFM69 SCK into Arduino GND & RFM69 GND into D13? Otherwise, are you sure there is no visible connection between SCK & one of the RFM69 GNDs? Beyond that, you start to wonder about defective RFM69 transceivers.

[lhw455]

It lives! I reseated all the connectors, resoldered a wire on the end node. I also found out that I had actually order the high power version of the RFM69, so I changed this back in the sketch, uploaded, and it's working! Excited to say the least, I see the end node sending messages, and also the gateway receiving them (checking via the serial monitor)! Now I just need to figure out how to get the gateway to publish correctly to the queue.

Thanks all!!!!

[papa]

Good to hear. This project often takes much patience & troubleshooting, but is such a thrill when it works. Congratulations.

[lhw455]

Thanks! Had a quick go at getting the raspberry pi receiving the messages via the MQTT broker, but (using something call mqtt-spy) I wasn't able to see anything come through. I've got it setup as follows:

1) End node is publishing messages via home/rfm_gw/nb/node02/dev48 and home/rfm_gw/nb/node02/dev49
2) End node is receiving at same address as far as I can tell
3) I can see the messages being received by the end node, but don't know if they're passing them along to the raspberry pi?

Is there anything I need to configure on the broker on the raspberry pi?

[papa]

Once I could tell that the Gateway & an end node was connecting, I got the OpenHAB server installed & created related config files. Then in the User Interface on a browser in my Windows Computer, I could tell if things were getting through.

Do you have OpenHAB server installed & running?

If not, here's some instructions github.com/openhab/openhab/wiki/Quick-Setup-an-openHAB-Server

In my OpenHAB / items config file, I have lines like the following to get Node 2 temp / humidity data into OpenHAB

* Indoor Conditions */
/* itemtype itemname ["labeltext"] [<iconname>] [(group1, group2, ...)] [{bindingconfig}] */
/* [%.1f °F] below for Fahrenheit. Use [%.1f °C] for Celsius */
Number Temperature_Node02 "Temperature_Node02 [%.1f °F]" <temperature> {mqtt="<[mosquitto:home/rfm_gw/nb/node02/dev48:state:default]"}
Number Humidity_Node02 "Humidity_Node02 [%d %%]" <humidity> {mqtt="<[mosquitto:home/rfm_gw/nb/node02/dev49:state:default]"}

In my Success... posts, starting on Oct 16, 2015 I detail installing & running OpenHAB on WINDOWS. Maybe you'll find some parallels doing it on the Pi.

At homeautomation.proboards.com/attachment/download/96, you can download a sample set of OpenHAB config files that I made for a Gateway & a Node 2 DHT End Node. I believe they have worked for those using OpenHAB on the Pi.

The original Uber Home Automation Instructable said, "I know how frustrating Linux can be. If all else fails, there's always the option of using the Windows version of OpenHAB and Mosquitto. The great thing about OpenHAB is that the operating system doesn't matter - you can start on Windows and move to Linux at a later date and all the configuration files can just be moved over and it'll work."  Not yet using a Pi, I don't know that is true, but I believe it is.

[lhw455]

It works! Again, thanks, I've now got my temp/humidity sensor updating my OpenHab interface! I'll post a small howto on what I did.

[papa]

Yes, it will be interesting on what did the trick. Congrats.

[lhw455]

Well, it was basically your code, plus adding in the references into the sitemap. A small lesson in OpenHab I guess, not sure it's worth repeating, but I don't know OpenHab that well yet so it was a bit of detective work and fiddling which got me there in the end. I'll document it at some point this weekend.

But super excited, next on the list is to create a home (i.e. container/case) for the gateway and relocate it to a better location, and the same plus a battery solution for the temp/humidity sensor. Then to figure out the strategy for the house and what I want to smarten up in the house. Potentially a garage door open/closed indicator and also a fixed location for a wall-mounted tablet.

[papa]

On locating the Gateway, I took the typical WiFI recommendation of putting it on the highest floor as central as possible. That signal reaches reliably down to my basement 2 floors below & throughout the house & even some ways in my yard (I did not test it any further).

On a solderless breadboard, I built a 3.3 volt "bare bones bones" Arduino with DHT11 sensor, that can be powered by battery or wall adapter. To save power, it does not have USB, etc. On batteries, it worked just fine in my yard. I have not tested how long it lasts on batteries. I adapted the battery powered unit in the Uber Home Automation Instructable. From that design, I removed the reed switch & removed voltage divider resistors which test battery voltage (the CompuTourist sketch takes care of that). I added circuitry for the DHT sensor. Among parts to buy are the AtMega microprocessor chips which one must program separately since the bare bones units do not have that. Also needed is a socket for the microprocessor so one can take it out & reprogram it. On circuit boards, I've built 5 volt bare bones Arduinos before & intend to build & document the 3.3 volt one someday. I recommend double sided boards instead of the single sided strip board in the Uber Home Automation Instructable.

I've built a node with a garage door open / closed indicator. I used a normally open reed switch & magnet at the bottom of the door so the magnet closed the reed switch & indicate the door was closed unless the bottom of the door was above the magnet. The sketch code detects the state of that switch & uses a new device to send to OpenHAB via the Gateway. That worked very well until it stopped a couple months after being deployed. Since the garage door is usually at the bottom & closed, I suspect the magnetically closed reed switch froze, maybe even welded (from the current) shut. I've ordered some normally closed reed switches & will rewire the circuit to the opposite logic: that is, if the (normally closed) reed switch is open (by the nearby magnet), then the door is totally closed. If the door rises above the magnet at the bottom, then (I hope) the reed switch will close itself & will indicate the door is open (from slightly to totally). I hope the reed switch will last longer this way. I intend to document this when I have the new approach working.

In case you want to try it, my Success... Posts  Sep 18, 2015 at 12:55pm, I document a Garage Door Opener end node. It's pretty cool to control opening / closing the door from my smart phone. (Even cooler when I can tell from my phone that the door is open or closed)

[lhw455]

OK, so while waiting for my new DHT22 and distance sensor to make a garage door node, I've been doing some research. My next plan is to create some temperature/humidity sensors, specifically for each part of the house. So I'll ideally need a low power, small size device. Now, I've been reading, and I found a site called mysensors.org, which had built a little ecosystem around the Arduino and the NRF24L01+ radio, which is a 2.4ghz radio. They've also got some great examples of tiny temp/humidity battery powered sensors, for example:

forum.mysensors.org/topic/2067/my-slim-2aa-battery-node
forum.mysensors.org/topic/486/my-2aa-battery-sensor
forum.mysensors.org/topic/1345/sensebender-micro (this is a good example)

The above seem to last for many months (as much as 12 months or more sometimes) on 2 AA batteries.

So my question, can we achieve something similar with Arduino and the RFM69? Similar battery use and importantly similar size?

[papa]

I'm currently working on some low power battery powered nodes. Will continue after weekend away.

[computourist]

gandalph posted his code for a battery powered node in this post.
Recently he reported it has been running on batteries for 5 months....

[lhw455]

Thanks! So my next question on the current setup! Now that I'm more comfortable, I wanted to add a distance sensor to my (only) end node, the temp/humidity sensor node. According to the original article (http://www.instructables.com/id/Uber-Home-Automation-w-Arduino-Pi/step4/Garage-Door-Monitor/), I need to connect the distance sensor to GND, 5V and pins 3 and 5 - is it as simple as doing that (for the hardware)?

And on the software side, I'm assuming I need to change the current DHT end node sketch and add the distance sensor details? What's the best way to achieve this, as I've looked through all the sketches on the github and I can't see anything which relates to a distance sensor node?

[papa]

I assume you're wanting to use an ultrasonic sensor. I worked with those back when, but not with this current project.

If / when I tried this, I would first search this forum to see if someone else provided something with an ultrasonic sensor. Then broaden the internet search.  Borrowing from the Uber sketch may be OK, too, but fit into the Computourist approach.

Once finding & testing code just for the sensor, you'd need to adapt it into something like the DHT end node sketch: reserve a device # for your sensor, add code for receiving commands & sending data for that device. To the respective OpenHAB config files, add items, sitemap lines, & (perhaps) rules.

To build the Node circuit, you'll need to stay aware of the voltage / current needs for ALL the components. From the data sheets, seems like Arduino's ATMega328P-PU operates fine 1.8 V to 5.5 V & RFM69 1.8 - 3.6v so 2 AA batteries should work well for them. However, data sheet says DHT11 should operate 3.3 - 6V. From its data sheet, add you new sensor's needs to whatever you hope to have on the node.  You'll need to connect sensor to Arduino pins not used by DHT & perhaps none of pins in the full DHT end node sketch.  Is it that simple as in the Uber Instructable?  It depends on whether the new sensor needs anything that conflicts (timing, power, etc) with whatever else you have on the node.

Good luck & when you get something working be sure to document so others can benefit as you have benefited from existing posts here. ;-)

More questions are OK, too, but please give enough details on what you've tried so far & the results.