ESP8266 OTA Update code for Sonoff, Ecoplugs, Wemos +similar

[jimkernsjr]

ESP8266 OTA Update code for Sonoff, Ecoplugs, Wemos +similar

Since Computourist did such a great job with the ESP8266 code, and specifically targeted a version toward Sonoffs, I decided this was in desperate need of OTA updating due to the fact that installed in some locations, Sonoffs are not accessed. Secondly, this device is a bit dangerous if proper care is not taken with it, so before I post the code, please heed these warnings! I've seen so much bad advice on the internet and youtube where people are so concerned with getting this device working they overlook these three most important safety steps:

-NEVER EVER hook your FTDI programmer to the board with the mains live!!! Let me repeat, NEVER EVER, not even to debug it, and I dont care if you left the 3.3v line disconnected or not, this thing is connected to dangerous mains voltage, and your device will be then also and possibly you! Therefore, it's particularly tricky to debug since you shouldnt be hooking up this FTDI adapter with it running! Additionally, most FTDI adapters cannot supply enough power to run it, much less activate the relay. I have found that some will, and some will not, and I'll put a document in the code with complete instructions.

    See this video starting at 9:30:   www.superhouse.tv/21-six-sonoff-secrets/

-It is never OK to leave ground wires disconnected!!! If you are outputting a ground, you need to CARRY IT AROUND the device. Just because there is no terminal for it doesnt mean you can skip it. Strip the ground long, and shorten the power connector, then use a WAGO connector to connect them in the box.

papa:  Also be careful to follow the N (so-called Neutral that can still shock) & L (Line or hot) mains connections on BOTH ends of the Sonoff case. In home wiring, neutral wires may have white insulation, line wires may have black insulation, & ground wires are often bare or in green insulation.  If you're installing a Sonoff into a two wire appliance or extension cord (taking to heart jimkernsjr's danger warnings), mark the neutral wire on both sides of where you cut the cord so you know where to connect to both of Sonoff's N terminals (& the other wire ends go to L).  On most 2-wire cords, the neutral wire ends in the wider prong on the plug.

-These Sonoff devices have precious little IP!!! (Ingress Protection) It has to be a 42 at best. IF you are going to put it anywhere that can be accessed easy, put it in an enclosure. A link to some nice boxes:
www.aliexpress.com/item/Waterproof-Clear-Cover-Plastic-Electronic-Project-Box-115x90x55mm/32758724664.html?spm=2114.13010608.0.0.P3V9IH
..for a double. They have a single 100x55mm also.  See attached pictures

-It has a rating of 90-250VAC. If you are going to put a dimmer, put it AFTER and not before. See attachment for a good way to enclose it, with a triac after it to dim my Christmas tree lights.

Additionally, watch the solder joints on the mains connector. I had one that wiggled loose that would have caused a loose connection and probably heat up. If it seems loose at all, reflow it with some lead based solder. Stupid counterproductive ROHS...

Next post, programming it and troubleshooting uploads:

Attachments:

[jimkernsjr]

Programming a Sonoff properly can be tricky. Best is to make up a premade cord for your programmer so you can eliminate errors.
     In this post, I attach a 2 pictures of my premade cord.
Starting from the fragile push button (sometimes an external one is better) left to right,
        the pins are connected as follows:
    Sonoff pin on strip        connect to FTDI
3V3                                          3V3
Rx                                            Tx
Tx                                             Rx
GND                                          GND
N/C

There is no need to solder a header. You can place the programmer in the holes, and hold some sideways pressure to make a good contact.

Make the proper settings in the Arduino environment AFTER installing support for the ESP8266 boards. Instructions here:

...but stop at the heading "Selecting the ESP8266 Thing Board", you don't need to go any further

Open Arduino(I use 1.6.8, some older versions don't have proper features, so use at least this version).
-In Tools menu, under boards, scroll down to the ESP8266 boards section, and select "Generic ESP8266 Board". This will give you all options!
-Don't touch Flash Mode, Frequency, and CPU frequency, leave those default.
-All Single Sonoffs have 8megabit flash chips, which is plenty for OTA of this code. You can upgrade them if you want to run Tasmota firmware, see the link in the above post to Superhouse. 8megabits/8 = 1 Megabyte flash. However, we need space in the FLASH area not the SPIFFS (SPI Flash File System) area for the upgrade code. Keep in mind when programming or adding new features, you cannot go over 50% of the program space and keep OTA working. In my next post I'll show you how to tell if you went over and troubleshoot uploading OTA. So we need to have as much program space and very little SPIFFS as possible. (I don't use it for anything yet, but it's a built in SSD drive for storing files, settings etc...) See the attached image to explain how it must fit. 
So, if we choose 1megabyte with 128kb spiffs, 1024-128=896.  896/2=448.  Currently, here is where my attached code is at:
Sketch uses 279,689 bytes (31%) of program storage space.
...so I can grow it 169 bytes 169kb or 169,000bytes more to 50%.
Now you understand the memory and SPIFFS, select 1M(128K SPIFFS) from the flash size.  If you want to verify your flash size, upload the sketch EXAMPLES/ESP8266/CHECKFLASHCONFIG then open serial monitor.  It will tell you the size and if you selected the correct flash.
-NEXT,  leave Debug Port and Debug Level to disabled and none.  Next post I'll tell you how to use them to troubleshoot upload OTA issues if you have them.
-Leave reset method on CK
-Speed up the upload speed to 230,400.   Wemos Mini's can take 921,600!
-for Port select the port your USB FTDI will be on when you plug it in in the next step.

Now place the programmer in the holes, and hold some sideways pressure to make a good contact. While doing this, HOLD DOWN the switch, and connect the USB of your FTDI to the computer. To Reiterate:

Connect FTDI, holding sideways pressure with one hand
Hold down the Sonoff's switch with the same hand
With your free hand, plug in the FTDI.
Check/Change the Port

Then upload.

After it uploads, you should be in business!
Close the Arduino IDE interface, and open it again.  Then under Tools/Ports your OTA ports should be alive!  For me, I had to restart Arduino IDE for them to show.
See image

If they don't show, make sure you followed my advice for naming this port(if you did it manually like I did and not let the sketch assign)!!  Limit about 20 characters, no underscores.
Sometimes, NONE showed up if even one followed the rules!

Papa: When the OTA port name appears under Arduino IDE Tools/Port/Network Ports, select that OTA port name & upload.  (To me, upload speed of 115200 seemed more reliable than a faster speed.) When it's done uploading (via OTA), wait a bit for the Sonoff's green LED to show WiFI & MQTT connection.  If you want, you can switch back to the serial port & check the Sonoff for the programming change via OTA.  You may need to restart IDE for serial port & monitoring to work.

jim: Tomorrow I explain troubleshooting, and I have to clean my personal info from the code and upload it.   Stay tuned...

[papa]

Great documentation on using the Basic Sonoff, jim. I really appreciate your making this detailed contribution to the forum. I know the effort that goes into careful documentation. I look forward to your next posts on this.

Some follow up comments in relation to what you posted so far above.  You give several important cautions, including about the possible presence of ground wires when the basic Sonoff has only connections for Line (Hot, black insulation?) & Neutral (white insulation?) wires.  Right, never leave ground wires disconnected.  Some extension & appliance cords have only two wires, Line & Neutral, but other situations have a third ground wire (green insulation or bare?).  As you indicate, jim, connect line & neutral wires to the Sonoff's input & output terminals, but connect one end of a ground wire directly to its matching ground wire.

 In this post, you mention pictures of your premade [programming] cord & your setup. I don't see those pics. Maybe you mean to add them later.

You say "no need to solder a header" on the Sonoff's programming holes." Maybe not, but it sounds awkward holding the cord's pins in the hole & holding the pushbutton down & plugging in the FTDI's USB cable. So far, I prefer soldering a header of female sockets & using male to male dupont cables between the Sonoff's added programming header & the female sockets of my FTDI device.  That makes good programming connections while I deal with the pushbutton & plugging into the computer's USB port.

Another caution: To avoid damaging the Sonoff, the FTDI device (for connecting to Arduino IDE via the computer's USB) must output 3.3 volts, not 5 volts. I have a FTDI device from Anarduino.com that switches between 3.3 volts & 5 volts.  It requires some soldering to complete.

About connecting wires to the Sonoff's mains terminals (input & output): Ads sometimes show a Sonoff installed in the middle of an extension cord or an appliance's power cord. I made such an extension cord which I believe will be useful, but it was a real pain to accomplish. The Sonoff's mains terminals & their holes for the wires are pretty small & not very deep. Extension or appliance cords typically use multi-strand wires that do not insert insert safely & securely in the holes. I tried adding some solder to the end of the multi-strand wires, but then they were too big for the hole. For a next time, I might install a short stub of insulated solid wire into each mains terminal. Then perhaps I could solder the multi-strand wires to those stubs & use some good heat shrink tubing to insulate where needed. I'm still mulling this over, but wanted to warn others & perhaps hear good ways to handle this.

Again, thanks, jim, for this great documentation on using Sonoff ESP8266.  I look forward to seeing more & to further collaboration.

[jimkernsjr]

Hi Papa...  Quick followup to your last post ...

Point one, yes, carry the green or bare ground THROUGH IF you have it.

Point two...FTDI progammer & premade cord pics attached:

Point three...yes, you  can of course solder a header, but waste of time and materials for me.  With my premade cord & pin setup you can push and hold the whole thing in the hole and its easy to hold the switch and header with one hand and insert USB with the other...  After you install OTA, youll never use it again anyway...

Point four...the FTDI I use has selectable voltage.  I always keep it on 3.3, and most time it still works with 5v as well, as 3.3v is over 1/2 of 5v and it still represents a logic high.  But thanks for pointing that out, because I never use 5v programming stuff I missed that!

EDIT!  For those of you wanting to build a sensor around the ESP8266 node code, and that sensor will use the analog pin (A0-only one), please note it works differently than Arduino!  With Arduino, you can set the reference with the bare chip, for ESP8266 it's a reference of 1V ONLY, and not 3.3.  This means you will have to make a voltage divider to deal with it.   Not really applicable to this Sonoff though, but if you bought a Wemos for testing and want to use it I thought I'd mention...

Point five...It's a pretty bad idea to put a sonoff by itself in line with an extension cord!!!  There is no doubt PAPA is careful and knows what he is doing, but other people you make this for or kids wont be!  This is why I linked the Superhouse video and took the time with my first post.  Unless you are absolutely sure no one will EVER trip over that, this solution offers NO strain relief for the cord!  You could yank bare live wires out if you trip over it or they work loose.  The little covers do not set firm enough into the plastic to hold anything firmly!  DO NOT DO THIS IF YOU HAVE KIDS AROUND!!!  Use the box I suggest and do it right, it's only $2 and you only paid $5 for the Sonoff!

[jimkernsjr]

p.s. on how I install stranded leads into Sonoff's mains terminals: I twist the bared ends tightly and solder them. This way you don't get a George Michael (Careless Whisker) sticking out of the terminal. DO not let the Solder build up, just enough to coat it. Back the screws almost all the way out of the terminals, and use a pick or sharp object to open the metal flaps up more. The cheap extension cords will fit right in if done properly, and you shouldn't be controlling heavy loads with this device anyway. As you can see above in my Christmas tree box, this is what I used and did...

[jimkernsjr]

Troubleshooting Preparation.  After a Sonoff is programmed with OTA (Over the Air) programming code ...

If the OTA ports do not show up (in Arduino IDE/tools/board/port):
-Limit OTAname to about 20 characters, no underscores.
-Use a very simple name to troubleshoot, and expand if need be.
-Make sure to use Arduino 1.6.8 at least with the extensions for ESP8266

If the port shows up, and you cannot upload:
Easiest thing to do is to recompile and re-upload with the debug options on.
From the Tools Menu, under the boards section, change the "Debug Port" to Serial, and change the "Debug Level" to "OTA + Uploader", and re-upload via serial like you did in a previous step.
At this point, you will need an alternate serial Terminal Emulator program like screen for Linux or Putty for Windoze.
Start your terminal program and watch the board via this terminal. Easy way on linux is "screen /dev/ttyS0 115200". Change of course the port (/dev/ttyS0" to your FTDI adapter and the port speed to match the serial baud IN THE SKETCH, not the upload speed.
I quickly unplug my adapter and replug and quickly execute this command. Then in screen or Putty you can see your sketch boot.
At this point, I may mention you may get stack traces if your FTDI adapter does not have enough power! So you may have to provide the Sonoff 3.3 volts separately. The one I have is this guy as you see in my other pictures and he does work:
www.aliexpress.com/item/Free-Shipping-FT232RL-FTDI-USB-3-3V-5-5V-to-TTL-Serial-Adapter-Module-forArduin-Mini/32519490747.html

This is also why I recommend you buy a Wemos D1 Mini and a relay shield to start off with. Its easy for a first troubleshooting with it's built in CH340 chip and no power problems. After you have that mastered, better then to move on to a Sonoff. But I digress...

At this point you should have live serial data on screen or Putty. Switch back to the Arduino environment, and switch the port to the OTA port of this device. Then retry your upload. When it fails, look at the output on screen or Putty and it will tell you the REAL REASON it failed, not the nonsense error from the Arduino Interface.

Next up the code:

[papa]

Thanks, jim, for added recommendations, including installing multi-strand wire into the Sonoff's mains terminals & trouble shooting the OTA (Over the Air programming) ports. & I second the caution about installing a Sonoff in the middle of an extension cord without added protection. I will be careful.

I also like the labels you put on various pins & connections (like with the FTDI).  I often use such labels & am convinced they helps avoid frustrating & even damaging mistakes.

For quite a bit of safe experimenting & troubleshooting, I removed the Sonoff circuit board from its case, connected the Sonoff to FTDI & computer's USB, used a solderless breadboard when expanded access to Sonoff connections was needed, & did NOT attach to the mains terminals.  Given that setup, Arduino IDE's serial monitor output gave me adequate feedback for testing & developing.  (Again while mains power was NOT connected) My FTDI's supplied enough power to show a change in relay state.  Only after I got the sketch working did I install mains power input & output & close the case.

Seems like no soldered programming header & OTA programming the Sonoff is working well for you. Unless an ESP8266 device (built in or upgraded) has larger memory than 1 megabyte, I'm wondering (with OTA requirements) if we'll have much memory slack to expand features in our node sketches. I guess I can better gauge that after you're able to post here your code, to which I look forward.

[jimkernsjr]

Hi Papa...

See above. The current sketch is at:
Sketch uses 279,689 bytes (31%) of program storage space.
...so I can grow it 169 bytes 169kb or 169,000bytes more to 50%.

Which is plenty considering there's no access to but 1 more GPIO and most of your control is in OpenHAB.
Remember its very easy and cheap to replace this flash memory chip if you want to run something bigger like Tasmota.
See the Superhouse video around 3:45 on how to, and here is a link also:
www.aliexpress.com/item/W25Q32FVSSIGTR-W25Q32FVSSIG-W25Q32FVSIG-25Q32FVSSIG-25Q32FVSIG-SOP8/32627854582.html

Part:
WINBOND W25Q32FVSIG SOP8
and another how to:
tech.scargill.net/32mb-esp01/

Its also another reason not to solder that silly header...that's one more thing in the way if you want to do this upgrade!

[jimkernsjr]

Onto the code:

I've separated into code and header (.h) files. There's little point to have separate projects targeting the same hardware and end up with different versions of the code base. Therefore, if we upgrade the code, we can just drop the code file in and upload without messing around with personalized settings. All that changes is the (hardware specific) header includes at the top.

.ino file:
in the first 18 lines, you will notice there are several includes, each to go with the corresponding header file that will open up in the tabs of the IDE. Uncomment the one you wish to use, and then choose the corresponding tab to customize your settings. I've given several examples there, some for Sonoff, some for Ecoplugs indoor, Ecoplugs with Power monitoring (to come, I haven't finished...but the code will be a drop in replacement.), and an EcoPlugs Outdoor and a test version to run on the Wemos D1 Mini with relay shield.

.header files:
-the version I customize and put that in the Openhab User Interface on a version pages. This way when I push an update I can see its version and the change took effect. Customize to your liking if you use this feature, or generalize it...
-Wifi access point A and B...see Computourist notes...
-node ID - obvious - must be unique
-debug - keep it on, pointless to turn it off it wont save any power and only 224 bytes. You have 169kb or 169,000bytes free. Please forgive my erroneous math earlier.
        Three new functions I added:
#define OnAtPowerup 1 //jim - 0=off, 1=on
#define RETAINMSG 0 //jim comment to make non retained. 1=retained, 0=no
#define STAONLY //to turn off the built in AP

-On at power up will make the Sonoff do just that. I like to still use my switch if I put them near the light. This way, I can turn the switch on/off and have it work normal, and then the home automation can take over if it's on. So you can interpret your light switch as OFF/ON AUTO. If you dont want that, ovbiously put it at 0

-Retainmsg - persistent messages annoy me. I like them to go away. Thanks Computourist for the help with getting rid of them, I put it as an option. Again, if you like this, put it back on to 1.

-STA only. As you deploy tons of these ESP things, they will host in AP mode as well. It annoys me to see them all in my Wifi networks, so I leave them off. You may want that if you implement a special function. (perhaps a fallback web page OTA update if you replace your AP and it cannot connect?) Anyway, looked like a bit of a security risk to me. I use the ESP8266 on my window shades and I dont want someone connecting to them and rolling them up at night (I wont go there....LOL)

#define ACT1 12 // Actuator pin (LED or relay to ground)
#define MQCON 13 // MQTT link indicator
#define BTN 0 // Button pin (also used for Flash setting)

-Act 1 is the activator pin the relay is on. 12 for Sonoff, D1 for Weemos, 15 for Ecoplugs indoor + outdoor.
-MQcon is the LED I think more research may be needed on this, I just do not care about it. I have some off the old version Sonoffs where the green light is tied to the relay. There is a hack to enable the red LED(see the support files directory or computourist's wiki). I think in the black label Sonoff this may be a bit different, maybe someone can look into it.
-BTN is obviously the pin the push button is wired to. 0 for sonoff, 13 for ECOplug, and the Wemos is mostly on D2 I believe although it depends on the shield. Some are configurable... I put you a spreadsheet in the support directory, along with a copy of these posts.

more...

[jimkernsjr]

...code continued.

//#define PMenable //enables the power management feature
//#define PMpin 5 //the PM read pin

These two are for the Power monitoring of the ECOplugs and the Sonoff POW I havent finished yet.  If someone has some research on that, pass it on, but right now I dont have the time or a direct need so it will be a while until I finish it.  Under images-ecoplug is a discussion, there is a head start in there about it if anyone is energetic.  It's as far as I got.  papa:  I don't believe PMenable & PMpin are in jimkernsjr's posted code yet, probably because it's not ready yet.

And the grand finale...the good stuff:

#define OTAenable //enable OTA code upgrades
#define OTAname "ESP8266-Sonoff-N46" //enable OTA code upgrades- --Max Chars about 20 & no _ (no underscore) or the port wont show!
//Limit 20 Chrs:|--------------------|

With those settings, you can turn on and off OTA. With it on and 1M/128kb SPIFFS:
Sketch uses 231,029 bytes (24%) of program storage space. Maximum is 958,448 bytes.
    With it off:
Sketch uses 236,522 bytes (24%) of program storage space. Maximum is 958,448 bytes.
Both are without debug on as well  (again, pointless to disable...not much savings).

With full debugging (OTA+Updater, and serial debugging enabled) and OTA enabled the max I came up with is:
Sketch uses 247,630 bytes (25%) of program storage space. Maximum is 958,448 bytes.

Last is the port name.  Keep it simple, avoid special characters except - (dash or minus sign) and keep it under 20, or the port won't show up (unless this is fixed in later versions of Arduino)

I wish you all good luck with it, and hope it makes your life as easy as it does mine.  Opening up the ECOplugs to reflash is a pain!!!
Special thanks to Computourist for this great work.  So much better and cheaper than those pesky RFM69 radio's when you need something simple.

code in the next post:

[jimkernsjr]

Jim:  OK, it's too big to attach here.  Find it on my Github:  github.com/jimkernsjr/ESP_SonoffAndEcoPlug

    papa: This seems a good time to make sure your WiFI router has a robust password for connecting to it.
At jim's github, each node's customizing is in a header file (ends in .h).  In each .h file, you must supply your network info in these lines:

#define wifi_ssid_B "YOURFIRSTACCESSPOINTNAME" // wifi station name (SSID)
#define wifi_password_B "YOURwifiPASSWORD" // wifi password

#define wifi_ssid_A "YOURsecondACCESSPOINTNAME" // fallback wifi station name; if not present use same as A
#define wifi_password_A "YOURwifiPASSWORD" // fallback wifi password; if not present use same as A

#define mqtt_server "10.1.1.15" // mqtt server IP  (change this to IP of YOUR computer hosting MQTT server)

    Jim continues:  Happy OTA Uploading!  (You'll be greatful, those cheap switches bust off the Sonoff after a while in the heat!  I have 2 down so far that have been in my garage.)
(if you want to fix those later, don't solder a programming header-you'll be sorry and they will break out in the heat!)

    Papa + Computourist:  Can you work together to merge it back to Computourist's master if he wants?  I put a lot of time in it already, and I must move on to other things...busy with work. I don't want to be the master and approve commits and such, I don't have the spare time to monitor it. I'd rather concentrate my efforts on the RFM95 now since this fits my needs.

If anyone else wants details of the ECOPlug, you can get them at Walmart in the home improvement electrical aisle, and on Walmart.com.  The Woods version of the same thing has power monitoring.  These are so much safer than a Sonoff on an extension cord (Don't do it without a case with strain relief!)  If you need details how to program it, let me know, its more tricky than the Sonoff and you must solder wires to the onboard ESP8266 itself, see the ecoplugs-images directory.  The large image is the way I program it.  To pull GPIO0 down, I use the green ground wire with a male plug connected to ground on my programmer.  When I'm done, I just tuck the wires behind the board and close them up in lieu of removing them.  It will pay off now when I put my new OTA code on!  But if you have questions, I'll do my best to answer.  You cant possibly have questions on the Sonoff with all the details I put!

Good luck to all.  Happy OTA uploading.

**EDIT**  I also have a version with the HTTP uploader, useful for those of you than may be using Atmel Studio  to work on your code.  If you want it, let me know but please don't ask unless you have a true need because it needs some cleanup yet, and you are on your own with Atmel Studio - I dont have time to document all of that too.

[jimkernsjr]

Hi everyone...

I left off one very important point!!!

You must install Python 2.7 for the upload to work(And it must be 2.7 and not 3.x). I had it already on my linux computer, I didnt think anything about it!

otherwise you get this wonderful stack trace:
java.io.IOException: Cannot run program "python.exe": CreateProcess error=2, The system cannot find the file specified
...
...
...


For Linux, easy...here is a link:
askubuntu.com/questions/101591/how-do-i-install-the-latest-python-2-7-x-or-3-x-on-ubuntu

Sorry I don't use Windoze, get Papa to help you with that or you can google it...this seems legit:
python-guide-pt-br.readthedocs.io/en/latest/starting/install/win/

[jimkernsjr]

I decided to update my Ecoplugs while I'm hot on the topic.  For those of you that have or plan to buy, here is the way to connect to FTDI:


Its big enough for you to download and zoom.  You can also see the pinout in the esp-images directory of my code.

You can easy see the connections.  The lower right green wire is the GPIO0 that needs grounded when you plug in the FTDI...Hook it into the ground hole of the FTDI, or loop it into the gig ground hole planes of the daughterboard that the ESP is on itself...either will work.

I don't think I need say to not fool with it plugged up...
To test, I do plug it into a power strip loose with the daughterboard off to the side before I put it back together (it's a pain).  But be careful where you touch, even the daughterboard is connected to the mains!!!  You have been warned.

[papa]

Thanks so much, jim, for this major effort in programming & careful documentation. You've added much more depth to our coverage of using ESP8266 devices with this forum's approach to DIY Home Automation. I'll aim to try as much as I can of what you have provided & solve my own problems as much as I can. I may have some questions, but will try to minimize those. I've begun researching how to use Github. Once I get some familiarity & confidence with what you provided & with Github, I'll look at what might be done at Github.

[jimkernsjr]

Thanks Papa...

If you have question, PM me. we can work it out then post. Then you can help the people. Sorry I contributed as much as I have resources for...
I tried this all from scratch on several different computers so it all worked well for me. Like I say I don't use Windows though so I don't know if you will run into a snag there or not. I tried on my Windows machine but it had no Python, and I dont waste time for that to install. I'm hoping you can check all these points and make sure it's smooth!
I dont know if anyone has any interest in the ecoplugs or not, but I put as much as I documented. There are alot of relay boards and plugs out there with the ESP8266, ESP8285, and the PSA module this will all work with. I have a collection I bought, I'll try to add more because on the PSA modules info is scarce. For $20, the Ecoplug is cheap and you can pick it up right away at Walmart and hack it and safe too.

I add that a good way to test your OTA's to turn your Sonoff or Ecoplug on. When it updates and reboots, you will see it power cycle.
Have fun all, but be careful! No need to create dodgy solutions when inexpensive or alternate products exist!

[papa]

I experimented with jim's above code & instructions for working with my Basic Sonoff & my Windows 7 computer which hosts Arduino IDE, OpenHAB, MQTT, & Python.  I made some corrections & clarification in this post above.

I customized the ESP_SonoffSingleNodes_Node_47-RearDeckFloodLights.h file to program a ShopLight Node 30. The serial upload & reboot of the Sonoff went fine & showed good results on Arduino IDE Serial Monitor & OpenHAB (given the right OpenHAB config entries).

Then I fussed several hours trying to get the Sonoff Node's OTA name to show under Arduino IDE Tools/Port/Network Ports so I could try an Over the Air upload to the Sonoff Node. The Sonoff's OTA name was already short with no illegal characters: ShopLt30

An internet search showed other Windows users had trouble with this & I tried mentioned suggestions: 1) Check the Windows Firewall. I temporarily turned it off (no change)
        On the WiFi Router (D-Link Dir-615 for me) ...
2) Try disabling client isolation. On D-Link, uncheck WLAN partition. (already disabled so no help).
3) Try tweaking router config related to MDNS. For D-Link, enable Multicast Streams.

Though the above 3 things might help other Windows users, I know the first two gave me no help. Suggestion 3 might have helped, but I don't know.

Turns out the my local network setup (which works well for me in other ways) was working against me for the OTA port: Internet Modem -> Wired (only) Router (DHCP activated to assign IPs) -> WiFi Router (DHCP OFF). Most computers & devices (including OpenHAB/ MQTT / Arduino IDE host) usually connect to my network via Ethernet cables connected to either router. Of course, the Sonoff node connects via WiFi. Maybe after a long time, the ShopLt30 OTA port might have gotten through to Arduino IDE, but so far it was not. I got the OTA port to appear much faster this way: The OpenHAB / MQTT / Arduino IDE host computer is a laptop with WiFi that was usually turned off. When I turned on the host computer's WiFi (in addition to wired Ethernet connection), the OTA port appeared.

Lesson learned ??  At least on Windows computers, trouble seeing the OTA port can have various possible reasons & solutions.

Once the OTA port reliably appeared in Arduino IDE, the upload via OTA worked nicely.  I had at least one OTA upload failure, but as jim explains below, this failure could not be caused by using a faster speed than 115200.  Such speed setting only apploes to the upload over the serial line via USB.  After failure, trying OTA again worked. Also below, jim explains shortcuts to repeating an OTA session.

Oh, BTW, the first time I successfully ran an OTA upload, Windows Firewall asked permission (once) to let Python be involved.  With permission given, upload proceeded.

PS to underscore what jim says, when we change ports (from serial port to OTA network port & back again), it seems best to restart Arduino IDE to make sure the port we need is live.  Sometimes live ports don't show until restart & sometimes disconnected ports still show in IDE until restarted.

[papa]

As indicated in the previous post, I have successfully tested jim's ESP8266 OTA (Over the Air programming) code & documentation (& made some corrections & clarification.)

What now? jim, I hear your reluctance & time constraints to be Github master & maintainer of the code & documentation. Obviously. computourist has access to this thread & can decide whether he has the time & interest to pull your code into his. (Probably off & on) I'll be figuring out how to use Github.

jim, it seems to me that unless you give me or someone else the login for your Github repository (questionable idea?), you will either have to master / maintain it OR let it stand unmaintained. Here's what (A & B) I suggest we use to provide support yet probably help you avoid unwanted involvement:

A) For now, jim, I suggest you consider these minor(?) updates to your Github repository: 1) At a minimum in the repository, you could add a prominent link to this OTA thread & even say it's users' best opportunity for help. With update 1), users of your Github repository could come to this thread for help & updates not in the repository (I'd be willing to provide as much help as I'm able & pass along what I cannot do). 2) You could also add the correction & clarification I made (indicated by dark blue text) in this post. With update 2), I believe your Github documentation could be in pretty good shape for now.

B) jim, what I may also do when able is fork your repository & then (giving credit to your contributions) adapt it according to my ideas. Your approach to customized programming is an interesting alternative to my choices version of computourist's RFM69 work, but I might adapt it some. I also have ideas & one working prototype about adding ESP8266 node features besides push buttons & relays to the main code. At my Github repository & this forum, I would commit to maintain the parts of your work I'm able to confirm & of course my adaptation of your work.

[jimkernsjr]

Hi ...

In my haste to complete, I inadvertently left out a good troubleshooting step for first time. 
Before you upload OTA, Click File, Preferences.
    Turn on:
Show Verbose output during UPLOAD.  That way you can see communication problems.

But one thing Papa had said that is not quite correct about the serial port speeds:
This speed selection is ONLY applicable to serial programming.  I told you to speed it up because I was holding down the Sonoff button while I uploaded (sometimes when I left off, it moved and it botched).  So quicker was better, but I got the hang of it and it was second nature.  Papa:  I corrected this misunderstanding above.  Thanks, jim for explaining.

        For Uploading OTA this is how it works:
Arduino Compiles a .BIN file and plops it in a temporary directory.  (If you only select "Export Compiled Binary" this is where it stops-no further action)
With Arduino's and serial, it calls AVRDUDE to do the upload.
With ESP's and serial, it calls ESPTOOL.        With ESP's and OTA it calls ESPOTA.

With OTA, the Baud rate has absolutely no affect - it's not passed to the command line.  See below examples where 1st scenario uses ESPTOOL and 5th parameter is the Baud passed.  With ESPOTA no Baud is passed.

ESPTOOL:
/home/jim/.arduino15/packages/esp8266/tools/esptool/0.4.9/esptool -vv -cd ck -cb 115200 -cp /dev/ttyS0 -ca 0x00000 -cf /tmp/build0e8350bfa3e8eb022f3b7a0ec772e118.tmp/ESP_SonoffAndEcoPlug.ino.bin

ESPOTA:
python /home/jim/.arduino15/packages/esp8266/hardware/esp8266/2.3.0/tools/espota.py -i 10.1.1.120 -p 8266 --auth= -f /tmp/build0e8350bfa3e8eb022f3b7a0ec772e118.tmp/ESP_SonoffAndEcoPlug.ino.bin

So Papa, your OTA uploading problem must have been something other than the Baud.

Also useful for troubleshooting: you can open a command prompt and copy paste the IDE's call to python in the command prompt to do a repeat if you fail.
Also, you will note that the OTA port name is not passed.  Another way to troubleshoot is to find your ESP/SONOFF's IP address, choose "export compiled binary" and fire off python with the parameters above and manually try.  Even if the port is missing from Arduino it will work - the ESP is still listening regardless. 

**EDIT**  If you still have troubles, it goes without saying to try a simple ping first, then a port scanner to troubleshoot comms between your Dev Environment IDE and the Host (ESP8266).  Remember that your node reports the IP address on startup if you have it enabled (Line 587) to MQTT-open up your broker, apply the filter, and reboot the node.  For Port scanners, there's Nmap for Linux, and I think they have a Windows too, but googleing will find any number of them.  Again I hate I omitted that, it would have saved PAPA and some of you who are trying this for the first time some headache.

Hope it can help all in the troubleshooting.

Papa...I'll personal message on the other topics ASAP.

[papa]

From jimkernsjr, I have accepted maintaining his version of Sonoff/ESP8266 OTA code with help as needed & inclination from Jim.

I prefer to do that maintenance mostly in this forum & thread, but (as I learn to use it) will also endeavor to maintain Jim's github repository.

I've done more experimenting & tinkering with Jim's update to computourist's Sonoff/ESP8266 code & Jims wealth of documentation. I repeat: It's great stuff & a major contribution to this forum's approach to DIY Home Automation.

I love using the .h header files to customize a node & preserve a record of that customization & its last OTA update.  Also once a Sonoff is OTA ready, as Jim predicted, I really love the OTA uploading. It seems faster & more reliable than the USB / FTDI serial programming. As Jim also predicted, available memory is so far quite enough for OTA programming the code we've used on this forum. So far I doubt I'll go to the trouble of upgrading Sonoff's memory.

[papa]

As indicated above, I highly respect what Jim has provided, but may also offer differing perspectives & approaches on some aspects.

Again take to heart Jim's caution about using only a Sonoff case to protect contact with mains voltage. ONLY IF you install mains wires as carefully & securely as possible AND better IF you install the Sonoff where access is restricted (behind a wall or ceiling, in a switch box or junction box), a Sonoff case might be enough.

Prevent mains wires' disconnecting from Sonoff terminals: In the gap between the lower teeth on the case bottom & on each side of the paired mains wire's insulation, we might also drill small holes. Again between the lower teeth, insert a piece of thin (24-26 gauge?) insulated wire under the mains' insulation. Wrap the small wire over the mains insulation & back down through the drilled holes. On the bottom of the case, pull & tightly twist together the small wire's ends. Help the plastic "teeth" grip the wires by using pliers to somewhat press the wire's insulation into the lower teeth. Then push the protective cap down hard while you install the fastening screws.

Or instead of a small insulated wire, my son recommends hot glue.  Between the bottom pair of plastic teeth & under the insulated mains cord, put a little hot glue & force the cord down on the teeth & glue.  Again push the protective cap down hard while you install the fastening screws. To keep someone from unsafely poking into mains terminals, hot glue could also fill a case exit gap next to the cord & on top of the cord:  Pile some hot glue between the bottom pair of plastic teeth & next to the insulated mains cord.  If needed put a little hot glue on top the insulated mains cord.  Just be careful not to glue the screwed down cap so it cannot be removed.

Repeating: Always make the mains wires as secure as possible AND better put the Sonoff where it is not very accessible. If there's any doubt, put the Sonoff in a secure added case & use recommended strain relief on mains wires. Be VERY, VERY, VERY careful.

[papa]

Not soldering a female socket header on the Sonoff's programming strip seems to work well enough for Jim. With the case top off, that requires pushing the button when the button's tall stem is not protected by peeking through the hole in the case's top & might break. So (like Jim) I recommend you soon move on to OTA programming & do not use the temporary wired connection any more than required. Since it took me a while to get the OTAname (port) to show up in the Arduino IDE, I appreciated having stable, hands-off connection via the header while I pressed the exposed push button to initiate Sonoff programming mode.

Via a header or a directly soldered wire, I plan to have some access at least to the two programming strip connections farthest from the push button. The next to the last connection is GND & the farthest connection is GPIO14. With some code (provided later), these connections can add another sensor feature to the Sonoff.  Adding any connections to that strip on the Sonoff circuit board are much easier before connecting wires to the mains terminal.

Again as Jim predicted, available memory is so far quite enough for OTA programming the code we've used on this forum. So I doubt I'll go to the trouble of upgrading Sonoff's memory. That's one less reason to avoid installing a header. If a project's code is big enough to require a memory upgrade, I'll just upgrade an untouched Sonoff (even if I need to buy another).

[jimkernsjr]

Hi Papa...

Actually, over time, that piece of junk button just blows apart by itself.  All of mine in the garage just fell loose when I took them apart one final time to put the ota code in.  I just solder a short piece of wire to either side the switch and connect together during upload, and run it out to an outside switch for remote on/off when my sonoff is protected. Or tape over it and tuck it back under the cover.  Maybe in an "easier" environment they may fare better.  Those were also some of the first origional Sonoffs that you had to do the Computourist hack to get the red light to work too, so maybe they improved but it looks the same button to me.

One note about Papa's safety recommendations above:
-Probably easier to just use wire ferrules.  In some countries it is actually illegal to use stranded wire in screw terminals.  Check your local rules.
-It's also highly illegal in the US and probably most other countries to put it "behind a wall".  You are not allowed to have hidden junctions of any kind.  Put it in a box, period.  Dont be cheap, you only paid $5-6 for it, you can afford another $3 for a real nice one like I have shown you.  And for $8, there's nothing else you can buy this cheap.

P.S.  Walmart has the Ecoplug variant on sale $10.  Pinout on this one same as the one I provided for you in the download package on github, Picture attached. Link:
www.walmart.com/ip/WorkChoice-1-Ol-Wi-Fi-Indoor-Switch-White/46721977
...so don't do crazy foolish things with the extension cord. 
...and it even does power monitoring.  If someone has some time to work on it, I'd love to hear your results.  I don't need them for another couple months (christmas lights) so it will be a while for me.


- now what you have this OTA code, you'll probably never take the cover off again, but I recommend just snipping the lower bar on the top cover in one place where it runs under the wire terminal.  This way you can remove the lid to reprogram.  THIS IS NOT A SHORTCUT TO CIRCUMVENT TURNING THE POWER OFF TO PROGRAM IT!!!  NEVER TOUCH IT LIVE!  Unplug it or throw the breaker - dont be lazy.  You already saved so much time with this code here and all the hints, take an extra second and be cautious.

[papa]

Thanks, jim, for being vigilant about everyone's safety.

Thanks also for the alert on the Walmart sale on the Ecoplug. I ordered a couple.

I like the idea of using wire ferrules. One vendor described them this way, "These products terminate stranded wire for insertion into terminal blocks."

Interesting, I'd been thinking about cutting a little piece from that lower bar on each end of the top cover under where the mains wire enters the case. As you say, that snip would make it possible (ONLY SAFELY WITH SONOFF DISCONNECTED FROM POWER) to remove the case top without also probably removing the mains cord (that may have been tricky to install right in the first place). That snip could also make it a bit easier to install the mains wires in the terminals because one could do it with the case top off.

[jimkernsjr]

Exactly why I cut all of my lower outside bars on the top cover!

I'm vigilant about it because I've seen so much bad advice out there, especially on Youtube. My purpose is to make sure everyone coming here to use this does it correctly. I dont want to lure anyone in with this code only to find out they fry themselves or a family member or kid / pet in the end.

Wire ferrules are the perfect thing for this, and as I said required in many countries.

By the way, if you are looking to bring the GPIO line out the side of the sonoff, consider Sonoff TH instead. It brings 2 of them out to the 3.5mm jack, and the two it brings out are perfect for I2C! And the button on the TH - it's soooooo much better. Perfect even. And the case is so industrial feeling. It's really worth the extra money. You can get him in 10 and 16 amp versions, and it has a sexy mounting tab. I sound like a Sonoff salesman, but I'm not-they are just great products. I've always liked Itead-they make nice things and normally pretty proper.

If anyone buys a Sonoff POW or Ecoplugs like you purchased, I'd appreciate some help on the Power Management code. Papa, since you bought the Ecoplug, maybe you have the time? Read the PDF post in my images-ecoplugs directory titled "ECOplugs-Woods-Replacement-Firmware-powermonitoring-discussion.pdf" I think there are some people talking in that forum that will give you a head start.

[papa]

I'm waiting on some Sonoff TH's from China. Supposed to get my Ecoplugs about Monday.

I have some busy periods this summer, including the week starting tomorrow, but will see what I can do with Ecoplug in between.

[jimkernsjr]

Hi All...

Some additional troubleshooting steps and general suggestions.

If sometimes you notice that your ESP8266 based solution suffers from frequent disconnects/reconnects, you may be over-occupying the CPU with your own code and routine.  Just as you never use a delay in an ISR, you should never do the following when utilizing OTA or the ESP code in general:

-Marshall the code away from the main loop for excessive amounts of time.  This means call a function/library from your loop or ISR that keeps the CPR busy for more than 200ms.  I came across this problem with my windowshades, and as it was rolling the shade up, it was occupied with the stepper motor and was not returning to the loop.  This is bad for WIFI connectivity and OTA.
     -What to do:  Use yield() in your function if it's going to be busy for a long time.  i.e., my code was broken down into steps, which make up approx 10 degrees, rotation, which makes up 36 steps, and inch movement which made up 1.25 rotations, and finally 52" for shade up/down.  This is 52*1.25*36*10*2ms worth of time it was away form the main loop.  I inserted yield in each degree, but not OTA handle.  So the shade will resume OTA only when it's done it's movements, but the wifi will stay connected.

-Again, I believe there is a time limit that the code needs to call OTAhandle in the main loop.  If you have trouble uploading OTA, make sure your are calling OTAhandle frequently enough.

It's not in the main code base, and I did not put it there and neither did Computourist because for what it is doing, just activating a switch, it is not needed. 
However, if you design complex nodes like I tend to, keep all of this in mind to save yourself some headaches.

[papa]

Via my router's settings, I gave my Sonoffs permanent IP addresses. That might make them more likely to reconnect automatically after a power outage.

[papa]

I had fairly easy success working with the Sonoff device as described in this thread.

However, for re-programming the currently available ECOPlug version of ESP8266, I had trouble making the wiring connections & with getting an upload to work. The current version is constructed differently from what is previously pictured & seems to enter flash mode somewhat differently. From my experience, successful uploading needs one more connection on the ECOPlug's ESP8266 board. Adding a large capacitor can help.  I also worked out a programming jig to avoid the difficult soldering to the board's pads & to avoid cramming wires into a crowded case. See this thread for documentation.  That thread is meant to depend this thread on & supplement it.

[papa]

Various options exist for re-programming ESp8266 devices. jimkernsjr & the rest of this forum uses computourist-derived sketches to program ESP8266s. Those sketches connect to a computer (possibly a Raspberry Pi) that hosts MQTT & OpenHAB. For help installing those, see here.

[papa]

You might program a Sonoff or ECOPlug as Node61 & use it to control a light in the living room.

        Here are OpenHAB entries to accomplish that:

        .items file
Switch Act_Node61 { mqtt="<[mosquitto:home/esp_gw/nb/node61/dev16:state:default::]", mqtt=">[mosquitto:home/esp_gw/sb/node61/dev16:command:*:default]" }

        .rules file
rule "Random timer for Output ON around 7:30pm" // Living Room Activity on (Node61)
    when
        Time cron "0 25 19 * * ?"
    then
        // Schedule a random ON event up to 15 minutes from now
        createTimer(now.plusMinutes((new java.util.Random).nextInt(15)) ) [|
            sendCommand(Act_Node61, ON) ]
end

rule "Random timer for Output OFF around 8:30pm" // Living Room Activity done (Node61)
    when
        Time cron "0 25 20 * * ?"
    then
        // Schedule a random OFF event up to 15 minutes from now
        createTimer(now.plusMinutes((new java.util.Random).nextInt(15)) ) [|
            sendCommand(Act_Node61, OFF) ]
end

        .sitemap file
Switch item=Act_Node61 label="LR_61"
==========================

To toggle the light on or off, one can use the Sonoff or ECOPlug push button OR use the virtual switch on the OpenHAB User Interface. The first rule will turn on the light semi-randomly around 7:30pm & the second rule will turn off the light about 8:30pm.

    For help interpreting the above OpenHAB entries see this thread.

This thread continues on Page 2.