Disable RFM69 Transmit So Local Node Features Work

[brump]

Papa,

If the connection between the node and gateway was lost, all functions not responding. Is it possible change the code structure for accept this?

In this situation, the node's local control of its functions is not hindered when RFM69 transmissions are not working.

Regards, Brump.

papa: In order to use the principles of this thread, one needs to have a working RFM69 Gateway & a working RFM69 node that has local features that are controlled by the node.  This other thread guides you through building & programming an RFM69 Gateway & an RFM69 DHT Node.

[papa]

brump: "If the connection between the node and gateway was lost, all functions not responding. Is it possible change the code structure for accept this?"

papa: Do you mean that when the node does not connect to the gateway, you lose local control on the node's functions (like using the node's push buttons)?

We can possibly address this. When the nodes, don't get a radio connection, they may not proceed very far. We may need to change the code used when a connection is not achieved. I had a quick look at the code, but am not seeing what might cause this yet.

I may need something like this for my programmable cooking controller. When it's not connected to the gateway, it works, but with problems. For example, the time elapsed displayed does not change for 30 seconds of each minute.

Or maybe the problem is the sketch tries to continue after no radio connection, but it is often lost in code that won't work without radio connection.

If our nodes lose RFM69 communication connection, sometimes we may wish that local node functions would still work.  Other times, for safety sake, we may want local functions to halt until communication is restored.  In the former case, it would be good to have a way to ensure local functions continue to work.

[papa]

brump: "If the connection between the node and gateway was lost, all functions not responding. Is it possible change the code structure for accept this?"

papa: "maybe the problem is the sketch tries to continue after no radio connection, but it is often lost in code that won't work without radio connection."

Here's how this might be approached. My Cooking Controller sketch worked without RFM69 connection.  Then adapting from code in the node choices sketch, I added the radio coding in Cooking Controller sections that start ifdef RADIO so one could program it with or without radio connection.

In comparison to your sketch, look at the Cooking Controller sketch in this post, especially some sections that start with ifdef RADIO.  One of those sections is the function txRadio() whose line 335 says
    if (retx) Serial.println("No connection...") // end of if (radio.sendWithRetry)

That is, if retx is not zero, radio communication is failing.

Your version of the node choices sketch uses mostly the same radio code that I bracketed with ifdef RADIO ... ENDIF.

Perhaps we can bracket some of the radio code [but not radio setup or the txRADIO() function] with
         if (retx) {     }
so that that radio code would not execute & perhaps hang up the whole sketch.
        =====================

Again line 335: if (retx) Serial.println("No connection...") // end of if (radio.sendWithRetry)
    This is what repeats "No connection..." on the serial monitor.

Is this  "trying to send & No connection..." maybe all or much of the sketch can do if the radio does not connect?  Try adding more Serial.print() debugging statements in the sketch to see if anything else is going on.

I have some other (non-electronic) projects that I need to work on so I may not be able to address this thoroughly for a while.

[papa]

    A fix for allowing a Node's local functions to work when RFM69 transmission is NOT working:

I tested with computourist's original RFM_DHT_node_22.ino.  Before I changed the sketch, I found that with the gateway running, the node button controlled the LED/relay. Without the gateway running, the node button did NOT control LED/relay & the serial monitor just spit out "No connection."

I started thinking more about flagging a transmission failure & disabling RFM69 transmission so a node's local functions could work (like the DHT node's push button togging its LED or relay).  Here's what worked for me ...

1)  Near the start of the sendMsg() function is a line whose variable is never used again. Disable this line so we can use the variable elsewhere:
     bool tx = false; // transmission flag        << Disable this with // for comment

2) Earlier in the sketch after "int  numtx; ...", add the re-purposed line:
        bool tx = true; // ok to transmit ? flag

3) Near the end of the txRadio() function, find the section:
    #ifdef DEBUG
        if (retx) Serial.println("No connection...")
    #endif

                Change that section to :
if (retx) {
    tx = false; // flag that it's not ok to transmit
        #ifdef DEBUG
    Serial.println("No connection...Disabling transmission") ;
        #endif
  } // end of if (retx)

4) Near the end of the main loop() find the line:
   sendMsg(); // send any radio messages

Change that line to:
    if (tx) sendMsg(); // send any radio messages if transmission is working

I believe that how I tweaked the DHT Node should work on other computourist-styled nodes since they are based on the DHT Node.

[papa]

   RFM_DHT_node_22_noTX_pub_01.ino (15.46 KB)  << Demonstration sketch

I uploaded this computourist DHT node sketch that was changed according to the previous post & tested again.

As with all computourist type sketches, you will need to customize these sketch options to your situation: encryption key, RFM69 radio frequency, & possibly to the high power version of the RFM69 radio.

As before & wanted, with the gateway running, the node transmitted & the node button controlled the LED/relay.

Then I turned off the Gateway. Sometimes after a little wait & sometimes almost immediately, the node button still controlled the LED/relay. Sometimes the serial monitor (during the little wait) showed "No connection...Disabling transmission" 7 times. Sometimes the serial monitor showed "No connection...Disabling transmission" only once.

If after disabled, the gateway was running again, we may want the node to transmit again. I had expected that would require rebooting the node or adding more code.  (Via the node's reset button, toggling the node's power, or stopping/restarting the Serial Monitor) Rebooting caused the Node to resume transmissions with a gateway that resumed running .

More code could probably let the node restart transmission without rebooting: Something like 1) timestamp when transmissions were disabled 2) If a defined interval passes the timestamp && tx = false, set tx = true so the node would try transmissions again.  One might lose local control of node features while the node tried to resume transmissions.

brump, see what you & others think about this. Again, I believe that how I tweaked the DHT Node should work on other computourist-styled nodes since they are based on the DHT Node. I'll later try this with my Cooking Controller so its local functions work better when not connected to a gateway. After successful testing, I can add these code changes to the choices sketch.

[brump]

Hello Papa,

Nice approach.
I'll analyze the code and test the functions in the coming days. I've been involved in a big project for the past few days.
Tomorrow I will board and I will have time to test the features.

papa: brump, I look forward to hearing how this works for you & others.

[papa]

Again in order to use this thread's principles, one needs to have a working RFM69 Gateway & a working RFM69 node that has local features that are controlled by the node. This other thread guides you through building & programming an RFM69 Gateway & an RFM69 DHT Node.

                In the original computourist node sketches,
why does RFM69 transmission failure (mostly) prevent local node functions from working?

For each device's data, the sketch tries to send it 7 times (0-6) plus waiting 1/2 second after each retry. That means at least 3.5 seconds (7 times .5) elapses per each device that is flagged for periodic transmission. I had 8 devices sending & that took 38.5 seconds of every transmission interval.

In the original DHT node sketch, TXinterval = 60 so the sketch tries to send all flagged devices every 60 seconds. However, for 38.5 seconds of that 60 seconds, the node is tied up trying & failing to send data. This matches what I saw on my programmable cooker when RFM69 transmission was not working. The clock would stop displaying at 0 seconds of each minute, but then resume displaying seconds after about 30-35 seconds.  Failing transmissions would use almost all of a smaller TXinterval.

The call to the sendMsg() function (every 60 seconds, TXinterval) tries to send data for ALL devices flagged for periodic transmission (including device 99, the wakeup). Therefore, my first fix above did not succeed in turning off transmissions until ALL devices tried their first transmission, in this case 38.5 seconds.

If RF69 transmissions are not working, my fix above stops radio sends after failing on sending ALL flagged device's data the first time. My fix made a definite improvement, but we can do better. I intended to disable transmission after the sketch failed to send only ONE device's data.

    Next, Improve how a node disables failing transmissions so local functions can still work.

[papa]

How can we improve the fix for allowing a Node's local functions to work
        when RFM69 transmission is NOT working?

If transmission retries are reduced to 3, failed transmission time reduces to 12.5 seconds, but I'm not inclined to do that.

My previous fix was to clear a flag (tx) if a device's send failed 7 times & then only call the sendMsg() function if the tx flag was still true. However the sendMsg() & txRadio() functions try 7 times to send ALL devices' data once.

A different approach is to move the if (tx) { } wrapper to the txRadio() function. This way the sendMsg() & txRadio() functions try 7 times to send ONLY ONE device's data (wakeup) & then disables transmission. Though 8 devices are flagged for sending, the sketch only needs 4 seconds to test & disable transmission from interfering with local node controls.

    Next, Sketch Details for the Improved Fix

[papa]

Improving the fix for allowing a Node's local functions when RFM69 transmission is not working.
        (Applies to a typical computourist type node with local functions.)

Here are the edits to get the improved fix described in the last post so that failing transmission is disabled & local node control is achieved sooner. In bright blue text, I highlight how these edits differ from the earlier fix.

1) Near the start of the sendMsg() function is a line whose variable is never used again. Disable this line so we can use the variable elsewhere:
        bool tx = false; // transmission flag << Disable this with // for comment

2) Earlier in the sketch after "int numtx; ...", add the re-purposed line:
        bool tx = true; // ok to transmit ? flag

3) Find the txRadio() function:
        change from:
void txRadio() // Transmits the 'mes'-struct to the gateway
{
        change to:
void txRadio() // Transmits the 'mes'-struct to the gateway
{
    if (tx) {

4) Near the end of the txRadio() function, find the section:
        #ifdef DEBUG
     if (retx) Serial.println("No connection...")
        #endif ... [through the end of the function]

        Change that section to :
            if (retx) {
    tx = false; // flag to disable transmitting, papa added
        #ifdef DEBUG
    Serial.println("No connection...Disabling transmission") ;
        #endif
            } // end of if (retx)
    } // end of if (tx), only send if transmission is working
;} // end txRadio()

4) Near the end of the main loop() find the previously changed line:
if (tx) sendMsg(); // send any radio messages if transmission is working

            Restore the line to its original form:
        sendMsg(); // send any radio messages

    Next, An Example Node Sketch with the Improved Fix

[papa]

   RFM_DHT_node_22_noTX_pub_04.ino (16.27 KB)   << Download this example sketch with the improved fix for node local functions to continue working even when RFM69 transmissions do NOT work.

As with all computourist type sketches, you will need to customize these sketch options to your situation:  encryption key, RFM69 radio frequency, & possibly to the high power version of the RFM69 radio.

Again in order to use this thread's principles, one needs to have a working RFM69 Gateway & a working RFM69 node that has local features that are controlled by the node. This other thread guides you through building, programming , & customizing an RFM69 Gateway & an RFM69 DHT Node.

This sketch only needs 4 seconds to test & disable transmission from interfering with local node controls.

This sketch includes the changes from the last post & some more debugging output to help see what is happening with transmissions. I believe that how I tweaked the DHT Node more should work on other computourist-styled nodes since they are based on the DHT Node.

    Next:  One more improvement:
Programming the Node to disable & re-enable RFM69 transmissions as needed.

[papa]

Changing the sketch so a node starts & stops RFM69 transmissions, depending on availability of transmissions. If the node has local functions, they should work most of the time & not be prevented by trying & failing at RFM69 transmissions.

When RFM69 transmissions, are not working, with the above improvements, the node's local functions are disabled about 4 seconds. This time could be reduced a little by reducing the 6 in this line of the txRadio function: while (retx && i<6) Reducing the 6 may also reduce the likelihood of the node reconnecting to transmission when a signal is available.

In this post, I said, "More code could probably let the node restart transmission without rebooting: Something like 1) timestamp when transmissions were disabled. 2) If a defined interval passes the timestamp && tx = false, set tx = true so the node would try transmissions again. One might lose local control of node features while the node tried to resume transmissions."

Instead of a timestamp, I took advantage of the built-in periodic transmission in a computourist type node sketch. When TXinterval = 60, the node tries to send ALL devices data every 60 seconds, but the above fixes sends only one device's data (wakeup) if transmission is not working. I decided the periodic transmission section is a good time for tx = true so the node tries again to transmit device data. If transmission fails, it will only try to send the one device. However, if transmission is now working, transmission will resume as normal.

With this addition, according to whether transmission is working, the node should shift back and forth between 1) transmission with local functions and 2) only local functions.

Note: Sometimes when RFM69 transmissions are stopped & restarted, the Serial Monitor will say "invalid message structure."  Ignore that temporary error.

    Next, Programming a Node to start & stop RFM69 transmissions,
        depending on availability of transmissions.

[papa]

    Programming a Node to start & stop RFM69 transmissions,
        depending on availability of transmissions.

In the main loop() of a computourist type node, find the section with the commented title PERIODIC TRANSMISSION (all capitals).
    From there, scroll down past the last line with the format sendXX = true (or false)
            ^^ After this line, add this new line:
                        tx = true; // ok to try transmitting

    The sketch changes in this post are also needed for this to work.

Note: I believe a TXinterval = 60 is a good setting. If transmission is available, this is not too long a wait. Also this setting should allow local node functions to work for 56 seconds out of 60 seconds.

   RFM_DHT_node_22_noTX_pub_06.ino (16.32 KB)
       ^^  Forum members (no charge, just register) may download this example sketch.

This example sketch includes the changes in this current post & some more debugging output to help see what is happening with transmissions. I believe that how I tweaked the DHT Node more should work on other computourist-styled nodes since they are based on the DHT Node.

Caution: If OpenHAB sends the node key information (like turning water on & off), it may not be wise to let the local node functions work when transmission is not working.  This concern is addressed below.

With the Cooking Controller sketch in this thread, local node functions (cooking, LCD, etc) now work with RFM69 radio transmissions success or failure.  When the controller attempts to transmit once a minute, it disables or enables radio transmission depending on the success of the transmission attempt.

    Next, Adding this code to the Node Choices sketch.

[brump]

Hi Papa,

Good implementation.
The options to solution the openHAB control a node:
1- The possibility to create a rule to send periodic “check status” for a node, and create a routine to response in a node. If the status are not OK, all controls to respective node are disabled.

2- If we applies a command to node, the node execute this command and return a status to openHAB. In rules, for all controls read a status, if status not received, the act was aborted. And return error in log or text item.

What do you think?

[papa]

Thanks, brump.  Working on this was an interesting & I believe useful exercise.

Yes, your "options to solution the openHAB control of a node" would probably work for some situations. We can probably create other approaches depending on the situation. I primarily wanted to caution users to consider the possible risks if the node was able to turn on & off its trying to connect to RFM69 transmissions.

In a parallel situation, OpenHAB controls when & how much my irrigation nodes water my garden beds.  So OpenHAB may successfully turn on a garden bed's water, but if the bed's node loses RFM69 transmission (especially for an extended period), OpenHAB could not get reliable sensor data or turn off the water at the right time.  Therefore, when my irrigation nodes lose RFM69 transmission, I have the nodes turn off the water to their garden beds.  I prefer to risk too little water to a bed for a while rather than risk flooding the bed with too much water.  I believe we must be aware of possible risks, choose our priorities, test, & program accordingly.

[brump]

Thinking about this question:

We can create the node basic configuration for control of the outputs in case of radio communications lost, with any options.

Case 1: if tx off, all loads are off. papa: Similar to the case papa cited above about controlling water to a garden.
Case 2: if tx off, all loads are on. papa: Used for a remote local power outlet where we want power available so we can toggle an appliance on & off with the appliance's switch.
               ESP8266 EcoPlugs can be programmed to turn on their relay/outlet when the EcoPlug connects to power.

Case 3: (Updated from a later post) The node controls its outputs locally, via keys, buttons, potentiometers, etc., and is also controlled remotely via OpenHAB. In this case, if the communication is interrupted with the gateway, the controls of the outputs are not modified, keeping its last state and waiting for a local control to change its state.  papa: Just like my code so local functions can work with or without RFM69 transmissions working.

Case 4: if tx off, all states stay in their states.  papa: if tx off, all states stay as they were before transmission failed.

It’s possible to create a map on eeprom for this item, and use a remote command from openHAB to set this mode.

Ps: I am struggling to write without the assistance of translators, so if there are grammatical errors, please do not despise me. (<< Updated from a later post)

[papa]

brump: "Thinking about ... creating the node basic configuration for control of outputs in case of radio communications lost."

papa: brump, your thoughts offer a practical, cautious approach to what happens with node output states when radio transmission fails. Especially in bright blue text above, I added some notes & tweaks to your cases. Please check my additions to see if they are correct. (PS we might apply different cases to different node states, depending on what is best for each node state.)

brump: "I am struggling to write without the assistance of translators, so if there are grammatical errors, please [do not?] despise me."

papa: Your struggle is quite understandable & forgivable.  Anytime you are away from translators, in a thread or in a personal message, you could write in your native language & I could put your words through Google Translate.

[brump]

Mar 20, 2018 17:40:06 GMT papa said:

brump: "Thinking about ... creating the node basic configuration for control of outputs in case of radio communications lost."

papa: brump, your thoughts offer a practical, cautious approach to what happens with node output states when radio transmission fails. Especially in bright blue text above, I added some notes & tweaks to your cases. Please check my additions to see if they are correct. (PS we might apply different cases to different node states, depending on what is best for each node state.)

brump: "I am struggling to write without the assistance of translators, so if there are grammatical errors, please [do not?] despise me."

papa: Your struggle is quite understandable & forgivable. Please say more about your Case 3. Anytime you are away from translators, in a thread or in a personal message, you could write in your native language & I could put your words through Google Translate.

Case 3: The node controls its outputs locally, via keys, buttons, potentiometers, etc., and is also controlled remotely via openHAB. In this case, if the communication is interrupted with the gateway, the controls of the outputs are not modified, keeping its last state and waiting for a local control to change its state. papa: Just like my code has been so local functions can work with or without RFM69 transmissions working.

Yes, please do not... despise me   Thanks for helping me !

[papa]

OK, brump, I believe I understand the cases you described for possible relationships between local functions & RFM69 transmissions (working or failing). Now to think about whether to code for those cases & if yes, how ...

Case 1 sounds like my concern about water continuing to run to my garden bed if OpenHAB turned it on, but RFM69 transmissions were off when OpenHAB commanded the water off. This case could be addressed as I did my irrigation automation: if RFM69 transmission is failing, set the relay actuator to OFF.

Case 2 sounds like the opposite of Case 1. To address: if RFM69 transmission is failing, set the relay actuator to ON.

I believe Cases 1 & 2 can be addressed by small code additions, esp in the txRadio() function & in the initial configuration variables.

I believe Cases 3 & 4 are the same.  They can be partly addressed by my code to disable RFM69 transmissions so local node functions can still work. Another initial configuration variable can flag whether output states remain steady when RFM69 transmission is disabled or be handled like Cases 1 or 2. A bonus of my code is that it re-establishes RFM69 transmissions if that becomes possible.

Assuming the code above for disabling & re-enabling RFM69 transmissions, the next post gives code additions to address these cases.  Another bonus is that in a configuration line, we can set a default, startup value for the actuator.

I have added the code additions to the node choices sketch & used it to create a DHT Node which successfully controls a solid state relay according to the needs of the above cases.

In another thread, I will post the latest node choices sketch when I have the time & energy to document it.

[papa]

    Addressing Concerns & Opportunities When RFM69 Transmissions Are Disabled

When RFM69 transmissions are disabled (so local node functions can still work), it is important to code what value the actuator (LED or relay) will hold afterwards: 1) OFF to avoid shocks, water overflows, etc. 2) ON so one could operate an attached appliance with its own switch. 3) Remain as it was before transmission was disabled.

        Added to the node sketch's configuration lines:

#define AutoRadio // Disables & re-enables RFM69 radio sends, acc to transmission success. Allows node local functions to always work.

bool ACT1default = 0 ; // Default state of actuator (LED or Relay?) at start up & PERHAPS when RFM69 transmission is disabled. 0 = OFF, 1 = ON

bool ACT1Steady = 0 ; // When RFM69 transmission is disabled, 1 means ACT1State retains its current value, 0 means ACT1State = ACT1default
----------------------------------
    In the setup() function, now:
#ifdef ACTOR // Papa added
    pinMode(ACT1, OUTPUT); // set actuator 1
    ACT1State = ACT1default; // Default value (0 or 1) is set above in configuration lines.
    digitalWrite(ACT1, ACT1State); // set status of ACT1 output.
#endif // ifdef ACTOR
--------------------------------
        In the txRadio() function:
#ifdef AutoRadio
    tx = false; // flag to disable transmitting, papa added
    Serial.println("Disabling transmission") ;
            // ?? leave ACT1State as is OR set to ACT1default ??
        if ( ! ACT1Steady) {
    ACT1State = ACT1default; // Default value (0 or 1) is set above in configuration lines.
    digitalWrite(ACT1, ACT1State); // set status of ACT1 output.
        } // end of if ( ! ACT1Steady)
#endif // end of ifdef AutoRadio

} // end of if (retx)

[papa]

        Updated Multi-Choice Sketch with New Code for Added Sensors & RFM69 Transmissions

I posted the latest multi-choice node sketch ( computourist_node_v2.2MhB_choices_pub.ino ) in this post of the thread, Building an RFM69 Home Automation Network (Beginners ??).

This sketch version includes the above code for setting a default value for the actuator (LED, Relay), disabling failing RFM69 radio transmission & re-enabling them (& how to handle actuator values when transmissions resume).

This sketch version also includes code for several sensors that forum members have added recently.

[brump]

Mar 21, 2018 19:22:55 GMT papa said:

    Addressing Concerns & Opportunities When RFM69 Transmissions Are Disabled

When RFM69 transmissions are disabled (so local node functions can still work), it is important to code what value the actuator (LED or relay) will hold afterwards: 1) OFF to avoid shocks, water overflows, etc. 2) ON so one could operate an attached appliance with its own switch. 3) Remain as it was before transmission was disabled.

        Added to the node sketch's configuration lines:

#define AutoRadio // Disables & re-enables RFM69 radio sends, acc to transmission success. Allows node local functions to always work.

bool ACT1default = 0 ; // Default state of actuator (LED or Relay?) at start up & PERHAPS when RFM69 transmission is disabled. 0 = OFF, 1 = ON

bool ACT1Steady = 0 ; // When RFM69 transmission is disabled, 1 means ACT1State retains its current value, 0 means ACT1State = ACT1default
----------------------------------
    In the setup() function, now:
#ifdef ACTOR // Papa added
    pinMode(ACT1, OUTPUT); // set actuator 1
    ACT1State = ACT1default; // Default value (0 or 1) is set above in configuration lines.
    digitalWrite(ACT1, ACT1State); // set status of ACT1 output.
#endif // ifdef ACTOR
--------------------------------
        In the txRadio() function:
#ifdef AutoRadio
    tx = false; // flag to disable transmitting, papa added
    Serial.println("Disabling transmission") ;
            // ?? leave ACT1State as is OR set to ACT1default ??
        if ( ! ACT1Steady) {
    ACT1State = ACT1default; // Default value (0 or 1) is set above in configuration lines.
    digitalWrite(ACT1, ACT1State); // set status of ACT1 output.
        } // end of if ( ! ACT1Steady)
#endif // end of ifdef AutoRadio

} // end of if (retx)

The solution to the cases was great.

I had some problems these days with ROV here on the vessel and had to leave.