WiFi_RFM69_MQTT Network

[papa]

        Introduction

Recently I was intrigued by the topic of the thread that Joshua started recently, Project to migrate RFM69 Gateway to an ESP32.

Joshua has been working with OpenMQTTGateway, He said he was aiming for a more secure connect to remote nodes via Wi-Fi.

I'm interested in adding a WiFi version of our DIY Home Automation RFM69 Gateway. I believe I have this WiFi Gateway working. Perhaps our forum members can benefit from it. Perhaps Joshua & others can build on what I learn for their versions.

         Next, Advantages of an ESP32 WiFi Gateway

[papa]

         Advantages of an ESP32 WiFi Gateway:

1) Since it does not depend on connecting to Ethernet cable, It can be placed in any location where it can get power & connect adequately to a WiFi router.

2) Advantage 1 might get the gateway closer to Arduino nodes that struggle to connect with the Ethernet Gateways.

3) If workable Arduino Ethernet Shields are difficult to obtain, we'll have another gateway option.

4) The ESP32 Gateway can be used in place of an existing Ethernet Gateway OR as a new, added Gateway. (The forum Gateway code is ready for 3 Gateways/3 Networks & one more Gateway/Network could be easily added in the code).

5) Since WiFi network connection is built in to the ESP32, we don't need to mount a separate network shield on the Arduino.  Less parts are involved in this Gateway.

This following posts give what I see was needed to use an ESP32 WiFi gateway to make a RFM69 radio network with Arduino nodes & interact with openHAB over WiFi/MQTT.

        Next, What Is Needed, #1 Hardware

[papa]

        What Is Needed for an ESP32_RFM69 Network, #1 Hardware

Arduino DIY Home Automation RFM69 Nodes. I found that our nodes' existing hardware & code work as is. For how to build the nodes, see this thread (just ignore instructions about the Ethernet Gateway, except about preparing the RFM69 radios). So you can use the nodes you already have. Or if starting at the beginning, you can make & use the Bare Bones node to test this approach.

An ESP32 board that breaks out all pins, or at least power, gnd, VSPI pins, an available pin for RFM69 chip select connection, & ways to connect the ESP32 pins. I bought a Sparkfun ESP32 Thing & 2 sets of female headers that were each 20 sockets long. The ESP32 Thing has all pins broken out & has an USB connector for programming & Arduino IDE serial output. It uses the standard ESP32 pin arrangements including for the V-SPI pins: MOSI (pin 23), MISO (19), SCK (18), & CS (5).  See the first 3 columns in this post's table.  Similarly qualified boards should also work.

        << Members, click pic for larger view
In the above pic ^^ I marked the pins I used with red bracket or dot. Pins marked by the red bracket are already selected for SPI communication use by the ESP32 add-on (see next post). Those bracketed pins & the GND pin I selected line up nicely with how I wire the RFM69 radios.

Micro USB cable to connect the ESP32 board to the Arduino IDE computer for programming & IDE Serial Monitor output.

The same frequency RFM69 radios for the ESP32 Gateway & the Arduino node. On the radios, solder connecting wires for the radios' pins: 3.3v, GND, MISO, MOSI, SCK, CS, DIO0, & antenna. See starting here for help with that. Note for the ESP32, the RFM69 radio's connecting wires (except antenna) may need to be a little longer than I instructed for the Arduino.
 
        Next, Add-on to Prepare Arduino IDE for ESP32

[papa]

        What Is Needed for an ESP32_RFM69 Network
            Prepare Arduino IDE for ESP32

In preparation for programming an ESP32 board (including a camera board), install the needed ESP32 add-on
               In Arduino IDE, go to File\Preferences

In the “Additional Board Manager URLs” near the bottom of Preferences, enter the URL below & click the OK button.  If other URLs are in the box, end the list with a comma & then add the URL. OR click the nearby "page & shadow" icon & enter the URL on a separate line.

        dl.espressif.com/dl/package_esp32_index.json

Go to Tools\Board\Boards Manager. In the box near the Boards Manager's top, search for ESP32 & press the install button for “ESP32 by Espressif Systems“   When the install finishes, you may close the Boards Manager. Installing that add-on gives us a Sparkfun ESP32 Thing option at IDE Tools/Board.
 
        Next, What Is Needed, Codewise

[papa]

        For an ESP32_RFM69 Network,
         What's Needed Code-wise?

We need code that will allow the ESP32 Gateway & the Arduino compatible nodes to use their RFM69 radios to communicate with each other. From my research & experiments, these issues had to be dealt with:

1) Apparently using the standard RFM69 library code on an ESP32 results in a conflict between interrupts for the RFM69 radio & SPI.

2) ESP32s & Arduinos store data in memory differently. For example, on an Arduino an int variable takes 16 bits. On an ESP32, an int variable takes 32 bits. ESP32 code must match up with the data struct sent by the Arduino node.

3) Also in the ESP32's data struct definition, it's important the type of variables we declare. Otherwise, the ESP32 received data struct will not match the data struct an Arduino node sends.

Two other needs are 4) connecting the ESP32 to WiFi & 5) then via WiFi, connecting the ESP32 Gateway to the MQTT messaging broker.

        Next, Addressing the Code Needs

[papa]

        Addressing the Code Needs

Again I've learned that the code on our RFM69 nodes works just fine (as long as the code is correct on the ESP32 Gateway. That's fortunate for those of us that already have many programmed nodes.

            How to address the needs in the last post:

1) rrobinet adapted the RFM69 library code to avoid interrupt conflicts.  Our thanks to rrobinet.

2 & 3) Based on what some Germans posted about data structs, mharizanov offered a sketch that points the way to matching up the data structs between Arduinos & ESP32s. Our thanks to them.

4 & 5) Our forum's existing ESP32 sketch provides adaptable code to connect WiFi & MQTT broker.

6) Using the above solutions 1-5 & some conditionals, our existing (Ethernet) RFM69 Gateway code can be adapted to work also for an ESP32 WiFi Gateway.

        Next, Removing RFM69/SPI Interrupt Conflicts

[papa]

                Working Out Solutions
        Removing RFM69/SPI Interrupt Conflicts

        From rrobinet's header in RFM69X.h:

// From [e-mail] 4/10/2016
//
// This is a modified RFM69 library used for ESP-32 processor.
// It is motivated by the fact the ESP-32 SPI transactions occur as interrupt mode while the RFM69 library starts SPI transactions in interrupt routine simultaneously.
// The main change is the move from the SPI transactions from the interruptHandler function, to a new one; the interruptHandling, which is tested during the receiveDone.
// This version works in a mixed environment (RFM69 and RFM69X libraries), however it is incompatible with virtualized versions such as the RFM_ATC one
//
// Notes :
// * The virtualized RFM69_SessionKey for ESP-32 is available (see RFM69X_SessionKey)
// * It also implement the SPI_HAS_TRANSACTION syntax from the recent SPI Library
// * A test of the presence or good functioning of the RFM transceiver is also added.
// * Because SPI transaction is now moved out of the Interrupt routine, this version is also compatible with Ethernet Shield (W5100) without further modifications
// * This version is tested on ARDUINO UNO, ARDUINO MEGA 2560, Moteino R6, MoteinoMEGA, ESP-32 (WeMoS LOLIN 32) and ESP8266 (WeMoS D1 mini)
// * For the time being, running on ESP-32 seems to be a little bit slower that on other processors.

//It is based on the lowpowerlab library version of July 2014

        Next, Matching Data Structs over RFM69 Radios

[papa]

        Matching Data Structs in RFM69 Communications

As I said, one change that achieves successful ESP32-Arduino RFM69 communication is using rrobinet's RFM69X library adapted from the lowpowerlab.com RFM69 library.

Another change is the the data struct definition that mharizanov used.

The struct definition for the Arduino Gateway & the Arduino nodes has been:

typedef struct { // Radio packet structure max 66 bytes
   int nodeID;      // node identifier
   int devID;      // device identifier
   int cmd;      // read or write
   long intVal;     // integer payload
   float fltVal;      // floating payload
   char payLoad[32]; // string / char array payload
} Message; // papa: creates uninitialized struct called Message.

The above definition works fine for the Arduino Gateway & will still work for the Arduino nodes. However, if an ESP32 Gateway defines this way, RFM69 communication is mismatched.

With this struct definition on the ESP32 Gateway, RFM69 communication is matched again:

            // DIY Home Automation struct adapted into TFT struct
#pragma pack(1) //https://www.arduinoforum.de/arduino-Thread-struct-wird-auf-einem-ESP8266-anders-behandelt-wie-auf-einem-Arduino
typedef struct {      // Radio packet structure max 66 bytes
    uint16_t nodeID;     // node identifier
    uint16_t devID;     // device identifier
    uint16_t cmd;          // read or write
    uint32_t intVal;     // integer payload
    float fltVal;         // floating payload
    char payLoad[32];      // string / char array payload
} Payload;
Payload theData;
#pragma pack(0)

        Next, Wiring an ESP32 Data Struct Receiver

[papa]

        Wiring an ESP32 Data Struct Receiver

See this post for the parts used in this device.

Use the following schematic, to connect an RFM69 radio to an ESP32 Data Struct Receiver (a Gateway that only receives node data) that we'll program below.

       << Members, click the pic for larger view.

In this post's image, see the ESP32 pins used (marked by red) in their actual locations.

        Next, An Example of a Wired Struct Receiver

[papa]

            An Example of a Wired Struct Receiver

The following pictures an actual ESP32 Struct Receiver. (Actually it shows a two-way ESP32 RFM69 Gateway which is wired the same as the Struct Receiver, except for the yellow & green LEDs.)

       << Members, click on the pic for a larger view.

With a Bare Bones node (see the first paragraph of this post) or other Arduino node & this ESP32 Struct Receiver (programmed as below), you can minimally test this RFM69 network.

        Next, Adapted Mharizanov Struct Receiver Sketch for RFM69 Communication

[papa]

        Adapted Mharizanov Struct Receiver Sketch
          for RFM69 Communication (One Way)

Here, mharizanov offered a sketch to program a ESP32 RFM69 Gateway (not with MQTT) with a TFT display. I've trimmed & adapted the code to receive data from one of our RFM69 nodes.

My adapted sketch needs a ESP32-RFM69 device built as instructed above. The sketch will program such an ESP32 board to receive data sent by a DIY Home Automation RFM69 radio node built & programmed on an Arduino. You can use nodes you've already created using this forum's instructions. If you have no RFM69 nodes yet, you can use my instructions to create a Bare Bones node.

     struct_receive_esp32_5_pub.ino (6.76 KB)  << Forum members, download this file to your Arduino sketch folder & open it in your Arduino IDE, then close the sketch.
                Updated ^^ Oct. 20,  2021

Don't upload this sketch to the ESP32 yet. Use the next post to download RFM69X library files & the following post to customize the sketch to your situation.

        Next, Needed RFM69X Library Files

[papa]

    RFM69X Library Files Needed
    for the Struct Receive Sketch

Here are 3 rrobinet-adapted RFM69 library files: RFM69X.cpp, RFM69X.h, & RFM69Xregisters.h.  Downloading the files together as a .zip may not work so I provide those 3 files in this post

       RFM69X.cpp (33.98 KB)         RFM69X.h (8.58 KB)         RFM69Xregisters.h (40.53 KB)

Download & save those 3 files ^^ in the same subfolder that Arduino IDE creates for the struct_receive sketch. Close the struct_receive sketch & reopen it. Now the RFM69X library files should appear in tabs with the struct_receive sketch.

    Again, don't upload the struct receive sketch yet until you check the sketch's customizing via the next post.

            Next, Customizing the Struct Receive Sketch

[papa]

        Customizing the Struct Receive Sketch

// #define DataDisplay // Disable to display raw data received & errors of data length
        ^^ If you get no Serial monitor results, delete ^^ this line's initial "//" to display raw data & errors. (Upload again)

#define RF69_IRQ 13     // Any available ESP pin connected to RFM69 DIO0 irq pin
        Above, the schematic wired RFM69 DIO0 to ESP32's pin 13. Change 13 if you wire differently.

#define RF69_SPI_CS 5    // ESP32 board's VSPI chip select pin (cs, ss, nss)
      Above, the schematic wired RFM69 NSS pin to ESP32's pin 5. This matches the standard ESP32 VSPI CS pin. Change 5 if your board is different.
 
#define IS_RFM69HW_HCW false    // change to true only for RFM69HW/HCW! Make false if you have RFM69W/CW!   Do NOT comment this line.
                                            ^^ Updated Oct. 14, 2021

#define NODEID 1                    // Unique NodeID of this ESP32 gateway
#define NETWORKID 100  // NetworkID for this gateway & nodes in its network
     If you have an existing Node 1/Network 100 with Ethernet Gateway & Arduino nodes, leave these 2 lines as is, but power down the Ethernet Gateway (which would conflict with the ESP32)
If you created a Bare Bones Arduino node with a NetworkID of 100, you can leave these 2 lines as is.

#define FREQUENCY RF69_915MHZ // Match this with Freq of your RFM69s (others: RF69_433MHZ, RF69_868MHZ)

#define KEY "xxxxxxxxxxxxxxxx" // #1 Network use same 16 characters on all devices in same network
       ^^ Use the same 16 characted key on the ESP32 as on the node(s) you have in the same network.

Recheck you customizing above & then you can upload to the ESP32 connected to your Arduino IDE computer.

        Next, What to Expect

[papa]

        What to Expect on the Arduino IDE Serial Monitor
            When RFM69 Communication Is Working
                    on the ESP32 Struct Receiver

Note: The previous version of this post showed the wrong Serial Monitor Output.

On the ESP32 Struct Receiver, I'm receiving data like the following from an existing Arduino RFM69 node. (I powered down my existing Ethernet Gateway that shared the same NodeID as the Struct Receiver.)

[5] [RX_RSSI:-29 dB] nodeID=5 devID=2 cmd=0 intVal=4294967247 fltVal=0.0000000000 payLoad=0 - ACK sent.
            From Node: 5; RSSI: -29
[5] [RX_RSSI:-33 dB] nodeID=5 devID=4 cmd=0 intVal=0 fltVal=3.2639999390 payLoad=0 - ACK sent. Pinging node 5 - ACK...nothing
            From Node: 0; RSSI: -29
[5] [RX_RSSI:-29 dB] nodeID=5 devID=9 cmd=0 intVal=1 fltVal=3.2639999390 payLoad=0 - ACK sent.
            From Node: 5; RSSI: -29
[5] [RX_RSSI:-29 dB] nodeID=5 devID=16 cmd=0 intVal=0 fltVal=3.2639999390 payLoad=0 - ACK sent.
            From Node: 5; RSSI: -29
[5] [RX_RSSI:-29 dB] nodeID=5 devID=48 cmd=0 intVal=0 fltVal=77.0000000000 payLoad=0 - ACK sent. Pinging node 5 - ACK...nothing
            From Node: 0; RSSI: -33
[5] [RX_RSSI:-34 dB] nodeID=5 devID=49 cmd=0 intVal=0 fltVal=33.0000000000 payLoad=0 - ACK sent.
            From Node: 5; RSSI: -34

            ^^ Expect similar results when RFM69 communication is working. (These results match how the Arduino node structured the packet.)

Our RFM69 nodes are usually programmed to send data every 60 seconds or less. So in my experience, if the Struct Receiver is working, we should see its Serial Monitor output within a minute or so after we open the Serial Monitor.

            Next, When RFM69 Communication is NOT Working on the Stuct Receiver

[papa]

                What to Expect on the Serial Monitor
        When RFM69 Communication is NOT Working

If #define DataDisplay is still commented out (// starts the line), you may see nothing on the Serial Monitor.
        If so, you can then delete the // that starts the line & upload.  Then you start seeing raw data & error complaints, such as

"Invalid payload received, not matching Payload struct!"
" Expected: " 

"Received: "  [Raw data like the following:]

[81] [RX_RSSI:-27 dB] size: 52; byte-data = 81 00 49 00 00 00 00 00 00 00 00 00 00 66 77 104 70 57 114 49 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
nodeID=3211345 devID=0 cmd=0 - ACK sent.
        From Node: 81; RSSI: -27

From the following, see that the node data was present in this case, it just did not match how the ESP32 expected to receive it.

                 node    dev   cmd   intVal----- fltVal--------- payLoad-------------------
byte-data = 81 00 49 00 00 00 00 00 00 00 00 00 00 66 77 104 70 57

            When your ESP32 Struct Receiver successfully receives data from an RFM69 node, then you can move on ...

       Next, A Two-Way RFM69 Gateway with WiFI & MQTT, Schematic

[papa]

            A Two-Way RFM69 Gateway with WiFI & MQTT, Schematic

The following wiring diagram for the two way RFM69 Gateway is the same as for the Struct Receiver above, EXCEPT we add two LEDs. Especially for when the Gateway will be disconnected from the Serial Monitor, one steady-burning LED will signal that WiFi/MQTT is connected. The flashing LED signals when radio communication is happening. For each LED, the originally shorter pin goes into an ESP32 ground. Each originally longer pin goes into the ESP32 pin designated in the Gateway sketch.

You may change the ESP32 pins for the LEDs, but you must also change the pin numbers in the Gateway sketch.

       << Members, click on pic for larger view.

In this post above, see the WiFi Gateway I built on a Sparkfun ESP32 Thing.

                Next, Prepare to Program the ESP32-RFM69 Gateway

[papa]

        Prepare to Program the ESP32-RFM69 Gateway

Note:  The following earlier version of the firmware had remaining Ethernet code which made things too complicated & the Ethernet code no longer fully works.

     RFM_MQTT_GW_25.6.1_pub.ino (30.14 KB)  << Do NOT download for programming your gateway. Only for archive purposes.
               ^^ updated Oct. 14, 2021

    Next: Prepare to Program the ESP32-RFM69 Gateway, Latest Version

[papa]

        Prepare to Program the ESP32-RFM69 Gateway
                        Latest Version

     RF69_MQTT_GW_25.6.3_esp_pub.ino (28.76 KB)  << As a forum member, download this & save it in your Arduino IDE sketch folder.
               ^^ updated Oct. 20, 2021

Note: This recommended sketch only contains code for a gateway built with an ESP32 board (WiFi) & an RFM69 radio. The earlier version had remaining Ethernet code which made things too complicated & did not fully work.

Open the Arduino IDE (program for loading, editing, & uploading Arduino sketch programs). From the Arduino IDE, open the sketch (program) file downloaded above. Accept the sketch's being put in a subfolder.

Before uploading (sending) the Gateway sketch to the Gateway, you must edit it to customize to your situation. Your IDE computer must also have the additional code library files that the Gateway sketch needs. The following posts offer guidance.

Note: In editing a sketch, sometimes we are told to "comment out a code line" which means put two forward slashes ("//") at the start of the code line to make the rest of a line a "comment" (not programming code) & disables the code. To UNcomment a line means to remove the two forward slashes ("//") at the start of the line to make the rest of the line into working code except "//" later in the line still indicates comment there. For example in the Gateway sketch, see "// #define DEBUGRADIO" which is a comment, not active code. The line "#define DEBUG // uncomment for MQTT debugging" starts with active code & ends with a comment (after //).

    Ahead, Get Your Computer's IP Address

[papa]

         Prepare to Program the ESP32 Gateway, Get Your Computer's IP Address

Your Gateway program needs the IP address of your computer that will host the home automation services (OpenHAB/MQTT). Also if your computer has a "dynamic" (rather than static or fixed) IP address via your router's DHCP, the address can change & disconnect it from the Gateway & other things.  So do the following ...

Get the present IP address of your computer that will host the home automation services (OpenHAB/MQTT): For example in Windows, click the Start button. In the search box type cmd [enter]. In that black & gray window type ipconfig [enter]. Look for Ethernet adapter, IPv4 Address ….. in my case it is 192.168.10.101  For Linux, in its Terminal, enter "ip a"  About two-thirds of the way down look for "inet" followed by a dotted number like my 192.168.10.101    Make note of all the ip address numbers to use below.

I recommend that you log in to your router & make sure your computer has a static or fixed IP address. You can use the same IP address you found just above for the static IP address.

    Ahead, Customizing the ESP32 Gateway Sketch Program

[papa]

    Prepare to Program the Gateway, Customize the Gateway Sketch Program
                Updated Oct. 20, 2021 for Latest Sketch

Here's where to edit & customize the Gateway sketch for your setup:

Find #define NetworkOne & two lines after it. UNcomment (no // at its start) only ONE of the three lines. In most cases, UNcomment #define NetworkOne. If you UNcomment one of the other 3 lines, be sure to do the same when you later use the choose_nodes sketch to program a node for that gateway.

    Find 3 lines with "#define FREQUENCY" (radio hardware version). 
UNcomment (no "//" at the start) the line that matches frequency of your RFM69 transceivers & comment the rest (start the line with "//").

    Find the line with #define IS_RFM69HW_HCW.  ONLY if your RFM69 transceivers have "H" in their part number, change the definition from false to true.

        Comment out #define DEBUGRADIO (change it to // #define DEBUGRADIO...)

    Make sure #define DEBUG is uncommented (NO "//" at the line's start). Later you may reverse the comment / uncomment of these two lines.

    Find #define mqtt_server "192.168.xx,xxx"
change the IP address to match that of your "computer" where MQTT service is running.

===================================================

    Starting with "#ifdef NetworkOne ...," find 3 sections of settings.

              In each Network section, edit these first two settings only:

    byte ip[] = { 192, 168, ##, ?? } "##" will be the same as in mqtt_server & "??" will be a UNIQUE & higher, but available network number than that in mqtt_server (following our made up example, ## could be 1 & ?? could be 150. For NetworkTwo, ?? could be 151, & for NetworkThree, ?? 152.).

    To put after ENCRYPTKEY, choose a UNIQUE 16 character encryption key for EACH of your networks, make note of it. Later help connect end nodes to each gateway by giving nodes the encryption key for its gateway. If you already chose an encryption key, use it in the section for GatewayOne.

Keep copies of the 3 ENCRYPTKEYs so you can use them for the 3 Gateway sections in the choose_nodes sketch.  Initial customizing of the choose_nodes sketch matches the Gateway customizing above.

    Next, WiFi Credentials for the Gateway Sketch

[papa]

                WiFi Credentials for the Gateway Sketch

        Find four lines that start with ...
#define wifi_ssid_A "aaa" // your wifi station name !!

For wifi_ssid_A  edit in your SSID's name.

For wifi_password_A edit in your WiFi password

If you have another WiFi station, edit in that info to wifi_ssid_B & wifi_password_B. Otherwise, give B the exact same WiFi info as A.

        Next, Check Sketch's ESP32 Pin Assignments

[papa]

        Check Sketch's ESP32 Pin Assignments
                Updated Oct. 20, 2021

In the ESP32 Gateway sketch, find the line that says "#define MQCON 16" & is followed by three lines of other pin assignments.

For MQCON & R_LED, I've chosen two available ESP32 pins that are not far from available GND pins (for ease of mounting LEDs). Keeping my criteria in mind, you may change to other available ESP32 pins.

RF69_SPI_CS needs to be your ESP32 board's pin to wire to the RFM69's NSS slave select pin. If you have an ESP32 board with standard pin assignments (see above), pin 5 should work for you, especially if the previous Struct_Receive sketch worked.

RF69_SPI_CS needs to be your ESP32 board's CS pin for connecting to the RFM69's NSS slave select pin. If your ESP32 board uses standard ESP32 pin assignments, pin 5 should work find, especially if the Struct_Receive sketch above worked. If your board's pinout's does not match the standard ESP32 pinout, then you need to find the pin numbers of your board's SPI pins & wire accordingly (See above) between the ESP32 & the RFM69 radio.

For RF69_IRQ. I chose an available ESP32 pin (13) that is located on the same side as the RFM69's DIO0 pin. You may change it to another available pin (in the wiring & in the sketch).

            Next, Be Sure To Have Code Libraries Needed by the Sketch

[papa]

        Be Sure To Have Code Libraries Needed by the Sketch

If you don't have the libraries to include, you'll get compiling errors. The WiFi.h library should come with the ESP32 add-on that you installed above. Into the same folder as the RF69_MQTT_GW_25.6.3_esp_pub sketch, copy or save the three RFM69X files you got from here.

At IDE/tools/Manage Libraries, search for RFM69 & install the LowPowerLab version.

At IDE/tools/Manage Libraries, search for PubSubClient & install the Nick O'Leary version.

Be sure to save your changes in the sketch.

        Next, Arduino IDE Board Settings & Upload

[papa]

            Arduino IDE Board Settings & Upload

In the Arduino IDE menu, before uploading, check/set something like these settings:

Plug the ESP32 board's cable into the computer's USB port. Avoid having other ESP32 or Arduino or ?? boards connected to the computer.

Tools/Board/ESP32 Arduino: I selected Sparkfun ESP32 Thing. If you also have an ESP32 board with standard ESP32 pinout, you could probably select ESP32 Dev Module. If you have a non-standard ESP32 board, perhaps you downloaded a board core file so then select that in the board list.

Tools/Board/Partition Scheme: Probably select a "large App" option.

Tools/Board/Core Debug Level: I choose "Verbose." On the Serial Monitor screen that gives extra debugging info when the board's reset button is pressed.

Tools/Board/Port: Select the USB port where the ESP32 board is plugged in. For Windows, it will be COM#. For Linux it will be /dev/........

If I did not mention a board setting, you can probably leave it as is.

In the Arduino IDE, have the RFM_MQTT_GW_25.6_pub sketch open & the Serial Monitor closed.

Upload the sketch to the USB-connected ESP32 board.

        Next, ESP32 WiFi/RFM69 Gateway, First Results

[papa]

                ESP32 WiFi/RFM69 Gateway, First Results
                   

You need to have at least one RFM69 Arduino node powered & sending RFM69 data on the same network as the ESP32 Gateway. If you have another Gateway on that network, power it down so it does not conflict.

Open the Arduino IDE Serial Monitor & make sure it is set at 115200 baud.

Updated Oct. 20, 2021: On Serial Monitor output, you should see
     GW RF69 25.6.3 esp  Nw# [# of your chosen network]

Fairly soon, you should also see [###] Mhz... [your chosen RFM69 frequency] signaling that the RFM69 radio has initialized. WiFi may take longer & then MQTT connection is fairly soon after that:
...............
Connecting to [your WiFi SSID]   this repeats until  ...

WiFi connected
IP address: x.x.xx.xxx
connecting   [MQTT messaging that is]
no link
MQTT OK  [connected]

Depending on the Core Debug Level you chose, you may see other
Serial Monitor output especially about the attempts for WiFi connection.

For WiFi connection, it may help to press the board's reset button every so often. Also you may try moving the board around & holding it at different angles. If the ESP32 Gateway disconnects from WiFi, it is programmed to try reconnecting.

Once the above works, you may place & power your ESP32 Gateway anywhere it can reliably connect to your WiFi.

        Next,  LED & Node Data Results

[ragoth]

            LEDs & Node Data Results on the Arduino IDE Serial Monitor

If you added the two LEDs to the ESP32 Gateway, they give some feedback (especially useful when not connected to Serial Monitor output on a computer). When WiFi & MQTT connect, one LED will light & stay steady. When RFM69 radio communication is happening, the other LED will flash.

My ESP32 is programmed to be Node 2 on Network 200. On the ESP32 Gateway, the Serial Monitor shows I'm receiving data like the following from existing Arduino RFM69 nodes on NetworkTwo. (I powered down my existing Node 2 Ethernet Gateway.) I added some explanatory notes after the data

MQTT msg: hom2/rfm_gw/nb/node99/dev02: -63    //   RSSI, radio signal strength
MQTT msg: hom2/rfm_gw/nb/node99/dev04: 3.34   //     node voltage
MQTT msg: hom2/rfm_gw/nb/node99/dev09: 1        //  # of transmit tries to accomplish send
MQTT msg: hom2/rfm_gw/nb/node99/dev43: ON       //   door locked
MQTT msg: hom2/rfm_gw/nb/node99/dev44: OFF            // door unlocked
MQTT msg: hom2/rfm_gw/nb/node99/dev48: nan   // temp not available, sensor disconnected
MQTT msg: hom2/rfm_gw/nb/node99/dev49: nan   //   humidity  "                         "
MQTT msg: hom2/rfm_gw/nb/node21/dev02: -47
MQTT msg: hom2/rfm_gw/nb/node21/dev04: 3.22
MQTT msg: hom2/rfm_gw/nb/node21/dev09: 1
MQTT msg: hom2/rfm_gw/nb/node21/dev66: 11     // code for sump cistern level
MQTT msg: hom2/rfm_gw/nb/node21/dev71: -6     // sump pump on for 6 seconds

            ^^ Expect similar results when RFM69 communication is working.

            Next, Using the ESP32 Gateway

[papa]

Oct 11, 2021 11:42:49 GMT ragoth said:

Papa, you are really really great!! Happy to see your continuous upgrades of this RFM69 projects.

My special thanks to Joshua for his initiation of "Project to migrate RFM69 Gateway to an ESP32".

[papa]

Thanks, ragoth. It's nice to get some response from my efforts.

Are you going to use the ESP32-RFM69 Gateway?  How?

I hope what I documented helps Joshua (& others) with his approach to having an RFM69 Gateway on an ESP32.

How about some of the rest of you giving some feedback to encourage content like this continuing?

        Next, Using the ESP32-RFM69 Gateway

[ragoth]

            Using the ESP32-RFM69 Gateway

This ESP32 Gateway (assigned to a different ID & network number) can work alongside another (Ethernet) RFM69 Gateway or as your sole RFM69 Gateway.

This ESP32 Gateway communicates just the same with RFM69 nodes as does the Ethernet RFM69 Gateway.

Also like the Ethernet Gateway, the ESP32 Gateway will communicate with openHAB via MQTT messaging. Using openHAB, one can gather data (including sensors) from nodes & take action based on that data. This thread can help you build & program an RFM69 network linked to openHAB. That thread assumes an Ethernet Gateway, but you can substitute this ESP32 Gateway.

This forum documents using a number of sensors on the RFM69 nodes. Those threads may still assume openHAB 2, but are adaptable to openHAB 3. (I am gradually updating).

What are your questions & comments on this project?

How do you think you might use the ESP32-RFM69 Gateway?

[papa]

Hi Papa, my plan is to replace Ethernet Gateway with a compact ESP32 based Gateway. I will try your ESP32-RFM69 Gateway setup and get back with my feedback. Thank you.