ESP32 Smart Doorbell (orig)

[papa]

        Introduction

ESP32 is a more robust platform than ESP8266. ESP32 camera boards caught my attention. I decided to explore programming such boards with Arduino IDE & seek openHAB integration. I was attracted to building DIY Smart Doorbells. The ESP-CAM board looks interesting. It's cheaper & smaller than what I bought & has an SD card slot, flash, 16 connector pins that could mount to a solderless breadboard, & much online documentation. Unlike what I bought, it does not have screen, PIR, or USB connection. Some online builds address those lacks.

At Amazon, I liked the features (see next post) & bought this TTGO T-Camera ESP32-WROVER-B OV2640 Camera Module ESP32 WROVER & PSRAM Camera Module 0.96 OLED ESP32-WROVER (Normal Camera). My board is currently NOT available at Amazon. Currently Amazon suggests this which has almost the same name: HiLetgo camera board with fish-eye camera ($24, same pin functions as my board).  For $18.50 Banggood.com has what I believe is the same board with a regular camera, 

        << Forum Members, click on pic for larger view

Banggood.com offers a similar board but also with a MIC for $25-$29, depending on camera & one chip. Its pin functions are different from my board which is addressed below.

There are a variety of these boards using sometimes similar names (LILYGO, TTGO, HiLetgo).  Andreas Spiess' helpful video & "SHOW MORE" comments calls my board "TTGO ESP32 camera White" which seems as specific as possible.  Below I'll use "TTGO ESP32 camera White" or "TTGO White" for boards with similar appearance & pinouts. I'll use "LILYGO White w MIC, for boards who appear similar to TTGO White, but with a MIC AND different pinouts.

What I write below specifically applies to this TTGO White camera board, but is probably adaptable to other ESP32 boards (even those without cameras).  Looks like one can adapt code to the board you get. Sometimes sketches have definition lists based on board pin names.  In those lists, you might match your board's pin assignments or using your board's pinouts (as I did), you might create a new definition list in a sketch (more below).

To upload a sketch, you select an ESP32 board's name under Arduino IDE's Tools/Board (often ESP32 Dev Board will work), but then more importantly, make sure the Tools/Board settings match specifications of your board. Below I model this for the TTGO White Board.

        Next, Specifications of My TTGO ESP32 camera White

[papa]

        Specifications of My TTGO ESP32 Camera White Board

        T-Camera specification
Master Chip: esp32 dual-core         CPU Frequency 240Mhz (WiFi, BT)
    ESP32 Module: Espressif WROVER-B     Crystal 40Mhz
Protocol: Wi-Fi 802.11 b/g/n & Bluetooth 4.2 BLE & BR/EDR
    Flash: 4MBytes        PSRAM: 8MBytes
Display chip: SSD1306 I2C        Display type: OLED        Display resolution: 128x64
    PIR: AS312        Uart Chip: CP2104     I2C

USB connector for operation & programming

    Camera: OV2640        Camera Resolution: 2Megapixel

[Battery ?] Charging chip: IP5306        Battery connector & cable included

One online sketch for the TTGO White Board had programming for a BME280 sensor (temp, humidity, atmospheric pressure). Using that sensor would require editing some library files to avoid conflicts between a struct for the camera & a struct for the sensor. The board I received had no BME280 mentioned in the specs & indeed some SSMD pads on the board were labeled BME280, but the chip was absent. Somewhere I may have seen that the BME sensor's presence on the board did not work accurately so the sensor was omitted.

        Next, Trying the TTGO White Board As Is & Preparing to Flash New Firmware

[papa]

       Trying the TTGO White Board As Is

       & Preparing to Flash New Firmware

DIY Smart Doorbell #2 + ESP32 & ESP8266 Firmware Backup | Home Assistant

     This youtube project was informative & inspiring. Here are time indexes to parts I used:

:35 Trying the ESP32 Camera Board As Is
1:28 DrZzs: "Enroll Face [for face recognition] is blocked out"
    papa: Actually, set much lower resolution & then [Enroll Face] is available

1:54 How to Backup any ESP-based firmware

2:07 Install ESPtool github.com/espressif/esptool

2:13 start with installing Python
(I used the latest version of Python 2.7 which we've used elsewhere on the forum)

Linux comes with Python. To check, in Terminal, enter: python --version
     My result: Python 2.7.17  [the version I want]

            Linux install esptool...
Check, Terminal:  esptool version
Terminal:  sudo apt-get update -y
                sudo apt install python-pip
        sudo pip install setuptools wheel pyaes esptool
            Check, Terminal:  pip -V
pip 9.0.1 from /usr/lib/python2.7/dist-packages (python 2.7)

sudo pip install setuptools wheel pyaes esptool

        esptool.py version            esptool.py v3.1
       
Github notes: github.com/espressif/esptool

sudo esptool.py -b 460800 read_flash 0 0x400000 ttgo_cam_orig.bin   (by default stored at /home/<username>)

esptool.py v3.1
Found 2 serial ports
Serial port /dev/ttyUSB0
Connecting....
Detecting chip type... ESP32
Chip is ESP32-D0WD (revision 1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz
MAC: 84:0d:8e:d2:93:58
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
4194304 (100 %)
4194304 (100 %)
Read 4194304 bytes at 0x0 in 98.4 seconds (341.1 kbit/s)...
Hard resetting via RTS pin...

        Linux: find port (device).  Terminal: ls -l /dev/serial/by-id            My results:

lrwxrwxrwx 1 root root 13 Aug 28 11:20 usb-Arduino__www.arduino.cc__Arduino_Uno_FB0001-if00 -> ../../ttyACM0
lrwxrwxrwx 1 root root 13 Sep 1 15:40 usb-Silicon_Labs_CP2104_USB_to_UART_Bridge_Controller_01DE0085-if00-port0 -> ../../ttyUSB0

sudo esptool.py -p  ttyUSB0 -b 460800 read_flash 0 0x400000 ttgo_cam_orig.bin
                Have only one ESP device connected to computer (esptool will find it)

            specify device port for reading
sudo esptool.py -p /dev/ttyUSB0 -b 460800 read_flash 0 0x400000 ttgo_cam_orig.bin

"Permission Denied" error? Add the user to the dialout group with a command like sudo usermod -a -G dialout <Your Linux USERNAME>

3:26 cmd window: Go to Python install folder

4:04 cmd: pip install esptool   Then cd "C:\Python27\Scripts"
7:20 cmd: to read TTGO White Board's 4MB of firmware & save in a .bin file (corrected Jan. 31, 2020)

        esptool.py -p COM[#] -b 460800 read_flash 0 0x400000 ttgo_cam.bin

                The file will save in the \Python??\Scripts folder

                That saved firmware can be restored to ESP board via ...

        esptool.py --p COM[#] -b 460800 write_flash 0 ttgo_cam.bin

8:44 quick summary of the above procedures

After the above, DrZzs moves to using Home Assistant, which looked promising.

    Next, New Firmware via ESPHome, HASSIO, & Home Assistant ??

[papa]

    New Firmware via ESPHome, HASSIO, & Home Assistant ??

After what I reported in the post above, DrZzs' youtube video moves to using Home Assistant. Actually, that seems like a pretty neat, easy method with a BIG IF ... if one does not use Windows 10 like I mostly do. To work at all on Windows, this method requires installing & configuring Oracle VM VirtualBox.

I took long detours on this. If you want to try this on Windows, I used another DrZzs youtube video & fairly detailed .pdf instructions by "Mark."  Here's another video on this.  I just found this DrZzs video (skip to about 7:30) on installing HASSIO on Windows which demonstrates a powerful one line script using Proxmox instead of Virtual Box to get HASSIO into a VM.  An Internet search for "DrZzs Proxmox" gives instructions on installing Proxmox.

At one point, I had Home Assistant working, but discovered I could not have the required HASSIO without the Oracle install. I never quite got the HASSIO VM working. At least for now, I abandoned this Home Assistant approach. Again, especially if one does not use Windows, the ESPHome, HASSIO, & Home Assistant approach looks promising.

Below I return to & continue the Arduino IDE progress I made before the Home Assistant detour.

        Next, Testing Firmware on TTGO White Board via Arduino IDE

[greginkansas]

Good work so far. I to have been looking for a good program for the ESP32 camera boards. Face or motion with MQTT would be great. All the streamers I have used so far crash after a few hours or day running on my zoneminder server.

[papa]

        Another Long Detour

Above I wrote, "DrZzs' youtube video moves to using Home Assistant. Actually, that seems like a pretty neat, easy method with a BIG IF ... if one does not use Windows 10 like I mostly do. To work at all on Windows, this method requires installing & configuring Oracle VM VirtualBox."

I tried the VirtualBox route & blew up my Windows install (including openHAB). Later I saw that such approach did not work well on laptops like mine. I had to reinstall Windows & will need to reinstall & configure openHAB, Arduino IDE, etc. before I can continue.

In the following, from memory, I'll reconstruct my experiments in programming the TTGO White Camera Board with currently the latest Arduino IDE (1.8.11).

Update, Jan. 31, 2020: Arduino IDE would not let me open sketches by double-clicking on their file name so I downgraded to Arduino IDE 1.8.10.

        Next, Getting the ESP32 Add-On

[papa]

     Arduino IDE & Getting the ESP32 Add-On

I started with something that seemed to use all features of the TTGO White Board, but I had trouble so I decided to try something basic & build on that. CameraWebServer.ino comes with the needed ESP32 Add-on for IDE & becomes available from Arduino IDE's File/Examples menu when one selects an ESP32 board in Arduino IDE's tools/boards menu.

This site was helpful. It instructed installing the needed Arduino IDE & the ESP32 Add-on for Windows & Mac/Linux.

In preparation for programming an ESP32 board (including a camera board), install the needed ESP32 add-on.

        I did so for Windows:   In Arduino IDE, go to File\Preferences

In the “Additional Board Manager URLs” near the bottom of Preferences, enter the following & click the OK button.  If other URLs are in the box, end the list with a comma & then add this URL:

   
https://dl.espressif.com/dl/package_esp32_index.json

Go to Tools\Board\Boards Manager. In the box near Boards Manager's top, search for ESP32 & press the install button for “ESP32 by Espressif Systems“   When the install finishes, you may close the Boards Manager.

        Next, Arduino IDE: Accessing an ESP32 Board

[papa]

        Arduino IDE: Accessing an ESP32 Board &

            the Example Sketch, CameraWebServer.ino

The following assumes first having completed what's in the post just above.

For the following to work, you MUST now connect an ESP32 board (perhaps through an FTDI device) to your Arduino IDE computer's USB. (The TTGO White Board has an USB connector.)

Then go to Arduino IDE's tools/boards menu & scroll down to see & select an ESP32 board. Perhaps your board will be listed, but I just use ESP32 Dev Module which seems generic. You may sometimes need to select a port in the config list that appeared when you selected the ESP32 board.

In File/Examples, scroll down to ESP32/Camera & select CameraWebServer. Arduino IDE should open 4 files in 4 tabs: CameraWebServer.ino, app_httpd.cpp, camera_index.h, & camera_pins.

Caution: Before we upload any ESP32 sketch (even when you've selected a board that resembles a name on your board), we must make sure the sketch pin assignments are correct for our board. The Arduino IDE's configuration list for the board may also need some tweaking. (See more below)

    Next, Preparing to Use CameraWebServer.ino

[papa]

        Preparing to Use New Firmware, CameraWebServer.ino

Hint: In Arduino IDE's File/Preferences menu, have verbose output turned on for compiling & uploading. That gives us steady feedback that things are working & any problems arising.

In Arduino IDE's Tools/Board, tweak some configs for the ESP32 Dev Module. For the TTGO White Board at least:
    Have Flash Frequency at 40MHz (the TTGO White Board's crystal frequency)
    Have Partition Scheme at Huge APP (to help it be large enough for the sketch)
    Have PSRAM at enabled (the TTGO White Board does have PSRAM, image RAM)  (Important for some performance)
        Using your board's specs, change any board configs as needed.  See this above post for TTGO White Board specs.

From installing Arduino IDE & the ESP32 add-on (see above), Arduino IDE has the libraries it needs for CameraWebServer.ino

However, we need to ensure the sketch has the correct pins configuration for our board (in this case the TTGO White).

    Next, Pins Configuration for Our Board

[papa]

        Pins Configuration for Our Board

As I cautioned above, "Before we upload any ESP32 sketch, we must make sure the sketch pin assignments are correct for our board." Sometimes different camera models' pin assignments may be listed within the sketch.  For CameraWebServer.ino, the models' pin assignments are in the header file, camera_pins.h

The TTGO White Board's pins are often given on sites where they are sold (See example in this post.) Mine also came with a chart on its protective box. This pin list did not match anything in camera_pins.h including under model names with "WROVER" or "TTGO" .  In cases like this, we put our camera board's pin assignments in a new camera_pins.h section.  I suggest you insert the following just above the "#else" near the end of camera_pins.h

#elif defined(CAMERA_MODEL_TTGO_WROVER_B) // TTGO White Board
#define PWDN_GPIO_NUM     26
#define RESET_GPIO_NUM    -1  // -1 means "not available"
#define XCLK_GPIO_NUM     32
#define SIOD_GPIO_NUM     13
#define SIOC_GPIO_NUM     12

#define Y9_GPIO_NUM       39
#define Y8_GPIO_NUM       36
#define Y7_GPIO_NUM       23
#define Y6_GPIO_NUM       18
#define Y5_GPIO_NUM       15
#define Y4_GPIO_NUM        4
#define Y3_GPIO_NUM       14
#define Y2_GPIO_NUM        5
#define VSYNC_GPIO_NUM    27
#define HREF_GPIO_NUM     25
#define PCLK_GPIO_NUM     19
Early in CameraWebServer.ino is a section where we UNcomment (activate) only one #define of a camera model to match ours.
In CameraWebServer.ino, at the end of the camera model list, add the line (based on the new pin list's heading):
        #define CAMERA_MODEL_TTGO_WROVER_B  // TTGO White Board

Deactivate/comment out (start the lines with //) all the other camera model defines.

        Next, Final Preparations & Uploading the Sketch

[papa]

        Final Preparations & Uploading the Sketch

Before you flash new firmware on the board, consider using ]this post to backup the existing firmware.

In the sketch, replace the two "*******" with your WiFi's SSID & Password.

Since you started with an example sketch, save as a new name, perhaps CameraWebServer01

Given the above preparations (including Tools/Board configuration), the sketch should now compile & upload to a USB attached ESP32 camera board.

BTW, I was able to flash the TTGO White Board just with a USB connection, no grounding a pin, no mashing a button.

Open the IDE's Serial Monitor & have it set to 115200 baud.

Once the upload completes, press the board's reset button (RST) once or twice to see in the Serial Monitor:

WiFi connected
Starting web server on port: '80'
Starting stream server on port: '81'
Camera Ready! Use 'http://xxx.xxx.xx.xxx' to connect

        Make note of the IP address in the above Serial Monitor output.

For camera board troubleshooting, see this at RandomNerdsTutorials.com.

    Using the CameraWebServer Firmware

[papa]

        Using the CameraWebServer Firmware

In a browser, enter the IP address you got from following the above post.

This displays a browser interface of several settings & buttons to work with the ESP32 camera board.

    This interface resulted from the 3 firmware sketches I used, probably due to their all including the esp_camera.h library.

        You may only need some of these settings/buttons, for example ...

Resolution sets the image size in the browser (starts at QVGA)

V-Flip & H-Flip help us orient the image as desired depending on how we orient the board.

Get Still displays an image in the browser

Start Stream displays continuously updated images in the browser.

        The following are only available for stills at CIF or lower resolution

Face Detection allows saving a face image for recognition.

    Enroll Face

Face Recognition activates recognizing saved faces (if Resolution is not set too high).

If you click the Start Stream button, a continually updated camera image displays in the browser & image info scrolls in the Serial Monitor.

With another sketch (provided later), I could add a sitemap entry for openHAB to display the image stream in the User Interface, but that did not seem to work with this firmware.

    Next, Using More Features of the TTGO White Board

[papa]

        Using More Features of the TTGO White Board

The first sketch I tried was esp32-camera-bme280.ino which LILYGO github provided seemingly tailored for the TTGO White Board (including its OLED screen). I had trouble with using the sketch. For one thing I discovered my board had the SMD pads for the BME280 sensor chip, but not the chip itself that the sketch included.  Even with BME280 code disabled in the sketch I still had problems.

It was helpful to adapt code from esp32-camera-bme280.ino into other sketches like CameraWebServer.ino. This enables using more features of the TTGO White Board (like the OLED).

    In the Arduino IDE's Tools/Manage Libraries, install OneButton library, freeRTOS, Adafruit SSD1306 library & "ESP8266 and ESP32 Oled Driver for SSD1306 display" (NOT the Adafruit SSD1306 library which works differently).

           In the camera_pins.h tab just above "#else" , add these TTGO White Board configs to those you added at this post above:

// ## added from www.banggood.com/LILYGO-TTGO-T-Camera-ESP32-WROVER-with-PSRAM-Camera-Module-OV2640-Camera-0_96-Inch-OLED-p-1418433.html

#define I2C_SDA 21
#define I2C_SCL 22 // ## & esp32_camera_bme280_org.ino
#define AS312_PIN 33 // ## " PIR
#define BUTTON_1 34 // ## " Right button

                                                                Save camera_pins.h

        Next, Using More TTGO White Features, Continued

   

[papa]

        Using More Features of the TTGO White Board, Continued

Now in Arduino IDE with CameraWebServer01.ino & the other 3 files opened, save as CameraWebServer06.

    CameraWebServer06pub.ino (8.5 KB)   << Download this file. Open it in a text editor like Notepad++. Select & copy all the file's lines.

Back to CameraWebServer06 in Arduino IDE, delete all its lines & paste in all lines you copied from CameraWebServer06pub.ino & save.

     Edit the resulting sketch to have your WiFi's SSID & password. Save.

In summary, a sketch folder should contain CameraWebServer06.ino (now with the contents of CameraWebServer06pub.ino & your WiFi info) AND the other 3 files (including an augmented camera_pins.h)

The firmware resulting from these edited files should now be ready to upload to the TTGO White Board.

        Next, Using the Augmented Firmware

[papa]

        Using the Augmented Firmware via the Edits of
          CameraWebServer.ino & camera_pins.h

After editing & uploading from the above posts (& the TTGO White Board is connected to USB), press its reset button (RST) once or twice. On the Serial Monitor, you should see results similar to the last half of this post.

On the OLED screen, you should also see "TTGO Camera" ""WiFi connected"  Then your OLED should alternate two screen shots 1) "[your board's IP address]" 2) A blank screen available for later. When the board detects motion, screen shot 1) will also display "PIR Trigger"

Follow this post on using this new firmware in a browser tab.

Via this new firmware, pressing the board's other button (labeled IO34) will vertically flip the browser tab's video stream or next still image.

        Next, Firmware to Display Camera Output in OpenHAB's UI

[papa]

        Firmware to Display Camera Output in OpenHAB's UI

The third sketch I worked with came from a helpful site called Random Nerd Tutorial

The sketch in the posting linked just above was set up for another ESP32 camera with pin assignment lists for some other cameras. Unlike the last example with 4 files, this is only one file that contains the board pin assignments.

   RN_VideoStreamttgo04_pub.ino (11.01 KB)   << Forum members, download my 04 version of this sketch, which changes some things:

    I added: pin assignments & instructions for the TTGO White Board & some others (not all tested)

                    some more instruction

                lowering of the displayed frame size (resolution) for higher initial frame rate

        setting vertical flip so the image is right side up when the TTGO White Board's USB connector is at the bottom

So far my version of this sketch uses only the TTGO White Board's camera, not other features (PIR, push button, OLED screen).

        Next, Customizing the Firmware Sketch

[papa]

        Customizing RN_VideoStreamttgo04_pub.ino

                You will need to customize the sketch:

Edit in your WiFi credentials.

As is, the sketch defines CAMERA_MODEL_TTGO_WROVER_B to use the TTGO White Board. For another board, comment (//) that board & uncomment another one. Match your board's pin assignments to one on the list or add a new define naming your board & a list of your board's pin assignments.

        Possible customizing:

Not too far from the sketch's end is a line setting the persisting display resolution which you could make larger or smaller.

            config.frame_size = FRAMESIZE_VGA;

Instead of VGA, you could use other resolution names.

    From esp_camera.h:  Size of the output image: FRAMESIZE_ + QVGA|CIF|VGA|SVGA|XGA|SXGA|UXGA

    A little later is a section starting with "#if defined(CAMERA_MODEL_TTGO_WROVER_B)"

This vertically flips the displayed image for my TTGO White Board. If you need this for another board, change the "#if defined..." to your camera model name or just delete "#if defined..." at the section's start & the "#endif" at the section's end.

    Next, Uploading RN_VideoStreamttgo04_pub.ino

[papa]

        Uploading RN_VideoStreamttgo04_pub.ino

I adapted & augmented uploading/flashing instruction from the original sketch:

IMPORTANT!!! For some boards (e.g. "AI Thinker ESP32-CAM") to upload a sketch:
    - GPIO 0 must be connected to GND
    - Then press the on-board RESET button to put board in flashing mode

IMPORTANT!!! To flash the TTGO White Board defined below as CAMERA_MODEL_TTGO_ESP32_White
    - Arduino IDE: Select Board "ESP32 Dev Module" flash freq 40mhz; partition: huge app; psram enabled
    - Do NOT try to GND GPIO 0 or hold down RESET button. Flashing works without this.

As the first instruction indicates, to flash ESPxx boards, we've expected to ground a pin &/or mash a push button. That may be required for your board. As the second instruction indicates, the TTGO White Board attached to a computer's USB is ready to flash as is. In fact, holding down the TTGO White Board's reset button seems to put the board in sleep mode.

    Next, Initial Results with RN_VideoStreamttgo04_pub.ino

[papa]

        Initial Results with RN_VideoStreamttgo04_pub.ino

So far, this sketch does not use the OLED screen on the TTGO White Board. So after uploading the sketch, open the Arduino IDE's Serial Monitor at 115200 baud. Maybe push the board's reset button (RST) once or twice.

On the Serial Monitor, we should see something like:

WiFi connected
Camera Stream Ready! Go to: http://[Camera Board IP Address]

Open a browser tab, enter the IP displayed on the Serial Monitor, & see the video stream in the browser.

Unlike with the other two firmware sketches used above, the browser will not display a console to change settings for the camera board. The perhaps inconvenient news is that (at least for now) to change camera settings, one needs to change & add those settings in the sketch & upload again. The good news follows in the next post.

        Note: Serial Monitor output can be increased by UNcommenting the sketch lines (may affect the stream):

// Serial.printf("MJPG: %uB\n",(uint32_t)(_jpg_buf_len));

//Serial.printf("Starting web server on port: '%d'\n", config.server_port);

        Next, Display Camera Output in OpenHAB's UI

[papa]

        Display Camera Output in OpenHAB's UI
        Using RN_VideoStreamttgo04_pub Firmware

From following the post just above, note the camera board's IP address displayed on the Arduino IDE's Serial Monitor.

Important: If you have a browser tab open hosting the video stream, CLOSE it now. (Otherwise the following won't work.)

Have your openHAB User Interface (UI) open in a browser tab.

        In an openHAB .sitemap file (Linux /etc/openhab/sitemaps), add this line, including your board's IP address where indicated (& save the file):

Image url="http://camera board's IP Address from above" refresh=1000

In openHAB 3 go to localhost:8080. Log in if needed. At the upper right, click on the other apps icon. Then click on Basic UI. Then click on the .sitemap file that contains the Image url=... line.

With occasional temporary blanks, the openHAB UI should display the video stream, refreshing it every 1000 milliseconds (1 second).

More: Disconnect the Camera Board from your computer's USB. Provide a power source via the USB cable or the provided LiPo battery cable & connector. Wherever the camera board is, as long as it can connect to your WiFi, it should display video stream on openHAB's UI. (You may need to push the board's reset button once or twice to get it connected to WiFi.)

Now from what I can learn about the three above sketches' code, I'll experiment in mixing their features to create firmware.  Perhaps adapt battery code from here or here.

        Next, Stay Tuned to This Video Station ;-)

[papa]

        More Progress Made

I have a firmware sketch that accesses the LILYGO camera TTGO White Board's RSSI, firmware version, IP address, PIR sensor, & push button. Also via MQTT, the firmware sketch publishes messages for those functions to openHAB. I've configured openHAB Things, Channels, items, & sitemap to collect the data from this camera node.

Beside the 3 above ESP32 camera sketches, my sketch also adapts computourist's ESP_SONOFF_12.ino.

I have the TTGO White Board's camera streaming server working (in a browser tab or openHAB's UI) with the above. I did more clean up & reorganizing in the sketch.

I experimented more with the OLED code from the above sketches & others. Through that, I was able to adapt workable OLED code into the sketch.

Update: In the above thread, I changed my camera board's designation from LILYGO to TTGO White Board (Thanks, Andreas Spiess).

Still to come: Using gained capabilities toward making a Video Doorbell Node.

        Next, Preparing to Use an ESP32 [Camera] Board

            with openHAB

[papa]

                    Preparing to Use
        an ESP32 [Camera] Board with openHAB

Needed, things covered above: an ESP32 Board, means to upload to the board (USB connector or FTDI device), Arduino IDE, the ESP32 Add-on for Arduino IDE (see this post), & your board's pin assignments (for hints, see this post) if they are not listed in a sketch already.

Also needed: A recent openHAB install (2.5.x) with MQTT 2.x Binding & MQTT Broker also installed into openHAB.

        See here for installing openHAB 2.5.x.

Start here for installing MQTT Binding & the embedded/system MQTT Broker.

See here for some troubleshooting for MQTT.     See here about Things.

Also if needed, add some code libraries, using Arduino IDE's Tool/Library Manager:

        PubSubClient

            ESP8266 and ESP32 OLED driver for SSD1306 displays

If you have not yet, perhaps save a backup copy of your board's original firmware or other firmware presently on your board. See this post.

    Next, a Sketch of Firmware for ESP32 Board

[papa]

        A Sketch of Firmware for ESP32 Board

        Connecting Camera Board to OpenHAB

       The sketch & .h header file previously posted here worked fairly well, but less than hour after the node was rebooted, the node lost communication with openHAB.  I've since added some example code for reconnecting MQTT messaging & so far, the Node & openHAB are staying connected over MQTT.

I added code from a PubSubClient example sketch for reconnecting MQTT if disconnected. So far for 24 hours, the node's MQTT has never stopped communicating with the openHAB User Interface. Previously, by 60 minutes, the node would have stopped communicating with openHAB.

Updated, Feb. 18, 2020 so openHAB can send messages to the OLED screen.  See more below.

    ESP32_mqtt_02d09e_pub.ino (42.88 KB)   << As a forum member, download this firmware sketch to where you can find it

       ^^ May 26, 2020, updated & successfully used with ESP32-CAM.

With Arduino IDE, open the firmware sketch & OK its being put in a folder (make note of the folder)

        Close the firmware sketch just downloaded & saved to a folder.

    ESP32_pins.h (11.1 KB)  << Download this .h header file to the folder now holding the firmware sketch that was downloaded above.

        ^^ May 26, 2020, updated & successfully used with ESP32-CAM.

(Note: as camera_pins.h, this came with CameraWebServer.h, but I renamed & augmented it for this sketch.)

Re-open the firmware sketch & see the ESP32_pins.h header file in another Arduino IDE tab
    From now on, when you save these two files, they should be in the same folder.

Go to the next post for customization of the firmware sketch & the .h header file & configuring for openHAB

        Next, Customizing the ESP32 Board Firmware

[papa]

        Customizing the ESP32 Board Firmware (ESP32_mqtt....ino)

nodeID must be a unique number for each WiFi device, but should be a unique node number in your openHAB system. I assume 30 below.

wifi_ssid_A, wifi_password_A         edit in your SSID & password

If you have another WiFi station, edit in that info to wifi_ssid_B, wifi_password_B. Otherwise, give B the same WiFi info as A.

For mqtt_server, edit in the IP address of your computer hosting the mqtt broker (See above for install)

In the list of Camera_Model_xxxxxx, UNcomment & thus select ONLY ONE board. As is, the sketch, activates CAMERA_MODEL_TTGO_ESP32_White, the only one I've tested so far. (Though as noted, RandomNerds.com tested the Camera code of some of the boards & other boards came with the official example, CameraWebServer.)

Caution 1: in ESP32_pins.h, for the Camera_Model you activate, check/correct the pin assignments to the best of your ability. If needed, create another pin assignment list & Camera_Model name.
                                    Updated, Feb. 18, 2020:

Caution 2: At the end of the firmware's (version ...02d09d) customization section are five defines: Enable_SSD1306, OLED_TXT, BTN, PIR, & Camera. Because CAMERA_MODEL_TTGO_ESP32_White can use all these features, all five are UNcommented/activated. Comment/DEactivate any define that does not apply to your board or to the functions you desire.

        Next, Uploading the Firmware & Expected Results

[papa]

        Uploading the ESP_mqtt Firmware & Expected Results

Connect the ESP32 board to your Arduino IDE computer's USB (perhaps via FTDI device) & upload the sketch.

        Nothing special should be required for CAMERA_MODEL_TTGO_ESP32_White

            Other boards may require grounding a pin &/or pressing a button.

                See some notes early in the sketch

        Next, Expected Results on Serial Monitor & OLED Screen (if present, & Browser Tab

[papa]

        Expected Results on Serial Monitor,

                OLED Screen (if present),

                   & Browser Tab

With the board connected to USB, on Arduino IDE, open the Serial Monitor at 115200 baud.

If the board does not connect to WiFi/MQTT shortly, press its Reset button once or twice.

Searching the sketch (Ctrl+F) for "ifdef SHOWME" will show you various success & error messages that can display on the serial monitor.

Including: push button states (up, dwn), PIR states (Motion, No Motion), & messages being sent for openHAB (which needs configuring as shown later).

Searching the sketch (Ctrl+F) for "ifdef ENABLE_SSD1306" will show you various success & error messages that can display on the OLED screen (if present).

Including: "TTGO Camera" IP,  Address of the ESP32 board, "PIR Triggered" (if PIR present & activated & motion in front of PIR), &

    a screen of simulated BME sensor data which could be repurposed to show other information.

Note the camera board's IP address displayed on the Arduino IDE's Serial Monitor & OLED screen.  Enter that IP address in a browser tab & see the camera stream there.

    Next, Configuring openHAB to Handle Board Data

[papa]

        Configuring openHAB to Handle Board Data

                    Introduction

My preferred method is using PaperUI for Add-ons, Things, Channels, & linking items to channels. I use text files to configure items, rules, & sitemaps.

PaperUI displays Things in alphabetical order, but displays a Thing's channels in the order they are created.

Hint: This PaperUI process is somewhat tedious. It's quicker, but "messier" if you create Things as you need them. If you like the Things in neater alphabetical order, create them in that order & create all you may eventually use. (It should hurt nothing if they are not used.)

For the TTGO White Board, I'll likely use devices/channels: 02 (RSSI), 10 (IP address), 40 (push button), & 41 (PIR) & will create those in order below. The device's firmware also has 03 (firmware version).

Again the following assumes the features of a TTGO White Board which has been programmed as Node 30.  When you use the instructions below, adjust them to the features of your board.

        Next, Create a Thing for Our Board

[papa]

            Configuring openHAB to Handle Board Data
                    Create a Thing for Our Board

Go to PaperUI\Configuration\Things   Click ( + ), (plus sign in a blue disc). Click "MQTT Binding". Click "ADD MANUALLY"

We must choose which type of Thing for the MQTT 2.xx Binding. All but one type are standards that may fit a standard used on our physical device. For physical devices (like our DIY RFM69 or ESPxxxx Nodes) which don't match listed standards, we use the Generic Thing & configure the Thing with the message standard used on our device.

          Click Generic MQTT Thing

    Configure the Generic MQTT Thing using something like the following fields:

                (Customize to your situation.)

Name [starts as Generic MQTT Thing] Node30db << This is the ESP32 Node to be accessed.

Thing ID [ randomly generated for us ?? ]

Location MAINFL [Thing's location on a tab of PaperUI\Control, may be blank ]

     

Bridge Selection MQTT Broker - mqtt:systemBroker:embedded-mqtt-broker

   ^^ If we install Mosquitto or another broker instead of the embedded broker, I believe that will show in the Bridge Selection drop down list.

To create this Node30db Thing with the above properties, click its  ( ✔ ), check mark in a blue disc.

      Next, Create Channels for the Thing

[papa]

        Configuring openHAB to Handle Board Data
             Create Channels for the Thing

Hint: Be careful as you add Channel type, Channel id & Label below. Clicking the channel's (pencil icon) for editing only lets us edit fields from MQTT State Topic & lower. To change Channel type, etc, we must delete (trash can icon) the channel, start over, & accept the channel's being listed at the end of its Thing's list.

First we add a channel 02 for the node's RSSI (signal strength).

To add channels to this Thing, click the ( + ) to the right of "Channels"

    Complete the Add channel fields:

            (Customize to your situation)

Channel type: Text value in the drop down list

Channel id: Node30Channel02

   Label: dBell_30_RSSI

MQTT state topic: home/esp_gw/nb/node30/dev02

   Check your work & Click "SAVE"

            Next, Continue Creating Other Channels

[papa]

        Continue Creating Other Channels
           For Node30db(or your Thing)

Follow the same process as in the last post. The following lists the field contents which you customize to your situation.

Channel type: Text value in the drop down list

Channel id: Node30Channel03

    Label: dBell_30_Version        [Node's Firmware version]

MQTT state topic: home/esp_gw/nb/node30/dev03     (Click SAVE)

  Channel type: Text value

Channel id: Node30Channel10

    Label: dBell_30_IP                [Node's IP address]

MQTT state topic: home/esp_gw/nb/node30/dev10      (Click SAVE)

  Channel type: Text value

Channel id: Node30Channel40

    Label: dBell_30_BTN            [state of Node's push button]

MQTT state topic: home/esp_gw/nb/node30/dev40 (Click SAVE)

    Next, Create a PIR Channel with a Transformation