ESP32 Smart Doorbell (MQTT, Camera, PIR, OLED, MIC)

[papa]

        Introduction

ESP32 is a more robust platform than ESP8266. ESP32 camera boards caught my attention. I decided to explore programming such boards with Arduino IDE & seek openHAB integration. I was attracted to building DIY Smart Doorbells. The ESP-CAM board looks interesting. It's cheaper & smaller than what I bought & has an SD card slot, flash, 16 connector pins that could mount to a solderless breadboard, & much online documentation. Unlike what I bought, it does not have screen, PIR, or USB connection. Some online builds address those lacks.

At Amazon, I liked the features (see next post) & bought this TTGO T-Camera ESP32-WROVER-B OV2640 Camera Module ESP32 WROVER & PSRAM Camera Module 0.96 OLED ESP32-WROVER (Normal Camera). My board is currently NOT available at Amazon. Currently Amazon suggests this which has almost the same name: HiLetgo camera board with fish-eye camera ($24, same pin functions as my board).  For $18.50 Banggood.com has what I believe is the same board with a regular camera, 

        << Forum Members, click on pic for larger view

Banggood.com offers a similar board but also with a MIC for $25-$29, depending on camera & one chip. Its pin functions are different from my board which is addressed below.

            Next: Specifications of My TTGO ESP32 Camera White Board

[papa]

        Specifications of My TTGO ESP32 Camera White Board

        T-Camera specification
Master Chip: esp32 dual-core         CPU Frequency 240Mhz (WiFi, BT)
    ESP32 Module: Espressif WROVER-B     Crystal 40Mhz
Protocol: Wi-Fi 802.11 b/g/n & Bluetooth 4.2 BLE & BR/EDR
    Flash: 4MBytes        PSRAM: 8MBytes
Display chip: SSD1306 I2C        Display type: OLED        Display resolution: 128x64
    PIR: AS312        Uart Chip: CP2104     I2C

USB connector for operation & programming

    Camera: OV2640        Camera Resolution: 2Megapixel

[Battery ?] Charging chip: IP5306        Battery connector & cable included

One online sketch for the TTGO White Board had programming for a BME280 sensor (temp, humidity, atmospheric pressure). Using that sensor would require editing some library files to avoid conflicts between a struct for the camera & a struct for the sensor. The board I received had no BME280 mentioned in the specs & indeed some SSMD pads on the board were labeled BME280, but the chip was absent. Somewhere I may have seen that the BME sensor's presence on the board did not work accurately so the sensor was omitted.

        Next, A Camera Board with Similar Appearance,

                        But with MIC & Different Pinout

[papa]

        A Camera Board with Similar Appearance, But with MIC & Different Pinout

Compared to my TTGO White Camera Board (no MIC), Banggood.com offers a similar board but also with a MIC for $25-$29, depending on camera & one chip (See chip chart below). Its pin functions are different from my board (See next post.) Using that different pinout will be addressed later in this thread.

    This T-Camera's Specification:

Master Chip: esp32 dual-core
Protocol: Wi-Fi 802.11 b/g/n & Bluetooth 4.2 BLE & BR/EDR
Flash: 4MBytes        PSRAM: 8MBytes
Display chip: SSD1306 I2C    Display type: OLED    Display resolution: 128x64
PIR: AS312        Uart Chip: CP2104
Fish-eye with Mic Uart Chip：CH9102F QFN24
Charging chip: IP5306 I2C（Mic T-Camera without I2C)
Camera: OV2640        Camera Resolution: 2Megapixel

        Github support:
github.com/Xinyuan-LilyGo/esp32-face-recognition
github.com/Xinyuan-LilyGO/LilyGo-Camera-Series

       << Chip info

        Next, Pinouts for Camera Board with MIC

[papa]

                Pinouts for Camera Board with MIC

       << Forum Members, click on pic for larger view

                Next, Reference Names for Camera Boards

[papa]

                Reference Names for Camera Boards

There are a variety of camera boards using sometimes similar names (LILYGO, TTGO, HiLetgo).  Andreas Spiess' helpful video & "SHOW MORE" comments call my board "TTGO ESP32 camera White" which seems as specific as possible.

Below I'll use "TTGO ESP32 Camera White" or "TTGO White" for boards with similar appearance & pinouts.

I'll use "LILYGO White w MIC" for boards who appear similar to TTGO White, but with a MIC AND different pinouts.

Unless indicated otherwise, what I write below specifically applies to my TTGO White camera board, but is probably adaptable to other ESP32 boards (even those without cameras).  Looks like one can adapt code to the board you get. Sometimes sketches have definition lists based on board pin names.  In those lists, you might match your board's pin assignments OR using your board's pinouts (as I did), you might create a new definition list for a camera board (more below).

To upload a sketch, you select an ESP32 board's name under Arduino IDE's Tools/Board (often ESP32 Dev Board will work), but then more importantly, make sure the Tools/Board settings match specifications of your board. Below I model this for the TTGO White Board.

Note: In the following thread, most posts were adapted & updated from early 2020 when Windows specific procedures were used. Linux procedures were currently used (2021). If an OS is NOT mentioned when I believe instructions will work for either OS.

        Next, Trying the TTGO White Board As Is & Preparing to Flash New Firmware (Windows)

                    Note: Linux will follow

[papa]

        Trying the TTGO White Board As Is & Preparing to Flash New Firmware (Windows)

If you use Linux, you can just watch the video below & go to the next post for what you'll do.

DIY Smart Doorbell #2 + ESP32 & ESP8266 Firmware Backup | Home Assistant

     This youtube project ^^ was informative & inspiring. Here are time indexes to parts I used:

   :35 Trying the ESP32 Camera Board As Is
1:28 DrZzs: "Enroll Face [for face recognition] is blocked out"
    papa: Actually, set a much lower resolution & then [Enroll Face] is available

1:54 How to Backup any ESP-based firmware

2:07 Install ESPtool     github.com/espressif/esptool

2:13 start with installing Python
(I used the latest version of Python 2.7 which we've used elsewhere on the forum)

3:26 Windows cmd window: Go to Python install folder
4:04 cmd: pip install esptool       Github notes for esptool: github.com/espressif/esptool
           Then cmd: cd "C:\Python27\Scripts"

7:20 To read TTGO White Board's 4MB of firmware & save in a .bin file ...

        cmd: esptool.py -p COM[#] -b 460800 read_flash 0 0x400000 ttgo_cam.bin
                    Replace [#] ^^ with the number of the port connected to your ESP device

             The file will save in the \PythonXX\Scripts folder

    That saved firmware can be restored to ESP board via ...

        esptool.py --p COM[#] -b 460800 write_flash 0 ttgo_cam.bin

8:44 quick summary of the above procedures

After the above, DrZzs moves to using Home Assistant, which looked promising. See more below.

    Next, Preparing to Flash New Firmware (Linux)

[papa]

        Trying the TTGO White Board As Is & Preparing to Flash New Firmware (Linux)

DIY Smart Doorbell #2 + ESP32 & ESP8266 Firmware Backup | Home Assistant

This youtube project ^^ was informative & inspiring. For Linux background, you might watch these indexed sections:

   :35 Trying the ESP32 Camera Board As Is
1:28 DrZzs: "Enroll Face [for face recognition] is blocked out"
    papa: Actually, set a much lower resolution & then [Enroll Face] is available.

Linux comes with Python installed. To check, in Terminal, enter: python --version
     My result: Python 2.7.17  [good to go]

            Linux install esptool...
Check, Terminal:  esptool version
Terminal:  sudo apt-get update -y                  sudo apt install python-pip
            Check install, Terminal:  pip -V
My result:  pip 9.0.1 from /usr/lib/python2.7/dist-packages (python 2.7)

Terminal: sudo pip install setuptools wheel pyaes esptool
      Check install, Terminal:  esptool.py version            Myresult: esptool.py v3.1
       
Github notes for esptool: github.com/espressif/esptool

                Next: Preparing to Flash New Firmware (Linux), Continued

[papa]

            Preparing to Flash New Firmware (Linux), Continued

Have only one ESP device connected to computer's USB (esptool will find it)
             To read camera board's 4MB of firmware & save in a .bin file, Linux Terminal:
sudo esptool.py -b 460800 read_flash 0 0x400000 ttgo_cam_orig.bin
                               (by default stored at /home/<username>)

        My results:       esptool.py v3.1
Found 2 serial ports    Serial port /dev/ttyUSB0    <<  Note the ESP serial port detected
Connecting....        
Detecting chip type... ESP32        Chip is ESP32-D0WD (revision 1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz            MAC: 84:0d:8e:d2:93:58
    Uploading stub...        Running stub...    Stub running...
Changing baud rate to 460800        Changed.
    4194304 (100 %)            4194304 (100 %)
Read 4194304 bytes at 0x0 in 98.4 seconds (341.1 kbit/s)...
      Hard resetting via RTS pin...

            To read & back up firmware via a specific device port, my Terminal command:  (Use your own device port.)
sudo esptool.py -p /dev/ttyUSB0 -b 460800 read_flash 0 0x400000 ttgo_cam_orig.bin

My saved firmware can be restored to the ESP board via Terminal:     (Use your own device port.)
    sudo esptool.py -p /dev/ttyUSB0 -b 460800 write_flash 0 ttgo_cam_orig.bin

        To find device port, Linux Terminal:     ls -l /dev/serial/by-id     My results:
lrwxrwxrwx 1 root root 13 Aug 28 11:20 usb-Arduino__www.arduino.cc__Arduino_Uno_FB0001-if00 -> ../../ttyACM0
lrwxrwxrwx 1 root root 13 Sep 1 15:40 usb-Silicon_Labs_CP2104_USB_to_UART_Bridge_Controller_01DE0085-if00-port0 -> ../../ttyUSB0

     The ESP device listing contains "UART_Bridge"      From the above result, the device port would be    /dev/ttyUSB0

Do you get a "Permission Denied" error from a port? Add the user to the dialout group with a Terminal command like sudo usermod -a -G dialout <Your Linux USERNAME>

            Next, New Firmware via ESPHome, HASSIO, & Home Assistant ??

[papa]

greginkansas said:

Good work so far. I to have been looking for a good program for the ESP32 camera boards. Face or motion with MQTT would be great. All the streamers I have used so far crash after a few hours or day running on my zoneminder server.

Papa: Later in the thread, I provided motion reported by MQTT.  Something later still may provide face recognition reported by MQTT.

[papa]

            New Firmware via ESPHome, HASSIO, & Home Assistant ??
                    (Not for Me ! ! !)

After what I reported in the post above, DrZzs' youtube video moves to using Home Assistant. Actually, that seemed like a pretty neat, easy method with a BIG IF ... if one does not use Windows 10 like I mostly do. To work at all on Windows, this method requires installing & configuring Oracle VM VirtualBox.

I took long detours on this. If you want to try this on Windows, I used another DrZzs youtube video & fairly detailed .pdf instructions by "Mark."  Here's another video on this.  I just found this DrZzs video (skip to about 7:30) on installing HASSIO on Windows which demonstrates a powerful one line script using Proxmox instead of Virtual Box to get HASSIO into a VM.  An Internet search for "DrZzs Proxmox" gives instructions on installing Proxmox.

At one point, I had Home Assistant working, but discovered I could not have the required HASSIO without the Oracle install. I never quite got the HASSIO VM working. For the time being, I abandoned this Home Assistant approach. Again, especially if one does not use Windows, the ESPHome, HASSIO, & Home Assistant approach might be interesting.

A few days later, I tried the VirtualBox route & blew up my Windows install (including openHAB). Later I saw that such approach did not work well on laptops like mine. I had to reinstall Windows & reinstall & configure openHAB, Arduino IDE, etc. before I could continue with the ESP32 Smart Doorbell project.

Below I return to & continue the Arduino IDE progress I made before the Home Assistant detour.

        Next, Getting the ESP32 Add-On for Arduino IDE

[papa]

          Getting the ESP32 Add-On for the Arduino IDE

I started with something that seemed to use all features of the TTGO White Board, but I had trouble so I decided to try something basic & build on that. CameraWebServer.ino comes with the needed ESP32 Add-on for IDE & becomes available from Arduino IDE's File/Examples menu when one selects an ESP32 board in Arduino IDE's tools/boards menu.

This site was helpful. It instructed installing the needed Arduino IDE & the ESP32 Add-on for Windows & Mac/Linux.

In preparation for programming an ESP32 board (including a camera board), install the needed ESP32 add-on.

           In Arduino IDE, go to File\Preferences

In the “Additional Board Manager URLs” near the bottom of Preferences, enter the URL below & click the OK button.  If other URLs are in the box, end the list with a comma & then add the URL. OR click the nearby "page & shadow" icon & enter the URL on a separate line.

        dl.espressif.com/dl/package_esp32_index.json

Go to Tools\Board\Boards Manager. In the box near Boards Manager's top, search for ESP32 & press the install button for “ESP32 by Espressif Systems“   When the install finishes, you may close the Boards Manager.

        Next, Arduino IDE: Accessing an ESP32 Board

[papa]

        Arduino IDE: Accessing an ESP32 Board &

            the Example Sketch, CameraWebServer.ino

The following assumes first having completed what's in posts above. (Did you save the board's original firmware as shown above in this Windows post or this Linux post.)

For the following to work, you MUST now connect an ESP32 board (perhaps through an FTDI device) to your Arduino IDE computer's USB. (The TTGO White Board has an USB connector.)

Then go to Arduino IDE's tools/boards menu & scroll down to see & select an ESP32 board. Perhaps your board will be listed, but I just use ESP32 Dev Module which seems generic. You may sometimes need to select a port in the config list that appeared when you selected the ESP32 board.

In File/Examples, scroll down to ESP32/Camera & select CameraWebServer. Arduino IDE should open 4 files in 4 tabs: CameraWebServer.ino, app_httpd.cpp, camera_index.h, & camera_pins.

Caution: Before we upload any ESP32 sketch (even when you've selected a board that resembles a name on your board), we must make sure the sketch pin assignments are correct for our board. The Arduino IDE's configuration list for the board may also need some tweaking. (See more below)

    Next, Preparing to Use CameraWebServer.ino

[papa]

        Preparing to Use New Firmware, CameraWebServer.ino

Hint: In Arduino IDE's File/Preferences menu, have verbose output turned on for compiling & uploading. That gives us steady feedback that things are working & any problems arising.

In Arduino IDE's Tools menu, tweak some configs for the ESP32 Dev Module. For the TTGO White Board at least:
    Have Flash Frequency at 40MHz (the TTGO White Board's crystal frequency)
    Have Partition Scheme at Huge APP (to help it be large enough for the sketch)
    Have PSRAM at enabled (the TTGO White Board does have PSRAM, image RAM)  (Important for some performance)
        Using your board's specs, change any board configs as needed.  See this above post for TTGO White Board specs or this post above for the LILYGO White with Mic.

            Next: Preparing to Use New Firmware, Continued

[papa]

            Preparing to Use New Firmware, CameraWebServer, Continued

Again CameraWebServer is an Example sketch, which means we can only read it, not change & save it & the related files. In the Arduino IDE menu go to /File/Save as... & give the sketch a different, but related name, like CameraWebServer002.

From installing Arduino IDE & the ESP32 add-on (see above), Arduino IDE should the libraries it needs for CameraWebServer.ino

However, we need to ensure the sketch has the correct pins configuration for our board (in this case the TTGO White).

    Next, Pins Configuration for Our Board

[papa]

        Pins Configuration for Our Board

As I cautioned above, "Before we upload any ESP32 sketch, we must make sure the sketch pin assignments are correct for our board." Sometimes different camera models' pin assignments may be listed within the sketch.  For CameraWebServer.ino, the models' pin assignments are in the header file, camera_pins.h (which loads in an Arduino IDE tab along with CameraWebServerXXX).

The TTGO White Board's pins are often given on sites where they are sold (See example in this post.)  My board also came with a chart on its protective box. This pin list did not match anything in camera_pins.h including under model names with "WROVER" or "TTGO" .  In cases like this, we put our camera board's pin assignments in a new camera_pins.h section.  For the TTGO White Board, copy the following, paste it just above the "#else" near the end of camera_pins.h & SAVE.

#elif defined(CAMERA_MODEL_TTGO_WROVER_B) // TTGO White Board
#define PWDN_GPIO_NUM     26
#define RESET_GPIO_NUM    -1  // -1 means "not available"
#define XCLK_GPIO_NUM     32
#define SIOD_GPIO_NUM     13
#define SIOC_GPIO_NUM     12

#define Y9_GPIO_NUM       39
#define Y8_GPIO_NUM       36
#define Y7_GPIO_NUM       23
#define Y6_GPIO_NUM       18
#define Y5_GPIO_NUM       15
#define Y4_GPIO_NUM        4
#define Y3_GPIO_NUM       14
#define Y2_GPIO_NUM        5
#define VSYNC_GPIO_NUM    27
#define HREF_GPIO_NUM     25
#define PCLK_GPIO_NUM     19
The above pin list for TTGO White has a unique name after #elif defined & then #defines for the TTGO pin functions. For a camera board whose pin assignments are NOT in camera_pins.h, imitate what I've done for the TTGO White.
 
        Next: Edit CameraWebServer Sketch to Use Our Board's Pin Assignments

[papa]

        Edit CameraWebServer Sketch to Use Our Board's Pin Assignments

Early in CameraWebServer is a section where we UNcomment (activate) only one #define of a camera model to match ours.

In CameraWebServer.ino, at the end of the camera model list, add the line (based on the new pin list's heading):

#define CAMERA_MODEL_TTGO_WROVER_B  // TTGO White Board

If you have added a different board with unique pin assignments, add its defined NAME at the end of the camera model list.

Deactivate/comment out (start the lines with //) ALL other camera model defines.  SAVE.

        Next, Final Preparations & Uploading the Sketch

[papa]

        Final Preparations & Uploading the CameraWebServer Sketch

Before you flash new firmware on the board, did you use this post to back up the existing firmware?

In the sketch, replace the two "*******" with your WiFi's SSID & Password.

Since you started with an example sketch, did you save it as a new name, perhaps CameraWebServer002?

Given the above preparations (including Tools/Board & Tools/Port configuration), the sketch should now compile & upload to a USB attached ESP32 camera board.

BTW, I was able to flash the TTGO White Board just with a USB connection, no grounding a pin, no mashing a button as some boards require.

Open the IDE's Serial Monitor &o have it set to 115200 baud.

Once the upload completes, press the board's reset button (RST) once or twice to see in the Serial Monitor:

WiFi connected
Starting web server on port: '80'
Starting stream server on port: '81'
Camera Ready! Use 'http://xxx.xxx.xx.xxx' to connect

        Make note of the IP address in the above Serial Monitor output.

For camera board troubleshooting, see this at RandomNerdsTutorials.com.

    Using the CameraWebServer Firmware

[papa]

        Using the CameraWebServer Firmware

In a browser, enter the IP address you got from following the above post.

This displays a browser interface of several settings & buttons to work with the ESP32 camera board.

    This interface was displayed by the 3 firmware sketches I used, probably due to their all including the esp_camera.h library.

        You may only need some of these settings/buttons, for example ...

              Scroll down to the bottom of the interface for
Get Still displays a camera-produced image in the browser

Start Stream displays continuously updated camera images in the browser.

     If you change the following, you may need to click [Get Still] or [Start Stream]
Resolution sets the image size in the browser.

V-Flip & H-Flip help us orient the image as desired depending on how we orient the board.

        The following are only available for stills at CIF or lower resolution

Face Detection allows saving a face image for recognition.

    Enroll Face

Face Recognition activates recognizing saved faces (if Resolution is not set too high).

If you click the Start Stream button, a continually updated camera image displays in the browser & image info scrolls in the Serial Monitor.

With another sketch (provided later), I could add a sitemap entry for openHAB to display the image stream in the User Interface, but that did not seem to work with this firmware.

    Next, Using More Features of the TTGO White Board

[papa]

        Using More Features of the TTGO White Board

The first sketch I tried was esp32-camera-bme280.ino which LILYGO github provided seemingly tailored for the TTGO White Board (including its OLED screen). I had trouble with using the sketch. For one thing I discovered my board had the SMD pads for the BME280 sensor chip, but not the chip itself that the sketch included.  Even with BME280 code disabled in the sketch I still had problems.

It was helpful to adapt code from esp32-camera-bme280.ino into other sketches like CameraWebServer.ino. This enables using more features of the TTGO White Board (like the OLED screen).

    In the Arduino IDE's Tools/Manage Libraries, install OneButton library, FreeRTOS, Adafruit SSD1306 library & "ESP8266 and ESP32 Oled Driver for SSD1306 display" (NOT the Adafruit SSD1306 library which works differently).

           In the camera-pins.h tab just above "#else", add these TTGO White Board configs to those you added at this post above:

// ## added from www.banggood.com/LILYGO-TTGO-T-Camera-ESP32-WROVER-with-PSRAM-Camera-vModule-OV2640-Camera-0_96-Inch-OLED-p-1418433.html

#define I2C_SDA 21
#define I2C_SCL 22 // ## & esp32_camera_bme280_org.ino
#define AS312_PIN 33 // ## " PIR
#define BUTTON_1 34 // ## " Right button

        Next, Using More TTGO White Features, Continued

   

[papa]

        Using More Features of the TTGO White Board, Continued

Now with CameraWebServer002.ino & the other 3 files open in Arduino IDE, save as CameraWebServer006.

     CameraWebServer006pub.ino (8.31 KB)  << Download this file. For Windows, open it in a text editor like Notepad++. For Linux, open in a text editor like Leafpad.

Using the text editor's Find, search for while (1). Be sure the second instance is disabled (//) & looks like this:

        // while (1); // wait for reset button pushed, causes error

Select & copy all lines of CameraWebServer006pub.ino.

Go back to CameraWebServer006 in Arduino IDE, delete all its lines & paste in all lines you copied from CameraWebServer006pub.ino & save.

     Edit the resulting CameraWebServer006 sketch to have your WiFi's SSID & password. Save.

In summary, a sketch folder should contain CameraWebServer006.ino (now with the contents of CameraWebServer006pub.ino & your WiFi info) AND the other 3 files (including an augmented camera_pins.h)

The firmware resulting from these edited files should now be almost ready. Tweak the configs in Arduino IDE as in this post.  Then upload CameraWebServer006.ino to the TTGO White Board.

        Next, Using the Augmented Firmware

[papa]

                Using the  CameraWebServer006 Firmware
                    Augmented via the Edits of
          CameraWebServer.ino & camera_pins.h

After editing & uploading from the above posts (& the TTGO White Board is connected to USB), press its reset button (RST) once or twice. On the Serial Monitor, you should see results similar to the last half of this post.

On the OLED screen, you should also see "TTGO Camera" ""WiFi connected"  Then your OLED should alternate two screen shots 1) "[your board's IP address]" 2) A blank screen available for later. When the board detects motion, screen shot 1) will also display "PIR Trigger"

You may briefly see the error message "Camera init Fail" Disregard that error message if you then see the messages in the previous paragraph.

Follow this post on using this new firmware in a browser tab.

Via this new firmware, pressing the board's other button (labeled IO34) will vertically flip the browser tab's video stream or next still image.

        Next, Firmware to Display Camera Output in OpenHAB's UI

[papa]

        Firmware to Display Camera Output in OpenHAB's UI

The third camera board sketch I worked with came from a helpful site called Random Nerd Tutorial

The sketch in the posting linked just above was set up for another ESP32 camera board with pin assignment lists for some other cameras. Unlike the last example with 4 files, this Random Nerd sketch is only one file that contains the board pin assignments.

    RN_VideoStreamttgo04_pub.ino (10.66 KB)  << Forum members, download my 04 version of this sketch, which changes some things:

    I added: pin assignments & instructions for the TTGO White Board & some others (not all tested)

                    some more instruction

                lowering of the displayed frame size (resolution) for higher initial frame rate

        setting vertical flip so the image is right side up when the TTGO White Board's USB connector is at the bottom

So far my version of this sketch uses only the TTGO White Board's camera, not other features (PIR, push button, OLED screen).

        Next, Customizing the Firmware Sketch

[papa]

        Customizing RN_VideoStreamttgo04_pub.ino

                You will need to customize the sketch:

Edit in your WiFi credentials.

As is, the sketch defines CAMERA_MODEL_TTGO_WROVER_B to use the TTGO White Board. For another board, comment (//) TTGO White & uncomment another board name. Match your board's pin assignments to one on the list or add a new define naming your board & a list of your board's pin assignments.

        Possible customizing:

Not too far from the sketch's end is a line setting the persisting display resolution which you could make larger or smaller.

            config.frame_size = FRAMESIZE_VGA;

Instead of VGA, you could use other resolution names which are listed

    in esp_camera.h:  Size of the output image: FRAMESIZE_ + QVGA|CIF|VGA|SVGA|XGA|SXGA|UXGA

    A little later is a section starting with "#if defined(CAMERA_MODEL_TTGO_WROVER_B)"

This vertically flips the displayed image for my TTGO White Board. If you need this for another board, change the "#if defined..." to your camera model name or just delete "#if defined..." at the section's start & the "#endif" at the section's end.

    Next, Uploading RN_VideoStreamttgo04_pub.ino

[papa]

        Uploading RN_VideoStreamttgo04_pub.ino

Before uploading, make sure the Arduino IDE configs are like those in this post.
 
I adapted & augmented the uploading/flashing instruction from the original sketch:

IMPORTANT!!! For some boards (e.g. "AI Thinker ESP32-CAM") to upload a sketch:
    - GPIO 0 must be connected to GND
    - Then press the on-board RESET button to put board in flashing mode

IMPORTANT!!! To flash the TTGO White Board defined below as CAMERA_MODEL_TTGO_ESP32_White
    - Arduino IDE: Select Board "ESP32 Dev Module" flash freq 40mhz; partition: huge app; psram enabled
    - Do NOT try to GND GPIO 0 or hold down RESET button. Flashing works without this.

As the first instruction indicates, to flash ESPxx boards, we've often expected to ground a pin &/or mash a push button. That may be required for your board. As the second instruction indicates, the TTGO White Board attached to a computer's USB is ready to flash as is. In fact, holding down the TTGO White Board's reset button seems to put the board in sleep mode.

    Next, Initial Results with RN_VideoStreamttgo04_pub.ino

[papa]

        Initial Results with RN_VideoStreamttgo04_pub.ino

So far, this sketch does not use the OLED screen on the TTGO White Board. Upload the sketch. Then open the Arduino IDE's Serial Monitor at 115200 baud. Maybe push the board's reset button (RST) once or twice.

On the Serial Monitor, we should see something like:

WiFi connected
Camera Stream Ready! Go to: http://[Camera Board IP Address]

Open a browser tab, enter the IP displayed on the Serial Monitor, & see the video stream in the browser.

Unlike with the other two firmware sketches used above, the browser will not display a console to change settings for the camera board. The perhaps inconvenient news is that (at least for now) to change camera settings, one needs to change & add those settings in the sketch & upload again. The good news follows in the next post.

        Note: More Serial Monitor details can be obtained by UNcommenting the sketch lines (may affect the stream):

// Serial.printf("MJPG: %uB\n",(uint32_t)(_jpg_buf_len));

//Serial.printf("Starting web server on port: '%d'\n", config.server_port);

After you make note of the camera board's IP address, you may close the Serial Monitor.

        Next, Display Camera Output in OpenHAB's UI

[papa]

        Display Camera Output in OpenHAB's Basic UI
        Using RN_VideoStreamttgo04_pub Firmware

From following the post just above, note the camera board's IP address displayed on the Arduino IDE's Serial Monitor.

Caution: If you have a browser tab open hosting the video stream, CLOSE it now. (Otherwise the following won't work.)

Have your openHAB User Interface (UI) open in a browser tab.

        In an openHAB .sitemap file (Linux /etc/openhab/sitemaps), add this line, including your board's IP address where indicated (& save the file):

Image url="http://camera board's IP Address from above" refresh=1000

In openHAB 3 go to localhost:8080. Log in if needed. At the upper right, click on the other apps icon. Then click on Basic UI. Then click on the .sitemap file that contains the line that starts with Image url=...

With occasional temporary blanks, the openHAB UI should display the video stream, refreshing it every 1000 milliseconds (1 second).

        Next, Display Camera Output in OpenHAB's HABPanel

[papa]

        Display Camera Output in OpenHAB's HABPanel

Caution: If you have camera output open in a browser tab or sitemap, end that display so it will not interfere with the following.

See this post on getting started on using HABPanel. Follow instructions in that post through "click on the [Add Widget] button."

 In [Add Widget], choose Image. Give the widget a name, like dbCamera. In the URL box, give the IP of your camera, in the pattern xxx.xxx.xxx.xxx        At the bottom right, click Save.

At the top left, click [Save] & [Run].

You may need to refresh your HABPanel browser tab & reboot the camera board for you to see camera output in the HABPanel widget.

        Next, Using a Camera Board Disconnected from Your Computer

[papa]

        Using a Camera Board Disconnected from Your Computer

Disconnect the Camera Board from your computer's USB. Provide a power source via the USB cable or the provided LiPo battery cable & connector. Wherever the camera board is, if it's powered & can connect to your WiFi, it should display video stream on openHAB's UI.

(To get the camera board connected to WiFi & displaying in the sitemap or browser tab, you may need to push the board's reset button once or twice &/or refresh the browser tab.)

In the following from what I can learn about the sketches' code used above, I'll experimented in mixing their features to create firmware.  Someday, I may adapt battery code from here or here.

        Next, Progress Made with the Camera Board

[papa]

        Progress Made with the Camera Board

Adapting from the above experience, I created a firmware sketch that accesses the TTGO White Board's RSSI, firmware version, IP address, PIR sensor, & push button. Also via MQTT, the firmware sketch publishes messages for those functions to openHAB. I configured openHAB Things, Channels, items, sitemap, & HABPanel to collect the data from this camera node.

Beside the 3 above ESP32 camera sketches, my sketch also adapted code from computourist's ESP_SONOFF_12.ino.

I have the TTGO White Board's camera streaming server working (in a browser tab or openHAB's Basic UI) with the above. I did more clean up & reorganizing in the sketch.

I experimented more with the OLED code from the above sketches & others. Through that, I was able to adapt workable OLED code into the sketch.

The resulting firmware will be offered below.

Still to come: Using gained capabilities toward making a Video Doorbell Node.

        Next, Preparing to Use an ESP32 [Camera] Board

            with openHAB

[papa]

                    Preparing to Use
        an ESP32 [Camera] Board with openHAB

Needed, things covered above: an ESP32 Board, means to upload to the board (USB connector or FTDI device), Arduino IDE, the ESP32 Add-on for Arduino IDE (see this post), & your board's pin assignments (for hints, see this post) if they are not listed in a sketch already.

Also needed: A MQTT Broker & openHAB 3.x installed; MQTT Binding also installed into openHAB & MQTT Things configured.

        See here for installing openHAB 3.x.

Start here for installing MQTT Binding & start here for Mosquitto MQTT Broker.

See here for some troubleshooting for MQTT.     See here about Things.

Also if needed, add some code libraries, using Arduino IDE's Tools/Library Manager:

            ESP8266 and ESP32 OLED driver for SSD1306 displays   See here.
                        PubSubClient

If you have not yet, perhaps save a backup copy of your board's original firmware or other firmware presently on your board. For Windows, see this post. For Linux, see this post.

    Next, a Sketch of Firmware for an ESP32 Board