ESP32 Smart Doorbell (orig)

[papa]

        Create a PIR Channel with a Transformation

If you have not already, go to PaperUI/Add-ons/Transformations/Map Transformation & click "Install"

        Mostly like previous posts, create a channel for the Node's PIR states.

    Channel type: Text value

Channel id: Node30Channel41

     Label: dBell_30_PIR

MQTT state topic: home/esp_gw/nb/node30/dev41

         [Click SHOW MORE]

    Transform incoming values     MAP:pir.map                [Click SAVE]

In openHAB's /conf/transform folder, create the text file:  pir.map

        [with the Contents]

ON=MOTION

OFF=NO MOTION      [save the file]

Note: The Node sends "ON" or "OFF".  The channel transforms that to "MOTION" or "NO MOTION"

    Next, Create Items for Channel Data

[papa]

        Create Items for Channel Data

In the following, I added to an existing Main-Fl.items file & created new items for Node30. Customize to your situation.

    Go to openHAB's /conf/items folder & in a new or existing .items file, create the following entries:

String Node30Rssi "DB_30_RSSI (db)[%s]" <network> (Nodes) { channel="mqtt:embedded-mqtt-broker:topic:mything:Node30Channel02" }

String Node30Version "DB_30_Vrsn (db)[%s]" { channel="mqtt:embedded-mqtt-broker:topic:mything:Node30Channel03" }

String Node30IP "DB_30_IP [%s]" <network> { channel="mqtt:embedded-mqtt-broker:topic:mything:Node30Channel10" }

String Node30BTN "DB_30_BTN [%s]" <wallswitch> { channel="mqtt:embedded-mqtt-broker:topic:mything:Node30Channel40" }

String Node30PIR "DB_30_PIR [%s]" <motion> { channel="mqtt:embedded-mqtt-broker:topic:mything:Node30Channel41" }

        Next, Linking Items to Their Channels

[papa]

        Linking Items to Their Channels

In the following, I instruct linking the PIR item to the PIR channel.  Do the same for the RSSI, Version, IP, & BTN items/channels/items.

        In PaperUI/Configuration/Things, scroll to Node30db (or your comparable Thing)

scroll to the Channel dBell_30_PIR & click on ( 0 ), its circle in a blue disc.

Complete the Link channel fields:
    Default
Select item to link from the dropdown list: DB_30_PIR (Node30PIR)      [click "Link"]

        Next, Add Linked Items to a Sitemap

[papa]

        Add Linked Items to a Sitemap

       & Display in a User Interface (UI)

        Go to openHAB's \conf\sitemaps folder &

add the following entries (or your comparables) to a new or existing .sitemap file (I used My.sitemap):

Image url="http://camera board's IP Address" refresh=1000

    [ Feb. 23, 2020 ^^ This easily worked with openHAB 2.5.1 & before.

        OH2.5.2 seems to resist changing sitemaps.  See this post. ]

Text item=Node30Rssi
Text item=Node30PIR
Text item=Node30IP
Text item=Node30BTN

Make sure your Node is powered & connected to WiFi/MQTT.  In a browser tab,enter http://127.0.0.1:8080/basicui/app?sitemap=My 

    Your User Interface should display the current output from Node30.

            See the board's camera streaming.

       Push the board's button & wave your hand over the board's PIR sensor & see states change on the UI.

Later, you might delete some of these entries from showing in the sitemap, but (through rules), do things based on the items' states.

    Next, An Enclosure Case for the TTGO White Camera Board

[papa]

        An Enclosure Case for the TTGO White Camera Board

The following could help protect & beautify the camera board node.

15:18 DrZzs discusses enclosures

        Also see in comments below the video:

Thingiverse Model

OnShape Model

Plastic Soap Box

DrZzs suggests waterproofing the soap box with silicone.  Maybe hot glue would work also.  I have a Plastic Soap Box, but have not yet tried making it into a case.

Update:  See more about the Thingiverse Model in this post below.

        To Be Continued

[papa]

        Update, Some Success, Some Glitches

On its own, the ESP32 Camera node firmware documented above has worked moderately well in combining its possibilities of camera, PIR motion detector, push button, & OLED screen. The screen shows the node's IP address & a message when motion is detected. The camera reliably streams to a browser tab or to openHAB's User Interface.

So far at least, I was not able to send a screen message from openHAB to the node's OLED. Then I experimented more with the node displaying messages on its OLED. I discovered that the node updated openHAB items for a time after its startup, but then updates stopped. The node still reliably shows messages on the OLED. The camera still streams to openHAB (& so the node is still connected to WiFi), but that's through http, not MQTT.  I experimented with a variety of ways to display text on the OLED, but all suffered this problem.

Perhaps after a while, the OLED code interferes with the MQTT code. I wonder if MQTT gets disconnected. If so, I also wonder if the MQTT could be reconnected.

Update: I'm trying some reconnect MQTT code from a PubSubClient example sketch.

[papa]

        Glitch Fixed

As mentioned in the last post, I added code from a PubSubClient example sketch for reconnecting MQTT if disconnected. So far for 24 hours, the node's MQTT has never stopped communicating with the openHAB User Interface.  Previously, by 60 minutes, the node would have stopped communicating with openHAB.

See this post above for updated firmware programming that stays reliably connected to the MQTT message service.

From there continue to the needed customizing & openHAB configuration.

[papa]

OpenHAB Can Now Send Messages to
   the ESP32 Node's OLED Screen
      Firmware Customization

As of firmware version ...02d09d (see this post to download), the ESP32 Camera node can receive messages from openHAB for its OLED screen.

To activate this ability, UNcomment & activate #define OLED_TXT in the firmware sketch.

        Next, Needed openHAB Configuration

[papa]

        OpenHAB Can Now Send Messages to
              an ESP32 Node's OLED Screen
               Configure OpenHAB Channel

First we add a channel 72 (command) to enable the node to receive messages for its OLED screen.

I assume Node30. Adapt the following if your node ID is a different number.

Go to PaperUI/Configuration/Things/Node30db

To add a channel to the Node30db Thing, click the ( + ) to the right of "Channels"
        Complete the Add channel fields:
            (Customize to your situation)

Channel type: Text value in the drop down list
    Channel id: Node30Channel72
        Label: dBell_30_OledTxt

MQTT command topic: home/esp_gw/sb/node30/dev72

    ^^ Corrected, March 23, 2020

        Check your work & Click "SAVE"

            Next, Configure OpenHAB Items

[papa]

        OpenHAB Can Now Send Messages to
            an ESP32 Node's OLED Screen
                Configure OpenHAB Items & Link Them

In the following, I added to an existing Main-Fl.items file & created new items for Node30. Customize to your situation.

Go to openHAB's /conf/items folder & in a new or existing .items file, create the following entries:

    String OLEDprint30 { channel="mqtt:embedded-mqtt-broker:topic:mything:Node30Channel72" }

    Switch SendOLEDmsg1 "0:push btn"
    Switch SendOLEDmsg2 "0:coming"
    Switch SendOLEDmsg3 "2:lv pkg"
    Switch SendOLEDmsg4 "2:no thx"

       Link one item to its channel:  (Updated Feb. 19, 2020)

Go to PaperUI/Configuration/Things/Node30db & click on Node30db's ( O )

     Fill the Link Channel fields.  Default

        Scroll thru the items until you can select the item OLEDprint30 & click Link

            Next, Configure OpenHAB Rules

[papa]

        OpenHAB Can Now Send Messages to
            an ESP32 Node's OLED Screen
                Configure OpenHAB Rules

In the following, I added to an existing Main-Fl.rules file & created new rules for Node30. Customize to your situation.

rule "write_OLED_Msg1"
    when
        Item SendOLEDmsg1 received update
    then
        if (SendOLEDmsg1.state==ON) {
            OLEDprint30.sendCommand("0:Push button")
            OLEDprint30.sendCommand("1:for attention.")
                Thread::sleep(250)
            SendOLEDmsg1.sendCommand(OFF)
        }
end

rule "write_OLED_Msg2"
    when
        Item SendOLEDmsg2 received update
    then
        if (SendOLEDmsg2.state==ON) {
            OLEDprint30.sendCommand("0:I'm on my way")
            OLEDprint30.sendCommand("1:to answer door")
                Thread::sleep(250)
            SendOLEDmsg2.sendCommand(OFF)
        }
end

rule "write_OLED_Msg3"
    when
        Item SendOLEDmsg3 received update
    then
        if (SendOLEDmsg3.state==ON) {
            OLEDprint30.sendCommand("2:Leave packages")
            OLEDprint30.sendCommand("3:left of door")
                Thread::sleep(250)
            SendOLEDmsg3.sendCommand(OFF)
        }
end

rule "write_OLED_Msg4"
    when
        Item SendOLEDmsg4 received update
    then
        if (SendOLEDmsg4.state==ON) {
            OLEDprint30.sendCommand("2:Not interested")
            OLEDprint30.sendCommand("3:No thank you")
                Thread::sleep(250)
            SendOLEDmsg4.sendCommand(OFF)
        }
end

        Next, Configure OpenHAB Sitemap

[papa]

OpenHAB Can Now Send Messages to
        an ESP32 Node's OLED Screen
         Configure OpenHAB Sitemap

In the following, I added to an existing xxxx.sitemap file & created new entries for Node30. Customize to your situation.

    Switch item=SendOLEDmsg1
    Switch item=SendOLEDmsg2
    Switch item=SendOLEDmsg3
    Switch item=SendOLEDmsg4

        Next, What to Expect, How This Works

[papa]

        OpenHAB Can Now Send Messages to
            an ESP32 Node's OLED Screen
            What to Expect, How This Works

Since the previous firmware version, the node displays standard (video doorbell) messages on an OLED screen. In the firmware sketch, search for msg01. You should find a list of Strings whose names start with msg.

Msg01 thru msg04 display on lines 0 thru 3 of the OLED's first screen. These messages are "TTGO Camera" & the node's IP address (for a while before blanking). When the PIR motion detector triggers, "Smile, you are on camera" appears on the bottom two lines of the first screen.

Msg05 thru msg08 display on the OLED's second screen. Via the openHAB UI, clicking switches with names starting with SendOLEDmsg will display the two lines associated with the switch. The label beside the UI switch hints at the message content & which screen line the message starts on.

You can customize second the screen's messages in the rules created above. The number before the colon sets the screen line (zero being the top line & 3 is the last).

Be aware that the node firmware will trim each screen line (after the colon) to the maximum length of  14 characters for the present font size.

[papa]

            Display 4 Motion Cameras on OpenHAB's User Interface
                                (Resizable via a Slider)

   Camera_feed_5_pub.html (2.1 KB)   << Members, download this .html file. Near the end, the 4 camera IP address are all the same. Edit the IP addresses to be the IP addresses of your one to four Motion-controlled cameras. (This site provided the .html file.)

On the computer hosting Motion, save the edited .html file in the conf/html folder nested in OpenHAB 2's config folder (/etc/openhab2/html for an apt-get install of OpenHAB2). You may rename the file, but use the new name consistently with the following instructions

        Continues ...

[papa]

            Display 4 Motion Cameras on OpenHAB's User Interface
                                (Resizable via a Slider)

This post above offered this sitemap entry which displays one camera feed:

Image url="http://camera board's IP Address" refresh=1000

To display up to 4 camera feeds, substitute this sitemap entry (which includes the edited .html file from the last post):

Webview url="http://127.0.0.1:8080/static/Camera_feed_5.html" icon="camera" height=20

Note: Apparently OpenHAB 2 knows that the /static folder is the /html folder nested in OpenHAB's configuration folder.

[papa]

        More on a 3D Printed Enclosure

        For the TTGO White Camera Board

I just obtained a 3D printed case based on Dr. Zzzs design posted above. The case works nicely to hold the board so everything works. It protects the board, but it's not waterproof. I can mount this on my mostly covered porch. Otherwise some other protection would be needed.

I'm unsure about one piece of Dr. Zzzs' design. It's wedge shaped with a screw hole. I assume it's for mounting the case.

Here's my thoughts: the wedge piece's longest side matches the case's width. Looks like the other 2 sides could fit in a corner (next to the outside of a door frame?) & a screw thru the hole (into wood door frame?) to hold the wedge in place. Gluing the case to the wedge's longest side would cover the ability to remove the case. But maybe Velcro could hold the case's back to the longest side of the wedge & allow removing the case (especially if an opening in the Velcro gives access to the screw head). Other thoughts?

[rubenkona]

Very very very very thanks to this post!
I have bought one of this Esp32 Cam on Aliexpress to do some DIY in home, adding it to other my OpenHAB stuff, I have several Esp32 and Esp8266 all of them with EspHome firmwares. So I have receiver this "seems white LilyGo", and directly burned with EspHome with all stuff of cam. First of all, cam seems to work but not connecting to anything. But after first reboot, cam not longer works. I test some EspHome stuff, device works but if I put cam, doesn't work. After some searching, I found that Cams with EspHome only works with HomeAssistant api, so I can't use it with OH.
I have found this post searching a lot, because all stuff for Esp32 Cams with arduino aren't very clear. I have followed all your steps, very great, thanks, camera works from first post, also all other stuff like PIR, OLED screen, etc. I am very agree to all this.
I am at last steps, only have to integrat with OH, the easy part!, but MQTT messages are working, same as camera capturing. So I am very happy!, and very thankful.
I only have to investigate a little to fix an IP instead of use a DHCP assigned one, not a big problem.
And only a little question? Have you tested the face detection and/or recognition? First server as samples from LilyGo seems to work.

Thanks again!
Ruben
PD: As my board seems the same as yours, my pin asignaments corresponds to CAMERA_MODEL_ESP_EYE

[papa]

rubenkona, I am glad that this thread proved useful for you.

It has been a while since I've worked on the board's firmware. When I first used the board with its original firmware, I believe I tried the face detection/recognition a little. I believe it worked sometimes & sometimes not & that others' reviewed indicated the same results.

[rubenkona]

Thanks a lot again, I have now it working, but I have some issues that I want to ask you before digging on source code, because maybe is more easy to ask:

- I can't send messages to the OLED screen. The program is compiled and send it to LilyGo with the ENABLE_SSD1306 defined. I am not still using OpenHAB for this (all itrms are working ok on OH), because I like to test before. I am using MQTT Explorer so I can see all messages on the MQTT Mosquito server. I have tried sending with topic home/esp_gw/sb/node30/dev72 and topic home/esp_gw/nb/node30/dev72, but none of them prints anything on screen apart from original message. I have also tried messages with "1:xxx" and "1,xxx" (as you mention "colon" in some explanation). On MQTT I see a topic send by device with topic "home/esp_gw/nb/node30/dev72" that always have "text received", if I send something to this topic, change for some seconds, but again turns to "text received", nothing apears on screen.  

- On the other hand, if I point my browser to the device IP, I am able to see the camera, but I can't see it on sitemap directly with Image tag. I have other IP cameras that can see without any problem, and I am also able to see it if I put his IP on your 4 cameras sample html. But not directly on OpenHAB Image tag. Very extrange. I am also surprised, because the image is not really a "still image", maybe the problem is that is some mpeg stream? It works for you with Image tag on sitemap?

In any case, thanks very much, I can dig on the Esp32 code to resolve the issues, but probably asking first is better. 

Ruben

[papa]

rubenkona, I'm glad you're making progress on using the camera board. My time is limited today so I'll give some limited responses.

It's been a while since I looked at the source code for the camera board so at present, your eyes may see more than mine. Let us know if you find ways to improve.

You wrote: "On the other hand, if I point my browser to the device IP, I am able to see the camera, but I can't see it on sitemap directly with Image tag. I have other IP cameras that can see without any problem, and I am also able to see it if I put his IP on your 4 cameras sample html. But not directly on OpenHAB Image tag. Very extrange. I am also surprised, because the image is not really a "still image", maybe the problem is that is some mpeg stream? It works for you with Image tag on sitemap?"

papa: Yes, it NOW, USUALLY works for me with IMAGE tag on a sitemap (or HABpanel). From what I remember, it did NOT work at first. I did things like this & eventually it mostly worked: Stopping & restarting openHAB, refreshing the browser or app holding the sitemap/HABpanel, stopping & restarting the camera board. As I mention in the thread above, the image/stream might only be accessible on one computer. However, lately, I've had some success accessing the image/stream (without refreshing the browser or restarting the camera board.

Regarding displaying messages on the SSD: I believe the same refreshing & restarting mentioned just above might have helped with this issue also.

One more thing that might help with these two issues: clearing the openHAB cache. See this post. (Though a caution: clearing the cache sometimes seems to introduce other issues.)

I have many features in my openHAB setup. My experience is that the more we set up in openHAB, the more resistant it becomes to adding new things or making other changes. Thus the need for refreshing/restarting/etc. This might also apply to your openHAB setup.

[rubenkona]

Hi, well, I want to tell you my progress with this. 

- Image: Strange... I am still not able to show it directly on Image tag on OpenHAB. I don't know, but I was trying to see the packets with Wireshark without succes. In any case, it works correctly if I put it on html like your sample using webview. But, I want to send a snapshot with Telegram, and it doesn't work. I will continue investigating this issue. 

- MQTT: Finally I was able do send comands to screen... very strange, because in theory, I haven't changed anything in the code. Only moved to Visual Studio Code from Arduino IDE, because I was trying to debug. Well, now I am working and sending the .ino from VS Code, and it is working (probably doesn't related to it!). 

- Recomendation to you on OpenHAB: 
instead of using this on rule (because using "Sleeps" on rules is not recommended on documentation):

if (SendOLEDmsg1.state==ON) {

               OLEDprint30.sendCommand("0:Push button")

               OLEDprint30.sendCommand("1:for attention.")

                   Thread::sleep(250)

               SendOLEDmsg1.sendCommand(OFF)

}

Is better to declare the item as "expire": 

Switch SendOLEDmsg1 "Ara vaig a la porta" <screen> { expire="30s,command=OFF" }
With this, the item actually "turns off" after a defined time. With OH 2.X you will need "Expire binding", with OH 3.X, it is supported directly. I have also added this rule to clean output after de "expire" time: 

rule "Neteja OLED"

   when

           Item SendOLEDmsg4 changed to OFF or

           Item SendOLEDmsg3 changed to OFF or

           Item SendOLEDmsg2 changed to OFF or

           Item SendOLEDmsg1 changed to OFF

   then

       it_lilygoText.sendCommand("0:                 ")

       it_lilygoText.sendCommand("1:                 ")

end

Edited: Do you know if could be done something with the Mic? I don't found information about it. Thanks a lot. 
Edited again: This is so strange!, if I start monitoring the Esp32 via serial monitor, sending commands to OLED with MQTT works... if I start it without serial monitoring, sending text to screen doesn't work. It is like when no serial monitor, it hasn't subcribed to the topic (but publishing from Esp32 is still working in both cases). So crazy 🤦‍♂️

[papa]

rubenkona, Thanks for reporting back on your progress. You seem to confirm my experience that openHAB's configuration for the camera board works better over time.

I'll take a look at using expire instead of sleep. I have used expire for other things.

rubenkona: "Do you know if could be done something with the Mic?"

papa: my camera board does not have a mic so I can only guess. A camera board mic might be used to signal whether sound (of a certain level) is present or not. One could code the board or openHAB to take action based on that signal. From board labels or docs, learn what pin reports that signal. Your board's mic might reflect my experience with mic/sound sensors, namely sound & the mic signal fluctuates in a way that makes taking action more challenging. Adapting this thread might give you a start on an approach to compensating for fluctuations.  Starting here I document using a mic sensor to monitor a pump's on & off.

[rubenkona]

Mar 24, 2021 14:17:29 GMT papa said:

rubenkona, Thanks for reporting back on your progress. You seem to confirm my experience that openHAB's configuration for the camera board works better over time.

I'll take a look at using expire instead of sleep. I have used expire for other things.

rubenkona: "Do you know if could be done something with the Mic?"

papa: my camera board does not have a mic so I can only guess. A camera board mic might be used to signal whether sound (of a certain level) is present or not. One could code the board or openHAB to take action based on that signal. From board labels or docs, learn what pin reports that signal. Your board's mic might reflect my experience with mic/sound sensors, namely sound & the mic signal fluctuates in a way that makes taking action more challenging. Adapting this thread might give you a start on an approach to compensating for fluctuations.  Starting here I document using a mic sensor to monitor a pump's on & off.

Thanks a lot, I try to search more information about the Mic on the links you provide me.

- On the other hand, my progress:

* I have resolved the issue with MQTT receiving commands. The problem is that the "reconnect" function doesn't subscribe to the topic, so it doesn't receive the topics when it has a reconnection to Mosquitto. Arranged.

* I have put the complete WebCamServer sample to your last code, so I can access to the full camera front-end to configure it, including face detection and recognition (not tested yet), but from here can change resolution and other camera parameters. Also have a URL "/capture" to take snapshots to OpenHAB (not tested yet , not many time).

Thanks for all information.
     Ruben

[papa]

You are welcome, rubenkona. Good to hear more on your findings & progress.

I encourage you to return here to document what you have added & improved with the camera board project. That way you will benefit others. AND you will give yourself a good record of what you have done in this project so you can duplicate & build on it later(especially when your memory of the project is not so strong as now).

Documentation takes time now, but saves much time & frustration later.

[rubenkona]

OMG, I have read this so much time after :-(


I have uploaded the main .ino code and the existing .h with camera definitions. 
ESP32_pins.h (11.1 KB)
Esp32CameraOH_ToConf.ino (44.51 KB)


Ruben

[papa]

Ruben,

Good to hear from you. I enjoyed reading back through our exchange of posts. I had forgotten about your recommendation to use expire. I was wondering if it was included in OH 3 (to which I moved recently) & you answered that question. I want to use expire where you suggested & in other code.

I especially admire how you came back to here document your improvements to the code. I want to get the opportunity soon to try what you've done.

I'm sure others will benefit from what you have offered.  Many thanks.