From 9e0dc5ecb71660b0f06b6b1527dc6a58923d6135 Mon Sep 17 00:00:00 2001
From: "github-actions[bot]" <github-actions[bot]@users.noreply.github.com>
Date: Sun, 22 Jan 2023 18:34:11 +0000
Subject: [PATCH] Deployed d88399d with MkDocs version: 1.4.2

---
 Installation/index.html  |  18 ++++++++++--------
 search/search_index.json |   2 +-
 sitemap.xml.gz           | Bin 660 -> 660 bytes
 3 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/Installation/index.html b/Installation/index.html
index df43bbc..0fd4bed 100644
--- a/Installation/index.html
+++ b/Installation/index.html
@@ -321,14 +321,16 @@ esptool write_flash 0x01000 bootloader.bin 0x08000 partitions.bin 0x10000 firmwa
 <h2 id="sd-card">SD-Card</h2>
 <p>The software expects a SD-Card prepared with certain directory and file structure in order to work properly.
 For the first setup take the <code>AI-on-the-edge-device__manual-setup__*.zip</code> from the <a href="https://github.com/jomjol/AI-on-the-edge-device/releases">Release</a> page, open the zip and extract the whole content of the in the setup file included <code>sd-card.zip</code> onto your SD-Card direclty to the root folder.</p>
-<p>SD-Card root should look like this:
-- config
-- demo
-- firmware
-- html
-- img_tmp
-- log
-- wlan.ini</p>
+<p>SD-Card root should look like this:</p>
+<ul>
+<li>config</li>
+<li>demo</li>
+<li>firmware</li>
+<li>html</li>
+<li>img_tmp</li>
+<li>log</li>
+<li>wlan.ini</li>
+</ul>
 <p>This initial setup needs to be done only once as further updates of the software are possible with an Over-The-Air update mechanismn.</p>
 <h3 id="notes">Notes</h3>
 <ul>
diff --git a/search/search_index.json b/search/search_index.json
index d5139d5..1f584a8 100644
--- a/search/search_index.json
+++ b/search/search_index.json
@@ -1 +1 @@
-{"config":{"indexing":"full","lang":["en"],"min_search_length":3,"prebuild_index":false,"separator":"[\\s\\-]+"},"docs":[{"location":"","text":"Welcome Welcome to the AI-on-the-edge-device project! This is the documentation. For the source code, please head to github.com/jomjol/AI-on-the-edge-device . Artificial intelligence based systems have been established in our every days live. Just think of speech or image recognition. Most of the systems relay on either powerful processors or a direct connection to the cloud for doing the calculations up there. With the increasing power of modern processors the AI systems are coming closer to the end user - which is usually called edge computing . Here this edge computing is brought into a practical oriented example, where a AI network is implemented on a ESP32 device so: AI on the edge . Key features Tensorflow Lite (TFlite) integration - including easy to use wrapper Inline Image processing (feature detection, alignment, ROI extraction) Small and cheap device (3x4.5x2 cm\u00b3, < 10 EUR) camera and illumination integrated Web surface to administrate and control OTA-Interface to update directly through the web interface Full integration into Homeassistant Support for Influx DB 1 MQTT REST API Idea Hardware Web interface Configuration Interface Have fun in studying the new possibilities and ideas This is about image recognition and digitalization, done totally on a cheap ESP32 board using artificial intelligence in form of convolutional neural networks (CNN). Everything, from image capture (OV2640), image preprocessing (auto alignment, ROI identification) all the way down to the image recognition (CNN structure) and result plausibility is done on a cheap 10 EUR device. This all is integrated in an easy to do setup and use environment, taking care for all the background processing and handling, including regular job scheduler. The user interface is an integrated web server, that can be easily adjusted an offers the data as an API in different options. The task to be demonstrated here is a automated readout of an analog water meter. The water consumption is to be recorded within a house automatization and the water meter is totally analog without any electronic interface. Therefore the task is solved by taking regularly an image of the water meter and digitize the reading. There are two types of CNN implemented, a classification network for reading the digital numbers and a single output network for digitalize the analog pointers for the sub digit readings. This project is a evolution of the water-meter-system-complete , which uses ESP32-CAM just for taking the image and a 1GB-Docker image to run the neural networks backbone. Here everything is integrated in an ESP32-CAM module with 8MB of RAM and a SD-Card as data storage.","title":"Welcome"},{"location":"#welcome","text":"Welcome to the AI-on-the-edge-device project! This is the documentation. For the source code, please head to github.com/jomjol/AI-on-the-edge-device . Artificial intelligence based systems have been established in our every days live. Just think of speech or image recognition. Most of the systems relay on either powerful processors or a direct connection to the cloud for doing the calculations up there. With the increasing power of modern processors the AI systems are coming closer to the end user - which is usually called edge computing . Here this edge computing is brought into a practical oriented example, where a AI network is implemented on a ESP32 device so: AI on the edge .","title":"Welcome"},{"location":"#key-features","text":"Tensorflow Lite (TFlite) integration - including easy to use wrapper Inline Image processing (feature detection, alignment, ROI extraction) Small and cheap device (3x4.5x2 cm\u00b3, < 10 EUR) camera and illumination integrated Web surface to administrate and control OTA-Interface to update directly through the web interface Full integration into Homeassistant Support for Influx DB 1 MQTT REST API","title":"Key features"},{"location":"#idea","text":"","title":"Idea"},{"location":"#hardware","text":"","title":"Hardware"},{"location":"#web-interface","text":"","title":"Web interface"},{"location":"#configuration-interface","text":"Have fun in studying the new possibilities and ideas This is about image recognition and digitalization, done totally on a cheap ESP32 board using artificial intelligence in form of convolutional neural networks (CNN). Everything, from image capture (OV2640), image preprocessing (auto alignment, ROI identification) all the way down to the image recognition (CNN structure) and result plausibility is done on a cheap 10 EUR device. This all is integrated in an easy to do setup and use environment, taking care for all the background processing and handling, including regular job scheduler. The user interface is an integrated web server, that can be easily adjusted an offers the data as an API in different options. The task to be demonstrated here is a automated readout of an analog water meter. The water consumption is to be recorded within a house automatization and the water meter is totally analog without any electronic interface. Therefore the task is solved by taking regularly an image of the water meter and digitize the reading. There are two types of CNN implemented, a classification network for reading the digital numbers and a single output network for digitalize the analog pointers for the sub digit readings. This project is a evolution of the water-meter-system-complete , which uses ESP32-CAM just for taking the image and a 1GB-Docker image to run the neural networks backbone. Here everything is integrated in an ESP32-CAM module with 8MB of RAM and a SD-Card as data storage.","title":"Configuration Interface"},{"location":"Additional-Information/","text":"The following links point to additional information in other repos: Digits Training and using a neural network to readout the value of a digital counter Training the CNN neural network Analog Training and using a neural network to read out the value of an analog display Training the CNN neural network","title":"Additional Information"},{"location":"Additional-Information/#digits","text":"Training and using a neural network to readout the value of a digital counter Training the CNN neural network","title":"Digits"},{"location":"Additional-Information/#analog","text":"Training and using a neural network to read out the value of an analog display Training the CNN neural network","title":"Analog"},{"location":"Best-Practice/","text":"Best Practice This page shows some best practices: Camera Placement Move the Camera as close as possible(~4cm), this will help get rid of reflections. -> focus can be adjusted by turning the outer black ring of the camera. If the LED reflections are too strong, put tape over the LED to defuse the light Change the ImageSize to QVGA under \"Expert mode\" configuration when close enough, this will be faster and is often good enough for digital recognition. Reflections Try to get ride of the reflections by rotating the camera, so that the reflections are at positions, where no number is. By using the external LED option, you can place WS2812 LEDs freely away from the main axis. Users report, that a handy cover foil could also help Post-processing Filter out the Number \"9\", as \"3\" will often be misread for a \"9\" and void every number between 3 and 9 due to it being negative flow. Split the readings into two, while the decimal numbers might move to fast to be recognized, at least the slower moving part will produce a correct reading. -> keep in mind that the offset needs to be adjusted, a.e if you have a comma reading of \"3\", it needs to become \"0.3\". This can be done wherever the data ends up being sent, like home assistant using sensor templates. If you are using a low resolution and only digital mode, processing can often be done in <1 minute. Check the logs to confirm how fast it is and then set the interval accordingly under \"Expert mode\" in configuration, as the normal mode will lock you to 3+ minutes.","title":"Best Practice"},{"location":"Best-Practice/#best-practice","text":"This page shows some best practices:","title":"Best Practice"},{"location":"Best-Practice/#camera-placement","text":"Move the Camera as close as possible(~4cm), this will help get rid of reflections. -> focus can be adjusted by turning the outer black ring of the camera. If the LED reflections are too strong, put tape over the LED to defuse the light Change the ImageSize to QVGA under \"Expert mode\" configuration when close enough, this will be faster and is often good enough for digital recognition.","title":"Camera Placement"},{"location":"Best-Practice/#reflections","text":"Try to get ride of the reflections by rotating the camera, so that the reflections are at positions, where no number is. By using the external LED option, you can place WS2812 LEDs freely away from the main axis. Users report, that a handy cover foil could also help","title":"Reflections"},{"location":"Best-Practice/#post-processing","text":"Filter out the Number \"9\", as \"3\" will often be misread for a \"9\" and void every number between 3 and 9 due to it being negative flow. Split the readings into two, while the decimal numbers might move to fast to be recognized, at least the slower moving part will produce a correct reading. -> keep in mind that the offset needs to be adjusted, a.e if you have a comma reading of \"3\", it needs to become \"0.3\". This can be done wherever the data ends up being sent, like home assistant using sensor templates. If you are using a low resolution and only digital mode, processing can often be done in <1 minute. Check the logs to confirm how fast it is and then set the interval accordingly under \"Expert mode\" in configuration, as the normal mode will lock you to 3+ minutes.","title":"Post-processing"},{"location":"Build-Instructions/","text":"Build the Project See README.md","title":"Build the Project"},{"location":"Build-Instructions/#build-the-project","text":"See README.md","title":"Build the Project"},{"location":"Choosing-the-Model/","text":"Model Selection Warning This page overlaps Neural Network Types . They should be merged to one page! In the Graphical Configuration Page , you can choose different models depending on your needs. This wiki page tries to help you on which model to select. For more technical/deeper explanations have a look on Neural-Network-Types . Digit Models For digits on water meters, gas-meters or power meters you can select between two main types of models. dig-class11 This model can recognize full digits. All intermediate states shown a \"N\" for not a number. But in post process it uses older values to fill up the \"N\" values if possible. Main features well suited for LCD digits with the ExtendedResolution option is not supported. (Only in conjunction with ana-class100 / ana-cont) dig-class100 / dig-cont These models are used to get a continuous reading with intermediate states. To see what the models are doing, you can go to the Recognition page. Main features suitable for all digit displays. Advantage over dig-class11 that results continue to be calculated in the transition between digits. With the ExtendedResolution option, higher accuracy is possible by adding another digit. Look here for a list of digit images used for the training dig-class100 vs. dig-cont The difference is in the internal processing. Take the one that gives you the best results. Analog pointer models ana-class100 / ana-cont For pointers on water meters use the analog models. You can only choose between ana-class100 and ana-cont. Both do mainly the same. Main features for all analogue pointers, especially for water meters. With the ExtendedResolution option, higher accuracy is possible by adding another digit. Look here for a list of pointer images used for the training ana-class100 vs. ana-cont The difference is in the internal processing. Take the one that gives you the best results. Both models learn from the same data.","title":"Model Selection"},{"location":"Choosing-the-Model/#model-selection","text":"Warning This page overlaps Neural Network Types . They should be merged to one page! In the Graphical Configuration Page , you can choose different models depending on your needs. This wiki page tries to help you on which model to select. For more technical/deeper explanations have a look on Neural-Network-Types .","title":"Model Selection"},{"location":"Choosing-the-Model/#digit-models","text":"For digits on water meters, gas-meters or power meters you can select between two main types of models.","title":"Digit Models"},{"location":"Choosing-the-Model/#dig-class11","text":"This model can recognize full digits. All intermediate states shown a \"N\" for not a number. But in post process it uses older values to fill up the \"N\" values if possible.","title":"dig-class11"},{"location":"Choosing-the-Model/#main-features","text":"well suited for LCD digits with the ExtendedResolution option is not supported. (Only in conjunction with ana-class100 / ana-cont)","title":"Main features"},{"location":"Choosing-the-Model/#dig-class100-dig-cont","text":"These models are used to get a continuous reading with intermediate states. To see what the models are doing, you can go to the Recognition page.","title":"dig-class100 / dig-cont"},{"location":"Choosing-the-Model/#main-features_1","text":"suitable for all digit displays. Advantage over dig-class11 that results continue to be calculated in the transition between digits. With the ExtendedResolution option, higher accuracy is possible by adding another digit. Look here for a list of digit images used for the training","title":"Main features"},{"location":"Choosing-the-Model/#dig-class100-vs-dig-cont","text":"The difference is in the internal processing. Take the one that gives you the best results.","title":"dig-class100 vs. dig-cont"},{"location":"Choosing-the-Model/#analog-pointer-models","text":"","title":"Analog pointer models"},{"location":"Choosing-the-Model/#ana-class100-ana-cont","text":"For pointers on water meters use the analog models. You can only choose between ana-class100 and ana-cont. Both do mainly the same.","title":"ana-class100 / ana-cont"},{"location":"Choosing-the-Model/#main-features_2","text":"for all analogue pointers, especially for water meters. With the ExtendedResolution option, higher accuracy is possible by adding another digit. Look here for a list of pointer images used for the training","title":"Main features"},{"location":"Choosing-the-Model/#ana-class100-vs-ana-cont","text":"The difference is in the internal processing. Take the one that gives you the best results. Both models learn from the same data.","title":"ana-class100 vs. ana-cont"},{"location":"Configuration-Parameter-Details/","text":"Configuration Parameter Details [MakeImage] [MakeImage] LogImageLocation = /log/source WaitBeforeTakingPicture = 5 LogfileRetentionInDays = 15 Brightness = -2 ;Contrast = 0 ;Saturation = 0 ImageQuality = 5 ImageSize = VGA FixedExposure = true Parameter Meaning Options/Examples LogImageLocation location for storing a copy of the image WaitBeforeTakingPicture waiting time between switch on the flash-light and taking the image in seconds LogfileRetentionInDays Number of days, for which the log files should be stored 0 = keep forever Brightness Adjustment of the camera brightness (-2 ... 2) Contrast NOT IMPLEMENTED Saturation NOT IMPLEMENTED ImageQuality Input image jpg-compression quality 0 (best) to 100 (lowest) 5 = default ImageSize Input Image Size from Camera only VGA, QVGA FixedExposure If enabled, the exposure settings are fixed at the beginning and the waiting time after switching on the illumination will be skipped [Alignment] [Alignment] InitialRotate = 179 InitialMirror = false SearchFieldX = 20 SearchFieldY = 20 AlignmentAlgo = Default FlipImageSize = false /config/ref0.jpg 104 271 /config/ref1.jpg 442 142 Parameter Meaning Options/Examples InitialMirror Option for initially mirroring the image on the original x-axis InitialRotate Initial rotation of image before alignment in degree (1...359) FlipImageSize Changes the aspect ratio after the image rotation to avoid cropping of the rotated imaged /config/refx.jpg 98, 257 Link to reference image and corresponding target coordinates file link is relative to sd-card root SearchFieldX/Y Search field size in X/Y for finding the reference images [pixel] Here two reference images are needed. Therefore rotation and shifting can be compensated. As the alignment is one of the most computing time using part, the search field needs to be limited. The calculation time goes quadratic with the search field size. [Digits] [Digits] Model=/config/digits.tfl ModelInputSize 20, 32 LogImageLocation = /log/digit LogfileRetentionInDays = 2 number1.digit1 292 120 37 67 number1.digit2 340 120 37 67 number1.digit3 389 120 37 67 number2.digit1 292 180 37 67 number2.digit2 340 180 37 67 Parameter Meaning Options/Examples Model Link to the CNN-tflite file used for AI-image recognition ModelInputSize Image input size for the CNN-Network [pixel] needed to resize the ROI to the input size LogImageLocation storage location for the recognized images, including the CNN-results in the file name/location numberX.digitY ROI for the corresponding digit in the aligned image. More than one number can be specified. Therefore the name consists of a naming of the number ( numberX ) and the region of interest ( digitY ) - separated by . LogfileRetentionInDays Number of days, for which the log files should be stored 0 = keep forever LogImageLocation location for storing a copy of the image [Analog] [Analog] Model=/config/analog.tfl ModelInputSize 32, 32 LogImageLocation = /log/analog LogfileRetentionInDays = 2 number1.analog1, 433, 207, 99, 99 number1.analog2, 378, 313, 99, 99 number1.analog3, 280, 356, 99, 99 number1.analog4, 149, 313, 99, 99 number2.analog1, 280, 456, 99, 99 number2.analog2, 149, 413, 99, 99 Same as for [digit], here only for the analog pointers [PostProcessing] [PostProcessing] number1.DecimalShift = 0 number2.DecimalShift = -1 PreValueUse = true PreValueAgeStartup = 720 AllowNegativeRates = false number1.MaxRateValue = 0.1 number2.MaxRateValue = 0.1 ErrorMessage = true CheckDigitIncreaseConsistency = false Here the post processing and consistency check for the readout can be adjusted Parameter Meaning Options/Examples PreValueUse Use the previous value for consistency check and substitution for NaN (True / False) PreValueAgeStartup Max age of PreValue after a reboot (downtime) AllowNegativeRates Allow decrease of the readout value numberX.MaxRateValue Maximum chance rate from one to the next readout. This can be specified for each number individual. ErrorMessage Show error messages numberX.DecimalShift Shifting of the decimal separator from the default position between digital and analog. This can be specified for each number individual. DecimalShift = 2: 123.456 --> 12345.6 DecimalShift = -1: 123.456 --> 12.3456 CheckDigitIncreaseConsistency This parameter controls, if the digits are checked for a consistent change in comparison to the previous value. This only makes sense, if the last digit is changing very slowly and every single digit is visible (e.g. 4.7 --> 4.8 --> 4.9 --> 5.0 --> 5.1). If single digits are skipped, for example because the digits changes to fast, this should be disabled (e.g. 4.7 --> 5.0 --> 5.1). [MQTT] [MQTT] Uri = mqtt://IP-ADRESS:1883 MainTopic = wasserzaehler ClientID = wasser user = USERNAME password = PASSWORD Here the post processing and consistency check for the readout can be adjusted Parameter Meaning Options/Examples Uri URI to the MQTT broker including port e.g.: mqtt://IP-Address:Port MainTopic MQTT main topic, under which the counters are published. The single value will be published with the following key: MainTopic/number/PARAMETER where parameters are: value, rate, timestamp, error and json The general connection status can be found in MainTopic/connection ClientID ClientID to connect to the MQTT broker user user for MQTT authentication (optional) password password for MQTT authentication (optional) [AutoTimer] [AutoTimer] AutoStart= true Intervall = 4.85 This paragraph is used to automatically trigger the periodic automated readout. Parameter Meaning Options/Examples AutoStart Automatically trigger the readout after startup Intervall Readout interval in minutes Values smaller than 2 minutes do not make sense, as this is the time for one detection [Debug] [Debug] Logfile = true LogfileRetentionInDays = 2 This paragraph is used to switch on an extended logging. It is optional and by default only a minimum logging is enabled. \u203c\ufe0f Attention: in case of extended logging the size of the log file ( /log.txt , /alignment.txt ) might increase rapidly, therefore manually deletion from time to time is recommended Parameter Meaning Options/Examples Logfile Turn on (true) or off (false) the extended logging parameter and section optional LogfileRetentionInDays Number of days, for which the log files should be stored 0 = keep forever [System] [System] TimeZone = CET-1CEST,M3.5.0,M10.5.0/3 ;TimeServer = TIMESERVER ;Hostname = undefined SetupMode = false This paragraph is used to switch on an extended logging. It is optional and by default only a minimum logging is enabled. \u203c\ufe0f Attention: in case of extended logging the size of the log file ( /log.txt , /alignment.txt ) might increase rapidly, therefore manually deletion from time to time is recommended Parameter Meaning Options/Examples TimeZone TimeZone of the system can be specified Central european, with summertime adjustement: CET-1CEST,M3.5.0,M10.5.0/3 TimeServer An dedicated time server can be specified. default = pool.ntp.org Hostname Additionally to the wlan.ini the hostname can be specified. It will be transferred to the wlan.ini and initiate a reboot SetupMode If enabled, the server starts in an initial setup mode. This is automatically disabled at the end of the setup [Ende] No function, just to mark, that the config is done!","title":"Configuration Parameter Details"},{"location":"Configuration-Parameter-Details/#configuration-parameter-details","text":"","title":"Configuration Parameter Details"},{"location":"Configuration-Parameter-Details/#makeimage","text":"[MakeImage] LogImageLocation = /log/source WaitBeforeTakingPicture = 5 LogfileRetentionInDays = 15 Brightness = -2 ;Contrast = 0 ;Saturation = 0 ImageQuality = 5 ImageSize = VGA FixedExposure = true Parameter Meaning Options/Examples LogImageLocation location for storing a copy of the image WaitBeforeTakingPicture waiting time between switch on the flash-light and taking the image in seconds LogfileRetentionInDays Number of days, for which the log files should be stored 0 = keep forever Brightness Adjustment of the camera brightness (-2 ... 2) Contrast NOT IMPLEMENTED Saturation NOT IMPLEMENTED ImageQuality Input image jpg-compression quality 0 (best) to 100 (lowest) 5 = default ImageSize Input Image Size from Camera only VGA, QVGA FixedExposure If enabled, the exposure settings are fixed at the beginning and the waiting time after switching on the illumination will be skipped","title":"[MakeImage]"},{"location":"Configuration-Parameter-Details/#alignment","text":"[Alignment] InitialRotate = 179 InitialMirror = false SearchFieldX = 20 SearchFieldY = 20 AlignmentAlgo = Default FlipImageSize = false /config/ref0.jpg 104 271 /config/ref1.jpg 442 142 Parameter Meaning Options/Examples InitialMirror Option for initially mirroring the image on the original x-axis InitialRotate Initial rotation of image before alignment in degree (1...359) FlipImageSize Changes the aspect ratio after the image rotation to avoid cropping of the rotated imaged /config/refx.jpg 98, 257 Link to reference image and corresponding target coordinates file link is relative to sd-card root SearchFieldX/Y Search field size in X/Y for finding the reference images [pixel] Here two reference images are needed. Therefore rotation and shifting can be compensated. As the alignment is one of the most computing time using part, the search field needs to be limited. The calculation time goes quadratic with the search field size.","title":"[Alignment]"},{"location":"Configuration-Parameter-Details/#digits","text":"[Digits] Model=/config/digits.tfl ModelInputSize 20, 32 LogImageLocation = /log/digit LogfileRetentionInDays = 2 number1.digit1 292 120 37 67 number1.digit2 340 120 37 67 number1.digit3 389 120 37 67 number2.digit1 292 180 37 67 number2.digit2 340 180 37 67 Parameter Meaning Options/Examples Model Link to the CNN-tflite file used for AI-image recognition ModelInputSize Image input size for the CNN-Network [pixel] needed to resize the ROI to the input size LogImageLocation storage location for the recognized images, including the CNN-results in the file name/location numberX.digitY ROI for the corresponding digit in the aligned image. More than one number can be specified. Therefore the name consists of a naming of the number ( numberX ) and the region of interest ( digitY ) - separated by . LogfileRetentionInDays Number of days, for which the log files should be stored 0 = keep forever LogImageLocation location for storing a copy of the image","title":"[Digits]"},{"location":"Configuration-Parameter-Details/#analog","text":"[Analog] Model=/config/analog.tfl ModelInputSize 32, 32 LogImageLocation = /log/analog LogfileRetentionInDays = 2 number1.analog1, 433, 207, 99, 99 number1.analog2, 378, 313, 99, 99 number1.analog3, 280, 356, 99, 99 number1.analog4, 149, 313, 99, 99 number2.analog1, 280, 456, 99, 99 number2.analog2, 149, 413, 99, 99 Same as for [digit], here only for the analog pointers","title":"[Analog]"},{"location":"Configuration-Parameter-Details/#postprocessing","text":"[PostProcessing] number1.DecimalShift = 0 number2.DecimalShift = -1 PreValueUse = true PreValueAgeStartup = 720 AllowNegativeRates = false number1.MaxRateValue = 0.1 number2.MaxRateValue = 0.1 ErrorMessage = true CheckDigitIncreaseConsistency = false Here the post processing and consistency check for the readout can be adjusted Parameter Meaning Options/Examples PreValueUse Use the previous value for consistency check and substitution for NaN (True / False) PreValueAgeStartup Max age of PreValue after a reboot (downtime) AllowNegativeRates Allow decrease of the readout value numberX.MaxRateValue Maximum chance rate from one to the next readout. This can be specified for each number individual. ErrorMessage Show error messages numberX.DecimalShift Shifting of the decimal separator from the default position between digital and analog. This can be specified for each number individual. DecimalShift = 2: 123.456 --> 12345.6 DecimalShift = -1: 123.456 --> 12.3456 CheckDigitIncreaseConsistency This parameter controls, if the digits are checked for a consistent change in comparison to the previous value. This only makes sense, if the last digit is changing very slowly and every single digit is visible (e.g. 4.7 --> 4.8 --> 4.9 --> 5.0 --> 5.1). If single digits are skipped, for example because the digits changes to fast, this should be disabled (e.g. 4.7 --> 5.0 --> 5.1).","title":"[PostProcessing]"},{"location":"Configuration-Parameter-Details/#mqtt","text":"[MQTT] Uri = mqtt://IP-ADRESS:1883 MainTopic = wasserzaehler ClientID = wasser user = USERNAME password = PASSWORD Here the post processing and consistency check for the readout can be adjusted Parameter Meaning Options/Examples Uri URI to the MQTT broker including port e.g.: mqtt://IP-Address:Port MainTopic MQTT main topic, under which the counters are published. The single value will be published with the following key: MainTopic/number/PARAMETER where parameters are: value, rate, timestamp, error and json The general connection status can be found in MainTopic/connection ClientID ClientID to connect to the MQTT broker user user for MQTT authentication (optional) password password for MQTT authentication (optional)","title":"[MQTT]"},{"location":"Configuration-Parameter-Details/#autotimer","text":"[AutoTimer] AutoStart= true Intervall = 4.85 This paragraph is used to automatically trigger the periodic automated readout. Parameter Meaning Options/Examples AutoStart Automatically trigger the readout after startup Intervall Readout interval in minutes Values smaller than 2 minutes do not make sense, as this is the time for one detection","title":"[AutoTimer]"},{"location":"Configuration-Parameter-Details/#debug","text":"[Debug] Logfile = true LogfileRetentionInDays = 2 This paragraph is used to switch on an extended logging. It is optional and by default only a minimum logging is enabled. \u203c\ufe0f Attention: in case of extended logging the size of the log file ( /log.txt , /alignment.txt ) might increase rapidly, therefore manually deletion from time to time is recommended Parameter Meaning Options/Examples Logfile Turn on (true) or off (false) the extended logging parameter and section optional LogfileRetentionInDays Number of days, for which the log files should be stored 0 = keep forever","title":"[Debug]"},{"location":"Configuration-Parameter-Details/#system","text":"[System] TimeZone = CET-1CEST,M3.5.0,M10.5.0/3 ;TimeServer = TIMESERVER ;Hostname = undefined SetupMode = false This paragraph is used to switch on an extended logging. It is optional and by default only a minimum logging is enabled. \u203c\ufe0f Attention: in case of extended logging the size of the log file ( /log.txt , /alignment.txt ) might increase rapidly, therefore manually deletion from time to time is recommended Parameter Meaning Options/Examples TimeZone TimeZone of the system can be specified Central european, with summertime adjustement: CET-1CEST,M3.5.0,M10.5.0/3 TimeServer An dedicated time server can be specified. default = pool.ntp.org Hostname Additionally to the wlan.ini the hostname can be specified. It will be transferred to the wlan.ini and initiate a reboot SetupMode If enabled, the server starts in an initial setup mode. This is automatically disabled at the end of the setup","title":"[System]"},{"location":"Configuration-Parameter-Details/#ende","text":"No function, just to mark, that the config is done!","title":"[Ende]"},{"location":"Configuration/","text":"Configuration Warning This page overlaps Graphical-Configuration . They should be merged to one page! Most of the settings can be modified with the help of a web based graphical user interface . This is hosted through the web server on the ESP32. More configuration parameters can be edited by hand in the config.ini and corresponding files in the /config directory on the SD-card. If you where using the Version 1 of the watermeter you can easily transfer the configuration to the new system by following the steps in this migration description Processing / Config.ini principle The principle is very simple and can most easily be described as a flow of processing steps. Each step has a dedicated parameter description in the config.ini , which is indicated by brackets [name_of_step] . The steps are processed in the order written in the config file. That means, that you first have to describe the image taking, then the aligning and cutting and only after that you can start to config a neural network. The last step is the post processing. Processing steps - Overview In the following you get a short overview over the available steps. This order is also the suggested order for the processing flow. Single steps can be left out, if not needed (e.g. omit the analog part, if only digits are present) 1. [MakeImage] This steps parametrises the taking of the image by the ESP32-CAM. Size, quality and storage for logging and debugging can be set. 2. [Alignment] Image preprocessing, including image alignment with reference images 3. [Digits] Neural network evaluation of an image for digits. The neural network is defined by a tflite formatted file and the output is a number between 0 .. 9 or NaN (if image is not unique enough) 4. [Analog] Neural network evaluation of analog counter. The neural network is defined by a tflite formatted file and the output is a number between 0.0 .. 9.9, representing the position of the pointer. 5. [PostProcessing] Summarized the individually converted pictures to the overall result. It also implements some error corrections and consistency checks to filter wrong reading. 6. [MQTT] Transfer of the readings to a MQTT server. 7. [AutoTimer] Configuration of the automated flow start at the start up of the ESP32. 8. [Debug] Configuration for debugging details 9. [Ende] No meaning, just an additional indication, that the configuration is finished. A detailed parameter description can be found here: [[Configuration Parameter Details]]. Graphical configuration interface It is recommended to do the configuration of the alignment structures and ROIs through the graphical user interface. A step by step instruction can be found here: [[Graphical Configuration]] Background for Image Alignment Details on the image recognition flow can be found in the other project here: https://github.com/jomjol/water-meter-system-complete/blob/master/images/Alignment_procedure_draft.pdf The config.ini here has the same functionality and options, but a slightly different syntax due to a own written ini-parser is used. Migration see here . Integration into Home Assistant Thanks to the help of the user @deadly667 here are some hints for the integration into the home assistant: [[Integration-Home-Assistant]]","title":"Configuration"},{"location":"Configuration/#configuration","text":"Warning This page overlaps Graphical-Configuration . They should be merged to one page! Most of the settings can be modified with the help of a web based graphical user interface . This is hosted through the web server on the ESP32. More configuration parameters can be edited by hand in the config.ini and corresponding files in the /config directory on the SD-card. If you where using the Version 1 of the watermeter you can easily transfer the configuration to the new system by following the steps in this migration description","title":"Configuration"},{"location":"Configuration/#processing-configini-principle","text":"The principle is very simple and can most easily be described as a flow of processing steps. Each step has a dedicated parameter description in the config.ini , which is indicated by brackets [name_of_step] . The steps are processed in the order written in the config file. That means, that you first have to describe the image taking, then the aligning and cutting and only after that you can start to config a neural network. The last step is the post processing.","title":"Processing / Config.ini principle"},{"location":"Configuration/#processing-steps-overview","text":"In the following you get a short overview over the available steps. This order is also the suggested order for the processing flow. Single steps can be left out, if not needed (e.g. omit the analog part, if only digits are present)","title":"Processing steps - Overview"},{"location":"Configuration/#1-makeimage","text":"This steps parametrises the taking of the image by the ESP32-CAM. Size, quality and storage for logging and debugging can be set.","title":"1. [MakeImage]"},{"location":"Configuration/#2-alignment","text":"Image preprocessing, including image alignment with reference images","title":"2. [Alignment]"},{"location":"Configuration/#3-digits","text":"Neural network evaluation of an image for digits. The neural network is defined by a tflite formatted file and the output is a number between 0 .. 9 or NaN (if image is not unique enough)","title":"3. [Digits]"},{"location":"Configuration/#4-analog","text":"Neural network evaluation of analog counter. The neural network is defined by a tflite formatted file and the output is a number between 0.0 .. 9.9, representing the position of the pointer.","title":"4. [Analog]"},{"location":"Configuration/#5-postprocessing","text":"Summarized the individually converted pictures to the overall result. It also implements some error corrections and consistency checks to filter wrong reading.","title":"5. [PostProcessing]"},{"location":"Configuration/#6-mqtt","text":"Transfer of the readings to a MQTT server.","title":"6. [MQTT]"},{"location":"Configuration/#7-autotimer","text":"Configuration of the automated flow start at the start up of the ESP32.","title":"7. [AutoTimer]"},{"location":"Configuration/#8-debug","text":"Configuration for debugging details","title":"8. [Debug]"},{"location":"Configuration/#9-ende","text":"No meaning, just an additional indication, that the configuration is finished. A detailed parameter description can be found here: [[Configuration Parameter Details]].","title":"9. [Ende]"},{"location":"Configuration/#graphical-configuration-interface","text":"It is recommended to do the configuration of the alignment structures and ROIs through the graphical user interface. A step by step instruction can be found here: [[Graphical Configuration]]","title":"Graphical configuration interface"},{"location":"Configuration/#background-for-image-alignment","text":"Details on the image recognition flow can be found in the other project here: https://github.com/jomjol/water-meter-system-complete/blob/master/images/Alignment_procedure_draft.pdf The config.ini here has the same functionality and options, but a slightly different syntax due to a own written ini-parser is used. Migration see here .","title":"Background for Image Alignment"},{"location":"Configuration/#integration-into-home-assistant","text":"Thanks to the help of the user @deadly667 here are some hints for the integration into the home assistant: [[Integration-Home-Assistant]]","title":"Integration into Home Assistant"},{"location":"Correction%20Algorithm/","text":"Correction Algorithm After the digitization of the images and the composition to a number a checking and correction algorithm is applied. This is explained here. There are several reasons, that a check might be necessary: In case of digits there is the output of \"N\" (=NaN = Not-a-Number) in case the digit cannot be detected correctly. This happens for example if the image shows a digit between to states The replacement of the \"N\" with a previous value could be not sufficient, due to the fact, that it might have changed. There is a misreading of one one of the numbers. This can always happen in case of neural network processing. Terms and definitions PreValue The last correct read value. Either from a previous correctly identified value or manual setting by the user. This is used to replace \"N\"s and make a check for the absolute change. Digits Value that are digitized from a digital number. There are 11 allowed values for this: Digits: 0, 1, 2, ... 9 N = Not-a-Number - representing a not unique state between two numbers Analogs This are value derived from a pointer like meter. This never has the state \"N\". CheckDigitIncreaseConsistency If this is enabled an \"inteligent\" algorithm is used to derive from zero-crossing of discrete digit positions, if the number should have been increased. This is relevant because in some of the digit meters, the increase of a digit to the next number can be seen, before the subdigit has gone through zero. For example: 16.6 --> 16.7 --> 1N.8 --> 17.9 corrected to 16.9 --> 17.0 --> 17.1 As you can see, the 17.9 is a false reading as the 7 is assumed to be already readable, although the subdigit has not crossed the zero. In this case the CheckDigitIncreaseConsistency algorithm will correct this to 16.9 A detailed description of the algorithm can be found below (not yet ready!) Negative Rate allowed Most of the meters only have increasing numbers and do not count backwards. Therefore a negative rate (= negative change compared to the PreValue) is surely a false value. This can be checked an flagged as false reading MaxRateValue / MaxRateType Here the maximum change from one to the next reading can be limited. If a false reading of the neural network results in a change larger than this, the reading is flagged as false. There a two types of comparisons possible 1) AbsolutChange : Here the difference between the PreValue and the current reading is compared directly, independent how much time has passed since the last reading. 2) RelativeRate : in this case a change rate in the unit of change/minute is calculated, taking the time between the last and the current reading into account. Be careful, that with increasing time, the absolute allowed change increases. Example: relative rate of 0.05 m\u00b3/minute --> after 20 minutes a maximum change of 20 minutes * 0.05 m\u00b3/minute = 1 m\u00b3 is possible. That means that a false reading of 1 m\u00b3 cannot be detected false after about 20 minutes in this case Assume, that there might me no change in the meter for hours (e.g. during the night) a much bigger change could also be accepted. Flow Chart CheckDigitIncreaseConsistency Algorithm The check digit increase consistency algorithm is functional for the digits only. Due to the fact, that the rotation might be a little bit earlier or later compared to the zero crossing of the digit before, errors during the reading before and after a zero crossing can be wrong. Therefore a simple algorithm can be applied, checking the consistency of zero crossing and changes in the following digit. This is applied to one after the other digit, starting with the lowest priority digits.","title":"Correction Algorithm"},{"location":"Correction%20Algorithm/#correction-algorithm","text":"After the digitization of the images and the composition to a number a checking and correction algorithm is applied. This is explained here. There are several reasons, that a check might be necessary: In case of digits there is the output of \"N\" (=NaN = Not-a-Number) in case the digit cannot be detected correctly. This happens for example if the image shows a digit between to states The replacement of the \"N\" with a previous value could be not sufficient, due to the fact, that it might have changed. There is a misreading of one one of the numbers. This can always happen in case of neural network processing.","title":"Correction Algorithm"},{"location":"Correction%20Algorithm/#terms-and-definitions","text":"","title":"Terms and definitions"},{"location":"Correction%20Algorithm/#prevalue","text":"The last correct read value. Either from a previous correctly identified value or manual setting by the user. This is used to replace \"N\"s and make a check for the absolute change.","title":"PreValue"},{"location":"Correction%20Algorithm/#digits","text":"Value that are digitized from a digital number. There are 11 allowed values for this: Digits: 0, 1, 2, ... 9 N = Not-a-Number - representing a not unique state between two numbers","title":"Digits"},{"location":"Correction%20Algorithm/#analogs","text":"This are value derived from a pointer like meter. This never has the state \"N\".","title":"Analogs"},{"location":"Correction%20Algorithm/#checkdigitincreaseconsistency","text":"If this is enabled an \"inteligent\" algorithm is used to derive from zero-crossing of discrete digit positions, if the number should have been increased. This is relevant because in some of the digit meters, the increase of a digit to the next number can be seen, before the subdigit has gone through zero. For example: 16.6 --> 16.7 --> 1N.8 --> 17.9 corrected to 16.9 --> 17.0 --> 17.1 As you can see, the 17.9 is a false reading as the 7 is assumed to be already readable, although the subdigit has not crossed the zero. In this case the CheckDigitIncreaseConsistency algorithm will correct this to 16.9 A detailed description of the algorithm can be found below (not yet ready!)","title":"CheckDigitIncreaseConsistency"},{"location":"Correction%20Algorithm/#negative-rate-allowed","text":"Most of the meters only have increasing numbers and do not count backwards. Therefore a negative rate (= negative change compared to the PreValue) is surely a false value. This can be checked an flagged as false reading","title":"Negative Rate allowed"},{"location":"Correction%20Algorithm/#maxratevalue-maxratetype","text":"Here the maximum change from one to the next reading can be limited. If a false reading of the neural network results in a change larger than this, the reading is flagged as false. There a two types of comparisons possible 1) AbsolutChange : Here the difference between the PreValue and the current reading is compared directly, independent how much time has passed since the last reading. 2) RelativeRate : in this case a change rate in the unit of change/minute is calculated, taking the time between the last and the current reading into account. Be careful, that with increasing time, the absolute allowed change increases. Example: relative rate of 0.05 m\u00b3/minute --> after 20 minutes a maximum change of 20 minutes * 0.05 m\u00b3/minute = 1 m\u00b3 is possible. That means that a false reading of 1 m\u00b3 cannot be detected false after about 20 minutes in this case Assume, that there might me no change in the meter for hours (e.g. during the night) a much bigger change could also be accepted.","title":"MaxRateValue / MaxRateType"},{"location":"Correction%20Algorithm/#flow-chart","text":"","title":"Flow Chart"},{"location":"Correction%20Algorithm/#checkdigitincreaseconsistency-algorithm","text":"The check digit increase consistency algorithm is functional for the digits only. Due to the fact, that the rotation might be a little bit earlier or later compared to the zero crossing of the digit before, errors during the reading before and after a zero crossing can be wrong. Therefore a simple algorithm can be applied, checking the consistency of zero crossing and changes in the following digit. This is applied to one after the other digit, starting with the lowest priority digits.","title":"CheckDigitIncreaseConsistency Algorithm"},{"location":"Demo-Mode/","text":"Demo Mode For Demo and Testing Purpose, the device can use pre-recorded images. You need to enable it in the configuration ( TakeImage > Demo ) and also provide the needed files on the SD-Card. For each round one image gets used, starting with the first image for the first round. For the reference image and the alignment also the first image gets used. Once the last image got reached, it starts again with the first one. SD-Card Structure demo/ \u251c\u2500\u2500 520.8983.jpg \u251c\u2500\u2500 520.9086.jpg \u251c\u2500\u2500 520.9351.jpg \u251c\u2500\u2500 ... \u2514\u2500\u2500 files.txt The jpg files can have any name The jpg files must be smaller than 30'000 bytes The files.txt must contains a list of those files, eg: 520.8983.jpg 520.9086.jpg 520.9351.jpg Recording To record real images of a meter, you have to periodically fetch http://<IP>/img_tmp/raw.jpg . To automate this, you can use the following shell script (Linux only): #!/bin/bash while [[ true ]]; do echo \"fetching value...\" wget -q http://192.168.1.151/value -O value.txt value=`cat value.txt` echo \"Value: $value\" diff=`diff value.txt value_previous.txt` changed=$? #echo \"Diff: $diff\" if [[ $changed -ne 0 ]]; then echo \"Value changed:\" echo $diff echo \"fetching image...\" wget -q http://192.168.1.151/img_tmp/raw.jpg -O $value.jpg else echo \"Value did not change, skipping image fetching!\" fi cp value.txt value_previous.txt echo \"waiting 60s...\" sleep 60 done Installation Just install the zip file using the OTA Update functionality. How does it work The Demo Mode tries to interfere as less as possible with the normal behavior. Whenever a Cam Framebuffer gets taken ( esp_camera_fb_get() ), it replaces the framebuffer with the image from the SD-Card. Example Data of a Watermeter You can use the following demo images if you want: It covers a meter range from 530.00688 to 531.85882 . Animation Animation of the watermeter (77 MB!) All images (843 images) Demo_Images_Watermeter_530.00688-532.08243_843_images.zip Animation of it (186 MB!) Selection of 84 images Demo_Images_Watermeter_530.00688-532.08243_84_images.zip Selection of 42 images Demo_Images_Watermeter_530.00688-532.08243_42_images.zip","title":"Demo Mode"},{"location":"Demo-Mode/#demo-mode","text":"For Demo and Testing Purpose, the device can use pre-recorded images. You need to enable it in the configuration ( TakeImage > Demo ) and also provide the needed files on the SD-Card. For each round one image gets used, starting with the first image for the first round. For the reference image and the alignment also the first image gets used. Once the last image got reached, it starts again with the first one.","title":"Demo Mode"},{"location":"Demo-Mode/#sd-card-structure","text":"demo/ \u251c\u2500\u2500 520.8983.jpg \u251c\u2500\u2500 520.9086.jpg \u251c\u2500\u2500 520.9351.jpg \u251c\u2500\u2500 ... \u2514\u2500\u2500 files.txt The jpg files can have any name The jpg files must be smaller than 30'000 bytes The files.txt must contains a list of those files, eg: 520.8983.jpg 520.9086.jpg 520.9351.jpg","title":"SD-Card Structure"},{"location":"Demo-Mode/#recording","text":"To record real images of a meter, you have to periodically fetch http://<IP>/img_tmp/raw.jpg . To automate this, you can use the following shell script (Linux only): #!/bin/bash while [[ true ]]; do echo \"fetching value...\" wget -q http://192.168.1.151/value -O value.txt value=`cat value.txt` echo \"Value: $value\" diff=`diff value.txt value_previous.txt` changed=$? #echo \"Diff: $diff\" if [[ $changed -ne 0 ]]; then echo \"Value changed:\" echo $diff echo \"fetching image...\" wget -q http://192.168.1.151/img_tmp/raw.jpg -O $value.jpg else echo \"Value did not change, skipping image fetching!\" fi cp value.txt value_previous.txt echo \"waiting 60s...\" sleep 60 done","title":"Recording"},{"location":"Demo-Mode/#installation","text":"Just install the zip file using the OTA Update functionality.","title":"Installation"},{"location":"Demo-Mode/#how-does-it-work","text":"The Demo Mode tries to interfere as less as possible with the normal behavior. Whenever a Cam Framebuffer gets taken ( esp_camera_fb_get() ), it replaces the framebuffer with the image from the SD-Card.","title":"How does it work"},{"location":"Demo-Mode/#example-data-of-a-watermeter","text":"You can use the following demo images if you want: It covers a meter range from 530.00688 to 531.85882 .","title":"Example Data of a Watermeter"},{"location":"Demo-Mode/#animation","text":"Animation of the watermeter (77 MB!)","title":"Animation"},{"location":"Demo-Mode/#all-images-843-images","text":"Demo_Images_Watermeter_530.00688-532.08243_843_images.zip Animation of it (186 MB!)","title":"All images (843 images)"},{"location":"Demo-Mode/#selection-of-84-images","text":"Demo_Images_Watermeter_530.00688-532.08243_84_images.zip","title":"Selection of 84 images"},{"location":"Demo-Mode/#selection-of-42-images","text":"Demo_Images_Watermeter_530.00688-532.08243_42_images.zip","title":"Selection of 42 images"},{"location":"Error-Codes/","text":"This page lists the possible error codes, their meaning and possible solutions. The effective error codes can be found here . Critical Errors Those Errors make the normal operation of the device impossible. Most likely they are caused by a hardware issue! 0x00000001 PSRAM bad Your device most likely has no PSRAM at all or it is too small (needs to have at least 4 MBytes)! See Hardware Compatibility . Usually the log shows something like this: psram: PSRAM ID read error: 0xffffffff cpu_start: Failed to init external RAM! 0x00000002 Heap too small The firmware failed to allocate enough memory. This most likely is a consequential error of a bad PSRAM! 0x00000004 Cam bad The attached camera can not be initialized. This usually is because on of the following reasons: - The camera is not supported, see Hardware Compatibility - The camera is not attached properly -> Try to remove and attach it again. Make sure you move the black part enough into the socket! - The camera or the camera cable is damaged Non-Critical Errors Those Errors can be caused by an error during initialization. It is possible that the error has no impact at all or that a reboot solves it. 0x00000100 Cam Framebuffer bad The firmware was unable to initialize the Camera Framebuffer. The firmware will continue to work, but other consequential error might arise. A reboot of the device might help. 0x00000200 NTP failed The firmware failed to get the world time from an NTP server. The firmware will continue to work, but has a wrong time.","title":"Error Codes"},{"location":"Error-Codes/#critical-errors","text":"Those Errors make the normal operation of the device impossible. Most likely they are caused by a hardware issue!","title":"Critical Errors"},{"location":"Error-Codes/#0x00000001-psram-bad","text":"Your device most likely has no PSRAM at all or it is too small (needs to have at least 4 MBytes)! See Hardware Compatibility . Usually the log shows something like this: psram: PSRAM ID read error: 0xffffffff cpu_start: Failed to init external RAM!","title":"0x00000001 PSRAM bad"},{"location":"Error-Codes/#0x00000002-heap-too-small","text":"The firmware failed to allocate enough memory. This most likely is a consequential error of a bad PSRAM!","title":"0x00000002 Heap too small"},{"location":"Error-Codes/#0x00000004-cam-bad","text":"The attached camera can not be initialized. This usually is because on of the following reasons: - The camera is not supported, see Hardware Compatibility - The camera is not attached properly -> Try to remove and attach it again. Make sure you move the black part enough into the socket! - The camera or the camera cable is damaged","title":"0x00000004 Cam bad"},{"location":"Error-Codes/#non-critical-errors","text":"Those Errors can be caused by an error during initialization. It is possible that the error has no impact at all or that a reboot solves it.","title":"Non-Critical Errors"},{"location":"Error-Codes/#0x00000100-cam-framebuffer-bad","text":"The firmware was unable to initialize the Camera Framebuffer. The firmware will continue to work, but other consequential error might arise. A reboot of the device might help.","title":"0x00000100 Cam Framebuffer bad"},{"location":"Error-Codes/#0x00000200-ntp-failed","text":"The firmware failed to get the world time from an NTP server. The firmware will continue to work, but has a wrong time.","title":"0x00000200 NTP failed"},{"location":"Error-Debugging/","text":"Error Debugging Rebooting General Remark Due to the rather complex code with a lot of external libraries and the limited availability of memory a reboot of the device from time to time is \"normal\". Background are memory leakages and therefore running out of free memory. The hardware of the ESP32CAM has a varying quality. I have one and the same hardware with a reboot range from every 5 detection runs to up to 250 detection runs. Getting deeper inside Have a look into the log file ( /log/message/... ). If the log file is very short you need to enable a enhanced logging in the config.ini (Debug --> logfile = true ) . Analyze the debugging output of the serial interface Connect a serial to USB interface (like for flashing) and make a logging of the serial communication There are a lot more intermediate information and the lines before the reboot tell you, where the firmware fails If you make an issue about this, please post these two information additionally Don't forget to remove your WLAN password in the serial log Often observed problems Hardware failure Camera not working --> check the interface, test another module Low cost module with only 2MB of PSRAM instead of 4MB --> image taking will fail first. This will never work due to too low memory ROI misaligned This typically happens if you have suboptimal \"Alignement Marks\". A very simple and working solution is to put put higly contrasted stickers on your meter and put \"Alignement Marks\" on it (see picture below) If after those adjustement you still have some issues, you can try to adjust your aligmenet settings in expert mode: My Analog Meter are recognized as Digital Counter or vice versa First, check that your ROI are correctly defined (yey!) Second, verify that the name of your ROI analog and digital ROIs are different Recognition is working well, but number aren't sorted correctly You have to sort your ROI correctly (Bigger to smaller). Select your ROI and click either \"move next\" or \"move previous\". Repeat until your ROI are correctly sorted","title":"Error Debugging"},{"location":"Error-Debugging/#error-debugging","text":"","title":"Error Debugging"},{"location":"Error-Debugging/#rebooting","text":"","title":"Rebooting"},{"location":"Error-Debugging/#general-remark","text":"Due to the rather complex code with a lot of external libraries and the limited availability of memory a reboot of the device from time to time is \"normal\". Background are memory leakages and therefore running out of free memory. The hardware of the ESP32CAM has a varying quality. I have one and the same hardware with a reboot range from every 5 detection runs to up to 250 detection runs.","title":"General Remark"},{"location":"Error-Debugging/#getting-deeper-inside","text":"Have a look into the log file ( /log/message/... ). If the log file is very short you need to enable a enhanced logging in the config.ini (Debug --> logfile = true ) . Analyze the debugging output of the serial interface Connect a serial to USB interface (like for flashing) and make a logging of the serial communication There are a lot more intermediate information and the lines before the reboot tell you, where the firmware fails If you make an issue about this, please post these two information additionally Don't forget to remove your WLAN password in the serial log","title":"Getting deeper inside"},{"location":"Error-Debugging/#often-observed-problems","text":"","title":"Often observed problems"},{"location":"Error-Debugging/#hardware-failure","text":"Camera not working --> check the interface, test another module Low cost module with only 2MB of PSRAM instead of 4MB --> image taking will fail first. This will never work due to too low memory","title":"Hardware failure"},{"location":"Error-Debugging/#roi-misaligned","text":"This typically happens if you have suboptimal \"Alignement Marks\". A very simple and working solution is to put put higly contrasted stickers on your meter and put \"Alignement Marks\" on it (see picture below) If after those adjustement you still have some issues, you can try to adjust your aligmenet settings in expert mode:","title":"ROI misaligned"},{"location":"Error-Debugging/#my-analog-meter-are-recognized-as-digital-counter-or-vice-versa","text":"First, check that your ROI are correctly defined (yey!) Second, verify that the name of your ROI analog and digital ROIs are different","title":"My Analog Meter are recognized as Digital Counter or vice versa"},{"location":"Error-Debugging/#recognition-is-working-well-but-number-arent-sorted-correctly","text":"You have to sort your ROI correctly (Bigger to smaller). Select your ROI and click either \"move next\" or \"move previous\". Repeat until your ROI are correctly sorted","title":"Recognition is working well, but number aren't sorted correctly"},{"location":"External-LED/","text":"External LED The internal flash LED is very close to the camera axis. This results in reflection, especially in case of flat glass surfaces such as for power meters. To circumvent this problem, it is now possible to control external LEDs, which than can be places somewhere else in the setup. As not simples LEDs are used, but RGB leds with a digital interface like WS2812 not only the position, but also the color and intensity of the illumination can now be adjusted. The following image shows a direct comparision of the \"old\" internal flash LED and two off axis LEDs. There is also a new meter adapter available. This has two features: designed for small clearings in front of the meter and prepared for WS2812 LEDs . 1. Hardware installation of the LED stripe The control line of the LED stripe is connected with a 470 Ohm resistor to the GPIO12. For power supply stabilization a capacitor between 5V and ground is recommended. Here a 470\u00b5F polymer capacitor is used. As a power supply a 5V from the ESP32 is used like in the following wiring. 2. Software configuration The handling of the WS2812 LED controller needs some other libraries, therefore it is controlled within a dedicated section called GPIO Settings . The external LED stripe is connected to GPIO12. After activating the \"GPIO Settings\" section, the internal flash is per default disabled. In order to activate the external LED, you need to activate GPIO 12 state and select \"extern flash light ws281x ...\" . Parameter Meaning LED-Type There are several types of controller implemented: WS2812(B), WS2813, SK6812 Numbers of LED Number of LEDs on the LED stripe LED Color The color and intensity can be controlled directly by a red/green/blue value, each within the range from 0 (off) to 255 (full) Enabling the GPIO settings automatically disables the flash LED. Therefore you can enable it here manually by checking GPIO4 and choose \"build-in led flash light\" . It is not recommended to use both illumination parallel.","title":"External LED"},{"location":"External-LED/#external-led","text":"The internal flash LED is very close to the camera axis. This results in reflection, especially in case of flat glass surfaces such as for power meters. To circumvent this problem, it is now possible to control external LEDs, which than can be places somewhere else in the setup. As not simples LEDs are used, but RGB leds with a digital interface like WS2812 not only the position, but also the color and intensity of the illumination can now be adjusted. The following image shows a direct comparision of the \"old\" internal flash LED and two off axis LEDs. There is also a new meter adapter available. This has two features: designed for small clearings in front of the meter and prepared for WS2812 LEDs .","title":"External LED"},{"location":"External-LED/#1-hardware-installation-of-the-led-stripe","text":"The control line of the LED stripe is connected with a 470 Ohm resistor to the GPIO12. For power supply stabilization a capacitor between 5V and ground is recommended. Here a 470\u00b5F polymer capacitor is used. As a power supply a 5V from the ESP32 is used like in the following wiring.","title":"1. Hardware installation of the LED stripe"},{"location":"External-LED/#2-software-configuration","text":"The handling of the WS2812 LED controller needs some other libraries, therefore it is controlled within a dedicated section called GPIO Settings . The external LED stripe is connected to GPIO12. After activating the \"GPIO Settings\" section, the internal flash is per default disabled. In order to activate the external LED, you need to activate GPIO 12 state and select \"extern flash light ws281x ...\" . Parameter Meaning LED-Type There are several types of controller implemented: WS2812(B), WS2813, SK6812 Numbers of LED Number of LEDs on the LED stripe LED Color The color and intensity can be controlled directly by a red/green/blue value, each within the range from 0 (off) to 255 (full) Enabling the GPIO settings automatically disables the flash LED. Therefore you can enable it here manually by checking GPIO4 and choose \"build-in led flash light\" . It is not recommended to use both illumination parallel.","title":"2. Software configuration"},{"location":"FAQs/","text":"Frequently Asked Questions My device is reboot frequently. What can I do? There are several reasons for the reboot: Frequent HTML requests Wrong configuration, missing configuration files Unstable hardware - see [[Hardware Compatibility]] There is a dedicated Wiki page about this: Frequent Reboots . How accurate are the detections? It is hard to give a specific accuracy number. It depends on many factors, e.g. How in-focus is your camera? How sturdy is the camera mount? Does it slightly move over extended periods of time? What type of meter are you reading? Is the meter already in the training data set? Are you trying to read digits, an analog dial, or both? etc. Anecdotally, the authors of this wiki have great success with the meter. While the AI algorithm itself is not perfect and sometimes returns NaN or incorrect values, other post-processing / prevalue / sanity checks help ensure such invalid values are filtered out. With the correct settings, one author has been running this device for 1 month without any incorrect values reported. See the FAQs below for more details and configuration hints. My numbers are not corrected detected. What can I do? There is a dedicated Wiki page about the correct setting ROI Configuration . This page also includes the instructions for gathering new images for the training. How can I ensure invalid numbers are never reported? As mentioned above, the AI algorithm is not perfect. Sometimes it may read an incorrect value. We can tune the software to almost never report an incorrect value. There is a tradeoff though: the software may report stale values - i.e. it will drop incorrect values for a potentially long period of time, resulting in the meter reading being outdated by hours. If never receiving an incorrect value is important to you, consider tolerating this tradeoff. You can change the following settings to reduce incorrect readings (but potentially increase staleness of data): * Set a prevalue via the UI, then change PostProcessing configuration option PreValueAgeStartup to a much larger number (e.g. 43200 = 30 days). * Change PostProcessing configuration option MaxRateType to be time based instead of absolute. Set MaxRateValue to something realistic (e.g. 5 gal/min). You can often find the max flow rate your meter supports directly on the cover. * Reduce AutoTimer configuration option Intervall to the lowest it can be (e.g. 3 min). The more often you take readings, the less likely for data staleness to occur. Even after I have setup everything perfect there is a false reading - especially around the zero crossing (roll over to next number) The roll over behavior is different for the different meters. E.g.: Rolling over start with different previous position (e.g. at 7, 8 or 9) The neutral position (no rolling) is not perfectly at zero, but rather at something like 7.9 or 8.1, even if it should be exactly 8 The \"PostProcessingAlgo\" is trying to judge out of the individual readings, what number it should be. For example if the previous number is a \"1\", but the next number seems to be a \"8.9\", mos probably there was a \"zero crossing\" and the number is a \"9\" and not still an \"8\" Currently the setting of the algorithm is set to fit most of the meters and cases. But the parameters do not fit perfectly for all situations. Therefore there might be intermediate states, where the reading is false. This is especially the case, at the positions, where the roll over (zero crossing) is just starting. To prevent a sending of false parameters, there is the possibility to limit the maximum allowed change (MaxRateChange). Usually after some time and movement of the counters a bit further, the reading is getting back to a stable reading. To handle this, a parametrized setting would be needed. This is rather complicated to implement as subtle changes make a relevant difference. Currently this is not implemented. So please be a bit patient with your meter :-)","title":"Frequently Asked Questions"},{"location":"FAQs/#frequently-asked-questions","text":"","title":"Frequently Asked Questions"},{"location":"FAQs/#my-device-is-reboot-frequently-what-can-i-do","text":"There are several reasons for the reboot: Frequent HTML requests Wrong configuration, missing configuration files Unstable hardware - see [[Hardware Compatibility]] There is a dedicated Wiki page about this: Frequent Reboots .","title":"My device is reboot frequently. What can I do?"},{"location":"FAQs/#how-accurate-are-the-detections","text":"It is hard to give a specific accuracy number. It depends on many factors, e.g. How in-focus is your camera? How sturdy is the camera mount? Does it slightly move over extended periods of time? What type of meter are you reading? Is the meter already in the training data set? Are you trying to read digits, an analog dial, or both? etc. Anecdotally, the authors of this wiki have great success with the meter. While the AI algorithm itself is not perfect and sometimes returns NaN or incorrect values, other post-processing / prevalue / sanity checks help ensure such invalid values are filtered out. With the correct settings, one author has been running this device for 1 month without any incorrect values reported. See the FAQs below for more details and configuration hints.","title":"How accurate are the detections?"},{"location":"FAQs/#my-numbers-are-not-corrected-detected-what-can-i-do","text":"There is a dedicated Wiki page about the correct setting ROI Configuration . This page also includes the instructions for gathering new images for the training.","title":"My numbers are not corrected detected. What can I do?"},{"location":"FAQs/#how-can-i-ensure-invalid-numbers-are-never-reported","text":"As mentioned above, the AI algorithm is not perfect. Sometimes it may read an incorrect value. We can tune the software to almost never report an incorrect value. There is a tradeoff though: the software may report stale values - i.e. it will drop incorrect values for a potentially long period of time, resulting in the meter reading being outdated by hours. If never receiving an incorrect value is important to you, consider tolerating this tradeoff. You can change the following settings to reduce incorrect readings (but potentially increase staleness of data): * Set a prevalue via the UI, then change PostProcessing configuration option PreValueAgeStartup to a much larger number (e.g. 43200 = 30 days). * Change PostProcessing configuration option MaxRateType to be time based instead of absolute. Set MaxRateValue to something realistic (e.g. 5 gal/min). You can often find the max flow rate your meter supports directly on the cover. * Reduce AutoTimer configuration option Intervall to the lowest it can be (e.g. 3 min). The more often you take readings, the less likely for data staleness to occur.","title":"How can I ensure invalid numbers are never reported?"},{"location":"FAQs/#even-after-i-have-setup-everything-perfect-there-is-a-false-reading-especially-around-the-zero-crossing-roll-over-to-next-number","text":"The roll over behavior is different for the different meters. E.g.: Rolling over start with different previous position (e.g. at 7, 8 or 9) The neutral position (no rolling) is not perfectly at zero, but rather at something like 7.9 or 8.1, even if it should be exactly 8 The \"PostProcessingAlgo\" is trying to judge out of the individual readings, what number it should be. For example if the previous number is a \"1\", but the next number seems to be a \"8.9\", mos probably there was a \"zero crossing\" and the number is a \"9\" and not still an \"8\" Currently the setting of the algorithm is set to fit most of the meters and cases. But the parameters do not fit perfectly for all situations. Therefore there might be intermediate states, where the reading is false. This is especially the case, at the positions, where the roll over (zero crossing) is just starting. To prevent a sending of false parameters, there is the possibility to limit the maximum allowed change (MaxRateChange). Usually after some time and movement of the counters a bit further, the reading is getting back to a stable reading. To handle this, a parametrized setting would be needed. This is rather complicated to implement as subtle changes make a relevant difference. Currently this is not implemented. So please be a bit patient with your meter :-)","title":"Even after I have setup everything perfect there is a false reading - especially around the zero crossing (roll over to next number)"},{"location":"Frequent-Reboots/","text":"Frequent Reboots There are several types of reboots. To get a deeper insight turn on the logging: Internal logging ( config.ini ) Serial log of the UART interface (same as for flashing the firmware) There are two principle types of reboots Random reboots (always different timing and situation) Permanent Reboots always at the same time Random reboots Random reboots have two reasons: overload during HTML access and unstable system In general: there are several mechanisms in the firmware (like saving previous values), to have a \"smooth\" reboot without too many notable disturbance. Overload during HTML access If you frequently access the web server over HTML requests, the firmware tends to reboot. This especially happens during the first run and when the ESP32 is busy with the digitization flow. The reason for this are running out of memory during a flow, minor memory leakage in combination with missing error handling. There is noting you can do about this kind of reboot, beside two thing: Support the firmware development with improved and tested part of code Be patient :-) Unstable system If your system is sometimes running smoothly over several runs and sometimes reboots obviously randomly, you have an partially unstable device. You can check this in the standard log file on the SD card: 2021-12-26T06:34:09: task_autodoFlow - round done 2021-12-26T06:34:09: CPU Temperature: 56.1 2021-12-26T06:38:00: task_autodoFlow - next round - Round #23 Here you see, that the round #23 is starting, so obviously there were no reboots in the last 22 rounds. There is hardware (ESP32CAM), where only 2-3 stable rounds are possible and others, where way more than 100 rounds without any reboots is possible. There is noting you can do about it, beside testing different hardware. Permanent reboots Permanent reboots at the same situation during the flow has a systematic problem either in the hardware or the configuration. It usually happens during the first run as there all needed parts of the firmware have been loaded for the first time. To find the reason mostly the serial log of the UART interface from the startup until the reboots is very helpful. It can be stored using the USB / UART interface - the same as for flashing the firmware - and logging the serial output of the ESP32. Possible problems: SD card PSRAM too low Configuration missing SD card problems The ESP32CAM is a little bit \"picky\" with the supported SD cards. Due to the limited availability of GPIOs the SD card can only be accessed via 1-wire mode. Therefore not all SD cards are supported. Several error cases can happen: No SD card Easy to detect: fast blinking red LED directly after startup, no reaction of the web server etc. at all SD card not supported at all Error message of no detectable SC card in the log file. Normal looking log for a 16GB SD card is like this: 09:38:25.037 -> \u001b[0;32mI (4789) main: Using SDMMC peripheral\u001b[0m 09:38:25.037 -> \u001b[0;32mI (4789) main: Using SDMMC peripheral\u001b[0m 09:38:25.138 -> Name: SC16G 09:38:25.138 -> Type: SDHC/SDXC 09:38:25.138 -> Speed: 20 MHz 09:38:25.138 -> Size: 15193MB Otherwise there is some error message. SD card recognized but not supported This is the most annoying error. The SD card is detected, but the files cannot be read. Most probably this results in a problem with the WLAN connection, as the first file needed is the wlan.ini in the root directory. PSRAM too low In order to work, there are 4 MB of PSRAM necessary. Normaly the ESP32CAM is equiped with 8 MB, whereof only 4 MB can be used effectively. Sometimes, there is hardware, where only 2 MB of PSRAM is present - even if you have bought a 8 MB module You can identify the amount of PSRAM in the serial log file: 09:38:21.224 -> \u001b[0;32mI (881) psram: This chip is ESP32-D0WD\u001b[0m 09:38:21.224 -> \u001b[0;32mI (885) spiram: Found 64MBit SPI RAM device\u001b[0m 09:38:21.224 -> \u001b[0;32mI (890) spiram: SPI RAM mode: flash 40m sram 40m\u001b[0m 09:38:21.224 -> \u001b[0;32mI (895) spiram: PSRAM initialized, cache is in low/high (2-core) mode.\u001b[0m Here you see 64MBit (= 8MByte) - which is okay. False reading would be: 16MBit The error in the SD log file is typically related with the taking of the image (tbd) as the first time, the system is running out of memory is usually, when it tries to transfer an image from the camera to the PSRAM. There is nothing to do, than to buy a new ESP32CAM with really 64MBit of PSRAM. Configuration missing There are several files needed during on run cycle. If one of this is missing, the firmware is missing information and tends to reboot due to missing error management: /wlan.ini /config/config.ini /config/XXXXX.tflite (1 time for analog and 1 time for digital) where XXXXX is the file name, that is written in the config.ini","title":"Frequent Reboots"},{"location":"Frequent-Reboots/#frequent-reboots","text":"There are several types of reboots. To get a deeper insight turn on the logging: Internal logging ( config.ini ) Serial log of the UART interface (same as for flashing the firmware) There are two principle types of reboots Random reboots (always different timing and situation) Permanent Reboots always at the same time","title":"Frequent Reboots"},{"location":"Frequent-Reboots/#random-reboots","text":"Random reboots have two reasons: overload during HTML access and unstable system In general: there are several mechanisms in the firmware (like saving previous values), to have a \"smooth\" reboot without too many notable disturbance.","title":"Random reboots"},{"location":"Frequent-Reboots/#overload-during-html-access","text":"If you frequently access the web server over HTML requests, the firmware tends to reboot. This especially happens during the first run and when the ESP32 is busy with the digitization flow. The reason for this are running out of memory during a flow, minor memory leakage in combination with missing error handling. There is noting you can do about this kind of reboot, beside two thing: Support the firmware development with improved and tested part of code Be patient :-)","title":"Overload during HTML access"},{"location":"Frequent-Reboots/#unstable-system","text":"If your system is sometimes running smoothly over several runs and sometimes reboots obviously randomly, you have an partially unstable device. You can check this in the standard log file on the SD card: 2021-12-26T06:34:09: task_autodoFlow - round done 2021-12-26T06:34:09: CPU Temperature: 56.1 2021-12-26T06:38:00: task_autodoFlow - next round - Round #23 Here you see, that the round #23 is starting, so obviously there were no reboots in the last 22 rounds. There is hardware (ESP32CAM), where only 2-3 stable rounds are possible and others, where way more than 100 rounds without any reboots is possible. There is noting you can do about it, beside testing different hardware.","title":"Unstable system"},{"location":"Frequent-Reboots/#permanent-reboots","text":"Permanent reboots at the same situation during the flow has a systematic problem either in the hardware or the configuration. It usually happens during the first run as there all needed parts of the firmware have been loaded for the first time. To find the reason mostly the serial log of the UART interface from the startup until the reboots is very helpful. It can be stored using the USB / UART interface - the same as for flashing the firmware - and logging the serial output of the ESP32. Possible problems: SD card PSRAM too low Configuration missing","title":"Permanent reboots"},{"location":"Frequent-Reboots/#sd-card-problems","text":"The ESP32CAM is a little bit \"picky\" with the supported SD cards. Due to the limited availability of GPIOs the SD card can only be accessed via 1-wire mode. Therefore not all SD cards are supported. Several error cases can happen:","title":"SD card problems"},{"location":"Frequent-Reboots/#no-sd-card","text":"Easy to detect: fast blinking red LED directly after startup, no reaction of the web server etc. at all","title":"No SD card"},{"location":"Frequent-Reboots/#sd-card-not-supported-at-all","text":"Error message of no detectable SC card in the log file. Normal looking log for a 16GB SD card is like this: 09:38:25.037 -> \u001b[0;32mI (4789) main: Using SDMMC peripheral\u001b[0m 09:38:25.037 -> \u001b[0;32mI (4789) main: Using SDMMC peripheral\u001b[0m 09:38:25.138 -> Name: SC16G 09:38:25.138 -> Type: SDHC/SDXC 09:38:25.138 -> Speed: 20 MHz 09:38:25.138 -> Size: 15193MB Otherwise there is some error message.","title":"SD card not supported at all"},{"location":"Frequent-Reboots/#sd-card-recognized-but-not-supported","text":"This is the most annoying error. The SD card is detected, but the files cannot be read. Most probably this results in a problem with the WLAN connection, as the first file needed is the wlan.ini in the root directory.","title":"SD card recognized but not supported"},{"location":"Frequent-Reboots/#psram-too-low","text":"In order to work, there are 4 MB of PSRAM necessary. Normaly the ESP32CAM is equiped with 8 MB, whereof only 4 MB can be used effectively. Sometimes, there is hardware, where only 2 MB of PSRAM is present - even if you have bought a 8 MB module You can identify the amount of PSRAM in the serial log file: 09:38:21.224 -> \u001b[0;32mI (881) psram: This chip is ESP32-D0WD\u001b[0m 09:38:21.224 -> \u001b[0;32mI (885) spiram: Found 64MBit SPI RAM device\u001b[0m 09:38:21.224 -> \u001b[0;32mI (890) spiram: SPI RAM mode: flash 40m sram 40m\u001b[0m 09:38:21.224 -> \u001b[0;32mI (895) spiram: PSRAM initialized, cache is in low/high (2-core) mode.\u001b[0m Here you see 64MBit (= 8MByte) - which is okay. False reading would be: 16MBit The error in the SD log file is typically related with the taking of the image (tbd) as the first time, the system is running out of memory is usually, when it tries to transfer an image from the camera to the PSRAM. There is nothing to do, than to buy a new ESP32CAM with really 64MBit of PSRAM.","title":"PSRAM too low"},{"location":"Frequent-Reboots/#configuration-missing","text":"There are several files needed during on run cycle. If one of this is missing, the firmware is missing information and tends to reboot due to missing error management: /wlan.ini /config/config.ini /config/XXXXX.tflite (1 time for analog and 1 time for digital) where XXXXX is the file name, that is written in the config.ini","title":"Configuration missing"},{"location":"Graphical-Configuration/","text":"Graphical Configuration Warning This page overlaps Configuration . They should be merged to one page! General remark: to activate the changes, currently the device needs a restart after saving the changes. partially the commands needs processing on the ESP32 device. This is not very fast - so please be patient. Access to the graphical user interface The graphical configuration mode can be reached via the \"Edit Configuration\" button in the main menue ( /index.html ): Overview function Direct edit config.ini in text editor Configuration of image alignment a. Create of reference image b. Define alignment structures Definition of ROIs for digits and analog pointers Test the settings Back to main menue (\"index.html\") 1. Edit Config.ini This is a text editor for the config.ini. Changes commited with the button on the lower left. Details see Configuration-Parameter-Details . 2a. Create Reference Image The reference image is the basis for the coordination of the ROIs. Therefore it is very important, to have a well aligned image, that is not rotated. \u203c\ufe0f Attention: Updating the reference image, also means, that all alignment images and ROIs needs to be teached again. Therefore do this step only with caution. At first the current image is shown and no adjustment is possible. To reload the actual image push the button \"Show actual Reference\" (1). To define a new reference image push the button \"Create new Reference\" (2). Then the last taken raw image from the camera is loaded. If you want to update this, you can push the button \"Make new raw image (raw.jpg)\". If you need to mirror your image (e.g. mirror before camera) you can do this by selecting \"mirror image\". After loading the mirroring (in case checked) and the prerotation angle from the config.ini are applied. Then use the rough and fine adjustment to get the image straight aligned (3). If everything is done, you can save the result with \"Update Reference Image\" (4). If you have problems with reflections, you can turn the camera in a positions, where the reflection is at a position, where no important information is. To reduce the intensity of the reflection you can also a peace of felt (\"Filz\") as diffusior at the LED. 2b. Define Alignment References The alignment references are used to realign every taken image to the reference coordinates. Therefore two alignment structures are identified and the image is shifted and rotated according to their position with the target to be in exactly the same position as the reference image. The alignment structures needs to be unique and have a good contrast. As this is the most calculation intensive process, only a field of view of 40x40 pixels around the original coordinates are scanned. This can be adjusted manually in the config.ini (Parameter: SearchFieldX / SearchFieldY ). In the upper part of the settings you can control the position and size of the selected reference image. You can define the ROI in the image directly via drag and drop with the mouse. Go to the starting point, push the left mouse button and drag your ROI. You will a red rectangle with the newly selected position. To make this active, you need to push \"Update Reference\" (2). You can change between the two reference images with the drop down box (\"Reference 0\", \"Reference 1\"). In some cases it might be useful to use a reference with a higher contrast. This can be achieved by pushing \"Enhance Contrast\" (3). The result will be calculated on the ESP32 - so be a bit patient, before you see it active. To save the modified reference to the config.ini push finally \"Save to config.ini\". 3a./3b. Define ROIs for image recognition Here the regions of interest for the digital and analog pointers are defined. As both are done identically, here as an example the digital images are shown. First of all you can define more than one number, for example in a dual meter counter. This can be done with defining a \"Number\" (1). Analog and digital ROIs belonging to the same \"Number\" are considered to be part of the same counter. As for the reference images you can change position, size and name of the ROI in the text fields or define them via drag and drop through the mouse button. You can iterate through the defined ROIs through the drop down box in the left upper area (2). To define new or delete ROIs use the corresponding button. Be careful: if you delete all ROIs, the tool will ask you to define minimum one manually in the config.ini . The order of the ROIs correspond to the position of the digit / analog pointer in the final readout number. The order can be changed with the button \"move Next\" / \"move Previous\" (3). In order to have a good recognition, the active ROI has two rectangles for alignment: The outer rectangle is the final size of the ROI More important is the inner smaller rectangle. This should tightly fit around the number itself in x- and in y-dimension. Maybe you need to unlock the aspect ratio to change x- and y-size independendly The line in the middle should go through the middle of the number (in case it is not moving in or out) To save the result push \"Save all to config.ini\" (4). \u203c\ufe0f Attention: Currently you have to reboot the ESP32 to take the changes in the config.ini to take place. This steps are running on the ESP32 directly. So be patient with the results.","title":"Graphical Configuration"},{"location":"Graphical-Configuration/#graphical-configuration","text":"Warning This page overlaps Configuration . They should be merged to one page!","title":"Graphical Configuration"},{"location":"Graphical-Configuration/#general-remark","text":"to activate the changes, currently the device needs a restart after saving the changes. partially the commands needs processing on the ESP32 device. This is not very fast - so please be patient.","title":"General remark:"},{"location":"Graphical-Configuration/#access-to-the-graphical-user-interface","text":"The graphical configuration mode can be reached via the \"Edit Configuration\" button in the main menue ( /index.html ):","title":"Access to the graphical user interface"},{"location":"Graphical-Configuration/#overview-function","text":"Direct edit config.ini in text editor Configuration of image alignment a. Create of reference image b. Define alignment structures Definition of ROIs for digits and analog pointers Test the settings Back to main menue (\"index.html\")","title":"Overview function"},{"location":"Graphical-Configuration/#1-edit-configini","text":"This is a text editor for the config.ini. Changes commited with the button on the lower left. Details see Configuration-Parameter-Details .","title":"1. Edit Config.ini"},{"location":"Graphical-Configuration/#2a-create-reference-image","text":"The reference image is the basis for the coordination of the ROIs. Therefore it is very important, to have a well aligned image, that is not rotated. \u203c\ufe0f Attention: Updating the reference image, also means, that all alignment images and ROIs needs to be teached again. Therefore do this step only with caution. At first the current image is shown and no adjustment is possible. To reload the actual image push the button \"Show actual Reference\" (1). To define a new reference image push the button \"Create new Reference\" (2). Then the last taken raw image from the camera is loaded. If you want to update this, you can push the button \"Make new raw image (raw.jpg)\". If you need to mirror your image (e.g. mirror before camera) you can do this by selecting \"mirror image\". After loading the mirroring (in case checked) and the prerotation angle from the config.ini are applied. Then use the rough and fine adjustment to get the image straight aligned (3). If everything is done, you can save the result with \"Update Reference Image\" (4). If you have problems with reflections, you can turn the camera in a positions, where the reflection is at a position, where no important information is. To reduce the intensity of the reflection you can also a peace of felt (\"Filz\") as diffusior at the LED.","title":"2a. Create Reference Image"},{"location":"Graphical-Configuration/#2b-define-alignment-references","text":"The alignment references are used to realign every taken image to the reference coordinates. Therefore two alignment structures are identified and the image is shifted and rotated according to their position with the target to be in exactly the same position as the reference image. The alignment structures needs to be unique and have a good contrast. As this is the most calculation intensive process, only a field of view of 40x40 pixels around the original coordinates are scanned. This can be adjusted manually in the config.ini (Parameter: SearchFieldX / SearchFieldY ). In the upper part of the settings you can control the position and size of the selected reference image. You can define the ROI in the image directly via drag and drop with the mouse. Go to the starting point, push the left mouse button and drag your ROI. You will a red rectangle with the newly selected position. To make this active, you need to push \"Update Reference\" (2). You can change between the two reference images with the drop down box (\"Reference 0\", \"Reference 1\"). In some cases it might be useful to use a reference with a higher contrast. This can be achieved by pushing \"Enhance Contrast\" (3). The result will be calculated on the ESP32 - so be a bit patient, before you see it active. To save the modified reference to the config.ini push finally \"Save to config.ini\".","title":"2b. Define Alignment References"},{"location":"Graphical-Configuration/#3a3b-define-rois-for-image-recognition","text":"Here the regions of interest for the digital and analog pointers are defined. As both are done identically, here as an example the digital images are shown. First of all you can define more than one number, for example in a dual meter counter. This can be done with defining a \"Number\" (1). Analog and digital ROIs belonging to the same \"Number\" are considered to be part of the same counter. As for the reference images you can change position, size and name of the ROI in the text fields or define them via drag and drop through the mouse button. You can iterate through the defined ROIs through the drop down box in the left upper area (2). To define new or delete ROIs use the corresponding button. Be careful: if you delete all ROIs, the tool will ask you to define minimum one manually in the config.ini . The order of the ROIs correspond to the position of the digit / analog pointer in the final readout number. The order can be changed with the button \"move Next\" / \"move Previous\" (3). In order to have a good recognition, the active ROI has two rectangles for alignment: The outer rectangle is the final size of the ROI More important is the inner smaller rectangle. This should tightly fit around the number itself in x- and in y-dimension. Maybe you need to unlock the aspect ratio to change x- and y-size independendly The line in the middle should go through the middle of the number (in case it is not moving in or out) To save the result push \"Save all to config.ini\" (4). \u203c\ufe0f Attention: Currently you have to reboot the ESP32 to take the changes in the config.ini to take place. This steps are running on the ESP32 directly. So be patient with the results.","title":"3a./3b. Define ROIs for image recognition"},{"location":"Hardware-Compatibility/","text":"Hardware Compatibility General Remark Altrough a board looks similair, it can have major differences, eg.: Processor Ram (Size! & Type) -> this Project needs at least 4MB RAM! Flashrom Camera Modules Onboard/External Antenna Quality of Components Manufacture Quality of the PCB and soldering Different Components \"Clone\" Components -> ESPxx etc. This can cause different Power Consumption, Power Requirements, compatibility issues, etc. Most manufacturers and sellers buy what's cheap today on the Asian markets. In the end, it looks like it is sometimes a trial and error approach which ESP32-CAM Module works reliable. Below you find some remarks and experiences from the community: ESP32 core itself Chip Version Image Status ESP32-D0WDQ6 (revision 1) \u2714\ufe0f PSRAM There seems to be a lot of \"fake\" chips, or maybe wrongly configured ESP32 Boards. For AP MEMORY, all \"real\" APS 64 04*3SQR Chips should work. For ESP PSRAM, all \"real\" PSRAM 64 * should work 64Mbit density = 8Mbyte PSRAM This Table is just a snapshot of chips which worked Labeling on PSRAM module Image Status IPUS / IPS640LS0 / 1815XBGN \u2714\ufe0f AP MEMORY / 64 04L-3SOR / 1040H / 110089G \u2714\ufe0f AP MEMORY / 64 04L-3SQR / 12205 / 150047G \u2714\ufe0f 8MB AP MEMORY / 64 04L-3SQR / 12208 / 150047G \u2714\ufe0f 8MB AP MEMORY / 64 04L-350R / 1120A / 130027G \u274c PSRAM not accessible AP MEMORY / 64 04L-35QR / 11208 / 130025G \u274c PSRAM not accessible AP MEMORY / 64 04L-3SQR / 13100 / 180026G \u274c PSRAM not accessible AP MEMORY / 64 04L-3SQR / 11207 / 130024G \u274c PSRAM not accessible AP MEMORY / 1604M-3SQR / 0280A / 070036G \u274c 2MB only! ESP PSRAM 64 H 462021 / 1B00286 \u2714\ufe0f ESP PSRAM 64 H 412021 /1A0039G \u2714\ufe0f (8MB) ESP PSRAM16M 302020 \u274c 2MB only! ESP PSRAM16H 202020 / 050022G \u274c 2MB only! OV2640 - Camera The experience with the camera only is based on single modules. It is well possible, that this module had a damage overall and other modules of the same type will work. Give it a try and report to me! Labeling on Flex-Connector Image Status TY-OV2 / 640-V2.0 \u2714\ufe0f DCX-OV2 / 640-V2 \u2714\ufe0f DC-26 / 40-V3 \u2714\ufe0f 3x \u274c 1x ESP32 Modules Module Image Status ESP32CAM / Different versions on the market! Especially the PSRAM is sometimes labeled wrong (Label: 4MB, Real: only 2 MB --> will not work!) \u2714\ufe0f with >=4 MB PSRAM! ESP32-S3-EYE No Flash LED, pins different used (e.g. LCD diskplay) NOT OKAY SD-Cards Due to the limited free available gpios (due to all the extensions needed like: camera, sd-card, LED-flash, ...) the sd card is connected in 1-wire mode. There are some cards, that are compatible with the esp32cam module for unknown reasons. It is observed, that smaller cards (up to 4 GB) tend to be more stable and larger cards have more problems. But quite some exceptions in the forums (4 GB cards not working, 16 G cards working like a charm). Devices known to work Modules (Old list, not up-to-date anymore): See https://github.com/jomjol/AI-on-the-edge-device/discussions/1732 for a more recent list. https://arduino-projekte.info/produkt/esp32-cam-v2-integriertem-ch340-mit-ov2640-kamera-modul/ (see https://github.com/jomjol/AI-on-the-edge-device/discussions/1041 ) https://www.amazon.de/-/en/gp/product/B0B51CQ13R https://www.reichelt.de/entwicklerboards-esp32-kamera-2mp-25--debo-cam-esp32-p266036.html?PROVID=2788&gclid=CjwKCAiAqaWdBhAvEiwAGAQlttJnV4azXWDYeaFUuNioMICh-jvxKp6Cifmcep9vvtoT2JRCDqBczRoC7Q0QAvD_BwE (27.12.2022) SD-Card Sandisk 2GB Micro SD Class 2 Sandisk 2GB AITRIP ESP32 and CAM ESP-32/CAM Amazon US - Aideepen ESP32-CAM W BT Board ESP32-CAM-MB Micro USB to Serial Port CH-340G with OV2640 2MP Camera Module Dual Mode with Amazon US - Cloudisk 5Pack 4GB Micro SD Card 4 GB MicroSD Memory Card Class6 Weak Wifi The ESP32-CAM supports an external antenna. It requires some soldering skills but can improve the connection quality. See https://randomnerdtutorials.com/esp32-cam-connect-external-antenna/","title":"Hardware Compatibility"},{"location":"Hardware-Compatibility/#hardware-compatibility","text":"","title":"Hardware Compatibility"},{"location":"Hardware-Compatibility/#general-remark","text":"Altrough a board looks similair, it can have major differences, eg.: Processor Ram (Size! & Type) -> this Project needs at least 4MB RAM! Flashrom Camera Modules Onboard/External Antenna Quality of Components Manufacture Quality of the PCB and soldering Different Components \"Clone\" Components -> ESPxx etc. This can cause different Power Consumption, Power Requirements, compatibility issues, etc. Most manufacturers and sellers buy what's cheap today on the Asian markets. In the end, it looks like it is sometimes a trial and error approach which ESP32-CAM Module works reliable. Below you find some remarks and experiences from the community:","title":"General Remark"},{"location":"Hardware-Compatibility/#esp32-core-itself","text":"Chip Version Image Status ESP32-D0WDQ6 (revision 1) \u2714\ufe0f","title":"ESP32 core itself"},{"location":"Hardware-Compatibility/#psram","text":"There seems to be a lot of \"fake\" chips, or maybe wrongly configured ESP32 Boards. For AP MEMORY, all \"real\" APS 64 04*3SQR Chips should work. For ESP PSRAM, all \"real\" PSRAM 64 * should work 64Mbit density = 8Mbyte PSRAM This Table is just a snapshot of chips which worked Labeling on PSRAM module Image Status IPUS / IPS640LS0 / 1815XBGN \u2714\ufe0f AP MEMORY / 64 04L-3SOR / 1040H / 110089G \u2714\ufe0f AP MEMORY / 64 04L-3SQR / 12205 / 150047G \u2714\ufe0f 8MB AP MEMORY / 64 04L-3SQR / 12208 / 150047G \u2714\ufe0f 8MB AP MEMORY / 64 04L-350R / 1120A / 130027G \u274c PSRAM not accessible AP MEMORY / 64 04L-35QR / 11208 / 130025G \u274c PSRAM not accessible AP MEMORY / 64 04L-3SQR / 13100 / 180026G \u274c PSRAM not accessible AP MEMORY / 64 04L-3SQR / 11207 / 130024G \u274c PSRAM not accessible AP MEMORY / 1604M-3SQR / 0280A / 070036G \u274c 2MB only! ESP PSRAM 64 H 462021 / 1B00286 \u2714\ufe0f ESP PSRAM 64 H 412021 /1A0039G \u2714\ufe0f (8MB) ESP PSRAM16M 302020 \u274c 2MB only! ESP PSRAM16H 202020 / 050022G \u274c 2MB only!","title":"PSRAM"},{"location":"Hardware-Compatibility/#ov2640-camera","text":"The experience with the camera only is based on single modules. It is well possible, that this module had a damage overall and other modules of the same type will work. Give it a try and report to me! Labeling on Flex-Connector Image Status TY-OV2 / 640-V2.0 \u2714\ufe0f DCX-OV2 / 640-V2 \u2714\ufe0f DC-26 / 40-V3 \u2714\ufe0f 3x \u274c 1x","title":"OV2640 - Camera"},{"location":"Hardware-Compatibility/#esp32-modules","text":"Module Image Status ESP32CAM / Different versions on the market! Especially the PSRAM is sometimes labeled wrong (Label: 4MB, Real: only 2 MB --> will not work!) \u2714\ufe0f with >=4 MB PSRAM! ESP32-S3-EYE No Flash LED, pins different used (e.g. LCD diskplay) NOT OKAY","title":"ESP32 Modules"},{"location":"Hardware-Compatibility/#sd-cards","text":"Due to the limited free available gpios (due to all the extensions needed like: camera, sd-card, LED-flash, ...) the sd card is connected in 1-wire mode. There are some cards, that are compatible with the esp32cam module for unknown reasons. It is observed, that smaller cards (up to 4 GB) tend to be more stable and larger cards have more problems. But quite some exceptions in the forums (4 GB cards not working, 16 G cards working like a charm).","title":"SD-Cards"},{"location":"Hardware-Compatibility/#devices-known-to-work","text":"","title":"Devices known to work"},{"location":"Hardware-Compatibility/#modules-old-list-not-up-to-date-anymore","text":"See https://github.com/jomjol/AI-on-the-edge-device/discussions/1732 for a more recent list. https://arduino-projekte.info/produkt/esp32-cam-v2-integriertem-ch340-mit-ov2640-kamera-modul/ (see https://github.com/jomjol/AI-on-the-edge-device/discussions/1041 ) https://www.amazon.de/-/en/gp/product/B0B51CQ13R https://www.reichelt.de/entwicklerboards-esp32-kamera-2mp-25--debo-cam-esp32-p266036.html?PROVID=2788&gclid=CjwKCAiAqaWdBhAvEiwAGAQlttJnV4azXWDYeaFUuNioMICh-jvxKp6Cifmcep9vvtoT2JRCDqBczRoC7Q0QAvD_BwE (27.12.2022)","title":"Modules (Old list, not up-to-date anymore):"},{"location":"Hardware-Compatibility/#sd-card","text":"Sandisk 2GB Micro SD Class 2 Sandisk 2GB AITRIP ESP32 and CAM ESP-32/CAM Amazon US - Aideepen ESP32-CAM W BT Board ESP32-CAM-MB Micro USB to Serial Port CH-340G with OV2640 2MP Camera Module Dual Mode with Amazon US - Cloudisk 5Pack 4GB Micro SD Card 4 GB MicroSD Memory Card Class6","title":"SD-Card"},{"location":"Hardware-Compatibility/#weak-wifi","text":"The ESP32-CAM supports an external antenna. It requires some soldering skills but can improve the connection quality. See https://randomnerdtutorials.com/esp32-cam-connect-external-antenna/","title":"Weak Wifi"},{"location":"Influx-DB/","text":"Influx DB The device also supports direct sending of data to an Influ DB. \u203c\ufe0f Only Influx DB 1 is supported!","title":"Influx DB"},{"location":"Influx-DB/#influx-db","text":"The device also supports direct sending of data to an Influ DB. \u203c\ufe0f Only Influx DB 1 is supported!","title":"Influx DB"},{"location":"Installation/","text":"Installation The installation requires multiple steps: Get the right hardware and wire it up Flash the firmware onto the ESP32 Write the data to the SD-Card Insert the SD-Card into the ESP32 board Power/restart it. Hardware ESP32-CAM OV2640 camera module Micro SD-Card slot 4 or 8 MB PSRAM. It can be easily found on the typical internet stores, searching for ESP32-CAM for less than 10 EUR. How ever since the hardware is cheap and coming from China, you unluckily could pick a malfunction device. See Hardware Compatibility for further advice! USB->UART interface For first time flashing the firmware a USB -> UART connector is needed. Later firmware upgrades than can be flashed via OTA. Power supply For power supply a 5V source is needed. Most easily this can be done via an USB power supply. The power supply should support minimum 500mA. For buffering current peaks some users reported to use a large elco condensator like a 2200uF between ground and VCC. \u203c\ufe0f Attention: in several internet forums there are problems reported, in case the ESP32-CAM is only supplied with 3.3V. Housing A small 3D-printable example for a very small case can be found in Thingiverse here: https://www.thingiverse.com/thing:4571627 \u203c\ufe0f Attention : the focus of the OV2640 needs to be adjusted, as it is normally set from ~40cm to infinity. In order to get an image that is big enough, it needs to be changed to about 10cm. Therefore the sealing glue on the objective ring needs to be removed with a scalpel or sharp knife. Afterwards the objective can be rotated clockwise until the image is sharp again. Wiring Beside the 5V power supply, only for the first flashing a connection to the USB-UART connector, including a short cut of GPIO0 to GND for bootloader start. A example for wiring can be found here: It is also possible to use external LEDs for the illumination instead of the internal flash LED. This is described here: [[External-LED]] Firmware flashing Files Grab the firmware from the Releases page (Stable, tested versions), or the Automatically build development branch (experimental, untested versions). Please have a look on https://github.com/jomjol/AI-on-the-edge-device/wiki/Install-a-rolling-%28unstable%29-release first! You need: partitions.bin bootloader.bin firmware.bin Flashing There are several options to flash the firmware. Here three are described: 1. Web Installer There is a Web Installer available which will work right out of the web browser Edge and Chrome. You can access it with the following link: https://jomjol.github.io/AI-on-the-edge-device This is the preferred way for beginners as it also allows access to the USB Log: 2. Using the Flash Tool from Espressif The flashing of the firmware can be done with the \"Flash Download Tool\" from espressif, that can found here Download and extract the Flash tool, after starting choose \"Developer Mode\", then \"ESP32-DownloadTool\" and you are in the setup of the flashing tool. Connect the ESP32-CAM with the USB-UART connection and identify the COM-Port. \u203c\ufe0f Attention : if you reflashing the code again, it is strongly recommended to erase the flash memory before flashing the firmware. Especially if you used OTA in between, which might cause remaining information on the flash, to still boot from an old image in the OTA-area, which is not erased by a normal flash. But your ESP32 in bootloader mode and push start, then it will identify the board and you can configure the bin-configuration according to the following table: Filename Offset bootloader.bin 0x1000 partitions.bin 0x8000 firmware.bin 0x10000 Alternatively it can be directly flashed from the development environment - here PlatformIO. But this is rather for experienced users, as the whole development chain needs to be installed for compilation. 3. Using esptool in python directly For this you need a python environment (e.g. Anaconda in Win10). Here you need to install the esptool: pip install esptool Then connect the ESP32 with the USB-UART connector to the system, put it in bootmode and with the following command you can erase the flash and flash bootloader, partitions and firmware in two steps: esptool erase_flash esptool write_flash 0x01000 bootloader.bin 0x08000 partitions.bin 0x10000 firmware.bin Maybe you need to specify the COM-port if it is not detected by default. If the erase command throws the error A fatal error occurred: ESP32 ROM does not support function erase_flash. , your esptool might be too old, see https://techoverflow.net/2022/02/08/how-to-fix-esp32-a-fatal-error-occurred-esp32-rom-does-not-support-function-erase_flash/ With some Python installations this may not work and you\u2019ll receive an error, try python -m pip install esptool or pip3 install esptool Further recommendations can be found on the espressif webpage SD-Card The software expects a SD-Card prepared with certain directory and file structure in order to work properly. For the first setup take the AI-on-the-edge-device__manual-setup__*.zip from the Release page, open the zip and extract the whole content of the in the setup file included sd-card.zip onto your SD-Card direclty to the root folder. SD-Card root should look like this: - config - demo - firmware - html - img_tmp - log - wlan.ini This initial setup needs to be done only once as further updates of the software are possible with an Over-The-Air update mechanismn. Notes Due to the limited availability of GPIOs (OV2640, Flash-Light, PSRAM & SD-Card) the communication mode to the SD card is limited to 1-line SD-Mode. It showed up, that this results in problems with very large SD-Cards (64GB, sometimes 32 GB) and some no name low cost SD-cards. There must be no partition table on the SD-card (no GPT, but only MBR for the single partition) Following setting are necessary for formating the SD-card: SINGLE PARTITION, MBR, FAT32 - 32K. NOT exFAT Some ESP32 devices share their SD-card and/or camera GPIOs with the pins for TX and RX. If you see errors like \u201cFailed to connect\u201d then your chip is probably not entering the bootloader properly. Remove the respective modules temporarily to free the GPIOs for flashing. You may find more information about troubleshooting on the homepage of Espressif . The ESP32 indicates problems with the SD card during startup with a fast not ending blinking. In this case, please try another SD card. WLAN The access to the WLAN is configured in the \"wlan.ini\" directly on the root directory of the sd-card. Just write the corresponding SSID and password before the startup of the ESP32. This file is hidden from external access (e.g. via Filemanager) to protect the password. After power on the connection status is indicated by 3x blinking of the red on board LED. WLAN-Status indication: 5 x fast blinking (< 1 second): connection still pending 3 x slow blinking (1 second on/off): WLAN connection established It is normal that at first one or two times a pending connection is indicated. Update (OTA / Over-The-Air) Update from version greater than 12.0.0 You can use the over the air update mechanism, which uploads the update via a ZIP files. The update file is located on the release page . Please choose the zip file with the following naming: AI-on-the-edge-device__update__*.zip Go to the menu System --> OTA Update and follow the instructions there. After a final automatic reboot you should have the new version running. Update from version older than 12.0.0 If you update from an version older than 12.0.1, you should firstly update to version 12.0.1. Background are not fully backward compatible changes in the config.ini , that are taken care of in this version. \u203c\ufe0f Make sure to read the instructions below carfully! Backup your configuration (use the System -> Backup/Restore page)! Upload and update the update-*.zip file from the release 12.0.1 see here . Let it restart and check on the System -> Info page that the Firmware as well as the Web UI got updated. If only one got updated, redo the update. If it fails several times, you also can update the Firmware and the Web UI separately. Safe way: Update first the firmware.bin (extract it from one of the provided zip files) and do the Reboot Update with the full zip file ( update-*.zip , ignore the version warning after the reboot) Please go to Settings -> Configuration and address the changed parameters: DataLogging (storing the values for data graph) Debug (extended by different debug reporting levels) Make sure it starts to do the digitalization (check the Error field on the overview page). If it does not start a round within a minute, restart the device. \u203c\ufe0f If the system is working now without any issues, please open the configuration editor once and save the config.ini . This will update the file to the newest content \u203c\ufe0f Now you can safely update to the newest version.","title":"Installation"},{"location":"Installation/#installation","text":"The installation requires multiple steps: Get the right hardware and wire it up Flash the firmware onto the ESP32 Write the data to the SD-Card Insert the SD-Card into the ESP32 board Power/restart it.","title":"Installation"},{"location":"Installation/#hardware","text":"","title":"Hardware"},{"location":"Installation/#esp32-cam","text":"OV2640 camera module Micro SD-Card slot 4 or 8 MB PSRAM. It can be easily found on the typical internet stores, searching for ESP32-CAM for less than 10 EUR. How ever since the hardware is cheap and coming from China, you unluckily could pick a malfunction device. See Hardware Compatibility for further advice!","title":"ESP32-CAM"},{"location":"Installation/#usb-uart-interface","text":"For first time flashing the firmware a USB -> UART connector is needed. Later firmware upgrades than can be flashed via OTA.","title":"USB-&gt;UART interface"},{"location":"Installation/#power-supply","text":"For power supply a 5V source is needed. Most easily this can be done via an USB power supply. The power supply should support minimum 500mA. For buffering current peaks some users reported to use a large elco condensator like a 2200uF between ground and VCC. \u203c\ufe0f Attention: in several internet forums there are problems reported, in case the ESP32-CAM is only supplied with 3.3V.","title":"Power supply"},{"location":"Installation/#housing","text":"A small 3D-printable example for a very small case can be found in Thingiverse here: https://www.thingiverse.com/thing:4571627 \u203c\ufe0f Attention : the focus of the OV2640 needs to be adjusted, as it is normally set from ~40cm to infinity. In order to get an image that is big enough, it needs to be changed to about 10cm. Therefore the sealing glue on the objective ring needs to be removed with a scalpel or sharp knife. Afterwards the objective can be rotated clockwise until the image is sharp again.","title":"Housing"},{"location":"Installation/#wiring","text":"Beside the 5V power supply, only for the first flashing a connection to the USB-UART connector, including a short cut of GPIO0 to GND for bootloader start. A example for wiring can be found here: It is also possible to use external LEDs for the illumination instead of the internal flash LED. This is described here: [[External-LED]]","title":"Wiring"},{"location":"Installation/#firmware-flashing","text":"","title":"Firmware flashing"},{"location":"Installation/#files","text":"Grab the firmware from the Releases page (Stable, tested versions), or the Automatically build development branch (experimental, untested versions). Please have a look on https://github.com/jomjol/AI-on-the-edge-device/wiki/Install-a-rolling-%28unstable%29-release first! You need: partitions.bin bootloader.bin firmware.bin","title":"Files"},{"location":"Installation/#flashing","text":"There are several options to flash the firmware. Here three are described:","title":"Flashing"},{"location":"Installation/#1-web-installer","text":"There is a Web Installer available which will work right out of the web browser Edge and Chrome. You can access it with the following link: https://jomjol.github.io/AI-on-the-edge-device This is the preferred way for beginners as it also allows access to the USB Log:","title":"1. Web Installer"},{"location":"Installation/#2-using-the-flash-tool-from-espressif","text":"The flashing of the firmware can be done with the \"Flash Download Tool\" from espressif, that can found here Download and extract the Flash tool, after starting choose \"Developer Mode\", then \"ESP32-DownloadTool\" and you are in the setup of the flashing tool. Connect the ESP32-CAM with the USB-UART connection and identify the COM-Port. \u203c\ufe0f Attention : if you reflashing the code again, it is strongly recommended to erase the flash memory before flashing the firmware. Especially if you used OTA in between, which might cause remaining information on the flash, to still boot from an old image in the OTA-area, which is not erased by a normal flash. But your ESP32 in bootloader mode and push start, then it will identify the board and you can configure the bin-configuration according to the following table: Filename Offset bootloader.bin 0x1000 partitions.bin 0x8000 firmware.bin 0x10000 Alternatively it can be directly flashed from the development environment - here PlatformIO. But this is rather for experienced users, as the whole development chain needs to be installed for compilation.","title":"2. Using the Flash Tool from Espressif"},{"location":"Installation/#3-using-esptool-in-python-directly","text":"For this you need a python environment (e.g. Anaconda in Win10). Here you need to install the esptool: pip install esptool Then connect the ESP32 with the USB-UART connector to the system, put it in bootmode and with the following command you can erase the flash and flash bootloader, partitions and firmware in two steps: esptool erase_flash esptool write_flash 0x01000 bootloader.bin 0x08000 partitions.bin 0x10000 firmware.bin Maybe you need to specify the COM-port if it is not detected by default. If the erase command throws the error A fatal error occurred: ESP32 ROM does not support function erase_flash. , your esptool might be too old, see https://techoverflow.net/2022/02/08/how-to-fix-esp32-a-fatal-error-occurred-esp32-rom-does-not-support-function-erase_flash/ With some Python installations this may not work and you\u2019ll receive an error, try python -m pip install esptool or pip3 install esptool Further recommendations can be found on the espressif webpage","title":"3. Using esptool in python directly"},{"location":"Installation/#sd-card","text":"The software expects a SD-Card prepared with certain directory and file structure in order to work properly. For the first setup take the AI-on-the-edge-device__manual-setup__*.zip from the Release page, open the zip and extract the whole content of the in the setup file included sd-card.zip onto your SD-Card direclty to the root folder. SD-Card root should look like this: - config - demo - firmware - html - img_tmp - log - wlan.ini This initial setup needs to be done only once as further updates of the software are possible with an Over-The-Air update mechanismn.","title":"SD-Card"},{"location":"Installation/#notes","text":"Due to the limited availability of GPIOs (OV2640, Flash-Light, PSRAM & SD-Card) the communication mode to the SD card is limited to 1-line SD-Mode. It showed up, that this results in problems with very large SD-Cards (64GB, sometimes 32 GB) and some no name low cost SD-cards. There must be no partition table on the SD-card (no GPT, but only MBR for the single partition) Following setting are necessary for formating the SD-card: SINGLE PARTITION, MBR, FAT32 - 32K. NOT exFAT Some ESP32 devices share their SD-card and/or camera GPIOs with the pins for TX and RX. If you see errors like \u201cFailed to connect\u201d then your chip is probably not entering the bootloader properly. Remove the respective modules temporarily to free the GPIOs for flashing. You may find more information about troubleshooting on the homepage of Espressif . The ESP32 indicates problems with the SD card during startup with a fast not ending blinking. In this case, please try another SD card.","title":"Notes"},{"location":"Installation/#wlan","text":"The access to the WLAN is configured in the \"wlan.ini\" directly on the root directory of the sd-card. Just write the corresponding SSID and password before the startup of the ESP32. This file is hidden from external access (e.g. via Filemanager) to protect the password. After power on the connection status is indicated by 3x blinking of the red on board LED. WLAN-Status indication: 5 x fast blinking (< 1 second): connection still pending 3 x slow blinking (1 second on/off): WLAN connection established It is normal that at first one or two times a pending connection is indicated.","title":"WLAN"},{"location":"Installation/#update-ota-over-the-air","text":"","title":"Update (OTA / Over-The-Air)"},{"location":"Installation/#update-from-version-greater-than-1200","text":"You can use the over the air update mechanism, which uploads the update via a ZIP files. The update file is located on the release page . Please choose the zip file with the following naming: AI-on-the-edge-device__update__*.zip Go to the menu System --> OTA Update and follow the instructions there. After a final automatic reboot you should have the new version running.","title":"Update from version greater than 12.0.0"},{"location":"Installation/#update-from-version-older-than-1200","text":"If you update from an version older than 12.0.1, you should firstly update to version 12.0.1. Background are not fully backward compatible changes in the config.ini , that are taken care of in this version. \u203c\ufe0f Make sure to read the instructions below carfully! Backup your configuration (use the System -> Backup/Restore page)! Upload and update the update-*.zip file from the release 12.0.1 see here . Let it restart and check on the System -> Info page that the Firmware as well as the Web UI got updated. If only one got updated, redo the update. If it fails several times, you also can update the Firmware and the Web UI separately. Safe way: Update first the firmware.bin (extract it from one of the provided zip files) and do the Reboot Update with the full zip file ( update-*.zip , ignore the version warning after the reboot) Please go to Settings -> Configuration and address the changed parameters: DataLogging (storing the values for data graph) Debug (extended by different debug reporting levels) Make sure it starts to do the digitalization (check the Error field on the overview page). If it does not start a round within a minute, restart the device. \u203c\ufe0f If the system is working now without any issues, please open the configuration editor once and save the config.ini . This will update the file to the newest content \u203c\ufe0f Now you can safely update to the newest version.","title":"Update from version older than 12.0.0"},{"location":"Integration-Home-Assistant/","text":"Integration into Home Assistant There are 3 ways to get the data into your Home Assistant: Using MQTT (Automatically Setup Entities using Homeassistant MQTT Discovery) Using MQTT (Manually Setup Entities) Using REST calls The first one is the easier way if you already have MQTT in use. Using MQTT (Automatically Setup Entities using Homeassistant MQTT Discovery) \u203c\ufe0f This feature will be available with the next release! Starting with Version >12.0.1 , AI-on-the-edge-devices support Homeassistant Discovery. Check here to learn more about it and how to enable it in Homeassistant. You also have to enable it in the MQTT settings of your device: Make sure to select the right Meter Type to get the right units! On the next start of the device, it will send discovery topics and Homeassistant should pick them up and show them under Settings > Integrations > MQTT : Using MQTT (Manually Setup Entities) First make sure with an MQTT client (for example MQTT Explorer ) that MQTT works as expected and to get a list of the available topics! Then add a sensor for each property: mqtt: sensor: - state_topic: \"wasserzaehler/main/value\" name: \"Watermeter Value\" unique_id: watermeter_value unit_of_measurement: 'm\u00b3' state_class: total_increasing device_class: water # Needs Homeassistant 2022.11! icon: 'mdi:water-pump' availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost - state_topic: \"wasserzaehler/main/rate\" name: \"Watermeter Rate\" unique_id: watermeter_rate unit_of_measurement: 'm\u00b3/min' state_class: measurement device_class: water # Needs Homeassistant 2022.11! icon: 'mdi:water-pump' availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost - state_topic: \"wasserzaehler/main/error\" name: \"Watermeter Error\" unique_id: watermeter_error icon: \"mdi:water-alert\" availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost - state_topic: \"wasserzaehler/uptime\" name: \"Watermeter Uptime\" unique_id: watermeter_uptime unit_of_measurement: 's' state_class: measurement device_class: duration entity_category: diagnostic icon: \"mdi:timer-outline\" availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost If you run the discovery once, you can also extract the information from there (MQTT Info, untested): mqtt: # Extracted form the Discovery but untested! sensor: - name: Value unique_id: wasserzaehler-main_value icon: mdi:gauge state_topic: wasserzaehler/main/value unit_of_measurement: m\u00b3 device_class: water state_class: total_increasing availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost If you want to convert the m\u00b3 to l , use a template sensor: template: - sensor: - name: \"Watermeter in l\" unique_id: watermeter_in_l icon: \"mdi:gauge\" state: \"{{ states('sensor.watermeter_value')|float(default=0) * 1000 }}\" # Convert 1 m3 => 1000 l unit_of_measurement: l availability: \"{{ states('sensor.watermeter_value') not in ['unknown', 'unavailable', 'none'] }}\" If you you want to have the consumption per day, you can use an Utility Meter . it is a helper and can be used to reset the total increasing values once a day utility_meter: utility_meter_gas_per_day: source: sensor.gasmeter_value cycle: daily utility_meter_water_per_day: source: sensor.watermeter_value cycle: daily Note that you also can add it using the UI. Examples Statistics Graph Creating Statistics Graphs (eg. usage per day) is easy using the Energy Dashboard : Note that there seems to be a bug in the graph, see https://github.com/home-assistant/frontend/issues/13995 ! InfluxDb Graphs See also Influx-DB . If you have setup InfluxDB already, it is also possible to fetch statistics from there, eg. daily usage: from(bucket: \"HomeAssistant\") |> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> filter(fn: (r) => r[\"entity_id\"] == \"wasserverbrauch_tag\") |> filter(fn: (r) => r[\"_field\"] == \"value\") |> timeShift(duration: -1d) |> aggregateWindow(every: 1d, fn: max, createEmpty: false) |> yield(name: \"mean\") Using REST When using REST, Home Assistant has to periodically call an URL on the ESP32 which in return provides the requested data. See REST API for a list of available URLs. The most practical one is the json entrypoint which provides the most relevant data JSON formated: http://<IP>/json This would return: { \"main\": { \"value\": \"512.3020\", \"raw\": \"0512.3020\", \"error\": \"no error\", \"rate\": 0.000000, \"timestamp\": \"2022-10-02T20:32:06\" [..] } } To do such a REST call, you need to create a REST sensor: sensor: - platform: rest name: \"Gasmeter JSON\" resource: http://<IP>/json json_attributes: - main value_template: '{{ value_json.value }}' headers: Content-Type: application/json scan_interval: 60 template: sensor: - name: \"Gasmeter Value from JSON\" unique_id: gas_meter_value_from_json state: \"{{ state_attr('sensor.gasmeter_json','main')['value'] }}\" unit_of_measurement: 'm\u00b3' - name: \"Watermeter Value from JSON\" unique_id: water_meter_value_from_json state: >- {{ state_attr('sensor.watermeter_json','main')['value'] | float }} unit_of_measurement: 'm\u00b3' device_class: water state_class: total_increasing icon: mdi:gauge See also https://community.home-assistant.io/t/rest-sensor-nested-json/243420/9 Photo REST can also be used to show the photo of the last round: To access it, use http://<IP>/img_tmp/alg_roi.jpg resp http://<IP>/img_tmp/raw.jpg .","title":"Integration into Home Assistant"},{"location":"Integration-Home-Assistant/#integration-into-home-assistant","text":"There are 3 ways to get the data into your Home Assistant: Using MQTT (Automatically Setup Entities using Homeassistant MQTT Discovery) Using MQTT (Manually Setup Entities) Using REST calls The first one is the easier way if you already have MQTT in use.","title":"Integration into Home Assistant"},{"location":"Integration-Home-Assistant/#using-mqtt-automatically-setup-entities-using-homeassistant-mqtt-discovery","text":"\u203c\ufe0f This feature will be available with the next release! Starting with Version >12.0.1 , AI-on-the-edge-devices support Homeassistant Discovery. Check here to learn more about it and how to enable it in Homeassistant. You also have to enable it in the MQTT settings of your device: Make sure to select the right Meter Type to get the right units! On the next start of the device, it will send discovery topics and Homeassistant should pick them up and show them under Settings > Integrations > MQTT :","title":"Using MQTT (Automatically Setup Entities using Homeassistant MQTT Discovery)"},{"location":"Integration-Home-Assistant/#using-mqtt-manually-setup-entities","text":"First make sure with an MQTT client (for example MQTT Explorer ) that MQTT works as expected and to get a list of the available topics! Then add a sensor for each property: mqtt: sensor: - state_topic: \"wasserzaehler/main/value\" name: \"Watermeter Value\" unique_id: watermeter_value unit_of_measurement: 'm\u00b3' state_class: total_increasing device_class: water # Needs Homeassistant 2022.11! icon: 'mdi:water-pump' availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost - state_topic: \"wasserzaehler/main/rate\" name: \"Watermeter Rate\" unique_id: watermeter_rate unit_of_measurement: 'm\u00b3/min' state_class: measurement device_class: water # Needs Homeassistant 2022.11! icon: 'mdi:water-pump' availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost - state_topic: \"wasserzaehler/main/error\" name: \"Watermeter Error\" unique_id: watermeter_error icon: \"mdi:water-alert\" availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost - state_topic: \"wasserzaehler/uptime\" name: \"Watermeter Uptime\" unique_id: watermeter_uptime unit_of_measurement: 's' state_class: measurement device_class: duration entity_category: diagnostic icon: \"mdi:timer-outline\" availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost If you run the discovery once, you can also extract the information from there (MQTT Info, untested): mqtt: # Extracted form the Discovery but untested! sensor: - name: Value unique_id: wasserzaehler-main_value icon: mdi:gauge state_topic: wasserzaehler/main/value unit_of_measurement: m\u00b3 device_class: water state_class: total_increasing availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost If you want to convert the m\u00b3 to l , use a template sensor: template: - sensor: - name: \"Watermeter in l\" unique_id: watermeter_in_l icon: \"mdi:gauge\" state: \"{{ states('sensor.watermeter_value')|float(default=0) * 1000 }}\" # Convert 1 m3 => 1000 l unit_of_measurement: l availability: \"{{ states('sensor.watermeter_value') not in ['unknown', 'unavailable', 'none'] }}\" If you you want to have the consumption per day, you can use an Utility Meter . it is a helper and can be used to reset the total increasing values once a day utility_meter: utility_meter_gas_per_day: source: sensor.gasmeter_value cycle: daily utility_meter_water_per_day: source: sensor.watermeter_value cycle: daily Note that you also can add it using the UI.","title":"Using MQTT (Manually Setup Entities)"},{"location":"Integration-Home-Assistant/#examples","text":"","title":"Examples"},{"location":"Integration-Home-Assistant/#statistics-graph","text":"Creating Statistics Graphs (eg. usage per day) is easy using the Energy Dashboard : Note that there seems to be a bug in the graph, see https://github.com/home-assistant/frontend/issues/13995 !","title":"Statistics Graph"},{"location":"Integration-Home-Assistant/#influxdb-graphs","text":"See also Influx-DB . If you have setup InfluxDB already, it is also possible to fetch statistics from there, eg. daily usage: from(bucket: \"HomeAssistant\") |> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> filter(fn: (r) => r[\"entity_id\"] == \"wasserverbrauch_tag\") |> filter(fn: (r) => r[\"_field\"] == \"value\") |> timeShift(duration: -1d) |> aggregateWindow(every: 1d, fn: max, createEmpty: false) |> yield(name: \"mean\")","title":"InfluxDb Graphs"},{"location":"Integration-Home-Assistant/#using-rest","text":"When using REST, Home Assistant has to periodically call an URL on the ESP32 which in return provides the requested data. See REST API for a list of available URLs. The most practical one is the json entrypoint which provides the most relevant data JSON formated: http://<IP>/json This would return: { \"main\": { \"value\": \"512.3020\", \"raw\": \"0512.3020\", \"error\": \"no error\", \"rate\": 0.000000, \"timestamp\": \"2022-10-02T20:32:06\" [..] } } To do such a REST call, you need to create a REST sensor: sensor: - platform: rest name: \"Gasmeter JSON\" resource: http://<IP>/json json_attributes: - main value_template: '{{ value_json.value }}' headers: Content-Type: application/json scan_interval: 60 template: sensor: - name: \"Gasmeter Value from JSON\" unique_id: gas_meter_value_from_json state: \"{{ state_attr('sensor.gasmeter_json','main')['value'] }}\" unit_of_measurement: 'm\u00b3' - name: \"Watermeter Value from JSON\" unique_id: water_meter_value_from_json state: >- {{ state_attr('sensor.watermeter_json','main')['value'] | float }} unit_of_measurement: 'm\u00b3' device_class: water state_class: total_increasing icon: mdi:gauge See also https://community.home-assistant.io/t/rest-sensor-nested-json/243420/9","title":"Using REST"},{"location":"Integration-Home-Assistant/#photo","text":"REST can also be used to show the photo of the last round: To access it, use http://<IP>/img_tmp/alg_roi.jpg resp http://<IP>/img_tmp/raw.jpg .","title":"Photo"},{"location":"Learn-models-with-your-own-images/","text":"Learn a model with your own images If your device has new, different digits and the existing models don't recognize them well, you can collect your own images and train the model. Of course, this also applies if you have a model that is already known, but the neural models do not produce good results. But before you do this, you please check if your type really is not contained yet in the training data, see digits resp. pointers for an overview of images used for the training. But we also like to take pictures of already known devices. Before you start Poor recognition is often caused by blurred images, low contrast or incorrect setting of the ROIs. Therefore, check these possibilities first, as additional training will bring little improvement here. You must not train the models by yourself (see section Share your images ). In most cases we will integrate you images in the training dataset of the models. Only if we fear a degradation of the models or you need a different behaviour, we will not include the data in the standard models. Collecting images The neural network is trained on base of a set of images, that have been collected over time. If your digits are included or at least very similar to included images, the chance is very high that the neural network is working fine for you as well. The neural network configuration is stored in the TensorFlow Lite format as filename.tfl or filename.tflite in the /config directory. It can be updated by uploading the new file and activating it on the configuration page or in the config file /config/config.ini . In order to incorporate new digits a training set of images is required. The training images needs to be collected in the final setup with the help of the Digits or Analog log settings (not to be confused with the Data or Debug log). Enable the logging of the images on the configuration page or in the config file /config/config.ini : Now wait, until you have an image of each digit of every type on the SD card. Ideally remove the SD card from the camera and search for two to three images of each digit ( not more! :-) ). The format can be jpg. But don't panic, the collector programmes will sort out duplicate images for you. Collecting images for dig-class100/dig-cont/ana-class100 Collectmeterdigits and collectmeteranalog helps you to collect the images easily. Read the project readme for detailed instructions. Train the model For training the model you will need a python and Jupyter installation. All current labeled images you can find under ziffer_sortiert_raw dig-class11 models (digits) Fork and checkout neural-network-digital-counter-readout . Install all requirements for running the notebooks. pip install -r requirements.txt Put your labeled images into /ziffer_sortiert_raw folder and run Image_Preparation.ipynb Train_CNN_Digital-Readout-Small-v2.ipynb It creates a dig-class11_xxxx_s2.tflite model, you can upload to the config folder on your device and test it. dig-class100 / dig-cont models (digits) Fork and checkout neural-network-analog-needle-readout . All labeled images you can find under Images Install all requirements for running the notebooks. pip install -r requirements.txt Put your labeled images into images/collected/<typeofdevice>/<your_short>/ Run dig-class100-s2.ipynb . The model to upload to your device you can find under '/output'. ana-class100/ana-cont models (analog pointers) Fork and checkout neural-network-analog-needle-readout . All labeled images you can find under data_raw_all Install all requirements for running the notebooks. pip install -r requirements.txt Put your labeled images into images/collected/<typeofdevice>/<your_short>/ After every adding of images you need to run Image_Preparation.ipynb before you train the models. Run Train_CNN_Analog-Readout_100-Small1_Dropout.ipynb and/or Train_CNN_Analog-Readout_Version-Small2.ipynb . The model to upload to your device you can find in the project folder. Share your images If the results are good you can share the images as pull-request. Please images only! If you not able to create a pull request or don't know what it is, open an issue and put the zipped images in it. Images can be rejected if As same as dig-class11 collected, more than 1000 images of your device are really to much. images are not good configured (ROIs) will be rejected. It reduces the accuracy of the networks. Images with too little focus will be rejected. Images with too much blur are rejected. Our models are to small to recognize everything in any quality. So we use only images of medium or good quality.","title":"Learn a model with your own images"},{"location":"Learn-models-with-your-own-images/#learn-a-model-with-your-own-images","text":"If your device has new, different digits and the existing models don't recognize them well, you can collect your own images and train the model. Of course, this also applies if you have a model that is already known, but the neural models do not produce good results. But before you do this, you please check if your type really is not contained yet in the training data, see digits resp. pointers for an overview of images used for the training. But we also like to take pictures of already known devices.","title":"Learn a model with your own images"},{"location":"Learn-models-with-your-own-images/#before-you-start","text":"Poor recognition is often caused by blurred images, low contrast or incorrect setting of the ROIs. Therefore, check these possibilities first, as additional training will bring little improvement here. You must not train the models by yourself (see section Share your images ). In most cases we will integrate you images in the training dataset of the models. Only if we fear a degradation of the models or you need a different behaviour, we will not include the data in the standard models.","title":"Before you start"},{"location":"Learn-models-with-your-own-images/#collecting-images","text":"The neural network is trained on base of a set of images, that have been collected over time. If your digits are included or at least very similar to included images, the chance is very high that the neural network is working fine for you as well. The neural network configuration is stored in the TensorFlow Lite format as filename.tfl or filename.tflite in the /config directory. It can be updated by uploading the new file and activating it on the configuration page or in the config file /config/config.ini . In order to incorporate new digits a training set of images is required. The training images needs to be collected in the final setup with the help of the Digits or Analog log settings (not to be confused with the Data or Debug log). Enable the logging of the images on the configuration page or in the config file /config/config.ini : Now wait, until you have an image of each digit of every type on the SD card. Ideally remove the SD card from the camera and search for two to three images of each digit ( not more! :-) ). The format can be jpg. But don't panic, the collector programmes will sort out duplicate images for you.","title":"Collecting images"},{"location":"Learn-models-with-your-own-images/#collecting-images-for-dig-class100dig-contana-class100","text":"Collectmeterdigits and collectmeteranalog helps you to collect the images easily. Read the project readme for detailed instructions.","title":"Collecting images for dig-class100/dig-cont/ana-class100"},{"location":"Learn-models-with-your-own-images/#train-the-model","text":"For training the model you will need a python and Jupyter installation. All current labeled images you can find under ziffer_sortiert_raw","title":"Train the model"},{"location":"Learn-models-with-your-own-images/#dig-class11-models-digits","text":"Fork and checkout neural-network-digital-counter-readout . Install all requirements for running the notebooks. pip install -r requirements.txt Put your labeled images into /ziffer_sortiert_raw folder and run Image_Preparation.ipynb Train_CNN_Digital-Readout-Small-v2.ipynb It creates a dig-class11_xxxx_s2.tflite model, you can upload to the config folder on your device and test it.","title":"dig-class11 models (digits)"},{"location":"Learn-models-with-your-own-images/#dig-class100-dig-cont-models-digits","text":"Fork and checkout neural-network-analog-needle-readout . All labeled images you can find under Images Install all requirements for running the notebooks. pip install -r requirements.txt Put your labeled images into images/collected/<typeofdevice>/<your_short>/ Run dig-class100-s2.ipynb . The model to upload to your device you can find under '/output'.","title":"dig-class100 / dig-cont models (digits)"},{"location":"Learn-models-with-your-own-images/#ana-class100ana-cont-models-analog-pointers","text":"Fork and checkout neural-network-analog-needle-readout . All labeled images you can find under data_raw_all Install all requirements for running the notebooks. pip install -r requirements.txt Put your labeled images into images/collected/<typeofdevice>/<your_short>/ After every adding of images you need to run Image_Preparation.ipynb before you train the models. Run Train_CNN_Analog-Readout_100-Small1_Dropout.ipynb and/or Train_CNN_Analog-Readout_Version-Small2.ipynb . The model to upload to your device you can find in the project folder.","title":"ana-class100/ana-cont models (analog pointers)"},{"location":"Learn-models-with-your-own-images/#share-your-images","text":"If the results are good you can share the images as pull-request. Please images only! If you not able to create a pull request or don't know what it is, open an issue and put the zipped images in it.","title":"Share your images"},{"location":"Learn-models-with-your-own-images/#images-can-be-rejected-if","text":"As same as dig-class11 collected, more than 1000 images of your device are really to much. images are not good configured (ROIs) will be rejected. It reduces the accuracy of the networks. Images with too little focus will be rejected. Images with too much blur are rejected. Our models are to small to recognize everything in any quality. So we use only images of medium or good quality.","title":"Images can be rejected if"},{"location":"MQTT-API/","text":"MQTT API The device is capable to register to a MQTT broker to publish data and subscribe to specific topics. The MQTT service has to be enabled and configured properly in the device configuration via web interface ( Settings -> Configuration -> section MQTT ) The following parameters have to be defined: * URI * MainTopic (optional, if not set, the hostname is used) * ClientID (optional, if not set, AIOTED- + the MAC address gets used to make sure the ID is unique) * User (optional) * Password (optional) * RetainFlag (optional) Published topics Status MainTopic /{status topic}, e.g. watermeter/status * #### Connection Interval MAC IP Hostname Uptime FreeMem WifiRSSI CPUTemp Status Result MainTopic /{NumberName}/{result topic}, e.g. watermeter/main/value Value Raw Error JSON Rate Rate_per_time_unit The time Unit gets set with the Homeassistant Discovery, eg. h or m (minutes) Rate_per_digitalization_round The interval defines when the next round gets triggered Changeabsolut Timestamp JSON All relevant results in JSON syntax GPIO MainTopic /{GPIO topic}, e.g. watermeter/GPIO/GPIO12 GPIO/GPIO{PinNumber} Depending on device configuration ( Settings --> Configuration --> Chapter GPIO ) Subscibed topics MainTopic /{subscribed topic}, e.g. watermeter/ctrl/flow_start Control ctrl/flow_start Trigger a flow start by publishing to this topic (any character, length > 0) GPIO/GPIO{PinNumber} Depending on device configuration ( Settings --> Configuration --> Chapter GPIO )","title":"MQTT API"},{"location":"MQTT-API/#mqtt-api","text":"The device is capable to register to a MQTT broker to publish data and subscribe to specific topics. The MQTT service has to be enabled and configured properly in the device configuration via web interface ( Settings -> Configuration -> section MQTT ) The following parameters have to be defined: * URI * MainTopic (optional, if not set, the hostname is used) * ClientID (optional, if not set, AIOTED- + the MAC address gets used to make sure the ID is unique) * User (optional) * Password (optional) * RetainFlag (optional)","title":"MQTT API"},{"location":"MQTT-API/#published-topics","text":"","title":"Published topics"},{"location":"MQTT-API/#status","text":"MainTopic /{status topic}, e.g. watermeter/status * #### Connection","title":"Status"},{"location":"MQTT-API/#interval","text":"","title":"Interval"},{"location":"MQTT-API/#mac","text":"","title":"MAC"},{"location":"MQTT-API/#ip","text":"","title":"IP"},{"location":"MQTT-API/#hostname","text":"","title":"Hostname"},{"location":"MQTT-API/#uptime","text":"","title":"Uptime"},{"location":"MQTT-API/#freemem","text":"","title":"FreeMem"},{"location":"MQTT-API/#wifirssi","text":"","title":"WifiRSSI"},{"location":"MQTT-API/#cputemp","text":"","title":"CPUTemp"},{"location":"MQTT-API/#status_1","text":"","title":"Status"},{"location":"MQTT-API/#result","text":"MainTopic /{NumberName}/{result topic}, e.g. watermeter/main/value","title":"Result"},{"location":"MQTT-API/#value","text":"","title":"Value"},{"location":"MQTT-API/#raw","text":"","title":"Raw"},{"location":"MQTT-API/#error","text":"","title":"Error"},{"location":"MQTT-API/#json","text":"","title":"JSON"},{"location":"MQTT-API/#rate","text":"","title":"Rate"},{"location":"MQTT-API/#rate_per_time_unit","text":"The time Unit gets set with the Homeassistant Discovery, eg. h or m (minutes)","title":"Rate_per_time_unit"},{"location":"MQTT-API/#rate_per_digitalization_round","text":"The interval defines when the next round gets triggered","title":"Rate_per_digitalization_round"},{"location":"MQTT-API/#changeabsolut","text":"","title":"Changeabsolut"},{"location":"MQTT-API/#timestamp","text":"","title":"Timestamp"},{"location":"MQTT-API/#json_1","text":"All relevant results in JSON syntax","title":"JSON"},{"location":"MQTT-API/#gpio","text":"MainTopic /{GPIO topic}, e.g. watermeter/GPIO/GPIO12","title":"GPIO"},{"location":"MQTT-API/#gpiogpiopinnumber","text":"Depending on device configuration ( Settings --> Configuration --> Chapter GPIO )","title":"GPIO/GPIO{PinNumber}"},{"location":"MQTT-API/#subscibed-topics","text":"MainTopic /{subscribed topic}, e.g. watermeter/ctrl/flow_start","title":"Subscibed topics"},{"location":"MQTT-API/#control","text":"","title":"Control"},{"location":"MQTT-API/#ctrlflow_start","text":"Trigger a flow start by publishing to this topic (any character, length > 0)","title":"ctrl/flow_start"},{"location":"MQTT-API/#gpiogpiopinnumber_1","text":"Depending on device configuration ( Settings --> Configuration --> Chapter GPIO )","title":"GPIO/GPIO{PinNumber}"},{"location":"Neural-Network-Types/","text":"Neural Network Types Warning This page overlaps Choosing the Model . They should be merged to one page! This section is describing the different types of neural networks, that are used with the AI-on-the-edge approach and gives an introduction on how and where to use them. Overview neural network type There are two types of input : digits with rolling number (top town) analog pointers (clockwise rotating pointer) There are two types of neural networks : classification networks with discrete output neurons for each result class: 11 classes for digits (0, 1, ... 8, 9 + \"Not-A-Number\") 100 classes for digits or analog pointers (0.1, 0.2, 0.3, ... , 9.7, 9.8, 9.9) continuous output networks with a continuous output in the interval [0, 10[ No setting of the type in the firmware is necessary. The type can detect by the output structure automatically. \u203c\ufe0f Attention: It is very important to choose the right network type (digits or analog pointers). Technically a wrong network will work and create output, but that would be totally arbitrary Not all type of pointers are trained in all networks. For the 11 classes digits network there many different types of digits trained. The reason is, that you 1) only need 20-30 training images and 2) the data collection is ongoing much longer For the continious and 100 classes network especially for the digits, there are only a few types of digits trained up to now Therefore sometimes for the digits it is more effective to choose the simpler 11 classes network type (= default). Naming convention Classification 11 classes 0, 1, ... 9 + \"N\" Classification 100 classes 0.0, 0.1, ... 9.9 Continuous Interval [0, 10[ Digits dig-class11 _XXX.tflite dig-class100 _XXX.tflite dig-cont _XXX.tflite Analog Pointers ana-class100 _XXX.tflite ana-cont _XXX.tflite XXX contains the versioning and a parameter for different sizes with the following naming: XXX = versioning_sY versioning = version or in newer networks the training data Y = Neural network size (typically s1, s2, ..., s4). Whereas s1 is the maximum sized neural network and s4 is the smallest. Optional the naming ends with an \"_q\" to signal, that the tflite file has been quantized (size reduction with minimum accuracy loss). Example: dig-class11_1410_s2_q.tflite Classification network for digits with 11 classes (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, N) Version 1410 = 14.1.0 s2 = Size 2 (Medium) q = Quantized Version Overview of trained types and details Analog Pointer (\"ana-cont_XXX.tflite\" & \"ana-class100_XXX.tflite\") This is to transfer the direction of a pointer into a continuous number between 0 and 1, whereas 0 (=1) is the upwards position (12 o'clock), 0.25 corresponds to the 3 o'clock positions and so on. This network is a envolop for all different types of pointers. Currently there are no dedicated network trainings for specific types of pointers. There are two types of network structure, currently both are supported. The \"class100\" is a pure classification network, that might need a bit more accuracy in the labeling. \"cont\" is a no classic approach with a continuous output off only 2 neurons (details see below). Types of counters trained: Training data needs Quadratic images, minimum size: 32x32 pixel Typically 100 - 200 images with a resultion of 1/100 of the full rotation (every 0.1 value or 3.6\u00b0) Naming: x.y_ARBITRARY.jpg, where x.y = value 0.0 ... 9.9 CNN Technical details: Input 32 x 32 RGB images Output ana-cont _XXX.tflite: 2 neurons with output in range [-1, 1] - representing a sinus / cosinus encoding of the angle needs to be converted to angle with arctan-hyperbolicus function ana-class100 _XXX.tflite 100 neurons representing the classes from 0.0, 0.1, ... 9.8, 9.9 Digits with 11 classes (\"dig-class11_XXX.tflite\") The digit type is a classical classification network, with 11 classes representing the numbers 0, 1, ... 9 and the special class \"N\". It is trained for the rolling ring of gas and electric meters. As there is sometime a status between two images, the special class \"N\" is representing Not-A-Number for the case, that the image cannot be unique classified to one number e.g. because it is between two digits. For this type the lowest amount of training data per type is needed, resulting in a large variety of type being already part of the training set. Types of counters trained: Training data needs RGB images, with minimum size: 20x32 pixel Typically 10 - 20 images (1-2 for each digit and an arbitrary number for the \"N\" class Naming: x_ARBITRARY.jpg, where x = value 0 ... 9 + N CNN Technical details: Input 20 x 32 RGB images Output 11 neurons for image classification (last layer normalized to 1) Neuron 0 to 9 represent the corresponding numbers \"0\" to \"9\" Neron 10 represent the \"Not-A-Number\" class, telling, that the image is not uniquely classified Digits with rolling results (\"dig-class100_XXX.tflite\" & \"dig-cont_XXX.tflite\") This type of network tries to overcome the problem, that there are intermediate values, when a rolling digit is between two numbers. Previous this was the \"N\" class. In this network type, there are also subdigit values trained, so that the intermediate state can be used as additional information for the algorithms. Types of counters trained: [[images/dig-cont/dig-cont_1.jpg) [[images/dig-cont/dig-cont_2a.jpg) [[images/dig-cont/dig-cont_2b.jpg) [[images/dig-cont/dig-cont_3a.jpg) [[images/dig-cont/dig-cont_3b.jpg) [[images/dig-cont/dig-cont_3c.jpg) Training data needs RGB images, with minimum size: 20x32 pixel Typically 100 - 200 images (1-2 for each possible position) Naming: x.y_ARBITRARY.jpg, where x.y = 0.0, 0.1, ... 9.9 representing the intermediate state CNN Technical details: Input 20 x 32 RGB images Output dig-cont _XXX.tflite: 10 neurons representing the digits 0, 1, ... 9. The intermediate values are represented by weighted normalized values of two neighboring output neurons needs to be converted to angle with arctan-hyperbolicus function dig-class100 _XXX.tflite 100 neurons representing the classes from 0.0, 0.1, ... 9.8, 9.9","title":"Neural Network Types"},{"location":"Neural-Network-Types/#neural-network-types","text":"Warning This page overlaps Choosing the Model . They should be merged to one page! This section is describing the different types of neural networks, that are used with the AI-on-the-edge approach and gives an introduction on how and where to use them.","title":"Neural Network Types"},{"location":"Neural-Network-Types/#overview-neural-network-type","text":"There are two types of input : digits with rolling number (top town) analog pointers (clockwise rotating pointer) There are two types of neural networks : classification networks with discrete output neurons for each result class: 11 classes for digits (0, 1, ... 8, 9 + \"Not-A-Number\") 100 classes for digits or analog pointers (0.1, 0.2, 0.3, ... , 9.7, 9.8, 9.9) continuous output networks with a continuous output in the interval [0, 10[ No setting of the type in the firmware is necessary. The type can detect by the output structure automatically. \u203c\ufe0f Attention: It is very important to choose the right network type (digits or analog pointers). Technically a wrong network will work and create output, but that would be totally arbitrary Not all type of pointers are trained in all networks. For the 11 classes digits network there many different types of digits trained. The reason is, that you 1) only need 20-30 training images and 2) the data collection is ongoing much longer For the continious and 100 classes network especially for the digits, there are only a few types of digits trained up to now Therefore sometimes for the digits it is more effective to choose the simpler 11 classes network type (= default).","title":"Overview neural network type"},{"location":"Neural-Network-Types/#naming-convention","text":"Classification 11 classes 0, 1, ... 9 + \"N\" Classification 100 classes 0.0, 0.1, ... 9.9 Continuous Interval [0, 10[ Digits dig-class11 _XXX.tflite dig-class100 _XXX.tflite dig-cont _XXX.tflite Analog Pointers ana-class100 _XXX.tflite ana-cont _XXX.tflite XXX contains the versioning and a parameter for different sizes with the following naming: XXX = versioning_sY versioning = version or in newer networks the training data Y = Neural network size (typically s1, s2, ..., s4). Whereas s1 is the maximum sized neural network and s4 is the smallest. Optional the naming ends with an \"_q\" to signal, that the tflite file has been quantized (size reduction with minimum accuracy loss). Example: dig-class11_1410_s2_q.tflite Classification network for digits with 11 classes (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, N) Version 1410 = 14.1.0 s2 = Size 2 (Medium) q = Quantized Version","title":"Naming convention"},{"location":"Neural-Network-Types/#overview-of-trained-types-and-details","text":"","title":"Overview of trained types and details"},{"location":"Neural-Network-Types/#analog-pointer-ana-cont_xxxtflite-ana-class100_xxxtflite","text":"This is to transfer the direction of a pointer into a continuous number between 0 and 1, whereas 0 (=1) is the upwards position (12 o'clock), 0.25 corresponds to the 3 o'clock positions and so on. This network is a envolop for all different types of pointers. Currently there are no dedicated network trainings for specific types of pointers. There are two types of network structure, currently both are supported. The \"class100\" is a pure classification network, that might need a bit more accuracy in the labeling. \"cont\" is a no classic approach with a continuous output off only 2 neurons (details see below).","title":"Analog Pointer (\"ana-cont_XXX.tflite\" &amp; \"ana-class100_XXX.tflite\")"},{"location":"Neural-Network-Types/#types-of-counters-trained","text":"","title":"Types of counters trained:"},{"location":"Neural-Network-Types/#training-data-needs","text":"Quadratic images, minimum size: 32x32 pixel Typically 100 - 200 images with a resultion of 1/100 of the full rotation (every 0.1 value or 3.6\u00b0) Naming: x.y_ARBITRARY.jpg, where x.y = value 0.0 ... 9.9","title":"Training data needs"},{"location":"Neural-Network-Types/#cnn-technical-details","text":"","title":"CNN Technical details:"},{"location":"Neural-Network-Types/#input","text":"32 x 32 RGB images","title":"Input"},{"location":"Neural-Network-Types/#output","text":"ana-cont _XXX.tflite: 2 neurons with output in range [-1, 1] - representing a sinus / cosinus encoding of the angle needs to be converted to angle with arctan-hyperbolicus function ana-class100 _XXX.tflite 100 neurons representing the classes from 0.0, 0.1, ... 9.8, 9.9","title":"Output"},{"location":"Neural-Network-Types/#digits-with-11-classes-dig-class11_xxxtflite","text":"The digit type is a classical classification network, with 11 classes representing the numbers 0, 1, ... 9 and the special class \"N\". It is trained for the rolling ring of gas and electric meters. As there is sometime a status between two images, the special class \"N\" is representing Not-A-Number for the case, that the image cannot be unique classified to one number e.g. because it is between two digits. For this type the lowest amount of training data per type is needed, resulting in a large variety of type being already part of the training set.","title":"Digits with 11 classes (\"dig-class11_XXX.tflite\")"},{"location":"Neural-Network-Types/#types-of-counters-trained_1","text":"","title":"Types of counters trained:"},{"location":"Neural-Network-Types/#training-data-needs_1","text":"RGB images, with minimum size: 20x32 pixel Typically 10 - 20 images (1-2 for each digit and an arbitrary number for the \"N\" class Naming: x_ARBITRARY.jpg, where x = value 0 ... 9 + N","title":"Training data needs"},{"location":"Neural-Network-Types/#cnn-technical-details_1","text":"","title":"CNN Technical details:"},{"location":"Neural-Network-Types/#input_1","text":"20 x 32 RGB images","title":"Input"},{"location":"Neural-Network-Types/#output_1","text":"11 neurons for image classification (last layer normalized to 1) Neuron 0 to 9 represent the corresponding numbers \"0\" to \"9\" Neron 10 represent the \"Not-A-Number\" class, telling, that the image is not uniquely classified","title":"Output"},{"location":"Neural-Network-Types/#digits-with-rolling-results-dig-class100_xxxtflite-dig-cont_xxxtflite","text":"This type of network tries to overcome the problem, that there are intermediate values, when a rolling digit is between two numbers. Previous this was the \"N\" class. In this network type, there are also subdigit values trained, so that the intermediate state can be used as additional information for the algorithms.","title":"Digits with rolling results (\"dig-class100_XXX.tflite\" &amp; \"dig-cont_XXX.tflite\")"},{"location":"Neural-Network-Types/#types-of-counters-trained_2","text":"[[images/dig-cont/dig-cont_1.jpg) [[images/dig-cont/dig-cont_2a.jpg) [[images/dig-cont/dig-cont_2b.jpg) [[images/dig-cont/dig-cont_3a.jpg) [[images/dig-cont/dig-cont_3b.jpg) [[images/dig-cont/dig-cont_3c.jpg)","title":"Types of counters trained:"},{"location":"Neural-Network-Types/#training-data-needs_2","text":"RGB images, with minimum size: 20x32 pixel Typically 100 - 200 images (1-2 for each possible position) Naming: x.y_ARBITRARY.jpg, where x.y = 0.0, 0.1, ... 9.9 representing the intermediate state","title":"Training data needs"},{"location":"Neural-Network-Types/#cnn-technical-details_2","text":"","title":"CNN Technical details:"},{"location":"Neural-Network-Types/#input_2","text":"20 x 32 RGB images","title":"Input"},{"location":"Neural-Network-Types/#output_2","text":"dig-cont _XXX.tflite: 10 neurons representing the digits 0, 1, ... 9. The intermediate values are represented by weighted normalized values of two neighboring output neurons needs to be converted to angle with arctan-hyperbolicus function dig-class100 _XXX.tflite 100 neurons representing the classes from 0.0, 0.1, ... 9.8, 9.9","title":"Output"},{"location":"New-Releases-Notification/","text":"Notification about new Releases Do you want to get notified about a new release? There are several ways for it: Github Notifications You will need a Github Account for this! Log into your Github account on Github . Go to AI-on-the-edge-device . On the top right side, click onto Watch and select Custom : Select Releases . You will get an email when a new release gets created. See also Github Documentation . CodeRelease.io Alternatively or if you do not want to create a Github account, CodeRelease.io can be an alternative. You also have to subscribe with an emailaddress but no account is required.","title":"Notification about new Releases"},{"location":"New-Releases-Notification/#notification-about-new-releases","text":"Do you want to get notified about a new release? There are several ways for it:","title":"Notification about new Releases"},{"location":"New-Releases-Notification/#github-notifications","text":"You will need a Github Account for this! Log into your Github account on Github . Go to AI-on-the-edge-device . On the top right side, click onto Watch and select Custom : Select Releases . You will get an email when a new release gets created. See also Github Documentation .","title":"Github Notifications"},{"location":"New-Releases-Notification/#codereleaseio","text":"Alternatively or if you do not want to create a Github account, CodeRelease.io can be an alternative. You also have to subscribe with an emailaddress but no account is required.","title":"CodeRelease.io"},{"location":"REST-API/","text":"REST API Various information is directly accessible over specific REST calls. To use it, just append them to the IP, separated with a / , eg. http://192.168.1.1/json Note: For more detailed information to the REST handler, have a look to the code in the repository: registered handlers Control flow_start Trigger a flow start (if not running) Set Pre Value Set the Previous Value /setPreValue?value=1234&numbers=main where 1234 is the new value and main the name of the number to be adjusted. GPIO Control a GPIO output The GPIO entrypoint also support parameters: /GPIO?GPIO={PinNumber}&Status=high /GPIO?GPIO={PinNumber}&Status=low Example: /GPIO?GPIO=12&Status=high Read a GPIO input The GPIO entrypoint also support parameters: /GPIO?GPIO={PinNumber} Example: /GPIO?GPIO=12 ota ota_page.html Opens the Over-The-Air update html page reboot Trigger a reboot of the device Results json Show result in JSON syntax Example: { \"main\": { \"value\": \"521.17108\", \"raw\": \"521.17108\", \"pre\": \"521.17108\", \"error\": \"no error\", \"rate\": \"0.023780\", \"timestamp\": \"2023-01-13T16:00:42+0100\" } } value Show single result values The value entrypoint also support parameters: http://<IP>/value?all=true&type=value http://<IP>/value?all=true&type=raw http://<IP>/value?all=true&type=error http://<IP>/value?all=true&type=prevalue img_tmp/raw.jpg Capture and show a new raw image img_tmp/roi.jpg Show last aligned image img_tmp/alg_roi.jpg Show last aligned image incuding ROI overlay Status statusflow Show the actual step of the flow incl. timestamp Example: Take Image (15:56:34) rssi Show the WIFI signal strength Example: -51dBm cpu_temperature Show the CPU temperature Example: 38.5\u00b0C sysinfo Show system infos in JSON syntax Example: [ { \"firmware\" : \"xxx\", \"buildtime\" : \"2023-01-15 21:27\", \"gitbranch\" : \"xxx\", \"gittag\" : \"\", \"gitrevision\" : \"a5e533f+\", \"html\" : \"Development-Branch: xxx (Commit: a5e533f+)\", \"cputemp\" : \"67.777779\", \"hostname\" : \"xxx\", \"IPv4\" : \"192.168.xxx.xxx\", \"freeHeapMem\" : \"2789466\" } ] starttime Show starttime Example: 20230113-154634 uptime Show uptime Example: 0d 00h 15m 50s Camera lighton Switch the camera flashlight on lightoff Switch the camera flashlight off capture Capture a new image (without flashlight) capture_with_flashlight Capture a new image with flashlight save Save a new image to SD card The save entrypoint also support parameters: http://<IP>/save?filename=test.jpg&delay=1 Logs log Last part of todays log (last 80 kBytes)) logfileact Full log of today log.html Opens the log html page Diagnostics heap print relevant memory (heap) information Example: Heap info: Heap Total: 1888926 | SPI Free: 1827431 | SPI Larg Block: 1802240 | SPI Min Free: 758155 | Int Free: 61495 | Int Larg Block: 55296 | Int Min Free: 36427","title":"REST API"},{"location":"REST-API/#rest-api","text":"Various information is directly accessible over specific REST calls. To use it, just append them to the IP, separated with a / , eg. http://192.168.1.1/json Note: For more detailed information to the REST handler, have a look to the code in the repository: registered handlers","title":"REST API"},{"location":"REST-API/#control","text":"","title":"Control"},{"location":"REST-API/#flow_start","text":"Trigger a flow start (if not running)","title":"flow_start"},{"location":"REST-API/#set-pre-value","text":"Set the Previous Value /setPreValue?value=1234&numbers=main where 1234 is the new value and main the name of the number to be adjusted.","title":"Set Pre Value"},{"location":"REST-API/#gpio","text":"Control a GPIO output The GPIO entrypoint also support parameters: /GPIO?GPIO={PinNumber}&Status=high /GPIO?GPIO={PinNumber}&Status=low Example: /GPIO?GPIO=12&Status=high Read a GPIO input The GPIO entrypoint also support parameters: /GPIO?GPIO={PinNumber} Example: /GPIO?GPIO=12","title":"GPIO"},{"location":"REST-API/#ota","text":"","title":"ota"},{"location":"REST-API/#ota_pagehtml","text":"Opens the Over-The-Air update html page","title":"ota_page.html"},{"location":"REST-API/#reboot","text":"Trigger a reboot of the device","title":"reboot"},{"location":"REST-API/#results","text":"","title":"Results"},{"location":"REST-API/#json","text":"Show result in JSON syntax Example: { \"main\": { \"value\": \"521.17108\", \"raw\": \"521.17108\", \"pre\": \"521.17108\", \"error\": \"no error\", \"rate\": \"0.023780\", \"timestamp\": \"2023-01-13T16:00:42+0100\" } }","title":"json"},{"location":"REST-API/#value","text":"Show single result values The value entrypoint also support parameters: http://<IP>/value?all=true&type=value http://<IP>/value?all=true&type=raw http://<IP>/value?all=true&type=error http://<IP>/value?all=true&type=prevalue","title":"value"},{"location":"REST-API/#img_tmprawjpg","text":"Capture and show a new raw image","title":"img_tmp/raw.jpg"},{"location":"REST-API/#img_tmproijpg","text":"Show last aligned image","title":"img_tmp/roi.jpg"},{"location":"REST-API/#img_tmpalg_roijpg","text":"Show last aligned image incuding ROI overlay","title":"img_tmp/alg_roi.jpg"},{"location":"REST-API/#status","text":"","title":"Status"},{"location":"REST-API/#statusflow","text":"Show the actual step of the flow incl. timestamp Example: Take Image (15:56:34)","title":"statusflow"},{"location":"REST-API/#rssi","text":"Show the WIFI signal strength Example: -51dBm","title":"rssi"},{"location":"REST-API/#cpu_temperature","text":"Show the CPU temperature Example: 38.5\u00b0C","title":"cpu_temperature"},{"location":"REST-API/#sysinfo","text":"Show system infos in JSON syntax Example: [ { \"firmware\" : \"xxx\", \"buildtime\" : \"2023-01-15 21:27\", \"gitbranch\" : \"xxx\", \"gittag\" : \"\", \"gitrevision\" : \"a5e533f+\", \"html\" : \"Development-Branch: xxx (Commit: a5e533f+)\", \"cputemp\" : \"67.777779\", \"hostname\" : \"xxx\", \"IPv4\" : \"192.168.xxx.xxx\", \"freeHeapMem\" : \"2789466\" } ]","title":"sysinfo"},{"location":"REST-API/#starttime","text":"Show starttime Example: 20230113-154634","title":"starttime"},{"location":"REST-API/#uptime","text":"Show uptime Example: 0d 00h 15m 50s","title":"uptime"},{"location":"REST-API/#camera","text":"","title":"Camera"},{"location":"REST-API/#lighton","text":"Switch the camera flashlight on","title":"lighton"},{"location":"REST-API/#lightoff","text":"Switch the camera flashlight off","title":"lightoff"},{"location":"REST-API/#capture","text":"Capture a new image (without flashlight)","title":"capture"},{"location":"REST-API/#capture_with_flashlight","text":"Capture a new image with flashlight","title":"capture_with_flashlight"},{"location":"REST-API/#save","text":"Save a new image to SD card The save entrypoint also support parameters: http://<IP>/save?filename=test.jpg&delay=1","title":"save"},{"location":"REST-API/#logs","text":"","title":"Logs"},{"location":"REST-API/#log","text":"Last part of todays log (last 80 kBytes))","title":"log"},{"location":"REST-API/#logfileact","text":"Full log of today","title":"logfileact"},{"location":"REST-API/#loghtml","text":"Opens the log html page","title":"log.html"},{"location":"REST-API/#diagnostics","text":"","title":"Diagnostics"},{"location":"REST-API/#heap","text":"print relevant memory (heap) information Example: Heap info: Heap Total: 1888926 | SPI Free: 1827431 | SPI Larg Block: 1802240 | SPI Min Free: 758155 | Int Free: 61495 | Int Larg Block: 55296 | Int Min Free: 36427","title":"heap"},{"location":"ROI-Configuration/","text":"ROI (Region of Interest) General remark: You are using a neural network approach which is trained to fit as many different type of meters as possible. The accuracy will never be 100%. It is normal to see a missing reading once in a while. There there are several precautions to detect this. For details see the section PostProcessing on the configuration page. The most critical settings for accurate detection are: Correct setting of the R egions O f I nterest (ROIs) for detection of the image. This must be done manually for each meter! Number type is part of the training set. Have a look on the Digital Counters resp. Analog Needles to check if your types are contained. If your number types are not contained, you should take the effort to record them so we can add them to the training data. See: Learn models with your own images on how to create new input. Correct Setup of ROI Please proceed in the following order! Don't forget to save after each step! Image Sharpness Ensure a sharp image of the camera by adjusting the focal length of the ESP OV2640 camera. Adjust the focus for the clearest possible image! In order to use it for reading a meter, the focal-length of the OV2640 camera has to be manipulated, as by default it only results in sharp image for distance bigger than ~40cm, causing the image of the water meter to be too small for automated readout processing. ATTENTION: this modification will void any warranty, as the sealing of the lens objective is broken! ATTENTION: This modification will render the camera unsuitable for general, web-cam type applications unless the focal length is changed back to the original setting. Remove the fixing glue of the OV2640 lens with a sharp knife. After this you can screw the lens in and out. Rotating it by about a quarter of a turn counterclockwise results in a focus plane of about 10cm. You need to figure out your best setting with a little bit of trial and error for your specific environment. Horizontal Alignment Ensure an exact horizontal alignment of the number via the alignment / reference setup: \u2714\ufe0f Okay \u274c Not Okay Correct Size for ROI Choose the right size of the ROI: The configuration of ROIs differs a bit on the model you choose. Below you find the differences between the different AI models. Pick the one you think fits best your purpose. If you don't get to good result, try another model. Model Selection dig-class11 Configuration dig-class11 - Models recognize the complete digit only . Here it is not relevant if the ROI fits the Border of the digit window. For this model, there should be a border of 20% of the image size around the number itself. This border is shown in the ROI setup image by the inner thinner rectangle. This rectangle should fit perfectly around the number when the number has not started to rotate to the next position: Example 1 Example 2 \u2714\ufe0f Okay \u274c Not Okay \u274c Not Okay If you have perfect alignment you and are not getting satisfying results, most probably your numbers are not part of the training data yet. Read on Learn models with your own images how to add your meter's type of numbers to the training set. dig-class100 / dig-cont Configuration These models recognize the tenths (fractions) between the numbers. This model requires a different ROI setup; the height must be set differently and more accurately. First, the width can be set as for dig-class11, i.e. 20% margin left and right. The height of the outer rectangle should be set to the upper and lower edge of the number window. To achieve this setting, you need to unlock the aspect ratio: Here an example: Example 1 \u2714\ufe0f Okay \u274c Not Okay","title":"ROI (Region of Interest)"},{"location":"ROI-Configuration/#roi-region-of-interest","text":"General remark: You are using a neural network approach which is trained to fit as many different type of meters as possible. The accuracy will never be 100%. It is normal to see a missing reading once in a while. There there are several precautions to detect this. For details see the section PostProcessing on the configuration page. The most critical settings for accurate detection are: Correct setting of the R egions O f I nterest (ROIs) for detection of the image. This must be done manually for each meter! Number type is part of the training set. Have a look on the Digital Counters resp. Analog Needles to check if your types are contained. If your number types are not contained, you should take the effort to record them so we can add them to the training data. See: Learn models with your own images on how to create new input.","title":"ROI (Region of Interest)"},{"location":"ROI-Configuration/#correct-setup-of-roi","text":"Please proceed in the following order! Don't forget to save after each step!","title":"Correct Setup of ROI"},{"location":"ROI-Configuration/#image-sharpness","text":"Ensure a sharp image of the camera by adjusting the focal length of the ESP OV2640 camera. Adjust the focus for the clearest possible image! In order to use it for reading a meter, the focal-length of the OV2640 camera has to be manipulated, as by default it only results in sharp image for distance bigger than ~40cm, causing the image of the water meter to be too small for automated readout processing. ATTENTION: this modification will void any warranty, as the sealing of the lens objective is broken! ATTENTION: This modification will render the camera unsuitable for general, web-cam type applications unless the focal length is changed back to the original setting. Remove the fixing glue of the OV2640 lens with a sharp knife. After this you can screw the lens in and out. Rotating it by about a quarter of a turn counterclockwise results in a focus plane of about 10cm. You need to figure out your best setting with a little bit of trial and error for your specific environment.","title":"Image Sharpness"},{"location":"ROI-Configuration/#horizontal-alignment","text":"Ensure an exact horizontal alignment of the number via the alignment / reference setup: \u2714\ufe0f Okay \u274c Not Okay","title":"Horizontal Alignment"},{"location":"ROI-Configuration/#correct-size-for-roi","text":"Choose the right size of the ROI: The configuration of ROIs differs a bit on the model you choose. Below you find the differences between the different AI models. Pick the one you think fits best your purpose. If you don't get to good result, try another model.","title":"Correct Size for ROI"},{"location":"ROI-Configuration/#model-selection","text":"","title":"Model Selection"},{"location":"ROI-Configuration/#dig-class11-configuration","text":"dig-class11 - Models recognize the complete digit only . Here it is not relevant if the ROI fits the Border of the digit window. For this model, there should be a border of 20% of the image size around the number itself. This border is shown in the ROI setup image by the inner thinner rectangle. This rectangle should fit perfectly around the number when the number has not started to rotate to the next position: Example 1 Example 2 \u2714\ufe0f Okay \u274c Not Okay \u274c Not Okay If you have perfect alignment you and are not getting satisfying results, most probably your numbers are not part of the training data yet. Read on Learn models with your own images how to add your meter's type of numbers to the training set.","title":"dig-class11 Configuration"},{"location":"ROI-Configuration/#dig-class100-dig-cont-configuration","text":"These models recognize the tenths (fractions) between the numbers. This model requires a different ROI setup; the height must be set differently and more accurately. First, the width can be set as for dig-class11, i.e. 20% margin left and right. The height of the outer rectangle should be set to the upper and lower edge of the number window. To achieve this setting, you need to unlock the aspect ratio: Here an example: Example 1 \u2714\ufe0f Okay \u274c Not Okay","title":"dig-class100 / dig-cont Configuration"},{"location":"Release-creation/","text":"Preparing for Release Changelog is merged back from master branch to rolling branch (should be the last step of the previous release creation) All changes are documented in the Changelog in rolling branch Release creation steps Merge rolling into master branch Best to wait for the GitHub action to run successfully On master branch tag the version like v11.3.1 and don't forget to push it. Wait for the GitHub-Action of release creation. After all is done: the release should be created the artifacts are downloadable from release The documented changes were applied to the release Merge master back in rolling Check that the Web Installer shows the right version","title":"Preparing for Release"},{"location":"Release-creation/#preparing-for-release","text":"Changelog is merged back from master branch to rolling branch (should be the last step of the previous release creation) All changes are documented in the Changelog in rolling branch","title":"Preparing for Release"},{"location":"Release-creation/#release-creation-steps","text":"Merge rolling into master branch Best to wait for the GitHub action to run successfully On master branch tag the version like v11.3.1 and don't forget to push it. Wait for the GitHub-Action of release creation. After all is done: the release should be created the artifacts are downloadable from release The documented changes were applied to the release Merge master back in rolling Check that the Web Installer shows the right version","title":"Release creation steps"},{"location":"Testing/","text":"Testing Option for VSCode You can test your functions directly on the device. Structure All tests are under directory \"test\" in the project and not compiled with default build option of platformio. The main function is in file test_suite_controlflow.cpp . In method app_main() you can add your own tests. Include my my own test In method app_main() of test_suite_controlflow.cpp you can add your own tests. Include your test-file in the top like #include \"components/jomjol-flowcontroll/test_flow_postrocess_helper.cpp\" components is a subfolder of tests here. Not the components directory of root source. In the bottom add your test function. RUN_TEST(testNegative); Your test function should have a TEST_ASSERT_EQUAL_* . For more information look at unity-testing . Run tests You will need a testing device. best with usb adapter. Before you upload your tests you will need to setup the device with initial setup procedure described in [[Installation]] Now you can use Visual Studio Code or a standard console to upload the test code. In VS Code (tab platformio) open Advanced and select Test . Alternativ you can run it in console/terminal with platformio test --environment esp32cam . In my environment the serial terminal not opens. I have to do it for myself. You will see much logging. If any test fails it logs it out. Else it logs all test passed in the end. Troubleshooting If you test very much cases in one function, the device runs in stackoverflow and an endless boot. Reduce the count of test cases or split the test function in multiple functions.","title":"Testing"},{"location":"Testing/#testing-option-for-vscode","text":"You can test your functions directly on the device.","title":"Testing Option for VSCode"},{"location":"Testing/#structure","text":"All tests are under directory \"test\" in the project and not compiled with default build option of platformio. The main function is in file test_suite_controlflow.cpp . In method app_main() you can add your own tests.","title":"Structure"},{"location":"Testing/#include-my-my-own-test","text":"In method app_main() of test_suite_controlflow.cpp you can add your own tests. Include your test-file in the top like #include \"components/jomjol-flowcontroll/test_flow_postrocess_helper.cpp\" components is a subfolder of tests here. Not the components directory of root source. In the bottom add your test function. RUN_TEST(testNegative); Your test function should have a TEST_ASSERT_EQUAL_* . For more information look at unity-testing .","title":"Include my my own test"},{"location":"Testing/#run-tests","text":"You will need a testing device. best with usb adapter. Before you upload your tests you will need to setup the device with initial setup procedure described in [[Installation]] Now you can use Visual Studio Code or a standard console to upload the test code. In VS Code (tab platformio) open Advanced and select Test . Alternativ you can run it in console/terminal with platformio test --environment esp32cam . In my environment the serial terminal not opens. I have to do it for myself. You will see much logging. If any test fails it logs it out. Else it logs all test passed in the end.","title":"Run tests"},{"location":"Testing/#troubleshooting","text":"If you test very much cases in one function, the device runs in stackoverflow and an endless boot. Reduce the count of test cases or split the test function in multiple functions.","title":"Troubleshooting"},{"location":"Upload-files-by-script/","text":"Scripted File Upload To upload a file eg. using curl , you first have to delete it and then upload it: curl -d '' http://192.168.1.153/delete/html/index.html curl --data-binary @ota_page.html http://192.168.1.153/upload/html/index.html","title":"Scripted File Upload"},{"location":"Upload-files-by-script/#scripted-file-upload","text":"To upload a file eg. using curl , you first have to delete it and then upload it: curl -d '' http://192.168.1.153/delete/html/index.html curl --data-binary @ota_page.html http://192.168.1.153/upload/html/index.html","title":"Scripted File Upload"},{"location":"Watermeter-specific-analog---digital-transition/","text":"Analog/Digital Transition on Watermeters At first, for the most watermeters the default configuration should be work. But the digit, especially the last digit differs in some devices. \"Normal\" transition In most cases, the transition of the last digit starts when the analogue pointer is > 9. Often the last digit \"hangs\" a bit on this devices and comes not over zero. So it is not easy to see which digit is correct. In the first example 4 or still 3? (3 is correct). Early transition Some units start the transition very early or run with the analogue pointer. In the third example, is it a 3 or a 2? Inaccuracies in image recognition The models for image recognition are good, but have inaccuracies in the range +/- 0.2. In order to obtain as many correct results as possible, a treatment is carried out in the post process in the range of 9.8-0.2 for the analogue pointer, which must start differently depending on the type of counter. How to configure for my meter type If you have a devices with \"normal\" transition you should not have any issues. On devices with \"early\" transition, you can set the option AnalogDigitalTransitionStart to a value between 6 and 8.","title":"Analog/Digital Transition on Watermeters"},{"location":"Watermeter-specific-analog---digital-transition/#analogdigital-transition-on-watermeters","text":"At first, for the most watermeters the default configuration should be work. But the digit, especially the last digit differs in some devices.","title":"Analog/Digital Transition on Watermeters"},{"location":"Watermeter-specific-analog---digital-transition/#normal-transition","text":"In most cases, the transition of the last digit starts when the analogue pointer is > 9. Often the last digit \"hangs\" a bit on this devices and comes not over zero. So it is not easy to see which digit is correct. In the first example 4 or still 3? (3 is correct).","title":"\"Normal\" transition"},{"location":"Watermeter-specific-analog---digital-transition/#early-transition","text":"Some units start the transition very early or run with the analogue pointer. In the third example, is it a 3 or a 2?","title":"Early transition"},{"location":"Watermeter-specific-analog---digital-transition/#inaccuracies-in-image-recognition","text":"The models for image recognition are good, but have inaccuracies in the range +/- 0.2. In order to obtain as many correct results as possible, a treatment is carried out in the post process in the range of 9.8-0.2 for the analogue pointer, which must start differently depending on the type of counter.","title":"Inaccuracies in image recognition"},{"location":"Watermeter-specific-analog---digital-transition/#how-to-configure-for-my-meter-type","text":"If you have a devices with \"normal\" transition you should not have any issues. On devices with \"early\" transition, you can set the option AnalogDigitalTransitionStart to a value between 6 and 8.","title":"How to configure for my meter type"},{"location":"inital-setup/","text":"Initial Setup After setting up the device (firmware, sd-card, wlan) the device will connect to the wifi access point and start in an initial setup configuration: In the top you can navigate through 5 steps, that guide you through the necessary setup. All settings can be accessed also later in the normal operation mode (see below). 1. Reference Image The reference image is the basis for the coordinate system of the ROIs. Therefore it is very important, to have a well aligned image, that is not rotated. At first the current image is shown. To define a new reference image push the button \"Create new Reference\" (2) and afterwards \"Take Image\" (2). It might take some seconds for processing, then your actual camera image should be shown. Attention: This is the first time, where you have access to the camera image. It might not be sharp yet. This is the first thing to do according to the following instruction ( TBD ). Correct Horizontal Alignment Ensure an exact horizontal alignment of the number via the alignment / reference setup: \u2714\ufe0f Okay \u274c Not Okay \u203c\ufe0f Attention: Updating the reference image, also means, that all alignment images and ROIs needs to be teached again. Therefore do this step later only with caution. If everything is done, you can save the result with \"Update Reference Image\" (4). Please do not reboot at this stage, but go to the next section. 2. Alignment References The alignment references are used to realign every taken image to the reference coordinates. Therefore two alignment structures are identified and the image is shifted and rotated according to their position with the target to be in exactly the same position as the reference image. The alignment structures needs to be unique and have a good contrast. You can switch between this two marks with (1). With the control in (3) position and size of the selected reference image. You can define the ROI in the image also directly via drag and drop with the mouse. To choose the currently marked image part you need to push \"Update Reference\" (2). In some cases it might be useful to use a reference with a higher contrast. This can be achieved by pushing \"Enhance Contrast\" (3). The result will be calculated on the ESP32 - so be a bit patient, before you see it active. To save the push finally \"Save to config.ini\" (4). Do not reboot at this stage, but proceed with the \"next\" button. 3./4. Define the ROIs for the number detection Here the regions of interest for the digital and analog pointers are defined. As both are done identically, here as an example the digital images are shown. General usage First of all, if you don't have that kind of numbers on you meter, you can disable it with the check box at the top (1). In contrast to the reference images, here there are more ROIs possible. You can switch between them with the drop down box (2). If you need additional ROIs or delete them you can do this with the control at (2). Attention: The order of the ROIs defines, how the indiviual digits are combined to the total number. The first ROI is the digit with the highest order, then the second and so on. You can control the order in the selector tab and change it with the buttons \"move Next\" or \"move Previous\". As for the reference images you can change position, size and name of the ROI in the text fields or define them via drag and drop through the mouse button. In most cases the digits are ordered in a aequidistant order and have the same size, you can synchronize them with the control in (4). Don' t forget to save the settings with \"Save\" and do not reboot at this stage. Detail for ROI configuration - Analog Meters For analog meters the ROI setting is rather straight forward as the meter is usually quadratic with a clear center. The circle should exactly fit to the outer size of the meter and the cross should be in the middle. Here is an example with the details for the ROI \"ana1\": Detail for ROI configuration - Digital Meters For the digital meters it is a little bit more complicated, as there are different options of digital models, that you can choose. Digital meter, that only recognized full digits (0, 1, 2, 3, ... 9) - Naming: dig-class11-....tfl Advantage: broad variety of types included in the training Disadvantage: partially rotated numbers cannot be detected Model with subdigit resulition (0.0, 0.1, 0.2, .... 9.8, 9.9) - Naming: dig-cont-....tfl or dig-class100-....tfl Advantage: partial numbers can be detected and a better post processing is possible Disadvantage: only limited types of meter types are trained due to the high effort for the training data Details and the corresponding \"perfect\" setting is explained here: Details ROI Configuration For a first run you can choose the following general settings: There is an inner and an outer frame for the ROIs. Make the inner frame exactly the size of the number. Example 1 Example 2 \u2714\ufe0f Okay \u274c Not Okay \u274c Not Okay 5. General Settings In the next steps you can configure the behavior and external interfaces in detail: The configuration is divided into different sub topics: TakeImage Digits Analog PostProcessing MQTT InfluxDB GPIOSettings Autotimer DataLogging Debug System The details are explained in other parts of the manual (see links ( TBD )) Some of the sections as well as parameters are mandatory. They can be en/disabled in the first column (1). The setting itself is done in the next column (2) and a brief explanation you can find in the last column (3). Don' t forget to save the settings with \"Save\" and do not reboot at this stage. Expert Modus With the normal parameters you should be able to make the needed settings for most of the system. Sometimes there is some fine tunning needed. For this there is an expert modus available. This can be enabled with the check box at the top (4). After this you see much more parameters. But before modifiying them you should be really sure, what they are about. Finish Setup and change to normal operation After setting up everything, there is a last step to be done: With (1) you leave the setup modus and reboot to normal operation mode. Access to setup in normal operation mode You can access all the settings also during the normal working mode via the \"Settings\" menue: (1) Access to configuration parameters (2) Update of reference image (3) Update of alignment marks (4)/(5) Update of the ROI setting","title":"Initial Setup"},{"location":"inital-setup/#initial-setup","text":"After setting up the device (firmware, sd-card, wlan) the device will connect to the wifi access point and start in an initial setup configuration: In the top you can navigate through 5 steps, that guide you through the necessary setup. All settings can be accessed also later in the normal operation mode (see below).","title":"Initial Setup"},{"location":"inital-setup/#1-reference-image","text":"The reference image is the basis for the coordinate system of the ROIs. Therefore it is very important, to have a well aligned image, that is not rotated. At first the current image is shown. To define a new reference image push the button \"Create new Reference\" (2) and afterwards \"Take Image\" (2). It might take some seconds for processing, then your actual camera image should be shown. Attention: This is the first time, where you have access to the camera image. It might not be sharp yet. This is the first thing to do according to the following instruction ( TBD ).","title":"1. Reference Image"},{"location":"inital-setup/#correct-horizontal-alignment","text":"Ensure an exact horizontal alignment of the number via the alignment / reference setup: \u2714\ufe0f Okay \u274c Not Okay \u203c\ufe0f Attention: Updating the reference image, also means, that all alignment images and ROIs needs to be teached again. Therefore do this step later only with caution. If everything is done, you can save the result with \"Update Reference Image\" (4). Please do not reboot at this stage, but go to the next section.","title":"Correct Horizontal Alignment"},{"location":"inital-setup/#2-alignment-references","text":"The alignment references are used to realign every taken image to the reference coordinates. Therefore two alignment structures are identified and the image is shifted and rotated according to their position with the target to be in exactly the same position as the reference image. The alignment structures needs to be unique and have a good contrast. You can switch between this two marks with (1). With the control in (3) position and size of the selected reference image. You can define the ROI in the image also directly via drag and drop with the mouse. To choose the currently marked image part you need to push \"Update Reference\" (2). In some cases it might be useful to use a reference with a higher contrast. This can be achieved by pushing \"Enhance Contrast\" (3). The result will be calculated on the ESP32 - so be a bit patient, before you see it active. To save the push finally \"Save to config.ini\" (4). Do not reboot at this stage, but proceed with the \"next\" button.","title":"2. Alignment References"},{"location":"inital-setup/#34-define-the-rois-for-the-number-detection","text":"Here the regions of interest for the digital and analog pointers are defined. As both are done identically, here as an example the digital images are shown.","title":"3./4. Define the ROIs for the number detection"},{"location":"inital-setup/#general-usage","text":"First of all, if you don't have that kind of numbers on you meter, you can disable it with the check box at the top (1). In contrast to the reference images, here there are more ROIs possible. You can switch between them with the drop down box (2). If you need additional ROIs or delete them you can do this with the control at (2). Attention: The order of the ROIs defines, how the indiviual digits are combined to the total number. The first ROI is the digit with the highest order, then the second and so on. You can control the order in the selector tab and change it with the buttons \"move Next\" or \"move Previous\". As for the reference images you can change position, size and name of the ROI in the text fields or define them via drag and drop through the mouse button. In most cases the digits are ordered in a aequidistant order and have the same size, you can synchronize them with the control in (4). Don' t forget to save the settings with \"Save\" and do not reboot at this stage.","title":"General usage"},{"location":"inital-setup/#detail-for-roi-configuration-analog-meters","text":"For analog meters the ROI setting is rather straight forward as the meter is usually quadratic with a clear center. The circle should exactly fit to the outer size of the meter and the cross should be in the middle. Here is an example with the details for the ROI \"ana1\":","title":"Detail for ROI configuration - Analog Meters"},{"location":"inital-setup/#detail-for-roi-configuration-digital-meters","text":"For the digital meters it is a little bit more complicated, as there are different options of digital models, that you can choose. Digital meter, that only recognized full digits (0, 1, 2, 3, ... 9) - Naming: dig-class11-....tfl Advantage: broad variety of types included in the training Disadvantage: partially rotated numbers cannot be detected Model with subdigit resulition (0.0, 0.1, 0.2, .... 9.8, 9.9) - Naming: dig-cont-....tfl or dig-class100-....tfl Advantage: partial numbers can be detected and a better post processing is possible Disadvantage: only limited types of meter types are trained due to the high effort for the training data Details and the corresponding \"perfect\" setting is explained here: Details ROI Configuration For a first run you can choose the following general settings: There is an inner and an outer frame for the ROIs. Make the inner frame exactly the size of the number. Example 1 Example 2 \u2714\ufe0f Okay \u274c Not Okay \u274c Not Okay","title":"Detail for ROI configuration - Digital Meters"},{"location":"inital-setup/#5-general-settings","text":"In the next steps you can configure the behavior and external interfaces in detail: The configuration is divided into different sub topics: TakeImage Digits Analog PostProcessing MQTT InfluxDB GPIOSettings Autotimer DataLogging Debug System The details are explained in other parts of the manual (see links ( TBD )) Some of the sections as well as parameters are mandatory. They can be en/disabled in the first column (1). The setting itself is done in the next column (2) and a brief explanation you can find in the last column (3). Don' t forget to save the settings with \"Save\" and do not reboot at this stage.","title":"5. General Settings"},{"location":"inital-setup/#expert-modus","text":"With the normal parameters you should be able to make the needed settings for most of the system. Sometimes there is some fine tunning needed. For this there is an expert modus available. This can be enabled with the check box at the top (4). After this you see much more parameters. But before modifiying them you should be really sure, what they are about.","title":"Expert Modus"},{"location":"inital-setup/#finish-setup-and-change-to-normal-operation","text":"After setting up everything, there is a last step to be done: With (1) you leave the setup modus and reboot to normal operation mode.","title":"Finish Setup and change to normal operation"},{"location":"inital-setup/#access-to-setup-in-normal-operation-mode","text":"You can access all the settings also during the normal working mode via the \"Settings\" menue: (1) Access to configuration parameters (2) Update of reference image (3) Update of alignment marks (4)/(5) Update of the ROI setting","title":"Access to setup in normal operation mode"},{"location":"ota/","text":"Over-The-Air (OTA) Update You can do an OTA (over-the-air) update via the graphical user interface. Grab the firmware from the Releases page (Stable, tested versions), or the Automatically build development branch (experimental, untested versions). Please have a look on first! You need: * firmware.bin * html.zip General remark: It is always recommended to upload both files, as they are coupled to each other If you make a major update, it might be needed to modify the config.ini as it's syntax or context has changed It is recommended to make a backup of the /config directory, minimum of the config.ini . Access to the update page: The graphical OTA update can be accessed in the menue \"System\": Update Just follow the steps 1 to 5 to perform the update: Select (a) and upload (b) the file firmware.bin Flash the firmware Select (a) and upload (b) the file html.zip Update the html-files Reboot After the reboot with a major change it is recommended to check the configuration settings and save them again","title":"Over-The-Air (OTA) Update"},{"location":"ota/#over-the-air-ota-update","text":"You can do an OTA (over-the-air) update via the graphical user interface. Grab the firmware from the Releases page (Stable, tested versions), or the Automatically build development branch (experimental, untested versions). Please have a look on first! You need: * firmware.bin * html.zip","title":"Over-The-Air (OTA) Update"},{"location":"ota/#general-remark","text":"It is always recommended to upload both files, as they are coupled to each other If you make a major update, it might be needed to modify the config.ini as it's syntax or context has changed It is recommended to make a backup of the /config directory, minimum of the config.ini .","title":"General remark:"},{"location":"ota/#access-to-the-update-page","text":"The graphical OTA update can be accessed in the menue \"System\":","title":"Access to the update page:"},{"location":"ota/#update","text":"Just follow the steps 1 to 5 to perform the update: Select (a) and upload (b) the file firmware.bin Flash the firmware Select (a) and upload (b) the file html.zip Update the html-files Reboot After the reboot with a major change it is recommended to check the configuration settings and save them again","title":"Update"},{"location":"outdated--Gasmeter-Log-Downloader/","text":"Gasmeter Log-Downloader Warning This page no longer is maintained! This small tool downloads the logfiles from your ESP32 and stores the last value of the day in an *.csv file. To use this tool you need to activate the debug logfile in your configuration (Configuration / Debug / Logfile). I go with 30 days of retention in days. It downloads only the past logfiles (yesterday and older). You can define the max. number of Logfiles to download (beginning from newest [yesterday]). I wrote this tool to get a chart of the daily gas consumption to optimize my gas powered heating. Variables to define by yourself: URL to Logfile-Path on Device: \"http://ESP32-IP-Address/fileserver/log/message/\" Download Logfiles to: enter a valid directory, e.g. \"D:\\Gaszaehler\\Auswertung\\Log-Downloads\\\" Output CSV-File: enter a valid directory, e.g. \"D:\\Gaszaehler\\Auswertung\\DailyValues.csv\" Download Logfiles from past # days: enter the max. number of logfiles you want to download (<= your logfile retention value in your device configuration) Feel free to optimize and modify it.","title":"Gasmeter Log-Downloader"},{"location":"outdated--Gasmeter-Log-Downloader/#gasmeter-log-downloader","text":"Warning This page no longer is maintained! This small tool downloads the logfiles from your ESP32 and stores the last value of the day in an *.csv file. To use this tool you need to activate the debug logfile in your configuration (Configuration / Debug / Logfile). I go with 30 days of retention in days. It downloads only the past logfiles (yesterday and older). You can define the max. number of Logfiles to download (beginning from newest [yesterday]). I wrote this tool to get a chart of the daily gas consumption to optimize my gas powered heating. Variables to define by yourself: URL to Logfile-Path on Device: \"http://ESP32-IP-Address/fileserver/log/message/\" Download Logfiles to: enter a valid directory, e.g. \"D:\\Gaszaehler\\Auswertung\\Log-Downloads\\\" Output CSV-File: enter a valid directory, e.g. \"D:\\Gaszaehler\\Auswertung\\DailyValues.csv\" Download Logfiles from past # days: enter the max. number of logfiles you want to download (<= your logfile retention value in your device configuration) Feel free to optimize and modify it.","title":"Gasmeter Log-Downloader"},{"location":"outdated--Integrated-Functions/","text":"Integrated Functions Warning This page no longer is maintained! wasserzaehler http://IP-ESP32/wasserzaehler.html This is the main purpose of this device. It returns the converted image as a number with different option. The output can be modified either by the configuration parameters or by HTML parameters. Details can be found here: tbd Picture Server http://IP-ESP32/capture http://IP-ESP32/capture_with_flashlight This is a implementation of the camera interface of https://github.com/jomjol/water-meter-picture-provider It is fully compatible including the parameters ( quality =..., size=... ) . This allows to use this ESP32 system in parallel to the corresponding docker system: https://github.com/jomjol/water-meter-system-complete, from which this project is basically the successor. File server Access: http://IP-ESP32/fileserver/ Simple file server, that allows viewing, upload, download and deleting of single files of the SD-card content. The usage is self explaining. The file path or file can directly be accessed by the URL after file server. Example for config.ini : http://IP-ESP/fileserver/config/config.ini OTA-Update http://IP-ESP32/ota?file=firmware.bin Here an over the air update can be triggered. The firmware file is expected to be located in the sub directory /firmware/ and can be uploaded with the file server. By the parameter file the name of the firmware file needs to be given. Reboot http://IP-ESP32/reboot A reboot with a delay of 5 seconds is initiated, e.g. after firmware update. ATTENTION : currently this is not working properly - hardware power off is needed instead. Work in progress! Simple Web Server If none of the above URLs are fitting, a very simple web server checks, if there is a fitting file from the sub directory /html This can be used for a very simple web server for information or simple web pages.","title":"Integrated Functions"},{"location":"outdated--Integrated-Functions/#integrated-functions","text":"Warning This page no longer is maintained!","title":"Integrated Functions"},{"location":"outdated--Integrated-Functions/#wasserzaehler","text":"http://IP-ESP32/wasserzaehler.html This is the main purpose of this device. It returns the converted image as a number with different option. The output can be modified either by the configuration parameters or by HTML parameters. Details can be found here: tbd","title":"wasserzaehler"},{"location":"outdated--Integrated-Functions/#picture-server","text":"http://IP-ESP32/capture http://IP-ESP32/capture_with_flashlight This is a implementation of the camera interface of https://github.com/jomjol/water-meter-picture-provider It is fully compatible including the parameters ( quality =..., size=... ) . This allows to use this ESP32 system in parallel to the corresponding docker system: https://github.com/jomjol/water-meter-system-complete, from which this project is basically the successor.","title":"Picture Server"},{"location":"outdated--Integrated-Functions/#file-server","text":"Access: http://IP-ESP32/fileserver/ Simple file server, that allows viewing, upload, download and deleting of single files of the SD-card content. The usage is self explaining. The file path or file can directly be accessed by the URL after file server. Example for config.ini : http://IP-ESP/fileserver/config/config.ini","title":"File server"},{"location":"outdated--Integrated-Functions/#ota-update","text":"http://IP-ESP32/ota?file=firmware.bin Here an over the air update can be triggered. The firmware file is expected to be located in the sub directory /firmware/ and can be uploaded with the file server. By the parameter file the name of the firmware file needs to be given.","title":"OTA-Update"},{"location":"outdated--Integrated-Functions/#reboot","text":"http://IP-ESP32/reboot A reboot with a delay of 5 seconds is initiated, e.g. after firmware update. ATTENTION : currently this is not working properly - hardware power off is needed instead. Work in progress!","title":"Reboot"},{"location":"outdated--Integrated-Functions/#simple-web-server","text":"If none of the above URLs are fitting, a very simple web server checks, if there is a fitting file from the sub directory /html This can be used for a very simple web server for information or simple web pages.","title":"Simple Web Server"},{"location":"outdated--Migrate-Old-Config-To-New-Config/","text":"Migration from water-meter \u201eold\u201c to water-meter \u201cAI-on-the-edge-device\u201d Warning This page no longer is maintained! There are only some few steps necessary to migrate your old system to the new one. Please follow the following steps: 1. Follow the installation guide to flash the ESP32CAM and prepare a SD-Card with the content of the master 2. Save the following files from the old Docker system on your PC: Reference Points 1-3 (only 2 needed) Config.ini 3. Copy Reference Points 1-3 onto the new water-meter system (Directory /config ) Please note only two Reference Points are supported in the new system. 4. Open new config.ini File: Insert from the old Config.ini file [alignment] and [alignment.ref0] and [alignment.ref1] section the two Ref x and y position and the initial_rotation_angle= 123 into the new Config.ini File, e.g.: Old: [alignment.ref0] image=./config/RB01_65x65.jpg pos_x=28 pos_y=63 [alignment.ref1] image=./config/RB02_50x35.jpg pos_x=497 pos_y=127 [alignment] initial_rotation_angle=180 New: [Alignment] InitalRotate=180 /config/RB01_65x65.jpg 28, 63 /config/RB02_50x35.jpg 497, 127 SearchFieldX = 20 SearchFieldY = 20 5. Insert the old Digit Values into the new Config.ini File, e.g.: Old: [Digital_Digit.ziffer1] pos_x=265 pos_y=117 dx=28 dy=51 [Digital_Digit.ziffer2] pos_x=310 pos_y=117 dx=28 dy=51 [Digital_Digit.ziffer3] pos_x=354 pos_y=117 dx=28 dy=51 [Digital_Digit.ziffer4] pos_x=399 pos_y=117 dx=28 dy=51 [Digital_Digit.ziffer5] pos_x=445 pos_y=115 dx=28 dy=51 New: [Digits] Model=/config/dig0630s3.tflite ;LogImageLocation = /log/digit ModelInputSize 20, 32 digit1, 265, 117, 28, 51 digit2, 310, 117, 28, 51 digit3, 354, 117, 28, 51 digit4, 399, 117, 28, 51 digit5, 445, 115, 28, 51 6. Make sure that you have the same quality and size settings as in your old Config.ini In the old configuration this was coded in the html-string for the image source: Old: URLImageSource=http://IP-ADRESS/capture_with_flashlight?quality=5&size=VGA Default was Quality=5 and VGA. New: ImageQuality = 5 ImageSize = VGA 7. Repeat the same for the analog section 8. Insert your SSID and Password into the new wlan.ini File 9. Compare and edit [ConsistencyCheck] Section with new [PostProcessing] Section 10. Save new config.ini File in the new System. 11. Restart the system. 12. After the first start set manually the PreValue in the new system","title":"Migration from water-meter \u201eold\u201c to water-meter \u201cAI-on-the-edge-device\u201d"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#migration-from-water-meter-old-to-water-meter-ai-on-the-edge-device","text":"Warning This page no longer is maintained! There are only some few steps necessary to migrate your old system to the new one. Please follow the following steps:","title":"Migration from water-meter \u201eold\u201c to water-meter \u201cAI-on-the-edge-device\u201d"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#1-follow-the-installation-guide-to-flash-the-esp32cam-and-prepare-a-sd-card-with-the-content-of-the-master","text":"","title":"1. Follow the installation guide to flash the ESP32CAM and prepare a SD-Card with the content of the master"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#2-save-the-following-files-from-the-old-docker-system-on-your-pc","text":"Reference Points 1-3 (only 2 needed) Config.ini","title":"2. Save the following files from the old Docker system on your PC:"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#3-copy-reference-points-1-3-onto-the-new-water-meter-system-directory-config","text":"Please note only two Reference Points are supported in the new system.","title":"3. Copy Reference Points 1-3 onto the new water-meter system (Directory /config)"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#4-open-new-configini-file","text":"Insert from the old Config.ini file [alignment] and [alignment.ref0] and [alignment.ref1] section the two Ref x and y position and the initial_rotation_angle= 123 into the new Config.ini File, e.g.:","title":"4. Open new config.ini File:"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#old","text":"[alignment.ref0] image=./config/RB01_65x65.jpg pos_x=28 pos_y=63 [alignment.ref1] image=./config/RB02_50x35.jpg pos_x=497 pos_y=127 [alignment] initial_rotation_angle=180","title":"Old:"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#new","text":"[Alignment] InitalRotate=180 /config/RB01_65x65.jpg 28, 63 /config/RB02_50x35.jpg 497, 127 SearchFieldX = 20 SearchFieldY = 20","title":"New:"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#5-insert-the-old-digit-values-into-the-new-configini-file-eg","text":"","title":"5. Insert the old Digit Values into the new Config.ini File, e.g.:"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#old_1","text":"[Digital_Digit.ziffer1] pos_x=265 pos_y=117 dx=28 dy=51 [Digital_Digit.ziffer2] pos_x=310 pos_y=117 dx=28 dy=51 [Digital_Digit.ziffer3] pos_x=354 pos_y=117 dx=28 dy=51 [Digital_Digit.ziffer4] pos_x=399 pos_y=117 dx=28 dy=51 [Digital_Digit.ziffer5] pos_x=445 pos_y=115 dx=28 dy=51","title":"Old:"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#new_1","text":"[Digits] Model=/config/dig0630s3.tflite ;LogImageLocation = /log/digit ModelInputSize 20, 32 digit1, 265, 117, 28, 51 digit2, 310, 117, 28, 51 digit3, 354, 117, 28, 51 digit4, 399, 117, 28, 51 digit5, 445, 115, 28, 51","title":"New:"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#6-make-sure-that-you-have-the-same-quality-and-size-settings-as-in-your-old-configini","text":"In the old configuration this was coded in the html-string for the image source:","title":"6. Make sure that you have the same quality and size settings as in your old Config.ini"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#old_2","text":"URLImageSource=http://IP-ADRESS/capture_with_flashlight?quality=5&size=VGA Default was Quality=5 and VGA.","title":"Old:"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#new_2","text":"ImageQuality = 5 ImageSize = VGA","title":"New:"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#7-repeat-the-same-for-the-analog-section","text":"","title":"7. Repeat the same for the analog section"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#8-insert-your-ssid-and-password-into-the-new-wlanini-file","text":"","title":"8. Insert your SSID and Password into the new wlan.ini File"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#9-compare-and-edit-consistencycheck-section-with-new-postprocessing-section","text":"","title":"9. Compare and edit [ConsistencyCheck] Section with new [PostProcessing] Section"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#10-save-new-configini-file-in-the-new-system","text":"","title":"10.  Save new config.ini File in the new System."},{"location":"outdated--Migrate-Old-Config-To-New-Config/#11-restart-the-system","text":"","title":"11. Restart the system."},{"location":"outdated--Migrate-Old-Config-To-New-Config/#12-after-the-first-start-set-manually-the-prevalue-in-the-new-system","text":"","title":"12. After the first start set manually the PreValue in the new system"},{"location":"rolling-installation/","text":"Living on the edge The branch rolling contains the latest version of the Firmware and the Web Interface. It might already contain a fix for your issue. But it is work in progress, don't expect it to work stable or be an improvement for your AI-on-the-edge-device! Also it might break the OTA Update and thus require manual flashing over USB! You still want to try it? Grab the latest rolling build from Github Actions Page and proceed as following: Pick the most top successful (green) build. Download the AI-on-the-edge-device__update__rolling_*.zip Flash the zip file using the OTA Update page of your device.","title":"Living on the edge"},{"location":"rolling-installation/#living-on-the-edge","text":"The branch rolling contains the latest version of the Firmware and the Web Interface. It might already contain a fix for your issue. But it is work in progress, don't expect it to work stable or be an improvement for your AI-on-the-edge-device! Also it might break the OTA Update and thus require manual flashing over USB!","title":"Living on the edge"},{"location":"rolling-installation/#you-still-want-to-try-it","text":"Grab the latest rolling build from Github Actions Page and proceed as following: Pick the most top successful (green) build. Download the AI-on-the-edge-device__update__rolling_*.zip Flash the zip file using the OTA Update page of your device.","title":"You still want to try it?"}]}
\ No newline at end of file
+{"config":{"indexing":"full","lang":["en"],"min_search_length":3,"prebuild_index":false,"separator":"[\\s\\-]+"},"docs":[{"location":"","text":"Welcome Welcome to the AI-on-the-edge-device project! This is the documentation. For the source code, please head to github.com/jomjol/AI-on-the-edge-device . Artificial intelligence based systems have been established in our every days live. Just think of speech or image recognition. Most of the systems relay on either powerful processors or a direct connection to the cloud for doing the calculations up there. With the increasing power of modern processors the AI systems are coming closer to the end user - which is usually called edge computing . Here this edge computing is brought into a practical oriented example, where a AI network is implemented on a ESP32 device so: AI on the edge . Key features Tensorflow Lite (TFlite) integration - including easy to use wrapper Inline Image processing (feature detection, alignment, ROI extraction) Small and cheap device (3x4.5x2 cm\u00b3, < 10 EUR) camera and illumination integrated Web surface to administrate and control OTA-Interface to update directly through the web interface Full integration into Homeassistant Support for Influx DB 1 MQTT REST API Idea Hardware Web interface Configuration Interface Have fun in studying the new possibilities and ideas This is about image recognition and digitalization, done totally on a cheap ESP32 board using artificial intelligence in form of convolutional neural networks (CNN). Everything, from image capture (OV2640), image preprocessing (auto alignment, ROI identification) all the way down to the image recognition (CNN structure) and result plausibility is done on a cheap 10 EUR device. This all is integrated in an easy to do setup and use environment, taking care for all the background processing and handling, including regular job scheduler. The user interface is an integrated web server, that can be easily adjusted an offers the data as an API in different options. The task to be demonstrated here is a automated readout of an analog water meter. The water consumption is to be recorded within a house automatization and the water meter is totally analog without any electronic interface. Therefore the task is solved by taking regularly an image of the water meter and digitize the reading. There are two types of CNN implemented, a classification network for reading the digital numbers and a single output network for digitalize the analog pointers for the sub digit readings. This project is a evolution of the water-meter-system-complete , which uses ESP32-CAM just for taking the image and a 1GB-Docker image to run the neural networks backbone. Here everything is integrated in an ESP32-CAM module with 8MB of RAM and a SD-Card as data storage.","title":"Welcome"},{"location":"#welcome","text":"Welcome to the AI-on-the-edge-device project! This is the documentation. For the source code, please head to github.com/jomjol/AI-on-the-edge-device . Artificial intelligence based systems have been established in our every days live. Just think of speech or image recognition. Most of the systems relay on either powerful processors or a direct connection to the cloud for doing the calculations up there. With the increasing power of modern processors the AI systems are coming closer to the end user - which is usually called edge computing . Here this edge computing is brought into a practical oriented example, where a AI network is implemented on a ESP32 device so: AI on the edge .","title":"Welcome"},{"location":"#key-features","text":"Tensorflow Lite (TFlite) integration - including easy to use wrapper Inline Image processing (feature detection, alignment, ROI extraction) Small and cheap device (3x4.5x2 cm\u00b3, < 10 EUR) camera and illumination integrated Web surface to administrate and control OTA-Interface to update directly through the web interface Full integration into Homeassistant Support for Influx DB 1 MQTT REST API","title":"Key features"},{"location":"#idea","text":"","title":"Idea"},{"location":"#hardware","text":"","title":"Hardware"},{"location":"#web-interface","text":"","title":"Web interface"},{"location":"#configuration-interface","text":"Have fun in studying the new possibilities and ideas This is about image recognition and digitalization, done totally on a cheap ESP32 board using artificial intelligence in form of convolutional neural networks (CNN). Everything, from image capture (OV2640), image preprocessing (auto alignment, ROI identification) all the way down to the image recognition (CNN structure) and result plausibility is done on a cheap 10 EUR device. This all is integrated in an easy to do setup and use environment, taking care for all the background processing and handling, including regular job scheduler. The user interface is an integrated web server, that can be easily adjusted an offers the data as an API in different options. The task to be demonstrated here is a automated readout of an analog water meter. The water consumption is to be recorded within a house automatization and the water meter is totally analog without any electronic interface. Therefore the task is solved by taking regularly an image of the water meter and digitize the reading. There are two types of CNN implemented, a classification network for reading the digital numbers and a single output network for digitalize the analog pointers for the sub digit readings. This project is a evolution of the water-meter-system-complete , which uses ESP32-CAM just for taking the image and a 1GB-Docker image to run the neural networks backbone. Here everything is integrated in an ESP32-CAM module with 8MB of RAM and a SD-Card as data storage.","title":"Configuration Interface"},{"location":"Additional-Information/","text":"The following links point to additional information in other repos: Digits Training and using a neural network to readout the value of a digital counter Training the CNN neural network Analog Training and using a neural network to read out the value of an analog display Training the CNN neural network","title":"Additional Information"},{"location":"Additional-Information/#digits","text":"Training and using a neural network to readout the value of a digital counter Training the CNN neural network","title":"Digits"},{"location":"Additional-Information/#analog","text":"Training and using a neural network to read out the value of an analog display Training the CNN neural network","title":"Analog"},{"location":"Best-Practice/","text":"Best Practice This page shows some best practices: Camera Placement Move the Camera as close as possible(~4cm), this will help get rid of reflections. -> focus can be adjusted by turning the outer black ring of the camera. If the LED reflections are too strong, put tape over the LED to defuse the light Change the ImageSize to QVGA under \"Expert mode\" configuration when close enough, this will be faster and is often good enough for digital recognition. Reflections Try to get ride of the reflections by rotating the camera, so that the reflections are at positions, where no number is. By using the external LED option, you can place WS2812 LEDs freely away from the main axis. Users report, that a handy cover foil could also help Post-processing Filter out the Number \"9\", as \"3\" will often be misread for a \"9\" and void every number between 3 and 9 due to it being negative flow. Split the readings into two, while the decimal numbers might move to fast to be recognized, at least the slower moving part will produce a correct reading. -> keep in mind that the offset needs to be adjusted, a.e if you have a comma reading of \"3\", it needs to become \"0.3\". This can be done wherever the data ends up being sent, like home assistant using sensor templates. If you are using a low resolution and only digital mode, processing can often be done in <1 minute. Check the logs to confirm how fast it is and then set the interval accordingly under \"Expert mode\" in configuration, as the normal mode will lock you to 3+ minutes.","title":"Best Practice"},{"location":"Best-Practice/#best-practice","text":"This page shows some best practices:","title":"Best Practice"},{"location":"Best-Practice/#camera-placement","text":"Move the Camera as close as possible(~4cm), this will help get rid of reflections. -> focus can be adjusted by turning the outer black ring of the camera. If the LED reflections are too strong, put tape over the LED to defuse the light Change the ImageSize to QVGA under \"Expert mode\" configuration when close enough, this will be faster and is often good enough for digital recognition.","title":"Camera Placement"},{"location":"Best-Practice/#reflections","text":"Try to get ride of the reflections by rotating the camera, so that the reflections are at positions, where no number is. By using the external LED option, you can place WS2812 LEDs freely away from the main axis. Users report, that a handy cover foil could also help","title":"Reflections"},{"location":"Best-Practice/#post-processing","text":"Filter out the Number \"9\", as \"3\" will often be misread for a \"9\" and void every number between 3 and 9 due to it being negative flow. Split the readings into two, while the decimal numbers might move to fast to be recognized, at least the slower moving part will produce a correct reading. -> keep in mind that the offset needs to be adjusted, a.e if you have a comma reading of \"3\", it needs to become \"0.3\". This can be done wherever the data ends up being sent, like home assistant using sensor templates. If you are using a low resolution and only digital mode, processing can often be done in <1 minute. Check the logs to confirm how fast it is and then set the interval accordingly under \"Expert mode\" in configuration, as the normal mode will lock you to 3+ minutes.","title":"Post-processing"},{"location":"Build-Instructions/","text":"Build the Project See README.md","title":"Build the Project"},{"location":"Build-Instructions/#build-the-project","text":"See README.md","title":"Build the Project"},{"location":"Choosing-the-Model/","text":"Model Selection Warning This page overlaps Neural Network Types . They should be merged to one page! In the Graphical Configuration Page , you can choose different models depending on your needs. This wiki page tries to help you on which model to select. For more technical/deeper explanations have a look on Neural-Network-Types . Digit Models For digits on water meters, gas-meters or power meters you can select between two main types of models. dig-class11 This model can recognize full digits. All intermediate states shown a \"N\" for not a number. But in post process it uses older values to fill up the \"N\" values if possible. Main features well suited for LCD digits with the ExtendedResolution option is not supported. (Only in conjunction with ana-class100 / ana-cont) dig-class100 / dig-cont These models are used to get a continuous reading with intermediate states. To see what the models are doing, you can go to the Recognition page. Main features suitable for all digit displays. Advantage over dig-class11 that results continue to be calculated in the transition between digits. With the ExtendedResolution option, higher accuracy is possible by adding another digit. Look here for a list of digit images used for the training dig-class100 vs. dig-cont The difference is in the internal processing. Take the one that gives you the best results. Analog pointer models ana-class100 / ana-cont For pointers on water meters use the analog models. You can only choose between ana-class100 and ana-cont. Both do mainly the same. Main features for all analogue pointers, especially for water meters. With the ExtendedResolution option, higher accuracy is possible by adding another digit. Look here for a list of pointer images used for the training ana-class100 vs. ana-cont The difference is in the internal processing. Take the one that gives you the best results. Both models learn from the same data.","title":"Model Selection"},{"location":"Choosing-the-Model/#model-selection","text":"Warning This page overlaps Neural Network Types . They should be merged to one page! In the Graphical Configuration Page , you can choose different models depending on your needs. This wiki page tries to help you on which model to select. For more technical/deeper explanations have a look on Neural-Network-Types .","title":"Model Selection"},{"location":"Choosing-the-Model/#digit-models","text":"For digits on water meters, gas-meters or power meters you can select between two main types of models.","title":"Digit Models"},{"location":"Choosing-the-Model/#dig-class11","text":"This model can recognize full digits. All intermediate states shown a \"N\" for not a number. But in post process it uses older values to fill up the \"N\" values if possible.","title":"dig-class11"},{"location":"Choosing-the-Model/#main-features","text":"well suited for LCD digits with the ExtendedResolution option is not supported. (Only in conjunction with ana-class100 / ana-cont)","title":"Main features"},{"location":"Choosing-the-Model/#dig-class100-dig-cont","text":"These models are used to get a continuous reading with intermediate states. To see what the models are doing, you can go to the Recognition page.","title":"dig-class100 / dig-cont"},{"location":"Choosing-the-Model/#main-features_1","text":"suitable for all digit displays. Advantage over dig-class11 that results continue to be calculated in the transition between digits. With the ExtendedResolution option, higher accuracy is possible by adding another digit. Look here for a list of digit images used for the training","title":"Main features"},{"location":"Choosing-the-Model/#dig-class100-vs-dig-cont","text":"The difference is in the internal processing. Take the one that gives you the best results.","title":"dig-class100 vs. dig-cont"},{"location":"Choosing-the-Model/#analog-pointer-models","text":"","title":"Analog pointer models"},{"location":"Choosing-the-Model/#ana-class100-ana-cont","text":"For pointers on water meters use the analog models. You can only choose between ana-class100 and ana-cont. Both do mainly the same.","title":"ana-class100 / ana-cont"},{"location":"Choosing-the-Model/#main-features_2","text":"for all analogue pointers, especially for water meters. With the ExtendedResolution option, higher accuracy is possible by adding another digit. Look here for a list of pointer images used for the training","title":"Main features"},{"location":"Choosing-the-Model/#ana-class100-vs-ana-cont","text":"The difference is in the internal processing. Take the one that gives you the best results. Both models learn from the same data.","title":"ana-class100 vs. ana-cont"},{"location":"Configuration-Parameter-Details/","text":"Configuration Parameter Details [MakeImage] [MakeImage] LogImageLocation = /log/source WaitBeforeTakingPicture = 5 LogfileRetentionInDays = 15 Brightness = -2 ;Contrast = 0 ;Saturation = 0 ImageQuality = 5 ImageSize = VGA FixedExposure = true Parameter Meaning Options/Examples LogImageLocation location for storing a copy of the image WaitBeforeTakingPicture waiting time between switch on the flash-light and taking the image in seconds LogfileRetentionInDays Number of days, for which the log files should be stored 0 = keep forever Brightness Adjustment of the camera brightness (-2 ... 2) Contrast NOT IMPLEMENTED Saturation NOT IMPLEMENTED ImageQuality Input image jpg-compression quality 0 (best) to 100 (lowest) 5 = default ImageSize Input Image Size from Camera only VGA, QVGA FixedExposure If enabled, the exposure settings are fixed at the beginning and the waiting time after switching on the illumination will be skipped [Alignment] [Alignment] InitialRotate = 179 InitialMirror = false SearchFieldX = 20 SearchFieldY = 20 AlignmentAlgo = Default FlipImageSize = false /config/ref0.jpg 104 271 /config/ref1.jpg 442 142 Parameter Meaning Options/Examples InitialMirror Option for initially mirroring the image on the original x-axis InitialRotate Initial rotation of image before alignment in degree (1...359) FlipImageSize Changes the aspect ratio after the image rotation to avoid cropping of the rotated imaged /config/refx.jpg 98, 257 Link to reference image and corresponding target coordinates file link is relative to sd-card root SearchFieldX/Y Search field size in X/Y for finding the reference images [pixel] Here two reference images are needed. Therefore rotation and shifting can be compensated. As the alignment is one of the most computing time using part, the search field needs to be limited. The calculation time goes quadratic with the search field size. [Digits] [Digits] Model=/config/digits.tfl ModelInputSize 20, 32 LogImageLocation = /log/digit LogfileRetentionInDays = 2 number1.digit1 292 120 37 67 number1.digit2 340 120 37 67 number1.digit3 389 120 37 67 number2.digit1 292 180 37 67 number2.digit2 340 180 37 67 Parameter Meaning Options/Examples Model Link to the CNN-tflite file used for AI-image recognition ModelInputSize Image input size for the CNN-Network [pixel] needed to resize the ROI to the input size LogImageLocation storage location for the recognized images, including the CNN-results in the file name/location numberX.digitY ROI for the corresponding digit in the aligned image. More than one number can be specified. Therefore the name consists of a naming of the number ( numberX ) and the region of interest ( digitY ) - separated by . LogfileRetentionInDays Number of days, for which the log files should be stored 0 = keep forever LogImageLocation location for storing a copy of the image [Analog] [Analog] Model=/config/analog.tfl ModelInputSize 32, 32 LogImageLocation = /log/analog LogfileRetentionInDays = 2 number1.analog1, 433, 207, 99, 99 number1.analog2, 378, 313, 99, 99 number1.analog3, 280, 356, 99, 99 number1.analog4, 149, 313, 99, 99 number2.analog1, 280, 456, 99, 99 number2.analog2, 149, 413, 99, 99 Same as for [digit], here only for the analog pointers [PostProcessing] [PostProcessing] number1.DecimalShift = 0 number2.DecimalShift = -1 PreValueUse = true PreValueAgeStartup = 720 AllowNegativeRates = false number1.MaxRateValue = 0.1 number2.MaxRateValue = 0.1 ErrorMessage = true CheckDigitIncreaseConsistency = false Here the post processing and consistency check for the readout can be adjusted Parameter Meaning Options/Examples PreValueUse Use the previous value for consistency check and substitution for NaN (True / False) PreValueAgeStartup Max age of PreValue after a reboot (downtime) AllowNegativeRates Allow decrease of the readout value numberX.MaxRateValue Maximum chance rate from one to the next readout. This can be specified for each number individual. ErrorMessage Show error messages numberX.DecimalShift Shifting of the decimal separator from the default position between digital and analog. This can be specified for each number individual. DecimalShift = 2: 123.456 --> 12345.6 DecimalShift = -1: 123.456 --> 12.3456 CheckDigitIncreaseConsistency This parameter controls, if the digits are checked for a consistent change in comparison to the previous value. This only makes sense, if the last digit is changing very slowly and every single digit is visible (e.g. 4.7 --> 4.8 --> 4.9 --> 5.0 --> 5.1). If single digits are skipped, for example because the digits changes to fast, this should be disabled (e.g. 4.7 --> 5.0 --> 5.1). [MQTT] [MQTT] Uri = mqtt://IP-ADRESS:1883 MainTopic = wasserzaehler ClientID = wasser user = USERNAME password = PASSWORD Here the post processing and consistency check for the readout can be adjusted Parameter Meaning Options/Examples Uri URI to the MQTT broker including port e.g.: mqtt://IP-Address:Port MainTopic MQTT main topic, under which the counters are published. The single value will be published with the following key: MainTopic/number/PARAMETER where parameters are: value, rate, timestamp, error and json The general connection status can be found in MainTopic/connection ClientID ClientID to connect to the MQTT broker user user for MQTT authentication (optional) password password for MQTT authentication (optional) [AutoTimer] [AutoTimer] AutoStart= true Intervall = 4.85 This paragraph is used to automatically trigger the periodic automated readout. Parameter Meaning Options/Examples AutoStart Automatically trigger the readout after startup Intervall Readout interval in minutes Values smaller than 2 minutes do not make sense, as this is the time for one detection [Debug] [Debug] Logfile = true LogfileRetentionInDays = 2 This paragraph is used to switch on an extended logging. It is optional and by default only a minimum logging is enabled. \u203c\ufe0f Attention: in case of extended logging the size of the log file ( /log.txt , /alignment.txt ) might increase rapidly, therefore manually deletion from time to time is recommended Parameter Meaning Options/Examples Logfile Turn on (true) or off (false) the extended logging parameter and section optional LogfileRetentionInDays Number of days, for which the log files should be stored 0 = keep forever [System] [System] TimeZone = CET-1CEST,M3.5.0,M10.5.0/3 ;TimeServer = TIMESERVER ;Hostname = undefined SetupMode = false This paragraph is used to switch on an extended logging. It is optional and by default only a minimum logging is enabled. \u203c\ufe0f Attention: in case of extended logging the size of the log file ( /log.txt , /alignment.txt ) might increase rapidly, therefore manually deletion from time to time is recommended Parameter Meaning Options/Examples TimeZone TimeZone of the system can be specified Central european, with summertime adjustement: CET-1CEST,M3.5.0,M10.5.0/3 TimeServer An dedicated time server can be specified. default = pool.ntp.org Hostname Additionally to the wlan.ini the hostname can be specified. It will be transferred to the wlan.ini and initiate a reboot SetupMode If enabled, the server starts in an initial setup mode. This is automatically disabled at the end of the setup [Ende] No function, just to mark, that the config is done!","title":"Configuration Parameter Details"},{"location":"Configuration-Parameter-Details/#configuration-parameter-details","text":"","title":"Configuration Parameter Details"},{"location":"Configuration-Parameter-Details/#makeimage","text":"[MakeImage] LogImageLocation = /log/source WaitBeforeTakingPicture = 5 LogfileRetentionInDays = 15 Brightness = -2 ;Contrast = 0 ;Saturation = 0 ImageQuality = 5 ImageSize = VGA FixedExposure = true Parameter Meaning Options/Examples LogImageLocation location for storing a copy of the image WaitBeforeTakingPicture waiting time between switch on the flash-light and taking the image in seconds LogfileRetentionInDays Number of days, for which the log files should be stored 0 = keep forever Brightness Adjustment of the camera brightness (-2 ... 2) Contrast NOT IMPLEMENTED Saturation NOT IMPLEMENTED ImageQuality Input image jpg-compression quality 0 (best) to 100 (lowest) 5 = default ImageSize Input Image Size from Camera only VGA, QVGA FixedExposure If enabled, the exposure settings are fixed at the beginning and the waiting time after switching on the illumination will be skipped","title":"[MakeImage]"},{"location":"Configuration-Parameter-Details/#alignment","text":"[Alignment] InitialRotate = 179 InitialMirror = false SearchFieldX = 20 SearchFieldY = 20 AlignmentAlgo = Default FlipImageSize = false /config/ref0.jpg 104 271 /config/ref1.jpg 442 142 Parameter Meaning Options/Examples InitialMirror Option for initially mirroring the image on the original x-axis InitialRotate Initial rotation of image before alignment in degree (1...359) FlipImageSize Changes the aspect ratio after the image rotation to avoid cropping of the rotated imaged /config/refx.jpg 98, 257 Link to reference image and corresponding target coordinates file link is relative to sd-card root SearchFieldX/Y Search field size in X/Y for finding the reference images [pixel] Here two reference images are needed. Therefore rotation and shifting can be compensated. As the alignment is one of the most computing time using part, the search field needs to be limited. The calculation time goes quadratic with the search field size.","title":"[Alignment]"},{"location":"Configuration-Parameter-Details/#digits","text":"[Digits] Model=/config/digits.tfl ModelInputSize 20, 32 LogImageLocation = /log/digit LogfileRetentionInDays = 2 number1.digit1 292 120 37 67 number1.digit2 340 120 37 67 number1.digit3 389 120 37 67 number2.digit1 292 180 37 67 number2.digit2 340 180 37 67 Parameter Meaning Options/Examples Model Link to the CNN-tflite file used for AI-image recognition ModelInputSize Image input size for the CNN-Network [pixel] needed to resize the ROI to the input size LogImageLocation storage location for the recognized images, including the CNN-results in the file name/location numberX.digitY ROI for the corresponding digit in the aligned image. More than one number can be specified. Therefore the name consists of a naming of the number ( numberX ) and the region of interest ( digitY ) - separated by . LogfileRetentionInDays Number of days, for which the log files should be stored 0 = keep forever LogImageLocation location for storing a copy of the image","title":"[Digits]"},{"location":"Configuration-Parameter-Details/#analog","text":"[Analog] Model=/config/analog.tfl ModelInputSize 32, 32 LogImageLocation = /log/analog LogfileRetentionInDays = 2 number1.analog1, 433, 207, 99, 99 number1.analog2, 378, 313, 99, 99 number1.analog3, 280, 356, 99, 99 number1.analog4, 149, 313, 99, 99 number2.analog1, 280, 456, 99, 99 number2.analog2, 149, 413, 99, 99 Same as for [digit], here only for the analog pointers","title":"[Analog]"},{"location":"Configuration-Parameter-Details/#postprocessing","text":"[PostProcessing] number1.DecimalShift = 0 number2.DecimalShift = -1 PreValueUse = true PreValueAgeStartup = 720 AllowNegativeRates = false number1.MaxRateValue = 0.1 number2.MaxRateValue = 0.1 ErrorMessage = true CheckDigitIncreaseConsistency = false Here the post processing and consistency check for the readout can be adjusted Parameter Meaning Options/Examples PreValueUse Use the previous value for consistency check and substitution for NaN (True / False) PreValueAgeStartup Max age of PreValue after a reboot (downtime) AllowNegativeRates Allow decrease of the readout value numberX.MaxRateValue Maximum chance rate from one to the next readout. This can be specified for each number individual. ErrorMessage Show error messages numberX.DecimalShift Shifting of the decimal separator from the default position between digital and analog. This can be specified for each number individual. DecimalShift = 2: 123.456 --> 12345.6 DecimalShift = -1: 123.456 --> 12.3456 CheckDigitIncreaseConsistency This parameter controls, if the digits are checked for a consistent change in comparison to the previous value. This only makes sense, if the last digit is changing very slowly and every single digit is visible (e.g. 4.7 --> 4.8 --> 4.9 --> 5.0 --> 5.1). If single digits are skipped, for example because the digits changes to fast, this should be disabled (e.g. 4.7 --> 5.0 --> 5.1).","title":"[PostProcessing]"},{"location":"Configuration-Parameter-Details/#mqtt","text":"[MQTT] Uri = mqtt://IP-ADRESS:1883 MainTopic = wasserzaehler ClientID = wasser user = USERNAME password = PASSWORD Here the post processing and consistency check for the readout can be adjusted Parameter Meaning Options/Examples Uri URI to the MQTT broker including port e.g.: mqtt://IP-Address:Port MainTopic MQTT main topic, under which the counters are published. The single value will be published with the following key: MainTopic/number/PARAMETER where parameters are: value, rate, timestamp, error and json The general connection status can be found in MainTopic/connection ClientID ClientID to connect to the MQTT broker user user for MQTT authentication (optional) password password for MQTT authentication (optional)","title":"[MQTT]"},{"location":"Configuration-Parameter-Details/#autotimer","text":"[AutoTimer] AutoStart= true Intervall = 4.85 This paragraph is used to automatically trigger the periodic automated readout. Parameter Meaning Options/Examples AutoStart Automatically trigger the readout after startup Intervall Readout interval in minutes Values smaller than 2 minutes do not make sense, as this is the time for one detection","title":"[AutoTimer]"},{"location":"Configuration-Parameter-Details/#debug","text":"[Debug] Logfile = true LogfileRetentionInDays = 2 This paragraph is used to switch on an extended logging. It is optional and by default only a minimum logging is enabled. \u203c\ufe0f Attention: in case of extended logging the size of the log file ( /log.txt , /alignment.txt ) might increase rapidly, therefore manually deletion from time to time is recommended Parameter Meaning Options/Examples Logfile Turn on (true) or off (false) the extended logging parameter and section optional LogfileRetentionInDays Number of days, for which the log files should be stored 0 = keep forever","title":"[Debug]"},{"location":"Configuration-Parameter-Details/#system","text":"[System] TimeZone = CET-1CEST,M3.5.0,M10.5.0/3 ;TimeServer = TIMESERVER ;Hostname = undefined SetupMode = false This paragraph is used to switch on an extended logging. It is optional and by default only a minimum logging is enabled. \u203c\ufe0f Attention: in case of extended logging the size of the log file ( /log.txt , /alignment.txt ) might increase rapidly, therefore manually deletion from time to time is recommended Parameter Meaning Options/Examples TimeZone TimeZone of the system can be specified Central european, with summertime adjustement: CET-1CEST,M3.5.0,M10.5.0/3 TimeServer An dedicated time server can be specified. default = pool.ntp.org Hostname Additionally to the wlan.ini the hostname can be specified. It will be transferred to the wlan.ini and initiate a reboot SetupMode If enabled, the server starts in an initial setup mode. This is automatically disabled at the end of the setup","title":"[System]"},{"location":"Configuration-Parameter-Details/#ende","text":"No function, just to mark, that the config is done!","title":"[Ende]"},{"location":"Configuration/","text":"Configuration Warning This page overlaps Graphical-Configuration . They should be merged to one page! Most of the settings can be modified with the help of a web based graphical user interface . This is hosted through the web server on the ESP32. More configuration parameters can be edited by hand in the config.ini and corresponding files in the /config directory on the SD-card. If you where using the Version 1 of the watermeter you can easily transfer the configuration to the new system by following the steps in this migration description Processing / Config.ini principle The principle is very simple and can most easily be described as a flow of processing steps. Each step has a dedicated parameter description in the config.ini , which is indicated by brackets [name_of_step] . The steps are processed in the order written in the config file. That means, that you first have to describe the image taking, then the aligning and cutting and only after that you can start to config a neural network. The last step is the post processing. Processing steps - Overview In the following you get a short overview over the available steps. This order is also the suggested order for the processing flow. Single steps can be left out, if not needed (e.g. omit the analog part, if only digits are present) 1. [MakeImage] This steps parametrises the taking of the image by the ESP32-CAM. Size, quality and storage for logging and debugging can be set. 2. [Alignment] Image preprocessing, including image alignment with reference images 3. [Digits] Neural network evaluation of an image for digits. The neural network is defined by a tflite formatted file and the output is a number between 0 .. 9 or NaN (if image is not unique enough) 4. [Analog] Neural network evaluation of analog counter. The neural network is defined by a tflite formatted file and the output is a number between 0.0 .. 9.9, representing the position of the pointer. 5. [PostProcessing] Summarized the individually converted pictures to the overall result. It also implements some error corrections and consistency checks to filter wrong reading. 6. [MQTT] Transfer of the readings to a MQTT server. 7. [AutoTimer] Configuration of the automated flow start at the start up of the ESP32. 8. [Debug] Configuration for debugging details 9. [Ende] No meaning, just an additional indication, that the configuration is finished. A detailed parameter description can be found here: [[Configuration Parameter Details]]. Graphical configuration interface It is recommended to do the configuration of the alignment structures and ROIs through the graphical user interface. A step by step instruction can be found here: [[Graphical Configuration]] Background for Image Alignment Details on the image recognition flow can be found in the other project here: https://github.com/jomjol/water-meter-system-complete/blob/master/images/Alignment_procedure_draft.pdf The config.ini here has the same functionality and options, but a slightly different syntax due to a own written ini-parser is used. Migration see here . Integration into Home Assistant Thanks to the help of the user @deadly667 here are some hints for the integration into the home assistant: [[Integration-Home-Assistant]]","title":"Configuration"},{"location":"Configuration/#configuration","text":"Warning This page overlaps Graphical-Configuration . They should be merged to one page! Most of the settings can be modified with the help of a web based graphical user interface . This is hosted through the web server on the ESP32. More configuration parameters can be edited by hand in the config.ini and corresponding files in the /config directory on the SD-card. If you where using the Version 1 of the watermeter you can easily transfer the configuration to the new system by following the steps in this migration description","title":"Configuration"},{"location":"Configuration/#processing-configini-principle","text":"The principle is very simple and can most easily be described as a flow of processing steps. Each step has a dedicated parameter description in the config.ini , which is indicated by brackets [name_of_step] . The steps are processed in the order written in the config file. That means, that you first have to describe the image taking, then the aligning and cutting and only after that you can start to config a neural network. The last step is the post processing.","title":"Processing / Config.ini principle"},{"location":"Configuration/#processing-steps-overview","text":"In the following you get a short overview over the available steps. This order is also the suggested order for the processing flow. Single steps can be left out, if not needed (e.g. omit the analog part, if only digits are present)","title":"Processing steps - Overview"},{"location":"Configuration/#1-makeimage","text":"This steps parametrises the taking of the image by the ESP32-CAM. Size, quality and storage for logging and debugging can be set.","title":"1. [MakeImage]"},{"location":"Configuration/#2-alignment","text":"Image preprocessing, including image alignment with reference images","title":"2. [Alignment]"},{"location":"Configuration/#3-digits","text":"Neural network evaluation of an image for digits. The neural network is defined by a tflite formatted file and the output is a number between 0 .. 9 or NaN (if image is not unique enough)","title":"3. [Digits]"},{"location":"Configuration/#4-analog","text":"Neural network evaluation of analog counter. The neural network is defined by a tflite formatted file and the output is a number between 0.0 .. 9.9, representing the position of the pointer.","title":"4. [Analog]"},{"location":"Configuration/#5-postprocessing","text":"Summarized the individually converted pictures to the overall result. It also implements some error corrections and consistency checks to filter wrong reading.","title":"5. [PostProcessing]"},{"location":"Configuration/#6-mqtt","text":"Transfer of the readings to a MQTT server.","title":"6. [MQTT]"},{"location":"Configuration/#7-autotimer","text":"Configuration of the automated flow start at the start up of the ESP32.","title":"7. [AutoTimer]"},{"location":"Configuration/#8-debug","text":"Configuration for debugging details","title":"8. [Debug]"},{"location":"Configuration/#9-ende","text":"No meaning, just an additional indication, that the configuration is finished. A detailed parameter description can be found here: [[Configuration Parameter Details]].","title":"9. [Ende]"},{"location":"Configuration/#graphical-configuration-interface","text":"It is recommended to do the configuration of the alignment structures and ROIs through the graphical user interface. A step by step instruction can be found here: [[Graphical Configuration]]","title":"Graphical configuration interface"},{"location":"Configuration/#background-for-image-alignment","text":"Details on the image recognition flow can be found in the other project here: https://github.com/jomjol/water-meter-system-complete/blob/master/images/Alignment_procedure_draft.pdf The config.ini here has the same functionality and options, but a slightly different syntax due to a own written ini-parser is used. Migration see here .","title":"Background for Image Alignment"},{"location":"Configuration/#integration-into-home-assistant","text":"Thanks to the help of the user @deadly667 here are some hints for the integration into the home assistant: [[Integration-Home-Assistant]]","title":"Integration into Home Assistant"},{"location":"Correction%20Algorithm/","text":"Correction Algorithm After the digitization of the images and the composition to a number a checking and correction algorithm is applied. This is explained here. There are several reasons, that a check might be necessary: In case of digits there is the output of \"N\" (=NaN = Not-a-Number) in case the digit cannot be detected correctly. This happens for example if the image shows a digit between to states The replacement of the \"N\" with a previous value could be not sufficient, due to the fact, that it might have changed. There is a misreading of one one of the numbers. This can always happen in case of neural network processing. Terms and definitions PreValue The last correct read value. Either from a previous correctly identified value or manual setting by the user. This is used to replace \"N\"s and make a check for the absolute change. Digits Value that are digitized from a digital number. There are 11 allowed values for this: Digits: 0, 1, 2, ... 9 N = Not-a-Number - representing a not unique state between two numbers Analogs This are value derived from a pointer like meter. This never has the state \"N\". CheckDigitIncreaseConsistency If this is enabled an \"inteligent\" algorithm is used to derive from zero-crossing of discrete digit positions, if the number should have been increased. This is relevant because in some of the digit meters, the increase of a digit to the next number can be seen, before the subdigit has gone through zero. For example: 16.6 --> 16.7 --> 1N.8 --> 17.9 corrected to 16.9 --> 17.0 --> 17.1 As you can see, the 17.9 is a false reading as the 7 is assumed to be already readable, although the subdigit has not crossed the zero. In this case the CheckDigitIncreaseConsistency algorithm will correct this to 16.9 A detailed description of the algorithm can be found below (not yet ready!) Negative Rate allowed Most of the meters only have increasing numbers and do not count backwards. Therefore a negative rate (= negative change compared to the PreValue) is surely a false value. This can be checked an flagged as false reading MaxRateValue / MaxRateType Here the maximum change from one to the next reading can be limited. If a false reading of the neural network results in a change larger than this, the reading is flagged as false. There a two types of comparisons possible 1) AbsolutChange : Here the difference between the PreValue and the current reading is compared directly, independent how much time has passed since the last reading. 2) RelativeRate : in this case a change rate in the unit of change/minute is calculated, taking the time between the last and the current reading into account. Be careful, that with increasing time, the absolute allowed change increases. Example: relative rate of 0.05 m\u00b3/minute --> after 20 minutes a maximum change of 20 minutes * 0.05 m\u00b3/minute = 1 m\u00b3 is possible. That means that a false reading of 1 m\u00b3 cannot be detected false after about 20 minutes in this case Assume, that there might me no change in the meter for hours (e.g. during the night) a much bigger change could also be accepted. Flow Chart CheckDigitIncreaseConsistency Algorithm The check digit increase consistency algorithm is functional for the digits only. Due to the fact, that the rotation might be a little bit earlier or later compared to the zero crossing of the digit before, errors during the reading before and after a zero crossing can be wrong. Therefore a simple algorithm can be applied, checking the consistency of zero crossing and changes in the following digit. This is applied to one after the other digit, starting with the lowest priority digits.","title":"Correction Algorithm"},{"location":"Correction%20Algorithm/#correction-algorithm","text":"After the digitization of the images and the composition to a number a checking and correction algorithm is applied. This is explained here. There are several reasons, that a check might be necessary: In case of digits there is the output of \"N\" (=NaN = Not-a-Number) in case the digit cannot be detected correctly. This happens for example if the image shows a digit between to states The replacement of the \"N\" with a previous value could be not sufficient, due to the fact, that it might have changed. There is a misreading of one one of the numbers. This can always happen in case of neural network processing.","title":"Correction Algorithm"},{"location":"Correction%20Algorithm/#terms-and-definitions","text":"","title":"Terms and definitions"},{"location":"Correction%20Algorithm/#prevalue","text":"The last correct read value. Either from a previous correctly identified value or manual setting by the user. This is used to replace \"N\"s and make a check for the absolute change.","title":"PreValue"},{"location":"Correction%20Algorithm/#digits","text":"Value that are digitized from a digital number. There are 11 allowed values for this: Digits: 0, 1, 2, ... 9 N = Not-a-Number - representing a not unique state between two numbers","title":"Digits"},{"location":"Correction%20Algorithm/#analogs","text":"This are value derived from a pointer like meter. This never has the state \"N\".","title":"Analogs"},{"location":"Correction%20Algorithm/#checkdigitincreaseconsistency","text":"If this is enabled an \"inteligent\" algorithm is used to derive from zero-crossing of discrete digit positions, if the number should have been increased. This is relevant because in some of the digit meters, the increase of a digit to the next number can be seen, before the subdigit has gone through zero. For example: 16.6 --> 16.7 --> 1N.8 --> 17.9 corrected to 16.9 --> 17.0 --> 17.1 As you can see, the 17.9 is a false reading as the 7 is assumed to be already readable, although the subdigit has not crossed the zero. In this case the CheckDigitIncreaseConsistency algorithm will correct this to 16.9 A detailed description of the algorithm can be found below (not yet ready!)","title":"CheckDigitIncreaseConsistency"},{"location":"Correction%20Algorithm/#negative-rate-allowed","text":"Most of the meters only have increasing numbers and do not count backwards. Therefore a negative rate (= negative change compared to the PreValue) is surely a false value. This can be checked an flagged as false reading","title":"Negative Rate allowed"},{"location":"Correction%20Algorithm/#maxratevalue-maxratetype","text":"Here the maximum change from one to the next reading can be limited. If a false reading of the neural network results in a change larger than this, the reading is flagged as false. There a two types of comparisons possible 1) AbsolutChange : Here the difference between the PreValue and the current reading is compared directly, independent how much time has passed since the last reading. 2) RelativeRate : in this case a change rate in the unit of change/minute is calculated, taking the time between the last and the current reading into account. Be careful, that with increasing time, the absolute allowed change increases. Example: relative rate of 0.05 m\u00b3/minute --> after 20 minutes a maximum change of 20 minutes * 0.05 m\u00b3/minute = 1 m\u00b3 is possible. That means that a false reading of 1 m\u00b3 cannot be detected false after about 20 minutes in this case Assume, that there might me no change in the meter for hours (e.g. during the night) a much bigger change could also be accepted.","title":"MaxRateValue / MaxRateType"},{"location":"Correction%20Algorithm/#flow-chart","text":"","title":"Flow Chart"},{"location":"Correction%20Algorithm/#checkdigitincreaseconsistency-algorithm","text":"The check digit increase consistency algorithm is functional for the digits only. Due to the fact, that the rotation might be a little bit earlier or later compared to the zero crossing of the digit before, errors during the reading before and after a zero crossing can be wrong. Therefore a simple algorithm can be applied, checking the consistency of zero crossing and changes in the following digit. This is applied to one after the other digit, starting with the lowest priority digits.","title":"CheckDigitIncreaseConsistency Algorithm"},{"location":"Demo-Mode/","text":"Demo Mode For Demo and Testing Purpose, the device can use pre-recorded images. You need to enable it in the configuration ( TakeImage > Demo ) and also provide the needed files on the SD-Card. For each round one image gets used, starting with the first image for the first round. For the reference image and the alignment also the first image gets used. Once the last image got reached, it starts again with the first one. SD-Card Structure demo/ \u251c\u2500\u2500 520.8983.jpg \u251c\u2500\u2500 520.9086.jpg \u251c\u2500\u2500 520.9351.jpg \u251c\u2500\u2500 ... \u2514\u2500\u2500 files.txt The jpg files can have any name The jpg files must be smaller than 30'000 bytes The files.txt must contains a list of those files, eg: 520.8983.jpg 520.9086.jpg 520.9351.jpg Recording To record real images of a meter, you have to periodically fetch http://<IP>/img_tmp/raw.jpg . To automate this, you can use the following shell script (Linux only): #!/bin/bash while [[ true ]]; do echo \"fetching value...\" wget -q http://192.168.1.151/value -O value.txt value=`cat value.txt` echo \"Value: $value\" diff=`diff value.txt value_previous.txt` changed=$? #echo \"Diff: $diff\" if [[ $changed -ne 0 ]]; then echo \"Value changed:\" echo $diff echo \"fetching image...\" wget -q http://192.168.1.151/img_tmp/raw.jpg -O $value.jpg else echo \"Value did not change, skipping image fetching!\" fi cp value.txt value_previous.txt echo \"waiting 60s...\" sleep 60 done Installation Just install the zip file using the OTA Update functionality. How does it work The Demo Mode tries to interfere as less as possible with the normal behavior. Whenever a Cam Framebuffer gets taken ( esp_camera_fb_get() ), it replaces the framebuffer with the image from the SD-Card. Example Data of a Watermeter You can use the following demo images if you want: It covers a meter range from 530.00688 to 531.85882 . Animation Animation of the watermeter (77 MB!) All images (843 images) Demo_Images_Watermeter_530.00688-532.08243_843_images.zip Animation of it (186 MB!) Selection of 84 images Demo_Images_Watermeter_530.00688-532.08243_84_images.zip Selection of 42 images Demo_Images_Watermeter_530.00688-532.08243_42_images.zip","title":"Demo Mode"},{"location":"Demo-Mode/#demo-mode","text":"For Demo and Testing Purpose, the device can use pre-recorded images. You need to enable it in the configuration ( TakeImage > Demo ) and also provide the needed files on the SD-Card. For each round one image gets used, starting with the first image for the first round. For the reference image and the alignment also the first image gets used. Once the last image got reached, it starts again with the first one.","title":"Demo Mode"},{"location":"Demo-Mode/#sd-card-structure","text":"demo/ \u251c\u2500\u2500 520.8983.jpg \u251c\u2500\u2500 520.9086.jpg \u251c\u2500\u2500 520.9351.jpg \u251c\u2500\u2500 ... \u2514\u2500\u2500 files.txt The jpg files can have any name The jpg files must be smaller than 30'000 bytes The files.txt must contains a list of those files, eg: 520.8983.jpg 520.9086.jpg 520.9351.jpg","title":"SD-Card Structure"},{"location":"Demo-Mode/#recording","text":"To record real images of a meter, you have to periodically fetch http://<IP>/img_tmp/raw.jpg . To automate this, you can use the following shell script (Linux only): #!/bin/bash while [[ true ]]; do echo \"fetching value...\" wget -q http://192.168.1.151/value -O value.txt value=`cat value.txt` echo \"Value: $value\" diff=`diff value.txt value_previous.txt` changed=$? #echo \"Diff: $diff\" if [[ $changed -ne 0 ]]; then echo \"Value changed:\" echo $diff echo \"fetching image...\" wget -q http://192.168.1.151/img_tmp/raw.jpg -O $value.jpg else echo \"Value did not change, skipping image fetching!\" fi cp value.txt value_previous.txt echo \"waiting 60s...\" sleep 60 done","title":"Recording"},{"location":"Demo-Mode/#installation","text":"Just install the zip file using the OTA Update functionality.","title":"Installation"},{"location":"Demo-Mode/#how-does-it-work","text":"The Demo Mode tries to interfere as less as possible with the normal behavior. Whenever a Cam Framebuffer gets taken ( esp_camera_fb_get() ), it replaces the framebuffer with the image from the SD-Card.","title":"How does it work"},{"location":"Demo-Mode/#example-data-of-a-watermeter","text":"You can use the following demo images if you want: It covers a meter range from 530.00688 to 531.85882 .","title":"Example Data of a Watermeter"},{"location":"Demo-Mode/#animation","text":"Animation of the watermeter (77 MB!)","title":"Animation"},{"location":"Demo-Mode/#all-images-843-images","text":"Demo_Images_Watermeter_530.00688-532.08243_843_images.zip Animation of it (186 MB!)","title":"All images (843 images)"},{"location":"Demo-Mode/#selection-of-84-images","text":"Demo_Images_Watermeter_530.00688-532.08243_84_images.zip","title":"Selection of 84 images"},{"location":"Demo-Mode/#selection-of-42-images","text":"Demo_Images_Watermeter_530.00688-532.08243_42_images.zip","title":"Selection of 42 images"},{"location":"Error-Codes/","text":"This page lists the possible error codes, their meaning and possible solutions. The effective error codes can be found here . Critical Errors Those Errors make the normal operation of the device impossible. Most likely they are caused by a hardware issue! 0x00000001 PSRAM bad Your device most likely has no PSRAM at all or it is too small (needs to have at least 4 MBytes)! See Hardware Compatibility . Usually the log shows something like this: psram: PSRAM ID read error: 0xffffffff cpu_start: Failed to init external RAM! 0x00000002 Heap too small The firmware failed to allocate enough memory. This most likely is a consequential error of a bad PSRAM! 0x00000004 Cam bad The attached camera can not be initialized. This usually is because on of the following reasons: - The camera is not supported, see Hardware Compatibility - The camera is not attached properly -> Try to remove and attach it again. Make sure you move the black part enough into the socket! - The camera or the camera cable is damaged Non-Critical Errors Those Errors can be caused by an error during initialization. It is possible that the error has no impact at all or that a reboot solves it. 0x00000100 Cam Framebuffer bad The firmware was unable to initialize the Camera Framebuffer. The firmware will continue to work, but other consequential error might arise. A reboot of the device might help. 0x00000200 NTP failed The firmware failed to get the world time from an NTP server. The firmware will continue to work, but has a wrong time.","title":"Error Codes"},{"location":"Error-Codes/#critical-errors","text":"Those Errors make the normal operation of the device impossible. Most likely they are caused by a hardware issue!","title":"Critical Errors"},{"location":"Error-Codes/#0x00000001-psram-bad","text":"Your device most likely has no PSRAM at all or it is too small (needs to have at least 4 MBytes)! See Hardware Compatibility . Usually the log shows something like this: psram: PSRAM ID read error: 0xffffffff cpu_start: Failed to init external RAM!","title":"0x00000001 PSRAM bad"},{"location":"Error-Codes/#0x00000002-heap-too-small","text":"The firmware failed to allocate enough memory. This most likely is a consequential error of a bad PSRAM!","title":"0x00000002 Heap too small"},{"location":"Error-Codes/#0x00000004-cam-bad","text":"The attached camera can not be initialized. This usually is because on of the following reasons: - The camera is not supported, see Hardware Compatibility - The camera is not attached properly -> Try to remove and attach it again. Make sure you move the black part enough into the socket! - The camera or the camera cable is damaged","title":"0x00000004 Cam bad"},{"location":"Error-Codes/#non-critical-errors","text":"Those Errors can be caused by an error during initialization. It is possible that the error has no impact at all or that a reboot solves it.","title":"Non-Critical Errors"},{"location":"Error-Codes/#0x00000100-cam-framebuffer-bad","text":"The firmware was unable to initialize the Camera Framebuffer. The firmware will continue to work, but other consequential error might arise. A reboot of the device might help.","title":"0x00000100 Cam Framebuffer bad"},{"location":"Error-Codes/#0x00000200-ntp-failed","text":"The firmware failed to get the world time from an NTP server. The firmware will continue to work, but has a wrong time.","title":"0x00000200 NTP failed"},{"location":"Error-Debugging/","text":"Error Debugging Rebooting General Remark Due to the rather complex code with a lot of external libraries and the limited availability of memory a reboot of the device from time to time is \"normal\". Background are memory leakages and therefore running out of free memory. The hardware of the ESP32CAM has a varying quality. I have one and the same hardware with a reboot range from every 5 detection runs to up to 250 detection runs. Getting deeper inside Have a look into the log file ( /log/message/... ). If the log file is very short you need to enable a enhanced logging in the config.ini (Debug --> logfile = true ) . Analyze the debugging output of the serial interface Connect a serial to USB interface (like for flashing) and make a logging of the serial communication There are a lot more intermediate information and the lines before the reboot tell you, where the firmware fails If you make an issue about this, please post these two information additionally Don't forget to remove your WLAN password in the serial log Often observed problems Hardware failure Camera not working --> check the interface, test another module Low cost module with only 2MB of PSRAM instead of 4MB --> image taking will fail first. This will never work due to too low memory ROI misaligned This typically happens if you have suboptimal \"Alignement Marks\". A very simple and working solution is to put put higly contrasted stickers on your meter and put \"Alignement Marks\" on it (see picture below) If after those adjustement you still have some issues, you can try to adjust your aligmenet settings in expert mode: My Analog Meter are recognized as Digital Counter or vice versa First, check that your ROI are correctly defined (yey!) Second, verify that the name of your ROI analog and digital ROIs are different Recognition is working well, but number aren't sorted correctly You have to sort your ROI correctly (Bigger to smaller). Select your ROI and click either \"move next\" or \"move previous\". Repeat until your ROI are correctly sorted","title":"Error Debugging"},{"location":"Error-Debugging/#error-debugging","text":"","title":"Error Debugging"},{"location":"Error-Debugging/#rebooting","text":"","title":"Rebooting"},{"location":"Error-Debugging/#general-remark","text":"Due to the rather complex code with a lot of external libraries and the limited availability of memory a reboot of the device from time to time is \"normal\". Background are memory leakages and therefore running out of free memory. The hardware of the ESP32CAM has a varying quality. I have one and the same hardware with a reboot range from every 5 detection runs to up to 250 detection runs.","title":"General Remark"},{"location":"Error-Debugging/#getting-deeper-inside","text":"Have a look into the log file ( /log/message/... ). If the log file is very short you need to enable a enhanced logging in the config.ini (Debug --> logfile = true ) . Analyze the debugging output of the serial interface Connect a serial to USB interface (like for flashing) and make a logging of the serial communication There are a lot more intermediate information and the lines before the reboot tell you, where the firmware fails If you make an issue about this, please post these two information additionally Don't forget to remove your WLAN password in the serial log","title":"Getting deeper inside"},{"location":"Error-Debugging/#often-observed-problems","text":"","title":"Often observed problems"},{"location":"Error-Debugging/#hardware-failure","text":"Camera not working --> check the interface, test another module Low cost module with only 2MB of PSRAM instead of 4MB --> image taking will fail first. This will never work due to too low memory","title":"Hardware failure"},{"location":"Error-Debugging/#roi-misaligned","text":"This typically happens if you have suboptimal \"Alignement Marks\". A very simple and working solution is to put put higly contrasted stickers on your meter and put \"Alignement Marks\" on it (see picture below) If after those adjustement you still have some issues, you can try to adjust your aligmenet settings in expert mode:","title":"ROI misaligned"},{"location":"Error-Debugging/#my-analog-meter-are-recognized-as-digital-counter-or-vice-versa","text":"First, check that your ROI are correctly defined (yey!) Second, verify that the name of your ROI analog and digital ROIs are different","title":"My Analog Meter are recognized as Digital Counter or vice versa"},{"location":"Error-Debugging/#recognition-is-working-well-but-number-arent-sorted-correctly","text":"You have to sort your ROI correctly (Bigger to smaller). Select your ROI and click either \"move next\" or \"move previous\". Repeat until your ROI are correctly sorted","title":"Recognition is working well, but number aren't sorted correctly"},{"location":"External-LED/","text":"External LED The internal flash LED is very close to the camera axis. This results in reflection, especially in case of flat glass surfaces such as for power meters. To circumvent this problem, it is now possible to control external LEDs, which than can be places somewhere else in the setup. As not simples LEDs are used, but RGB leds with a digital interface like WS2812 not only the position, but also the color and intensity of the illumination can now be adjusted. The following image shows a direct comparision of the \"old\" internal flash LED and two off axis LEDs. There is also a new meter adapter available. This has two features: designed for small clearings in front of the meter and prepared for WS2812 LEDs . 1. Hardware installation of the LED stripe The control line of the LED stripe is connected with a 470 Ohm resistor to the GPIO12. For power supply stabilization a capacitor between 5V and ground is recommended. Here a 470\u00b5F polymer capacitor is used. As a power supply a 5V from the ESP32 is used like in the following wiring. 2. Software configuration The handling of the WS2812 LED controller needs some other libraries, therefore it is controlled within a dedicated section called GPIO Settings . The external LED stripe is connected to GPIO12. After activating the \"GPIO Settings\" section, the internal flash is per default disabled. In order to activate the external LED, you need to activate GPIO 12 state and select \"extern flash light ws281x ...\" . Parameter Meaning LED-Type There are several types of controller implemented: WS2812(B), WS2813, SK6812 Numbers of LED Number of LEDs on the LED stripe LED Color The color and intensity can be controlled directly by a red/green/blue value, each within the range from 0 (off) to 255 (full) Enabling the GPIO settings automatically disables the flash LED. Therefore you can enable it here manually by checking GPIO4 and choose \"build-in led flash light\" . It is not recommended to use both illumination parallel.","title":"External LED"},{"location":"External-LED/#external-led","text":"The internal flash LED is very close to the camera axis. This results in reflection, especially in case of flat glass surfaces such as for power meters. To circumvent this problem, it is now possible to control external LEDs, which than can be places somewhere else in the setup. As not simples LEDs are used, but RGB leds with a digital interface like WS2812 not only the position, but also the color and intensity of the illumination can now be adjusted. The following image shows a direct comparision of the \"old\" internal flash LED and two off axis LEDs. There is also a new meter adapter available. This has two features: designed for small clearings in front of the meter and prepared for WS2812 LEDs .","title":"External LED"},{"location":"External-LED/#1-hardware-installation-of-the-led-stripe","text":"The control line of the LED stripe is connected with a 470 Ohm resistor to the GPIO12. For power supply stabilization a capacitor between 5V and ground is recommended. Here a 470\u00b5F polymer capacitor is used. As a power supply a 5V from the ESP32 is used like in the following wiring.","title":"1. Hardware installation of the LED stripe"},{"location":"External-LED/#2-software-configuration","text":"The handling of the WS2812 LED controller needs some other libraries, therefore it is controlled within a dedicated section called GPIO Settings . The external LED stripe is connected to GPIO12. After activating the \"GPIO Settings\" section, the internal flash is per default disabled. In order to activate the external LED, you need to activate GPIO 12 state and select \"extern flash light ws281x ...\" . Parameter Meaning LED-Type There are several types of controller implemented: WS2812(B), WS2813, SK6812 Numbers of LED Number of LEDs on the LED stripe LED Color The color and intensity can be controlled directly by a red/green/blue value, each within the range from 0 (off) to 255 (full) Enabling the GPIO settings automatically disables the flash LED. Therefore you can enable it here manually by checking GPIO4 and choose \"build-in led flash light\" . It is not recommended to use both illumination parallel.","title":"2. Software configuration"},{"location":"FAQs/","text":"Frequently Asked Questions My device is reboot frequently. What can I do? There are several reasons for the reboot: Frequent HTML requests Wrong configuration, missing configuration files Unstable hardware - see [[Hardware Compatibility]] There is a dedicated Wiki page about this: Frequent Reboots . How accurate are the detections? It is hard to give a specific accuracy number. It depends on many factors, e.g. How in-focus is your camera? How sturdy is the camera mount? Does it slightly move over extended periods of time? What type of meter are you reading? Is the meter already in the training data set? Are you trying to read digits, an analog dial, or both? etc. Anecdotally, the authors of this wiki have great success with the meter. While the AI algorithm itself is not perfect and sometimes returns NaN or incorrect values, other post-processing / prevalue / sanity checks help ensure such invalid values are filtered out. With the correct settings, one author has been running this device for 1 month without any incorrect values reported. See the FAQs below for more details and configuration hints. My numbers are not corrected detected. What can I do? There is a dedicated Wiki page about the correct setting ROI Configuration . This page also includes the instructions for gathering new images for the training. How can I ensure invalid numbers are never reported? As mentioned above, the AI algorithm is not perfect. Sometimes it may read an incorrect value. We can tune the software to almost never report an incorrect value. There is a tradeoff though: the software may report stale values - i.e. it will drop incorrect values for a potentially long period of time, resulting in the meter reading being outdated by hours. If never receiving an incorrect value is important to you, consider tolerating this tradeoff. You can change the following settings to reduce incorrect readings (but potentially increase staleness of data): * Set a prevalue via the UI, then change PostProcessing configuration option PreValueAgeStartup to a much larger number (e.g. 43200 = 30 days). * Change PostProcessing configuration option MaxRateType to be time based instead of absolute. Set MaxRateValue to something realistic (e.g. 5 gal/min). You can often find the max flow rate your meter supports directly on the cover. * Reduce AutoTimer configuration option Intervall to the lowest it can be (e.g. 3 min). The more often you take readings, the less likely for data staleness to occur. Even after I have setup everything perfect there is a false reading - especially around the zero crossing (roll over to next number) The roll over behavior is different for the different meters. E.g.: Rolling over start with different previous position (e.g. at 7, 8 or 9) The neutral position (no rolling) is not perfectly at zero, but rather at something like 7.9 or 8.1, even if it should be exactly 8 The \"PostProcessingAlgo\" is trying to judge out of the individual readings, what number it should be. For example if the previous number is a \"1\", but the next number seems to be a \"8.9\", mos probably there was a \"zero crossing\" and the number is a \"9\" and not still an \"8\" Currently the setting of the algorithm is set to fit most of the meters and cases. But the parameters do not fit perfectly for all situations. Therefore there might be intermediate states, where the reading is false. This is especially the case, at the positions, where the roll over (zero crossing) is just starting. To prevent a sending of false parameters, there is the possibility to limit the maximum allowed change (MaxRateChange). Usually after some time and movement of the counters a bit further, the reading is getting back to a stable reading. To handle this, a parametrized setting would be needed. This is rather complicated to implement as subtle changes make a relevant difference. Currently this is not implemented. So please be a bit patient with your meter :-)","title":"Frequently Asked Questions"},{"location":"FAQs/#frequently-asked-questions","text":"","title":"Frequently Asked Questions"},{"location":"FAQs/#my-device-is-reboot-frequently-what-can-i-do","text":"There are several reasons for the reboot: Frequent HTML requests Wrong configuration, missing configuration files Unstable hardware - see [[Hardware Compatibility]] There is a dedicated Wiki page about this: Frequent Reboots .","title":"My device is reboot frequently. What can I do?"},{"location":"FAQs/#how-accurate-are-the-detections","text":"It is hard to give a specific accuracy number. It depends on many factors, e.g. How in-focus is your camera? How sturdy is the camera mount? Does it slightly move over extended periods of time? What type of meter are you reading? Is the meter already in the training data set? Are you trying to read digits, an analog dial, or both? etc. Anecdotally, the authors of this wiki have great success with the meter. While the AI algorithm itself is not perfect and sometimes returns NaN or incorrect values, other post-processing / prevalue / sanity checks help ensure such invalid values are filtered out. With the correct settings, one author has been running this device for 1 month without any incorrect values reported. See the FAQs below for more details and configuration hints.","title":"How accurate are the detections?"},{"location":"FAQs/#my-numbers-are-not-corrected-detected-what-can-i-do","text":"There is a dedicated Wiki page about the correct setting ROI Configuration . This page also includes the instructions for gathering new images for the training.","title":"My numbers are not corrected detected. What can I do?"},{"location":"FAQs/#how-can-i-ensure-invalid-numbers-are-never-reported","text":"As mentioned above, the AI algorithm is not perfect. Sometimes it may read an incorrect value. We can tune the software to almost never report an incorrect value. There is a tradeoff though: the software may report stale values - i.e. it will drop incorrect values for a potentially long period of time, resulting in the meter reading being outdated by hours. If never receiving an incorrect value is important to you, consider tolerating this tradeoff. You can change the following settings to reduce incorrect readings (but potentially increase staleness of data): * Set a prevalue via the UI, then change PostProcessing configuration option PreValueAgeStartup to a much larger number (e.g. 43200 = 30 days). * Change PostProcessing configuration option MaxRateType to be time based instead of absolute. Set MaxRateValue to something realistic (e.g. 5 gal/min). You can often find the max flow rate your meter supports directly on the cover. * Reduce AutoTimer configuration option Intervall to the lowest it can be (e.g. 3 min). The more often you take readings, the less likely for data staleness to occur.","title":"How can I ensure invalid numbers are never reported?"},{"location":"FAQs/#even-after-i-have-setup-everything-perfect-there-is-a-false-reading-especially-around-the-zero-crossing-roll-over-to-next-number","text":"The roll over behavior is different for the different meters. E.g.: Rolling over start with different previous position (e.g. at 7, 8 or 9) The neutral position (no rolling) is not perfectly at zero, but rather at something like 7.9 or 8.1, even if it should be exactly 8 The \"PostProcessingAlgo\" is trying to judge out of the individual readings, what number it should be. For example if the previous number is a \"1\", but the next number seems to be a \"8.9\", mos probably there was a \"zero crossing\" and the number is a \"9\" and not still an \"8\" Currently the setting of the algorithm is set to fit most of the meters and cases. But the parameters do not fit perfectly for all situations. Therefore there might be intermediate states, where the reading is false. This is especially the case, at the positions, where the roll over (zero crossing) is just starting. To prevent a sending of false parameters, there is the possibility to limit the maximum allowed change (MaxRateChange). Usually after some time and movement of the counters a bit further, the reading is getting back to a stable reading. To handle this, a parametrized setting would be needed. This is rather complicated to implement as subtle changes make a relevant difference. Currently this is not implemented. So please be a bit patient with your meter :-)","title":"Even after I have setup everything perfect there is a false reading - especially around the zero crossing (roll over to next number)"},{"location":"Frequent-Reboots/","text":"Frequent Reboots There are several types of reboots. To get a deeper insight turn on the logging: Internal logging ( config.ini ) Serial log of the UART interface (same as for flashing the firmware) There are two principle types of reboots Random reboots (always different timing and situation) Permanent Reboots always at the same time Random reboots Random reboots have two reasons: overload during HTML access and unstable system In general: there are several mechanisms in the firmware (like saving previous values), to have a \"smooth\" reboot without too many notable disturbance. Overload during HTML access If you frequently access the web server over HTML requests, the firmware tends to reboot. This especially happens during the first run and when the ESP32 is busy with the digitization flow. The reason for this are running out of memory during a flow, minor memory leakage in combination with missing error handling. There is noting you can do about this kind of reboot, beside two thing: Support the firmware development with improved and tested part of code Be patient :-) Unstable system If your system is sometimes running smoothly over several runs and sometimes reboots obviously randomly, you have an partially unstable device. You can check this in the standard log file on the SD card: 2021-12-26T06:34:09: task_autodoFlow - round done 2021-12-26T06:34:09: CPU Temperature: 56.1 2021-12-26T06:38:00: task_autodoFlow - next round - Round #23 Here you see, that the round #23 is starting, so obviously there were no reboots in the last 22 rounds. There is hardware (ESP32CAM), where only 2-3 stable rounds are possible and others, where way more than 100 rounds without any reboots is possible. There is noting you can do about it, beside testing different hardware. Permanent reboots Permanent reboots at the same situation during the flow has a systematic problem either in the hardware or the configuration. It usually happens during the first run as there all needed parts of the firmware have been loaded for the first time. To find the reason mostly the serial log of the UART interface from the startup until the reboots is very helpful. It can be stored using the USB / UART interface - the same as for flashing the firmware - and logging the serial output of the ESP32. Possible problems: SD card PSRAM too low Configuration missing SD card problems The ESP32CAM is a little bit \"picky\" with the supported SD cards. Due to the limited availability of GPIOs the SD card can only be accessed via 1-wire mode. Therefore not all SD cards are supported. Several error cases can happen: No SD card Easy to detect: fast blinking red LED directly after startup, no reaction of the web server etc. at all SD card not supported at all Error message of no detectable SC card in the log file. Normal looking log for a 16GB SD card is like this: 09:38:25.037 -> \u001b[0;32mI (4789) main: Using SDMMC peripheral\u001b[0m 09:38:25.037 -> \u001b[0;32mI (4789) main: Using SDMMC peripheral\u001b[0m 09:38:25.138 -> Name: SC16G 09:38:25.138 -> Type: SDHC/SDXC 09:38:25.138 -> Speed: 20 MHz 09:38:25.138 -> Size: 15193MB Otherwise there is some error message. SD card recognized but not supported This is the most annoying error. The SD card is detected, but the files cannot be read. Most probably this results in a problem with the WLAN connection, as the first file needed is the wlan.ini in the root directory. PSRAM too low In order to work, there are 4 MB of PSRAM necessary. Normaly the ESP32CAM is equiped with 8 MB, whereof only 4 MB can be used effectively. Sometimes, there is hardware, where only 2 MB of PSRAM is present - even if you have bought a 8 MB module You can identify the amount of PSRAM in the serial log file: 09:38:21.224 -> \u001b[0;32mI (881) psram: This chip is ESP32-D0WD\u001b[0m 09:38:21.224 -> \u001b[0;32mI (885) spiram: Found 64MBit SPI RAM device\u001b[0m 09:38:21.224 -> \u001b[0;32mI (890) spiram: SPI RAM mode: flash 40m sram 40m\u001b[0m 09:38:21.224 -> \u001b[0;32mI (895) spiram: PSRAM initialized, cache is in low/high (2-core) mode.\u001b[0m Here you see 64MBit (= 8MByte) - which is okay. False reading would be: 16MBit The error in the SD log file is typically related with the taking of the image (tbd) as the first time, the system is running out of memory is usually, when it tries to transfer an image from the camera to the PSRAM. There is nothing to do, than to buy a new ESP32CAM with really 64MBit of PSRAM. Configuration missing There are several files needed during on run cycle. If one of this is missing, the firmware is missing information and tends to reboot due to missing error management: /wlan.ini /config/config.ini /config/XXXXX.tflite (1 time for analog and 1 time for digital) where XXXXX is the file name, that is written in the config.ini","title":"Frequent Reboots"},{"location":"Frequent-Reboots/#frequent-reboots","text":"There are several types of reboots. To get a deeper insight turn on the logging: Internal logging ( config.ini ) Serial log of the UART interface (same as for flashing the firmware) There are two principle types of reboots Random reboots (always different timing and situation) Permanent Reboots always at the same time","title":"Frequent Reboots"},{"location":"Frequent-Reboots/#random-reboots","text":"Random reboots have two reasons: overload during HTML access and unstable system In general: there are several mechanisms in the firmware (like saving previous values), to have a \"smooth\" reboot without too many notable disturbance.","title":"Random reboots"},{"location":"Frequent-Reboots/#overload-during-html-access","text":"If you frequently access the web server over HTML requests, the firmware tends to reboot. This especially happens during the first run and when the ESP32 is busy with the digitization flow. The reason for this are running out of memory during a flow, minor memory leakage in combination with missing error handling. There is noting you can do about this kind of reboot, beside two thing: Support the firmware development with improved and tested part of code Be patient :-)","title":"Overload during HTML access"},{"location":"Frequent-Reboots/#unstable-system","text":"If your system is sometimes running smoothly over several runs and sometimes reboots obviously randomly, you have an partially unstable device. You can check this in the standard log file on the SD card: 2021-12-26T06:34:09: task_autodoFlow - round done 2021-12-26T06:34:09: CPU Temperature: 56.1 2021-12-26T06:38:00: task_autodoFlow - next round - Round #23 Here you see, that the round #23 is starting, so obviously there were no reboots in the last 22 rounds. There is hardware (ESP32CAM), where only 2-3 stable rounds are possible and others, where way more than 100 rounds without any reboots is possible. There is noting you can do about it, beside testing different hardware.","title":"Unstable system"},{"location":"Frequent-Reboots/#permanent-reboots","text":"Permanent reboots at the same situation during the flow has a systematic problem either in the hardware or the configuration. It usually happens during the first run as there all needed parts of the firmware have been loaded for the first time. To find the reason mostly the serial log of the UART interface from the startup until the reboots is very helpful. It can be stored using the USB / UART interface - the same as for flashing the firmware - and logging the serial output of the ESP32. Possible problems: SD card PSRAM too low Configuration missing","title":"Permanent reboots"},{"location":"Frequent-Reboots/#sd-card-problems","text":"The ESP32CAM is a little bit \"picky\" with the supported SD cards. Due to the limited availability of GPIOs the SD card can only be accessed via 1-wire mode. Therefore not all SD cards are supported. Several error cases can happen:","title":"SD card problems"},{"location":"Frequent-Reboots/#no-sd-card","text":"Easy to detect: fast blinking red LED directly after startup, no reaction of the web server etc. at all","title":"No SD card"},{"location":"Frequent-Reboots/#sd-card-not-supported-at-all","text":"Error message of no detectable SC card in the log file. Normal looking log for a 16GB SD card is like this: 09:38:25.037 -> \u001b[0;32mI (4789) main: Using SDMMC peripheral\u001b[0m 09:38:25.037 -> \u001b[0;32mI (4789) main: Using SDMMC peripheral\u001b[0m 09:38:25.138 -> Name: SC16G 09:38:25.138 -> Type: SDHC/SDXC 09:38:25.138 -> Speed: 20 MHz 09:38:25.138 -> Size: 15193MB Otherwise there is some error message.","title":"SD card not supported at all"},{"location":"Frequent-Reboots/#sd-card-recognized-but-not-supported","text":"This is the most annoying error. The SD card is detected, but the files cannot be read. Most probably this results in a problem with the WLAN connection, as the first file needed is the wlan.ini in the root directory.","title":"SD card recognized but not supported"},{"location":"Frequent-Reboots/#psram-too-low","text":"In order to work, there are 4 MB of PSRAM necessary. Normaly the ESP32CAM is equiped with 8 MB, whereof only 4 MB can be used effectively. Sometimes, there is hardware, where only 2 MB of PSRAM is present - even if you have bought a 8 MB module You can identify the amount of PSRAM in the serial log file: 09:38:21.224 -> \u001b[0;32mI (881) psram: This chip is ESP32-D0WD\u001b[0m 09:38:21.224 -> \u001b[0;32mI (885) spiram: Found 64MBit SPI RAM device\u001b[0m 09:38:21.224 -> \u001b[0;32mI (890) spiram: SPI RAM mode: flash 40m sram 40m\u001b[0m 09:38:21.224 -> \u001b[0;32mI (895) spiram: PSRAM initialized, cache is in low/high (2-core) mode.\u001b[0m Here you see 64MBit (= 8MByte) - which is okay. False reading would be: 16MBit The error in the SD log file is typically related with the taking of the image (tbd) as the first time, the system is running out of memory is usually, when it tries to transfer an image from the camera to the PSRAM. There is nothing to do, than to buy a new ESP32CAM with really 64MBit of PSRAM.","title":"PSRAM too low"},{"location":"Frequent-Reboots/#configuration-missing","text":"There are several files needed during on run cycle. If one of this is missing, the firmware is missing information and tends to reboot due to missing error management: /wlan.ini /config/config.ini /config/XXXXX.tflite (1 time for analog and 1 time for digital) where XXXXX is the file name, that is written in the config.ini","title":"Configuration missing"},{"location":"Graphical-Configuration/","text":"Graphical Configuration Warning This page overlaps Configuration . They should be merged to one page! General remark: to activate the changes, currently the device needs a restart after saving the changes. partially the commands needs processing on the ESP32 device. This is not very fast - so please be patient. Access to the graphical user interface The graphical configuration mode can be reached via the \"Edit Configuration\" button in the main menue ( /index.html ): Overview function Direct edit config.ini in text editor Configuration of image alignment a. Create of reference image b. Define alignment structures Definition of ROIs for digits and analog pointers Test the settings Back to main menue (\"index.html\") 1. Edit Config.ini This is a text editor for the config.ini. Changes commited with the button on the lower left. Details see Configuration-Parameter-Details . 2a. Create Reference Image The reference image is the basis for the coordination of the ROIs. Therefore it is very important, to have a well aligned image, that is not rotated. \u203c\ufe0f Attention: Updating the reference image, also means, that all alignment images and ROIs needs to be teached again. Therefore do this step only with caution. At first the current image is shown and no adjustment is possible. To reload the actual image push the button \"Show actual Reference\" (1). To define a new reference image push the button \"Create new Reference\" (2). Then the last taken raw image from the camera is loaded. If you want to update this, you can push the button \"Make new raw image (raw.jpg)\". If you need to mirror your image (e.g. mirror before camera) you can do this by selecting \"mirror image\". After loading the mirroring (in case checked) and the prerotation angle from the config.ini are applied. Then use the rough and fine adjustment to get the image straight aligned (3). If everything is done, you can save the result with \"Update Reference Image\" (4). If you have problems with reflections, you can turn the camera in a positions, where the reflection is at a position, where no important information is. To reduce the intensity of the reflection you can also a peace of felt (\"Filz\") as diffusior at the LED. 2b. Define Alignment References The alignment references are used to realign every taken image to the reference coordinates. Therefore two alignment structures are identified and the image is shifted and rotated according to their position with the target to be in exactly the same position as the reference image. The alignment structures needs to be unique and have a good contrast. As this is the most calculation intensive process, only a field of view of 40x40 pixels around the original coordinates are scanned. This can be adjusted manually in the config.ini (Parameter: SearchFieldX / SearchFieldY ). In the upper part of the settings you can control the position and size of the selected reference image. You can define the ROI in the image directly via drag and drop with the mouse. Go to the starting point, push the left mouse button and drag your ROI. You will a red rectangle with the newly selected position. To make this active, you need to push \"Update Reference\" (2). You can change between the two reference images with the drop down box (\"Reference 0\", \"Reference 1\"). In some cases it might be useful to use a reference with a higher contrast. This can be achieved by pushing \"Enhance Contrast\" (3). The result will be calculated on the ESP32 - so be a bit patient, before you see it active. To save the modified reference to the config.ini push finally \"Save to config.ini\". 3a./3b. Define ROIs for image recognition Here the regions of interest for the digital and analog pointers are defined. As both are done identically, here as an example the digital images are shown. First of all you can define more than one number, for example in a dual meter counter. This can be done with defining a \"Number\" (1). Analog and digital ROIs belonging to the same \"Number\" are considered to be part of the same counter. As for the reference images you can change position, size and name of the ROI in the text fields or define them via drag and drop through the mouse button. You can iterate through the defined ROIs through the drop down box in the left upper area (2). To define new or delete ROIs use the corresponding button. Be careful: if you delete all ROIs, the tool will ask you to define minimum one manually in the config.ini . The order of the ROIs correspond to the position of the digit / analog pointer in the final readout number. The order can be changed with the button \"move Next\" / \"move Previous\" (3). In order to have a good recognition, the active ROI has two rectangles for alignment: The outer rectangle is the final size of the ROI More important is the inner smaller rectangle. This should tightly fit around the number itself in x- and in y-dimension. Maybe you need to unlock the aspect ratio to change x- and y-size independendly The line in the middle should go through the middle of the number (in case it is not moving in or out) To save the result push \"Save all to config.ini\" (4). \u203c\ufe0f Attention: Currently you have to reboot the ESP32 to take the changes in the config.ini to take place. This steps are running on the ESP32 directly. So be patient with the results.","title":"Graphical Configuration"},{"location":"Graphical-Configuration/#graphical-configuration","text":"Warning This page overlaps Configuration . They should be merged to one page!","title":"Graphical Configuration"},{"location":"Graphical-Configuration/#general-remark","text":"to activate the changes, currently the device needs a restart after saving the changes. partially the commands needs processing on the ESP32 device. This is not very fast - so please be patient.","title":"General remark:"},{"location":"Graphical-Configuration/#access-to-the-graphical-user-interface","text":"The graphical configuration mode can be reached via the \"Edit Configuration\" button in the main menue ( /index.html ):","title":"Access to the graphical user interface"},{"location":"Graphical-Configuration/#overview-function","text":"Direct edit config.ini in text editor Configuration of image alignment a. Create of reference image b. Define alignment structures Definition of ROIs for digits and analog pointers Test the settings Back to main menue (\"index.html\")","title":"Overview function"},{"location":"Graphical-Configuration/#1-edit-configini","text":"This is a text editor for the config.ini. Changes commited with the button on the lower left. Details see Configuration-Parameter-Details .","title":"1. Edit Config.ini"},{"location":"Graphical-Configuration/#2a-create-reference-image","text":"The reference image is the basis for the coordination of the ROIs. Therefore it is very important, to have a well aligned image, that is not rotated. \u203c\ufe0f Attention: Updating the reference image, also means, that all alignment images and ROIs needs to be teached again. Therefore do this step only with caution. At first the current image is shown and no adjustment is possible. To reload the actual image push the button \"Show actual Reference\" (1). To define a new reference image push the button \"Create new Reference\" (2). Then the last taken raw image from the camera is loaded. If you want to update this, you can push the button \"Make new raw image (raw.jpg)\". If you need to mirror your image (e.g. mirror before camera) you can do this by selecting \"mirror image\". After loading the mirroring (in case checked) and the prerotation angle from the config.ini are applied. Then use the rough and fine adjustment to get the image straight aligned (3). If everything is done, you can save the result with \"Update Reference Image\" (4). If you have problems with reflections, you can turn the camera in a positions, where the reflection is at a position, where no important information is. To reduce the intensity of the reflection you can also a peace of felt (\"Filz\") as diffusior at the LED.","title":"2a. Create Reference Image"},{"location":"Graphical-Configuration/#2b-define-alignment-references","text":"The alignment references are used to realign every taken image to the reference coordinates. Therefore two alignment structures are identified and the image is shifted and rotated according to their position with the target to be in exactly the same position as the reference image. The alignment structures needs to be unique and have a good contrast. As this is the most calculation intensive process, only a field of view of 40x40 pixels around the original coordinates are scanned. This can be adjusted manually in the config.ini (Parameter: SearchFieldX / SearchFieldY ). In the upper part of the settings you can control the position and size of the selected reference image. You can define the ROI in the image directly via drag and drop with the mouse. Go to the starting point, push the left mouse button and drag your ROI. You will a red rectangle with the newly selected position. To make this active, you need to push \"Update Reference\" (2). You can change between the two reference images with the drop down box (\"Reference 0\", \"Reference 1\"). In some cases it might be useful to use a reference with a higher contrast. This can be achieved by pushing \"Enhance Contrast\" (3). The result will be calculated on the ESP32 - so be a bit patient, before you see it active. To save the modified reference to the config.ini push finally \"Save to config.ini\".","title":"2b. Define Alignment References"},{"location":"Graphical-Configuration/#3a3b-define-rois-for-image-recognition","text":"Here the regions of interest for the digital and analog pointers are defined. As both are done identically, here as an example the digital images are shown. First of all you can define more than one number, for example in a dual meter counter. This can be done with defining a \"Number\" (1). Analog and digital ROIs belonging to the same \"Number\" are considered to be part of the same counter. As for the reference images you can change position, size and name of the ROI in the text fields or define them via drag and drop through the mouse button. You can iterate through the defined ROIs through the drop down box in the left upper area (2). To define new or delete ROIs use the corresponding button. Be careful: if you delete all ROIs, the tool will ask you to define minimum one manually in the config.ini . The order of the ROIs correspond to the position of the digit / analog pointer in the final readout number. The order can be changed with the button \"move Next\" / \"move Previous\" (3). In order to have a good recognition, the active ROI has two rectangles for alignment: The outer rectangle is the final size of the ROI More important is the inner smaller rectangle. This should tightly fit around the number itself in x- and in y-dimension. Maybe you need to unlock the aspect ratio to change x- and y-size independendly The line in the middle should go through the middle of the number (in case it is not moving in or out) To save the result push \"Save all to config.ini\" (4). \u203c\ufe0f Attention: Currently you have to reboot the ESP32 to take the changes in the config.ini to take place. This steps are running on the ESP32 directly. So be patient with the results.","title":"3a./3b. Define ROIs for image recognition"},{"location":"Hardware-Compatibility/","text":"Hardware Compatibility General Remark Altrough a board looks similair, it can have major differences, eg.: Processor Ram (Size! & Type) -> this Project needs at least 4MB RAM! Flashrom Camera Modules Onboard/External Antenna Quality of Components Manufacture Quality of the PCB and soldering Different Components \"Clone\" Components -> ESPxx etc. This can cause different Power Consumption, Power Requirements, compatibility issues, etc. Most manufacturers and sellers buy what's cheap today on the Asian markets. In the end, it looks like it is sometimes a trial and error approach which ESP32-CAM Module works reliable. Below you find some remarks and experiences from the community: ESP32 core itself Chip Version Image Status ESP32-D0WDQ6 (revision 1) \u2714\ufe0f PSRAM There seems to be a lot of \"fake\" chips, or maybe wrongly configured ESP32 Boards. For AP MEMORY, all \"real\" APS 64 04*3SQR Chips should work. For ESP PSRAM, all \"real\" PSRAM 64 * should work 64Mbit density = 8Mbyte PSRAM This Table is just a snapshot of chips which worked Labeling on PSRAM module Image Status IPUS / IPS640LS0 / 1815XBGN \u2714\ufe0f AP MEMORY / 64 04L-3SOR / 1040H / 110089G \u2714\ufe0f AP MEMORY / 64 04L-3SQR / 12205 / 150047G \u2714\ufe0f 8MB AP MEMORY / 64 04L-3SQR / 12208 / 150047G \u2714\ufe0f 8MB AP MEMORY / 64 04L-350R / 1120A / 130027G \u274c PSRAM not accessible AP MEMORY / 64 04L-35QR / 11208 / 130025G \u274c PSRAM not accessible AP MEMORY / 64 04L-3SQR / 13100 / 180026G \u274c PSRAM not accessible AP MEMORY / 64 04L-3SQR / 11207 / 130024G \u274c PSRAM not accessible AP MEMORY / 1604M-3SQR / 0280A / 070036G \u274c 2MB only! ESP PSRAM 64 H 462021 / 1B00286 \u2714\ufe0f ESP PSRAM 64 H 412021 /1A0039G \u2714\ufe0f (8MB) ESP PSRAM16M 302020 \u274c 2MB only! ESP PSRAM16H 202020 / 050022G \u274c 2MB only! OV2640 - Camera The experience with the camera only is based on single modules. It is well possible, that this module had a damage overall and other modules of the same type will work. Give it a try and report to me! Labeling on Flex-Connector Image Status TY-OV2 / 640-V2.0 \u2714\ufe0f DCX-OV2 / 640-V2 \u2714\ufe0f DC-26 / 40-V3 \u2714\ufe0f 3x \u274c 1x ESP32 Modules Module Image Status ESP32CAM / Different versions on the market! Especially the PSRAM is sometimes labeled wrong (Label: 4MB, Real: only 2 MB --> will not work!) \u2714\ufe0f with >=4 MB PSRAM! ESP32-S3-EYE No Flash LED, pins different used (e.g. LCD diskplay) NOT OKAY SD-Cards Due to the limited free available gpios (due to all the extensions needed like: camera, sd-card, LED-flash, ...) the sd card is connected in 1-wire mode. There are some cards, that are compatible with the esp32cam module for unknown reasons. It is observed, that smaller cards (up to 4 GB) tend to be more stable and larger cards have more problems. But quite some exceptions in the forums (4 GB cards not working, 16 G cards working like a charm). Devices known to work Modules (Old list, not up-to-date anymore): See https://github.com/jomjol/AI-on-the-edge-device/discussions/1732 for a more recent list. https://arduino-projekte.info/produkt/esp32-cam-v2-integriertem-ch340-mit-ov2640-kamera-modul/ (see https://github.com/jomjol/AI-on-the-edge-device/discussions/1041 ) https://www.amazon.de/-/en/gp/product/B0B51CQ13R https://www.reichelt.de/entwicklerboards-esp32-kamera-2mp-25--debo-cam-esp32-p266036.html?PROVID=2788&gclid=CjwKCAiAqaWdBhAvEiwAGAQlttJnV4azXWDYeaFUuNioMICh-jvxKp6Cifmcep9vvtoT2JRCDqBczRoC7Q0QAvD_BwE (27.12.2022) SD-Card Sandisk 2GB Micro SD Class 2 Sandisk 2GB AITRIP ESP32 and CAM ESP-32/CAM Amazon US - Aideepen ESP32-CAM W BT Board ESP32-CAM-MB Micro USB to Serial Port CH-340G with OV2640 2MP Camera Module Dual Mode with Amazon US - Cloudisk 5Pack 4GB Micro SD Card 4 GB MicroSD Memory Card Class6 Weak Wifi The ESP32-CAM supports an external antenna. It requires some soldering skills but can improve the connection quality. See https://randomnerdtutorials.com/esp32-cam-connect-external-antenna/","title":"Hardware Compatibility"},{"location":"Hardware-Compatibility/#hardware-compatibility","text":"","title":"Hardware Compatibility"},{"location":"Hardware-Compatibility/#general-remark","text":"Altrough a board looks similair, it can have major differences, eg.: Processor Ram (Size! & Type) -> this Project needs at least 4MB RAM! Flashrom Camera Modules Onboard/External Antenna Quality of Components Manufacture Quality of the PCB and soldering Different Components \"Clone\" Components -> ESPxx etc. This can cause different Power Consumption, Power Requirements, compatibility issues, etc. Most manufacturers and sellers buy what's cheap today on the Asian markets. In the end, it looks like it is sometimes a trial and error approach which ESP32-CAM Module works reliable. Below you find some remarks and experiences from the community:","title":"General Remark"},{"location":"Hardware-Compatibility/#esp32-core-itself","text":"Chip Version Image Status ESP32-D0WDQ6 (revision 1) \u2714\ufe0f","title":"ESP32 core itself"},{"location":"Hardware-Compatibility/#psram","text":"There seems to be a lot of \"fake\" chips, or maybe wrongly configured ESP32 Boards. For AP MEMORY, all \"real\" APS 64 04*3SQR Chips should work. For ESP PSRAM, all \"real\" PSRAM 64 * should work 64Mbit density = 8Mbyte PSRAM This Table is just a snapshot of chips which worked Labeling on PSRAM module Image Status IPUS / IPS640LS0 / 1815XBGN \u2714\ufe0f AP MEMORY / 64 04L-3SOR / 1040H / 110089G \u2714\ufe0f AP MEMORY / 64 04L-3SQR / 12205 / 150047G \u2714\ufe0f 8MB AP MEMORY / 64 04L-3SQR / 12208 / 150047G \u2714\ufe0f 8MB AP MEMORY / 64 04L-350R / 1120A / 130027G \u274c PSRAM not accessible AP MEMORY / 64 04L-35QR / 11208 / 130025G \u274c PSRAM not accessible AP MEMORY / 64 04L-3SQR / 13100 / 180026G \u274c PSRAM not accessible AP MEMORY / 64 04L-3SQR / 11207 / 130024G \u274c PSRAM not accessible AP MEMORY / 1604M-3SQR / 0280A / 070036G \u274c 2MB only! ESP PSRAM 64 H 462021 / 1B00286 \u2714\ufe0f ESP PSRAM 64 H 412021 /1A0039G \u2714\ufe0f (8MB) ESP PSRAM16M 302020 \u274c 2MB only! ESP PSRAM16H 202020 / 050022G \u274c 2MB only!","title":"PSRAM"},{"location":"Hardware-Compatibility/#ov2640-camera","text":"The experience with the camera only is based on single modules. It is well possible, that this module had a damage overall and other modules of the same type will work. Give it a try and report to me! Labeling on Flex-Connector Image Status TY-OV2 / 640-V2.0 \u2714\ufe0f DCX-OV2 / 640-V2 \u2714\ufe0f DC-26 / 40-V3 \u2714\ufe0f 3x \u274c 1x","title":"OV2640 - Camera"},{"location":"Hardware-Compatibility/#esp32-modules","text":"Module Image Status ESP32CAM / Different versions on the market! Especially the PSRAM is sometimes labeled wrong (Label: 4MB, Real: only 2 MB --> will not work!) \u2714\ufe0f with >=4 MB PSRAM! ESP32-S3-EYE No Flash LED, pins different used (e.g. LCD diskplay) NOT OKAY","title":"ESP32 Modules"},{"location":"Hardware-Compatibility/#sd-cards","text":"Due to the limited free available gpios (due to all the extensions needed like: camera, sd-card, LED-flash, ...) the sd card is connected in 1-wire mode. There are some cards, that are compatible with the esp32cam module for unknown reasons. It is observed, that smaller cards (up to 4 GB) tend to be more stable and larger cards have more problems. But quite some exceptions in the forums (4 GB cards not working, 16 G cards working like a charm).","title":"SD-Cards"},{"location":"Hardware-Compatibility/#devices-known-to-work","text":"","title":"Devices known to work"},{"location":"Hardware-Compatibility/#modules-old-list-not-up-to-date-anymore","text":"See https://github.com/jomjol/AI-on-the-edge-device/discussions/1732 for a more recent list. https://arduino-projekte.info/produkt/esp32-cam-v2-integriertem-ch340-mit-ov2640-kamera-modul/ (see https://github.com/jomjol/AI-on-the-edge-device/discussions/1041 ) https://www.amazon.de/-/en/gp/product/B0B51CQ13R https://www.reichelt.de/entwicklerboards-esp32-kamera-2mp-25--debo-cam-esp32-p266036.html?PROVID=2788&gclid=CjwKCAiAqaWdBhAvEiwAGAQlttJnV4azXWDYeaFUuNioMICh-jvxKp6Cifmcep9vvtoT2JRCDqBczRoC7Q0QAvD_BwE (27.12.2022)","title":"Modules (Old list, not up-to-date anymore):"},{"location":"Hardware-Compatibility/#sd-card","text":"Sandisk 2GB Micro SD Class 2 Sandisk 2GB AITRIP ESP32 and CAM ESP-32/CAM Amazon US - Aideepen ESP32-CAM W BT Board ESP32-CAM-MB Micro USB to Serial Port CH-340G with OV2640 2MP Camera Module Dual Mode with Amazon US - Cloudisk 5Pack 4GB Micro SD Card 4 GB MicroSD Memory Card Class6","title":"SD-Card"},{"location":"Hardware-Compatibility/#weak-wifi","text":"The ESP32-CAM supports an external antenna. It requires some soldering skills but can improve the connection quality. See https://randomnerdtutorials.com/esp32-cam-connect-external-antenna/","title":"Weak Wifi"},{"location":"Influx-DB/","text":"Influx DB The device also supports direct sending of data to an Influ DB. \u203c\ufe0f Only Influx DB 1 is supported!","title":"Influx DB"},{"location":"Influx-DB/#influx-db","text":"The device also supports direct sending of data to an Influ DB. \u203c\ufe0f Only Influx DB 1 is supported!","title":"Influx DB"},{"location":"Installation/","text":"Installation The installation requires multiple steps: Get the right hardware and wire it up Flash the firmware onto the ESP32 Write the data to the SD-Card Insert the SD-Card into the ESP32 board Power/restart it. Hardware ESP32-CAM OV2640 camera module Micro SD-Card slot 4 or 8 MB PSRAM. It can be easily found on the typical internet stores, searching for ESP32-CAM for less than 10 EUR. How ever since the hardware is cheap and coming from China, you unluckily could pick a malfunction device. See Hardware Compatibility for further advice! USB->UART interface For first time flashing the firmware a USB -> UART connector is needed. Later firmware upgrades than can be flashed via OTA. Power supply For power supply a 5V source is needed. Most easily this can be done via an USB power supply. The power supply should support minimum 500mA. For buffering current peaks some users reported to use a large elco condensator like a 2200uF between ground and VCC. \u203c\ufe0f Attention: in several internet forums there are problems reported, in case the ESP32-CAM is only supplied with 3.3V. Housing A small 3D-printable example for a very small case can be found in Thingiverse here: https://www.thingiverse.com/thing:4571627 \u203c\ufe0f Attention : the focus of the OV2640 needs to be adjusted, as it is normally set from ~40cm to infinity. In order to get an image that is big enough, it needs to be changed to about 10cm. Therefore the sealing glue on the objective ring needs to be removed with a scalpel or sharp knife. Afterwards the objective can be rotated clockwise until the image is sharp again. Wiring Beside the 5V power supply, only for the first flashing a connection to the USB-UART connector, including a short cut of GPIO0 to GND for bootloader start. A example for wiring can be found here: It is also possible to use external LEDs for the illumination instead of the internal flash LED. This is described here: [[External-LED]] Firmware flashing Files Grab the firmware from the Releases page (Stable, tested versions), or the Automatically build development branch (experimental, untested versions). Please have a look on https://github.com/jomjol/AI-on-the-edge-device/wiki/Install-a-rolling-%28unstable%29-release first! You need: partitions.bin bootloader.bin firmware.bin Flashing There are several options to flash the firmware. Here three are described: 1. Web Installer There is a Web Installer available which will work right out of the web browser Edge and Chrome. You can access it with the following link: https://jomjol.github.io/AI-on-the-edge-device This is the preferred way for beginners as it also allows access to the USB Log: 2. Using the Flash Tool from Espressif The flashing of the firmware can be done with the \"Flash Download Tool\" from espressif, that can found here Download and extract the Flash tool, after starting choose \"Developer Mode\", then \"ESP32-DownloadTool\" and you are in the setup of the flashing tool. Connect the ESP32-CAM with the USB-UART connection and identify the COM-Port. \u203c\ufe0f Attention : if you reflashing the code again, it is strongly recommended to erase the flash memory before flashing the firmware. Especially if you used OTA in between, which might cause remaining information on the flash, to still boot from an old image in the OTA-area, which is not erased by a normal flash. But your ESP32 in bootloader mode and push start, then it will identify the board and you can configure the bin-configuration according to the following table: Filename Offset bootloader.bin 0x1000 partitions.bin 0x8000 firmware.bin 0x10000 Alternatively it can be directly flashed from the development environment - here PlatformIO. But this is rather for experienced users, as the whole development chain needs to be installed for compilation. 3. Using esptool in python directly For this you need a python environment (e.g. Anaconda in Win10). Here you need to install the esptool: pip install esptool Then connect the ESP32 with the USB-UART connector to the system, put it in bootmode and with the following command you can erase the flash and flash bootloader, partitions and firmware in two steps: esptool erase_flash esptool write_flash 0x01000 bootloader.bin 0x08000 partitions.bin 0x10000 firmware.bin Maybe you need to specify the COM-port if it is not detected by default. If the erase command throws the error A fatal error occurred: ESP32 ROM does not support function erase_flash. , your esptool might be too old, see https://techoverflow.net/2022/02/08/how-to-fix-esp32-a-fatal-error-occurred-esp32-rom-does-not-support-function-erase_flash/ With some Python installations this may not work and you\u2019ll receive an error, try python -m pip install esptool or pip3 install esptool Further recommendations can be found on the espressif webpage SD-Card The software expects a SD-Card prepared with certain directory and file structure in order to work properly. For the first setup take the AI-on-the-edge-device__manual-setup__*.zip from the Release page, open the zip and extract the whole content of the in the setup file included sd-card.zip onto your SD-Card direclty to the root folder. SD-Card root should look like this: config demo firmware html img_tmp log wlan.ini This initial setup needs to be done only once as further updates of the software are possible with an Over-The-Air update mechanismn. Notes Due to the limited availability of GPIOs (OV2640, Flash-Light, PSRAM & SD-Card) the communication mode to the SD card is limited to 1-line SD-Mode. It showed up, that this results in problems with very large SD-Cards (64GB, sometimes 32 GB) and some no name low cost SD-cards. There must be no partition table on the SD-card (no GPT, but only MBR for the single partition) Following setting are necessary for formating the SD-card: SINGLE PARTITION, MBR, FAT32 - 32K. NOT exFAT Some ESP32 devices share their SD-card and/or camera GPIOs with the pins for TX and RX. If you see errors like \u201cFailed to connect\u201d then your chip is probably not entering the bootloader properly. Remove the respective modules temporarily to free the GPIOs for flashing. You may find more information about troubleshooting on the homepage of Espressif . The ESP32 indicates problems with the SD card during startup with a fast not ending blinking. In this case, please try another SD card. WLAN The access to the WLAN is configured in the \"wlan.ini\" directly on the root directory of the sd-card. Just write the corresponding SSID and password before the startup of the ESP32. This file is hidden from external access (e.g. via Filemanager) to protect the password. After power on the connection status is indicated by 3x blinking of the red on board LED. WLAN-Status indication: 5 x fast blinking (< 1 second): connection still pending 3 x slow blinking (1 second on/off): WLAN connection established It is normal that at first one or two times a pending connection is indicated. Update (OTA / Over-The-Air) Update from version greater than 12.0.0 You can use the over the air update mechanism, which uploads the update via a ZIP files. The update file is located on the release page . Please choose the zip file with the following naming: AI-on-the-edge-device__update__*.zip Go to the menu System --> OTA Update and follow the instructions there. After a final automatic reboot you should have the new version running. Update from version older than 12.0.0 If you update from an version older than 12.0.1, you should firstly update to version 12.0.1. Background are not fully backward compatible changes in the config.ini , that are taken care of in this version. \u203c\ufe0f Make sure to read the instructions below carfully! Backup your configuration (use the System -> Backup/Restore page)! Upload and update the update-*.zip file from the release 12.0.1 see here . Let it restart and check on the System -> Info page that the Firmware as well as the Web UI got updated. If only one got updated, redo the update. If it fails several times, you also can update the Firmware and the Web UI separately. Safe way: Update first the firmware.bin (extract it from one of the provided zip files) and do the Reboot Update with the full zip file ( update-*.zip , ignore the version warning after the reboot) Please go to Settings -> Configuration and address the changed parameters: DataLogging (storing the values for data graph) Debug (extended by different debug reporting levels) Make sure it starts to do the digitalization (check the Error field on the overview page). If it does not start a round within a minute, restart the device. \u203c\ufe0f If the system is working now without any issues, please open the configuration editor once and save the config.ini . This will update the file to the newest content \u203c\ufe0f Now you can safely update to the newest version.","title":"Installation"},{"location":"Installation/#installation","text":"The installation requires multiple steps: Get the right hardware and wire it up Flash the firmware onto the ESP32 Write the data to the SD-Card Insert the SD-Card into the ESP32 board Power/restart it.","title":"Installation"},{"location":"Installation/#hardware","text":"","title":"Hardware"},{"location":"Installation/#esp32-cam","text":"OV2640 camera module Micro SD-Card slot 4 or 8 MB PSRAM. It can be easily found on the typical internet stores, searching for ESP32-CAM for less than 10 EUR. How ever since the hardware is cheap and coming from China, you unluckily could pick a malfunction device. See Hardware Compatibility for further advice!","title":"ESP32-CAM"},{"location":"Installation/#usb-uart-interface","text":"For first time flashing the firmware a USB -> UART connector is needed. Later firmware upgrades than can be flashed via OTA.","title":"USB-&gt;UART interface"},{"location":"Installation/#power-supply","text":"For power supply a 5V source is needed. Most easily this can be done via an USB power supply. The power supply should support minimum 500mA. For buffering current peaks some users reported to use a large elco condensator like a 2200uF between ground and VCC. \u203c\ufe0f Attention: in several internet forums there are problems reported, in case the ESP32-CAM is only supplied with 3.3V.","title":"Power supply"},{"location":"Installation/#housing","text":"A small 3D-printable example for a very small case can be found in Thingiverse here: https://www.thingiverse.com/thing:4571627 \u203c\ufe0f Attention : the focus of the OV2640 needs to be adjusted, as it is normally set from ~40cm to infinity. In order to get an image that is big enough, it needs to be changed to about 10cm. Therefore the sealing glue on the objective ring needs to be removed with a scalpel or sharp knife. Afterwards the objective can be rotated clockwise until the image is sharp again.","title":"Housing"},{"location":"Installation/#wiring","text":"Beside the 5V power supply, only for the first flashing a connection to the USB-UART connector, including a short cut of GPIO0 to GND for bootloader start. A example for wiring can be found here: It is also possible to use external LEDs for the illumination instead of the internal flash LED. This is described here: [[External-LED]]","title":"Wiring"},{"location":"Installation/#firmware-flashing","text":"","title":"Firmware flashing"},{"location":"Installation/#files","text":"Grab the firmware from the Releases page (Stable, tested versions), or the Automatically build development branch (experimental, untested versions). Please have a look on https://github.com/jomjol/AI-on-the-edge-device/wiki/Install-a-rolling-%28unstable%29-release first! You need: partitions.bin bootloader.bin firmware.bin","title":"Files"},{"location":"Installation/#flashing","text":"There are several options to flash the firmware. Here three are described:","title":"Flashing"},{"location":"Installation/#1-web-installer","text":"There is a Web Installer available which will work right out of the web browser Edge and Chrome. You can access it with the following link: https://jomjol.github.io/AI-on-the-edge-device This is the preferred way for beginners as it also allows access to the USB Log:","title":"1. Web Installer"},{"location":"Installation/#2-using-the-flash-tool-from-espressif","text":"The flashing of the firmware can be done with the \"Flash Download Tool\" from espressif, that can found here Download and extract the Flash tool, after starting choose \"Developer Mode\", then \"ESP32-DownloadTool\" and you are in the setup of the flashing tool. Connect the ESP32-CAM with the USB-UART connection and identify the COM-Port. \u203c\ufe0f Attention : if you reflashing the code again, it is strongly recommended to erase the flash memory before flashing the firmware. Especially if you used OTA in between, which might cause remaining information on the flash, to still boot from an old image in the OTA-area, which is not erased by a normal flash. But your ESP32 in bootloader mode and push start, then it will identify the board and you can configure the bin-configuration according to the following table: Filename Offset bootloader.bin 0x1000 partitions.bin 0x8000 firmware.bin 0x10000 Alternatively it can be directly flashed from the development environment - here PlatformIO. But this is rather for experienced users, as the whole development chain needs to be installed for compilation.","title":"2. Using the Flash Tool from Espressif"},{"location":"Installation/#3-using-esptool-in-python-directly","text":"For this you need a python environment (e.g. Anaconda in Win10). Here you need to install the esptool: pip install esptool Then connect the ESP32 with the USB-UART connector to the system, put it in bootmode and with the following command you can erase the flash and flash bootloader, partitions and firmware in two steps: esptool erase_flash esptool write_flash 0x01000 bootloader.bin 0x08000 partitions.bin 0x10000 firmware.bin Maybe you need to specify the COM-port if it is not detected by default. If the erase command throws the error A fatal error occurred: ESP32 ROM does not support function erase_flash. , your esptool might be too old, see https://techoverflow.net/2022/02/08/how-to-fix-esp32-a-fatal-error-occurred-esp32-rom-does-not-support-function-erase_flash/ With some Python installations this may not work and you\u2019ll receive an error, try python -m pip install esptool or pip3 install esptool Further recommendations can be found on the espressif webpage","title":"3. Using esptool in python directly"},{"location":"Installation/#sd-card","text":"The software expects a SD-Card prepared with certain directory and file structure in order to work properly. For the first setup take the AI-on-the-edge-device__manual-setup__*.zip from the Release page, open the zip and extract the whole content of the in the setup file included sd-card.zip onto your SD-Card direclty to the root folder. SD-Card root should look like this: config demo firmware html img_tmp log wlan.ini This initial setup needs to be done only once as further updates of the software are possible with an Over-The-Air update mechanismn.","title":"SD-Card"},{"location":"Installation/#notes","text":"Due to the limited availability of GPIOs (OV2640, Flash-Light, PSRAM & SD-Card) the communication mode to the SD card is limited to 1-line SD-Mode. It showed up, that this results in problems with very large SD-Cards (64GB, sometimes 32 GB) and some no name low cost SD-cards. There must be no partition table on the SD-card (no GPT, but only MBR for the single partition) Following setting are necessary for formating the SD-card: SINGLE PARTITION, MBR, FAT32 - 32K. NOT exFAT Some ESP32 devices share their SD-card and/or camera GPIOs with the pins for TX and RX. If you see errors like \u201cFailed to connect\u201d then your chip is probably not entering the bootloader properly. Remove the respective modules temporarily to free the GPIOs for flashing. You may find more information about troubleshooting on the homepage of Espressif . The ESP32 indicates problems with the SD card during startup with a fast not ending blinking. In this case, please try another SD card.","title":"Notes"},{"location":"Installation/#wlan","text":"The access to the WLAN is configured in the \"wlan.ini\" directly on the root directory of the sd-card. Just write the corresponding SSID and password before the startup of the ESP32. This file is hidden from external access (e.g. via Filemanager) to protect the password. After power on the connection status is indicated by 3x blinking of the red on board LED. WLAN-Status indication: 5 x fast blinking (< 1 second): connection still pending 3 x slow blinking (1 second on/off): WLAN connection established It is normal that at first one or two times a pending connection is indicated.","title":"WLAN"},{"location":"Installation/#update-ota-over-the-air","text":"","title":"Update (OTA / Over-The-Air)"},{"location":"Installation/#update-from-version-greater-than-1200","text":"You can use the over the air update mechanism, which uploads the update via a ZIP files. The update file is located on the release page . Please choose the zip file with the following naming: AI-on-the-edge-device__update__*.zip Go to the menu System --> OTA Update and follow the instructions there. After a final automatic reboot you should have the new version running.","title":"Update from version greater than 12.0.0"},{"location":"Installation/#update-from-version-older-than-1200","text":"If you update from an version older than 12.0.1, you should firstly update to version 12.0.1. Background are not fully backward compatible changes in the config.ini , that are taken care of in this version. \u203c\ufe0f Make sure to read the instructions below carfully! Backup your configuration (use the System -> Backup/Restore page)! Upload and update the update-*.zip file from the release 12.0.1 see here . Let it restart and check on the System -> Info page that the Firmware as well as the Web UI got updated. If only one got updated, redo the update. If it fails several times, you also can update the Firmware and the Web UI separately. Safe way: Update first the firmware.bin (extract it from one of the provided zip files) and do the Reboot Update with the full zip file ( update-*.zip , ignore the version warning after the reboot) Please go to Settings -> Configuration and address the changed parameters: DataLogging (storing the values for data graph) Debug (extended by different debug reporting levels) Make sure it starts to do the digitalization (check the Error field on the overview page). If it does not start a round within a minute, restart the device. \u203c\ufe0f If the system is working now without any issues, please open the configuration editor once and save the config.ini . This will update the file to the newest content \u203c\ufe0f Now you can safely update to the newest version.","title":"Update from version older than 12.0.0"},{"location":"Integration-Home-Assistant/","text":"Integration into Home Assistant There are 3 ways to get the data into your Home Assistant: Using MQTT (Automatically Setup Entities using Homeassistant MQTT Discovery) Using MQTT (Manually Setup Entities) Using REST calls The first one is the easier way if you already have MQTT in use. Using MQTT (Automatically Setup Entities using Homeassistant MQTT Discovery) \u203c\ufe0f This feature will be available with the next release! Starting with Version >12.0.1 , AI-on-the-edge-devices support Homeassistant Discovery. Check here to learn more about it and how to enable it in Homeassistant. You also have to enable it in the MQTT settings of your device: Make sure to select the right Meter Type to get the right units! On the next start of the device, it will send discovery topics and Homeassistant should pick them up and show them under Settings > Integrations > MQTT : Using MQTT (Manually Setup Entities) First make sure with an MQTT client (for example MQTT Explorer ) that MQTT works as expected and to get a list of the available topics! Then add a sensor for each property: mqtt: sensor: - state_topic: \"wasserzaehler/main/value\" name: \"Watermeter Value\" unique_id: watermeter_value unit_of_measurement: 'm\u00b3' state_class: total_increasing device_class: water # Needs Homeassistant 2022.11! icon: 'mdi:water-pump' availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost - state_topic: \"wasserzaehler/main/rate\" name: \"Watermeter Rate\" unique_id: watermeter_rate unit_of_measurement: 'm\u00b3/min' state_class: measurement device_class: water # Needs Homeassistant 2022.11! icon: 'mdi:water-pump' availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost - state_topic: \"wasserzaehler/main/error\" name: \"Watermeter Error\" unique_id: watermeter_error icon: \"mdi:water-alert\" availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost - state_topic: \"wasserzaehler/uptime\" name: \"Watermeter Uptime\" unique_id: watermeter_uptime unit_of_measurement: 's' state_class: measurement device_class: duration entity_category: diagnostic icon: \"mdi:timer-outline\" availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost If you run the discovery once, you can also extract the information from there (MQTT Info, untested): mqtt: # Extracted form the Discovery but untested! sensor: - name: Value unique_id: wasserzaehler-main_value icon: mdi:gauge state_topic: wasserzaehler/main/value unit_of_measurement: m\u00b3 device_class: water state_class: total_increasing availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost If you want to convert the m\u00b3 to l , use a template sensor: template: - sensor: - name: \"Watermeter in l\" unique_id: watermeter_in_l icon: \"mdi:gauge\" state: \"{{ states('sensor.watermeter_value')|float(default=0) * 1000 }}\" # Convert 1 m3 => 1000 l unit_of_measurement: l availability: \"{{ states('sensor.watermeter_value') not in ['unknown', 'unavailable', 'none'] }}\" If you you want to have the consumption per day, you can use an Utility Meter . it is a helper and can be used to reset the total increasing values once a day utility_meter: utility_meter_gas_per_day: source: sensor.gasmeter_value cycle: daily utility_meter_water_per_day: source: sensor.watermeter_value cycle: daily Note that you also can add it using the UI. Examples Statistics Graph Creating Statistics Graphs (eg. usage per day) is easy using the Energy Dashboard : Note that there seems to be a bug in the graph, see https://github.com/home-assistant/frontend/issues/13995 ! InfluxDb Graphs See also Influx-DB . If you have setup InfluxDB already, it is also possible to fetch statistics from there, eg. daily usage: from(bucket: \"HomeAssistant\") |> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> filter(fn: (r) => r[\"entity_id\"] == \"wasserverbrauch_tag\") |> filter(fn: (r) => r[\"_field\"] == \"value\") |> timeShift(duration: -1d) |> aggregateWindow(every: 1d, fn: max, createEmpty: false) |> yield(name: \"mean\") Using REST When using REST, Home Assistant has to periodically call an URL on the ESP32 which in return provides the requested data. See REST API for a list of available URLs. The most practical one is the json entrypoint which provides the most relevant data JSON formated: http://<IP>/json This would return: { \"main\": { \"value\": \"512.3020\", \"raw\": \"0512.3020\", \"error\": \"no error\", \"rate\": 0.000000, \"timestamp\": \"2022-10-02T20:32:06\" [..] } } To do such a REST call, you need to create a REST sensor: sensor: - platform: rest name: \"Gasmeter JSON\" resource: http://<IP>/json json_attributes: - main value_template: '{{ value_json.value }}' headers: Content-Type: application/json scan_interval: 60 template: sensor: - name: \"Gasmeter Value from JSON\" unique_id: gas_meter_value_from_json state: \"{{ state_attr('sensor.gasmeter_json','main')['value'] }}\" unit_of_measurement: 'm\u00b3' - name: \"Watermeter Value from JSON\" unique_id: water_meter_value_from_json state: >- {{ state_attr('sensor.watermeter_json','main')['value'] | float }} unit_of_measurement: 'm\u00b3' device_class: water state_class: total_increasing icon: mdi:gauge See also https://community.home-assistant.io/t/rest-sensor-nested-json/243420/9 Photo REST can also be used to show the photo of the last round: To access it, use http://<IP>/img_tmp/alg_roi.jpg resp http://<IP>/img_tmp/raw.jpg .","title":"Integration into Home Assistant"},{"location":"Integration-Home-Assistant/#integration-into-home-assistant","text":"There are 3 ways to get the data into your Home Assistant: Using MQTT (Automatically Setup Entities using Homeassistant MQTT Discovery) Using MQTT (Manually Setup Entities) Using REST calls The first one is the easier way if you already have MQTT in use.","title":"Integration into Home Assistant"},{"location":"Integration-Home-Assistant/#using-mqtt-automatically-setup-entities-using-homeassistant-mqtt-discovery","text":"\u203c\ufe0f This feature will be available with the next release! Starting with Version >12.0.1 , AI-on-the-edge-devices support Homeassistant Discovery. Check here to learn more about it and how to enable it in Homeassistant. You also have to enable it in the MQTT settings of your device: Make sure to select the right Meter Type to get the right units! On the next start of the device, it will send discovery topics and Homeassistant should pick them up and show them under Settings > Integrations > MQTT :","title":"Using MQTT (Automatically Setup Entities using Homeassistant MQTT Discovery)"},{"location":"Integration-Home-Assistant/#using-mqtt-manually-setup-entities","text":"First make sure with an MQTT client (for example MQTT Explorer ) that MQTT works as expected and to get a list of the available topics! Then add a sensor for each property: mqtt: sensor: - state_topic: \"wasserzaehler/main/value\" name: \"Watermeter Value\" unique_id: watermeter_value unit_of_measurement: 'm\u00b3' state_class: total_increasing device_class: water # Needs Homeassistant 2022.11! icon: 'mdi:water-pump' availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost - state_topic: \"wasserzaehler/main/rate\" name: \"Watermeter Rate\" unique_id: watermeter_rate unit_of_measurement: 'm\u00b3/min' state_class: measurement device_class: water # Needs Homeassistant 2022.11! icon: 'mdi:water-pump' availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost - state_topic: \"wasserzaehler/main/error\" name: \"Watermeter Error\" unique_id: watermeter_error icon: \"mdi:water-alert\" availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost - state_topic: \"wasserzaehler/uptime\" name: \"Watermeter Uptime\" unique_id: watermeter_uptime unit_of_measurement: 's' state_class: measurement device_class: duration entity_category: diagnostic icon: \"mdi:timer-outline\" availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost If you run the discovery once, you can also extract the information from there (MQTT Info, untested): mqtt: # Extracted form the Discovery but untested! sensor: - name: Value unique_id: wasserzaehler-main_value icon: mdi:gauge state_topic: wasserzaehler/main/value unit_of_measurement: m\u00b3 device_class: water state_class: total_increasing availability_topic: wasserzaehler/connection payload_available: connected payload_not_available: connection lost If you want to convert the m\u00b3 to l , use a template sensor: template: - sensor: - name: \"Watermeter in l\" unique_id: watermeter_in_l icon: \"mdi:gauge\" state: \"{{ states('sensor.watermeter_value')|float(default=0) * 1000 }}\" # Convert 1 m3 => 1000 l unit_of_measurement: l availability: \"{{ states('sensor.watermeter_value') not in ['unknown', 'unavailable', 'none'] }}\" If you you want to have the consumption per day, you can use an Utility Meter . it is a helper and can be used to reset the total increasing values once a day utility_meter: utility_meter_gas_per_day: source: sensor.gasmeter_value cycle: daily utility_meter_water_per_day: source: sensor.watermeter_value cycle: daily Note that you also can add it using the UI.","title":"Using MQTT (Manually Setup Entities)"},{"location":"Integration-Home-Assistant/#examples","text":"","title":"Examples"},{"location":"Integration-Home-Assistant/#statistics-graph","text":"Creating Statistics Graphs (eg. usage per day) is easy using the Energy Dashboard : Note that there seems to be a bug in the graph, see https://github.com/home-assistant/frontend/issues/13995 !","title":"Statistics Graph"},{"location":"Integration-Home-Assistant/#influxdb-graphs","text":"See also Influx-DB . If you have setup InfluxDB already, it is also possible to fetch statistics from there, eg. daily usage: from(bucket: \"HomeAssistant\") |> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> filter(fn: (r) => r[\"entity_id\"] == \"wasserverbrauch_tag\") |> filter(fn: (r) => r[\"_field\"] == \"value\") |> timeShift(duration: -1d) |> aggregateWindow(every: 1d, fn: max, createEmpty: false) |> yield(name: \"mean\")","title":"InfluxDb Graphs"},{"location":"Integration-Home-Assistant/#using-rest","text":"When using REST, Home Assistant has to periodically call an URL on the ESP32 which in return provides the requested data. See REST API for a list of available URLs. The most practical one is the json entrypoint which provides the most relevant data JSON formated: http://<IP>/json This would return: { \"main\": { \"value\": \"512.3020\", \"raw\": \"0512.3020\", \"error\": \"no error\", \"rate\": 0.000000, \"timestamp\": \"2022-10-02T20:32:06\" [..] } } To do such a REST call, you need to create a REST sensor: sensor: - platform: rest name: \"Gasmeter JSON\" resource: http://<IP>/json json_attributes: - main value_template: '{{ value_json.value }}' headers: Content-Type: application/json scan_interval: 60 template: sensor: - name: \"Gasmeter Value from JSON\" unique_id: gas_meter_value_from_json state: \"{{ state_attr('sensor.gasmeter_json','main')['value'] }}\" unit_of_measurement: 'm\u00b3' - name: \"Watermeter Value from JSON\" unique_id: water_meter_value_from_json state: >- {{ state_attr('sensor.watermeter_json','main')['value'] | float }} unit_of_measurement: 'm\u00b3' device_class: water state_class: total_increasing icon: mdi:gauge See also https://community.home-assistant.io/t/rest-sensor-nested-json/243420/9","title":"Using REST"},{"location":"Integration-Home-Assistant/#photo","text":"REST can also be used to show the photo of the last round: To access it, use http://<IP>/img_tmp/alg_roi.jpg resp http://<IP>/img_tmp/raw.jpg .","title":"Photo"},{"location":"Learn-models-with-your-own-images/","text":"Learn a model with your own images If your device has new, different digits and the existing models don't recognize them well, you can collect your own images and train the model. Of course, this also applies if you have a model that is already known, but the neural models do not produce good results. But before you do this, you please check if your type really is not contained yet in the training data, see digits resp. pointers for an overview of images used for the training. But we also like to take pictures of already known devices. Before you start Poor recognition is often caused by blurred images, low contrast or incorrect setting of the ROIs. Therefore, check these possibilities first, as additional training will bring little improvement here. You must not train the models by yourself (see section Share your images ). In most cases we will integrate you images in the training dataset of the models. Only if we fear a degradation of the models or you need a different behaviour, we will not include the data in the standard models. Collecting images The neural network is trained on base of a set of images, that have been collected over time. If your digits are included or at least very similar to included images, the chance is very high that the neural network is working fine for you as well. The neural network configuration is stored in the TensorFlow Lite format as filename.tfl or filename.tflite in the /config directory. It can be updated by uploading the new file and activating it on the configuration page or in the config file /config/config.ini . In order to incorporate new digits a training set of images is required. The training images needs to be collected in the final setup with the help of the Digits or Analog log settings (not to be confused with the Data or Debug log). Enable the logging of the images on the configuration page or in the config file /config/config.ini : Now wait, until you have an image of each digit of every type on the SD card. Ideally remove the SD card from the camera and search for two to three images of each digit ( not more! :-) ). The format can be jpg. But don't panic, the collector programmes will sort out duplicate images for you. Collecting images for dig-class100/dig-cont/ana-class100 Collectmeterdigits and collectmeteranalog helps you to collect the images easily. Read the project readme for detailed instructions. Train the model For training the model you will need a python and Jupyter installation. All current labeled images you can find under ziffer_sortiert_raw dig-class11 models (digits) Fork and checkout neural-network-digital-counter-readout . Install all requirements for running the notebooks. pip install -r requirements.txt Put your labeled images into /ziffer_sortiert_raw folder and run Image_Preparation.ipynb Train_CNN_Digital-Readout-Small-v2.ipynb It creates a dig-class11_xxxx_s2.tflite model, you can upload to the config folder on your device and test it. dig-class100 / dig-cont models (digits) Fork and checkout neural-network-analog-needle-readout . All labeled images you can find under Images Install all requirements for running the notebooks. pip install -r requirements.txt Put your labeled images into images/collected/<typeofdevice>/<your_short>/ Run dig-class100-s2.ipynb . The model to upload to your device you can find under '/output'. ana-class100/ana-cont models (analog pointers) Fork and checkout neural-network-analog-needle-readout . All labeled images you can find under data_raw_all Install all requirements for running the notebooks. pip install -r requirements.txt Put your labeled images into images/collected/<typeofdevice>/<your_short>/ After every adding of images you need to run Image_Preparation.ipynb before you train the models. Run Train_CNN_Analog-Readout_100-Small1_Dropout.ipynb and/or Train_CNN_Analog-Readout_Version-Small2.ipynb . The model to upload to your device you can find in the project folder. Share your images If the results are good you can share the images as pull-request. Please images only! If you not able to create a pull request or don't know what it is, open an issue and put the zipped images in it. Images can be rejected if As same as dig-class11 collected, more than 1000 images of your device are really to much. images are not good configured (ROIs) will be rejected. It reduces the accuracy of the networks. Images with too little focus will be rejected. Images with too much blur are rejected. Our models are to small to recognize everything in any quality. So we use only images of medium or good quality.","title":"Learn a model with your own images"},{"location":"Learn-models-with-your-own-images/#learn-a-model-with-your-own-images","text":"If your device has new, different digits and the existing models don't recognize them well, you can collect your own images and train the model. Of course, this also applies if you have a model that is already known, but the neural models do not produce good results. But before you do this, you please check if your type really is not contained yet in the training data, see digits resp. pointers for an overview of images used for the training. But we also like to take pictures of already known devices.","title":"Learn a model with your own images"},{"location":"Learn-models-with-your-own-images/#before-you-start","text":"Poor recognition is often caused by blurred images, low contrast or incorrect setting of the ROIs. Therefore, check these possibilities first, as additional training will bring little improvement here. You must not train the models by yourself (see section Share your images ). In most cases we will integrate you images in the training dataset of the models. Only if we fear a degradation of the models or you need a different behaviour, we will not include the data in the standard models.","title":"Before you start"},{"location":"Learn-models-with-your-own-images/#collecting-images","text":"The neural network is trained on base of a set of images, that have been collected over time. If your digits are included or at least very similar to included images, the chance is very high that the neural network is working fine for you as well. The neural network configuration is stored in the TensorFlow Lite format as filename.tfl or filename.tflite in the /config directory. It can be updated by uploading the new file and activating it on the configuration page or in the config file /config/config.ini . In order to incorporate new digits a training set of images is required. The training images needs to be collected in the final setup with the help of the Digits or Analog log settings (not to be confused with the Data or Debug log). Enable the logging of the images on the configuration page or in the config file /config/config.ini : Now wait, until you have an image of each digit of every type on the SD card. Ideally remove the SD card from the camera and search for two to three images of each digit ( not more! :-) ). The format can be jpg. But don't panic, the collector programmes will sort out duplicate images for you.","title":"Collecting images"},{"location":"Learn-models-with-your-own-images/#collecting-images-for-dig-class100dig-contana-class100","text":"Collectmeterdigits and collectmeteranalog helps you to collect the images easily. Read the project readme for detailed instructions.","title":"Collecting images for dig-class100/dig-cont/ana-class100"},{"location":"Learn-models-with-your-own-images/#train-the-model","text":"For training the model you will need a python and Jupyter installation. All current labeled images you can find under ziffer_sortiert_raw","title":"Train the model"},{"location":"Learn-models-with-your-own-images/#dig-class11-models-digits","text":"Fork and checkout neural-network-digital-counter-readout . Install all requirements for running the notebooks. pip install -r requirements.txt Put your labeled images into /ziffer_sortiert_raw folder and run Image_Preparation.ipynb Train_CNN_Digital-Readout-Small-v2.ipynb It creates a dig-class11_xxxx_s2.tflite model, you can upload to the config folder on your device and test it.","title":"dig-class11 models (digits)"},{"location":"Learn-models-with-your-own-images/#dig-class100-dig-cont-models-digits","text":"Fork and checkout neural-network-analog-needle-readout . All labeled images you can find under Images Install all requirements for running the notebooks. pip install -r requirements.txt Put your labeled images into images/collected/<typeofdevice>/<your_short>/ Run dig-class100-s2.ipynb . The model to upload to your device you can find under '/output'.","title":"dig-class100 / dig-cont models (digits)"},{"location":"Learn-models-with-your-own-images/#ana-class100ana-cont-models-analog-pointers","text":"Fork and checkout neural-network-analog-needle-readout . All labeled images you can find under data_raw_all Install all requirements for running the notebooks. pip install -r requirements.txt Put your labeled images into images/collected/<typeofdevice>/<your_short>/ After every adding of images you need to run Image_Preparation.ipynb before you train the models. Run Train_CNN_Analog-Readout_100-Small1_Dropout.ipynb and/or Train_CNN_Analog-Readout_Version-Small2.ipynb . The model to upload to your device you can find in the project folder.","title":"ana-class100/ana-cont models (analog pointers)"},{"location":"Learn-models-with-your-own-images/#share-your-images","text":"If the results are good you can share the images as pull-request. Please images only! If you not able to create a pull request or don't know what it is, open an issue and put the zipped images in it.","title":"Share your images"},{"location":"Learn-models-with-your-own-images/#images-can-be-rejected-if","text":"As same as dig-class11 collected, more than 1000 images of your device are really to much. images are not good configured (ROIs) will be rejected. It reduces the accuracy of the networks. Images with too little focus will be rejected. Images with too much blur are rejected. Our models are to small to recognize everything in any quality. So we use only images of medium or good quality.","title":"Images can be rejected if"},{"location":"MQTT-API/","text":"MQTT API The device is capable to register to a MQTT broker to publish data and subscribe to specific topics. The MQTT service has to be enabled and configured properly in the device configuration via web interface ( Settings -> Configuration -> section MQTT ) The following parameters have to be defined: * URI * MainTopic (optional, if not set, the hostname is used) * ClientID (optional, if not set, AIOTED- + the MAC address gets used to make sure the ID is unique) * User (optional) * Password (optional) * RetainFlag (optional) Published topics Status MainTopic /{status topic}, e.g. watermeter/status * #### Connection Interval MAC IP Hostname Uptime FreeMem WifiRSSI CPUTemp Status Result MainTopic /{NumberName}/{result topic}, e.g. watermeter/main/value Value Raw Error JSON Rate Rate_per_time_unit The time Unit gets set with the Homeassistant Discovery, eg. h or m (minutes) Rate_per_digitalization_round The interval defines when the next round gets triggered Changeabsolut Timestamp JSON All relevant results in JSON syntax GPIO MainTopic /{GPIO topic}, e.g. watermeter/GPIO/GPIO12 GPIO/GPIO{PinNumber} Depending on device configuration ( Settings --> Configuration --> Chapter GPIO ) Subscibed topics MainTopic /{subscribed topic}, e.g. watermeter/ctrl/flow_start Control ctrl/flow_start Trigger a flow start by publishing to this topic (any character, length > 0) GPIO/GPIO{PinNumber} Depending on device configuration ( Settings --> Configuration --> Chapter GPIO )","title":"MQTT API"},{"location":"MQTT-API/#mqtt-api","text":"The device is capable to register to a MQTT broker to publish data and subscribe to specific topics. The MQTT service has to be enabled and configured properly in the device configuration via web interface ( Settings -> Configuration -> section MQTT ) The following parameters have to be defined: * URI * MainTopic (optional, if not set, the hostname is used) * ClientID (optional, if not set, AIOTED- + the MAC address gets used to make sure the ID is unique) * User (optional) * Password (optional) * RetainFlag (optional)","title":"MQTT API"},{"location":"MQTT-API/#published-topics","text":"","title":"Published topics"},{"location":"MQTT-API/#status","text":"MainTopic /{status topic}, e.g. watermeter/status * #### Connection","title":"Status"},{"location":"MQTT-API/#interval","text":"","title":"Interval"},{"location":"MQTT-API/#mac","text":"","title":"MAC"},{"location":"MQTT-API/#ip","text":"","title":"IP"},{"location":"MQTT-API/#hostname","text":"","title":"Hostname"},{"location":"MQTT-API/#uptime","text":"","title":"Uptime"},{"location":"MQTT-API/#freemem","text":"","title":"FreeMem"},{"location":"MQTT-API/#wifirssi","text":"","title":"WifiRSSI"},{"location":"MQTT-API/#cputemp","text":"","title":"CPUTemp"},{"location":"MQTT-API/#status_1","text":"","title":"Status"},{"location":"MQTT-API/#result","text":"MainTopic /{NumberName}/{result topic}, e.g. watermeter/main/value","title":"Result"},{"location":"MQTT-API/#value","text":"","title":"Value"},{"location":"MQTT-API/#raw","text":"","title":"Raw"},{"location":"MQTT-API/#error","text":"","title":"Error"},{"location":"MQTT-API/#json","text":"","title":"JSON"},{"location":"MQTT-API/#rate","text":"","title":"Rate"},{"location":"MQTT-API/#rate_per_time_unit","text":"The time Unit gets set with the Homeassistant Discovery, eg. h or m (minutes)","title":"Rate_per_time_unit"},{"location":"MQTT-API/#rate_per_digitalization_round","text":"The interval defines when the next round gets triggered","title":"Rate_per_digitalization_round"},{"location":"MQTT-API/#changeabsolut","text":"","title":"Changeabsolut"},{"location":"MQTT-API/#timestamp","text":"","title":"Timestamp"},{"location":"MQTT-API/#json_1","text":"All relevant results in JSON syntax","title":"JSON"},{"location":"MQTT-API/#gpio","text":"MainTopic /{GPIO topic}, e.g. watermeter/GPIO/GPIO12","title":"GPIO"},{"location":"MQTT-API/#gpiogpiopinnumber","text":"Depending on device configuration ( Settings --> Configuration --> Chapter GPIO )","title":"GPIO/GPIO{PinNumber}"},{"location":"MQTT-API/#subscibed-topics","text":"MainTopic /{subscribed topic}, e.g. watermeter/ctrl/flow_start","title":"Subscibed topics"},{"location":"MQTT-API/#control","text":"","title":"Control"},{"location":"MQTT-API/#ctrlflow_start","text":"Trigger a flow start by publishing to this topic (any character, length > 0)","title":"ctrl/flow_start"},{"location":"MQTT-API/#gpiogpiopinnumber_1","text":"Depending on device configuration ( Settings --> Configuration --> Chapter GPIO )","title":"GPIO/GPIO{PinNumber}"},{"location":"Neural-Network-Types/","text":"Neural Network Types Warning This page overlaps Choosing the Model . They should be merged to one page! This section is describing the different types of neural networks, that are used with the AI-on-the-edge approach and gives an introduction on how and where to use them. Overview neural network type There are two types of input : digits with rolling number (top town) analog pointers (clockwise rotating pointer) There are two types of neural networks : classification networks with discrete output neurons for each result class: 11 classes for digits (0, 1, ... 8, 9 + \"Not-A-Number\") 100 classes for digits or analog pointers (0.1, 0.2, 0.3, ... , 9.7, 9.8, 9.9) continuous output networks with a continuous output in the interval [0, 10[ No setting of the type in the firmware is necessary. The type can detect by the output structure automatically. \u203c\ufe0f Attention: It is very important to choose the right network type (digits or analog pointers). Technically a wrong network will work and create output, but that would be totally arbitrary Not all type of pointers are trained in all networks. For the 11 classes digits network there many different types of digits trained. The reason is, that you 1) only need 20-30 training images and 2) the data collection is ongoing much longer For the continious and 100 classes network especially for the digits, there are only a few types of digits trained up to now Therefore sometimes for the digits it is more effective to choose the simpler 11 classes network type (= default). Naming convention Classification 11 classes 0, 1, ... 9 + \"N\" Classification 100 classes 0.0, 0.1, ... 9.9 Continuous Interval [0, 10[ Digits dig-class11 _XXX.tflite dig-class100 _XXX.tflite dig-cont _XXX.tflite Analog Pointers ana-class100 _XXX.tflite ana-cont _XXX.tflite XXX contains the versioning and a parameter for different sizes with the following naming: XXX = versioning_sY versioning = version or in newer networks the training data Y = Neural network size (typically s1, s2, ..., s4). Whereas s1 is the maximum sized neural network and s4 is the smallest. Optional the naming ends with an \"_q\" to signal, that the tflite file has been quantized (size reduction with minimum accuracy loss). Example: dig-class11_1410_s2_q.tflite Classification network for digits with 11 classes (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, N) Version 1410 = 14.1.0 s2 = Size 2 (Medium) q = Quantized Version Overview of trained types and details Analog Pointer (\"ana-cont_XXX.tflite\" & \"ana-class100_XXX.tflite\") This is to transfer the direction of a pointer into a continuous number between 0 and 1, whereas 0 (=1) is the upwards position (12 o'clock), 0.25 corresponds to the 3 o'clock positions and so on. This network is a envolop for all different types of pointers. Currently there are no dedicated network trainings for specific types of pointers. There are two types of network structure, currently both are supported. The \"class100\" is a pure classification network, that might need a bit more accuracy in the labeling. \"cont\" is a no classic approach with a continuous output off only 2 neurons (details see below). Types of counters trained: Training data needs Quadratic images, minimum size: 32x32 pixel Typically 100 - 200 images with a resultion of 1/100 of the full rotation (every 0.1 value or 3.6\u00b0) Naming: x.y_ARBITRARY.jpg, where x.y = value 0.0 ... 9.9 CNN Technical details: Input 32 x 32 RGB images Output ana-cont _XXX.tflite: 2 neurons with output in range [-1, 1] - representing a sinus / cosinus encoding of the angle needs to be converted to angle with arctan-hyperbolicus function ana-class100 _XXX.tflite 100 neurons representing the classes from 0.0, 0.1, ... 9.8, 9.9 Digits with 11 classes (\"dig-class11_XXX.tflite\") The digit type is a classical classification network, with 11 classes representing the numbers 0, 1, ... 9 and the special class \"N\". It is trained for the rolling ring of gas and electric meters. As there is sometime a status between two images, the special class \"N\" is representing Not-A-Number for the case, that the image cannot be unique classified to one number e.g. because it is between two digits. For this type the lowest amount of training data per type is needed, resulting in a large variety of type being already part of the training set. Types of counters trained: Training data needs RGB images, with minimum size: 20x32 pixel Typically 10 - 20 images (1-2 for each digit and an arbitrary number for the \"N\" class Naming: x_ARBITRARY.jpg, where x = value 0 ... 9 + N CNN Technical details: Input 20 x 32 RGB images Output 11 neurons for image classification (last layer normalized to 1) Neuron 0 to 9 represent the corresponding numbers \"0\" to \"9\" Neron 10 represent the \"Not-A-Number\" class, telling, that the image is not uniquely classified Digits with rolling results (\"dig-class100_XXX.tflite\" & \"dig-cont_XXX.tflite\") This type of network tries to overcome the problem, that there are intermediate values, when a rolling digit is between two numbers. Previous this was the \"N\" class. In this network type, there are also subdigit values trained, so that the intermediate state can be used as additional information for the algorithms. Types of counters trained: [[images/dig-cont/dig-cont_1.jpg) [[images/dig-cont/dig-cont_2a.jpg) [[images/dig-cont/dig-cont_2b.jpg) [[images/dig-cont/dig-cont_3a.jpg) [[images/dig-cont/dig-cont_3b.jpg) [[images/dig-cont/dig-cont_3c.jpg) Training data needs RGB images, with minimum size: 20x32 pixel Typically 100 - 200 images (1-2 for each possible position) Naming: x.y_ARBITRARY.jpg, where x.y = 0.0, 0.1, ... 9.9 representing the intermediate state CNN Technical details: Input 20 x 32 RGB images Output dig-cont _XXX.tflite: 10 neurons representing the digits 0, 1, ... 9. The intermediate values are represented by weighted normalized values of two neighboring output neurons needs to be converted to angle with arctan-hyperbolicus function dig-class100 _XXX.tflite 100 neurons representing the classes from 0.0, 0.1, ... 9.8, 9.9","title":"Neural Network Types"},{"location":"Neural-Network-Types/#neural-network-types","text":"Warning This page overlaps Choosing the Model . They should be merged to one page! This section is describing the different types of neural networks, that are used with the AI-on-the-edge approach and gives an introduction on how and where to use them.","title":"Neural Network Types"},{"location":"Neural-Network-Types/#overview-neural-network-type","text":"There are two types of input : digits with rolling number (top town) analog pointers (clockwise rotating pointer) There are two types of neural networks : classification networks with discrete output neurons for each result class: 11 classes for digits (0, 1, ... 8, 9 + \"Not-A-Number\") 100 classes for digits or analog pointers (0.1, 0.2, 0.3, ... , 9.7, 9.8, 9.9) continuous output networks with a continuous output in the interval [0, 10[ No setting of the type in the firmware is necessary. The type can detect by the output structure automatically. \u203c\ufe0f Attention: It is very important to choose the right network type (digits or analog pointers). Technically a wrong network will work and create output, but that would be totally arbitrary Not all type of pointers are trained in all networks. For the 11 classes digits network there many different types of digits trained. The reason is, that you 1) only need 20-30 training images and 2) the data collection is ongoing much longer For the continious and 100 classes network especially for the digits, there are only a few types of digits trained up to now Therefore sometimes for the digits it is more effective to choose the simpler 11 classes network type (= default).","title":"Overview neural network type"},{"location":"Neural-Network-Types/#naming-convention","text":"Classification 11 classes 0, 1, ... 9 + \"N\" Classification 100 classes 0.0, 0.1, ... 9.9 Continuous Interval [0, 10[ Digits dig-class11 _XXX.tflite dig-class100 _XXX.tflite dig-cont _XXX.tflite Analog Pointers ana-class100 _XXX.tflite ana-cont _XXX.tflite XXX contains the versioning and a parameter for different sizes with the following naming: XXX = versioning_sY versioning = version or in newer networks the training data Y = Neural network size (typically s1, s2, ..., s4). Whereas s1 is the maximum sized neural network and s4 is the smallest. Optional the naming ends with an \"_q\" to signal, that the tflite file has been quantized (size reduction with minimum accuracy loss). Example: dig-class11_1410_s2_q.tflite Classification network for digits with 11 classes (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, N) Version 1410 = 14.1.0 s2 = Size 2 (Medium) q = Quantized Version","title":"Naming convention"},{"location":"Neural-Network-Types/#overview-of-trained-types-and-details","text":"","title":"Overview of trained types and details"},{"location":"Neural-Network-Types/#analog-pointer-ana-cont_xxxtflite-ana-class100_xxxtflite","text":"This is to transfer the direction of a pointer into a continuous number between 0 and 1, whereas 0 (=1) is the upwards position (12 o'clock), 0.25 corresponds to the 3 o'clock positions and so on. This network is a envolop for all different types of pointers. Currently there are no dedicated network trainings for specific types of pointers. There are two types of network structure, currently both are supported. The \"class100\" is a pure classification network, that might need a bit more accuracy in the labeling. \"cont\" is a no classic approach with a continuous output off only 2 neurons (details see below).","title":"Analog Pointer (\"ana-cont_XXX.tflite\" &amp; \"ana-class100_XXX.tflite\")"},{"location":"Neural-Network-Types/#types-of-counters-trained","text":"","title":"Types of counters trained:"},{"location":"Neural-Network-Types/#training-data-needs","text":"Quadratic images, minimum size: 32x32 pixel Typically 100 - 200 images with a resultion of 1/100 of the full rotation (every 0.1 value or 3.6\u00b0) Naming: x.y_ARBITRARY.jpg, where x.y = value 0.0 ... 9.9","title":"Training data needs"},{"location":"Neural-Network-Types/#cnn-technical-details","text":"","title":"CNN Technical details:"},{"location":"Neural-Network-Types/#input","text":"32 x 32 RGB images","title":"Input"},{"location":"Neural-Network-Types/#output","text":"ana-cont _XXX.tflite: 2 neurons with output in range [-1, 1] - representing a sinus / cosinus encoding of the angle needs to be converted to angle with arctan-hyperbolicus function ana-class100 _XXX.tflite 100 neurons representing the classes from 0.0, 0.1, ... 9.8, 9.9","title":"Output"},{"location":"Neural-Network-Types/#digits-with-11-classes-dig-class11_xxxtflite","text":"The digit type is a classical classification network, with 11 classes representing the numbers 0, 1, ... 9 and the special class \"N\". It is trained for the rolling ring of gas and electric meters. As there is sometime a status between two images, the special class \"N\" is representing Not-A-Number for the case, that the image cannot be unique classified to one number e.g. because it is between two digits. For this type the lowest amount of training data per type is needed, resulting in a large variety of type being already part of the training set.","title":"Digits with 11 classes (\"dig-class11_XXX.tflite\")"},{"location":"Neural-Network-Types/#types-of-counters-trained_1","text":"","title":"Types of counters trained:"},{"location":"Neural-Network-Types/#training-data-needs_1","text":"RGB images, with minimum size: 20x32 pixel Typically 10 - 20 images (1-2 for each digit and an arbitrary number for the \"N\" class Naming: x_ARBITRARY.jpg, where x = value 0 ... 9 + N","title":"Training data needs"},{"location":"Neural-Network-Types/#cnn-technical-details_1","text":"","title":"CNN Technical details:"},{"location":"Neural-Network-Types/#input_1","text":"20 x 32 RGB images","title":"Input"},{"location":"Neural-Network-Types/#output_1","text":"11 neurons for image classification (last layer normalized to 1) Neuron 0 to 9 represent the corresponding numbers \"0\" to \"9\" Neron 10 represent the \"Not-A-Number\" class, telling, that the image is not uniquely classified","title":"Output"},{"location":"Neural-Network-Types/#digits-with-rolling-results-dig-class100_xxxtflite-dig-cont_xxxtflite","text":"This type of network tries to overcome the problem, that there are intermediate values, when a rolling digit is between two numbers. Previous this was the \"N\" class. In this network type, there are also subdigit values trained, so that the intermediate state can be used as additional information for the algorithms.","title":"Digits with rolling results (\"dig-class100_XXX.tflite\" &amp; \"dig-cont_XXX.tflite\")"},{"location":"Neural-Network-Types/#types-of-counters-trained_2","text":"[[images/dig-cont/dig-cont_1.jpg) [[images/dig-cont/dig-cont_2a.jpg) [[images/dig-cont/dig-cont_2b.jpg) [[images/dig-cont/dig-cont_3a.jpg) [[images/dig-cont/dig-cont_3b.jpg) [[images/dig-cont/dig-cont_3c.jpg)","title":"Types of counters trained:"},{"location":"Neural-Network-Types/#training-data-needs_2","text":"RGB images, with minimum size: 20x32 pixel Typically 100 - 200 images (1-2 for each possible position) Naming: x.y_ARBITRARY.jpg, where x.y = 0.0, 0.1, ... 9.9 representing the intermediate state","title":"Training data needs"},{"location":"Neural-Network-Types/#cnn-technical-details_2","text":"","title":"CNN Technical details:"},{"location":"Neural-Network-Types/#input_2","text":"20 x 32 RGB images","title":"Input"},{"location":"Neural-Network-Types/#output_2","text":"dig-cont _XXX.tflite: 10 neurons representing the digits 0, 1, ... 9. The intermediate values are represented by weighted normalized values of two neighboring output neurons needs to be converted to angle with arctan-hyperbolicus function dig-class100 _XXX.tflite 100 neurons representing the classes from 0.0, 0.1, ... 9.8, 9.9","title":"Output"},{"location":"New-Releases-Notification/","text":"Notification about new Releases Do you want to get notified about a new release? There are several ways for it: Github Notifications You will need a Github Account for this! Log into your Github account on Github . Go to AI-on-the-edge-device . On the top right side, click onto Watch and select Custom : Select Releases . You will get an email when a new release gets created. See also Github Documentation . CodeRelease.io Alternatively or if you do not want to create a Github account, CodeRelease.io can be an alternative. You also have to subscribe with an emailaddress but no account is required.","title":"Notification about new Releases"},{"location":"New-Releases-Notification/#notification-about-new-releases","text":"Do you want to get notified about a new release? There are several ways for it:","title":"Notification about new Releases"},{"location":"New-Releases-Notification/#github-notifications","text":"You will need a Github Account for this! Log into your Github account on Github . Go to AI-on-the-edge-device . On the top right side, click onto Watch and select Custom : Select Releases . You will get an email when a new release gets created. See also Github Documentation .","title":"Github Notifications"},{"location":"New-Releases-Notification/#codereleaseio","text":"Alternatively or if you do not want to create a Github account, CodeRelease.io can be an alternative. You also have to subscribe with an emailaddress but no account is required.","title":"CodeRelease.io"},{"location":"REST-API/","text":"REST API Various information is directly accessible over specific REST calls. To use it, just append them to the IP, separated with a / , eg. http://192.168.1.1/json Note: For more detailed information to the REST handler, have a look to the code in the repository: registered handlers Control flow_start Trigger a flow start (if not running) Set Pre Value Set the Previous Value /setPreValue?value=1234&numbers=main where 1234 is the new value and main the name of the number to be adjusted. GPIO Control a GPIO output The GPIO entrypoint also support parameters: /GPIO?GPIO={PinNumber}&Status=high /GPIO?GPIO={PinNumber}&Status=low Example: /GPIO?GPIO=12&Status=high Read a GPIO input The GPIO entrypoint also support parameters: /GPIO?GPIO={PinNumber} Example: /GPIO?GPIO=12 ota ota_page.html Opens the Over-The-Air update html page reboot Trigger a reboot of the device Results json Show result in JSON syntax Example: { \"main\": { \"value\": \"521.17108\", \"raw\": \"521.17108\", \"pre\": \"521.17108\", \"error\": \"no error\", \"rate\": \"0.023780\", \"timestamp\": \"2023-01-13T16:00:42+0100\" } } value Show single result values The value entrypoint also support parameters: http://<IP>/value?all=true&type=value http://<IP>/value?all=true&type=raw http://<IP>/value?all=true&type=error http://<IP>/value?all=true&type=prevalue img_tmp/raw.jpg Capture and show a new raw image img_tmp/roi.jpg Show last aligned image img_tmp/alg_roi.jpg Show last aligned image incuding ROI overlay Status statusflow Show the actual step of the flow incl. timestamp Example: Take Image (15:56:34) rssi Show the WIFI signal strength Example: -51dBm cpu_temperature Show the CPU temperature Example: 38.5\u00b0C sysinfo Show system infos in JSON syntax Example: [ { \"firmware\" : \"xxx\", \"buildtime\" : \"2023-01-15 21:27\", \"gitbranch\" : \"xxx\", \"gittag\" : \"\", \"gitrevision\" : \"a5e533f+\", \"html\" : \"Development-Branch: xxx (Commit: a5e533f+)\", \"cputemp\" : \"67.777779\", \"hostname\" : \"xxx\", \"IPv4\" : \"192.168.xxx.xxx\", \"freeHeapMem\" : \"2789466\" } ] starttime Show starttime Example: 20230113-154634 uptime Show uptime Example: 0d 00h 15m 50s Camera lighton Switch the camera flashlight on lightoff Switch the camera flashlight off capture Capture a new image (without flashlight) capture_with_flashlight Capture a new image with flashlight save Save a new image to SD card The save entrypoint also support parameters: http://<IP>/save?filename=test.jpg&delay=1 Logs log Last part of todays log (last 80 kBytes)) logfileact Full log of today log.html Opens the log html page Diagnostics heap print relevant memory (heap) information Example: Heap info: Heap Total: 1888926 | SPI Free: 1827431 | SPI Larg Block: 1802240 | SPI Min Free: 758155 | Int Free: 61495 | Int Larg Block: 55296 | Int Min Free: 36427","title":"REST API"},{"location":"REST-API/#rest-api","text":"Various information is directly accessible over specific REST calls. To use it, just append them to the IP, separated with a / , eg. http://192.168.1.1/json Note: For more detailed information to the REST handler, have a look to the code in the repository: registered handlers","title":"REST API"},{"location":"REST-API/#control","text":"","title":"Control"},{"location":"REST-API/#flow_start","text":"Trigger a flow start (if not running)","title":"flow_start"},{"location":"REST-API/#set-pre-value","text":"Set the Previous Value /setPreValue?value=1234&numbers=main where 1234 is the new value and main the name of the number to be adjusted.","title":"Set Pre Value"},{"location":"REST-API/#gpio","text":"Control a GPIO output The GPIO entrypoint also support parameters: /GPIO?GPIO={PinNumber}&Status=high /GPIO?GPIO={PinNumber}&Status=low Example: /GPIO?GPIO=12&Status=high Read a GPIO input The GPIO entrypoint also support parameters: /GPIO?GPIO={PinNumber} Example: /GPIO?GPIO=12","title":"GPIO"},{"location":"REST-API/#ota","text":"","title":"ota"},{"location":"REST-API/#ota_pagehtml","text":"Opens the Over-The-Air update html page","title":"ota_page.html"},{"location":"REST-API/#reboot","text":"Trigger a reboot of the device","title":"reboot"},{"location":"REST-API/#results","text":"","title":"Results"},{"location":"REST-API/#json","text":"Show result in JSON syntax Example: { \"main\": { \"value\": \"521.17108\", \"raw\": \"521.17108\", \"pre\": \"521.17108\", \"error\": \"no error\", \"rate\": \"0.023780\", \"timestamp\": \"2023-01-13T16:00:42+0100\" } }","title":"json"},{"location":"REST-API/#value","text":"Show single result values The value entrypoint also support parameters: http://<IP>/value?all=true&type=value http://<IP>/value?all=true&type=raw http://<IP>/value?all=true&type=error http://<IP>/value?all=true&type=prevalue","title":"value"},{"location":"REST-API/#img_tmprawjpg","text":"Capture and show a new raw image","title":"img_tmp/raw.jpg"},{"location":"REST-API/#img_tmproijpg","text":"Show last aligned image","title":"img_tmp/roi.jpg"},{"location":"REST-API/#img_tmpalg_roijpg","text":"Show last aligned image incuding ROI overlay","title":"img_tmp/alg_roi.jpg"},{"location":"REST-API/#status","text":"","title":"Status"},{"location":"REST-API/#statusflow","text":"Show the actual step of the flow incl. timestamp Example: Take Image (15:56:34)","title":"statusflow"},{"location":"REST-API/#rssi","text":"Show the WIFI signal strength Example: -51dBm","title":"rssi"},{"location":"REST-API/#cpu_temperature","text":"Show the CPU temperature Example: 38.5\u00b0C","title":"cpu_temperature"},{"location":"REST-API/#sysinfo","text":"Show system infos in JSON syntax Example: [ { \"firmware\" : \"xxx\", \"buildtime\" : \"2023-01-15 21:27\", \"gitbranch\" : \"xxx\", \"gittag\" : \"\", \"gitrevision\" : \"a5e533f+\", \"html\" : \"Development-Branch: xxx (Commit: a5e533f+)\", \"cputemp\" : \"67.777779\", \"hostname\" : \"xxx\", \"IPv4\" : \"192.168.xxx.xxx\", \"freeHeapMem\" : \"2789466\" } ]","title":"sysinfo"},{"location":"REST-API/#starttime","text":"Show starttime Example: 20230113-154634","title":"starttime"},{"location":"REST-API/#uptime","text":"Show uptime Example: 0d 00h 15m 50s","title":"uptime"},{"location":"REST-API/#camera","text":"","title":"Camera"},{"location":"REST-API/#lighton","text":"Switch the camera flashlight on","title":"lighton"},{"location":"REST-API/#lightoff","text":"Switch the camera flashlight off","title":"lightoff"},{"location":"REST-API/#capture","text":"Capture a new image (without flashlight)","title":"capture"},{"location":"REST-API/#capture_with_flashlight","text":"Capture a new image with flashlight","title":"capture_with_flashlight"},{"location":"REST-API/#save","text":"Save a new image to SD card The save entrypoint also support parameters: http://<IP>/save?filename=test.jpg&delay=1","title":"save"},{"location":"REST-API/#logs","text":"","title":"Logs"},{"location":"REST-API/#log","text":"Last part of todays log (last 80 kBytes))","title":"log"},{"location":"REST-API/#logfileact","text":"Full log of today","title":"logfileact"},{"location":"REST-API/#loghtml","text":"Opens the log html page","title":"log.html"},{"location":"REST-API/#diagnostics","text":"","title":"Diagnostics"},{"location":"REST-API/#heap","text":"print relevant memory (heap) information Example: Heap info: Heap Total: 1888926 | SPI Free: 1827431 | SPI Larg Block: 1802240 | SPI Min Free: 758155 | Int Free: 61495 | Int Larg Block: 55296 | Int Min Free: 36427","title":"heap"},{"location":"ROI-Configuration/","text":"ROI (Region of Interest) General remark: You are using a neural network approach which is trained to fit as many different type of meters as possible. The accuracy will never be 100%. It is normal to see a missing reading once in a while. There there are several precautions to detect this. For details see the section PostProcessing on the configuration page. The most critical settings for accurate detection are: Correct setting of the R egions O f I nterest (ROIs) for detection of the image. This must be done manually for each meter! Number type is part of the training set. Have a look on the Digital Counters resp. Analog Needles to check if your types are contained. If your number types are not contained, you should take the effort to record them so we can add them to the training data. See: Learn models with your own images on how to create new input. Correct Setup of ROI Please proceed in the following order! Don't forget to save after each step! Image Sharpness Ensure a sharp image of the camera by adjusting the focal length of the ESP OV2640 camera. Adjust the focus for the clearest possible image! In order to use it for reading a meter, the focal-length of the OV2640 camera has to be manipulated, as by default it only results in sharp image for distance bigger than ~40cm, causing the image of the water meter to be too small for automated readout processing. ATTENTION: this modification will void any warranty, as the sealing of the lens objective is broken! ATTENTION: This modification will render the camera unsuitable for general, web-cam type applications unless the focal length is changed back to the original setting. Remove the fixing glue of the OV2640 lens with a sharp knife. After this you can screw the lens in and out. Rotating it by about a quarter of a turn counterclockwise results in a focus plane of about 10cm. You need to figure out your best setting with a little bit of trial and error for your specific environment. Horizontal Alignment Ensure an exact horizontal alignment of the number via the alignment / reference setup: \u2714\ufe0f Okay \u274c Not Okay Correct Size for ROI Choose the right size of the ROI: The configuration of ROIs differs a bit on the model you choose. Below you find the differences between the different AI models. Pick the one you think fits best your purpose. If you don't get to good result, try another model. Model Selection dig-class11 Configuration dig-class11 - Models recognize the complete digit only . Here it is not relevant if the ROI fits the Border of the digit window. For this model, there should be a border of 20% of the image size around the number itself. This border is shown in the ROI setup image by the inner thinner rectangle. This rectangle should fit perfectly around the number when the number has not started to rotate to the next position: Example 1 Example 2 \u2714\ufe0f Okay \u274c Not Okay \u274c Not Okay If you have perfect alignment you and are not getting satisfying results, most probably your numbers are not part of the training data yet. Read on Learn models with your own images how to add your meter's type of numbers to the training set. dig-class100 / dig-cont Configuration These models recognize the tenths (fractions) between the numbers. This model requires a different ROI setup; the height must be set differently and more accurately. First, the width can be set as for dig-class11, i.e. 20% margin left and right. The height of the outer rectangle should be set to the upper and lower edge of the number window. To achieve this setting, you need to unlock the aspect ratio: Here an example: Example 1 \u2714\ufe0f Okay \u274c Not Okay","title":"ROI (Region of Interest)"},{"location":"ROI-Configuration/#roi-region-of-interest","text":"General remark: You are using a neural network approach which is trained to fit as many different type of meters as possible. The accuracy will never be 100%. It is normal to see a missing reading once in a while. There there are several precautions to detect this. For details see the section PostProcessing on the configuration page. The most critical settings for accurate detection are: Correct setting of the R egions O f I nterest (ROIs) for detection of the image. This must be done manually for each meter! Number type is part of the training set. Have a look on the Digital Counters resp. Analog Needles to check if your types are contained. If your number types are not contained, you should take the effort to record them so we can add them to the training data. See: Learn models with your own images on how to create new input.","title":"ROI (Region of Interest)"},{"location":"ROI-Configuration/#correct-setup-of-roi","text":"Please proceed in the following order! Don't forget to save after each step!","title":"Correct Setup of ROI"},{"location":"ROI-Configuration/#image-sharpness","text":"Ensure a sharp image of the camera by adjusting the focal length of the ESP OV2640 camera. Adjust the focus for the clearest possible image! In order to use it for reading a meter, the focal-length of the OV2640 camera has to be manipulated, as by default it only results in sharp image for distance bigger than ~40cm, causing the image of the water meter to be too small for automated readout processing. ATTENTION: this modification will void any warranty, as the sealing of the lens objective is broken! ATTENTION: This modification will render the camera unsuitable for general, web-cam type applications unless the focal length is changed back to the original setting. Remove the fixing glue of the OV2640 lens with a sharp knife. After this you can screw the lens in and out. Rotating it by about a quarter of a turn counterclockwise results in a focus plane of about 10cm. You need to figure out your best setting with a little bit of trial and error for your specific environment.","title":"Image Sharpness"},{"location":"ROI-Configuration/#horizontal-alignment","text":"Ensure an exact horizontal alignment of the number via the alignment / reference setup: \u2714\ufe0f Okay \u274c Not Okay","title":"Horizontal Alignment"},{"location":"ROI-Configuration/#correct-size-for-roi","text":"Choose the right size of the ROI: The configuration of ROIs differs a bit on the model you choose. Below you find the differences between the different AI models. Pick the one you think fits best your purpose. If you don't get to good result, try another model.","title":"Correct Size for ROI"},{"location":"ROI-Configuration/#model-selection","text":"","title":"Model Selection"},{"location":"ROI-Configuration/#dig-class11-configuration","text":"dig-class11 - Models recognize the complete digit only . Here it is not relevant if the ROI fits the Border of the digit window. For this model, there should be a border of 20% of the image size around the number itself. This border is shown in the ROI setup image by the inner thinner rectangle. This rectangle should fit perfectly around the number when the number has not started to rotate to the next position: Example 1 Example 2 \u2714\ufe0f Okay \u274c Not Okay \u274c Not Okay If you have perfect alignment you and are not getting satisfying results, most probably your numbers are not part of the training data yet. Read on Learn models with your own images how to add your meter's type of numbers to the training set.","title":"dig-class11 Configuration"},{"location":"ROI-Configuration/#dig-class100-dig-cont-configuration","text":"These models recognize the tenths (fractions) between the numbers. This model requires a different ROI setup; the height must be set differently and more accurately. First, the width can be set as for dig-class11, i.e. 20% margin left and right. The height of the outer rectangle should be set to the upper and lower edge of the number window. To achieve this setting, you need to unlock the aspect ratio: Here an example: Example 1 \u2714\ufe0f Okay \u274c Not Okay","title":"dig-class100 / dig-cont Configuration"},{"location":"Release-creation/","text":"Preparing for Release Changelog is merged back from master branch to rolling branch (should be the last step of the previous release creation) All changes are documented in the Changelog in rolling branch Release creation steps Merge rolling into master branch Best to wait for the GitHub action to run successfully On master branch tag the version like v11.3.1 and don't forget to push it. Wait for the GitHub-Action of release creation. After all is done: the release should be created the artifacts are downloadable from release The documented changes were applied to the release Merge master back in rolling Check that the Web Installer shows the right version","title":"Preparing for Release"},{"location":"Release-creation/#preparing-for-release","text":"Changelog is merged back from master branch to rolling branch (should be the last step of the previous release creation) All changes are documented in the Changelog in rolling branch","title":"Preparing for Release"},{"location":"Release-creation/#release-creation-steps","text":"Merge rolling into master branch Best to wait for the GitHub action to run successfully On master branch tag the version like v11.3.1 and don't forget to push it. Wait for the GitHub-Action of release creation. After all is done: the release should be created the artifacts are downloadable from release The documented changes were applied to the release Merge master back in rolling Check that the Web Installer shows the right version","title":"Release creation steps"},{"location":"Testing/","text":"Testing Option for VSCode You can test your functions directly on the device. Structure All tests are under directory \"test\" in the project and not compiled with default build option of platformio. The main function is in file test_suite_controlflow.cpp . In method app_main() you can add your own tests. Include my my own test In method app_main() of test_suite_controlflow.cpp you can add your own tests. Include your test-file in the top like #include \"components/jomjol-flowcontroll/test_flow_postrocess_helper.cpp\" components is a subfolder of tests here. Not the components directory of root source. In the bottom add your test function. RUN_TEST(testNegative); Your test function should have a TEST_ASSERT_EQUAL_* . For more information look at unity-testing . Run tests You will need a testing device. best with usb adapter. Before you upload your tests you will need to setup the device with initial setup procedure described in [[Installation]] Now you can use Visual Studio Code or a standard console to upload the test code. In VS Code (tab platformio) open Advanced and select Test . Alternativ you can run it in console/terminal with platformio test --environment esp32cam . In my environment the serial terminal not opens. I have to do it for myself. You will see much logging. If any test fails it logs it out. Else it logs all test passed in the end. Troubleshooting If you test very much cases in one function, the device runs in stackoverflow and an endless boot. Reduce the count of test cases or split the test function in multiple functions.","title":"Testing"},{"location":"Testing/#testing-option-for-vscode","text":"You can test your functions directly on the device.","title":"Testing Option for VSCode"},{"location":"Testing/#structure","text":"All tests are under directory \"test\" in the project and not compiled with default build option of platformio. The main function is in file test_suite_controlflow.cpp . In method app_main() you can add your own tests.","title":"Structure"},{"location":"Testing/#include-my-my-own-test","text":"In method app_main() of test_suite_controlflow.cpp you can add your own tests. Include your test-file in the top like #include \"components/jomjol-flowcontroll/test_flow_postrocess_helper.cpp\" components is a subfolder of tests here. Not the components directory of root source. In the bottom add your test function. RUN_TEST(testNegative); Your test function should have a TEST_ASSERT_EQUAL_* . For more information look at unity-testing .","title":"Include my my own test"},{"location":"Testing/#run-tests","text":"You will need a testing device. best with usb adapter. Before you upload your tests you will need to setup the device with initial setup procedure described in [[Installation]] Now you can use Visual Studio Code or a standard console to upload the test code. In VS Code (tab platformio) open Advanced and select Test . Alternativ you can run it in console/terminal with platformio test --environment esp32cam . In my environment the serial terminal not opens. I have to do it for myself. You will see much logging. If any test fails it logs it out. Else it logs all test passed in the end.","title":"Run tests"},{"location":"Testing/#troubleshooting","text":"If you test very much cases in one function, the device runs in stackoverflow and an endless boot. Reduce the count of test cases or split the test function in multiple functions.","title":"Troubleshooting"},{"location":"Upload-files-by-script/","text":"Scripted File Upload To upload a file eg. using curl , you first have to delete it and then upload it: curl -d '' http://192.168.1.153/delete/html/index.html curl --data-binary @ota_page.html http://192.168.1.153/upload/html/index.html","title":"Scripted File Upload"},{"location":"Upload-files-by-script/#scripted-file-upload","text":"To upload a file eg. using curl , you first have to delete it and then upload it: curl -d '' http://192.168.1.153/delete/html/index.html curl --data-binary @ota_page.html http://192.168.1.153/upload/html/index.html","title":"Scripted File Upload"},{"location":"Watermeter-specific-analog---digital-transition/","text":"Analog/Digital Transition on Watermeters At first, for the most watermeters the default configuration should be work. But the digit, especially the last digit differs in some devices. \"Normal\" transition In most cases, the transition of the last digit starts when the analogue pointer is > 9. Often the last digit \"hangs\" a bit on this devices and comes not over zero. So it is not easy to see which digit is correct. In the first example 4 or still 3? (3 is correct). Early transition Some units start the transition very early or run with the analogue pointer. In the third example, is it a 3 or a 2? Inaccuracies in image recognition The models for image recognition are good, but have inaccuracies in the range +/- 0.2. In order to obtain as many correct results as possible, a treatment is carried out in the post process in the range of 9.8-0.2 for the analogue pointer, which must start differently depending on the type of counter. How to configure for my meter type If you have a devices with \"normal\" transition you should not have any issues. On devices with \"early\" transition, you can set the option AnalogDigitalTransitionStart to a value between 6 and 8.","title":"Analog/Digital Transition on Watermeters"},{"location":"Watermeter-specific-analog---digital-transition/#analogdigital-transition-on-watermeters","text":"At first, for the most watermeters the default configuration should be work. But the digit, especially the last digit differs in some devices.","title":"Analog/Digital Transition on Watermeters"},{"location":"Watermeter-specific-analog---digital-transition/#normal-transition","text":"In most cases, the transition of the last digit starts when the analogue pointer is > 9. Often the last digit \"hangs\" a bit on this devices and comes not over zero. So it is not easy to see which digit is correct. In the first example 4 or still 3? (3 is correct).","title":"\"Normal\" transition"},{"location":"Watermeter-specific-analog---digital-transition/#early-transition","text":"Some units start the transition very early or run with the analogue pointer. In the third example, is it a 3 or a 2?","title":"Early transition"},{"location":"Watermeter-specific-analog---digital-transition/#inaccuracies-in-image-recognition","text":"The models for image recognition are good, but have inaccuracies in the range +/- 0.2. In order to obtain as many correct results as possible, a treatment is carried out in the post process in the range of 9.8-0.2 for the analogue pointer, which must start differently depending on the type of counter.","title":"Inaccuracies in image recognition"},{"location":"Watermeter-specific-analog---digital-transition/#how-to-configure-for-my-meter-type","text":"If you have a devices with \"normal\" transition you should not have any issues. On devices with \"early\" transition, you can set the option AnalogDigitalTransitionStart to a value between 6 and 8.","title":"How to configure for my meter type"},{"location":"inital-setup/","text":"Initial Setup After setting up the device (firmware, sd-card, wlan) the device will connect to the wifi access point and start in an initial setup configuration: In the top you can navigate through 5 steps, that guide you through the necessary setup. All settings can be accessed also later in the normal operation mode (see below). 1. Reference Image The reference image is the basis for the coordinate system of the ROIs. Therefore it is very important, to have a well aligned image, that is not rotated. At first the current image is shown. To define a new reference image push the button \"Create new Reference\" (2) and afterwards \"Take Image\" (2). It might take some seconds for processing, then your actual camera image should be shown. Attention: This is the first time, where you have access to the camera image. It might not be sharp yet. This is the first thing to do according to the following instruction ( TBD ). Correct Horizontal Alignment Ensure an exact horizontal alignment of the number via the alignment / reference setup: \u2714\ufe0f Okay \u274c Not Okay \u203c\ufe0f Attention: Updating the reference image, also means, that all alignment images and ROIs needs to be teached again. Therefore do this step later only with caution. If everything is done, you can save the result with \"Update Reference Image\" (4). Please do not reboot at this stage, but go to the next section. 2. Alignment References The alignment references are used to realign every taken image to the reference coordinates. Therefore two alignment structures are identified and the image is shifted and rotated according to their position with the target to be in exactly the same position as the reference image. The alignment structures needs to be unique and have a good contrast. You can switch between this two marks with (1). With the control in (3) position and size of the selected reference image. You can define the ROI in the image also directly via drag and drop with the mouse. To choose the currently marked image part you need to push \"Update Reference\" (2). In some cases it might be useful to use a reference with a higher contrast. This can be achieved by pushing \"Enhance Contrast\" (3). The result will be calculated on the ESP32 - so be a bit patient, before you see it active. To save the push finally \"Save to config.ini\" (4). Do not reboot at this stage, but proceed with the \"next\" button. 3./4. Define the ROIs for the number detection Here the regions of interest for the digital and analog pointers are defined. As both are done identically, here as an example the digital images are shown. General usage First of all, if you don't have that kind of numbers on you meter, you can disable it with the check box at the top (1). In contrast to the reference images, here there are more ROIs possible. You can switch between them with the drop down box (2). If you need additional ROIs or delete them you can do this with the control at (2). Attention: The order of the ROIs defines, how the indiviual digits are combined to the total number. The first ROI is the digit with the highest order, then the second and so on. You can control the order in the selector tab and change it with the buttons \"move Next\" or \"move Previous\". As for the reference images you can change position, size and name of the ROI in the text fields or define them via drag and drop through the mouse button. In most cases the digits are ordered in a aequidistant order and have the same size, you can synchronize them with the control in (4). Don' t forget to save the settings with \"Save\" and do not reboot at this stage. Detail for ROI configuration - Analog Meters For analog meters the ROI setting is rather straight forward as the meter is usually quadratic with a clear center. The circle should exactly fit to the outer size of the meter and the cross should be in the middle. Here is an example with the details for the ROI \"ana1\": Detail for ROI configuration - Digital Meters For the digital meters it is a little bit more complicated, as there are different options of digital models, that you can choose. Digital meter, that only recognized full digits (0, 1, 2, 3, ... 9) - Naming: dig-class11-....tfl Advantage: broad variety of types included in the training Disadvantage: partially rotated numbers cannot be detected Model with subdigit resulition (0.0, 0.1, 0.2, .... 9.8, 9.9) - Naming: dig-cont-....tfl or dig-class100-....tfl Advantage: partial numbers can be detected and a better post processing is possible Disadvantage: only limited types of meter types are trained due to the high effort for the training data Details and the corresponding \"perfect\" setting is explained here: Details ROI Configuration For a first run you can choose the following general settings: There is an inner and an outer frame for the ROIs. Make the inner frame exactly the size of the number. Example 1 Example 2 \u2714\ufe0f Okay \u274c Not Okay \u274c Not Okay 5. General Settings In the next steps you can configure the behavior and external interfaces in detail: The configuration is divided into different sub topics: TakeImage Digits Analog PostProcessing MQTT InfluxDB GPIOSettings Autotimer DataLogging Debug System The details are explained in other parts of the manual (see links ( TBD )) Some of the sections as well as parameters are mandatory. They can be en/disabled in the first column (1). The setting itself is done in the next column (2) and a brief explanation you can find in the last column (3). Don' t forget to save the settings with \"Save\" and do not reboot at this stage. Expert Modus With the normal parameters you should be able to make the needed settings for most of the system. Sometimes there is some fine tunning needed. For this there is an expert modus available. This can be enabled with the check box at the top (4). After this you see much more parameters. But before modifiying them you should be really sure, what they are about. Finish Setup and change to normal operation After setting up everything, there is a last step to be done: With (1) you leave the setup modus and reboot to normal operation mode. Access to setup in normal operation mode You can access all the settings also during the normal working mode via the \"Settings\" menue: (1) Access to configuration parameters (2) Update of reference image (3) Update of alignment marks (4)/(5) Update of the ROI setting","title":"Initial Setup"},{"location":"inital-setup/#initial-setup","text":"After setting up the device (firmware, sd-card, wlan) the device will connect to the wifi access point and start in an initial setup configuration: In the top you can navigate through 5 steps, that guide you through the necessary setup. All settings can be accessed also later in the normal operation mode (see below).","title":"Initial Setup"},{"location":"inital-setup/#1-reference-image","text":"The reference image is the basis for the coordinate system of the ROIs. Therefore it is very important, to have a well aligned image, that is not rotated. At first the current image is shown. To define a new reference image push the button \"Create new Reference\" (2) and afterwards \"Take Image\" (2). It might take some seconds for processing, then your actual camera image should be shown. Attention: This is the first time, where you have access to the camera image. It might not be sharp yet. This is the first thing to do according to the following instruction ( TBD ).","title":"1. Reference Image"},{"location":"inital-setup/#correct-horizontal-alignment","text":"Ensure an exact horizontal alignment of the number via the alignment / reference setup: \u2714\ufe0f Okay \u274c Not Okay \u203c\ufe0f Attention: Updating the reference image, also means, that all alignment images and ROIs needs to be teached again. Therefore do this step later only with caution. If everything is done, you can save the result with \"Update Reference Image\" (4). Please do not reboot at this stage, but go to the next section.","title":"Correct Horizontal Alignment"},{"location":"inital-setup/#2-alignment-references","text":"The alignment references are used to realign every taken image to the reference coordinates. Therefore two alignment structures are identified and the image is shifted and rotated according to their position with the target to be in exactly the same position as the reference image. The alignment structures needs to be unique and have a good contrast. You can switch between this two marks with (1). With the control in (3) position and size of the selected reference image. You can define the ROI in the image also directly via drag and drop with the mouse. To choose the currently marked image part you need to push \"Update Reference\" (2). In some cases it might be useful to use a reference with a higher contrast. This can be achieved by pushing \"Enhance Contrast\" (3). The result will be calculated on the ESP32 - so be a bit patient, before you see it active. To save the push finally \"Save to config.ini\" (4). Do not reboot at this stage, but proceed with the \"next\" button.","title":"2. Alignment References"},{"location":"inital-setup/#34-define-the-rois-for-the-number-detection","text":"Here the regions of interest for the digital and analog pointers are defined. As both are done identically, here as an example the digital images are shown.","title":"3./4. Define the ROIs for the number detection"},{"location":"inital-setup/#general-usage","text":"First of all, if you don't have that kind of numbers on you meter, you can disable it with the check box at the top (1). In contrast to the reference images, here there are more ROIs possible. You can switch between them with the drop down box (2). If you need additional ROIs or delete them you can do this with the control at (2). Attention: The order of the ROIs defines, how the indiviual digits are combined to the total number. The first ROI is the digit with the highest order, then the second and so on. You can control the order in the selector tab and change it with the buttons \"move Next\" or \"move Previous\". As for the reference images you can change position, size and name of the ROI in the text fields or define them via drag and drop through the mouse button. In most cases the digits are ordered in a aequidistant order and have the same size, you can synchronize them with the control in (4). Don' t forget to save the settings with \"Save\" and do not reboot at this stage.","title":"General usage"},{"location":"inital-setup/#detail-for-roi-configuration-analog-meters","text":"For analog meters the ROI setting is rather straight forward as the meter is usually quadratic with a clear center. The circle should exactly fit to the outer size of the meter and the cross should be in the middle. Here is an example with the details for the ROI \"ana1\":","title":"Detail for ROI configuration - Analog Meters"},{"location":"inital-setup/#detail-for-roi-configuration-digital-meters","text":"For the digital meters it is a little bit more complicated, as there are different options of digital models, that you can choose. Digital meter, that only recognized full digits (0, 1, 2, 3, ... 9) - Naming: dig-class11-....tfl Advantage: broad variety of types included in the training Disadvantage: partially rotated numbers cannot be detected Model with subdigit resulition (0.0, 0.1, 0.2, .... 9.8, 9.9) - Naming: dig-cont-....tfl or dig-class100-....tfl Advantage: partial numbers can be detected and a better post processing is possible Disadvantage: only limited types of meter types are trained due to the high effort for the training data Details and the corresponding \"perfect\" setting is explained here: Details ROI Configuration For a first run you can choose the following general settings: There is an inner and an outer frame for the ROIs. Make the inner frame exactly the size of the number. Example 1 Example 2 \u2714\ufe0f Okay \u274c Not Okay \u274c Not Okay","title":"Detail for ROI configuration - Digital Meters"},{"location":"inital-setup/#5-general-settings","text":"In the next steps you can configure the behavior and external interfaces in detail: The configuration is divided into different sub topics: TakeImage Digits Analog PostProcessing MQTT InfluxDB GPIOSettings Autotimer DataLogging Debug System The details are explained in other parts of the manual (see links ( TBD )) Some of the sections as well as parameters are mandatory. They can be en/disabled in the first column (1). The setting itself is done in the next column (2) and a brief explanation you can find in the last column (3). Don' t forget to save the settings with \"Save\" and do not reboot at this stage.","title":"5. General Settings"},{"location":"inital-setup/#expert-modus","text":"With the normal parameters you should be able to make the needed settings for most of the system. Sometimes there is some fine tunning needed. For this there is an expert modus available. This can be enabled with the check box at the top (4). After this you see much more parameters. But before modifiying them you should be really sure, what they are about.","title":"Expert Modus"},{"location":"inital-setup/#finish-setup-and-change-to-normal-operation","text":"After setting up everything, there is a last step to be done: With (1) you leave the setup modus and reboot to normal operation mode.","title":"Finish Setup and change to normal operation"},{"location":"inital-setup/#access-to-setup-in-normal-operation-mode","text":"You can access all the settings also during the normal working mode via the \"Settings\" menue: (1) Access to configuration parameters (2) Update of reference image (3) Update of alignment marks (4)/(5) Update of the ROI setting","title":"Access to setup in normal operation mode"},{"location":"ota/","text":"Over-The-Air (OTA) Update You can do an OTA (over-the-air) update via the graphical user interface. Grab the firmware from the Releases page (Stable, tested versions), or the Automatically build development branch (experimental, untested versions). Please have a look on first! You need: * firmware.bin * html.zip General remark: It is always recommended to upload both files, as they are coupled to each other If you make a major update, it might be needed to modify the config.ini as it's syntax or context has changed It is recommended to make a backup of the /config directory, minimum of the config.ini . Access to the update page: The graphical OTA update can be accessed in the menue \"System\": Update Just follow the steps 1 to 5 to perform the update: Select (a) and upload (b) the file firmware.bin Flash the firmware Select (a) and upload (b) the file html.zip Update the html-files Reboot After the reboot with a major change it is recommended to check the configuration settings and save them again","title":"Over-The-Air (OTA) Update"},{"location":"ota/#over-the-air-ota-update","text":"You can do an OTA (over-the-air) update via the graphical user interface. Grab the firmware from the Releases page (Stable, tested versions), or the Automatically build development branch (experimental, untested versions). Please have a look on first! You need: * firmware.bin * html.zip","title":"Over-The-Air (OTA) Update"},{"location":"ota/#general-remark","text":"It is always recommended to upload both files, as they are coupled to each other If you make a major update, it might be needed to modify the config.ini as it's syntax or context has changed It is recommended to make a backup of the /config directory, minimum of the config.ini .","title":"General remark:"},{"location":"ota/#access-to-the-update-page","text":"The graphical OTA update can be accessed in the menue \"System\":","title":"Access to the update page:"},{"location":"ota/#update","text":"Just follow the steps 1 to 5 to perform the update: Select (a) and upload (b) the file firmware.bin Flash the firmware Select (a) and upload (b) the file html.zip Update the html-files Reboot After the reboot with a major change it is recommended to check the configuration settings and save them again","title":"Update"},{"location":"outdated--Gasmeter-Log-Downloader/","text":"Gasmeter Log-Downloader Warning This page no longer is maintained! This small tool downloads the logfiles from your ESP32 and stores the last value of the day in an *.csv file. To use this tool you need to activate the debug logfile in your configuration (Configuration / Debug / Logfile). I go with 30 days of retention in days. It downloads only the past logfiles (yesterday and older). You can define the max. number of Logfiles to download (beginning from newest [yesterday]). I wrote this tool to get a chart of the daily gas consumption to optimize my gas powered heating. Variables to define by yourself: URL to Logfile-Path on Device: \"http://ESP32-IP-Address/fileserver/log/message/\" Download Logfiles to: enter a valid directory, e.g. \"D:\\Gaszaehler\\Auswertung\\Log-Downloads\\\" Output CSV-File: enter a valid directory, e.g. \"D:\\Gaszaehler\\Auswertung\\DailyValues.csv\" Download Logfiles from past # days: enter the max. number of logfiles you want to download (<= your logfile retention value in your device configuration) Feel free to optimize and modify it.","title":"Gasmeter Log-Downloader"},{"location":"outdated--Gasmeter-Log-Downloader/#gasmeter-log-downloader","text":"Warning This page no longer is maintained! This small tool downloads the logfiles from your ESP32 and stores the last value of the day in an *.csv file. To use this tool you need to activate the debug logfile in your configuration (Configuration / Debug / Logfile). I go with 30 days of retention in days. It downloads only the past logfiles (yesterday and older). You can define the max. number of Logfiles to download (beginning from newest [yesterday]). I wrote this tool to get a chart of the daily gas consumption to optimize my gas powered heating. Variables to define by yourself: URL to Logfile-Path on Device: \"http://ESP32-IP-Address/fileserver/log/message/\" Download Logfiles to: enter a valid directory, e.g. \"D:\\Gaszaehler\\Auswertung\\Log-Downloads\\\" Output CSV-File: enter a valid directory, e.g. \"D:\\Gaszaehler\\Auswertung\\DailyValues.csv\" Download Logfiles from past # days: enter the max. number of logfiles you want to download (<= your logfile retention value in your device configuration) Feel free to optimize and modify it.","title":"Gasmeter Log-Downloader"},{"location":"outdated--Integrated-Functions/","text":"Integrated Functions Warning This page no longer is maintained! wasserzaehler http://IP-ESP32/wasserzaehler.html This is the main purpose of this device. It returns the converted image as a number with different option. The output can be modified either by the configuration parameters or by HTML parameters. Details can be found here: tbd Picture Server http://IP-ESP32/capture http://IP-ESP32/capture_with_flashlight This is a implementation of the camera interface of https://github.com/jomjol/water-meter-picture-provider It is fully compatible including the parameters ( quality =..., size=... ) . This allows to use this ESP32 system in parallel to the corresponding docker system: https://github.com/jomjol/water-meter-system-complete, from which this project is basically the successor. File server Access: http://IP-ESP32/fileserver/ Simple file server, that allows viewing, upload, download and deleting of single files of the SD-card content. The usage is self explaining. The file path or file can directly be accessed by the URL after file server. Example for config.ini : http://IP-ESP/fileserver/config/config.ini OTA-Update http://IP-ESP32/ota?file=firmware.bin Here an over the air update can be triggered. The firmware file is expected to be located in the sub directory /firmware/ and can be uploaded with the file server. By the parameter file the name of the firmware file needs to be given. Reboot http://IP-ESP32/reboot A reboot with a delay of 5 seconds is initiated, e.g. after firmware update. ATTENTION : currently this is not working properly - hardware power off is needed instead. Work in progress! Simple Web Server If none of the above URLs are fitting, a very simple web server checks, if there is a fitting file from the sub directory /html This can be used for a very simple web server for information or simple web pages.","title":"Integrated Functions"},{"location":"outdated--Integrated-Functions/#integrated-functions","text":"Warning This page no longer is maintained!","title":"Integrated Functions"},{"location":"outdated--Integrated-Functions/#wasserzaehler","text":"http://IP-ESP32/wasserzaehler.html This is the main purpose of this device. It returns the converted image as a number with different option. The output can be modified either by the configuration parameters or by HTML parameters. Details can be found here: tbd","title":"wasserzaehler"},{"location":"outdated--Integrated-Functions/#picture-server","text":"http://IP-ESP32/capture http://IP-ESP32/capture_with_flashlight This is a implementation of the camera interface of https://github.com/jomjol/water-meter-picture-provider It is fully compatible including the parameters ( quality =..., size=... ) . This allows to use this ESP32 system in parallel to the corresponding docker system: https://github.com/jomjol/water-meter-system-complete, from which this project is basically the successor.","title":"Picture Server"},{"location":"outdated--Integrated-Functions/#file-server","text":"Access: http://IP-ESP32/fileserver/ Simple file server, that allows viewing, upload, download and deleting of single files of the SD-card content. The usage is self explaining. The file path or file can directly be accessed by the URL after file server. Example for config.ini : http://IP-ESP/fileserver/config/config.ini","title":"File server"},{"location":"outdated--Integrated-Functions/#ota-update","text":"http://IP-ESP32/ota?file=firmware.bin Here an over the air update can be triggered. The firmware file is expected to be located in the sub directory /firmware/ and can be uploaded with the file server. By the parameter file the name of the firmware file needs to be given.","title":"OTA-Update"},{"location":"outdated--Integrated-Functions/#reboot","text":"http://IP-ESP32/reboot A reboot with a delay of 5 seconds is initiated, e.g. after firmware update. ATTENTION : currently this is not working properly - hardware power off is needed instead. Work in progress!","title":"Reboot"},{"location":"outdated--Integrated-Functions/#simple-web-server","text":"If none of the above URLs are fitting, a very simple web server checks, if there is a fitting file from the sub directory /html This can be used for a very simple web server for information or simple web pages.","title":"Simple Web Server"},{"location":"outdated--Migrate-Old-Config-To-New-Config/","text":"Migration from water-meter \u201eold\u201c to water-meter \u201cAI-on-the-edge-device\u201d Warning This page no longer is maintained! There are only some few steps necessary to migrate your old system to the new one. Please follow the following steps: 1. Follow the installation guide to flash the ESP32CAM and prepare a SD-Card with the content of the master 2. Save the following files from the old Docker system on your PC: Reference Points 1-3 (only 2 needed) Config.ini 3. Copy Reference Points 1-3 onto the new water-meter system (Directory /config ) Please note only two Reference Points are supported in the new system. 4. Open new config.ini File: Insert from the old Config.ini file [alignment] and [alignment.ref0] and [alignment.ref1] section the two Ref x and y position and the initial_rotation_angle= 123 into the new Config.ini File, e.g.: Old: [alignment.ref0] image=./config/RB01_65x65.jpg pos_x=28 pos_y=63 [alignment.ref1] image=./config/RB02_50x35.jpg pos_x=497 pos_y=127 [alignment] initial_rotation_angle=180 New: [Alignment] InitalRotate=180 /config/RB01_65x65.jpg 28, 63 /config/RB02_50x35.jpg 497, 127 SearchFieldX = 20 SearchFieldY = 20 5. Insert the old Digit Values into the new Config.ini File, e.g.: Old: [Digital_Digit.ziffer1] pos_x=265 pos_y=117 dx=28 dy=51 [Digital_Digit.ziffer2] pos_x=310 pos_y=117 dx=28 dy=51 [Digital_Digit.ziffer3] pos_x=354 pos_y=117 dx=28 dy=51 [Digital_Digit.ziffer4] pos_x=399 pos_y=117 dx=28 dy=51 [Digital_Digit.ziffer5] pos_x=445 pos_y=115 dx=28 dy=51 New: [Digits] Model=/config/dig0630s3.tflite ;LogImageLocation = /log/digit ModelInputSize 20, 32 digit1, 265, 117, 28, 51 digit2, 310, 117, 28, 51 digit3, 354, 117, 28, 51 digit4, 399, 117, 28, 51 digit5, 445, 115, 28, 51 6. Make sure that you have the same quality and size settings as in your old Config.ini In the old configuration this was coded in the html-string for the image source: Old: URLImageSource=http://IP-ADRESS/capture_with_flashlight?quality=5&size=VGA Default was Quality=5 and VGA. New: ImageQuality = 5 ImageSize = VGA 7. Repeat the same for the analog section 8. Insert your SSID and Password into the new wlan.ini File 9. Compare and edit [ConsistencyCheck] Section with new [PostProcessing] Section 10. Save new config.ini File in the new System. 11. Restart the system. 12. After the first start set manually the PreValue in the new system","title":"Migration from water-meter \u201eold\u201c to water-meter \u201cAI-on-the-edge-device\u201d"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#migration-from-water-meter-old-to-water-meter-ai-on-the-edge-device","text":"Warning This page no longer is maintained! There are only some few steps necessary to migrate your old system to the new one. Please follow the following steps:","title":"Migration from water-meter \u201eold\u201c to water-meter \u201cAI-on-the-edge-device\u201d"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#1-follow-the-installation-guide-to-flash-the-esp32cam-and-prepare-a-sd-card-with-the-content-of-the-master","text":"","title":"1. Follow the installation guide to flash the ESP32CAM and prepare a SD-Card with the content of the master"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#2-save-the-following-files-from-the-old-docker-system-on-your-pc","text":"Reference Points 1-3 (only 2 needed) Config.ini","title":"2. Save the following files from the old Docker system on your PC:"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#3-copy-reference-points-1-3-onto-the-new-water-meter-system-directory-config","text":"Please note only two Reference Points are supported in the new system.","title":"3. Copy Reference Points 1-3 onto the new water-meter system (Directory /config)"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#4-open-new-configini-file","text":"Insert from the old Config.ini file [alignment] and [alignment.ref0] and [alignment.ref1] section the two Ref x and y position and the initial_rotation_angle= 123 into the new Config.ini File, e.g.:","title":"4. Open new config.ini File:"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#old","text":"[alignment.ref0] image=./config/RB01_65x65.jpg pos_x=28 pos_y=63 [alignment.ref1] image=./config/RB02_50x35.jpg pos_x=497 pos_y=127 [alignment] initial_rotation_angle=180","title":"Old:"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#new","text":"[Alignment] InitalRotate=180 /config/RB01_65x65.jpg 28, 63 /config/RB02_50x35.jpg 497, 127 SearchFieldX = 20 SearchFieldY = 20","title":"New:"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#5-insert-the-old-digit-values-into-the-new-configini-file-eg","text":"","title":"5. Insert the old Digit Values into the new Config.ini File, e.g.:"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#old_1","text":"[Digital_Digit.ziffer1] pos_x=265 pos_y=117 dx=28 dy=51 [Digital_Digit.ziffer2] pos_x=310 pos_y=117 dx=28 dy=51 [Digital_Digit.ziffer3] pos_x=354 pos_y=117 dx=28 dy=51 [Digital_Digit.ziffer4] pos_x=399 pos_y=117 dx=28 dy=51 [Digital_Digit.ziffer5] pos_x=445 pos_y=115 dx=28 dy=51","title":"Old:"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#new_1","text":"[Digits] Model=/config/dig0630s3.tflite ;LogImageLocation = /log/digit ModelInputSize 20, 32 digit1, 265, 117, 28, 51 digit2, 310, 117, 28, 51 digit3, 354, 117, 28, 51 digit4, 399, 117, 28, 51 digit5, 445, 115, 28, 51","title":"New:"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#6-make-sure-that-you-have-the-same-quality-and-size-settings-as-in-your-old-configini","text":"In the old configuration this was coded in the html-string for the image source:","title":"6. Make sure that you have the same quality and size settings as in your old Config.ini"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#old_2","text":"URLImageSource=http://IP-ADRESS/capture_with_flashlight?quality=5&size=VGA Default was Quality=5 and VGA.","title":"Old:"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#new_2","text":"ImageQuality = 5 ImageSize = VGA","title":"New:"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#7-repeat-the-same-for-the-analog-section","text":"","title":"7. Repeat the same for the analog section"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#8-insert-your-ssid-and-password-into-the-new-wlanini-file","text":"","title":"8. Insert your SSID and Password into the new wlan.ini File"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#9-compare-and-edit-consistencycheck-section-with-new-postprocessing-section","text":"","title":"9. Compare and edit [ConsistencyCheck] Section with new [PostProcessing] Section"},{"location":"outdated--Migrate-Old-Config-To-New-Config/#10-save-new-configini-file-in-the-new-system","text":"","title":"10.  Save new config.ini File in the new System."},{"location":"outdated--Migrate-Old-Config-To-New-Config/#11-restart-the-system","text":"","title":"11. Restart the system."},{"location":"outdated--Migrate-Old-Config-To-New-Config/#12-after-the-first-start-set-manually-the-prevalue-in-the-new-system","text":"","title":"12. After the first start set manually the PreValue in the new system"},{"location":"rolling-installation/","text":"Living on the edge The branch rolling contains the latest version of the Firmware and the Web Interface. It might already contain a fix for your issue. But it is work in progress, don't expect it to work stable or be an improvement for your AI-on-the-edge-device! Also it might break the OTA Update and thus require manual flashing over USB! You still want to try it? Grab the latest rolling build from Github Actions Page and proceed as following: Pick the most top successful (green) build. Download the AI-on-the-edge-device__update__rolling_*.zip Flash the zip file using the OTA Update page of your device.","title":"Living on the edge"},{"location":"rolling-installation/#living-on-the-edge","text":"The branch rolling contains the latest version of the Firmware and the Web Interface. It might already contain a fix for your issue. But it is work in progress, don't expect it to work stable or be an improvement for your AI-on-the-edge-device! Also it might break the OTA Update and thus require manual flashing over USB!","title":"Living on the edge"},{"location":"rolling-installation/#you-still-want-to-try-it","text":"Grab the latest rolling build from Github Actions Page and proceed as following: Pick the most top successful (green) build. Download the AI-on-the-edge-device__update__rolling_*.zip Flash the zip file using the OTA Update page of your device.","title":"You still want to try it?"}]}
\ No newline at end of file
diff --git a/sitemap.xml.gz b/sitemap.xml.gz
index 8416818254abba204eda2adf9ef04d3e9128e3ff..96c6c5f15e8f45a30d1f5e23ff758fbb63ed7bd9 100644
GIT binary patch
delta 15
WcmbQjI)#-@zMF$%apOj|J|+Mm-vnL&

delta 15
WcmbQjI)#-@zMF$1r+y<_9}@r|iv%SA