Mirror/AI-on-the-edge-device-docs
1
0
Fork 0
mirror of https://github.com/jomjol/AI-on-the-edge-device-docs.git synced 2025-12-06 03:26:54 +03:00
Code Issues Packages Projects Releases Wiki Activity

Add Makefile and Markdown linter (#73)

Browse Source 
* Add Makefile

* Update .gitignore to exclude venv

* Add requirements.txt for venv

* Add pymarkdown lint Github action

* Update .md from lint tool

* Update README with make commands

* Add linter config
This commit is contained in:
Adriaan Van Niekerk
2025-10-31 01:06:36 +02:00
committed by GitHub
parent bbe3bd79db
commit 0cd98c67e2
44 changed files with 524 additions and 279 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
.github/workflows/markdown-style-checks.yml vendored Normal file
View File

@@ -0,0 +1,25 @@
name: Markdown style checks
on:
push:
branches:
- main
paths:
- 'docs/**' # Only run on changes to the docs directory
pull_request:
branches:
- '*'
paths:
- 'docs/**' # Only run on changes to the docs directory
jobs:
markdown-lint:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v5
with:
fetch-depth: 0
- name: Create venv
run: make install
- name: Lint markdown
run: make lint-md

3
.gitignore vendored
View File

@@ -1,4 +1,5 @@
docs/Parameters.md docs/Parameters.md
docs/img/* docs/img/*
.idea .idea
site site
venv

46
.pymarkdown.json Normal file
View File

@@ -0,0 +1,46 @@
{
"plugins": {
"selectively_enable_rules": true,
"heading-style": {
"enabled": true,
"style": "atx"
},
"commands-show-output": {
"enabled": true
},
"no-missing-space-atx": {
"enabled": true
},
"blanks-around-headings": {
"enabled": true
},
"heading-start-left": {
"enabled": true
},
"no-trailing-punctuation": {
"enabled": true,
"punctuation": ".,;。，；"
},
"blanks-around-fences": {
"enabled": true,
"list_items": false
},
"blanks-around-lists": {
"enabled": true
},
"hr-style": {
"enabled": true
},
"no-empty-links": {
"enabled": true
},
"no-alt-text": {
"enabled": true
}
},
"extensions": {
"front-matter" : {
"enabled" : true
}
}
}

47
Makefile Normal file
View File

@@ -0,0 +1,47 @@
DOCSDIR = docs
VENVDIR = venv
VENV = $(VENVDIR)/bin/activate
MKDOCS_CONFIG ?= mkdocs.yml
BUILDDIR ?= site
TARGET = docs
VALE_CONFIG = .vale.ini
.PHONY: build install serve spelling vale-install
# If requirements are updated, venv should be rebuilt and timestamped.
$(VENVDIR):
@echo "... setting up virtualenv"
python3 -m venv $(VENVDIR) || { echo "You must install python3-venv before you can build the documentation."; exit 1; }
. $(VENV); pip install $(PIPOPTS) --require-virtualenv \
--upgrade -r requirements.txt \
--log $(VENVDIR)/pip_install.log
@test ! -f $(VENVDIR)/pip_list.txt || \
mv $(VENVDIR)/pip_list.txt $(VENVDIR)/pip_list.txt.bak
@. $(VENV); pip list --local --format=freeze > $(VENVDIR)/pip_list.txt
@touch $(VENVDIR)
install: $(VENVDIR)
build: install
. $(VENV); mkdocs build -f $(MKDOCS_CONFIG)
lint-md: pymarkdownlnt-install
@. $(VENV); pymarkdownlnt --config .pymarkdown.json scan docs --recurse --exclude=./$(SPHINXDIR)/** $(SOURCEDIR)
lint-md-fix: pymarkdownlnt-install
@. $(VENV); pymarkdownlnt fix --recurse $(DOCSDIR)
pymarkdownlnt-install:
@. $(VENV); test -d $(VENVDIR)/lib/python*/site-packages/pymarkdown || pip install pymarkdownlnt
serve: install
. $(VENV); mkdocs serve -f $(MKDOCS_CONFIG) --livereload
vale-install: install
@. $(VENV);
@. $(VENV); find $(VENVDIR)/lib/python*/site-packages/vale/vale_bin -size 195c -exec vale --version \;
spelling: vale-install
@echo "Running Vale against $(TARGET). To change target set TARGET= with make command"
@. $(VENV); vale --config="$(VALE_CONFIG)" --glob='*.{md,rst}' $(TARGET)

15
README.md
View File

@@ -53,13 +53,8 @@ See https://python-markdown.github.io/extensions/admonition/
## Local Test ## Local Test
To test it locally: To test it locally:
1. Clone this repo 1. Clone this repo
1. Install the required tools (See also [.github/workflows/build-docs.yaml](.github/workflows/build-docs.yaml)): 2. Run `make install` that creates a virtual environment and installs all the dependencies
``` 3. Run `make serve`. This will locally generate the documentation. You can access it under http://127.0.0.1:8000/AI-on-the-edge-device-docs/
pip install --upgrade pip Any change to the files will automatically be applied.
pip install mkdocs mkdocs-gen-files mkdocs-awesome-pages-plugin mkdocs-material pymdown-extensions mkdocs-enumerate-headings-plugin 4. Add your edits.
``` 5. Run `make lint-md` and correct any issues before opening a PR.
1. In the main folder of the repo, call `mkdocs serve` (and keep it running).
This will locally generate the documentation.
You can access it under http://127.0.0.1:8000/AI-on-the-edge-device-docs/
Any change to the files will automatically be applied.

4
docs/Additional-Information.md
View File

@@ -1,9 +1,11 @@
The following links point to additional information in other repos: The following links point to additional information in other repos:
# Digits # Digits
* [Training the CNN neural network](https://github.com/jomjol/neural-network-digital-counter-readout/blob/master/Train_Network.md) * [Training the CNN neural network](https://github.com/jomjol/neural-network-digital-counter-readout/blob/master/Train_Network.md)
* [Training and using a neural network to readout the value of a digit counter](https://github.com/jomjol/neural-network-digital-counter-readout) * [Training and using a neural network to readout the value of a digit counter](https://github.com/jomjol/neural-network-digital-counter-readout)
# Analog # Analog
* [Training and using a neural network to read out the value of an analog display](https://github.com/jomjol/neural-network-analog-needle-readout) * [Training and using a neural network to read out the value of an analog display](https://github.com/jomjol/neural-network-analog-needle-readout)
* [Training the CNN neural network](https://github.com/jomjol/neural-network-analog-needle-readout/blob/master/Train_Network.md) * [Training the CNN neural network](https://github.com/jomjol/neural-network-analog-needle-readout/blob/master/Train_Network.md)

11
docs/Alignment.md
View File

@@ -1,26 +1,25 @@
# Alignment References # Alignment References
The alignment references are used in every round to re-align the taken image to the reference coordinates. The alignment references are used in every round to re-align the taken image to the reference coordinates.
Two alignment structures must be defined and the taken image then in each round is shifted and rotated according to their position Two alignment structures must be defined and the taken image then in each round is shifted and rotated according to their position
with the target to be in exactly the same position as the reference image. with the target to be in exactly the same position as the reference image.
!!! Note !!! Note
The alignment structures needs to be unique and have a good contrast. The alignment structures needs to be unique and have a good contrast.
It is advised to have them as far apart as possible. It is advised to have them as far apart as possible.
## Precondition ## Precondition
Please make sure to have setup your camera properly and taken a good [Reference Image](Reference-Image.md). Please make sure to have setup your camera properly and taken a good [Reference Image](Reference-Image.md).
## Define two Reference Images ## Define two Reference Images
![](img/initial_setup_2_alignment_marks.jpg){: style="width:500px"} ![Initial Setup Alignment Marks](img/initial_setup_2_alignment_marks.jpg){: style="width:500px"}
You can switch between this two marks with `(1)`. You can switch between this two marks with `(1)`.
Then define the reference area in the image by either directly drag and drop with the mouse or use the input boxes below. Then define the reference area in the image by either directly drag and drop with the mouse or use the input boxes below.
To apply the currently marked image part you need to push `"Update Reference" (2)`. To apply the currently marked image part you need to push `"Update Reference" (2)`.
In some cases it might be useful to use a reference with a higher contrast. This can be achieved by pushing `Enhance Contrast" (3)`. In some cases it might be useful to use a reference with a higher contrast. This can be achieved by pushing `Enhance Contrast" (3)`.
The result will be calculated on the ESP32 - so be a bit patient, before you see it active. The result will be calculated on the ESP32 - so be a bit patient, before you see it active.

6
docs/Best-Practice.md
View File

@@ -1,18 +1,22 @@
# Best Practice # Best Practice
This page shows some best practices: This page shows some best practices:
## Camera Placement ## Camera Placement
* Move the Camera as close as possible (~4cm), this will help get rid of reflections. * Move the Camera as close as possible (~4cm), this will help get rid of reflections.
-> focus can be adjusted by turning the outer black ring of the camera. -> focus can be adjusted by turning the outer black ring of the camera.
* If the LED reflections are too strong, put tape over the LED to diffuse the light * If the LED reflections are too strong, put tape over the LED to diffuse the light
* Change the ImageSize to QVGA under "Expert mode" configuration when close enough, this will be faster and is often good enough for digit recognition. * Change the ImageSize to QVGA under "Expert mode" configuration when close enough, this will be faster and is often good enough for digit recognition.
## Reflections ## Reflections
* Try to get rid of the reflections by rotating the camera, so that the reflections are at positions, where no number is. * Try to get rid of the reflections by rotating the camera, so that the reflections are at positions, where no number is.
* By using the external LED option, you can place WS2812 LEDs freely away from the main axis. * By using the external LED option, you can place WS2812 LEDs freely away from the main axis.
* Users report, that a handy cover foil could also help * Users report, that a handy cover foil could also help
## Post-processing ## Post-processing
* Filter out the Number "9", as "3" will often be misread for a "9" and void every number between 3 and 9 due to it being negative flow. * Filter out the Number "9", as "3" will often be misread for a "9" and void every number between 3 and 9 due to it being negative flow.
* Split the readings into two, while the decimal numbers might move to fast to be recognized, at least the slower moving part will produce a correct reading. * Split the readings into two, while the decimal numbers might move to fast to be recognized, at least the slower moving part will produce a correct reading.
-> keep in mind that the offset needs to be adjusted, a.e if you have a comma reading of "3", it needs to become "0.3". This can be done wherever the data ends up being sent, like home assistant using sensor templates. -> keep in mind that the offset needs to be adjusted, a.e if you have a comma reading of "3", it needs to become "0.3". This can be done wherever the data ends up being sent, like home assistant using sensor templates.

1
docs/Build-Instructions.md
View File

@@ -1,2 +1,3 @@
# Build the Project # Build the Project
See [README.md](https://github.com/jomjol/AI-on-the-edge-device/blob/master/code/README.md) in the main repo. See [README.md](https://github.com/jomjol/AI-on-the-edge-device/blob/master/code/README.md) in the main repo.

33
docs/Choosing-the-Model.md
View File

@@ -9,6 +9,7 @@ This page tries to help you on which model to select.
For more technical/deeper explanations have a look on [Neural-Network-Types](Neural-Network-Types.md). For more technical/deeper explanations have a look on [Neural-Network-Types](Neural-Network-Types.md).
## Digit Models ## Digit Models
For digits on water meters, gas-meters or power meters you can select between two main types of models: For digits on water meters, gas-meters or power meters you can select between two main types of models:
- `dig-class11` - `dig-class11`
@@ -19,27 +20,29 @@ For digits on water meters, gas-meters or power meters you can select between tw
**continuous** means, that there is no discrete model, that has discreate states, but there is a different mechanism, that provides a not discrete value in the interval between [0, 1[. **continuous** means, that there is no discrete model, that has discreate states, but there is a different mechanism, that provides a not discrete value in the interval between [0, 1[.
### `dig-class11` ### `dig-class11`
This model can recognize **full digits**. It was the first model version. All intermediate states shown a `N` for not-a-number (aka. `NaN`). But in post process it uses older values to fill up the `N` values if possible. This model can recognize **full digits**. It was the first model version. All intermediate states shown a `N` for not-a-number (aka. `NaN`). But in post process it uses older values to fill up the `N` values if possible.
![](img/dig-class11.png){: style="width:300px"} ![Dig Class11](img/dig-class11.png){: style="width:300px"}
It's possibly a good fallback, if `dig-cont` or `dig-class100` results are not good. It's possibly a good fallback, if `dig-cont` or `dig-class100` results are not good.
Main features: Main features:
* well suited for LCD digits - well suited for LCD digits
* the ExtendedResolution option is not supported. (Only in conjunction with `ana-class100` or `ana-cont`) - the ExtendedResolution option is not supported. (Only in conjunction with `ana-class100` or `ana-cont`)
### `dig-class100` and `dig-cont` ### `dig-class100` and `dig-cont`
These models are used to get a **continuous reading** with intermediate states. To see what the models are doing, you can go to the Recognition page of your device. These models are used to get a **continuous reading** with intermediate states. To see what the models are doing, you can go to the Recognition page of your device.
![](img/dig-class100.png){: style="width:300px"} ![Did Class100](img/dig-class100.png){: style="width:300px"}
Main features: Main features:
* suitable for all digit displays. - suitable for all digit displays.
* Advantage over `dig-class11` that results continue to be calculated in the transition between digits. - Advantage over `dig-class11` that results continue to be calculated in the transition between digits.
* With the ExtendedResolution option, higher accuracy is possible by adding another digit. - With the ExtendedResolution option, higher accuracy is possible by adding another digit.
The difference between `dig-class100` and `dig-cont` is in the internal processing. The difference between `dig-class100` and `dig-cont` is in the internal processing.
The `dig-class100` is a standard classification model. Each tenth step is an output. The `dig-class100` is a standard classification model. Each tenth step is an output.
@@ -51,13 +54,14 @@ The `dig-class100` is a standard classification model. Each tenth step is an out
Look [here](https://jomjol.github.io/neural-network-digital-counter-readout) for a list of digit images used for the training. Look [here](https://jomjol.github.io/neural-network-digital-counter-readout) for a list of digit images used for the training.
## Analog pointer models ## Analog pointer models
For pointers on water meters use the analog models: For pointers on water meters use the analog models:
![](img/ana-class100.png){: style="width:250px"} ![ANA Class100](img/ana-class100.png){: style="width:250px"}
You can choose between two models: You can choose between two models:
- `ana-class100` - `ana-class100`
- `ana-cont` - `ana-cont`
Both do mainly the same. Both do mainly the same.
@@ -65,8 +69,8 @@ Both do mainly the same.
Main features: Main features:
* for all analogue pointers, especially for water meters. - for all analogue pointers, especially for water meters.
* With the ExtendedResolution option, higher accuracy is possible by adding another digit. - With the ExtendedResolution option, higher accuracy is possible by adding another digit.
Again, the difference between `ana-class100` and `ana-cont` is in the internal processing. Again, the difference between `ana-class100` and `ana-cont` is in the internal processing.
@@ -76,6 +80,7 @@ Again, the difference between `ana-class100` and `ana-cont` is in the internal p
Look [here](https://jomjol.github.io/neural-network-analog-needle-readout/) for a list of pointer images used for the training Look [here](https://jomjol.github.io/neural-network-analog-needle-readout/) for a list of pointer images used for the training
## Different types of models (normal vs. quantized) ## Different types of models (normal vs. quantized)
The normally trained network is calculating with internal floating point numbers. The saving of floating point numbers naturally takes more space than an integer type. Often the increased accuracy is not needed. Therefore there is the option, to "quantize" a neural network. In this case the internal values are rescaled to integer values, which is called "quantization". The stored tflite files are usually much smaller and runs faster on the edgeAI-device. The normally trained network is calculating with internal floating point numbers. The saving of floating point numbers naturally takes more space than an integer type. Often the increased accuracy is not needed. Therefore there is the option, to "quantize" a neural network. In this case the internal values are rescaled to integer values, which is called "quantization". The stored tflite files are usually much smaller and runs faster on the edgeAI-device.
Usually the models are distrusted therefore in both versions. They can be distinguished by a `q` at the end of the logfile. Usually the models are distrusted therefore in both versions. They can be distinguished by a `q` at the end of the logfile.
@@ -87,6 +92,7 @@ Example:
| Quantized | `dig-cont_0610_s3-q.tflite` | | Quantized | `dig-cont_0610_s3-q.tflite` |
## Model Naming Convention ## Model Naming Convention
Model filenames follow a specific structure composed of several parts: Model filenames follow a specific structure composed of several parts:
1. **Model Type**: Indicates the kind of model, such as `ana/dig`, `cont`, `class11`, or `class100`. 1. **Model Type**: Indicates the kind of model, such as `ana/dig`, `cont`, `class11`, or `class100`.
@@ -96,6 +102,7 @@ Model filenames follow a specific structure composed of several parts:
5. **File Type**: Always `.tflite`. 5. **File Type**: Always `.tflite`.
### Version Number ### Version Number
The version number consists of four digits: The version number consists of four digits:
- The **first two digits** represent the *main version*. - The **first two digits** represent the *main version*.
@@ -106,6 +113,7 @@ Example:
In general, higher numbers correspond to newer models. In general, higher numbers correspond to newer models.
### Size Indicator ### Size Indicator
- The size indicator typically starts with `s` followed by a number, e.g., `s3`. - The size indicator typically starts with `s` followed by a number, e.g., `s3`.
- This naming is not strictly standardized yet. - This naming is not strictly standardized yet.
- Training usually begins with a larger model (`s0`), and successive versions (`s1`, `s2`, ...) reduce the number of parameters to create smaller, faster networks. - Training usually begins with a larger model (`s0`), and successive versions (`s1`, `s2`, ...) reduce the number of parameters to create smaller, faster networks.
@@ -113,5 +121,6 @@ Example:
- In newer model generations, only the best-performing sizes are retained and further trained. - In newer model generations, only the best-performing sizes are retained and further trained.
### Quantization Indicator ### Quantization Indicator
- If the model name ends with `-q`, it indicates that **quantization** was applied after training. - If the model name ends with `-q`, it indicates that **quantization** was applied after training.
- Quantization significantly reduces the model size, typically without a noticeable impact on recognition performance. - Quantization significantly reduces the model size, typically without a noticeable impact on recognition performance.

22
docs/Configuration.md
View File

@@ -1,4 +1,5 @@
# Graphical Configuration # Graphical Configuration
Most of the settings can be modified on the Settings page: Most of the settings can be modified on the Settings page:
<img src="../img/config_s1_access.jpg" width="600" align="middle"> <img src="../img/config_s1_access.jpg" width="600" align="middle">
@@ -12,18 +13,20 @@ It can be reached via the menu `Settings > Configuration`.
All parameters are documented on the [Parameters](Parameters.md) page and as tooltips on the config page. All parameters are documented on the [Parameters](Parameters.md) page and as tooltips on the config page.
## Expert Parameters ## Expert Parameters
Some parameters are treated as **Expert Parameters** and are hidden by default. Some parameters are treated as **Expert Parameters** and are hidden by default.
Tick the checkbox in the top left corner to enable them: Tick the checkbox in the top left corner to enable them:
![](img/expert-parameters.png) ![Expert Parameters](img/expert-parameters.png)
The **Expert Parameters** then will be shown with a red background: The **Expert Parameters** then will be shown with a red background:
![](img/expert-parameters2.png) ![Second Expert Parameters](img/expert-parameters2.png)
## Manual Editing of the Config File ## Manual Editing of the Config File
Even more configuration parameters can be edited manually in the `config.ini`: Even more configuration parameters can be edited manually in the `config.ini`:
![](img/manual-config-editing.png) ![Manual Config Editing](img/manual-config-editing.png)
To edit the `config.ini` file directly, click on the `Edit Config.ini directly` button. To edit the `config.ini` file directly, click on the `Edit Config.ini directly` button.
@@ -35,31 +38,40 @@ To edit the `config.ini` file directly, click on the `Edit Config.ini directly`
The principle is very simple and can most easily be described as a flow of processing steps. Each step has a dedicated parameter description in the `config.ini`, which is indicated by brackets ```[name_of_step]```. The steps are processed in the order written in the config file. That means, that you first have to describe the image taking, then the aligning and cutting and only after that you can start to config a neural network. The last step is the post processing. The principle is very simple and can most easily be described as a flow of processing steps. Each step has a dedicated parameter description in the `config.ini`, which is indicated by brackets ```[name_of_step]```. The steps are processed in the order written in the config file. That means, that you first have to describe the image taking, then the aligning and cutting and only after that you can start to config a neural network. The last step is the post processing.
### Processing steps - Overview ### Processing steps - Overview
In the following you get a short overview over the available steps. This order is also the suggested order for the processing flow. Single steps can be left out, if not needed (e.g. omit the analog part, if only digits are present) In the following you get a short overview over the available steps. This order is also the suggested order for the processing flow. Single steps can be left out, if not needed (e.g. omit the analog part, if only digits are present)
#### 1. `[TakeImage]` #### 1. `[TakeImage]`
This steps parametrises the taking of the image by the ESP32-CAM. Size, quality and storage for logging and debugging can be set. This steps parametrises the taking of the image by the ESP32-CAM. Size, quality and storage for logging and debugging can be set.
#### 2. `[Alignment]` #### 2. `[Alignment]`
Image preprocessing, including image alignment with reference images Image preprocessing, including image alignment with reference images
#### 3. `[Digits]` #### 3. `[Digits]`
Neural network evaluation of an image for digits. The neural network is defined by a tflite formatted file and the output is a number between 0 .. 9 or NaN (if image is not unique enough) Neural network evaluation of an image for digits. The neural network is defined by a tflite formatted file and the output is a number between 0 .. 9 or NaN (if image is not unique enough)
#### 4. `[Analog]` #### 4. `[Analog]`
Neural network evaluation of analog counter. The neural network is defined by a tflite formatted file and the output is a number between 0.0 .. 9.9, representing the position of the pointer. Neural network evaluation of analog counter. The neural network is defined by a tflite formatted file and the output is a number between 0.0 .. 9.9, representing the position of the pointer.
#### 5. `[PostProcessing]` #### 5. `[PostProcessing]`
Summarized the individually converted pictures to the overall result. It also implements some error corrections and consistency checks to filter wrong reading. Summarized the individually converted pictures to the overall result. It also implements some error corrections and consistency checks to filter wrong reading.
For more details look at [Correction Algorithm](Correction%20Algorithm.md)). For more details look at [Correction Algorithm](Correction%20Algorithm.md)).
#### 6. `[MQTT]` #### 6. `[MQTT]`
Transfer of the readings to a MQTT server. Transfer of the readings to a MQTT server.
#### 7. `[AutoTimer]` #### 7. `[AutoTimer]`
Configuration of the automated flow start at the start up of the ESP32.
Configuration of the automated flow start at the start up of the ESP32.
#### 8. `[Debug]` #### 8. `[Debug]`
Configuration for debugging details Configuration for debugging details

33
docs/Correction Algorithm.md
View File

@@ -8,17 +8,15 @@ There are several reasons, that a check might be necessary:
2. The replacement of the "N" with a previous value could be not sufficient, due to the fact, that it might have changed. 2. The replacement of the "N" with a previous value could be not sufficient, due to the fact, that it might have changed.
3. There is a misreading of one one of the numbers. This can always happen in case of neural network processing. 3. There is a misreading of one one of the numbers. This can always happen in case of neural network processing.
## Terms and definitions
### PreValue
### Terms and definitions
##### PreValue
The last correct read value. PreValue is here a bit missleading, because normally it is the same as the last value. In the next round of reading it will be used to check negative rates, high rates (MaxRateValue / MaxRateType) and CCheckDigitIncreaseConsistency (dig-class11 only). Either from a previous correctly identified value or manual setting by the user. The last correct read value. PreValue is here a bit missleading, because normally it is the same as the last value. In the next round of reading it will be used to check negative rates, high rates (MaxRateValue / MaxRateType) and CCheckDigitIncreaseConsistency (dig-class11 only). Either from a previous correctly identified value or manual setting by the user.
##### Digits #### Digits
Value that are digitized from a digit number. There are 11 allowed values for this: Value that are digitized from a digit number. There are 11 allowed values for this:
1. Digits: 0, 1, 2, ... 9 1. Digits: 0, 1, 2, ... 9
2. N = Not-a-Number - representing a not unique state between two numbers 2. N = Not-a-Number - representing a not unique state between two numbers
@@ -31,7 +29,7 @@ This are value derived from a pointer like meter. This never has the state "N".
If this is enabled an "intelligent" algorithm is used to derive from zero-crossing of discrete digit positions, if the number should have been increased. This is relevant because in some of the digit meters, the increase of a digit to the next number can be seen, before the sub-digit has gone through zero. If this is enabled an "intelligent" algorithm is used to derive from zero-crossing of discrete digit positions, if the number should have been increased. This is relevant because in some of the digit meters, the increase of a digit to the next number can be seen, before the sub-digit has gone through zero.
For example: 16.6 --> 16.7 --> 1N.8 --> **17.9** corrected to 16.9 --> 17.0 --> 17.1 For example: 16.6 --> 16.7 --> 1N.8 --> **17.9** corrected to 16.9 --> 17.0 --> 17.1
As you can see, the 17.9 is a false reading as the 7 is assumed to be already readable, although the sub-digit has not crossed the zero. In this case the CheckDigitIncreaseConsistency algorithm will correct this to 16.9 As you can see, the 17.9 is a false reading as the 7 is assumed to be already readable, although the sub-digit has not crossed the zero. In this case the CheckDigitIncreaseConsistency algorithm will correct this to 16.9
@@ -62,37 +60,32 @@ Here the maximum change from one to the next reading can be limited. If a false
1) **AbsolutChange**: Here the difference between the PreValue and the current reading is compared directly, independent how much time has passed since the last reading. 1) **AbsolutChange**: Here the difference between the PreValue and the current reading is compared directly, independent how much time has passed since the last reading.
2) **RelativeRate**: in this case a change rate in the unit of change/minute is calculated, taking the time between the last and the current reading into account. Be careful, that with increasing time, the absolute allowed change increases. 2) **RelativeRate**: in this case a change rate in the unit of change/minute is calculated, taking the time between the last and the current reading into account. Be careful, that with increasing time, the absolute allowed change increases.
Example: relative rate of 0.05 m³/minute --> after 20 minutes a maximum change of 20 minutes * 0.05 m³/minute = 1 m³ is possible. That means that a false reading of 1 m³ cannot be detected false after about 20 minutes in this case Example: relative rate of 0.05 m³/minute --> after 20 minutes a maximum change of 20 minutes * 0.05 m³/minute = 1 m³ is possible. That means that a false reading of 1 m³ cannot be detected false after about 20 minutes in this case
Assume, that there might be no change in the meter for hours (e.g. during the night) a much bigger change could also be accepted. Assume, that there might be no change in the meter for hours (e.g. during the night) a much bigger change could also be accepted.
###### `ExtendedResolution` ###### `ExtendedResolution`
Newer models such as dig-cont and dig-class100 have a high resolution of the values and can thus represent another digit by using the value of the last digit or pointer (ex. 7.8 in the last digit). Newer models such as dig-cont and dig-class100 have a high resolution of the values and can thus represent another digit by using the value of the last digit or pointer (ex. 7.8 in the last digit).
If the value is set to true, the result of the last digit is used completely. If the value is set to true, the result of the last digit is used completely.
When using dig-class11 models, the setting is ignored. When using dig-class11 models, the setting is ignored.
Due to inaccuracies of the neural networks, it sometimes happens that the results jump back and forth between two decimal places. Therefore, when using `ÀllowNegatives`= false, no error is output if the value is only off by 0.2. Nevertheless, the value then remains at the higher determined value. Due to inaccuracies of the neural networks, it sometimes happens that the results jump back and forth between two decimal places. Therefore, when using `ÀllowNegatives`= false, no error is output if the value is only off by 0.2. Nevertheless, the value then remains at the higher determined value.
###### `IgnoreLeadingNaN` ###### `IgnoreLeadingNaN`
The parameter is only be used, if a dig-class11 model is selected. `ÌgnoreLeadingNaN` removes in the CheckDigitIncreaseConsistency process the leading `N` values. The parameter is only be used, if a dig-class11 model is selected. `ÌgnoreLeadingNaN` removes in the CheckDigitIncreaseConsistency process the leading `N` values.
#### Flow Chart #### Flow Chart
![](img/correct_algo_1.jpg) ![Correction Algorithm 1](img/correct_algo_1.jpg)
![](img/correct_algo_2.jpg)
![](img/correct_algo_3.jpg)
![Correction Algorithm 2](img/correct_algo_2.jpg)
![Correction Algorithm 3](img/correct_algo_3.jpg)
## CheckDigitIncreaseConsistency Algorithm ## CheckDigitIncreaseConsistency Algorithm
The check digit increase consistency algorithm is functional for the digits only. Due to the fact, that the rotation might be a little bit earlier or later compared to the zero crossing of the digit before, errors during the reading before and after a zero crossing can be wrong. Therefore a simple algorithm can be applied, checking the consistency of zero crossing and changes in the following digit. This is applied to one after the other digit, starting with the lowest priority digits. The check digit increase consistency algorithm is functional for the digits only. Due to the fact, that the rotation might be a little bit earlier or later compared to the zero crossing of the digit before, errors during the reading before and after a zero crossing can be wrong. Therefore a simple algorithm can be applied, checking the consistency of zero crossing and changes in the following digit. This is applied to one after the other digit, starting with the lowest priority digits.
![](img/correct_algo_zero_crossing.jpg) ![Correction Algorithm Zero Crossing](img/correct_algo_zero_crossing.jpg)

21
docs/Demo-Mode.md
View File

@@ -1,4 +1,5 @@
# Demo Mode # Demo Mode
For Demo and Testing Purpose, the device can use pre-recorded raw images. For Demo and Testing Purpose, the device can use pre-recorded raw images.
You need to enable it in the configuration (`TakeImage > Demo`) and also provide the needed files on the SD card. You need to enable it in the configuration (`TakeImage > Demo`) and also provide the needed files on the SD card.
@@ -10,6 +11,7 @@ For the reference image and the alignment also the first image gets used.
Once the last image got reached, it starts again with the first one. Once the last image got reached, it starts again with the first one.
## SD Card Structure ## SD Card Structure
``` ```
demo/ demo/
├── 520.8983.jpg ├── 520.8983.jpg
@@ -23,11 +25,14 @@ demo/
- The jpg files must be smaller than 30'000 bytes - The jpg files must be smaller than 30'000 bytes
- The `files.txt` must contains a list of those files, eg: - The `files.txt` must contains a list of those files, eg:
520.8983.jpg ```
520.9086.jpg 520.8983.jpg
520.9351.jpg 520.9086.jpg
520.9351.jpg
```
## Collecting images of your device ## Collecting images of your device
There are several ways to collect images from your device: There are several ways to collect images from your device:
1. Use the [Parameter `RawImagesLocation`](../Parameters/#parameter-rawimageslocation) to store them directly onto your SD card. 1. Use the [Parameter `RawImagesLocation`](../Parameters/#parameter-rawimageslocation) to store them directly onto your SD card.
@@ -64,12 +69,15 @@ There are several ways to collect images from your device:
``` ```
## Use prepared images ## Use prepared images
Instead of recoding your own images, you can use one of the selections of prepared images below. Instead of recoding your own images, you can use one of the selections of prepared images below.
## How does it work ## How does it work
The Demo Mode tries to interfere as less as possible with the normal behavior. Whenever a Cam framebuffer gets taken (`esp_camera_fb_get()`), it replaces the framebuffer with the image from the SD card. The Demo Mode tries to interfere as less as possible with the normal behavior. Whenever a Cam framebuffer gets taken (`esp_camera_fb_get()`), it replaces the framebuffer with the image from the SD card.
## Example Data of a Water Meter ## Example Data of a Water Meter
You can use the following demo images if you want: You can use the following demo images if you want:
![530 00688](https://user-images.githubusercontent.com/1783586/211902363-1b8e4115-5f08-4e25-ace6-bb52e43b3741.jpg){:style="width:400px"} ![530 00688](https://user-images.githubusercontent.com/1783586/211902363-1b8e4115-5f08-4e25-ace6-bb52e43b3741.jpg){:style="width:400px"}
@@ -77,22 +85,23 @@ You can use the following demo images if you want:
It covers a meter range from `530.00688` to `531.85882`. It covers a meter range from `530.00688` to `531.85882`.
### Animation ### Animation
[Animation of the watermeter (77 MB!)](img/demo-watermeter-animated.png) [Animation of the watermeter (77 MB!)](img/demo-watermeter-animated.png)
### Selection of 84 images ### Selection of 84 images
[![grafik](https://user-images.githubusercontent.com/1783586/211915870-aa5c1342-c61c-4e1f-afe0-10e222f1499d.png){:style="width:400px"}](https://user-images.githubusercontent.com/1783586/211915870-aa5c1342-c61c-4e1f-afe0-10e222f1499d.png) [![grafik](https://user-images.githubusercontent.com/1783586/211915870-aa5c1342-c61c-4e1f-afe0-10e222f1499d.png){:style="width:400px"}](https://user-images.githubusercontent.com/1783586/211915870-aa5c1342-c61c-4e1f-afe0-10e222f1499d.png)
[Demo_Images_Watermeter_530.00688-532.08243_84_images.zip](https://github.com/jomjol/AI-on-the-edge-device-docs/files/10395892/Demo_Images_Watermeter_530.00688-532.08243_84_images.zip) [Demo_Images_Watermeter_530.00688-532.08243_84_images.zip](https://github.com/jomjol/AI-on-the-edge-device-docs/files/10395892/Demo_Images_Watermeter_530.00688-532.08243_84_images.zip)
### Selection of 42 images ### Selection of 42 images
[![grafik](https://user-images.githubusercontent.com/1783586/211915898-b499e109-7b63-4e21-ba5b-c0a370022f7a.png){:style="width:400px"}](https://user-images.githubusercontent.com/1783586/211915898-b499e109-7b63-4e21-ba5b-c0a370022f7a.png) [![grafik](https://user-images.githubusercontent.com/1783586/211915898-b499e109-7b63-4e21-ba5b-c0a370022f7a.png){:style="width:400px"}](https://user-images.githubusercontent.com/1783586/211915898-b499e109-7b63-4e21-ba5b-c0a370022f7a.png)
[Demo_Images_Watermeter_530.00688-532.08243_42_images.zip](https://github.com/jomjol/AI-on-the-edge-device-docs/files/10395893/Demo_Images_Watermeter_530.00688-532.08243_42_images.zip) [Demo_Images_Watermeter_530.00688-532.08243_42_images.zip](https://github.com/jomjol/AI-on-the-edge-device-docs/files/10395893/Demo_Images_Watermeter_530.00688-532.08243_42_images.zip)
### All images (843 images) ### All images (843 images)
[![grafik](https://user-images.githubusercontent.com/1783586/211915731-9a2a3cd3-390b-4b1f-a064-5e7e443ab113.png){:style="width:400px"}](https://user-images.githubusercontent.com/1783586/211915731-9a2a3cd3-390b-4b1f-a064-5e7e443ab113.png) [![grafik](https://user-images.githubusercontent.com/1783586/211915731-9a2a3cd3-390b-4b1f-a064-5e7e443ab113.png){:style="width:400px"}](https://user-images.githubusercontent.com/1783586/211915731-9a2a3cd3-390b-4b1f-a064-5e7e443ab113.png)
[Demo_Images_Watermeter_530.00688-532.08243_843_images.zip](https://github.com/jomjol/AI-on-the-edge-device-docs/files/10395941/Demo_Images_Watermeter_530.00688-532.08243_843_images.zip) [Demo_Images_Watermeter_530.00688-532.08243_843_images.zip](https://github.com/jomjol/AI-on-the-edge-device-docs/files/10395941/Demo_Images_Watermeter_530.00688-532.08243_843_images.zip)

50
docs/Error-Codes.md
View File

@@ -8,63 +8,70 @@ The error code(s) get printed with specific error codes. This page lists the pos
Here the error codes are defined in source code: [error codes](https://github.com/jomjol/AI-on-the-edge-device/blob/main/code/components/jomjol_helper/Helper.h). Here the error codes are defined in source code: [error codes](https://github.com/jomjol/AI-on-the-edge-device/blob/main/code/components/jomjol_helper/Helper.h).
## Critical Errors ## Critical Errors
Those Errors make the normal operation of the device impossible. Those Errors make the normal operation of the device impossible.
Most likely they are caused by a hardware issue! Most likely they are caused by a hardware issue!
### `0x00000001` PSRAM bad ### `0x00000001` PSRAM bad
Your device most likely has no PSRAM at all or it is too small (needs to have at least 4 MBytes)! Your device most likely has no PSRAM at all or it is too small (needs to have at least 4 MBytes)!
See [Hardware Compatibility](Hardware-Compatibility.md). See [Hardware Compatibility](Hardware-Compatibility.md).
Usually the log shows something like this: Usually the log shows something like this:
``` ```
psram: PSRAM ID read error: 0xffffffff psram: PSRAM ID read error: 0xffffffff
cpu_start: Failed to init external RAM! cpu_start: Failed to init external RAM!
``` ```
### `0x00000002` Heap too small ### `0x00000002` Heap too small
The firmware failed to allocate enough memory. This most likely is a consequential error of a bad PSRAM! The firmware failed to allocate enough memory. This most likely is a consequential error of a bad PSRAM!
### `0x00000004` Cam bad ### `0x00000004` Cam bad
The attached camera can not be initialized. The attached camera can not be initialized.
This usually is because on of the following reasons: This usually is because on of the following reasons:
* The camera is not supported, see [Hardware Compatibility](Hardware-Compatibility.md) * The camera is not supported, see [Hardware Compatibility](Hardware-Compatibility.md)
* The camera is not attached properly -> Try to remove and attach it again. Make sure you move the black part enough into the socket! * The camera is not attached properly -> Try to remove and attach it again. Make sure you move the black part enough into the socket!
* The camera or the camera cable is damaged * The camera or the camera cable is damaged
### `0x00000008` SD card basic check failed ### `0x00000008` SD card basic check failed
One or more basic SD card checks failed. One or more basic SD card checks failed.
The following checks are performed during boot sequence: The following checks are performed during boot sequence:
* Write a file (sdcard/sdcheck.txt) to SD card with some generic text * Write a file (sdcard/sdcheck.txt) to SD card with some generic text
* Read the written file back * Read the written file back
* CRC verification * CRC verification
* Delete the file * Delete the file
Detailed error indication (write, rerad or delete error) can be derived from blinking code of red board status LED. Please refer to [StatusLED-BlinkCodes](https://jomjol.github.io/AI-on-the-edge-device-docs/StatusLED-BlinkCodes/) Detailed error indication (write, rerad or delete error) can be derived from blinking code of red board status LED. Please refer to [StatusLED-BlinkCodes](https://jomjol.github.io/AI-on-the-edge-device-docs/StatusLED-BlinkCodes/)
Recommendation: Reformat SD card and check again or try another SD card Recommendation: Reformat SD card and check again or try another SD card
### `0x00000010` SD folder or file presence check failed ### `0x00000010` SD folder or file presence check failed
One or more mandatory folders and/or files are missing on SD card. One or more mandatory folders and/or files are missing on SD card.
To have early indication that SD card is potentially ready for operation, some folder and files, which are mandatory are presence checked. This is not a 100% check and a successful test does not mean everthing is OK. To have early indication that SD card is potentially ready for operation, some folder and files, which are mandatory are presence checked. This is not a 100% check and a successful test does not mean everthing is OK.
The following folders / files get checked during boot sequence: The following folders / files get checked during boot sequence:
* /sdcard/config * /sdcard/config
* /sdcard/html * /sdcard/html
* /sdcard/demo --> created automatically in firmware * /sdcard/demo --> created automatically in firmware
* /sdcard/firmware --> created automatically in firmware * /sdcard/firmware --> created automatically in firmware
* /sdcard/img_tmp --> created automatically in firmware * /sdcard/img_tmp --> created automatically in firmware
* /sdcard/log --> created automatically in firmware * /sdcard/log --> created automatically in firmware
* /sdcard/wlan.ini * /sdcard/wlan.ini
* /sdcard/config/config.ini * /sdcard/config/config.ini
* /sdcard/html/index.html * /sdcard/html/index.html
* /sdcard/html/ota_page.html * /sdcard/html/ota_page.html
* /sdcard/html/log.html * /sdcard/html/log.html
* /sdcard/html/common.js * /sdcard/html/common.js
* /sdcard/html/version.txt * /sdcard/html/version.txt
!!! Notes !!! Notes
This list might be outdated, see the source code for the latest implementation: [SDCardCheckRW()](https://github.com/jomjol/AI-on-the-edge-device/blob/main/code/components/jomjol_helper/sdcard_check.cpp#L14) This list might be outdated, see the source code for the latest implementation: [SDCardCheckRW()](https://github.com/jomjol/AI-on-the-edge-device/blob/main/code/components/jomjol_helper/sdcard_check.cpp#L14)
@@ -72,9 +79,11 @@ The following folders / files get checked during boot sequence:
Recommendation: Check logs and / or redo a Over-The-Air Update (OTA Update) to ensure proper SD card structure Recommendation: Check logs and / or redo a Over-The-Air Update (OTA Update) to ensure proper SD card structure
## Non-Critical Errors ## Non-Critical Errors
Those Errors can be caused by an error during initialization. It is possible that the error has no impact at all or that a reboot solves it. Those Errors can be caused by an error during initialization. It is possible that the error has no impact at all or that a reboot solves it.
### `0x00000100` Cam Framebuffer bad ### `0x00000100` Cam Framebuffer bad
The firmware was unable to initialize the Camera Framebuffer. The firmware was unable to initialize the Camera Framebuffer.
The firmware will continue to work, but other consequential error might arise. The firmware will continue to work, but other consequential error might arise.
A reboot of the device might help. A reboot of the device might help.
@@ -82,4 +91,5 @@ A reboot of the device might help.
This might also be caused by a corrupred SD-Card, see [CAM is not working anymore" on init #2390](https://github.com/jomjol/AI-on-the-edge-device/discussions/2390#discussioncomment-6430819) This might also be caused by a corrupred SD-Card, see [CAM is not working anymore" on init #2390](https://github.com/jomjol/AI-on-the-edge-device/discussions/2390#discussioncomment-6430819)
### `0x00000200` NTP failed ### `0x00000200` NTP failed
The firmware failed to get the world time from an NTP server. The firmware will continue to work, but has a wrong time. The firmware failed to get the world time from an NTP server. The firmware will continue to work, but has a wrong time.

16
docs/Error-Debugging.md
View File

@@ -1,10 +1,11 @@
# Often observed issues # Often observed issues
## Hardware failure ## Hardware failure
* Camera not working --> check the interface, test another module
* Low cost module with no or only 2MB memory instead of 4MB --> test another module * Camera not working --> check the interface, test another module
* SD card issues --> test another SD card * Low cost module with no or only 2MB memory instead of 4MB --> test another module
* Wifi reception bad / unstable --> bad antenna, test another module or use external antenna * SD card issues --> test another SD card
* Wifi reception bad / unstable --> bad antenna, test another module or use external antenna
More information in terms of hardware, component and basic configuration issues can be found here: [Reboot reasons](https://jomjol.github.io/AI-on-the-edge-device-docs/Frequent-Reboots/) More information in terms of hardware, component and basic configuration issues can be found here: [Reboot reasons](https://jomjol.github.io/AI-on-the-edge-device-docs/Frequent-Reboots/)
@@ -19,18 +20,15 @@ This typically happens if you have suboptimal "Alignment Marks". A very simple a
If after those adjustment you still have some issues, you can try to adjust your alignment settings in expert mode: If after those adjustment you still have some issues, you can try to adjust your alignment settings in expert mode:
<img src="https://user-images.githubusercontent.com/108122193/188382213-68c4a015-6582-4911-81bc-cdce8ef60ed2.png" width=75% height=75%> <img src="https://user-images.githubusercontent.com/108122193/188382213-68c4a015-6582-4911-81bc-cdce8ef60ed2.png" width=75% height=75%>
## My Analog Meter are recognized as Digit Counter or vice versa
## My Analog Meter are recognized as Digit Counter or vice versa
<img src="https://user-images.githubusercontent.com/108122193/188265470-001a392f-d1f4-46a3-b1e8-f29ec41c8621.png" width=40% height=40%> <img src="https://user-images.githubusercontent.com/108122193/188265470-001a392f-d1f4-46a3-b1e8-f29ec41c8621.png" width=40% height=40%>
1. First, check that your ROI are correctly defined (yey!) 1. First, check that your ROI are correctly defined (yey!)
2. Second, verify that the name of your ROI analog and digit ROIs are different 2. Second, verify that the name of your ROI analog and digit ROIs are different
## Recognition is working well, but number aren't sorted correctly ## Recognition is working well, but number aren't sorted correctly
You have to sort your ROI correctly (Bigger to smaller). Select your ROI and click either "move next" or "move previous". Repeat until your ROI are correctly sorted You have to sort your ROI correctly (Bigger to smaller). Select your ROI and click either "move next" or "move previous". Repeat until your ROI are correctly sorted
<img src="https://user-images.githubusercontent.com/108122193/188264916-03befff1-4e61-4370-bd5a-9168a88c57f2.png" width=50% height=50%> <img src="https://user-images.githubusercontent.com/108122193/188264916-03befff1-4e61-4370-bd5a-9168a88c57f2.png" width=50% height=50%>

29
docs/External-LED.md
View File

@@ -5,39 +5,22 @@ To circumvent this problem, it is now possible to control external LEDs, which t
<img src="https://raw.githubusercontent.com/jomjol/ai-on-the-edge-device/master/images/intern_vs_external.jpg" width="700"> <img src="https://raw.githubusercontent.com/jomjol/ai-on-the-edge-device/master/images/intern_vs_external.jpg" width="700">
There is also a new [meter adapter](https://www.thingiverse.com/thing:5028229) available. This has two features: designed for **small clearings** in front of the meter and prepared for **WS2812 LEDs**. There is also a new [meter adapter](https://www.thingiverse.com/thing:5028229) available. This has two features: designed for **small clearings** in front of the meter and prepared for **WS2812 LEDs**.
<img src="https://raw.githubusercontent.com/jomjol/ai-on-the-edge-device/master/images/Power_Meter_Mounted.jpg" width="500"> <img src="https://raw.githubusercontent.com/jomjol/ai-on-the-edge-device/master/images/Power_Meter_Mounted.jpg" width="500">
## 1. Hardware installation of the LED stripe ## 1. Hardware installation of the LED stripe
The control line of the LED stripe is connected with a 470 Ohm resistor to the GPIO12. The control line of the LED stripe is connected with a 470 Ohm resistor to the GPIO12.
For power supply stabilization a capacitor between 5V and ground is recommended. Here a 470µF polymer capacitor is used. As a power supply a 5V from the ESP32 is used like in the following wiring. For power supply stabilization a capacitor between 5V and ground is recommended. Here a 470µF polymer capacitor is used. As a power supply a 5V from the ESP32 is used like in the following wiring.
<img src="https://raw.githubusercontent.com/jomjol/ai-on-the-edge-device/master/images/install_external_led.jpg" width="500">
<img src="https://raw.githubusercontent.com/jomjol/ai-on-the-edge-device/master/images/install_external_led.jpg" width="500">
## 2. Software configuration ## 2. Software configuration
The handling of the WS2812 LED controller needs some other libraries, therefore it is controlled within a dedicated section called `GPIO Settings`. The external LED stripe is connected to GPIO12. After activating the "GPIO Settings" section, the internal flash is per default disabled. In order to activate the external LED, you need to activate `GPIO 12 state` and select `"extern flash light ws281x ..."`. The handling of the WS2812 LED controller needs some other libraries, therefore it is controlled within a dedicated section called `GPIO Settings`. The external LED stripe is connected to GPIO12. After activating the "GPIO Settings" section, the internal flash is per default disabled. In order to activate the external LED, you need to activate `GPIO 12 state` and select `"extern flash light ws281x ..."`.
<img src="https://raw.githubusercontent.com/jomjol/ai-on-the-edge-device/master/images/external_GPIO_settings.jpg" width="700">
<img src="https://raw.githubusercontent.com/jomjol/ai-on-the-edge-device/master/images/external_GPIO_settings.jpg" width="700">
| Parameter | Meaning | | Parameter | Meaning |
| -------------- | ------------------------------------------------------------ | | -------------- | ------------------------------------------------------------ |
@@ -45,6 +28,4 @@ The handling of the WS2812 LED controller needs some other libraries, therefore
| Numbers of LED | Number of LEDs on the LED stripe | | Numbers of LED | Number of LEDs on the LED stripe |
| LED Color | The color and intensity can be controlled directly by a red/green/blue value, each within the range from 0 (off) to 255 (full) | | LED Color | The color and intensity can be controlled directly by a red/green/blue value, each within the range from 0 (off) to 255 (full) |
Enabling the GPIO settings automatically disables the flash LED. Therefore you can enable it here manually by checking GPIO4 and choose `"build-in led flash light"`. It is not recommended to use both illumination parallel.
Enabling the GPIO settings automatically disables the flash LED. Therefore you can enable it here manually by checking GPIO4 and choose `"build-in led flash light"`. It is not recommended to use both illumination parallel.

34
docs/FAQs.md
View File

@@ -1,6 +1,7 @@
# Frequently Asked Questions # Frequently Asked Questions
## My device reboots frequently. What can I do? ## My device reboots frequently. What can I do?
There are several reasons for frequent reboots: There are several reasons for frequent reboots:
* Frequent HTML requests * Frequent HTML requests
@@ -9,15 +10,15 @@ There are several reasons for frequent reboots:
There is a dedicated page about this: [Frequent Reboots](Frequent-Reboots.md). There is a dedicated page about this: [Frequent Reboots](Frequent-Reboots.md).
## Bad WebUI responsiveness. What can I do? ## Bad WebUI responsiveness. What can I do?
This is usually due to hardware or WLAN issues. There are already many entries in discussion section, some of which have good tipps. This is usually due to hardware or WLAN issues. There are already many entries in discussion section, some of which have good tipps.
Possible checks / ideas: Possible checks / ideas:
* ESP32CAM hardware antenna design is very poor in connection with camera frequency. * ESP32CAM hardware antenna design is very poor in connection with camera frequency.
* Simple test: When the device is in operation, putting your thumb on the camera connector and the directly adjacent components should make the device respond more quickly. * Simple test: When the device is in operation, putting your thumb on the camera connector and the directly adjacent components should make the device respond more quickly.
* Possible optimization: Here, an attempt was made to dampen the frequency influences somewhat by shielding. * Possible optimization: Here, an attempt was made to dampen the frequency influences somewhat by shielding.
[Shielding Example](https://www.reddit.com/r/esp32/comments/r9g5jc/fixing_ymmv_the_poor_frame_rate_on_the_esp32cam/) [Shielding Example](https://www.reddit.com/r/esp32/comments/r9g5jc/fixing_ymmv_the_poor_frame_rate_on_the_esp32cam/)
* WLAN channel: Preferably use channel 1, 6 or 11 * WLAN channel: Preferably use channel 1, 6 or 11
* Performance can vary depending on the AP manufacturer. If necessary, check with a mobile hotspot or other device to exclude AP influence * Performance can vary depending on the AP manufacturer. If necessary, check with a mobile hotspot or other device to exclude AP influence
@@ -29,8 +30,8 @@ Possible checks / ideas:
Check [discussion section](https://github.com/jomjol/AI-on-the-edge-device/discussions) for possible further tipps. Check [discussion section](https://github.com/jomjol/AI-on-the-edge-device/discussions) for possible further tipps.
## How accurate are the detections? ## How accurate are the detections?
It is hard to give a specific accuracy number. It depends on many factors, e.g. It is hard to give a specific accuracy number. It depends on many factors, e.g.
* How in-focus is your camera? * How in-focus is your camera?
@@ -39,12 +40,12 @@ It is hard to give a specific accuracy number. It depends on many factors, e.g.
* Are you trying to read digits, an analog dial, or both? * Are you trying to read digits, an analog dial, or both?
* etc. * etc.
Anecdotally, the authors of this page have great success with the meter. While the AI algorithm itself is not perfect and sometimes returns `NaN` or incorrect values, other post-processing / prevalue / sanity checks help ensure such invalid values are filtered out. With the correct settings, one author has been running this device for 1 month without any incorrect values reported. Anecdotally, the authors of this page have great success with the meter. While the AI algorithm itself is not perfect and sometimes returns `NaN` or incorrect values, other post-processing / prevalue / sanity checks help ensure such invalid values are filtered out. With the correct settings, one author has been running this device for 1 month without any incorrect values reported.
See the FAQs below for more details and configuration hints. See the FAQs below for more details and configuration hints.
## My numbers are not correctly detected. What can I do? ## My numbers are not correctly detected. What can I do?
* There is a dedicated page about the correct setting [ROI Configuration](ROI-Configuration.md). * There is a dedicated page about the correct setting [ROI Configuration](ROI-Configuration.md).
* This page also includes the instructions for gathering new images for the training. * This page also includes the instructions for gathering new images for the training.
@@ -61,43 +62,48 @@ You can change the following settings to reduce incorrect readings (but potentia
* Reduce `AutoTimer` configuration option `Interval` to the lowest it can be (e.g. `3` min). The more often you take readings, the less likely for data staleness to occur. * Reduce `AutoTimer` configuration option `Interval` to the lowest it can be (e.g. `3` min). The more often you take readings, the less likely for data staleness to occur.
## Even after I have setup everything perfect there is a false reading - especially around the zero crossing (roll over to next number) ## Even after I have setup everything perfect there is a false reading - especially around the zero crossing (roll over to next number)
* The roll over behavior is different for the different meters. E.g.: * The roll over behavior is different for the different meters. E.g.:
* Rolling over start with different previous position (e.g. at 7, 8 or 9) * Rolling over start with different previous position (e.g. at 7, 8 or 9)
* The neutral position (no rolling) is not perfectly at zero, but rather at something like 7.9 or 8.1, even if it should be exactly 8 * The neutral position (no rolling) is not perfectly at zero, but rather at something like 7.9 or 8.1, even if it should be exactly 8
* The "PostProcessingAlgo" is trying to judge out of the individual readings, what number it should be. * The "PostProcessingAlgo" is trying to judge out of the individual readings, what number it should be.
* For example if the previous number is a "1", but the next number seems to be a "8.9", most probably there was a "zero crossing" and the number is a "9" and not still an "8" * For example if the previous number is a "1", but the next number seems to be a "8.9", most probably there was a "zero crossing" and the number is a "9" and not still an "8"
* Currently the setting of the algorithm is set to fit most of the meters and cases. But the parameters do not fit perfectly for all situations. Therefore there might be intermediate states, where the reading is false. * Currently the setting of the algorithm is set to fit most of the meters and cases. But the parameters do not fit perfectly for all situations. Therefore there might be intermediate states, where the reading is false.
This is especially the case, at the positions, where the roll over (zero crossing) is just starting. This is especially the case, at the positions, where the roll over (zero crossing) is just starting.
* To prevent a sending of false parameters, there is the possibility to limit the maximum allowed change (MaxRateChange). * To prevent a sending of false parameters, there is the possibility to limit the maximum allowed change (MaxRateChange).
Usually after some time and movement of the counters a bit further, the reading is getting back to a stable reading. Usually after some time and movement of the counters a bit further, the reading is getting back to a stable reading.
* To handle this, a parametrized setting would be needed. This is rather complicated to implement as subtle changes make a relevant difference. Currently this is not implemented. * To handle this, a parametrized setting would be needed. This is rather complicated to implement as subtle changes make a relevant difference. Currently this is not implemented.
So please be a bit patient with your meter :-) So please be a bit patient with your meter :-)
## Pre-Value ## Pre-Value
PreValue is here a bit missleading, because normally it is the same as the last value. In the next round of reading it will be used to check negative rates, high rates (MaxRateValue / MaxRateType) and CheckDigitIncreaseConsistency (dig-class11 only). Either from a previous correctly identified value or manual setting by the user. PreValue is here a bit missleading, because normally it is the same as the last value. In the next round of reading it will be used to check negative rates, high rates (MaxRateValue / MaxRateType) and CheckDigitIncreaseConsistency (dig-class11 only). Either from a previous correctly identified value or manual setting by the user.
If you use post processes, enable the pre-value. The pre-value must be set at first time. Set it to the current raw value. If you use post processes, enable the pre-value. The pre-value must be set at first time. Set it to the current raw value.
If the device runs in errors, the pre-value will not be updated, as long as the `preValueAgeStartup` time between the last valid value (or startup time) and current time is not exceeded. After it the preValue will be set again, if no other error occured. So the device can not run in an endless error, like high rate. If the device runs in errors, the pre-value will not be updated, as long as the `preValueAgeStartup` time between the last valid value (or startup time) and current time is not exceeded. After it the preValue will be set again, if no other error occured. So the device can not run in an endless error, like high rate.
## "Rate too high - Read: ..." ## "Rate too high - Read: ..."
In configuration you can set the `MaxRateValue` and `MaxRateType`. The settings suppress improbably high values that can come from false readings. To do this, the value must be set correctly depending on your meter. In configuration you can set the `MaxRateValue` and `MaxRateType`. The settings suppress improbably high values that can come from false readings. To do this, the value must be set correctly depending on your meter.
Before doing this, you should be clear about the type of rating you want to use. Before doing this, you should be clear about the type of rating you want to use.
- `Absolute change` is the interval between two readings - no matter how often the readings happen. * `Absolute change` is the interval between two readings - no matter how often the readings happen.
- `RateChange` is the change per minute. This is calculated from the time difference between the last and the current reading. * `RateChange` is the change per minute. This is calculated from the time difference between the last and the current reading.
If there is an interval of 5 minutes between readings and a MaxRateValue of 1, an error "Rate too high - Read: ..." if If there is an interval of 5 minutes between readings and a MaxRateValue of 1, an error "Rate too high - Read: ..." if
- Absolute change: the difference is `> 1` * Absolute change: the difference is `> 1`
- RateChange: the difference is `> 1 / 5` * RateChange: the difference is `> 1 / 5`
## Train on my own images ## Train on my own images
Look at [Learn models with your own images](https://jomjol.github.io/AI-on-the-edge-device-docs/Learn-models-with-your-own-images/) Look at [Learn models with your own images](https://jomjol.github.io/AI-on-the-edge-device-docs/Learn-models-with-your-own-images/)
and [Cookbook running the jupyter notebook with my own data](https://github.com/haverland/Tenth-of-step-of-a-meter-digit/wiki/Cookbook-running-the-jupyter-notebook-with-my-own-data). and [Cookbook running the jupyter notebook with my own data](https://github.com/haverland/Tenth-of-step-of-a-meter-digit/wiki/Cookbook-running-the-jupyter-notebook-with-my-own-data).
## What does AIOTED mean? ## What does AIOTED mean?
This is just the abbreviation of __AI-on-the-edge-device__. This is just the abbreviation of __AI-on-the-edge-device__.

51
docs/Frequent-Reboots.md
View File

@@ -1,22 +1,22 @@
# Basic hardware / configuration issues # Basic hardware / configuration issues
If the device is behaving eratically or not running as expected you can use the following tools trying to identify the root cause: If the device is behaving eratically or not running as expected you can use the following tools trying to identify the root cause:
1. Internal logging (`config.ini`) 1. Internal logging (`config.ini`)
--> Set to DEBUG log level --> Set to DEBUG log level
2. Reduced web interface (only error indication visualization, [Error codes on reduced webinterface](https://jomjol.github.io/AI-on-the-edge-device-docs/Error-Codes/)) 2. Reduced web interface (only error indication visualization, [Error codes on reduced webinterface](https://jomjol.github.io/AI-on-the-edge-device-docs/Error-Codes/))
2. Red board LED: [Status LED Blinkcodes](https://jomjol.github.io/AI-on-the-edge-device-docs/StatusLED-BlinkCodes/) 3. Red board LED: [Status LED Blinkcodes](https://jomjol.github.io/AI-on-the-edge-device-docs/StatusLED-BlinkCodes/)
3. Serial log of the UART interface (USB access needed, only local, same as for flashing the firmware) 4. Serial log of the UART interface (USB access needed, only local, same as for flashing the firmware)
There are in principle two reboots types: There are in principle two reboots types:
1. Sporadic random reboots (always different timing and situation) 1. Sporadic random reboots (always different timing and situation)
2. Repeating boot loops (reoccuring, always stop working after same precondition) 2. Repeating boot loops (reoccuring, always stop working after same precondition)
______ ______
### Sporadic random reboots ## Sporadic random reboots
Sporadic random reboots could have the following reasons: Sporadic random reboots could have the following reasons:
* In general: Unstable system due to software issues (e.g. overload during HTML access, ...) * In general: Unstable system due to software issues (e.g. overload during HTML access, ...)
@@ -26,12 +26,12 @@ Sporadic random reboots could have the following reasons:
In general: There are several mechanisms in the firmware (like saving previous values), to have a "smooth" reboot without too many notable disturbance. In general: There are several mechanisms in the firmware (like saving previous values), to have a "smooth" reboot without too many notable disturbance.
##### System instabilities ### System instabilities
If your system is sometimes running smoothly over several runs and sometimes reboots obviously randomly, you have an partially unstable device.
If your system is sometimes running smoothly over several runs and sometimes reboots obviously randomly, you have an partially unstable device.
You can check this in the standard log file on the SD card: You can check this in the standard log file on the SD card:
``` ```
2021-12-26T06:34:09: task_autodoFlow - round done 2021-12-26T06:34:09: task_autodoFlow - round done
2021-12-26T06:34:09: CPU Temperature: 56.1 2021-12-26T06:34:09: CPU Temperature: 56.1
@@ -41,9 +41,9 @@ You can check this in the standard log file on the SD card:
Here you see, that the round #23 is starting, so obviously there were no reboots in the last 22 rounds. There is hardware (ESP32CAM), where only 2-3 stable rounds are possible and others, where way more than 100 rounds without any reboots is possible. Here you see, that the round #23 is starting, so obviously there were no reboots in the last 22 rounds. There is hardware (ESP32CAM), where only 2-3 stable rounds are possible and others, where way more than 100 rounds without any reboots is possible.
There is noting you can do about it, beside testing different hardware. There is noting you can do about it, beside testing different hardware.
#### Overload during HTML access #### Overload during HTML access
If you frequently access the web server over HTML requests, the firmware tends to reboot. This especially happens during the first run and when the ESP32 is busy with the digitization flow.
If you frequently access the web server over HTML requests, the firmware tends to reboot. This especially happens during the first run and when the ESP32 is busy with the digitization flow.
The reason for this are running out of memory during a flow, minor memory leakage in combination with missing error handling. The reason for this are running out of memory during a flow, minor memory leakage in combination with missing error handling.
@@ -52,15 +52,15 @@ There is noting you can do about this kind of reboot, beside two thing:
1. Support the firmware development with improved and tested part of code 1. Support the firmware development with improved and tested part of code
2. Be patient :-) 2. Be patient :-)
##### Bad or insufficient power supply ##### Bad or insufficient power supply
A good and stabilized power supply is essential to have error free operation. The device is quite picky in terms of proper power supply. Especially the wifi module have some load spikes which the power supply needs to cover. If the power is not stable enough, the brwonout mechanism is protecting against strange behaviour and force a reboot whenever the voltage drops below a specific level. You can see this in random reboots which indication is logfile: --> Reset reason: Brownout A good and stabilized power supply is essential to have error free operation. The device is quite picky in terms of proper power supply. Especially the wifi module have some load spikes which the power supply needs to cover. If the power is not stable enough, the brwonout mechanism is protecting against strange behaviour and force a reboot whenever the voltage drops below a specific level. You can see this in random reboots which indication is logfile: --> Reset reason: Brownout
### Repeating boot loops ### Repeating boot loops
Repeating boot loops at the same situation during the flow has a systematic problem either in the hardware or the configuration. It usually happens during initialization state or processing the first round as there all needed parts of the firmware have been loaded for the first time. Repeating boot loops at the same situation during the flow has a systematic problem either in the hardware or the configuration. It usually happens during initialization state or processing the first round as there all needed parts of the firmware have been loaded for the first time.
To identify the root cause the logfiles, the reduced web interface, the red board LED or the serial log of the UART interface (no remote access, USB access needed) is helpful. To identify the root cause the logfiles, the reduced web interface, the red board LED or the serial log of the UART interface (no remote access, USB access needed) is helpful.
Possible issues: Possible issues:
@@ -68,25 +68,26 @@ Possible issues:
* RAM related issues * RAM related issues
* Configuration related issues * Configuration related issues
##### SD card related issues #### SD card related issues
The ESP32CAM is a little bit "picky" with the supported SD cards. Due to the limited availability of GPIOs the SD card can only be accessed via 1-wire mode. Therefore not all SD cards are supported. The following error cases can occur: The ESP32CAM is a little bit "picky" with the supported SD cards. Due to the limited availability of GPIOs the SD card can only be accessed via 1-wire mode. Therefore not all SD cards are supported. The following error cases can occur:
##### SD card: Wrong filesystem (only FAT32 is supported)
###### SD card: Wrong filesystem (only FAT32 is supported)
If this SD card error is detected only the following indications are available. No web interface will be accessible. If this SD card error is detected only the following indications are available. No web interface will be accessible.
* Red board LED is blinking. The blinking codes are described here: [Status LED Blinkcodes](https://jomjol.github.io/AI-on-the-edge-device-docs/StatusLED-BlinkCodes/) * Red board LED is blinking. The blinking codes are described here: [Status LED Blinkcodes](https://jomjol.github.io/AI-on-the-edge-device-docs/StatusLED-BlinkCodes/)
* Error messages on serial log (UART interface) * Error messages on serial log (UART interface)
###### SD card not detected / not supported ###### SD card not detected / not supported
If this SD card error is detected the following indication are available. No web interface will be accessible. If this SD card error is detected the following indication are available. No web interface will be accessible.
* Red board LED is blinking. The blinking codes are described here: [Status LED Blinkcodes](https://jomjol.github.io/AI-on-the-edge-device-docs/StatusLED-BlinkCodes/) * Red board LED is blinking. The blinking codes are described here: [Status LED Blinkcodes](https://jomjol.github.io/AI-on-the-edge-device-docs/StatusLED-BlinkCodes/)
* Error messages on serial log (UART interface) * Error messages on serial log (UART interface)
###### SD card detected but files are not readable / writeable ###### SD card detected but files are not readable / writeable
The SD card is detected, but the files cannot be read or written. A basic SD card check for SD reading / writing is performed on every boot. This not 100% guarantee that SD card is working but it's at least a indication. The SD card is detected, but the files cannot be read or written. A basic SD card check for SD reading / writing is performed on every boot. This not 100% guarantee that SD card is working but it's at least a indication.
If this SD card error is detected the following indications are available: If this SD card error is detected the following indications are available:
@@ -97,11 +98,12 @@ If this SD card error is detected the following indications are available:
* Error messages on serial log (UART interface) * Error messages on serial log (UART interface)
##### RAM related issues ##### RAM related issues
In order to run the firmware, 4 MB of external RAM (PSRAM) are mandatory. Usually, the ESP32CAM is equipped with 8MB (64Mbit) PSRAM chip, whereof only 4MB can be used effectively (direct addressable).
Unfortunately, there is hardware around, where no PSRAM or only 2MB of PSRAM is present - **even if you have bought a device where a 8MB PSRAM was promoted**. These modules are not suiable for this firmware because the external RAM is needed to handle the CNN files and camera images. There is nothing to do, than to buy a new ESP32CAM with **really** 64MBit of PSRAM.
In order to run the firmware, 4 MB of external RAM (PSRAM) are mandatory. Usually, the ESP32CAM is equipped with 8MB (64Mbit) PSRAM chip, whereof only 4MB can be used effectively (direct addressable).
Unfortunately, there is hardware around, where no PSRAM or only 2MB of PSRAM is present - **even if you have bought a device where a 8MB PSRAM was promoted**. These modules are not suiable for this firmware because the external RAM is needed to handle the CNN files and camera images. There is nothing to do, than to buy a new ESP32CAM with **really** 64MBit of PSRAM.
###### Too less external RAM (PSRAM) ###### Too less external RAM (PSRAM)
During the boot process the available RAM is going to be checked. During the boot process the available RAM is going to be checked.
If there is too less RAM (PSRAM or total HEAP < 4MB) detected, the follwoing indications are available: If there is too less RAM (PSRAM or total HEAP < 4MB) detected, the follwoing indications are available:
@@ -112,7 +114,9 @@ If there is too less RAM (PSRAM or total HEAP < 4MB) detected, the follwoing ind
* Error messages on serial log (UART interface) * Error messages on serial log (UART interface)
##### Configuration related issues ##### Configuration related issues
###### Folders and files missing ###### Folders and files missing
Most of the relevant folders and files are checked during boot. The complete list can be found here: [Error codes on reduced webinterface](https://jomjol.github.io/AI-on-the-edge-device-docs/Error-Codes/) Most of the relevant folders and files are checked during boot. The complete list can be found here: [Error codes on reduced webinterface](https://jomjol.github.io/AI-on-the-edge-device-docs/Error-Codes/)
If a relevant folder or file is missing the following indications are available: If a relevant folder or file is missing the following indications are available:
@@ -122,8 +126,8 @@ If a relevant folder or file is missing the following indications are available:
* Red board LED is blinking. The blinking codes are described here: [Status LED Blinkcodes](https://jomjol.github.io/AI-on-the-edge-device-docs/StatusLED-BlinkCodes/) * Red board LED is blinking. The blinking codes are described here: [Status LED Blinkcodes](https://jomjol.github.io/AI-on-the-edge-device-docs/StatusLED-BlinkCodes/)
* Error messages on serial log (UART interface) * Error messages on serial log (UART interface)
###### CNN model file not available / corrupt ###### CNN model file not available / corrupt
Additionally for operation CNN model files on SD card are mandatory, one CNN model file for analog counter and for for digit numbers each. Additionally for operation CNN model files on SD card are mandatory, one CNN model file for analog counter and for for digit numbers each.
* `/config/XXXXX.tflite` (XXXXX is the file name, that is written in the `config.ini`) * `/config/XXXXX.tflite` (XXXXX is the file name, that is written in the `config.ini`)
@@ -131,6 +135,7 @@ Additionally for operation CNN model files on SD card are mandatory, one CNN mod
If the files which are configured in `config.ini` are not present or corrupt, the process is going to be interrupted (or at worst case a device crash occurs). Please check logs files to have an indicator for the root cause. If the files which are configured in `config.ini` are not present or corrupt, the process is going to be interrupted (or at worst case a device crash occurs). Please check logs files to have an indicator for the root cause.
This a logfile extract (DEBUB log level) where digit CNN model file is not present. The system is initializing the system and trying to load the model files: This a logfile extract (DEBUB log level) where digit CNN model file is not present. The system is initializing the system and trying to load the model files:
``` ```
[0d00h05m11s] 2023-03-27T12:25:14 [TFLITE] CTfLiteClass::LoadModel [0d00h05m11s] 2023-03-27T12:25:14 [TFLITE] CTfLiteClass::LoadModel
[0d00h05m11s] 2023-03-27T12:25:14 [TFLITE] CTfLiteClass::ReadFileToModel: /sdcard [0d00h05m11s] 2023-03-27T12:25:14 [TFLITE] CTfLiteClass::ReadFileToModel: /sdcard
@@ -151,9 +156,9 @@ This a logfile extract (DEBUB log level) where digit CNN model file is not prese
``` ```
* Bad config example: * Bad config example:
- `[0d00h05m11s] 2023-03-27T12:25:14 [TFLITE] CTfLiteClass::ReadFileToModel: /sdcard` * `[0d00h05m11s] 2023-03-27T12:25:14 [TFLITE] CTfLiteClass::ReadFileToModel: /sdcard`
--> model file missing: check configuration or file presence --> model file missing: check configuration or file presence
* Good config example: * Good config example:
- `[0d00h05m12s] 2023-03-27T12:25:14 [TFLITE] CTfLiteClass::ReadFileToModel: /sdcard/config/ana-cont_1105_s2_q.tflite` * `[0d00h05m12s] 2023-03-27T12:25:14 [TFLITE] CTfLiteClass::ReadFileToModel: /sdcard/config/ana-cont_1105_s2_q.tflite`
--> model file found: config OK --> model file found: config OK

15
docs/Hardware-Compatibility.md
View File

@@ -1,6 +1,7 @@
# Hardware Compatibility # Hardware Compatibility
## General Remark ## General Remark
Although a board looks similar, it can have major differences, e.g.: Although a board looks similar, it can have major differences, e.g.:
- Processor - Processor
@@ -30,7 +31,6 @@ Below you find some remarks and experiences from the community:
There seems to be a lot of "fake" chips, or maybe wrongly configured ESP32 Boards. There seems to be a lot of "fake" chips, or maybe wrongly configured ESP32 Boards.
For AP MEMORY, all "real" APS**64**04\*3SQR chips should work. For AP MEMORY, all "real" APS**64**04\*3SQR chips should work.
For ESP PSRAM, all "real" PSRAM**64**\* should work. For ESP PSRAM, all "real" PSRAM**64**\* should work.
@@ -55,7 +55,7 @@ This Table is just a snapshot of chips which worked
| AP MEMORY / 1604M-3SQR / 0280A / 070036G| | :x: 2MB only! | | AP MEMORY / 1604M-3SQR / 0280A / 070036G| | :x: 2MB only! |
| ESP PSRAM**64**H 462021 / 1B00286 | | :heavy_check_mark: | | ESP PSRAM**64**H 462021 / 1B00286 | | :heavy_check_mark: |
| ESP PSRAM**64**H 412021 / 1A0039G | | :heavy_check_mark: 8MB | | ESP PSRAM**64**H 412021 / 1A0039G | | :heavy_check_mark: 8MB |
| ESP PSRAM**64**H 402021 / 1A0017N | | :x: PSRAM not accessible | | ESP PSRAM**64**H 402021 / 1A0017N | | :x: PSRAM not accessible |
| ESP PSRAM16M 302020 | | :x: 2MB only! | | ESP PSRAM16M 302020 | | :x: 2MB only! |
| ESP PSRAM16H 202020 / 050022G | | :x: 2MB only! | | ESP PSRAM16H 202020 / 050022G | | :x: 2MB only! |
@@ -69,8 +69,6 @@ The experience with the camera only is based on single modules. It is well possi
| DCX-OV2 / 640-V2 | | :heavy_check_mark: | | DCX-OV2 / 640-V2 | | :heavy_check_mark: |
| DC-26 / 40-V3 | | :heavy_check_mark: 3x<br> :x: 1x | | DC-26 / 40-V3 | | :heavy_check_mark: 3x<br> :x: 1x |
# ESP32 Modules # ESP32 Modules
| Module | Image | Status | | Module | Image | Status |
@@ -78,32 +76,29 @@ The experience with the camera only is based on single modules. It is well possi
| ESP32CAM / Different versions on the market!<br>Especially the PSRAM is sometimes labeled wrong<br>(Label: 4MB, Real: only 2 MB --> will not work!) | | :heavy_check_mark:<br />with >=4 MB PSRAM! | | ESP32CAM / Different versions on the market!<br>Especially the PSRAM is sometimes labeled wrong<br>(Label: 4MB, Real: only 2 MB --> will not work!) | | :heavy_check_mark:<br />with >=4 MB PSRAM! |
| ESP32-S3-EYE<br />No Flash LED, pins different used (e.g. LCD display) | | **NOT OKAY** | | ESP32-S3-EYE<br />No Flash LED, pins different used (e.g. LCD display) | | **NOT OKAY** |
# SD Cards # SD Cards
Due to the limited free available GPIOs (due to all the extensions needed like: camera, SD card, LED-flash, ...) the SD card is connected in 1-wire mode. There are some cards, that are compatible with the esp32cam module for unknown reasons. Due to the limited free available GPIOs (due to all the extensions needed like: camera, SD card, LED-flash, ...) the SD card is connected in 1-wire mode. There are some cards, that are compatible with the esp32cam module for unknown reasons.
It is observed, that smaller cards (up to 4 GB) tend to be more stable and larger cards have more problems. But quite some exceptions in the forums (4 GB cards not working, 16 GB cards working like a charm). It is observed, that smaller cards (up to 4 GB) tend to be more stable and larger cards have more problems. But quite some exceptions in the forums (4 GB cards not working, 16 GB cards working like a charm).
# Devices known to work # Devices known to work
## Modules (Old list, not up-to-date anymore): ## Modules (Old list, not up-to-date anymore):
See [https://github.com/jomjol/AI-on-the-edge-device/discussions/1732](https://github.com/jomjol/AI-on-the-edge-device/discussions/1732) for a more recent list. See [https://github.com/jomjol/AI-on-the-edge-device/discussions/1732](https://github.com/jomjol/AI-on-the-edge-device/discussions/1732) for a more recent list.
- [https://arduino-projekte.info/produkt/esp32-cam-v2-integriertem-ch340-mit-ov2640-kamera-modul/](https://arduino-projekte.info/produkt/esp32-cam-v2-integriertem-ch340-mit-ov2640-kamera-modul/) (see [https://github.com/jomjol/AI-on-the-edge-device/discussions/1041](https://github.com/jomjol/AI-on-the-edge-device/discussions/1041)) - [https://arduino-projekte.info/produkt/esp32-cam-v2-integriertem-ch340-mit-ov2640-kamera-modul/](https://arduino-projekte.info/produkt/esp32-cam-v2-integriertem-ch340-mit-ov2640-kamera-modul/) (see [https://github.com/jomjol/AI-on-the-edge-device/discussions/1041](https://github.com/jomjol/AI-on-the-edge-device/discussions/1041))
- [https://www.amazon.de/-/en/gp/product/B0B51CQ13R](https://www.amazon.de/-/en/gp/product/B0B51CQ13R) - [https://www.amazon.de/-/en/gp/product/B0B51CQ13R](https://www.amazon.de/-/en/gp/product/B0B51CQ13R)
- [https://www.reichelt.de/entwicklerboards-esp32-kamera-2mp-25--debo-cam-esp32-p266036.html?PROVID=2788&gclid=CjwKCAiAqaWdBhAvEiwAGAQlttJnV4azXWDYeaFUuNioMICh-jvxKp6Cifmcep9vvtoT2JRCDqBczRoC7Q0QAvD_BwE](https://www.reichelt.de/entwicklerboards-esp32-kamera-2mp-25--debo-cam-esp32-p266036.html?PROVID=2788&gclid=CjwKCAiAqaWdBhAvEiwAGAQlttJnV4azXWDYeaFUuNioMICh-jvxKp6Cifmcep9vvtoT2JRCDqBczRoC7Q0QAvD_BwE) (27.12.2022) - [https://www.reichelt.de/entwicklerboards-esp32-kamera-2mp-25--debo-cam-esp32-p266036.html?PROVID=2788&gclid=CjwKCAiAqaWdBhAvEiwAGAQlttJnV4azXWDYeaFUuNioMICh-jvxKp6Cifmcep9vvtoT2JRCDqBczRoC7Q0QAvD_BwE](https://www.reichelt.de/entwicklerboards-esp32-kamera-2mp-25--debo-cam-esp32-p266036.html?PROVID=2788&gclid=CjwKCAiAqaWdBhAvEiwAGAQlttJnV4azXWDYeaFUuNioMICh-jvxKp6Cifmcep9vvtoT2JRCDqBczRoC7Q0QAvD_BwE) (27.12.2022)
## SD Card ## SD Card
- Sandisk 2GB Micro SD Class 2 [Sandisk 2GB](https://www.amazon.co.uk/gp/product/B000N3LL02/ref=ppx_yo_dt_b_asin_title_o01_s00?ie=UTF8&psc=1) - Sandisk 2GB Micro SD Class 2 [Sandisk 2GB](https://www.amazon.co.uk/gp/product/B000N3LL02/ref=ppx_yo_dt_b_asin_title_o01_s00?ie=UTF8&psc=1)
AITRIP ESP32 and CAM [ESP-32/CAM](https://www.amazon.co.uk/gp/product/B08X49P8P3/ref=ppx_yo_dt_b_asin_title_o03_s00?ie=UTF8&psc=1) AITRIP ESP32 and CAM [ESP-32/CAM](https://www.amazon.co.uk/gp/product/B08X49P8P3/ref=ppx_yo_dt_b_asin_title_o03_s00?ie=UTF8&psc=1)
- [Amazon US - Aideepen ESP32-CAM W BT Board ESP32-CAM-MB Micro USB to Serial Port CH-340G with OV2640 2MP Camera Module Dual Mode](https://www.amazon.com/gp/product/B0948ZFTQZ) with [Amazon US - Cloudisk 5Pack 4GB Micro SD Card 4 GB MicroSD Memory Card Class6](https://www.amazon.com/gp/product/B07QYTP4VN) - [Amazon US - Aideepen ESP32-CAM W BT Board ESP32-CAM-MB Micro USB to Serial Port CH-340G with OV2640 2MP Camera Module Dual Mode](https://www.amazon.com/gp/product/B0948ZFTQZ) with [Amazon US - Cloudisk 5Pack 4GB Micro SD Card 4 GB MicroSD Memory Card Class6](https://www.amazon.com/gp/product/B07QYTP4VN)
# Weak Wifi # Weak Wifi
The ESP32-CAM supports an external antenna. It requires some soldering skills but can improve the connection quality. See [https://randomnerdtutorials.com/esp32-cam-connect-external-antenna/](https://randomnerdtutorials.com/esp32-cam-connect-external-antenna/) The ESP32-CAM supports an external antenna. It requires some soldering skills but can improve the connection quality. See [https://randomnerdtutorials.com/esp32-cam-connect-external-antenna/](https://randomnerdtutorials.com/esp32-cam-connect-external-antenna/)

1
docs/Influx-DB.md
View File

@@ -1,4 +1,5 @@
# Influx DB # Influx DB
The device also supports direct sending of data to an Influx DB. The device also supports direct sending of data to an Influx DB.
See also [Influx Graph in Home Assistant](Integration-Home-Assistant.md#influxdb-graphs). See also [Influx Graph in Home Assistant](Integration-Home-Assistant.md#influxdb-graphs).

64
docs/Installation.md
View File

@@ -1,4 +1,5 @@
# Installation # Installation
The installation requires multiple steps: The installation requires multiple steps:
1. Get the right hardware and wire it up 1. Get the right hardware and wire it up
@@ -9,30 +10,36 @@ The installation requires multiple steps:
For point 2 and 3 we provide multiple ways to do it. Pick the one that looks the easiest for you! For point 2 and 3 we provide multiple ways to do it. Pick the one that looks the easiest for you!
## 1. Hardware ## 1. Hardware
### ESP32-CAM ### ESP32-CAM
* OV2640 camera module * OV2640 camera module
* Micro SD card slot * Micro SD card slot
* 4 or 8 MB PSRAM. * 4 or 8 MB PSRAM.
It can be easily found on the typical internet stores, searching for ESP32-CAM for less than 10 EUR. It can be easily found on the typical internet stores, searching for ESP32-CAM for less than 10 EUR.
How ever since the hardware is cheap and coming from China, you unluckily could pick a malfunctioning device. See [Hardware Compatibility](Hardware-Compatibility.md) for further advice! How ever since the hardware is cheap and coming from China, you unluckily could pick a malfunctioning device. See [Hardware Compatibility](Hardware-Compatibility.md) for further advice!
### USB->UART interface ### USB->UART interface
For first time flashing the firmware a USB -> UART connector is needed. Later firmware upgrades than can be flashed via OTA. For first time flashing the firmware a USB -> UART connector is needed. Later firmware upgrades than can be flashed via OTA.
### Power supply ### Power supply
For power supply a 5V source is needed. Most easily this can be done via a USB power supply. The power supply should support minimum 500mA. For buffering current peaks some users reported to use a large electrolytic capacitor like a 2200uF between ground and VCC. For power supply a 5V source is needed. Most easily this can be done via a USB power supply. The power supply should support minimum 500mA. For buffering current peaks some users reported to use a large electrolytic capacitor like a 2200uF between ground and VCC.
!!! Warning !!! Warning
In several internet forums there are problems reported, in case the ESP32-CAM is only supplied with 3.3V. In several internet forums there are problems reported, in case the ESP32-CAM is only supplied with 3.3V.
### Housing ### Housing
A small 3D-printable example for a very small case can be found in Thingiverse here: [https://www.thingiverse.com/thing:4571627](https://www.thingiverse.com/thing:4571627) A small 3D-printable example for a very small case can be found in Thingiverse here: [https://www.thingiverse.com/thing:4571627](https://www.thingiverse.com/thing:4571627)
![](img/main.jpg){: style="width:200px"} ![Main](img/main.jpg){: style="width:200px"}
![](img/size.png){: style="width:200px"} ![Size](img/size.png){: style="width:200px"}
### Focusing the lense ### Focusing the lense
The focus of the OV2640 needs to be adjusted, as it is normally set from ~40cm to infinity. In order to get an image that is big enough, it needs to be changed to about 10cm. The focus of the OV2640 needs to be adjusted, as it is normally set from ~40cm to infinity. In order to get an image that is big enough, it needs to be changed to about 10cm.
Therefore the sealing glue on the objective ring needs to be removed with a scalpel or sharp knife. Afterwards the objective can be rotated clockwise until the image is sharp again. Therefore the sealing glue on the objective ring needs to be removed with a scalpel or sharp knife. Afterwards the objective can be rotated clockwise until the image is sharp again.
@@ -42,31 +49,35 @@ Therefore the sealing glue on the objective ring needs to be removed with a scal
There are some useful 3D-printable tools available, see [esp32-cam OV2640 focus adjustment wrench](https://www.printables.com/model/877739-esp32-cam-ov2640-focus-adjustment-wrench/comments) or [ETSY shop](https://www.etsy.com/ch/listing/4313477325/esp32-cam-ov2640-fokus-einstellring) if you don't have a 3D printer. There are some useful 3D-printable tools available, see [esp32-cam OV2640 focus adjustment wrench](https://www.printables.com/model/877739-esp32-cam-ov2640-focus-adjustment-wrench/comments) or [ETSY shop](https://www.etsy.com/ch/listing/4313477325/esp32-cam-ov2640-fokus-einstellring) if you don't have a 3D printer.
Also check the discussions for some hints, eg. [discussion 3777](https://github.com/jomjol/AI-on-the-edge-device/discussions/3777). Also check the discussions for some hints, eg. [discussion 3777](https://github.com/jomjol/AI-on-the-edge-device/discussions/3777).
![](img/focus_adjustment.jpg){: style="width:200px"} ![Focus Adjustment](img/focus_adjustment.jpg){: style="width:200px"}
### Wiring ### Wiring
Beside the 5V power supply, only for the first flashing a connection to the USB-UART connector, including a short cut of GPIO0 to GND for bootloader start. Beside the 5V power supply, only for the first flashing a connection to the USB-UART connector, including a short cut of GPIO0 to GND for bootloader start.
A example for wiring can be found here: A example for wiring can be found here:
![](img/wiring.png) ![Wiring](img/wiring.png)
![](img/progammer_manual.jpg) ![Programmer Manual](img/progammer_manual.jpg)
It is also possible to use external LEDs for the illumination instead of the internal flash LED. This is described [here](External-LED.md) It is also possible to use external LEDs for the illumination instead of the internal flash LED. This is described [here](External-LED.md)
## 2. Firmware ## 2. Firmware
### Web Installer ### Web Installer
There is a Web Installer available which will work right out of the web browser Edge and Chrome. There is a Web Installer available which will work right out of the web browser Edge and Chrome.
You can access it with the following link: [Web Installer](https://jomjol.github.io/AI-on-the-edge-device) You can access it with the following link: [Web Installer](https://jomjol.github.io/AI-on-the-edge-device)
**This is the preferred way for beginners** as it also allows access to the USB Log: **This is the preferred way for beginners** as it also allows access to the USB Log:
[![](img/web-console.png)](img/web-console.png) [![Web Console](img/web-console.png)](img/web-console.png)
### Manual Flashing ### Manual Flashing
#### Files #### Files
Grab the firmware from the Grab the firmware from the
- [Releases page](https://github.com/jomjol/AI-on-the-edge-device/releases) (Stable, tested versions), or the - [Releases page](https://github.com/jomjol/AI-on-the-edge-device/releases) (Stable, tested versions), or the
@@ -78,11 +89,11 @@ You need:
* bootloader.bin * bootloader.bin
* firmware.bin * firmware.bin
#### Flashing using the Flash Tool from Espressif (GUI) #### Flashing using the Flash Tool from Espressif (GUI)
Get the [Flash Download Tool](https://www.espressif.com/en/support/download/other-tools) from Espressif. Get the [Flash Download Tool](https://www.espressif.com/en/support/download/other-tools) from Espressif.
Download and extract the Flash tool, after starting choose "Developer Mode", then "ESP32-DownloadTool" and you are in the setup of the flashing tool. Connect the ESP32-CAM with the USB-UART connection and identify the COM-Port. Download and extract the Flash tool, after starting choose "Developer Mode", then "ESP32-DownloadTool" and you are in the setup of the flashing tool. Connect the ESP32-CAM with the USB-UART connection and identify the COM-Port.
!!! Warning !!! Warning
If you are re-flashing the code again, it is strongly recommended to erase the flash memory before flashing the firmware. Especially if you used OTA in between, which might cause remaining information on the flash, to still boot from an old image in the OTA-area, which is not erased by a normal flash. If you are re-flashing the code again, it is strongly recommended to erase the flash memory before flashing the firmware. Especially if you used OTA in between, which might cause remaining information on the flash, to still boot from an old image in the OTA-area, which is not erased by a normal flash.
@@ -95,22 +106,24 @@ Put your ESP32 in bootloader mode and push start, then it will identify the boar
| partitions.bin | 0x8000 | | partitions.bin | 0x8000 |
| firmware.bin | 0x10000 | | firmware.bin | 0x10000 |
![](img/Flash_Settings.png) ![Flash Settings](img/Flash_Settings.png)
#### Flashing using the Python based esptool (Console) #### Flashing using the Python based esptool (Console)
For this you need a python environment (e.g. Anaconda in Win10).
For this you need a python environment (e.g. Anaconda in Win10).
Here you need to install the esptool: Here you need to install the esptool:
``` ```
pip install esptool pip install esptool
``` ```
Then connect the ESP32 with the USB-UART connector to the system, put it in boot mode and with the following command you can erase the flash and flash bootloader, partitions and firmware in two steps: Then connect the ESP32 with the USB-UART connector to the system, put it in boot mode and with the following command you can erase the flash and flash bootloader, partitions and firmware in two steps:
``` ```
esptool erase_flash esptool erase_flash
esptool write_flash 0x01000 bootloader.bin 0x08000 partitions.bin 0x10000 firmware.bin esptool write_flash 0x01000 bootloader.bin 0x08000 partitions.bin 0x10000 firmware.bin
``` ```
- Maybe you need to specify the COM-port if it is not detected by default. - Maybe you need to specify the COM-port if it is not detected by default.
- If the erase command throws the error `A fatal error occurred: ESP32 ROM does not support function erase_flash.`, your `esptool` might be too old, see <https://techoverflow.net/2022/02/08/how-to-fix-esp32-a-fatal-error-occurred-esp32-rom-does-not-support-function-erase_flash/> - If the erase command throws the error `A fatal error occurred: ESP32 ROM does not support function erase_flash.`, your `esptool` might be too old, see <https://techoverflow.net/2022/02/08/how-to-fix-esp32-a-fatal-error-occurred-esp32-rom-does-not-support-function-erase_flash/>
@@ -119,23 +132,26 @@ With some Python installations this may not work and you’ll receive an error,
Further recommendations can be found on the [espressif webpage](https://docs.espressif.com/projects/esptool/en/latest/esp32/installation.html). Further recommendations can be found on the [espressif webpage](https://docs.espressif.com/projects/esptool/en/latest/esp32/installation.html).
## 3. SD Card ## 3. SD Card
The software expects an SD card prepared with certain directory and file structure in order to work properly. The software expects an SD card prepared with certain directory and file structure in order to work properly.
SD card most top directory should look like this: SD card most top directory should look like this:
![](sd-card-content.png) ![SD Card Content](sd-card-content.png)
This initial setup needs only to be done **once** as further updates (Firmware as well as SD card content) are possible with the [Over-The-Air Update](ota.md) mechanism. This initial setup needs only to be done **once** as further updates (Firmware as well as SD card content) are possible with the [Over-The-Air Update](ota.md) mechanism.
#### Notes #### Notes
- Due to the limited availability of GPIOs (OV2640, Flash-Light, PSRAM & SD card) the communication mode to the SD card is limited to 1-line SD-Mode. It showed up, that this results in problems with very large SD-Cards (64GB, sometimes 32 GB) and some no name low cost SD-cards. - Due to the limited availability of GPIOs (OV2640, Flash-Light, PSRAM & SD card) the communication mode to the SD card is limited to 1-line SD-Mode. It showed up, that this results in problems with very large SD-Cards (64GB, sometimes 32 GB) and some no name low cost SD-cards.
- There must be no partition table on the SD-card (no GPT, but only MBR for the single partition) - There must be no partition table on the SD-card (no GPT, but only MBR for the single partition)
- Following setting are necessary for formatting the SD-card: **SINGLE PARTITION, MBR, FAT32 - 32K. NOT exFAT** - Following setting are necessary for formatting the SD-card: **SINGLE PARTITION, MBR, FAT32 - 32K. NOT exFAT**
- Some ESP32 devices share their SD-card and/or camera GPIOs with the pins for TX and RX. If you see errors like “Failed to connect” then your chip is probably not entering the bootloader properly. Remove the respective modules temporarily to free the GPIOs for flashing. You may find more information about troubleshooting on the [homepage of Espressif](https://docs.espressif.com/projects/esptool/en/latest/esp8266/troubleshooting.html). - Some ESP32 devices share their SD-card and/or camera GPIOs with the pins for TX and RX. If you see errors like “Failed to connect” then your chip is probably not entering the bootloader properly. Remove the respective modules temporarily to free the GPIOs for flashing. You may find more information about troubleshooting on the [homepage of Espressif](https://docs.espressif.com/projects/esptool/en/latest/esp8266/troubleshooting.html).
**The ESP32 indicates problems with the SD card during startup with a fast, endless blinking.** **The ESP32 indicates problems with the SD card during startup with a fast, endless blinking.**
**In this case, please try another SD card.** **In this case, please try another SD card.**
### Manual Setup with an SD Card Reader on a PC ### Manual Setup with an SD Card Reader on a PC
1. Take the `AI-on-the-edge-device__manual-setup__*.zip` from the [Release](https://github.com/jomjol/AI-on-the-edge-device/releases) page. 1. Take the `AI-on-the-edge-device__manual-setup__*.zip` from the [Release](https://github.com/jomjol/AI-on-the-edge-device/releases) page.
1. Open it and extract the `sd-card.zip`. 1. Open it and extract the `sd-card.zip`.
1. Open it and extract all files onto onto your SD card. 1. Open it and extract all files onto onto your SD card.
@@ -149,19 +165,20 @@ This initial setup needs only to be done **once** as further updates (Firmware a
After this, you can insert the SD card into the ESP32 board and start it. After this, you can insert the SD card into the ESP32 board and start it.
### Remote Setup using the built-in Access Point ### Remote Setup using the built-in Access Point
On startup of the ESP32, it checks if the `wlan.ini` or the `config/config.ini` are available on the SD card. On startup of the ESP32, it checks if the `wlan.ini` or the `config/config.ini` are available on the SD card.
If not, the ESP32 switches to a special mode. In this mode, it provides a Wifi Access Point which can be used to add the missing `wlan.ini` or the `config/config.ini` file. If not, the ESP32 switches to a special mode. In this mode, it provides a Wifi Access Point which can be used to add the missing `wlan.ini` or the `config/config.ini` file.
1. Take the `AI-on-the-edge-device__remote-setup__*.zip` from the [Release](https://github.com/jomjol/AI-on-the-edge-device/releases) page. 1. Take the `AI-on-the-edge-device__remote-setup__*.zip` from the [Release](https://github.com/jomjol/AI-on-the-edge-device/releases) page.
1. Connect to Access Point of the device. The SSID is "AI-on-the-Edge" and you can access it without any password: 1. Connect to Access Point of the device. The SSID is "AI-on-the-Edge" and you can access it without any password:
![](img/access-point.png) ![Access Point](img/access-point.png)
The device has the following fixed IP: [http://192.168.4.1](http://192.168.4.1). The device has the following fixed IP: [http://192.168.4.1](http://192.168.4.1).
1. Upload initial configuration to SD card 1. Upload initial configuration to SD card
![](img/setup-config.png) ![Setup Config](img/setup-config.png)
Use the `select file` and `upload` button to start the upload. Use the `select file` and `upload` button to start the upload.
A warning will show up if you have chosen a possible wrong file (without default configuration). A warning will show up if you have chosen a possible wrong file (without default configuration).
@@ -170,14 +187,14 @@ If not, the ESP32 switches to a special mode. In this mode, it provides a Wifi A
After the upload, a new page will be shown: After the upload, a new page will be shown:
![](img/setup-wlan.png) ![Setup WLAN](img/setup-wlan.png)
Enter your SSID and password. Enter your SSID and password.
!!! Note !!! Note
Only basic settings are supported. If you need advanced configuration (fixed ip, ...), you need to use the manual setup as documented above. Only basic settings are supported. If you need advanced configuration (fixed ip, ...), you need to use the manual setup as documented above.
!!! Warning !!! Warning
- Carefully check your wifi settings. To change them later on, you need to take out the SD card and edit the `wlan.ini` manually (or delete it and start again). - Carefully check your wifi settings. To change them later on, you need to take out the SD card and edit the `wlan.ini` manually (or delete it and start again).
- The information is transferred without encryption! - The information is transferred without encryption!
@@ -187,15 +204,14 @@ If not, the ESP32 switches to a special mode. In this mode, it provides a Wifi A
The final step is the reboot: The final step is the reboot:
![](img/setup-reboot.png) ![Setup Reboot](img/setup-reboot.png)
!!! Warning !!! Warning
It will take up to 3 minutes. Afterwards you can find your device in the local network. Check your router for the IP. You can find it also in the USB Console output. It will take up to 3 minutes. Afterwards you can find your device in the local network. Check your router for the IP. You can find it also in the USB Console output.
## 4. Initial Startup ## 4. Initial Startup
After the firmware is flashed and the SD card is setup properly, you can start it. After the firmware is flashed and the SD card is setup properly, you can start it.
After power on the connection status is indicated by 3x blinking of the red on board LED. After power on the connection status is indicated by 3x blinking of the red on board LED.

18
docs/Integration-Home-Assistant.md
View File

@@ -1,4 +1,5 @@
# Integration into Home Assistant # Integration into Home Assistant
There are 3 ways to get the data into your Home Assistant: There are 3 ways to get the data into your Home Assistant:
1. Using MQTT (Automatically Setup Entities using Home Assistant MQTT Discovery) 1. Using MQTT (Automatically Setup Entities using Home Assistant MQTT Discovery)
@@ -8,6 +9,7 @@ There are 3 ways to get the data into your Home Assistant:
The first one is the easier way if you already have MQTT in use. The first one is the easier way if you already have MQTT in use.
## Using MQTT (Automatically Setup Entities using Home Assistant MQTT Discovery) ## Using MQTT (Automatically Setup Entities using Home Assistant MQTT Discovery)
Starting with Version `>12.0.1`, AI-on-the-edge-devices support Home Assistant Discovery. Starting with Version `>12.0.1`, AI-on-the-edge-devices support Home Assistant Discovery.
1. Check [here](https://www.home-assistant.io/integrations/mqtt/#mqtt-discovery) to learn more about it and how to enable it in Homeassistant. 1. Check [here](https://www.home-assistant.io/integrations/mqtt/#mqtt-discovery) to learn more about it and how to enable it in Homeassistant.
@@ -23,11 +25,12 @@ On the next start of the device, it will send discovery topics and Home Assistan
![grafik](https://user-images.githubusercontent.com/1783586/199352565-9b0ade28-cb43-47b4-821f-7909cad41a73.png) ![grafik](https://user-images.githubusercontent.com/1783586/199352565-9b0ade28-cb43-47b4-821f-7909cad41a73.png)
![grafik](https://user-images.githubusercontent.com/1783586/199352619-217df627-4b87-4fa0-86a2-f5347c452fdb.png) ![grafik](https://user-images.githubusercontent.com/1783586/199352619-217df627-4b87-4fa0-86a2-f5347c452fdb.png)
### Using MQTT (Manually Setup Entities) ### Using MQTT (Manually Setup Entities)
First make sure with an MQTT client (for example [MQTT Explorer](http://mqtt-explorer.com/)) that MQTT works as expected and to get a list of the available topics! First make sure with an MQTT client (for example [MQTT Explorer](http://mqtt-explorer.com/)) that MQTT works as expected and to get a list of the available topics!
Then add a sensor for each property: Then add a sensor for each property:
```yaml ```yaml
mqtt: mqtt:
sensor: sensor:
@@ -73,7 +76,9 @@ mqtt:
payload_available: connected payload_available: connected
payload_not_available: connection lost payload_not_available: connection lost
``` ```
If you run the discovery once, you can also extract the information from there (MQTT Info, untested): If you run the discovery once, you can also extract the information from there (MQTT Info, untested):
```yaml ```yaml
mqtt: # Extracted form the Discovery but untested! mqtt: # Extracted form the Discovery but untested!
sensor: sensor:
@@ -90,6 +95,7 @@ mqtt: # Extracted form the Discovery but untested!
``` ```
If you want to convert the `m³` to `l`, use a template sensor: If you want to convert the `m³` to `l`, use a template sensor:
```yaml ```yaml
template: template:
- sensor: - sensor:
@@ -118,22 +124,24 @@ utility_meter:
Note that you also can add it using the UI. Note that you also can add it using the UI.
### Examples ### Examples
![grafik](https://user-images.githubusercontent.com/1783586/193472069-4135736e-e63a-4afb-8009-5b97aa5c9ac5.png) ![grafik](https://user-images.githubusercontent.com/1783586/193472069-4135736e-e63a-4afb-8009-5b97aa5c9ac5.png)
![grafik](https://user-images.githubusercontent.com/1783586/193472091-1484aac4-ddc2-48ba-896c-28370963fc2d.png) ![grafik](https://user-images.githubusercontent.com/1783586/193472091-1484aac4-ddc2-48ba-896c-28370963fc2d.png)
### Statistics Graph ### Statistics Graph
Creating Statistics Graphs (e.g. usage per day) is easy using the [Energy Dashboard](https://www.home-assistant.io/home-energy-management/): Creating Statistics Graphs (e.g. usage per day) is easy using the [Energy Dashboard](https://www.home-assistant.io/home-energy-management/):
![grafik](https://user-images.githubusercontent.com/1783586/193471893-d8ab8f5f-0906-4076-8926-8b5a69a24bce.png) ![grafik](https://user-images.githubusercontent.com/1783586/193471893-d8ab8f5f-0906-4076-8926-8b5a69a24bce.png)
Note that there seems to be a bug in the graph, see [https://github.com/home-assistant/frontend/issues/13995](https://github.com/home-assistant/frontend/issues/13995)! Note that there seems to be a bug in the graph, see [https://github.com/home-assistant/frontend/issues/13995](https://github.com/home-assistant/frontend/issues/13995)!
### InfluxDb Graphs ### InfluxDb Graphs
See also [Influx-DB](Influx-DB.md). See also [Influx-DB](Influx-DB.md).
If you have setup InfluxDB already, it is also possible to fetch statistics from there, e.g. daily usage: If you have setup InfluxDB already, it is also possible to fetch statistics from there, e.g. daily usage:
``` ```
from(bucket: "HomeAssistant") from(bucket: "HomeAssistant")
|> range(start: v.timeRangeStart, stop: v.timeRangeStop) |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
@@ -146,8 +154,8 @@ from(bucket: "HomeAssistant")
![grafik](https://user-images.githubusercontent.com/1783586/193473347-c81fc301-c52f-4af0-9fcb-56fab12cacac.png) ![grafik](https://user-images.githubusercontent.com/1783586/193473347-c81fc301-c52f-4af0-9fcb-56fab12cacac.png)
## Using REST ## Using REST
When using REST, Home Assistant has to periodically call an URL on the ESP32 which in return provides the requested data. When using REST, Home Assistant has to periodically call an URL on the ESP32 which in return provides the requested data.
See [REST API](REST-API.md) for a list of available URLs. See [REST API](REST-API.md) for a list of available URLs.
@@ -155,6 +163,7 @@ See [REST API](REST-API.md) for a list of available URLs.
The most practical one is the `json` entrypoint which provides the most relevant data JSON formatted: The most practical one is the `json` entrypoint which provides the most relevant data JSON formatted:
`http://<IP>/json` `http://<IP>/json`
This would return: This would return:
```JSON ```JSON
{ {
"main": "main":
@@ -170,6 +179,7 @@ This would return:
``` ```
To do such a REST call, you need to create a REST sensor: To do such a REST call, you need to create a REST sensor:
```yaml ```yaml
sensor: sensor:
@@ -218,8 +228,8 @@ sensor:
See also https://community.home-assistant.io/t/rest-sensor-nested-json/243420/9 See also https://community.home-assistant.io/t/rest-sensor-nested-json/243420/9
#### Photo #### Photo
REST can also be used to show the photo of the last round: REST can also be used to show the photo of the last round:
![grafik](https://user-images.githubusercontent.com/1783586/193546075-b247942f-9106-47a4-a64b-42ff96dd9078.png) ![grafik](https://user-images.githubusercontent.com/1783586/193546075-b247942f-9106-47a4-a64b-42ff96dd9078.png)

5
docs/Learn-models-with-your-own-images.md
View File

@@ -1,4 +1,5 @@
# Learn a model with your own images # Learn a model with your own images
Once you have collected and selected your own images (see [Collect images to improve the models](collect-new-images.md)), you can train your very own model with them. Once you have collected and selected your own images (see [Collect images to improve the models](collect-new-images.md)), you can train your very own model with them.
**This is an optional step and only suggested for advances users!** **This is an optional step and only suggested for advances users!**
@@ -8,6 +9,7 @@ For training the model you will need a python and Jupyter installation.
All current labeled images you can find under [ziffer_sortiert_raw](https://github.com/jomjol/neural-network-digital-counter-readout/tree/master/ziffer_sortiert_raw) All current labeled images you can find under [ziffer_sortiert_raw](https://github.com/jomjol/neural-network-digital-counter-readout/tree/master/ziffer_sortiert_raw)
### dig-class11 models (digits) ### dig-class11 models (digits)
Fork and checkout [neural-network-digital-counter-readout](https://github.com/jomjol/neural-network-digital-counter-readout). Fork and checkout [neural-network-digital-counter-readout](https://github.com/jomjol/neural-network-digital-counter-readout).
Install all requirements for running the notebooks. Install all requirements for running the notebooks.
@@ -24,6 +26,7 @@ Put your labeled images into `/ziffer_sortiert_raw` folder and run
It creates a dig-class11_xxxx_s2.tflite model, you can upload to the `config` folder on your device and test it. It creates a dig-class11_xxxx_s2.tflite model, you can upload to the `config` folder on your device and test it.
### dig-class100 / dig-cont models (digits) ### dig-class100 / dig-cont models (digits)
Fork and checkout [neural-network-digital-counter-readout](https://github.com/jomjol/neural-network-digital-counter-readout). Fork and checkout [neural-network-digital-counter-readout](https://github.com/jomjol/neural-network-digital-counter-readout).
All labeled images you can find under [Images](https://github.com/haverland/Tenth-of-step-of-a-meter-digit/tree/master/images) All labeled images you can find under [Images](https://github.com/haverland/Tenth-of-step-of-a-meter-digit/tree/master/images)
@@ -39,6 +42,7 @@ Put your labeled images into `images/collected/<typeofdevice>/<your_short>/`
Run [dig-class100-s2.ipynb](https://github.com/haverland/Tenth-of-step-of-a-meter-digit/blob/master/dig-class100-s2.ipynb). The model to upload to your device you can find under '/output'. Run [dig-class100-s2.ipynb](https://github.com/haverland/Tenth-of-step-of-a-meter-digit/blob/master/dig-class100-s2.ipynb). The model to upload to your device you can find under '/output'.
### ana-class100/ana-cont models (analog pointers) ### ana-class100/ana-cont models (analog pointers)
Fork and checkout [neural-network-analog-needle-readout](https://github.com/jomjol/neural-network-analog-needle-readout). Fork and checkout [neural-network-analog-needle-readout](https://github.com/jomjol/neural-network-analog-needle-readout).
All labeled images you can find under [data_raw_all](https://github.com/jomjol/neural-network-analog-needle-readout/tree/main/data_raw_all) All labeled images you can find under [data_raw_all](https://github.com/jomjol/neural-network-analog-needle-readout/tree/main/data_raw_all)
@@ -56,6 +60,7 @@ After every adding of images you need to run [Image_Preparation.ipynb](https://g
Run [Train_CNN_Analog-Readout_100-Small1_Dropout.ipynb](https://github.com/jomjol/neural-network-analog-needle-readout/blob/main/Train_CNN_Analog-Readout_100-Small1_Dropout.ipynb) and/or [Train_CNN_Analog-Readout_Version-Small2.ipynb](https://github.com/jomjol/neural-network-analog-needle-readout/blob/main/Train_CNN_Analog-Readout_Version-Small2.ipynb). The model to upload to your device you can find in the project folder. Run [Train_CNN_Analog-Readout_100-Small1_Dropout.ipynb](https://github.com/jomjol/neural-network-analog-needle-readout/blob/main/Train_CNN_Analog-Readout_100-Small1_Dropout.ipynb) and/or [Train_CNN_Analog-Readout_Version-Small2.ipynb](https://github.com/jomjol/neural-network-analog-needle-readout/blob/main/Train_CNN_Analog-Readout_Version-Small2.ipynb). The model to upload to your device you can find in the project folder.
## Share your images ## Share your images
If the results are good you can share the images as pull-request. Please images only! If the results are good you can share the images as pull-request. Please images only!
See [Share your images](collect-new-images.md#share-your-images) for details. See [Share your images](collect-new-images.md#share-your-images) for details.

7
docs/MQTT-API.md
View File

@@ -1,4 +1,5 @@
# MQTT API # MQTT API
The device is capable to register to a MQTT broker to publish data and subscribe to specific topics. The device is capable to register to a MQTT broker to publish data and subscribe to specific topics.
!!! Note !!! Note
@@ -51,18 +52,21 @@ Topic | Description
`watermeter/main/json` | This is a JSON formatted object containing the following values: `value`, `raw`, `pre`, `error`, `rate`, `timestamp`. `watermeter/main/json` | This is a JSON formatted object containing the following values: `value`, `raw`, `pre`, `error`, `rate`, `timestamp`.
### GPIO ### GPIO
`MainTopic`/{GPIO topic}, e.g. `watermeter/GPIO/GPIO12` `MainTopic`/{GPIO topic}, e.g. `watermeter/GPIO/GPIO12`
#### GPIO/GPIO{PinNumber} #### GPIO/GPIO{PinNumber}
Depending on device configuration (`Settings` --> `Configuration` --> section `GPIO`) Depending on device configuration (`Settings` --> `Configuration` --> section `GPIO`)
## Subscribed topics ## Subscribed topics
`MainTopic`/{subscribed topic}, e.g. `watermeter/ctrl/flow_start` `MainTopic`/{subscribed topic}, e.g. `watermeter/ctrl/flow_start`
### Control ### Control
#### ctrl/flow_start #### ctrl/flow_start
Trigger a flow start by publishing to this topic. Trigger a flow start by publishing to this topic.
This will automatically reset the flow interval. This will automatically reset the flow interval.
@@ -85,4 +89,5 @@ __Payload:__
* `"value":` Provide any negative number * `"value":` Provide any negative number
#### GPIO/GPIO{PinNumber} #### GPIO/GPIO{PinNumber}
Depending on device configuration (`Settings` --> `Configuration` --> section `GPIO`) Depending on device configuration (`Settings` --> `Configuration` --> section `GPIO`)

41
docs/Neural-Network-Types.md
View File

@@ -1,11 +1,12 @@
# Neural Network Types # Neural Network Types
!!! Note !!! Note
For an overview, see [Choosing the Model](Choosing-the-Model.md). For an overview, see [Choosing the Model](Choosing-the-Model.md).
This section is describing the different types of neural networks, that are used with the AI-on-the-edge approach and gives an introduction on how and where to use them. This section is describing the different types of neural networks, that are used with the AI-on-the-edge approach and gives an introduction on how and where to use them.
## Overview neural network type ## Overview neural network type
There are two **types of input**: There are two **types of input**:
* digits with rolling number (top town) * digits with rolling number (top town)
@@ -29,12 +30,12 @@ No setting of the type in the firmware is necessary. The type can detect by the
* For the continuous and 100 classes network especially for the digits, there are only a few types of digits trained up to now * For the continuous and 100 classes network especially for the digits, there are only a few types of digits trained up to now
* Therefore sometimes for the digits it is more effective to choose the simpler 11 classes network type (= default). * Therefore sometimes for the digits it is more effective to choose the simpler 11 classes network type (= default).
## Naming convention ## Naming convention
| | Classification<br />11 classes<br />0, 1, ... 9 + "N" | Classification<br />100 classes<br />0.0, 0.1, ... 9.9 | Continuous<br />Interval<br />[0, 10[ | | | Classification<br />11 classes<br />0, 1, ... 9 + "N" | Classification<br />100 classes<br />0.0, 0.1, ... 9.9 | Continuous<br />Interval<br />[0, 10[ |
| ---------------------------------------------------- | ----------------------------------------------------- | ------------------------------------------------------ | ------------------------------------- | | ---------------------------------------------------- | ----------------------------------------------------- | ------------------------------------------------------ | ------------------------------------- |
| **Digits** <br />![](img/0_arbitrary.jpg) | **dig-class11**_XXX.tflite | **dig-class100**_XXX.tflite | **dig-cont**_XXX.tflite | | **Digits** <br />![Arbitrary](img/0_arbitrary.jpg) | **dig-class11**_XXX.tflite | **dig-class100**_XXX.tflite | **dig-cont**_XXX.tflite |
| **Analog Pointers** <br />![](img/ana-examp.jpg) | | **ana-class100**_XXX.tflite | **ana-cont**_XXX.tflite | | **Analog Pointers** <br />![ANA Example](img/ana-examp.jpg) | | **ana-class100**_XXX.tflite | **ana-cont**_XXX.tflite |
XXX contains the versioning and a parameter for different sizes with the following naming: XXX contains the versioning and a parameter for different sizes with the following naming:
@@ -53,29 +54,35 @@ Example: `dig-class11_1410_s2_q.tflite`
* s2 = Size 2 (Medium) * s2 = Size 2 (Medium)
* q = Quantized Version * q = Quantized Version
## Overview of trained types and details ## Overview of trained types and details
### Analog Pointer ("ana-cont_XXX.tflite" & "ana-class100_XXX.tflite") ### Analog Pointer ("ana-cont_XXX.tflite" & "ana-class100_XXX.tflite")
This is to transfer the direction of a pointer into a continuous number between 0 and 1, whereas 0 (=1) is the upwards position (12 o'clock), 0.25 corresponds to the 3 o'clock positions and so on. This network is a envelop for all different types of pointers. Currently there are no dedicated network trainings for specific types of pointers. This is to transfer the direction of a pointer into a continuous number between 0 and 1, whereas 0 (=1) is the upwards position (12 o'clock), 0.25 corresponds to the 3 o'clock positions and so on. This network is a envelop for all different types of pointers. Currently there are no dedicated network trainings for specific types of pointers.
There are two types of network structure, currently both are supported. The "class100" is a pure classification network, that might need a bit more accuracy in the labeling. "cont" is a no classic approach with a continuous output off only 2 neurons (details see below). There are two types of network structure, currently both are supported. The "class100" is a pure classification network, that might need a bit more accuracy in the labeling. "cont" is a no classic approach with a continuous output off only 2 neurons (details see below).
#### Types of counters trained #### Types of counters trained
| | | | | | | | | |
| ----------------------------------- | ----------------------------------- | ----------------------------------- | ----------------------------------- | | ----------------------------------- | ----------------------------------- | ----------------------------------- | ----------------------------------- |
| ![](img/ana-cont/examp-ana1.jpg) | ![](img/ana-cont/examp-ana2.jpg) | ![](img/ana-cont/examp-ana3.jpg) | ![](img/ana-cont/examp-ana4.jpg) | | ![Example ana1](img/ana-cont/examp-ana1.jpg) | ![Example ana2](img/ana-cont/examp-ana2.jpg) | ![Example ana3](img/ana-cont/examp-ana3.jpg) | ![Example ana4](img/ana-cont/examp-ana4.jpg) |
| ![](img/ana-cont/examp-ana5.jpg) | ![](img/ana-cont/examp-ana6.jpg) | ![](img/ana-cont/examp-ana7.jpg) | | | ![Example ana5](img/ana-cont/examp-ana5.jpg) | ![Example ana6](img/ana-cont/examp-ana6.jpg) | ![Example ana7](img/ana-cont/examp-ana7.jpg) | |
#### Training data needs #### Training data needs
* Quadratic images, minimum size: 32x32 pixel * Quadratic images, minimum size: 32x32 pixel
* Typically 100 - 200 images with a resolution of 1/100 of the full rotation (every 0.1 value or 3.6°) * Typically 100 - 200 images with a resolution of 1/100 of the full rotation (every 0.1 value or 3.6°)
* Naming: x.y_ARBITRARY.jpg, where x.y = value 0.0 ... 9.9 * Naming: x.y_ARBITRARY.jpg, where x.y = value 0.0 ... 9.9
#### CNN Technical details: #### CNN Technical details:
##### Input ##### Input
* 32 x 32 RGB images * 32 x 32 RGB images
##### Output ##### Output
* **ana-cont**_XXX.tflite: * **ana-cont**_XXX.tflite:
* 2 neurons with output in range [-1, 1] - representing a sinus / cosine encoding of the angle * 2 neurons with output in range [-1, 1] - representing a sinus / cosine encoding of the angle
* needs to be converted to angle with arctan-hyperbolic function * needs to be converted to angle with arctan-hyperbolic function
@@ -83,53 +90,61 @@ There are two types of network structure, currently both are supported. The "cla
* **ana-class100**_XXX.tflite * **ana-class100**_XXX.tflite
* 100 neurons representing the classes from 0.0, 0.1, ... 9.8, 9.9 * 100 neurons representing the classes from 0.0, 0.1, ... 9.8, 9.9
### Digits with 11 classes ("dig-class11_XXX.tflite") ### Digits with 11 classes ("dig-class11_XXX.tflite")
The digit type is a classical classification network, with 11 classes representing the numbers 0, 1, ... 9 and the special class "N". It is trained for the rolling ring of gas and electric meters. As there is sometime a status between two images, the special class "N" is representing Not-A-Number for the case, that the image cannot be unique classified to one number e.g. because it is between two digits. For this type the lowest amount of training data per type is needed, resulting in a large variety of type being already part of the training set. The digit type is a classical classification network, with 11 classes representing the numbers 0, 1, ... 9 and the special class "N". It is trained for the rolling ring of gas and electric meters. As there is sometime a status between two images, the special class "N" is representing Not-A-Number for the case, that the image cannot be unique classified to one number e.g. because it is between two digits. For this type the lowest amount of training data per type is needed, resulting in a large variety of type being already part of the training set.
#### Types of counters trained #### Types of counters trained
| | | | | | | | | | | | | | | |
| -------------------------- | -------------------------- | -------------------------- | -------------------------- | -------------------------- | -------------------------- | -------------------------- | | -------------------------- | -------------------------- | -------------------------- | -------------------------- | -------------------------- | -------------------------- | -------------------------- |
| ![](img/dig-class11/examp-dig1.jpg) | ![](img/dig-class11/examp-dig2.jpg) | ![](img/dig-class11/examp-dig3.jpg) | ![](img/dig-class11/examp-dig4.jpg) | ![](img/dig-class11/examp-dig13.jpg) | ![](img/dig-class11/examp-dig12.jpg) | ![](img/dig-class11/examp-dig9.jpg) | | ![Example Dig 1](img/dig-class11/examp-dig1.jpg) | ![Example Dig 2](img/dig-class11/examp-dig2.jpg) | ![Example Dig 3](img/dig-class11/examp-dig3.jpg) | ![Example Dig 4](img/dig-class11/examp-dig4.jpg) | ![Example Dig 13](img/dig-class11/examp-dig13.jpg) | ![Example Dig 12](img/dig-class11/examp-dig12.jpg) | ![Example Dig 9](img/dig-class11/examp-dig9.jpg) |
| ![](img/dig-class11/examp-dig5.jpg) | ![](img/dig-class11/examp-dig6.jpg) | ![](img/dig-class11/examp-dig7.jpg) | ![](img/dig-class11/examp-dig8.jpg) | ![](img/dig-class11/examp-dig11.jpg) | ![](img/dig-class11/examp-dig10.jpg) | | | ![Example Dig 5](img/dig-class11/examp-dig5.jpg) | ![Example Dig 6](img/dig-class11/examp-dig6.jpg) | ![Example Dig 7](img/dig-class11/examp-dig7.jpg) | ![Example Dig 8](img/dig-class11/examp-dig8.jpg) | ![Example Dig 11](img/dig-class11/examp-dig11.jpg) | ![Example Dig 10](img/dig-class11/examp-dig10.jpg) | |
#### Training data needs #### Training data needs
* RGB images, with minimum size: 20x32 pixel * RGB images, with minimum size: 20x32 pixel
* Typically 10 - 20 images (1-2 for each digit and an arbitrary number for the "N" class * Typically 10 - 20 images (1-2 for each digit and an arbitrary number for the "N" class
* Naming: x_ARBITRARY.jpg, where x = value 0 ... 9 + N * Naming: x_ARBITRARY.jpg, where x = value 0 ... 9 + N
#### CNN Technical details #### CNN Technical details
##### Input ##### Input
* 20 x 32 RGB images * 20 x 32 RGB images
##### Output ##### Output
* 11 neurons for image classification (last layer normalized to 1) * 11 neurons for image classification (last layer normalized to 1)
* Neuron 0 to 9 represent the corresponding numbers "0" to "9" * Neuron 0 to 9 represent the corresponding numbers "0" to "9"
* Neuron 10 represents the "Not-A-Number" class, telling, that the image is not uniquely classified * Neuron 10 represents the "Not-A-Number" class, telling, that the image is not uniquely classified
### Digits with rolling results ("dig-class100_XXX.tflite" & "dig-cont_XXX.tflite") ### Digits with rolling results ("dig-class100_XXX.tflite" & "dig-cont_XXX.tflite")
This type of network tries to overcome the problem, that there are intermediate values, when a rolling digit is between two numbers. Previous this was the "N" class. In this network type, there are also sub-digit values trained, so that the intermediate state can be used as additional information for the algorithms. This type of network tries to overcome the problem, that there are intermediate values, when a rolling digit is between two numbers. Previous this was the "N" class. In this network type, there are also sub-digit values trained, so that the intermediate state can be used as additional information for the algorithms.
#### Types of counters trained #### Types of counters trained
| | | | | | | | | |
| ---------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ---- | | ---------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ---- |
| ![](img/dig-cont/dig-cont_1.jpg) | ![](img/dig-cont/dig-cont_2a.jpg) ![](img/dig-cont/dig-cont_2b.jpg) | ![](img/dig-cont/dig-cont_3a.jpg) ![](img/dig-cont/dig-cont_3b.jpg) ![](img/dig-cont/dig-cont_3c.jpg) | | | ![Digi Cont 1](img/dig-cont/dig-cont_1.jpg) | ![Dig Cont 2a](img/dig-cont/dig-cont_2a.jpg) ![Dig Cont 2b](img/dig-cont/dig-cont_2b.jpg) | ![Dig Cont 3a](img/dig-cont/dig-cont_3a.jpg) ![Dig Cont 3b](img/dig-cont/dig-cont_3b.jpg) ![Dig Cont 3c](img/dig-cont/dig-cont_3c.jpg) | |
#### Training data needs #### Training data needs
* RGB images, with minimum size: 20x32 pixel * RGB images, with minimum size: 20x32 pixel
* Typically 100 - 200 images (1-2 for each possible position) * Typically 100 - 200 images (1-2 for each possible position)
* Naming: x.y_ARBITRARY.jpg, where x.y = 0.0, 0.1, ... 9.9 representing the intermediate state * Naming: x.y_ARBITRARY.jpg, where x.y = 0.0, 0.1, ... 9.9 representing the intermediate state
#### CNN Technical details #### CNN Technical details
##### Input ##### Input
* 20 x 32 RGB images * 20 x 32 RGB images
##### Output ##### Output
* **dig-cont**_XXX.tflite: * **dig-cont**_XXX.tflite:
* 10 neurons representing the digits 0, 1, ... 9. The intermediate values are represented by weighted normalized values of two neighboring output neurons * 10 neurons representing the digits 0, 1, ... 9. The intermediate values are represented by weighted normalized values of two neighboring output neurons
* needs to be converted to angle with arctan-hyperbolic function * needs to be converted to angle with arctan-hyperbolic function

3
docs/New-Releases-Notification.md
View File

@@ -4,12 +4,13 @@ Do you want to get notified about a new release?
There are several ways for it: There are several ways for it:
## Github Notifications ## Github Notifications
You will need a Github Account for this! You will need a Github Account for this!
1. Log into your Github account on [Github](https://github.com). 1. Log into your Github account on [Github](https://github.com).
1. Go to [AI-on-the-edge-device](https://github.com/jomjol/AI-on-the-edge-device). 1. Go to [AI-on-the-edge-device](https://github.com/jomjol/AI-on-the-edge-device).
1. On the top right side, click onto `Watch` and select `Custom`: 1. On the top right side, click onto `Watch` and select `Custom`:
![](img/release-notification.png) ![Release Notification](img/release-notification.png)
1. Select `Releases`. 1. Select `Releases`.
You will get an email when a new release gets created. You will get an email when a new release gets created.

2
docs/Password-Protection.md
View File

@@ -1,8 +1,10 @@
# Password Protection # Password Protection
The Web Interface and the REST API can be protected by a password. The Web Interface and the REST API can be protected by a password.
To do so, you have to set the username and password in the `wlan.ini` file on the SD-Card. To do so, you have to set the username and password in the `wlan.ini` file on the SD-Card.
Uncomment (remove the leading `;`) and update the two lines with `http_username` and `http_password`: Uncomment (remove the leading `;`) and update the two lines with `http_username` and `http_password`:
``` ```
http_username = "myusername" http_username = "myusername"
http_password = "mypassword" http_password = "mypassword"

37
docs/REST-API.md
View File

@@ -1,4 +1,5 @@
# REST API # REST API
Various information is directly accessible over specific REST calls. Various information is directly accessible over specific REST calls.
To use it, just append them to the IP, separated with a `/`, e.g. `http://192.168.1.1/json` To use it, just append them to the IP, separated with a `/`, e.g. `http://192.168.1.1/json`
@@ -7,7 +8,9 @@ To use it, just append them to the IP, separated with a `/`, e.g. `http://192.16
For more detailed information to the REST handler, have a look to the code in the repository: [registered handlers](https://github.com/jomjol/AI-on-the-edge-device/search?q=camuri.uri) For more detailed information to the REST handler, have a look to the code in the repository: [registered handlers](https://github.com/jomjol/AI-on-the-edge-device/search?q=camuri.uri)
## Control ## Control
### flow_start ### flow_start
Trigger the next flow Trigger the next flow
+ Payload: + Payload:
- No payload needed - No payload needed
@@ -15,6 +18,7 @@ To use it, just append them to the IP, separated with a `/`, e.g. `http://192.16
This will automatically reset the flow interval. This will automatically reset the flow interval.
### setPreValue ### setPreValue
Set the last valid value (previous value) to given value or the actual RAW value. Set the last valid value (previous value) to given value or the actual RAW value.
+ Payload: + Payload:
- Set to given value (value >= 0), e.g. `/setPreValue?numbers=main&value=1234.5678` - Set to given value (value >= 0), e.g. `/setPreValue?numbers=main&value=1234.5678`
@@ -26,6 +30,7 @@ This will automatically reset the flow interval.
* `value=` provide yna negative number * `value=` provide yna negative number
### GPIO ### GPIO
- Control a GPIO output - Control a GPIO output
- The `GPIO` entrypoint also support parameters: - The `GPIO` entrypoint also support parameters:
- `/GPIO?GPIO={PinNumber}&Status=high` - `/GPIO?GPIO={PinNumber}&Status=high`
@@ -38,14 +43,18 @@ This will automatically reset the flow interval.
- Example: `/GPIO?GPIO=12` - Example: `/GPIO?GPIO=12`
### reboot ### reboot
Trigger a reboot of the device Trigger a reboot of the device
### mqtt_publish_discovery ### mqtt_publish_discovery
Trigger re-sending of the Home Assistant discovery topics. Trigger re-sending of the Home Assistant discovery topics.
The topics will get send at the end of the next round. The topics will get send at the end of the next round.
## Results ## Results
### json ### json
Show result in JSON syntax Show result in JSON syntax
- Example: - Example:
`{ `{
@@ -61,6 +70,7 @@ This will automatically reset the flow interval.
}` }`
### value ### value
Show single result values Show single result values
- The `value` entrypoint also support parameters: - The `value` entrypoint also support parameters:
- `http://<IP>/value?all=true&type=value` - `http://<IP>/value?all=true&type=value`
@@ -69,53 +79,69 @@ This will automatically reset the flow interval.
- `http://<IP>/value?all=true&type=prevalue` - `http://<IP>/value?all=true&type=prevalue`
### img_tmp/raw.jpg ### img_tmp/raw.jpg
Capture and show a new raw image Capture and show a new raw image
### img_tmp/alg.jpg ### img_tmp/alg.jpg
Show last aligned image Show last aligned image
### img_tmp/alg_roi.jpg ### img_tmp/alg_roi.jpg
Show last aligned image including ROI overlay Show last aligned image including ROI overlay
## Status ## Status
### statusflow ### statusflow
Show the actual step of the flow incl. timestamp Show the actual step of the flow incl. timestamp
- Example: `Take Image (15:56:34)` - Example: `Take Image (15:56:34)`
### rssi ### rssi
Show the WIFI signal strength (Unit: dBm) Show the WIFI signal strength (Unit: dBm)
- Example: `-51` - Example: `-51`
### cpu_temperature ### cpu_temperature
Show the CPU temperature (Unit: °C) Show the CPU temperature (Unit: °C)
- Example: `38` - Example: `38`
### sysinfo ### sysinfo
Show system infos in JSON syntax Show system infos in JSON syntax
- Example: `[{"firmware": "","buildtime": "2023-01-25 12:41","gitbranch": "HEAD","gittag": "","gitrevision": "af13c68+","html": "Development-Branch: HEAD (Commit: af13c68+)","cputemp": "64","hostname": "WaterMeterTest","IPv4": "192.168.xxx.xxx","freeHeapMem": "2818330"}]` - Example: `[{"firmware": "","buildtime": "2023-01-25 12:41","gitbranch": "HEAD","gittag": "","gitrevision": "af13c68+","html": "Development-Branch: HEAD (Commit: af13c68+)","cputemp": "64","hostname": "WaterMeterTest","IPv4": "192.168.xxx.xxx","freeHeapMem": "2818330"}]`
### starttime ### starttime
Show starttime Show starttime
- Example: `20230113-154634` - Example: `20230113-154634`
### uptime ### uptime
Show uptime Show uptime
- Example: `0d 00h 15m 50s` - Example: `0d 00h 15m 50s`
## Camera ## Camera
### lighton ### lighton
Switch the camera flashlight on Switch the camera flashlight on
### lightoff ### lightoff
Switch the camera flashlight off Switch the camera flashlight off
### capture ### capture
Capture a new image (without flashlight) Capture a new image (without flashlight)
### capture_with_flashlight ### capture_with_flashlight
Capture a new image with flashlight Capture a new image with flashlight
### stream ### stream
Stream a live video of the camera. Stream a live video of the camera.
It has a slow refresh rate of only 2 FPS to avoid stressing the system. Flow processing continues to work in the background, albeit possibly a bit slower. It has a slow refresh rate of only 2 FPS to avoid stressing the system. Flow processing continues to work in the background, albeit possibly a bit slower.
@@ -124,30 +150,39 @@ This will automatically reset the flow interval.
**LIMITATION:** To avoid using extra memory, no additional dedicated stream webserver is implemented for this feature (instead, the web interface server is reused in a kind of "blocking way"). This means that either the web interface is fully functional or the stream is active, but not both at the same time. However, this is sufficient for the intended use case. **LIMITATION:** To avoid using extra memory, no additional dedicated stream webserver is implemented for this feature (instead, the web interface server is reused in a kind of "blocking way"). This means that either the web interface is fully functional or the stream is active, but not both at the same time. However, this is sufficient for the intended use case.
### save ### save
Save a new image to SD card Save a new image to SD card
- The `save` entrypoint also support parameters: - The `save` entrypoint also support parameters:
- `http://<IP>/save?filename=test.jpg&delay=1000` - `http://<IP>/save?filename=test.jpg&delay=1000`
## Logs ## Logs
### log ### log
Last part of todays log (last 80 kBytes)) Last part of todays log (last 80 kBytes))
### logfileact ### logfileact
Full log of today Full log of today
### log.html ### log.html
Opens the log html page Opens the log html page
## Diagnostics ## Diagnostics
### heap ### heap
print relevant memory (heap) information print relevant memory (heap) information
- Example: `Heap info: Heap Total: 1888926 | SPI Free: 1827431 | SPI Larg Block: 1802240 | SPI Min Free: 758155 | Int Free: 61495 | Int Larg Block: 55296 | Int Min Free: 36427` - Example: `Heap info: Heap Total: 1888926 | SPI Free: 1827431 | SPI Larg Block: 1802240 | SPI Min Free: 758155 | Int Free: 61495 | Int Larg Block: 55296 | Int Min Free: 36427`
## Prometheus/OpenMetrics ## Prometheus/OpenMetrics
### metrics ### metrics
Provides a set of metrics that can be scraped by prometheus. See [Prometheus/OpenMetrics](prometheus-openmetrics.md) for details. Provides a set of metrics that can be scraped by prometheus. See [Prometheus/OpenMetrics](prometheus-openmetrics.md) for details.
## Password Protection ## Password Protection
The Web Interface and the REST API can be protected by a password, see [Password-Protection](https://jomjol.github.io/AI-on-the-edge-device-docs/Password-Protection/). The Web Interface and the REST API can be protected by a password, see [Password-Protection](https://jomjol.github.io/AI-on-the-edge-device-docs/Password-Protection/).

32
docs/ROI-Configuration.md
View File

@@ -1,4 +1,5 @@
# ROIs (Regions of Interest) # ROIs (Regions of Interest)
!!! Notes !!! Notes
You are using a neural network approach which is trained to fit as many different type of meters as possible. You are using a neural network approach which is trained to fit as many different type of meters as possible.
The accuracy will never be 100%. It is normal to see a missing reading once in a while. The accuracy will never be 100%. It is normal to see a missing reading once in a while.
@@ -14,22 +15,22 @@ The most critical components for an accurate detection are:
Have a look on the [Digit Counters](https://jomjol.github.io/neural-network-digital-counter-readout/) resp. [Analog Needles](https://jomjol.github.io/neural-network-analog-needle-readout) to check if your types are contained. If your number types are **not** contained, you should take the effort to record them so we can add them to the training data. See [Collect images to improve the models](Learn-models-with-your-own-images.md) on how to collect new training data. Have a look on the [Digit Counters](https://jomjol.github.io/neural-network-digital-counter-readout/) resp. [Analog Needles](https://jomjol.github.io/neural-network-analog-needle-readout) to check if your types are contained. If your number types are **not** contained, you should take the effort to record them so we can add them to the training data. See [Collect images to improve the models](Learn-models-with-your-own-images.md) on how to collect new training data.
## Precondition ## Precondition
Please make sure to have: Please make sure to have:
1. Setup your camera properly and taken a good [Reference Image](Reference-Image.md). 1. Setup your camera properly and taken a good [Reference Image](Reference-Image.md).
1. Selected good [Alignment References](Alignment.md). 1. Selected good [Alignment References](Alignment.md).
## Define the ROIs ## Define the ROIs
For each digit or analog pointer, a ROI must be defined. For each digit or analog pointer, a ROI must be defined.
You can even have multiple independent **Numbers** (eg. electerical meters mostly have 2 numbers for the high and low tariff). You can even have multiple independent **Numbers** (eg. electerical meters mostly have 2 numbers for the high and low tariff).
Depending if you have only one of those types, you can enable/disable `(1)` it on the top left corner: Depending if you have only one of those types, you can enable/disable `(1)` it on the top left corner:
![](img/initial_setup_3_rois.jpg){: style="width:500px"} ![Initial Setup ROI](img/initial_setup_3_rois.jpg){: style="width:500px"}
You can switch between the individual ROIs with the Drop down box `(2)`. You can switch between the individual ROIs with the Drop down box `(2)`.
If you need additional ROIs or delete them you can do this with the control at `(3)`. If you need additional ROIs or delete them you can do this with the control at `(3)`.
@@ -51,20 +52,21 @@ Same for the **analog counters**, the sizes should be identical and the `x` and
!!! Note !!! Note
Don't forget to save the settings with "Save" and do not reboot at this stage. Don't forget to save the settings with "Save" and do not reboot at this stage.
### Analog Counters ### Analog Counters
For analog counters the ROI setting is rather straight forward as the meter is usually quadratic with a clear center. For analog counters the ROI setting is rather straight forward as the meter is usually quadratic with a clear center.
The circle should exactly fit to the outer size of the meter and the cross should be in the middle of the pointer. The circle should exactly fit to the outer size of the meter and the cross should be in the middle of the pointer.
Here is an example with the details for the ROI `ana1`: Here is an example with the details for the ROI `ana1`:
![](img/initial_setup_3_analog_example.jpg){: style="width:500px"} ![Initial Setup Analog Example](img/initial_setup_3_analog_example.jpg){: style="width:500px"}
### Digits ### Digits
For the Digit Meters it is a little bit more complicated, as there are different options of digit models which can be choosen. For the Digit Meters it is a little bit more complicated, as there are different options of digit models which can be choosen.
#### Correct Size for ROI #### Correct Size for ROI
First of all, choose the right size of the ROI. First of all, choose the right size of the ROI.
The configuration of ROIs differs a bit on the selected model (see below). The configuration of ROIs differs a bit on the selected model (see below).
@@ -74,7 +76,6 @@ In [Model Selection](Choosing-the-Model.md) you find the differences between the
Here we only show the different configuration of the ROIs. Here we only show the different configuration of the ROIs.
1. Digit Meters with only recognized full digits (`0, 1, 2, 3, ... 9`) 1. Digit Meters with only recognized full digits (`0, 1, 2, 3, ... 9`)
Suggested Model: `dig-class11-*.tfl` Suggested Model: `dig-class11-*.tfl`
@@ -92,6 +93,7 @@ Here we only show the different configuration of the ROIs.
**Disadvantage:** only limited types of meter types are trained due to the high effort for the training data. **Disadvantage:** only limited types of meter types are trained due to the high effort for the training data.
#### How to setup the digit ROIs perfectly #### How to setup the digit ROIs perfectly
Details and the corresponding "perfect" setting is explained below. Details and the corresponding "perfect" setting is explained below.
For a first run you can choose the following general settings: For a first run you can choose the following general settings:
@@ -100,12 +102,12 @@ For a first run you can choose the following general settings:
| |Example 1|Example 2| | |Example 1|Example 2|
|---|---|---| |---|---|---|
| :heavy_check_mark: **Okay** | ![](img/cont_img_ok.png){: style="width:80px"} | ![](img/wb_okay.jpg){: style="width:80px"} | | :heavy_check_mark: **Okay** | ![Image Okay](img/cont_img_ok.png){: style="width:80px"} | ![WB Okay](img/wb_okay.jpg){: style="width:80px"} |
| :x: **Not** Okay | ![](img/bw_not_okay_small.jpg){: style="width:80px"} | ![](img/wb_not_okay_small.jpg){: style="width:80px"} | | :x: **Not** Okay | ![BW Not Okay Small](img/bw_not_okay_small.jpg){: style="width:80px"} | ![WB Not Okay Small](img/wb_not_okay_small.jpg){: style="width:80px"} |
| :x: **Not** Okay | ![](img/bw_not_okay_big.jpg){: style="width:80px"}| ![](img/wb_not_okay_big.jpg){: style="width:80px"}| | :x: **Not** Okay | ![BW Not Okay Big](img/bw_not_okay_big.jpg){: style="width:80px"}| ![WB Not Okay Big](img/wb_not_okay_big.jpg){: style="width:80px"}|
##### Setup using `dig-class11` models ##### Setup using `dig-class11` models
`dig-class11` - Models recognize the **complete digit only**. Here it is not relevant if the ROI fits the Border of the digit window. `dig-class11` - Models recognize the **complete digit only**. Here it is not relevant if the ROI fits the Border of the digit window.
For this model, there should be a border of `20%` of the image size around the number itself. This border is shown in the ROI setup image by the inner thinner rectangle. This rectangle should fit perfectly around the number when the number has not started to rotate to the next position: For this model, there should be a border of `20%` of the image size around the number itself. This border is shown in the ROI setup image by the inner thinner rectangle. This rectangle should fit perfectly around the number when the number has not started to rotate to the next position:
@@ -114,14 +116,13 @@ For this model, there should be a border of `20%` of the image size around the n
| | Example 1 | Example 2 | | | Example 1 | Example 2 |
| ------------ | --------------------------------- | --------------------------------- | | ------------ | --------------------------------- | --------------------------------- |
| :heavy_check_mark: **Okay** | ![](img/bw_okay.jpg) | ![](img/wb_okay.jpg) | | :heavy_check_mark: **Okay** | ![BW Okay](img/bw_okay.jpg) | ![WB Okay](img/wb_okay.jpg) |
| :x: **Not** Okay | ![](img/bw_not_okay_small.jpg) | ![](img/wb_not_okay_small.jpg) | | :x: **Not** Okay | ![BW Not Okay Small](img/bw_not_okay_small.jpg) | ![WB Not Okay Small](img/wb_not_okay_small.jpg) |
| :x: **Not** Okay | ![](img/bw_not_okay_big.jpg) | ![](img/wb_not_okay_big.jpg) | | :x: **Not** Okay | ![BW Not Okay Big](img/bw_not_okay_big.jpg) | ![WB Not Okay Big](img/wb_not_okay_big.jpg) |
If you have perfect alignment and still are not getting satisfying results, most probably your numbers are not part of the training data yet. See [Collect images to improve the models](Learn-models-with-your-own-images.md) on how to collect new training data. If you have perfect alignment and still are not getting satisfying results, most probably your numbers are not part of the training data yet. See [Collect images to improve the models](Learn-models-with-your-own-images.md) on how to collect new training data.
##### Setup using `dig-class100` or `dig-cont` Models ##### Setup using `dig-class100` or `dig-cont` Models
These models recognize the **tenths** (fractions) between the numbers. These models recognize the **tenths** (fractions) between the numbers.
@@ -144,6 +145,7 @@ Here an example:
| :x: **Not** Okay | <img width="125" alt="dig-class100_NOK" src="https://user-images.githubusercontent.com/412645/199028469-3a69ed31-e5c9-4038-a8dc-6d44a42437ed.png"> | | :x: **Not** Okay | <img width="125" alt="dig-class100_NOK" src="https://user-images.githubusercontent.com/412645/199028469-3a69ed31-e5c9-4038-a8dc-6d44a42437ed.png"> |
### Saving ### Saving
Once you are done, push `Save` to persist your setup. Once you are done, push `Save` to persist your setup.
A reboot is required to apply the changed configuration! A reboot is required to apply the changed configuration!

11
docs/Reference-Image.md
View File

@@ -7,16 +7,16 @@ At first an example image is shown.
To define a new reference image push the button `"Create new Reference" (2)` and afterwards `"Take Image" (2)`. It might take some seconds for processing, then your actual camera image should be shown. To define a new reference image push the button `"Create new Reference" (2)` and afterwards `"Take Image" (2)`. It might take some seconds for processing, then your actual camera image should be shown.
Then play with the provided parameters to get a good result. Then play with the provided parameters to get a good result.
![](img/initial_setup_1_reference_image.jpg){: style="width:500px"} ![Initial Setup Reference Image](img/initial_setup_1_reference_image.jpg){: style="width:500px"}
## Focus ## Focus
This is the first time, where you have access to the camera image. It most likely is out of focus and not sharp! This is the first time, where you have access to the camera image. It most likely is out of focus and not sharp!
Ensure a sharp image of the camera by adjusting the focal length of the ESP OV2640 camera. Ensure a sharp image of the camera by adjusting the focal length of the ESP OV2640 camera.
!!! Note !!! Note
Try to adjust the focus for the clearest possible image! Try to adjust the focus for the clearest possible image!
In order to use it for reading a meter, the focal-length of the OV2640 camera has to be manipulated. In order to use it for reading a meter, the focal-length of the OV2640 camera has to be manipulated.
By default it only results in sharp image for distance bigger than around `~40cm` which is not ideal for our purpose. By default it only results in sharp image for distance bigger than around `~40cm` which is not ideal for our purpose.
@@ -32,9 +32,7 @@ Therefore you need to remove the fixing glue of the OV2640 lens with a sharp kni
!!! Warning !!! Warning
This modification will render the camera unsuitable for general, web-cam type applications unless the focal length is changed back to the original setting. This modification will render the camera unsuitable for general, web-cam type applications unless the focal length is changed back to the original setting.
![](img/focus_adjustment.jpg) ![Focus Adjustment](img/focus_adjustment.jpg)
## Correct Horizontal Alignment ## Correct Horizontal Alignment
@@ -42,7 +40,7 @@ Ensure an **exact horizontal alignment** of the number:
| :heavy_check_mark: Okay | :x: Not Okay | | :heavy_check_mark: Okay | :x: Not Okay |
| --------------------------- | ------------------------------- | | --------------------------- | ------------------------------- |
| ![](img/alignment_okay.jpg) | ![](img/alignment_not_okay.jpg) | | ![Alignment Okay](img/alignment_okay.jpg) | ![Alignment Not Okay](img/alignment_not_okay.jpg) |
!!! Warning !!! Warning
Updating the reference image also means that all alignment images and ROIs needs to be configured again. Updating the reference image also means that all alignment images and ROIs needs to be configured again.
@@ -56,6 +54,7 @@ If everything is done, you can save the result with `"Update Reference Image" (4
As next you should update the [Alignment References](Alignment.md). As next you should update the [Alignment References](Alignment.md).
## Dealing with Reflections ## Dealing with Reflections
Reflections can be caused by the flash LED and make it hard to provide a reliable detection. Reflections can be caused by the flash LED and make it hard to provide a reliable detection.
There are various ways to deal with them: There are various ways to deal with them:

1
docs/Release-creation.md
View File

@@ -1,4 +1,5 @@
# Release Procedure # Release Procedure
Follow these steps to create a release: Follow these steps to create a release:
1. Document all relevant changes in the [Changelog](https://github.com/jomjol/AI-on-the-edge-device/blob/main/Changelog.md) in the `main` branch. 1. Document all relevant changes in the [Changelog](https://github.com/jomjol/AI-on-the-edge-device/blob/main/Changelog.md) in the `main` branch.

43
docs/StatusLED-BlinkCodes.md
View File

@@ -37,8 +37,6 @@ The error code source definition can be found [here](https://github.com/jomjol/A
| OTA_OR_AP | 8 | Soft AP started (for remote config) | 2 | X | OTA_OR_AP | 8 | Soft AP started (for remote config) | 2 | X
| FLASHLIGHT | N/A | LED on when flashlight is on | solid, <br> no blink | FLASHLIGHT | N/A | LED on when flashlight is on | solid, <br> no blink
# ERROR / WARNING # ERROR / WARNING
## Source WLAN_CONN: WLAN disconnected ## Source WLAN_CONN: WLAN disconnected
@@ -48,79 +46,86 @@ The error code source definition can be found [here](https://github.com/jomjol/A
--> General info: [WLAN disconnect reason code description](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/wifi.html#wi-fi-reason-code) --> General info: [WLAN disconnect reason code description](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/wifi.html#wi-fi-reason-code)
### `WLAN disconnected (No Access Point)` ### `WLAN disconnected (No Access Point)`
WLAN connection is interrupted due to no access point in range. WLAN connection is interrupted due to no access point in range.
### `WLAN Disconnected (Authentication failure)` ### `WLAN Disconnected (Authentication failure)`
WLAN connection is interrupted due to an authentication failure. If error repeats check WLAN config in `wlan.ini` (username, password) WLAN connection is interrupted due to an authentication failure. If error repeats check WLAN config in `wlan.ini` (username, password)
### `WLAN Disconnected (Timeout)` ### `WLAN Disconnected (Timeout)`
WLAN connection is interrupted due to an timeout because no beacon from AP is received in a timely manner. Most probably access point is not available anymore or connection is not reliable. WLAN connection is interrupted due to an timeout because no beacon from AP is received in a timely manner. Most probably access point is not available anymore or connection is not reliable.
### `WLAN Disconnected (Further reasons)` ### `WLAN Disconnected (Further reasons)`
WLAN connection is interrupted due to further reasons. Disconnect reason is printed in warining message. Please check serial console output or logfile from sd card (using another device to retrieve logfile /sdcard/log/message/). Please refer to this page to have additional infos in terms of WLAN disconnect reasons --> [WLAN disconnect reason code description](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/wifi.html#wi-fi-reason-code) WLAN connection is interrupted due to further reasons. Disconnect reason is printed in warining message. Please check serial console output or logfile from sd card (using another device to retrieve logfile /sdcard/log/message/). Please refer to this page to have additional infos in terms of WLAN disconnect reasons --> [WLAN disconnect reason code description](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/wifi.html#wi-fi-reason-code)
## Source WLAN_INIT: WLAN initialization ## Source WLAN_INIT: WLAN initialization
!!! NOTE !!! NOTE
All critical errors, regular boot not possible All critical errors, regular boot not possible
### `wlan.ini empty or not readable` ### `wlan.ini empty or not readable`
The `wlan.ini` file is present but content is either not readable or no content present. Please check for further errors in terms of SD card readability or content of `wlan.ini` which is located in /sdcard (most top folder od SD card) The `wlan.ini` file is present but content is either not readable or no content present. Please check for further errors in terms of SD card readability or content of `wlan.ini` which is located in /sdcard (most top folder od SD card)
### `SSID or password empty` ### `SSID or password empty`
The mandatory parameters SSID (name of WIFI network) and / or password is empty. Please configure those parameters in `wlan.ini` and try again. The mandatory parameters SSID (name of WIFI network) and / or password is empty. Please configure those parameters in `wlan.ini` and try again.
### `WIFI init error (details console)` ### `WIFI init error (details console)`
A general WIFI initialization error occured. Please check serial console output or logfile from sd card (using another device to retrieve logfile /sdcard/log/message/) A general WIFI initialization error occured. Please check serial console output or logfile from sd card (using another device to retrieve logfile /sdcard/log/message/)
## Source SDCARD_INIT: SD card initialization ## Source SDCARD_INIT: SD card initialization
!!! NOTE !!! NOTE
All critical errors, regular boot not possible All critical errors, regular boot not possible
### `SD card filesystem mount failed` ### `SD card filesystem mount failed`
Failed to mount FAT filesystem on SD card. Check SD card filesystem (only FAT supported) or try another card. Possible further infos: Please check serial console output. Failed to mount FAT filesystem on SD card. Check SD card filesystem (only FAT supported) or try another card. Possible further infos: Please check serial console output.
### `SD card not found (Error code 0x107)` ### `SD card not found (Error code 0x107)`
SD card init failed. Check if SD card is properly inserted into SD card slot or try another card. Possible further infos: Please check serial console output. SD card init failed. Check if SD card is properly inserted into SD card slot or try another card. Possible further infos: Please check serial console output.
### `SD card init failed (details console)` ### `SD card init failed (details console)`
A general SD card initialization error occured. Please check serial console output. A general SD card initialization error occured. Please check serial console output.
## Source SDCARD_CHECK: SD card basic check ## Source SDCARD_CHECK: SD card basic check
!!! NOTE !!! NOTE
All critical errors, normal boot not possible. Reduced WebUI is going to be loaded for further diagnostic possibilities or redo firmware update. All critical errors, normal boot not possible. Reduced WebUI is going to be loaded for further diagnostic possibilities or redo firmware update.
### `File creation / write error` ### `File creation / write error`
A basic check of SD card is performed at boot. Failed to create the test file or writing content to the file failed. Most likely SD card is defective or not supported. Please check logs with log viewer in reduced web interface, serial console output or try another card. A basic check of SD card is performed at boot. Failed to create the test file or writing content to the file failed. Most likely SD card is defective or not supported. Please check logs with log viewer in reduced web interface, serial console output or try another card.
Recommendation: Format or try another card Recommendation: Format or try another card
### `File read / CRC verfication error` ### `File read / CRC verfication error`
A basic check of SD card is performed at boot. Failed to read the test file or CRC of read back content failed. Most likely SD card is defective. Please check logs with log viewer in reduced web interface or serial console output for further error indication or try another card. A basic check of SD card is performed at boot. Failed to read the test file or CRC of read back content failed. Most likely SD card is defective. Please check logs with log viewer in reduced web interface or serial console output for further error indication or try another card.
Recommendation: Format or try another card Recommendation: Format or try another card
### `File delete error` ### `File delete error`
A basic check of SD card is performed at boot. Failed to delelte the test file. Most likely SD card is defective. Please check logs with log viewer in reduced web interface or serial console output for further error indication or try another card. A basic check of SD card is performed at boot. Failed to delelte the test file. Most likely SD card is defective. Please check logs with log viewer in reduced web interface or serial console output for further error indication or try another card.
Recommendation: Format or try another card Recommendation: Format or try another card
### `Folder / File presence failed` ### `Folder / File presence failed`
A basic check of SD card is performed at boot. One or more menadatory folder / file are not found on SD card. Please check logs with log viewer in reduced web interface or serial console output for further error indication. A basic check of SD card is performed at boot. One or more menadatory folder / file are not found on SD card. Please check logs with log viewer in reduced web interface or serial console output for further error indication.
Recommendation: Repeat installation using AI-on-the-edge-device__update__*.zip Recommendation: Repeat installation using AI-on-the-edge-device__update__*.zip
## Source CAM_INIT: Camera initialization ## Source CAM_INIT: Camera initialization
### `Camera init failed (details console)` ### `Camera init failed (details console)`
!!! NOTE !!! NOTE
@@ -131,29 +136,30 @@ A general camera initialization error occured. Please check logs with log viewer
Recommendation: Check for proper electrical connection, whether camera model is supported and whether power supply is sufficient. Recommendation: Check for proper electrical connection, whether camera model is supported and whether power supply is sufficient.
### `Camera framebuffer check failed` ### `Camera framebuffer check failed`
The framebuffer of the camera was not readable. The firmware will trying to continue regular boot, but further errors can occur which block regular processing. Please check logs with logfile viewer if processing is behaving irregular. The framebuffer of the camera was not readable. The firmware will trying to continue regular boot, but further errors can occur which block regular processing. Please check logs with logfile viewer if processing is behaving irregular.
Recommendation: Check for proper electrical commenection, wether camera model is supported and wether power supply is suffcient. Recommendation: Check for proper electrical commenection, wether camera model is supported and wether power supply is suffcient.
## Source PSRAM_INIT: External RAM (SPI RAM) initialization ## Source PSRAM_INIT: External RAM (SPI RAM) initialization
!!! NOTE !!! NOTE
A critical errors, normal boot not possible. Reduced WebUI is going to be loaded for further diagnostic possibilities or redo firmware update. A critical errors, normal boot not possible. Reduced WebUI is going to be loaded for further diagnostic possibilities or redo firmware update.
### `SPI RAM init failed: Not found/defective` ### `SPI RAM init failed: Not found/defective`
External RAM (SPI RAM) initialization failed. Most likely external RAM not accessable or defective. Normal operation is not possible without having external RAM. External RAM (SPI RAM) initialization failed. Most likely external RAM not accessable or defective. Normal operation is not possible without having external RAM.
### `External SPI RAM < 4MB` ### `External SPI RAM < 4MB`
External RAM (SPI RAM) initialization successful, but external RAM size is too small. A size of >= 4MB is necessary to run this firmware. External RAM (SPI RAM) initialization successful, but external RAM size is too small. A size of >= 4MB is necessary to run this firmware.
### `Total heap < 4MB` ### `Total heap < 4MB`
Total available system memory (heap) is too small. A size of >= 4MB is necessary to run this firmware. Total available system memory (heap) is too small. A size of >= 4MB is necessary to run this firmware.
## Source TIME_CHECK: External RAM (SPI RAM) initialization ## Source TIME_CHECK: External RAM (SPI RAM) initialization
### `Missing time sync (check every round)` ### `Missing time sync (check every round)`
!!! NOTE !!! NOTE
@@ -161,8 +167,6 @@ Total available system memory (heap) is too small. A size of >= 4MB is necessary
If system is configured to be synced with a NTP server the sync status is checked after every round (in state: "Flow finished". An warming message is also printed to log). If the time is not synced after serveral rounds, please check for proper configuration. If system is configured to be synced with a NTP server the sync status is checked after every round (in state: "Flow finished". An warming message is also printed to log). If the time is not synced after serveral rounds, please check for proper configuration.
# STATUS # STATUS
!!! NOTE !!! NOTE
@@ -171,14 +175,15 @@ If system is configured to be synced with a NTP server the sync status is checke
## Source OTA_OR_AP: OTA Update / Access point mode ## Source OTA_OR_AP: OTA Update / Access point mode
### `OTA process ongoing` ### `OTA process ongoing`
An OTA is performed right now. Please wait until OTA is completed. System is rebooting automatically. If system is not coming up, please check serial console output. An OTA is performed right now. Please wait until OTA is completed. System is rebooting automatically. If system is not coming up, please check serial console output.
### `Soft AP started (for remote config)` ### `Soft AP started (for remote config)`
The built-in access point functionality is started to perform initial remote remote setup. Further description: [Installtion --> `Section Remote Setup using the built-in Access Point`](https://jomjol.github.io/AI-on-the-edge-device-docs/Installation/) The built-in access point functionality is started to perform initial remote remote setup. Further description: [Installtion --> `Section Remote Setup using the built-in Access Point`](https://jomjol.github.io/AI-on-the-edge-device-docs/Installation/)
## Source FLASHLIGHT: Flashlight ## Source FLASHLIGHT: Flashlight
### `LED on when flashlight is on` ### `LED on when flashlight is on`
The LED is solid on as long the flashlight is on. This feature has lower priority than the other LED codes. Whenever another code occurs this feature will be overrided. The LED is solid on as long the flashlight is on. This feature has lower priority than the other LED codes. Whenever another code occurs this feature will be overrided.

3
docs/Testing.md
View File

@@ -28,17 +28,14 @@ You will need a testing device. best with usb adapter. Before you upload your te
<img width="300" alt="image" src="https://user-images.githubusercontent.com/412645/209813215-e0ea7405-6ff4-48d0-8dab-97bfab6962af.png"> <img width="300" alt="image" src="https://user-images.githubusercontent.com/412645/209813215-e0ea7405-6ff4-48d0-8dab-97bfab6962af.png">
Now you can use Visual Studio Code or a standard console to upload the test code. In VS Code (tab platformio) open _Advanced_ and select _Test_. Now you can use Visual Studio Code or a standard console to upload the test code. In VS Code (tab platformio) open _Advanced_ and select _Test_.
<img width="467" alt="image" src="https://user-images.githubusercontent.com/412645/209813917-ea7fca50-2553-4acf-a8af-ecdac84a01ea.png"> <img width="467" alt="image" src="https://user-images.githubusercontent.com/412645/209813917-ea7fca50-2553-4acf-a8af-ecdac84a01ea.png">
Alternatively you can run it in console/terminal with `platformio test --environment esp32cam`. Alternatively you can run it in console/terminal with `platformio test --environment esp32cam`.
In my environment the serial terminal not opens. I have to do it for myself. You will see much logging. If any test fails it logs it out. Else it logs all test passed in the end. In my environment the serial terminal not opens. I have to do it for myself. You will see much logging. If any test fails it logs it out. Else it logs all test passed in the end.
## Troubleshooting ## Troubleshooting
If you test very much cases in one function, the device runs in Stack Overflow and an endless boot. Reduce the count of test cases or split the test function in multiple functions. If you test very much cases in one function, the device runs in Stack Overflow and an endless boot. Reduce the count of test cases or split the test function in multiple functions.

2
docs/Upload-files-by-script.md
View File

@@ -1,5 +1,7 @@
# Scripted File Upload # Scripted File Upload
To upload a file e.g. using `curl`, you first have to delete it and then upload it: To upload a file e.g. using `curl`, you first have to delete it and then upload it:
``` ```
curl -d '' http://192.168.1.153/delete/html/index.html curl -d '' http://192.168.1.153/delete/html/index.html
curl --data-binary @ota_page.html http://192.168.1.153/upload/html/index.html curl --data-binary @ota_page.html http://192.168.1.153/upload/html/index.html

3
docs/Watermeter-specific-analog---digit-transition.md
View File

@@ -11,7 +11,6 @@ Often the last digit "hangs" a bit on this devices and comes not over zero. So i
<img width="122" alt="image" src="https://user-images.githubusercontent.com/412645/209808192-5ff67e9f-ea7c-4d82-a8e4-54b3643c7e24.png"> <img width="122" alt="image" src="https://user-images.githubusercontent.com/412645/209808192-5ff67e9f-ea7c-4d82-a8e4-54b3643c7e24.png">
<img width="122" alt="image" src="https://user-images.githubusercontent.com/412645/209808306-359cce2e-ec84-4390-82d1-6747e1ec056c.png"> <img width="122" alt="image" src="https://user-images.githubusercontent.com/412645/209808306-359cce2e-ec84-4390-82d1-6747e1ec056c.png">
## Early transition ## Early transition
Some units start the transition very early or run with the analogue pointer. In the third example, is it a 3 or a 2? Some units start the transition very early or run with the analogue pointer. In the third example, is it a 3 or a 2?
@@ -20,12 +19,10 @@ Some units start the transition very early or run with the analogue pointer. In
<img width="122" alt="image" src="https://user-images.githubusercontent.com/412645/209808972-448bb6d0-7b7e-4030-abb2-9c966ceffc4a.png"> <img width="122" alt="image" src="https://user-images.githubusercontent.com/412645/209808972-448bb6d0-7b7e-4030-abb2-9c966ceffc4a.png">
<img width="122" alt="image" src="https://user-images.githubusercontent.com/412645/209809116-d4acc5f2-ab5c-4304-9559-598b1dfc59c2.png"> <img width="122" alt="image" src="https://user-images.githubusercontent.com/412645/209809116-d4acc5f2-ab5c-4304-9559-598b1dfc59c2.png">
## Inaccuracies in image recognition ## Inaccuracies in image recognition
The models for image recognition are good, but have inaccuracies in the range +/- 0.2. In order to obtain as many correct results as possible, a treatment is carried out in the post process in the range of 9.8-0.2 for the analogue pointer, which must start differently depending on the type of counter. The models for image recognition are good, but have inaccuracies in the range +/- 0.2. In order to obtain as many correct results as possible, a treatment is carried out in the post process in the range of 9.8-0.2 for the analogue pointer, which must start differently depending on the type of counter.
## How to configure for my meter type ## How to configure for my meter type
If you have a devices with "normal" transition you should not have any issues. On devices with "early" transition, you can set the option `AnalogToDigitTransitionStart` to a value between 6 and 8. If you have a devices with "normal" transition you should not have any issues. On devices with "early" transition, you can set the option `AnalogToDigitTransitionStart` to a value between 6 and 8.

2
docs/Webhook.md
View File

@@ -1,4 +1,5 @@
# Webhook # Webhook
## Purpose ## Purpose
The idea behind this webhook feature is to provide an alternative to MQTT and InfluxDB for transmitting data, such as water meter readings from a vacation home, to a standard PHP webspace. The idea behind this webhook feature is to provide an alternative to MQTT and InfluxDB for transmitting data, such as water meter readings from a vacation home, to a standard PHP webspace.
@@ -141,6 +142,7 @@ if ($method === 'POST') {
``` ```
## Basic Example of a server using Python ## Basic Example of a server using Python
```Python ```Python
from flask import Flask, request, jsonify from flask import Flask, request, jsonify
import csv import csv

9
docs/collect-new-images.md
View File

@@ -1,21 +1,24 @@
# Collect images to improve the models # Collect images to improve the models
If your device has new, different digits or pointers it might be that the existing models don't recognize them well. In such case you can collect your images and so we can train the model better. This helps you and also others as the models get more accurate. Adding more images also helps if you have a model that is already known, but the neural models do not produce good results. If your device has new, different digits or pointers it might be that the existing models don't recognize them well. In such case you can collect your images and so we can train the model better. This helps you and also others as the models get more accurate. Adding more images also helps if you have a model that is already known, but the neural models do not produce good results.
Experienced users can do the training also by themselves, see [Learn a model with your own images](Learn-models-with-your-own-images.md). Experienced users can do the training also by themselves, see [Learn a model with your own images](Learn-models-with-your-own-images.md).
## Before you start ## Before you start
Before you go ahead, please check if your digits/pointers are not yet contained in the training data. A visual overview is available at [digits](https://jomjol.github.io/neural-network-digital-counter-readout) resp. [pointers](https://jomjol.github.io/neural-network-analog-needle-readout/). Before you go ahead, please check if your digits/pointers are not yet contained in the training data. A visual overview is available at [digits](https://jomjol.github.io/neural-network-digital-counter-readout) resp. [pointers](https://jomjol.github.io/neural-network-analog-needle-readout/).
Poor recognition is often caused by blurred images, low contrast or incorrect setting of the ROIs. Therefore, check these possibilities first, as additional training will bring little improvement here. See [ROI Configuration](ROI-Configuration.md) for details. Poor recognition is often caused by blurred images, low contrast or incorrect setting of the ROIs. Therefore, check these possibilities first, as additional training will bring little improvement here. See [ROI Configuration](ROI-Configuration.md) for details.
## Collecting images ## Collecting images
The neural network is trained based on a set of images that have already been collected over time. If your digits are included or at least very similar to included images, the chance is very high that the neural network is working fine for you as well. The neural network is trained based on a set of images that have already been collected over time. If your digits are included or at least very similar to included images, the chance is very high that the neural network is working fine for you as well.
The neural network configuration is stored in the TensorFlow Lite format as `*.tfl` or `*.tflite` in the `/config` directory on the SD card. A model can be updated (or a new one added) by uploading the new file and activating it on the configuration page or in the config file `/config/config.ini`. The neural network configuration is stored in the TensorFlow Lite format as `*.tfl` or `*.tflite` in the `/config` directory on the SD card. A model can be updated (or a new one added) by uploading the new file and activating it on the configuration page or in the config file `/config/config.ini`.
In order to incorporate new digits a training set of images is required. The training images needs to be collected in the final setup with the help of the `Digits` or `Analog` log settings (not to be confused with the `Data` or `Debug` log). Enable the logging of the images on the configuration page or in the config file `/config/config.ini`: In order to incorporate new digits a training set of images is required. The training images needs to be collected in the final setup with the help of the `Digits` or `Analog` log settings (not to be confused with the `Data` or `Debug` log). Enable the logging of the images on the configuration page or in the config file `/config/config.ini`:
![](img/enable_log_image.jpg) ![Enable Log Image](img/enable_log_image.jpg)
Now be patient! You have to wait until it has collected an image of each digit of every type. They will be placed on the SD card in the folder `/log/digit/` resp. `/log/analog/`. Now be patient! You have to wait until it has collected an image of each digit of every type. They will be placed on the SD card in the folder `/log/digit/` resp. `/log/analog/`.
@@ -24,18 +27,20 @@ After some days, there will be a lot of images, many of them very similar. Becau
The tools shown below can help you with that. The tools shown below can help you with that.
## Collecting images for dig-class100/dig-cont/ana-class100 ## Collecting images for dig-class100/dig-cont/ana-class100
For digits use [Collectmeterdigits](https://github.com/haverland/collectmeterdigits) resp. for pointers use [collectmeteranalog](https://github.com/haverland/collectmeteranalog) to fetch the images from the device and select a subset of them. Please read the detailed instructions on the mentioned links for details! For digits use [Collectmeterdigits](https://github.com/haverland/collectmeterdigits) resp. for pointers use [collectmeteranalog](https://github.com/haverland/collectmeteranalog) to fetch the images from the device and select a subset of them. Please read the detailed instructions on the mentioned links for details!
If the fetching of the images is too slow for you, a faster way to get the images to your PC is to remove the SD-card from the ESP32 module and insert it into the card reader of yur PC. If the fetching of the images is too slow for you, a faster way to get the images to your PC is to remove the SD-card from the ESP32 module and insert it into the card reader of yur PC.
Then search for two..three images of each digit (**not more! :-)**). You will have to make sure to label the images yourself matching the effective value they are supposed to show. Then search for two..three images of each digit (**not more! :-)**). You will have to make sure to label the images yourself matching the effective value they are supposed to show.
## Share your images ## Share your images
In most cases we will integrate your images in the training dataset of the models. Only if we fear a degradation of the models or you need a different behavior, we might not include the data in the standard models (see at bottom of page for reasons). In most cases we will integrate your images in the training dataset of the models. Only if we fear a degradation of the models or you need a different behavior, we might not include the data in the standard models (see at bottom of page for reasons).
To provide your images to us for training the model, open an [Github Issue](https://github.com/jomjol/AI-on-the-edge-device/issues/new/choose) and append the zipped images ito it. To provide your images to us for training the model, open an [Github Issue](https://github.com/jomjol/AI-on-the-edge-device/issues/new/choose) and append the zipped images ito it.
### Images can be rejected if ### Images can be rejected if
* You provide too many images. More than 1000 images of your device are really to much. * You provide too many images. More than 1000 images of your device are really to much.
* Images which are not good enough (see [ROI Configuration](ROI-Configuration.md)) will be rejected. It would reduce the accuracy of the networks. * Images which are not good enough (see [ROI Configuration](ROI-Configuration.md)) will be rejected. It would reduce the accuracy of the networks.
* Images with too little focus will be rejected. * Images with too little focus will be rejected.

1
docs/data-logging.md
View File

@@ -6,5 +6,6 @@ the results of every round gets written to the SD-Card.
The data files are stored in `/log/data` on the SD-Card. The data files are stored in `/log/data` on the SD-Card.
## Data Format ## Data Format
The data is stored as CSV with the following columns: The data is stored as CSV with the following columns:
`time`, `name-of-number`, `raw-value`, `return-value`, `pre-value`, `change-rate`, `change-absolute`, `error-text`, `cnn-digit`, `cnn-analog` `time`, `name-of-number`, `raw-value`, `return-value`, `pre-value`, `change-rate`, `change-absolute`, `error-text`, `cnn-digit`, `cnn-analog`

15
docs/index.md
View File

@@ -8,6 +8,7 @@ Artificial intelligence based systems have been established in our everyday live
Here this edge computing is brought into a practice-oriented example, where a AI network is implemented on a ESP32 device so: **AI on the edge**. Here this edge computing is brought into a practice-oriented example, where a AI network is implemented on a ESP32 device so: **AI on the edge**.
## Key features ## Key features
- Tensorflow Lite (TFlite) integration - including easy to use wrapper - Tensorflow Lite (TFlite) integration - including easy to use wrapper
- Inline image processing (feature detection, alignment, ROI extraction) - Inline image processing (feature detection, alignment, ROI extraction)
- **Small** and **cheap** device (3x4.5x2 cm³, < 10 EUR) - **Small** and **cheap** device (3x4.5x2 cm³, < 10 EUR)
@@ -23,22 +24,19 @@ Here this edge computing is brought into a practice-oriented example, where a AI
![Idea](img/idea.jpg){: style="width:600px"} ![Idea](img/idea.jpg){: style="width:600px"}
### Hardware ### Hardware
![](img/watermeter_all.jpg){: style="width:200px"} ![Watermeter All](img/watermeter_all.jpg){: style="width:200px"}
![](img/main.jpg){: style="width:200px"} ![Main](img/main.jpg){: style="width:200px"}
![](img/size.png){: style="width:200px"} ![Size](img/size.png){: style="width:200px"}
### Web interface ### Web interface
![](img/watermeter.jpg){: style="width:600px"} ![Watermeter](img/watermeter.jpg){: style="width:600px"}
### Configuration Interface ### Configuration Interface
![](img/edit_reference.jpg){: style="width:600px"} ![Edit Reference](img/edit_reference.jpg){: style="width:600px"}
**Have fun in studying the new possibilities and ideas** **Have fun in studying the new possibilities and ideas**
@@ -53,6 +51,7 @@ There are two types of CNN implemented, a classification network for reading the
This project is an evolution of the [water-meter-system-complete](https://github.com/jomjol/water-meter-system-complete), which uses ESP32-CAM just for taking the image and a 1GB-Docker image to run the neural network's backbone. Here everything is integrated in an ESP32-CAM module with 8MB of RAM and a SD card as data storage. This project is an evolution of the [water-meter-system-complete](https://github.com/jomjol/water-meter-system-complete), which uses ESP32-CAM just for taking the image and a 1GB-Docker image to run the neural network's backbone. Here everything is integrated in an ESP32-CAM module with 8MB of RAM and a SD card as data storage.
# Additional Tutorials # Additional Tutorials
A lot of people created useful Youtube videos which might help you getting started. A lot of people created useful Youtube videos which might help you getting started.
Here a small selection: Here a small selection:

9
docs/initial-setup.md
View File

@@ -2,7 +2,7 @@
After setting up the device (firmware, SD card, WLAN) the device will connect to the wifi access point and start in an initial setup configuration: After setting up the device (firmware, SD card, WLAN) the device will connect to the wifi access point and start in an initial setup configuration:
![](img/setup_initial_welcome.png){: style="width:500px"} ![Setup Initial Welcome](img/setup_initial_welcome.png){: style="width:500px"}
With the buttons on the top you can navigate through 5 steps which guide you through the necessary setup: With the buttons on the top you can navigate through 5 steps which guide you through the necessary setup:
@@ -18,15 +18,16 @@ All settings can be accessed later in the normal operation mode.
Don't forget to save each step with "Save", and do not reboot at this stage. Don't forget to save each step with "Save", and do not reboot at this stage.
## Finish the Setup and change to the Normal Operation mode ## Finish the Setup and change to the Normal Operation mode
With the last step `(1)` you leave the **Setup Mode** and reboot to the **Normal Operation mode**. With the last step `(1)` you leave the **Setup Mode** and reboot to the **Normal Operation mode**.
![](img/initial_setup_6_finish_reboot.jpg){: style="width:500px"} ![Initial Setup Finish Boot](img/initial_setup_6_finish_reboot.jpg){: style="width:500px"}
## Access to the Setup Pages in the Normal Operation mode ## Access to the Setup Pages in the Normal Operation mode
You always can access all the settings during the normal operation mode via the `Settings` menu: You always can access all the settings during the normal operation mode via the `Settings` menu:
![](img/initial_setup_7_access_normal_mode.jpg){: style="width:500px"} ![Initial Setup Normal Mode](img/initial_setup_7_access_normal_mode.jpg){: style="width:500px"}
- `(1)` Access to the [General Settings](Configuration.md). - `(1)` Access to the [General Settings](Configuration.md).
- `(2)` Update of the [Reference Image](Reference-Image.md). - `(2)` Update of the [Reference Image](Reference-Image.md).

5
docs/ota.md
View File

@@ -1,4 +1,5 @@
# Over-The-Air (OTA) Update # Over-The-Air (OTA) Update
You can do an OTA (over-the-air) update via the Web Interface. You can do an OTA (over-the-air) update via the Web Interface.
Grab the firmware from the Grab the firmware from the
@@ -6,14 +7,15 @@ Grab the firmware from the
* [Automatically build development branch](https://github.com/jomjol/AI-on-the-edge-device/actions?query=branch%3Amain) (experimental, untested versions). Please inform yourself on [Living on the Edge](unstable-installation.md) first! * [Automatically build development branch](https://github.com/jomjol/AI-on-the-edge-device/actions?query=branch%3Amain) (experimental, untested versions). Please inform yourself on [Living on the Edge](unstable-installation.md) first!
## Update Procedure ## Update Procedure
1. Create a backup of your configuration. Either use the Backup/Restore function of your device for this (menu `System > Backup/Restore`) or back the files manually up using the File Server (menu `File Server`, folder `config`). It is recommended to at least save the config file `config.ini`! 1. Create a backup of your configuration. Either use the Backup/Restore function of your device for this (menu `System > Backup/Restore`) or back the files manually up using the File Server (menu `File Server`, folder `config`). It is recommended to at least save the config file `config.ini`!
1. Head to the menu `System > OTA Update` and follow the instructions there. 1. Head to the menu `System > OTA Update` and follow the instructions there.
If you do an update between major versions, it might be needed to modify the config file `config.ini` as it's syntax or context has changed. If you do an update between major versions, it might be needed to modify the config file `config.ini` as it's syntax or context has changed.
To do so, go to the menu `Settings > Configuration` (after the update completed and the device restarted) and check if it warns you about an unset parameter. To do so, go to the menu `Settings > Configuration` (after the update completed and the device restarted) and check if it warns you about an unset parameter.
### Update from version `v12.0.0` or newer ### Update from version `v12.0.0` or newer
You can use the over the air update mechanism, which uploads the update via a ZIP files. You can use the over the air update mechanism, which uploads the update via a ZIP files.
The update file is located on the [release page](https://github.com/jomjol/AI-on-the-edge-device/releases). Please choose the zip file with the following naming: `AI-on-the-edge-device__update__*.zip` The update file is located on the [release page](https://github.com/jomjol/AI-on-the-edge-device/releases). Please choose the zip file with the following naming: `AI-on-the-edge-device__update__*.zip`
@@ -21,6 +23,7 @@ The update file is located on the [release page](https://github.com/jomjol/AI-on
Go to the menu `System --> OTA Update` and follow the instructions there. After a final automatic reboot you should have the new version running. Go to the menu `System --> OTA Update` and follow the instructions there. After a final automatic reboot you should have the new version running.
### Update from version older than `v12.0.0` ### Update from version older than `v12.0.0`
If you update from an version older than 12.0.1, you should firstly update to version 12.0.1. Background are not fully backward compatible changes in the `config.ini`, that are taken care of in this version. If you update from an version older than 12.0.1, you should firstly update to version 12.0.1. Background are not fully backward compatible changes in the `config.ini`, that are taken care of in this version.
!!! Warning !!! Warning

10
docs/outdated--Integrated-Functions.md
View File

@@ -11,8 +11,6 @@ This is the main purpose of this device. It returns the converted image as a num
Details can be found here: tbd Details can be found here: tbd
## Picture Server ## Picture Server
```http://IP-ESP32/capture``` ```http://IP-ESP32/capture```
@@ -23,8 +21,6 @@ This is a implementation of the camera interface of https://github.com/jomjol/wa
It is fully compatible including the parameters (```quality```=..., `size=...` ) . This allows to use this ESP32 system in parallel to the corresponding docker system: https://github.com/jomjol/water-meter-system-complete, from which this project is basically the successor. It is fully compatible including the parameters (```quality```=..., `size=...` ) . This allows to use this ESP32 system in parallel to the corresponding docker system: https://github.com/jomjol/water-meter-system-complete, from which this project is basically the successor.
## File server ## File server
Access: ```http://IP-ESP32/fileserver/``` Access: ```http://IP-ESP32/fileserver/```
@@ -35,16 +31,12 @@ The usage is self explaining. The file path or file can directly be accessed by
Example for ```config.ini``` : ```http://IP-ESP/fileserver/config/config.ini``` Example for ```config.ini``` : ```http://IP-ESP/fileserver/config/config.ini```
## OTA-Update ## OTA-Update
```http://IP-ESP32/ota?file=firmware.bin``` ```http://IP-ESP32/ota?file=firmware.bin```
Here an over the air update can be triggered. The firmware file is expected to be located in the sub directory ```/firmware/``` and can be uploaded with the file server. By the parameter ```file``` the name of the firmware file needs to be given. Here an over the air update can be triggered. The firmware file is expected to be located in the sub directory ```/firmware/``` and can be uploaded with the file server. By the parameter ```file``` the name of the firmware file needs to be given.
## Reboot ## Reboot
```http://IP-ESP32/reboot``` ```http://IP-ESP32/reboot```
@@ -53,8 +45,6 @@ A reboot with a delay of 5 seconds is initiated, e.g. after firmware update.
**ATTENTION**: currently this is not working properly - hardware power off is needed instead. **Work in progress!** **ATTENTION**: currently this is not working properly - hardware power off is needed instead. **Work in progress!**
## Simple Web Server ## Simple Web Server
If none of the above URLs are fitting, a very simple web server checks, if there is a fitting file from the sub directory ```/html``` If none of the above URLs are fitting, a very simple web server checks, if there is a fitting file from the sub directory ```/html```

4
docs/unstable-installation.md
View File

@@ -8,7 +8,7 @@
Ok, then grab the latest `main` build from [Github Actions](https://github.com/jomjol/AI-on-the-edge-device/actions/workflows/build.yaml?query=branch%3Amain) Page and proceed as following: Ok, then grab the latest `main` build from [Github Actions](https://github.com/jomjol/AI-on-the-edge-device/actions/workflows/build.yaml?query=branch%3Amain) Page and proceed as following:
1. Pick the most top successful (green) build: 1. Pick the most top successful (green) build:
![](actions.png) ![Actions](actions.png)
2. Scroll down and download the `AI-on-the-edge-device__update__*.zip`: 2. Scroll down and download the `AI-on-the-edge-device__update__*.zip`:
![](update-artifact.png) ![Update Artifact](update-artifact.png)
5. Flash the zip file using the [OTA Update](ota.md) page of your device. 5. Flash the zip file using the [OTA Update](ota.md) page of your device.

13
requirements.txt Normal file
View File

@@ -0,0 +1,13 @@
mkdocs
mkdocs-gen-files
mkdocs-awesome-pages-plugin
mkdocs-material
pymdown-extensions
mkdocs-enumerate-headings-plugin
# Linting
pymarkdownlnt
# Vale dependencies
rst2html
vale