Update descriptions of post process settings (#24)

* Describe AllowNegativeRates

* Describe AllowNegativeRates

* Move Configuration details to Correction Algorithm.md

* Describe 'IgnoreLeadingNan'

* described Pre-Value and Rate to high in FAQ #23

* Update docs/Choosing-the-Model.md

---------

Co-authored-by: CaCO3 <caco3@ruinelli.ch>
This commit is contained in:
Frank Haverland
2023-03-13 21:04:35 +01:00
committed by GitHub
parent 0ff9904f1a
commit 9bd60edcfa
4 changed files with 72 additions and 12 deletions

View File

@@ -14,14 +14,16 @@ For digits on water meters, gas-meters or power meters you can select between tw
### dig-class11
This model can recognize full digits. All intermediate states shown a "N" for not a number. But in post process it uses older values to fill up the "N" values if possible.
This model can recognize full digits. It was the first model version. All intermediate states shown a "N" for not a number. But in post process it uses older values to fill up the "N" values if possible.
![](img/dig-class11.png){: style="width:300px"}
It's possibly a good fallback, if `dig-cont/dig-class100` results are not good.
#### Main features
* well suited for LCD digits
* with the ExtendedResolution option is not supported. (Only in conjunction with ana-class100 / ana-cont)
* the ExtendedResolution option is not supported. (Only in conjunction with ana-class100 / ana-cont)
### dig-class100 / dig-cont
@@ -37,10 +39,16 @@ These models are used to get a continuous reading with intermediate states. To s
* Advantage over dig-class11 that results continue to be calculated in the transition between digits.
* With the ExtendedResolution option, higher accuracy is possible by adding another digit.
Look [here](https://jomjol.github.io/neural-network-digital-counter-readout) for a list of digit images used for the training
Look [here](https://jomjol.github.io/neural-network-digital-counter-readout) for a list of digit images used for the training.
#### dig-class100 vs. dig-cont
The difference is in the internal processing. Take the one that gives you the best results.
The difference is in the internal processing.
The dig-class100 is a standard classification model. Each tenth step is an output.
dig-cont uses two outputs and arctangent to get the result. You see very complicated.
Try both models on your device and take the one that gives you the best results.
## Analog pointer models
@@ -58,13 +66,16 @@ For pointers on water meters use the analog models. You can only choose between
Look [here](https://jomjol.github.io/neural-network-analog-needle-readout/) for a list of pointer images used for the training
#### ana-class100 vs. ana-cont
The difference is in the internal processing. Take the one that gives you the best results. Both models learn from the same data.
The difference is in the internal processing.
Take the one that gives you the best results. Both models learn from the same data.
## Different types of models (normal vs. quantized)
The normally trained network is calculating with internal floating point numbers. The saving of floating point numbers naturally takes more space than an integer type. Often the increased accuracy is not needed. Therefore there is the option, to "quantize" a neural network. In this case the internal values are rescaled to integer values, which is called "quantization". The stored tflite files are usually much smaller.
The normally trained network is calculating with internal floating point numbers. The saving of floating point numbers naturally takes more space than an integer type. Often the increased accuracy is not needed. Therefore there is the option, to "quantize" a neural network. In this case the internal values are rescaled to integer values, which is called "quantization". The stored tflite files are usually much smaller and runs faster on the edgeAI-device.
Usually the models are distrusted therefore in both versions. They can be distinguished by a "-q" at the end of the logfile.
#### Example:

View File

@@ -53,6 +53,8 @@ Neural network evaluation of analog counter. The neural network is defined by a
#### 5. ``[PostProcessing]``
Summarized the individually converted pictures to the overall result. It also implements some error corrections and consistency checks to filter wrong reading.
For more details look at [Correction Algorithm](Correction%20Algorithm.md)).
#### 6. ``[MQTT]``
Transfer of the readings to a MQTT server.

View File

@@ -14,10 +14,7 @@ There are several reasons, that a check might be necessary:
##### PreValue
The last correct read value. Either from a previous correctly identified value or manual setting by the user.
This is used to replace "N"s and make a check for the absolute change.
The last correct read value. PreValue is here a bit missleading, because normally it is the same as the last value. In the next round of reading it will be used to check nagtive rates, high rates (MaxRateValue / MaxRateType) and CCheckDigitIncreaseConsistency (dig-class11 only). Either from a previous correctly identified value or manual setting by the user.
##### Digits
@@ -40,11 +37,25 @@ As you can see, the 17.9 is a false reading as the 7 is assumed to be already re
A detailed description of the algorithm can be found below (not yet ready!)
##### Negative Rate allowed
##### ``AllowNegativeRates``
Most of the meters only have increasing numbers and do not count backwards. Therefore a negative rate (= negative change compared to the PreValue) is surely a false value. This can be checked an flagged as false reading
##### MaxRateValue / MaxRateType
The ``AllowNegativeRates`` property ensures that the result does not become negative in the event of incorrect readings. This can happen, for example, if the alignment of the image did not work properly. But also the neural networks sometimes flip between two states for similar images.
If ``AllowNegativeRates = true``, the result is discarded if it is smaller than the pre-value of the last readout and the output shows an error "Negative rate.."
If the ``ExtendedResolution`` setting is true, there is an exception where the value does not become smaller, but no error is output. This only applies if the value +/- 0.2 of the last digit is inaccurate.
###### ``DecimalShift``
The ``DecimalShift`` setting puts the decimal point in the right place. It acts like a shift. Negative values shift the decimal point to the left. Positive values move the decimal point to the right, filling with zero.
###### ``AnalogDigitalTransitionStart``
For detailed description see [Analog-digital-transition](Watermeter-specific-analog---digital-transition.md).
###### ``MaxRateValue`` and ``MaxRateType``
Here the maximum change from one to the next reading can be limited. If a false reading of the neural network results in a change larger than this, the reading is flagged as false. There a two types of comparisons possible
@@ -53,7 +64,17 @@ Here the maximum change from one to the next reading can be limited. If a false
Example: relative rate of 0.05 m³/minute --> after 20 minutes a maximum change of 20 minutes * 0.05 m³/minute = 1 m³ is possible. That means that a false reading of 1 m³ cannot be detected false after about 20 minutes in this case
Assume, that there might me no change in the meter for hours (e.g. during the night) a much bigger change could also be accepted.
###### ``ExtendedResolution``
Newer models such as dig-cont and dig-class100 have a high resolution of the values and can thus represent another digit by using the value of the last digit or pointer (ex. 7.8 in the last digit).
If the value is set to true, the result of the last digit is used completely.
When using dig-class11 models, the setting is ignored.
###### ``IgnoreLeadingNaN``
The parameter is only be used, if a dig-class11 model is selected. ``ÌgnoreLeadingNaN`` removes in the CheckDigitIncreaseConsistency process the leading `N` values.
#### Flow Chart

View File

@@ -44,6 +44,7 @@ You can change the following settings to reduce incorrect readings (but potentia
* Reduce `AutoTimer` configuration option `Interval` to the lowest it can be (e.g. `3` min). The more often you take readings, the less likely for data staleness to occur.
## Even after I have setup everything perfect there is a false reading - especially around the zero crossing (roll over to next number)
* The roll over behavior is different for the different meters. E.g.:
* Rolling over start with different previous position (e.g. at 7, 8 or 9)
* The neutral position (no rolling) is not perfectly at zero, but rather at something like 7.9 or 8.1, even if it should be exactly 8
@@ -57,3 +58,28 @@ You can change the following settings to reduce incorrect readings (but potentia
Usually after some time and movement of the counters a bit further, the reading is getting back to a stable reading.
* To handle this, a parametrized setting would be needed. This is rather complicated to implement as subtle changes make a relevant difference. Currently this is not implemented.
So please be a bit patient with your meter :-)
## Pre-Value
PreValue is here a bit missleading, because normally it is the same as the last value. In the next round of reading it will be used to check nagtive rates, high rates (MaxRateValue / MaxRateType) and CCheckDigitIncreaseConsistency (dig-class11 only). Either from a previous correctly identified value or manual setting by the user.
If you use post processes, enable the pre-value. The pre-value must be set at first time. Set it to the current raw value.
If the device runs in errors, the pre-value will not be updated, as long as the ``preValueAgeStartup`` time between the last valid value (or startup time) and current time is not exceeded. After it the preValue will be set again, if no other error occured. So the device can not run in an endless error, like high rate.
## "Rate too high - Read: ..."
In configuration you can set the ``MaxRateValue`` and ``MaxRateType``. The settings suppress improbably high values that can come from false readings. To do this, the value must be set correctly depending on your meter.
Before doing this, you should be clear about the type of rating you want to use.
"Absolute change" is the interval between two readings - no matter how often the readings happen.
"RateCange" is the change per minute. This is calculated from the time difference between the last and the current reading.
If there is an interval of 5 minutes between readings and a MaxRateValue of 1, an error "Rate too high - Read: ..." if
Absolute change: the difference is > 1
RateChange: the difference is > 1 / 5