v10.1.1

tabindex to html

Changelog.md

@@ -2,6 +2,99 @@
 
 
 
+##### 8.5.0 - Multi Meter Support (2021-10-07)
+
+* Upgrade digital CNN to v13.1.0 (added new images)
+* bug fix: wlan password with space, double digit output
+
+##### 8.4.0 - Multi Meter Support (2021-09-25)
+
+* License change (remove MIT license, remark see below)
+
+* html: show hostname in title and main page
+
+* configuration: 
+
+  * moved setting `ExtendedResolution` to individual number settings
+  * New parameter `IgnoreLeadingNaN` (delete leading NaN's specifically)
+  * **ATTENTION**: update of the `config.ini` needed (open, adjust `ExtendedResolution`, save)
+
+* Bug fixing (html, images of recognized numbers)
+
+  
+
+### **ATTENTION: LICENSE CHANGE - removal of MIT License.** 
+
+- Currently no licence published - copyright belongs to author
+- If you are interested in a commercial usage or dedicated versions please contact the developer
+  - no limits to private usage
+
+
+
+##### 8.3.0 - Multi Meter Support (2021-09-12)
+
+* Upgrade digital CNN to v12.1.0 (added new images)
+* Dedicated NaN handling, internal refactoring (CNN-Handling)
+* HTML: confirmation after config.ini update
+* Bug fixing
+
+##### 8.2.0 - Multi Meter Support (2021-08-24)
+
+* Improve server responsiveness
+* Flow status and prevalue status in overview
+* Improved prevalue handling 
+
+##### 8.1.0 - Multi Meter Support (2021-08-12)
+
+* GPIO: using the general mqtt main topic for GPIO
+
+* Upgrade digital CNN to v12.0.0  (added new images)
+* Update tfmicro to new master (2021-08-07)
+* Bug fix: remove text in mqtt value, remove connect limit in wlan reconnet
+
+##### 8.0.5 - Multi Meter Support (2021-08-01)
+
+* NEW 8.0.5: bug fix: saving prevalue
+* NEW 8.0.4: bug fix: load config.ini after upgrade
+* NEW 8.0.3: bug fix: reboot during `config.ini` handling, html error
+* NEW 8.0.2: saving roundes prevalue, bug fix html server
+* NEW 8.0.1: bug fix: html handling of parameter `FixedExposure` and `ImageSize`
+* Dual / multi meter support (more than 1 number to be recognized)
+  This is implemented with the feature "number" on the ROI definition as well as selected options
+* MQTT: standardization of the naming - including new topics (`json`,  `freeMem `, `uptime`)c
+* Preparation for extended GPIO support (thanks to Zwerk2k) - not tested and fully functional yet
+* Bug fixing: html server, memory leak, MQTT connect, hostname, turn of flash LED
+
+<span style="color: red;">**ATTENTION: the configuration and prevalue files are modified automatically and will not be backward compatible!**</span> 
+
+##### 7.1.2 MQTT-Update - (2021-06-17)
+
+* NEW: 7.1.2: bug fix setting hostname, Flash-LED not off during reboot
+
+* NEW: 7.1.1: bug fix wlan password with "="  (again)
+
+* MQTT error message: changes "no error", send retain flag
+
+* Update wlan handling to esp-idf 4.1
+
+* Upgrade digital CNN to v8.7.0  (added new images)
+
+* Bug fix: MQTT, WLAN, LED-Controll, GPIO usage, fixed IP, calculation flow rate
+
+  
+
+##### 7.0.1 MQTT-Update - (2021-05-13)
+
+* NEW: 7.0.1: bug fix wlan password with "=" 
+
+* Upgrade digital CNN to v8.5.0  (added new images)
+
+* New MQTT topics: flow rate (units/minute), time stamp (last correct read readout)
+
+* Update MQTT/Error topic to " " in case no error (instead of empty string)
+
+* Portrait or landscape image orientation in rotated image (avoid cropping)
+
 ##### 6.7.2 Image Processing in Memory - (2021-05-01)
 
 * NEW 6.7.2: Updated html for setup modus - remove reboot on edit configuration)

FeatureRequest.md

@@ -11,13 +11,100 @@
 
 ____
 
-#### #6 Check for double ROI names
+#### #17 Direct InfluxDB connection
 
-Check during configuration, that ROI names are unique.
+* https://github.com/jomjol/AI-on-the-edge-device/issues/534
+* Direct interface to a InfluxDB data base
+* Integrate InfluxDB interface in firmware
+* Adapt html web page for configuration
+
+
+#### #16 Serial Communication
+
+* https://github.com/jomjol/AI-on-the-edge-device/issues/512
+* Send the readout value via RX/TX interface with a dedicated TAG
+* Make dedicated communication FlowModule
+* Modification of RX/TX communication
+* Configuration interfache
+
+
+#### #15 Calibration for FishEye image
+
+* https://github.com/jomjol/AI-on-the-edge-device/issues/507
+
+1.  The development of such a correction algorithm with the libraries, that are available for the ESP32 environment.
+2. New module for integration of the flow into the image processing flow.
+3. Extension of the configuration (config.ini) and html-pages
+4. Parameter adjustment and testing for every different fish-eye module
+5. Maintenance for further updates / modules, ...
+
+
+
+#### #14 Backup and restore option for configuration
+
+* https://github.com/jomjol/AI-on-the-edge-device/issues/459
+
+* Implement a zip file compression for store and restore
+
+* Update the html to handle it
+
+  
+
+#### #13 Manage non linear gauge without CNN re-training
+
+* https://github.com/jomjol/AI-on-the-edge-device/issues/443
+
+* Implement a look up table for non linear analog meters
+
+  
+
+#### #12 Less reboots due to memory leakage
+
+* Issue: #414 & #425  #430
+
+  
+
+#### #11 MQTT - configurable payload
+
+* https://github.com/jomjol/AI-on-the-edge-device/issues/344
+
+  
+
+#### #10 Improve and bug fix logging of images
+
+* https://github.com/jomjol/AI-on-the-edge-device/issues/307
+
+  
+
+#### #9 Basic auth for the UI
+
+* https://github.com/jomjol/AI-on-the-edge-device/issues/283
+
+* Implementation of an authentication mechanism.
+
+#### #8 MQTT configurable readout intervall
+
+Make the readout intervall configurable via MQTT.
+
+* Change the mqtt part to receive and process input and not only sending
+
+#### #7 Extended Error Handling
+
+Check different types of error (e.g. tflite not availabe) and generate an error on the html page.
 
 To do:
 
-* Implementation of ROI name checking in html code before saving analog or digital ROIs
+* Make a list of "important" errors
+* Implement a checking algo
+* Extend the firmware and html page for the error handling
+
+#### ~~#6 Check for double ROI names~~ - implemented v8.0.0
+
+~~Check during configuration, that ROI names are unique.~~
+
+~~To do:~~
+
+* ~~Implementation of ROI name checking in html code before saving analog or digital ROIs~~
 
   
 
@@ -33,31 +120,31 @@ To do:
 
 
 
-#### #4 Initial Shifting and Rotation
+#### ~~#4 Initial Shifting and Rotation~~ - implemented v7.0.0
 
-* https://github.com/jomjol/AI-on-the-edge-device/issues/123
+* ~~https://github.com/jomjol/AI-on-the-edge-device/issues/123~~
 
-Implementation of a shifting additional to the initial rotation of the raw camera input
+~~Implementation of a shifting additional to the initial rotation of the raw camera input~~
 
-To do:
+~~To do:~~
 
-* Implementation of shifting
-* Extension of configuration
-* Adaption of the html configuration to implement shifting
+* ~~Implementation of shifting~~
+* ~~Extension of configuration~~
+* ~~Adaption of the html configuration to implement shifting~~
 
 
 
-#### #3 Allow grouping of digits to multiple reading values
+#### ~~#3 Allow grouping of digits to multiple reading values~~ - implemented v8.0.0
 
-* https://github.com/jomjol/AI-on-the-edge-device/issues/123
+* ~~https://github.com/jomjol/AI-on-the-edge-device/issues/123~~
 
-Implementation of two different independent readouts in one setup
+~~Implementation of two different independent readouts in one setup~~
 
-To do:
+~~To do:~~
+
+* ~~Extend the configuration, setting and processing flow for two independend readouts~~
 
-* Extend the configuration, setting and processing flow for two independend readouts
 
-https://github.com/jomjol/AI-on-the-edge-device/issues/123
 
 
 
@@ -80,15 +167,16 @@ To do:
 
 ____
 
-#### #1 Optional GPIO for external flash/lighting
+#### ~~#1 Optional GPIO for external flash/lighting~~ - implemented (v8.0.0)
 
-* https://github.com/jomjol/AI-on-the-edge-device/issues/133
+* ~~https://github.com/jomjol/AI-on-the-edge-device/issues/133~~
 
-Implementation of an an extrnal flash / lightning through GPIOs.
-* available GPIOs: 12 & 13 (currently in use for html switching)
+~~Implementation of an an extrnal flash / lightning through GPIOs.~~
 
-To do:
+* ~~available GPIOs: 12 & 13 (currently in use for html switching)~~
 
-* Implementation of a software module for external light source (e.g. WS8132 LED controller, ...)
-* Update of the camera module to use the external light instead of the internal flash light
-* Adopt the configuration algorithm with a configurable light source
+~~To do:~~
+
+* ~~Implementation of a software module for external light source (e.g. WS8132 LED controller, ...)~~
+* ~~Update of the camera module to use the external light instead of the internal flash light~~
+* ~~Adopt the configuration algorithm with a configurable light source~~

LICENSE

@@ -1,21 +0,0 @@
-MIT License
-
-Copyright (c) 2020 jomjol
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.

README.md

@@ -6,12 +6,16 @@ This is an example of Artificial Intelligence (AI) calculations on a very cheap
 
 A 3d-printable housing can be found here: https://www.thingiverse.com/thing:4573481
 
+or here https://www.thingiverse.com/thing:5028229
+
 respectively ESP32-Cam housing only: https://www.thingiverse.com/thing:4571627
 
 <img src="https://raw.githubusercontent.com/jomjol/AI-on-the-edge-device/master/images/watermeter_all.jpg" width="200"><img src="https://raw.githubusercontent.com/jomjol/AI-on-the-edge-device/master/images/main.jpg" width="200"><img src="https://raw.githubusercontent.com/jomjol/AI-on-the-edge-device/master/images/size.png" width="200"> 
 
 <img src="https://raw.githubusercontent.com/jomjol/AI-on-the-edge-device/master/images/watermeter.jpg" width="600"> 
 
+<img src="https://raw.githubusercontent.com/jomjol/AI-on-the-edge-device/master/images/powermeter.jpg" width="600"> 
+
 
 
 
@@ -30,46 +34,93 @@ If you have any technical topics, you can file a issue in this repository.
 
 In other cases you can contact the developer via email: <img src="https://raw.githubusercontent.com/jomjol/AI-on-the-edge-device/master/images/mail.jpg" height="25"> 
 
-## Change log
+------
+## Coming next
+
+* Automated update of the neural network file (tflite) to make the learing of additional pictures much easier and automated (GitHub action)
+* New "hyprid" neural network for digital numbers --> allowing the detection of intermediate states ("ring between two numbers") as a subdigit
+
 
 ------
-
+## Change log
 ### Known Issues
 
 * slow response of web server during picture analysis
 * spontaneous reboots (mostly due to html access during image processing) - self recovery implemented
 
-------
-
 **General remark:** Beside the `firmware.bin`, typically also the content of `/html` needs to be updated!
 
+------
 
 
 
-##### 7.1.2 MQTT-Update - (2021-06-17)
+##### 10.1.1 - Stability Increase (2022-01-12)
 
-* NEW: 7.1.2: bug fix setting hostname, Flash-LED not off during reboot
-* NEW: 7.1.1: bug fix wlan password with "="  (again)
-* MQTT error message: changes "no error", send retain flag
-* Update wlan handling to esp-idf 4.1
-* Upgrade digital CNN to v8.7.0  (added new images)
-* Bug fix: MQTT, WLAN, LED-Controll, GPIO usage, fixed IP, calculation flow rate
+- **Bug Fix MQTT problem**
+- Issue: 
+  - Changing from v9.x to 10.x the MQTT-paramter "Topic" was renamed into "MainTopic" to address multiple number meters
+    This renaming should have been done automatically in the background within the graphical configuration, but was not working. Instead the parameter "Topic" was deleted and "MainTopic" was set to disabled and "undefined".
+- ToDo
+  - Update the `html.zip`
+  - If old `config.ini` available: copy it to `/config`, open the graphical configuration and save it again.
+  - If old `config.ini` not available: reset the parameter "MainTopic" within the `config.ini` manually 
+  - Reboot
 
+##### 10.1.0 - Stability Increase (2022-01-12)
 
-##### 7.0.1 MQTT-Update - (2021-05-13)
+- Reduce ESP32 frequency to 160MHz
+- Update tflite (new source: https://github.com/espressif/tflite-micro-esp-examples)
+- Update analog neural network (ana-s3-q-20220105.tflite)
+- Update digital neural network (dig-s1-q-20220102.tflite)
+- Increased web-server buffers
+- bug fix: compiler compatibility
 
-* NEW: 7.0.1: bug fix wlan password with "=" 
+##### 10.0.2 - Stability Increase (2022-01-01)
 
-* Upgrade digital CNN to v8.5.0  (added new images)
+- NEW v10.0.2: Corrected JSON error
 
-* New MQTT topics: flow rate (units/minute), time stamp (last correct read readout)
+- Updated compiler toolchain to ESP-IDF 4.3
 
-* Update MQTT/Error topic to " " in case no error (instead of empty string)
+- Removal of memory leak
 
-* Portrait or landscape image orientation in rotated image (avoid cropping)
+- Improved error handling during startup (check PSRAM and camera with remark in logfile)
+
+- MQTT: implemented raw value additionally, removal of regex contrain
+
+- Normalized Parameter ``MaxRateValue``  to "change per minute" 
+
+- HTML: improved input handling
+
+- Corrected error handling: in case of error the old value, rate, timestamp are not transmitted any more
 
   
 
+##### 9.2.0 - External Illumination (2021-12-02)
+
+- Direct JSON access: ``http://IP-ADRESS/json`` 
+- Error message in log file in case camera error during startup
+- Upgrade analog CNN to v9.1.0
+- Upgrade digital CNN to v13.3.0 (added new images)
+- html: support of different ports
+
+##### 9.1.1 - External Illumination (2021-11-16)
+
+- NEW 9.1.1 bug fix: LED implemenetation
+- External LEDs: change control mode (resolve bug with more than 2 LEDs)
+- Additional info into log file
+- Bug fix: decimal shift, html, log file
+
+##### 9.0.0 - External Illumination (2021-10-23)
+
+* Implementation of external illumination to adjust positioning, brightness and color of the illumination now individually
+  * Technical details can be found in the wiki: https://github.com/jomjol/AI-on-the-edge-device/wiki/External-LED
+  <img src="https://raw.githubusercontent.com/jomjol/ai-on-the-edge-device/master/images/intern_vs_external.jpg" width="500">
+* New housing published for external LEDs and small clearing: https://www.thingiverse.com/thing:5028229
+
+
+
+
+
 
 ## Additional ideas
 
@@ -81,6 +132,10 @@ There are some ideas and feature request, which are not followed currently - mai
 
 ## History
 
+##### 8.5.0 - Multi Meter Support (2021-10-07)
+
+##### 7.1.2 MQTT-Update - (2021-06-17)
+
 ##### 6.7.2 Image Processing in Memory - (2021-05-01)
 
 ##### 5.0.0 Setup Modus - (2020-12-06)
@@ -103,8 +158,3 @@ There are some ideas and feature request, which are not followed currently - mai
 
 #### [Full Changelog](Changelog.md)
 
-
-
-## Solved topics
-
-* n.a.

code/.helper/copy-esp-idf.bat

@@ -0,0 +1,3 @@
+copy "..\..\code\build\esp32cam-server-only.bin" "..\..\firmware\firmware.bin"
+copy "..\..\code\build\bootloader\bootloader.bin" "..\..\firmware\bootloader.bin"
+copy "..\..\code\build\partition_table\partition-table.bin" "..\..\firmware\partitions.bin"

code/.helper/copy.bat

@@ -1,3 +1,3 @@
-copy "C:\Users\Muell\Documents\Programmieren\GitHub\AI-on-the-edge-device\code\.pio\build\esp32cam\firmware.bin" "C:\Users\Muell\Documents\Programmieren\GitHub\AI-on-the-edge-device\firmware\firmware.bin"
-copy "C:\Users\Muell\Documents\Programmieren\GitHub\AI-on-the-edge-device\code\.pio\build\esp32cam\bootloader.bin" "C:\Users\Muell\Documents\Programmieren\GitHub\AI-on-the-edge-device\firmware\bootloader.bin"
-copy "C:\Users\Muell\Documents\Programmieren\GitHub\AI-on-the-edge-device\code\.pio\build\esp32cam\partitions.bin" "C:\Users\Muell\Documents\Programmieren\GitHub\AI-on-the-edge-device\firmware\partitions.bin"
+copy "..\..\code\.pio\build\esp32cam\firmware.bin" "..\..\firmware\firmware.bin"
+copy "..\..\code\.pio\build\esp32cam\bootloader.bin" "..\..\firmware\bootloader.bin"
+copy "..\..\code\.pio\build\esp32cam\partitions.bin" "..\..\firmware\partitions.bin"

code/.helper/makezip.bat

@@ -1 +1 @@
-powershell Compress-Archive "C:\Users\Muell\Documents\Programmieren\GitHub\AI-on-the-edge-device\sd-card\html\*.*" "C:\Users\Muell\Documents\Programmieren\GitHub\AI-on-the-edge-device\firmware\html.zip"
+powershell Compress-Archive "..\..\sd-card\html\*.*" "..\..\firmware\html.zip"

code/components/esp32-camera-master/examples/take_picture.c

@@ -29,7 +29,7 @@
 
 // ================================ CODE ======================================
 
-#include <esp_event_loop.h>
+#include <esp_event.h>
 #include <esp_log.h>
 #include <esp_system.h>
 #include <nvs_flash.h>

code/components/jomjol_configfile/CMakeLists.txt

@@ -0,0 +1,7 @@
+FILE(GLOB_RECURSE app_sources ${CMAKE_CURRENT_SOURCE_DIR}/*.*)
+
+idf_component_register(SRCS ${app_sources}
+                    INCLUDE_DIRS "."
+                    REQUIRES jomjol_logfile)
+
+

code/components/jomjol_configfile/configFile.cpp

@@ -0,0 +1,105 @@
+#include <string.h>
+#include "freertos/FreeRTOS.h"
+#include "freertos/task.h"
+
+#include "Helper.h"
+#include "configFile.h"
+
+//static const char *TAGCONFIGFILE = "configFile";
+
+ConfigFile::ConfigFile(std::string filePath)
+{
+    std::string config = FormatFileName(filePath);
+    pFile = OpenFileAndWait(config.c_str(), "r");
+}
+
+ConfigFile::~ConfigFile()
+{
+    fclose(pFile);
+}
+
+bool ConfigFile::isNewParagraph(std::string input)
+{
+	if ((input[0] == '[') || ((input[0] == ';') && (input[1] == '[')))
+	{
+		return true;
+	}
+	return false;
+}
+
+bool ConfigFile::GetNextParagraph(std::string& aktparamgraph, bool &disabled, bool &eof)
+{
+	while (getNextLine(&aktparamgraph, disabled, eof) && !isNewParagraph(aktparamgraph));
+
+	if (isNewParagraph(aktparamgraph))
+		return true;
+	return false;
+}
+
+bool ConfigFile::getNextLine(std::string *rt, bool &disabled, bool &eof)
+{
+    eof = false;
+	char zw[1024] = "";
+	if (pFile == NULL)
+	{
+		*rt = "";
+		return false;
+	}
+
+	if (fgets(zw, 1024, pFile))
+	{
+		printf("%s", zw);
+		if ((strlen(zw) == 0) && feof(pFile))
+		{
+			*rt = "";
+			eof = true;
+			return false;
+		}
+	}
+	else
+	{
+		*rt = "";
+		eof = true;
+		return false;
+	}
+	*rt = zw;
+	*rt = trim(*rt);
+	while ((zw[0] == ';' || zw[0] == '#' || (rt->size() == 0)) && !(zw[1] == '['))			// Kommentarzeilen (; oder #) und Leerzeilen überspringen, es sei denn es ist ein neuer auskommentierter Paragraph
+	{
+		fgets(zw, 1024, pFile);
+		printf("%s", zw);		
+		if (feof(pFile))
+		{
+			*rt = "";
+            eof = true;
+			return false;
+		}
+		*rt = zw;
+		*rt = trim(*rt);
+	}
+
+    disabled = ((*rt)[0] == ';');
+	return true;
+}
+
+std::vector<string> ConfigFile::ZerlegeZeile(std::string input, std::string delimiter)
+{
+	std::vector<string> Output;
+//	std::string delimiter = " =,";
+
+	input = trim(input, delimiter);
+	size_t pos = findDelimiterPos(input, delimiter);
+	std::string token;
+	while (pos != std::string::npos) {
+		token = input.substr(0, pos);
+		token = trim(token, delimiter);
+		Output.push_back(token);
+		input.erase(0, pos + 1);
+		input = trim(input, delimiter);
+		pos = findDelimiterPos(input, delimiter);
+	}
+	Output.push_back(input);
+
+	return Output;
+
+}

code/components/jomjol_configfile/configFile.h

@@ -0,0 +1,16 @@
+#include <string>
+#include <vector>
+
+class ConfigFile {
+public:
+    ConfigFile(std::string filePath);
+    ~ConfigFile();
+
+    bool isNewParagraph(std::string input);
+    bool GetNextParagraph(std::string& aktparamgraph, bool &disabled, bool &eof);
+	bool getNextLine(std::string* rt, bool &disabled, bool &eof);
+    std::vector<std::string> ZerlegeZeile(std::string input, std::string delimiter = " =, \t");
+    
+private:
+    FILE* pFile;
+};

code/components/jomjol_controlGPIO/CMakeLists.txt

@@ -3,7 +3,7 @@ FILE(GLOB_RECURSE app_sources ${CMAKE_CURRENT_SOURCE_DIR}/*.*)
 list(APPEND EXTRA_COMPONENT_DIRS $ENV{IDF_PATH}/examples/common_components/protocol_examples_common)
 
 idf_component_register(SRCS ${app_sources}
-                    INCLUDE_DIRS "."
-                    REQUIRES esp_http_server jomjol_logfile)
+                    INCLUDE_DIRS "." "../../include"
+                    REQUIRES esp_http_server jomjol_logfile jomjol_configfile jomjol_mqtt jomjol_flowcontroll)
 
 

code/components/jomjol_controlGPIO/Color.cpp

@@ -0,0 +1,132 @@
+#include "Color.h"
+#include <algorithm>
+#include <cmath>
+#include <cassert>
+
+namespace {
+
+// Int -> fixed point
+int up( int x ) { return x * 255; }
+
+} // namespace
+
+int iRgbSqrt( int num ) {
+    // https://en.wikipedia.org/wiki/Methods_of_computing_square_roots#Binary_numeral_system_.28base_2.29
+    assert( "sqrt input should be non-negative" && num >= 0 );
+    assert( "sqrt input should no exceed 16 bits" && num <= 0xFFFF );
+    int res = 0;
+    int bit = 1 << 16;
+    while ( bit > num )
+        bit >>= 2;
+    while ( bit != 0 ) {
+        if ( num >= res + bit ) {
+            num -= res + bit;
+            res = ( res >> 1 ) + bit;
+        } else
+            res >>= 1;
+        bit >>= 2;
+    }
+    return res;
+}
+
+Rgb::Rgb( Hsv y ) {
+    // https://stackoverflow.com/questions/24152553/hsv-to-rgb-and-back-without-floating-point-math-in-python
+    // greyscale
+    if( y.s == 0 ) {
+        r = g = b = y.v;
+        return;
+    }
+
+    const int region = y.h / 43;
+    const int remainder = ( y.h - ( region * 43 ) ) * 6;
+
+    const int p = ( y.v * ( 255 - y.s ) ) >> 8;
+    const int q = ( y.v * ( 255 - ( ( y.s * remainder ) >> 8 ) ) ) >> 8;
+    const int t = ( y.v * ( 255 - ( ( y.s * (255 -remainder ) ) >> 8 ) ) ) >> 8;
+
+    switch( region ) {
+        case 0: r = y.v; g = t; b = p; break;
+        case 1: r = q; g = y.v; b = p; break;
+        case 2: r = p; g = y.v; b = t; break;
+        case 3: r = p; g = q; b = y.v; break;
+        case 4: r = t; g = p; b = y.v; break;
+        case 5: r = y.v; g = p; b = q; break;
+        default: __builtin_trap();
+    }
+
+    a = y.a;
+}
+
+Rgb& Rgb::operator=( Hsv hsv ) {
+    Rgb r{ hsv };
+    swap( r );
+    return *this;
+}
+
+Rgb Rgb::operator+( Rgb in ) const {
+    auto copy = *this;
+    copy += in;
+    return copy;
+}
+
+Rgb& Rgb::operator+=( Rgb in ) {
+    unsigned int red = r + in.r;
+    r = ( red < 255 ) ? red : 255;
+    unsigned int green = g + in.g;
+    g = ( green < 255 ) ? green : 255;
+    unsigned int blue = b + in.b;
+    b = ( blue < 255 ) ? blue : 255;
+    return *this;
+}
+
+Rgb& Rgb::blend( Rgb in ) {
+    unsigned int inAlpha = in.a * ( 255 - a );
+    unsigned int alpha = a + inAlpha;
+    r = iRgbSqrt( ( ( r * r * a ) + ( in.r * in.r * inAlpha ) ) / alpha );
+    g = iRgbSqrt( ( ( g * g * a ) + ( in.g * in.g * inAlpha ) ) / alpha );
+    b = iRgbSqrt( ( ( b * b * a ) + ( in.b * in.b * inAlpha ) ) / alpha );
+    a = alpha;
+    return *this;
+}
+
+uint8_t IRAM_ATTR Rgb::getGrb( int idx ) {
+    switch ( idx ) {
+        case 0: return g;
+        case 1: return r;
+        case 2: return b;
+    }
+    __builtin_unreachable();
+}
+
+Hsv::Hsv( Rgb r ) {
+    int min = std::min( r.r, std::min( r.g, r.b ) );
+    int max = std::max( r.r, std::max( r.g, r.b ) );
+    int chroma = max - min;
+
+    v = max;
+    if ( chroma == 0 ) {
+        h = s = 0;
+        return;
+    }
+
+    s = up( chroma ) / max;
+    int hh;
+    if ( max == r.r )
+        hh = ( up( int( r.g ) - int( r.b ) ) ) / chroma / 6;
+    else if ( max == r.g )
+        hh = 255 / 3 + ( up( int( r.b ) - int( r.r ) ) ) / chroma / 6;
+    else
+        hh = 2 * 255 / 3 + ( up( int( r.r ) - int( r.g ) ) ) / chroma / 6;
+
+    if ( hh < 0 )
+        hh += 255;
+    h = hh;
+
+    a = r.a;
+}
+
+Hsv& Hsv::operator=( Rgb rgb ) {
+    Hsv h{ rgb };
+    swap( h );
+    return *this;
+}

code/components/jomjol_controlGPIO/Color.h

@@ -0,0 +1,69 @@
+#pragma once
+
+#include <cstdint>
+#include "esp_attr.h"
+union Hsv;
+
+union Rgb {
+    struct __attribute__ ((packed)) {
+        uint8_t r, g, b, a;
+    };
+    uint32_t value;
+
+    Rgb( uint8_t r = 0, uint8_t g = 0, uint8_t b = 0, uint8_t a = 255 ) : r( r ), g( g ), b( b ), a( a ) {}
+    Rgb( Hsv c );
+    Rgb& operator=( Rgb rgb ) { swap( rgb ); return *this; }
+    Rgb& operator=( Hsv hsv );
+    Rgb operator+( Rgb in ) const;
+    Rgb& operator+=( Rgb in );
+    bool operator==( Rgb in ) const { return in.value == value; }
+    Rgb& blend( Rgb in );
+    void swap( Rgb& o ) {  value = o.value; }
+    void linearize() {
+        r = channelGamma(r);
+        g = channelGamma(g);
+        b = channelGamma(b);
+    }
+
+    uint8_t IRAM_ATTR getGrb( int idx );
+
+    void stretchChannels( uint8_t maxR, uint8_t maxG, uint8_t maxB ) {
+        r = stretch( r, maxR );
+        g = stretch( g, maxG );
+        b = stretch( b, maxB );
+    }
+
+    void stretchChannelsEvenly( uint8_t max ) {
+        stretchChannels( max, max, max );
+    }
+
+private:
+    uint8_t stretch( int value, uint8_t max ) {
+        return ( value * max ) >> 8;
+    }
+
+    uint8_t channelGamma( int channel ) {
+        /* The optimal gamma correction is x^2.8. However, this is expensive to
+         * compute. Therefore, we use x^3 for gamma correction. Also, we add a
+         * bias as the WS2812 LEDs do not turn on for values less than 4. */
+        if (channel == 0)
+            return channel;
+        channel = channel * channel * channel * 251;
+        channel >>= 24;
+        return static_cast< uint8_t >( 4 + channel );
+    }
+};
+
+union Hsv {
+    struct __attribute__ ((packed)) {
+        uint8_t h, s, v, a;
+    };
+    uint32_t value;
+
+    Hsv( uint8_t h, uint8_t s = 0, uint8_t v = 0, uint8_t a = 255 ) : h( h ), s( s ), v( v ), a( a ) {}
+    Hsv( Rgb r );
+    Hsv& operator=( Hsv h ) { swap( h ); return *this; }
+    Hsv& operator=( Rgb rgb );
+    bool operator==( Hsv in ) const { return in.value == value; }
+    void swap( Hsv& o ) { value = o.value; }
+};

code/components/jomjol_controlGPIO/SmartLeds.cpp

@@ -0,0 +1,63 @@
+#include "SmartLeds.h"
+
+IsrCore SmartLed::_interruptCore = CoreCurrent;
+intr_handle_t SmartLed::_interruptHandle = NULL;
+
+SmartLed*& IRAM_ATTR SmartLed::ledForChannel( int channel ) {
+    static SmartLed* table[8] = { nullptr };
+    assert( channel < 8 );
+    return table[ channel ];
+}
+
+void IRAM_ATTR SmartLed::interruptHandler(void*) {
+    for (int channel = 0; channel != 8; channel++) {
+        auto self = ledForChannel( channel );
+
+        if ( RMT.int_st.val & (1 << (24 + channel ) ) ) { // tx_thr_event
+            if ( self )
+                self->copyRmtHalfBlock();
+            RMT.int_clr.val |= 1 << ( 24 + channel );
+        } else if ( RMT.int_st.val & ( 1 << (3 * channel ) ) ) { // tx_end
+            if ( self )
+                xSemaphoreGiveFromISR( self->_finishedFlag, nullptr );
+            RMT.int_clr.val |= 1 << ( 3 * channel );
+        }
+    }
+}
+
+void IRAM_ATTR SmartLed::copyRmtHalfBlock() {
+    int offset = detail::MAX_PULSES * _halfIdx;
+    _halfIdx = !_halfIdx;
+    int len = 3 - _componentPosition + 3 * ( _count - 1 );
+    len = std::min( len, detail::MAX_PULSES / 8 );
+
+    if ( !len ) {
+        for ( int i = 0; i < detail::MAX_PULSES; i++) {
+            RMTMEM.chan[ _channel].data32[i + offset ].val = 0;
+        }
+    }
+
+    int i;
+    for ( i = 0; i != len && _pixelPosition != _count; i++ ) {
+        uint8_t val = _buffer[ _pixelPosition ].getGrb( _componentPosition );
+        for ( int j = 0; j != 8; j++, val <<= 1 ) {
+            int bit = val >> 7;
+            int idx = i * 8 + offset + j;
+            RMTMEM.chan[ _channel ].data32[ idx ].val = _bitToRmt[ bit & 0x01 ].value;
+        }
+        if ( _pixelPosition == _count - 1 && _componentPosition == 2 ) {
+            RMTMEM.chan[ _channel ].data32[ i * 8 + offset + 7 ].duration1 =
+                _timing.TRS / ( detail::RMT_DURATION_NS * detail::DIVIDER );
+        }
+
+        _componentPosition++;
+        if ( _componentPosition == 3 ) {
+            _componentPosition = 0;
+            _pixelPosition++;
+        }
+    }
+
+    for ( i *= 8; i != detail::MAX_PULSES; i++ ) {
+        RMTMEM.chan[ _channel ].data32[ i + offset ].val = 0;
+    }
+}

code/components/jomjol_controlGPIO/SmartLeds.h

@@ -0,0 +1,530 @@
+#pragma once
+
+/*
+ * A C++ driver for the WS2812 LEDs using the RMT peripheral on the ESP32.
+ *
+ * Jan "yaqwsx" Mrázek <email@honzamrazek.cz>
+ *
+ * Based on the work by Martin F. Falatic - https://github.com/FozzTexx/ws2812-demo
+ */
+
+/*
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include <memory>
+#include <cassert>
+#include <cstring>
+
+#if defined ( ARDUINO )
+    extern "C" { // ...someone forgot to put in the includes...
+        #include "esp32-hal.h"
+        #include "esp_intr_alloc.h"
+        #include "esp_ipc.h"
+        #include "driver/gpio.h"
+        #include "driver/periph_ctrl.h"
+        #include "freertos/semphr.h"
+        #include "soc/rmt_struct.h"
+        #include <driver/spi_master.h>
+        #include "esp_idf_version.h"
+#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL( 4, 0, 0 )
+        #include "soc/dport_reg.h"
+#endif
+    }
+#elif defined ( ESP_PLATFORM )
+    extern "C" { // ...someone forgot to put in the includes...
+        #include <esp_intr_alloc.h>
+        #include <esp_ipc.h>
+        #include <driver/gpio.h>
+        #include <freertos/FreeRTOS.h>
+        #include <freertos/semphr.h>
+        #include <soc/dport_reg.h>
+        #include <soc/gpio_sig_map.h>
+        #include <soc/rmt_struct.h>
+        #include <driver/spi_master.h>
+    }
+    #include <stdio.h>
+#endif
+
+#include "Color.h"
+
+namespace detail {
+
+struct TimingParams {
+    uint32_t T0H;
+    uint32_t T1H;
+    uint32_t T0L;
+    uint32_t T1L;
+    uint32_t TRS;
+};
+
+union RmtPulsePair {
+    struct {
+        int duration0:15;
+        int level0:1;
+        int duration1:15;
+        int level1:1;
+    };
+    uint32_t value;
+};
+
+static const int DIVIDER = 4; // 8 still seems to work, but timings become marginal
+static const int MAX_PULSES = 32; // A channel has a 64 "pulse" buffer - we use half per pass
+static const double RMT_DURATION_NS = 12.5; // minimum time of a single RMT duration based on clock ns
+
+} // namespace detail
+
+using LedType = detail::TimingParams;
+
+static const LedType LED_WS2812  = { 350, 700, 800, 600, 50000 };
+static const LedType LED_WS2812B = { 400, 850, 850, 400, 50100 };
+static const LedType LED_SK6812  = { 300, 600, 900, 600, 80000 };
+static const LedType LED_WS2813  = { 350, 800, 350, 350, 300000 };
+
+enum BufferType { SingleBuffer = 0, DoubleBuffer };
+
+enum IsrCore { CoreFirst = 0, CoreSecond = 1, CoreCurrent = 2};
+
+class SmartLed {
+public:
+    // The RMT interrupt must not run on the same core as WiFi interrupts, otherwise SmartLeds
+    // can't fill the RMT buffer fast enough, resulting in rendering artifacts.
+    // Usually, that means you have to set isrCore == CoreSecond.
+    //
+    // If you use anything other than CoreCurrent, the FreeRTOS scheduler MUST be already running,
+    // so you can't use it if you define SmartLed as global variable.
+    SmartLed( const LedType& type, int count, int pin, int channel = 0, BufferType doubleBuffer = SingleBuffer, IsrCore isrCore = CoreCurrent)
+        : _timing( type ),
+          _channel( channel ),
+          _count( count ),
+          _firstBuffer( new Rgb[ count ] ),
+          _secondBuffer( doubleBuffer ? new Rgb[ count ] : nullptr ),
+          _finishedFlag( xSemaphoreCreateBinary() )
+    {
+        assert( channel >= 0 && channel < 8 );
+        assert( ledForChannel( channel ) == nullptr );
+
+        xSemaphoreGive( _finishedFlag );
+
+        DPORT_SET_PERI_REG_MASK( DPORT_PERIP_CLK_EN_REG, DPORT_RMT_CLK_EN );
+        DPORT_CLEAR_PERI_REG_MASK( DPORT_PERIP_RST_EN_REG, DPORT_RMT_RST );
+
+        PIN_FUNC_SELECT( GPIO_PIN_MUX_REG[ pin ], 2 );
+        gpio_set_direction( static_cast< gpio_num_t >( pin ), GPIO_MODE_OUTPUT );
+        gpio_matrix_out( static_cast< gpio_num_t >( pin ), RMT_SIG_OUT0_IDX + _channel, 0, 0 );
+        initChannel( _channel );
+
+        RMT.tx_lim_ch[ _channel ].limit = detail::MAX_PULSES;
+        RMT.int_ena.val |= 1 << ( 24 + _channel );
+        RMT.int_ena.val |= 1 << ( 3 * _channel );
+
+        _bitToRmt[ 0 ].level0 = 1;
+        _bitToRmt[ 0 ].level1 = 0;
+        _bitToRmt[ 0 ].duration0 = _timing.T0H / ( detail::RMT_DURATION_NS * detail::DIVIDER );
+        _bitToRmt[ 0 ].duration1 = _timing.T0L / ( detail::RMT_DURATION_NS * detail::DIVIDER );
+
+        _bitToRmt[ 1 ].level0 = 1;
+        _bitToRmt[ 1 ].level1 = 0;
+        _bitToRmt[ 1 ].duration0 = _timing.T1H / ( detail::RMT_DURATION_NS * detail::DIVIDER );
+        _bitToRmt[ 1 ].duration1 = _timing.T1L / ( detail::RMT_DURATION_NS * detail::DIVIDER );
+
+        if ( !anyAlive() ) {
+            _interruptCore = isrCore;
+            if(isrCore != CoreCurrent) {
+                ESP_ERROR_CHECK(esp_ipc_call_blocking(isrCore, registerInterrupt, NULL));
+            } else {
+                registerInterrupt(NULL);
+            }
+        }
+
+        ledForChannel( channel ) = this;
+    }
+
+    ~SmartLed() {
+        ledForChannel( _channel ) = nullptr;
+        if ( !anyAlive() ) {
+            if(_interruptCore != CoreCurrent) {
+                ESP_ERROR_CHECK(esp_ipc_call_blocking(_interruptCore, unregisterInterrupt, NULL));
+            } else {
+                unregisterInterrupt(NULL);
+            }
+        }
+        vSemaphoreDelete( _finishedFlag );
+    }
+
+    Rgb& operator[]( int idx ) {
+        return _firstBuffer[ idx ];
+    }
+
+    const Rgb& operator[]( int idx ) const {
+        return _firstBuffer[ idx ];
+    }
+
+    void show() {
+        _buffer = _firstBuffer.get();
+        startTransmission();
+        swapBuffers();
+    }
+
+    bool wait( TickType_t timeout = portMAX_DELAY ) {
+        if( xSemaphoreTake( _finishedFlag, timeout ) == pdTRUE ) {
+            xSemaphoreGive( _finishedFlag );
+            return true;
+        }
+        return false;
+    }
+
+    int size() const {
+        return _count;
+    }
+
+    Rgb *begin() { return _firstBuffer.get(); }
+    const Rgb *begin() const { return _firstBuffer.get(); }
+    const Rgb *cbegin() const { return _firstBuffer.get(); }
+
+    Rgb *end() { return _firstBuffer.get() + _count; }
+    const Rgb *end() const { return _firstBuffer.get() + _count; }
+    const Rgb *cend() const { return _firstBuffer.get() + _count; }
+
+private:
+    static intr_handle_t _interruptHandle;
+    static IsrCore _interruptCore;
+
+    static void initChannel( int channel ) {
+        RMT.apb_conf.fifo_mask = 1;  //enable memory access, instead of FIFO mode.
+        RMT.apb_conf.mem_tx_wrap_en = 1; //wrap around when hitting end of buffer
+        RMT.conf_ch[ channel ].conf0.div_cnt = detail::DIVIDER;
+        RMT.conf_ch[ channel ].conf0.mem_size = 1;
+        RMT.conf_ch[ channel ].conf0.carrier_en = 0;
+        RMT.conf_ch[ channel ].conf0.carrier_out_lv = 1;
+        RMT.conf_ch[ channel ].conf0.mem_pd = 0;
+
+        RMT.conf_ch[ channel ].conf1.rx_en = 0;
+        RMT.conf_ch[ channel ].conf1.mem_owner = 0;
+        RMT.conf_ch[ channel ].conf1.tx_conti_mode = 0;    //loop back mode.
+        RMT.conf_ch[ channel ].conf1.ref_always_on = 1;    // use apb clock: 80M
+        RMT.conf_ch[ channel ].conf1.idle_out_en = 1;
+        RMT.conf_ch[ channel ].conf1.idle_out_lv = 0;
+    }
+
+    static void registerInterrupt(void *) {
+        ESP_ERROR_CHECK(esp_intr_alloc( ETS_RMT_INTR_SOURCE, 0, interruptHandler, nullptr, &_interruptHandle));
+    }
+
+    static void unregisterInterrupt(void*) {
+        esp_intr_free( _interruptHandle );
+    }
+
+    static SmartLed*& IRAM_ATTR ledForChannel( int channel );
+    static void IRAM_ATTR interruptHandler( void* );
+
+    void IRAM_ATTR copyRmtHalfBlock();
+
+    void swapBuffers() {
+        if ( _secondBuffer )
+            _firstBuffer.swap( _secondBuffer );
+    }
+
+    void startTransmission() {
+        // Invalid use of the library
+        if( xSemaphoreTake( _finishedFlag, 0 ) != pdTRUE )
+            abort();
+
+        _pixelPosition = _componentPosition = _halfIdx = 0;
+        copyRmtHalfBlock();
+        if ( _pixelPosition < _count )
+            copyRmtHalfBlock();
+
+        RMT.conf_ch[ _channel ].conf1.mem_rd_rst = 1;
+        RMT.conf_ch[ _channel ].conf1.tx_start = 1;
+    }
+
+    static bool anyAlive() {
+        for ( int i = 0; i != 8; i++ )
+            if ( ledForChannel( i ) != nullptr ) return true;
+        return false;
+    }
+
+    const LedType& _timing;
+    int _channel;
+    detail::RmtPulsePair _bitToRmt[ 2 ];
+    int _count;
+    std::unique_ptr< Rgb[] > _firstBuffer;
+    std::unique_ptr< Rgb[] > _secondBuffer;
+    Rgb *_buffer;
+
+    xSemaphoreHandle _finishedFlag;
+
+    int _pixelPosition;
+    int _componentPosition;
+    int _halfIdx;
+};
+
+class Apa102 {
+public:
+    struct ApaRgb {
+        ApaRgb( uint8_t r = 0, uint8_t g = 0, uint32_t b = 0, uint32_t v = 0xFF )
+            : v( 0xE0 | v ), b( b ), g( g ), r( r )
+        {}
+
+        ApaRgb& operator=( const Rgb& o ) {
+            r = o.r;
+            g = o.g;
+            b = o.b;
+            return *this;
+        }
+
+        ApaRgb& operator=( const Hsv& o ) {
+            *this = Rgb{ o };
+            return *this;
+        }
+
+        uint8_t v, b, g, r;
+    };
+
+    static const int FINAL_FRAME_SIZE = 4;
+    static const int TRANS_COUNT = 2 + 8;
+
+    Apa102( int count, int clkpin, int datapin, BufferType doubleBuffer = SingleBuffer )
+        : _count( count ),
+          _firstBuffer( new ApaRgb[ count ] ),
+          _secondBuffer( doubleBuffer ? new ApaRgb[ count ] : nullptr ),
+          _initFrame( 0 )
+    {
+        spi_bus_config_t buscfg;
+        memset( &buscfg, 0, sizeof( buscfg ) );
+        buscfg.mosi_io_num = datapin;
+        buscfg.miso_io_num = -1;
+        buscfg.sclk_io_num = clkpin;
+        buscfg.quadwp_io_num = -1;
+        buscfg.quadhd_io_num = -1;
+        buscfg.max_transfer_sz = 65535;
+
+        spi_device_interface_config_t devcfg;
+        memset( &devcfg, 0, sizeof( devcfg ) );
+        devcfg.clock_speed_hz = 1000000;
+        devcfg.mode = 0;
+        devcfg.spics_io_num = -1;
+        devcfg.queue_size = TRANS_COUNT;
+        devcfg.pre_cb = nullptr;
+
+        auto ret = spi_bus_initialize( HSPI_HOST, &buscfg, 1 );
+        assert( ret == ESP_OK );
+
+        ret = spi_bus_add_device( HSPI_HOST, &devcfg, &_spi );
+        assert( ret == ESP_OK );
+
+        std::fill_n( _finalFrame, FINAL_FRAME_SIZE, 0xFFFFFFFF );
+    }
+
+    ~Apa102() {
+        // ToDo
+    }
+
+    ApaRgb& operator[]( int idx ) {
+        return _firstBuffer[ idx ];
+    }
+
+    const ApaRgb& operator[]( int idx ) const {
+        return _firstBuffer[ idx ];
+    }
+
+    void show() {
+        _buffer = _firstBuffer.get();
+        startTransmission();
+        swapBuffers();
+    }
+
+    void wait() {
+        for ( int i = 0; i != _transCount; i++ ) {
+            spi_transaction_t *t;
+            spi_device_get_trans_result( _spi, &t, portMAX_DELAY );
+        }
+    }
+private:
+    void swapBuffers() {
+        if ( _secondBuffer )
+            _firstBuffer.swap( _secondBuffer );
+    }
+
+    void startTransmission() {
+        for ( int i = 0; i != TRANS_COUNT; i++ ) {
+            _transactions[ i ].cmd = 0;
+            _transactions[ i ].addr = 0;
+            _transactions[ i ].flags = 0;
+            _transactions[ i ].rxlength = 0;
+            _transactions[ i ].rx_buffer = nullptr;
+        }
+        // Init frame
+        _transactions[ 0 ].length = 32;
+        _transactions[ 0 ].tx_buffer = &_initFrame;
+        spi_device_queue_trans( _spi, _transactions + 0, portMAX_DELAY );
+        // Data
+        _transactions[ 1 ].length = 32 * _count;
+        _transactions[ 1 ].tx_buffer = _buffer;
+        spi_device_queue_trans( _spi, _transactions + 1, portMAX_DELAY );
+        _transCount = 2;
+        // End frame
+        for ( int i = 0; i != 1 + _count / 32 / FINAL_FRAME_SIZE; i++ ) {
+            _transactions[ 2 + i ].length = 32 * FINAL_FRAME_SIZE;
+            _transactions[ 2 + i ].tx_buffer = _finalFrame;
+            spi_device_queue_trans( _spi, _transactions + 2 + i, portMAX_DELAY );
+            _transCount++;
+        }
+    }
+
+    spi_device_handle_t _spi;
+    int _count;
+    std::unique_ptr< ApaRgb[] > _firstBuffer, _secondBuffer;
+    ApaRgb *_buffer;
+
+    spi_transaction_t _transactions[ TRANS_COUNT ];
+    int _transCount;
+
+    uint32_t _initFrame;
+    uint32_t _finalFrame[ FINAL_FRAME_SIZE ];
+};
+
+class LDP8806 {
+public:
+    struct LDP8806_GRB {
+
+        LDP8806_GRB( uint8_t g_7bit = 0, uint8_t r_7bit = 0, uint32_t b_7bit = 0 )
+            : g( g_7bit ), r( r_7bit ), b( b_7bit )
+        {
+        }
+
+        LDP8806_GRB& operator=( const Rgb& o ) {
+            //Convert 8->7bit colour
+            r = ( o.r * 127 / 256 ) | 0x80;
+            g = ( o.g * 127 / 256 ) | 0x80;
+            b = ( o.b * 127 / 256 ) | 0x80;
+            return *this;
+        }
+
+        LDP8806_GRB& operator=( const Hsv& o ) {
+            *this = Rgb{ o };
+            return *this;
+        }
+
+        uint8_t g, r, b;
+    };
+
+    static const int LED_FRAME_SIZE_BYTES = sizeof( LDP8806_GRB );
+    static const int LATCH_FRAME_SIZE_BYTES = 3;
+    static const int TRANS_COUNT_MAX = 20;//Arbitrary, supports up to 600 LED
+
+    LDP8806( int count, int clkpin, int datapin, BufferType doubleBuffer = SingleBuffer, uint32_t clock_speed_hz = 2000000 )
+        : _count( count ),
+          _firstBuffer( new LDP8806_GRB[ count ] ),
+          _secondBuffer( doubleBuffer ? new LDP8806_GRB[ count ] : nullptr ),
+          // one 'latch'/start-of-data mark frame for every 32 leds
+          _latchFrames( ( count + 31 ) / 32 )
+    {
+        spi_bus_config_t buscfg;
+        memset( &buscfg, 0, sizeof( buscfg ) );
+        buscfg.mosi_io_num = datapin;
+        buscfg.miso_io_num = -1;
+        buscfg.sclk_io_num = clkpin;
+        buscfg.quadwp_io_num = -1;
+        buscfg.quadhd_io_num = -1;
+        buscfg.max_transfer_sz = 65535;
+
+        spi_device_interface_config_t devcfg;
+        memset( &devcfg, 0, sizeof( devcfg ) );
+        devcfg.clock_speed_hz = clock_speed_hz;
+        devcfg.mode = 0;
+        devcfg.spics_io_num = -1;
+        devcfg.queue_size = TRANS_COUNT_MAX;
+        devcfg.pre_cb = nullptr;
+
+        auto ret = spi_bus_initialize( HSPI_HOST, &buscfg, 1 );
+        assert( ret == ESP_OK );
+
+        ret = spi_bus_add_device( HSPI_HOST, &devcfg, &_spi );
+        assert( ret == ESP_OK );
+
+        std::fill_n( _latchBuffer, LATCH_FRAME_SIZE_BYTES, 0x0 );
+    }
+
+    ~LDP8806() {
+        // noop
+    }
+
+    LDP8806_GRB& operator[]( int idx ) {
+        return _firstBuffer[ idx ];
+    }
+
+    const LDP8806_GRB& operator[]( int idx ) const {
+        return _firstBuffer[ idx ];
+    }
+
+    void show() {
+        _buffer = _firstBuffer.get();
+        startTransmission();
+        swapBuffers();
+    }
+
+    void wait() {
+        while ( _transCount-- ) {
+            spi_transaction_t *t;
+            spi_device_get_trans_result( _spi, &t, portMAX_DELAY );
+        }
+    }
+private:
+    void swapBuffers() {
+        if ( _secondBuffer )
+            _firstBuffer.swap( _secondBuffer );
+    }
+
+    void startTransmission() {
+        _transCount = 0;
+        for ( int i = 0; i != TRANS_COUNT_MAX; i++ ) {
+            _transactions[ i ].cmd = 0;
+            _transactions[ i ].addr = 0;
+            _transactions[ i ].flags = 0;
+            _transactions[ i ].rxlength = 0;
+            _transactions[ i ].rx_buffer = nullptr;
+        }
+        // LED Data
+        _transactions[ 0 ].length = ( LED_FRAME_SIZE_BYTES * 8 ) * _count;
+        _transactions[ 0 ].tx_buffer = _buffer;
+        spi_device_queue_trans( _spi, _transactions + _transCount, portMAX_DELAY );
+        _transCount++;
+
+        // 'latch'/start-of-data marker frames
+        for ( int i = 0; i < _latchFrames; i++ ) {
+            _transactions[ _transCount ].length = ( LATCH_FRAME_SIZE_BYTES * 8 );
+            _transactions[ _transCount ].tx_buffer = _latchBuffer;
+            spi_device_queue_trans( _spi, _transactions + _transCount, portMAX_DELAY );
+            _transCount++;
+        }
+    }
+
+    spi_device_handle_t _spi;
+    int _count;
+    std::unique_ptr< LDP8806_GRB[] > _firstBuffer, _secondBuffer;
+    LDP8806_GRB *_buffer;
+
+    spi_transaction_t _transactions[ TRANS_COUNT_MAX ];
+    int _transCount;
+
+    int _latchFrames;
+    uint8_t _latchBuffer[ LATCH_FRAME_SIZE_BYTES ];
+};

code/components/jomjol_controlGPIO/server_GPIO.cpp

@@ -1,4 +1,5 @@
 #include <string>
+#include <functional>
 #include "string.h"
 
 #include <string.h>
@@ -6,22 +7,453 @@
 #include "freertos/task.h"
 #include "esp_system.h"
 #include "esp_event.h"
+
+#include "server_tflite.h"
+
+//#define LOG_LOCAL_LEVEL ESP_LOG_DEBUG
 #include "esp_log.h"
-#include "driver/gpio.h"
 //#include "errno.h"
 
 #include <sys/stat.h>
+#include <vector>
+//#include <regex>
+
+#include "defines.h"
 
 #include "server_GPIO.h"
 
 #include "ClassLogFile.h"
-
+#include "configFile.h"
 #include "Helper.h"
+#include "interface_mqtt.h"
 
-// #define DEBUG_DETAIL_ON 
+static const char *TAG_SERVERGPIO = "server_GPIO";
+QueueHandle_t gpio_queue_handle = NULL;
 
-esp_err_t handler_switch_GPIO(httpd_req_t *req)
+#define DEBUG_DETAIL_ON 
+
+GpioPin::GpioPin(gpio_num_t gpio, const char* name, gpio_pin_mode_t mode, gpio_int_type_t interruptType, uint8_t dutyResolution, std::string mqttTopic, bool httpEnable) 
 {
+    _gpio = gpio;
+    _name = name; 
+    _mode = mode;
+    _interruptType = interruptType;    
+    _mqttTopic = mqttTopic;
+}
+
+GpioPin::~GpioPin()
+{
+    ESP_LOGD(TAG_SERVERGPIO,"reset GPIO pin %d", _gpio);
+    if (_interruptType != GPIO_INTR_DISABLE) {
+        //hook isr handler for specific gpio pin
+        gpio_isr_handler_remove(_gpio);
+    }
+    gpio_reset_pin(_gpio);
+}
+
+static void IRAM_ATTR gpio_isr_handler(void* arg)
+{
+    GpioResult gpioResult;
+    gpioResult.gpio = *(gpio_num_t*) arg;
+    gpioResult.value = gpio_get_level(gpioResult.gpio);
+    BaseType_t ContextSwitchRequest = pdFALSE;
+ 
+    xQueueSendToBackFromISR(gpio_queue_handle,(void*)&gpioResult,&ContextSwitchRequest);
+   
+    if(ContextSwitchRequest){
+        taskYIELD();
+    }
+}
+
+static void gpioHandlerTask(void *arg) {
+    ESP_LOGD(TAG_SERVERGPIO,"start interrupt task");
+    while(1){
+        if(uxQueueMessagesWaiting(gpio_queue_handle)){
+            while(uxQueueMessagesWaiting(gpio_queue_handle)){
+                GpioResult gpioResult;
+                xQueueReceive(gpio_queue_handle,(void*)&gpioResult,10);
+                ESP_LOGD(TAG_SERVERGPIO,"gpio: %d state: %d", gpioResult.gpio, gpioResult.value);
+                ((GpioHandler*)arg)->gpioInterrupt(&gpioResult);
+            }  
+        }
+
+        ((GpioHandler*)arg)->taskHandler();
+        vTaskDelay(pdMS_TO_TICKS(1000));
+    }
+}
+
+void GpioPin::gpioInterrupt(int value) {
+    if (_mqttTopic != "") {
+        ESP_LOGD(TAG_SERVERGPIO, "gpioInterrupt %s %d", _mqttTopic.c_str(), value);
+
+        MQTTPublish(_mqttTopic, value ? "true" : "false");
+        currentState = value;
+    }
+}
+
+void GpioPin::init()
+{
+    gpio_config_t io_conf;
+    //set interrupt
+    io_conf.intr_type = _interruptType;
+    //set as output mode
+    io_conf.mode = (_mode == GPIO_PIN_MODE_OUTPUT) || (_mode == GPIO_PIN_MODE_BUILT_IN_FLASH_LED) ? gpio_mode_t::GPIO_MODE_OUTPUT : gpio_mode_t::GPIO_MODE_INPUT;
+    //bit mask of the pins that you want to set,e.g.GPIO18/19
+    io_conf.pin_bit_mask = (1ULL << _gpio);
+    //set pull-down mode
+    io_conf.pull_down_en = _mode == GPIO_PIN_MODE_INPUT_PULLDOWN ? gpio_pulldown_t::GPIO_PULLDOWN_ENABLE : gpio_pulldown_t::GPIO_PULLDOWN_DISABLE;
+    //set pull-up mode
+    io_conf.pull_up_en = _mode == GPIO_PIN_MODE_INPUT_PULLDOWN ? gpio_pullup_t::GPIO_PULLUP_ENABLE : gpio_pullup_t::GPIO_PULLUP_DISABLE;
+    //configure GPIO with the given settings
+    gpio_config(&io_conf);
+
+//    if (_interruptType != GPIO_INTR_DISABLE) {                // ohne GPIO_PIN_MODE_EXTERNAL_FLASH_WS281X, wenn das genutzt wird, dann soll auch der Handler hier nicht initialisiert werden, da das dann über SmartLED erfolgt.
+    if ((_interruptType != GPIO_INTR_DISABLE) && (_interruptType != GPIO_PIN_MODE_EXTERNAL_FLASH_WS281X)) {
+        //hook isr handler for specific gpio pin
+        ESP_LOGD(TAG_SERVERGPIO, "GpioPin::init add isr handler for GPIO %d\r\n", _gpio);
+        gpio_isr_handler_add(_gpio, gpio_isr_handler, (void*)&_gpio);
+    }
+
+    if ((_mqttTopic != "") && ((_mode == GPIO_PIN_MODE_OUTPUT) || (_mode == GPIO_PIN_MODE_OUTPUT_PWM) || (_mode == GPIO_PIN_MODE_BUILT_IN_FLASH_LED))) {
+        std::function<bool(std::string, char*, int)> f = std::bind(&GpioPin::handleMQTT, this, std::placeholders::_1, std::placeholders::_2, std::placeholders::_3);
+        MQTTregisterSubscribeFunction(_mqttTopic, f);
+    }
+}
+
+bool GpioPin::getValue(std::string* errorText)
+{   
+    if ((_mode != GPIO_PIN_MODE_INPUT) && (_mode != GPIO_PIN_MODE_INPUT_PULLUP) && (_mode != GPIO_PIN_MODE_INPUT_PULLDOWN)) {
+        (*errorText) = "GPIO is not in input mode";
+    }
+
+    return gpio_get_level(_gpio) == 1;
+}
+
+void GpioPin::setValue(bool value, gpio_set_source setSource, std::string* errorText)
+{
+    ESP_LOGD(TAG_SERVERGPIO, "GpioPin::setValue %d\r\n", value);
+
+    if ((_mode != GPIO_PIN_MODE_OUTPUT) && (_mode != GPIO_PIN_MODE_OUTPUT_PWM) && (_mode != GPIO_PIN_MODE_BUILT_IN_FLASH_LED)) {
+        (*errorText) = "GPIO is not in output mode";
+    } else {
+        gpio_set_level(_gpio, value);
+
+        if ((_mqttTopic != "") && (setSource != GPIO_SET_SOURCE_MQTT)) {
+            MQTTPublish(_mqttTopic, value ? "true" : "false");
+        }
+    }
+}
+
+void GpioPin::publishState() {
+    int newState = gpio_get_level(_gpio);
+    if (newState != currentState) {
+        ESP_LOGD(TAG_SERVERGPIO,"publish state of GPIO %d new state %d", _gpio, newState);
+        MQTTPublish(_mqttTopic, newState ? "true" : "false");
+        currentState = newState;
+    }
+}
+
+bool GpioPin::handleMQTT(std::string, char* data, int data_len) {
+    ESP_LOGD(TAG_SERVERGPIO, "GpioPin::handleMQTT data %.*s\r\n", data_len, data);
+
+    std::string dataStr(data, data_len);
+    dataStr = toLower(dataStr);
+    std::string errorText = "";
+    if ((dataStr == "true") || (dataStr == "1")) {
+        setValue(true, GPIO_SET_SOURCE_MQTT, &errorText);
+    } else if ((dataStr == "false") || (dataStr == "0")) {
+        setValue(false, GPIO_SET_SOURCE_MQTT, &errorText);    
+    } else {
+        errorText = "wrong value ";
+        errorText.append(data, data_len);
+    }
+
+    if (errorText != "") {
+        ESP_LOGE(TAG_SERVERGPIO, "%s", errorText.c_str());
+    }
+
+    return (errorText == "");
+}
+
+
+esp_err_t callHandleHttpRequest(httpd_req_t *req)
+{
+    ESP_LOGD(TAG_SERVERGPIO,"callHandleHttpRequest");
+
+    GpioHandler *gpioHandler = (GpioHandler*)req->user_ctx;
+    return gpioHandler->handleHttpRequest(req);
+}
+
+void taskGpioHandler(void *pvParameter)
+{
+    ESP_LOGD(TAG_SERVERGPIO,"taskGpioHandler");
+    ((GpioHandler*)pvParameter)->init();
+}
+
+GpioHandler::GpioHandler(std::string configFile, httpd_handle_t httpServer) 
+{
+    ESP_LOGI(TAG_SERVERGPIO,"start GpioHandler");
+    _configFile = configFile;
+    _httpServer = httpServer;
+
+    ESP_LOGI(TAG_SERVERGPIO, "register GPIO Uri");
+    registerGpioUri();
+}
+
+GpioHandler::~GpioHandler()  {
+    if (gpioMap != NULL) {
+        clear();
+        delete gpioMap;
+    }
+}
+
+void GpioHandler::init()
+{
+    // TickType_t xDelay = 60000 / portTICK_PERIOD_MS;
+    // printf("wait before start %ldms\r\n", (long) xDelay);
+    // vTaskDelay( xDelay );
+
+    printf("*************** Start GPIOHandler_Init *****************\n");
+
+    if (gpioMap == NULL) {
+        gpioMap = new std::map<gpio_num_t, GpioPin*>();
+    } else {
+        clear();
+    }
+    
+    
+    ESP_LOGI(TAG_SERVERGPIO, "read GPIO config and init GPIO");
+    if (!readConfig()) {
+        clear();
+        delete gpioMap;
+        gpioMap = NULL;
+        ESP_LOGI(TAG_SERVERGPIO, "GPIO init comleted, handler is disabled");
+        return;
+    }
+
+
+    for(std::map<gpio_num_t, GpioPin*>::iterator it = gpioMap->begin(); it != gpioMap->end(); ++it) {
+        it->second->init();
+    }
+
+    std::function<void()> f = std::bind(&GpioHandler::handleMQTTconnect, this);
+    MQTTregisterConnectFunction("gpio-handler", f);
+
+    if (xHandleTaskGpio == NULL) {
+        gpio_queue_handle = xQueueCreate(10,sizeof(GpioResult));
+        BaseType_t  xReturned = xTaskCreate(&gpioHandlerTask, "gpio_int", configMINIMAL_STACK_SIZE * 8, (void *)this, tskIDLE_PRIORITY + 2, &xHandleTaskGpio);
+        if(xReturned == pdPASS ) {
+            ESP_LOGD(TAG_SERVERGPIO, "xHandletaskGpioHandler started");
+        } else {
+            ESP_LOGD(TAG_SERVERGPIO, "xHandletaskGpioHandler not started %d ", (int)xHandleTaskGpio);
+        }
+    }
+
+    ESP_LOGI(TAG_SERVERGPIO, "GPIO init comleted, is enabled");
+}
+
+void GpioHandler::taskHandler() {
+    if (gpioMap != NULL) {
+        for(std::map<gpio_num_t, GpioPin*>::iterator it = gpioMap->begin(); it != gpioMap->end(); ++it) {
+            if ((it->second->getInterruptType() == GPIO_INTR_DISABLE))
+                it->second->publishState();
+        }
+    }
+}
+
+
+void GpioHandler::handleMQTTconnect()
+{
+    if (gpioMap != NULL) {
+        for(std::map<gpio_num_t, GpioPin*>::iterator it = gpioMap->begin(); it != gpioMap->end(); ++it) {
+            if ((it->second->getMode() == GPIO_PIN_MODE_INPUT) || (it->second->getMode() == GPIO_PIN_MODE_INPUT_PULLDOWN) || (it->second->getMode() == GPIO_PIN_MODE_INPUT_PULLUP))
+                it->second->publishState();
+        }
+    }
+}
+
+void GpioHandler::deinit() {
+    MQTTunregisterConnectFunction("gpio-handler");
+    clear();
+    if (xHandleTaskGpio != NULL) {
+        vTaskDelete(xHandleTaskGpio);
+        xHandleTaskGpio = NULL;
+    }
+}
+
+void GpioHandler::gpioInterrupt(GpioResult* gpioResult) {
+    if ((gpioMap != NULL) && (gpioMap->find(gpioResult->gpio) != gpioMap->end())) {
+        (*gpioMap)[gpioResult->gpio]->gpioInterrupt(gpioResult->value);
+    }
+}
+
+bool GpioHandler::readConfig() 
+{
+    if (!gpioMap->empty())
+        clear();
+
+    ConfigFile configFile = ConfigFile(_configFile); 
+
+    std::vector<std::string> zerlegt;
+    std::string line = "";
+    bool disabledLine = false;
+    bool eof = false;
+    gpio_num_t gpioExtLED = (gpio_num_t) 0;
+    
+//    printf("readConfig - Start 1\n");
+        
+    while ((!configFile.GetNextParagraph(line, disabledLine, eof) || (line.compare("[GPIO]") != 0)) && !eof) {}
+    if (eof)
+        return false;
+
+//    printf("readConfig - Start 2 line: %s, disabbledLine: %d\n", line.c_str(), (int) disabledLine);
+
+
+    _isEnabled = !disabledLine;
+
+    if (!_isEnabled)
+        return false;
+
+//    printf("readConfig - Start 3\n");
+
+//    std::string mainTopicMQTT = "";
+    std::string mainTopicMQTT = GetMQTTMainTopic();
+    if (mainTopicMQTT.length() > 0)
+    {
+        mainTopicMQTT = mainTopicMQTT + "/GPIO";
+        ESP_LOGD(TAG_SERVERGPIO, "MAINTOPICMQTT found\r\n");
+    }
+
+    bool registerISR = false;
+    while (configFile.getNextLine(&line, disabledLine, eof) && !configFile.isNewParagraph(line))
+    {
+        zerlegt = configFile.ZerlegeZeile(line);
+        // const std::regex pieces_regex("IO([0-9]{1,2})");
+        // std::smatch pieces_match;
+        // if (std::regex_match(zerlegt[0], pieces_match, pieces_regex) && (pieces_match.size() == 2))
+        // {
+        //     std::string gpioStr = pieces_match[1];
+        ESP_LOGD(TAG_SERVERGPIO, "conf param %s\r\n", toUpper(zerlegt[0]).c_str());
+        if (toUpper(zerlegt[0]) == "MAINTOPICMQTT") {
+//            ESP_LOGD(TAG_SERVERGPIO, "MAINTOPICMQTT found\r\n");
+//            mainTopicMQTT = zerlegt[1];
+        } else if ((zerlegt[0].rfind("IO", 0) == 0) && (zerlegt.size() >= 6))
+        {
+            ESP_LOGI(TAG_SERVERGPIO,"Enable GP%s in %s mode", zerlegt[0].c_str(), zerlegt[1].c_str());
+            std::string gpioStr = zerlegt[0].substr(2, 2);
+            gpio_num_t gpioNr = (gpio_num_t)atoi(gpioStr.c_str());
+            gpio_pin_mode_t pinMode = resolvePinMode(toLower(zerlegt[1]));
+            gpio_int_type_t intType = resolveIntType(toLower(zerlegt[2]));
+            uint16_t dutyResolution = (uint8_t)atoi(zerlegt[3].c_str());
+            bool mqttEnabled = toLower(zerlegt[4]) == "true";
+            bool httpEnabled = toLower(zerlegt[5]) == "true";
+            char gpioName[100];
+            if (zerlegt.size() >= 7) {
+                strcpy(gpioName, trim(zerlegt[6]).c_str());
+            } else {
+                sprintf(gpioName, "GPIO%d", gpioNr);
+            }
+            std::string mqttTopic = mqttEnabled ? (mainTopicMQTT + "/" + gpioName) : "";
+            GpioPin* gpioPin = new GpioPin(gpioNr, gpioName, pinMode, intType,dutyResolution, mqttTopic, httpEnabled);
+            (*gpioMap)[gpioNr] = gpioPin;
+
+            if (pinMode == GPIO_PIN_MODE_EXTERNAL_FLASH_WS281X)
+            {
+                printf("Set WS2812 to GPIO %d\n", gpioNr);
+                gpioExtLED = gpioNr;
+            }
+
+            if (intType != GPIO_INTR_DISABLE) {
+                registerISR = true;
+            }
+        }
+        if (toUpper(zerlegt[0]) == "LEDNUMBERS")
+        {
+            LEDNumbers = stoi(zerlegt[1]);
+        }
+        if (toUpper(zerlegt[0]) == "LEDCOLOR")
+        {
+            uint8_t _r, _g, _b;
+            _r = stoi(zerlegt[1]);
+            _g = stoi(zerlegt[2]);
+            _b = stoi(zerlegt[3]);
+
+            LEDColor = Rgb{_r, _g, _b};
+        }
+        if (toUpper(zerlegt[0]) == "LEDTYPE")
+        {
+            if (zerlegt[1] == "WS2812")
+                LEDType = LED_WS2812;
+            if (zerlegt[1] == "WS2812B")
+                LEDType = LED_WS2812B;
+            if (zerlegt[1] == "SK6812")
+                LEDType = LED_SK6812;
+            if (zerlegt[1] == "WS2813")
+                LEDType = LED_WS2813;
+        }
+    }
+
+    if (registerISR) {
+        //install gpio isr service
+        gpio_install_isr_service(ESP_INTR_FLAG_LOWMED | ESP_INTR_FLAG_IRAM);
+    }
+
+    if (gpioExtLED > 0)
+    {
+    //     LogFile.WriteToFile("Startsequence 06");      // Nremove
+//        vTaskDelay( xDelay );   
+//        xDelay = 5000 / portTICK_PERIOD_MS;
+//        printf("main: sleep for : %ldms\n", (long) xDelay);
+
+//        SmartLed leds( LED_WS2812, 2, GPIO_NUM_12, 0, DoubleBuffer );
+
+
+//        leds[ 0 ] = Rgb{ 255, 0, 0 };
+//        leds[ 1 ] = Rgb{ 255, 255, 255 };
+//        leds.show();    
+//        SmartLed leds = new SmartLed(LEDType, LEDNumbers, gpioExtLED, 0, DoubleBuffer);
+//        _SmartLED = new SmartLed( LED_WS2812, 2, GPIO_NUM_12, 0, DoubleBuffer );
+    }
+
+    return true;
+}
+
+void GpioHandler::clear() 
+{
+    ESP_LOGD(TAG_SERVERGPIO, "GpioHandler::clear\r\n");
+
+    if (gpioMap != NULL) {
+        for(std::map<gpio_num_t, GpioPin*>::iterator it = gpioMap->begin(); it != gpioMap->end(); ++it) {
+            delete it->second;
+        }
+        gpioMap->clear();
+    }
+
+    // gpio_uninstall_isr_service(); can't uninstall, isr service is used by camera
+}
+ 
+void GpioHandler::registerGpioUri() 
+{
+    ESP_LOGI(TAG_SERVERGPIO, "server_GPIO - Registering URI handlers");
+    
+    httpd_uri_t camuri = { };
+    camuri.method    = HTTP_GET;
+    camuri.uri       = "/GPIO";
+    camuri.handler   = callHandleHttpRequest;
+    camuri.user_ctx  = (void*)this;    
+    httpd_register_uri_handler(_httpServer, &camuri);
+}
+
+esp_err_t GpioHandler::handleHttpRequest(httpd_req_t *req)
+{
+    ESP_LOGD(TAG_SERVERGPIO, "handleHttpRequest");
+
+    if (gpioMap == NULL) {
+        std::string resp_str = "GPIO handler not initialized";
+        httpd_resp_send(req, resp_str.c_str(), resp_str.length());    
+        return ESP_OK;
+    }
+
 #ifdef DEBUG_DETAIL_ON 
     LogFile.WriteHeapInfo("handler_switch_GPIO - Start");    
 #endif
@@ -30,95 +462,223 @@ esp_err_t handler_switch_GPIO(httpd_req_t *req)
     char _query[200];
     char _valueGPIO[30];    
     char _valueStatus[30];    
-    std::string gpio, status, zw;
-    int gpionum = 0;
-    gpio_num_t gpio_num;
+    std::string gpio, status;
 
-    if (httpd_req_get_url_query_str(req, _query, 200) == ESP_OK)
-    {
-        printf("Query: "); printf(_query); printf("\n");
+    if (httpd_req_get_url_query_str(req, _query, 200) == ESP_OK) {
+        ESP_LOGD(TAG_SERVERGPIO, "Query: %s", _query);
         
         if (httpd_query_key_value(_query, "GPIO", _valueGPIO, 30) == ESP_OK)
         {
-            printf("GPIO is found"); printf(_valueGPIO); printf("\n"); 
+            ESP_LOGD(TAG_SERVERGPIO, "GPIO is found %s", _valueGPIO); 
             gpio = std::string(_valueGPIO);
+        } else {
+            std::string resp_str = "GPIO No is not defined";
+            httpd_resp_send(req, resp_str.c_str(), resp_str.length());    
+            return ESP_OK;
         }
         if (httpd_query_key_value(_query, "Status", _valueStatus, 30) == ESP_OK)
         {
-            printf("Status is found"); printf(_valueStatus); printf("\n"); 
+            ESP_LOGD(TAG_SERVERGPIO, "Status is found %s", _valueStatus); 
             status = std::string(_valueStatus);
         }
-    };
+    } else {
+        const char* resp_str = "Error in call. Use /GPIO?GPIO=12&Status=high";
+        httpd_resp_send(req, resp_str, strlen(resp_str));    
+        return ESP_OK;
+    }
 
     status = toUpper(status);
-    if (!(status == "HIGH") && !(status == "LOW"))
+    if ((status != "HIGH") && (status != "LOW") && (status != "TRUE") && (status != "FALSE") && (status != "0") && (status != "1") && (status != ""))
     {
-        zw = "Status not valid: " + status;;
+        std::string zw = "Status not valid: " + status;
         httpd_resp_sendstr_chunk(req, zw.c_str());
         httpd_resp_sendstr_chunk(req, NULL);          
         return ESP_OK;    
     }
 
-    gpionum = stoi(gpio);
-    
-    // frei: 16; 12-15; 2; 4  // nur 12 und 13 funktionieren 2: reboot, 4: BlitzLED, 14/15: DMA für SDKarte ???
+    int gpionum = stoi(gpio);
 
-    switch (gpionum) {
-        case 12:
-            gpio_num = GPIO_NUM_12;
-            break;
-        case 13:
-            gpio_num = GPIO_NUM_13;
-            break;
-        default:
-            zw = "GPIO" + std::to_string(gpionum) + " not support - only 12 & 13 free";
+    // frei: 16; 12-15; 2; 4  // nur 12 und 13 funktionieren 2: reboot, 4: BlitzLED, 15: PSRAM, 14/15: DMA für SDKarte ???
+    gpio_num_t gpio_num = resolvePinNr(gpionum);
+    if (gpio_num == GPIO_NUM_NC)
+    {
+        std::string zw = "GPIO" + std::to_string(gpionum) + " not support - only 12 & 13 free";
             httpd_resp_sendstr_chunk(req, zw.c_str());
             httpd_resp_sendstr_chunk(req, NULL);          
-            return ESP_OK;    
+            return ESP_OK;
     }
 
-    if (status == "HIGH")  
-        gpio_set_level(gpio_num, 1);
+    if (gpioMap->count(gpio_num) == 0) {
+        char resp_str [30];
+        sprintf(resp_str, "GPIO%d is not registred", gpio_num);
+        httpd_resp_send(req, resp_str, strlen(resp_str));  
+        return ESP_OK;     
+    }
+    
+    if (status == "") 
+    {
+        std::string resp_str = "";
+        status = (*gpioMap)[gpio_num]->getValue(&resp_str) ? "HIGH" : "LOW";
+        if (resp_str == "") {
+            resp_str = status;
+        }
+        httpd_resp_sendstr_chunk(req, resp_str.c_str());
+        httpd_resp_sendstr_chunk(req, NULL);
+    }
     else
-        gpio_set_level(gpio_num, 0); 
-
-
-    zw = "GPIO" + std::to_string(gpionum) + " switched to " + status;
-    httpd_resp_sendstr_chunk(req, zw.c_str());
-    httpd_resp_sendstr_chunk(req, NULL);          
+    {
+        std::string resp_str = "";
+        (*gpioMap)[gpio_num]->setValue((status == "HIGH") || (status == "TRUE") || (status == "1"), GPIO_SET_SOURCE_HTTP, &resp_str);
+        if (resp_str == "") {
+            resp_str = "GPIO" + std::to_string(gpionum) + " switched to " + status;
+        }
+        httpd_resp_sendstr_chunk(req, resp_str.c_str());
+        httpd_resp_sendstr_chunk(req, NULL);
+    }
+          
     return ESP_OK;    
 };
 
-void initGPIO()
+void GpioHandler::flashLightEnable(bool value) 
 {
-    gpio_config_t io_conf;
-    //disable interrupt
-    io_conf.intr_type = GPIO_INTR_DISABLE;
-    //set as output mode
-    io_conf.mode = GPIO_MODE_OUTPUT;
-    //bit mask of the pins that you want to set,e.g.GPIO18/19
-//    io_conf.pin_bit_mask = ((1ULL<<GPIO_OUTPUT_IO_0) | (1ULL<<GPIO_OUTPUT_IO_1));
-//    io_conf.pin_bit_mask = ((1ULL << GPIO_NUM_12) | (1ULL << GPIO_NUM_2) | (1ULL << GPIO_NUM_4) | (1ULL << GPIO_NUM_12) | (1ULL << GPIO_NUM_13) | (1ULL << GPIO_NUM_14) | (1ULL << GPIO_NUM_15));
-    io_conf.pin_bit_mask = ((1ULL << GPIO_NUM_12) | (1ULL << GPIO_NUM_13));
-    //disable pull-down mode
-    io_conf.pull_down_en = (gpio_pulldown_t) 0;
-    //disable pull-up mode
-    io_conf.pull_up_en = (gpio_pullup_t) 0;
-    //configure GPIO with the given settings
-    gpio_config(&io_conf);
+    ESP_LOGD(TAG_SERVERGPIO, "GpioHandler::flashLightEnable %s\r\n", value ? "true" : "false");
+
+    if (gpioMap != NULL) {
+        for(std::map<gpio_num_t, GpioPin*>::iterator it = gpioMap->begin(); it != gpioMap->end(); ++it) 
+        {
+            if (it->second->getMode() == GPIO_PIN_MODE_BUILT_IN_FLASH_LED) //|| (it->second->getMode() == GPIO_PIN_MODE_EXTERNAL_FLASH_PWM) || (it->second->getMode() == GPIO_PIN_MODE_EXTERNAL_FLASH_WS281X))
+            {
+                std::string resp_str = "";
+                it->second->setValue(value, GPIO_SET_SOURCE_INTERNAL, &resp_str);
+
+                if (resp_str == "") {
+                    ESP_LOGD(TAG_SERVERGPIO, "Flash light pin GPIO %d switched to %s\r\n", (int)it->first, (value ? "on" : "off"));
+                } else {
+                    ESP_LOGE(TAG_SERVERGPIO, "Can't set flash light pin GPIO %d.  Error: %s\r\n", (int)it->first, resp_str.c_str());
+                }
+            } else 
+                {
+                    if (it->second->getMode() == GPIO_PIN_MODE_EXTERNAL_FLASH_WS281X)
+                    {
+#ifdef __LEDGLOBAL
+                        if (leds_global == NULL) {
+                            ESP_LOGI(TAG_SERVERGPIO, "init SmartLed: LEDNumber=%d, GPIO=%d", LEDNumbers, (int)it->second->getGPIO());
+                            leds_global = new SmartLed( LEDType, LEDNumbers, it->second->getGPIO(), 0, DoubleBuffer );
+                        } else {
+                            // wait until we can update: https://github.com/RoboticsBrno/SmartLeds/issues/10#issuecomment-386921623
+                            leds_global->wait();
+                        }
+#else
+                        SmartLed leds( LEDType, LEDNumbers, it->second->getGPIO(), 0, DoubleBuffer );
+#endif
+  
+                        if (value)
+                        {
+                            for (int i = 0; i < LEDNumbers; ++i)
+#ifdef __LEDGLOBAL
+                                (*leds_global)[i] = LEDColor;
+#else
+                                leds[i] = LEDColor;
+#endif
+                        }
+                        else
+                        {
+                            for (int i = 0; i < LEDNumbers; ++i)
+#ifdef __LEDGLOBAL
+                                (*leds_global)[i] = Rgb{0, 0, 0};
+#else
+                                leds[i] = Rgb{0, 0, 0};
+#endif
+                        }
+#ifdef __LEDGLOBAL
+                        leds_global->show(); 
+#else
+                        leds.show(); 
+#endif 
+                    }
+                }
+        }
+    }
 }
 
-
-void register_server_GPIO_uri(httpd_handle_t server)
+gpio_num_t GpioHandler::resolvePinNr(uint8_t pinNr) 
 {
-    ESP_LOGI(TAGPARTGPIO, "server_GPIO - Registering URI handlers");
-    
-    httpd_uri_t camuri = { };
-    camuri.method    = HTTP_GET;
-    camuri.uri       = "/GPIO";
-    camuri.handler   = handler_switch_GPIO;
-    camuri.user_ctx  = (void*) "switch GPIO";    
-    httpd_register_uri_handler(server, &camuri);
-
-    initGPIO();
+    switch(pinNr)  {
+        case 0:
+            return GPIO_NUM_0;
+        case 1:
+            return GPIO_NUM_1;
+        case 3:
+            return GPIO_NUM_3;
+        case 4:
+            return GPIO_NUM_4;
+        case 12:
+            return GPIO_NUM_12;
+        case 13:
+            return GPIO_NUM_13;
+        default: 
+            return GPIO_NUM_NC;   
+    }
+}
+
+gpio_pin_mode_t GpioHandler::resolvePinMode(std::string input) 
+{
+    if( input == "disabled" ) return GPIO_PIN_MODE_DISABLED;
+    if( input == "input" ) return GPIO_PIN_MODE_INPUT;
+    if( input == "input-pullup" ) return GPIO_PIN_MODE_INPUT_PULLUP;
+    if( input == "input-pulldown" ) return GPIO_PIN_MODE_INPUT_PULLDOWN;
+    if( input == "output" ) return GPIO_PIN_MODE_OUTPUT;
+    if( input == "built-in-led" ) return GPIO_PIN_MODE_BUILT_IN_FLASH_LED;
+    if( input == "output-pwm" ) return GPIO_PIN_MODE_OUTPUT_PWM;
+    if( input == "external-flash-pwm" ) return GPIO_PIN_MODE_EXTERNAL_FLASH_PWM;
+    if( input == "external-flash-ws281x" ) return GPIO_PIN_MODE_EXTERNAL_FLASH_WS281X;
+
+    return GPIO_PIN_MODE_DISABLED;
+}
+
+gpio_int_type_t GpioHandler::resolveIntType(std::string input) 
+{
+    if( input == "disabled" ) return GPIO_INTR_DISABLE;
+    if( input == "rising-edge" ) return GPIO_INTR_POSEDGE;
+    if( input == "falling-edge" ) return GPIO_INTR_NEGEDGE;
+    if( input == "rising-and-falling" ) return GPIO_INTR_ANYEDGE ;
+    if( input == "low-level-trigger" ) return GPIO_INTR_LOW_LEVEL;
+    if( input == "high-level-trigger" ) return GPIO_INTR_HIGH_LEVEL;
+
+
+    return GPIO_INTR_DISABLE;
+}
+
+static GpioHandler *gpioHandler = NULL;
+
+void gpio_handler_create(httpd_handle_t server) 
+{
+    if (gpioHandler == NULL)
+        gpioHandler = new GpioHandler(CONFIG_FILE, server);
+}
+
+void gpio_handler_init() 
+{
+    if (gpioHandler != NULL) {
+        gpioHandler->init();
+    }
+}
+
+void gpio_handler_deinit() {
+    if (gpioHandler != NULL) {
+        gpioHandler->deinit();
+   }
+}
+
+void gpio_handler_destroy()
+{
+    if (gpioHandler != NULL) {
+        delete gpioHandler;
+        gpioHandler = NULL;
+    }
+}
+
+GpioHandler* gpio_handler_get()
+{
+    return gpioHandler;
 }

code/components/jomjol_controlGPIO/server_GPIO.h

@@ -1,10 +1,112 @@
+#ifndef SERVER_GPIO_H
+#define SERVER_GPIO_H
+
 #include <esp_log.h>
 
 #include <esp_http_server.h>
+#include <map>
+#include "driver/gpio.h"
+
+#include "SmartLeds.h"
 
 //#include "ClassControllCamera.h"
 
-static const char *TAGPARTGPIO = "server_GPIO";
+// wenn __LEDGLOBAL definiert ist, wird eine globale Variable für die LED-Ansteuerung verwendet, ansonsten lokal und jedesmal neu
+#define __LEDGLOBAL
 
-void register_server_GPIO_uri(httpd_handle_t server);
+typedef enum {
+    GPIO_PIN_MODE_DISABLED              = 0x0,
+    GPIO_PIN_MODE_INPUT                 = 0x1,
+    GPIO_PIN_MODE_INPUT_PULLUP          = 0x2,
+    GPIO_PIN_MODE_INPUT_PULLDOWN        = 0x3,
+    GPIO_PIN_MODE_OUTPUT                = 0x4,
+    GPIO_PIN_MODE_BUILT_IN_FLASH_LED    = 0x5,
+    GPIO_PIN_MODE_OUTPUT_PWM            = 0x6,
+    GPIO_PIN_MODE_EXTERNAL_FLASH_PWM    = 0x7,
+    GPIO_PIN_MODE_EXTERNAL_FLASH_WS281X = 0x8,
+} gpio_pin_mode_t;
 
+struct GpioResult {
+    gpio_num_t gpio;
+    int value;
+};
+
+typedef enum {
+    GPIO_SET_SOURCE_INTERNAL  = 0,
+    GPIO_SET_SOURCE_MQTT  = 1,
+    GPIO_SET_SOURCE_HTTP  = 2,
+} gpio_set_source;
+
+class GpioPin {
+public:
+    GpioPin(gpio_num_t gpio, const char* name, gpio_pin_mode_t mode, gpio_int_type_t interruptType, uint8_t dutyResolution, std::string mqttTopic, bool httpEnable);
+    ~GpioPin();
+
+    void init();
+    bool getValue(std::string* errorText);
+    void setValue(bool value, gpio_set_source setSource, std::string* errorText);
+    bool handleMQTT(std::string, char* data, int data_len);
+    void publishState();
+    void gpioInterrupt(int value);
+    gpio_int_type_t getInterruptType() { return _interruptType; }
+    gpio_pin_mode_t getMode() { return _mode; }
+    gpio_num_t getGPIO(){return _gpio;};
+
+private:
+    gpio_num_t _gpio;
+    const char* _name;
+    gpio_pin_mode_t _mode;
+    gpio_int_type_t _interruptType;
+    std::string _mqttTopic;
+    int currentState = -1;
+};
+
+esp_err_t callHandleHttpRequest(httpd_req_t *req);
+void taskGpioHandler(void *pvParameter);
+
+class GpioHandler {
+public:
+    GpioHandler(std::string configFile, httpd_handle_t httpServer);
+    ~GpioHandler();
+    
+    void init();
+    void deinit();
+    void registerGpioUri();
+    esp_err_t handleHttpRequest(httpd_req_t *req);
+    void taskHandler();
+    void gpioInterrupt(GpioResult* gpioResult);  
+    void flashLightEnable(bool value);
+    bool isEnabled() { return _isEnabled; }
+    void handleMQTTconnect();
+
+private:
+    std::string _configFile;
+    httpd_handle_t _httpServer;
+    std::map<gpio_num_t, GpioPin*> *gpioMap = NULL;
+    TaskHandle_t xHandleTaskGpio = NULL;
+    bool _isEnabled = false;
+
+    int LEDNumbers = 2;
+    Rgb LEDColor = Rgb{ 255, 255, 255 };
+    LedType LEDType = LED_WS2812;
+#ifdef __LEDGLOBAL
+    SmartLed *leds_global = NULL;
+#endif
+
+    bool readConfig();
+    void clear();
+    
+    gpio_num_t resolvePinNr(uint8_t pinNr);
+    gpio_pin_mode_t resolvePinMode(std::string input);
+    gpio_int_type_t resolveIntType(std::string input);
+};
+
+void gpio_handler_create(httpd_handle_t server);
+void gpio_handler_init();
+void gpio_handler_deinit();
+void gpio_handler_destroy();
+GpioHandler* gpio_handler_get();
+
+
+
+#endif //SERVER_GPIO_H

code/components/jomjol_controlcamera/CMakeLists.txt

@@ -4,6 +4,6 @@ list(APPEND EXTRA_COMPONENT_DIRS $ENV{IDF_PATH}/examples/common_components/proto
 
 idf_component_register(SRCS ${app_sources}
                     INCLUDE_DIRS "."
-                    REQUIRES esp32-camera-master esp_http_server jomjol_logfile jomjol_image_proc nvs_flash jomjol_fileserver_ota)
+                    REQUIRES esp32-camera-master esp_http_server jomjol_logfile jomjol_image_proc nvs_flash jomjol_fileserver_ota jomjol_controlGPIO)
 
 

code/components/jomjol_controlcamera/ClassControllCamera.cpp

@@ -10,12 +10,13 @@
 #include "CImageBasis.h"
 
 #include "server_ota.h"
+#include "server_GPIO.h"
 
 
 #define BOARD_ESP32CAM_AITHINKER
 
 
-#include <esp_event_loop.h>
+#include <esp_event.h>
 #include <esp_log.h>
 #include <esp_system.h>
 #include <nvs_flash.h>
@@ -50,7 +51,7 @@
 #define CAM_PIN_HREF 23
 #define CAM_PIN_PCLK 22
 
-static const char *TAG = "example:take_picture";
+static const char *TAGCAMERACLASS = "server_part_camera"; 
 
 static camera_config_t camera_config = {
     .pin_pwdn = CAM_PIN_PWDN,
@@ -73,7 +74,7 @@ static camera_config_t camera_config = {
 
     //XCLK 20MHz or 10MHz for OV2640 double FPS (Experimental)
 //    .xclk_freq_hz = 20000000,             // Orginalwert
-    .xclk_freq_hz = 5000000,               // Test, um die Bildfehler los zu werden !!!!
+    .xclk_freq_hz =    5000000,               // Test, um die Bildfehler los zu werden !!!!
     .ledc_timer = LEDC_TIMER_0,
     .ledc_channel = LEDC_CHANNEL_0,
 
@@ -85,6 +86,8 @@ static camera_config_t camera_config = {
 
     .jpeg_quality = 5, //0-63 lower number means higher quality
     .fb_count = 1       //if more than one, i2s runs in continuous mode. Use only with JPEG
+//    .grab_mode = CAMERA_GRAB_WHEN_EMPTY,
+
 };
 
 
@@ -275,16 +278,26 @@ esp_err_t CCamera::CaptureToBasisImage(CImageBasis *_Image, int delay)
 
     camera_fb_t * fb = esp_camera_fb_get();
     if (!fb) {
-        ESP_LOGE(TAGCAMERACLASS, "Camera Capture Failed");
+        ESP_LOGE(TAGCAMERACLASS, "CaptureToBasisImage: Camera Capture Failed");
         LEDOnOff(false);
         LightOnOff(false);
-        doReboot();
+
+        LogFile.SwitchOnOff(true);
+        LogFile.WriteToFile("Camera is not working anymore - most propably hardware problem (instablility, ...). "
+                "System will reboot.");
+         doReboot();
 
         return ESP_FAIL;
     }
 
     int _size = fb->len;
     zwischenspeicher = (uint8_t*) malloc(_size);
+    if (!zwischenspeicher)
+    {
+        ESP_LOGE(TAGCAMERACLASS, "Nicht ausreichend Speicherplatz für Bild in Funktion CaptureToBasisImage()");
+        LogFile.SwitchOnOff(true);
+        LogFile.WriteToFile("Nicht ausreichend Speicherplatz für Bild in Funktion CaptureToBasisImage()");
+    }
     for (int i = 0; i < _size; ++i)
         *(zwischenspeicher + i) = *(fb->buf + i);
     esp_camera_fb_return(fb);        
@@ -362,8 +375,7 @@ esp_err_t CCamera::CaptureToFile(std::string nm, int delay)
 
     camera_fb_t * fb = esp_camera_fb_get();
     if (!fb) {
-        ESP_LOGE(TAGCAMERACLASS, "Camera Capture Failed");
-        ESP_LOGE(TAGCAMERACLASS, "Reboot ?????");
+        ESP_LOGE(TAGCAMERACLASS, "CaptureToFile: Camera Capture Failed");
         LEDOnOff(false);
         LightOnOff(false);
         doReboot();
@@ -497,15 +509,21 @@ esp_err_t CCamera::CaptureToHTTP(httpd_req_t *req, int delay)
 
 void CCamera::LightOnOff(bool status)
 {
-	// Init the GPIO
-    gpio_pad_select_gpio(FLASH_GPIO);
-    /* Set the GPIO as a push/pull output */
-    gpio_set_direction(FLASH_GPIO, GPIO_MODE_OUTPUT);  
+    GpioHandler* gpioHandler = gpio_handler_get();
+    if ((gpioHandler != NULL) && (gpioHandler->isEnabled())) {
+        printf("Use gpioHandler flashLigh\n");
+        gpioHandler->flashLightEnable(status);
+    }  else {
+        // Init the GPIO
+        gpio_pad_select_gpio(FLASH_GPIO);
+        /* Set the GPIO as a push/pull output */
+        gpio_set_direction(FLASH_GPIO, GPIO_MODE_OUTPUT);  
 
-    if (status)  
-        gpio_set_level(FLASH_GPIO, 1);
-    else
-        gpio_set_level(FLASH_GPIO, 0);      
+        if (status)  
+            gpio_set_level(FLASH_GPIO, 1);
+        else
+            gpio_set_level(FLASH_GPIO, 0);
+    }
 }
 
 void CCamera::LEDOnOff(bool status)

code/components/jomjol_controlcamera/ClassControllCamera.h

@@ -16,9 +16,6 @@
 #define CAMERA_MODEL_AI_THINKER
 
 
-static const char *TAGCAMERACLASS = "server_part_camera"; 
-
-
 class CCamera {
     protected:
         int ActualQuality;

code/components/jomjol_controlcamera/sensor.h

@@ -11,11 +11,48 @@
 #include <stdint.h>
 #include <stdbool.h>
 
-#define OV9650_PID     (0x96)
-#define OV7725_PID     (0x77)
-#define OV2640_PID     (0x26)
-#define OV3660_PID     (0x36)
-#define OV5640_PID     (0x56)
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+typedef enum {
+    OV9650_PID = 0x96,
+    OV7725_PID = 0x77,
+    OV2640_PID = 0x26,
+    OV3660_PID = 0x3660,
+    OV5640_PID = 0x5640,
+    OV7670_PID = 0x76,
+    NT99141_PID = 0x1410,
+    GC2145_PID = 0x2145,
+    GC032A_PID = 0x232a,
+    GC0308_PID = 0x9b,
+} camera_pid_t;
+
+typedef enum {
+    CAMERA_OV7725,
+    CAMERA_OV2640,
+    CAMERA_OV3660,
+    CAMERA_OV5640,
+    CAMERA_OV7670,
+    CAMERA_NT99141,
+    CAMERA_GC2145,
+    CAMERA_GC032A,
+    CAMERA_GC0308,
+    CAMERA_MODEL_MAX,
+    CAMERA_NONE,
+} camera_model_t;
+
+typedef enum {
+    OV2640_SCCB_ADDR   = 0x30,// 0x60 >> 1
+    OV5640_SCCB_ADDR   = 0x3C,// 0x78 >> 1
+    OV3660_SCCB_ADDR   = 0x3C,// 0x78 >> 1
+    OV7725_SCCB_ADDR   = 0x21,// 0x42 >> 1
+    OV7670_SCCB_ADDR   = 0x21,// 0x42 >> 1
+    NT99141_SCCB_ADDR  = 0x2A,// 0x54 >> 1
+    GC2145_SCCB_ADDR   = 0x3C,// 0x78 >> 1
+    GC032A_SCCB_ADDR   = 0x21,// 0x42 >> 1
+    GC0308_SCCB_ADDR   = 0x21,// 0x42 >> 1
+} camera_sccb_addr_t;
 
 typedef enum {
     PIXFORMAT_RGB565,    // 2BPP/RGB565
@@ -56,6 +93,15 @@ typedef enum {
     FRAMESIZE_INVALID
 } framesize_t;
 
+typedef struct {
+    const camera_model_t model;
+    const char *name;
+    const camera_sccb_addr_t sccb_addr;
+    const camera_pid_t pid;
+    const framesize_t max_size;
+    const bool support_jpeg;
+} camera_sensor_info_t;
+
 typedef enum {
     ASPECT_RATIO_4X3,
     ASPECT_RATIO_3X2,
@@ -99,11 +145,13 @@ typedef struct {
 
 // Resolution table (in sensor.c)
 extern const resolution_info_t resolution[];
+// camera sensor table (in sensor.c)
+extern const camera_sensor_info_t camera_sensor[];
 
 typedef struct {
     uint8_t MIDH;
     uint8_t MIDL;
-    uint8_t PID;
+    uint16_t PID;
     uint8_t VER;
 } sensor_id_t;
 
@@ -188,4 +236,10 @@ typedef struct _sensor {
     int  (*set_xclk)            (sensor_t *sensor, int timer, int xclk);
 } sensor_t;
 
+camera_sensor_info_t *esp_camera_sensor_get_info(sensor_id_t *id);
+
+#ifdef __cplusplus
+}
+#endif
+
 #endif /* __SENSOR_H__ */

code/components/jomjol_controlcamera/server_camera.cpp

@@ -8,18 +8,21 @@
 
 #include "ClassLogFile.h"
 
-#define SCRATCH_BUFSIZE2  8192 
-char scratch2[SCRATCH_BUFSIZE2];
+// #define SCRATCH_BUFSIZE2  8192 
+// char scratch2[SCRATCH_BUFSIZE2];
 
 //#define DEBUG_DETAIL_ON   
-
+static const char *TAGPARTCAMERA = "server_camera";
 
 
 void PowerResetCamera(){
         ESP_LOGD(TAGPARTCAMERA, "Resetting camera by power down line");
-        gpio_config_t conf = { 0 };
+        gpio_config_t conf;
+        conf.intr_type = GPIO_INTR_DISABLE;
         conf.pin_bit_mask = 1LL << GPIO_NUM_32;
         conf.mode = GPIO_MODE_OUTPUT;
+        conf.pull_down_en = GPIO_PULLDOWN_DISABLE;
+        conf.pull_up_en = GPIO_PULLUP_DISABLE;
         gpio_config(&conf);
 
         // carefull, logic is inverted compared to reset pin

code/components/jomjol_controlcamera/server_camera.h

@@ -7,8 +7,6 @@
 
 //#include "ClassControllCamera.h"
 
-static const char *TAGPARTCAMERA = "server_camera";
-
 void register_server_camera_uri(httpd_handle_t server);
 
 void PowerResetCamera();

code/components/jomjol_fileserver_ota/CMakeLists.txt

@@ -1,7 +1,7 @@
 FILE(GLOB_RECURSE app_sources ${CMAKE_CURRENT_SOURCE_DIR}/*.*)
 
 idf_component_register(SRCS ${app_sources}
-                    INCLUDE_DIRS "."
-                    REQUIRES tfmicro esp_http_server app_update esp_http_client nvs_flash jomjol_tfliteclass jomjol_flowcontroll spiffs jomjol_helper)
+                    INCLUDE_DIRS "." "../../include"
+                    REQUIRES tflite-lib esp_http_server app_update esp_http_client nvs_flash jomjol_tfliteclass jomjol_flowcontroll spiffs jomjol_helper jomjol_controlGPIO)
 
 

code/components/jomjol_fileserver_ota/server_file.cpp

@@ -17,7 +17,14 @@
 #include <sys/param.h>
 #include <sys/unistd.h>
 #include <sys/stat.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
 #include <dirent.h>
+#ifdef __cplusplus
+}
+#endif
 
 #include "esp_err.h"
 #include "esp_log.h"
@@ -26,9 +33,12 @@
 #include <esp_spiffs.h>
 #include "esp_http_server.h"
 
+#include "defines.h"
 #include "ClassLogFile.h"
 
 #include "server_help.h"
+#include "interface_mqtt.h"
+#include "server_GPIO.h"
 
 #include "Helper.h"
 #include "miniz.h"
@@ -53,17 +63,63 @@ struct file_server_data {
     char scratch[SCRATCH_BUFSIZE];
 };
 
-static const char *TAG = "file_server";
+static const char *TAG_FILESERVER = "file_server";
+
+
+#include <iostream>
+#include <sys/types.h>
+#include <dirent.h>
+
+using namespace std;
+
+static esp_err_t get_tflite_file_handler(httpd_req_t *req){
+    DIR *verzeichnis;
+    struct dirent *files;
+
+    std::string _filename, _fileext, _result = "";
+    std::string _delimiter = ".";
+    size_t pos = 0;
+
+    verzeichnis=opendir("/sdcard/config");
+
+    printf("Suche TFLITE in /sdcard/config\n");
+
+    while((files = readdir(verzeichnis)))
+    {
+        _filename = files->d_name;
+        _fileext = _filename;
+        printf("File: %s\t", _filename.c_str());
+
+        while ((pos = _fileext.find(_delimiter))) {
+            _fileext.erase(0, pos + _delimiter.length());
+        }
+
+        printf(" Extension: %s\n", _fileext.c_str());
+
+        if ((_fileext == "tfl") || (_fileext == "tflite"))
+        {
+            _result = _result + _filename + "\t";
+        }
+    }
+    closedir(verzeichnis);
+
+    httpd_resp_set_hdr(req, "Access-Control-Allow-Origin", "*");
+    httpd_resp_set_type(req, "text/plain");
+    httpd_resp_sendstr_chunk(req, _result.c_str());
+    httpd_resp_sendstr_chunk(req, NULL);
+    return ESP_OK;
+}
+
 
 /* Handler to redirect incoming GET request for /index.html to /
  * This can be overridden by uploading file with same name */
-static esp_err_t index_html_get_handler(httpd_req_t *req)
-{
-    httpd_resp_set_status(req, "307 Temporary Redirect");
-    httpd_resp_set_hdr(req, "Location", "/");
-    httpd_resp_send(req, NULL, 0);  // Response body can be empty
-    return ESP_OK;
-}
+// static esp_err_t index_html_get_handler(httpd_req_t *req)
+// {
+//     httpd_resp_set_status(req, "307 Temporary Redirect");
+//     httpd_resp_set_hdr(req, "Location", "/");
+//     httpd_resp_send(req, NULL, 0);  // Response body can be empty
+//     return ESP_OK;
+// }
 
 /* Send HTTP response with a run-time generated html consisting of
  * a list of all files and folders under the requested path.
@@ -95,7 +151,7 @@ static esp_err_t http_resp_dir_html(httpd_req_t *req, const char *dirpath, const
     printf("entrypath: <%s>\n", entrypath);    
 
     if (!dir) {
-        ESP_LOGE(TAG, "Failed to stat dir : %s", dirpath);
+        ESP_LOGE(TAG_FILESERVER, "Failed to stat dir : %s", dirpath);
         /* Respond with 404 Not Found */
         httpd_resp_send_err(req, HTTPD_404_NOT_FOUND, "Directory does not exist");
         return ESP_FAIL;
@@ -115,7 +171,7 @@ static esp_err_t http_resp_dir_html(httpd_req_t *req, const char *dirpath, const
             if (chunksize > 0){
                 if (httpd_resp_send_chunk(req, chunk, chunksize) != ESP_OK) {
                 fclose(fd);
-                ESP_LOGE(TAG, "File sending failed!");
+                ESP_LOGE(TAG_FILESERVER, "File sending failed!");
                 return ESP_FAIL;
                 }
             }
@@ -154,11 +210,11 @@ static esp_err_t http_resp_dir_html(httpd_req_t *req, const char *dirpath, const
             strlcpy(entrypath + dirpath_len, entry->d_name, sizeof(entrypath) - dirpath_len);
             printf("Entrypath: %s\n", entrypath);
             if (stat(entrypath, &entry_stat) == -1) {
-                ESP_LOGE(TAG, "Failed to stat %s : %s", entrytype, entry->d_name);
+                ESP_LOGE(TAG_FILESERVER, "Failed to stat %s : %s", entrytype, entry->d_name);
                 continue;
             }
             sprintf(entrysize, "%ld", entry_stat.st_size);
-            ESP_LOGI(TAG, "Found %s : %s (%s bytes)", entrytype, entry->d_name, entrysize);
+            ESP_LOGI(TAG_FILESERVER, "Found %s : %s (%s bytes)", entrytype, entry->d_name, entrysize);
 
             /* Send chunk of HTML file containing table entries with file name and size */
             httpd_resp_sendstr_chunk(req, "<tr><td><a href=\"");
@@ -206,19 +262,19 @@ static esp_err_t logfileact_get_handler(httpd_req_t *req)
     LogFile.WriteToFile("logfileact_get_handler");
     char filepath[FILE_PATH_MAX];
     FILE *fd = NULL;
-    struct stat file_stat;
+    //struct stat file_stat;
     printf("uri: %s\n", req->uri);
 
-    const char filename = 'log_current.txt'; 
+    const char* filename = "log_current.txt"; 
 
-    printf("uri: %s, filename: %s, filepath: %s\n", req->uri, &filename, filepath);
+    printf("uri: %s, filename: %s, filepath: %s\n", req->uri, filename, filepath);
 
     std::string currentfilename = LogFile.GetCurrentFileName();
 
 
     fd = OpenFileAndWait(currentfilename.c_str(), "r");
     if (!fd) {
-        ESP_LOGE(TAG, "Failed to read existing file : %s", filepath);
+        ESP_LOGE(TAG_FILESERVER, "Failed to read existing file : %s", filepath);
         /* Respond with 500 Internal Server Error */
         httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, "Failed to read existing file");
         return ESP_FAIL;
@@ -226,8 +282,8 @@ static esp_err_t logfileact_get_handler(httpd_req_t *req)
 
     httpd_resp_set_hdr(req, "Access-Control-Allow-Origin", "*");
 
-//    ESP_LOGI(TAG, "Sending file : %s (%ld bytes)...", &filename, file_stat.st_size);
-    set_content_type_from_file(req, &filename);
+//    ESP_LOGI(TAG_FILESERVER, "Sending file : %s (%ld bytes)...", &filename, file_stat.st_size);
+    set_content_type_from_file(req, filename);
 
     /* Retrieve the pointer to scratch buffer for temporary storage */
     char *chunk = ((struct file_server_data *)req->user_ctx)->scratch;
@@ -239,7 +295,7 @@ static esp_err_t logfileact_get_handler(httpd_req_t *req)
         /* Send the buffer contents as HTTP response chunk */
         if (httpd_resp_send_chunk(req, chunk, chunksize) != ESP_OK) {
             fclose(fd);
-            ESP_LOGE(TAG, "File sending failed!");
+            ESP_LOGE(TAG_FILESERVER, "File sending failed!");
             /* Abort sending file */
             httpd_resp_sendstr_chunk(req, NULL);
             /* Respond with 500 Internal Server Error */
@@ -252,7 +308,7 @@ static esp_err_t logfileact_get_handler(httpd_req_t *req)
 
     /* Close file after sending complete */
     fclose(fd);
-    ESP_LOGI(TAG, "File sending complete");
+    ESP_LOGI(TAG_FILESERVER, "File sending complete");
 
     /* Respond with an empty chunk to signal HTTP response completion */
     httpd_resp_send_chunk(req, NULL, 0);
@@ -284,7 +340,7 @@ static esp_err_t download_get_handler(httpd_req_t *req)
 
 
     if (!filename) {
-        ESP_LOGE(TAG, "Filename is too long");
+        ESP_LOGE(TAG_FILESERVER, "Filename is too long");
         /* Respond with 500 Internal Server Error */
         httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, "Filename too long");
         return ESP_FAIL;
@@ -297,11 +353,11 @@ static esp_err_t download_get_handler(httpd_req_t *req)
         if (buf_len > 1) {
             char buf[buf_len];
             if (httpd_req_get_url_query_str(req, buf, buf_len) == ESP_OK) {
-                ESP_LOGI(TAG, "Found URL query => %s", buf);
+                ESP_LOGI(TAG_FILESERVER, "Found URL query => %s", buf);
                 char param[32];
                 /* Get value of expected key from query string */
                 if (httpd_query_key_value(buf, "readonly", param, sizeof(param)) == ESP_OK) {
-                    ESP_LOGI(TAG, "Found URL query parameter => readonly=%s", param);
+                    ESP_LOGI(TAG_FILESERVER, "Found URL query parameter => readonly=%s", param);
                     readonly = param && strcmp(param,"true")==0;
                 }
             }
@@ -316,7 +372,7 @@ static esp_err_t download_get_handler(httpd_req_t *req)
 
         /* If file not present on SPIFFS check if URI
          * corresponds to one of the hardcoded paths */
-        ESP_LOGE(TAG, "Failed to stat file : %s", filepath);
+        ESP_LOGE(TAG_FILESERVER, "Failed to stat file : %s", filepath);
         /* Respond with 404 Not Found */
         httpd_resp_send_err(req, HTTPD_404_NOT_FOUND, "File does not exist");
         return ESP_FAIL;
@@ -324,7 +380,7 @@ static esp_err_t download_get_handler(httpd_req_t *req)
 
     fd = OpenFileAndWait(filepath, "r");
     if (!fd) {
-        ESP_LOGE(TAG, "Failed to read existing file : %s", filepath);
+        ESP_LOGE(TAG_FILESERVER, "Failed to read existing file : %s", filepath);
         /* Respond with 500 Internal Server Error */
         httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, "Failed to read existing file");
         return ESP_FAIL;
@@ -332,7 +388,7 @@ static esp_err_t download_get_handler(httpd_req_t *req)
 
     httpd_resp_set_hdr(req, "Access-Control-Allow-Origin", "*");
 
-    ESP_LOGI(TAG, "Sending file : %s (%ld bytes)...", filename, file_stat.st_size);
+    ESP_LOGI(TAG_FILESERVER, "Sending file : %s (%ld bytes)...", filename, file_stat.st_size);
     set_content_type_from_file(req, filename);
 
     /* Retrieve the pointer to scratch buffer for temporary storage */
@@ -345,7 +401,7 @@ static esp_err_t download_get_handler(httpd_req_t *req)
         /* Send the buffer contents as HTTP response chunk */
         if (httpd_resp_send_chunk(req, chunk, chunksize) != ESP_OK) {
             fclose(fd);
-            ESP_LOGE(TAG, "File sending failed!");
+            ESP_LOGE(TAG_FILESERVER, "File sending failed!");
             /* Abort sending file */
             httpd_resp_sendstr_chunk(req, NULL);
             /* Respond with 500 Internal Server Error */
@@ -358,7 +414,7 @@ static esp_err_t download_get_handler(httpd_req_t *req)
 
     /* Close file after sending complete */
     fclose(fd);
-    ESP_LOGI(TAG, "File sending complete");
+    ESP_LOGI(TAG_FILESERVER, "File sending complete");
 
     /* Respond with an empty chunk to signal HTTP response completion */
     httpd_resp_send_chunk(req, NULL, 0);
@@ -385,13 +441,13 @@ static esp_err_t upload_post_handler(httpd_req_t *req)
 
     /* Filename cannot have a trailing '/' */
     if (filename[strlen(filename) - 1] == '/') {
-        ESP_LOGE(TAG, "Invalid filename : %s", filename);
+        ESP_LOGE(TAG_FILESERVER, "Invalid filename : %s", filename);
         httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, "Invalid filename");
         return ESP_FAIL;
     }
 
     if (stat(filepath, &file_stat) == 0) {
-        ESP_LOGE(TAG, "File already exists : %s", filepath);
+        ESP_LOGE(TAG_FILESERVER, "File already exists : %s", filepath);
         /* Respond with 400 Bad Request */
         httpd_resp_send_err(req, HTTPD_400_BAD_REQUEST, "File already exists");
         return ESP_FAIL;
@@ -399,7 +455,7 @@ static esp_err_t upload_post_handler(httpd_req_t *req)
 
     /* File cannot be larger than a limit */
     if (req->content_len > MAX_FILE_SIZE) {
-        ESP_LOGE(TAG, "File too large : %d bytes", req->content_len);
+        ESP_LOGE(TAG_FILESERVER, "File too large : %d bytes", req->content_len);
         /* Respond with 400 Bad Request */
         httpd_resp_send_err(req, HTTPD_400_BAD_REQUEST,
                             "File size must be less than "
@@ -411,13 +467,13 @@ static esp_err_t upload_post_handler(httpd_req_t *req)
 
     fd = OpenFileAndWait(filepath, "w");
     if (!fd) {
-        ESP_LOGE(TAG, "Failed to create file : %s", filepath);
+        ESP_LOGE(TAG_FILESERVER, "Failed to create file : %s", filepath);
         /* Respond with 500 Internal Server Error */
         httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, "Failed to create file");
         return ESP_FAIL;
     }
 
-    ESP_LOGI(TAG, "Receiving file : %s...", filename);
+    ESP_LOGI(TAG_FILESERVER, "Receiving file : %s...", filename);
 
     /* Retrieve the pointer to scratch buffer for temporary storage */
     char *buf = ((struct file_server_data *)req->user_ctx)->scratch;
@@ -429,7 +485,7 @@ static esp_err_t upload_post_handler(httpd_req_t *req)
 
     while (remaining > 0) {
 
-        ESP_LOGI(TAG, "Remaining size : %d", remaining);
+        ESP_LOGI(TAG_FILESERVER, "Remaining size : %d", remaining);
         /* Receive the file part by part into a buffer */
         if ((received = httpd_req_recv(req, buf, MIN(remaining, SCRATCH_BUFSIZE))) <= 0) {
             if (received == HTTPD_SOCK_ERR_TIMEOUT) {
@@ -442,7 +498,7 @@ static esp_err_t upload_post_handler(httpd_req_t *req)
             fclose(fd);
             unlink(filepath);
 
-            ESP_LOGE(TAG, "File reception failed!");
+            ESP_LOGE(TAG_FILESERVER, "File reception failed!");
             /* Respond with 500 Internal Server Error */
             httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, "Failed to receive file");
             return ESP_FAIL;
@@ -455,7 +511,7 @@ static esp_err_t upload_post_handler(httpd_req_t *req)
             fclose(fd);
             unlink(filepath);
 
-            ESP_LOGE(TAG, "File write failed!");
+            ESP_LOGE(TAG_FILESERVER, "File write failed!");
             /* Respond with 500 Internal Server Error */
             httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, "Failed to write file to storage");
             return ESP_FAIL;
@@ -468,7 +524,7 @@ static esp_err_t upload_post_handler(httpd_req_t *req)
 
     /* Close file upon upload completion */
     fclose(fd);
-    ESP_LOGI(TAG, "File reception complete");
+    ESP_LOGI(TAG_FILESERVER, "File reception complete");
 
     std::string directory = std::string(filepath);
 	size_t zw = directory.find("/");
@@ -483,10 +539,10 @@ static esp_err_t upload_post_handler(httpd_req_t *req)
     int start_fn = strlen(((struct file_server_data *)req->user_ctx)->base_path);
     printf("Directory: %s, start_fn: %d, found: %d\n", directory.c_str(), start_fn, found);
 	directory = directory.substr(start_fn, found - start_fn + 1);
-    printf("Directory danach: %s\n", directory.c_str());    
+    printf("Directory danach 1: %s\n", directory.c_str());    
 
     directory = "/fileserver" + directory;
-    printf("Directory danach: %s\n", directory.c_str());   
+    printf("Directory danach 2: %s\n", directory.c_str());   
 
     /* Redirect onto root to see the updated file list */
     httpd_resp_set_status(req, "303 See Other");
@@ -496,6 +552,15 @@ static esp_err_t upload_post_handler(httpd_req_t *req)
     httpd_resp_set_status(req, "303 See Other");
     httpd_resp_set_hdr(req, "Location", directory.c_str());
     httpd_resp_sendstr(req, "File uploaded successfully");
+
+/*
+    if (strcmp(filepath, CONFIG_FILE) == 0) {
+        printf("New config found. Reload handler.");
+        gpio_handler_deinit();
+        MQTTdestroy();
+    }
+*/
+
     return ESP_OK;
 }
 
@@ -567,19 +632,19 @@ static esp_err_t delete_post_handler(httpd_req_t *req)
 
         /* Filename cannot have a trailing '/' */
         if (filename[strlen(filename) - 1] == '/') {
-            ESP_LOGE(TAG, "Invalid filename : %s", filename);
+            ESP_LOGE(TAG_FILESERVER, "Invalid filename : %s", filename);
             httpd_resp_send_err(req, HTTPD_500_INTERNAL_SERVER_ERROR, "Invalid filename");
             return ESP_FAIL;
         }
 
         if (stat(filepath, &file_stat) == -1) {
-            ESP_LOGE(TAG, "File does not exist : %s", filename);
+            ESP_LOGE(TAG_FILESERVER, "File does not exist : %s", filename);
             /* Respond with 400 Bad Request */
             httpd_resp_send_err(req, HTTPD_400_BAD_REQUEST, "File does not exist");
             return ESP_FAIL;
         }
 
-        ESP_LOGI(TAG, "Deleting file : %s", filename);
+        ESP_LOGI(TAG_FILESERVER, "Deleting file : %s", filename);
         /* Delete file */
         unlink(filepath);
 
@@ -596,10 +661,10 @@ static esp_err_t delete_post_handler(httpd_req_t *req)
         int start_fn = strlen(((struct file_server_data *)req->user_ctx)->base_path);
         printf("Directory: %s, start_fn: %d, found: %d\n", directory.c_str(), start_fn, found);
         directory = directory.substr(start_fn, found - start_fn + 1);
-        printf("Directory danach: %s\n", directory.c_str());    
+        printf("Directory danach 3: %s\n", directory.c_str());    
 
         directory = "/fileserver" + directory;
-        printf("Directory danach: %s\n", directory.c_str());   
+        printf("Directory danach 4: %s\n", directory.c_str());   
     }
     
 
@@ -623,7 +688,7 @@ void delete_all_in_directory(std::string _directory)
     std::string filename;
 
     if (!dir) {
-        ESP_LOGE(TAG, "Failed to stat dir : %s", _directory.c_str());
+        ESP_LOGE(TAG_FILESERVER, "Failed to stat dir : %s", _directory.c_str());
         return;
     }
 
@@ -632,7 +697,7 @@ void delete_all_in_directory(std::string _directory)
         if (!(entry->d_type == DT_DIR)){
             if (strcmp("wlan.ini", entry->d_name) != 0){                    // auf wlan.ini soll nicht zugegriffen werden !!!
                 filename = _directory + "/" + std::string(entry->d_name);
-                ESP_LOGI(TAG, "Deleting file : %s", filename.c_str());
+                ESP_LOGI(TAG_FILESERVER, "Deleting file : %s", filename.c_str());
                 /* Delete file */
                 unlink(filename.c_str());    
             }
@@ -722,19 +787,19 @@ void register_server_file_uri(httpd_handle_t server, const char *base_path)
     /* Validate file storage base path */
     if (!base_path) {
 //    if (!base_path || strcmp(base_path, "/spiffs") != 0) {
-        ESP_LOGE(TAG, "File server base_path not set");
+        ESP_LOGE(TAG_FILESERVER, "File server base_path not set");
 //        return ESP_ERR_INVALID_ARG;
     }
 
     if (server_data) {
-        ESP_LOGE(TAG, "File server already started");
+        ESP_LOGE(TAG_FILESERVER, "File server already started");
 //        return ESP_ERR_INVALID_STATE;
     }
 
     /* Allocate memory for server data */
     server_data = (file_server_data *) calloc(1, sizeof(struct file_server_data));
     if (!server_data) {
-        ESP_LOGE(TAG, "Failed to allocate memory for server data");
+        ESP_LOGE(TAG_FILESERVER, "Failed to allocate memory for server data");
 //        return ESP_ERR_NO_MEM;
     }
     strlcpy(server_data->base_path, base_path,
@@ -785,4 +850,15 @@ void register_server_file_uri(httpd_handle_t server, const char *base_path)
     };
     httpd_register_uri_handler(server, &file_delete);
 
+
+    /* URI handler for getting tflite files from server */
+/*
+    httpd_uri_t file_tflite = {
+        .uri       = "/tflite",   // Match all URIs of type /delete/path/to/file
+        .method    = HTTP_GET,
+        .handler   = get_tflite_file_handler,
+        .user_ctx  = server_data    // Pass server data as context
+    };
+    httpd_register_uri_handler(server, &file_tflite);
+*/
 }

code/components/jomjol_fileserver_ota/server_help.cpp

@@ -5,7 +5,14 @@
 #include <sys/param.h>
 #include <sys/unistd.h>
 #include <sys/stat.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
 #include <dirent.h>
+#ifdef __cplusplus
+}
+#endif
 
 #include "esp_err.h"
 #include "esp_log.h"
@@ -107,8 +114,12 @@ esp_err_t set_content_type_from_file(httpd_req_t *req, const char *filename)
         return httpd_resp_set_type(req, "text/html");
     } else if (IS_FILE_EXT(filename, ".jpeg")) {
         return httpd_resp_set_type(req, "image/jpeg");
+    } else if (IS_FILE_EXT(filename, ".jpg")) {
+        return httpd_resp_set_type(req, "image/jpeg");
     } else if (IS_FILE_EXT(filename, ".ico")) {
         return httpd_resp_set_type(req, "image/x-icon");
+    } else if (IS_FILE_EXT(filename, ".js")) {
+        return httpd_resp_set_type(req, "text/javascript");
     }
     /* This is a limited set only */
     /* For any other type always set as plain text */

code/components/jomjol_fileserver_ota/server_ota.cpp

@@ -12,7 +12,7 @@
 #include "freertos/task.h"
 #include "esp_system.h"
 #include "esp_event.h"
-#include "esp_event_loop.h"
+#include "esp_event.h"
 #include "esp_log.h"
 #include <esp_ota_ops.h>
 #include "esp_http_client.h"
@@ -28,6 +28,8 @@
 
 #include "server_tflite.h"
 #include "server_file.h"
+#include "server_GPIO.h"
+
 
 #include "ClassLogFile.h"
 
@@ -46,6 +48,7 @@ static char ota_write_data[BUFFSIZE + 1] = { 0 };
 
 
 #define OTA_URL_SIZE 256
+static const char *TAGPARTOTA = "server_ota";
 
 
 static void infinite_loop(void)
@@ -60,14 +63,14 @@ static void infinite_loop(void)
 
 
 
-static bool ota_example_task(std::string fn)
+static bool ota_update_task(std::string fn)
 {
     esp_err_t err;
     /* update handle : set by esp_ota_begin(), must be freed via esp_ota_end() */
     esp_ota_handle_t update_handle = 0 ;
     const esp_partition_t *update_partition = NULL;
 
-    ESP_LOGI(TAGPARTOTA, "Starting OTA example");
+    ESP_LOGI(TAGPARTOTA, "Starting OTA update");
 
     const esp_partition_t *configured = esp_ota_get_boot_partition();
     const esp_partition_t *running = esp_ota_get_running_partition();
@@ -374,7 +377,9 @@ esp_err_t handler_ota_update(httpd_req_t *req)
 
     const char* resp_str;    
 
-    if (ota_example_task(fn))
+    KillTFliteTasks();
+    gpio_handler_deinit();
+    if (ota_update_task(fn))
     {
         resp_str = "Firmware Update Successfull!<br><br>You can restart now.";
     }
@@ -400,8 +405,6 @@ void hard_restart() {
 
 void task_reboot(void *pvParameter)
 {
-
-
     while(1)
     {
         vTaskDelay(5000 / portTICK_PERIOD_MS);
@@ -413,12 +416,14 @@ void task_reboot(void *pvParameter)
 }
 
 void doReboot(){
-    LogFile.WriteToFile("Reboot - now");
-    KillTFliteTasks();
+    ESP_LOGI(TAGPARTOTA, "Reboot in 5sec");
+    LogFile.WriteToFile("Reboot in 5sec");
     xTaskCreate(&task_reboot, "reboot", configMINIMAL_STACK_SIZE * 64, NULL, 10, NULL);
+    // KillTFliteTasks(); // kills itself 
+    gpio_handler_destroy();
     vTaskDelay(5000 / portTICK_PERIOD_MS);
     esp_restart();
-    hard_restart();    
+    hard_restart();
 }
 
 

code/components/jomjol_fileserver_ota/server_ota.h

@@ -4,8 +4,6 @@
 
 //#include "ClassControllCamera.h"
 
-static const char *TAGPARTOTA = "server_ota";
-
 void register_server_ota_sdcard_uri(httpd_handle_t server);
 void CheckOTAUpdate();
 void doReboot();

code/components/jomjol_flowcontroll/ClassFlow.cpp

@@ -94,6 +94,23 @@ string ClassFlow::getReadout()
 	return string();
 }
 
+std::string ClassFlow::GetParameterName(std::string _input)
+{
+    string _param;
+    int _pospunkt = _input.find_first_of(".");
+    if (_pospunkt > -1)
+    {
+        _param = _input.substr(_pospunkt+1, _input.length() - _pospunkt - 1);
+    }
+    else
+    {
+        _param = _input;
+    }
+//    printf("Parameter: %s, Pospunkt: %d\n", _param.c_str(), _pospunkt);
+	return _param;
+}
+
+
 bool ClassFlow::getNextLine(FILE* pfile, string *rt)
 {
 	char zw[1024];
@@ -102,24 +119,21 @@ bool ClassFlow::getNextLine(FILE* pfile, string *rt)
 		*rt = "";
 		return false;
 	}
-	fgets(zw, 1024, pfile);
-	printf("%s", zw);
-	if ((strlen(zw) == 0) && feof(pfile))
+	if (!fgets(zw, 1024, pfile))
 	{
 		*rt = "";
+		printf("END OF FILE\n");
 		return false;
 	}
+	printf("%s", zw);
 	*rt = zw;
 	*rt = trim(*rt);
 	while ((zw[0] == ';' || zw[0] == '#' || (rt->size() == 0)) && !(zw[1] == '['))			// Kommentarzeilen (; oder #) und Leerzeilen überspringen, es sei denn es ist ein neuer auskommentierter Paragraph
 	{
-		fgets(zw, 1024, pfile);
-		printf("%s", zw);		
-		if (feof(pfile))
-		{
-			*rt = "";
+		*rt = "";
+		if (!fgets(zw, 1024, pfile))
 			return false;
-		}
+		printf("%s", zw);		
 		*rt = zw;
 		*rt = trim(*rt);
 	}

code/components/jomjol_flowcontroll/ClassFlow.h

@@ -37,6 +37,8 @@ protected:
 
 	virtual void SetInitialParameter(void);
 
+	std::string GetParameterName(std::string _input);
+
 	bool disabled;
 
 public:

code/components/jomjol_flowcontroll/ClassFlowAnalog.cpp

@@ -1,350 +0,0 @@
-#include "ClassFlowAnalog.h"
-
-#include <math.h>
-#include <iomanip>
-#include <sys/types.h>
-#include <sstream>      // std::stringstream
-
-  
-// #define OHNETFLITE
-
-#ifndef OHNETFLITE
-#include "CTfLiteClass.h"
-#endif
-
-#include "ClassLogFile.h"
-
-static const char* TAG = "flow_analog";
-
-bool debugdetailanalog = false;
-
-void ClassFlowAnalog::SetInitialParameter(void)
-{
-    string cnnmodelfile = "";
-    modelxsize = 1;
-    modelysize = 1;
-    ListFlowControll = NULL;
-    previousElement = NULL;   
-    SaveAllFiles = false; 
-    disabled = false;
-    extendedResolution = false;
-}   
-
-ClassFlowAnalog::ClassFlowAnalog(std::vector<ClassFlow*>* lfc) : ClassFlowImage(lfc, TAG)
-{
-    SetInitialParameter();
-    ListFlowControll = lfc;
-
-    for (int i = 0; i < ListFlowControll->size(); ++i)
-    {
-        if (((*ListFlowControll)[i])->name().compare("ClassFlowAlignment") == 0)
-        {
-            flowpostalignment = (ClassFlowAlignment*) (*ListFlowControll)[i];
-        }
-    }
-
-}
-
-
-int ClassFlowAnalog::AnzahlROIs()
-{
-    int zw = ROI.size();
-    if (extendedResolution)
-        zw++;
-    
-    return zw;
-} 
-
-
-string ClassFlowAnalog::getReadout()
-{
-    string result = "";    
-    if (ROI.size() == 0)
-        return result;
-
-
-    float zahl = ROI[ROI.size() - 1]->result;
-    int ergebnis_nachkomma = ((int) floor(zahl * 10)) % 10;
-
-    int prev = -1;
-
-    prev = ZeigerEval(ROI[ROI.size() - 1]->result, prev);
-    result = std::to_string(prev);
-
-    if (extendedResolution)
-        result = result + std::to_string(ergebnis_nachkomma);
-
-    for (int i = ROI.size() - 2; i >= 0; --i)
-    {
-        prev = ZeigerEval(ROI[i]->result, prev);
-        result = std::to_string(prev) + result;
-    }
-
-    return result;
-}
-
-int ClassFlowAnalog::ZeigerEval(float zahl, int ziffer_vorgaenger)
-{
-    int ergebnis_nachkomma = ((int) floor(zahl * 10)) % 10;
-    int ergebnis_vorkomma = ((int) floor(zahl)) % 10;
-    int ergebnis, ergebnis_rating;
-
-    if (ziffer_vorgaenger == -1)
-        return ergebnis_vorkomma % 10;
-
-    ergebnis_rating = ergebnis_nachkomma - ziffer_vorgaenger;
-    if (ergebnis_nachkomma >= 5)
-        ergebnis_rating-=5;
-    else
-        ergebnis_rating+=5;
-    ergebnis = (int) round(zahl);
-    if (ergebnis_rating < 0)
-        ergebnis-=1;
-    if (ergebnis == -1)
-        ergebnis+=10;
-
-    ergebnis = ergebnis % 10;
-    return ergebnis;
-}
-
-bool ClassFlowAnalog::ReadParameter(FILE* pfile, string& aktparamgraph)
-{
-    std::vector<string> zerlegt;
-
-    aktparamgraph = trim(aktparamgraph);
-
-    if (aktparamgraph.size() == 0)
-        if (!this->GetNextParagraph(pfile, aktparamgraph))
-            return false;
-
-
-    if ((aktparamgraph.compare("[Analog]") != 0) && (aktparamgraph.compare(";[Analog]") != 0))       // Paragraph passt nich zu MakeImage
-        return false;
-
-    if (aktparamgraph[0] == ';')
-    {
-        disabled = true;
-        while (getNextLine(pfile, &aktparamgraph) && !isNewParagraph(aktparamgraph));
-        printf("[Analog] is disabled !!!\n");
-        return true;
-    }
-
-
-    while (this->getNextLine(pfile, &aktparamgraph) && !this->isNewParagraph(aktparamgraph))
-    {
-        zerlegt = this->ZerlegeZeile(aktparamgraph);
-        if ((zerlegt[0] == "LogImageLocation") && (zerlegt.size() > 1))
-        {
-            this->LogImageLocation = "/sdcard" + zerlegt[1];
-            this->isLogImage = true;
-        }
-        if ((toUpper(zerlegt[0]) == "LOGFILERETENTIONINDAYS") && (zerlegt.size() > 1))
-        {
-            this->logfileRetentionInDays = std::stoi(zerlegt[1]);
-        }
-        if ((zerlegt[0] == "Model") && (zerlegt.size() > 1))
-        {
-            this->cnnmodelfile = zerlegt[1];
-        }
-        if ((zerlegt[0] == "ModelInputSize") && (zerlegt.size() > 2))
-        {
-            this->modelxsize = std::stoi(zerlegt[1]);
-            this->modelysize = std::stoi(zerlegt[2]);
-        }
-        if (zerlegt.size() >= 5)
-        {
-            roianalog* neuroi = new roianalog;
-            neuroi->name = zerlegt[0];
-            neuroi->posx = std::stoi(zerlegt[1]);
-            neuroi->posy = std::stoi(zerlegt[2]);
-            neuroi->deltax = std::stoi(zerlegt[3]);
-            neuroi->deltay = std::stoi(zerlegt[4]);
-            neuroi->result = -1;
-            neuroi->image = NULL;
-            neuroi->image_org = NULL;
-            ROI.push_back(neuroi);
-        }
-
-        if ((toUpper(zerlegt[0]) == "SAVEALLFILES") && (zerlegt.size() > 1))
-        {
-            if (toUpper(zerlegt[1]) == "TRUE")
-                SaveAllFiles = true;
-        }
-
-        if ((toUpper(zerlegt[0]) == "EXTENDEDRESOLUTION") && (zerlegt.size() > 1))
-        {
-            if (toUpper(zerlegt[1]) == "TRUE")
-                extendedResolution = true;
-        }
-    }
-
-    for (int i = 0; i < ROI.size(); ++i)
-    {
-        ROI[i]->image = new CImageBasis(modelxsize, modelysize, 3);
-        ROI[i]->image_org = new CImageBasis(ROI[i]->deltax, ROI[i]->deltay, 3);
-    }
-
-    return true;
-}
-
-
-string ClassFlowAnalog::getHTMLSingleStep(string host)
-{
-    string result, zw;
-    std::vector<HTMLInfo*> htmlinfo;
-
-    result = "<p>Found ROIs: </p> <p><img src=\"" + host + "/img_tmp/alg_roi.jpg\"></p>\n";
-    result = result + "Analog Pointers: <p> ";
-
-    htmlinfo = GetHTMLInfo();
-    for (int i = 0; i < htmlinfo.size(); ++i)
-    {
-        std::stringstream stream;
-        stream << std::fixed << std::setprecision(1) << htmlinfo[i]->val;
-        zw = stream.str();
-
-        result = result + "<img src=\"" + host + "/img_tmp/" +  htmlinfo[i]->filename + "\"> " + zw;
-        delete htmlinfo[i];
-    }
-    htmlinfo.clear();         
-
-    return result;
-}
-
-
-
-bool ClassFlowAnalog::doFlow(string time)
-{
-    if (disabled)
-      return true;
-
-    if (!doAlignAndCut(time)){
-        return false;
-    };
-
-    if (debugdetailanalog) LogFile.WriteToFile("ClassFlowAnalog::doFlow nach Alignment");
-
-    doNeuralNetwork(time);
-
-    RemoveOldLogs();
-
-    return true;
-}
-
-bool ClassFlowAnalog::doAlignAndCut(string time)
-{
-    if (disabled)
-        return true;
-
-    CAlignAndCutImage *caic = flowpostalignment->GetAlignAndCutImage();    
-
-    for (int i = 0; i < ROI.size(); ++i)
-    {
-        printf("Analog %d - Align&Cut\n", i);
-        
-        caic->CutAndSave(ROI[i]->posx, ROI[i]->posy, ROI[i]->deltax, ROI[i]->deltay, ROI[i]->image_org);
-        if (SaveAllFiles) ROI[i]->image_org->SaveToFile(FormatFileName("/sdcard/img_tmp/" + ROI[i]->name + ".jpg"));
-
-        ROI[i]->image_org->Resize(modelxsize, modelysize, ROI[i]->image);
-        if (SaveAllFiles) ROI[i]->image->SaveToFile(FormatFileName("/sdcard/img_tmp/" + ROI[i]->name + ".bmp"));
-    }
-
-    return true;
-} 
-
-void ClassFlowAnalog::DrawROI(CImageBasis *_zw)
-{
-    int r = 0;
-    int g = 255;
-    int b = 0;
-
-    for (int i = 0; i < ROI.size(); ++i)
-    {
-        _zw->drawRect(ROI[i]->posx, ROI[i]->posy, ROI[i]->deltax, ROI[i]->deltay, r, g, b, 1);
-        _zw->drawCircle((int) (ROI[i]->posx + ROI[i]->deltax/2), (int)  (ROI[i]->posy + ROI[i]->deltay/2), (int) (ROI[i]->deltax/2), r, g, b, 2);
-        _zw->drawLine((int) (ROI[i]->posx + ROI[i]->deltax/2), (int) ROI[i]->posy, (int) (ROI[i]->posx + ROI[i]->deltax/2), (int) (ROI[i]->posy + ROI[i]->deltay), r, g, b, 2);
-        _zw->drawLine((int) ROI[i]->posx, (int) (ROI[i]->posy + ROI[i]->deltay/2), (int) ROI[i]->posx + ROI[i]->deltax, (int) (ROI[i]->posy + ROI[i]->deltay/2), r, g, b, 2);
-    }
-} 
-
-bool ClassFlowAnalog::doNeuralNetwork(string time)
-{
-    if (disabled)
-        return true;
-
-    string logPath = CreateLogFolder(time);
-    
-    string input = "/sdcard/img_tmp/alg.jpg";
-    string ioresize = "/sdcard/img_tmp/resize.bmp";
-    string output;
-    input = FormatFileName(input);
-
-#ifndef OHNETFLITE
-    CTfLiteClass *tflite = new CTfLiteClass;  
-    string zwcnn = "/sdcard" + cnnmodelfile;
-    zwcnn = FormatFileName(zwcnn);
-    printf(zwcnn.c_str());printf("\n");
-    tflite->LoadModel(zwcnn); 
-    tflite->MakeAllocate();
-#endif
-
-    for (int i = 0; i < ROI.size(); ++i)
-    {
-        printf("Analog %d - TfLite\n", i);
-        ioresize = "/sdcard/img_tmp/ra" + std::to_string(i) + ".bmp";
-        ioresize = FormatFileName(ioresize);
-
-
-        float f1, f2;
-        f1 = 0; f2 = 0;
-
-#ifndef OHNETFLITE
-//        LogFile.WriteToFile("ClassFlowAnalog::doNeuralNetwork vor CNN tflite->LoadInputImage(ioresize)");
-//        tflite->LoadInputImage(ioresize);
-        tflite->LoadInputImageBasis(ROI[i]->image);        
-        tflite->Invoke();
-        if (debugdetailanalog) LogFile.WriteToFile("Nach Invoke");
-
-
-        f1 = tflite->GetOutputValue(0);
-        f2 = tflite->GetOutputValue(1);
-#endif
-
-        float result = fmod(atan2(f1, f2) / (M_PI * 2) + 2, 1);
-//        printf("Result sin, cos, ziffer: %f, %f, %f\n", f1, f2, result);  
-        ROI[i]->result = result * 10;
-
-        printf("Result Analog%i: %f\n", i, ROI[i]->result); 
-
-        if (isLogImage)
-        {
-            LogImage(logPath, ROI[i]->name, &ROI[i]->result, NULL, time, ROI[i]->image_org);
-        }
-    }
-#ifndef OHNETFLITE
-        delete tflite;
-#endif    
-
-    return true;
-}
-
-
-std::vector<HTMLInfo*> ClassFlowAnalog::GetHTMLInfo()
-{
-    std::vector<HTMLInfo*> result;
-
-    for (int i = 0; i < ROI.size(); ++i)
-    {
-        HTMLInfo *zw = new HTMLInfo;
-        zw->filename = ROI[i]->name + ".bmp";
-        zw->filename_org = ROI[i]->name + ".jpg";
-        zw->val = ROI[i]->result;
-        zw->image = ROI[i]->image;
-        zw->image_org = ROI[i]->image_org;
-        result.push_back(zw);
-    }
-
-    return result;
-}
-
-

code/components/jomjol_flowcontroll/ClassFlowAnalog.h

@@ -1,48 +0,0 @@
-#pragma once
-#include "ClassFlowImage.h"
-#include "ClassFlowAlignment.h"
-// #include "CTfLiteClass.h"
-
-struct roianalog {
-    int posx, posy, deltax, deltay;
-    float result;
-    CImageBasis *image, *image_org;
-    string name;
-};
-
-
-class ClassFlowAnalog :
-    public ClassFlowImage
-{
-protected:
-    std::vector<roianalog*> ROI;
-    string cnnmodelfile;
-    int modelxsize, modelysize;
-    int ZeigerEval(float zahl, int ziffer_vorgaenger);
-    bool SaveAllFiles;    
-
-
-    ClassFlowAlignment* flowpostalignment;
-
-	void SetInitialParameter(void);        
-
-public:
-    bool extendedResolution;
-
-    ClassFlowAnalog(std::vector<ClassFlow*>* lfc);
-
-    bool ReadParameter(FILE* pfile, string& aktparamgraph);
-    bool doFlow(string time);
-    string getHTMLSingleStep(string host);
-    string getReadout();   
-
-    void DrawROI(CImageBasis *_zw); 
-
-    bool doNeuralNetwork(string time); 
-    bool doAlignAndCut(string time);
-   	std::vector<HTMLInfo*> GetHTMLInfo();   
-    int AnzahlROIs(); 
-
-    string name(){return "ClassFlowAnalog";}; 
-};
-

code/components/jomjol_flowcontroll/ClassFlowCNNGeneral.cpp

@@ -0,0 +1,665 @@
+#include "ClassFlowCNNGeneral.h"
+
+#include <math.h>
+#include <iomanip> 
+#include <sys/types.h>
+#include <sstream>      // std::stringstream
+
+#include "CTfLiteClass.h"
+#include "ClassLogFile.h"
+
+static const char* TAG = "flow_analog";
+
+bool debugdetailgeneral = false;
+
+ClassFlowCNNGeneral::ClassFlowCNNGeneral(ClassFlowAlignment *_flowalign, t_CNNType _cnntype) : ClassFlowImage(NULL, TAG)
+{
+    string cnnmodelfile = "";
+    modelxsize = 1;
+    modelysize = 1;
+    ListFlowControll = NULL;
+    previousElement = NULL;   
+    SaveAllFiles = false; 
+    disabled = false;
+//    extendedResolution = false;
+    isLogImageSelect = false;
+    CNNType = AutoDetect;
+    CNNType = _cnntype;
+    flowpostalignment = _flowalign;
+}
+
+/*
+int ClassFlowCNNGeneral::AnzahlROIs(int _analog = 0)
+{
+    int zw = GENERAL[_analog]->ROI.size();
+    if (extendedResolution && (CNNType != Digital)) zw++;   // da letzte Ziffer inkl Nachhkomma, es sei denn, das Nachkomma gibt es nicht (Digital)
+    return zw;
+} 
+*/
+
+string ClassFlowCNNGeneral::getReadout(int _analog = 0, bool _extendedResolution = false)
+{
+    string result = "";    
+    if (GENERAL[_analog]->ROI.size() == 0)
+        return result;
+
+    if (CNNType == Analogue)
+    {
+        float zahl = GENERAL[_analog]->ROI[GENERAL[_analog]->ROI.size() - 1]->result_float;
+        int ergebnis_nachkomma = ((int) floor(zahl * 10) + 10) % 10;
+
+        int prev = -1;
+
+        prev = ZeigerEval(GENERAL[_analog]->ROI[GENERAL[_analog]->ROI.size() - 1]->result_float, prev);
+        result = std::to_string(prev);
+
+        if (_extendedResolution && (CNNType != Digital))
+            result = result + std::to_string(ergebnis_nachkomma);
+
+        for (int i = GENERAL[_analog]->ROI.size() - 2; i >= 0; --i)
+        {
+            prev = ZeigerEval(GENERAL[_analog]->ROI[i]->result_float, prev);
+            result = std::to_string(prev) + result;
+        }
+        return result;
+    }
+
+    if (CNNType == Digital)
+    {
+        for (int i = 0; i < GENERAL[_analog]->ROI.size(); ++i)
+        {
+            if (GENERAL[_analog]->ROI[i]->result_klasse >= 10)
+                result = result + "N";
+            else
+                result = result + std::to_string(GENERAL[_analog]->ROI[i]->result_klasse);
+        }
+        return result;
+    }
+
+    if (CNNType == DigitalHyprid)
+    {
+//        int ergebnis_nachkomma = -1;
+        int zif_akt = -1;
+
+        float zahl = GENERAL[_analog]->ROI[GENERAL[_analog]->ROI.size() - 1]->result_float;
+        if (zahl >= 0)       // NaN?
+        {
+            if (_extendedResolution)
+            {
+                int ergebnis_nachkomma = ((int) floor(zahl * 10)) % 10;
+                int ergebnis_vorkomma = ((int) floor(zahl)) % 10;
+
+                result = std::to_string(ergebnis_vorkomma) + std::to_string(ergebnis_nachkomma);
+                zif_akt = ergebnis_vorkomma;
+            }
+            else
+            {
+                zif_akt = ZeigerEvalHybrid(GENERAL[_analog]->ROI[GENERAL[_analog]->ROI.size() - 1]->result_float, -1, -1);
+                result = std::to_string(zif_akt);
+            }
+        }
+        else
+        {
+            result = "N";
+            if (_extendedResolution && (CNNType != Digital))
+                result = "NN";
+        }
+
+        for (int i = GENERAL[_analog]->ROI.size() - 2; i >= 0; --i)
+        {
+            if (GENERAL[_analog]->ROI[i]->result_float >= 0)
+            {
+                zif_akt = ZeigerEvalHybrid(GENERAL[_analog]->ROI[i]->result_float, GENERAL[_analog]->ROI[i+1]->result_float, zif_akt);
+                result = std::to_string(zif_akt) + result;
+            }
+            else
+            {
+                zif_akt = -1;
+                result = "N" + result;
+            }
+        }
+        return result;
+    }
+
+    return result;
+}
+
+int ClassFlowCNNGeneral::ZeigerEvalHybrid(float zahl, float zahl_vorgaenger, int eval_vorgaenger)
+{
+    int ergebnis_nachkomma = ((int) floor(zahl * 10)) % 10;
+//    int ergebnis_vorkomma = ((int) floor(zahl)) % 10;
+
+    if (zahl_vorgaenger < 0)                // keine Vorzahl vorhanden !!! --> Runde die Zahl
+    {
+        if ((ergebnis_nachkomma <= 2) || (ergebnis_nachkomma >= 8))     // Band um die Ziffer --> Runden, da Ziffer im Rahmen Ungenauigkeit erreicht
+            return ((int) round(zahl) + 10) % 10;
+        else
+            return ((int) trunc(zahl) + 10) % 10;
+    }
+
+    if (zahl_vorgaenger > 9.2)              // Ziffernwechsel beginnt
+    {
+        if (eval_vorgaenger == 0)           // Wechsel hat schon stattgefunden
+        {
+            return ((int) round(zahl) + 10) % 10;      // Annahme, dass die neue Zahl schon in der Nähe des Ziels ist
+        }
+        else
+        {
+            if (zahl_vorgaenger <= 9.5)     // Wechsel startet gerade, aber beginnt erst
+            {
+                if ((ergebnis_nachkomma <= 2) || (ergebnis_nachkomma >= 8))     // Band um die Ziffer --> Runden, da Ziffer im Rahmen Ungenauigkeit erreicht
+                    return ((int) round(zahl) + 10) % 10;
+                else
+                    return ((int) trunc(zahl) + 10) % 10;
+            }
+            else
+            {
+                return ((int) trunc(zahl) + 10) % 10;   // Wechsel schon weiter fortgeschritten, d.h. über 2 als Nachkomma
+            }
+        }
+    }
+
+    if ((ergebnis_nachkomma <= 2) || (ergebnis_nachkomma >= 8))     // Band um die Ziffer --> Runden, da Ziffer im Rahmen Ungenauigkeit erreicht
+        return ((int) round(zahl) + 10) % 10;
+
+    return ((int) trunc(zahl) + 10) % 10;
+}
+
+int ClassFlowCNNGeneral::ZeigerEval(float zahl, int ziffer_vorgaenger)
+{
+    int ergebnis_nachkomma = ((int) floor(zahl * 10) + 10) % 10;
+    int ergebnis_vorkomma = ((int) floor(zahl) + 10) % 10;
+    int ergebnis, ergebnis_rating;
+
+    if (ziffer_vorgaenger == -1)
+        return ergebnis_vorkomma % 10;
+
+    ergebnis_rating = ergebnis_nachkomma - ziffer_vorgaenger;
+    if (ergebnis_nachkomma >= 5)
+        ergebnis_rating-=5;
+    else
+        ergebnis_rating+=5;
+    ergebnis = (int) round(zahl);
+    if (ergebnis_rating < 0)
+        ergebnis-=1;
+    if (ergebnis == -1)
+        ergebnis+=10;
+
+    ergebnis = (ergebnis + 10) % 10;
+    return ergebnis;
+}
+
+bool ClassFlowCNNGeneral::ReadParameter(FILE* pfile, string& aktparamgraph)
+{
+    std::vector<string> zerlegt;
+
+    aktparamgraph = trim(aktparamgraph);
+
+    if (aktparamgraph.size() == 0)
+        if (!this->GetNextParagraph(pfile, aktparamgraph))
+            return false;
+
+
+    if ((toUpper(aktparamgraph) != "[ANALOG]") && (toUpper(aktparamgraph) != ";[ANALOG]") 
+        && (toUpper(aktparamgraph) != "[DIGIT]") && (toUpper(aktparamgraph) != ";[DIGIT]")
+        && (toUpper(aktparamgraph) != "[DIGITS]") && (toUpper(aktparamgraph) != ";[DIGITS]")
+        )       // Paragraph passt nicht
+        return false;
+
+
+/*
+    if ((aktparamgraph.compare("[Analog]") != 0) && (aktparamgraph.compare(";[Analog]") != 0) 
+        && (aktparamgraph.compare("[Digit]") != 0) && (aktparamgraph.compare(";[Digit]")))       // Paragraph passt nicht
+        return false;
+*/
+
+    if (aktparamgraph[0] == ';')
+    {
+        disabled = true;
+        while (getNextLine(pfile, &aktparamgraph) && !isNewParagraph(aktparamgraph));
+        printf("[Analog/Digit] is disabled !!!\n");
+        return true;
+    }
+
+
+    while (this->getNextLine(pfile, &aktparamgraph) && !this->isNewParagraph(aktparamgraph))
+    {
+        zerlegt = this->ZerlegeZeile(aktparamgraph);
+        if ((zerlegt[0] == "LogImageLocation") && (zerlegt.size() > 1))
+        {
+            this->LogImageLocation = "/sdcard" + zerlegt[1];
+            this->isLogImage = true;
+        }
+        if ((zerlegt[0] == "LogImageSelect") && (zerlegt.size() > 1))
+        {
+            LogImageSelect = zerlegt[1];
+            isLogImageSelect = true;            
+        }
+
+        if ((toUpper(zerlegt[0]) == "LOGFILERETENTIONINDAYS") && (zerlegt.size() > 1))
+        {
+            this->logfileRetentionInDays = std::stoi(zerlegt[1]);
+        }
+        if ((toUpper(zerlegt[0]) == "MODELTYPE") && (zerlegt.size() > 1))
+        {
+            if (toUpper(zerlegt[1]) == "DIGITHYPRID")
+                CNNType = DigitalHyprid;
+        }
+
+        if ((zerlegt[0] == "Model") && (zerlegt.size() > 1))
+        {
+            this->cnnmodelfile = zerlegt[1];
+        }
+        if ((zerlegt[0] == "ModelInputSize") && (zerlegt.size() > 2))
+        {
+            this->modelxsize = std::stoi(zerlegt[1]);
+            this->modelysize = std::stoi(zerlegt[2]);
+        }
+        if (zerlegt.size() >= 5)
+        {
+            general* _analog = GetGENERAL(zerlegt[0], true);
+            roi* neuroi = _analog->ROI[_analog->ROI.size()-1];
+            neuroi->posx = std::stoi(zerlegt[1]);
+            neuroi->posy = std::stoi(zerlegt[2]);
+            neuroi->deltax = std::stoi(zerlegt[3]);
+            neuroi->deltay = std::stoi(zerlegt[4]);
+            neuroi->result_float = -1;
+            neuroi->image = NULL;
+            neuroi->image_org = NULL;
+        }
+
+        if ((toUpper(zerlegt[0]) == "SAVEALLFILES") && (zerlegt.size() > 1))
+        {
+            if (toUpper(zerlegt[1]) == "TRUE")
+                SaveAllFiles = true;
+        }
+
+/*
+        if ((toUpper(zerlegt[0]) == "EXTENDEDRESOLUTION") && (zerlegt.size() > 1))
+        {
+            if (toUpper(zerlegt[1]) == "TRUE")
+                extendedResolution = true;
+        }
+*/
+    }
+
+
+   for (int _ana = 0; _ana < GENERAL.size(); ++_ana)
+        for (int i = 0; i < GENERAL[_ana]->ROI.size(); ++i)
+        {
+            GENERAL[_ana]->ROI[i]->image = new CImageBasis(modelxsize, modelysize, 3);
+            GENERAL[_ana]->ROI[i]->image_org = new CImageBasis(GENERAL[_ana]->ROI[i]->deltax, GENERAL[_ana]->ROI[i]->deltay, 3);
+        }
+
+    return true;
+}
+
+general* ClassFlowCNNGeneral::FindGENERAL(string _name_number)
+{
+    for (int i = 0; i < GENERAL.size(); ++i)
+        if (GENERAL[i]->name == _name_number)
+            return GENERAL[i];
+    return NULL;
+}
+
+
+
+general* ClassFlowCNNGeneral::GetGENERAL(string _name, bool _create = true)
+{
+    string _analog, _roi;
+    int _pospunkt = _name.find_first_of(".");
+
+    if (_pospunkt > -1)
+    {
+        _analog = _name.substr(0, _pospunkt);
+        _roi = _name.substr(_pospunkt+1, _name.length() - _pospunkt - 1);
+    }
+    else
+    {
+        _analog = "default";
+        _roi = _name;
+    }
+
+    general *_ret = NULL;
+
+    for (int i = 0; i < GENERAL.size(); ++i)
+        if (GENERAL[i]->name == _analog)
+            _ret = GENERAL[i];
+
+    if (!_create)         // nicht gefunden und soll auch nicht erzeugt werden
+        return _ret;
+
+    if (_ret == NULL)
+    {
+        _ret = new general;
+        _ret->name = _analog;
+        GENERAL.push_back(_ret);
+    }
+
+    roi* neuroi = new roi;
+    neuroi->name = _roi;
+    _ret->ROI.push_back(neuroi);
+
+    printf("GetGENERAL - GENERAL %s - roi %s\n", _analog.c_str(), _roi.c_str());
+
+    return _ret;
+}
+
+
+
+string ClassFlowCNNGeneral::getHTMLSingleStep(string host)
+{
+    string result, zw;
+    std::vector<HTMLInfo*> htmlinfo;
+
+    result = "<p>Found ROIs: </p> <p><img src=\"" + host + "/img_tmp/alg_roi.jpg\"></p>\n";
+    result = result + "Analog Pointers: <p> ";
+
+    htmlinfo = GetHTMLInfo();
+    for (int i = 0; i < htmlinfo.size(); ++i)
+    {
+        std::stringstream stream;
+        stream << std::fixed << std::setprecision(1) << htmlinfo[i]->val;
+        zw = stream.str();
+
+        result = result + "<img src=\"" + host + "/img_tmp/" +  htmlinfo[i]->filename + "\"> " + zw;
+        delete htmlinfo[i];
+    }
+    htmlinfo.clear();         
+
+    return result;
+}
+
+
+
+bool ClassFlowCNNGeneral::doFlow(string time)
+{
+    if (disabled)
+      return true;
+
+    if (!doAlignAndCut(time)){
+        return false;
+    };
+
+    if (debugdetailgeneral) LogFile.WriteToFile("ClassFlowCNNGeneral::doFlow nach Alignment");
+
+    doNeuralNetwork(time);
+
+    RemoveOldLogs();
+    return true;
+}
+
+bool ClassFlowCNNGeneral::doAlignAndCut(string time)
+{
+    if (disabled)
+        return true;
+
+    CAlignAndCutImage *caic = flowpostalignment->GetAlignAndCutImage();    
+
+    for (int _ana = 0; _ana < GENERAL.size(); ++_ana)
+        for (int i = 0; i < GENERAL[_ana]->ROI.size(); ++i)
+        {
+            printf("General %d - Align&Cut\n", i);
+            
+            caic->CutAndSave(GENERAL[_ana]->ROI[i]->posx, GENERAL[_ana]->ROI[i]->posy, GENERAL[_ana]->ROI[i]->deltax, GENERAL[_ana]->ROI[i]->deltay, GENERAL[_ana]->ROI[i]->image_org);
+            if (SaveAllFiles)
+            {
+                if (GENERAL[_ana]->name == "default")
+                    GENERAL[_ana]->ROI[i]->image_org->SaveToFile(FormatFileName("/sdcard/img_tmp/" + GENERAL[_ana]->ROI[i]->name + ".jpg"));
+                else
+                    GENERAL[_ana]->ROI[i]->image_org->SaveToFile(FormatFileName("/sdcard/img_tmp/" + GENERAL[_ana]->name + "_" + GENERAL[_ana]->ROI[i]->name + ".jpg"));
+            } 
+
+            GENERAL[_ana]->ROI[i]->image_org->Resize(modelxsize, modelysize, GENERAL[_ana]->ROI[i]->image);
+            if (SaveAllFiles)
+            {
+                if (GENERAL[_ana]->name == "default")
+                    GENERAL[_ana]->ROI[i]->image->SaveToFile(FormatFileName("/sdcard/img_tmp/" + GENERAL[_ana]->ROI[i]->name + ".bmp"));
+                else
+                    GENERAL[_ana]->ROI[i]->image->SaveToFile(FormatFileName("/sdcard/img_tmp/" + GENERAL[_ana]->name + "_" + GENERAL[_ana]->ROI[i]->name + ".bmp"));
+            } 
+        }
+
+    return true;
+} 
+
+void ClassFlowCNNGeneral::DrawROI(CImageBasis *_zw)
+{
+    if (CNNType == Analogue)
+    {
+        int r = 0;
+        int g = 255;
+        int b = 0;
+
+        for (int _ana = 0; _ana < GENERAL.size(); ++_ana)
+            for (int i = 0; i < GENERAL[_ana]->ROI.size(); ++i)
+            {
+                _zw->drawRect(GENERAL[_ana]->ROI[i]->posx, GENERAL[_ana]->ROI[i]->posy, GENERAL[_ana]->ROI[i]->deltax, GENERAL[_ana]->ROI[i]->deltay, r, g, b, 1);
+                _zw->drawCircle((int) (GENERAL[_ana]->ROI[i]->posx + GENERAL[_ana]->ROI[i]->deltax/2), (int)  (GENERAL[_ana]->ROI[i]->posy + GENERAL[_ana]->ROI[i]->deltay/2), (int) (GENERAL[_ana]->ROI[i]->deltax/2), r, g, b, 2);
+                _zw->drawLine((int) (GENERAL[_ana]->ROI[i]->posx + GENERAL[_ana]->ROI[i]->deltax/2), (int) GENERAL[_ana]->ROI[i]->posy, (int) (GENERAL[_ana]->ROI[i]->posx + GENERAL[_ana]->ROI[i]->deltax/2), (int) (GENERAL[_ana]->ROI[i]->posy + GENERAL[_ana]->ROI[i]->deltay), r, g, b, 2);
+                _zw->drawLine((int) GENERAL[_ana]->ROI[i]->posx, (int) (GENERAL[_ana]->ROI[i]->posy + GENERAL[_ana]->ROI[i]->deltay/2), (int) GENERAL[_ana]->ROI[i]->posx + GENERAL[_ana]->ROI[i]->deltax, (int) (GENERAL[_ana]->ROI[i]->posy + GENERAL[_ana]->ROI[i]->deltay/2), r, g, b, 2);
+            }
+    }
+    else
+    {
+        for (int _dig = 0; _dig < GENERAL.size(); ++_dig)
+            for (int i = 0; i < GENERAL[_dig]->ROI.size(); ++i)
+                _zw->drawRect(GENERAL[_dig]->ROI[i]->posx, GENERAL[_dig]->ROI[i]->posy, GENERAL[_dig]->ROI[i]->deltax, GENERAL[_dig]->ROI[i]->deltay, 0, 0, (255 - _dig*100), 2);
+    }
+} 
+
+bool ClassFlowCNNGeneral::doNeuralNetwork(string time)
+{
+    if (disabled)
+        return true;
+
+    string logPath = CreateLogFolder(time);
+
+    CTfLiteClass *tflite = new CTfLiteClass;  
+    string zwcnn = "/sdcard" + cnnmodelfile;
+    zwcnn = FormatFileName(zwcnn);
+    printf(zwcnn.c_str());printf("\n");
+    if (!tflite->LoadModel(zwcnn)) {
+        printf("Can't read model file /sdcard%s\n", cnnmodelfile.c_str());
+        LogFile.WriteToFile("Cannot load model");
+
+        delete tflite;
+        return false;
+    } 
+    tflite->MakeAllocate();
+
+    if (CNNType == AutoDetect)
+    {
+        int _anzoutputdimensions = tflite->GetAnzOutPut();
+        switch (_anzoutputdimensions) 
+        {
+            case 2:
+                CNNType = Analogue;
+                printf("TFlite-Type set to Analogue\n");
+                break;
+            case 11:
+                CNNType = Digital;
+                printf("TFlite-Type set to Digital\n");
+                break;
+            case 22:
+                CNNType = DigitalHyprid;
+                printf("TFlite-Type set to DigitalHyprid\n");
+                break;
+            default:
+                printf("ERROR ERROR ERROR - tflite passt nicht zur Firmware - ERROR ERROR ERROR\n");
+        }
+//        flowpostprocessing->UpdateNachkommaDecimalShift();
+    }
+
+    for (int _ana = 0; _ana < GENERAL.size(); ++_ana)
+    {
+        for (int i = 0; i < GENERAL[_ana]->ROI.size(); ++i)
+        {
+            printf("General %d - TfLite\n", i);
+
+            switch (CNNType) {
+                case Analogue:
+                    {
+                        float f1, f2;
+                        f1 = 0; f2 = 0;
+
+                        tflite->LoadInputImageBasis(GENERAL[_ana]->ROI[i]->image);        
+                        tflite->Invoke();
+                        if (debugdetailgeneral) LogFile.WriteToFile("Nach Invoke");
+
+                        f1 = tflite->GetOutputValue(0);
+                        f2 = tflite->GetOutputValue(1);
+                        float result = fmod(atan2(f1, f2) / (M_PI * 2) + 2, 1);
+                        GENERAL[_ana]->ROI[i]->result_float = result * 10;
+                        printf("Result General(Analog)%i: %f\n", i, GENERAL[_ana]->ROI[i]->result_float); 
+                        if (isLogImage)
+                            LogImage(logPath, GENERAL[_ana]->ROI[i]->name, &GENERAL[_ana]->ROI[i]->result_float, NULL, time, GENERAL[_ana]->ROI[i]->image_org);
+                    } break;
+                case Digital:
+                    {
+                        GENERAL[_ana]->ROI[i]->result_klasse = 0;
+                        GENERAL[_ana]->ROI[i]->result_klasse = tflite->GetClassFromImageBasis(GENERAL[_ana]->ROI[i]->image);
+                        printf("Result General(Digit)%i: %d\n", i, GENERAL[_ana]->ROI[i]->result_klasse);
+
+                        if (isLogImage)
+                        {
+                            if (isLogImageSelect)
+                            {
+                                if (LogImageSelect.find(GENERAL[_ana]->ROI[i]->name) != std::string::npos)
+                                    LogImage(logPath, GENERAL[_ana]->ROI[i]->name, NULL, &GENERAL[_ana]->ROI[i]->result_klasse, time, GENERAL[_ana]->ROI[i]->image_org);
+                            }
+                            else
+                            {
+                                LogImage(logPath, GENERAL[_ana]->ROI[i]->name, NULL, &GENERAL[_ana]->ROI[i]->result_klasse, time, GENERAL[_ana]->ROI[i]->image_org);
+                            }
+                        }
+                    } break;
+                case DigitalHyprid:
+                    {
+                        int _num, _nachkomma;
+
+                        tflite->LoadInputImageBasis(GENERAL[_ana]->ROI[i]->image);        
+                        tflite->Invoke();
+                        if (debugdetailgeneral) LogFile.WriteToFile("Nach Invoke");
+
+                        _num = tflite->GetOutClassification(0, 10);
+                        _nachkomma = tflite->GetOutClassification(11, 21);
+
+
+                        string _zwres = "Nach Invoke - Nummer: " + to_string(_num) + " Nachkomma: " + to_string(_nachkomma);
+                        if (debugdetailgeneral) LogFile.WriteToFile(_zwres);
+
+                        if ((_num == 10) || (_nachkomma == 10))                      // NaN detektiert
+                            GENERAL[_ana]->ROI[i]->result_float = -1;
+                        else
+                            GENERAL[_ana]->ROI[i]->result_float = fmod((double) _num + (((double)_nachkomma)-5)/10 + (double) 10, 10);
+
+                        printf("Result General(DigitalHyprid)%i: %f\n", i, GENERAL[_ana]->ROI[i]->result_float); 
+                        _zwres = "Result General(DigitalHyprid)" + to_string(i) + ": " + to_string(GENERAL[_ana]->ROI[i]->result_float);
+                        if (debugdetailgeneral) LogFile.WriteToFile(_zwres);
+
+                        if (isLogImage)
+                            LogImage(logPath, GENERAL[_ana]->ROI[i]->name, &GENERAL[_ana]->ROI[i]->result_float, NULL, time, GENERAL[_ana]->ROI[i]->image_org);
+                    } break;
+                default:
+                    break;
+            }
+        }
+    }
+
+    delete tflite;
+
+    return true;
+}
+
+bool ClassFlowCNNGeneral::isExtendedResolution(int _number)
+{
+//    if (extendedResolution && !(CNNType == Digital))
+    if (!(CNNType == Digital))
+        return true;
+
+    return false;
+}
+
+
+
+std::vector<HTMLInfo*> ClassFlowCNNGeneral::GetHTMLInfo()
+{
+    std::vector<HTMLInfo*> result;
+
+    for (int _ana = 0; _ana < GENERAL.size(); ++_ana)
+        for (int i = 0; i < GENERAL[_ana]->ROI.size(); ++i)
+        {
+                if (GENERAL[_ana]->name == "default")
+                    GENERAL[_ana]->ROI[i]->image->SaveToFile(FormatFileName("/sdcard/img_tmp/" + GENERAL[_ana]->ROI[i]->name + ".bmp"));
+                else
+                    GENERAL[_ana]->ROI[i]->image->SaveToFile(FormatFileName("/sdcard/img_tmp/" + GENERAL[_ana]->name + "_" + GENERAL[_ana]->ROI[i]->name + ".bmp"));
+
+
+            HTMLInfo *zw = new HTMLInfo;
+            if (GENERAL[_ana]->name == "default")
+            {
+                zw->filename = GENERAL[_ana]->ROI[i]->name + ".bmp";
+                zw->filename_org = GENERAL[_ana]->ROI[i]->name + ".jpg";
+            }
+            else
+            {
+                zw->filename = GENERAL[_ana]->name + "_" + GENERAL[_ana]->ROI[i]->name + ".bmp";
+                zw->filename_org = GENERAL[_ana]->name + "_" + GENERAL[_ana]->ROI[i]->name + ".jpg";
+            }
+
+            if (CNNType == Digital)
+                zw->val = GENERAL[_ana]->ROI[i]->result_klasse;
+            else
+                zw->val = GENERAL[_ana]->ROI[i]->result_float;
+            zw->image = GENERAL[_ana]->ROI[i]->image;
+            zw->image_org = GENERAL[_ana]->ROI[i]->image_org;
+
+//            printf("Push %s\n", zw->filename.c_str());
+
+            result.push_back(zw);
+        }
+
+//    printf("größe: %d\n", result.size());
+
+    return result;
+}
+
+int ClassFlowCNNGeneral::getAnzahlGENERAL()
+{
+    return GENERAL.size();
+}
+
+string ClassFlowCNNGeneral::getNameGENERAL(int _analog)
+{
+    if (_analog < GENERAL.size())
+        return GENERAL[_analog]->name;
+
+    return "GENERAL DOES NOT EXIST";
+}
+
+general* ClassFlowCNNGeneral::GetGENERAL(int _analog)
+{
+    if (_analog < GENERAL.size())
+        return GENERAL[_analog];
+
+    return NULL;
+}
+
+
+
+void ClassFlowCNNGeneral::UpdateNameNumbers(std::vector<std::string> *_name_numbers)
+{
+    for (int _dig = 0; _dig < GENERAL.size(); _dig++)
+    {
+        std::string _name = GENERAL[_dig]->name;
+        bool found = false;
+        for (int i = 0; i < (*_name_numbers).size(); ++i)
+        {
+            if ((*_name_numbers)[i] == _name)
+                found = true;
+        }
+        if (!found)
+            (*_name_numbers).push_back(_name);
+    }
+}

code/components/jomjol_flowcontroll/ClassFlowCNNGeneral.h

@@ -0,0 +1,72 @@
+#ifndef __CLASSCNNGENERAL__
+#define __CLASSCNNGENERAL__
+
+#include"ClassFlowDefineTypes.h"
+#include "ClassFlowAlignment.h"
+// #include "ClassFlowPostProcessing.h"
+
+
+enum t_CNNType {
+    AutoDetect,
+    Analogue,
+    Digital,
+    DigitalHyprid,
+    None
+ };
+
+class ClassFlowCNNGeneral :
+    public ClassFlowImage
+{
+protected:
+    t_CNNType CNNType;
+    std::vector<general*> GENERAL;
+
+    string cnnmodelfile;
+    int modelxsize, modelysize;
+    bool isLogImageSelect;
+    string LogImageSelect;
+    ClassFlowAlignment* flowpostalignment;
+//    ClassFlowPostProcessing *flowpostprocessing = NULL;
+    bool SaveAllFiles;   
+//    bool extendedResolution;
+
+    int ZeigerEval(float zahl, int ziffer_vorgaenger);
+    int ZeigerEvalHybrid(float zahl, float zahl_vorgaenger, int eval_vorgaenger);
+
+
+    bool doNeuralNetwork(string time); 
+    bool doAlignAndCut(string time);
+
+public:
+    ClassFlowCNNGeneral(ClassFlowAlignment *_flowalign, t_CNNType _cnntype = AutoDetect);
+
+    bool ReadParameter(FILE* pfile, string& aktparamgraph);
+    bool doFlow(string time);
+
+    string getHTMLSingleStep(string host);
+    string getReadout(int _analog, bool _extendedResolution);   
+
+    void DrawROI(CImageBasis *_zw); 
+
+   	std::vector<HTMLInfo*> GetHTMLInfo();   
+
+//    int AnzahlROIs(int _analog);
+    int getAnzahlGENERAL();
+    general* GetGENERAL(int _analog);
+    general* GetGENERAL(string _name, bool _create);
+    general* FindGENERAL(string _name_number);    
+    string getNameGENERAL(int _analog);    
+
+    bool isExtendedResolution(int _number = 0);
+
+//    void setPostprocessing(ClassFlowPostProcessing *_fpp){flowpostprocessing = _fpp;}; 
+
+    void UpdateNameNumbers(std::vector<std::string> *_name_numbers);
+
+    t_CNNType getCNNType(){return CNNType;};
+
+    string name(){return "ClassFlowCNNGeneral";}; 
+};
+
+#endif
+

code/components/jomjol_flowcontroll/ClassFlowControll.cpp

@@ -6,12 +6,23 @@
 #include "freertos/task.h"
 
 #include <sys/stat.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
 #include <dirent.h>
+#ifdef __cplusplus
+}
+#endif
+
 #include "ClassLogFile.h"
 #include "time_sntp.h"
 #include "Helper.h"
 #include "server_ota.h"
 
+
+//#include "CImg.h"
+
 #include "server_help.h"
 
 //#define DEBUG_DETAIL_ON  
@@ -32,10 +43,10 @@ std::string ClassFlowControll::doSingleStep(std::string _stepname, std::string _
     if ((_stepname.compare(0, 7, "[Digits") == 0) || (_stepname.compare(0, 8, ";[Digits") == 0)) {
 //    if ((_stepname.compare("[Digits]") == 0) || (_stepname.compare(";[Digits]") == 0)){
 //        printf("Digits!!!\n");
-        _classname = "ClassFlowDigit";
+        _classname = "ClassFlowCNNGeneral";
     }
     if ((_stepname.compare("[Analog]") == 0) || (_stepname.compare(";[Analog]") == 0)){
-        _classname = "ClassFlowAnalog";
+        _classname = "ClassFlowCNNGeneral";
     }
     if ((_stepname.compare("[MQTT]") == 0) || (_stepname.compare(";[MQTT]") == 0)){
         _classname = "ClassFlowMQTT";
@@ -51,11 +62,33 @@ std::string ClassFlowControll::doSingleStep(std::string _stepname, std::string _
     return result;
 }
 
-std::vector<HTMLInfo*> ClassFlowControll::GetAllDigital()
+
+std::string ClassFlowControll::TranslateAktstatus(std::string _input)
 {
-    for (int i = 0; i < FlowControll.size(); ++i)
-        if (FlowControll[i]->name().compare("ClassFlowDigit") == 0)
-            return ((ClassFlowDigit*) (FlowControll[i]))->GetHTMLInfo();
+    if (_input.compare("ClassFlowMakeImage") == 0)
+        return ("Take Image");
+    if (_input.compare("ClassFlowAlignment") == 0)
+        return ("Aligning");
+    //if (_input.compare("ClassFlowAnalog") == 0)
+    //    return ("Analog ROIs");
+    if (_input.compare("ClassFlowCNNGeneral") == 0)
+        return ("Digitalization of ROIs");
+    if (_input.compare("ClassFlowMQTT") == 0)
+        return ("Sending MQTT");
+    if (_input.compare("ClassFlowPostProcessing") == 0)
+        return ("Processing");
+
+    return "Unkown Status";
+}
+
+
+std::vector<HTMLInfo*> ClassFlowControll::GetAllDigital() 
+{
+    if (flowdigit)
+    {
+        printf("ClassFlowControll::GetAllDigital - flowdigit != NULL\n");
+        return flowdigit->GetHTMLInfo();
+    }
 
     std::vector<HTMLInfo*> empty;
     return empty;
@@ -63,14 +96,43 @@ std::vector<HTMLInfo*> ClassFlowControll::GetAllDigital()
 
 std::vector<HTMLInfo*> ClassFlowControll::GetAllAnalog()
 {
-    for (int i = 0; i < FlowControll.size(); ++i)
-        if (FlowControll[i]->name().compare("ClassFlowAnalog") == 0)
-            return ((ClassFlowAnalog*) (FlowControll[i]))->GetHTMLInfo();
+    if (flowanalog)
+        return flowanalog->GetHTMLInfo();
 
     std::vector<HTMLInfo*> empty;
     return empty;
 }
 
+t_CNNType ClassFlowControll::GetTypeDigital()
+{
+    if (flowdigit)
+        return flowdigit->getCNNType();
+
+    return t_CNNType::None;
+}
+
+t_CNNType ClassFlowControll::GetTypeAnalog()
+{
+    if (flowanalog)
+        return flowanalog->getCNNType();
+
+    return t_CNNType::None;
+}
+
+
+
+
+string ClassFlowControll::GetMQTTMainTopic()
+{
+    for (int i = 0; i < FlowControll.size(); ++i)
+        if (FlowControll[i]->name().compare("ClassFlowMQTT") == 0)
+            return ((ClassFlowMQTT*) (FlowControll[i]))->GetMQTTMainTopic();
+
+
+    return "";
+}
+
+
 
 void ClassFlowControll::SetInitialParameter(void)
 {
@@ -82,7 +144,7 @@ void ClassFlowControll::SetInitialParameter(void)
     flowpostprocessing = NULL;
     disabled = false;
     aktRunNr = 0;
-    aktstatus = "Startup";
+    aktstatus = "Booting ...";
 
 }
 
@@ -110,19 +172,20 @@ ClassFlow* ClassFlowControll::CreateClassFlow(std::string _type)
     }
     if (toUpper(_type).compare("[ANALOG]") == 0)
     {
-        cfc = new ClassFlowAnalog(&FlowControll);
-        flowanalog = (ClassFlowAnalog*) cfc;
+        cfc = new ClassFlowCNNGeneral(flowalignment);
+        flowanalog = (ClassFlowCNNGeneral*) cfc;
     }
     if (toUpper(_type).compare(0, 7, "[DIGITS") == 0)
     {
-        cfc = new ClassFlowDigit(&FlowControll);
-        flowdigit = (ClassFlowDigit*) cfc;
+        cfc = new ClassFlowCNNGeneral(flowalignment);
+        flowdigit = (ClassFlowCNNGeneral*) cfc;
     }
     if (toUpper(_type).compare("[MQTT]") == 0)
         cfc = new ClassFlowMQTT(&FlowControll);
+        
     if (toUpper(_type).compare("[POSTPROCESSING]") == 0)
     {
-        cfc = new ClassFlowPostProcessing(&FlowControll); 
+        cfc = new ClassFlowPostProcessing(&FlowControll, flowanalog, flowdigit); 
         flowpostprocessing = (ClassFlowPostProcessing*) cfc;
     }
 
@@ -167,14 +230,17 @@ void ClassFlowControll::InitFlow(std::string config)
         cfc = CreateClassFlow(line);
         if (cfc)
         {
-            printf("Start ReadParameter\n");
+            printf("Start ReadParameter (%s)\n", line.c_str());
             cfc->ReadParameter(pFile, line);
         }
         else
         {
-            fgets(zw, 1024, pFile);
-            printf("%s", zw);
-            line = std::string(zw);
+            line = "";
+            if (fgets(zw, 1024, pFile) && !feof(pFile))
+                {
+                    printf("Read: %s", zw);
+                    line = std::string(zw);
+                }
         }
     }
 
@@ -182,8 +248,8 @@ void ClassFlowControll::InitFlow(std::string config)
 
 }
 
-std::string ClassFlowControll::getActStatus(){
-    return aktstatus;
+std::string* ClassFlowControll::getActStatus(){
+    return &aktstatus;
 }
 
 void ClassFlowControll::doFlowMakeImageOnly(string time){
@@ -192,9 +258,9 @@ void ClassFlowControll::doFlowMakeImageOnly(string time){
     for (int i = 0; i < FlowControll.size(); ++i)
     {
         if (FlowControll[i]->name() == "ClassFlowMakeImage") {
-            zw_time = gettimestring("%Y%m%d-%H%M%S");
-            aktstatus = zw_time + ": " + FlowControll[i]->name();
-            string zw = "FlowControll.doFlowMakeImageOnly - " + FlowControll[i]->name();
+//            zw_time = gettimestring("%Y%m%d-%H%M%S");
+            zw_time = gettimestring("%H:%M:%S");
+            aktstatus = TranslateAktstatus(FlowControll[i]->name()) + " (" + zw_time + ")";
             FlowControll[i]->doFlow(time);
         }
     }
@@ -209,13 +275,16 @@ bool ClassFlowControll::doFlow(string time)
     int repeat = 0;
 
 #ifdef DEBUG_DETAIL_ON 
-    LogFile.WriteHeapInfo("ClassFlowAnalog::doFlow - Start");
+    LogFile.WriteHeapInfo("ClassFlowControll::doFlow - Start");
 #endif
 
     for (int i = 0; i < FlowControll.size(); ++i)
     {
-        zw_time = gettimestring("%Y%m%d-%H%M%S");
-        aktstatus = zw_time + ": " + FlowControll[i]->name();
+        zw_time = gettimestring("%H:%M:%S");
+        aktstatus = TranslateAktstatus(FlowControll[i]->name()) + " (" + zw_time + ")";
+
+//        zw_time = gettimestring("%Y%m%d-%H%M%S");
+//        aktstatus = zw_time + ": " + FlowControll[i]->name();
         
        
         string zw = "FlowControll.doFlow - " + FlowControll[i]->name();
@@ -224,7 +293,7 @@ bool ClassFlowControll::doFlow(string time)
         if (!FlowControll[i]->doFlow(time)){
             repeat++;
             LogFile.WriteToFile("Fehler im vorheriger Schritt - wird zum " + to_string(repeat) + ". Mal wiederholt");
-            i = -1;    // vorheriger Schritt muss wiederholt werden (vermutlich Bilder aufnehmen)
+            if (i) i -= 1;    // vorheriger Schritt muss wiederholt werden (vermutlich Bilder aufnehmen)
             result = false;
             if (repeat > 5) {
                 LogFile.WriteToFile("Wiederholung 5x nicht erfolgreich --> reboot");
@@ -238,28 +307,56 @@ bool ClassFlowControll::doFlow(string time)
         }
         
 #ifdef DEBUG_DETAIL_ON  
-        LogFile.WriteHeapInfo("ClassFlowAnalog::doFlow");
+        LogFile.WriteHeapInfo("ClassFlowControll::doFlow");
 #endif
 
     }
-    zw_time = gettimestring("%Y%m%d-%H%M%S");    
-    aktstatus = zw_time + ": Flow is done";
+    zw_time = gettimestring("%H:%M:%S");
+    aktstatus = "Flow finished (" + zw_time + ")";
     return result;
 }
 
-void ClassFlowControll::UpdateAktStatus(std::string _flow)
+
+string ClassFlowControll::getReadoutAll(int _type)
 {
-    aktstatus = gettimestring("%Y%m%d-%H%M%S");
-    aktstatus = aktstatus + "\t" + std::to_string(aktRunNr) + "\t";
-    
-    if (_flow == "ClassFlowMakeImage")
-        aktstatus = aktstatus + "Taking Raw Image";
-    else
-        if (_flow == "ClassFlowAlignment")
-            aktstatus = aktstatus + "Aligning Image";
+    std::string out = "";
+    if (flowpostprocessing)
+    {
+        std::vector<NumberPost*> *numbers = flowpostprocessing->GetNumbers();
 
+        for (int i = 0; i < (*numbers).size(); ++i)
+        {
+            out = out + (*numbers)[i]->name + "\t";
+            switch (_type) {
+                case READOUT_TYPE_VALUE:
+                    out = out + (*numbers)[i]->ReturnValueNoError;
+                    break;
+                case READOUT_TYPE_PREVALUE:
+                    if (flowpostprocessing->PreValueUse)
+                    {
+                        if ((*numbers)[i]->PreValueOkay)
+                            out = out + (*numbers)[i]->ReturnPreValue;
+                        else
+                            out = out + "PreValue too old";                
+                    }
+                    else
+                        out = out + "PreValue deactivated";
+                    break;
+                case READOUT_TYPE_RAWVALUE:
+                    out = out + (*numbers)[i]->ReturnRawValue;
+                    break;
+                case READOUT_TYPE_ERROR:
+                    out = out + (*numbers)[i]->ErrorMessageText;
+                    break;
+            }
+            if (i < (*numbers).size()-1)
+                out = out + "\r\n";
+        }
+    //    printf("OUT: %s", out.c_str());
+    }
 
-}
+    return out;
+}	
 
 
 string ClassFlowControll::getReadout(bool _rawvalue = false, bool _noerror = false)
@@ -285,17 +382,17 @@ string ClassFlowControll::getReadout(bool _rawvalue = false, bool _noerror = fal
     return result;
 }
 
-string ClassFlowControll::GetPrevalue()	
+string ClassFlowControll::GetPrevalue(std::string _number)	
 {
     if (flowpostprocessing)
     {
-        return flowpostprocessing->GetPreValue();   
+        return flowpostprocessing->GetPreValue(_number);   
     }
 
     return std::string();    
 }
 
-std::string ClassFlowControll::UpdatePrevalue(std::string _newvalue)
+std::string ClassFlowControll::UpdatePrevalue(std::string _newvalue, std::string _numbers, bool _extern)
 {
     float zw;
     char* p;
@@ -317,7 +414,7 @@ std::string ClassFlowControll::UpdatePrevalue(std::string _newvalue)
 
     if (flowpostprocessing)
     {
-        flowpostprocessing->SavePreValue(zw);
+        flowpostprocessing->SetPreValue(zw, _numbers, _extern);
         return _newvalue;    
     }
 
@@ -409,6 +506,7 @@ bool ClassFlowControll::ReadParameter(FILE* pfile, string& aktparamgraph)
     return true;
 }
 
+
 int ClassFlowControll::CleanTempFolder() {
     const char* folderPath = "/sdcard/img_tmp";
     
@@ -442,7 +540,7 @@ int ClassFlowControll::CleanTempFolder() {
 
 esp_err_t ClassFlowControll::SendRawJPG(httpd_req_t *req)
 {
-    return flowmakeimage->SendRawJPG(req);
+    return flowmakeimage != NULL ? flowmakeimage->SendRawJPG(req) : ESP_FAIL;
 }
 
 
@@ -454,51 +552,67 @@ esp_err_t ClassFlowControll::GetJPGStream(std::string _fn, httpd_req_t *req)
     esp_err_t result = ESP_FAIL;
     bool Dodelete = false;    
 
+    if (flowalignment == NULL)
+    {
+        printf("Can't continue, flowalignment is NULL\n");
+        return ESP_FAIL;
+    }
+
     if (_fn == "alg.jpg")
     {
         _send = flowalignment->ImageBasis;  
     }
-
-
-
-    if (_fn == "alg_roi.jpg")
+    else
     {
-        CImageBasis* _imgzw = new CImageBasis(flowalignment->ImageBasis);
-        flowalignment->DrawRef(_imgzw);
-        if (flowdigit) flowdigit->DrawROI(_imgzw);
-        if (flowanalog) flowanalog->DrawROI(_imgzw);
-        _send = _imgzw;
-        Dodelete = true;
-    }
+        if (_fn == "alg_roi.jpg")
+        {
+            CImageBasis* _imgzw = new CImageBasis(flowalignment->ImageBasis);
+            flowalignment->DrawRef(_imgzw);
+            if (flowdigit) flowdigit->DrawROI(_imgzw);
+            if (flowanalog) flowanalog->DrawROI(_imgzw);
+            _send = _imgzw;
+            Dodelete = true;
+        }
+        else
+        {
+            std::vector<HTMLInfo*> htmlinfo;
+            htmlinfo = GetAllDigital();
+            for (int i = 0; i < htmlinfo.size(); ++i)
+            {
+                if (_fn == htmlinfo[i]->filename)
+                {
+                    if (htmlinfo[i]->image)
+                        _send = htmlinfo[i]->image;
+                }
+                if (_fn == htmlinfo[i]->filename_org)
+                {
+                    if (htmlinfo[i]->image_org)
+                        _send = htmlinfo[i]->image_org;        
+                }
+                delete htmlinfo[i];
+            }
+            htmlinfo.clear();
 
-    std::vector<HTMLInfo*> htmlinfo;
-    htmlinfo = GetAllDigital();
-    for (int i = 0; i < htmlinfo.size(); ++i)
-    {
-        if (_fn == htmlinfo[i]->filename)
-        {
-            if (htmlinfo[i]->image)
-                _send = htmlinfo[i]->image;
-        }
-        if (_fn == htmlinfo[i]->filename_org)
-        {
-            if (htmlinfo[i]->image_org)
-                _send = htmlinfo[i]->image_org;        
-        }
-    }
+            if (!_send)
+            {
+                htmlinfo = GetAllAnalog();
+                for (int i = 0; i < htmlinfo.size(); ++i)
+                {
+                    if (_fn == htmlinfo[i]->filename)
+                    {
+                        if (htmlinfo[i]->image)
+                            _send = htmlinfo[i]->image;
+                    }
+                    if (_fn == htmlinfo[i]->filename_org)
+                    {
+                        if (htmlinfo[i]->image_org)
+                            _send = htmlinfo[i]->image_org;        
+                    }
+                    delete htmlinfo[i];
+                }
+                htmlinfo.clear();
 
-    htmlinfo = GetAllAnalog();
-    for (int i = 0; i < htmlinfo.size(); ++i)
-    {
-        if (_fn == htmlinfo[i]->filename)
-        {
-            if (htmlinfo[i]->image)
-                _send = htmlinfo[i]->image;
-        }
-        if (_fn == htmlinfo[i]->filename_org)
-        {
-            if (htmlinfo[i]->image_org)
-                _send = htmlinfo[i]->image_org;        
+            }
         }
     }
 
@@ -518,4 +632,37 @@ esp_err_t ClassFlowControll::GetJPGStream(std::string _fn, httpd_req_t *req)
     }
 
     return result;
-}
+}
+
+
+string ClassFlowControll::getJSON()
+{
+    std::vector<NumberPost*>* NUMBERS = flowpostprocessing->GetNumbers();
+
+    std::string json="{\n";
+
+    for (int i = 0; i < (*NUMBERS).size(); ++i)
+    {
+        json += "\"" + (*NUMBERS)[i]->name + "\":\n";
+        json += "  {\n";
+        if ((*NUMBERS)[i]->ReturnValueNoError.length() > 0)
+            json += "    \"value\": "      + (*NUMBERS)[i]->ReturnValueNoError          + ",\n";
+        else
+            json += "    \"value\": \"\",\n";
+        json += "    \"raw\": \""        + (*NUMBERS)[i]->ReturnRawValue              + "\",\n";
+        json += "    \"error\": \""     + (*NUMBERS)[i]->ErrorMessageText             + "\",\n";
+        if ((*NUMBERS)[i]->ReturnRateValue.length() > 0)
+            json += "    \"rate\": "      + (*NUMBERS)[i]->ReturnRateValue                + ",\n";
+        else
+            json += "    \"rate\": \"\",\n";
+
+        json += "    \"timestamp\": \"" + (*NUMBERS)[i]->timeStamp                    + "\"\n";
+        if ((i+1) < (*NUMBERS).size())
+            json += "  },\n";
+        else
+            json += "  }\n";
+    }
+    json += "}";
+
+    return json;
+}

code/components/jomjol_flowcontroll/ClassFlowControll.h

@@ -1,14 +1,21 @@
-#pragma once
+#ifndef __FLOWCONTROLL__
+#define __FLOWCONTROLL__
 
 #include <string>
 
 #include "ClassFlow.h"
 #include "ClassFlowMakeImage.h"
 #include "ClassFlowAlignment.h"
-#include "ClassFlowDigit.h"
-#include "ClassFlowAnalog.h"
+#include "ClassFlowCNNGeneral.h"
 #include "ClassFlowPostProcessing.h"
 #include "ClassFlowMQTT.h"
+#include "ClassFlowCNNGeneral.h"
+
+
+#define READOUT_TYPE_VALUE 0
+#define READOUT_TYPE_PREVALUE 1
+#define READOUT_TYPE_RAWVALUE 2
+#define READOUT_TYPE_ERROR 3
 
 
 class ClassFlowControll :
@@ -18,8 +25,9 @@ protected:
 	std::vector<ClassFlow*> FlowControll;
 	ClassFlowPostProcessing* flowpostprocessing;
 	ClassFlowAlignment* flowalignment;	
-	ClassFlowAnalog* flowanalog;
-	ClassFlowDigit* flowdigit;
+	ClassFlowCNNGeneral* flowanalog;
+	ClassFlowCNNGeneral* flowdigit;
+//	ClassFlowDigit* flowdigit;
 	ClassFlowMakeImage* flowmakeimage;
 	ClassFlow* CreateClassFlow(std::string _type);
 
@@ -30,17 +38,21 @@ protected:
 	std::string aktstatus;
 	int aktRunNr;
 
-	void UpdateAktStatus(std::string _flow);
-
 public:
 	void InitFlow(std::string config);
 	bool doFlow(string time);
 	void doFlowMakeImageOnly(string time);
 	bool getStatusSetupModus(){return SetupModeActive;};
 	string getReadout(bool _rawvalue, bool _noerror);
-	string UpdatePrevalue(std::string _newvalue);
-	string GetPrevalue();	
+	string getReadoutAll(int _type);	
+	string UpdatePrevalue(std::string _newvalue, std::string _numbers, bool _extern);
+	string GetPrevalue(std::string _number = "");	
 	bool ReadParameter(FILE* pfile, string& aktparamgraph);	
+	string getJSON();
+
+	string TranslateAktstatus(std::string _input);
+
+	string GetMQTTMainTopic();
 
 	esp_err_t GetJPGStream(std::string _fn, httpd_req_t *req);
 	esp_err_t SendRawJPG(httpd_req_t *req);
@@ -49,14 +61,19 @@ public:
 
 	bool isAutoStart(long &_intervall);
 
-	std::string getActStatus();
+	std::string* getActStatus();
 
 	std::vector<HTMLInfo*> GetAllDigital();
 	std::vector<HTMLInfo*> GetAllAnalog();	
 
+	t_CNNType GetTypeDigital();
+	t_CNNType GetTypeAnalog();
+
 	int CleanTempFolder();
 
 	string name(){return "ClassFlowControll";};
 };
 
+#endif
+
 

code/components/jomjol_flowcontroll/ClassFlowDefineTypes.h

@@ -0,0 +1,53 @@
+#ifndef __CLASSFLOWIMAGE_CLASS__
+#define __CLASSFLOWIMAGE_CLASS__
+
+#include "ClassFlowImage.h"
+
+struct roi {
+    int posx, posy, deltax, deltay;
+    float result_float;
+    int result_klasse;
+    string name;
+    CImageBasis *image, *image_org;
+};
+
+struct general {
+    string name;
+    std::vector<roi*> ROI;
+};
+
+
+struct NumberPost {
+    float MaxRateValue;
+    bool useMaxRateValue;
+    bool ErrorMessage;
+    bool PreValueOkay;
+    bool AllowNegativeRates;
+    bool checkDigitIncreaseConsistency;
+    time_t lastvalue;
+    string timeStamp;
+    float FlowRateAct;          // m3 / min
+    float PreValue;             // letzter Wert, der gut ausgelesen wurde
+    float Value;                // letzer ausgelesener Wert, inkl. Korrekturen
+    string ReturnRateValue;      // RückgabewertRate
+    string ReturnRawValue;      // Rohwert (mit N & führenden 0)    
+    string ReturnValue;         // korrigierter Rückgabewert, ggf. mit Fehlermeldung
+    string ReturnPreValue;  // korrigierter Rückgabewert ohne Fehlermeldung
+    string ReturnValueNoError;
+    string ErrorMessageText;        // Fehlermeldung bei Consistency Check
+    int AnzahlAnalog;
+    int AnzahlDigital;
+    int DecimalShift;
+    int DecimalShiftInitial;
+    int Nachkomma;
+
+    bool isExtendedResolution;
+
+    general *digit_roi;
+    general *analog_roi;
+
+    string name;
+};
+
+#endif
+

code/components/jomjol_flowcontroll/ClassFlowDigit.cpp

@@ -1,293 +0,0 @@
-#include "ClassFlowDigit.h"
-
-
-//#include "CFindTemplate.h"
-//#include "CTfLiteClass.h"
-
-// #define OHNETFLITE
-
-#ifndef OHNETFLITE
-#include "CTfLiteClass.h"
-#endif
-
-// #include "bitmap_image.hpp"
-
-#include "ClassLogFile.h"
-
-static const char* TAG = "flow_digital";
-
-
-void ClassFlowDigit::SetInitialParameter(void)
-{
-    string cnnmodelfile = "";
-    modelxsize = 1;
-    modelysize = 1;
-    ListFlowControll = NULL;
-    previousElement = NULL;    
-    SaveAllFiles = false;
-    disabled = false;
-    DecimalShift = 0;
-    DecimalShiftEnabled = false;
-}    
-
-ClassFlowDigit::ClassFlowDigit() : ClassFlowImage(TAG)
-{
-    SetInitialParameter();
-}
-
-ClassFlowDigit::ClassFlowDigit(std::vector<ClassFlow*>* lfc) : ClassFlowImage(lfc, TAG)
-{
-    SetInitialParameter();
-    ListFlowControll = lfc;
-
-    for (int i = 0; i < ListFlowControll->size(); ++i)
-    {
-        if (((*ListFlowControll)[i])->name().compare("ClassFlowAlignment") == 0)
-        {
-            flowpostalignment = (ClassFlowAlignment*) (*ListFlowControll)[i];
-        }
-    }
-}
-
-ClassFlowDigit::ClassFlowDigit(std::vector<ClassFlow*>* lfc, ClassFlow *_prev) : ClassFlowImage(lfc, _prev, TAG)
-{
-    SetInitialParameter();
-    ListFlowControll = lfc;
-    previousElement = _prev;
-
-    for (int i = 0; i < ListFlowControll->size(); ++i)
-    {
-        if (((*ListFlowControll)[i])->name().compare("ClassFlowAlignment") == 0)
-        {
-            flowpostalignment = (ClassFlowAlignment*) (*ListFlowControll)[i];
-        }
-    }    
-}
-
-string ClassFlowDigit::getReadout()
-{
-    string rst = "";
-
-    for (int i = 0; i < ROI.size(); ++i)
-    {
-        if (ROI[i]->resultklasse == 10)
-            rst = rst + "N";
-        else
-            rst = rst + std::to_string(ROI[i]->resultklasse);
-    }
-
-    return rst;
-}
-
-bool ClassFlowDigit::ReadParameter(FILE* pfile, string& aktparamgraph)
-{
-    std::vector<string> zerlegt;
-
-    aktparamgraph = trim(aktparamgraph);
-
-    if (aktparamgraph.size() == 0)
-        if (!this->GetNextParagraph(pfile, aktparamgraph)) 
-            return false;
-
-    printf("aktparamgraph: %s\n", aktparamgraph.c_str());
-
-
-/*
-    if ((aktparamgraph.compare("[Digits]") != 0) && (aktparamgraph.compare(";[Digits]") != 0))       // Paragraph passt nich zu MakeImage
-        return false;
-*/
-
-    if ((aktparamgraph.compare(0, 7, "[Digits") != 0) && (aktparamgraph.compare(0, 8, ";[Digits") != 0))       // Paragraph passt nich zu MakeImage
-        return false;
-
-    int _pospkt = aktparamgraph.find_first_of(".");
-    int _posklammerzu = aktparamgraph.find_first_of("]");
-//    printf("Pos: %d, %d\n", _pospkt, _posklammerzu);
-    if (_pospkt > -1)
-        NameDigit = aktparamgraph.substr(_pospkt+1, _posklammerzu - _pospkt-1);
-    else
-        NameDigit = "";
-    printf("Name Digit: %s\n", NameDigit.c_str());
-
-    if (aktparamgraph[0] == ';')
-    {
-        disabled = true;
-        while (getNextLine(pfile, &aktparamgraph) && !isNewParagraph(aktparamgraph));
-        printf("[Digits] is disabled !!!\n");
-        return true;
-    }
-
-
-    while (getNextLine(pfile, &aktparamgraph) && !isNewParagraph(aktparamgraph))
-    {
-        zerlegt = this->ZerlegeZeile(aktparamgraph);
-        if ((zerlegt[0] == "LogImageLocation") && (zerlegt.size() > 1))
-        {
-            LogImageLocation = "/sdcard" + zerlegt[1];
-            isLogImage = true;            
-        }
-        if ((zerlegt[0] == "Model") && (zerlegt.size() > 1))
-        {
-            cnnmodelfile = zerlegt[1];
-        }
-        if ((zerlegt[0] == "ModelInputSize") && (zerlegt.size() > 2))
-        {
-            modelxsize = std::stoi(zerlegt[1]);
-            modelysize = std::stoi(zerlegt[2]);
-        }
-        if (zerlegt.size() >= 5)
-        {
-            roi* neuroi = new roi;
-            neuroi->name = zerlegt[0];
-            neuroi->posx = std::stoi(zerlegt[1]);
-            neuroi->posy = std::stoi(zerlegt[2]);
-            neuroi->deltax = std::stoi(zerlegt[3]);
-            neuroi->deltay = std::stoi(zerlegt[4]);
-            neuroi->resultklasse = -1;
-            neuroi->image = NULL;
-            neuroi->image_org = NULL;            
-            ROI.push_back(neuroi);
-        }
-
-        if ((toUpper(zerlegt[0]) == "SAVEALLFILES") && (zerlegt.size() > 1))
-        {
-            if (toUpper(zerlegt[1]) == "TRUE")
-                SaveAllFiles = true;
-        }
-
-    }
-
-    for (int i = 0; i < ROI.size(); ++i)
-    {
-        ROI[i]->image = new CImageBasis(modelxsize, modelysize, 3);
-        ROI[i]->image_org = new CImageBasis(ROI[i]->deltax, ROI[i]->deltay, 3);
-    }
-
-    return true;
-}
-
-
-string ClassFlowDigit::getHTMLSingleStep(string host)
-{
-    string result, zw;
-    std::vector<HTMLInfo*> htmlinfo;
-
-    result = "<p>Found ROIs: </p> <p><img src=\"" + host + "/img_tmp/alg_roi.jpg\"></p>\n";
-    result = result + "Digital Counter: <p> ";
-
-    htmlinfo = GetHTMLInfo();
-    for (int i = 0; i < htmlinfo.size(); ++i)
-    {
-        if (htmlinfo[i]->val == 10)
-            zw = "NaN";
-        else
-        {
-            zw = to_string((int) htmlinfo[i]->val);
-        }
-        result = result + "<img src=\"" + host + "/img_tmp/" +  htmlinfo[i]->filename + "\"> " + zw;
-        delete htmlinfo[i];
-    }
-    htmlinfo.clear();    
-
-    return result;
-}
-
-
-bool ClassFlowDigit::doFlow(string time)
-{
-    if (disabled)
-        return true;
-        
-    if (!doAlignAndCut(time)){
-        return false;
-    };
-
-    doNeuralNetwork(time);
-
-    RemoveOldLogs();
-
-    return true;
-}
-
-bool ClassFlowDigit::doAlignAndCut(string time)
-{
-    if (disabled)
-        return true;
-
-    CAlignAndCutImage *caic = flowpostalignment->GetAlignAndCutImage();
-
-    for (int i = 0; i < ROI.size(); ++i)
-    {
-        printf("DigitalDigit %d - Align&Cut\n", i);
-        
-        caic->CutAndSave(ROI[i]->posx, ROI[i]->posy, ROI[i]->deltax, ROI[i]->deltay, ROI[i]->image_org);
-        if (SaveAllFiles) ROI[i]->image_org->SaveToFile(FormatFileName("/sdcard/img_tmp/" + ROI[i]->name + ".jpg"));
-
-        ROI[i]->image_org->Resize(modelxsize, modelysize, ROI[i]->image);
-        if (SaveAllFiles) ROI[i]->image->SaveToFile(FormatFileName("/sdcard/img_tmp/" + ROI[i]->name + ".bmp"));
-    }
-
-    return true;
-} 
-
-bool ClassFlowDigit::doNeuralNetwork(string time)
-{
-    if (disabled)
-        return true;
-            
-    string logPath = CreateLogFolder(time);
-
-#ifndef OHNETFLITE
-    CTfLiteClass *tflite = new CTfLiteClass;  
-    string zwcnn =  FormatFileName("/sdcard" + cnnmodelfile);
-    printf(zwcnn.c_str());printf("\n");
-    tflite->LoadModel(zwcnn); 
-    tflite->MakeAllocate();
-#endif
-
-    for (int i = 0; i < ROI.size(); ++i)
-    {
-        printf("DigitalDigit %d - TfLite\n", i);
-
-        ROI[i]->resultklasse = 0;
-#ifndef OHNETFLITE
-        ROI[i]->resultklasse = tflite->GetClassFromImageBasis(ROI[i]->image);
-
-#endif
-        printf("Result Digit%i: %d\n", i, ROI[i]->resultklasse);
-
-        if (isLogImage)
-        {
-            LogImage(logPath, ROI[i]->name, NULL, &ROI[i]->resultklasse, time, ROI[i]->image_org);
-        }
-    }
-#ifndef OHNETFLITE
-        delete tflite;
-#endif
-    return true;
-}
-
-void ClassFlowDigit::DrawROI(CImageBasis *_zw)
-{
-    for (int i = 0; i < ROI.size(); ++i)
-        _zw->drawRect(ROI[i]->posx, ROI[i]->posy, ROI[i]->deltax, ROI[i]->deltay, 0, 0, 255, 2);
-}     
-
-std::vector<HTMLInfo*> ClassFlowDigit::GetHTMLInfo()
-{
-    std::vector<HTMLInfo*> result;
-
-    for (int i = 0; i < ROI.size(); ++i)
-    {
-        HTMLInfo *zw = new HTMLInfo;
-        zw->filename = ROI[i]->name + ".bmp";
-        zw->filename_org = ROI[i]->name + ".jpg";
-        zw->val = ROI[i]->resultklasse;
-        zw->image = ROI[i]->image;
-        zw->image_org = ROI[i]->image_org;
-        result.push_back(zw);
-    }
-
-    return result;
-}
-

code/components/jomjol_flowcontroll/ClassFlowDigit.h

@@ -1,50 +0,0 @@
-#pragma once
-#include "ClassFlowImage.h"
-#include "ClassFlowAlignment.h"
-#include "Helper.h"
-
-#include <string>
-
-struct roi {
-    int posx, posy, deltax, deltay;
-    int resultklasse;
-    string name;
-    CImageBasis *image, *image_org;
-    roi* next;
-};
-
-class ClassFlowDigit :
-    public ClassFlowImage
-{
-protected:
-    std::vector<roi*> ROI;
-    string cnnmodelfile;
-    int modelxsize, modelysize;
-    bool SaveAllFiles;
-    string NameDigit;
-    int DecimalShift;
-    bool DecimalShiftEnabled;
-
-
-    ClassFlowAlignment* flowpostalignment;
- 
-    bool doNeuralNetwork(string time); 
-    bool doAlignAndCut(string time); 
-
-	void SetInitialParameter(void);    
-
-public:
-    ClassFlowDigit();
-    ClassFlowDigit(std::vector<ClassFlow*>* lfc);
-    ClassFlowDigit(std::vector<ClassFlow*>* lfc, ClassFlow *_prev);
-    bool ReadParameter(FILE* pfile, string& aktparamgraph);
-    bool doFlow(string time);
-    string getHTMLSingleStep(string host); 
-    string getReadout();
-   	std::vector<HTMLInfo*> GetHTMLInfo();
-
-    void DrawROI(CImageBasis *_zw);        
-
-    string name(){return "ClassFlowDigit";};
-};
-

code/components/jomjol_flowcontroll/ClassFlowImage.cpp

@@ -2,7 +2,15 @@
 #include <string>
 #include <string.h>
 #include <sys/stat.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
 #include <dirent.h>
+#ifdef __cplusplus
+}
+#endif
+
 #include "time_sntp.h"
 #include "ClassLogFile.h"
 #include "CImageBasis.h"
@@ -48,9 +56,14 @@ void ClassFlowImage::LogImage(string logPath, string name, float *resultFloat, i
 	if (!isLogImage)
 		return;
 	
+    
 	char buf[10];
+
 	if (resultFloat != NULL) {
-		sprintf(buf, "%.1f_", *resultFloat);
+        if (*resultFloat < 0)
+            sprintf(buf, "N.N_");
+        else
+            sprintf(buf, "%.1f_", *resultFloat);
 	} else if (resultInt != NULL) {
 		sprintf(buf, "%d_", *resultInt);
 	} else {

code/components/jomjol_flowcontroll/ClassFlowMQTT.cpp

@@ -1,6 +1,8 @@
+#include <sstream>
 #include "ClassFlowMQTT.h"
 #include "Helper.h"
 
+#include "time_sntp.h"
 #include "interface_mqtt.h"
 #include "ClassFlowPostProcessing.h"
 
@@ -13,6 +15,11 @@ void ClassFlowMQTT::SetInitialParameter(void)
     topicError = "";
     topicRate = "";
     topicTimeStamp = "";
+    maintopic = "";
+    mainerrortopic = ""; 
+
+    topicUptime = "";
+    topicFreeMem = "";
     clientname = "watermeter";
     OldValue = "";
     flowpostprocessing = NULL;  
@@ -21,6 +28,9 @@ void ClassFlowMQTT::SetInitialParameter(void)
     previousElement = NULL;
     ListFlowControll = NULL; 
     disabled = false;
+    MQTTenable = false;
+    
+    
 
 }       
 
@@ -88,53 +98,118 @@ bool ClassFlowMQTT::ReadParameter(FILE* pfile, string& aktparamgraph)
         {
             this->uri = zerlegt[1];
         }
-        if ((toUpper(zerlegt[0]) == "TOPIC") && (zerlegt.size() > 1))
-        {
-            this->topic = zerlegt[1];
-        }
-        if ((toUpper(zerlegt[0]) == "TOPICERROR") && (zerlegt.size() > 1))
-        {
-            this->topicError = zerlegt[1];
-        }
-        if ((toUpper(zerlegt[0]) == "TOPICRATE") && (zerlegt.size() > 1))
-        {
-            this->topicRate  = zerlegt[1];
-        }
-        if ((toUpper(zerlegt[0]) == "TOPICTIMESTAMP") && (zerlegt.size() > 1))
-        {
-            this->topicTimeStamp  = zerlegt[1];
-        }
 
         if ((toUpper(zerlegt[0]) == "CLIENTID") && (zerlegt.size() > 1))
         {
             this->clientname = zerlegt[1];
         }
 
+        if (((toUpper(zerlegt[0]) == "TOPIC") || (toUpper(zerlegt[0]) == "MAINTOPIC")) && (zerlegt.size() > 1))
+        {
+            maintopic = zerlegt[1];
+        }
     }
 
-    if ((uri.length() > 0) && (topic.length() > 0)) 
+    if (!MQTTisConnected() && (uri.length() > 0) && (maintopic.length() > 0)) 
     {
-        MQTTInit(uri, clientname, user, password, topicError, 60);
+        mainerrortopic = maintopic + "/connection";
+        MQTTInit(uri, clientname, user, password, mainerrortopic, 60); 
+        MQTTPublish(mainerrortopic, "connected");
+        MQTTenable = true;
     }
    
     return true;
 }
 
 
+string ClassFlowMQTT::GetMQTTMainTopic()
+{
+    return maintopic;
+}
+
+
 bool ClassFlowMQTT::doFlow(string zwtime)
 {
+    if (!MQTTenable)
+        return true;
+
     std::string result;
     std::string resulterror = "";
+    std::string resultraw = "";
     std::string resultrate = "";
     std::string resulttimestamp = "";
     string zw = "";
+    string namenumber = "";
+
+    MQTTPublish(mainerrortopic, "connected");
     
+    zw = maintopic + "/" + "uptime";
+    char uptimeStr[11];
+    sprintf(uptimeStr, "%ld", (long)getUpTime());
+    MQTTPublish(zw, uptimeStr);
+
+    zw = maintopic + "/" + "freeMem";
+    char freeheapmem[11];
+    sprintf(freeheapmem, "%zu", esp_get_free_heap_size());
+    MQTTPublish(zw, freeheapmem);
+
     if (flowpostprocessing)
     {
-        result =  flowpostprocessing->getReadoutParam(false, true);
-        resulterror = flowpostprocessing->getReadoutError();
-        resultrate = flowpostprocessing->getReadoutRate();
-        resulttimestamp = flowpostprocessing->getReadoutTimeStamp();
+        std::vector<NumberPost*>* NUMBERS = flowpostprocessing->GetNumbers();
+
+        for (int i = 0; i < (*NUMBERS).size(); ++i)
+        {
+            result =  (*NUMBERS)[i]->ReturnValueNoError;
+            resultraw =  (*NUMBERS)[i]->ReturnRawValue;
+            resulterror = (*NUMBERS)[i]->ErrorMessageText;
+            resultrate = (*NUMBERS)[i]->ReturnRateValue;
+            resulttimestamp = (*NUMBERS)[i]->timeStamp;
+
+            namenumber = (*NUMBERS)[i]->name;
+            if (namenumber == "default")
+                namenumber = maintopic + "/";
+            else
+                namenumber = maintopic + "/" + namenumber + "/";
+
+            zw = namenumber + "value"; 
+            if (result.length() > 0)   
+                MQTTPublish(zw, result);
+
+            zw = namenumber + "error"; 
+            if (resulterror.length() > 0)  
+                MQTTPublish(zw, resulterror, 1);
+
+            zw = namenumber + "rate"; 
+            if (resultrate.length() > 0)   
+                MQTTPublish(zw, resultrate);
+
+            zw = namenumber + "raw"; 
+            if (resultraw.length() > 0)   
+                MQTTPublish(zw, resultraw);
+
+            zw = namenumber + "timestamp";
+            if (resulttimestamp.length() > 0)
+                MQTTPublish(zw, resulttimestamp);
+
+
+            std::string json = "";
+            
+            if (result.length() > 0)
+                json += "{\"value\":"+result;
+            else
+                json += "{\"value\":\"\"";
+
+            json += ",\"raw\":\""+resultraw;
+            json += "\",\"error\":\""+resulterror;
+            if (resultrate.length() > 0)
+                json += "\",\"rate\":"+resultrate;
+            else
+                json += "\",\"rate\":\"\"";
+            json += ",\"timestamp\":\""+resulttimestamp+"\"}";
+
+            zw = namenumber + "json";
+            MQTTPublish(zw, json);
+        }
     }
     else
     {
@@ -149,26 +224,9 @@ bool ClassFlowMQTT::doFlow(string zwtime)
                     result = result + "\t" + zw;
             }
         }
+        MQTTPublish(topic, result);
     }
     
-    MQTTPublish(topic, result);
-
-    if (topicError.length() > 0) {
-        if (resulterror.length() == 0)
-        {
-            resulterror = " ";
-        }
-        MQTTPublish(topicError, resulterror, 1);
-    }
-
-    if (topicRate.length() > 0) {
-        MQTTPublish(topicRate, resultrate);
-    }
-
-    if (topicTimeStamp.length() > 0) {
-        MQTTPublish(topicTimeStamp, resulttimestamp);
-    }
-
     OldValue = result;
     
     return true;

code/components/jomjol_flowcontroll/ClassFlowMQTT.h

@@ -9,10 +9,13 @@ class ClassFlowMQTT :
     public ClassFlow
 {
 protected:
-    std::string uri, topic, topicError, clientname, topicRate, topicTimeStamp;
+    std::string uri, topic, topicError, clientname, topicRate, topicTimeStamp, topicUptime, topicFreeMem;
     std::string OldValue;
 	ClassFlowPostProcessing* flowpostprocessing;  
-    std::string user, password;  
+    std::string user, password; 
+    bool MQTTenable;
+
+    std::string maintopic, mainerrortopic; 
 	void SetInitialParameter(void);        
 
 public:
@@ -20,6 +23,8 @@ public:
     ClassFlowMQTT(std::vector<ClassFlow*>* lfc);
     ClassFlowMQTT(std::vector<ClassFlow*>* lfc, ClassFlow *_prev);
 
+    string GetMQTTMainTopic();
+
     bool ReadParameter(FILE* pfile, string& aktparamgraph);
     bool doFlow(string time);
     string name(){return "ClassFlowMQTT";};

code/components/jomjol_flowcontroll/ClassFlowPostProcessing.h

@@ -1,59 +1,68 @@
-#pragma once
+#ifndef __FLOWPOSTPROCESSING__
+#define __FLOWPOSTPROCESSING__
+
 #include "ClassFlow.h"
 #include "ClassFlowMakeImage.h"
+#include "ClassFlowCNNGeneral.h"
+#include "ClassFlowDefineTypes.h"
 
 #include <string>
 
-
-
-
 class ClassFlowPostProcessing :
     public ClassFlow
 {
 protected:
-    bool PreValueUse;
+    std::vector<NumberPost*> NUMBERS;
+    bool UpdatePreValueINI;
+
     int PreValueAgeStartup; 
-    bool AllowNegativeRates;
-    float MaxRateValue;
-    bool useMaxRateValue;
     bool ErrorMessage;
-    bool PreValueOkay;
-    bool checkDigitIncreaseConsistency;
-    int DecimalShift;
-    time_t lastvalue;
-    float FlowRateAct;          // m3 / min
+    bool IgnoreLeadingNaN;          // SPEZIALFALL für User Gustl
+
+
+    ClassFlowCNNGeneral* flowAnalog;
+    ClassFlowCNNGeneral* flowDigit;    
 
 
     string FilePreValue;
-    float PreValue;             // letzter Wert, der gut ausgelesen wurde
-    float Value;                // letzer ausgelesener Wert, inkl. Korrekturen
-    string ReturnRawValue;      // Rohwert (mit N & führenden 0)    
-    string ReturnValue;         // korrigierter Rückgabewert, ggf. mit Fehlermeldung
-    string ReturnValueNoError;  // korrigierter Rückgabewert ohne Fehlermeldung
-    string ErrorMessageText;        // Fehlermeldung bei Consistency Check
-    string timeStamp;
 
     ClassFlowMakeImage *flowMakeImage;
 
     bool LoadPreValue(void);
     string ShiftDecimal(string in, int _decShift);
 
-    string ErsetzteN(string);
-    float checkDigitConsistency(float input, int _decilamshift, bool _isanalog);
+    string ErsetzteN(string, float _prevalue);
+    float checkDigitConsistency(float input, int _decilamshift, bool _isanalog, float _preValue);
     string RundeOutput(float _in, int _anzNachkomma);
 
+    void InitNUMBERS();
+    void handleDecimalSeparator(string _decsep, string _value);
+    void handleMaxRateValue(string _decsep, string _value);
+    void handleDecimalExtendedResolution(string _decsep, string _value);    
+
+
+
 public:
-    ClassFlowPostProcessing(std::vector<ClassFlow*>* lfc);
+    bool PreValueUse;
+
+    ClassFlowPostProcessing(std::vector<ClassFlow*>* lfc, ClassFlowCNNGeneral *_analog, ClassFlowCNNGeneral *_digit);
     bool ReadParameter(FILE* pfile, string& aktparamgraph);
     bool doFlow(string time);
-    string getReadout();
-    string getReadoutParam(bool _rawValue, bool _noerror);
-    string getReadoutError();
-    string getReadoutRate();
-    string getReadoutTimeStamp();
-    void SavePreValue(float value, string time = "");
-    string GetPreValue();
+    string getReadout(int _number);
+    string getReadoutParam(bool _rawValue, bool _noerror, int _number = 0);
+    string getReadoutError(int _number = 0);
+    string getReadoutRate(int _number = 0);
+    string getReadoutTimeStamp(int _number = 0);
+    void SavePreValue();
+    string GetPreValue(std::string _number = "");
+    void SetPreValue(float zw, string _numbers, bool _extern = false);
+
+    void UpdateNachkommaDecimalShift();
+
+    std::vector<NumberPost*>* GetNumbers(){return &NUMBERS;};
 
     string name(){return "ClassFlowPostProcessing";};
 };
 
+
+#endif

code/components/jomjol_helper/CMakeLists.txt

@@ -2,6 +2,6 @@ FILE(GLOB_RECURSE app_sources ${CMAKE_CURRENT_SOURCE_DIR}/*.*)
 
 idf_component_register(SRCS ${app_sources}
                     INCLUDE_DIRS "."
-                    REQUIRES tfmicro jomjol_logfile)
+                    REQUIRES tflite-lib jomjol_logfile)
 
 

code/components/jomjol_helper/Helper.cpp

@@ -6,10 +6,19 @@
 #include "Helper.h"
 #include <sys/types.h>
 #include <sys/stat.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
 #include <dirent.h>
+#ifdef __cplusplus
+}
+#endif
+
 #include <string.h>
 #include <esp_log.h>
 
+
 #include "ClassLogFile.h"
 //#include "ClassLogFile.h"
 
@@ -77,21 +86,25 @@ void memCopyGen(uint8_t* _source, uint8_t* _target, int _size)
 
 
 
-FILE* OpenFileAndWait(const char* nm, char* _mode, int _waitsec)
+FILE* OpenFileAndWait(const char* nm, const char* _mode, int _waitsec)
 {
+	printf("open file %s in mode %s\n", nm, _mode);
 	FILE *pfile = fopen(nm, _mode);
 
+/*
 	if (pfile == NULL)
 	{
 		TickType_t xDelay;
 		xDelay = _waitsec * 1000 / portTICK_PERIOD_MS;
-		std::string zw = "File is locked: " + std::string(nm) + " - wait for " + std::to_string(_waitsec);
+		std::string zw = "File is locked: " + std::string(nm) + " - wait for " + std::to_string(_waitsec) + " seconds";
 	    printf(zw.c_str());
 		printf("\n");
 		LogFile.WriteToFile(zw);      
 		vTaskDelay( xDelay );
 		pfile = fopen(nm, _mode);
 	}
+*/
+
 	return pfile;
 }
 
@@ -313,6 +326,14 @@ string toUpper(string in)
 	return in;
 }
 
+string toLower(string in)
+{
+	for (int i = 0; i < in.length(); ++i)
+		in[i] = tolower(in[i]);
+	
+	return in;
+}
+
 // CPU Temp
 extern "C" uint8_t temprature_sens_read();
 float temperatureRead()
@@ -358,3 +379,30 @@ int removeFolder(const char* folderPath, const char* logTag) {
 
 	return deleted;
 }
+
+
+
+std::vector<string> HelperZerlegeZeile(std::string input, std::string _delimiter = "")
+{
+	std::vector<string> Output;
+	std::string delimiter = " =,";
+    if (_delimiter.length() > 0){
+        delimiter = _delimiter;
+    }
+
+	input = trim(input, delimiter);
+	size_t pos = findDelimiterPos(input, delimiter);
+	std::string token;
+	while (pos != std::string::npos) {
+		token = input.substr(0, pos);
+		token = trim(token, delimiter);
+		Output.push_back(token);
+		input.erase(0, pos + 1);
+		input = trim(input, delimiter);
+		pos = findDelimiterPos(input, delimiter);
+	}
+	Output.push_back(input);
+
+	return Output;
+}
+

code/components/jomjol_helper/Helper.h

@@ -1,6 +1,7 @@
 #pragma once
 #include <string>
 #include <fstream>
+#include <vector>
 
 
 using namespace std;
@@ -10,7 +11,7 @@ void FindReplace(std::string& line, std::string& oldString, std::string& newStri
 
 void CopyFile(string input, string output);
 
-FILE* OpenFileAndWait(const char* nm, char* _mode, int _waitsec = 1);
+FILE* OpenFileAndWait(const char* nm, const char* _mode, int _waitsec = 1);
 
 size_t findDelimiterPos(string input, string delimiter);
 //string trim(string istring);
@@ -22,6 +23,7 @@ string getFileType(string filename);
 int mkdir_r(const char *dir, const mode_t mode);
 int removeFolder(const char* folderPath, const char* logTag);
 
+string toLower(string in);
 string toUpper(string in);
 
 float temperatureRead();
@@ -30,6 +32,8 @@ time_t addDays(time_t startTime, int days);
 
 void memCopyGen(uint8_t* _source, uint8_t* _target, int _size);
 
+std::vector<string> HelperZerlegeZeile(std::string input, std::string _delimiter);
+
 ///////////////////////////
 size_t getInternalESPHeapSize();
 size_t getESPHeapSize();

code/components/jomjol_image_proc/CImageBasis.cpp

@@ -1,6 +1,7 @@
 #include "CImageBasis.h"
 #include "Helper.h"
 #include "ClassLogFile.h"
+#include "server_ota.h"
 
 #include <esp_log.h>
 
@@ -337,6 +338,18 @@ void CImageBasis::LoadFromMemory(stbi_uc *_buffer, int len)
     rgb_image = stbi_load_from_memory(_buffer, len, &width, &height, &channels, 3);
     bpp = channels;
     printf("Image loaded from memory: %d, %d, %d\n", width, height, channels);
+    if ((width * height * channels) == 0)
+    {
+        ESP_LOGE(TAG, "Image with size 0 loaded --> reboot to be done! "
+            "Check that your camera module is working and connected properly.");
+
+        LogFile.SwitchOnOff(true);
+        LogFile.WriteToFile("Image with size 0 loaded --> reboot to be done! "
+                "Check that your camera module is working and connected properly.");
+
+        doReboot();
+
+    }
     RGBImageRelease();
 }
 
@@ -354,12 +367,10 @@ CImageBasis::CImageBasis(CImageBasis *_copyfrom, int _anzrepeat)
     int memsize = width * height * channels;
     rgb_image = (unsigned char*)GET_MEMORY(memsize);
 
-    TickType_t xDelay;
     int anz = 1;
     while (!rgb_image && (anz < _anzrepeat))    
     {
-		    printf("Create Image from Copy - Speicher ist voll - Versuche es erneut: %d.\n", anz);
-        xDelay = 1000 / portTICK_PERIOD_MS;
+	    printf("Create Image from Copy - Speicher ist voll - Versuche es erneut: %d.\n", anz);
         rgb_image = (unsigned char*) malloc(memsize);
         anz++;
     }

code/components/jomjol_image_proc/CMakeLists.txt

@@ -2,6 +2,6 @@ FILE(GLOB_RECURSE app_sources ${CMAKE_CURRENT_SOURCE_DIR}/*.*)
 
 idf_component_register(SRCS ${app_sources}
                     INCLUDE_DIRS "."
-                    REQUIRES jomjol_helper jomjol_logfile esp_http_server)
+                    REQUIRES jomjol_helper jomjol_logfile esp_http_server jomjol_fileserver_ota) 
 
 

code/components/jomjol_logfile/ClassLogFile.cpp

@@ -3,7 +3,15 @@
 #include <string.h>
 #include <sys/types.h>
 #include <sys/stat.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
 #include <dirent.h>
+#ifdef __cplusplus
+}
+#endif
+
 #include "Helper.h"
 
 static const char *TAG = "log";
@@ -12,7 +20,7 @@ ClassLogFile LogFile("/sdcard/log/message", "log_%Y-%m-%d.txt");
 
 void ClassLogFile::WriteHeapInfo(std::string _id)
 {
-std::string _zw = "\t" + _id;
+    std::string _zw = "\t" + _id;
     if (loglevel > 0)
         _zw =  _zw + "\t" + getESPHeapInfo();
 

code/components/jomjol_mqtt/CMakeLists.txt

@@ -2,6 +2,6 @@ FILE(GLOB_RECURSE app_sources ${CMAKE_CURRENT_SOURCE_DIR}/*.*)
 
 idf_component_register(SRCS ${app_sources}
                     INCLUDE_DIRS "."
-                    REQUIRES tfmicro mqtt jomjol_logfile)
+                    REQUIRES tflite-lib mqtt jomjol_logfile)
 
 

code/components/jomjol_mqtt/interface_mqtt.cpp

@@ -1,12 +1,14 @@
 #include "interface_mqtt.h"
 
-
+//#define LOG_LOCAL_LEVEL ESP_LOG_DEBUG
 #include "esp_log.h"
 #include "mqtt_client.h"
 #include "ClassLogFile.h"
 
-static const char *TAG = "interface_mqtt";
+static const char *TAG_INTERFACEMQTT = "interface_mqtt";
 
+std::map<std::string, std::function<void()>>* connectFunktionMap = NULL;  
+std::map<std::string, std::function<bool(std::string, char*, int)>>* subscribeFunktionMap = NULL;  
 bool debugdetail = true;
 
 // #define CONFIG_BROKER_URL "mqtt://192.168.178.43:1883"
@@ -23,44 +25,67 @@ void MQTTPublish(std::string _key, std::string _content, int retained_flag){
         msg_id = esp_mqtt_client_publish(client, _key.c_str(), _content.c_str(), 0, 1, retained_flag);
         zw = "sent publish successful in MQTTPublish, msg_id=" + std::to_string(msg_id) + ", " + _key + ", " + _content;
         if (debugdetail) LogFile.WriteToFile(zw);
-        ESP_LOGI(TAG, "sent publish successful in MQTTPublish, msg_id=%d, %s, %s", msg_id, _key.c_str(), _content.c_str());
+        ESP_LOGD(TAG_INTERFACEMQTT, "sent publish successful in MQTTPublish, msg_id=%d, %s, %s", msg_id, _key.c_str(), _content.c_str());
     }
     else {
-        ESP_LOGI(TAG, "Problem with Publish, client=%d, mqtt_connected %d", (int) client, (int) mqtt_connected);
+        ESP_LOGW(TAG_INTERFACEMQTT, "Problem with Publish, client=%d, mqtt_connected %d", (int) client, (int) mqtt_connected);
     }
 }
 
 
 static esp_err_t mqtt_event_handler_cb(esp_mqtt_event_handle_t event)
 {
+    int msg_id;
+    std::string topic = "";
     switch (event->event_id) {
+        case MQTT_EVENT_BEFORE_CONNECT:
+            ESP_LOGI(TAG_INTERFACEMQTT, "MQTT_EVENT_BEFORE_CONNECT");
+            break;
         case MQTT_EVENT_CONNECTED:
-            ESP_LOGI(TAG, "MQTT_EVENT_CONNECTED");
+            ESP_LOGI(TAG_INTERFACEMQTT, "MQTT_EVENT_CONNECTED");
             mqtt_connected = true;
+            MQTTconnected();
             break;
         case MQTT_EVENT_DISCONNECTED:
-            ESP_LOGI(TAG, "MQTT_EVENT_DISCONNECTED");
+            ESP_LOGI(TAG_INTERFACEMQTT, "MQTT_EVENT_DISCONNECTED");
+            break;
+        case MQTT_EVENT_SUBSCRIBED:
+            ESP_LOGI(TAG_INTERFACEMQTT, "MQTT_EVENT_SUBSCRIBED, msg_id=%d", event->msg_id);
+            msg_id = esp_mqtt_client_publish(client, "/topic/qos0", "data", 0, 0, 0);
+            ESP_LOGI(TAG_INTERFACEMQTT, "sent publish successful, msg_id=%d", msg_id);
+            break;
+        case MQTT_EVENT_UNSUBSCRIBED:
+            ESP_LOGI(TAG_INTERFACEMQTT, "MQTT_EVENT_UNSUBSCRIBED, msg_id=%d", event->msg_id);
             break;
         case MQTT_EVENT_PUBLISHED:
-            ESP_LOGI(TAG, "MQTT_EVENT_PUBLISHED, msg_id=%d", event->msg_id);
+            ESP_LOGI(TAG_INTERFACEMQTT, "MQTT_EVENT_PUBLISHED, msg_id=%d", event->msg_id);
             break;
         case MQTT_EVENT_DATA:
-            ESP_LOGI(TAG, "MQTT_EVENT_DATA");
-            printf("TOPIC=%.*s\r\n", event->topic_len, event->topic);
-            printf("DATA=%.*s\r\n", event->data_len, event->data);
+            ESP_LOGI(TAG_INTERFACEMQTT, "MQTT_EVENT_DATA");
+            ESP_LOGI(TAG_INTERFACEMQTT, "TOPIC=%.*s\r\n", event->topic_len, event->topic);
+            ESP_LOGI(TAG_INTERFACEMQTT, "DATA=%.*s\r\n", event->data_len, event->data);
+            topic.assign(event->topic, event->topic_len);
+            if (subscribeFunktionMap != NULL) {
+                if (subscribeFunktionMap->find(topic) != subscribeFunktionMap->end()) {
+                    ESP_LOGD(TAG_INTERFACEMQTT, "call handler function\r\n");
+                    (*subscribeFunktionMap)[topic](topic, event->data, event->data_len);
+                }
+            } else {
+                ESP_LOGW(TAG_INTERFACEMQTT, "no handler available\r\n");
+            }
             break;
         case MQTT_EVENT_ERROR:
-            ESP_LOGI(TAG, "MQTT_EVENT_ERROR");
+            ESP_LOGI(TAG_INTERFACEMQTT, "MQTT_EVENT_ERROR");
             break;
         default:
-            ESP_LOGI(TAG, "Other event id:%d", event->event_id);
+            ESP_LOGI(TAG_INTERFACEMQTT, "Other event id:%d", event->event_id);
             break;
     }
     return ESP_OK;
 }
 
 static void mqtt_event_handler(void *handler_args, esp_event_base_t base, int32_t event_id, void *event_data) {
-    ESP_LOGD(TAG, "Event dispatched from event loop base=%s, event_id=%d", base, event_id);
+    ESP_LOGD(TAG_INTERFACEMQTT, "Event dispatched from event loop base=%s, event_id=%d", base, event_id);
     mqtt_event_handler_cb((esp_mqtt_event_handle_t) event_data);
 }
 
@@ -82,7 +107,7 @@ void MQTTInit(std::string _mqttURI, std::string _clientid, std::string _user, st
     if (_user.length() && _password.length()){
         mqtt_cfg.username = _user.c_str();
         mqtt_cfg.password = _password.c_str();
-        printf("Connect to MQTT: %s, %s", mqtt_cfg.username, mqtt_cfg.password);
+        ESP_LOGI(TAG_INTERFACEMQTT, "Connect to MQTT: %s, %s", mqtt_cfg.username, mqtt_cfg.password);
     };
 
     client = esp_mqtt_client_init(&mqtt_cfg);
@@ -91,3 +116,91 @@ void MQTTInit(std::string _mqttURI, std::string _clientid, std::string _user, st
 
     MQTTPublish(_LWTContext, "", 1);
 }
+
+void MQTTdestroy() {
+    if (client != NULL) {
+        esp_mqtt_client_stop(client);
+        esp_mqtt_client_destroy(client);
+    }
+}
+
+bool MQTTisConnected() {
+    return mqtt_connected;
+}
+
+void MQTTregisterConnectFunction(std::string name, std::function<void()> func){
+    ESP_LOGD(TAG_INTERFACEMQTT, "MQTTregisteronnectFunction %s\r\n", name.c_str());
+    if (connectFunktionMap == NULL) {
+        connectFunktionMap = new std::map<std::string, std::function<void()>>();
+    }
+
+    if ((*connectFunktionMap)[name] != NULL) {
+        ESP_LOGW(TAG_INTERFACEMQTT, "connect function %s already registred", name.c_str());
+        return;
+    }
+
+    (*connectFunktionMap)[name] = func;
+
+    if (mqtt_connected) {
+        func();
+    }
+}
+
+void MQTTunregisterConnectFunction(std::string name){
+    ESP_LOGD(TAG_INTERFACEMQTT, "MQTTregisteronnectFunction %s\r\n", name.c_str());
+    if ((connectFunktionMap != NULL) && (connectFunktionMap->find(name) != connectFunktionMap->end())) {
+        connectFunktionMap->erase(name);
+    }
+}
+
+void MQTTregisterSubscribeFunction(std::string topic, std::function<bool(std::string, char*, int)> func){
+    ESP_LOGD(TAG_INTERFACEMQTT, "MQTTregisterSubscribeFunction %s\r\n", topic.c_str());
+    if (subscribeFunktionMap == NULL) {
+        subscribeFunktionMap = new std::map<std::string, std::function<bool(std::string, char*, int)>>();
+    }
+
+    if ((*subscribeFunktionMap)[topic] != NULL) {
+        ESP_LOGW(TAG_INTERFACEMQTT, "topic %s already registred for subscription", topic.c_str());
+        return;
+    }
+
+    (*subscribeFunktionMap)[topic] = func;
+
+    if (mqtt_connected) {
+        int msg_id = esp_mqtt_client_subscribe(client, topic.c_str(), 0);
+        ESP_LOGD(TAG_INTERFACEMQTT, "topic %s subscribe successful, msg_id=%d", topic.c_str(), msg_id);
+    }
+}
+
+void MQTTconnected(){
+    if (mqtt_connected) {
+        if (connectFunktionMap != NULL) {
+            for(std::map<std::string, std::function<void()>>::iterator it = connectFunktionMap->begin(); it != connectFunktionMap->end(); ++it) {
+                it->second();
+                ESP_LOGD(TAG_INTERFACEMQTT, "call connect function %s", it->first.c_str());
+            }
+        }
+
+        if (subscribeFunktionMap != NULL) {
+            for(std::map<std::string, std::function<bool(std::string, char*, int)>>::iterator it = subscribeFunktionMap->begin(); it != subscribeFunktionMap->end(); ++it) {
+                int msg_id = esp_mqtt_client_subscribe(client, it->first.c_str(), 0);
+                ESP_LOGD(TAG_INTERFACEMQTT, "topic %s subscribe successful, msg_id=%d", it->first.c_str(), msg_id);
+            }
+        }
+    }
+}
+
+void MQTTdestroySubscribeFunction(){
+    if (subscribeFunktionMap != NULL) {
+        if (mqtt_connected) {
+            for(std::map<std::string, std::function<bool(std::string, char*, int)>>::iterator it = subscribeFunktionMap->begin(); it != subscribeFunktionMap->end(); ++it) {
+                int msg_id = esp_mqtt_client_unsubscribe(client, it->first.c_str());
+                ESP_LOGI(TAG_INTERFACEMQTT, "topic %s unsubscribe successful, msg_id=%d", it->first.c_str(), msg_id);
+            }
+        }
+
+        subscribeFunktionMap->clear();
+        delete subscribeFunktionMap;
+        subscribeFunktionMap = NULL;
+    }
+}

code/components/jomjol_mqtt/interface_mqtt.h

@@ -1,7 +1,23 @@
+#ifndef INTERFACE_MQTT_H
+#define INTERFACE_MQTT_H
+
 #include <string>
+#include <map>
+#include <functional>
 
 void MQTTInit(std::string _mqttURI, std::string _clientid, std::string _user, std::string _password, std::string _LWTContext, int _keepalive);
+void MQTTdestroy();
 
 //void MQTTInit(std::string _mqttURI, std::string _clientid, std::string _user = "", std::string _password = "");
 
-void MQTTPublish(std::string _key, std::string _content, int retained_flag = 0);
+void MQTTPublish(std::string _key, std::string _content, int retained_flag = 0);
+
+bool MQTTisConnected();
+
+void MQTTregisterConnectFunction(std::string name, std::function<void()> func);
+void MQTTunregisterConnectFunction(std::string name);
+void MQTTregisterSubscribeFunction(std::string topic, std::function<bool(std::string, char*, int)> func);
+void MQTTdestroySubscribeFunction();
+void MQTTconnected();
+
+#endif //INTERFACE_MQTT_H

code/components/jomjol_tfliteclass/CTfLiteClass.cpp

@@ -17,6 +17,7 @@ float CTfLiteClass::GetOutputValue(int nr)
     return output2->data.f[nr];
 }
 
+
 int CTfLiteClass::GetClassFromImageBasis(CImageBasis *rs)
 {
     if (!LoadInputImageBasis(rs))
@@ -27,19 +28,36 @@ int CTfLiteClass::GetClassFromImageBasis(CImageBasis *rs)
     return GetOutClassification();
 }
 
-int CTfLiteClass::GetOutClassification()
+
+int CTfLiteClass::GetOutClassification(int _von, int _bis)
 {
   TfLiteTensor* output2 = interpreter->output(0);
 
-  float zw_max = 0;
+  float zw_max;
   float zw;
-  int zw_class = -1;
+  int zw_class;
 
   if (output2 == NULL)
     return -1;
 
   int numeroutput = output2->dims->data[1];
-  for (int i = 0; i < numeroutput; ++i)
+  //printf("\n number output neurons: %d\n\n", numeroutput);
+
+  if (_bis == -1)
+    _bis = numeroutput -1;
+
+  if (_von == -1)
+    _von = 0;
+
+  if (_bis >= numeroutput)
+  {
+    printf("ANZAHL OUTPUT NEURONS passt nicht zu geforderter Classifizierung!");
+    return -1;
+  }
+
+  zw_max = output2->data.f[_von];
+  zw_class = _von;
+  for (int i = _von + 1; i <= _bis; ++i)
   {
     zw = output2->data.f[i];
     if (zw > zw_max)
@@ -48,7 +66,7 @@ int CTfLiteClass::GetOutClassification()
         zw_class = i;
     }
   }
-  return zw_class;
+  return (zw_class - _von);
 }
 
 void CTfLiteClass::GetInputDimension(bool silent = false)
@@ -70,18 +88,18 @@ void CTfLiteClass::GetInputDimension(bool silent = false)
 }
 
 
-void CTfLiteClass::GetOutPut()
+int CTfLiteClass::GetAnzOutPut(bool silent)
 {
   TfLiteTensor* output2 = this->interpreter->output(0);
 
   int numdim = output2->dims->size;
-  printf("NumDimension: %d\n", numdim);  
+  if (!silent) printf("NumDimension: %d\n", numdim);  
 
   int sizeofdim;
   for (int j = 0; j < numdim; ++j)
   {
     sizeofdim = output2->dims->data[j];
-    printf("SizeOfDimension %d: %d\n", j, sizeofdim);  
+    if (!silent) printf("SizeOfDimension %d: %d\n", j, sizeofdim);  
   }
 
 
@@ -92,20 +110,22 @@ void CTfLiteClass::GetOutPut()
   for (int i = 0; i < numeroutput; ++i)
   {
    fo = output2->data.f[i];
-    printf("Result %d: %f\n", i, fo);  
+    if (!silent) printf("Result %d: %f\n", i, fo);  
   }
+  return numeroutput;
 }
 
 void CTfLiteClass::Invoke()
 {
-    interpreter->Invoke();
+    if (interpreter != nullptr)
+      interpreter->Invoke();
 }
 
 
 
 bool CTfLiteClass::LoadInputImageBasis(CImageBasis *rs)
 {
-    std::string zw = "ClassFlowAnalog::doNeuralNetwork nach LoadInputResizeImage: ";
+    std::string zw = "ClassFlowCNNGeneral::doNeuralNetwork nach LoadInputResizeImage: ";
 
     unsigned int w = rs->width;
     unsigned int h = rs->height;
@@ -148,6 +168,8 @@ void CTfLiteClass::MakeAllocate()
     TfLiteStatus allocate_status = this->interpreter->AllocateTensors();
     if (allocate_status != kTfLiteOk) {
         TF_LITE_REPORT_ERROR(error_reporter, "AllocateTensors() failed");
+        LogFile.WriteToFile("AllocateTensors() failed");
+
     this->GetInputDimension();   
     return;
   }
@@ -155,9 +177,9 @@ void CTfLiteClass::MakeAllocate()
 }
 
 void CTfLiteClass::GetInputTensorSize(){
+#ifdef DEBUG_DETAIL_ON    
     float *zw = this->input;
     int test = sizeof(zw);
-#ifdef DEBUG_DETAIL_ON    
     printf("Input Tensor Dimension: %d\n", test);       
 #endif
 }
@@ -184,13 +206,11 @@ unsigned char* CTfLiteClass::ReadFileToCharArray(std::string _fn)
 
     unsigned char *result = (unsigned char*) malloc(size);
     int anz = 1;
-    TickType_t xDelay;
     while (!result && (anz < 6))    // maximal 5x versuchen (= 5s)
     {
 #ifdef DEBUG_DETAIL_ON      
 		    printf("Speicher ist voll - Versuche es erneut: %d.\n", anz);
 #endif
-        xDelay = 1000 / portTICK_PERIOD_MS;
         result = (unsigned char*) malloc(size);
         anz++;
     }
@@ -208,7 +228,7 @@ unsigned char* CTfLiteClass::ReadFileToCharArray(std::string _fn)
     return result;
 }
 
-void CTfLiteClass::LoadModel(std::string _fn){
+bool CTfLiteClass::LoadModel(std::string _fn){
 
 #ifdef SUPRESS_TFLITE_ERRORS
     this->error_reporter = new tflite::OwnMicroErrorReporter;
@@ -216,12 +236,16 @@ void CTfLiteClass::LoadModel(std::string _fn){
     this->error_reporter = new tflite::MicroErrorReporter;
 #endif
 
-    unsigned char *rd;
-    rd = ReadFileToCharArray(_fn.c_str());
+    modelload = ReadFileToCharArray(_fn.c_str());
 
-    this->model = tflite::GetModel(rd);
-    free(rd);
+    if (modelload == NULL) 
+      return false;
+
+    model = tflite::GetModel(modelload);
+//    free(rd);
     TFLITE_MINIMAL_CHECK(model != nullptr); 
+    
+    return true;
 }
 
 
@@ -232,7 +256,7 @@ CTfLiteClass::CTfLiteClass()
     this->interpreter = nullptr;
     this->input = nullptr;
     this->output = nullptr;  
-    this->kTensorArenaSize = 200 * 1024;   /// laut testfile: 108000 - bisher 600
+    this->kTensorArenaSize = 800 * 1024;   /// laut testfile: 108000 - bisher 600;; 2021-09-11: 200 * 1024
     this->tensor_arena = new uint8_t[kTensorArenaSize]; 
 }
 
@@ -241,6 +265,8 @@ CTfLiteClass::~CTfLiteClass()
   delete this->tensor_arena;
   delete this->interpreter;
   delete this->error_reporter;
+  if (modelload)
+    free(modelload);
 }        
 
 

code/components/jomjol_tfliteclass/CTfLiteClass.h

@@ -46,6 +46,9 @@ class CTfLiteClass
         int kTensorArenaSize;
         uint8_t *tensor_arena;
 
+        unsigned char *modelload = NULL;
+
+
         float* input;
         int input_i;
         int im_height, im_width, im_channel;
@@ -56,14 +59,18 @@ class CTfLiteClass
     public:
         CTfLiteClass();
         ~CTfLiteClass();        
-        void LoadModel(std::string _fn);
+        bool LoadModel(std::string _fn);
         void MakeAllocate();
         void GetInputTensorSize();
         bool LoadInputImageBasis(CImageBasis *rs);
         void Invoke();
-        void GetOutPut();
-        int GetOutClassification();
+        int GetAnzOutPut(bool silent = true);        
+//        void GetOutPut();
+//        int GetOutClassification();
+        int GetOutClassification(int _von = -1, int _bis = -1);
+
         int GetClassFromImageBasis(CImageBasis *rs);
+        std::string GetStatusFlow();
 
         float GetOutputValue(int nr);
         void GetInputDimension(bool silent);

code/components/jomjol_tfliteclass/server_tflite.cpp

@@ -8,6 +8,7 @@
 #include <iomanip>
 #include <sstream>
 
+#include "defines.h"
 #include "Helper.h"
 
 #include "esp_camera.h"
@@ -17,8 +18,9 @@
 #include "ClassFlowControll.h"
 
 #include "ClassLogFile.h"
+#include "server_GPIO.h"
 
-//#define DEBUG_DETAIL_ON       
+// #define DEBUG_DETAIL_ON       
 
 
 ClassFlowControll tfliteflow;
@@ -26,9 +28,6 @@ ClassFlowControll tfliteflow;
 TaskHandle_t xHandleblink_task_doFlow = NULL;
 TaskHandle_t xHandletask_autodoFlow = NULL;
 
-
-
-
 bool flowisrunning = false;
 
 long auto_intervall = 0;
@@ -37,6 +36,9 @@ bool auto_isrunning = false;
 
 int countRounds = 0;
 
+static const char *TAGTFLITE = "server_tflite";
+
+
 int getCountFlowRounds() {
     return countRounds;
 }
@@ -64,9 +66,11 @@ void KillTFliteTasks()
 #ifdef DEBUG_DETAIL_ON          
     printf("Handle: xHandleblink_task_doFlow: %ld\n", (long) xHandleblink_task_doFlow);  
 #endif  
-    if (xHandleblink_task_doFlow)
+    if (xHandleblink_task_doFlow != NULL)
     {
-        vTaskDelete(xHandleblink_task_doFlow);
+        TaskHandle_t xHandleblink_task_doFlowTmp = xHandleblink_task_doFlow;
+        xHandleblink_task_doFlow = NULL;
+        vTaskDelete(xHandleblink_task_doFlowTmp);
 #ifdef DEBUG_DETAIL_ON      
         printf("Killed: xHandleblink_task_doFlow\n");
 #endif
@@ -75,9 +79,11 @@ void KillTFliteTasks()
 #ifdef DEBUG_DETAIL_ON      
     printf("Handle: xHandletask_autodoFlow: %ld\n", (long) xHandletask_autodoFlow);  
 #endif
-    if (xHandletask_autodoFlow)
+    if (xHandletask_autodoFlow != NULL)
     {
-        vTaskDelete(xHandletask_autodoFlow);
+        TaskHandle_t xHandletask_autodoFlowTmp = xHandletask_autodoFlow;
+        xHandletask_autodoFlow = NULL;
+        vTaskDelete(xHandletask_autodoFlowTmp);
 #ifdef DEBUG_DETAIL_ON      
         printf("Killed: xHandletask_autodoFlow\n");
 #endif
@@ -87,11 +93,10 @@ void KillTFliteTasks()
 
 void doInit(void)
 {
-    string config = "/sdcard/config/config.ini";
 #ifdef DEBUG_DETAIL_ON             
     printf("Start tfliteflow.InitFlow(config);\n");
 #endif
-    tfliteflow.InitFlow(config);
+    tfliteflow.InitFlow(CONFIG_FILE);
 #ifdef DEBUG_DETAIL_ON      
     printf("Finished tfliteflow.InitFlow(config);\n");
 #endif
@@ -136,7 +141,7 @@ esp_err_t handler_init(httpd_req_t *req)
     printf("handler_doinit uri:\n"); printf(req->uri); printf("\n");
 #endif
 
-    char* resp_str = "Init started<br>";
+    const char* resp_str = "Init started<br>";
     httpd_resp_send(req, resp_str, strlen(resp_str));     
 
     doInit();
@@ -159,8 +164,6 @@ esp_err_t handler_doflow(httpd_req_t *req)
     LogFile.WriteHeapInfo("handler_doflow - Start");       
 #endif
 
-    char* resp_str;
-
     printf("handler_doFlow uri: "); printf(req->uri); printf("\n");
 
     if (flowisrunning)
@@ -173,7 +176,7 @@ esp_err_t handler_doflow(httpd_req_t *req)
     {
         xTaskCreate(&blink_task_doFlow, "blink_doFlow", configMINIMAL_STACK_SIZE * 64, NULL, tskIDLE_PRIORITY+1, &xHandleblink_task_doFlow);
     }
-    resp_str = "doFlow gestartet - dauert ca. 60 Sekunden";
+    const char* resp_str = "doFlow gestartet - dauert ca. 60 Sekunden";
     httpd_resp_send(req, resp_str, strlen(resp_str));  
     /* Respond with an empty chunk to signal HTTP response completion */
     httpd_resp_send_chunk(req, NULL, 0);       
@@ -186,6 +189,36 @@ esp_err_t handler_doflow(httpd_req_t *req)
 };
 
 
+esp_err_t handler_json(httpd_req_t *req)
+{
+#ifdef DEBUG_DETAIL_ON       
+    LogFile.WriteHeapInfo("handler_json - Start");    
+#endif
+
+
+    printf("handler_JSON uri:\n"); printf(req->uri); printf("\n");
+
+    char _query[100];
+    char _size[10];
+
+    httpd_resp_set_hdr(req, "Access-Control-Allow-Origin", "*");
+    httpd_resp_set_type(req, "application/json");
+
+    std::string zw = tfliteflow.getJSON();
+    if (zw.length() > 0)
+        httpd_resp_sendstr_chunk(req, zw.c_str()); 
+
+    string query = std::string(_query);
+
+    /* Respond with an empty chunk to signal HTTP response completion */
+    httpd_resp_sendstr_chunk(req, NULL);   
+
+#ifdef DEBUG_DETAIL_ON       
+    LogFile.WriteHeapInfo("handler_JSON - Done");   
+#endif
+    return ESP_OK;
+};
+
 
 
 esp_err_t handler_wasserzaehler(httpd_req_t *req)
@@ -196,6 +229,8 @@ esp_err_t handler_wasserzaehler(httpd_req_t *req)
 
     bool _rawValue = false;
     bool _noerror = false;
+    bool _all = false;
+    std::string _type = "value";
     string zw;
 
     printf("handler_wasserzaehler uri:\n"); printf(req->uri); printf("\n");
@@ -206,6 +241,22 @@ esp_err_t handler_wasserzaehler(httpd_req_t *req)
     if (httpd_req_get_url_query_str(req, _query, 100) == ESP_OK)
     {
 //        printf("Query: "); printf(_query); printf("\n");
+        if (httpd_query_key_value(_query, "all", _size, 10) == ESP_OK)
+        {
+#ifdef DEBUG_DETAIL_ON       
+            printf("all is found"); printf(_size); printf("\n"); 
+#endif
+            _all = true;
+        }
+
+        if (httpd_query_key_value(_query, "type", _size, 10) == ESP_OK)
+        {
+#ifdef DEBUG_DETAIL_ON       
+            printf("all is found"); printf(_size); printf("\n"); 
+#endif
+            _type = std::string(_size);
+        }
+
         if (httpd_query_key_value(_query, "rawvalue", _size, 10) == ESP_OK)
         {
 #ifdef DEBUG_DETAIL_ON       
@@ -222,6 +273,29 @@ esp_err_t handler_wasserzaehler(httpd_req_t *req)
         }        
     }  
 
+    httpd_resp_set_hdr(req, "Access-Control-Allow-Origin", "*");
+
+    if (_all)
+    {
+        httpd_resp_set_type(req, "text/plain");
+        printf("TYPE: %s\n", _type.c_str());
+        int _intype = READOUT_TYPE_VALUE;
+        if (_type == "prevalue")
+            _intype = READOUT_TYPE_PREVALUE;
+        if (_type == "raw")
+            _intype = READOUT_TYPE_RAWVALUE;
+        if (_type == "error")
+            _intype = READOUT_TYPE_ERROR;
+
+
+        zw = tfliteflow.getReadoutAll(_intype);
+        printf("ZW: %s\n", zw.c_str());
+        if (zw.length() > 0)
+            httpd_resp_sendstr_chunk(req, zw.c_str()); 
+        httpd_resp_sendstr_chunk(req, NULL);   
+        return ESP_OK;
+    }
+
     zw = tfliteflow.getReadout(_rawValue, _noerror);
     if (zw.length() > 0)
         httpd_resp_sendstr_chunk(req, zw.c_str()); 
@@ -236,37 +310,48 @@ esp_err_t handler_wasserzaehler(httpd_req_t *req)
         txt = txt + "Digital Counter: <p> ";
         httpd_resp_sendstr_chunk(req, txt.c_str()); 
         
-        std::vector<HTMLInfo*> htmlinfo;
-        htmlinfo = tfliteflow.GetAllDigital();
-        for (int i = 0; i < htmlinfo.size(); ++i)
+        std::vector<HTMLInfo*> htmlinfodig;
+        htmlinfodig = tfliteflow.GetAllDigital();  
+        for (int i = 0; i < htmlinfodig.size(); ++i)
         {
-            if (htmlinfo[i]->val == 10)
-                zw = "NaN";
+            if (tfliteflow.GetTypeDigital() == Digital)
+            {
+                if (htmlinfodig[i]->val == 10)
+                    zw = "NaN";
+                else
+                    zw = to_string((int) htmlinfodig[i]->val);
+
+                txt = "<img src=\"/img_tmp/" +  htmlinfodig[i]->filename + "\"> " + zw;
+            }
             else
             {
-                zw = to_string((int) htmlinfo[i]->val);
+                std::stringstream stream;
+                stream << std::fixed << std::setprecision(1) << htmlinfodig[i]->val;
+                zw = stream.str();
+
+                txt = "<img src=\"/img_tmp/" +  htmlinfodig[i]->filename + "\"> " + zw;
             }
-            txt = "<img src=\"/img_tmp/" +  htmlinfo[i]->filename + "\"> " + zw;
             httpd_resp_sendstr_chunk(req, txt.c_str()); 
-            delete htmlinfo[i];
+            delete htmlinfodig[i];
         }
-        htmlinfo.clear();
+        htmlinfodig.clear();
       
         txt = " <p> Analog Meter: <p> ";
         httpd_resp_sendstr_chunk(req, txt.c_str()); 
         
-        htmlinfo = tfliteflow.GetAllAnalog();
-        for (int i = 0; i < htmlinfo.size(); ++i)
+        std::vector<HTMLInfo*> htmlinfoana;
+        htmlinfoana = tfliteflow.GetAllAnalog();
+        for (int i = 0; i < htmlinfoana.size(); ++i)
         {
             std::stringstream stream;
-            stream << std::fixed << std::setprecision(1) << htmlinfo[i]->val;
+            stream << std::fixed << std::setprecision(1) << htmlinfoana[i]->val;
             zw = stream.str();
 
-            txt = "<img src=\"/img_tmp/" +  htmlinfo[i]->filename + "\"> " + zw;
+            txt = "<img src=\"/img_tmp/" +  htmlinfoana[i]->filename + "\"> " + zw;
             httpd_resp_sendstr_chunk(req, txt.c_str()); 
-            delete htmlinfo[i];
+            delete htmlinfoana[i];
         }
-        htmlinfo.clear();   
+        htmlinfoana.clear();   
 
     }   
 
@@ -429,7 +514,7 @@ esp_err_t handler_editflow(httpd_req_t *req)
 
 //        printf("Parameter host: "); printf(_host.c_str()); printf("\n"); 
 //        string zwzw = "Do " + _task + " start\n"; printf(zwzw.c_str());
-        bool changed = Camera.SetBrightnessContrastSaturation(bri, con, sat);
+        Camera.SetBrightnessContrastSaturation(bri, con, sat);
         std::string zw = tfliteflow.doSingleStep("[MakeImage]", _host);
         httpd_resp_sendstr_chunk(req, zw.c_str()); 
     } 
@@ -446,31 +531,7 @@ esp_err_t handler_editflow(httpd_req_t *req)
 //        string zwzw = "Do " + _task + " start\n"; printf(zwzw.c_str());
         std::string zw = tfliteflow.doSingleStep("[Alignment]", _host);
         httpd_resp_sendstr_chunk(req, zw.c_str()); 
-    }  
-    if (_task.compare("test_analog") == 0)
-    {
-        std::string _host = "";
-        if (httpd_query_key_value(_query, "host", _valuechar, 30) == ESP_OK) {
-            _host = std::string(_valuechar);
-        }
-//        printf("Parameter host: "); printf(_host.c_str()); printf("\n"); 
-//        string zwzw = "Do " + _task + " start\n"; printf(zwzw.c_str());
-        std::string zw = tfliteflow.doSingleStep("[Analog]", _host);
-        httpd_resp_sendstr_chunk(req, zw.c_str()); 
-    }  
-    if (_task.compare("test_digits") == 0)
-    {
-        std::string _host = "";
-        if (httpd_query_key_value(_query, "host", _valuechar, 30) == ESP_OK) {
-            _host = std::string(_valuechar);
-        }
-//        printf("Parameter host: "); printf(_host.c_str()); printf("\n"); 
-
-//        string zwzw = "Do " + _task + " start\n"; printf(zwzw.c_str());
-        std::string zw = tfliteflow.doSingleStep("[Digits]", _host);
-        httpd_resp_sendstr_chunk(req, zw.c_str()); 
-    } 
-
+    }
 
     /* Respond with an empty chunk to signal HTTP response completion */
     httpd_resp_sendstr_chunk(req, NULL);   
@@ -483,6 +544,34 @@ esp_err_t handler_editflow(httpd_req_t *req)
 };
 
 
+esp_err_t handler_statusflow(httpd_req_t *req)
+{
+#ifdef DEBUG_DETAIL_ON       
+    LogFile.WriteHeapInfo("handler_prevalue - Start");       
+#endif
+
+    const char* resp_str;
+
+#ifdef DEBUG_DETAIL_ON       
+    printf("handler_prevalue:\n"); printf(req->uri); printf("\n");
+#endif
+
+    string* zw = tfliteflow.getActStatus();
+    resp_str = zw->c_str();
+
+    httpd_resp_set_hdr(req, "Access-Control-Allow-Origin", "*");
+    httpd_resp_send(req, resp_str, strlen(resp_str));   
+    /* Respond with an empty chunk to signal HTTP response completion */
+    httpd_resp_send_chunk(req, NULL, 0);      
+
+#ifdef DEBUG_DETAIL_ON       
+    LogFile.WriteHeapInfo("handler_prevalue - Start");       
+#endif
+
+    return ESP_OK;
+};
+
+
 esp_err_t handler_prevalue(httpd_req_t *req)
 {
 #ifdef DEBUG_DETAIL_ON       
@@ -498,6 +587,7 @@ esp_err_t handler_prevalue(httpd_req_t *req)
 
     char _query[100];
     char _size[10] = "";
+    char _numbers[50] = "default";
 
     if (httpd_req_get_url_query_str(req, _query, 100) == ESP_OK)
     {
@@ -511,15 +601,24 @@ esp_err_t handler_prevalue(httpd_req_t *req)
             printf("Value: "); printf(_size); printf("\n"); 
 #endif
         }
+
+        httpd_query_key_value(_query, "numbers", _numbers, 50);
     }      
 
     if (strlen(_size) == 0)
-        zw = tfliteflow.GetPrevalue();
+    {
+        zw = tfliteflow.GetPrevalue(std::string(_numbers));
+    }
     else
-        zw = "SetPrevalue to " + tfliteflow.UpdatePrevalue(_size);
+    {
+        zw = "SetPrevalue to " + tfliteflow.UpdatePrevalue(_size, _numbers, true);
+    }
     
     resp_str = zw.c_str();
 
+    httpd_resp_set_hdr(req, "Access-Control-Allow-Origin", "*");
+
+
     httpd_resp_send(req, resp_str, strlen(resp_str));   
     /* Respond with an empty chunk to signal HTTP response completion */
     httpd_resp_send_chunk(req, NULL, 0);      
@@ -535,17 +634,17 @@ void task_autodoFlow(void *pvParameter)
 {
     int64_t fr_start, fr_delta_ms;
 
+    printf("task_autodoFlow: start\r\n");
     doInit();
-    
-    auto_isrunning = tfliteflow.isAutoStart(auto_intervall);
+    gpio_handler_init();
 
+    auto_isrunning = tfliteflow.isAutoStart(auto_intervall);
     if (isSetupModusActive()) {
         auto_isrunning = false;
         std::string zw_time = gettimestring(LOGFILE_TIME_FORMAT);
         tfliteflow.doFlowMakeImageOnly(zw_time);
 
     }
-
     while (auto_isrunning)
     {
         std::string _zw = "task_autodoFlow - next round - Round #" + std::to_string(++countRounds);
@@ -590,11 +689,33 @@ void task_autodoFlow(void *pvParameter)
     }
     vTaskDelete(NULL); //Delete this task if it exits from the loop above
     xHandletask_autodoFlow = NULL;
+    printf("task_autodoFlow: end\r\n");
 }
 
 void TFliteDoAutoStart()
 {
-    xTaskCreate(&task_autodoFlow, "task_autodoFlow", configMINIMAL_STACK_SIZE * 64, NULL, tskIDLE_PRIORITY+1, &xHandletask_autodoFlow);
+    BaseType_t xReturned;
+
+    int _i = configMINIMAL_STACK_SIZE;
+
+    printf("task_autodoFlow configMINIMAL_STACK_SIZE: %d\n", _i);
+    printf("getESPHeapInfo: %s\n", getESPHeapInfo().c_str());
+
+    xReturned = xTaskCreate(&task_autodoFlow, "task_autodoFlow", configMINIMAL_STACK_SIZE * 60, NULL, tskIDLE_PRIORITY+1, &xHandletask_autodoFlow);
+    if( xReturned != pdPASS )
+    {
+
+       //Memory: 64 --> 48 --> 35 --> 25
+       printf("ERROR task_autodoFlow konnte nicht erzeugt werden !!\r\n");
+    }
+    printf("getESPHeapInfo: %s\n", getESPHeapInfo().c_str());
+
+
+}
+
+std::string GetMQTTMainTopic()
+{
+    return tfliteflow.GetMQTTMainTopic();
 }
 
 
@@ -621,6 +742,10 @@ void register_server_tflite_uri(httpd_handle_t server)
     camuri.user_ctx  = (void*) "Light Off"; 
     httpd_register_uri_handler(server, &camuri);  
 
+    camuri.uri       = "/statusflow.html";
+    camuri.handler   = handler_statusflow;
+    camuri.user_ctx  = (void*) "Light Off"; 
+    httpd_register_uri_handler(server, &camuri);  
     
     camuri.uri       = "/editflow.html";
     camuri.handler   = handler_editflow;
@@ -631,4 +756,10 @@ void register_server_tflite_uri(httpd_handle_t server)
     camuri.handler   = handler_wasserzaehler;
     camuri.user_ctx  = (void*) "Wasserzaehler"; 
     httpd_register_uri_handler(server, &camuri);  
+
+    camuri.uri       = "/json";
+    camuri.handler   = handler_json;
+    camuri.user_ctx  = (void*) "JSON"; 
+    httpd_register_uri_handler(server, &camuri);     
+
 }

code/components/jomjol_tfliteclass/server_tflite.h

@@ -1,12 +1,11 @@
 #include <esp_log.h>
+#include <string>
 
 #include <esp_http_server.h>
 #include "CImageBasis.h"
 
 //#include "ClassControllCamera.h"
 
-static const char *TAGTFLITE = "server_tflite";
-
 void register_server_tflite_uri(httpd_handle_t server);
 
 void KillTFliteTasks();
@@ -15,6 +14,8 @@ void TFliteDoAutoStart();
 
 bool isSetupModusActive();
 
+std::string GetMQTTMainTopic();
+
 esp_err_t GetJPG(std::string _filename, httpd_req_t *req);
 
 esp_err_t GetRawJPG(httpd_req_t *req);

code/components/jomjol_time_sntp/CMakeLists.txt

@@ -2,6 +2,6 @@ FILE(GLOB_RECURSE app_sources ${CMAKE_CURRENT_SOURCE_DIR}/*.*)
 
 idf_component_register(SRCS ${app_sources}
                     INCLUDE_DIRS "."
-                    REQUIRES tfmicro jomjol_logfile)
+                    REQUIRES tflite-lib jomjol_logfile)
 
 

code/components/jomjol_time_sntp/time_sntp.cpp

@@ -18,6 +18,7 @@
 static const char *TAG = "sntp";
 
 bool setTimeAlwaysOnReboot = true;
+time_t bootTime;
 
 static void obtain_time(void);
 static void initialize_sntp(void);
@@ -125,4 +126,17 @@ static void initialize_sntp(void)
     sntp_setservername(0, "pool.ntp.org");
 //    sntp_set_time_sync_notification_cb(time_sync_notification_cb);
     sntp_init();
+}
+
+void setBootTime() 
+{
+    time(&bootTime);
+}
+
+time_t getUpTime() 
+{
+    time_t now;
+    time(&now);
+
+    return now - bootTime;
 }

code/components/jomjol_time_sntp/time_sntp.h

@@ -18,4 +18,7 @@ std::string gettimestring(const char * frm);
 std::string ConvertTimeToString(time_t _time, const char * frm);
 
 void setTimeZone(std::string _tzstring);
-void reset_servername(std::string _servername);
+void reset_servername(std::string _servername);
+
+void setBootTime();
+time_t getUpTime();

code/components/jomjol_wlan/connect_wlan.cpp

@@ -50,14 +50,14 @@ std::string std_hostname = "watermeter";
 std::string ipadress = "";
 std::string ssid = "";
 
-std::string getIPAddress()
+std::string* getIPAddress()
 {
-    return ipadress;
+    return &ipadress;
 }
 
-std::string getSSID()
+std::string* getSSID()
 {
-    return ssid;
+    return &ssid;
 }
 
 
@@ -115,14 +115,10 @@ static void event_handler(void* arg, esp_event_base_t event_base,
         LEDBlinkTask(200, 1, true);
         esp_wifi_connect();
     } else if (event_base == WIFI_EVENT && event_id == WIFI_EVENT_STA_DISCONNECTED) {
-        if (s_retry_num < EXAMPLE_ESP_MAXIMUM_RETRY) {
-//            LEDBlinkTask(200, 1, true);
+//        if (s_retry_num < EXAMPLE_ESP_MAXIMUM_RETRY){ 
             esp_wifi_connect();
             s_retry_num++;
             ESP_LOGI(TAG, "retry to connect to the AP");
-        } else {
-            xEventGroupSetBits(s_wifi_event_group, WIFI_FAIL_BIT);
-        }
         ESP_LOGI(TAG,"connect to the AP fail");
     } else if (event_base == IP_EVENT && event_id == IP_EVENT_STA_GOT_IP) {
         ip_event_got_ip_t* event = (ip_event_got_ip_t*) event_data;
@@ -188,32 +184,6 @@ void wifi_init_sta(const char *_ssid, const char *_password, const char *_hostna
         ip_addr_set_ip4_u32(&dns_info.ip, ip.addr);
         ESP_ERROR_CHECK(esp_netif_set_dns_info(my_sta, ESP_NETIF_DNS_MAIN, &dns_info));
     }
-/////////////////////////////////////////////////////////////////
-
-
-//    esp_netif_create_default_wifi_sta();
-
-//    wifi_init_config_t cfg = WIFI_INIT_CONFIG_DEFAULT();
-//    ESP_ERROR_CHECK(esp_wifi_init(&cfg));
-
-
-
-/* 
-////////////////////////////// esp-idf 4.2 //////////////////////////
-    esp_event_handler_instance_t instance_any_id;
-    esp_event_handler_instance_t instance_got_ip;
-    ESP_ERROR_CHECK(esp_event_handler_instance_register(WIFI_EVENT,
-                                                        ESP_EVENT_ANY_ID,
-                                                        &event_handler,
-                                                        NULL,
-                                                        &instance_any_id));
-    ESP_ERROR_CHECK(esp_event_handler_instance_register(IP_EVENT,
-                                                        IP_EVENT_STA_GOT_IP,
-                                                        &event_handler,
-                                                        NULL,
-                                                        &instance_got_ip));
-////////////////////////// ENDE esp-idf 4.2 ///////////////////////////
-*/
 
     ESP_ERROR_CHECK(esp_event_handler_register(WIFI_EVENT, ESP_EVENT_ANY_ID, &event_handler, NULL));
     ESP_ERROR_CHECK(esp_event_handler_register(IP_EVENT, IP_EVENT_STA_GOT_IP, &event_handler, NULL));
@@ -257,27 +227,6 @@ void wifi_init_sta(const char *_ssid, const char *_password, const char *_hostna
     } else {
         ESP_LOGE(TAG, "UNEXPECTED EVENT");
     }
-
-    /* The event will not be processed after unregister */
-/* 
-////////////////////////////// esp-idf 4.2 //////////////////////////
-    ESP_ERROR_CHECK(esp_event_handler_instance_unregister(IP_EVENT, IP_EVENT_STA_GOT_IP, instance_got_ip));
-    ESP_ERROR_CHECK(esp_event_handler_instance_unregister(WIFI_EVENT, ESP_EVENT_ANY_ID, instance_any_id));
-////////////////////////// ENDE esp-idf 4.2 ///////////////////////////
-*/
-
-/* Deaktiveren, damit bei einen Verbindungsabbruch neu aufgebaut wird
-    ESP_ERROR_CHECK(esp_event_handler_unregister(IP_EVENT, IP_EVENT_STA_GOT_IP, &event_handler));
-    ESP_ERROR_CHECK(esp_event_handler_unregister(WIFI_EVENT, ESP_EVENT_ANY_ID, &event_handler));
-    vEventGroupDelete(s_wifi_event_group);
-*/ 
-
-/*
-    while (BlinkIsRunning)
-    {
-        vTaskDelay(100 / portTICK_PERIOD_MS);
-    }
-*/
 }
 
 

code/components/jomjol_wlan/connect_wlan.h

@@ -7,8 +7,8 @@ void wifi_init_sta(const char *_ssid, const char *_password, const char *_hostna
 void wifi_init_sta(const char *_ssid, const char *_password, const char *_hostname);
 void wifi_init_sta(const char *_ssid, const char *_password);
 
-std::string getIPAddress();
-std::string getSSID();
+std::string* getIPAddress();
+std::string* getSSID();
 
 extern std::string hostname;
 extern std::string std_hostname;

code/components/jomjol_wlan/read_wlanini.cpp

@@ -12,7 +12,7 @@
 #include <string.h>
 
 
-std::vector<string> ZerlegeZeile(std::string input, std::string _delimiter = "")
+std::vector<string> ZerlegeZeileWLAN(std::string input, std::string _delimiter = "")
 {
 	std::vector<string> Output;
 	std::string delimiter = " =,";
@@ -23,13 +23,13 @@ std::vector<string> ZerlegeZeile(std::string input, std::string _delimiter = "")
 	input = trim(input, delimiter);
 	size_t pos = findDelimiterPos(input, delimiter);
 	std::string token;
-	while (pos != std::string::npos) {
+    if (pos != std::string::npos)           // Zerlegt nur bis ersten Gleichheitszeichen !!! Sonderfall für WLAN.ini
+    {
 		token = input.substr(0, pos);
 		token = trim(token, delimiter);
 		Output.push_back(token);
 		input.erase(0, pos + 1);
 		input = trim(input, delimiter);
-		pos = findDelimiterPos(input, delimiter);
 	}
 	Output.push_back(input);
 
@@ -67,10 +67,8 @@ void LoadWlanFromFile(std::string fn, char *&_ssid, char *&_password, char *&_ho
     while ((line.size() > 0) || !(feof(pFile)))
     {
 //        printf("%s", line.c_str());
-        zerlegt = ZerlegeZeile(line, "=");
+        zerlegt = ZerlegeZeileWLAN(line, "=");
         zerlegt[0] = trim(zerlegt[0], " ");
-        for (int i = 2; i < zerlegt.size(); ++i)
-            zerlegt[1] = zerlegt[1] + "=" + zerlegt[i];
 
         if ((zerlegt.size() > 1) && (toUpper(zerlegt[0]) == "HOSTNAME")){
             hostname = trim(zerlegt[1]);
@@ -212,7 +210,7 @@ bool ChangeHostName(std::string fn, std::string _newhostname)
     while ((line.size() > 0) || !(feof(pFile)))
     {
         printf("%s", line.c_str());
-        zerlegt = ZerlegeZeile(line, "=");
+        zerlegt = ZerlegeZeileWLAN(line, "=");
         zerlegt[0] = trim(zerlegt[0], " ");
 
         if ((zerlegt.size() > 1) && (toUpper(zerlegt[0]) == "HOSTNAME")){

code/components/tflite-lib/CMakeLists.txt

@@ -0,0 +1,50 @@
+cmake_minimum_required(VERSION 3.5)
+
+set(tflite_dir "${CMAKE_CURRENT_SOURCE_DIR}/tensorflow/lite")
+set(tfmicro_dir "${tflite_dir}/micro")
+set(tfmicro_frontend_dir "${tflite_dir}/experimental/microfrontend/lib")
+set(tfmicro_kernels_dir "${tfmicro_dir}/kernels")
+
+file(GLOB srcs_micro
+          "${tfmicro_dir}/*.cc"
+          "${tfmicro_dir}/*.c")
+
+file(GLOB src_micro_frontend
+          "${tfmicro_frontend_dir}/*.c"
+          "${tfmicro_frontend_dir}/*.cc")
+file(GLOB srcs_kernels
+          "${tfmicro_kernels_dir}/*.c"
+          "${tfmicro_kernels_dir}/*.cc")
+
+set(lib_srcs
+          "${srcs_micro}"
+          "${srcs_kernels}"
+          "${src_micro_frontend}"
+          "${tflite_dir}/kernels/kernel_util.cc"
+          "${tflite_dir}/micro/memory_planner/greedy_memory_planner.cc"
+          "${tflite_dir}/micro/memory_planner/linear_memory_planner.cc"
+          "${tflite_dir}/c/common.c"
+          "${tflite_dir}/core/api/error_reporter.cc"
+          "${tflite_dir}/core/api/flatbuffer_conversions.cc"
+          "${tflite_dir}/core/api/op_resolver.cc"
+          "${tflite_dir}/core/api/tensor_utils.cc"
+          "${tflite_dir}/kernels/internal/quantization_util.cc"
+          "${tflite_dir}/schema/schema_utils.cc")
+
+idf_component_register(
+            SRCS "${lib_srcs}"
+            INCLUDE_DIRS "." "third_party/gemmlowp"
+                         "third_party/flatbuffers/include"
+                         "third_party/ruy"
+                         "third_party/kissfft")
+
+# Reduce the level of paranoia to be able to compile TF sources
+target_compile_options(${COMPONENT_LIB} PRIVATE
+  -Wno-maybe-uninitialized
+  -Wno-missing-field-initializers
+  -Wno-type-limits)
+
+target_compile_options(${COMPONENT_LIB} PRIVATE -fno-unwind-tables -ffunction-sections -fdata-sections -fmessage-length=0 -DTF_LITE_STATIC_MEMORY -DTF_LITE_DISABLE_X86_NEON -O3 -Wsign-compare -Wdouble-promotion -Wshadow -Wunused-variable -Wmissing-field-initializers -Wunused-function -Wswitch -Wvla -Wall -Wextra -Wstrict-aliasing -Wno-unused-parameter -DESP -DESP_NN -Wno-nonnull -Wno-nonnull -Wno-nonnull)
+target_compile_options(${COMPONENT_LIB} PRIVATE $<$<COMPILE_LANGUAGE:CXX>: -std=c++11 -fno-rtti -fno-exceptions -fno-threadsafe-statics -fno-unwind-tables -ffunction-sections -fdata-sections -fmessage-length=0 -DTF_LITE_STATIC_MEMORY -DTF_LITE_DISABLE_X86_NEON -O3 -Werror -Wsign-compare -Wdouble-promotion -Wshadow -Wunused-variable -Wmissing-field-initializers -Wunused-function -Wswitch -Wvla -Wall -Wextra -Wstrict-aliasing -Wno-unused-parameter -DESP -DESP_NN -Wno-return-type -Wno-strict-aliasing -std=gnu++14 -Wno-return-type -Wno-strict-aliasing -std=gnu++14 -Wno-return-type -Wno-strict-aliasing -std=gnu++14 >)
+target_compile_options(${COMPONENT_LIB} INTERFACE $<$<IN_LIST:-DTF_LITE_STATIC_MEMORY,$<TARGET_PROPERTY:${COMPONENT_LIB},COMPILE_OPTIONS>>:-DTF_LITE_STATIC_MEMORY>)
+target_link_libraries(${COMPONENT_LIB} PRIVATE -lm)

code/components/tflite-lib/tensorflow/lite/c/builtin_op_data.h

@@ -63,7 +63,6 @@ typedef struct {
 } TfLiteMirrorPaddingParams;
 
 // Possible fused activation functions.
-// TODO(aselle): rename to TfLiteActivation
 typedef enum {
   kTfLiteActNone = 0,
   kTfLiteActRelu,
@@ -98,6 +97,8 @@ typedef struct {
   TfLiteFusedActivation activation;
 } TfLiteConv3DParams;
 
+typedef TfLiteConv3DParams TfLiteConv3DTransposeParams;
+
 typedef struct {
   TfLitePadding padding;
   int stride_width;
@@ -328,8 +329,9 @@ typedef struct {
 } TfLitePadV2Params;
 
 typedef struct {
-  // TODO(ahentz): We can't have dynamic data in this struct, at least not yet.
-  // For now we will fix the maximum possible number of dimensions.
+  // These fields are only used in old models for backward compatibility.
+  // In the current implementation, we use the 2nd input of the op as the shape,
+  // and these fields are unused.
   int shape[TFLITE_RESHAPE_PARAMS_MAX_DIMENSION_COUNT];
   int num_dimensions;
 } TfLiteReshapeParams;
@@ -495,6 +497,27 @@ typedef struct {
   TfLiteType value_dtype;
 } TfLiteHashtableParams;
 
+typedef struct {
+  const char* container;
+  const char* shared_name;
+} TfLiteVarHandleParams;
+
+typedef struct {
+  int seed;
+  int seed2;
+} TfLiteRandomParams;
+
+typedef struct {
+  int num_boundaries;
+  // This points to the memory stored in the model (flatbuffer),
+  // and is not owned.
+  const float* boundaries;
+} TfLiteBucketizeParams;
+
+typedef struct {
+  bool approximate;
+} TfLiteGeluParams;
+
 #ifdef __cplusplus
 }  // extern "C"
 #endif  // __cplusplus

code/components/tflite-lib/tensorflow/lite/c/c_api_types.h

@@ -29,7 +29,9 @@ extern "C" {
 // library.
 #ifdef SWIG
 #define TFL_CAPI_EXPORT
-#else
+#elif defined(TFL_STATIC_LIBRARY_BUILD)
+#define TFL_CAPI_EXPORT
+#else  // not definded TFL_STATIC_LIBRARY_BUILD
 #if defined(_WIN32)
 #ifdef TFL_COMPILE_LIBRARY
 #define TFL_CAPI_EXPORT __declspec(dllexport)
@@ -41,6 +43,9 @@ extern "C" {
 #endif  // _WIN32
 #endif  // SWIG
 
+// Note that new error status values may be added in future in order to
+// indicate more fine-grained internal states, therefore, applications should
+// not rely on status values being members of the enum.
 typedef enum TfLiteStatus {
   kTfLiteOk = 0,
 
@@ -52,9 +57,26 @@ typedef enum TfLiteStatus {
 
   // Generally referring to an error in applying a delegate due to
   // incompatibility between runtime and delegate, e.g., this error is returned
-  // when trying to apply a TfLite delegate onto a model graph that's already
+  // when trying to apply a TF Lite delegate onto a model graph that's already
   // immutable.
-  kTfLiteApplicationError = 3
+  kTfLiteApplicationError = 3,
+
+  // Generally referring to serialized delegate data not being found.
+  // See tflite::delegates::Serialization.
+  kTfLiteDelegateDataNotFound = 4,
+
+  // Generally referring to data-writing issues in delegate serialization.
+  // See tflite::delegates::Serialization.
+  kTfLiteDelegateDataWriteError = 5,
+
+  // Generally referring to data-reading issues in delegate serialization.
+  // See tflite::delegates::Serialization.
+  kTfLiteDelegateDataReadError = 6,
+
+  // Generally referring to issues when the TF Lite model has ops that cannot be
+  // resolved at runtime. This could happen when the specific op is not
+  // registered or built with the TF Lite framework.
+  kTfLiteUnresolvedOps = 7,
 } TfLiteStatus;
 
 // Types supported by tensor

code/components/tflite-lib/tensorflow/lite/c/common.c

@@ -21,9 +21,15 @@ limitations under the License.
 #include <string.h>
 #endif  // TF_LITE_STATIC_MEMORY
 
-int TfLiteIntArrayGetSizeInBytes(int size) {
+size_t TfLiteIntArrayGetSizeInBytes(int size) {
   static TfLiteIntArray dummy;
-  return sizeof(dummy) + sizeof(dummy.data[0]) * size;
+
+  size_t computed_size = sizeof(dummy) + sizeof(dummy.data[0]) * size;
+#if defined(_MSC_VER)
+  // Context for why this is needed is in http://b/189926408#comment21
+  computed_size -= sizeof(dummy.data[0]);
+#endif
+  return computed_size;
 }
 
 int TfLiteIntArrayEqual(const TfLiteIntArray* a, const TfLiteIntArray* b) {
@@ -45,8 +51,10 @@ int TfLiteIntArrayEqualsArray(const TfLiteIntArray* a, int b_size,
 #ifndef TF_LITE_STATIC_MEMORY
 
 TfLiteIntArray* TfLiteIntArrayCreate(int size) {
-  TfLiteIntArray* ret =
-      (TfLiteIntArray*)malloc(TfLiteIntArrayGetSizeInBytes(size));
+  size_t alloc_size = TfLiteIntArrayGetSizeInBytes(size);
+  if (alloc_size <= 0) return NULL;
+  TfLiteIntArray* ret = (TfLiteIntArray*)malloc(alloc_size);
+  if (!ret) return ret;
   ret->size = size;
   return ret;
 }
@@ -66,7 +74,13 @@ void TfLiteIntArrayFree(TfLiteIntArray* a) { free(a); }
 
 int TfLiteFloatArrayGetSizeInBytes(int size) {
   static TfLiteFloatArray dummy;
-  return sizeof(dummy) + sizeof(dummy.data[0]) * size;
+
+  int computed_size = sizeof(dummy) + sizeof(dummy.data[0]) * size;
+#if defined(_MSC_VER)
+  // Context for why this is needed is in http://b/189926408#comment21
+  computed_size -= sizeof(dummy.data[0]);
+#endif
+  return computed_size;
 }
 
 #ifndef TF_LITE_STATIC_MEMORY
@@ -174,6 +188,26 @@ void TfLiteTensorReset(TfLiteType type, const char* name, TfLiteIntArray* dims,
   tensor->quantization.params = NULL;
 }
 
+TfLiteStatus TfLiteTensorCopy(const TfLiteTensor* src, TfLiteTensor* dst) {
+  if (!src || !dst)
+    return kTfLiteOk;
+  if (src->bytes != dst->bytes)
+    return kTfLiteError;
+  if (src == dst)
+    return kTfLiteOk;
+
+  dst->type = src->type;
+  if (dst->dims)
+    TfLiteIntArrayFree(dst->dims);
+  dst->dims = TfLiteIntArrayCopy(src->dims);
+  memcpy(dst->data.raw, src->data.raw, src->bytes);
+  dst->buffer_handle = src->buffer_handle;
+  dst->data_is_stale = src->data_is_stale;
+  dst->delegate = src->delegate;
+
+  return kTfLiteOk;
+}
+
 void TfLiteTensorRealloc(size_t num_bytes, TfLiteTensor* tensor) {
   if (tensor->allocation_type != kTfLiteDynamic &&
       tensor->allocation_type != kTfLitePersistentRo) {
@@ -181,9 +215,9 @@ void TfLiteTensorRealloc(size_t num_bytes, TfLiteTensor* tensor) {
   }
   // TODO(b/145340303): Tensor data should be aligned.
   if (!tensor->data.raw) {
-    tensor->data.raw = malloc(num_bytes);
+    tensor->data.raw = (char*)malloc(num_bytes);
   } else if (num_bytes > tensor->bytes) {
-    tensor->data.raw = realloc(tensor->data.raw, num_bytes);
+    tensor->data.raw = (char*)realloc(tensor->data.raw, num_bytes);
   }
   tensor->bytes = num_bytes;
 }
@@ -229,7 +263,7 @@ const char* TfLiteTypeGetName(TfLiteType type) {
   return "Unknown type";
 }
 
-TfLiteDelegate TfLiteDelegateCreate() {
+TfLiteDelegate TfLiteDelegateCreate(void) {
   TfLiteDelegate d = {
       .data_ = NULL,
       .Prepare = NULL,

code/components/tflite-lib/tensorflow/lite/c/common.h

@@ -80,12 +80,16 @@ typedef struct TfLiteExternalContext {
 // indices
 typedef struct TfLiteIntArray {
   int size;
-// gcc 6.1+ have a bug where flexible members aren't properly handled
-// https://github.com/google/re2/commit/b94b7cd42e9f02673cd748c1ac1d16db4052514c
-#if (!defined(__clang__) && defined(__GNUC__) && __GNUC__ == 6 && \
-     __GNUC_MINOR__ >= 1) ||                                      \
-    defined(HEXAGON) ||                                           \
+
+#if defined(_MSC_VER)
+  // Context for why this is needed is in http://b/189926408#comment21
+  int data[1];
+#elif (!defined(__clang__) && defined(__GNUC__) && __GNUC__ == 6 && \
+       __GNUC_MINOR__ >= 1) ||                                      \
+    defined(HEXAGON) ||                                             \
     (defined(__clang__) && __clang_major__ == 7 && __clang_minor__ == 1)
+  // gcc 6.1+ have a bug where flexible members aren't properly handled
+  // https://github.com/google/re2/commit/b94b7cd42e9f02673cd748c1ac1d16db4052514c
   int data[0];
 #else
   int data[];
@@ -94,7 +98,7 @@ typedef struct TfLiteIntArray {
 
 // Given the size (number of elements) in a TfLiteIntArray, calculate its size
 // in bytes.
-int TfLiteIntArrayGetSizeInBytes(int size);
+size_t TfLiteIntArrayGetSizeInBytes(int size);
 
 #ifndef TF_LITE_STATIC_MEMORY
 // Create a array of a given `size` (uninitialized entries).
@@ -121,11 +125,15 @@ void TfLiteIntArrayFree(TfLiteIntArray* a);
 // Fixed size list of floats. Used for per-channel quantization.
 typedef struct TfLiteFloatArray {
   int size;
-// gcc 6.1+ have a bug where flexible members aren't properly handled
-// https://github.com/google/re2/commit/b94b7cd42e9f02673cd748c1ac1d16db4052514c
-// This also applies to the toolchain used for Qualcomm Hexagon DSPs.
-#if !defined(__clang__) && defined(__GNUC__) && __GNUC__ == 6 && \
-    __GNUC_MINOR__ >= 1
+#if defined(_MSC_VER)
+  // Context for why this is needed is in http://b/189926408#comment21
+  float data[1];
+#elif (!defined(__clang__) && defined(__GNUC__) && __GNUC__ == 6 && \
+       __GNUC_MINOR__ >= 1) ||                                      \
+    defined(HEXAGON) ||                                             \
+    (defined(__clang__) && __clang_major__ == 7 && __clang_minor__ == 1)
+  // gcc 6.1+ have a bug where flexible members aren't properly handled
+  // https://github.com/google/re2/commit/b94b7cd42e9f02673cd748c1ac1d16db4052514c
   float data[0];
 #else
   float data[];
@@ -456,8 +464,8 @@ typedef struct TfLiteTensor {
 } TfLiteTensor;
 
 // A structure representing an instance of a node.
-// This structure only exhibits the inputs, outputs and user defined data, not
-// other features like the type.
+// This structure only exhibits the inputs, outputs, user defined data and some
+// node properties (like statefulness), not other features like the type.
 typedef struct TfLiteNode {
   // Inputs to this node expressed as indices into the simulator's tensors.
   TfLiteIntArray* inputs;
@@ -490,6 +498,9 @@ typedef struct TfLiteNode {
   // created by calling `interpreter.ModifyGraphWithDelegate`.
   // WARNING: This is an experimental interface that is subject to change.
   struct TfLiteDelegate* delegate;
+
+  // Whether this op might have side effect (e.g. stateful op).
+  bool might_have_side_effect;
 } TfLiteNode;
 #else   // defined(TF_LITE_STATIC_MEMORY)?
 // NOTE: This flag is opt-in only at compile time.
@@ -559,6 +570,10 @@ typedef struct TfLiteNode {
   // Outputs to this node expressed as indices into the simulator's tensors.
   TfLiteIntArray* outputs;
 
+  // intermediate tensors to this node expressed as indices into the simulator's
+  // tensors.
+  TfLiteIntArray* intermediates;
+
   // Opaque data provided by the node implementer through `Registration.init`.
   void* user_data;
 
@@ -611,6 +626,16 @@ void TfLiteTensorReset(TfLiteType type, const char* name, TfLiteIntArray* dims,
                        const void* allocation, bool is_variable,
                        TfLiteTensor* tensor);
 
+// Copies the contents of 'src' in 'dst'.
+// Function does nothing if either 'src' or 'dst' is passed as nullptr and
+// return kTfLiteOk.
+// Returns kTfLiteError if 'src' and 'dst' doesn't have matching data size.
+// Note function copies contents, so it won't create new data pointer
+// or change allocation type.
+// All Tensor related properties will be copied from 'src' to 'dst' like
+// quantization, sparsity, ...
+TfLiteStatus TfLiteTensorCopy(const TfLiteTensor* src, TfLiteTensor* dst);
+
 // Resize the allocated data of a (dynamic) tensor. Tensors with allocation
 // types other than kTfLiteDynamic will be ignored.
 void TfLiteTensorRealloc(size_t num_bytes, TfLiteTensor* tensor);
@@ -640,6 +665,7 @@ typedef struct TfLiteContext {
   // TfLiteDelegates can traverse the current execution plan by iterating
   // through each member of this array and using GetNodeAndRegistration() to
   // access details about a node. i.e.
+  //
   // TfLiteIntArray* execution_plan;
   // TF_LITE_ENSURE_STATUS(context->GetExecutionPlan(context, &execution_plan));
   // for (int exec_index = 0; exec_index < execution_plan->size; exec_index++) {
@@ -648,6 +674,28 @@ typedef struct TfLiteContext {
   //    TfLiteRegistration* reg;
   //    context->GetNodeAndRegistration(context, node_index, &node, &reg);
   // }
+  // Note: the memory pointed by '`*execution_plan` is OWNED by TfLite runtime.
+  // Future calls to GetExecutionPlan invalidates earlier outputs. The following
+  // code snippet shows the issue of such an invocation pattern. After calling
+  // CheckNode, subsequent access to `plan_1st` is undefined.
+  //
+  // void CheckNode(const TfLiteNode* node) {
+  //   ...
+  //   TfLiteIntArray* plan_2nd;
+  //   TF_LITE_ENSURE_STATUS(context->GetExecutionPlan(context, &plan_2nd));
+  //   ...
+  // }
+  //
+  // TfLiteIntArray* plan_1st;
+  // TF_LITE_ENSURE_STATUS(context->GetExecutionPlan(context, &plan_1st));
+  // for (int exec_index = 0; exec_index < plan_1st->size; exec_index++) {
+  //    int node_index = plan_1st->data[exec_index];
+  //    TfLiteNode* node;
+  //    TfLiteRegistration* reg;
+  //    context->GetNodeAndRegistration(context, node_index, &node, &reg);
+  //    CheckNode(node);
+  // }
+  //
   // WARNING: This is an experimental interface that is subject to change.
   TfLiteStatus (*GetExecutionPlan)(struct TfLiteContext* context,
                                    TfLiteIntArray** execution_plan);
@@ -777,10 +825,25 @@ typedef struct TfLiteContext {
   // WARNING: This method may not be available on all platforms.
   TfLiteEvalTensor* (*GetEvalTensor)(const struct TfLiteContext* context,
                                      int tensor_idx);
+
+  // Retrieves named metadata buffer from the TFLite model.
+  // Returns kTfLiteOk if metadata is successfully obtained from the flatbuffer
+  // Model: that is, there exists a `metadata` entry with given `name` string.
+  // (see TFLite's schema.fbs).
+  // The corresponding `buffer` information is populated in `ptr` & `bytes`.
+  // The data from `ptr` is valid for the lifetime of the Interpreter.
+  //
+  // WARNING: This is an experimental interface that is subject to change.
+  TfLiteStatus (*GetModelMetadata)(const struct TfLiteContext* context,
+                                   const char* name, const char** ptr,
+                                   size_t* bytes);
 } TfLiteContext;
 
 typedef struct TfLiteRegistration {
   // Initializes the op from serialized data.
+  // Called only *once* for the lifetime of the op, so any one-time allocations
+  // should be made here (unless they depend on tensor sizes).
+  //
   // If a built-in op:
   //   `buffer` is the op's params data (TfLiteLSTMParams*).
   //   `length` is zero.
@@ -803,6 +866,7 @@ typedef struct TfLiteRegistration {
   // prepare is called when the inputs this node depends on have been resized.
   // context->ResizeTensor() can be called to request output tensors to be
   // resized.
+  // Can be called multiple times for the lifetime of the op.
   //
   // Returns kTfLiteOk on success.
   TfLiteStatus (*prepare)(TfLiteContext* context, TfLiteNode* node);
@@ -918,7 +982,7 @@ typedef struct TfLiteDelegate {
 
 // Build a 'null' delegate, with all the fields properly set to their default
 // values.
-TfLiteDelegate TfLiteDelegateCreate();
+TfLiteDelegate TfLiteDelegateCreate(void);
 
 #ifdef __cplusplus
 }  // extern "C"

code/components/tflite-lib/tensorflow/lite/core/api/flatbuffer_conversions.cc

@@ -1,4 +1,4 @@
-/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -131,6 +131,17 @@ TfLitePadding ConvertPadding(Padding padding) {
   return kTfLitePaddingUnknown;
 }
 
+// Converts the flatbuffer mirror padding enum to what is used at runtime.
+TfLiteMirrorPaddingMode ConvertMirrorPadding(MirrorPadMode padding) {
+  switch (padding) {
+    case MirrorPadMode_REFLECT:
+      return kTfLiteMirrorPaddingReflect;
+    case MirrorPadMode_SYMMETRIC:
+      return kTfLiteMirrorPaddingSymmetric;
+  }
+  return kTfLiteMirrorPaddingUnknown;
+}
+
 #ifndef TF_LITE_STATIC_MEMORY
 TfLiteStatus ParseOpDataTfLite(const Operator* op, BuiltinOperator op_type,
                                ErrorReporter* error_reporter,
@@ -181,6 +192,10 @@ TfLiteStatus ParseOpDataTfLite(const Operator* op, BuiltinOperator op_type,
       return ParseArgMin(op, error_reporter, allocator, builtin_data);
     }
 
+    case BuiltinOperator_ASSIGN_VARIABLE: {
+      return ParseAssignVariable(op, error_reporter, allocator, builtin_data);
+    }
+
     case BuiltinOperator_AVERAGE_POOL_2D: {
       return ParsePool(op, error_reporter, allocator, builtin_data);
     }
@@ -193,6 +208,10 @@ TfLiteStatus ParseOpDataTfLite(const Operator* op, BuiltinOperator op_type,
       return ParseBatchToSpaceNd(op, error_reporter, allocator, builtin_data);
     }
 
+    case BuiltinOperator_CALL_ONCE: {
+      return ParseCallOnce(op, error_reporter, allocator, builtin_data);
+    }
+
     case BuiltinOperator_CEIL: {
       return ParseCeil(op, error_reporter, allocator, builtin_data);
     }
@@ -325,6 +344,10 @@ TfLiteStatus ParseOpDataTfLite(const Operator* op, BuiltinOperator op_type,
       return ParsePool(op, error_reporter, allocator, builtin_data);
     }
 
+    case BuiltinOperator_MIRROR_PAD: {
+      return ParseMirrorPad(op, error_reporter, allocator, builtin_data);
+    }
+
     case BuiltinOperator_MEAN: {
       return ParseReducer(op, error_reporter, allocator, builtin_data);
     }
@@ -369,10 +392,18 @@ TfLiteStatus ParseOpDataTfLite(const Operator* op, BuiltinOperator op_type,
       return ParseQuantize(op, error_reporter, allocator, builtin_data);
     }
 
+    case BuiltinOperator_READ_VARIABLE: {
+      return ParseReadVariable(op, error_reporter, allocator, builtin_data);
+    }
+
     case BuiltinOperator_REDUCE_ANY: {
       return ParseReducer(op, error_reporter, allocator, builtin_data);
     }
 
+    case BuiltinOperator_REDUCE_ALL: {
+      return ParseReducer(op, error_reporter, allocator, builtin_data);
+    }
+
     case BuiltinOperator_REDUCE_MAX: {
       return ParseReducer(op, error_reporter, allocator, builtin_data);
     }
@@ -482,6 +513,10 @@ TfLiteStatus ParseOpDataTfLite(const Operator* op, BuiltinOperator op_type,
       return ParseUnpack(op, error_reporter, allocator, builtin_data);
     }
 
+    case BuiltinOperator_VAR_HANDLE: {
+      return ParseVarHandle(op, error_reporter, allocator, builtin_data);
+    }
+
     case BuiltinOperator_ZEROS_LIKE: {
       return ParseZerosLike(op, error_reporter, allocator, builtin_data);
     }
@@ -602,21 +637,8 @@ TfLiteStatus ParseOpDataTfLite(const Operator* op, BuiltinOperator op_type,
       return kTfLiteOk;
     }
     case BuiltinOperator_UNIDIRECTIONAL_SEQUENCE_LSTM: {
-      auto params =
-          safe_allocator.Allocate<TfLiteUnidirectionalSequenceLSTMParams>();
-      TF_LITE_ENSURE(error_reporter, params != nullptr);
-      if (const auto* seq_lstm_params =
-              op->builtin_options_as_UnidirectionalSequenceLSTMOptions()) {
-        params->activation =
-            ConvertActivation(seq_lstm_params->fused_activation_function());
-        params->cell_clip = seq_lstm_params->cell_clip();
-        params->proj_clip = seq_lstm_params->proj_clip();
-        params->time_major = seq_lstm_params->time_major();
-        params->asymmetric_quantize_inputs =
-            seq_lstm_params->asymmetric_quantize_inputs();
-      }
-      *builtin_data = params.release();
-      return kTfLiteOk;
+      return ParseUnidirectionalSequenceLSTM(op, error_reporter, allocator,
+                                             builtin_data);
     }
     case BuiltinOperator_BIDIRECTIONAL_SEQUENCE_LSTM: {
       auto params =
@@ -663,7 +685,6 @@ TfLiteStatus ParseOpDataTfLite(const Operator* op, BuiltinOperator op_type,
       return kTfLiteOk;
     }
     case BuiltinOperator_DELEGATE: {
-      // TODO(ycling): Revisit when supporting saving delegated models.
       TF_LITE_REPORT_ERROR(error_reporter,
                            "DELEGATE op shouldn't exist in model.");
       return kTfLiteError;
@@ -690,19 +711,6 @@ TfLiteStatus ParseOpDataTfLite(const Operator* op, BuiltinOperator op_type,
       *builtin_data = params.release();
       return kTfLiteOk;
     }
-    case BuiltinOperator_MIRROR_PAD: {
-      auto params = safe_allocator.Allocate<TfLiteMirrorPaddingParams>();
-      TF_LITE_ENSURE(error_reporter, params != nullptr);
-      const auto* mirror_pad_params = op->builtin_options_as_MirrorPadOptions();
-      if (mirror_pad_params != nullptr) {
-        params->mode =
-            mirror_pad_params->mode() == tflite::MirrorPadMode_REFLECT
-                ? TfLiteMirrorPaddingMode::kTfLiteMirrorPaddingReflect
-                : TfLiteMirrorPaddingMode::kTfLiteMirrorPaddingSymmetric;
-      }
-      *builtin_data = params.release();
-      return kTfLiteOk;
-    }
     case BuiltinOperator_UNIQUE: {
       auto params = safe_allocator.Allocate<TfLiteUniqueParams>();
       TF_LITE_ENSURE(error_reporter, params != nullptr);
@@ -747,17 +755,8 @@ TfLiteStatus ParseOpDataTfLite(const Operator* op, BuiltinOperator op_type,
       *builtin_data = params.release();
       return kTfLiteOk;
     }
-    case BuiltinOperator_CALL_ONCE: {
-      auto params = safe_allocator.Allocate<TfLiteCallOnceParams>();
-      TF_LITE_ENSURE(error_reporter, params != nullptr);
-      if (const auto* call_once_params =
-              op->builtin_options_as_CallOnceOptions()) {
-        params->init_subgraph_index = call_once_params->init_subgraph_index();
-      }
-      *builtin_data = params.release();
-      return kTfLiteOk;
-    }
-    case BuiltinOperator_CONV_3D: {
+    case BuiltinOperator_CONV_3D:
+    case BuiltinOperator_CONV_3D_TRANSPOSE: {
       auto params = safe_allocator.Allocate<TfLiteConv3DParams>();
       TF_LITE_ENSURE(error_reporter, params != nullptr);
       if (const auto* conv3d_params = op->builtin_options_as_Conv3DOptions()) {
@@ -789,6 +788,73 @@ TfLiteStatus ParseOpDataTfLite(const Operator* op, BuiltinOperator op_type,
       *builtin_data = params.release();
       return kTfLiteOk;
     }
+    case BuiltinOperator_MULTINOMIAL: {
+      auto params = safe_allocator.Allocate<TfLiteRandomParams>();
+      TF_LITE_ENSURE(error_reporter, params != nullptr);
+      if (const auto* multinomial_params =
+              op->builtin_options_as_RandomOptions()) {
+        params->seed = multinomial_params->seed();
+        params->seed2 = multinomial_params->seed2();
+      }
+      *builtin_data = params.release();
+      return kTfLiteOk;
+    }
+    case BuiltinOperator_RANDOM_STANDARD_NORMAL: {
+      auto params = safe_allocator.Allocate<TfLiteRandomParams>();
+      TF_LITE_ENSURE(error_reporter, params != nullptr);
+      if (const auto* random_std_normal_params =
+              op->builtin_options_as_RandomOptions()) {
+        params->seed = random_std_normal_params->seed();
+        params->seed2 = random_std_normal_params->seed2();
+      }
+      *builtin_data = params.release();
+      return kTfLiteOk;
+    }
+    case BuiltinOperator_BUCKETIZE: {
+      auto params = safe_allocator.Allocate<TfLiteBucketizeParams>();
+      TF_LITE_ENSURE(error_reporter, params != nullptr);
+      if (const auto* bucketize_params =
+              op->builtin_options_as_BucketizeOptions()) {
+        const flatbuffers::Vector<float>* boundaries =
+            bucketize_params->boundaries();
+        if (boundaries == nullptr) {
+          TF_LITE_REPORT_ERROR(
+              error_reporter,
+              "boundaries array not provided for operation 'bucketize'.\n");
+          return kTfLiteError;
+        }
+        params->num_boundaries = boundaries->size();
+        if (boundaries->data() == nullptr) {
+          TF_LITE_REPORT_ERROR(error_reporter,
+                               "boundaries.data() returned nullptr for "
+                               "operation 'bucketize'.\n");
+          return kTfLiteError;
+        }
+        params->boundaries = boundaries->data();
+      }
+      *builtin_data = params.release();
+      return kTfLiteOk;
+    }
+    case BuiltinOperator_RANDOM_UNIFORM: {
+      auto params = safe_allocator.Allocate<TfLiteRandomParams>();
+      TF_LITE_ENSURE(error_reporter, params != nullptr);
+      if (const auto* random_uniform_params =
+              op->builtin_options_as_RandomOptions()) {
+        params->seed = random_uniform_params->seed();
+        params->seed2 = random_uniform_params->seed2();
+      }
+      *builtin_data = params.release();
+      return kTfLiteOk;
+    }
+    case BuiltinOperator_GELU: {
+      auto params = safe_allocator.Allocate<TfLiteGeluParams>();
+      TF_LITE_ENSURE(error_reporter, params != nullptr);
+      if (const auto* gelu_params = op->builtin_options_as_GeluOptions()) {
+        params->approximate = gelu_params->approximate();
+      }
+      *builtin_data = params.release();
+      return kTfLiteOk;
+    }
     // Below are the ops with no builtin_data structure.
     // TODO(aselle): Implement call in BuiltinOptions, but nullptrs are
     // ok for now, since there is no call implementation either.
@@ -825,6 +891,7 @@ TfLiteStatus ParseOpDataTfLite(const Operator* op, BuiltinOperator op_type,
     case BuiltinOperator_HASHTABLE_FIND:
     case BuiltinOperator_HASHTABLE_IMPORT:
     case BuiltinOperator_HASHTABLE_SIZE:
+    case BuiltinOperator_BROADCAST_ARGS:
       return kTfLiteOk;
     case BuiltinOperator_PLACEHOLDER_FOR_GREATER_OP_CODES:
       return kTfLiteError;
@@ -981,6 +1048,14 @@ TfLiteStatus ParseArgMin(const Operator* op, ErrorReporter* error_reporter,
   return kTfLiteOk;
 }
 
+// We have this parse function instead of directly returning kTfLiteOk from the
+// switch-case in ParseOpData because this function is used as part of the
+// selective registration for the OpResolver implementation in micro.
+TfLiteStatus ParseAssignVariable(const Operator*, ErrorReporter*,
+                                 BuiltinDataAllocator*, void**) {
+  return kTfLiteOk;
+}
+
 // We have this parse function instead of directly returning kTfLiteOk from the
 // switch-case in ParseOpData because this function is used as part of the
 // selective registration for the OpResolver implementation in micro.
@@ -1010,6 +1085,33 @@ TfLiteStatus ParseBatchToSpaceNd(const Operator*, ErrorReporter*,
   return kTfLiteOk;
 }
 
+TfLiteStatus ParseCallOnce(const Operator* op, ErrorReporter* error_reporter,
+                           BuiltinDataAllocator* allocator,
+                           void** builtin_data) {
+  CheckParsePointerParams(op, error_reporter, allocator, builtin_data);
+
+  SafeBuiltinDataAllocator safe_allocator(allocator);
+  std::unique_ptr<TfLiteCallOnceParams,
+                  SafeBuiltinDataAllocator::BuiltinDataDeleter>
+      params = safe_allocator.Allocate<TfLiteCallOnceParams>();
+  TF_LITE_ENSURE(error_reporter, params != nullptr);
+
+  const CallOnceOptions* schema_params =
+      op->builtin_options_as_CallOnceOptions();
+
+  if (schema_params != nullptr) {
+    params->init_subgraph_index = schema_params->init_subgraph_index();
+
+  } else {
+    // TODO(b/157480169): We should either return kTfLiteError or fill in some
+    // reasonable defaults in the params struct. We are not doing so until we
+    // better undertand the ramifications of changing the legacy behavior.
+  }
+
+  *builtin_data = params.release();
+  return kTfLiteOk;
+}
+
 // We have this parse function instead of directly returning kTfLiteOk from the
 // switch-case in ParseOpData because this function is used as part of the
 // selective registration for the OpResolver implementation in micro.
@@ -1372,6 +1474,30 @@ TfLiteStatus ParseHardSwish(const Operator*, ErrorReporter*,
   return kTfLiteOk;
 }
 
+TfLiteStatus ParseIf(const Operator* op, ErrorReporter* error_reporter,
+                     BuiltinDataAllocator* allocator, void** builtin_data) {
+  CheckParsePointerParams(op, error_reporter, allocator, builtin_data);
+
+  SafeBuiltinDataAllocator safe_allocator(allocator);
+  std::unique_ptr<TfLiteIfParams, SafeBuiltinDataAllocator::BuiltinDataDeleter>
+      params = safe_allocator.Allocate<TfLiteIfParams>();
+  TF_LITE_ENSURE(error_reporter, params != nullptr);
+
+  const IfOptions* schema_params = op->builtin_options_as_IfOptions();
+
+  if (schema_params != nullptr) {
+    params->then_subgraph_index = schema_params->then_subgraph_index();
+    params->else_subgraph_index = schema_params->else_subgraph_index();
+  } else {
+    // TODO(b/157480169): We should either return kTfLiteError or fill in some
+    // reasonable defaults in the params struct. We are not doing so until we
+    // better undertand the ramifications of changing the legacy behavior.
+  }
+
+  *builtin_data = params.release();
+  return kTfLiteOk;
+}
+
 TfLiteStatus ParseL2Normalization(const Operator* op,
                                   ErrorReporter* error_reporter,
                                   BuiltinDataAllocator* allocator,
@@ -1495,6 +1621,32 @@ TfLiteStatus ParseMinimum(const Operator*, ErrorReporter*,
   return kTfLiteOk;
 }
 
+TfLiteStatus ParseMirrorPad(const Operator* op, ErrorReporter* error_reporter,
+                            BuiltinDataAllocator* allocator,
+                            void** builtin_data) {
+  CheckParsePointerParams(op, error_reporter, allocator, builtin_data);
+
+  SafeBuiltinDataAllocator safe_allocator(allocator);
+  std::unique_ptr<TfLiteMirrorPaddingParams,
+                  SafeBuiltinDataAllocator::BuiltinDataDeleter>
+      params = safe_allocator.Allocate<TfLiteMirrorPaddingParams>();
+  TF_LITE_ENSURE(error_reporter, params != nullptr);
+
+  const MirrorPadOptions* schema_params =
+      op->builtin_options_as_MirrorPadOptions();
+
+  if (schema_params != nullptr) {
+    params->mode = ConvertMirrorPadding(schema_params->mode());
+  } else {
+    // TODO(b/157480169): We should either return kTfLiteError or fill in some
+    // reasonable defaults in the params struct. We are not doing so until we
+    // better undertand the ramifications of changing the legacy behavior.
+  }
+
+  *builtin_data = params.release();
+  return kTfLiteOk;
+}
+
 TfLiteStatus ParseMul(const Operator* op, ErrorReporter* error_reporter,
                       BuiltinDataAllocator* allocator, void** builtin_data) {
   CheckParsePointerParams(op, error_reporter, allocator, builtin_data);
@@ -1630,6 +1782,14 @@ TfLiteStatus ParseQuantize(const Operator*, ErrorReporter*,
   return kTfLiteOk;
 }
 
+// We have this parse function instead of directly returning kTfLiteOk from the
+// switch-case in ParseOpData because this function is used as part of the
+// selective registration for the OpResolver implementation in micro.
+TfLiteStatus ParseReadVariable(const Operator*, ErrorReporter*,
+                               BuiltinDataAllocator*, void**) {
+  return kTfLiteOk;
+}
+
 TfLiteStatus ParseReducer(const Operator* op, ErrorReporter* error_reporter,
                           BuiltinDataAllocator* allocator,
                           void** builtin_data) {
@@ -1810,6 +1970,14 @@ TfLiteStatus ParseSin(const Operator*, ErrorReporter*, BuiltinDataAllocator*,
   return kTfLiteOk;
 }
 
+// We have this parse function instead of directly returning kTfLiteOk from the
+// switch-case in ParseOpData because this function is used as part of the
+// selective registration for the OpResolver implementation in micro.
+TfLiteStatus ParseSlice(const Operator*, ErrorReporter*, BuiltinDataAllocator*,
+                        void**) {
+  return kTfLiteOk;
+}
+
 TfLiteStatus ParseSoftmax(const Operator* op, ErrorReporter* error_reporter,
                           BuiltinDataAllocator* allocator,
                           void** builtin_data) {
@@ -1916,6 +2084,29 @@ TfLiteStatus ParseSplitV(const Operator* op, ErrorReporter* error_reporter,
   return kTfLiteOk;
 }
 
+TfLiteStatus ParseUnidirectionalSequenceLSTM(const Operator* op,
+                                             ErrorReporter* error_reporter,
+                                             BuiltinDataAllocator* allocator,
+                                             void** builtin_data) {
+  CheckParsePointerParams(op, error_reporter, allocator, builtin_data);
+  SafeBuiltinDataAllocator safe_allocator(allocator);
+  auto params =
+      safe_allocator.Allocate<TfLiteUnidirectionalSequenceLSTMParams>();
+  TF_LITE_ENSURE(error_reporter, params != nullptr);
+  if (const auto* seq_lstm_params =
+          op->builtin_options_as_UnidirectionalSequenceLSTMOptions()) {
+    params->activation =
+        ConvertActivation(seq_lstm_params->fused_activation_function());
+    params->cell_clip = seq_lstm_params->cell_clip();
+    params->proj_clip = seq_lstm_params->proj_clip();
+    params->time_major = seq_lstm_params->time_major();
+    params->asymmetric_quantize_inputs =
+        seq_lstm_params->asymmetric_quantize_inputs();
+  }
+  *builtin_data = params.release();
+  return kTfLiteOk;
+}
+
 TfLiteStatus ParseSqueeze(const Operator* op, ErrorReporter* error_reporter,
                           BuiltinDataAllocator* allocator,
                           void** builtin_data) {
@@ -2115,6 +2306,37 @@ TfLiteStatus ParseUnpack(const Operator* op, ErrorReporter* error_reporter,
   return kTfLiteOk;
 }
 
+TfLiteStatus ParseVarHandle(const Operator* op, ErrorReporter* error_reporter,
+                            BuiltinDataAllocator* allocator,
+                            void** builtin_data) {
+  CheckParsePointerParams(op, error_reporter, allocator, builtin_data);
+
+  SafeBuiltinDataAllocator safe_allocator(allocator);
+  std::unique_ptr<TfLiteVarHandleParams,
+                  SafeBuiltinDataAllocator::BuiltinDataDeleter>
+      params = safe_allocator.Allocate<TfLiteVarHandleParams>();
+  TF_LITE_ENSURE(error_reporter, params != nullptr);
+
+  const VarHandleOptions* schema_params =
+      op->builtin_options_as_VarHandleOptions();
+
+  if (schema_params != nullptr) {
+    if (schema_params->container()) {
+      params->container = schema_params->container()->c_str();
+    }
+    if (schema_params->shared_name()) {
+      params->shared_name = schema_params->shared_name()->c_str();
+    }
+  } else {
+    // TODO(b/157480169): We should either return kTfLiteError or fill in some
+    // reasonable defaults in the params struct. We are not doing so until we
+    // better undertand the ramifications of changing the legacy behavior.
+  }
+
+  *builtin_data = params.release();
+  return kTfLiteOk;
+}
+
 // We have this parse function instead of directly returning kTfLiteOk from the
 // switch-case in ParseOpData because this function is used as part of the
 // selective registration for the OpResolver implementation in micro.

code/components/tflite-lib/tensorflow/lite/core/api/flatbuffer_conversions.h

@@ -1,4 +1,4 @@
-/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+/* Copyright 2021 The TensorFlow Authors. All Rights Reserved.
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -84,6 +84,11 @@ TfLiteStatus ParseArgMax(const Operator* op, ErrorReporter* error_reporter,
 TfLiteStatus ParseArgMin(const Operator* op, ErrorReporter* error_reporter,
                          BuiltinDataAllocator* allocator, void** builtin_data);
 
+TfLiteStatus ParseAssignVariable(const Operator* op,
+                                 ErrorReporter* error_reporter,
+                                 BuiltinDataAllocator* allocator,
+                                 void** builtin_data);
+
 TfLiteStatus ParseBatchMatMul(const Operator* op, ErrorReporter* error_reporter,
                               BuiltinDataAllocator* allocator,
                               void** builtin_data);
@@ -93,6 +98,10 @@ TfLiteStatus ParseBatchToSpaceNd(const Operator* op,
                                  BuiltinDataAllocator* allocator,
                                  void** builtin_data);
 
+TfLiteStatus ParseCallOnce(const Operator* op, ErrorReporter* error_reporter,
+                           BuiltinDataAllocator* allocator,
+                           void** builtin_data);
+
 TfLiteStatus ParseCeil(const Operator* op, ErrorReporter* error_reporter,
                        BuiltinDataAllocator* allocator, void** builtin_data);
 
@@ -181,6 +190,9 @@ TfLiteStatus ParseHardSwish(const Operator* op, ErrorReporter* error_reporter,
                             BuiltinDataAllocator* allocator,
                             void** builtin_data);
 
+TfLiteStatus ParseIf(const Operator* op, ErrorReporter* error_reporter,
+                     BuiltinDataAllocator* allocator, void** builtin_data);
+
 TfLiteStatus ParseL2Normalization(const Operator* op,
                                   ErrorReporter* error_reporter,
                                   BuiltinDataAllocator* allocator,
@@ -226,6 +238,10 @@ TfLiteStatus ParseMaximum(const Operator* op, ErrorReporter* error_reporter,
 TfLiteStatus ParseMinimum(const Operator* op, ErrorReporter* error_reporter,
                           BuiltinDataAllocator* allocator, void** builtin_data);
 
+TfLiteStatus ParseMirrorPad(const Operator* op, ErrorReporter* error_reporter,
+                            BuiltinDataAllocator* allocator,
+                            void** builtin_data);
+
 TfLiteStatus ParseMul(const Operator* op, ErrorReporter* error_reporter,
                       BuiltinDataAllocator* allocator, void** builtin_data);
 
@@ -258,6 +274,11 @@ TfLiteStatus ParseQuantize(const Operator* op, ErrorReporter* error_reporter,
                            BuiltinDataAllocator* allocator,
                            void** builtin_data);
 
+TfLiteStatus ParseReadVariable(const Operator* op,
+                               ErrorReporter* error_reporter,
+                               BuiltinDataAllocator* allocator,
+                               void** builtin_data);
+
 TfLiteStatus ParseReducer(const Operator* op, ErrorReporter* error_reporter,
                           BuiltinDataAllocator* allocator, void** builtin_data);
 
@@ -292,6 +313,9 @@ TfLiteStatus ParseShape(const Operator* op, ErrorReporter* error_reporter,
 TfLiteStatus ParseSin(const Operator* op, ErrorReporter* error_reporter,
                       BuiltinDataAllocator* allocator, void** builtin_data);
 
+TfLiteStatus ParseSlice(const Operator* op, ErrorReporter* error_reporter,
+                        BuiltinDataAllocator* allocator, void** builtin_data);
+
 TfLiteStatus ParseSoftmax(const Operator* op, ErrorReporter* error_reporter,
                           BuiltinDataAllocator* allocator, void** builtin_data);
 
@@ -346,6 +370,15 @@ TfLiteStatus ParseTransposeConv(const Operator* op,
 TfLiteStatus ParseUnpack(const Operator* op, ErrorReporter* error_reporter,
                          BuiltinDataAllocator* allocator, void** builtin_data);
 
+TfLiteStatus ParseUnidirectionalSequenceLSTM(const Operator* op,
+                                             ErrorReporter* error_reporter,
+                                             BuiltinDataAllocator* allocator,
+                                             void** builtin_data);
+
+TfLiteStatus ParseVarHandle(const Operator* op, ErrorReporter* error_reporter,
+                            BuiltinDataAllocator* allocator,
+                            void** builtin_data);
+
 TfLiteStatus ParseZerosLike(const Operator* op, ErrorReporter* error_reporter,
                             BuiltinDataAllocator* allocator,
                             void** builtin_data);

code/components/tflite-lib/tensorflow/lite/core/api/op_resolver.cc

@@ -30,8 +30,7 @@ TfLiteStatus GetRegistrationFromOpCode(
   auto builtin_code = GetBuiltinCode(opcode);
   int version = opcode->version();
 
-  if (builtin_code > BuiltinOperator_MAX ||
-      builtin_code < BuiltinOperator_MIN) {
+  if (builtin_code > BuiltinOperator_MAX) {
     TF_LITE_REPORT_ERROR(
         error_reporter,
         "Op builtin_code out of range: %d. Are you using old TFLite binary "

code/components/tflite-lib/tensorflow/lite/core/api/op_resolver.h

@@ -15,6 +15,7 @@ limitations under the License.
 #ifndef TENSORFLOW_LITE_CORE_API_OP_RESOLVER_H_
 #define TENSORFLOW_LITE_CORE_API_OP_RESOLVER_H_
 
+#include <functional>
 #include <memory>
 #include <vector>
 
@@ -36,16 +37,43 @@ class OpResolver {
   virtual const TfLiteRegistration* FindOp(const char* op,
                                            int version) const = 0;
 
+  using TfLiteDelegatePtrVector =
+      std::vector<std::unique_ptr<TfLiteDelegate, void (*)(TfLiteDelegate*)>>;
   // Returns optional delegates for resolving and handling ops in the flatbuffer
   // model. This may be used in addition to the standard TfLiteRegistration
   // lookup for graph resolution.
-  using TfLiteDelegatePtrVector =
-      std::vector<std::unique_ptr<TfLiteDelegate, void (*)(TfLiteDelegate*)>>;
+  // WARNING: This API is deprecated, GetDelegateCreators is preferred.
   virtual TfLiteDelegatePtrVector GetDelegates(int num_threads) const {
-    return TfLiteDelegatePtrVector();
+    return {};
   }
 
+  // Represent a function that creates a TfLite delegate instance.
+  using TfLiteDelegateCreator =
+      std::function<std::unique_ptr<TfLiteDelegate, void (*)(TfLiteDelegate*)>(
+          int /*num_threads*/)>;
+  using TfLiteDelegateCreators = std::vector<TfLiteDelegateCreator>;
+  // Returns a vector of delegate creators to create optional delegates for
+  // resolving and handling ops in the flatbuffer model. This may be used in
+  // addition to the standard TfLiteRegistration lookup for graph resolution.
+  virtual TfLiteDelegateCreators GetDelegateCreators() const { return {}; }
+
   virtual ~OpResolver() {}
+
+ private:
+  /// Returns true if this OpResolver may contain any "user defined" ops.
+  /// By "user defined" ops, we mean any op definitions other than those
+  /// contained in tflite::ops::builtin::BuiltinOpResolver.
+  ///
+  /// If this method returns true, it doesn't necessarily mean that the
+  /// OpResolver contains a user-defined op, just that the absence of
+  /// user-defined ops can't be guaranteed.
+  ///
+  /// Note that "user-defined" ops are not the same as "custom" ops;
+  /// BuiltinOpResolver may support certain "custom" ops, in addition to
+  /// "builtin" ops, and may not support all of the "builtin" op enum values.
+  virtual bool MayContainUserDefinedOps() const { return true; }
+
+  friend class OpResolverInternal;
 };
 
 // Handles the logic for converting between an OperatorCode structure extracted

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/bits.h

@@ -0,0 +1,102 @@
+/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+#ifndef TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_BITS_H_
+#define TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_BITS_H_
+
+#ifdef __cplusplus
+#include <cstdint>
+
+extern "C" {
+#endif
+
+static inline int CountLeadingZeros32Slow(uint64_t n) {
+  int zeroes = 28;
+  if (n >> 16) zeroes -= 16, n >>= 16;
+  if (n >> 8) zeroes -= 8, n >>= 8;
+  if (n >> 4) zeroes -= 4, n >>= 4;
+  return "\4\3\2\2\1\1\1\1\0\0\0\0\0\0\0"[n] + zeroes;
+}
+
+static inline int CountLeadingZeros32(uint32_t n) {
+#if defined(_MSC_VER)
+  unsigned long result = 0;  // NOLINT(runtime/int)
+  if (_BitScanReverse(&result, n)) {
+    return 31 - result;
+  }
+  return 32;
+#elif defined(__GNUC__)
+
+  // Handle 0 as a special case because __builtin_clz(0) is undefined.
+  if (n == 0) {
+    return 32;
+  }
+  return __builtin_clz(n);
+#else
+  return CountLeadingZeros32Slow(n);
+#endif
+}
+
+static inline int MostSignificantBit32(uint32_t n) {
+  return 32 - CountLeadingZeros32(n);
+}
+
+static inline int CountLeadingZeros64Slow(uint64_t n) {
+  int zeroes = 60;
+  if (n >> 32) zeroes -= 32, n >>= 32;
+  if (n >> 16) zeroes -= 16, n >>= 16;
+  if (n >> 8) zeroes -= 8, n >>= 8;
+  if (n >> 4) zeroes -= 4, n >>= 4;
+  return "\4\3\2\2\1\1\1\1\0\0\0\0\0\0\0"[n] + zeroes;
+}
+
+static inline int CountLeadingZeros64(uint64_t n) {
+#if defined(_MSC_VER) && defined(_M_X64)
+  // MSVC does not have __builtin_clzll. Use _BitScanReverse64.
+  unsigned long result = 0;  // NOLINT(runtime/int)
+  if (_BitScanReverse64(&result, n)) {
+    return 63 - result;
+  }
+  return 64;
+#elif defined(_MSC_VER)
+  // MSVC does not have __builtin_clzll. Compose two calls to _BitScanReverse
+  unsigned long result = 0;  // NOLINT(runtime/int)
+  if ((n >> 32) && _BitScanReverse(&result, n >> 32)) {
+    return 31 - result;
+  }
+  if (_BitScanReverse(&result, n)) {
+    return 63 - result;
+  }
+  return 64;
+#elif defined(__GNUC__)
+
+  // Handle 0 as a special case because __builtin_clzll(0) is undefined.
+  if (n == 0) {
+    return 64;
+  }
+  return __builtin_clzll(n);
+#else
+  return CountLeadingZeros64Slow(n);
+#endif
+}
+
+static inline int MostSignificantBit64(uint64_t n) {
+  return 64 - CountLeadingZeros64(n);
+}
+
+#ifdef __cplusplus
+}  // extern "C"
+#endif
+
+#endif  // TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_BITS_H_

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/fft.cc

@@ -0,0 +1,52 @@
+/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+#include "tensorflow/lite/experimental/microfrontend/lib/fft.h"
+#include "tensorflow/lite/experimental/microfrontend/lib/kiss_fft_int16.h"
+
+#include <string.h>
+
+
+void FftCompute(struct FftState* state, const int16_t* input,
+                int input_scale_shift) {
+  const size_t input_size = state->input_size;
+  const size_t fft_size = state->fft_size;
+
+  int16_t* fft_input = state->input;
+  // First, scale the input by the given shift.
+  size_t i;
+  for (i = 0; i < input_size; ++i) {
+    fft_input[i] = static_cast<int16_t>(static_cast<uint16_t>(input[i])
+                                        << input_scale_shift);
+  }
+  // Zero out whatever else remains in the top part of the input.
+  for (; i < fft_size; ++i) {
+    fft_input[i] = 0;
+  }
+
+  // Apply the FFT.
+  kissfft_fixed16::kiss_fftr(
+    reinterpret_cast<kissfft_fixed16::kiss_fftr_cfg>(state->scratch),
+    state->input,
+    reinterpret_cast<kissfft_fixed16::kiss_fft_cpx*>(state->output));
+}
+
+void FftInit(struct FftState* state) {
+  // All the initialization is done in FftPopulateState()
+}
+
+void FftReset(struct FftState* state) {
+  memset(state->input, 0, state->fft_size * sizeof(*state->input));
+  memset(state->output, 0, (state->fft_size / 2 + 1) * sizeof(*state->output));
+}

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/fft.h

@@ -0,0 +1,50 @@
+/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+#ifndef TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_FFT_H_
+#define TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_FFT_H_
+
+#include <stdint.h>
+#include <stdlib.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct complex_int16_t {
+  int16_t real;
+  int16_t imag;
+};
+
+struct FftState {
+  int16_t* input;
+  struct complex_int16_t* output;
+  size_t fft_size;
+  size_t input_size;
+  void* scratch;
+  size_t scratch_size;
+};
+
+void FftCompute(struct FftState* state, const int16_t* input,
+                int input_scale_shift);
+
+void FftInit(struct FftState* state);
+
+void FftReset(struct FftState* state);
+
+#ifdef __cplusplus
+}  // extern "C"
+#endif
+
+#endif  // TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_FFT_H_

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/fft_util.cc

@@ -0,0 +1,69 @@
+/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+#include "tensorflow/lite/experimental/microfrontend/lib/fft_util.h"
+#include "tensorflow/lite/experimental/microfrontend/lib/kiss_fft_int16.h"
+
+#include <stdio.h>
+
+int FftPopulateState(struct FftState* state, size_t input_size) {
+  state->input_size = input_size;
+  state->fft_size = 1;
+  while (state->fft_size < state->input_size) {
+    state->fft_size <<= 1;
+  }
+
+  state->input = reinterpret_cast<int16_t*>(
+      malloc(state->fft_size * sizeof(*state->input)));
+  if (state->input == nullptr) {
+    fprintf(stderr, "Failed to alloc fft input buffer\n");
+    return 0;
+  }
+
+  state->output = reinterpret_cast<complex_int16_t*>(
+      malloc((state->fft_size / 2 + 1) * sizeof(*state->output) * 2));
+  if (state->output == nullptr) {
+    fprintf(stderr, "Failed to alloc fft output buffer\n");
+    return 0;
+  }
+
+  // Ask kissfft how much memory it wants.
+  size_t scratch_size = 0;
+  kissfft_fixed16::kiss_fftr_cfg kfft_cfg = kissfft_fixed16::kiss_fftr_alloc(
+      state->fft_size, 0, nullptr, &scratch_size);
+  if (kfft_cfg != nullptr) {
+    fprintf(stderr, "Kiss memory sizing failed.\n");
+    return 0;
+  }
+  state->scratch = malloc(scratch_size);
+  if (state->scratch == nullptr) {
+    fprintf(stderr, "Failed to alloc fft scratch buffer\n");
+    return 0;
+  }
+  state->scratch_size = scratch_size;
+  // Let kissfft configure the scratch space we just allocated
+  kfft_cfg = kissfft_fixed16::kiss_fftr_alloc(state->fft_size, 0,
+                                              state->scratch, &scratch_size);
+  if (kfft_cfg != state->scratch) {
+    fprintf(stderr, "Kiss memory preallocation strategy failed.\n");
+    return 0;
+  }
+  return 1;
+}
+
+void FftFreeStateContents(struct FftState* state) {
+  free(state->input);
+  free(state->output);
+  free(state->scratch);
+}

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/fft_util.h

@@ -0,0 +1,34 @@
+/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+#ifndef TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_FFT_UTIL_H_
+#define TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_FFT_UTIL_H_
+
+#include "tensorflow/lite/experimental/microfrontend/lib/fft.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+// Prepares and FFT for the given input size.
+int FftPopulateState(struct FftState* state, size_t input_size);
+
+// Frees any allocated buffers.
+void FftFreeStateContents(struct FftState* state);
+
+#ifdef __cplusplus
+}  // extern "C"
+#endif
+
+#endif  // TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_FFT_UTIL_H_

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/filterbank.c

@@ -0,0 +1,134 @@
+/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+#include "tensorflow/lite/experimental/microfrontend/lib/filterbank.h"
+
+#include <string.h>
+
+#include "tensorflow/lite/experimental/microfrontend/lib/bits.h"
+
+void FilterbankConvertFftComplexToEnergy(struct FilterbankState* state,
+                                         struct complex_int16_t* fft_output,
+                                         int32_t* energy) {
+  const int end_index = state->end_index;
+  int i;
+  energy += state->start_index;
+  fft_output += state->start_index;
+  for (i = state->start_index; i < end_index; ++i) {
+    const int32_t real = fft_output->real;
+    const int32_t imag = fft_output->imag;
+    fft_output++;
+    const uint32_t mag_squared = (real * real) + (imag * imag);
+    *energy++ = mag_squared;
+  }
+}
+
+void FilterbankAccumulateChannels(struct FilterbankState* state,
+                                  const int32_t* energy) {
+  uint64_t* work = state->work;
+  uint64_t weight_accumulator = 0;
+  uint64_t unweight_accumulator = 0;
+
+  const int16_t* channel_frequency_starts = state->channel_frequency_starts;
+  const int16_t* channel_weight_starts = state->channel_weight_starts;
+  const int16_t* channel_widths = state->channel_widths;
+
+  int num_channels_plus_1 = state->num_channels + 1;
+  int i;
+  for (i = 0; i < num_channels_plus_1; ++i) {
+    const int32_t* magnitudes = energy + *channel_frequency_starts++;
+    const int16_t* weights = state->weights + *channel_weight_starts;
+    const int16_t* unweights = state->unweights + *channel_weight_starts++;
+    const int width = *channel_widths++;
+    int j;
+    for (j = 0; j < width; ++j) {
+      weight_accumulator += *weights++ * ((uint64_t)*magnitudes);
+      unweight_accumulator += *unweights++ * ((uint64_t)*magnitudes);
+      ++magnitudes;
+    }
+    *work++ = weight_accumulator;
+    weight_accumulator = unweight_accumulator;
+    unweight_accumulator = 0;
+  }
+}
+
+static uint16_t Sqrt32(uint32_t num) {
+  if (num == 0) {
+    return 0;
+  }
+  uint32_t res = 0;
+  int max_bit_number = 32 - MostSignificantBit32(num);
+  max_bit_number |= 1;
+  uint32_t bit = 1U << (31 - max_bit_number);
+  int iterations = (31 - max_bit_number) / 2 + 1;
+  while (iterations--) {
+    if (num >= res + bit) {
+      num -= res + bit;
+      res = (res >> 1U) + bit;
+    } else {
+      res >>= 1U;
+    }
+    bit >>= 2U;
+  }
+  // Do rounding - if we have the bits.
+  if (num > res && res != 0xFFFF) {
+    ++res;
+  }
+  return res;
+}
+
+static uint32_t Sqrt64(uint64_t num) {
+  // Take a shortcut and just use 32 bit operations if the upper word is all
+  // clear. This will cause a slight off by one issue for numbers close to 2^32,
+  // but it probably isn't going to matter (and gives us a big performance win).
+  if ((num >> 32) == 0) {
+    return Sqrt32((uint32_t)num);
+  }
+  uint64_t res = 0;
+  int max_bit_number = 64 - MostSignificantBit64(num);
+  max_bit_number |= 1;
+  uint64_t bit = 1ULL << (63 - max_bit_number);
+  int iterations = (63 - max_bit_number) / 2 + 1;
+  while (iterations--) {
+    if (num >= res + bit) {
+      num -= res + bit;
+      res = (res >> 1U) + bit;
+    } else {
+      res >>= 1U;
+    }
+    bit >>= 2U;
+  }
+  // Do rounding - if we have the bits.
+  if (num > res && res != 0xFFFFFFFFLL) {
+    ++res;
+  }
+  return res;
+}
+
+uint32_t* FilterbankSqrt(struct FilterbankState* state, int scale_down_shift) {
+  const int num_channels = state->num_channels;
+  const uint64_t* work = state->work + 1;
+  // Reuse the work buffer since we're fine clobbering it at this point to hold
+  // the output.
+  uint32_t* output = (uint32_t*)state->work;
+  int i;
+  for (i = 0; i < num_channels; ++i) {
+    *output++ = Sqrt64(*work++) >> scale_down_shift;
+  }
+  return (uint32_t*)state->work;
+}
+
+void FilterbankReset(struct FilterbankState* state) {
+  memset(state->work, 0, (state->num_channels + 1) * sizeof(*state->work));
+}

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/filterbank.h

@@ -0,0 +1,63 @@
+/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+#ifndef TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_FILTERBANK_H_
+#define TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_FILTERBANK_H_
+
+#include <stdint.h>
+#include <stdlib.h>
+
+#include "tensorflow/lite/experimental/microfrontend/lib/fft.h"
+
+#define kFilterbankBits 12
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct FilterbankState {
+  int num_channels;
+  int start_index;
+  int end_index;
+  int16_t* channel_frequency_starts;
+  int16_t* channel_weight_starts;
+  int16_t* channel_widths;
+  int16_t* weights;
+  int16_t* unweights;
+  uint64_t* work;
+};
+
+// Converts the relevant complex values of an FFT output into energy (the
+// square magnitude).
+void FilterbankConvertFftComplexToEnergy(struct FilterbankState* state,
+                                         struct complex_int16_t* fft_output,
+                                         int32_t* energy);
+
+// Computes the mel-scale filterbank on the given energy array. Output is cached
+// internally - to fetch it, you need to call FilterbankSqrt.
+void FilterbankAccumulateChannels(struct FilterbankState* state,
+                                  const int32_t* energy);
+
+// Applies an integer square root to the 64 bit intermediate values of the
+// filterbank, and returns a pointer to them. Memory will be invalidated the
+// next time FilterbankAccumulateChannels is called.
+uint32_t* FilterbankSqrt(struct FilterbankState* state, int scale_down_shift);
+
+void FilterbankReset(struct FilterbankState* state);
+
+#ifdef __cplusplus
+}  // extern "C"
+#endif
+
+#endif  // TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_FILTERBANK_H_

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/filterbank_util.c

@@ -0,0 +1,220 @@
+/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+#include "tensorflow/lite/experimental/microfrontend/lib/filterbank_util.h"
+
+#include <assert.h>
+#include <math.h>
+#include <stdio.h>
+
+#define kFilterbankIndexAlignment 4
+#define kFilterbankChannelBlockSize 4
+
+void FilterbankFillConfigWithDefaults(struct FilterbankConfig* config) {
+  config->num_channels = 32;
+  config->lower_band_limit = 125.0f;
+  config->upper_band_limit = 7500.0f;
+  config->output_scale_shift = 7;
+}
+
+static float FreqToMel(float freq) { return 1127.0 * log1p(freq / 700.0); }
+
+static void CalculateCenterFrequencies(const int num_channels,
+                                       const float lower_frequency_limit,
+                                       const float upper_frequency_limit,
+                                       float* center_frequencies) {
+  assert(lower_frequency_limit >= 0.0f);
+  assert(upper_frequency_limit > lower_frequency_limit);
+
+  const float mel_low = FreqToMel(lower_frequency_limit);
+  const float mel_hi = FreqToMel(upper_frequency_limit);
+  const float mel_span = mel_hi - mel_low;
+  const float mel_spacing = mel_span / ((float)num_channels);
+  int i;
+  for (i = 0; i < num_channels; ++i) {
+    center_frequencies[i] = mel_low + (mel_spacing * (i + 1));
+  }
+}
+
+static void QuantizeFilterbankWeights(const float float_weight, int16_t* weight,
+                                      int16_t* unweight) {
+  *weight = floor(float_weight * (1 << kFilterbankBits) + 0.5);
+  *unweight = floor((1.0 - float_weight) * (1 << kFilterbankBits) + 0.5);
+}
+
+int FilterbankPopulateState(const struct FilterbankConfig* config,
+                            struct FilterbankState* state, int sample_rate,
+                            int spectrum_size) {
+  state->num_channels = config->num_channels;
+  const int num_channels_plus_1 = config->num_channels + 1;
+
+  // How should we align things to index counts given the byte alignment?
+  const int index_alignment =
+      (kFilterbankIndexAlignment < sizeof(int16_t)
+           ? 1
+           : kFilterbankIndexAlignment / sizeof(int16_t));
+
+  state->channel_frequency_starts =
+      malloc(num_channels_plus_1 * sizeof(*state->channel_frequency_starts));
+  state->channel_weight_starts =
+      malloc(num_channels_plus_1 * sizeof(*state->channel_weight_starts));
+  state->channel_widths =
+      malloc(num_channels_plus_1 * sizeof(*state->channel_widths));
+  state->work = malloc(num_channels_plus_1 * sizeof(*state->work));
+
+  float* center_mel_freqs =
+      malloc(num_channels_plus_1 * sizeof(*center_mel_freqs));
+  int16_t* actual_channel_starts =
+      malloc(num_channels_plus_1 * sizeof(*actual_channel_starts));
+  int16_t* actual_channel_widths =
+      malloc(num_channels_plus_1 * sizeof(*actual_channel_widths));
+
+  if (state->channel_frequency_starts == NULL ||
+      state->channel_weight_starts == NULL || state->channel_widths == NULL ||
+      center_mel_freqs == NULL || actual_channel_starts == NULL ||
+      actual_channel_widths == NULL) {
+    free(center_mel_freqs);
+    free(actual_channel_starts);
+    free(actual_channel_widths);
+    fprintf(stderr, "Failed to allocate channel buffers\n");
+    return 0;
+  }
+
+  CalculateCenterFrequencies(num_channels_plus_1, config->lower_band_limit,
+                             config->upper_band_limit, center_mel_freqs);
+
+  // Always exclude DC.
+  const float hz_per_sbin = 0.5 * sample_rate / ((float)spectrum_size - 1);
+  state->start_index = 1.5 + config->lower_band_limit / hz_per_sbin;
+  state->end_index = 0;  // Initialized to zero here, but actually set below.
+
+  // For each channel, we need to figure out what frequencies belong to it, and
+  // how much padding we need to add so that we can efficiently multiply the
+  // weights and unweights for accumulation. To simplify the multiplication
+  // logic, all channels will have some multiplication to do (even if there are
+  // no frequencies that accumulate to that channel) - they will be directed to
+  // a set of zero weights.
+  int chan_freq_index_start = state->start_index;
+  int weight_index_start = 0;
+  int needs_zeros = 0;
+
+  int chan;
+  for (chan = 0; chan < num_channels_plus_1; ++chan) {
+    // Keep jumping frequencies until we overshoot the bound on this channel.
+    int freq_index = chan_freq_index_start;
+    while (FreqToMel((freq_index)*hz_per_sbin) <= center_mel_freqs[chan]) {
+      ++freq_index;
+    }
+
+    const int width = freq_index - chan_freq_index_start;
+    actual_channel_starts[chan] = chan_freq_index_start;
+    actual_channel_widths[chan] = width;
+
+    if (width == 0) {
+      // This channel doesn't actually get anything from the frequencies, it's
+      // always zero. We need then to insert some 'zero' weights into the
+      // output, and just redirect this channel to do a single multiplication at
+      // this point. For simplicity, the zeros are placed at the beginning of
+      // the weights arrays, so we have to go and update all the other
+      // weight_starts to reflect this shift (but only once).
+      state->channel_frequency_starts[chan] = 0;
+      state->channel_weight_starts[chan] = 0;
+      state->channel_widths[chan] = kFilterbankChannelBlockSize;
+      if (!needs_zeros) {
+        needs_zeros = 1;
+        int j;
+        for (j = 0; j < chan; ++j) {
+          state->channel_weight_starts[j] += kFilterbankChannelBlockSize;
+        }
+        weight_index_start += kFilterbankChannelBlockSize;
+      }
+    } else {
+      // How far back do we need to go to ensure that we have the proper
+      // alignment?
+      const int aligned_start =
+          (chan_freq_index_start / index_alignment) * index_alignment;
+      const int aligned_width = (chan_freq_index_start - aligned_start + width);
+      const int padded_width =
+          (((aligned_width - 1) / kFilterbankChannelBlockSize) + 1) *
+          kFilterbankChannelBlockSize;
+
+      state->channel_frequency_starts[chan] = aligned_start;
+      state->channel_weight_starts[chan] = weight_index_start;
+      state->channel_widths[chan] = padded_width;
+      weight_index_start += padded_width;
+    }
+    chan_freq_index_start = freq_index;
+  }
+
+  // Allocate the two arrays to store the weights - weight_index_start contains
+  // the index of what would be the next set of weights that we would need to
+  // add, so that's how many weights we need to allocate.
+  state->weights = calloc(weight_index_start, sizeof(*state->weights));
+  state->unweights = calloc(weight_index_start, sizeof(*state->unweights));
+
+  // If the alloc failed, we also need to nuke the arrays.
+  if (state->weights == NULL || state->unweights == NULL) {
+    free(center_mel_freqs);
+    free(actual_channel_starts);
+    free(actual_channel_widths);
+    fprintf(stderr, "Failed to allocate weights or unweights\n");
+    return 0;
+  }
+
+  // Next pass, compute all the weights. Since everything has been memset to
+  // zero, we only need to fill in the weights that correspond to some frequency
+  // for a channel.
+  const float mel_low = FreqToMel(config->lower_band_limit);
+  for (chan = 0; chan < num_channels_plus_1; ++chan) {
+    int frequency = actual_channel_starts[chan];
+    const int num_frequencies = actual_channel_widths[chan];
+    const int frequency_offset =
+        frequency - state->channel_frequency_starts[chan];
+    const int weight_start = state->channel_weight_starts[chan];
+    const float denom_val = (chan == 0) ? mel_low : center_mel_freqs[chan - 1];
+
+    int j;
+    for (j = 0; j < num_frequencies; ++j, ++frequency) {
+      const float weight =
+          (center_mel_freqs[chan] - FreqToMel(frequency * hz_per_sbin)) /
+          (center_mel_freqs[chan] - denom_val);
+
+      // Make the float into an integer for the weights (and unweights).
+      const int weight_index = weight_start + frequency_offset + j;
+      QuantizeFilterbankWeights(weight, state->weights + weight_index,
+                                state->unweights + weight_index);
+    }
+    if (frequency > state->end_index) {
+      state->end_index = frequency;
+    }
+  }
+
+  free(center_mel_freqs);
+  free(actual_channel_starts);
+  free(actual_channel_widths);
+  if (state->end_index >= spectrum_size) {
+    fprintf(stderr, "Filterbank end_index is above spectrum size.\n");
+    return 0;
+  }
+  return 1;
+}
+
+void FilterbankFreeStateContents(struct FilterbankState* state) {
+  free(state->channel_frequency_starts);
+  free(state->channel_weight_starts);
+  free(state->channel_widths);
+  free(state->weights);
+  free(state->unweights);
+  free(state->work);
+}

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/filterbank_util.h

@@ -0,0 +1,50 @@
+/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+#ifndef TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_FILTERBANK_UTIL_H_
+#define TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_FILTERBANK_UTIL_H_
+
+#include "tensorflow/lite/experimental/microfrontend/lib/filterbank.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct FilterbankConfig {
+  // number of frequency channel buckets for filterbank
+  int num_channels;
+  // maximum frequency to include
+  float upper_band_limit;
+  // minimum frequency to include
+  float lower_band_limit;
+  // unused
+  int output_scale_shift;
+};
+
+// Fills the frontendConfig with "sane" defaults.
+void FilterbankFillConfigWithDefaults(struct FilterbankConfig* config);
+
+// Allocates any buffers.
+int FilterbankPopulateState(const struct FilterbankConfig* config,
+                            struct FilterbankState* state, int sample_rate,
+                            int spectrum_size);
+
+// Frees any allocated buffers.
+void FilterbankFreeStateContents(struct FilterbankState* state);
+
+#ifdef __cplusplus
+}  // extern "C"
+#endif
+
+#endif  // TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_FILTERBANK_UTIL_H_

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/frontend.c

@@ -0,0 +1,72 @@
+/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+#include "tensorflow/lite/experimental/microfrontend/lib/frontend.h"
+
+#include "tensorflow/lite/experimental/microfrontend/lib/bits.h"
+
+struct FrontendOutput FrontendProcessSamples(struct FrontendState* state,
+                                             const int16_t* samples,
+                                             size_t num_samples,
+                                             size_t* num_samples_read) {
+  struct FrontendOutput output;
+  output.values = NULL;
+  output.size = 0;
+
+  // Try to apply the window - if it fails, return and wait for more data.
+  if (!WindowProcessSamples(&state->window, samples, num_samples,
+                            num_samples_read)) {
+    return output;
+  }
+
+  // Apply the FFT to the window's output (and scale it so that the fixed point
+  // FFT can have as much resolution as possible).
+  int input_shift =
+      15 - MostSignificantBit32(state->window.max_abs_output_value);
+  FftCompute(&state->fft, state->window.output, input_shift);
+
+  // We can re-ruse the fft's output buffer to hold the energy.
+  int32_t* energy = (int32_t*)state->fft.output;
+
+  FilterbankConvertFftComplexToEnergy(&state->filterbank, state->fft.output,
+                                      energy);
+
+  FilterbankAccumulateChannels(&state->filterbank, energy);
+  uint32_t* scaled_filterbank = FilterbankSqrt(&state->filterbank, input_shift);
+
+  // Apply noise reduction.
+  NoiseReductionApply(&state->noise_reduction, scaled_filterbank);
+
+  if (state->pcan_gain_control.enable_pcan) {
+    PcanGainControlApply(&state->pcan_gain_control, scaled_filterbank);
+  }
+
+  // Apply the log and scale.
+  int correction_bits =
+      MostSignificantBit32(state->fft.fft_size) - 1 - (kFilterbankBits / 2);
+  uint16_t* logged_filterbank =
+      LogScaleApply(&state->log_scale, scaled_filterbank,
+                    state->filterbank.num_channels, correction_bits);
+
+  output.size = state->filterbank.num_channels;
+  output.values = logged_filterbank;
+  return output;
+}
+
+void FrontendReset(struct FrontendState* state) {
+  WindowReset(&state->window);
+  FftReset(&state->fft);
+  FilterbankReset(&state->filterbank);
+  NoiseReductionReset(&state->noise_reduction);
+}

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/frontend.h

@@ -0,0 +1,64 @@
+/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+#ifndef TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_FRONTEND_H_
+#define TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_FRONTEND_H_
+
+#include <stdint.h>
+#include <stdlib.h>
+
+#include "tensorflow/lite/experimental/microfrontend/lib/fft.h"
+#include "tensorflow/lite/experimental/microfrontend/lib/filterbank.h"
+#include "tensorflow/lite/experimental/microfrontend/lib/log_scale.h"
+#include "tensorflow/lite/experimental/microfrontend/lib/noise_reduction.h"
+#include "tensorflow/lite/experimental/microfrontend/lib/pcan_gain_control.h"
+#include "tensorflow/lite/experimental/microfrontend/lib/window.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct FrontendState {
+  struct WindowState window;
+  struct FftState fft;
+  struct FilterbankState filterbank;
+  struct NoiseReductionState noise_reduction;
+  struct PcanGainControlState pcan_gain_control;
+  struct LogScaleState log_scale;
+};
+
+struct FrontendOutput {
+  const uint16_t* values;
+  size_t size;
+};
+
+// Main entry point to processing frontend samples. Updates num_samples_read to
+// contain the number of samples that have been consumed from the input array.
+// Returns a struct containing the generated output. If not enough samples were
+// added to generate a feature vector, the returned size will be 0 and the
+// values pointer will be NULL. Note that the output pointer will be invalidated
+// as soon as FrontendProcessSamples is called again, so copy the contents
+// elsewhere if you need to use them later.
+struct FrontendOutput FrontendProcessSamples(struct FrontendState* state,
+                                             const int16_t* samples,
+                                             size_t num_samples,
+                                             size_t* num_samples_read);
+
+void FrontendReset(struct FrontendState* state);
+
+#ifdef __cplusplus
+}  // extern "C"
+#endif
+
+#endif  // TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_FRONTEND_H_

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/frontend_util.c

@@ -0,0 +1,85 @@
+/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+#include "tensorflow/lite/experimental/microfrontend/lib/frontend_util.h"
+
+#include <stdio.h>
+#include <string.h>
+
+#include "tensorflow/lite/experimental/microfrontend/lib/bits.h"
+
+void FrontendFillConfigWithDefaults(struct FrontendConfig* config) {
+  WindowFillConfigWithDefaults(&config->window);
+  FilterbankFillConfigWithDefaults(&config->filterbank);
+  NoiseReductionFillConfigWithDefaults(&config->noise_reduction);
+  PcanGainControlFillConfigWithDefaults(&config->pcan_gain_control);
+  LogScaleFillConfigWithDefaults(&config->log_scale);
+}
+
+int FrontendPopulateState(const struct FrontendConfig* config,
+                          struct FrontendState* state, int sample_rate) {
+  memset(state, 0, sizeof(*state));
+
+  if (!WindowPopulateState(&config->window, &state->window, sample_rate)) {
+    fprintf(stderr, "Failed to populate window state\n");
+    return 0;
+  }
+
+  if (!FftPopulateState(&state->fft, state->window.size)) {
+    fprintf(stderr, "Failed to populate fft state\n");
+    return 0;
+  }
+  FftInit(&state->fft);
+
+  if (!FilterbankPopulateState(&config->filterbank, &state->filterbank,
+                               sample_rate, state->fft.fft_size / 2 + 1)) {
+    fprintf(stderr, "Failed to populate filterbank state\n");
+    return 0;
+  }
+
+  if (!NoiseReductionPopulateState(&config->noise_reduction,
+                                   &state->noise_reduction,
+                                   state->filterbank.num_channels)) {
+    fprintf(stderr, "Failed to populate noise reduction state\n");
+    return 0;
+  }
+
+  int input_correction_bits =
+      MostSignificantBit32(state->fft.fft_size) - 1 - (kFilterbankBits / 2);
+  if (!PcanGainControlPopulateState(
+          &config->pcan_gain_control, &state->pcan_gain_control,
+          state->noise_reduction.estimate, state->filterbank.num_channels,
+          state->noise_reduction.smoothing_bits, input_correction_bits)) {
+    fprintf(stderr, "Failed to populate pcan gain control state\n");
+    return 0;
+  }
+
+  if (!LogScalePopulateState(&config->log_scale, &state->log_scale)) {
+    fprintf(stderr, "Failed to populate log scale state\n");
+    return 0;
+  }
+
+  FrontendReset(state);
+
+  // All good, return a true value.
+  return 1;
+}
+
+void FrontendFreeStateContents(struct FrontendState* state) {
+  WindowFreeStateContents(&state->window);
+  FftFreeStateContents(&state->fft);
+  FilterbankFreeStateContents(&state->filterbank);
+  NoiseReductionFreeStateContents(&state->noise_reduction);
+  PcanGainControlFreeStateContents(&state->pcan_gain_control);
+}

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/frontend_util.h

@@ -0,0 +1,52 @@
+/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+#ifndef TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_FRONTEND_UTIL_H_
+#define TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_FRONTEND_UTIL_H_
+
+#include "tensorflow/lite/experimental/microfrontend/lib/fft_util.h"
+#include "tensorflow/lite/experimental/microfrontend/lib/filterbank_util.h"
+#include "tensorflow/lite/experimental/microfrontend/lib/frontend.h"
+#include "tensorflow/lite/experimental/microfrontend/lib/log_scale_util.h"
+#include "tensorflow/lite/experimental/microfrontend/lib/noise_reduction_util.h"
+#include "tensorflow/lite/experimental/microfrontend/lib/pcan_gain_control_util.h"
+#include "tensorflow/lite/experimental/microfrontend/lib/window_util.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct FrontendConfig {
+  struct WindowConfig window;
+  struct FilterbankConfig filterbank;
+  struct NoiseReductionConfig noise_reduction;
+  struct PcanGainControlConfig pcan_gain_control;
+  struct LogScaleConfig log_scale;
+};
+
+// Fills the frontendConfig with "sane" defaults.
+void FrontendFillConfigWithDefaults(struct FrontendConfig* config);
+
+// Allocates any buffers.
+int FrontendPopulateState(const struct FrontendConfig* config,
+                          struct FrontendState* state, int sample_rate);
+
+// Frees any allocated buffers.
+void FrontendFreeStateContents(struct FrontendState* state);
+
+#ifdef __cplusplus
+}  // extern "C"
+#endif
+
+#endif  // TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_FRONTEND_UTIL_H_

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/kiss_fft_common.h

@@ -0,0 +1,48 @@
+/* Copyright 2019 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+
+#ifndef TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_KISS_FFT_COMMON_H_
+#define TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_KISS_FFT_COMMON_H_
+
+// This header file should be included in all variants of kiss_fft_$type.{h,cc}
+// so that their sub-included source files do not mistakenly wrap libc header
+// files within their kissfft_$type namespaces.
+// E.g, This header avoids kissfft_int16.h containing:
+//   namespace kiss_fft_int16 {
+//     #include "kiss_fft.h"
+//   }
+// where kiss_fft_.h contains:
+//   #include <math.h>
+//
+// TRICK: By including the following header files here, their preprocessor
+// header guards prevent them being re-defined inside of the kiss_fft_$type
+// namespaces declared within the kiss_fft_$type.{h,cc} sources.
+// Note that the original kiss_fft*.h files are untouched since they
+// may be used in libraries that include them directly.
+
+#include <limits.h>
+#include <math.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#ifdef FIXED_POINT
+#include <sys/types.h>
+#endif
+
+#ifdef USE_SIMD
+#include <xmmintrin.h>
+#endif
+#endif  // TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_KISS_FFT_COMMON_H_

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/kiss_fft_int16.cc

@@ -0,0 +1,8 @@
+#include "tensorflow/lite/experimental/microfrontend/lib/kiss_fft_common.h"
+
+#define FIXED_POINT 16
+namespace kissfft_fixed16 {
+#include "kiss_fft.c"
+#include "tools/kiss_fftr.c"
+}  // namespace kissfft_fixed16
+#undef FIXED_POINT

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/kiss_fft_int16.h

@@ -0,0 +1,34 @@
+/* Copyright 2019 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+
+#ifndef TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_KISS_FFT_INT16_H_
+#define TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_KISS_FFT_INT16_H_
+
+#include "tensorflow/lite/experimental/microfrontend/lib/kiss_fft_common.h"
+
+// Wrap 16-bit kiss fft in its own namespace. Enables us to link an application
+// with different kiss fft resultions (16/32 bit interger, float, double)
+// without getting a linker error.
+#define FIXED_POINT 16
+namespace kissfft_fixed16 {
+#include "kiss_fft.h"
+#include "tools/kiss_fftr.h"
+}  // namespace kissfft_fixed16
+#undef FIXED_POINT
+#undef kiss_fft_scalar
+#undef KISS_FFT_H
+
+#endif  // TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_KISS_FFT_INT16_H_
+

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/log_lut.c

@@ -0,0 +1,30 @@
+/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+#include "tensorflow/lite/experimental/microfrontend/lib/log_lut.h"
+const uint16_t kLogLut[]
+#ifndef _MSC_VER
+    __attribute__((aligned(4)))
+#endif  // _MSV_VER
+    = {0,    224,  442,  654,  861,  1063, 1259, 1450, 1636, 1817, 1992, 2163,
+       2329, 2490, 2646, 2797, 2944, 3087, 3224, 3358, 3487, 3611, 3732, 3848,
+       3960, 4068, 4172, 4272, 4368, 4460, 4549, 4633, 4714, 4791, 4864, 4934,
+       5001, 5063, 5123, 5178, 5231, 5280, 5326, 5368, 5408, 5444, 5477, 5507,
+       5533, 5557, 5578, 5595, 5610, 5622, 5631, 5637, 5640, 5641, 5638, 5633,
+       5626, 5615, 5602, 5586, 5568, 5547, 5524, 5498, 5470, 5439, 5406, 5370,
+       5332, 5291, 5249, 5203, 5156, 5106, 5054, 5000, 4944, 4885, 4825, 4762,
+       4697, 4630, 4561, 4490, 4416, 4341, 4264, 4184, 4103, 4020, 3935, 3848,
+       3759, 3668, 3575, 3481, 3384, 3286, 3186, 3084, 2981, 2875, 2768, 2659,
+       2549, 2437, 2323, 2207, 2090, 1971, 1851, 1729, 1605, 1480, 1353, 1224,
+       1094, 963,  830,  695,  559,  421,  282,  142,  0,    0};

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/log_lut.h

@@ -0,0 +1,40 @@
+/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+#ifndef TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_LOG_LUT_H_
+#define TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_LOG_LUT_H_
+
+#include <stdint.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+// Number of segments in the log lookup table. The table will be kLogSegments+1
+// in length (with some padding).
+#define kLogSegments 128
+#define kLogSegmentsLog2 7
+
+// Scale used by lookup table.
+#define kLogScale 65536
+#define kLogScaleLog2 16
+#define kLogCoeff 45426
+
+extern const uint16_t kLogLut[];
+
+#ifdef __cplusplus
+}  // extern "C"
+#endif
+
+#endif  // TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_LOG_LUT_H_

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/log_scale.c

@@ -0,0 +1,83 @@
+/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+#include "tensorflow/lite/experimental/microfrontend/lib/log_scale.h"
+
+#include "tensorflow/lite/experimental/microfrontend/lib/bits.h"
+#include "tensorflow/lite/experimental/microfrontend/lib/log_lut.h"
+
+#define kuint16max 0x0000FFFF
+
+// The following functions implement integer logarithms of various sizes. The
+// approximation is calculated according to method described in
+//       www.inti.gob.ar/electronicaeinformatica/instrumentacion/utic/
+//       publicaciones/SPL2007/Log10-spl07.pdf
+// It first calculates log2 of the input and then converts it to natural
+// logarithm.
+
+static uint32_t Log2FractionPart(const uint32_t x, const uint32_t log2x) {
+  // Part 1
+  int32_t frac = x - (1LL << log2x);
+  if (log2x < kLogScaleLog2) {
+    frac <<= kLogScaleLog2 - log2x;
+  } else {
+    frac >>= log2x - kLogScaleLog2;
+  }
+  // Part 2
+  const uint32_t base_seg = frac >> (kLogScaleLog2 - kLogSegmentsLog2);
+  const uint32_t seg_unit =
+      (((uint32_t)1) << kLogScaleLog2) >> kLogSegmentsLog2;
+
+  const int32_t c0 = kLogLut[base_seg];
+  const int32_t c1 = kLogLut[base_seg + 1];
+  const int32_t seg_base = seg_unit * base_seg;
+  const int32_t rel_pos = ((c1 - c0) * (frac - seg_base)) >> kLogScaleLog2;
+  return frac + c0 + rel_pos;
+}
+
+static uint32_t Log(const uint32_t x, const uint32_t scale_shift) {
+  const uint32_t integer = MostSignificantBit32(x) - 1;
+  const uint32_t fraction = Log2FractionPart(x, integer);
+  const uint32_t log2 = (integer << kLogScaleLog2) + fraction;
+  const uint32_t round = kLogScale / 2;
+  const uint32_t loge = (((uint64_t)kLogCoeff) * log2 + round) >> kLogScaleLog2;
+  // Finally scale to our output scale
+  const uint32_t loge_scaled = ((loge << scale_shift) + round) >> kLogScaleLog2;
+  return loge_scaled;
+}
+
+uint16_t* LogScaleApply(struct LogScaleState* state, uint32_t* signal,
+                        int signal_size, int correction_bits) {
+  const int scale_shift = state->scale_shift;
+  uint16_t* output = (uint16_t*)signal;
+  uint16_t* ret = output;
+  int i;
+  for (i = 0; i < signal_size; ++i) {
+    uint32_t value = *signal++;
+    if (state->enable_log) {
+      if (correction_bits < 0) {
+        value >>= -correction_bits;
+      } else {
+        value <<= correction_bits;
+      }
+      if (value > 1) {
+        value = Log(value, scale_shift);
+      } else {
+        value = 0;
+      }
+    }
+    *output++ = (value < kuint16max) ? value : kuint16max;
+  }
+  return ret;
+}

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/log_scale.h

@@ -0,0 +1,39 @@
+/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+#ifndef TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_LOG_SCALE_H_
+#define TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_LOG_SCALE_H_
+
+#include <stdint.h>
+#include <stdlib.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct LogScaleState {
+  int enable_log;
+  int scale_shift;
+};
+
+// Applies a fixed point logarithm to the signal and converts it to 16 bit. Note
+// that the signal array will be modified.
+uint16_t* LogScaleApply(struct LogScaleState* state, uint32_t* signal,
+                        int signal_size, int correction_bits);
+
+#ifdef __cplusplus
+}  // extern "C"
+#endif
+
+#endif  // TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_LOG_SCALE_H_

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/log_scale_util.c

@@ -1,4 +1,4 @@
-/* Copyright 2020 The TensorFlow Authors. All Rights Reserved.
+/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -12,11 +12,16 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
 ==============================================================================*/
+#include "tensorflow/lite/experimental/microfrontend/lib/log_scale_util.h"
 
-#ifndef TENSORFLOW_LITE_MICRO_BENCHMARKS_KEYWORD_SCRAMBLED_MODEL_DATA_H_
-#define TENSORFLOW_LITE_MICRO_BENCHMARKS_KEYWORD_SCRAMBLED_MODEL_DATA_H_
+void LogScaleFillConfigWithDefaults(struct LogScaleConfig* config) {
+  config->enable_log = 1;
+  config->scale_shift = 6;
+}
 
-extern const unsigned char g_keyword_scrambled_model_data[];
-extern const unsigned int g_keyword_scrambled_model_data_length;
-
-#endif  // TENSORFLOW_LITE_MICRO_BENCHMARKS_KEYWORD_SCRAMBLED_MODEL_DATA_H_
+int LogScalePopulateState(const struct LogScaleConfig* config,
+                          struct LogScaleState* state) {
+  state->enable_log = config->enable_log;
+  state->scale_shift = config->scale_shift;
+  return 1;
+}

code/components/tflite-lib/tensorflow/lite/experimental/microfrontend/lib/log_scale_util.h

@@ -0,0 +1,45 @@
+/* Copyright 2018 The TensorFlow Authors. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+==============================================================================*/
+#ifndef TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_LOG_SCALE_UTIL_H_
+#define TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_LOG_SCALE_UTIL_H_
+
+#include <stdint.h>
+#include <stdlib.h>
+
+#include "tensorflow/lite/experimental/microfrontend/lib/log_scale.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct LogScaleConfig {
+  // set to false (0) to disable this module
+  int enable_log;
+  // scale results by 2^(scale_shift)
+  int scale_shift;
+};
+
+// Populates the LogScaleConfig with "sane" default values.
+void LogScaleFillConfigWithDefaults(struct LogScaleConfig* config);
+
+// Allocates any buffers.
+int LogScalePopulateState(const struct LogScaleConfig* config,
+                          struct LogScaleState* state);
+
+#ifdef __cplusplus
+}  // extern "C"
+#endif
+
+#endif  // TENSORFLOW_LITE_EXPERIMENTAL_MICROFRONTEND_LIB_LOG_SCALE_UTIL_H_