mirror of
https://github.com/jomjol/AI-on-the-edge-device.git
synced 2025-12-12 14:37:06 +03:00
cbd14a267f
* Testcase for #2145 and debug-log (#2151) * new models ana-cont-11.0.5, ana-class100-1.5.7, dig-class100-1.6.0 * Testcase for #2145 Added debug log, if allowNegativeRates is handeled * Fix timezone config parser (#2169) * make sure to parse the whole config line * fix crash on empty timezone parameter --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> * Enhance ROI pages (#2161) * Check if the ROIs are equidistant. Only if not, untick the checkbox * renaming * Check if the ROIs have same y, dy and dx. If so, tick the sync checkbox * only allow editing space when box is checked * fix sync check * show inner frame on all ROIs * cleanup * Check if the ROIs have same dy and dx. If so, tick the sync checkbox * checkbox position * renaming * renaming * show inner frame and cross hairs on all ROIs * update ROIs on ticking checkboxes * show timezone hint * fix deleting last ROI * cleanup --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> * restart timeout on progress, catch error (#2170) * restart timeout on progress, catch error * . --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> * BugFix #2167 * Release 15.1 preparations (#2171) * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update changelog * Fix links to PR * Formating * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update Changelog.md --------- Co-authored-by: Slider0007 <jobbelle@gmx.net> Co-authored-by: Slider0007 <115730895+Slider0007@users.noreply.github.com> * fix typo * Replace relative documentation links with absolute ones pointing to the external documentation (#2180) Co-authored-by: CaCO3 <caco@ruinelli.ch> * Sort model files in configuration combobox (#2189) * new models ana-cont-11.0.5, ana-class100-1.5.7, dig-class100-1.6.0 * Testcase for #2145 Added debug log, if allowNegativeRates is handeled * Sort model files in combobox * reboot task - increase stack size (#2201) Avoid stack overflow * Update interface_influxdb.cpp * Update Changelog.md * Show PSRAM usage (#2206) * centralize PSRAM usage (application code only) * update logging * update logging * fix use after free * initialize buffer * free rgb_image before ussing it for new allocation * use wrapper function * switch log level to debug * . * undo adding free() calls * . * add names to all CImage instances * . * . * . * revert changes of stbi_image_free() with free_psram_heap() on the places where is is not in PSRAM * . * typos * typo * Added MQTT Outbox explanation/warning * added CONFIG_SPIRAM_USE_MEMMAP explanation --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> * Disable custom MQTT Outbox. This also moves the MQTT Publishing memory usage back to the internal RAM! * log MQTT connection refused reasons (#2216) * Revert PSRAM usage as it lead to memory fragmentation. See https://github.com/jomjol/AI-on-the-edge-device/issues/2200 for details * fix missing value data * Revert PSRAM usage as it lead to memory fragmentation. (#2224) See https://github.com/jomjol/AI-on-the-edge-device/issues/2200 for details Co-authored-by: CaCO3 <caco@ruinelli.ch> * Fix missing value data in graph (#2230) * fix missing value data --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> * Update Changelog.md (#2231) * Update interface_influxdb.cpp (#2233) * update copyright year * Cleanup * Set prevalue using MQTT + set prevalue to RAW value (REST+MQTT) (#2252) * Use double instead of float * Error handling + set to RAW if newvalue < 0 * REST SetPrevalue: Set to RAW if newvalue < 0 * set prevalue with MQTT * removed the stb_image files and re-add them as a submodule. (#2223) - stb_image.h: Version update 2.25 -> 2.28 - stb_resize.h: Version update 0.96 -> 0.97 - stb_write.h: Version update 1.14 -> 1.16 Co-authored-by: CaCO3 <caco@ruinelli.ch> * Remove obsolete ClassFlowWriteList (#2264) * Renaming & cleanup of some modules / functions in source code (#2265) * Rename module tag name * Rename server_tflite.cpp -> MainFlowControl.cpp * Remove redundandant MQTTMainTopic function * Update * Remove obsolete GetMQTTMainTopic * Fix last element missing in digit model drop down (#2282) * Debug influxdb (#2283) * Fix time offset issues in InfluxDB component. (#2278) Closes #2273 Closes #2150 * Update interface_influxdb.cpp * Update interface_influxdb.cpp * Improve Logging * Implement TimeSync at beginning * Update time_sntp.cpp * Update time_sntp.cpp * Set Time After WLAN Init --------- Co-authored-by: Antonin Delpeuch <antonin@delpeuch.eu> * Implement a camera livestream handler (#2286) * fix leading NaN (#2310) * analogROI: Activate save button after ROI creation (#2326) * Migration of PlatformIO 5.2.0 to 6.1.0 (resp. ESP IDF from 4.4.2 to 5.0.1) (#2305) * Migration to PlatformIO 6.1.0 * Disable RMTMEM usage as it is no longer allowed -> Smart LEDs not functional! * moved miniz into subfolder of jomjol_fileserver_ota, else it does not build anymore. * cleanup * fix leading NaN (#2310) * Migration to PlatformIO 6.1.0 * Disable RMTMEM usage as it is no longer allowed -> Smart LEDs not functional! * moved miniz into subfolder of jomjol_fileserver_ota, else it does not build anymore. * cleanup * Task watchdog has new config name * Fix return value check. It must be something else than ESP_FAIL, but it does not need to be ESP_OK! * add missing strucures to work around new RMTMEM restriction (untested) --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> * Keep MainFlowTask alive to handle reboot (#2325) * Shared PSRAM memory (#2285) * enable PSRAM logging * add extra functions for psram shared memroy handling * CImageBasis objects still should used dynamic memory (eg. rawImage), haw ever tmpImage must be placed inside the shared memory * Place all STBI allocs inside the shared memory * The models are placed in the shared PSRAM reagion and must be allocated through the dedicated functions * . * renaming * fix cast warning * add flag to switch STBI PSRAM usage * improve PSRAM shared handling * reserve shared PSRAM as early as possible * init logging eralier so we can use it in PSRAM shared alloc * move Wifi_LWIP, BSS_SEG and MQTT Outbox into PSRAM to ffree internal memory * Check if model fits into reserved shared memory * Update code/components/jomjol_tfliteclass/CTfLiteClass.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_flowcontroll/ClassFlowControll.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_image_proc/CImageBasis.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * . * . * . * . * Korrektur Merge Conflict in main.cpp --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> Co-authored-by: jomjol <30766535+jomjol@users.noreply.github.com> * fix PSRAM init return value check * Extend InfluxDBv1 with individual topic names (#2319) * Implement individual influx topic * Update interface_influxdb.cpp * Update interface_influxdb.cpp * Update FieldName * analogROI: Activate save button after ROI creation (#2326) * Migration of PlatformIO 5.2.0 to 6.1.0 (resp. ESP IDF from 4.4.2 to 5.0.1) (#2305) * Migration to PlatformIO 6.1.0 * Disable RMTMEM usage as it is no longer allowed -> Smart LEDs not functional! * moved miniz into subfolder of jomjol_fileserver_ota, else it does not build anymore. * cleanup * fix leading NaN (#2310) * Migration to PlatformIO 6.1.0 * Disable RMTMEM usage as it is no longer allowed -> Smart LEDs not functional! * moved miniz into subfolder of jomjol_fileserver_ota, else it does not build anymore. * cleanup * Task watchdog has new config name * Fix return value check. It must be something else than ESP_FAIL, but it does not need to be ESP_OK! * add missing strucures to work around new RMTMEM restriction (untested) --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> * Keep MainFlowTask alive to handle reboot (#2325) * Shared PSRAM memory (#2285) * enable PSRAM logging * add extra functions for psram shared memroy handling * CImageBasis objects still should used dynamic memory (eg. rawImage), haw ever tmpImage must be placed inside the shared memory * Place all STBI allocs inside the shared memory * The models are placed in the shared PSRAM reagion and must be allocated through the dedicated functions * . * renaming * fix cast warning * add flag to switch STBI PSRAM usage * improve PSRAM shared handling * reserve shared PSRAM as early as possible * init logging eralier so we can use it in PSRAM shared alloc * move Wifi_LWIP, BSS_SEG and MQTT Outbox into PSRAM to ffree internal memory * Check if model fits into reserved shared memory * Update code/components/jomjol_tfliteclass/CTfLiteClass.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_flowcontroll/ClassFlowControll.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_image_proc/CImageBasis.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * . * . * . * . * Korrektur Merge Conflict in main.cpp --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> Co-authored-by: jomjol <30766535+jomjol@users.noreply.github.com> * fix PSRAM init return value check * Extend incl. indiv. Measurement * Implement UX * Update ClassFlowInfluxDBv2.cpp * Implement individual influx topic * Update interface_influxdb.cpp * Update interface_influxdb.cpp * Update FieldName * Extend incl. indiv. Measurement * Implement UX * Update ClassFlowInfluxDBv2.cpp * Update main.cpp --------- Co-authored-by: Slider0007 <115730895+Slider0007@users.noreply.github.com> Co-authored-by: CaCO3 <caco3@ruinelli.ch> Co-authored-by: CaCO3 <caco@ruinelli.ch> * Update Changelog.md (#2344) * Update Changelog.md * Update Changelog.md * merge conflicts * merge conflicts * update changelog * Update Changelog.md * Fix alignment mark PSRAM issue (#2357) * Add lock for shared PSRAM memory, use it for the relevant round steps and the Alignment Mark Task * only allow taking a new image for the Alignment Mark while round got completed * show success/denial of Alignment Mark Update Request * . --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> * Update Changelog.md (#2360) * Update platformio.ini (#2368) * Allow the Alignment Mark step while status is "Initializing" or "Initialization (delayed)" or while in setup mode (#2373) Co-authored-by: CaCO3 <caco@ruinelli.ch> * Improve logging if Autostart is not enabled. Renamed/splitted functions (#2376) Co-authored-by: CaCO3 <caco@ruinelli.ch> * Cleanup web UI (#2378) * fix wrong index on backup log * remove no longer used files/functions * improve logging on Alignment Mark Update --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> * Update Changelog.md * fix broken sysinfo (#2381) Co-authored-by: CaCO3 <caco@ruinelli.ch> * Update Changelog.md * Update Changelog.md * Update sdcard_check.cpp (#2384) * Update Changelog.md --------- Co-authored-by: Frank Haverland <fspapaping@googlemail.com> Co-authored-by: CaCO3 <caco@ruinelli.ch> Co-authored-by: jomjol <30766535+jomjol@users.noreply.github.com> Co-authored-by: Slider0007 <jobbelle@gmx.net> Co-authored-by: Slider0007 <115730895+Slider0007@users.noreply.github.com> Co-authored-by: Antonin Delpeuch <antonin@delpeuch.eu>
Welcome to the AI-on-the-edge-device
Artificial intelligence based systems have been established in our every days live. Just think of speech or image recognition. Most of the systems relay on either powerful processors or a direct connection to the cloud for doing the calculations up there. With the increasing power of modern processors the AI systems are coming closer to the end user - which is usually called edge computing. Here this edge computing is brought into a practical oriented example, where a AI network is implemented on a ESP32 device so: AI on the edge.
This projects allows you to digitalize your analoge water, gas, power and other meters using cheap and easily available hardware.
All you need is an ESP32 board with a supported camera and a bit of a practical hand.
Key features
- Tensorflow Lite (TFlite) integration - including easy to use wrapper
- Inline Image processing (feature detection, alignment, ROI extraction)
- Small and cheap device (3x4.5x2 cm³, < 10 EUR)
- camera and illumination integrated
- Web surface to administrate and control
- OTA-Interface to update directly through the web interface
- Full integration into Homeassistant
- Support for Influx DB 1
- MQTT
- REST API
Workflow
The device takes a photo of your meter at a defined interval. It then extracts the Regions of Interest (ROI's) out of it and runs them through an artificial inteligence. As a result, you get the digitalized value of your meter.
There are several options what to do with that value. Either send it to a MQTT broker, write it to an InfluxDb or simply provide it throug a REST API.
Impressions
AI-on-the-edge-device on a Water Meter
Web Interface (Water Meter)
AI-on-the-edge-device on a Electrical Power Meter
Setup
There is a growing documentation which provides you with a lot of information. Head there to get a start, set it up and configure it.
There are also a articles in the German Heise magazine "make:" about the setup and the technical background (behind a paywall) : DIY - Setup
For further background information, head to Neural Networks, Training Neural Networks and Programming on the ESP32
Download
The latest available version is available on the Releases page.
Flashing of the ESP32
Initially you will have to flash the ESP32 through an USB connection. Later an update is possible directly over the Air (OTA).
There are different ways to flash your ESP32:
- Web Installer and Console (Webbrowser based tool to flash the ESP32 and extract the Log over USB)
- Flash Tool from Espressif
- ESPtool (Command Line Tool)
See the Docu for more information.
Flashing the SD-Card
The SD-Card must be flashed separately, see the Docu for details.
Casing
A 3d-printable housing can be found here:
- https://www.thingiverse.com/thing:4573481 (Water Meter)
- https://www.thingiverse.com/thing:5028229 (Power Meter)
- https://www.thingiverse.com/thing:5224101 (Gas Meter)
- https://www.thingiverse.com/thing:4571627 (ESP32-Cam housing only)
Build it yourself
See Build Instructions.
Donate
If you would like to support the developer with a cup of coffee you can do that via Paypal.
If you have any technical topics, you can create an [Issue](https://github.com/jomjol/AI-on-the-edge-device/issues).
In other cases you can contact the developer via email: 
Changes and History
See Changelog
Tools
- Logfile downloader and combiner (Thx to reserve85)
- Files see '/tools/logfile-tool', How-to see Docu
Additional Ideas
There are some ideas and feature requests which are not followed currently - mainly due to capacity reasons on side of the developer. They are collected here: FeatureRequest.md