Master (#2104)

* Replace deprecated actions (#2016) * Update build.yaml * Update build.yaml * Update manual-update-webinstaller.yml * Update manual-update-webinstaller.yml * Update manual-update-webinstaller.yml * Update manual-update-webinstaller.yml * Update build.yaml * preparations for v15.0 (#2063) * Migrated parameters, see https://github.com/jomjol/AI-on-the-edge-device/pull/2023 * remove no longer used "topic" parameter. This is a backport from b21e3c6c9d * Fix wrong url-encoding, see https://github.com/jomjol/AI-on-the-edge-device/issues/2036 resp. https://github.com/jomjol/AI-on-the-edge-device/issues/2036 * Threashold -> Threshold * updated changelog --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> * DataLogActive is true by default * updated changelog * Bugfix #1933 (again :-)) * Update Influxdb * re-add missing dropdownbox filling for Postprocessing Individual Parameters * stop auto filling the release notes, it causes more confusion than it helps * Update Changelog.md --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> Co-authored-by: jomjol <30766535+jomjol@users.noreply.github.com>
2025-12-06 11:36:51 +03:00 · 2023-03-01 20:54:14 +01:00
parent a7dc37761b
commit 8c7e86fea4
4 changed files with 274 additions and 15 deletions
--- a/.github/workflows/build.yaml
+++ b/.github/workflows/build.yaml
@@ -343,13 +343,13 @@ jobs:
 #        changelogPath: Changelog.md
 #        version: ${{ steps.get_version.outputs.version-without-v }}
            
-    # the release notes will be extracted from changelog 
-    - name: Extract release notes
-      id: extract-release-notes
-      if: startsWith(github.ref, 'refs/tags/') 
-      uses: ffurrer2/extract-release-notes@v1
-      with:
-          changelog_file: Changelog.md
+#    # the release notes will be extracted from changelog 
+#    - name: Extract release notes
+#      id: extract-release-notes
+#      if: startsWith(github.ref, 'refs/tags/') 
+#      uses: ffurrer2/extract-release-notes@v1
+#      with:
+#          changelog_file: Changelog.md

    # Releases should only be created on master by tagging the last commit.
    # all artifacts in firmware folder pushed to the release
--- a/Changelog.md
+++ b/Changelog.md
@@ -1,8 +1,22 @@
 ## [Unreleased]

-**Rolling**
+xxx

+### Update Procedure

+Update Procedure see [online documentation](https://jomjol.github.io/AI-on-the-edge-device-docs/Installation/#update-ota-over-the-air)
+
+:bangbang: Afterwards you should force-reload the Web Interface (usually Ctrl-F5 will do it).
+
+### Changes
+
+This release only migrates some parameters, see #2023 for details and a list of all parameter changes.
+The parameter migration happens automatically on the next startup. No user interaction is required.
+A backup of the config is stored on the SD-card as `config.bak`.
+
+Beside of the parameter change and the bugfix listed below, no changes are contained in this release!
+
+If you want to revert back to `v14` or earlier, you will have to revert the migration changes in `config.ini` manually!

 #### Added

@@ -14,18 +28,18 @@

 #### Changed

-   n.a.
+-   [#2023](https://github.com/jomjol/AI-on-the-edge-device/pull/2023) Migrated Parameters
+-   Removed old `Topic` parameter, it is not used anymore

 #### Fixed
-
-   
+-  n.a.

 #### Removed

 -   n.a.


-## [15.0.1] - 2023-02-23
+## [15.0.3] - 2023-02-28

 **Parameter Migration**

@@ -33,6 +47,8 @@

 Update Procedure see [online documentation](https://jomjol.github.io/AI-on-the-edge-device-docs/Installation/#update-ota-over-the-air)

+:bangbang: Afterwards you should force-reload the Web Interface (usually Ctrl-F5 will do it).
+
 ### Changes

 This release only migrates some parameters, see #2023 for details and a list of all parameter changes.
@@ -54,7 +70,13 @@ If you want to revert back to `v14` or earlier, you will have to revert the migr

 #### Fixed

-   [2036](https://github.com/jomjol/AI-on-the-edge-device/issues/2036) Fix wrong url-encoding
+-   [#2036](https://github.com/jomjol/AI-on-the-edge-device/issues/2036) Fix wrong url-encoding
+-   **NEW v15.0.2:**  [#1933](https://github.com/jomjol/AI-on-the-edge-device/issues/1933) Bugfix InfluxDB Timestamp
+-   **NEW v15.0.3:**  Re-added lost dropdownbox filling for Postprocessing Individual Parameters
+
+#### Removed
+
+-   n.a.


 ## [14.0.3] -2023-02-05
@@ -106,7 +128,6 @@ For a full list of changes see [Full list of changes](https://github.com/jomjol/
 -   [1530](https://github.com/jomjol/AI-on-the-edge-device/pull/1530) Homeassistant `Problem Sensor`
 -   [1518](https://github.com/jomjol/AI-on-the-edge-device/pull/1518) JSON Strings
 -   [1817](https://github.com/jomjol/AI-on-the-edge-device/pull/1817) DataGraph: datafiles sorted -> newest on top
-   **New 14.0.4:** Fix for InfluxDB timeshift problem [#1991](https://github.com/jomjol/AI-on-the-edge-device/issues/1991)

 #### Removed

@@ -862,7 +883,7 @@ External Illumination
 -   Initial Version


-[15.0.1]: https://github.com/jomjol/AI-on-the-edge-device/compare/v14.0.3...v15.0.1
+[15.0.3]: https://github.com/jomjol/AI-on-the-edge-device/compare/v14.0.3...v15.0.3
 [14.0.3]: https://github.com/jomjol/AI-on-the-edge-device/compare/v13.0.8...v14.0.3
 [13.0.8]: https://github.com/jomjol/AI-on-the-edge-device/compare/v12.0.1...v13.0.8
 [13.0.7]: https://github.com/jomjol/AI-on-the-edge-device/compare/v12.0.1...v13.0.7
--- a/code/components/jomjol_influxdb/interface_influxdb._c_pp_old
+++ b/code/components/jomjol_influxdb/interface_influxdb._c_pp_old
@@ -0,0 +1,214 @@
+#ifdef ENABLE_INFLUXDB
+#include "interface_influxdb.h"
+
+#include "esp_log.h"
+#include <time.h>
+#include "ClassLogFile.h"
+#include "esp_http_client.h"
+#include "../../include/defines.h"
+
+
+static const char *TAG = "INFLUXDB";
+
+std::string _influxDBURI;
+std::string _influxDBDatabase;
+std::string _influxDBMeasurement;
+std::string _influxDBUser;
+std::string _influxDBPassword;
+
+static esp_err_t http_event_handler(esp_http_client_event_t *evt)
+{
+    switch(evt->event_id)
+    {
+        case HTTP_EVENT_ERROR:
+            LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client Error encountered");
+            break;
+        case HTTP_EVENT_ON_CONNECTED:
+            LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client Error encountered");
+            ESP_LOGI(TAG, "HTTP Client Connected");
+            break;
+        case HTTP_EVENT_HEADERS_SENT:
+            LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client sent all request headers");
+            break;
+        case HTTP_EVENT_ON_HEADER:
+            LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "Header: key=" + std::string(evt->header_key) + ", value="  + std::string(evt->header_value));
+            break;
+        case HTTP_EVENT_ON_DATA:
+            LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client data recevied: len=" + std::to_string(evt->data_len));
+            break;
+        case HTTP_EVENT_ON_FINISH:
+            LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client finished");
+            break;
+         case HTTP_EVENT_DISCONNECTED:
+            LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client Disconnected");
+            break;
+    }
+    return ESP_OK;
+}
+
+void InfluxDBPublish(std::string _key, std::string _content, std::string _timestamp) {
+    char response_buffer[MAX_HTTP_OUTPUT_BUFFER] = {0};
+    esp_http_client_config_t http_config = {
+       .user_agent = "ESP32 Meter reader",
+       .method = HTTP_METHOD_POST,
+       .event_handler = http_event_handler,
+       .buffer_size = MAX_HTTP_OUTPUT_BUFFER,
+       .user_data = response_buffer
+    };
+
+    if (_influxDBUser.length() && _influxDBPassword.length()){
+       http_config.username = _influxDBUser.c_str();
+       http_config.password = _influxDBPassword.c_str();
+       http_config.auth_type = HTTP_AUTH_TYPE_BASIC;
+    }
+
+    LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "InfluxDBPublish - Key: " + _key + ", Content: " + _content + ", Timestamp: " + _timestamp);
+
+    char nowTimestamp[21];
+    std::string payload;
+
+    if (_timestamp.length() > 0)
+    {
+        struct tm tm;
+        strptime(_timestamp.c_str(), PREVALUE_TIME_FORMAT_OUTPUT, &tm);
+        time_t t = mktime(&tm); // Time in Localtime (looks like timezone is not used by strptime)
+
+        struct tm * ptm;
+        ptm = gmtime ( &t );
+        time_t utc = mktime(ptm);
+        utc = 2*t - utc;        // Take care of timezone (looks difficult, but is easy: t = t + (t - utc), weil t-utc = timezone)
+
+        sprintf(nowTimestamp,"%ld000000000", (long) utc);           // UTC
+
+        payload = _influxDBMeasurement + " " + _key + "=" + _content + " " + nowTimestamp;
+//        payload = _influxDBMeasurement + " " + _key + "=774 " + nowTimestamp;
+    }
+    else
+    {
+        payload = _influxDBMeasurement + " " + _key + "=" + _content;
+    }
+
+    payload.shrink_to_fit();
+
+    LogFile.WriteToFile(ESP_LOG_INFO, TAG, "sending line to influxdb:" + payload);
+
+
+    // use the default retention policy of the database
+    std::string apiURI = _influxDBURI + "/api/v2/write?bucket=" + _influxDBDatabase + "/";
+    apiURI.shrink_to_fit();
+    http_config.url = apiURI.c_str();
+
+    LogFile.WriteToFile(ESP_LOG_INFO, TAG, "API URI: " + apiURI);
+
+    esp_http_client_handle_t http_client = esp_http_client_init(&http_config);
+    LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "client is initialized");
+
+    esp_http_client_set_header(http_client, "Content-Type", "text/plain");
+    LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "header is set");
+
+    ESP_ERROR_CHECK(esp_http_client_set_post_field(http_client, payload.c_str(), payload.length()));
+    LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "post payload is set");
+
+    esp_err_t err = ESP_ERROR_CHECK_WITHOUT_ABORT(esp_http_client_perform(http_client));
+
+    if( err == ESP_OK ) {
+      LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP request was performed");
+      int status_code = esp_http_client_get_status_code(http_client);
+      LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP status code" + std::to_string(status_code));
+    } else {
+      LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP request failed");
+    }
+    esp_http_client_cleanup(http_client);
+}
+
+
+
+/*
+void InfluxDBPublish(std::string _key, std::string _content, std::string _timestamp) {
+    char response_buffer[MAX_HTTP_OUTPUT_BUFFER] = {0};
+    esp_http_client_config_t http_config = {
+       .user_agent = "ESP32 Meter reader",
+       .method = HTTP_METHOD_POST,
+       .event_handler = http_event_handler,
+       .buffer_size = MAX_HTTP_OUTPUT_BUFFER,
+       .user_data = response_buffer
+    };
+
+    if (_influxDBUser.length() && _influxDBPassword.length()){
+       http_config.username = _influxDBUser.c_str();
+       http_config.password = _influxDBPassword.c_str();
+       http_config.auth_type = HTTP_AUTH_TYPE_BASIC;
+    }
+
+    LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "InfluxDBPublish - Key: " + _key + ", Content: " + _content + ", Timestamp: " + _timestamp);
+
+    // Format:     #define PREVALUE_TIME_FORMAT_OUTPUT "%Y-%m-%dT%H:%M:%S%z"
+    struct tm tm;
+    strptime(_timestamp.c_str(), PREVALUE_TIME_FORMAT_OUTPUT, &tm);
+    time_t t = mktime(&tm);  // t is now your desired time_t
+
+    struct tm * ptm;
+    ptm = gmtime ( &t );
+    time_t utc = mktime(ptm);
+
+//    time_t now;
+//    time(&now);
+    char nowTimestamp[21];
+    // pad with zeroes to get nanoseconds
+//    sprintf(nowTimestamp,"%ld000000000", (long) now);
+//    sprintf(nowTimestamp,"%ld000000000", (long) t);           // Localtime
+    sprintf(nowTimestamp,"%ld000000000", (long) utc);           // UTC
+    
+
+//    LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "Test Time Conversion - t: " + std::to_string(t) + ", utc: " + std::to_string(utc));
+//    LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "Test Time Conversion - now: " + std::to_string(now) + ", timestamp: " + std::to_string(t)  + "(correct time not used yet)");
+
+    std::string payload = _influxDBMeasurement + " " + _key + "=" + _content + " " + nowTimestamp;
+    payload.shrink_to_fit();
+
+    LogFile.WriteToFile(ESP_LOG_INFO, TAG, "sending line to influxdb:" + payload);
+
+
+    // use the default retention policy of the database
+    std::string apiURI = _influxDBURI + "/api/v2/write?bucket=" + _influxDBDatabase + "/";
+    apiURI.shrink_to_fit();
+    http_config.url = apiURI.c_str();
+
+    LogFile.WriteToFile(ESP_LOG_INFO, TAG, "API URI: " + apiURI);
+
+    esp_http_client_handle_t http_client = esp_http_client_init(&http_config);
+    LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "client is initialized");
+
+    esp_http_client_set_header(http_client, "Content-Type", "text/plain");
+    LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "header is set");
+
+    ESP_ERROR_CHECK(esp_http_client_set_post_field(http_client, payload.c_str(), payload.length()));
+    LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "post payload is set");
+
+    esp_err_t err = ESP_ERROR_CHECK_WITHOUT_ABORT(esp_http_client_perform(http_client));
+
+    if( err == ESP_OK ) {
+      LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP request was performed");
+      int status_code = esp_http_client_get_status_code(http_client);
+      LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP status code" + std::to_string(status_code));
+    } else {
+      LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP request failed");
+    }
+    esp_http_client_cleanup(http_client);
+}
+*/
+
+
+void InfluxDBInit(std::string _uri, std::string _database, std::string _measurement, std::string _user, std::string _password){
+    _influxDBURI = _uri;
+    _influxDBDatabase = _database;
+    _influxDBMeasurement = _measurement;
+    _influxDBUser = _user;
+    _influxDBPassword = _password;
+ 
+}
+
+void InfluxDBdestroy() {
+}
+
+#endif //ENABLE_INFLUXDB
--- a/code/components/jomjol_influxdb/interface_influxdb._h_old
+++ b/code/components/jomjol_influxdb/interface_influxdb._h_old
@@ -0,0 +1,24 @@
+#ifdef ENABLE_INFLUXDB
+
+#pragma once
+#ifndef INTERFACE_INFLUXDB_H
+#define INTERFACE_INFLUXDB_H
+
+#include <string>
+#include <map>
+#include <functional>
+
+// Interface to InfluxDB v1.x
+void InfluxDBInit(std::string _influxDBURI, std::string _database, std::string _measurement, std::string _user, std::string _password);
+void InfluxDBPublish(std::string _key, std::string _content, std::string _timestamp);
+
+// Interface to InfluxDB v2.x
+void InfluxDB_V2_Init(std::string _uri, std::string _database, std::string _measurement, std::string _org, std::string _token);
+void InfluxDB_V2_Publish(std::string _key, std::string _content, std::string _timestamp);
+
+
+
+void InfluxDBdestroy();
+
+#endif //INTERFACE_INFLUXDB_H
+#endif //ENABLE_INFLUXDB