mirror of
https://github.com/jomjol/AI-on-the-edge-device.git
synced 2025-12-07 20:16:55 +03:00
* Testcase for #2145 and debug-log (#2151) * new models ana-cont-11.0.5, ana-class100-1.5.7, dig-class100-1.6.0 * Testcase for #2145 Added debug log, if allowNegativeRates is handeled * Fix timezone config parser (#2169) * make sure to parse the whole config line * fix crash on empty timezone parameter --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> * Enhance ROI pages (#2161) * Check if the ROIs are equidistant. Only if not, untick the checkbox * renaming * Check if the ROIs have same y, dy and dx. If so, tick the sync checkbox * only allow editing space when box is checked * fix sync check * show inner frame on all ROIs * cleanup * Check if the ROIs have same dy and dx. If so, tick the sync checkbox * checkbox position * renaming * renaming * show inner frame and cross hairs on all ROIs * update ROIs on ticking checkboxes * show timezone hint * fix deleting last ROI * cleanup --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> * restart timeout on progress, catch error (#2170) * restart timeout on progress, catch error * . --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> * BugFix #2167 * Release 15.1 preparations (#2171) * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update changelog * Fix links to PR * Formating * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update Changelog.md * Update Changelog.md --------- Co-authored-by: Slider0007 <jobbelle@gmx.net> Co-authored-by: Slider0007 <115730895+Slider0007@users.noreply.github.com> * fix typo * Replace relative documentation links with absolute ones pointing to the external documentation (#2180) Co-authored-by: CaCO3 <caco@ruinelli.ch> * Sort model files in configuration combobox (#2189) * new models ana-cont-11.0.5, ana-class100-1.5.7, dig-class100-1.6.0 * Testcase for #2145 Added debug log, if allowNegativeRates is handeled * Sort model files in combobox * reboot task - increase stack size (#2201) Avoid stack overflow * Update interface_influxdb.cpp * Update Changelog.md * Show PSRAM usage (#2206) * centralize PSRAM usage (application code only) * update logging * update logging * fix use after free * initialize buffer * free rgb_image before ussing it for new allocation * use wrapper function * switch log level to debug * . * undo adding free() calls * . * add names to all CImage instances * . * . * . * revert changes of stbi_image_free() with free_psram_heap() on the places where is is not in PSRAM * . * typos * typo * Added MQTT Outbox explanation/warning * added CONFIG_SPIRAM_USE_MEMMAP explanation --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> * Disable custom MQTT Outbox. This also moves the MQTT Publishing memory usage back to the internal RAM! * log MQTT connection refused reasons (#2216) * Revert PSRAM usage as it lead to memory fragmentation. See https://github.com/jomjol/AI-on-the-edge-device/issues/2200 for details * fix missing value data * Revert PSRAM usage as it lead to memory fragmentation. (#2224) See https://github.com/jomjol/AI-on-the-edge-device/issues/2200 for details Co-authored-by: CaCO3 <caco@ruinelli.ch> * Fix missing value data in graph (#2230) * fix missing value data --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> * Update Changelog.md (#2231) * Update interface_influxdb.cpp (#2233) * update copyright year * Cleanup * Set prevalue using MQTT + set prevalue to RAW value (REST+MQTT) (#2252) * Use double instead of float * Error handling + set to RAW if newvalue < 0 * REST SetPrevalue: Set to RAW if newvalue < 0 * set prevalue with MQTT * removed the stb_image files and re-add them as a submodule. (#2223) - stb_image.h: Version update 2.25 -> 2.28 - stb_resize.h: Version update 0.96 -> 0.97 - stb_write.h: Version update 1.14 -> 1.16 Co-authored-by: CaCO3 <caco@ruinelli.ch> * Remove obsolete ClassFlowWriteList (#2264) * Renaming & cleanup of some modules / functions in source code (#2265) * Rename module tag name * Rename server_tflite.cpp -> MainFlowControl.cpp * Remove redundandant MQTTMainTopic function * Update * Remove obsolete GetMQTTMainTopic * Fix last element missing in digit model drop down (#2282) * Debug influxdb (#2283) * Fix time offset issues in InfluxDB component. (#2278) Closes #2273 Closes #2150 * Update interface_influxdb.cpp * Update interface_influxdb.cpp * Improve Logging * Implement TimeSync at beginning * Update time_sntp.cpp * Update time_sntp.cpp * Set Time After WLAN Init --------- Co-authored-by: Antonin Delpeuch <antonin@delpeuch.eu> * Implement a camera livestream handler (#2286) * fix leading NaN (#2310) * analogROI: Activate save button after ROI creation (#2326) * Migration of PlatformIO 5.2.0 to 6.1.0 (resp. ESP IDF from 4.4.2 to 5.0.1) (#2305) * Migration to PlatformIO 6.1.0 * Disable RMTMEM usage as it is no longer allowed -> Smart LEDs not functional! * moved miniz into subfolder of jomjol_fileserver_ota, else it does not build anymore. * cleanup * fix leading NaN (#2310) * Migration to PlatformIO 6.1.0 * Disable RMTMEM usage as it is no longer allowed -> Smart LEDs not functional! * moved miniz into subfolder of jomjol_fileserver_ota, else it does not build anymore. * cleanup * Task watchdog has new config name * Fix return value check. It must be something else than ESP_FAIL, but it does not need to be ESP_OK! * add missing strucures to work around new RMTMEM restriction (untested) --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> * Keep MainFlowTask alive to handle reboot (#2325) * Shared PSRAM memory (#2285) * enable PSRAM logging * add extra functions for psram shared memroy handling * CImageBasis objects still should used dynamic memory (eg. rawImage), haw ever tmpImage must be placed inside the shared memory * Place all STBI allocs inside the shared memory * The models are placed in the shared PSRAM reagion and must be allocated through the dedicated functions * . * renaming * fix cast warning * add flag to switch STBI PSRAM usage * improve PSRAM shared handling * reserve shared PSRAM as early as possible * init logging eralier so we can use it in PSRAM shared alloc * move Wifi_LWIP, BSS_SEG and MQTT Outbox into PSRAM to ffree internal memory * Check if model fits into reserved shared memory * Update code/components/jomjol_tfliteclass/CTfLiteClass.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_flowcontroll/ClassFlowControll.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_image_proc/CImageBasis.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * . * . * . * . * Korrektur Merge Conflict in main.cpp --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> Co-authored-by: jomjol <30766535+jomjol@users.noreply.github.com> * fix PSRAM init return value check * Extend InfluxDBv1 with individual topic names (#2319) * Implement individual influx topic * Update interface_influxdb.cpp * Update interface_influxdb.cpp * Update FieldName * analogROI: Activate save button after ROI creation (#2326) * Migration of PlatformIO 5.2.0 to 6.1.0 (resp. ESP IDF from 4.4.2 to 5.0.1) (#2305) * Migration to PlatformIO 6.1.0 * Disable RMTMEM usage as it is no longer allowed -> Smart LEDs not functional! * moved miniz into subfolder of jomjol_fileserver_ota, else it does not build anymore. * cleanup * fix leading NaN (#2310) * Migration to PlatformIO 6.1.0 * Disable RMTMEM usage as it is no longer allowed -> Smart LEDs not functional! * moved miniz into subfolder of jomjol_fileserver_ota, else it does not build anymore. * cleanup * Task watchdog has new config name * Fix return value check. It must be something else than ESP_FAIL, but it does not need to be ESP_OK! * add missing strucures to work around new RMTMEM restriction (untested) --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> * Keep MainFlowTask alive to handle reboot (#2325) * Shared PSRAM memory (#2285) * enable PSRAM logging * add extra functions for psram shared memroy handling * CImageBasis objects still should used dynamic memory (eg. rawImage), haw ever tmpImage must be placed inside the shared memory * Place all STBI allocs inside the shared memory * The models are placed in the shared PSRAM reagion and must be allocated through the dedicated functions * . * renaming * fix cast warning * add flag to switch STBI PSRAM usage * improve PSRAM shared handling * reserve shared PSRAM as early as possible * init logging eralier so we can use it in PSRAM shared alloc * move Wifi_LWIP, BSS_SEG and MQTT Outbox into PSRAM to ffree internal memory * Check if model fits into reserved shared memory * Update code/components/jomjol_tfliteclass/CTfLiteClass.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_flowcontroll/ClassFlowControll.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_image_proc/CImageBasis.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * Update code/components/jomjol_helper/psram.cpp * . * . * . * . * Korrektur Merge Conflict in main.cpp --------- Co-authored-by: CaCO3 <caco@ruinelli.ch> Co-authored-by: jomjol <30766535+jomjol@users.noreply.github.com> * fix PSRAM init return value check * Extend incl. indiv. Measurement * Implement UX * Update ClassFlowInfluxDBv2.cpp * Implement individual influx topic * Update interface_influxdb.cpp * Update interface_influxdb.cpp * Update FieldName * Extend incl. indiv. Measurement * Implement UX * Update ClassFlowInfluxDBv2.cpp * Update main.cpp --------- Co-authored-by: Slider0007 <115730895+Slider0007@users.noreply.github.com> Co-authored-by: CaCO3 <caco3@ruinelli.ch> Co-authored-by: CaCO3 <caco@ruinelli.ch> * Update Changelog.md (#2344) * Update Changelog.md * Update Changelog.md * merge conflicts * merge conflicts * update changelog --------- Co-authored-by: Frank Haverland <fspapaping@googlemail.com> Co-authored-by: CaCO3 <caco@ruinelli.ch> Co-authored-by: jomjol <30766535+jomjol@users.noreply.github.com> Co-authored-by: Slider0007 <jobbelle@gmx.net> Co-authored-by: Slider0007 <115730895+Slider0007@users.noreply.github.com> Co-authored-by: Antonin Delpeuch <antonin@delpeuch.eu>
237 lines
8.7 KiB
C++
237 lines
8.7 KiB
C++
#ifdef ENABLE_INFLUXDB
|
|
#include "interface_influxdb.h"
|
|
|
|
#include "esp_log.h"
|
|
#include <time.h>
|
|
#include "ClassLogFile.h"
|
|
#include "esp_http_client.h"
|
|
#include "../../include/defines.h"
|
|
|
|
|
|
static const char *TAG = "INFLUXDB";
|
|
|
|
std::string _influxDBURI;
|
|
std::string _influxDBDatabase;
|
|
std::string _influxDBUser;
|
|
std::string _influxDBPassword;
|
|
|
|
std::string _influxDB_V2_URI;
|
|
std::string _influxDB_V2_Database;
|
|
std::string _influxDB_V2_Token;
|
|
std::string _influxDB_V2_Org;
|
|
|
|
static esp_err_t http_event_handler(esp_http_client_event_t *evt);
|
|
|
|
void InfluxDB_V2_Init(std::string _uri, std::string _database, std::string _org, std::string _token)
|
|
{
|
|
_influxDB_V2_URI = _uri;
|
|
_influxDB_V2_Database = _database;
|
|
_influxDB_V2_Org = _org;
|
|
_influxDB_V2_Token = _token;
|
|
}
|
|
|
|
void InfluxDB_V2_Publish(std::string _measurement, std::string _key, std::string _content, std::string _timestamp)
|
|
{
|
|
char response_buffer[MAX_HTTP_OUTPUT_BUFFER] = {0};
|
|
esp_http_client_config_t http_config = {
|
|
.user_agent = "ESP32 Meter reader",
|
|
.method = HTTP_METHOD_POST,
|
|
.event_handler = http_event_handler,
|
|
.buffer_size = MAX_HTTP_OUTPUT_BUFFER,
|
|
.user_data = response_buffer
|
|
};
|
|
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "InfluxDB_V2_Publish - Key: " + _key + ", Content: " + _content + ", Timestamp: " + _timestamp);
|
|
|
|
std::string payload;
|
|
char nowTimestamp[21];
|
|
|
|
if (_timestamp.length() > 0)
|
|
{
|
|
struct tm tm;
|
|
strptime(_timestamp.c_str(), PREVALUE_TIME_FORMAT_OUTPUT, &tm);
|
|
time_t t = mktime(&tm); // Time in Localtime (looks like timezone is not used by strptime)
|
|
|
|
// struct tm * ptm;
|
|
// ptm = gmtime ( &t );
|
|
// time_t utc = mktime(ptm);
|
|
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "Use handover timestamp: " + _timestamp + " converted GMT timestamp: " + std::to_string(t));
|
|
|
|
// utc = 2*t - utc; // Take care of timezone (looks difficult, but is easy: t = t + (t - utc), weil t-utc = timezone)
|
|
// LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "time conversion utc after: " + std::to_string(utc));
|
|
|
|
sprintf(nowTimestamp,"%ld000000000", (long) t); // UTC
|
|
|
|
payload = _measurement + " " + _key + "=" + _content + " " + nowTimestamp;
|
|
}
|
|
else
|
|
{
|
|
payload = _measurement + " " + _key + "=" + _content;
|
|
}
|
|
|
|
payload.shrink_to_fit();
|
|
|
|
LogFile.WriteToFile(ESP_LOG_INFO, TAG, "sending line to influxdb:" + payload);
|
|
|
|
std::string apiURI = _influxDB_V2_URI + "/api/v2/write?org=" + _influxDB_V2_Org + "&bucket=" + _influxDB_V2_Database;
|
|
apiURI.shrink_to_fit();
|
|
http_config.url = apiURI.c_str();
|
|
ESP_LOGI(TAG, "http_config: %s", http_config.url); // Add mark on log to see when it restarted
|
|
|
|
LogFile.WriteToFile(ESP_LOG_INFO, TAG, "API URI: " + apiURI);
|
|
|
|
esp_http_client_handle_t http_client = esp_http_client_init(&http_config);
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "client is initialized");
|
|
|
|
esp_http_client_set_header(http_client, "Content-Type", "text/plain");
|
|
std::string _zw = "Token " + _influxDB_V2_Token;
|
|
// LogFile.WriteToFile(ESP_LOG_INFO, TAG, "Tokenheader: %s\n", _zw.c_str());
|
|
esp_http_client_set_header(http_client, "Authorization", _zw.c_str());
|
|
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "header is set");
|
|
|
|
ESP_ERROR_CHECK(esp_http_client_set_post_field(http_client, payload.c_str(), payload.length()));
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "post payload is set");
|
|
|
|
esp_err_t err = ESP_ERROR_CHECK_WITHOUT_ABORT(esp_http_client_perform(http_client));
|
|
|
|
if( err == ESP_OK ) {
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP request was performed");
|
|
int status_code = esp_http_client_get_status_code(http_client);
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP status code" + std::to_string(status_code));
|
|
} else {
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP request failed");
|
|
}
|
|
esp_http_client_cleanup(http_client);
|
|
}
|
|
|
|
|
|
|
|
static esp_err_t http_event_handler(esp_http_client_event_t *evt)
|
|
{
|
|
switch(evt->event_id)
|
|
{
|
|
case HTTP_EVENT_ERROR:
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client Error encountered");
|
|
break;
|
|
case HTTP_EVENT_ON_CONNECTED:
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client Error encountered");
|
|
ESP_LOGI(TAG, "HTTP Client Connected");
|
|
break;
|
|
case HTTP_EVENT_HEADERS_SENT:
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client sent all request headers");
|
|
break;
|
|
case HTTP_EVENT_ON_HEADER:
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "Header: key=" + std::string(evt->header_key) + ", value=" + std::string(evt->header_value));
|
|
break;
|
|
case HTTP_EVENT_ON_DATA:
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client data recevied: len=" + std::to_string(evt->data_len));
|
|
break;
|
|
case HTTP_EVENT_ON_FINISH:
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client finished");
|
|
break;
|
|
case HTTP_EVENT_DISCONNECTED:
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client Disconnected");
|
|
break;
|
|
case HTTP_EVENT_REDIRECT:
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Redirect");
|
|
break;
|
|
}
|
|
return ESP_OK;
|
|
}
|
|
|
|
void InfluxDBPublish(std::string _measurement, std::string _key, std::string _content, std::string _timestamp) {
|
|
char response_buffer[MAX_HTTP_OUTPUT_BUFFER] = {0};
|
|
esp_http_client_config_t http_config = {
|
|
.user_agent = "ESP32 Meter reader",
|
|
.method = HTTP_METHOD_POST,
|
|
.event_handler = http_event_handler,
|
|
.buffer_size = MAX_HTTP_OUTPUT_BUFFER,
|
|
.user_data = response_buffer
|
|
};
|
|
|
|
if (_influxDBUser.length() && _influxDBPassword.length()){
|
|
http_config.username = _influxDBUser.c_str();
|
|
http_config.password = _influxDBPassword.c_str();
|
|
http_config.auth_type = HTTP_AUTH_TYPE_BASIC;
|
|
}
|
|
|
|
std::string payload;
|
|
char nowTimestamp[21];
|
|
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "InfluxDBPublish - Key: " + _key + ", Content: " + _content + ", Timestamp: " + _timestamp);
|
|
|
|
if (_timestamp.length() > 0)
|
|
{
|
|
struct tm tm;
|
|
strptime(_timestamp.c_str(), PREVALUE_TIME_FORMAT_OUTPUT, &tm);
|
|
time_t t = mktime(&tm); // Time in Localtime (looks like timezone is not used by strptime)
|
|
|
|
// struct tm * ptm;
|
|
// ptm = gmtime ( &t );
|
|
// time_t utc = mktime(ptm);
|
|
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "Use handover timestamp: " + _timestamp + " converted GMT timestamp: " + std::to_string(t));
|
|
|
|
// utc = 2*t - utc; // Take care of timezone (looks difficult, but is easy: t = t + (t - utc), weil t-utc = timezone)
|
|
// LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "time conversion utc after: " + std::to_string(utc));
|
|
|
|
sprintf(nowTimestamp,"%ld000000000", (long) t); // UTC
|
|
|
|
payload = _measurement + " " + _key + "=" + _content + " " + nowTimestamp;
|
|
}
|
|
else
|
|
{
|
|
payload = _measurement + " " + _key + "=" + _content;
|
|
}
|
|
|
|
payload.shrink_to_fit();
|
|
|
|
LogFile.WriteToFile(ESP_LOG_INFO, TAG, "sending line to influxdb:" + payload);
|
|
|
|
|
|
// use the default retention policy of the database
|
|
std::string apiURI = _influxDBURI + "/write?db=" + _influxDBDatabase;
|
|
// std::string apiURI = _influxDBURI + "/api/v2/write?bucket=" + _influxDBDatabase + "/";
|
|
|
|
apiURI.shrink_to_fit();
|
|
http_config.url = apiURI.c_str();
|
|
|
|
LogFile.WriteToFile(ESP_LOG_INFO, TAG, "API URI: " + apiURI);
|
|
|
|
esp_http_client_handle_t http_client = esp_http_client_init(&http_config);
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "client is initialized");
|
|
|
|
esp_http_client_set_header(http_client, "Content-Type", "text/plain");
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "header is set");
|
|
|
|
ESP_ERROR_CHECK(esp_http_client_set_post_field(http_client, payload.c_str(), payload.length()));
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "post payload is set");
|
|
|
|
esp_err_t err = ESP_ERROR_CHECK_WITHOUT_ABORT(esp_http_client_perform(http_client));
|
|
|
|
if( err == ESP_OK ) {
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP request was performed");
|
|
int status_code = esp_http_client_get_status_code(http_client);
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP status code" + std::to_string(status_code));
|
|
} else {
|
|
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP request failed");
|
|
}
|
|
esp_http_client_cleanup(http_client);
|
|
}
|
|
|
|
|
|
void InfluxDBInit(std::string _uri, std::string _database, std::string _user, std::string _password){
|
|
_influxDBURI = _uri;
|
|
_influxDBDatabase = _database;
|
|
_influxDBUser = _user;
|
|
_influxDBPassword = _password;
|
|
|
|
}
|
|
|
|
void InfluxDBdestroy() {
|
|
}
|
|
|
|
#endif //ENABLE_INFLUXDB
|