Files
AI-on-the-edge-device/code/components/jomjol_influxdb/interface_influxdb.cpp
CaCO3 c2a55e7c86 v15.2.0 (#2345)
* Testcase for #2145 and debug-log (#2151)

* new models ana-cont-11.0.5, ana-class100-1.5.7, dig-class100-1.6.0

* Testcase for #2145
Added debug log, if allowNegativeRates is handeled

* Fix timezone config parser (#2169)

* make sure to parse the whole config line

* fix crash on empty timezone parameter

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Enhance ROI pages (#2161)

* Check if the ROIs are equidistant. Only if not, untick the checkbox

* renaming

* Check if the ROIs have same y, dy and dx. If so, tick the sync checkbox

* only allow editing space when box is checked

* fix sync check

* show inner frame on all ROIs

* cleanup

* Check if the ROIs have same dy and dx. If so, tick the sync checkbox

* checkbox position

* renaming

* renaming

* show inner frame and cross hairs on all ROIs

* update ROIs on ticking checkboxes

* show timezone hint

* fix deleting last ROI

* cleanup

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* restart timeout on progress, catch error (#2170)

* restart timeout on progress, catch error

* .

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* BugFix #2167

* Release 15.1 preparations (#2171)

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update changelog

* Fix links to PR

* Formating

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

---------

Co-authored-by: Slider0007 <jobbelle@gmx.net>
Co-authored-by: Slider0007 <115730895+Slider0007@users.noreply.github.com>

* fix typo

* Replace relative documentation links with absolute ones pointing to the external documentation (#2180)

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Sort model files in configuration combobox (#2189)

* new models ana-cont-11.0.5, ana-class100-1.5.7, dig-class100-1.6.0

* Testcase for #2145
Added debug log, if allowNegativeRates is handeled

* Sort model files in combobox

* reboot task - increase stack size (#2201)

Avoid stack overflow

* Update interface_influxdb.cpp

* Update Changelog.md

* Show PSRAM usage (#2206)

* centralize PSRAM usage (application code only)

* update logging

* update logging

* fix use after free

* initialize buffer

* free rgb_image before ussing it for new allocation

* use wrapper function

* switch log level to debug

* .

* undo adding free() calls

* .

* add names to all CImage instances

* .

* .

* .

* revert changes of stbi_image_free() with free_psram_heap() on the places where is is not in PSRAM

* .

* typos

* typo

* Added MQTT Outbox explanation/warning

* added CONFIG_SPIRAM_USE_MEMMAP explanation

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Disable custom MQTT Outbox. This also moves the MQTT Publishing memory usage back to the internal RAM!

* log MQTT connection refused reasons (#2216)

* Revert PSRAM usage as it lead to memory fragmentation.
See https://github.com/jomjol/AI-on-the-edge-device/issues/2200 for details

* fix missing value data

* Revert PSRAM usage as it lead to memory fragmentation. (#2224)

See https://github.com/jomjol/AI-on-the-edge-device/issues/2200 for details

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Fix missing value data in graph (#2230)

* fix missing value data

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Update Changelog.md (#2231)

* Update interface_influxdb.cpp (#2233)

* update copyright year

* Cleanup

* Set prevalue using MQTT + set prevalue to RAW value (REST+MQTT) (#2252)

* Use double instead of float

* Error handling + set to RAW if newvalue < 0

* REST SetPrevalue: Set to RAW if newvalue < 0

* set prevalue with MQTT

* removed the stb_image files and re-add them as a submodule. (#2223)

- stb_image.h: Version update 2.25 -> 2.28
- stb_resize.h: Version update 0.96 -> 0.97
- stb_write.h: Version update 1.14 -> 1.16

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Remove obsolete ClassFlowWriteList (#2264)

* Renaming & cleanup of some modules / functions in source code (#2265)

* Rename module tag name

* Rename server_tflite.cpp -> MainFlowControl.cpp

* Remove redundandant MQTTMainTopic function

* Update

* Remove obsolete GetMQTTMainTopic

* Fix last element missing in digit model drop down (#2282)

* Debug influxdb (#2283)

* Fix time offset issues in InfluxDB component. (#2278)

Closes #2273
Closes #2150

* Update interface_influxdb.cpp

* Update interface_influxdb.cpp

* Improve Logging

* Implement TimeSync at beginning

* Update time_sntp.cpp

* Update time_sntp.cpp

* Set Time After WLAN Init

---------

Co-authored-by: Antonin Delpeuch <antonin@delpeuch.eu>

* Implement a camera livestream handler (#2286)

* fix leading NaN (#2310)

* analogROI: Activate save button after ROI creation (#2326)

* Migration of PlatformIO 5.2.0 to 6.1.0 (resp. ESP IDF from 4.4.2 to 5.0.1) (#2305)

* Migration to PlatformIO 6.1.0

* Disable RMTMEM usage as it is no longer allowed -> Smart LEDs not functional!

* moved miniz into subfolder of jomjol_fileserver_ota, else it does not build anymore.

* cleanup

* fix leading NaN (#2310)

* Migration to PlatformIO 6.1.0

* Disable RMTMEM usage as it is no longer allowed -> Smart LEDs not functional!

* moved miniz into subfolder of jomjol_fileserver_ota, else it does not build anymore.

* cleanup

* Task watchdog has new config name

* Fix return value check. It must be something else than ESP_FAIL, but it does not need to be ESP_OK!

* add missing strucures to work around new RMTMEM restriction (untested)

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Keep MainFlowTask alive to handle reboot (#2325)

* Shared PSRAM memory (#2285)

* enable PSRAM logging

* add extra functions for psram shared memroy handling

* CImageBasis objects still should used dynamic memory (eg. rawImage), haw ever tmpImage must be placed inside the shared memory

* Place all STBI allocs inside the shared memory

* The models are placed in the shared PSRAM reagion and must be allocated through the dedicated functions

* .

* renaming

* fix cast warning

* add flag to switch STBI PSRAM usage

* improve PSRAM shared handling

* reserve shared PSRAM as early as possible

* init logging eralier so we can use it in PSRAM shared alloc

* move Wifi_LWIP, BSS_SEG and MQTT Outbox into PSRAM to ffree internal memory

* Check if model fits into reserved shared memory

* Update code/components/jomjol_tfliteclass/CTfLiteClass.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_flowcontroll/ClassFlowControll.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_image_proc/CImageBasis.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* .

* .

* .

* .

* Korrektur Merge Conflict in main.cpp

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>
Co-authored-by: jomjol <30766535+jomjol@users.noreply.github.com>

* fix PSRAM init return value check

* Extend InfluxDBv1 with individual topic names (#2319)

* Implement individual influx topic

* Update interface_influxdb.cpp

* Update interface_influxdb.cpp

* Update FieldName

* analogROI: Activate save button after ROI creation (#2326)

* Migration of PlatformIO 5.2.0 to 6.1.0 (resp. ESP IDF from 4.4.2 to 5.0.1) (#2305)

* Migration to PlatformIO 6.1.0

* Disable RMTMEM usage as it is no longer allowed -> Smart LEDs not functional!

* moved miniz into subfolder of jomjol_fileserver_ota, else it does not build anymore.

* cleanup

* fix leading NaN (#2310)

* Migration to PlatformIO 6.1.0

* Disable RMTMEM usage as it is no longer allowed -> Smart LEDs not functional!

* moved miniz into subfolder of jomjol_fileserver_ota, else it does not build anymore.

* cleanup

* Task watchdog has new config name

* Fix return value check. It must be something else than ESP_FAIL, but it does not need to be ESP_OK!

* add missing strucures to work around new RMTMEM restriction (untested)

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Keep MainFlowTask alive to handle reboot (#2325)

* Shared PSRAM memory (#2285)

* enable PSRAM logging

* add extra functions for psram shared memroy handling

* CImageBasis objects still should used dynamic memory (eg. rawImage), haw ever tmpImage must be placed inside the shared memory

* Place all STBI allocs inside the shared memory

* The models are placed in the shared PSRAM reagion and must be allocated through the dedicated functions

* .

* renaming

* fix cast warning

* add flag to switch STBI PSRAM usage

* improve PSRAM shared handling

* reserve shared PSRAM as early as possible

* init logging eralier so we can use it in PSRAM shared alloc

* move Wifi_LWIP, BSS_SEG and MQTT Outbox into PSRAM to ffree internal memory

* Check if model fits into reserved shared memory

* Update code/components/jomjol_tfliteclass/CTfLiteClass.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_flowcontroll/ClassFlowControll.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_image_proc/CImageBasis.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* .

* .

* .

* .

* Korrektur Merge Conflict in main.cpp

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>
Co-authored-by: jomjol <30766535+jomjol@users.noreply.github.com>

* fix PSRAM init return value check

* Extend incl. indiv. Measurement

* Implement UX

* Update ClassFlowInfluxDBv2.cpp

* Implement individual influx topic

* Update interface_influxdb.cpp

* Update interface_influxdb.cpp

* Update FieldName

* Extend incl. indiv. Measurement

* Implement UX

* Update ClassFlowInfluxDBv2.cpp

* Update main.cpp

---------

Co-authored-by: Slider0007 <115730895+Slider0007@users.noreply.github.com>
Co-authored-by: CaCO3 <caco3@ruinelli.ch>
Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Update Changelog.md (#2344)

* Update Changelog.md

* Update Changelog.md

* merge conflicts

* merge conflicts

* update changelog

---------

Co-authored-by: Frank Haverland <fspapaping@googlemail.com>
Co-authored-by: CaCO3 <caco@ruinelli.ch>
Co-authored-by: jomjol <30766535+jomjol@users.noreply.github.com>
Co-authored-by: Slider0007 <jobbelle@gmx.net>
Co-authored-by: Slider0007 <115730895+Slider0007@users.noreply.github.com>
Co-authored-by: Antonin Delpeuch <antonin@delpeuch.eu>
2023-04-23 17:13:18 +02:00

237 lines
8.7 KiB
C++

#ifdef ENABLE_INFLUXDB
#include "interface_influxdb.h"
#include "esp_log.h"
#include <time.h>
#include "ClassLogFile.h"
#include "esp_http_client.h"
#include "../../include/defines.h"
static const char *TAG = "INFLUXDB";
std::string _influxDBURI;
std::string _influxDBDatabase;
std::string _influxDBUser;
std::string _influxDBPassword;
std::string _influxDB_V2_URI;
std::string _influxDB_V2_Database;
std::string _influxDB_V2_Token;
std::string _influxDB_V2_Org;
static esp_err_t http_event_handler(esp_http_client_event_t *evt);
void InfluxDB_V2_Init(std::string _uri, std::string _database, std::string _org, std::string _token)
{
_influxDB_V2_URI = _uri;
_influxDB_V2_Database = _database;
_influxDB_V2_Org = _org;
_influxDB_V2_Token = _token;
}
void InfluxDB_V2_Publish(std::string _measurement, std::string _key, std::string _content, std::string _timestamp)
{
char response_buffer[MAX_HTTP_OUTPUT_BUFFER] = {0};
esp_http_client_config_t http_config = {
.user_agent = "ESP32 Meter reader",
.method = HTTP_METHOD_POST,
.event_handler = http_event_handler,
.buffer_size = MAX_HTTP_OUTPUT_BUFFER,
.user_data = response_buffer
};
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "InfluxDB_V2_Publish - Key: " + _key + ", Content: " + _content + ", Timestamp: " + _timestamp);
std::string payload;
char nowTimestamp[21];
if (_timestamp.length() > 0)
{
struct tm tm;
strptime(_timestamp.c_str(), PREVALUE_TIME_FORMAT_OUTPUT, &tm);
time_t t = mktime(&tm); // Time in Localtime (looks like timezone is not used by strptime)
// struct tm * ptm;
// ptm = gmtime ( &t );
// time_t utc = mktime(ptm);
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "Use handover timestamp: " + _timestamp + " converted GMT timestamp: " + std::to_string(t));
// utc = 2*t - utc; // Take care of timezone (looks difficult, but is easy: t = t + (t - utc), weil t-utc = timezone)
// LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "time conversion utc after: " + std::to_string(utc));
sprintf(nowTimestamp,"%ld000000000", (long) t); // UTC
payload = _measurement + " " + _key + "=" + _content + " " + nowTimestamp;
}
else
{
payload = _measurement + " " + _key + "=" + _content;
}
payload.shrink_to_fit();
LogFile.WriteToFile(ESP_LOG_INFO, TAG, "sending line to influxdb:" + payload);
std::string apiURI = _influxDB_V2_URI + "/api/v2/write?org=" + _influxDB_V2_Org + "&bucket=" + _influxDB_V2_Database;
apiURI.shrink_to_fit();
http_config.url = apiURI.c_str();
ESP_LOGI(TAG, "http_config: %s", http_config.url); // Add mark on log to see when it restarted
LogFile.WriteToFile(ESP_LOG_INFO, TAG, "API URI: " + apiURI);
esp_http_client_handle_t http_client = esp_http_client_init(&http_config);
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "client is initialized");
esp_http_client_set_header(http_client, "Content-Type", "text/plain");
std::string _zw = "Token " + _influxDB_V2_Token;
// LogFile.WriteToFile(ESP_LOG_INFO, TAG, "Tokenheader: %s\n", _zw.c_str());
esp_http_client_set_header(http_client, "Authorization", _zw.c_str());
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "header is set");
ESP_ERROR_CHECK(esp_http_client_set_post_field(http_client, payload.c_str(), payload.length()));
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "post payload is set");
esp_err_t err = ESP_ERROR_CHECK_WITHOUT_ABORT(esp_http_client_perform(http_client));
if( err == ESP_OK ) {
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP request was performed");
int status_code = esp_http_client_get_status_code(http_client);
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP status code" + std::to_string(status_code));
} else {
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP request failed");
}
esp_http_client_cleanup(http_client);
}
static esp_err_t http_event_handler(esp_http_client_event_t *evt)
{
switch(evt->event_id)
{
case HTTP_EVENT_ERROR:
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client Error encountered");
break;
case HTTP_EVENT_ON_CONNECTED:
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client Error encountered");
ESP_LOGI(TAG, "HTTP Client Connected");
break;
case HTTP_EVENT_HEADERS_SENT:
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client sent all request headers");
break;
case HTTP_EVENT_ON_HEADER:
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "Header: key=" + std::string(evt->header_key) + ", value=" + std::string(evt->header_value));
break;
case HTTP_EVENT_ON_DATA:
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client data recevied: len=" + std::to_string(evt->data_len));
break;
case HTTP_EVENT_ON_FINISH:
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client finished");
break;
case HTTP_EVENT_DISCONNECTED:
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client Disconnected");
break;
case HTTP_EVENT_REDIRECT:
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Redirect");
break;
}
return ESP_OK;
}
void InfluxDBPublish(std::string _measurement, std::string _key, std::string _content, std::string _timestamp) {
char response_buffer[MAX_HTTP_OUTPUT_BUFFER] = {0};
esp_http_client_config_t http_config = {
.user_agent = "ESP32 Meter reader",
.method = HTTP_METHOD_POST,
.event_handler = http_event_handler,
.buffer_size = MAX_HTTP_OUTPUT_BUFFER,
.user_data = response_buffer
};
if (_influxDBUser.length() && _influxDBPassword.length()){
http_config.username = _influxDBUser.c_str();
http_config.password = _influxDBPassword.c_str();
http_config.auth_type = HTTP_AUTH_TYPE_BASIC;
}
std::string payload;
char nowTimestamp[21];
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "InfluxDBPublish - Key: " + _key + ", Content: " + _content + ", Timestamp: " + _timestamp);
if (_timestamp.length() > 0)
{
struct tm tm;
strptime(_timestamp.c_str(), PREVALUE_TIME_FORMAT_OUTPUT, &tm);
time_t t = mktime(&tm); // Time in Localtime (looks like timezone is not used by strptime)
// struct tm * ptm;
// ptm = gmtime ( &t );
// time_t utc = mktime(ptm);
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "Use handover timestamp: " + _timestamp + " converted GMT timestamp: " + std::to_string(t));
// utc = 2*t - utc; // Take care of timezone (looks difficult, but is easy: t = t + (t - utc), weil t-utc = timezone)
// LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "time conversion utc after: " + std::to_string(utc));
sprintf(nowTimestamp,"%ld000000000", (long) t); // UTC
payload = _measurement + " " + _key + "=" + _content + " " + nowTimestamp;
}
else
{
payload = _measurement + " " + _key + "=" + _content;
}
payload.shrink_to_fit();
LogFile.WriteToFile(ESP_LOG_INFO, TAG, "sending line to influxdb:" + payload);
// use the default retention policy of the database
std::string apiURI = _influxDBURI + "/write?db=" + _influxDBDatabase;
// std::string apiURI = _influxDBURI + "/api/v2/write?bucket=" + _influxDBDatabase + "/";
apiURI.shrink_to_fit();
http_config.url = apiURI.c_str();
LogFile.WriteToFile(ESP_LOG_INFO, TAG, "API URI: " + apiURI);
esp_http_client_handle_t http_client = esp_http_client_init(&http_config);
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "client is initialized");
esp_http_client_set_header(http_client, "Content-Type", "text/plain");
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "header is set");
ESP_ERROR_CHECK(esp_http_client_set_post_field(http_client, payload.c_str(), payload.length()));
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "post payload is set");
esp_err_t err = ESP_ERROR_CHECK_WITHOUT_ABORT(esp_http_client_perform(http_client));
if( err == ESP_OK ) {
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP request was performed");
int status_code = esp_http_client_get_status_code(http_client);
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP status code" + std::to_string(status_code));
} else {
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP request failed");
}
esp_http_client_cleanup(http_client);
}
void InfluxDBInit(std::string _uri, std::string _database, std::string _user, std::string _password){
_influxDBURI = _uri;
_influxDBDatabase = _database;
_influxDBUser = _user;
_influxDBPassword = _password;
}
void InfluxDBdestroy() {
}
#endif //ENABLE_INFLUXDB