Files
AI-on-the-edge-device/code/components/jomjol_flowcontroll/ClassFlowMQTT.cpp
CaCO3 c2a55e7c86 v15.2.0 (#2345)
* Testcase for #2145 and debug-log (#2151)

* new models ana-cont-11.0.5, ana-class100-1.5.7, dig-class100-1.6.0

* Testcase for #2145
Added debug log, if allowNegativeRates is handeled

* Fix timezone config parser (#2169)

* make sure to parse the whole config line

* fix crash on empty timezone parameter

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Enhance ROI pages (#2161)

* Check if the ROIs are equidistant. Only if not, untick the checkbox

* renaming

* Check if the ROIs have same y, dy and dx. If so, tick the sync checkbox

* only allow editing space when box is checked

* fix sync check

* show inner frame on all ROIs

* cleanup

* Check if the ROIs have same dy and dx. If so, tick the sync checkbox

* checkbox position

* renaming

* renaming

* show inner frame and cross hairs on all ROIs

* update ROIs on ticking checkboxes

* show timezone hint

* fix deleting last ROI

* cleanup

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* restart timeout on progress, catch error (#2170)

* restart timeout on progress, catch error

* .

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* BugFix #2167

* Release 15.1 preparations (#2171)

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update changelog

* Fix links to PR

* Formating

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

* Update Changelog.md

---------

Co-authored-by: Slider0007 <jobbelle@gmx.net>
Co-authored-by: Slider0007 <115730895+Slider0007@users.noreply.github.com>

* fix typo

* Replace relative documentation links with absolute ones pointing to the external documentation (#2180)

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Sort model files in configuration combobox (#2189)

* new models ana-cont-11.0.5, ana-class100-1.5.7, dig-class100-1.6.0

* Testcase for #2145
Added debug log, if allowNegativeRates is handeled

* Sort model files in combobox

* reboot task - increase stack size (#2201)

Avoid stack overflow

* Update interface_influxdb.cpp

* Update Changelog.md

* Show PSRAM usage (#2206)

* centralize PSRAM usage (application code only)

* update logging

* update logging

* fix use after free

* initialize buffer

* free rgb_image before ussing it for new allocation

* use wrapper function

* switch log level to debug

* .

* undo adding free() calls

* .

* add names to all CImage instances

* .

* .

* .

* revert changes of stbi_image_free() with free_psram_heap() on the places where is is not in PSRAM

* .

* typos

* typo

* Added MQTT Outbox explanation/warning

* added CONFIG_SPIRAM_USE_MEMMAP explanation

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Disable custom MQTT Outbox. This also moves the MQTT Publishing memory usage back to the internal RAM!

* log MQTT connection refused reasons (#2216)

* Revert PSRAM usage as it lead to memory fragmentation.
See https://github.com/jomjol/AI-on-the-edge-device/issues/2200 for details

* fix missing value data

* Revert PSRAM usage as it lead to memory fragmentation. (#2224)

See https://github.com/jomjol/AI-on-the-edge-device/issues/2200 for details

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Fix missing value data in graph (#2230)

* fix missing value data

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Update Changelog.md (#2231)

* Update interface_influxdb.cpp (#2233)

* update copyright year

* Cleanup

* Set prevalue using MQTT + set prevalue to RAW value (REST+MQTT) (#2252)

* Use double instead of float

* Error handling + set to RAW if newvalue < 0

* REST SetPrevalue: Set to RAW if newvalue < 0

* set prevalue with MQTT

* removed the stb_image files and re-add them as a submodule. (#2223)

- stb_image.h: Version update 2.25 -> 2.28
- stb_resize.h: Version update 0.96 -> 0.97
- stb_write.h: Version update 1.14 -> 1.16

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Remove obsolete ClassFlowWriteList (#2264)

* Renaming & cleanup of some modules / functions in source code (#2265)

* Rename module tag name

* Rename server_tflite.cpp -> MainFlowControl.cpp

* Remove redundandant MQTTMainTopic function

* Update

* Remove obsolete GetMQTTMainTopic

* Fix last element missing in digit model drop down (#2282)

* Debug influxdb (#2283)

* Fix time offset issues in InfluxDB component. (#2278)

Closes #2273
Closes #2150

* Update interface_influxdb.cpp

* Update interface_influxdb.cpp

* Improve Logging

* Implement TimeSync at beginning

* Update time_sntp.cpp

* Update time_sntp.cpp

* Set Time After WLAN Init

---------

Co-authored-by: Antonin Delpeuch <antonin@delpeuch.eu>

* Implement a camera livestream handler (#2286)

* fix leading NaN (#2310)

* analogROI: Activate save button after ROI creation (#2326)

* Migration of PlatformIO 5.2.0 to 6.1.0 (resp. ESP IDF from 4.4.2 to 5.0.1) (#2305)

* Migration to PlatformIO 6.1.0

* Disable RMTMEM usage as it is no longer allowed -> Smart LEDs not functional!

* moved miniz into subfolder of jomjol_fileserver_ota, else it does not build anymore.

* cleanup

* fix leading NaN (#2310)

* Migration to PlatformIO 6.1.0

* Disable RMTMEM usage as it is no longer allowed -> Smart LEDs not functional!

* moved miniz into subfolder of jomjol_fileserver_ota, else it does not build anymore.

* cleanup

* Task watchdog has new config name

* Fix return value check. It must be something else than ESP_FAIL, but it does not need to be ESP_OK!

* add missing strucures to work around new RMTMEM restriction (untested)

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Keep MainFlowTask alive to handle reboot (#2325)

* Shared PSRAM memory (#2285)

* enable PSRAM logging

* add extra functions for psram shared memroy handling

* CImageBasis objects still should used dynamic memory (eg. rawImage), haw ever tmpImage must be placed inside the shared memory

* Place all STBI allocs inside the shared memory

* The models are placed in the shared PSRAM reagion and must be allocated through the dedicated functions

* .

* renaming

* fix cast warning

* add flag to switch STBI PSRAM usage

* improve PSRAM shared handling

* reserve shared PSRAM as early as possible

* init logging eralier so we can use it in PSRAM shared alloc

* move Wifi_LWIP, BSS_SEG and MQTT Outbox into PSRAM to ffree internal memory

* Check if model fits into reserved shared memory

* Update code/components/jomjol_tfliteclass/CTfLiteClass.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_flowcontroll/ClassFlowControll.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_image_proc/CImageBasis.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* .

* .

* .

* .

* Korrektur Merge Conflict in main.cpp

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>
Co-authored-by: jomjol <30766535+jomjol@users.noreply.github.com>

* fix PSRAM init return value check

* Extend InfluxDBv1 with individual topic names (#2319)

* Implement individual influx topic

* Update interface_influxdb.cpp

* Update interface_influxdb.cpp

* Update FieldName

* analogROI: Activate save button after ROI creation (#2326)

* Migration of PlatformIO 5.2.0 to 6.1.0 (resp. ESP IDF from 4.4.2 to 5.0.1) (#2305)

* Migration to PlatformIO 6.1.0

* Disable RMTMEM usage as it is no longer allowed -> Smart LEDs not functional!

* moved miniz into subfolder of jomjol_fileserver_ota, else it does not build anymore.

* cleanup

* fix leading NaN (#2310)

* Migration to PlatformIO 6.1.0

* Disable RMTMEM usage as it is no longer allowed -> Smart LEDs not functional!

* moved miniz into subfolder of jomjol_fileserver_ota, else it does not build anymore.

* cleanup

* Task watchdog has new config name

* Fix return value check. It must be something else than ESP_FAIL, but it does not need to be ESP_OK!

* add missing strucures to work around new RMTMEM restriction (untested)

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Keep MainFlowTask alive to handle reboot (#2325)

* Shared PSRAM memory (#2285)

* enable PSRAM logging

* add extra functions for psram shared memroy handling

* CImageBasis objects still should used dynamic memory (eg. rawImage), haw ever tmpImage must be placed inside the shared memory

* Place all STBI allocs inside the shared memory

* The models are placed in the shared PSRAM reagion and must be allocated through the dedicated functions

* .

* renaming

* fix cast warning

* add flag to switch STBI PSRAM usage

* improve PSRAM shared handling

* reserve shared PSRAM as early as possible

* init logging eralier so we can use it in PSRAM shared alloc

* move Wifi_LWIP, BSS_SEG and MQTT Outbox into PSRAM to ffree internal memory

* Check if model fits into reserved shared memory

* Update code/components/jomjol_tfliteclass/CTfLiteClass.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_flowcontroll/ClassFlowControll.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_image_proc/CImageBasis.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* Update code/components/jomjol_helper/psram.cpp

* .

* .

* .

* .

* Korrektur Merge Conflict in main.cpp

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>
Co-authored-by: jomjol <30766535+jomjol@users.noreply.github.com>

* fix PSRAM init return value check

* Extend incl. indiv. Measurement

* Implement UX

* Update ClassFlowInfluxDBv2.cpp

* Implement individual influx topic

* Update interface_influxdb.cpp

* Update interface_influxdb.cpp

* Update FieldName

* Extend incl. indiv. Measurement

* Implement UX

* Update ClassFlowInfluxDBv2.cpp

* Update main.cpp

---------

Co-authored-by: Slider0007 <115730895+Slider0007@users.noreply.github.com>
Co-authored-by: CaCO3 <caco3@ruinelli.ch>
Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Update Changelog.md (#2344)

* Update Changelog.md

* Update Changelog.md

* merge conflicts

* merge conflicts

* update changelog

---------

Co-authored-by: Frank Haverland <fspapaping@googlemail.com>
Co-authored-by: CaCO3 <caco@ruinelli.ch>
Co-authored-by: jomjol <30766535+jomjol@users.noreply.github.com>
Co-authored-by: Slider0007 <jobbelle@gmx.net>
Co-authored-by: Slider0007 <115730895+Slider0007@users.noreply.github.com>
Co-authored-by: Antonin Delpeuch <antonin@delpeuch.eu>
2023-04-23 17:13:18 +02:00

314 lines
10 KiB
C++

#ifdef ENABLE_MQTT
#include <sstream>
#include <iomanip>
#include "ClassFlowMQTT.h"
#include "Helper.h"
#include "connect_wlan.h"
#include "read_wlanini.h"
#include "ClassLogFile.h"
#include "time_sntp.h"
#include "interface_mqtt.h"
#include "ClassFlowPostProcessing.h"
#include "ClassFlowControll.h"
#include "server_mqtt.h"
#include <time.h>
#include "../../include/defines.h"
static const char *TAG = "MQTT";
extern const char* libfive_git_version(void);
extern const char* libfive_git_revision(void);
extern const char* libfive_git_branch(void);
void ClassFlowMQTT::SetInitialParameter(void)
{
uri = "";
topic = "";
topicError = "";
topicRate = "";
topicTimeStamp = "";
maintopic = wlan_config.hostname;
topicUptime = "";
topicFreeMem = "";
clientname = wlan_config.hostname;
OldValue = "";
flowpostprocessing = NULL;
user = "";
password = "";
SetRetainFlag = false;
previousElement = NULL;
ListFlowControll = NULL;
disabled = false;
keepAlive = 25*60;
}
ClassFlowMQTT::ClassFlowMQTT()
{
SetInitialParameter();
}
ClassFlowMQTT::ClassFlowMQTT(std::vector<ClassFlow*>* lfc)
{
SetInitialParameter();
ListFlowControll = lfc;
for (int i = 0; i < ListFlowControll->size(); ++i)
{
if (((*ListFlowControll)[i])->name().compare("ClassFlowPostProcessing") == 0)
{
flowpostprocessing = (ClassFlowPostProcessing*) (*ListFlowControll)[i];
}
}
}
ClassFlowMQTT::ClassFlowMQTT(std::vector<ClassFlow*>* lfc, ClassFlow *_prev)
{
SetInitialParameter();
previousElement = _prev;
ListFlowControll = lfc;
for (int i = 0; i < ListFlowControll->size(); ++i)
{
if (((*ListFlowControll)[i])->name().compare("ClassFlowPostProcessing") == 0)
{
flowpostprocessing = (ClassFlowPostProcessing*) (*ListFlowControll)[i];
}
}
}
bool ClassFlowMQTT::ReadParameter(FILE* pfile, string& aktparamgraph)
{
std::vector<string> splitted;
aktparamgraph = trim(aktparamgraph);
if (aktparamgraph.size() == 0)
if (!this->GetNextParagraph(pfile, aktparamgraph))
return false;
if (toUpper(aktparamgraph).compare("[MQTT]") != 0) // Paragraph does not fit MQTT
return false;
while (this->getNextLine(pfile, &aktparamgraph) && !this->isNewParagraph(aktparamgraph))
{
splitted = ZerlegeZeile(aktparamgraph);
if ((toUpper(splitted[0]) == "USER") && (splitted.size() > 1))
{
this->user = splitted[1];
}
if ((toUpper(splitted[0]) == "PASSWORD") && (splitted.size() > 1))
{
this->password = splitted[1];
}
if ((toUpper(splitted[0]) == "URI") && (splitted.size() > 1))
{
this->uri = splitted[1];
}
if ((toUpper(splitted[0]) == "RETAINMESSAGES") && (splitted.size() > 1))
{
if (toUpper(splitted[1]) == "TRUE") {
SetRetainFlag = true;
setMqtt_Server_Retain(SetRetainFlag);
}
}
if ((toUpper(splitted[0]) == "HOMEASSISTANTDISCOVERY") && (splitted.size() > 1))
{
if (toUpper(splitted[1]) == "TRUE")
SetHomeassistantDiscoveryEnabled(true);
}
if ((toUpper(splitted[0]) == "METERTYPE") && (splitted.size() > 1)) {
/* Use meter type for the device class
Make sure it is a listed one on https://developers.home-assistant.io/docs/core/entity/sensor/#available-device-classes */
if (toUpper(splitted[1]) == "WATER_M3") {
mqttServer_setMeterType("water", "", "h", "m³/h");
}
else if (toUpper(splitted[1]) == "WATER_L") {
mqttServer_setMeterType("water", "L", "h", "L/h");
}
else if (toUpper(splitted[1]) == "WATER_FT3") {
mqttServer_setMeterType("water", "ft³", "m", "ft³/m"); // Minutes
}
else if (toUpper(splitted[1]) == "WATER_GAL") {
mqttServer_setMeterType("water", "gal", "h", "gal/h");
}
else if (toUpper(splitted[1]) == "GAS_M3") {
mqttServer_setMeterType("gas", "", "h", "m³/h");
}
else if (toUpper(splitted[1]) == "GAS_FT3") {
mqttServer_setMeterType("gas", "ft³", "m", "ft³/m"); // Minutes
}
else if (toUpper(splitted[1]) == "ENERGY_WH") {
mqttServer_setMeterType("energy", "Wh", "h", "W");
}
else if (toUpper(splitted[1]) == "ENERGY_KWH") {
mqttServer_setMeterType("energy", "kWh", "h", "kW");
}
else if (toUpper(splitted[1]) == "ENERGY_MWH") {
mqttServer_setMeterType("energy", "MWh", "h", "MW");
}
else if (toUpper(splitted[1]) == "ENERGY_GJ") {
mqttServer_setMeterType("energy", "GJ", "h", "GJ/h");
}
}
if ((toUpper(splitted[0]) == "CLIENTID") && (splitted.size() > 1))
{
this->clientname = splitted[1];
}
if (((toUpper(splitted[0]) == "TOPIC") || (toUpper(splitted[0]) == "MAINTOPIC")) && (splitted.size() > 1))
{
maintopic = splitted[1];
}
}
/* Note:
* Originally, we started the MQTT client here.
* How ever we need the interval parameter from the ClassFlowControll, but that only gets started later.
* To work around this, we delay the start and trigger it from ClassFlowControll::ReadParameter() */
mqttServer_setMainTopic(maintopic);
return true;
}
bool ClassFlowMQTT::Start(float AutoInterval)
{
roundInterval = AutoInterval; // Minutes
keepAlive = roundInterval * 60 * 2.5; // Seconds, make sure it is greater thatn 2 rounds!
std::stringstream stream;
stream << std::fixed << std::setprecision(1) << "Digitizer interval is " << roundInterval <<
" minutes => setting MQTT LWT timeout to " << ((float)keepAlive/60) << " minutes.";
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, stream.str());
mqttServer_setParameter(flowpostprocessing->GetNumbers(), keepAlive, roundInterval);
bool MQTTConfigCheck = MQTT_Configure(uri, clientname, user, password, maintopic, LWT_TOPIC, LWT_CONNECTED,
LWT_DISCONNECTED, keepAlive, SetRetainFlag, (void *)&GotConnected);
if (!MQTTConfigCheck) {
return false;
}
return (MQTT_Init() == 1);
}
bool ClassFlowMQTT::doFlow(string zwtime)
{
bool success;
std::string result;
std::string resulterror = "";
std::string resultraw = "";
std::string resultpre = "";
std::string resultrate = ""; // Always Unit / Minute
std::string resultRatePerTimeUnit = ""; // According to selection
std::string resulttimestamp = "";
std::string resultchangabs = "";
string zw = "";
string namenumber = "";
int qos = 1;
/* Send the the Homeassistant Discovery and the Static Topics in case they where scheduled */
sendDiscovery_and_static_Topics();
success = publishSystemData(qos);
if (flowpostprocessing && getMQTTisConnected())
{
std::vector<NumberPost*>* NUMBERS = flowpostprocessing->GetNumbers();
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "Publishing MQTT topics...");
for (int i = 0; i < (*NUMBERS).size(); ++i)
{
result = (*NUMBERS)[i]->ReturnValue;
resultraw = (*NUMBERS)[i]->ReturnRawValue;
resultpre = (*NUMBERS)[i]->ReturnPreValue;
resulterror = (*NUMBERS)[i]->ErrorMessageText;
resultrate = (*NUMBERS)[i]->ReturnRateValue; // Unit per minutes
resultchangabs = (*NUMBERS)[i]->ReturnChangeAbsolute; // Units per round
resulttimestamp = (*NUMBERS)[i]->timeStamp;
namenumber = (*NUMBERS)[i]->name;
if (namenumber == "default")
namenumber = maintopic + "/";
else
namenumber = maintopic + "/" + namenumber + "/";
if (result.length() > 0)
success |= MQTTPublish(namenumber + "value", result, qos, SetRetainFlag);
if (resulterror.length() > 0)
success |= MQTTPublish(namenumber + "error", resulterror, qos, SetRetainFlag);
if (resultrate.length() > 0) {
success |= MQTTPublish(namenumber + "rate", resultrate, qos, SetRetainFlag);
std::string resultRatePerTimeUnit;
if (getTimeUnit() == "h") { // Need conversion to be per hour
resultRatePerTimeUnit = resultRatePerTimeUnit = to_string((*NUMBERS)[i]->FlowRateAct * 60); // per minutes => per hour
}
else { // Keep per minute
resultRatePerTimeUnit = resultrate;
}
success |= MQTTPublish(namenumber + "rate_per_time_unit", resultRatePerTimeUnit, qos, SetRetainFlag);
}
if (resultchangabs.length() > 0) {
success |= MQTTPublish(namenumber + "changeabsolut", resultchangabs, qos, SetRetainFlag); // Legacy API
success |= MQTTPublish(namenumber + "rate_per_digitalization_round", resultchangabs, qos, SetRetainFlag);
}
if (resultraw.length() > 0)
success |= MQTTPublish(namenumber + "raw", resultraw, qos, SetRetainFlag);
if (resulttimestamp.length() > 0)
success |= MQTTPublish(namenumber + "timestamp", resulttimestamp, qos, SetRetainFlag);
std::string json = flowpostprocessing->getJsonFromNumber(i, "\n");
success |= MQTTPublish(namenumber + "json", json, qos, SetRetainFlag);
}
}
/* Disabled because this is no longer a use case */
// else
// {
// for (int i = 0; i < ListFlowControll->size(); ++i)
// {
// zw = (*ListFlowControll)[i]->getReadout();
// if (zw.length() > 0)
// {
// if (result.length() == 0)
// result = zw;
// else
// result = result + "\t" + zw;
// }
// }
// success |= MQTTPublish(topic, result, qos, SetRetainFlag);
// }
OldValue = result;
if (!success) {
LogFile.WriteToFile(ESP_LOG_WARN, TAG, "One or more MQTT topics failed to be published!");
}
return true;
}
#endif //ENABLE_MQTT