Files
AI-on-the-edge-device/code/components/jomjol_flowcontroll/ClassFlowMQTT.cpp
CaCO3 ca01f5a38f Prepare next release (#3267)
* Create dig-cont_0640_s3_q.tflite

* Update edit_reference.html (#2924)

Zoom-Einstellungen ausblenden, wenn zoom deaktiviert ist

* Update edit_config_template.html (#2925)

Zoom-Einstellungen ausblenden, wenn zoom deaktiviert ist +
Kameraeinstellungen nicht nur in der config.ini speichern, sondern auch setzen

* The parameter "negative" is processed on the MCU

* Renamed menu entry Alignment -> References

* renamed model file to make naming consistent

* Fix test (#2933)

* always reset change absolute (#2956)

* Create dig-class11_1701_s2.tflite

* Update tflite

* Update tflite

* Camera settings (#3029)

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Fix minor typo and harmonize spaces (#3030)

* Update edit_digits.html

* Update edit_analog.html

* Update overview.html

* Update edit_reference.html

---------

Co-authored-by: jomjol <30766535+jomjol@users.noreply.github.com>

* Source code formatting, ClassFlowPostProcessing.cpp

* fix edit_config_template.html

Unfortunately, I didn't take out some entries from other experiments because I overlooked/forgot them^^

* fix readconfigparam.js

Unfortunately, I didn't take out some entries from other experiments because I overlooked/forgot them^^

* fix edit_config_template.html

TakeImage_CamZoomMode_value1 was replaced by TakeImage_CamZoomSize_value1, but not deleted.
This caused problems when saving the camera settings.

* Update info.html

function loadWLANSSID() was present three times

* Update overview.html

CamFrameSize is no longer needed/used for zoom

* Update reply-bot.yaml

* homeassistant service discovery: derive node_id when using nested topics (#3088)

* derive correct node_id for homeassistant service discovery in nested topics (fixes #1792)

* explicit use of std::string

* move nodeId creation to separate function
add unit-tests

* add documentation about node_id generation for Home Assistant MQTT Service Discovery

* add Prometheus/OpenMetrics exporter (#3081)

* add prometheus endpoint

* refine metrics implementation

* move metrics generator to ClassFlowControll

* add more metrics
align prefix

* add more metrics
clean up

* refine documentation

* revert dependencies change

* sanitize labels

* create separate module for openmetrics

* move openmetrics to separate folder

* clean up

* add basic unit-tests

* work with const numbers
add replaceAll for string replacement
avoid opening std namespace
adapt unit-tests

* Update code/main/server_main.cpp

---------

Co-authored-by: CaCO3 <caco3@ruinelli.ch>

* Typo

* update platformIO to 6.7.0 (ESP IDF 5.2.1) (#3098)

* update to platformio/espressif32 @ 6.7.0

* remove unused getReadout() as it throws errors (error: 'virtual std::string ClassFlow::getReadout()' was hidden).

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Update reply-bot.yaml (#3107)

* Fix actions-label-commenter

* Update Helper.cpp

* Update Helper.h

* Update Helper.cpp

* Update readconfigparam.js

fix for:
In the selected field the value '1' in the section 'TakeImage' in the field 'CamGainceiling' is invalid. PLEASE CHECK BEFORE SAVING!

* proposal for renaming and documentation (#3115)

* Update server_camera.cpp

 Fix building with -D DEBUG_DETAIL_ON 

better alternative to:
https://github.com/jomjol/AI-on-the-edge-device/pull/3160

* Update ClassFlowPostProcessing.cpp

* Update ClassFlowAlignment.cpp

* Fix building with `-D DEBUG_DETAIL_ON` (#3160)

Building with `-D DEBUG_DETAIL_ON` has been broken since #3029.

Co-authored-by: CaCO3 <caco3@ruinelli.ch>

* Handle empty prevalue.ini gracefully (#3162)

Fixes #2149.

* Bugfix for boot loop (#3175)

* Add files via upload

* Add files via upload

* Add files via upload

* Delete param-docs/parameter-pages/TakeImage/Aec2.md

has been replaced by CamAec2.md

* Delete param-docs/parameter-pages/TakeImage/AutoExposureLevel.md

has been replaced by CamAeLevel.md

* Delete param-docs/parameter-pages/TakeImage/Brightness.md

has been replaced by CamBrightness.md

* Delete param-docs/parameter-pages/TakeImage/Contrast.md

has been replaced by CamContrast.md

* Delete param-docs/parameter-pages/TakeImage/Grayscale.md

has been replaced by CamSpecialEffect.md

* Delete param-docs/parameter-pages/TakeImage/Negative.md

has been replaced by CamSpecialEffect.md

* Delete param-docs/parameter-pages/TakeImage/Saturation.md

has been replaced by CamSaturation.md

* Delete param-docs/parameter-pages/TakeImage/Sharpness.md

has been replaced by CamSharpness.md

* Delete param-docs/parameter-pages/TakeImage/ImageQuality.md

has been replaced by CamQuality.md

* Delete param-docs/parameter-pages/TakeImage/Zoom.md

has been replaced by CamZoom.md

* Delete param-docs/parameter-pages/TakeImage/ZoomMode.md

has been replaced by CamZoomSize.md

* Delete param-docs/parameter-pages/TakeImage/ZoomOffsetX.md

has been replaced by CamZoomOffsetX.md

* Delete param-docs/parameter-pages/TakeImage/ZoomOffsetY.md

has been replaced by CamZoomOffsetY.md

* Delete param-docs/parameter-pages/TakeImage/ImageSize.md

has been replaced by CamZoomSize.md

* Delete param-docs/parameter-pages/TakeImage/FixedExposure.md

has been replaced by CamAec.md

* Delete param-docs/parameter-pages/Alignment/FlipImageSize.md

has been replaced by CamVflip.md

* Delete param-docs/parameter-pages/Alignment/InitialMirror.md

has been replaced by CamHmirror.md

* CamParameter documentation update

https://github.com/jomjol/AI-on-the-edge-device/issues/3185

* typo

* add Webhook #3148 (#3163)

* WIP add Webhook

* fix config html for webhook
add tooltips for webhook

* webhook: fix not enabling webhook

* send webhook as json

* Update ApiKey.md

* webhook: fix only sending last "Number"

* webhook JSON is now closer to the data log in CSV format

* webhook: drop timeStampTimeUTC and switch from timeStamp to lastvalue like lokal csv to fix no timestamp on error

---------

Co-authored-by: CaCO3 <caco3@ruinelli.ch>

* Bugfix for time stamp (#3180)

* Update ClassFlowPostProcessing.cpp

* Update ClassFlowDefineTypes.h

* Update ClassFlowPostProcessing.cpp

* Update ClassFlowPostProcessing.cpp

* Update ClassFlowPostProcessing.cpp

* Update ClassFlowPostProcessing.cpp

* Update interface_webhook.cpp

* Update readconfigcommon.js

fix for: The same message("Image Contrast got enhanced") came up with "Update Marker" and "Enhance Image Contrast".

* fix svg favicon 

The svg one got added in 33893eb566 but does not work on Firefox

* Update platformIO to 6.8.1 (Contains ESP IDF 5.3) (#3196)

* Update platformIO to 6.8.1 (ESP IDF 5.3)

* removed now redundant typedef

* updated IDF manifest hash

* Add files via upload

so it should work now

* Update server_main.cpp

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>
Co-authored-by: michael <Heinrich-Tuning@web.de>

* Update MainFlowControl.cpp

* Add support for OV5640 camera (#3063)

* Add support for OV5640 camera

* clean up sharpness handling

* limit sharpness range to -2 and +2

* refactor

* Fix OV3660 sharpness handling

* refactor sharpness handling

* fix OV3660 zoom mode

* reinstate aspect ratio via imageSize

* Changed OV5640 full frame size to match datasheet

* various fixes

* add denoise config and general clean up

* fix line endings to LF

* Support enabling red blue swap via web interface

* update jpeg quality limits

* remove color swap config; color swap workaround dependent on vflip

* fix missing commit

* fix gain ceiling

* Update cam vflip param page

* fix typo: camdenoise, not camsdenoise

* fix compile errors

* Update MainFlowControl.cpp

* Add rate threshold parameter (#3195)

* still needs to be tested

https://github.com/jomjol/AI-on-the-edge-device/issues/3143

* Update ClassFlowPostProcessing.cpp

code formatting

* Update ClassFlowDefineTypes.h

code formatting

* Update ClassFlowPostProcessing.h

code formatting

* Update edit_config_template.html

* fix

* Update config.ini

* Update edit_config_template.html

* Updated param doc

* Rename parameters

* Update edit_config_template.html

* Update NUMBER.ChangeRateThreshold.md

* Update NUMBER.ChangeRateThreshold.md

---------

Co-authored-by: CaCO3 <caco3@ruinelli.ch>

* Update main.cpp

* Update config.ini

Adjusted camera settings to make the image brighter.

* Update readconfigparam.js

Adjusted camera settings to make the image brighter.

* add optional ImageUpload for Webhook (#3174)

* WIP add Webhook

* fix config html for webhook
add tooltips for webhook

* webhook: fix not enabling webhook

* send webhook as json

* Update ApiKey.md

* webhook: fix only sending last "Number"

* webhook JSON is now closer to the data log in CSV format

* webhook: add img upload

* webhoop added config for imgupload

* webhook html fixes

* webhook: drop timeStampTimeUTC and switch from timeStamp to lastvalue like lokal csv to fix no timestamp on error

* add checkbox for Webhook_UploadImg

* Update sd-card/html/edit_config_template.html

* Update edit_config_template.html

* Update edit_config_template.html

* Update edit_config_template.html

* added a long timestamp to both webhook requests

---------

Co-authored-by: CaCO3 <caco3@ruinelli.ch>

* Add files via upload (#3207)

* Update ClassFlowPostProcessing.cpp

deleted some unnecessary double entries
RateType renamed to MaxRateType

* Update ClassFlowDefineTypes.h

RateType renamed to MaxRateType

* Update text on recognition page

* Update digital CNN

* add a Delay between the WiFi reconnections (#3068)

* add a Delay between the WiFi reconnections

* log the delay between the WiFi reconnections

move the delay after the log

* cleanup

* cleanup

* cleanup

* Update edit_alignment.html

* cleanup

* cleanup

* Update platformIO to 6.9.0 (Contains ESP IDF 5.3.1)

* Handle crash on corrupted model (#3220)

* Upgrade esp-tflite-micro to 1.3.1

* Added log message to hint in case it crashes on loading a corrupted model

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* new dig-class100-173-s2-q on 23.800 images (#3257)

* new dig-class100-173-s2-q on 23.800 images

* platformio/espressif32 @ 6.8.1 for esp32cam-dev

* Revert "platformio/espressif32 @ 6.8.1 for esp32cam-dev"

This reverts commit cc9297d483.

* not using platformio 6.1.16

* Revert "not using platformio 6.1.16"

This reverts commit ef18e4fae7.

* moved number edit box styles into new file edit_style.css (#3262)

changed input[type=number] from 60px to 3em, to show 3 digits with current font size.

* added note about only TLS 1.2 is supported (#3213)

* Renamed digital to digit (#3219)

* renamed Digital to Digit

* added param migration

* Update .github/label-commenter-config.yaml

* renamed AnalogDigitTransition* to AnalogToDigitTransition*

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* Update Changelog.md

* Update Changelog.md

---------

Co-authored-by: jomjol <30766535+jomjol@users.noreply.github.com>
Co-authored-by: michael <Heinrich-Tuning@web.de>
Co-authored-by: Frank Haverland <fspapaping@googlemail.com>
Co-authored-by: kub3let <95883234+kub3let@users.noreply.github.com>
Co-authored-by: Marco H <myxor@users.noreply.github.com>
Co-authored-by: Henry Thasler <henrythasler@users.noreply.github.com>
Co-authored-by: CaCO3 <caco@ruinelli.ch>
Co-authored-by: Sebastian Lövdahl <slovdahl@hibox.fi>
Co-authored-by: Raphael Hehl <raphael@rhehl.de>
Co-authored-by: jasaw <jasaw@dius.com.au>
Co-authored-by: Francesco Carnielli <hex7c0@gmail.com>
Co-authored-by: kalwados <kalwados@gmx.de>
2024-09-29 23:22:33 +02:00

330 lines
11 KiB
C++

#ifdef ENABLE_MQTT
#include <sstream>
#include <iomanip>
#include "ClassFlowMQTT.h"
#include "Helper.h"
#include "connect_wlan.h"
#include "read_wlanini.h"
#include "ClassLogFile.h"
#include "time_sntp.h"
#include "interface_mqtt.h"
#include "ClassFlowPostProcessing.h"
#include "ClassFlowControll.h"
#include "server_mqtt.h"
#include <time.h>
#include "../../include/defines.h"
static const char *TAG = "MQTT";
extern const char* libfive_git_version(void);
extern const char* libfive_git_revision(void);
extern const char* libfive_git_branch(void);
void ClassFlowMQTT::SetInitialParameter(void)
{
uri = "";
topic = "";
topicError = "";
topicRate = "";
topicTimeStamp = "";
maintopic = wlan_config.hostname;
topicUptime = "";
topicFreeMem = "";
caCertFilename = "";
clientCertFilename = "";
clientKeyFilename = "";
clientname = wlan_config.hostname;
OldValue = "";
flowpostprocessing = NULL;
user = "";
password = "";
SetRetainFlag = false;
previousElement = NULL;
ListFlowControll = NULL;
disabled = false;
keepAlive = 25*60;
}
ClassFlowMQTT::ClassFlowMQTT()
{
SetInitialParameter();
}
ClassFlowMQTT::ClassFlowMQTT(std::vector<ClassFlow*>* lfc)
{
SetInitialParameter();
ListFlowControll = lfc;
for (int i = 0; i < ListFlowControll->size(); ++i)
{
if (((*ListFlowControll)[i])->name().compare("ClassFlowPostProcessing") == 0)
{
flowpostprocessing = (ClassFlowPostProcessing*) (*ListFlowControll)[i];
}
}
}
ClassFlowMQTT::ClassFlowMQTT(std::vector<ClassFlow*>* lfc, ClassFlow *_prev)
{
SetInitialParameter();
previousElement = _prev;
ListFlowControll = lfc;
for (int i = 0; i < ListFlowControll->size(); ++i)
{
if (((*ListFlowControll)[i])->name().compare("ClassFlowPostProcessing") == 0)
{
flowpostprocessing = (ClassFlowPostProcessing*) (*ListFlowControll)[i];
}
}
}
bool ClassFlowMQTT::ReadParameter(FILE* pfile, string& aktparamgraph)
{
std::vector<string> splitted;
aktparamgraph = trim(aktparamgraph);
if (aktparamgraph.size() == 0)
if (!this->GetNextParagraph(pfile, aktparamgraph))
return false;
if (toUpper(aktparamgraph).compare("[MQTT]") != 0) // Paragraph does not fit MQTT
return false;
while (this->getNextLine(pfile, &aktparamgraph) && !this->isNewParagraph(aktparamgraph))
{
splitted = ZerlegeZeile(aktparamgraph);
if ((toUpper(splitted[0]) == "CACERT") && (splitted.size() > 1))
{
this->caCertFilename = splitted[1];
}
if ((toUpper(splitted[0]) == "CLIENTCERT") && (splitted.size() > 1))
{
this->clientCertFilename = splitted[1];
}
if ((toUpper(splitted[0]) == "CLIENTKEY") && (splitted.size() > 1))
{
this->clientKeyFilename = splitted[1];
}
if ((toUpper(splitted[0]) == "USER") && (splitted.size() > 1))
{
this->user = splitted[1];
}
if ((toUpper(splitted[0]) == "PASSWORD") && (splitted.size() > 1))
{
this->password = splitted[1];
}
if ((toUpper(splitted[0]) == "URI") && (splitted.size() > 1))
{
this->uri = splitted[1];
}
if ((toUpper(splitted[0]) == "RETAINMESSAGES") && (splitted.size() > 1))
{
if (toUpper(splitted[1]) == "TRUE") {
SetRetainFlag = true;
setMqtt_Server_Retain(SetRetainFlag);
}
}
if ((toUpper(splitted[0]) == "HOMEASSISTANTDISCOVERY") && (splitted.size() > 1))
{
if (toUpper(splitted[1]) == "TRUE")
SetHomeassistantDiscoveryEnabled(true);
}
if ((toUpper(splitted[0]) == "METERTYPE") && (splitted.size() > 1)) {
/* Use meter type for the device class
Make sure it is a listed one on https://developers.home-assistant.io/docs/core/entity/sensor/#available-device-classes */
if (toUpper(splitted[1]) == "WATER_M3") {
mqttServer_setMeterType("water", "", "h", "m³/h");
}
else if (toUpper(splitted[1]) == "WATER_L") {
mqttServer_setMeterType("water", "L", "h", "L/h");
}
else if (toUpper(splitted[1]) == "WATER_FT3") {
mqttServer_setMeterType("water", "ft³", "m", "ft³/m"); // Minutes
}
else if (toUpper(splitted[1]) == "WATER_GAL") {
mqttServer_setMeterType("water", "gal", "h", "gal/h");
}
else if (toUpper(splitted[1]) == "GAS_M3") {
mqttServer_setMeterType("gas", "", "h", "m³/h");
}
else if (toUpper(splitted[1]) == "GAS_FT3") {
mqttServer_setMeterType("gas", "ft³", "m", "ft³/m"); // Minutes
}
else if (toUpper(splitted[1]) == "ENERGY_WH") {
mqttServer_setMeterType("energy", "Wh", "h", "W");
}
else if (toUpper(splitted[1]) == "ENERGY_KWH") {
mqttServer_setMeterType("energy", "kWh", "h", "kW");
}
else if (toUpper(splitted[1]) == "ENERGY_MWH") {
mqttServer_setMeterType("energy", "MWh", "h", "MW");
}
else if (toUpper(splitted[1]) == "ENERGY_GJ") {
mqttServer_setMeterType("energy", "GJ", "h", "GJ/h");
}
}
if ((toUpper(splitted[0]) == "CLIENTID") && (splitted.size() > 1))
{
this->clientname = splitted[1];
}
if (((toUpper(splitted[0]) == "TOPIC") || (toUpper(splitted[0]) == "MAINTOPIC")) && (splitted.size() > 1))
{
maintopic = splitted[1];
}
}
/* Note:
* Originally, we started the MQTT client here.
* How ever we need the interval parameter from the ClassFlowControll, but that only gets started later.
* To work around this, we delay the start and trigger it from ClassFlowControll::ReadParameter() */
mqttServer_setMainTopic(maintopic);
return true;
}
bool ClassFlowMQTT::Start(float AutoInterval)
{
roundInterval = AutoInterval; // Minutes
keepAlive = roundInterval * 60 * 2.5; // Seconds, make sure it is greater thatn 2 rounds!
std::stringstream stream;
stream << std::fixed << std::setprecision(1) << "Digitizer interval is " << roundInterval <<
" minutes => setting MQTT LWT timeout to " << ((float)keepAlive/60) << " minutes.";
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, stream.str());
mqttServer_setParameter(flowpostprocessing->GetNumbers(), keepAlive, roundInterval);
bool MQTTConfigCheck = MQTT_Configure(uri, clientname, user, password, maintopic, LWT_TOPIC, LWT_CONNECTED,
LWT_DISCONNECTED, caCertFilename, clientCertFilename, clientKeyFilename,
keepAlive, SetRetainFlag, (void *)&GotConnected);
if (!MQTTConfigCheck) {
return false;
}
return (MQTT_Init() == 1);
}
bool ClassFlowMQTT::doFlow(string zwtime)
{
bool success;
std::string result;
std::string resulterror = "";
std::string resultraw = "";
std::string resultpre = "";
std::string resultrate = ""; // Always Unit / Minute
std::string resultRatePerTimeUnit = ""; // According to selection
std::string resulttimestamp = "";
std::string resultchangabs = "";
string zw = "";
string namenumber = "";
int qos = 1;
/* Send the the Homeassistant Discovery and the Static Topics in case they where scheduled */
sendDiscovery_and_static_Topics();
success = publishSystemData(qos);
if (flowpostprocessing && getMQTTisConnected())
{
std::vector<NumberPost*>* NUMBERS = flowpostprocessing->GetNumbers();
LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "Publishing MQTT topics...");
for (int i = 0; i < (*NUMBERS).size(); ++i)
{
result = (*NUMBERS)[i]->ReturnValue;
resultraw = (*NUMBERS)[i]->ReturnRawValue;
resultpre = (*NUMBERS)[i]->ReturnPreValue;
resulterror = (*NUMBERS)[i]->ErrorMessageText;
resultrate = (*NUMBERS)[i]->ReturnRateValue; // Unit per minutes
resultchangabs = (*NUMBERS)[i]->ReturnChangeAbsolute; // Units per round
resulttimestamp = (*NUMBERS)[i]->timeStamp;
namenumber = (*NUMBERS)[i]->name;
if (namenumber == "default")
namenumber = maintopic + "/";
else
namenumber = maintopic + "/" + namenumber + "/";
if (result.length() > 0)
success |= MQTTPublish(namenumber + "value", result, qos, SetRetainFlag);
if (resulterror.length() > 0)
success |= MQTTPublish(namenumber + "error", resulterror, qos, SetRetainFlag);
if (resultrate.length() > 0) {
success |= MQTTPublish(namenumber + "rate", resultrate, qos, SetRetainFlag);
std::string resultRatePerTimeUnit;
if (getTimeUnit() == "h") { // Need conversion to be per hour
resultRatePerTimeUnit = resultRatePerTimeUnit = to_string((*NUMBERS)[i]->FlowRateAct * 60); // per minutes => per hour
}
else { // Keep per minute
resultRatePerTimeUnit = resultrate;
}
success |= MQTTPublish(namenumber + "rate_per_time_unit", resultRatePerTimeUnit, qos, SetRetainFlag);
}
if (resultchangabs.length() > 0) {
success |= MQTTPublish(namenumber + "changeabsolut", resultchangabs, qos, SetRetainFlag); // Legacy API
success |= MQTTPublish(namenumber + "rate_per_Digitization_round", resultchangabs, qos, SetRetainFlag);
}
if (resultraw.length() > 0)
success |= MQTTPublish(namenumber + "raw", resultraw, qos, SetRetainFlag);
if (resulttimestamp.length() > 0)
success |= MQTTPublish(namenumber + "timestamp", resulttimestamp, qos, SetRetainFlag);
std::string json = flowpostprocessing->getJsonFromNumber(i, "\n");
success |= MQTTPublish(namenumber + "json", json, qos, SetRetainFlag);
}
}
/* Disabled because this is no longer a use case */
// else
// {
// for (int i = 0; i < ListFlowControll->size(); ++i)
// {
// zw = (*ListFlowControll)[i]->getReadout();
// if (zw.length() > 0)
// {
// if (result.length() == 0)
// result = zw;
// else
// result = result + "\t" + zw;
// }
// }
// success |= MQTTPublish(topic, result, qos, SetRetainFlag);
// }
OldValue = result;
if (!success) {
LogFile.WriteToFile(ESP_LOG_WARN, TAG, "One or more MQTT topics failed to be published!");
}
return true;
}
#endif //ENABLE_MQTT