Auto gen parameter doc (#14)

* added script to generate parameter documentation

* extended readme

* added templated parameter pages

* Update build-docs.yaml

* .

* .

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>

.github/workflows/build-docs.yaml

@@ -11,6 +11,7 @@ on:
   push:
     branches:
       - 'main'
+      - 'auto-gen-parameter-doc'
 
 jobs:
   build-documentation:
@@ -39,4 +40,11 @@ jobs:
           
       - name: Store docs in gh-pages branch
         run: |
+          # Update Parameter Documentation
+          cd generate-param-docs
+          python generate-param-doc-pages.py
+          python concat-parameter-pages.py
+          cd ..
+        
+          # Deploy it
           mkdocs gh-deploy

.gitignore

@@ -1 +1,3 @@
+docs/Parameters.md
+.idea
 site

README.md

@@ -25,6 +25,13 @@ Each page has a link on its top-right corner `Edit on GitHub` which brings you d
 1. Add a new `*.md` document in the [docs](docs) folder.
 1. Add the **filename** to the [docs/nav.yml](docs/nav.yml) at the wished position in the **Links** section.
 
+## Parameter Docs Generator
+The script `generate-param-docs/generate-param-doc-pages.py` should be run whenever the [configfile](https://github.com/jomjol/AI-on-the-edge-device/blob/rolling/sd-card/config/config.ini) in the main project repo changed.
+It then checks if there is a markdown file  for each of the contained parameters. if not, it generates a templated page.
+
+The script `generate-param-docs/concat-parameter-pages.py` should be run whenever one of the parameter documentation pages changed.
+It then concatenates all parameter pages into a single `Parameters.md` which can be added to the documentation.
+
 ## Local Test
 To test it locally:
 1. Clone this repo

docs/nav.yml

@@ -22,6 +22,7 @@ nav:
     - Choosing-the-Model.md
     - Neural-Network-Types.md
     - ota.md
+    - Parameters.md
 
   - Advanced:
     - Integration-Home-Assistant.md

generate-param-docs/concat-parameter-pages.py

@@ -0,0 +1,48 @@
+"""
+Grab all parameter files and concat them into a single file.
+The header structure gets moved down 1 level
+"""
+
+import os
+import shutil
+import glob
+
+
+parameterDocsFolder = "parameter-docs"
+parameterOverviewFile = "../docs/Parameters.md"
+parameterOverviewTemplateFile = "./templates/overview.md"
+
+def appendParameterFile(folder, file):
+    print(folder, file)
+
+    with open(file, 'r') as parameterFileHandle:
+        parameterDoc = parameterFileHandle.read()
+        parameterDoc = parameterDoc.replace("# ", "### ") # Move all headings 2 level down
+
+    # Add parameter doc to overview page
+    with open(parameterOverviewFile, 'a') as overviewFileHandle:
+        overviewFileHandle.write(parameterDoc + "\n---\n\n")
+
+
+
+# Create templated overview markdown file
+if os.path.exists(parameterOverviewFile):
+    os.remove(parameterOverviewFile)
+shutil.copy(parameterOverviewTemplateFile, parameterOverviewFile)
+
+"""
+Append all parameter pages in a sorted manner
+"""
+folders = sorted( filter( os.path.isdir, glob.glob(parameterDocsFolder + '/*') ) )
+for folder in folders:
+    folder = folder.split("/")[-1]
+#    print(folder)
+
+    # Add section
+    with open(parameterOverviewFile, 'a') as overviewFileHandle:
+        overviewFileHandle.write("\n## [%s]\n\n" % folder)
+
+    files = sorted(filter(os.path.isfile, glob.glob(parameterDocsFolder + "/" + folder + '/*')))
+    for file in files:
+#        print("  %s" % file)
+        appendParameterFile(folder, file)

generate-param-docs/generate-param-doc-pages.py

@@ -0,0 +1,68 @@
+"""
+For each parameter which can be found in the config file,
+create a markdown file with a templated contentf it does not exist yet.
+The files are grouped in sub folders representing the config sections.
+"""
+
+import os
+import configparser
+import urllib.request
+
+configFileUrl = "https://raw.githubusercontent.com/jomjol/AI-on-the-edge-device/rolling/sd-card/config/config.ini"
+
+parameterDocsFolder = "parameter-docs"
+parameterTemplateFile = "./templates/parameter.md"
+
+# Fetch default config file from URL
+with urllib.request.urlopen(configFileUrl) as response:
+   content = response.read().decode("utf-8")
+
+lines = str(content).split("\n")
+
+for l in range(len(lines)):
+    lines[l] = lines[l].strip() + "\n"
+    if lines[l][0] == ";":
+        lines[l] = lines[l][1:] # Remove comment
+
+content = "".join(lines)
+
+
+config = configparser.ConfigParser(allow_no_value=True)
+config.optionxform = str # Make it case-insensitive
+config.read_string(content)
+
+#shutil.rmtree(parameterDocsFolder)
+if not os.path.exists(parameterDocsFolder):
+    os.mkdir(parameterDocsFolder)
+
+with open(parameterTemplateFile, 'r') as parameterTemplateFileHandle:
+    parameterTemplate = parameterTemplateFileHandle.read()
+
+
+
+for section in config:
+    if section != "DEFAULT":
+        #print(section)
+
+        subFolder = parameterDocsFolder + "/" + section
+
+        if not os.path.exists(subFolder):
+            os.mkdir(subFolder)
+
+        for parameter in config[section]:
+            if not " " in parameter: # Ignore parameters with whitespaces in them (special format, not part of editable config)
+                value = config[section][parameter]
+                #print("  %s = %s" % (parameter, value))
+
+                """
+                For each config line, create a markdown file
+                """
+                parameterDocFile = subFolder + '/' + parameter + ".md"
+
+                if not os.path.exists(parameterDocFile): # File does not exist yet, generate template
+                    with open(parameterDocFile, 'w') as paramFileHandle:
+                        content = parameterTemplate
+                        content = content.replace("$NAME", parameter)
+                        content = content.replace("$DEFAULT", value)
+
+                        paramFileHandle.write(content)

generate-param-docs/parameter-docs/Alignment/AlignmentAlgo.md

@@ -0,0 +1,6 @@
+# Parameter `AlignmentAlgo`
+Default Value: `Default`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/Alignment/FlipImageSize.md

@@ -0,0 +1,6 @@
+# Parameter `FlipImageSize`
+Default Value: `false`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/Alignment/InitialMirror.md

@@ -0,0 +1,6 @@
+# Parameter `InitialMirror`
+Default Value: `false`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/Alignment/InitialRotate.md

@@ -0,0 +1,6 @@
+# Parameter `InitialRotate`
+Default Value: `179`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/Alignment/SearchFieldX.md

@@ -0,0 +1,6 @@
+# Parameter `SearchFieldX`
+Default Value: `20`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/Alignment/SearchFieldY.md

@@ -0,0 +1,6 @@
+# Parameter `SearchFieldY`
+Default Value: `20`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/Analog/CNNGoodThreshold.md

@@ -0,0 +1,6 @@
+# Parameter `CNNGoodThreshold`
+Default Value: `0.5`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/Analog/ExtendedResolution.md

@@ -0,0 +1,6 @@
+# Parameter `ExtendedResolution`
+Default Value: `true`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/Analog/LogImageLocation.md

@@ -0,0 +1,6 @@
+# Parameter `LogImageLocation`
+Default Value: `/log/analog`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/Analog/LogfileRetentionInDays.md

@@ -0,0 +1,6 @@
+# Parameter `LogfileRetentionInDays`
+Default Value: `3`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/Analog/Model.md

@@ -0,0 +1,6 @@
+# Parameter `Model`
+Default Value: `/config/ana-cont_11.3.1_s2.tflite`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/AutoTimer/AutoStart.md

@@ -0,0 +1,6 @@
+# Parameter `AutoStart`
+Default Value: `true`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/AutoTimer/Intervall.md

@@ -0,0 +1,6 @@
+# Parameter `Intervall`
+Default Value: `5`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/DataLogging/DataLogActive.md

@@ -0,0 +1,6 @@
+# Parameter `DataLogActive`
+Default Value: `true`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/DataLogging/DataLogRetentionInDays.md

@@ -0,0 +1,6 @@
+# Parameter `DataLogRetentionInDays`
+Default Value: `3`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/Debug/Logfile.md

@@ -0,0 +1,6 @@
+# Parameter `Logfile`
+Default Value: `1`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/Debug/LogfileRetentionInDays.md

@@ -0,0 +1,6 @@
+# Parameter `LogfileRetentionInDays`
+Default Value: `3`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/Digits/CNNGoodThreshold.md

@@ -0,0 +1,6 @@
+# Parameter `CNNGoodThreshold`
+Default Value: `0.5`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/Digits/LogImageLocation.md

@@ -0,0 +1,6 @@
+# Parameter `LogImageLocation`
+Default Value: `/log/digit`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/Digits/LogfileRetentionInDays.md

@@ -0,0 +1,6 @@
+# Parameter `LogfileRetentionInDays`
+Default Value: `3`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/Digits/Model.md

@@ -0,0 +1,6 @@
+# Parameter `Model`
+Default Value: `/config/dig-cont_0600_s3.tflite`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/GPIO/IO0.md

@@ -0,0 +1,6 @@
+# Parameter `IO0`
+Default Value: `input disabled 10 false false`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/GPIO/IO1.md

@@ -0,0 +1,6 @@
+# Parameter `IO1`
+Default Value: `input disabled 10 false false`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/GPIO/IO12.md

@@ -0,0 +1,6 @@
+# Parameter `IO12`
+Default Value: `input-pullup disabled 10 false false`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/GPIO/IO13.md

@@ -0,0 +1,6 @@
+# Parameter `IO13`
+Default Value: `input-pullup disabled 10 false false`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/GPIO/IO3.md

@@ -0,0 +1,6 @@
+# Parameter `IO3`
+Default Value: `input disabled 10 false false`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/GPIO/IO4.md

@@ -0,0 +1,6 @@
+# Parameter `IO4`
+Default Value: `built-in-led disabled 10 false false`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/GPIO/LEDColor.md

@@ -0,0 +1,6 @@
+# Parameter `LEDColor`
+Default Value: `150 150 150`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/GPIO/LEDNumbers.md

@@ -0,0 +1,6 @@
+# Parameter `LEDNumbers`
+Default Value: `2`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/GPIO/LEDType.md

@@ -0,0 +1,6 @@
+# Parameter `LEDType`
+Default Value: `WS2812`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/GPIO/MainTopicMQTT.md

@@ -0,0 +1,6 @@
+# Parameter `MainTopicMQTT`
+Default Value: `wasserzaehler/GPIO`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/InfluxDB/Database.md

@@ -0,0 +1,6 @@
+# Parameter `Database`
+Default Value: ``
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/InfluxDB/Measurement.md

@@ -0,0 +1,6 @@
+# Parameter `Measurement`
+Default Value: `undefined`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/InfluxDB/Uri.md

@@ -0,0 +1,6 @@
+# Parameter `Uri`
+Default Value: `undefined`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/InfluxDB/password.md

@@ -0,0 +1,6 @@
+# Parameter `password`
+Default Value: `undefined`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/InfluxDB/user.md

@@ -0,0 +1,6 @@
+# Parameter `user`
+Default Value: `undefined`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/MQTT/ClientID.md

@@ -0,0 +1,6 @@
+# Parameter `ClientID`
+Default Value: `watermeter`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/MQTT/HomeassistantDiscovery.md

@@ -0,0 +1,6 @@
+# Parameter `HomeassistantDiscovery`
+Default Value: `true`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/MQTT/MainTopic.md

@@ -0,0 +1,6 @@
+# Parameter `MainTopic`
+Default Value: `watermeter`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/MQTT/MeterType.md

@@ -0,0 +1,6 @@
+# Parameter `MeterType`
+Default Value: `other`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/MQTT/SetRetainFlag.md

@@ -0,0 +1,6 @@
+# Parameter `SetRetainFlag`
+Default Value: `true`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/MQTT/Uri.md

@@ -0,0 +1,6 @@
+# Parameter `Uri`
+Default Value: `mqtt://IP-ADRESS:1883`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/MQTT/password.md

@@ -0,0 +1,6 @@
+# Parameter `password`
+Default Value: `PASSWORD`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/MQTT/user.md

@@ -0,0 +1,6 @@
+# Parameter `user`
+Default Value: `USERNAME`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/MakeImage/Brightness.md

@@ -0,0 +1,6 @@
+# Parameter `Brightness`
+Default Value: `0`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/MakeImage/Contrast.md

@@ -0,0 +1,6 @@
+# Parameter `Contrast`
+Default Value: `0`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/MakeImage/Demo.md

@@ -0,0 +1,6 @@
+# Parameter `Demo`
+Default Value: `true`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/MakeImage/FixedExposure.md

@@ -0,0 +1,6 @@
+# Parameter `FixedExposure`
+Default Value: `false`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/MakeImage/ImageQuality.md

@@ -0,0 +1,6 @@
+# Parameter `ImageQuality`
+Default Value: `12`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/MakeImage/ImageSize.md

@@ -0,0 +1,6 @@
+# Parameter `ImageSize`
+Default Value: `VGA`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/MakeImage/LEDIntensity.md

@@ -0,0 +1,6 @@
+# Parameter `LEDIntensity`
+Default Value: `50`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/MakeImage/LogImageLocation.md

@@ -0,0 +1,6 @@
+# Parameter `LogImageLocation`
+Default Value: `/log/source`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/MakeImage/LogfileRetentionInDays.md

@@ -0,0 +1,6 @@
+# Parameter `LogfileRetentionInDays`
+Default Value: `15`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/MakeImage/Saturation.md

@@ -0,0 +1,6 @@
+# Parameter `Saturation`
+Default Value: `0`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/MakeImage/WaitBeforeTakingPicture.md

@@ -0,0 +1,6 @@
+# Parameter `WaitBeforeTakingPicture`
+Default Value: `5`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/PostProcessing/AllowNegativeRates.md

@@ -0,0 +1,6 @@
+# Parameter `AllowNegativeRates`
+Default Value: `false`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/PostProcessing/CheckDigitIncreaseConsistency.md

@@ -0,0 +1,6 @@
+# Parameter `CheckDigitIncreaseConsistency`
+Default Value: `false`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/PostProcessing/ErrorMessage.md

@@ -0,0 +1,6 @@
+# Parameter `ErrorMessage`
+Default Value: `true`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/PostProcessing/PreValueAgeStartup.md

@@ -0,0 +1,6 @@
+# Parameter `PreValueAgeStartup`
+Default Value: `720`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/PostProcessing/PreValueUse.md

@@ -0,0 +1,6 @@
+# Parameter `PreValueUse`
+Default Value: `true`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/PostProcessing/main.AnalogDigitalTransitionStart.md

@@ -0,0 +1,6 @@
+# Parameter `main.AnalogDigitalTransitionStart`
+Default Value: `9.2`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/PostProcessing/main.DecimalShift.md

@@ -0,0 +1,6 @@
+# Parameter `main.DecimalShift`
+Default Value: `0`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/PostProcessing/main.ExtendedResolution.md

@@ -0,0 +1,6 @@
+# Parameter `main.ExtendedResolution`
+Default Value: `false`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/PostProcessing/main.IgnoreLeadingNaN.md

@@ -0,0 +1,6 @@
+# Parameter `main.IgnoreLeadingNaN`
+Default Value: `true`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/PostProcessing/main.MaxRateType.md

@@ -0,0 +1,6 @@
+# Parameter `main.MaxRateType`
+Default Value: `AbsoluteChange`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/PostProcessing/main.MaxRateValue.md

@@ -0,0 +1,6 @@
+# Parameter `main.MaxRateValue`
+Default Value: `0.05`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/System/AutoAdjustSummertime.md

@@ -0,0 +1,6 @@
+# Parameter `AutoAdjustSummertime`
+Default Value: `false`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/System/Hostname.md

@@ -0,0 +1,6 @@
+# Parameter `Hostname`
+Default Value: `undefined`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/System/SetupMode.md

@@ -0,0 +1,6 @@
+# Parameter `SetupMode`
+Default Value: `true`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/System/TimeServer.md

@@ -0,0 +1,6 @@
+# Parameter `TimeServer`
+Default Value: `pool.ntp.org`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/parameter-docs/System/TimeZone.md

@@ -0,0 +1,6 @@
+# Parameter `TimeZone`
+Default Value: `CET-1CEST,M3.5.0,M10.5.0/3`
+
+## Description
+Please fill me with an explanation and useful links
+

generate-param-docs/templates/overview.md

@@ -0,0 +1,7 @@
+# Parameters
+This page lists all available [Configuration](../Configuration) Parameters.
+ 
+!!! Note
+    This is an auto-generated page! See the [README](https://github.com/jomjol/AI-on-the-edge-device-docs/blob/main/README.md) for details!
+
+---

generate-param-docs/templates/parameter.md

@@ -0,0 +1,6 @@
+# Parameter `$NAME`
+Default Value: `$DEFAULT`
+
+## Description
+Please fill me with an explanation and useful links
+