Compare commits

...

188 Commits

Author SHA1 Message Date
jomjol
aca60465f0 v11.1.1 2022-08-22 18:20:00 +02:00
jomjol
57bdca37fc v11.1.1 2022-08-22 18:12:08 +02:00
jomjol
6409397770 Merge pull request #933 from haverland/rolling
Fix for #921, #919
2022-08-22 17:58:30 +02:00
Frank Haverland
974044adf0 last analog result_float as Readout parameter, testcases for #921, #919 2022-08-22 16:10:07 +02:00
Frank Haverland
59aeeda786 Merge branch 'rolling' into analogtodig_as_float 2022-08-22 10:33:10 +02:00
Frank Haverland
b6bf8d992f Test case for postprocessing 2022-08-22 10:31:25 +02:00
jomjol
9d31edc67a Rolling 2022-08-21 2022-08-21 21:33:56 +02:00
jomjol
c9a879d329 v11.1.0 2022-08-21 17:56:46 +02:00
jomjol
ea69b1be00 v11.1.0 2022-08-21 17:44:34 +02:00
jomjol
2a8b3a87ea Merge pull request #922 from haverland/rolling
Rolling
2022-08-21 16:47:07 +02:00
jomjol
52783734ce Merge pull request #925 from jochenchrist/master
Remove .DS_Store
2022-08-21 16:44:12 +02:00
jomjol
0e1b390ec6 Merge pull request #918 from ppisljar/patch-1
Update FeatureRequest.md
2022-08-21 16:41:30 +02:00
jochen
ab49bdf82f .DS_Store removed 2022-08-20 19:25:52 +02:00
jochenchrist
25e7051271 Delete .DS_Store 2022-08-20 19:23:08 +02:00
Frank Haverland
7315f9adfc Merge branch 'jomjol:rolling' into rolling 2022-08-19 21:33:08 +02:00
Frank Haverland
af1aee4ac3 add testcase for #921 2022-08-19 21:30:11 +02:00
Frank Haverland
d6ff7eef88 fix problems with early transition of digits if analog pointers. #921 2022-08-19 21:05:23 +02:00
Frank Haverland
7706b4dbc3 fix for #919 the prev is int, so <9.0 instead of <9.5 2022-08-18 19:28:13 +02:00
Peter Pisljar
3561ecd2b7 Update FeatureRequest.md 2022-08-18 15:16:57 +02:00
jomjol
74c7ff7fdf v11.0.1 2022-08-15 22:48:42 +02:00
jomjol
a68ce353ad Merge pull request #910 from haverland/rolling
Fix naming of models and new version
2022-08-13 15:58:57 +02:00
Frank Haverland
0d168f3445 Merge branch 'jomjol:rolling' into rolling 2022-08-13 15:38:57 +02:00
Frank Haverland
073e04a3cc fix naming of models and new versions 2022-08-13 15:37:04 +02:00
jomjol
591dc048d4 v11.0.0 2022-08-13 14:26:04 +02:00
jomjol
bfe8d3b37a v11.0.0 2022-08-13 14:20:40 +02:00
jomjol
9695dba415 Merge branch 'master' into rolling 2022-08-07 21:20:28 +02:00
jomjol
6a48f0502e Merge pull request #885 from haverland/rolling
CNNThreshold removed vor Analog100 and Digital100
2022-08-07 21:19:37 +02:00
Frank Haverland
4a8d6592ab CNNThreshold removed for Analog100 and Digital100 2022-07-28 19:43:45 +02:00
jomjol
434aebd641 Merge pull request #881 from haverland/rolling
Ignore hidden files in configuration->model selection
2022-07-25 19:11:29 +02:00
Frank Haverland
c124c38e70 ignore hidden files in configuration->model selection 2022-07-25 16:30:11 +02:00
Frank Haverland
e6d60bb124 Merge branch 'jomjol:rolling' into rolling 2022-07-24 20:20:53 +02:00
jomjol
085ea2028c v10.6.1 2022-07-24 19:07:43 +02:00
jomjol
0e7c600cf7 Rolling v10.6.1 2022-07-24 18:53:25 +02:00
Frank Haverland
3f3532defe Revert "Fix for #712 "Incorrect rollover digital numbers""
This reverts commit 11bfaf0e91.
2022-07-20 19:12:19 +02:00
Frank Haverland
a0ffc88e47 Merge branch 'rolling' of https://github.com/haverland/AI-on-the-edge-device into rolling 2022-07-20 18:37:05 +02:00
Frank Haverland
11bfaf0e91 Fix for #712 "Incorrect rollover digital numbers" 2022-07-20 18:35:42 +02:00
jomjol
189093d548 Merge pull request #862 from haverland/rolling
Version 1.0 of digital and analog categorical models
2022-07-18 20:33:02 +02:00
Frank Haverland
34eb89b1b6 fix extension finding for tlfite modellist 2022-07-18 19:18:09 +02:00
Frank Haverland
b725d242d3 Add Error to Logfile if output-dimenstion of model inconsistent. 2022-07-18 18:36:42 +02:00
Frank Haverland
568e88314b Merge branch 'jomjol:rolling' into rolling 2022-07-17 20:00:28 +02:00
jomjol
aaad952782 v10.6.0 2022-07-17 09:34:27 +02:00
jomjol
dc27911174 Preparation for v10.6.0 2022-07-17 09:00:10 +02:00
Frank Haverland
42678ae8e1 Merge branch 'jomjol:rolling' into rolling 2022-07-16 23:09:41 +02:00
Frank Haverland
b6341992f6 Version 1.0 of digital and analog categorical models 2022-07-16 21:29:56 +02:00
jomjol
17fd0f96df Rolling 20220716_2 2022-07-16 20:59:09 +02:00
jomjol
0b039e8d8c Merge pull request #861 from haverland/rolling
fix false value if analog + dighybrid on last digit if previous >=9.5
2022-07-16 20:07:02 +02:00
Frank Haverland
eab8a6d96b fix false value if analog + dighybrid on last digit if previous >=9.5 2022-07-16 19:37:30 +02:00
jomjol
9f72bf117e Rolling 20220716 2022-07-16 08:09:18 +02:00
jomjol
058e9436f0 Merge pull request #858 from haverland/rolling
Rolling: Add analog classification model
2022-07-16 07:50:00 +02:00
Frank Haverland
ebd8fe0e4f Merge branch 'rolling' of https://github.com/haverland/AI-on-the-edge-device into rolling 2022-07-03 21:02:00 +02:00
Frank Haverland
7970f5f691 Add analog classification (100) as modeltype analog100 2022-07-03 21:01:42 +02:00
jomjol
0689ca86c5 Merge branch 'rolling' of https://github.com/jomjol/AI-on-the-edge-device into rolling 2022-07-01 07:27:28 +02:00
jomjol
5783e0f182 Rolling 20220107 2022-07-01 07:27:26 +02:00
jomjol
e190aa8d03 Merge pull request #839 from haverland/rolling
Rolling: Add testing for CNNFlowcontroll
2022-06-28 07:09:35 +02:00
Frank Haverland
15bac02cbf Merge branch 'jomjol:rolling' into rolling 2022-06-26 23:23:16 +02:00
Frank Haverland
7fdacce9a2 Merge branch 'rolling' of https://github.com/haverland/AI-on-the-edge-device into rolling 2022-06-26 23:21:39 +02:00
Frank Haverland
78ea179e9a added testing to cnnflowcontroll::ZeigerEval/ZeigerEvalHybrid 2022-06-26 23:21:01 +02:00
jomjol
2cfc3fadb2 Rolling 20220626 2022-06-26 21:57:20 +02:00
jomjol
60cdee2ea6 Merge pull request #834 from haverland/rolling
Rolling: Fix -1 issue on watermeter (digital100 + analog)
2022-06-26 21:12:43 +02:00
Frank Haverland
a17ac6860d Merge branch 'jomjol:rolling' into rolling 2022-06-26 13:47:06 +02:00
Frank Haverland
ef24466702 revert comment out in ZeigerEval
use ZeigerEvalHybrid instead of ZeigerEval in DoubleHyprid10/Digital100 branch
2022-06-26 13:44:55 +02:00
jomjol
de78027b0e Merge pull request #837 from toolsfactory/master
Added Homie Mqtt request
2022-06-26 12:11:35 +02:00
jomjol
28c40a0ad2 Merge branch 'rolling' into master 2022-06-26 12:11:02 +02:00
Michael Geissler
230bbb2239 Added Homie Mqtt request 2022-06-26 08:14:41 +02:00
Frank Haverland
bda2913a32 cleanup, disable debugging 2022-06-24 14:59:51 +02:00
Frank Haverland
78daaf6e5b fix -1 on the last digital digit if using digital100 model 2022-06-24 14:58:00 +02:00
jomjol
8939167a39 Rolling 20220618_v3 2022-06-18 16:01:25 +02:00
jomjol
b48c6111d9 Merge pull request #827 from haverland/rolling
Rolling: Add hostname to title of configuration
2022-06-18 16:00:24 +02:00
Frank Haverland
de1e919b96 add hostname to configure 2022-06-18 15:39:54 +02:00
jomjol
35a96027f1 Rolling 20220618_v2 2022-06-18 14:42:30 +02:00
jomjol
7a36bfa2ff Rolling 20220618 2022-06-18 11:10:31 +02:00
jomjol
dfeac0cb7f Rolling 20220609 2022-06-09 21:16:05 +02:00
jomjol
dfec780157 Merge pull request #817 from haverland/rolling
Digital100 model processing added
2022-06-01 20:34:43 +02:00
Frank Haverland
4cc93324f8 Digital100 model 2022-05-31 22:33:12 +02:00
Frank Haverland
bf8833eae6 add Digital100 model processing with output 0.0-9.9 as categorical network 2022-05-30 23:22:17 +02:00
jomjol
00028010ee Rolling 20220526 2022-05-26 20:31:26 +02:00
jomjol
cce812ff11 Rolling 20220417 2022-04-17 21:51:26 +02:00
jomjol
ccb4bd595c Rolling 20220415 2022-04-15 14:49:58 +02:00
jomjol
dc7eedcd8f Merge pull request #774 from wetneb/534-influxdb
InfluxDB integration
2022-04-15 14:39:53 +02:00
jomjol
40faa78929 Merge branch 'rolling' into 534-influxdb 2022-04-15 14:39:09 +02:00
Antonin Delpeuch
eb53db00d0 First draft of InfluxDB integration, for #534 2022-04-15 10:19:14 +02:00
jomjol
658ef39736 Rolling 20220322 2022-03-22 20:51:55 +01:00
jomjol
0e90bcb2ef Rolling 20220320 2022-03-20 21:36:44 +01:00
jomjol
a020fce2d7 Merge pull request #725 from mkelley88/master
Update README.md
2022-03-15 20:27:34 +01:00
Matthew T. Kelley
525ccf7aba Update README.md
Made changes to increase clarity of documentation.
2022-03-15 04:50:37 -04:00
jomjol
ebcfc16f63 Create dig-s0-q-20220224.tflite 2022-02-24 21:17:54 +01:00
jomjol
4b825efffb v10.5.2 2022-02-22 19:17:20 +01:00
jomjol
71871016d2 v10.5.1 2022-02-20 16:24:05 +01:00
jomjol
c07ef23d5b v10.5.1 2022-02-20 16:05:36 +01:00
jomjol
cbe884ad63 v10.5.0 2022-02-18 21:22:00 +01:00
jomjol
1dd703b337 v10.5.0 2022-02-18 21:14:04 +01:00
jomjol
bcb1d98191 Rolling 20220215 2022-02-15 21:29:09 +01:00
jomjol
1f5486e8cc Rolling 20220215 2022-02-15 21:26:38 +01:00
jomjol
1371be6f2c v10.4.0 2022-02-12 09:11:43 +01:00
jomjol
b0d8ed6248 v10.4.0 2022-02-12 09:02:00 +01:00
jomjol
379f4585e3 Rolling 20220211 2022-02-11 21:38:40 +01:00
jomjol
45a71981c8 Rolling 20220208 2022-02-08 19:36:25 +01:00
jomjol
ac3409f644 Update FeatureRequest.md 2022-02-06 20:24:59 +01:00
jomjol
11c33f81dd Rolling 20220206 2022-02-06 19:50:47 +01:00
jomjol
641cc860d8 v10.3.0 2022-01-29 15:59:13 +01:00
jomjol
2029bd6e8a Rolling 20220122 2022-01-29 15:41:56 +01:00
jomjol
1ca5e1218d Rolling 20220128 2022-01-28 19:17:05 +01:00
jomjol
887c704f63 Rolling 20220127 2022-01-27 21:43:54 +01:00
jomjol
567dc74cd7 Rolling 20220123 2022-01-23 19:47:34 +01:00
jomjol
53606d5055 Rolling 20220121 2022-01-21 20:50:54 +01:00
jomjol
19a6c21c44 Rolling 20220118 2022-01-18 07:16:38 +01:00
jomjol
3a4b11e4b3 v10.2.0 2022-01-14 21:03:38 +01:00
jomjol
d981574ab2 Update FeatureRequest.md 2022-01-14 20:42:24 +01:00
jomjol
eb817739b9 v10.2.0 2022-01-14 20:34:59 +01:00
jomjol
8972f17638 Merge branch 'wifi-mod' 2022-01-14 20:28:42 +01:00
jomjol
4d997d9473 v10.2.0 2022-01-14 20:28:01 +01:00
jomjol
e79c86c7b6 v10.2.0 2022-01-14 20:18:24 +01:00
jomjol
96bb86536f v10.1.1 2022-01-12 21:02:05 +01:00
jomjol
2daf6c8b3f v10.1.0 2022-01-09 09:47:16 +01:00
jomjol
24b46158de v10.1.0 2022-01-09 09:43:22 +01:00
jomjol
63d336ba28 Rolling 20220104 2022-01-04 21:17:20 +01:00
jomjol
8dd3a92671 Rolling 20220103 2022-01-03 17:32:01 +01:00
jomjol
8e8897c70d v10.0.2 2022-01-01 08:57:07 +01:00
jomjol
eac513411e v10.0.2 2022-01-01 08:53:55 +01:00
jomjol
98f9274085 V10.0.1 2021-12-31 09:02:59 +01:00
jomjol
884dd9fc3b V10.0.0 2021-12-30 18:38:15 +01:00
jomjol
04c564936a Master 10.0.0 2021-12-30 18:29:01 +01:00
jomjol
957138960a Rolling 20211230 2021-12-30 11:02:20 +01:00
jomjol
58a0297915 Rolling 20211223 2021-12-23 08:03:46 +01:00
jomjol
61bf536207 Rolling 20211212 2021-12-12 19:30:07 +01:00
jomjol
4136a7b372 Merge pull request #456 from mad2xlc/master
tabindex to html
2021-12-12 17:45:25 +01:00
Stefan
b3afc9961d Merge branch 'jomjol:master' into master 2021-12-12 10:36:53 +01:00
jomjol
27e2e042da Rolling 2021-21-03 2021-12-03 18:12:45 +01:00
Frederik Kemner
cc659bad30 Time based flow rate limiting
Take time since last valid value into account for flow rate limiting
2021-12-03 11:27:20 +01:00
jomjol
776931c0ad v9.2.0 2021-12-02 21:56:05 +01:00
jomjol
e22b4b6fe1 v9.2.0 2021-12-02 21:52:45 +01:00
jomjol
cad534bffe update wiki 2021-12-02 21:47:37 +01:00
jomjol
3b44adb6fa Rolling 20211128 2021-11-28 09:09:24 +01:00
jomjol
cc0bd1473f Rolling 20211124 2021-11-24 07:32:03 +01:00
jomjol
58124d27bf Rolling 2021-11-23 2021-11-23 17:57:07 +01:00
Sven Rojek
9ad118814a add basic exception handling for the camera module 2021-11-21 19:05:17 +01:00
jomjol
270f8dd093 v9.1.1 2021-11-16 07:15:55 +01:00
jomjol
fde0ae4781 v9.1.0 2021-11-14 08:57:19 +01:00
jomjol
b87ce8c75d Merge branch 'master' into rolling 2021-11-14 08:52:40 +01:00
jomjol
e372c87836 v9.1.0 2021-11-14 08:52:28 +01:00
jomjol
f8478d7742 Merge branch 'master' into rolling 2021-11-14 08:48:42 +01:00
jomjol
a3d6923fec v9.1.0 2021-11-14 08:47:55 +01:00
jomjol
7bfa22187d Merge branch 'pr/398' into rolling 2021-11-14 08:19:13 +01:00
pixel::doc
7a8affae32 Update Helper.cpp
Change "open config file" to "open file"
2021-11-14 01:05:40 +01:00
jomjol
e48dd7c4c4 rolling 20211106 2021-11-06 22:46:16 +01:00
mad2xlc
6d298c1746 added tabindex for coordinate fields 2021-11-04 17:17:22 +01:00
jomjol
4fe9ab92e0 image update 2021-10-27 18:57:07 +02:00
jomjol
f2ac4cdc64 v9.0.0 2021-10-23 16:35:40 +02:00
jomjol
be902401d1 Merge branch 'rolling' 2021-10-23 16:32:44 +02:00
jomjol
020e93bca2 v9.0.0 2021-10-23 16:31:55 +02:00
jomjol
61dfdc2258 Add files via upload 2021-10-23 15:44:46 +02:00
jomjol
a98e3c8eab Add files via upload 2021-10-22 21:31:21 +02:00
jomjol
58a90ff706 v8.5.0 2021-10-07 07:45:40 +02:00
jomjol
d0bf12f3d4 v8.5.0 2021-10-07 07:16:46 +02:00
jomjol
af16785bbf Rolling 20211002 2021-10-02 14:59:09 +02:00
jomjol
18f6e83a2c v8.4.0 2021-09-25 18:57:40 +02:00
jomjol
147d97421b Merge branch 'rolling' 2021-09-25 18:53:47 +02:00
jomjol
dcf2feb7aa v8.4.0 2021-09-25 18:53:14 +02:00
jomjol
e63e940b96 v8.4.0 2021-09-25 08:08:21 +02:00
jomjol
68b0fb83ee v8.4.0 2021-09-24 19:57:48 +02:00
jomjol
f15e5f060a v8.4.0 2021-09-23 18:43:53 +02:00
jomjol
e2a403441f Rolling 20210922 2021-09-22 22:13:08 +02:00
jomjol
9b3665b9c6 Rolling 20210921 v2 2021-09-21 19:41:20 +02:00
jomjol
f4c8bf9206 Rolling 20210921 2021-09-21 18:49:32 +02:00
jomjol
c033db9c31 Rolling 20210921 2021-09-21 07:27:46 +02:00
jomjol
9300526f49 Rolling 2021-09-20 2021-09-20 21:18:34 +02:00
jomjol
b6dd1f7f2d Update 2021-09-14 20:00:45 +02:00
jomjol
1e6eddca04 Rolling 20210913 2021-09-13 20:05:54 +02:00
jomjol
19ca0d7dd7 Update Versioninfo 2021-09-12 07:33:30 +02:00
jomjol
7fcb5d1c0c v8.3.0 2021-09-12 07:29:30 +02:00
jomjol
dd995ec28a Rolling 20210910 2021-09-10 09:26:52 +02:00
jomjol
af99de3535 IgnoreLeadingNaN 2021-09-02 11:04:01 +02:00
jomjol
3567cc2fb0 Merge pull request #330 from pixeldoc2000/pixeldoc2000-patch-1
Pixeldoc2000 patch 1
2021-09-02 11:00:49 +02:00
pixel::doc
5e9d9bd264 Update edit_config_param.html
Fixed some more Text.
2021-09-02 10:09:36 +02:00
pixel::doc
62447c1bb9 Update edit_config_param.html
Fixed some Text
2021-09-02 00:23:59 +02:00
jomjol
a86434c9a2 Rolling 20210831 2021-08-31 11:40:29 +02:00
jomjol
b7b70299f7 Rolling 20210830 2021-08-30 21:21:18 +02:00
jomjol
eb02e0aec1 new images 2021-08-29 20:57:21 +02:00
jomjol
7816e53db7 v8.2.0 2021-08-24 08:37:18 +02:00
jomjol
7ae08e572a Merge branch 'rolling' 2021-08-24 08:34:32 +02:00
jomjol
47d15d8adb v8.2.0 2021-08-24 08:33:56 +02:00
jomjol
0dac0e87e4 rolling 20210823 2021-08-23 18:57:48 +02:00
jomjol
b290099d5b v12.0.0 2021-08-12 07:25:12 +02:00
jomjol
f6b1a41a0b v12.0.0 2021-08-12 07:20:58 +02:00
jomjol
e529af04cf Update FeatureRequest.md 2021-08-10 20:19:05 +02:00
jomjol
6c365dd949 rolling v20210809 2021-08-09 21:53:07 +02:00
jomjol
32f15fc557 rolling 20210708 2021-08-07 15:25:27 +02:00
jomjol
6f06af1d5f Update README.md 2021-08-01 21:52:00 +02:00
636 changed files with 52747 additions and 15100 deletions

BIN
.DS_Store vendored

Binary file not shown.

3
.gitignore vendored
View File

@@ -4,6 +4,7 @@
.code-workspace
/sd-card/htm./.vscode/
/code/build
/sd-card/html/debug/
CMakeLists.txt.user
CMakeCache.txt
@@ -15,3 +16,5 @@ install_manifest.txt
compile_commands.json
CTestTestfile.cmake
_deps
code/edgeAI.code-workspace
.DS_Store

View File

@@ -1,6 +1,245 @@
# Versions
##### 10.6.2 - Stability Increase (2022-07-24)
- **NEW 10.6.2**: ignore hidden files in model selection (configuration page)
- **NEW 10.6.1**: Revoke esp32cam & tflite update
- **NEW 10.6.1**: Bug Fix: tflite-filename with ".", HTML spelling error
- IndluxDB: direct injection into InfluxDB - thanks to **[wetneb](https://github.com/wetneb)**
- MQTT: implemented "Retain Flag" and extend with absolute Change (in addition to rate)
- `config.ini`: removal of modelsize (readout from tflite)
- Updated analog neural network file (`ana1000s2.tflite`) & digital neural network file (`dig1400s2q.tflite`)
- TFMicro/Lite: Update (espressif Version 20220716)
- Updated esp32cam (v20220716)
- ESP-IDF: Update to 4.4
- Internal update (CNN algorithm optimizations, reparation for new neural network type)
- Bug Fix: no time with fixed IP, Postprocessing, MQTT
##### 10.5.2 - Stability Increase (2022-02-22)
- NEW 10.5.2: Bug Fix: wrong `firmware.bin` (no rate update)
- NEW 10.5.1: Bug Fix: wrong return value, rate value & PreValue status, HTML: SSID & IP were not displayed
- MQTT: changed wifi naming to "wifiRSSI"
- HTML: check selectable values for consistency
- Refactoring of check postprocessing consistency (e.g. max rate, negative rate, ...)
- Bug Fix: corrected error in "Check Consistency Increase"
##### 10.4.0 - Stability Increase (2022-02-12)
- Graphical configuration: select available neural network files (*.tfl, *.tflite) from drop down menu
- OTA-update: add option to upload tfl / tflite files to the correct location (`/config/`)
- In the future the new files will also be copied to the `firmware` directory of the repository
- Added Wifi RSSI to MQTT information
- Updated analog neural network file (`ana-s3-q-20220105.tflite`)
- Updated digital neural network file (`dig-s1-q-20220102.tflite`)
- Updated build environment to `Espressif 3.5.0`
##### 10.3.0 - Stability Increase (2022-01-29)
- Implemented LED flash dimming (`LEDIntensity`).
Remark: as auto illumination in the camera is used, this is rather for energy saving. It will not help reducing reflections
- Additional camera parameters: saturation, contrast (although not too much impact yet)
- Some readings will have removable "N"s that can not be removed automatically and are handled with an "error" --> no return value in the field "value" anymore (still reported back via field "raw value")
- Updated esp32 camera hardware driver
- Bug fix: MQTT, HTML improvements
**ATTENTION: The new ESP32 camera hardware driver is much more stable on newer OV2640 versions (no or much less reboots) but seems to be not fully compatible with older versions.**
* If you have problem with stalled systems you can try the following
- Update the parameter `ImageQuality` to `12` instead of current value `5` (manually in the `config.ini`)
- If this is not helping, you might need to update your hardware or stay with version 9.2
##### 10.2.0 - Stability Increase (2022-01-14)
- Due to the updated camera driver, the image looks different and a new setup might be needed
- Update reference image
- Update Alignment marks
- Reduce reboot due to camera problems
- Update esp32-camera to new version (master as of 2022-01-09)
##### 10.1.1 - Stability Increase (2022-01-12)
- Bug Fix MQTT problem
- Issue:
- Changing from v9.x to 10.x the MQTT-parameter "Topic" was renamed into "MainTopic" to address multiple number meters. This renaming should have been done automatically in the background within the graphical configuration, but was not working. Instead the parameter "Topic" was deleted and "MainTopic" was set to disabled and "undefined".
- ToDo
- Update the `html.zip`
- If old `config.ini` available: copy it to `/config`, open the graphical configuration and save it again.
- If old `config.ini` not available: reset the parameter "MainTopic" within the `config.ini` manually
- Reboot
##### 10.1.0 - Stability Increase (2022-01-09)
- Reduce ESP32 frequency to 160MHz
- Update tflite (new source: https://github.com/espressif/tflite-micro-esp-examples)
- Update analog neural network (ana-s3-q-20220105.tflite)
- Update digital neural network (dig-s1-q-20220102.tflite)
- Increased web-server buffers
- bug fix: compiler compatibility
##### 10.0.2 - Stability Increase (2022-01-01)
- NEW v10.0.2: Corrected JSON error
- Updated compiler toolchain to ESP-IDF 4.3
- Removal of memory leak
- Improved error handling during startup (check PSRAM and camera with remark in logfile)
- MQTT: implemented raw value additionally, removal of regex contrain
- Normalized Parameter ``MaxRateValue`` to "change per minute"
- HTML: improved input handling
- Corrected error handling: in case of error the old value, rate, timestamp are not transmitted any more
##### 9.2.0 - External Illumination (2021-12-02)
- Direct JSON access: ``http://IP-ADRESS/json``
- Error message in log file in case camera error during startup
- Upgrade analog CNN to v9.1.0
- Upgrade digital CNN to v13.3.0 (added new images)
- html: support of different ports
##### 9.1.1 - External Illumination (2021-11-16)
- NEW 9.1.1 bug fix: LED implemenetation
- External LEDs: change control mode (resolve bug with more than 2 LEDs)
- Additional info into log file
- Bug fix: decimal shift, html, log file
##### 9.0.0 - External Illumination (2021-10-23)
* Implementation of external illumination to adjust positioning, brightness and color of the illumination now set individually
* Technical details can be found in the wiki: https://github.com/jomjol/AI-on-the-edge-device/wiki/External-LED
<img src="https://raw.githubusercontent.com/jomjol/ai-on-the-edge-device/master/images/intern_vs_external.jpg" width="500">
* New housing published for external LEDs and small clearing: https://www.thingiverse.com/thing:5028229
##### 8.5.0 - Multi Meter Support (2021-10-07)
* Upgrade digital CNN to v13.1.0 (added new images)
* bug fix: wlan password with space, double digit output
##### 8.4.0 - Multi Meter Support (2021-09-25)
* License change (remove MIT license, remark see below)
* html: show hostname in title and main page
* configuration:
* moved setting `ExtendedResolution` to individual number settings
* New parameter `IgnoreLeadingNaN` (delete leading NaN's specifically)
* **ATTENTION**: update of the `config.ini` needed (open, adjust `ExtendedResolution`, save)
* Bug fixing (html, images of recognized numbers)
### **ATTENTION: LICENSE CHANGE - removal of MIT License.**
- Currently no licence published - copyright belongs to author
- If you are interested in a commercial usage or dedicated versions please contact the developer
- no limits to private usage
##### 8.3.0 - Multi Meter Support (2021-09-12)
* Upgrade digital CNN to v12.1.0 (added new images)
* Dedicated NaN handling, internal refactoring (CNN-Handling)
* HTML: confirmation after config.ini update
* Bug fixing
##### 8.2.0 - Multi Meter Support (2021-08-24)
* Improve server responsiveness
* Flow status and prevalue status in overview
* Improved prevalue handling
##### 8.1.0 - Multi Meter Support (2021-08-12)
* GPIO: using the general mqtt main topic for GPIO
* Upgrade digital CNN to v12.0.0 (added new images)
* Update tfmicro to new master (2021-08-07)
* Bug fix: remove text in mqtt value, remove connect limit in wlan reconnet
##### 8.0.5 - Multi Meter Support (2021-08-01)
* NEW 8.0.5: bug fix: saving prevalue
* NEW 8.0.4: bug fix: load config.ini after upgrade
* NEW 8.0.3: bug fix: reboot during `config.ini` handling, html error
* NEW 8.0.2: saving roundes prevalue, bug fix html server
* NEW 8.0.1: bug fix: html handling of parameter `FixedExposure` and `ImageSize`
* Dual / multi meter support (more than 1 number to be recognized)
This is implemented with the feature "number" on the ROI definition as well as selected options
* MQTT: standardization of the naming - including new topics (`json`, `freeMem `, `uptime`)c
* Preparation for extended GPIO support (thanks to Zwerk2k) - not tested and fully functional yet
* Bug fixing: html server, memory leak, MQTT connect, hostname, turn of flash LED
<span style="color: red;">**ATTENTION: the configuration and prevalue files are modified automatically and will not be backward compatible!**</span>
##### 7.1.2 MQTT-Update - (2021-06-17)
* NEW: 7.1.2: bug fix setting hostname, Flash-LED not off during reboot
* NEW: 7.1.1: bug fix wlan password with "=" (again)
* MQTT error message: changes "no error", send retain flag
* Update wlan handling to esp-idf 4.1
* Upgrade digital CNN to v8.7.0 (added new images)
* Bug fix: MQTT, WLAN, LED-Controll, GPIO usage, fixed IP, calculation flow rate
##### 7.0.1 MQTT-Update - (2021-05-13)
* NEW: 7.0.1: bug fix wlan password with "="
* Upgrade digital CNN to v8.5.0 (added new images)
* New MQTT topics: flow rate (units/minute), time stamp (last correct read readout)
* Update MQTT/Error topic to " " in case no error (instead of empty string)
* Portrait or landscape image orientation in rotated image (avoid cropping)
##### 6.7.2 Image Processing in Memory - (2021-05-01)

View File

@@ -11,6 +11,137 @@
____
#### #29 Add favicon and use the hostname for the website
* https://github.com/jomjol/AI-on-the-edge-device/issues/927
#### #28 Improved error handling for ROIs
* In case a ROI is out of the image, there is no error message, but a non sense image is used
* Implement a error message for wrong configuratioin of ROI
#### #27 Use Homie Spec for Mqtt binding
* Use the standardized Home Protocol for the Mqtt binding
* https://homieiot.github.io/
#### #26 Changes behaviour for "N" replacement
* in case the higher digits has already increased by minium 1 - don't set the "N" to the last value, but to "0"
* https://github.com/jomjol/AI-on-the-edge-device/issues/792
#### #25 Trigger Measurement via MQTT
* https://github.com/jomjol/AI-on-the-edge-device/issues/727
#### #24 Show Mqtt state directly in Webserver
* Show MQTT log in Web page. E.g. connection established or failed to connect...
#### #23 CPU Temp and Mqtt values
* Show the CPU Temp directly in Webpage. Also add the value to MQTT sending
#### #22 Direct hint to the different neural network files in the other repositories
* https://github.com/jomjol/AI-on-the-edge-device/issues/644
#### #21 Extended "CheckDigitalConsistency" Logik
* https://github.com/jomjol/AI-on-the-edge-device/issues/590
#### #20 Deep sleep and push mode
* Let the device be normally in deep sleep state, and wake it up periodically to collect data and push it via MQTT or HTTP post.
* Support ESP-NOW to reduce the overhead of connecting to wifi and mqtt
* the above should enable battery powered applications
#### #19 Extended log informations
* https://github.com/jomjol/AI-on-the-edge-device/issues/580
#### ~~#18 Document WLAN-strength in web page~~
* ~~https://github.com/jomjol/AI-on-the-edge-device/issues/563~~
#### ~~#17 Direct InfluxDB connection~~
* ~~Done in v10.6.0~~
#### #16 Serial Communication
* https://github.com/jomjol/AI-on-the-edge-device/issues/512
* Send the readout value via RX/TX interface with a dedicated TAG
* Make dedicated communication FlowModule
* Modification of RX/TX communication
* Configuration interfache
#### #15 Calibration for FishEye image
* https://github.com/jomjol/AI-on-the-edge-device/issues/507
1. The development of such a correction algorithm with the libraries, that are available for the ESP32 environment.
2. New module for integration of the flow into the image processing flow.
3. Extension of the configuration (config.ini) and html-pages
4. Parameter adjustment and testing for every different fish-eye module
5. Maintenance for further updates / modules, ...
#### #14 Backup and restore option for configuration
* https://github.com/jomjol/AI-on-the-edge-device/issues/459
* Implement a zip file compression for store and restore
* Update the html to handle it
#### #13 Manage non linear gauge without CNN re-training
* https://github.com/jomjol/AI-on-the-edge-device/issues/443
* Implement a look up table for non linear analog meters
#### ~~#12 Less reboots due to memory leakage~~
* ~~Issue: #414 & #425 #430~~
#### #11 MQTT - configurable payload
* https://github.com/jomjol/AI-on-the-edge-device/issues/344
#### #10 Improve and bug fix logging of images
* https://github.com/jomjol/AI-on-the-edge-device/issues/307
#### #9 Basic auth for the UI
* https://github.com/jomjol/AI-on-the-edge-device/issues/283
@@ -114,4 +245,4 @@ ____
* ~~Implementation of a software module for external light source (e.g. WS8132 LED controller, ...)~~
* ~~Update of the camera module to use the external light instead of the internal flash light~~
* ~~Adopt the configuration algorithm with a configurable light source~~
* ~~Adopt the configuration algorithm with a configurable light source~~

21
LICENSE
View File

@@ -1,21 +0,0 @@
MIT License
Copyright (c) 2020 jomjol
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -4,9 +4,10 @@ This is an example of Artificial Intelligence (AI) calculations on a very cheap
### Details on **function**, **installation** and **configuration** can be found on the **[Wiki Page](https://github.com/jomjol/AI-on-the-edge-device/wiki)**
A 3d-printable housing can be found here: https://www.thingiverse.com/thing:4573481
respectively ESP32-Cam housing only: https://www.thingiverse.com/thing:4571627
A 3d-printable housing can be found here:
- https://www.thingiverse.com/thing:4573481 (Water Meter)
- https://www.thingiverse.com/thing:5028229 (Power Meter)
- https://www.thingiverse.com/thing:4571627 (ESP32-Cam housing only)
<img src="https://raw.githubusercontent.com/jomjol/AI-on-the-edge-device/master/images/watermeter_all.jpg" width="200"><img src="https://raw.githubusercontent.com/jomjol/AI-on-the-edge-device/master/images/main.jpg" width="200"><img src="https://raw.githubusercontent.com/jomjol/AI-on-the-edge-device/master/images/size.png" width="200">
@@ -32,66 +33,48 @@ If you have any technical topics, you can file a issue in this repository.
In other cases you can contact the developer via email: <img src="https://raw.githubusercontent.com/jomjol/AI-on-the-edge-device/master/images/mail.jpg" height="25">
------
## Change log
**General remark:** Besides the file `firmware.bin`, typically the content of `/html` will need to be updated!
------
### Known Issues
##### 11.1.1 - Intermediate Digits (2022-08-22)
* slow response of web server during picture analysis
* spontaneous reboots (mostly due to html access during image processing) - self recovery implemented
- New and improved consistency check (especially with analog and digital counters mixed)
- Bug Fix: digital counter algorithm
------
##### 11.0.1 - Intermediate Digits (2022-08-18)
**General remark:** Beside the `firmware.bin`, typically also the content of `/html` needs to be updated!
- **NEW v11.0.1**: Bug Fix InfluxDB configuration (only update of html.zip necessary)
- Implementation of new CNN types to detect intermediate values of digits with rolling numbers
- By default the old algo (0, 1, ..., 9, "N") is active (due to the limited types of digits trained so far)
- Activation can be done by selection a tflite file with the new trained model in the 'config.ini'
- **Details can be found in the [wiki](https://github.com/jomjol/AI-on-the-edge-device/wiki/Neural-Network-Types)** (different types, trained image types, naming convention)
##### 8.0.5 - Multi Meter Support (2021-07-29)
- Updated neural network files (and adaption to new naming convention)
* NEW 8.0.5: bug fix: saving prevalue
* NEW 8.0.4: bug fix: load config.ini after upgrade
* NEW 8.0.3: bug fix: reboot during `config.ini` handling, html error
* NEW 8.0.2: saving roundes prevalue, bug fix html server
* NEW 8.0.1: bug fix: html handling of parameter `FixedExposure` and `ImageSize`
* Dual / multi meter support (more than 1 number to be recognized)
This is implemented with the feature "number" on the ROI definition as well as selected options
* MQTT: standardization of the naming - including new topics (`json`, `freeMem `, `uptime`)c
* Preparation for extended GPIO support (thanks to Zwerk2k) - not tested and fully functional yet
* Bug fixing: html server, memory leak, MQTT connect, hostname, turn of flash LED
- Published a tool to download and combine log files - **Thanks to **
<span style="color: red;">**ATTENTION: the configuration and prevalue files are modified automatically and will not be backward compatible!**</span>
- Files see ['/tools/logfile-tool'](tbd), How-to see [wiki](https://github.com/jomjol/AI-on-the-edge-device/wiki/Gasmeter-Log-Downloader)
##### 7.1.2 MQTT-Update - (2021-06-17)
* NEW: 7.1.2: bug fix setting hostname, Flash-LED not off during rebootNEW: 7.1.1: bug fix wlan password with "=" (again)
* MQTT error message: changes "no error", send retain flag
* Update wlan handling to esp-idf 4.1
* Upgrade digital CNN to v8.7.0 (added new images)
* Bug fix: MQTT, WLAN, LED-Controll, GPIO usage, fixed IP, calculation flow rate
- Bug Fix: InfluxDB enabling in grahic configuration
##### 7.0.1 MQTT-Update - (2021-05-13)
## Tools
* NEW: 7.0.1: bug fix wlan password with "="
* Upgrade digital CNN to v8.5.0 (added new images)
* New MQTT topics: flow rate (units/minute), time stamp (last correct read readout)
* Update MQTT/Error topic to " " in case no error (instead of empty string)
* Portrait or landscape image orientation in rotated image (avoid cropping)
* Logfile downloader and combiner (Thx to [reserve85](https://github.com/reserve85))
* Files see ['/tools/logfile-tool'](tbd), How-to see [wiki](https://github.com/jomjol/AI-on-the-edge-device/wiki/Gasmeter-Log-Downloader)
## Additional ideas
There are some ideas and feature request, which are not followed currently - mainly due to capacity reasons on side of the developer. They are collected here: [FeatureRequest.md](FeatureRequest.md)
## Additional Ideas
There are some ideas and feature requests which are not followed currently - mainly due to capacity reasons on side of the developer. They are collected here: [FeatureRequest.md](FeatureRequest.md)
@@ -99,6 +82,14 @@ There are some ideas and feature request, which are not followed currently - mai
## History
##### 10.6.2 - Stability Increase (2022-07-24)
##### 9.2.0 - External Illumination (2021-12-02)
##### 8.5.0 Multi Meter Support (2021-10-07)
##### 7.1.2 MQTT-Update - (2021-06-17)
##### 6.7.2 Image Processing in Memory - (2021-05-01)
##### 5.0.0 Setup Modus - (2020-12-06)
@@ -121,8 +112,3 @@ There are some ideas and feature request, which are not followed currently - mai
#### [Full Changelog](Changelog.md)
## Solved topics
* n.a.

View File

@@ -0,0 +1,3 @@
copy "..\..\code\build\esp32cam-server-only.bin" "..\..\firmware\firmware.bin"
copy "..\..\code\build\bootloader\bootloader.bin" "..\..\firmware\bootloader.bin"
copy "..\..\code\build\partition_table\partition-table.bin" "..\..\firmware\partitions.bin"

57
code/components/esp-nn/.gitignore vendored Normal file
View File

@@ -0,0 +1,57 @@
.config
*.o
*.i
*.s
*.orig
*.pyc
# gtags
GTAGS
GRTAGS
GPATH
# emacs
.dir-locals.el
# emacs temp file suffixes
*~
.#*
\#*#
# eclipse setting
.settings
# MacOS directory files
.DS_Store
# Example project files
examples/**/sdkconfig
examples/**/sdkconfig.old
examples/**/build
# Test app files
test_app/build
test_app/sdkconfig
test_app/sdkconfig.old
# Doc build artifacts
docs/_build/
docs/doxygen-warning-log.txt
docs/sphinx-warning-log.txt
docs/sphinx-warning-log-sanitized.txt
docs/xml/
docs/xml_in/
docs/man/
docs/doxygen_sqlite3.db
TEST_LOGS
# gcov coverage reports
*.gcda
*.gcno
coverage.info
coverage_report/
# VS Code Settings
.vscode/

View File

@@ -0,0 +1,55 @@
stages:
- build
variables:
BATCH_BUILD: "1"
V: "0"
MAKEFLAGS: "-j8 --no-keep-going"
IDF_PATH: "$CI_PROJECT_DIR/esp-idf"
LOG_PATH: "$CI_PROJECT_DIR"
.set_git_config: &set_git_config
# Set git config
- git config user.email "test@espressif.com"
- git config user.name "Espressif"
.add_ssh_key: &add_ssh_key
# Add gitlab ssh key
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo -n $GITLAB_KEY > ~/.ssh/id_rsa_base64
- base64 --decode --ignore-garbage ~/.ssh/id_rsa_base64 > ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa
- echo -e "Host gitlab.espressif.cn\n\tStrictHostKeyChecking no\n" >> ~/.ssh/config
before_script:
# Add gitlab ssh key
- *add_ssh_key
# Set git config
- *set_git_config
.build_esp32s3: &build_esp32s3
- idf.py set-target esp32s3 build
.build_esp32: &build_esp32
- idf.py set-target esp32 build
build_demo:
stage: build
image: $CI_DOCKER_REGISTRY/esp32-ci-env:esp-nn
tags:
- build
script:
# Clone IDF
- git clone --recursive --single-branch -b release/v4.4 --reference-if-able /local_references/gitlab/ https://gitlab-ci-token:${BOT_TOKEN}@gitlab.espressif.cn:6688/espressif/esp-idf.git
- cd esp-idf
- ./install.sh
- . ./export.sh
- cd ..
# Build examples now
- cd test_app
# Build esp32s3
- *build_esp32s3
# Build esp32
- *build_esp32
- cd -

View File

@@ -0,0 +1,48 @@
idf_build_get_property(idf_target IDF_TARGET)
set(c_srcs
"src/activation_functions/esp_nn_relu_ansi.c"
"src/basic_math/esp_nn_add_ansi.c"
"src/basic_math/esp_nn_mul_ansi.c"
"src/convolution/esp_nn_conv_ansi.c"
"src/convolution/esp_nn_depthwise_conv_ansi.c"
"src/fully_connected/esp_nn_fully_connected_ansi.c"
"src/softmax/esp_nn_softmax_ansi.c"
"src/softmax/esp_nn_softmax_opt.c"
"src/pooling/esp_nn_avg_pool_ansi.c"
"src/pooling/esp_nn_max_pool_ansi.c")
if(CONFIG_IDF_TARGET_ESP32S3)
set(s3_srcs
"src/common/esp_nn_common_functions_esp32s3.S"
"src/common/esp_nn_multiply_by_quantized_mult_esp32s3.S"
"src/common/esp_nn_multiply_by_quantized_mult_ver1_esp32s3.S"
"src/activation_functions/esp_nn_relu_s8_esp32s3.S"
"src/basic_math/esp_nn_add_s8_esp32s3.S"
"src/basic_math/esp_nn_mul_s8_esp32s3.S"
"src/convolution/esp_nn_conv_esp32s3.c"
"src/convolution/esp_nn_depthwise_conv_s8_esp32s3.c"
"src/convolution/esp_nn_conv_s16_mult8_esp32s3.S"
"src/convolution/esp_nn_conv_s16_mult8_1x1_esp32s3.S"
"src/convolution/esp_nn_conv_s16_mult4_1x1_esp32s3.S"
"src/convolution/esp_nn_depthwise_conv_s8_mult1_3x3_padded_esp32s3.S"
"src/convolution/esp_nn_depthwise_conv_s16_mult1_esp32s3.S"
"src/convolution/esp_nn_depthwise_conv_s16_mult1_3x3_esp32s3.S"
"src/convolution/esp_nn_depthwise_conv_s16_mult1_3x3_no_pad_esp32s3.S"
"src/convolution/esp_nn_depthwise_conv_s16_mult8_3x3_esp32s3.S"
"src/convolution/esp_nn_depthwise_conv_s16_mult4_esp32s3.S"
"src/convolution/esp_nn_depthwise_conv_s16_mult8_esp32s3.S"
"src/fully_connected/esp_nn_fully_connected_s8_esp32s3.S"
"src/pooling/esp_nn_max_pool_s8_esp32s3.S"
"src/pooling/esp_nn_avg_pool_s8_esp32s3.S")
endif()
idf_component_register(SRCS "${c_srcs}"
"${s3_srcs}"
INCLUDE_DIRS "include" "src/common")
if(CONFIG_IDF_TARGET_ESP32S3)
target_compile_options(${COMPONENT_LIB} PRIVATE -mlongcalls -fno-unroll-loops -O2 -Wno-unused-function)
else()
target_compile_options(${COMPONENT_LIB} PRIVATE -Wno-unused-function)
endif()

View File

@@ -0,0 +1,29 @@
menu "ESP-NN"
choice NN_OPTIMIZATIONS
bool "Optimization for nn functions"
default NN_OPTIMIZED
help
Use ANSI-C versions for verification and debug purpose.
Optimisations are automatically picked up for a chipset.
For ESP32-S3, assembly Optimisations are selected.
For ESP32, just the ANSI C versions are selected for now.
config NN_ANSI_C
bool "ANSI C"
help
ANSI C versions for verification and debug purposes.
config NN_OPTIMIZED
bool "Optimized versions"
help
Optimisations are automatically picked up for a chipset.
For ESP32-S3, assembly Optimisations are selected.
For ESP32, just the ANSI C versions are selected for now.
endchoice
config NN_OPTIMIZATIONS
int
default 0 if NN_ANSI_C
default 1 if NN_OPTIMIZED
endmenu

View File

@@ -0,0 +1,54 @@
# ESP-NN
The library contains optimised NN (Neural Network) functions for various Espressif chipsets.
* Supported platforms:
* TensorFlow Lite Micro (TFLite Micro). Repo can be found [here](https://github.com/espressif/tflite-micro-esp-examples)
* Supported ESP chipsets include:
* ESP32-S3 (Assembly versions optimised to benefit from vector instructions of ESP32-S3)
* ESP32 (ANSI C versions)
## Performance
### Kernelwise performance for s8 versions:
* Kernelwise performance on ESP32-S3 chip
* Numbers are ticks taken for kernel to execute
* Chip config: 240MHz, SPI: QPI 80MHz, Data cache: 64KB
| Function | ANSI C | ESP32-S3 Opt | Opt Ratio | Data info | Memory |
| ----------------| --------|---------|---------|-------------|-----------|
| elementwise_add | 320397 | 87119 | 3.68 | size = 1615 | External |
| elementwise_mul | 125958 | 44239 | 2.85 | size = 1615 | External |
| convolution | 4663012 | 428675 | 10.88 | input(10,10), filter(64x1x1x64) | External |
| convolution | 301014 | 32433 | 9.28 | input(8,8), filter(16x1x1x16) | External |
| convolution | 2115418 | 1020923 | 2.07 | input(10,10), filter(64x3x3x3) | External |
| depthwise conv | 1190062 | 203278 | 5.85 | input (18, 18), pad(0,0), stride(1,1) filter: 1x3x3x16 | External |
| depthwise conv | 837072 | 182335 | 4.59 | input (12, 12), pad(1,1), stride(1,1) filter: 8x5x5x4 | External |
| max pool | 485714 | 76747 | 6.33 | input(16,16), filter (1x3x3x16) | Internal |
| avg pool | 541462 | 160580 | 3.37 | input(16,16), filter (1x3x3x16) | Internal |
| fully connected | 15853 | 9547 | 1.66 | len: 265, ch = 3 | Internal |
| prelu (relu6) | 19472 | 2734 | 7.12 | size, 1615 | Internal |
## Configuration
* To configure, please use `idf.py menuconfig` and under `ESP-NN` select `NN_OPTIMIZATIONS`
* There are two options presented:
* Optimized versions
* ANSI C
* Default selection is for `Optimized versions`. For ESP32-S3, assembly versions are automatically selected, whereas for ESP32, ANSI-C versions are selected by default.
* For debugging purposes, you may want to select `ANSI C`
## Contributing
If you encounter an issue with ESP-NN, or wish to submit a feature request, please use the Issues section on the Github.
For general questions related to this library, please use the esp32.com forum.
## Copyrights and License
All original source code in this repository is Copyright (C) 2020-2021 Espressif Systems. This source code is licensed under the Apache License 2.0 as described in the file LICENSE.

View File

@@ -0,0 +1,46 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#pragma once
#if defined(CONFIG_NN_OPTIMIZED)
#ifdef CONFIG_IDF_TARGET_ESP32S3
#define ARCH_ESP32_S3 1
#endif
#ifdef CONFIG_IDF_TARGET_ESP32
#define ARCH_ESP32 1
#endif
#endif
#ifdef __cplusplus
extern "C" {
#endif
/* reference kernels included by default */
#include "esp_nn_ansi_headers.h"
#if defined(CONFIG_NN_OPTIMIZED)
#ifdef ARCH_ESP32_S3
#include "esp_nn_esp32s3.h"
#endif
#ifdef ARCH_ESP32
#include "esp_nn_esp32.h"
#endif
#else
#include "esp_nn_ansi_c.h"
#endif
#ifdef __cplusplus
}
#endif

View File

@@ -0,0 +1,46 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
/**
* @file Header definitions to include for ANSI C versions.
* These are just typedefs to pick up ANSI versions.
*/
#pragma once
#include "esp_nn_ansi_headers.h"
#define esp_nn_add_elementwise_s8 esp_nn_add_elementwise_s8_ansi
#define esp_nn_mul_elementwise_s8 esp_nn_mul_elementwise_s8_ansi
#define esp_nn_depthwise_conv_s8 esp_nn_depthwise_conv_s8_ansi
#define esp_nn_conv_s8 esp_nn_conv_s8_ansi
#define esp_nn_get_conv_scratch_size esp_nn_get_conv_scratch_size_ansi
#define esp_nn_set_conv_scratch_buf esp_nn_set_conv_scratch_buf_ansi
#define esp_nn_get_depthwise_conv_scratch_size esp_nn_get_depthwise_conv_scratch_size_ansi
#define esp_nn_set_depthwise_conv_scratch_buf esp_nn_set_depthwise_conv_scratch_buf_ansi
#define esp_nn_relu6_s8 esp_nn_relu6_s8_ansi
#define esp_nn_avg_pool_s8 esp_nn_avg_pool_s8_ansi
#define esp_nn_max_pool_s8 esp_nn_max_pool_s8_ansi
#define esp_nn_fully_connected_s8 esp_nn_fully_connected_s8_ansi
#define esp_nn_get_softmax_scratch_size esp_nn_get_softmax_scratch_size_ansi
#define esp_nn_set_softmax_scratch_buf esp_nn_set_softmax_scratch_buf_ansi
#define esp_nn_softmax_s8 esp_nn_softmax_s8_ansi

View File

@@ -0,0 +1,283 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#pragma once
/**
* @file Header definitions to include for esp_nn reference functions
*/
#include <stdint.h>
/************************** Basic math functions ****************************/
/**
* @brief elementwise addition
*
* @note inputs type: int8_t, output: int8_t
* input offsets: although int32_t, they are contained in 8 bits [-128, 127]
*
* shift values are expected to be <= 0
*/
void esp_nn_add_elementwise_s8_ansi(const int8_t *input1_data,
const int8_t *input2_data,
const int32_t input1_offset,
const int32_t input2_offset,
const int32_t input1_mult,
const int32_t input2_mult,
const int32_t input1_shift,
const int32_t input2_shift,
const int32_t left_shift,
int8_t *output,
const int32_t out_offset,
const int32_t out_mult,
const int32_t out_shift,
const int32_t activation_min,
const int32_t activation_max,
const int32_t size);
/**
* @brief elementwise multiplication
*
* @note inputs type: int8_t, output: int8_t
* input offsets: although int32_t, they are contained in 8 bits [-128, 127]
*
* output shift is expected to be <= 0
*/
void esp_nn_mul_elementwise_s8_ansi(const int8_t *input1_data,
const int8_t *input2_data,
const int32_t input1_offset,
const int32_t input2_offset,
int8_t *output,
const int32_t out_offset,
const int32_t out_mult,
const int32_t out_shift,
const int32_t activation_min,
const int32_t activation_max,
const int32_t size);
/************************** Convolution functions *****************************/
/**
* @brief depthwise convolution per channel
*
* @note inputs type: int8_t, output: int8_t
* Version used in tflite is per channel.
* This version follows the same footsprints.
* Meaning, it has per out_channel shift and multiplier for
* requantization
*
* optimization notes: Though input_offset is int32 type,
* offset values are contained in 8 bits [-128, 127]
*/
void esp_nn_depthwise_conv_s8_ansi(const int8_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const int32_t input_offset,
const uint16_t pad_wd,
const uint16_t pad_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const uint16_t ch_mult,
const int8_t *filter_data,
const uint16_t filter_wd,
const uint16_t filter_ht,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max);
/**
* @brief 2d-convolution channelwise
*
* @note operation: result += (input + offset) * filter
*
* inputs type: int8_t, output: int8_t
* input offsets: although int32_t, they are contained in 8 bits [-128, 127]
*/
void esp_nn_conv_s8_ansi(const int8_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t in_channels,
const int32_t input_offset,
const uint16_t pad_wd,
const uint16_t pad_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const int8_t *filter_data,
const uint16_t filter_wd,
const uint16_t filter_ht,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const uint16_t out_channels,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max);
int esp_nn_get_conv_scratch_size_ansi(const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t in_ch,
const uint16_t out_ch,
const uint16_t filter_wd,
const uint16_t filter_ht);
void esp_nn_set_conv_scratch_buf_ansi(const void *buf);
int esp_nn_get_depthwise_conv_scratch_size_ansi(const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const uint16_t ch_mult,
const uint16_t filter_wd,
const uint16_t filter_ht);
void esp_nn_set_depthwise_conv_scratch_buf_ansi(const void *buf);
/************************** Activation functions *****************************/
/**
* @brief relu6
*
* @note inout: int8_t
*/
void esp_nn_relu6_s8_ansi(int8_t *data, uint16_t size);
/************************** Pooling functions *****************************/
/**
* @brief max_pool
*
* @note inputs type: int8_t, output: int8_t
* input offsets: although int32_t, they are contained in 8 bits [-128, 127]
*/
void esp_nn_max_pool_s8_ansi(const int8_t *input,
const uint16_t input_wd,
const uint16_t input_ht,
int8_t *output,
const uint16_t output_wd,
const uint16_t output_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const uint16_t filter_wd,
const uint16_t filter_ht,
const uint16_t pad_wd,
const uint16_t pad_ht,
const int32_t activation_min,
const int32_t activation_max,
const uint16_t channels);
/**
* @brief avg_pool
*
* @note inputs type: int8_t, output: int8_t
* input offsets: although int32_t, they are contained in 8 bits [-128, 127]
*/
void esp_nn_avg_pool_s8_ansi(const int8_t *input,
const uint16_t input_wd,
const uint16_t input_ht,
int8_t *output,
const uint16_t output_wd,
const uint16_t output_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const uint16_t filter_wd,
const uint16_t filter_ht,
const uint16_t pad_wd,
const uint16_t pad_ht,
const int32_t activation_min,
const int32_t activation_max,
const uint16_t channels);
/************************** Fully connected functions ***********************/
/**
* @brief fully connected
*
* @note inputs type: int8_t, output: int8_t
* input offsets: although int32_t, they are contained in 8 bits [-128, 127]
*/
void esp_nn_fully_connected_s8_ansi(const int8_t *input_data,
const int32_t input_offset,
const uint16_t row_len,
const int8_t *filter_data,
const int32_t filter_offset,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_channels,
const int32_t out_offset,
const int32_t out_shift,
const int32_t out_mult,
const int32_t activation_min,
const int32_t activation_max);
/**
* @brief Get scratch buffer size needed by softmax function
*
* @param width
* @param height
* @return size in bytes
*
* @note buffer must be 4 byte aligned
*/
int32_t esp_nn_get_softmax_scratch_size_ansi(const int32_t width, const int32_t height);
/* ANSI C function to be hooked up when optimised version needed */
int32_t esp_nn_get_softmax_scratch_size_opt(const int32_t width, const int32_t height);
/**
* @brief Set scratch buffer to be used by softmax function
*
* @param buffer this can be NULL if one needs to unset it
* must be aligned to 4 bytes
*/
void esp_nn_set_softmax_scratch_buf_ansi(void *buffer);
/* ANSI C function to be hooked up when optimised version needed */
void esp_nn_set_softmax_scratch_buf_opt(void *buffer);
/**
* @brief reference softmax function
*
* @note inputs type: int8_t, output: int8_t
*/
void esp_nn_softmax_s8_ansi(const int8_t *input_data,
const int32_t height,
const int32_t width,
const int32_t mult,
const int32_t shift,
const int32_t diff_min,
int8_t *output_data);
/**
* @brief optimised version of softmax function
*
* @note the function uses extra buffer (4 * width bytes)
* hence, scratch buffers must be set before calling this.
*/
void esp_nn_softmax_s8_opt(const int8_t *input_data,
const int32_t height,
const int32_t width,
const int32_t mult,
const int32_t shift,
const int32_t diff_min,
int8_t *output_data);

View File

@@ -0,0 +1,48 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
/**
* @file Header definitions to include for esp_nn optimized functions for
* the ESP32 platform.
* We are hooking up just the C versions for now.
* The file hence is exactly same as `esp_nn_ansi_c.h`
*/
#pragma once
#include "esp_nn_ansi_headers.h"
#define esp_nn_add_elementwise_s8 esp_nn_add_elementwise_s8_ansi
#define esp_nn_mul_elementwise_s8 esp_nn_mul_elementwise_s8_ansi
#define esp_nn_depthwise_conv_s8 esp_nn_depthwise_conv_s8_ansi
#define esp_nn_conv_s8 esp_nn_conv_s8_ansi
#define esp_nn_get_conv_scratch_size esp_nn_get_conv_scratch_size_ansi
#define esp_nn_set_conv_scratch_buf esp_nn_set_conv_scratch_buf_ansi
#define esp_nn_get_depthwise_conv_scratch_size esp_nn_get_depthwise_conv_scratch_size_ansi
#define esp_nn_set_depthwise_conv_scratch_buf esp_nn_set_depthwise_conv_scratch_buf_ansi
#define esp_nn_relu6_s8 esp_nn_relu6_s8_ansi
#define esp_nn_avg_pool_s8 esp_nn_avg_pool_s8_ansi
#define esp_nn_max_pool_s8 esp_nn_max_pool_s8_ansi
#define esp_nn_fully_connected_s8 esp_nn_fully_connected_s8_ansi
#define esp_nn_get_softmax_scratch_size esp_nn_get_softmax_scratch_size_opt
#define esp_nn_set_softmax_scratch_buf esp_nn_set_softmax_scratch_buf_opt
#define esp_nn_softmax_s8 esp_nn_softmax_s8_opt

View File

@@ -0,0 +1,261 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
/**
* @file Header definitions to include for esp_nn optimized functions for
* the ESP32-S3 platform
*/
#pragma once
#include <stdint.h>
#include "esp_nn_ansi_headers.h"
/************************** Basic math functions *****************************/
/**
* @brief elementwise addition
*
* @note inputs type: int8_t, output: int8_t
* input offsets: although int32_t, they are contained in 8 bits [-128, 127]
*
* shift values are expected to be <= 0
*/
void esp_nn_add_elementwise_s8_esp32s3(const int8_t *input1_data,
const int8_t *input2_data,
const int32_t input1_offset,
const int32_t input2_offset,
const int32_t input1_mult,
const int32_t input2_mult,
const int32_t input1_shift,
const int32_t input2_shift,
const int32_t left_shift,
int8_t *output,
const int32_t out_offset,
const int32_t out_mult,
const int32_t out_shift,
const int32_t activation_min,
const int32_t activation_max,
const int32_t size);
/**
* @brief elementwise multiplication
*
* @note inputs type: int8_t, output: int8_t
* input offsets: although int32_t, they are contained in 8 bits [-128, 127]
*
* output shift is expected to be <= 0
*/
void esp_nn_mul_elementwise_s8_esp32s3(const int8_t *input1_data,
const int8_t *input2_data,
const int32_t input1_offset,
const int32_t input2_offset,
int8_t *output,
const int32_t out_offset,
const int32_t out_mult,
const int32_t out_shift,
const int32_t activation_min,
const int32_t activation_max,
const int32_t size);
/************************** Convolution functions *****************************/
/**
* @brief depthwise convolution per channel
*
* @note inputs type: int8_t, output: int8_t
* Version used in tflite is per channel.
* This version follows the same footsprints.
* Meaning, it has per out_channel shift and multiplier for
* requantization
*
* optimization notes: Though input_offset is int32 type,
* offset values are contained in 8 bits [-128, 127]
*/
void esp_nn_depthwise_conv_s8_esp32s3(const int8_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const int32_t input_offset,
const uint16_t pad_wd,
const uint16_t pad_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const uint16_t ch_mult,
const int8_t *filter_data,
const uint16_t filter_wd,
const uint16_t filter_ht,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max);
/**
* @brief 2d - convolution channelwise
*
* @note operation: result += (input + offset) * filter
*
* inputs type: int8_t, output: int8_t
* input offsets: although int32_t, they are contained in 8 bits [-128, 127]
*/
void esp_nn_conv_s8_esp32s3(const int8_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t in_channels,
const int32_t input_offset,
const uint16_t pad_wd,
const uint16_t pad_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const int8_t *filter_data,
const uint16_t filter_wd,
const uint16_t filter_ht,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const uint16_t out_channels,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max);
int esp_nn_get_conv_scratch_size_esp32s3(const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t in_ch,
const uint16_t out_ch,
const uint16_t filter_wd,
const uint16_t filter_ht);
void esp_nn_set_conv_scratch_buf_esp32s3(const void *buf);
int esp_nn_get_depthwise_conv_scratch_size_esp32s3(const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const uint16_t ch_mult,
const uint16_t filter_wd,
const uint16_t filter_ht);
void esp_nn_set_depthwise_conv_scratch_buf_esp32s3(const void *buf);
/************************** Pooling functions *****************************/
/**
* @brief max_pool
*
* @note inputs type: int8_t, output: int8_t
* input offsets: although int32_t, they are contained in 8 bits [-128, 127]
*/
void esp_nn_max_pool_s8_esp32s3(const int8_t *input,
const uint16_t input_wd,
const uint16_t input_ht,
int8_t *output,
const uint16_t output_wd,
const uint16_t output_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const uint16_t filter_wd,
const uint16_t filter_ht,
const uint16_t pad_wd,
const uint16_t pad_ht,
const int32_t activation_min,
const int32_t activation_max,
const uint16_t channels);
/**
* @brief avg_pool
*
* @note inputs type: int8_t, output: int8_t
* input offsets: although int32_t, they are contained in 8 bits [-128, 127]
*/
void esp_nn_avg_pool_s8_esp32s3(const int8_t *input,
const uint16_t input_wd,
const uint16_t input_ht,
int8_t *output,
const uint16_t output_wd,
const uint16_t output_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const uint16_t filter_wd,
const uint16_t filter_ht,
const uint16_t pad_wd,
const uint16_t pad_ht,
const int32_t activation_min,
const int32_t activation_max,
const uint16_t channels);
/************************** Fully connected functions *****************************/
/**
* @brief fully connected
*
* @note inputs type: int8_t, output: int8_t
* input offsets: although int32_t, they are contained in 8 bits [-128, 127]
*
* Current version works only on aligned input.
* row_len and channels should both be multiple of 8.
*/
void esp_nn_fully_connected_s8_esp32s3(const int8_t *input_data,
const int32_t input_offset,
const uint16_t row_len,
const int8_t *filter_data,
const int32_t filter_offset,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_channels,
const int32_t out_offset,
const int32_t out_shift,
const int32_t out_mult,
const int32_t activation_min,
const int32_t activation_max);
/**
* @brief relu6
*
* @note inout: int8_t
*/
void esp_nn_relu6_s8_esp32s3(int8_t *data, uint16_t size);
/********************** function defines ***************************/
#define esp_nn_add_elementwise_s8 esp_nn_add_elementwise_s8_esp32s3
#define esp_nn_mul_elementwise_s8 esp_nn_mul_elementwise_s8_esp32s3
#define esp_nn_depthwise_conv_s8 esp_nn_depthwise_conv_s8_esp32s3
#define esp_nn_get_conv_scratch_size esp_nn_get_conv_scratch_size_esp32s3
#define esp_nn_set_conv_scratch_buf esp_nn_set_conv_scratch_buf_esp32s3
#define esp_nn_get_depthwise_conv_scratch_size esp_nn_get_depthwise_conv_scratch_size_esp32s3
#define esp_nn_set_depthwise_conv_scratch_buf esp_nn_set_depthwise_conv_scratch_buf_esp32s3
#define esp_nn_conv_s8 esp_nn_conv_s8_esp32s3
#define esp_nn_relu6_s8 esp_nn_relu6_s8_esp32s3
#define esp_nn_avg_pool_s8 esp_nn_avg_pool_s8_esp32s3
#define esp_nn_max_pool_s8 esp_nn_max_pool_s8_esp32s3
#define esp_nn_fully_connected_s8 esp_nn_fully_connected_s8_esp32s3
#define esp_nn_get_softmax_scratch_size esp_nn_get_softmax_scratch_size_opt
#define esp_nn_set_softmax_scratch_buf esp_nn_set_softmax_scratch_buf_opt
#define esp_nn_softmax_s8 esp_nn_softmax_s8_opt

View File

@@ -0,0 +1,30 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <stdlib.h>
#include <common_functions.h>
void esp_nn_relu6_s8_ansi(int8_t *data, uint16_t size)
{
int32_t i;
for (i = 0; i < size; i++) {
int32_t ip = data[i];
ip = max(ip, 0);
data[i] = min(ip, 6);
}
}

View File

@@ -0,0 +1,97 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <common_functions.h>
void esp_nn_add_elementwise_u8_ansi(const uint8_t *input1_data,
const uint8_t *input2_data,
const int32_t input1_offset,
const int32_t input2_offset,
const int32_t input1_mult,
const int32_t input2_mult,
const int32_t input1_shift,
const int32_t input2_shift,
const int32_t left_shift,
uint8_t *output,
const int32_t out_offset,
const int32_t out_mult,
const int32_t out_shift,
const int32_t activation_min,
const int32_t activation_max,
const int32_t size)
{
for (int i = 0; i < size; i++) {
int32_t tmp1 = input1_data[i] + input1_offset;
int32_t tmp2 = input2_data[i] + input2_offset;
tmp1 <<= left_shift;
tmp2 <<= left_shift;
tmp1 = esp_nn_sat_round_doubling_high_mul(tmp1, input1_mult);
tmp2 = esp_nn_sat_round_doubling_high_mul(tmp2, input2_mult);
tmp1 = esp_nn_div_by_power_of_two(tmp1, -input1_shift);
tmp2 = esp_nn_div_by_power_of_two(tmp2, -input2_shift);
int32_t out = tmp1 + tmp2;
out = esp_nn_sat_round_doubling_high_mul(out, out_mult);
out = esp_nn_div_by_power_of_two(out, -out_shift);
out = out + out_offset;
out = max(activation_min, min(out, activation_max));
output[i] = (uint8_t) out;
}
}
void esp_nn_add_elementwise_s8_ansi(const int8_t *input1_data,
const int8_t *input2_data,
const int32_t input1_offset,
const int32_t input2_offset,
const int32_t input1_mult,
const int32_t input2_mult,
const int32_t input1_shift,
const int32_t input2_shift,
const int32_t left_shift,
int8_t *output,
const int32_t out_offset,
const int32_t out_mult,
const int32_t out_shift,
const int32_t activation_min,
const int32_t activation_max,
const int32_t size)
{
for (int i = 0; i < size; i++) {
int32_t tmp1 = input1_data[i] + input1_offset;
int32_t tmp2 = input2_data[i] + input2_offset;
tmp1 <<= left_shift;
tmp2 <<= left_shift;
tmp1 = esp_nn_sat_round_doubling_high_mul(tmp1, input1_mult);
tmp2 = esp_nn_sat_round_doubling_high_mul(tmp2, input2_mult);
tmp1 = esp_nn_div_by_power_of_two(tmp1, -input1_shift);
tmp2 = esp_nn_div_by_power_of_two(tmp2, -input2_shift);
int32_t out = tmp1 + tmp2;
out = esp_nn_sat_round_doubling_high_mul(out, out_mult);
out = esp_nn_div_by_power_of_two(out, -out_shift);
out = out + out_offset;
out = max(activation_min, min(out, activation_max));
output[i] = (int8_t) out;
}
}

View File

@@ -0,0 +1,42 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <common_functions.h>
void esp_nn_mul_elementwise_s8_ansi(const int8_t *input1_data,
const int8_t *input2_data,
const int32_t input1_offset,
const int32_t input2_offset,
int8_t *output,
const int32_t out_offset,
const int32_t out_mult,
const int32_t out_shift,
const int32_t activation_min,
const int32_t activation_max,
const int32_t size)
{
for (int i = 0; i < size; i++) {
int32_t tmp1 = input1_data[i] + input1_offset;
int32_t tmp2 = input2_data[i] + input2_offset;
int32_t out = tmp1 * tmp2;
out = esp_nn_multiply_by_quantized_mult(out, out_mult, out_shift);
out = out + out_offset;
out = max(activation_min, min(out, activation_max));
output[i] = (int8_t) out;
}
}

View File

@@ -0,0 +1,218 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#pragma once
#include <stdint.h>
#include <stdbool.h>
#include <string.h>
/**
* c99 standard still doesn't strictly inline functions
* We need to use attribute as well to do this.
*/
#define __NN_FORCE_INLINE__ __attribute((always_inline)) static inline
/* min/max macros */
#ifndef max
#define max(a, b) ({ \
__typeof__ (a) _a = (a); \
__typeof__ (b) _b = (b); \
_a > _b ? _a : _b; \
})
#define min(a, b) ({ \
__typeof__ (a) _a = (a); \
__typeof__ (b) _b = (b); \
_a < _b ? _a : _b; \
})
#endif
__NN_FORCE_INLINE__ int32_t esp_nn_clz32(uint32_t in)
{
__asm__ volatile("nsau %0, %0" : "+r" (in));
return in;
}
__NN_FORCE_INLINE__ int32_t esp_nn_pick_sat_high32_of64(int64_t val64)
{
int32_t sign = (int32_t) (val64 >> 63);
int32_t to_add = sign & ((1ul << 31) - 1);
return (int32_t) ((int64_t) (val64 + to_add) >> 31);
}
/**
* Signed saturate a 32 bit value to 8 bits keeping output in 32 bit variable.
*/
__NN_FORCE_INLINE__ int32_t esp_nn_saturate8(int32_t in)
{
__asm__ volatile("clamps %0, %0, 7" : "+a"(in));
return in;
}
__NN_FORCE_INLINE__ int32_t esp_nn_sat_round_doubling_high_mul(int32_t in0, int32_t in1)
{
int32_t result;
int64_t in0_64 = (int64_t) in0;
bool overflow = (in0 == in1) && (in0 == (int32_t) INT32_MIN);
/* Nudge value */
int64_t nudge_val = 1 << 30;
if ((in0 < 0) ^ (in1 < 0)) {
nudge_val = 1 - nudge_val;
}
/* Multiply and add nudge */
int64_t mult = in0_64 * in1 + nudge_val;
/* Round and pickup 32 bits */
result = esp_nn_pick_sat_high32_of64(mult);
return overflow ? INT32_MAX : result;
}
/**
* fast version
* this will fail for values closer to INT32_MAX and INT32_MIN by `1 << (exponent - 1)`.
* We can afford to do this because we are at the very last stage of filter.
* Also it is pretty rare condition as our output is going to be 8 bit.
*/
__NN_FORCE_INLINE__ int32_t esp_nn_div_by_power_of_two_fast(int32_t val, int32_t exponent)
{
int32_t to_add = (1 << (exponent - 1)) - (val < 0);
return (int32_t) ((val + to_add) >> exponent);
}
__NN_FORCE_INLINE__ int32_t esp_nn_div_by_power_of_two(int32_t val, int32_t exponent)
{
int32_t result;
const int32_t mask = (1 << exponent) - 1;
const int32_t remainder = val & mask;
result = val >> exponent;
int32_t threshold = (mask >> 1) + (result < 0);
if (remainder > threshold) {
result += 1;
}
return result;
}
__NN_FORCE_INLINE__ int32_t esp_nn_multiply_by_quantized_mult(int32_t x, int32_t mult, int32_t shift)
{
int32_t left_shift = shift > 0 ? shift : 0;
int32_t right_shift = shift > 0 ? 0 : -shift;
int32_t result = esp_nn_sat_round_doubling_high_mul(x * (1 << left_shift), mult);
return esp_nn_div_by_power_of_two(result, right_shift);
}
__NN_FORCE_INLINE__ int32_t esp_nn_multiply_by_quantized_mult_fast(int32_t x, int32_t mult, int32_t shift)
{
int32_t left_shift = max(shift, 0);
int32_t right_shift = left_shift - shift;
int64_t nudge_val = 1 << 30;
int64_t in0_64 = (int64_t) (x << left_shift);
/* Multiply and add nudge */
int64_t mult_64 = in0_64 * mult + nudge_val;
int32_t result = (int32_t) (mult_64 >> 31);
if (right_shift) {
result = esp_nn_div_by_power_of_two_fast(result, right_shift);
}
return result;
}
static void esp_nn_aligned_s8_pad_with_value(const int8_t *src, int8_t *dst,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const int32_t pad_val,
const uint16_t pad_wd,
const uint16_t pad_ht)
{
/* memset with pad_val */
memset(dst, pad_val, ((input_wd + 2 * pad_wd) * (input_ht + 2 * pad_ht)) * channels * 2);
dst += (pad_wd + input_wd + pad_wd) * channels;
for (int i = 0; i < input_ht; i++) {
dst += pad_wd * channels;
for (int j = 0; j < input_wd * channels; j++) {
*dst++ = *src++;
}
dst += pad_wd * channels;
}
}
#if 0
static void esp_nn_aligned_s8_pad_end_with_value(const int8_t *src, int8_t *dst,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const int32_t pad_val,
const uint16_t pad_wd,
const uint16_t pad_ht)
{
for (int i = 0; i < input_ht; i++) {
for (int j = 0; j < input_wd * channels; j++) {
*dst++ = *src++;
}
memset(dst, pad_val, pad_wd * channels);
dst += pad_wd * channels;
}
/* pad end `pad_ht` lines at end */
memset(dst, pad_val, (input_wd + pad_wd) * pad_ht * channels);
}
#endif
/**
* @brief convert 8 bit input data to 16 bit
*
* @param src int8_t source data
* @param dst int16_t dst data
* @param size length of data
* @param offset offset to be added to src data. Range: [-128, 127]
*/
__NN_FORCE_INLINE__ void esp_nn_s8_to_s16_with_offset(const int8_t *src, int16_t *dst,
const int size, const int32_t offset)
{
int i = 0;
for (; i < size; i += 2) {
dst[i + 0] = src[i + 0] + offset;
dst[i + 1] = src[i + 1] + offset;
}
if(i < size) {
dst[i] = src[i] + offset;
}
}
/**
* @brief convert 8 bit input data to 16 bit
*
* @param src int8_t source data
* @param dst int16_t dst data
* @param size length of data
*/
__NN_FORCE_INLINE__ void esp_nn_s8_to_s16(const int8_t *src, int16_t *dst, const int size)
{
int i = 0;
for (; i < size; i += 2) {
dst[i + 0] = src[i + 0];
dst[i + 1] = src[i + 1];
}
if(i < size) {
dst[i] = src[i];
}
}

View File

@@ -0,0 +1,175 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <common_functions.h>
int esp_nn_get_conv_scratch_size_ansi(const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t in_ch,
const uint16_t out_ch,
const uint16_t filter_wd,
const uint16_t filter_ht)
{
return 0;
}
void esp_nn_set_conv_scratch_buf_ansi(const void *buf)
{
}
/**
* Assumption 1: i/p channels == o/p channels
* Assumption 2: Pointers are valid
* Assumption 3: dialation width = 1
*/
void esp_nn_conv_u8_ansi(const uint8_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t in_channels,
const int32_t input_offset,
const uint16_t pad_wd,
const uint16_t pad_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const uint8_t *filter_data,
const uint16_t filter_wd,
const uint16_t filter_ht,
const int32_t filter_offset,
const int32_t *bias,
uint8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const uint16_t out_channels,
const int32_t out_offset,
const int32_t out_shift,
const int32_t out_mult,
const int32_t activation_min,
const int32_t activation_max)
{
for (int out_y = 0; out_y < out_ht; out_y++) { //height loop
const int16_t base_y = (out_y * stride_ht) - pad_ht;
for (int out_x = 0; out_x < out_wd; out_x++) { //width_loop
const int16_t base_x = (out_x * stride_wd) - pad_wd;
for (int out_ch_idx = 0; out_ch_idx < out_channels; out_ch_idx++) {//channel_loop
int32_t result = 0;
/* Select filter so as the point doesn't lie outside block */
int filter_y_start = max(0, -base_y);
int filter_x_start = max(0, -base_x);
int filter_y_end = min(filter_ht, input_ht - base_y);
int filter_x_end = min(filter_wd, input_wd - base_x);
for (int filter_y_idx = filter_y_start; filter_y_idx < filter_y_end; filter_y_idx++) {
const int32_t idx_y = base_y + filter_y_idx;
for (int filter_x_idx = filter_x_start; filter_x_idx < filter_x_end; filter_x_idx++) {
const int32_t idx_x = base_x + filter_x_idx;
for (int in_ch_idx = 0; in_ch_idx < in_channels; in_ch_idx++) {
int32_t input_index = (idx_y * input_wd + idx_x) * in_channels + in_ch_idx;
int32_t filter_index = ((out_ch_idx * filter_ht + filter_y_idx)
* filter_wd + filter_x_idx) * in_channels
+ in_ch_idx;
int32_t input_val = input_data[input_index] + input_offset;
int32_t filter_val = filter_data[filter_index] + filter_offset;
result += input_val * filter_val;
}
}
}
if (bias) {
result += bias[out_ch_idx];
}
result = esp_nn_multiply_by_quantized_mult(result, out_mult, out_shift);
result += out_offset;
result = max(result, activation_min);
result = min(result, activation_max);
int out_index = (out_y * out_wd + out_x) * out_channels + out_ch_idx;
out_data[out_index] = (uint8_t) result;
}
}
}
}
/**
* Assumption 1: i/p channels == o/p channels
* Assumption 2: Pointers are valid
* Assumption 3: dialation width = 1
*/
void esp_nn_conv_s8_ansi(const int8_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t in_channels,
const int32_t input_offset,
const uint16_t pad_wd,
const uint16_t pad_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const int8_t *filter_data,
const uint16_t filter_wd,
const uint16_t filter_ht,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const uint16_t out_channels,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max)
{
int32_t out_ch_idx, out_y, out_x, in_ch_idx, filter_y_idx, filter_x_idx;
for (out_y = 0; out_y < out_ht; out_y++) {
for (out_x = 0; out_x < out_wd; out_x++) {
for (out_ch_idx = 0; out_ch_idx < out_channels; out_ch_idx++) {
int32_t conv_out = 0;
const int32_t base_y = stride_ht * out_y - pad_ht;
const int32_t base_x = stride_wd * out_x - pad_wd;
const int32_t filter_y_start = max(0, -base_y);
const int32_t filter_x_start = max(0, -base_x);
const int32_t filter_y_end = min(filter_ht, input_ht - base_y);
const int32_t filter_x_end = min(filter_wd, input_wd - base_x);
for (filter_y_idx = filter_y_start; filter_y_idx < filter_y_end; filter_y_idx++) {
for (filter_x_idx = filter_x_start; filter_x_idx < filter_x_end; filter_x_idx++) {
const int32_t in_row = base_y + filter_y_idx;
const int32_t in_col = base_x + filter_x_idx;
int32_t input_base_offset = (in_row * input_wd + in_col) * in_channels;
int32_t filter_base_offset = out_ch_idx * in_channels * filter_ht * filter_wd +
(filter_y_idx * filter_wd + filter_x_idx) * in_channels;
for (in_ch_idx = 0; in_ch_idx < in_channels; in_ch_idx++) {
conv_out +=
(input_data[input_base_offset + in_ch_idx] + input_offset) *
filter_data[filter_base_offset + in_ch_idx];
}
}
}
if (bias) {
conv_out += bias[out_ch_idx];
}
conv_out = esp_nn_multiply_by_quantized_mult(conv_out, out_mult[out_ch_idx], out_shift[out_ch_idx]);
conv_out += out_offset;
conv_out = max(conv_out, activation_min);
conv_out = min(conv_out, activation_max);
*out_data++ = (int8_t) conv_out;
}
}
}
}

View File

@@ -0,0 +1,436 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <stdio.h>
#include <common_functions.h>
static int16_t *scratch_buffer = NULL;
extern void esp_nn_conv_s16_mult8_1x1_esp32s3(const int8_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t in_channels,
const int32_t input_offset,
const int16_t *filter_data,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const uint16_t out_channels,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max,
void *buffer /* scratch buffer */);
extern void esp_nn_conv_s16_mult4_1x1_esp32s3(const int16_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t in_channels,
const int16_t *filter_data,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const uint16_t out_channels,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max,
void *buffer /* scratch buffer */);
extern void esp_nn_conv_s16_mult8_esp32s3(const int16_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t in_channels,
const uint16_t pad_wd,
const uint16_t pad_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const int16_t *filter_data,
const uint16_t filter_wd,
const uint16_t filter_ht,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const uint16_t out_channels,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max);
extern void esp_nn_aligned_s8_to_s16_with_offset_esp32s3(const int8_t *src, int16_t *dst,
const int size, const int32_t offset);
extern void esp_nn_s8_to_s16_esp32s3(const int8_t *src, int16_t *dst, const int size);
static void esp_nn_conv_s8_unrolled(const int8_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t in_channels,
const int32_t input_offset,
const uint16_t pad_wd,
const uint16_t pad_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const int8_t *filter_data,
const uint16_t filter_wd,
const uint16_t filter_ht,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const uint16_t out_channels,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max)
{
int32_t out_ch_idx, out_y, out_x, in_ch_idx, filter_y_idx, filter_x_idx;
for (out_y = 0; out_y < out_ht; out_y++) {
for (out_x = 0; out_x < out_wd; out_x++) {
for (out_ch_idx = 0; out_ch_idx < out_channels; out_ch_idx++) {
int32_t conv_out = 0;
const int32_t base_y = stride_ht * out_y - pad_ht;
const int32_t base_x = stride_wd * out_x - pad_wd;
const int32_t filter_y_start = max(0, -base_y);
const int32_t filter_x_start = max(0, -base_x);
const int32_t filter_y_end = min(filter_ht, input_ht - base_y);
const int32_t filter_x_end = min(filter_wd, input_wd - base_x);
for (filter_y_idx = filter_y_start; filter_y_idx < filter_y_end; filter_y_idx++) {
for (filter_x_idx = filter_x_start; filter_x_idx < filter_x_end; filter_x_idx++) {
const int32_t in_row = base_y + filter_y_idx;
const int32_t in_col = base_x + filter_x_idx;
int32_t input_base_offset = (in_row * input_wd + in_col) * in_channels;
int32_t filter_base_offset = out_ch_idx * in_channels * filter_ht * filter_wd +
(filter_y_idx * filter_wd + filter_x_idx) * in_channels;
for (in_ch_idx = 0; in_ch_idx < in_channels; in_ch_idx++) {
conv_out +=
(input_data[input_base_offset + in_ch_idx] + input_offset) *
filter_data[filter_base_offset + in_ch_idx];
}
}
}
if (bias) {
conv_out += bias[out_ch_idx];
}
conv_out = esp_nn_multiply_by_quantized_mult_fast(conv_out, out_mult[out_ch_idx], out_shift[out_ch_idx]);
conv_out += out_offset;
conv_out = max(conv_out, activation_min);
conv_out = min(conv_out, activation_max);
*out_data++ = (int8_t) conv_out;
}
}
}
}
static void esp_nn_conv_s8_pad_valid(const int8_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t in_channels,
const int32_t input_offset,
const uint16_t stride_wd,
const uint16_t stride_ht,
const int8_t *filter_data,
const uint16_t filter_wd,
const uint16_t filter_ht,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const uint16_t out_channels,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max)
{
int32_t out_ch_idx, out_y, out_x, in_ch_idx, filter_y_idx, filter_x_idx;
for (out_y = 0; out_y < out_ht; out_y++) {
for (out_x = 0; out_x < out_wd; out_x++) {
for (out_ch_idx = 0; out_ch_idx < out_channels; out_ch_idx++) {
int32_t conv_out = 0;
const int32_t base_y = stride_ht * out_y;
const int32_t base_x = stride_wd * out_x;
for (filter_y_idx = 0; filter_y_idx < filter_ht; filter_y_idx++) {
for (filter_x_idx = 0; filter_x_idx < filter_wd; filter_x_idx++) {
const int32_t in_row = base_y + filter_y_idx;
const int32_t in_col = base_x + filter_x_idx;
int32_t input_base_offset = (in_row * input_wd + in_col) * in_channels;
int32_t filter_base_offset = out_ch_idx * in_channels * filter_ht * filter_wd +
(filter_y_idx * filter_wd + filter_x_idx) * in_channels;
const int8_t *input_data_ptr = input_data + input_base_offset;
const int8_t *filter_data_ptr = filter_data + filter_base_offset;
for (in_ch_idx = 0; in_ch_idx < in_channels; in_ch_idx++) {
conv_out += (*input_data_ptr++ + input_offset) * *filter_data_ptr++;
}
}
}
if (bias) {
conv_out += bias[out_ch_idx];
}
conv_out = esp_nn_multiply_by_quantized_mult_fast(conv_out, out_mult[out_ch_idx], out_shift[out_ch_idx]);
conv_out += out_offset;
conv_out = max(conv_out, activation_min);
conv_out = min(conv_out, activation_max);
*out_data++ = (int8_t) conv_out;
}
}
}
}
static void esp_nn_conv_s8_pad_valid_3x3(const int8_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t in_channels,
const int32_t input_offset,
const uint16_t stride_wd,
const uint16_t stride_ht,
const int8_t *filter_data,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const uint16_t out_channels,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max)
{
int32_t out_ch_idx, out_y, out_x, in_ch_idx, filter_y_idx, filter_x_idx;
for (out_y = 0; out_y < out_ht; out_y++) {
for (out_x = 0; out_x < out_wd; out_x++) {
const int32_t base_y = stride_ht * out_y;
const int32_t base_x = stride_wd * out_x;
for (out_ch_idx = 0; out_ch_idx < out_channels; out_ch_idx++) {
int32_t conv_out = 0;
for (filter_y_idx = 0; filter_y_idx < 3; filter_y_idx++) {
for (filter_x_idx = 0; filter_x_idx < 3; filter_x_idx++) {
const int32_t in_row = base_y + filter_y_idx;
const int32_t in_col = base_x + filter_x_idx;
int32_t input_base_offset = (in_row * input_wd + in_col) * in_channels;
int32_t filter_base_offset = out_ch_idx * in_channels * 3 * 3 +
(filter_y_idx * 3 + filter_x_idx) * in_channels;
const int8_t *input_data_ptr = input_data + input_base_offset;
const int8_t *filter_data_ptr = filter_data + filter_base_offset;
for (in_ch_idx = 0; in_ch_idx < in_channels; in_ch_idx++) {
conv_out += (*input_data_ptr++ + input_offset) * *filter_data_ptr++;
}
}
}
if (bias) {
conv_out += bias[out_ch_idx];
}
conv_out = esp_nn_multiply_by_quantized_mult_fast(conv_out, out_mult[out_ch_idx], out_shift[out_ch_idx]);
conv_out += out_offset;
conv_out = max(conv_out, activation_min);
conv_out = min(conv_out, activation_max);
*out_data++ = (int8_t) conv_out;
}
}
}
}
static void esp_nn_conv_s8_pad_valid_ch3_3x3(const int8_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const int32_t input_offset,
const uint16_t stride_wd,
const uint16_t stride_ht,
const int8_t *filter_data,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const uint16_t out_channels,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max)
{
int32_t out_ch_idx, out_y, out_x, filter_y_idx;
/* use scratch_buffer to pre-compute offset factor */
int16_t *filter_sum = (int16_t *) scratch_buffer;
const int8_t *filter_ptr = filter_data;
for (out_ch_idx = 0; out_ch_idx < out_channels; out_ch_idx++) {
int16_t sum_val = 0;
for (int i = 0; i < 9; i++) {
sum_val += *filter_ptr++;
sum_val += *filter_ptr++;
sum_val += *filter_ptr++;
}
*filter_sum++ = sum_val;
}
for (out_y = 0; out_y < out_ht; out_y++) {
for (out_x = 0; out_x < out_wd; out_x++) {
const int8_t *filter_data_ptr = filter_data;
const int32_t base_y = stride_ht * out_y;
const int32_t base_x = stride_wd * out_x;
const int8_t *input_base_ptr = input_data + (base_y * input_wd + base_x) * 3;
int16_t *filter_sum = (int16_t *) scratch_buffer;
for (out_ch_idx = 0; out_ch_idx < out_channels; out_ch_idx++) {
int32_t conv_out = 0;
for (filter_y_idx = 0; filter_y_idx < 3; filter_y_idx++) {
const int8_t *input_data_ptr = input_base_ptr + (filter_y_idx * input_wd) * 3;
conv_out += (*input_data_ptr++) * (*filter_data_ptr++);
conv_out += (*input_data_ptr++) * (*filter_data_ptr++);
conv_out += (*input_data_ptr++) * (*filter_data_ptr++);
conv_out += (*input_data_ptr++) * (*filter_data_ptr++);
conv_out += (*input_data_ptr++) * (*filter_data_ptr++);
conv_out += (*input_data_ptr++) * (*filter_data_ptr++);
conv_out += (*input_data_ptr++) * (*filter_data_ptr++);
conv_out += (*input_data_ptr++) * (*filter_data_ptr++);
conv_out += (*input_data_ptr++) * (*filter_data_ptr++);
}
conv_out += *filter_sum++ * input_offset;
if (bias) {
conv_out += bias[out_ch_idx];
}
conv_out = esp_nn_multiply_by_quantized_mult_fast(conv_out, out_mult[out_ch_idx], out_shift[out_ch_idx]);
conv_out += out_offset;
conv_out = max(conv_out, activation_min);
conv_out = min(conv_out, activation_max);
*out_data++ = (int8_t) conv_out;
}
}
}
}
int esp_nn_get_conv_scratch_size_esp32s3(const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t in_ch,
const uint16_t out_ch,
const uint16_t filter_wd,
const uint16_t filter_ht)
{
int filter_size = filter_wd * filter_ht * in_ch * out_ch;
int input_size = input_wd * input_ht * in_ch;
int transpose_buf_size = 8 * in_ch; /* to store intermediate data */
int align_buf_size = 32; /* extra buffer for alignment */
return 2 * (filter_size + input_size + transpose_buf_size) + align_buf_size;
}
void esp_nn_set_conv_scratch_buf_esp32s3(void *buf)
{
scratch_buffer = (int16_t *) buf;
}
void esp_nn_conv_s8_esp32s3(const int8_t *input,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const int32_t input_offset,
const uint16_t pad_wd,
const uint16_t pad_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const int8_t *filter_data,
const uint16_t filter_wd,
const uint16_t filter_ht,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const uint16_t out_channels,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max)
{
int filter_size = filter_wd * filter_ht * channels * out_channels;
int input_size = input_wd * input_ht * channels;
int align_len = 16 - (filter_size & 15);
int16_t *filter_data16 = scratch_buffer;
int16_t *input_data16 = scratch_buffer + filter_size + align_len;
if (scratch_buffer == NULL) {
printf("esp_nn_conv error! scratch_buffer not set!\n");
return;
}
if (channels % 8 == 0 && filter_wd == 1 && filter_ht == 1 &&
pad_wd == 0 && pad_ht == 0 && stride_wd == 1 && stride_ht == 1) {
int scratch_offset = (int) (filter_data16 + filter_size);
void *scratch_buf = (void *) (scratch_offset + 16 - (scratch_offset & 15));
esp_nn_s8_to_s16_esp32s3(filter_data, filter_data16, filter_size);
esp_nn_conv_s16_mult8_1x1_esp32s3(
input, input_wd, input_ht, channels, input_offset, filter_data16,
bias, out_data, out_wd, out_ht, out_channels, out_offset,
out_shift, out_mult, activation_min, activation_max, scratch_buf);
} else if (channels % 4 == 0 && filter_wd == 1 && filter_ht == 1 &&
(input_wd * input_ht) % 16 == 0 && /* TODO: remove this check */
pad_wd == 0 && pad_ht == 0 && stride_wd == 1 && stride_ht == 1) {
int scratch_offset = (int) (input_data16 + input_size);
void *scratch_buf = (void *) (scratch_offset + 16 - (scratch_offset & 15));
esp_nn_s8_to_s16_esp32s3(filter_data, filter_data16, filter_size);
esp_nn_aligned_s8_to_s16_with_offset_esp32s3(input, input_data16, input_size, input_offset);
esp_nn_conv_s16_mult4_1x1_esp32s3(
input_data16, input_wd, input_ht, channels, filter_data16,
bias, out_data, out_wd, out_ht, out_channels, out_offset,
out_shift, out_mult, activation_min, activation_max, scratch_buf);
} else if (channels % 8 == 0) {
esp_nn_s8_to_s16_esp32s3(filter_data, filter_data16, filter_size);
esp_nn_aligned_s8_to_s16_with_offset_esp32s3(input, input_data16, input_size, input_offset);
esp_nn_conv_s16_mult8_esp32s3(
input_data16, input_wd, input_ht, channels, pad_wd, pad_ht,
stride_wd, stride_ht, filter_data16, filter_wd, filter_ht, bias,
out_data, out_wd, out_ht, out_channels, out_offset, out_shift,
out_mult, activation_min, activation_max);
} else if (pad_wd == 0 && pad_ht == 0) {
if (filter_wd == 3 && filter_ht == 3 && channels == 3) {
esp_nn_conv_s8_pad_valid_ch3_3x3(input, input_wd, input_ht, input_offset,
stride_wd, stride_ht, filter_data, bias,
out_data, out_wd, out_ht, out_channels, out_offset,
out_shift, out_mult, activation_min, activation_max);
} else {
esp_nn_conv_s8_pad_valid(input, input_wd, input_ht, channels, input_offset,
stride_wd, stride_ht, filter_data, filter_wd, filter_ht, bias,
out_data, out_wd, out_ht, out_channels, out_offset, out_shift,
out_mult, activation_min, activation_max);
}
} else {
/* Basic unrolled version */
esp_nn_conv_s8_unrolled(input, input_wd, input_ht, channels, input_offset,
pad_wd, pad_ht, stride_wd, stride_ht,
filter_data, filter_wd, filter_ht, bias,
out_data, out_wd, out_ht, out_channels, out_offset, out_shift,
out_mult, activation_min, activation_max);
}
}

View File

@@ -0,0 +1,97 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <common_functions.h>
int esp_nn_get_depthwise_conv_scratch_size_ansi(const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const uint16_t ch_mult,
const uint16_t filter_wd,
const uint16_t filter_ht)
{
return 0;
}
void esp_nn_set_depthwise_conv_scratch_buf_ansi(const void *buf)
{
}
void esp_nn_depthwise_conv_s8_ansi(const int8_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const int32_t input_offset,
const uint16_t pad_wd,
const uint16_t pad_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const uint16_t ch_mult,
const int8_t *filter_data,
const uint16_t filter_wd,
const uint16_t filter_ht,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max)
{
int out_idx = 0;
for (int out_y = 0; out_y < out_ht; out_y++) { //height loop
const int16_t base_y = (out_y * stride_ht) - pad_ht;
for (int out_x = 0; out_x < out_wd; out_x++) { //width_loop
const int16_t base_x = (out_x * stride_wd) - pad_wd;
for (int ch_idx = 0; ch_idx < channels; ch_idx++) {//channel_loop
for (int ch_mult_idx = 0; ch_mult_idx < ch_mult; ch_mult_idx++) {
int32_t result = 0;
const int out_ch_idx = ch_mult_idx + ch_idx * ch_mult;
/* Select filter so as the point doesn't lie outside block */
int filter_y_start = max(0, -base_y);
int filter_x_start = max(0, -base_x);
int filter_y_end = min(filter_ht, input_ht - base_y);
int filter_x_end = min(filter_wd, input_wd - base_x);
for (int filter_y_idx = filter_y_start; filter_y_idx < filter_y_end; filter_y_idx++) {
const int32_t idx_y = base_y + filter_y_idx;
for (int filter_x_idx = filter_x_start; filter_x_idx < filter_x_end; filter_x_idx++) {
const int32_t idx_x = base_x + filter_x_idx;
int32_t input_index = (idx_y * input_wd + idx_x) * channels + ch_idx;
int32_t filter_index = (filter_y_idx * filter_wd + filter_x_idx) * (channels * ch_mult) + out_ch_idx;
int32_t input_val = input_data[input_index] + input_offset;
int32_t filter_val = filter_data[filter_index];
result += input_val * filter_val;
}
}
if (bias) {
result += bias[out_ch_idx];
}
result = esp_nn_multiply_by_quantized_mult(result, out_mult[out_ch_idx], out_shift[out_ch_idx]);
result += out_offset;
result = max(result, activation_min);
result = min(result, activation_max);
out_data[out_idx++] = result;
}
}
}
}
}

View File

@@ -0,0 +1,483 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <stdio.h>
#include <common_functions.h>
static int16_t *scratch_buffer = NULL;
extern void esp_nn_depthwise_conv_s16_mult8_3x3_esp32s3(const int16_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const uint16_t pad_wd,
const uint16_t pad_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const uint16_t ch_mult,
const int16_t *filter_data,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max);
extern void esp_nn_depthwise_conv_s8_mult1_3x3_padded_esp32s3(const int8_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const int32_t input_offset,
const uint16_t stride_wd,
const uint16_t stride_ht,
const int8_t *filter_data,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max);
extern void esp_nn_depthwise_conv_s16_mult1_3x3_no_pad_esp32s3(const int16_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const uint16_t stride_wd,
const uint16_t stride_ht,
const int16_t *filter_data,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max);
extern void esp_nn_depthwise_conv_s16_mult8_esp32s3(const int16_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const uint16_t pad_wd,
const uint16_t pad_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const uint16_t ch_mult,
const int16_t *filter_data,
const uint16_t filter_wd,
const uint16_t filter_ht,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max);
extern void esp_nn_depthwise_conv_s16_mult4_esp32s3(const int16_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const uint16_t pad_wd,
const uint16_t pad_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const uint16_t ch_mult,
const int16_t *filter_data,
const uint16_t filter_wd,
const uint16_t filter_ht,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max);
extern void esp_nn_depthwise_conv_s16_mult1_3x3_esp32s3(const int16_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const uint16_t pad_wd,
const uint16_t pad_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const int16_t *filter_data,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max);
extern void esp_nn_depthwise_conv_s16_mult1_esp32s3(const int16_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const uint16_t pad_wd,
const uint16_t pad_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const int16_t *filter_data,
const uint16_t filter_wd,
const uint16_t filter_ht,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max);
extern void esp_nn_s8_to_s16_esp32s3(const int8_t *src, int16_t *dst, const int size);
extern void esp_nn_aligned_s8_to_s16_with_offset_esp32s3(const int8_t *src, int16_t *dst,
const int size, const int32_t offset);
static void esp_nn_depthwise_conv_s8_unrolled(const int8_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const int32_t input_offset,
const uint16_t pad_wd,
const uint16_t pad_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const uint16_t ch_mult,
const int8_t *filter_data,
const uint16_t filter_wd,
const uint16_t filter_ht,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max)
{
int out_idx = 0;
for (int out_y = 0; out_y < out_ht; out_y++) { //height loop
const int16_t base_y = (out_y * stride_ht) - pad_ht;
for (int out_x = 0; out_x < out_wd; out_x++) { //width_loop
const int16_t base_x = (out_x * stride_wd) - pad_wd;
for (int ch_idx = 0; ch_idx < channels; ch_idx++) {//channel_loop
int ch_mult_idx = 0;
for (; ch_mult_idx < ch_mult - 3; ch_mult_idx += 4) {
int32_t result0 = 0, result1 = 0, result2 = 0, result3 = 0;
const int out_ch_idx = ch_mult_idx + ch_idx * ch_mult;
/* Select filter so as the point doesn't lie outside block */
int filter_y_start = max(0, -base_y);
int filter_x_start = max(0, -base_x);
int filter_y_end = min(filter_ht, input_ht - base_y);
int filter_x_end = min(filter_wd, input_wd - base_x);
for (int filter_y_idx = filter_y_start; filter_y_idx < filter_y_end; filter_y_idx++) {
const int32_t idx_y = base_y + filter_y_idx;
for (int filter_x_idx = filter_x_start; filter_x_idx < filter_x_end; filter_x_idx++) {
const int32_t idx_x = base_x + filter_x_idx;
int32_t input_index = (idx_y * input_wd + idx_x) * channels + ch_idx;
int32_t filter_index = (filter_y_idx * filter_wd + filter_x_idx) * (channels * ch_mult) + out_ch_idx;
int32_t input_val = input_data[input_index] + input_offset;
int32_t filter_val0 = filter_data[filter_index + 0];
int32_t filter_val1 = filter_data[filter_index + 1];
int32_t filter_val2 = filter_data[filter_index + 2];
int32_t filter_val3 = filter_data[filter_index + 3];
result0 += input_val * filter_val0;
result1 += input_val * filter_val1;
result2 += input_val * filter_val2;
result3 += input_val * filter_val3;
}
}
if (bias) {
result0 += bias[out_ch_idx + 0];
result1 += bias[out_ch_idx + 1];
result2 += bias[out_ch_idx + 2];
result3 += bias[out_ch_idx + 3];
}
result0 = esp_nn_multiply_by_quantized_mult(result0,
out_mult[out_ch_idx + 0], out_shift[out_ch_idx + 0]);
result1 = esp_nn_multiply_by_quantized_mult(result1,
out_mult[out_ch_idx + 1], out_shift[out_ch_idx + 1]);
result2 = esp_nn_multiply_by_quantized_mult(result2,
out_mult[out_ch_idx + 2], out_shift[out_ch_idx + 2]);
result3 = esp_nn_multiply_by_quantized_mult(result3,
out_mult[out_ch_idx + 3], out_shift[out_ch_idx + 3]);
result0 += out_offset;
result1 += out_offset;
result2 += out_offset;
result3 += out_offset;
result0 = max(result0, activation_min);
result1 = max(result1, activation_min);
result2 = max(result2, activation_min);
result3 = max(result3, activation_min);
result0 = min(result0, activation_max);
result1 = min(result1, activation_max);
result2 = min(result2, activation_max);
result3 = min(result3, activation_max);
out_data[out_idx++] = result0;
out_data[out_idx++] = result1;
out_data[out_idx++] = result2;
out_data[out_idx++] = result3;
}
/* left-over */
for (; ch_mult_idx < ch_mult; ch_mult_idx++) {
int32_t result = 0;
const int out_ch_idx = ch_mult_idx + ch_idx * ch_mult;
/* Select filter so as the point doesn't lie outside block */
int filter_y_start = max(0, -base_y);
int filter_x_start = max(0, -base_x);
int filter_y_end = min(filter_ht, input_ht - base_y);
int filter_x_end = min(filter_wd, input_wd - base_x);
for (int filter_y_idx = filter_y_start; filter_y_idx < filter_y_end; filter_y_idx++) {
const int32_t idx_y = base_y + filter_y_idx;
for (int filter_x_idx = filter_x_start; filter_x_idx < filter_x_end; filter_x_idx++) {
const int32_t idx_x = base_x + filter_x_idx;
int32_t input_index = (idx_y * input_wd + idx_x) * channels + ch_idx;
int32_t filter_index = (filter_y_idx * filter_wd + filter_x_idx) * (channels * ch_mult) + out_ch_idx;
int32_t input_val = input_data[input_index] + input_offset;
int32_t filter_val = filter_data[filter_index];
result += input_val * filter_val;
}
}
if (bias) {
result += bias[out_ch_idx];
}
result = esp_nn_multiply_by_quantized_mult(result, out_mult[out_ch_idx], out_shift[out_ch_idx]);
result += out_offset;
result = max(result, activation_min);
result = min(result, activation_max);
out_data[out_idx++] = result;
}
}
}
}
}
void esp_nn_depthwise_conv_s8_ch_mult1(const int8_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const int32_t input_offset,
const uint16_t pad_wd,
const uint16_t pad_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const int8_t *filter_data,
const uint16_t filter_wd,
const uint16_t filter_ht,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max)
{
int out_idx = 0;
for (int out_y = 0; out_y < out_ht; out_y++) { //height loop
const int16_t base_y = (out_y * stride_ht) - pad_ht;
for (int out_x = 0; out_x < out_wd; out_x++) { //width_loop
const int16_t base_x = (out_x * stride_wd) - pad_wd;
for (int ch_idx = 0; ch_idx < channels; ch_idx++) {//channel_loop
int32_t result = 0;
/* Select filter so as the point doesn't lie outside block */
int filter_y_start = max(0, -base_y);
int filter_x_start = max(0, -base_x);
int filter_y_end = min(filter_ht, input_ht - base_y);
int filter_x_end = min(filter_wd, input_wd - base_x);
for (int filter_y_idx = filter_y_start; filter_y_idx < filter_y_end; filter_y_idx++) {
const int32_t idx_y = base_y + filter_y_idx;
for (int filter_x_idx = filter_x_start; filter_x_idx < filter_x_end; filter_x_idx++) {
const int32_t idx_x = base_x + filter_x_idx;
int32_t input_index = (idx_y * input_wd + idx_x) * channels + ch_idx;
int32_t filter_index = (filter_y_idx * filter_wd + filter_x_idx) * channels + ch_idx;
int32_t input_val = input_data[input_index] + input_offset;
int32_t filter_val = filter_data[filter_index];
result += input_val * filter_val;
}
}
if (bias) {
result += bias[ch_idx];
}
result = esp_nn_multiply_by_quantized_mult(result, out_mult[ch_idx], out_shift[ch_idx]);
result += out_offset;
result = max(result, activation_min);
result = min(result, activation_max);
out_data[out_idx++] = result;
}
}
}
}
int esp_nn_get_depthwise_conv_scratch_size_esp32s3(const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const uint16_t ch_mult,
const uint16_t filter_wd,
const uint16_t filter_ht)
{
int filter_size = filter_wd * filter_ht * channels * ch_mult;
int padding_used = ((filter_wd == 3) && (filter_ht == 3)) * 2;
int input_size = (input_wd + padding_used) * (input_ht + padding_used) * channels;
return 2 * (filter_size + input_size) + 16; //16 for alignment
}
void esp_nn_set_depthwise_conv_scratch_buf_esp32s3(void *buf)
{
scratch_buffer = (int16_t *) buf;
}
/**
* Assumption 1: i/p channels == o/p channels
* Assumption 2: Pointers are valid
* Assumption 3: dialation width = 1
*/
void esp_nn_depthwise_conv_s8_esp32s3(const int8_t *input_data,
const uint16_t input_wd,
const uint16_t input_ht,
const uint16_t channels,
const int32_t input_offset,
const uint16_t pad_wd,
const uint16_t pad_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const uint16_t ch_mult,
const int8_t *filter_data,
const uint16_t filter_wd,
const uint16_t filter_ht,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_wd,
const uint16_t out_ht,
const int32_t out_offset,
const int32_t *out_shift,
const int32_t *out_mult,
const int32_t activation_min,
const int32_t activation_max)
{
int filter_size = filter_wd * filter_ht * channels * ch_mult;
int align_len = 16 - (filter_size & 15);
int input_size = input_wd * input_ht * channels;
int16_t *filter_data16 = scratch_buffer;
int16_t *input_data16 = scratch_buffer + filter_size + align_len;
if (scratch_buffer == NULL) {
printf("esp_nn_depthwise_conv error! scratch_buffer not set!\n");
return;
}
if ((ch_mult == 1) && (channels % 8 == 0)) {
if ((filter_wd == 3) && (filter_ht == 3)) {
if ((channels % 16 == 0) && (pad_wd == 1) && (pad_ht == 1)) {
/* process in 8 bits */
int8_t *filter_aligned = (int8_t *) scratch_buffer;
int8_t *input_padded = (int8_t *) scratch_buffer + filter_size + align_len;
memcpy(filter_aligned, filter_data, filter_size);
esp_nn_aligned_s8_pad_with_value(input_data, input_padded, input_wd, input_ht, channels,
-input_offset, pad_wd, pad_ht);
esp_nn_depthwise_conv_s8_mult1_3x3_padded_esp32s3(input_padded, input_wd + 2 * pad_wd,
input_ht + 2 * pad_ht, channels, input_offset,
stride_wd, stride_ht, filter_aligned, bias,
out_data, out_wd, out_ht, out_offset, out_shift,
out_mult, activation_min, activation_max);
} else if ((pad_wd == 0) && (pad_ht == 0) &&
// because this does not handle padding offset cases yet, run just for stride (1, 1).
// end padding of input with `-input_offset` should solve this
(stride_wd == 1) && (stride_ht == 1)) {
/* process in 8 bits */
int8_t *filter_aligned = (int8_t *) scratch_buffer;
memcpy(filter_aligned, filter_data, filter_size);
esp_nn_depthwise_conv_s8_mult1_3x3_padded_esp32s3(input_data, input_wd, input_ht, channels, input_offset,
stride_wd, stride_ht, filter_aligned,
bias, out_data, out_wd, out_ht, out_offset, out_shift,
out_mult, activation_min, activation_max);
} else { /* (channels % 8) == 0 && pad_wd == 1 && pad_ht == 1 */
esp_nn_s8_to_s16_esp32s3(filter_data, filter_data16, filter_size);
esp_nn_aligned_s8_to_s16_with_offset_esp32s3(input_data, input_data16, input_size, input_offset);
esp_nn_depthwise_conv_s16_mult1_3x3_esp32s3(input_data16, input_wd, input_ht, channels,
pad_wd, pad_ht, stride_wd, stride_ht, filter_data16,
bias, out_data, out_wd, out_ht, out_offset, out_shift,
out_mult, activation_min, activation_max);
}
} else { // all other ch_mult == 1, `channels % 8 == 0`
esp_nn_depthwise_conv_s8_ch_mult1(input_data, input_wd, input_ht, channels, input_offset,
pad_wd, pad_ht, stride_wd, stride_ht,
filter_data, filter_wd, filter_ht,
bias, out_data, out_wd, out_ht, out_offset, out_shift,
out_mult, activation_min, activation_max);
}
} else if (ch_mult % 8 == 0) {
esp_nn_s8_to_s16_esp32s3(filter_data, filter_data16, filter_size);
esp_nn_aligned_s8_to_s16_with_offset_esp32s3(input_data, input_data16, input_size, input_offset);
if (filter_wd == 3 && filter_ht == 3) {
esp_nn_depthwise_conv_s16_mult8_3x3_esp32s3(input_data16, input_wd, input_ht, channels,
pad_wd, pad_ht, stride_wd, stride_ht, ch_mult,
filter_data16, bias,
out_data, out_wd, out_ht, out_offset, out_shift,
out_mult, activation_min, activation_max);
} else {
esp_nn_depthwise_conv_s16_mult8_esp32s3(input_data16, input_wd, input_ht, channels,
pad_wd, pad_ht, stride_wd, stride_ht, ch_mult,
filter_data16, filter_wd, filter_ht, bias,
out_data, out_wd, out_ht, out_offset, out_shift,
out_mult, activation_min, activation_max);
}
} else if (ch_mult % 4 == 0) {
esp_nn_s8_to_s16_esp32s3(filter_data, filter_data16, filter_size);
esp_nn_aligned_s8_to_s16_with_offset_esp32s3(input_data, input_data16, input_size, input_offset);
esp_nn_depthwise_conv_s16_mult4_esp32s3(input_data16, input_wd, input_ht, channels,
pad_wd, pad_ht, stride_wd, stride_ht, ch_mult,
filter_data16, filter_wd, filter_ht, bias,
out_data, out_wd, out_ht, out_offset, out_shift,
out_mult, activation_min, activation_max);
} else {
esp_nn_depthwise_conv_s8_unrolled(input_data, input_wd, input_ht, channels, input_offset,
pad_wd, pad_ht, stride_wd, stride_ht, ch_mult,
filter_data, filter_wd, filter_ht,
bias, out_data, out_wd, out_ht, out_offset, out_shift,
out_mult, activation_min, activation_max);
}
}

View File

@@ -0,0 +1,50 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <common_functions.h>
void esp_nn_fully_connected_s8_ansi(const int8_t *input_data,
const int32_t input_offset,
const uint16_t row_len,
const int8_t *filter_data,
const int32_t filter_offset,
const int32_t *bias,
int8_t *out_data,
const uint16_t out_channels,
const int32_t out_offset,
const int32_t out_shift,
const int32_t out_mult,
const int32_t activation_min,
const int32_t activation_max)
{
for (int32_t out_c = 0; out_c < out_channels; ++out_c) {
int32_t result = 0;
for (int32_t data_idx = 0; data_idx < row_len; data_idx++) {
int32_t filter_index = row_len * out_c + data_idx;
int32_t input_val = input_data[data_idx];
int32_t filter_val = filter_data[filter_index];
result += (filter_val + filter_offset) * (input_val + input_offset);
}
if (bias) {
result += bias[out_c];
}
result = esp_nn_multiply_by_quantized_mult(result, out_mult, out_shift);
result += out_offset;
result = max(result, activation_min);
result = min(result, activation_max);
out_data[out_c] = (int8_t) result;
}
}

View File

@@ -0,0 +1,72 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <common_functions.h>
void esp_nn_avg_pool_s8_ansi(const int8_t *input,
const uint16_t input_wd,
const uint16_t input_ht,
int8_t *output,
const uint16_t output_wd,
const uint16_t output_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const uint16_t filter_wd,
const uint16_t filter_ht,
const uint16_t pad_wd,
const uint16_t pad_ht,
const int32_t activation_min,
const int32_t activation_max,
const uint16_t channels)
{
int32_t base_y = -pad_ht;
for (int32_t out_y = 0; out_y < output_ht; out_y++, base_y += stride_ht) {
int32_t base_x = -pad_wd;
for (int32_t out_x = 0; out_x < output_wd; out_x++, base_x += stride_wd) {
for (int32_t ch_idx = 0; ch_idx < channels; ch_idx++) {
int32_t result = 0;
int32_t filter_cnt = 0;
/* Make sure filter does not cross the input box */
int32_t filter_y_start = max(0, -base_y);
int32_t filter_x_start = max(0, -base_x);
int32_t filter_y_end = min(filter_ht, input_ht - base_y);
int32_t filter_x_end = min(filter_wd, input_wd - base_x);
for (int32_t filter_y = filter_y_start; filter_y < filter_y_end; filter_y++) {
for (int32_t filter_x = filter_x_start; filter_x < filter_x_end; filter_x++) {
int32_t in_x_idx = base_x + filter_x;
int32_t in_y_idx = base_y + filter_y;
int32_t input_index = (in_y_idx * input_wd + in_x_idx) * channels + ch_idx;
result += input[input_index];
filter_cnt++;
}
}
/* Rounded average */
result = result > 0 ? (result + filter_cnt / 2) / filter_cnt
: (result - filter_cnt / 2) / filter_cnt;
/* Activation function */
result = max(result, activation_min);
result = min(result, activation_max);
int32_t output_index = (out_y * output_wd + out_x) * channels + ch_idx;
output[output_index] = (int8_t) result;
}
}
}
}

View File

@@ -0,0 +1,66 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <common_functions.h>
void esp_nn_max_pool_s8_ansi(const int8_t *input,
const uint16_t input_wd,
const uint16_t input_ht,
int8_t *output,
const uint16_t output_wd,
const uint16_t output_ht,
const uint16_t stride_wd,
const uint16_t stride_ht,
const uint16_t filter_wd,
const uint16_t filter_ht,
const uint16_t pad_wd,
const uint16_t pad_ht,
const int32_t activation_min,
const int32_t activation_max,
const uint16_t channels)
{
int32_t base_y = -pad_ht;
for (int32_t out_y = 0; out_y < output_ht; out_y++, base_y += stride_ht) {
int32_t base_x = -pad_wd;
for (int32_t out_x = 0; out_x < output_wd; out_x++, base_x += stride_wd) {
/* Make sure filter does not cross the input box */
int32_t filter_y_start = max(0, -base_y);
int32_t filter_x_start = max(0, -base_x);
int32_t filter_y_end = min(filter_ht, input_ht - base_y);
int32_t filter_x_end = min(filter_wd, input_wd - base_x);
for (int32_t ch_idx = 0; ch_idx < channels; ch_idx++) {
int8_t result = INT8_MIN;
for (int32_t filter_y = filter_y_start; filter_y < filter_y_end; filter_y++) {
for (int32_t filter_x = filter_x_start; filter_x < filter_x_end; filter_x++) {
int32_t in_x_idx = base_x + filter_x;
int32_t in_y_idx = base_y + filter_y;
int32_t input_index = (in_y_idx * input_wd + in_x_idx) * channels + ch_idx;
result = max(input[input_index], result);
}
}
/* Activation function */
result = max(result, activation_min);
result = min(result, activation_max);
int32_t output_index = (out_y * output_wd + out_x) * channels + ch_idx;
output[output_index] = result;
}
}
}
}

View File

@@ -0,0 +1,88 @@
// Copyright 2022 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "softmax_common.h"
int32_t esp_nn_get_softmax_scratch_size_ansi(const int32_t width, const int32_t height)
{
(void) width;
(void) height;
return 0;
}
void esp_nn_set_softmax_scratch_buf_ansi(void *buffer)
{
(void) buffer;
return;
}
void esp_nn_softmax_s8_ansi(const int8_t *input_data,
const int32_t height,
const int32_t width,
const int32_t mult,
const int32_t shift,
const int32_t diff_min,
int8_t *output_data)
{
// The representation chosen for the input to the exp() function is Q5.26.
// We need to leave extra space since values that we skip might be as large as
// -32 before multiplying by input mult, and therefore as large as
// -16 afterwards. Note that exp(-8) is definitely not insignificant to
// accumulation, but exp(-16) definitely is.
#define ACCUM_BITS 12
#define DIFF_BITS 5
const int32_t mask = (1 << shift);
int32_t col = 0;
const int8_t *in_ptr = input_data;
int8_t *out_ptr = output_data;
for (int row_idx = 0; row_idx < height; row_idx++) {
int8_t max_in_row = in_ptr[0];
for (col = 1; col < width; col++) {
max_in_row = max(max_in_row, in_ptr[col]);
}
int32_t input_diff = 0;
int32_t sum_of_exps = 0;
for (col = 0; col < width; col++) {
input_diff = in_ptr[col] - max_in_row;
if (input_diff >= diff_min) {
const int32_t input_diff_rescaled = SAT_HIGH_MUL(input_diff * mask, mult);
const int32_t exp_raw = esp_nn_exp_on_negative_values(input_diff_rescaled);
sum_of_exps += DIV_POW2(exp_raw, ACCUM_BITS);
}
}
const int32_t headroom_plus1 = esp_nn_clz32((uint32_t) sum_of_exps);
const int32_t shifted_scale = ONE_OVER_ONE_X((sum_of_exps << headroom_plus1) - (1 << 31));
const int32_t bits_over_unit = ACCUM_BITS - headroom_plus1 + 31 - sizeof(int8_t) * 8;
for (col = 0; col < width; col++) {
input_diff = in_ptr[col] - max_in_row;
if (input_diff >= diff_min) {
const int32_t input_diff_rescaled = SAT_HIGH_MUL(input_diff * mask, mult);
const int32_t exp_raw = esp_nn_exp_on_negative_values(input_diff_rescaled);
const int32_t shifted_output = SAT_HIGH_MUL(shifted_scale, exp_raw);
const int32_t result = DIV_POW2(shifted_output, bits_over_unit) - 128;
out_ptr[col] = (int8_t) esp_nn_saturate8(result);
} else {
out_ptr[col] = -128;
}
}
in_ptr += width;
out_ptr += width;
}
}

View File

@@ -0,0 +1,108 @@
// Copyright 2022 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "softmax_common.h"
#include <stdio.h>
static int32_t *scratch_buf = NULL;
/**
* @brief Get scratch buffer size needed by softmax function
*
* @param width
* @param height
* @return size in bytes
*
* @note buffer must be 4 byte aligned
*/
int32_t esp_nn_get_softmax_scratch_size_opt(const int32_t width, const int32_t height)
{
(void) height;
return width * 4;
}
/**
* @brief Set scratch buffer to be used by softmax function
*
* @param buffer this can be NULL if one needs to unset it
* must be aligned to 4 bytes
*/
void esp_nn_set_softmax_scratch_buf_opt(void *buffer)
{
scratch_buf = (int32_t *) buffer;
}
void esp_nn_softmax_s8_opt(const int8_t *input_data,
const int32_t height,
const int32_t width,
const int32_t mult,
const int32_t shift,
const int32_t diff_min,
int8_t *output_data)
{
if (scratch_buf == NULL) {
printf("%s error! scratch buffer not set\n", __FUNCTION__);
return;
}
// The representation chosen for the input to the exp() function is Q5.26.
// We need to leave extra space since values that we skip might be as large as
// -32 before multiplying by input mult, and therefore as large as
// -16 afterwards. Note that exp(-8) is definitely not insignificant to
// accumulation, but exp(-16) definitely is.
#define ACCUM_BITS 12
#define DIFF_BITS 5
const int32_t mask = (1 << shift);
int32_t col = 0;
const int8_t *in_ptr = input_data;
int8_t *out_ptr = output_data;
for (int row_idx = 0; row_idx < height; row_idx++) {
int8_t max_in_row = in_ptr[0];
for (col = 1; col < width; col++) {
max_in_row = max(max_in_row, in_ptr[col]);
}
int32_t input_diff = 0;
int32_t sum_of_exps = 0;
for (col = 0; col < width; col++) {
input_diff = in_ptr[col] - max_in_row;
if (input_diff >= diff_min) {
const int32_t input_diff_rescaled = SAT_HIGH_MUL(input_diff * mask, mult);
const int32_t exp_raw = esp_nn_exp_on_negative_values(input_diff_rescaled);
scratch_buf[col] = exp_raw; // store to avoid duplicate calculation later
sum_of_exps += DIV_POW2(exp_raw, ACCUM_BITS);
}
}
const int32_t headroom_plus1 = esp_nn_clz32((uint32_t) sum_of_exps);
const int32_t shifted_scale = ONE_OVER_ONE_X((sum_of_exps << headroom_plus1) - (1 << 31));
const int32_t bits_over_unit = ACCUM_BITS - headroom_plus1 + 31 - sizeof(int8_t) * 8;
for (col = 0; col < width; col++) {
input_diff = in_ptr[col] - max_in_row;
if (input_diff >= diff_min) {
int32_t exp_raw = scratch_buf[col];
const int32_t shifted_output = SAT_HIGH_MUL(shifted_scale, exp_raw);
const int32_t result = DIV_POW2(shifted_output, bits_over_unit) - 128;
out_ptr[col] = (int8_t) esp_nn_saturate8(result);
} else {
out_ptr[col] = -128;
}
}
in_ptr += width;
out_ptr += width;
}
}

View File

@@ -0,0 +1,104 @@
// Copyright 2022 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <common_functions.h>
#define MASK_IF_ZERO(x) (x) == 0 ? ~0 : 0
#define MASK_IF_NON_ZERO(x) (x) != 0 ? ~0 : 0
#define SELECT_USING_MASK(mask, a, b) ((mask) & (a)) ^ (~(mask) & (b))
#define SAT_HIGH_MUL(x, y) esp_nn_sat_round_doubling_high_mul((x), (y))
#define DIV_POW2(x,y) esp_nn_div_by_power_of_two((x), (y))
__NN_FORCE_INLINE__ int32_t mul_power_of_2(int val, int exp)
{
const int32_t thresh = ((1 << (31 - exp)) - 1);
int32_t result = val << exp;
result = SELECT_USING_MASK(MASK_IF_NON_ZERO(val > thresh), INT32_MAX, result);
result = SELECT_USING_MASK(MASK_IF_NON_ZERO(val < -thresh), INT32_MIN, result);
return result;
}
/**
* @brief Calculate `1 / (1 + x)` for x in [0, 1]
*
* @param val input value to calculate `1/(1+x)` for
* @return `int32_t` result
* @note Newton-Raphson division
*
* https://en.wikipedia.org/wiki/Division_algorithm#Newton.E2.80.93Raphson_division
* Refer to that page for the logic behind the 48/17 and 32/17 constants.
* Pseudocode: https://en.wikipedia.org/wiki/Division_algorithm#Pseudocode
*/
__NN_FORCE_INLINE__ int32_t esp_nn_one_over_one_plus_x_for_x_in_0_1(int32_t val)
{
const int64_t sum = (int64_t) val + INT32_MAX;
const int32_t half_denominator = (int32_t) ((sum + (sum >= 0 ? 1 : -1)) / 2L);
int32_t constant_48_over_17 = 1515870810;
int32_t constant_neg_32_over_17 = -1010580540;
int32_t x = constant_48_over_17 + SAT_HIGH_MUL(half_denominator, constant_neg_32_over_17);
const int32_t fixed_2_one = (1 << 29);
x += mul_power_of_2(SAT_HIGH_MUL(x, fixed_2_one - SAT_HIGH_MUL(half_denominator, x)), 2);
x += mul_power_of_2(SAT_HIGH_MUL(x, fixed_2_one - SAT_HIGH_MUL(half_denominator, x)), 2);
x += mul_power_of_2(SAT_HIGH_MUL(x, fixed_2_one - SAT_HIGH_MUL(half_denominator, x)), 2);
return mul_power_of_2(x, 1);
}
#define ONE_OVER_ONE_X(x) esp_nn_one_over_one_plus_x_for_x_in_0_1((x))
/**
* @brief Return exp(x) for x < 0.
*
*/
__NN_FORCE_INLINE__ int32_t esp_nn_exp_on_negative_values(int32_t val)
{
int32_t shift = 24;
const int32_t one_quarter = (1 << shift);
int32_t mask = one_quarter - 1;
const int32_t val_mod_minus_quarter = (val & mask) - one_quarter;
const int32_t remainder = val_mod_minus_quarter - val;
// calculate exponent for x in [-1/4, 0) in `result`
const int32_t x = (val_mod_minus_quarter << 5) + (1 << 28);
const int32_t x2 = SAT_HIGH_MUL(x, x);
const int32_t x3 = SAT_HIGH_MUL(x2, x);
const int32_t x4 = SAT_HIGH_MUL(x2, x2);
const int32_t one_over_3 = 715827883;
const int32_t one_over_8 = 1895147668;
const int32_t x4_over_4 = DIV_POW2(x4, 2);
const int32_t x4_over_4_plus_x3_over_6_plus_x2_over_2 = DIV_POW2(SAT_HIGH_MUL(x4_over_4 + x3, one_over_3) + x2, 1);
int32_t result = one_over_8 + SAT_HIGH_MUL(one_over_8, x + x4_over_4_plus_x3_over_6_plus_x2_over_2);
#define SELECT_IF_NON_ZERO(x) { \
mask = MASK_IF_NON_ZERO(remainder & (1 << shift++)); \
result = SELECT_USING_MASK(mask, SAT_HIGH_MUL(result, x), result); \
}
SELECT_IF_NON_ZERO(1672461947)
SELECT_IF_NON_ZERO(1302514674)
SELECT_IF_NON_ZERO(790015084)
SELECT_IF_NON_ZERO(290630308)
SELECT_IF_NON_ZERO(39332535)
SELECT_IF_NON_ZERO(720401)
SELECT_IF_NON_ZERO(242)
#undef SELECT_IF_NON_ZERO
mask = MASK_IF_ZERO(val);
return SELECT_USING_MASK(mask, INT32_MAX, result);
}

View File

@@ -0,0 +1,9 @@
# The following lines of boilerplate have to be in your project's
# CMakeLists in this exact order for cmake to work correctly
cmake_minimum_required(VERSION 3.5)
set(EXTRA_COMPONENT_DIRS "../" "../tests/")
set(IDF_EXCLUDE_COMPONENTS test test_app)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(test_app)

View File

@@ -0,0 +1,7 @@
set(COMPONENT_SRCS "main.c")
set(COMPONENT_ADD_INCLUDEDIRS "")
set(COMPONENT_PRIV_REQUIRES tests)
register_component()

View File

@@ -0,0 +1,8 @@
#
# Main component makefile.
#
# This Makefile can be left empty. By default, it will take the sources in the
# src/ directory, compile them and link them into lib(subdirectory_name).a
# in the build directory. This behaviour is entirely configurable,
# please read the ESP-IDF documents if you need to do this.
#

View File

@@ -0,0 +1,87 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>
#include <esp_log.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <test_functions.h>
#include <esp_timer.h>
static const char *TAG = "test_app";
static uint32_t start_c, start_opt, total_c, total_opt;
void profile_c_start()
{
/* initiate profiling */
start_c = esp_cpu_get_ccount();
}
void profile_c_end()
{
/* record profile number */
total_c = esp_cpu_get_ccount() - start_c;
}
void profile_opt_start()
{
/* initiate profiling */
start_opt = esp_cpu_get_ccount();
}
void profile_opt_end()
{
/* record profile number */
total_opt = esp_cpu_get_ccount() - start_opt;
}
void app_main()
{
/* s8 tests */
ESP_LOGI(TAG, "Running s8 tests...");
esp_nn_add_elementwise_s8_test();
printf("add, c %u opt %u\n", total_c, total_opt);
esp_nn_mul_elementwise_s8_test();
printf("mul, c %u opt %u\n", total_c, total_opt);
esp_nn_depthwise_conv_s8_test();
printf("depthwise, c %u opt %u\n", total_c, total_opt);
esp_nn_conv_s8_test();
printf("conv2d, c %u opt %u\n", total_c, total_opt);
esp_nn_relu6_s8_test();
printf("relu, c %u opt %u\n", total_c, total_opt);
esp_nn_avg_pool_s8_test();
printf("avg_pool, c %u opt %u\n", total_c, total_opt);
esp_nn_max_pool_s8_test();
printf("max_pool, c %u opt %u\n", total_c, total_opt);
esp_nn_fully_connected_s8_test();
printf("fully_connected, c %u opt %u\n", total_c, total_opt);
esp_nn_softmax_s8_test();
printf("softmax, c %u opt %u\n", total_c, total_opt);
ESP_LOGI(TAG, "s8 tests done!\n");
/* u8 tests */
//ESP_LOGI(TAG, "Running u8 tests...");
//esp_nn_add_elementwise_u8_test();
//esp_nn_depthwise_conv_u8_test();
//esp_nn_conv_u8_test();
//esp_nn_avg_pool_u8_test();
//esp_nn_max_pool_u8_test();
//esp_nn_fully_connected_u8_test();
//ESP_LOGI(TAG, "u8 tests done!\n");
}

View File

@@ -0,0 +1,5 @@
#
# esp-nn
#
CONFIG_NN_ESP32=y

View File

@@ -0,0 +1,15 @@
set(COMPONENT_ADD_INCLUDEDIRS ./include/)
set(COMPONENT_SRCS "src/basic_math_test.c"
"src/convolution_test.c"
"src/fully_connected_test.c"
"src/pooling_test.c"
"src/relu_test.c"
"src/softmax_test.c")
set(COMPONENT_REQUIRES )
set(COMPONENT_PRIV_REQUIRES esp-nn)
register_component()
target_compile_options(${COMPONENT_LIB} PRIVATE -Wno-unused-function)

View File

@@ -0,0 +1,4 @@
# Tests for esp_nn library
- Include these in your test framework and run the framework.
- For IDF test please refer `test_app`

View File

@@ -0,0 +1,5 @@
#FIXME
COMPONENT_ADD_INCLUDEDIRS := include/
COMPONENT_SRCDIRS := src/

View File

@@ -0,0 +1,48 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
/* int8_t ops tests */
void esp_nn_add_elementwise_s8_test();
void esp_nn_mul_elementwise_s8_test();
void esp_nn_depthwise_conv_s8_test();
void esp_nn_conv_s8_test();
void esp_nn_avg_pool_s8_test();
void esp_nn_max_pool_s8_test();
void esp_nn_fully_connected_s8_test();
void esp_nn_relu6_s8_test();
void esp_nn_softmax_s8_test();
/* uint8_t ops tests */
void esp_nn_add_elementwise_u8_test();
void esp_nn_depthwise_conv_u8_test();
void esp_nn_conv_u8_test();
void esp_nn_avg_pool_u8_test();
void esp_nn_max_pool_u8_test();
void esp_nn_fully_connected_u8_test();
/* instructions test functions */
void compare_instructions_test();
void arith_instructions_test();
void min_max_instructions_test();
void bitwise_instructions_test();
void load_store_instructions_test();

View File

@@ -0,0 +1,87 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <stdbool.h>
#include <common_functions.h>
#include <stdio.h>
/* mult value range */
#define MULT_MAX INT32_MAX
#define MULT_MIN 0
/* shift value range */
#define SHIFT_MIN -31
#define SHIFT_MAX 30
/**
* @brief callback function to run before C function
*/
void profile_c_start();
/**
* @brief callback function to run after C function
*/
void profile_c_end();
/**
* @brief callback function to run before optimized function
*/
void profile_opt_start();
/**
* @brief callback function to run after optimized function
*/
void profile_opt_end();
#define ANSI_COLOR_RED "\x1b[31m"
#define ANSI_COLOR_GREEN "\x1b[32m"
#define ANSI_COLOR_YELLOW "\x1b[33m"
#define ANSI_COLOR_BLUE "\x1b[34m"
#define ANSI_COLOR_MAGENTA "\x1b[35m"
#define ANSI_COLOR_CYAN "\x1b[36m"
#define ANSI_COLOR_RESET "\x1b[0m"
#define CHECK_EQUAL(ARRAY1, ARRAY2, size) ({ \
bool res = true; \
for (int _i = 0; _i < size; _i++) { \
if (ARRAY1[_i] != ARRAY2[_i]) { \
res = false; \
break; \
} \
} \
res; \
})
#define PRINT_ARRAY_INT(ARRAY, width, height) ({ \
int *_array = (int *) ARRAY; \
for (int _j = 0; _j < height; _j++) { \
for (int _i = 0; _i < width; _i++) { \
printf("%d\t", _array[width * _j + _i]); \
} \
printf("\n"); \
} \
printf("\n"); \
})
#define PRINT_ARRAY_HEX(ARRAY, width, height) ({ \
uint8_t *_array = (uint8_t *) ARRAY; \
for (int _j = 0; _j < height; _j++) { \
for (int _i = 0; _i < width; _i++) { \
printf("%02x\t", _array[width * _j + _i]); \
} \
printf("\n"); \
} \
printf("\n"); \
})

View File

@@ -0,0 +1,343 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <malloc.h>
#include <common_functions.h>
#include <esp_nn.h>
#include "test_utils.h"
#if CONFIG_IDF_CMAKE
#define IDF_HEAP_CAPS 1
#if IDF_HEAP_CAPS
#include "esp_heap_caps.h"
#endif
#endif
void esp_nn_add_elementwise_s8_test()
{
/* prepare data */
const int size = 1600 + 8 + 7; /* odd len to test leftover */
int8_t *input1;
int8_t *input2;
int8_t *out_data_c;
int8_t *out_data_opt;
int8_t *input1_orig = NULL;
int8_t *input2_orig = NULL;
int8_t *out_c_orig = NULL;
int8_t *out_opt_orig = NULL;
int32_t input1_offset = 34;
int32_t input2_offset = 35;
int32_t output_offset = 36;
int32_t input1_shift = -8; // right_shift amt always <= 0
int32_t input2_shift = -8; // right_shift amt always <= 0
int32_t output_shift = -9; // right_shift amt always <= 0
int32_t left_shift = 15; // always +ve
int32_t input1_mult = INT32_MAX;
int32_t input2_mult = INT32_MAX;
int32_t output_mult = INT32_MAX;
int32_t activation_min = -128;
int32_t activation_max = 127;
for (int itr = 0; itr < 10; itr++) {
switch (itr) {
case 0: // all zeros
input1_offset = 0;
input2_offset = 0;
output_offset = 0;
input1_mult = 0;
input2_mult = 0;
output_mult = 0;
input1_shift = 0;
input2_shift = 0;
output_shift = 0;
left_shift = 0;
break;
case 1: // hit min
input1_offset = -127;
input2_offset = -127;
output_offset = -128;
input1_mult = MULT_MIN;
input2_mult = MULT_MIN;
output_mult = MULT_MIN;
input1_shift = 0;
input2_shift = 0;
output_shift = 0;
left_shift = 0;
break;
case 2: // hit max
input1_offset = 128;
input2_offset = 128;
output_offset = -127;
input1_mult = MULT_MAX;
input2_mult = MULT_MAX;
output_mult = MULT_MAX;
input1_shift = SHIFT_MIN;
input2_shift = SHIFT_MIN;
output_shift = SHIFT_MIN;
left_shift = 30 - 8; // since input is 8 bits
break;
case 3: // hit extreme max
input1_offset = 128;
input2_offset = 128;
output_offset = -127;
input1_mult = MULT_MAX;
input2_mult = MULT_MAX;
output_mult = MULT_MAX;
input1_shift = 0;
input2_shift = 0;
output_shift = 0;
left_shift = 30 - 8; // -8 since input is 8 bit
break;
default: // practical random input
input1_offset = rand() % 256 - 127; // range [-127, 128]
input2_offset = rand() % 256 - 127; // range [-127, 128]
output_offset = rand() % 256 - 128; // range [-128, 127]
input1_mult = MULT_MAX / 2 + rand() % INT16_MAX;
input2_mult = MULT_MAX / 2 + rand() % INT16_MAX;
output_mult = MULT_MAX / 2 + rand() % INT16_MAX;
input1_shift = -8 + rand() % 4;
input2_shift = -8 + rand() % 4;
output_shift = -8 + rand() % 4;
left_shift = rand() % 15;
}
#if IDF_HEAP_CAPS
input1_orig = (int8_t *) heap_caps_malloc(size + 32, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
input2_orig = (int8_t *) heap_caps_malloc(size + 32, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
out_c_orig = (int8_t *) heap_caps_malloc(size + 32, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
out_opt_orig = (int8_t *) heap_caps_malloc(size + 32, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
input1 = 16 + input1_orig - ((uint32_t) input1_orig & 0xf);
input2 = 16 + input2_orig - ((uint32_t) input2_orig & 0xf);
out_data_c = 16 + out_c_orig - ((uint32_t) out_c_orig & 0xf);
out_data_opt = 16 + out_opt_orig - ((uint32_t) out_opt_orig & 0xf);
#else
input1 = memalign(16, size);
input2 = memalign(16, size);
out_data_c = memalign(16, size);
out_data_opt = memalign(16, size);
input1_orig = input1;
input2_orig = input2;
out_c_orig = out_data_c;
out_opt_orig = out_data_opt;
#endif
for (int i = 0; i < size; ++i) {
input1[i] = rand() % 256 - 128;
input2[i] = rand() % 256 - 128;
}
if (itr == 0) {
/* enable profiler */
profile_c_start();
}
/* C function */
esp_nn_add_elementwise_s8_ansi(input1, input2, input1_offset, input2_offset,
input1_mult, input2_mult, input1_shift, input2_shift,
left_shift, out_data_c, output_offset, output_mult,
output_shift, activation_min, activation_max, size);
if (itr == 0) {
profile_c_end();
profile_opt_start();
}
/* Optimized function */
esp_nn_add_elementwise_s8(input1, input2, input1_offset, input2_offset,
input1_mult, input2_mult, input1_shift, input2_shift,
left_shift, out_data_opt, output_offset, output_mult,
output_shift, activation_min, activation_max, size);
if (itr == 0) {
/* disable profiler */
profile_opt_end();
}
bool ret = CHECK_EQUAL(out_data_c, out_data_opt, size);
if (ret == false) {
printf(ANSI_COLOR_RED"%s[%d] failed\n"ANSI_COLOR_RESET, __FUNCTION__, itr);
printf("Output: \n");
PRINT_ARRAY_HEX(out_data_opt, size, 1);
printf("Expected: \n");
PRINT_ARRAY_HEX(out_data_c, size, 1);
printf("Input1:\n");
PRINT_ARRAY_HEX(input1, size, 1);
printf("Input2:\n");
PRINT_ARRAY_HEX(input2, size, 1);
printf("in1_shift %d, in2_shift %d, left_shift %d, out_shift %d\n",
input1_shift, input2_shift, left_shift, output_shift);
printf("in1_mult %d, in2_mult %d, out_mult %d\n", input1_mult, input2_mult, output_mult);
goto elementwise_add_test_cleanup;
}
printf(ANSI_COLOR_GREEN"%s[%d] passed\n"ANSI_COLOR_RESET, __FUNCTION__, itr);
elementwise_add_test_cleanup:
if (input1_orig) {
free(input1_orig);
}
if (input2_orig) {
free(input2_orig);
}
if (out_data_c) {
free(out_c_orig);
}
if (out_data_opt) {
free(out_opt_orig);
}
}
}
void esp_nn_mul_elementwise_s8_test()
{
/* prepare data */
const int size = 1600 + 8 + 7; /* odd len to test leftover */
int8_t *input1;
int8_t *input2;
int8_t *out_data_c;
int8_t *out_data_opt;
int32_t input1_offset = 34;
int32_t input2_offset = 35;
int32_t output_offset = 36;
int32_t output_shift = -7;
int32_t output_mult = MULT_MAX; // max out_mult
int32_t activation_min = -128;
int32_t activation_max = 127;
int8_t *input1_orig = NULL;
int8_t *input2_orig = NULL;
int8_t *out_c_orig = NULL;
int8_t *out_opt_orig = NULL;
for (int itr = 0; itr < 10; itr++) {
switch (itr) {
case 0: // all zeros
input1_offset = 0;
input2_offset = 0;
output_offset = 0;
output_mult = 0;
output_shift = 0;
break;
case 1: // hit min
input1_offset = -127;
input2_offset = -127;
output_offset = -128;
output_mult = MULT_MIN;
output_shift = 0;
break;
case 2: // hit max
input1_offset = 128;
input2_offset = 128;
output_offset = -127;
output_mult = MULT_MAX;
output_shift = SHIFT_MIN;
break;
case 3: // hit extreme max
input1_offset = 128;
input2_offset = 128;
output_offset = -127;
output_mult = MULT_MAX;
output_shift = 0;
break;
default: // practical random input
input1_offset = rand() % 256 - 127; // range [-127, 128]
input2_offset = rand() % 256 - 127; // range [-127, 128]
output_offset = rand() % 256 - 128; // range [-128, 127]
output_mult = MULT_MAX / 2 + rand() % INT16_MAX;
output_shift = -8 + rand() % 4;
}
#if IDF_HEAP_CAPS
input1_orig = (int8_t *) heap_caps_malloc(size + 32, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
input2_orig = (int8_t *) heap_caps_malloc(size + 32, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
out_c_orig = (int8_t *) heap_caps_malloc(size + 32, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
out_opt_orig = (int8_t *) heap_caps_malloc(size + 32, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
input1 = 16 + input1_orig - ((uint32_t) input1_orig & 0xf);
input2 = 16 + input2_orig - ((uint32_t) input2_orig & 0xf);
out_data_c = 16 + out_c_orig - ((uint32_t) out_c_orig & 0xf);
out_data_opt = 16 + out_opt_orig - ((uint32_t) out_opt_orig & 0xf);
#else
input1 = memalign(16, size);
input2 = memalign(16, size);
out_data_c = memalign(16, size);
out_data_opt = memalign(16, size);
input1_orig = input1;
input2_orig = input2;
out_c_orig = out_data_c;
out_opt_orig = out_data_opt;
#endif
for (int i = 0; i < size; ++i) {
input1[i] = rand() % 256 - 128;
input2[i] = rand() % 256 - 128;
}
if (itr == 0) {
/* enable profiler */
profile_c_start();
}
/* C function */
esp_nn_mul_elementwise_s8_ansi(input1, input2, input1_offset, input2_offset,
out_data_c, output_offset, output_mult, output_shift,
activation_min, activation_max, size);
if (itr == 0) {
profile_c_end();
profile_opt_start();
}
/* Optimized function */
esp_nn_mul_elementwise_s8(input1, input2, input1_offset, input2_offset,
out_data_opt, output_offset, output_mult, output_shift,
activation_min, activation_max, size);
if (itr == 0) {
/* disable profiler */
profile_opt_end();
}
bool ret = CHECK_EQUAL(out_data_c, out_data_opt, size);
if (ret == false) {
printf(ANSI_COLOR_RED"%s[%d] failed\n"ANSI_COLOR_RESET, __FUNCTION__, itr);
printf("Output: \n");
PRINT_ARRAY_HEX(out_data_opt, size, 1);
printf("Expected: \n");
PRINT_ARRAY_HEX(out_data_c, size, 1);
printf("Input1:\n");
PRINT_ARRAY_HEX(input1, size, 1);
printf("Input2:\n");
PRINT_ARRAY_HEX(input2, size, 1);
goto elementwise_mult_test_cleanup;
}
printf(ANSI_COLOR_GREEN"%s[%d] passed\n"ANSI_COLOR_RESET, __FUNCTION__, itr);
elementwise_mult_test_cleanup:
if (input1_orig) {
free(input1_orig);
}
if (input2_orig) {
free(input2_orig);
}
if (out_data_c) {
free(out_c_orig);
}
if (out_data_opt) {
free(out_opt_orig);
}
}
}

View File

@@ -0,0 +1,571 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <malloc.h>
#include <esp_nn.h>
#include "test_utils.h"
#if CONFIG_IDF_CMAKE
#define IDF_HEAP_CAPS 1
#if IDF_HEAP_CAPS
#include "esp_heap_caps.h"
#endif
#endif
void esp_nn_depthwise_conv_s8_test()
{
int8_t *input = NULL, *filter_data = NULL, *out_data_c = NULL, *out_data_opt = NULL;
int32_t *bias = NULL;
int32_t input_offset = 5; /* some number in [-128, 127] */
int32_t out_offset = 7;
int32_t activation_min = -125;
int32_t activation_max = 120;
void *scratch_buf = NULL;
/* independent variables */
int input_wd, input_ht, channels;
uint16_t filter_ht, filter_wd, ch_mult;
uint16_t pad_wd, pad_ht, stride_wd, stride_ht;
// run for 10 iterations
for (int itr = 0; itr < 10; itr++) {
/* prepare data */
switch (itr) {
case 0: // (ch_mult 1, (channels % 16) = 0), filter (3,3), pad (0,0)
input_wd = 18;
input_ht = 18;
filter_ht = 3;
filter_wd = 3;
ch_mult = 1;
channels = 16;
pad_wd = 0;
pad_ht = 0;
stride_wd = 1;
stride_ht = 1;
break;
case 1: // (ch_mult 1, (channels % 16) = 0), filter (3,3), pad (1,1)
input_wd = 10;
input_ht = 10;
filter_ht = 3;
filter_wd = 3;
ch_mult = 1;
channels = 16;
pad_wd = 1;
pad_ht = 1;
stride_wd = 1;
stride_ht = 1;
break;
case 2: // (ch_mult 1, (channels % 8) = 0), filter (3,3), pad (1,1)
input_wd = 10;
input_ht = 10;
filter_ht = 3;
filter_wd = 3;
ch_mult = 1;
channels = 24;
pad_wd = 1;
pad_ht = 1;
stride_wd = 1;
stride_ht = 1;
break;
case 3: // other filter sizes (ch_mult 1, (channels % 8) = 0)
input_wd = 10;
input_ht = 10;
filter_ht = 3;
filter_wd = 3;
ch_mult = 1;
channels = 24;
pad_wd = 1;
pad_ht = 1;
stride_wd = 1;
stride_ht = 1;
break;
case 4: // other filter sizes (ch_mult 8 = 0)
input_wd = 6;
input_ht = 6;
filter_ht = 3;
filter_wd = 3;
ch_mult = 8;
channels = 4;
pad_wd = 1;
pad_ht = 1;
stride_wd = 1;
stride_ht = 1;
break;
case 5: // other filter sizes (ch_mult 8 = 0)
input_wd = 12;
input_ht = 12;
filter_ht = 5;
filter_wd = 5;
ch_mult = 8;
channels = 4;
pad_wd = 1;
pad_ht = 1;
stride_wd = 1;
stride_ht = 1;
break;
case 6: // other filter sizes (ch_mult 4 = 0)
input_wd = 6;
input_ht = 6;
filter_ht = 3;
filter_wd = 3;
ch_mult = 4;
channels = 4;
pad_wd = 1;
pad_ht = 1;
stride_wd = 1;
stride_ht = 1;
break;
case 7: // (ch_mult 1, (channels % 16) = 0), filter (3,3), pad (0,0) stride (2,2)
input_wd = 6;
input_ht = 6;
filter_ht = 3;
filter_wd = 3;
ch_mult = 1;
channels = 16;
pad_wd = 0;
pad_ht = 0;
stride_wd = 2;
stride_ht = 2;
break;
default:
input_wd = 4;
input_ht = 4;
filter_ht = 3;
filter_wd = 3;
ch_mult = 4;
channels = 4;
pad_wd = 1;
pad_ht = 1;
stride_wd = 1;
stride_ht = 1;
break;
}
uint16_t out_wd = (input_wd - filter_wd + 1) / stride_wd;
uint16_t out_ht = (input_ht - filter_ht + 1) / stride_ht;
int in_size = input_wd * input_ht * channels;
int out_size = out_wd * out_ht * channels * ch_mult;
int filter_size = filter_wd * filter_ht * channels * ch_mult + 4;
int bias_size = channels * ch_mult + 1;
int32_t out_shift[channels * ch_mult];
int32_t out_mult[channels * ch_mult];
#if IDF_HEAP_CAPS
int8_t *input_orig = (int8_t *) heap_caps_malloc(in_size + 32, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
int8_t *out_c_orig = (int8_t *) heap_caps_malloc(out_size + 32, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
int8_t *out_opt_orig = (int8_t *) heap_caps_malloc(out_size + 32, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
filter_data = (int8_t *) heap_caps_malloc(filter_size, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
bias = (int32_t *) heap_caps_malloc(bias_size * 4, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
input = 16 + input_orig - ((uint32_t) input_orig & 0xf);
out_data_c = 16 + out_c_orig - ((uint32_t) out_c_orig & 0xf);
out_data_opt = 16 + out_opt_orig - ((uint32_t) out_opt_orig & 0xf);
#else
input = memalign(16, in_size + 16);
filter_data = memalign(16, filter_size);
out_data_c = memalign(16, out_size + 16);
out_data_opt = memalign(16, out_size + 16);
bias = memalign(16, bias_size * 4);
int8_t *input_orig = input;
int8_t *out_c_orig = out_data_c;
int8_t *out_opt_orig = out_data_opt;
#endif
if (bias == NULL || input == NULL || filter_data == NULL ||
out_data_c == NULL || out_data_opt == NULL || bias == NULL) {
printf(ANSI_COLOR_RED"%s[%d] allocations failed\n"ANSI_COLOR_RESET, __FUNCTION__, itr);
goto dc_s8_cleanup;
}
/* Generate input data */
for (int i = 0; i < in_size; ++i) {
input[i] = rand() % 128;
}
/* Generate filter data */
for (int i = 0; i < filter_size; ++i) {
filter_data[i] = rand() % 256 - 128;
}
/* Generate bias data */
for (int i = 0; i < channels * ch_mult; ++i) {
bias[i + 1] = rand() % INT16_MAX; //0th index left for unalignment
out_shift[i] = -8 + rand() % 3;
out_mult[i] = 0x7eb0e200 + rand() % 50;
}
int scratch_buf_size = esp_nn_get_depthwise_conv_scratch_size(input_wd, input_ht,
channels, ch_mult,
filter_wd, filter_ht);
if (scratch_buf_size > 0) {
#if IDF_HEAP_CAPS
scratch_buf = heap_caps_malloc(scratch_buf_size + 32, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
int align_sz = 16 - (((int32_t) scratch_buf) & 0xf);
#else
scratch_buf = memalign(16, scratch_buf_size);
int align_sz = 0;
#endif
if (scratch_buf == NULL) {
printf(ANSI_COLOR_RED"%s[%d] scratch_buf alloc failed size %d\n"ANSI_COLOR_RESET,
__FUNCTION__, itr, scratch_buf_size);
goto dc_s8_cleanup;
}
esp_nn_set_depthwise_conv_scratch_buf(scratch_buf + align_sz);
}
if (itr == 0) {
/* enable profiler */
profile_c_start();
}
/* C function */
esp_nn_depthwise_conv_s8_ansi(input, input_wd, input_ht, channels, input_offset,
pad_wd, pad_ht, stride_wd, stride_ht, ch_mult,
filter_data + 4, filter_wd, filter_ht,
bias + 1, out_data_c, out_wd, out_ht, out_offset, out_shift,
out_mult, activation_min, activation_max);
if (itr == 0) {
profile_c_end();
profile_opt_start();
}
/* Optimized function */
esp_nn_depthwise_conv_s8(input, input_wd, input_ht, channels, input_offset,
pad_wd, pad_ht, stride_wd, stride_ht, ch_mult,
filter_data + 4, filter_wd, filter_ht,
bias + 1, out_data_opt, out_wd, out_ht, out_offset, out_shift,
out_mult, activation_min, activation_max);
if (itr == 0) {
/* disable profiler */
profile_opt_end();
}
bool ret = CHECK_EQUAL(out_data_c, out_data_opt, out_size);
if (ret == false) {
printf(ANSI_COLOR_RED"%s[%d] failed\n"ANSI_COLOR_RESET, __FUNCTION__, itr);
printf("Output: \n");
PRINT_ARRAY_HEX(out_data_opt, out_size / out_ht, out_ht);
printf("Expected: \n");
PRINT_ARRAY_HEX(out_data_c, out_size / out_ht, out_ht);
printf("Input:\n");
PRINT_ARRAY_HEX(input, in_size / input_ht, input_ht);
printf("Filter data:\n");
PRINT_ARRAY_HEX(filter_data + 4, (filter_size - 4) / filter_ht, filter_ht);
printf("bias data:\n");
PRINT_ARRAY_INT(bias + 1, ch_mult * channels, 1);
goto dc_s8_cleanup;
}
printf(ANSI_COLOR_GREEN"%s[%d] passed\n"ANSI_COLOR_RESET, __FUNCTION__, itr);
dc_s8_cleanup:
if (input) {
free(input_orig);
}
if (filter_data) {
free(filter_data);
}
if (out_data_c) {
free(out_c_orig);
}
if (out_data_opt) {
free(out_opt_orig);
}
if (bias) {
free(bias);
}
if (scratch_buf) {
free(scratch_buf);
}
}
}
void esp_nn_conv_s8_test()
{
const int32_t input_offset = 5; /* some number in [-128, 127] */
const int32_t activation_min = -125;
const int32_t activation_max = 122;
const int32_t out_offset = 3;
void *scratch_buf = NULL;
int8_t *input_orig;
int8_t *out_c_orig;
int8_t *out_opt_orig;
int8_t *filter_data;
int32_t *bias;
/* independent variable */
int in_wd, in_ht, in_channels, out_channels;
uint16_t filter_ht, filter_wd;
uint16_t pad_wd, pad_ht, stride_wd, stride_ht;
// run for 10 iterations
for (int itr = 0; itr < 10; itr++) {
switch (itr) {
case 0: // ch % 8 == 0 && filter (1,1), padding (0,0)
in_wd = 10;
in_ht = 10;
in_channels = 64;
out_channels = 64;
filter_ht = 1;
filter_wd = 1;
pad_wd = 0;
pad_ht = 0;
stride_wd = 1;
stride_ht = 1;
break;
case 1: // ch % 4 == 0 && (in_wd * in_ht) % 16 == 0
in_wd = 4;
in_ht = 4;
in_channels = 20;
out_channels = 8;
filter_ht = 1;
filter_wd = 1;
pad_wd = 0;
pad_ht = 0;
stride_wd = 1;
stride_ht = 1;
break;
case 2: // ch, filter (3x3x3)
in_wd = 10;
in_ht = 10;
in_channels = 3;
out_channels = 64;
filter_ht = 3;
filter_wd = 3;
pad_wd = 0;
pad_ht = 0;
stride_wd = 1;
stride_ht = 1;
break;
case 3: // remaining pad (0, 0)
in_wd = 10;
in_ht = 10;
in_channels = 3;
out_channels = 64;
filter_ht = 1;
filter_wd = 1;
pad_wd = 0;
pad_ht = 0;
stride_wd = 1;
stride_ht = 1;
break;
case 4: // unopt case
in_wd = 10;
in_ht = 10;
in_channels = 12;
out_channels = 64;
filter_ht = 3;
filter_wd = 3;
pad_wd = 1;
pad_ht = 1;
stride_wd = 1;
stride_ht = 1;
break;
case 5: // ch % 8 == 0 & stride (2,2)
in_wd = 16;
in_ht = 16;
in_channels = 16;
out_channels = 16;
filter_ht = 1;
filter_wd = 1;
pad_wd = 0;
pad_ht = 0;
stride_wd = 2;
stride_ht = 2;
break;
case 6: // ch % 8 == 0 && filter (1,1), padding (0,0)
in_wd = 2;
in_ht = 2;
in_channels = 8;
out_channels = 8;
filter_ht = 1;
filter_wd = 1;
pad_wd = 0;
pad_ht = 0;
stride_wd = 1;
stride_ht = 1;
break;
default: // ch % 8 == 0
in_wd = 8;
in_ht = 8;
in_channels = 16;
out_channels = 16;
filter_ht = 1;
filter_wd = 1;
pad_wd = 0;
pad_ht = 0;
stride_wd = 1;
stride_ht = 1;
break;
}
/* prepare data */
uint16_t out_wd = (in_wd - filter_wd + 1) / stride_wd;
uint16_t out_ht = (in_ht - filter_ht + 1) / stride_ht;
int in_size = in_wd * in_ht * in_channels;
int filter_size = filter_wd * filter_ht * in_channels * out_channels + 2;
int out_size = out_wd * out_ht * out_channels;
#if IDF_HEAP_CAPS
input_orig = (int8_t *) heap_caps_malloc(in_size + 32, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
out_c_orig = (int8_t *) heap_caps_malloc(out_size + 32, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
out_opt_orig = (int8_t *) heap_caps_malloc(out_size + 32, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
filter_data = (int8_t *) heap_caps_malloc(filter_size + 32, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
bias = (int32_t *) heap_caps_malloc(128 + sizeof (int32_t) * out_channels, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
int8_t *input = 16 + input_orig - ((uint32_t) input_orig & 0xf);
int8_t *out_data_c = 16 + out_c_orig - ((uint32_t) out_c_orig & 0xf);
int8_t *out_data_opt = 16 + out_opt_orig - ((uint32_t) out_opt_orig & 0xf);
#else
int8_t *input = memalign(16, in_size);
int8_t *out_data_c = memalign(16, out_size);
int8_t *out_data_opt = memalign(16, out_size);
filter_data = memalign(16, filter_size);
bias = calloc(1, 128 + sizeof (int32_t) * out_channels);
input_orig = input;
out_c_orig = out_data_c;
out_opt_orig = out_data_opt;
#endif
int32_t *out_shift = calloc(1, 128 + sizeof (int32_t) * out_channels);
int32_t *out_mult = calloc(1, 128 + sizeof (int32_t) * out_channels);
if (input == NULL || filter_data == NULL ||
out_data_c == NULL || out_data_opt == NULL) {
printf(ANSI_COLOR_RED"%s allocations failed\n"ANSI_COLOR_RESET, __FUNCTION__);
goto conv_s8_cleanup;
}
if (bias == NULL || out_shift == NULL || out_mult == NULL) {
printf(ANSI_COLOR_RED"%s allocations failed\n"ANSI_COLOR_RESET, __FUNCTION__);
goto conv_s8_cleanup;
}
/* Generate input data between -128 -> +127 */
for (int i = 0; i < in_size; ++i) {
input[i] = rand() % 255 - 128;
}
/* Generate filter data between -128 -> +127 */
for (int i = 0; i < filter_size; ++i) {
filter_data[i] = rand() % 256 - 128;
}
/* Generate bias data */
for (int i = 0; i < out_channels; ++i) {
bias[i] = (int32_t)rand() % UINT16_MAX + UINT8_MAX;
}
/* Shift and multiplier */
for (int i = 0; i < out_channels; ++i) {
out_shift[i] = -10 + rand() % 2;
out_mult[i] = 0x7f67f4f8 + rand() % 50;
}
int scratch_buf_size = esp_nn_get_conv_scratch_size(in_wd, in_ht, in_channels,
out_channels, filter_wd, filter_ht);
if (scratch_buf_size > 0) {
#if IDF_HEAP_CAPS
void *scratch_buf = heap_caps_malloc(scratch_buf_size + 32, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
int align_sz = 16 - (((int32_t) scratch_buf) & 0xf);
#else
void *scratch_buf = memalign(16, scratch_buf_size);
int align_sz = 0;
#endif
if (scratch_buf == NULL) {
printf(ANSI_COLOR_RED"%s scratch_buf alloc failed size %d\n"ANSI_COLOR_RESET, __FUNCTION__, scratch_buf_size);
goto conv_s8_cleanup;
}
esp_nn_set_conv_scratch_buf(scratch_buf + align_sz);
}
if (itr == 0) {
/* enable profiler */
profile_c_start();
}
/* C function */
esp_nn_conv_s8_ansi(input, in_wd, in_ht, in_channels, input_offset,
pad_wd, pad_ht, stride_wd, stride_ht,
filter_data + 2, filter_wd, filter_ht, bias,
out_data_c, out_wd, out_ht, out_channels, out_offset, out_shift,
out_mult, activation_min, activation_max);
if (itr == 0) {
profile_c_end();
profile_opt_start();
}
/* Optimized function */
esp_nn_conv_s8(input, in_wd, in_ht, in_channels, input_offset,
pad_wd, pad_ht, stride_wd, stride_ht,
filter_data + 2, filter_wd, filter_ht, bias,
out_data_opt, out_wd, out_ht, out_channels, out_offset, out_shift,
out_mult, activation_min, activation_max);
if (itr == 0) {
/* disable profiler */
profile_opt_end();
}
bool ret = CHECK_EQUAL(out_data_c, out_data_opt, out_size);
if (ret == false) {
printf(ANSI_COLOR_RED"%s[%d] failed\n"ANSI_COLOR_RESET, __FUNCTION__, itr);
printf("Output: \n");
PRINT_ARRAY_HEX(out_data_opt, out_size / out_ht, out_ht);
printf("Expected: \n");
PRINT_ARRAY_HEX(out_data_c, out_size / out_ht, out_ht);
printf("Input:\n");
PRINT_ARRAY_HEX(input, in_size / in_ht, in_ht);
printf("Filter data:\n");
PRINT_ARRAY_HEX(filter_data + 2, (filter_size - 2) / filter_ht, filter_ht);
printf("bias data:\n");
PRINT_ARRAY_INT(bias, out_channels, 1);
goto conv_s8_cleanup;
}
printf(ANSI_COLOR_GREEN"%s[%d] passed\n"ANSI_COLOR_RESET, __FUNCTION__, itr);
conv_s8_cleanup:
if (input) {
free(input_orig);
}
if (filter_data) {
free(filter_data);
}
if (out_data_c) {
free(out_c_orig);
}
if (out_data_opt) {
free(out_opt_orig);
}
if (bias) {
free(bias);
}
if (out_shift) {
free(out_shift);
}
if (out_mult) {
free(out_mult);
}
if (scratch_buf) {
free(scratch_buf);
}
}
}

View File

@@ -0,0 +1,111 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <esp_nn.h>
#include "test_utils.h"
void esp_nn_fully_connected_s8_test()
{
/* prepare data */
static uint16_t row_len = 256 + 8 + 7; /* odd len to test unaligned+left-over */
static uint16_t out_channels = 3;
int8_t input[row_len];
int8_t filter_data[row_len * out_channels];
int8_t output_c[out_channels], output_opt[out_channels];
static int32_t activation_min = -128;
static int32_t activation_max = 127;
static int32_t input_offset = 0;
static int32_t filter_offset = 0;
int32_t out_shift = -10;
static int32_t out_offset = 127;
int32_t out_mult = 0x59e492c4;
for (int itr = 0; itr < 5; itr++) {
out_mult = INT32_MAX / row_len + rand() % INT16_MAX;
switch (itr) {
case 0:
out_shift = -10;
break;
case 1:
out_shift = SHIFT_MIN;
break;
case 2:
out_shift = SHIFT_MAX;
break;
case 3:
out_shift = 0;
break;
default:
out_shift = -10 + rand() % 5;
break;
}
if (itr == 0) {
out_shift = SHIFT_MAX;
}
/* Generate input and filter data */
for (int i = 0; i < row_len; ++i) {
input[i] = rand() % 256 - 128;
}
for (int i = 0; i < row_len * out_channels; ++i) {
filter_data[i] = rand() % 256 - 128;
}
if (itr == 0) {
/* enable profiler */
profile_c_start();
}
/* C function */
esp_nn_fully_connected_s8_ansi(input, input_offset, row_len, filter_data, filter_offset,
NULL, output_c, out_channels, out_offset, out_shift, out_mult,
activation_min, activation_max);
if (itr == 0) {
profile_c_end();
profile_opt_start();
}
/* Optimized function */
esp_nn_fully_connected_s8(input, input_offset, row_len, filter_data, filter_offset,
NULL, output_opt, out_channels, out_offset, out_shift, out_mult,
activation_min, activation_max);
if (itr == 0) {
/* disable profiler */
profile_opt_end();
}
bool ret = CHECK_EQUAL(output_c, output_opt, out_channels);
if (ret == false) {
printf(ANSI_COLOR_RED"%s[%d] failed\n"ANSI_COLOR_RESET, __FUNCTION__, itr);
printf("Output: \n");
PRINT_ARRAY_HEX(output_opt, out_channels, 1);
printf("Expected: \n");
PRINT_ARRAY_HEX(output_c, out_channels, 1);
printf("Input:\n");
PRINT_ARRAY_HEX(input, row_len, 1);
printf("Filter data:\n");
PRINT_ARRAY_HEX(filter_data, row_len, out_channels);
printf("Out shift: %d\n", out_shift);
printf("Out mult: %x\n", out_mult);
return;
}
printf(ANSI_COLOR_GREEN"%s[%d] passed\n"ANSI_COLOR_RESET, __FUNCTION__, itr);
}
}

View File

@@ -0,0 +1,184 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <malloc.h>
#include <esp_nn.h>
#include "test_utils.h"
void esp_nn_avg_pool_s8_test()
{
/* prepare data */
const uint16_t input_wd = 16;
const uint16_t input_ht = 16;
const uint16_t channels = 16; /* With TFLite example, I have seen it 256 */
const int size = input_wd * input_ht * channels;
int8_t *input, *output_c, *output_opt;
const int32_t activation_min = -128;
const int32_t activation_max = 127;
const uint16_t pad_wd = 1;
const uint16_t pad_ht = 1;
const uint16_t stride_wd = 1;
const uint16_t stride_ht = 1;
const uint16_t filter_ht = 3;
const uint16_t filter_wd = 3;
const uint16_t out_wd = input_wd / stride_wd;
const uint16_t out_ht = input_ht / stride_ht;
const int out_size = out_wd * out_ht * channels;
input = memalign(16, size);
output_c = memalign(16, out_size);
output_opt = memalign(16, out_size);
if (input == NULL || output_c == NULL || output_opt == NULL) {
printf(ANSI_COLOR_RED"%s allocations failed\n"ANSI_COLOR_RESET, __FUNCTION__);
goto avg_pool_s8_cleanup;
}
/**
* width/height, channels etc look suspicious but it it true.
* It actually depends upon where in model this is actually placed.
* If at the end wd/ht tends to be smaller and depth larger.
*/
for (int i = 0; i < size; ++i) {
input[i] = rand() % 256 - 128;
}
/* enable profiler */
profile_c_start();
/* C function */
esp_nn_avg_pool_s8_ansi(input, input_wd, input_ht, output_c, out_wd, out_ht,
stride_wd, stride_ht, filter_wd, filter_ht, pad_wd, pad_ht,
activation_min, activation_max, channels);
profile_c_end();
profile_opt_start();
/* Optimized function */
esp_nn_avg_pool_s8(input, input_wd, input_ht, output_opt, out_wd, out_ht,
stride_wd, stride_ht, filter_wd, filter_ht, pad_wd, pad_ht,
activation_min, activation_max, channels);
/* disable profiler */
profile_opt_end();
bool ret = CHECK_EQUAL(output_c, output_opt, out_size);
if (ret == false) {
printf(ANSI_COLOR_RED"%s failed\n"ANSI_COLOR_RESET, __FUNCTION__);
printf("Output: \n");
PRINT_ARRAY_HEX(output_opt, out_wd * channels, out_ht);
printf("Expected: \n");
PRINT_ARRAY_HEX(output_c, out_wd * channels, out_ht);
printf("Input:\n");
PRINT_ARRAY_HEX(input, input_wd * channels, input_ht);
goto avg_pool_s8_cleanup;
}
printf(ANSI_COLOR_GREEN"%s passed\n"ANSI_COLOR_RESET, __FUNCTION__);
avg_pool_s8_cleanup:
if (input) {
free(input);
}
if (output_c) {
free(output_c);
}
if (output_opt) {
free(output_opt);
}
}
void esp_nn_max_pool_s8_test()
{
/* prepare data */
const uint16_t input_wd = 16;
const uint16_t input_ht = 16;
const uint16_t channels = 16; /* With TFLite example, I have seen it 256 */
int8_t *input, *output_c, *output_opt;
const int size = input_wd * input_ht * channels;
const int32_t activation_min = -128;
const int32_t activation_max = 127;
const uint16_t pad_wd = 1;
const uint16_t pad_ht = 1;
const uint16_t stride_wd = 1;
const uint16_t stride_ht = 1;
const uint16_t filter_ht = 3;
const uint16_t filter_wd = 3;
const uint16_t out_wd = input_wd / stride_wd;
const uint16_t out_ht = input_ht / stride_ht;
const int out_size = out_wd * out_ht * channels;
input = memalign(16, size);
output_c = memalign(16, out_size);
output_opt = memalign(16, out_size);
if (input == NULL || output_c == NULL || output_opt == NULL) {
printf(ANSI_COLOR_RED"%s allocations failed\n"ANSI_COLOR_RESET, __FUNCTION__);
goto max_pool_s8_cleanup;
}
for (int i = 0; i < size; ++i) {
input[i] = rand() % 256 - 128;
}
/* enable profiler */
profile_c_start();
/* C function */
esp_nn_max_pool_s8_ansi(input, input_wd, input_ht, output_c, out_wd, out_ht,
stride_wd, stride_ht, filter_wd, filter_ht, pad_wd, pad_ht,
activation_min, activation_max, channels);
profile_c_end();
profile_opt_start();
/* Optimized function */
esp_nn_max_pool_s8(input, input_wd, input_ht, output_opt, out_wd, out_ht,
stride_wd, stride_ht, filter_wd, filter_ht, pad_wd, pad_ht,
activation_min, activation_max, channels);
/* disable profiler */
profile_opt_end();
bool ret = CHECK_EQUAL(output_c, output_opt, out_wd * out_ht * channels);
if (ret == false) {
printf(ANSI_COLOR_RED"%s failed\n"ANSI_COLOR_RESET, __FUNCTION__);
printf("Output: \n");
PRINT_ARRAY_HEX(output_opt, out_wd * out_ht * channels, 1);
printf("Expected: \n");
PRINT_ARRAY_HEX(output_c, out_wd * out_ht * channels, 1);
printf("Input:\n");
PRINT_ARRAY_HEX(input, 8, size / 8);
goto max_pool_s8_cleanup;
}
printf(ANSI_COLOR_GREEN"%s passed\n"ANSI_COLOR_RESET, __FUNCTION__);
max_pool_s8_cleanup:
if (input) {
free(input);
}
if (output_c) {
free(output_c);
}
if (output_opt) {
free(output_opt);
}
}

View File

@@ -0,0 +1,83 @@
// Copyright 2020-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <malloc.h>
#include <esp_nn.h>
#include "test_utils.h"
void esp_nn_relu6_s8_test()
{
const int size = 1600 + 8 + 7;
int8_t *input, *inout_ansi, *inout_opt;
input = memalign(16, size);
inout_ansi = memalign(16, size);
inout_opt = memalign(16, size);
if (input == NULL || inout_ansi == NULL || inout_opt == NULL) {
printf(ANSI_COLOR_RED"%s allocations failed\n"ANSI_COLOR_RESET, __FUNCTION__);
goto relu6_s8_cleanup;
}
/* Generate filter data between -128 -> +127 */
for (int i = 0; i < size; ++i) {
input[i] = rand() % 255 - 128;
inout_ansi[i] = input[i];
inout_opt[i] = input[i];
}
/* enable profiler */
profile_c_start();
/* C function */
esp_nn_relu6_s8_ansi(inout_ansi, size);
profile_c_end();
profile_opt_start();
/* Optimized function */
esp_nn_relu6_s8(inout_opt, size);
/* disable profiler */
profile_opt_end();
bool ret = CHECK_EQUAL(inout_ansi, inout_opt, size);
if (ret == false) {
printf(ANSI_COLOR_RED"%s failed\n"ANSI_COLOR_RESET, __FUNCTION__);
printf("Output: \n");
PRINT_ARRAY_HEX(inout_opt, size, 1);
printf("Expected: \n");
PRINT_ARRAY_HEX(inout_ansi, size, 1);
printf("Input:\n");
PRINT_ARRAY_HEX(input, size, 1);
goto relu6_s8_cleanup;
}
printf(ANSI_COLOR_GREEN"%s passed\n"ANSI_COLOR_RESET, __FUNCTION__);
relu6_s8_cleanup:
if (input) {
free (input);
}
if (inout_ansi) {
free (inout_ansi);
}
if (inout_opt) {
free (inout_opt);
}
}

View File

@@ -0,0 +1,101 @@
// Copyright 2022 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <malloc.h>
#include <esp_nn.h>
#include "test_utils.h"
void esp_nn_softmax_s8_test()
{
const int32_t height = 8;
const int32_t width = 32;
const int32_t diff_min = -128;
const int32_t mult = INT32_MAX / 2;
const int32_t shift = 7;
void *scratch_buf = NULL;
const int size = width * height;
int8_t *input, *out_ansi, *out_opt;
input = memalign(16, size);
out_ansi = memalign(16, size);
out_opt = memalign(16, size);
if (input == NULL || out_ansi == NULL || out_opt == NULL) {
printf(ANSI_COLOR_RED"%s buffer allocations failed\n"ANSI_COLOR_RESET, __FUNCTION__);
goto softmax_s8_cleanup;
}
/* Generate input data between -128 -> +127 */
for (int i = 0; i < size; ++i) {
input[i] = rand() % 255 - 128;
}
/* enable profiler */
profile_c_start();
/* C function */
esp_nn_softmax_s8_ansi(input, height, width, mult, shift, diff_min, out_ansi);
profile_c_end();
int32_t scratch_buf_size = esp_nn_get_softmax_scratch_size(width, height);
if (scratch_buf_size) {
scratch_buf = memalign(4, scratch_buf_size);
if (scratch_buf == NULL) {
printf(ANSI_COLOR_RED"%s scratch_buf alloc failed size %d\n"ANSI_COLOR_RESET, __FUNCTION__, scratch_buf_size);
goto softmax_s8_cleanup;
}
esp_nn_set_softmax_scratch_buf(scratch_buf);
}
profile_opt_start();
/* Optimized function */
esp_nn_softmax_s8(input, height, width, mult, shift, diff_min, out_opt);
/* disable profiler */
profile_opt_end();
bool ret = CHECK_EQUAL(out_ansi, out_opt, size);
if (ret == false) {
printf(ANSI_COLOR_RED"%s failed\n"ANSI_COLOR_RESET, __FUNCTION__);
printf("Output: \n");
PRINT_ARRAY_HEX(out_opt, width, height);
printf("Expected: \n");
PRINT_ARRAY_HEX(out_ansi, width, height);
printf("Input:\n");
PRINT_ARRAY_HEX(input, width, height);
goto softmax_s8_cleanup;
}
printf(ANSI_COLOR_GREEN"%s passed\n"ANSI_COLOR_RESET, __FUNCTION__);
softmax_s8_cleanup:
if (input) {
free (input);
}
if (out_ansi) {
free (out_ansi);
}
if (out_opt) {
free (out_opt);
}
if (scratch_buf) {
free (scratch_buf);
}
}

Binary file not shown.

Binary file not shown.

Binary file not shown.

View File

@@ -0,0 +1,79 @@
name: Build examples
on:
push:
branches:
- master
pull_request:
jobs:
build-master:
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v2
with:
submodules: 'recursive'
- name: esp-idf build
uses: espressif/esp-idf-ci-action@latest
with:
path: 'examples'
build-release-v4_0:
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v2
with:
submodules: 'recursive'
- name: esp-idf build
uses: espressif/esp-idf-ci-action@release-v4.0
with:
path: 'examples'
build-release-v4_1:
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v2
with:
submodules: 'recursive'
- name: esp-idf build
uses: espressif/esp-idf-ci-action@release-v4.1
with:
path: 'examples'
build-release-v4_2:
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v2
with:
submodules: 'recursive'
- name: esp-idf build
uses: espressif/esp-idf-ci-action@release-v4.2
with:
path: 'examples'
build-release-v4_3:
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v2
with:
submodules: 'recursive'
- name: esp-idf build
uses: espressif/esp-idf-ci-action@release-v4.3
with:
path: 'examples'
build-release-v3_3:
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v2
with:
submodules: 'recursive'
- name: esp-idf build
uses: espressif/esp-idf-ci-action@release-v3.3
with:
path: 'examples'

View File

@@ -0,0 +1,27 @@
# This workflow warns and then closes issues and PRs that have had no activity for a specified amount of time.
#
# You can adjust the behavior by modifying this file.
# For more information, see:
# https://github.com/actions/stale
name: Mark stale issues and pull requests
on:
schedule:
- cron: '20 9 * * *'
jobs:
stale:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/stale@v3
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
stale-issue-message: 'This issue appears to be stale. Please close it if its no longer valid.'
stale-pr-message: 'This pull request appears to be stale. Please close it if its no longer valid.'
stale-issue-label: 'no-issue-activity'
stale-pr-label: 'no-pr-activity'

View File

@@ -0,0 +1,21 @@
name: Push component to https://components.espressif.com
on:
push:
tags:
- v*
jobs:
upload_components:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@master
with:
submodules: "recursive"
- name: Upload component to the component registry
uses: espressif/github-actions/upload_components@master
with:
name: "esp32-camera"
version: "git"
namespace: "espressif"
service_url: ${{ secrets.IDF_COMPONENT_API_URL }}
api_token: ${{ secrets.IDF_COMPONENT_API_TOKEN }}

View File

@@ -1 +1,5 @@
*.DS_Store
.vscode
**/build
**/sdkconfig
**/sdkconfig.old

View File

@@ -1,15 +1,19 @@
if(IDF_TARGET STREQUAL "esp32")
if(IDF_TARGET STREQUAL "esp32" OR IDF_TARGET STREQUAL "esp32s2" OR IDF_TARGET STREQUAL "esp32s3")
set(COMPONENT_SRCS
driver/camera.c
driver/esp_camera.c
driver/cam_hal.c
driver/sccb.c
driver/sensor.c
driver/xclk.c
sensors/ov2640.c
sensors/ov3660.c
sensors/ov5640.c
sensors/ov7725.c
sensors/ov7670.c
sensors/nt99141.c
sensors/gc0308.c
sensors/gc2145.c
sensors/gc032a.c
sensors/bf3005.c
conversions/yuv.c
conversions/to_jpg.cpp
conversions/to_bmp.c
@@ -26,8 +30,34 @@ if(IDF_TARGET STREQUAL "esp32")
driver/private_include
sensors/private_include
conversions/private_include
target/private_include
)
if(IDF_TARGET STREQUAL "esp32")
list(APPEND COMPONENT_SRCS
target/xclk.c
target/esp32/ll_cam.c
)
endif()
if(IDF_TARGET STREQUAL "esp32s2")
list(APPEND COMPONENT_SRCS
target/xclk.c
target/esp32s2/ll_cam.c
target/esp32s2/tjpgd.c
)
list(APPEND COMPONENT_PRIV_INCLUDEDIRS
target/esp32s2/private_include
)
endif()
if(IDF_TARGET STREQUAL "esp32s3")
list(APPEND COMPONENT_SRCS
target/esp32s3/ll_cam.c
)
endif()
set(COMPONENT_REQUIRES driver)
set(COMPONENT_PRIV_REQUIRES freertos nvs_flash)

View File

@@ -5,11 +5,11 @@ menu "Camera configuration"
default y
help
Enable this option if you want to use the OV7670.
Disable this option to safe memory.
Disable this option to save memory.
config OV7725_SUPPORT
bool "Support OV7725 SVGA"
default n
bool "Support OV7725 VGA"
default y
help
Enable this option if you want to use the OV7725.
Disable this option to save memory.
@@ -42,6 +42,34 @@ menu "Camera configuration"
Enable this option if you want to use the OV5640.
Disable this option to save memory.
config GC2145_SUPPORT
bool "Support GC2145 2MP"
default y
help
Enable this option if you want to use the GC2145.
Disable this option to save memory.
config GC032A_SUPPORT
bool "Support GC032A VGA"
default y
help
Enable this option if you want to use the GC032A.
Disable this option to save memory.
config GC0308_SUPPORT
bool "Support GC0308 VGA"
default y
help
Enable this option if you want to use the GC0308.
Disable this option to save memory.
config BF3005_SUPPORT
bool "Support BF3005(BYD3005) VGA"
default y
help
Enable this option if you want to use the BF3005.
Disable this option to save memory.
choice SCCB_HARDWARE_I2C_PORT
bool "I2C peripheral to use for SCCB"
default SCCB_HARDWARE_I2C_PORT1
@@ -53,6 +81,28 @@ menu "Camera configuration"
endchoice
config SCCB_CLK_FREQ
int "SCCB clk frequency"
default 100000
range 100000 400000
help
Increasing this value can reduce the initialization time of the sensor.
Please refer to the relevant instructions of the sensor to adjust the value.
choice GC_SENSOR_WINDOW_MODE
bool "GalaxyCore Sensor Window Mode"
depends on (GC2145_SUPPORT || GC032A_SUPPORT || GC0308_SUPPORT)
default GC_SENSOR_SUBSAMPLE_MODE
help
This option determines how to reduce the output size when the resolution you set is less than the maximum resolution.
SUBSAMPLE_MODE has a bigger perspective and WINDOWING_MODE has a higher frame rate.
config GC_SENSOR_WINDOWING_MODE
bool "Windowing Mode"
config GC_SENSOR_SUBSAMPLE_MODE
bool "Subsample Mode"
endchoice
choice CAMERA_TASK_PINNED_TO_CORE
bool "Camera task pinned to core"
default CAMERA_CORE0
@@ -68,4 +118,12 @@ menu "Camera configuration"
endchoice
config CAMERA_DMA_BUFFER_SIZE_MAX
int "DMA buffer size"
range 8192 32768
default 32768
help
Maximum value of DMA buffer
Larger values may fail to allocate due to insufficient contiguous memory blocks, and smaller value may cause DMA interrupt to be too frequent
endmenu

View File

@@ -1,8 +1,30 @@
# ESP32 Camera Driver
[![Build examples](https://github.com/espressif/esp32-camera/actions/workflows/build.yml/badge.svg)](https://github.com/espressif/esp32-camera/actions/workflows/build.yml)
## General Information
This repository hosts ESP32 compatible driver for OV2640, OV3660, OV5640, OV7670 and OV7725 image sensors. Additionally it provides a few tools, which allow converting the captured frame data to the more common BMP and JPEG formats.
This repository hosts ESP32 series Soc compatible driver for image sensors. Additionally it provides a few tools, which allow converting the captured frame data to the more common BMP and JPEG formats.
### Supported Soc
- ESP32
- ESP32-S2
- ESP32-S3
### Supported Sensor
| model | max resolution | color type | output format | Len Size |
| ------- | -------------- | ---------- | ------------------------------------------------------------ | -------- |
| OV2640 | 1600 x 1200 | color | YUV(422/420)/YCbCr422<br>RGB565/555<br>8-bit compressed data<br>8/10-bit Raw RGB data | 1/4" |
| OV3660 | 2048 x 1536 | color | raw RGB data<br/>RGB565/555/444<br/>CCIR656<br/>YCbCr422<br/>compression | 1/5" |
| OV5640 | 2592 x 1944 | color | RAW RGB<br/>RGB565/555/444<br/>CCIR656<br/>YUV422/420<br/>YCbCr422<br/>compression | 1/4" |
| OV7670 | 640 x 480 | color | Raw Bayer RGB<br/>Processed Bayer RGB<br>YUV/YCbCr422<br>GRB422<br>RGB565/555 | 1/6" |
| OV7725 | 640 x 480 | color | Raw RGB<br/>GRB 422<br/>RGB565/555/444<br/>YCbCr 422 | 1/4" |
| NT99141 | 1280 x 720 | color | YCbCr 422<br/>RGB565/555/444<br/>Raw<br/>CCIR656<br/>JPEG compression | 1/4" |
| GC032A | 640 x 480 | color | YUV/YCbCr422<br/>RAW Bayer<br/>RGB565 | 1/10" |
| GC0308 | 640 x 480 | color | YUV/YCbCr422<br/>RAW Bayer<br/>RGB565 | 1/6.5" |
| GC2145 | 1600 x 1200 | color | YUV/YCbCr422<br/>RAW Bayer<br/>RGB565 | 1/5" |
| BF3005 | 640 x 480 | color | YUV/YCbCr422<br/>RAW Bayer<br/>RGB565 | 1/4" |
## Important to Remember
@@ -17,7 +39,7 @@ This repository hosts ESP32 compatible driver for OV2640, OV3660, OV5640, OV7670
### Using esp-idf
- Clone or download and extract the repository to the components folder of your ESP-IDF project
- Enable PSRAM in `menuconfig`
- Enable PSRAM in `menuconfig` (also set Flash and PSRAM frequiencies to 80MHz)
- Include `esp_camera.h` in your code
### Using PlatformIO
@@ -75,17 +97,6 @@ However with a bit of patience and experimenting you'll figure the Kconfig out.
If you miss-skip-ignore this critical step the camera module will compile but camera logic inside the library will be 'empty' because the Kconfig sets the proper #ifdef statements during the build process to initialize the selected cameras. It's very not optional!
### Kconfig options
| config | description | default |
| --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------ |
| CONFIG_OV2640_SUPPORT | Support for OV2640 camera | enabled |
| CONFIG_OV7725_SUPPORT | Support for OV7725 camera | disabled |
| CONFIG_OV3660_SUPPORT | Support for OV3660 camera | enabled |
| CONFIG_OV5640_SUPPORT | Support for OV5640 camera | enabled |
| CONFIG_SCCB_HARDWARE_I2C | Enable this option if you want to use hardware I2C to control the camera. Disable this option to use software I2C. | enabled |
| CONFIG_SCCB_HARDWARE_I2C_PORT | I2C peripheral to use for SCCB. Can be I2C0 and I2C1. | CONFIG_SCCB_HARDWARE_I2C_PORT1 |
| CONFIG_CAMERA_TASK_PINNED_TO_CORE | Pin the camera handle task to a certain core(0/1). It can also be done automatically choosing NO_AFFINITY. Can be CAMERA_CORE0, CAMERA_CORE1 or NO_AFFINITY. | CONFIG_CAMERA_CORE0 |
## Examples
@@ -132,8 +143,7 @@ static camera_config_t camera_config = {
.pin_href = CAM_PIN_HREF,
.pin_pclk = CAM_PIN_PCLK,
//XCLK 20MHz or 10MHz for OV2640 double FPS (Experimental)
.xclk_freq_hz = 20000000,
.xclk_freq_hz = 20000000,//EXPERIMENTAL: Set to 16MHz on ESP32-S2 or ESP32-S3 to enable EDMA mode
.ledc_timer = LEDC_TIMER_0,
.ledc_channel = LEDC_CHANNEL_0,
@@ -141,7 +151,8 @@ static camera_config_t camera_config = {
.frame_size = FRAMESIZE_UXGA,//QQVGA-QXGA Do not use sizes above QVGA when not JPEG
.jpeg_quality = 12, //0-63 lower number means higher quality
.fb_count = 1 //if more than one, i2s runs in continuous mode. Use only with JPEG
.fb_count = 1, //if more than one, i2s runs in continuous mode. Use only with JPEG
.grab_mode = CAMERA_GRAB_WHEN_EMPTY//CAMERA_GRAB_LATEST. Sets when buffers should be filled
};
esp_err_t camera_init(){

View File

@@ -1,4 +1,4 @@
COMPONENT_ADD_INCLUDEDIRS := driver/include conversions/include
COMPONENT_PRIV_INCLUDEDIRS := driver/private_include conversions/private_include sensors/private_include
COMPONENT_SRCDIRS := driver conversions sensors
COMPONENT_PRIV_INCLUDEDIRS := driver/private_include conversions/private_include sensors/private_include target/private_include
COMPONENT_SRCDIRS := driver conversions sensors target target/esp32
CXXFLAGS += -fno-rtti

View File

@@ -17,7 +17,11 @@
#if ESP_IDF_VERSION_MAJOR >= 4 // IDF 4+
#if CONFIG_IDF_TARGET_ESP32 // ESP32/PICO-D4
#include "esp32/rom/tjpgd.h"
#else
#elif CONFIG_IDF_TARGET_ESP32S2
#include "tjpgd.h"
#elif CONFIG_IDF_TARGET_ESP32S3
#include "esp32s3/rom/tjpgd.h"
#else
#error Target CONFIG_IDF_TARGET is not supported
#endif
#else // ESP32 Before IDF 4.0

View File

@@ -22,6 +22,7 @@ extern "C" {
#include <stdint.h>
#include <stdbool.h>
#include "esp_camera.h"
#include "esp_jpg_decode.h"
typedef size_t (* jpg_out_cb)(void * arg, size_t index, const void* data, size_t len);
@@ -120,6 +121,8 @@ bool frame2bmp(camera_fb_t * fb, uint8_t ** out, size_t * out_len);
*/
bool fmt2rgb888(const uint8_t *src_buf, size_t src_len, pixformat_t format, uint8_t * rgb_buf);
bool jpg2rgb565(const uint8_t *src, size_t src_len, uint8_t * out, jpg_scale_t scale);
#ifdef __cplusplus
}
#endif

View File

@@ -24,6 +24,10 @@
#if ESP_IDF_VERSION_MAJOR >= 4 // IDF 4+
#if CONFIG_IDF_TARGET_ESP32 // ESP32/PICO-D4
#include "esp32/spiram.h"
#elif CONFIG_IDF_TARGET_ESP32S2
#include "esp32s2/spiram.h"
#elif CONFIG_IDF_TARGET_ESP32S3
#include "esp32s3/spiram.h"
#else
#error Target CONFIG_IDF_TARGET is not supported
#endif
@@ -115,6 +119,54 @@ static bool _rgb_write(void * arg, uint16_t x, uint16_t y, uint16_t w, uint16_t
return true;
}
static bool _rgb565_write(void * arg, uint16_t x, uint16_t y, uint16_t w, uint16_t h, uint8_t *data)
{
rgb_jpg_decoder * jpeg = (rgb_jpg_decoder *)arg;
if(!data){
if(x == 0 && y == 0){
//write start
jpeg->width = w;
jpeg->height = h;
//if output is null, this is BMP
if(!jpeg->output){
jpeg->output = (uint8_t *)_malloc((w*h*3)+jpeg->data_offset);
if(!jpeg->output){
return false;
}
}
} else {
//write end
}
return true;
}
size_t jw = jpeg->width*3;
size_t jw2 = jpeg->width*2;
size_t t = y * jw;
size_t t2 = y * jw2;
size_t b = t + (h * jw);
size_t l = x * 2;
uint8_t *out = jpeg->output+jpeg->data_offset;
uint8_t *o = out;
size_t iy, iy2, ix, ix2;
w = w * 3;
for(iy=t, iy2=t2; iy<b; iy+=jw, iy2+=jw2) {
o = out+iy2+l;
for(ix2=ix=0; ix<w; ix+= 3, ix2 +=2) {
uint16_t r = data[ix];
uint16_t g = data[ix+1];
uint16_t b = data[ix+2];
uint16_t c = ((r & 0xF8) << 8) | ((g & 0xFC) << 3) | (b >> 3);
o[ix2+1] = c>>8;
o[ix2] = c&0xff;
}
data+=w;
}
return true;
}
//input buffer
static uint32_t _jpg_read(void * arg, size_t index, uint8_t *buf, size_t len)
{
@@ -140,6 +192,21 @@ static bool jpg2rgb888(const uint8_t *src, size_t src_len, uint8_t * out, jpg_sc
return true;
}
bool jpg2rgb565(const uint8_t *src, size_t src_len, uint8_t * out, jpg_scale_t scale)
{
rgb_jpg_decoder jpeg;
jpeg.width = 0;
jpeg.height = 0;
jpeg.input = src;
jpeg.output = out;
jpeg.data_offset = 0;
if(esp_jpg_decode(src_len, scale, _jpg_read, _rgb565_write, (void*)&jpeg) != ESP_OK){
return false;
}
return true;
}
bool jpg2bmp(const uint8_t *src, size_t src_len, uint8_t ** out, size_t * out_len)
{
@@ -317,7 +384,7 @@ bool fmt2bmp(uint8_t *src, size_t src_len, uint16_t width, uint16_t height, pixf
}
*out = out_buf;
*out_len = out_size;
return true;
return true;
}
bool frame2bmp(camera_fb_t * fb, uint8_t ** out, size_t * out_len)

View File

@@ -25,6 +25,10 @@
#if ESP_IDF_VERSION_MAJOR >= 4 // IDF 4+
#if CONFIG_IDF_TARGET_ESP32 // ESP32/PICO-D4
#include "esp32/spiram.h"
#elif CONFIG_IDF_TARGET_ESP32S2
#include "esp32s2/spiram.h"
#elif CONFIG_IDF_TARGET_ESP32S3
#include "esp32s3/spiram.h"
#else
#error Target CONFIG_IDF_TARGET is not supported
#endif
@@ -195,7 +199,7 @@ public:
return true;
}
if ((size_t)len > (max_len - index)) {
ESP_LOGW(TAG, "JPG output overflow: %d bytes", len - (max_len - index));
//ESP_LOGW(TAG, "JPG output overflow: %d bytes (%d,%d,%d)", len - (max_len - index), len, index, max_len);
len = max_len - index;
}
if (len) {
@@ -215,7 +219,7 @@ bool fmt2jpg(uint8_t *src, size_t src_len, uint16_t width, uint16_t height, pixf
{
//todo: allocate proper buffer for holding JPEG data
//this should be enough for CIF frame size
int jpg_buf_len = 64*1024;
int jpg_buf_len = 128*1024;
uint8_t * jpg_buf = (uint8_t *)_malloc(jpg_buf_len);

View File

@@ -0,0 +1,483 @@
// Copyright 2010-2020 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdio.h>
#include <string.h>
#include "esp_heap_caps.h"
#include "ll_cam.h"
#include "cam_hal.h"
static const char *TAG = "cam_hal";
static cam_obj_t *cam_obj = NULL;
static const uint32_t JPEG_SOI_MARKER = 0xFFD8FF; // written in little-endian for esp32
static const uint16_t JPEG_EOI_MARKER = 0xD9FF; // written in little-endian for esp32
static int cam_verify_jpeg_soi(const uint8_t *inbuf, uint32_t length)
{
uint32_t sig = *((uint32_t *)inbuf) & 0xFFFFFF;
if(sig != JPEG_SOI_MARKER) {
for (uint32_t i = 0; i < length; i++) {
sig = *((uint32_t *)(&inbuf[i])) & 0xFFFFFF;
if (sig == JPEG_SOI_MARKER) {
ESP_LOGW(TAG, "SOI: %d", i);
return i;
}
}
ESP_LOGW(TAG, "NO-SOI");
return -1;
}
return 0;
}
static int cam_verify_jpeg_eoi(const uint8_t *inbuf, uint32_t length)
{
int offset = -1;
uint8_t *dptr = (uint8_t *)inbuf + length - 2;
while (dptr > inbuf) {
uint16_t sig = *((uint16_t *)dptr);
if (JPEG_EOI_MARKER == sig) {
offset = dptr - inbuf;
//ESP_LOGW(TAG, "EOI: %d", length - (offset + 2));
return offset;
}
dptr--;
}
return -1;
}
static bool cam_get_next_frame(int * frame_pos)
{
if(!cam_obj->frames[*frame_pos].en){
for (int x = 0; x < cam_obj->frame_cnt; x++) {
if (cam_obj->frames[x].en) {
*frame_pos = x;
return true;
}
}
} else {
return true;
}
return false;
}
static bool cam_start_frame(int * frame_pos)
{
if (cam_get_next_frame(frame_pos)) {
if(ll_cam_start(cam_obj, *frame_pos)){
// Vsync the frame manually
ll_cam_do_vsync(cam_obj);
uint64_t us = (uint64_t)esp_timer_get_time();
cam_obj->frames[*frame_pos].fb.timestamp.tv_sec = us / 1000000UL;
cam_obj->frames[*frame_pos].fb.timestamp.tv_usec = us % 1000000UL;
return true;
}
}
return false;
}
void IRAM_ATTR ll_cam_send_event(cam_obj_t *cam, cam_event_t cam_event, BaseType_t * HPTaskAwoken)
{
if (xQueueSendFromISR(cam->event_queue, (void *)&cam_event, HPTaskAwoken) != pdTRUE) {
ll_cam_stop(cam);
cam->state = CAM_STATE_IDLE;
ESP_EARLY_LOGE(TAG, "EV-%s-OVF", cam_event==CAM_IN_SUC_EOF_EVENT ? "EOF" : "VSYNC");
}
}
//Copy fram from DMA dma_buffer to fram dma_buffer
static void cam_task(void *arg)
{
int cnt = 0;
int frame_pos = 0;
cam_obj->state = CAM_STATE_IDLE;
cam_event_t cam_event = 0;
xQueueReset(cam_obj->event_queue);
while (1) {
xQueueReceive(cam_obj->event_queue, (void *)&cam_event, portMAX_DELAY);
DBG_PIN_SET(1);
switch (cam_obj->state) {
case CAM_STATE_IDLE: {
if (cam_event == CAM_VSYNC_EVENT) {
//DBG_PIN_SET(1);
if(cam_start_frame(&frame_pos)){
cam_obj->frames[frame_pos].fb.len = 0;
cam_obj->state = CAM_STATE_READ_BUF;
}
cnt = 0;
}
}
break;
case CAM_STATE_READ_BUF: {
camera_fb_t * frame_buffer_event = &cam_obj->frames[frame_pos].fb;
size_t pixels_per_dma = (cam_obj->dma_half_buffer_size * cam_obj->fb_bytes_per_pixel) / (cam_obj->dma_bytes_per_item * cam_obj->in_bytes_per_pixel);
if (cam_event == CAM_IN_SUC_EOF_EVENT) {
if(!cam_obj->psram_mode){
if (cam_obj->fb_size < (frame_buffer_event->len + pixels_per_dma)) {
ESP_LOGW(TAG, "FB-OVF");
ll_cam_stop(cam_obj);
DBG_PIN_SET(0);
continue;
}
frame_buffer_event->len += ll_cam_memcpy(cam_obj,
&frame_buffer_event->buf[frame_buffer_event->len],
&cam_obj->dma_buffer[(cnt % cam_obj->dma_half_buffer_cnt) * cam_obj->dma_half_buffer_size],
cam_obj->dma_half_buffer_size);
}
//Check for JPEG SOI in the first buffer. stop if not found
if (cam_obj->jpeg_mode && cnt == 0 && cam_verify_jpeg_soi(frame_buffer_event->buf, frame_buffer_event->len) != 0) {
ll_cam_stop(cam_obj);
cam_obj->state = CAM_STATE_IDLE;
}
cnt++;
} else if (cam_event == CAM_VSYNC_EVENT) {
//DBG_PIN_SET(1);
ll_cam_stop(cam_obj);
if (cnt || !cam_obj->jpeg_mode || cam_obj->psram_mode) {
if (cam_obj->jpeg_mode) {
if (!cam_obj->psram_mode) {
if (cam_obj->fb_size < (frame_buffer_event->len + pixels_per_dma)) {
ESP_LOGW(TAG, "FB-OVF");
cnt--;
} else {
frame_buffer_event->len += ll_cam_memcpy(cam_obj,
&frame_buffer_event->buf[frame_buffer_event->len],
&cam_obj->dma_buffer[(cnt % cam_obj->dma_half_buffer_cnt) * cam_obj->dma_half_buffer_size],
cam_obj->dma_half_buffer_size);
}
}
cnt++;
}
cam_obj->frames[frame_pos].en = 0;
if (cam_obj->psram_mode) {
if (cam_obj->jpeg_mode) {
frame_buffer_event->len = cnt * cam_obj->dma_half_buffer_size;
} else {
frame_buffer_event->len = cam_obj->recv_size;
}
} else if (!cam_obj->jpeg_mode) {
if (frame_buffer_event->len != cam_obj->fb_size) {
cam_obj->frames[frame_pos].en = 1;
ESP_LOGE(TAG, "FB-SIZE: %u != %u", frame_buffer_event->len, cam_obj->fb_size);
}
}
//send frame
if(!cam_obj->frames[frame_pos].en && xQueueSend(cam_obj->frame_buffer_queue, (void *)&frame_buffer_event, 0) != pdTRUE) {
//pop frame buffer from the queue
camera_fb_t * fb2 = NULL;
if(xQueueReceive(cam_obj->frame_buffer_queue, &fb2, 0) == pdTRUE) {
//push the new frame to the end of the queue
if (xQueueSend(cam_obj->frame_buffer_queue, (void *)&frame_buffer_event, 0) != pdTRUE) {
cam_obj->frames[frame_pos].en = 1;
ESP_LOGE(TAG, "FBQ-SND");
}
//free the popped buffer
cam_give(fb2);
} else {
//queue is full and we could not pop a frame from it
cam_obj->frames[frame_pos].en = 1;
ESP_LOGE(TAG, "FBQ-RCV");
}
}
}
if(!cam_start_frame(&frame_pos)){
cam_obj->state = CAM_STATE_IDLE;
} else {
cam_obj->frames[frame_pos].fb.len = 0;
}
cnt = 0;
}
}
break;
}
DBG_PIN_SET(0);
}
}
static lldesc_t * allocate_dma_descriptors(uint32_t count, uint16_t size, uint8_t * buffer)
{
lldesc_t *dma = (lldesc_t *)heap_caps_malloc(count * sizeof(lldesc_t), MALLOC_CAP_DMA);
if (dma == NULL) {
return dma;
}
for (int x = 0; x < count; x++) {
dma[x].size = size;
dma[x].length = 0;
dma[x].sosf = 0;
dma[x].eof = 0;
dma[x].owner = 1;
dma[x].buf = (buffer + size * x);
dma[x].empty = (uint32_t)&dma[(x + 1) % count];
}
return dma;
}
static esp_err_t cam_dma_config(const camera_config_t *config)
{
bool ret = ll_cam_dma_sizes(cam_obj);
if (0 == ret) {
return ESP_FAIL;
}
cam_obj->dma_node_cnt = (cam_obj->dma_buffer_size) / cam_obj->dma_node_buffer_size; // Number of DMA nodes
cam_obj->frame_copy_cnt = cam_obj->recv_size / cam_obj->dma_half_buffer_size; // Number of interrupted copies, ping-pong copy
ESP_LOGI(TAG, "buffer_size: %d, half_buffer_size: %d, node_buffer_size: %d, node_cnt: %d, total_cnt: %d",
cam_obj->dma_buffer_size, cam_obj->dma_half_buffer_size, cam_obj->dma_node_buffer_size, cam_obj->dma_node_cnt, cam_obj->frame_copy_cnt);
cam_obj->dma_buffer = NULL;
cam_obj->dma = NULL;
cam_obj->frames = (cam_frame_t *)heap_caps_calloc(1, cam_obj->frame_cnt * sizeof(cam_frame_t), MALLOC_CAP_DEFAULT);
CAM_CHECK(cam_obj->frames != NULL, "frames malloc failed", ESP_FAIL);
uint8_t dma_align = 0;
size_t fb_size = cam_obj->fb_size;
if (cam_obj->psram_mode) {
dma_align = ll_cam_get_dma_align(cam_obj);
if (cam_obj->fb_size < cam_obj->recv_size) {
fb_size = cam_obj->recv_size;
}
}
/* Allocate memory for frame buffer */
size_t alloc_size = fb_size * sizeof(uint8_t) + dma_align;
uint32_t _caps = MALLOC_CAP_8BIT;
if (CAMERA_FB_IN_DRAM == config->fb_location) {
_caps |= MALLOC_CAP_INTERNAL;
} else {
_caps |= MALLOC_CAP_SPIRAM;
}
for (int x = 0; x < cam_obj->frame_cnt; x++) {
cam_obj->frames[x].dma = NULL;
cam_obj->frames[x].fb_offset = 0;
cam_obj->frames[x].en = 0;
ESP_LOGI(TAG, "Allocating %d Byte frame buffer in %s", alloc_size, _caps & MALLOC_CAP_SPIRAM ? "PSRAM" : "OnBoard RAM");
cam_obj->frames[x].fb.buf = (uint8_t *)heap_caps_malloc(alloc_size, _caps);
CAM_CHECK(cam_obj->frames[x].fb.buf != NULL, "frame buffer malloc failed", ESP_FAIL);
if (cam_obj->psram_mode) {
//align PSRAM buffer. TODO: save the offset so proper address can be freed later
cam_obj->frames[x].fb_offset = dma_align - ((uint32_t)cam_obj->frames[x].fb.buf & (dma_align - 1));
cam_obj->frames[x].fb.buf += cam_obj->frames[x].fb_offset;
ESP_LOGI(TAG, "Frame[%d]: Offset: %u, Addr: 0x%08X", x, cam_obj->frames[x].fb_offset, (uint32_t)cam_obj->frames[x].fb.buf);
cam_obj->frames[x].dma = allocate_dma_descriptors(cam_obj->dma_node_cnt, cam_obj->dma_node_buffer_size, cam_obj->frames[x].fb.buf);
CAM_CHECK(cam_obj->frames[x].dma != NULL, "frame dma malloc failed", ESP_FAIL);
}
cam_obj->frames[x].en = 1;
}
if (!cam_obj->psram_mode) {
cam_obj->dma_buffer = (uint8_t *)heap_caps_malloc(cam_obj->dma_buffer_size * sizeof(uint8_t), MALLOC_CAP_DMA);
if(NULL == cam_obj->dma_buffer) {
ESP_LOGE(TAG,"%s(%d): DMA buffer %d Byte malloc failed, the current largest free block:%d Byte", __FUNCTION__, __LINE__,
cam_obj->dma_buffer_size, heap_caps_get_largest_free_block(MALLOC_CAP_DMA));
return ESP_FAIL;
}
cam_obj->dma = allocate_dma_descriptors(cam_obj->dma_node_cnt, cam_obj->dma_node_buffer_size, cam_obj->dma_buffer);
CAM_CHECK(cam_obj->dma != NULL, "dma malloc failed", ESP_FAIL);
}
return ESP_OK;
}
esp_err_t cam_init(const camera_config_t *config)
{
CAM_CHECK(NULL != config, "config pointer is invalid", ESP_ERR_INVALID_ARG);
esp_err_t ret = ESP_OK;
cam_obj = (cam_obj_t *)heap_caps_calloc(1, sizeof(cam_obj_t), MALLOC_CAP_DMA);
CAM_CHECK(NULL != cam_obj, "lcd_cam object malloc error", ESP_ERR_NO_MEM);
cam_obj->swap_data = 0;
cam_obj->vsync_pin = config->pin_vsync;
cam_obj->vsync_invert = true;
ll_cam_set_pin(cam_obj, config);
ret = ll_cam_config(cam_obj, config);
CAM_CHECK_GOTO(ret == ESP_OK, "ll_cam initialize failed", err);
#if CAMERA_DBG_PIN_ENABLE
PIN_FUNC_SELECT(GPIO_PIN_MUX_REG[DBG_PIN_NUM], PIN_FUNC_GPIO);
gpio_set_direction(DBG_PIN_NUM, GPIO_MODE_OUTPUT);
gpio_set_pull_mode(DBG_PIN_NUM, GPIO_FLOATING);
#endif
ESP_LOGI(TAG, "cam init ok");
return ESP_OK;
err:
free(cam_obj);
cam_obj = NULL;
return ESP_FAIL;
}
esp_err_t cam_config(const camera_config_t *config, framesize_t frame_size, uint16_t sensor_pid)
{
CAM_CHECK(NULL != config, "config pointer is invalid", ESP_ERR_INVALID_ARG);
esp_err_t ret = ESP_OK;
ret = ll_cam_set_sample_mode(cam_obj, (pixformat_t)config->pixel_format, config->xclk_freq_hz, sensor_pid);
cam_obj->jpeg_mode = config->pixel_format == PIXFORMAT_JPEG;
#if CONFIG_IDF_TARGET_ESP32
cam_obj->psram_mode = false;
#else
cam_obj->psram_mode = (config->xclk_freq_hz == 16000000);
#endif
cam_obj->frame_cnt = config->fb_count;
cam_obj->width = resolution[frame_size].width;
cam_obj->height = resolution[frame_size].height;
if(cam_obj->jpeg_mode){
cam_obj->recv_size = cam_obj->width * cam_obj->height / 5;
cam_obj->fb_size = cam_obj->recv_size;
} else {
cam_obj->recv_size = cam_obj->width * cam_obj->height * cam_obj->in_bytes_per_pixel;
cam_obj->fb_size = cam_obj->width * cam_obj->height * cam_obj->fb_bytes_per_pixel;
}
ret = cam_dma_config(config);
CAM_CHECK_GOTO(ret == ESP_OK, "cam_dma_config failed", err);
cam_obj->event_queue = xQueueCreate(cam_obj->dma_half_buffer_cnt - 1, sizeof(cam_event_t));
CAM_CHECK_GOTO(cam_obj->event_queue != NULL, "event_queue create failed", err);
size_t frame_buffer_queue_len = cam_obj->frame_cnt;
if (config->grab_mode == CAMERA_GRAB_LATEST && cam_obj->frame_cnt > 1) {
frame_buffer_queue_len = cam_obj->frame_cnt - 1;
}
cam_obj->frame_buffer_queue = xQueueCreate(frame_buffer_queue_len, sizeof(camera_fb_t*));
CAM_CHECK_GOTO(cam_obj->frame_buffer_queue != NULL, "frame_buffer_queue create failed", err);
ret = ll_cam_init_isr(cam_obj);
CAM_CHECK_GOTO(ret == ESP_OK, "cam intr alloc failed", err);
#if CONFIG_CAMERA_CORE0
xTaskCreatePinnedToCore(cam_task, "cam_task", 2048, NULL, configMAX_PRIORITIES - 2, &cam_obj->task_handle, 0);
#elif CONFIG_CAMERA_CORE1
xTaskCreatePinnedToCore(cam_task, "cam_task", 2048, NULL, configMAX_PRIORITIES - 2, &cam_obj->task_handle, 1);
#else
xTaskCreate(cam_task, "cam_task", 2048, NULL, configMAX_PRIORITIES - 2, &cam_obj->task_handle);
#endif
ESP_LOGI(TAG, "cam config ok");
return ESP_OK;
err:
cam_deinit();
return ESP_FAIL;
}
esp_err_t cam_deinit(void)
{
if (!cam_obj) {
return ESP_FAIL;
}
cam_stop();
if (cam_obj->task_handle) {
vTaskDelete(cam_obj->task_handle);
}
if (cam_obj->event_queue) {
vQueueDelete(cam_obj->event_queue);
}
if (cam_obj->frame_buffer_queue) {
vQueueDelete(cam_obj->frame_buffer_queue);
}
if (cam_obj->dma) {
free(cam_obj->dma);
}
if (cam_obj->dma_buffer) {
free(cam_obj->dma_buffer);
}
if (cam_obj->frames) {
for (int x = 0; x < cam_obj->frame_cnt; x++) {
free(cam_obj->frames[x].fb.buf - cam_obj->frames[x].fb_offset);
if (cam_obj->frames[x].dma) {
free(cam_obj->frames[x].dma);
}
}
free(cam_obj->frames);
}
ll_cam_deinit(cam_obj);
free(cam_obj);
cam_obj = NULL;
return ESP_OK;
}
void cam_stop(void)
{
ll_cam_vsync_intr_enable(cam_obj, false);
ll_cam_stop(cam_obj);
}
void cam_start(void)
{
ll_cam_vsync_intr_enable(cam_obj, true);
}
camera_fb_t *cam_take(TickType_t timeout)
{
camera_fb_t *dma_buffer = NULL;
TickType_t start = xTaskGetTickCount();
xQueueReceive(cam_obj->frame_buffer_queue, (void *)&dma_buffer, timeout);
if (dma_buffer) {
if(cam_obj->jpeg_mode){
// find the end marker for JPEG. Data after that can be discarded
int offset_e = cam_verify_jpeg_eoi(dma_buffer->buf, dma_buffer->len);
if (offset_e >= 0) {
// adjust buffer length
dma_buffer->len = offset_e + sizeof(JPEG_EOI_MARKER);
return dma_buffer;
} else {
ESP_LOGW(TAG, "NO-EOI");
cam_give(dma_buffer);
return cam_take(timeout - (xTaskGetTickCount() - start));//recurse!!!!
}
} else if(cam_obj->psram_mode && cam_obj->in_bytes_per_pixel != cam_obj->fb_bytes_per_pixel){
//currently this is used only for YUV to GRAYSCALE
dma_buffer->len = ll_cam_memcpy(cam_obj, dma_buffer->buf, dma_buffer->buf, dma_buffer->len);
}
return dma_buffer;
} else {
ESP_LOGW(TAG, "Failed to get the frame on time!");
}
return NULL;
}
void cam_give(camera_fb_t *dma_buffer)
{
for (int x = 0; x < cam_obj->frame_cnt; x++) {
if (&cam_obj->frames[x].fb == dma_buffer) {
cam_obj->frames[x].en = 1;
break;
}
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,421 @@
// Copyright 2015-2016 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "time.h"
#include "sys/time.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "driver/gpio.h"
#include "esp_system.h"
#include "nvs_flash.h"
#include "nvs.h"
#include "sensor.h"
#include "sccb.h"
#include "cam_hal.h"
#include "esp_camera.h"
#include "xclk.h"
#if CONFIG_OV2640_SUPPORT
#include "ov2640.h"
#endif
#if CONFIG_OV7725_SUPPORT
#include "ov7725.h"
#endif
#if CONFIG_OV3660_SUPPORT
#include "ov3660.h"
#endif
#if CONFIG_OV5640_SUPPORT
#include "ov5640.h"
#endif
#if CONFIG_NT99141_SUPPORT
#include "nt99141.h"
#endif
#if CONFIG_OV7670_SUPPORT
#include "ov7670.h"
#endif
#if CONFIG_GC2145_SUPPORT
#include "gc2145.h"
#endif
#if CONFIG_GC032A_SUPPORT
#include "gc032a.h"
#endif
#if CONFIG_GC0308_SUPPORT
#include "gc0308.h"
#endif
#if CONFIG_BF3005_SUPPORT
#include "bf3005.h"
#endif
#if defined(ARDUINO_ARCH_ESP32) && defined(CONFIG_ARDUHAL_ESP_LOG)
#include "esp32-hal-log.h"
#define TAG ""
#else
#include "esp_log.h"
static const char *TAG = "camera";
#endif
typedef struct {
sensor_t sensor;
camera_fb_t fb;
} camera_state_t;
static const char *CAMERA_SENSOR_NVS_KEY = "sensor";
static const char *CAMERA_PIXFORMAT_NVS_KEY = "pixformat";
static camera_state_t *s_state = NULL;
#if CONFIG_IDF_TARGET_ESP32S3 // LCD_CAM module of ESP32-S3 will generate xclk
#define CAMERA_ENABLE_OUT_CLOCK(v)
#define CAMERA_DISABLE_OUT_CLOCK()
#else
#define CAMERA_ENABLE_OUT_CLOCK(v) camera_enable_out_clock((v))
#define CAMERA_DISABLE_OUT_CLOCK() camera_disable_out_clock()
#endif
typedef struct {
int (*detect)(int slv_addr, sensor_id_t *id);
int (*init)(sensor_t *sensor);
} sensor_func_t;
static const sensor_func_t g_sensors[] = {
#if CONFIG_OV7725_SUPPORT
{ov7725_detect, ov7725_init},
#endif
#if CONFIG_OV7670_SUPPORT
{ov7670_detect, ov7670_init},
#endif
#if CONFIG_OV2640_SUPPORT
{ov2640_detect, ov2640_init},
#endif
#if CONFIG_OV3660_SUPPORT
{ov3660_detect, ov3660_init},
#endif
#if CONFIG_OV5640_SUPPORT
{ov5640_detect, ov5640_init},
#endif
#if CONFIG_NT99141_SUPPORT
{nt99141_detect, nt99141_init},
#endif
#if CONFIG_GC2145_SUPPORT
{gc2145_detect, gc2145_init},
#endif
#if CONFIG_GC032A_SUPPORT
{gc032a_detect, gc032a_init},
#endif
#if CONFIG_GC0308_SUPPORT
{gc0308_detect, gc0308_init},
#endif
#if CONFIG_BF3005_SUPPORT
{bf3005_detect, bf3005_init},
#endif
};
static esp_err_t camera_probe(const camera_config_t *config, camera_model_t *out_camera_model)
{
*out_camera_model = CAMERA_NONE;
if (s_state != NULL) {
return ESP_ERR_INVALID_STATE;
}
s_state = (camera_state_t *) calloc(sizeof(camera_state_t), 1);
if (!s_state) {
return ESP_ERR_NO_MEM;
}
if (config->pin_xclk >= 0) {
ESP_LOGD(TAG, "Enabling XCLK output");
CAMERA_ENABLE_OUT_CLOCK(config);
}
if (config->pin_sscb_sda != -1) {
ESP_LOGD(TAG, "Initializing SSCB");
SCCB_Init(config->pin_sscb_sda, config->pin_sscb_scl);
}
if (config->pin_pwdn >= 0) {
ESP_LOGD(TAG, "Resetting camera by power down line");
gpio_config_t conf = { 0 };
conf.pin_bit_mask = 1LL << config->pin_pwdn;
conf.mode = GPIO_MODE_OUTPUT;
gpio_config(&conf);
// carefull, logic is inverted compared to reset pin
gpio_set_level(config->pin_pwdn, 1);
vTaskDelay(10 / portTICK_PERIOD_MS);
gpio_set_level(config->pin_pwdn, 0);
vTaskDelay(10 / portTICK_PERIOD_MS);
}
if (config->pin_reset >= 0) {
ESP_LOGD(TAG, "Resetting camera");
gpio_config_t conf = { 0 };
conf.pin_bit_mask = 1LL << config->pin_reset;
conf.mode = GPIO_MODE_OUTPUT;
gpio_config(&conf);
gpio_set_level(config->pin_reset, 0);
vTaskDelay(10 / portTICK_PERIOD_MS);
gpio_set_level(config->pin_reset, 1);
vTaskDelay(10 / portTICK_PERIOD_MS);
}
ESP_LOGD(TAG, "Searching for camera address");
vTaskDelay(10 / portTICK_PERIOD_MS);
uint8_t slv_addr = SCCB_Probe();
if (slv_addr == 0) {
CAMERA_DISABLE_OUT_CLOCK();
return ESP_ERR_NOT_FOUND;
}
ESP_LOGI(TAG, "Detected camera at address=0x%02x", slv_addr);
s_state->sensor.slv_addr = slv_addr;
s_state->sensor.xclk_freq_hz = config->xclk_freq_hz;
/**
* Read sensor ID and then initialize sensor
* Attention: Some sensors have the same SCCB address. Therefore, several attempts may be made in the detection process
*/
sensor_id_t *id = &s_state->sensor.id;
for (size_t i = 0; i < sizeof(g_sensors) / sizeof(sensor_func_t); i++) {
if (g_sensors[i].detect(slv_addr, id)) {
camera_sensor_info_t *info = esp_camera_sensor_get_info(id);
if (NULL != info) {
*out_camera_model = info->model;
ESP_LOGI(TAG, "Detected %s camera", info->name);
g_sensors[i].init(&s_state->sensor);
break;
}
}
}
if (CAMERA_NONE == *out_camera_model) { //If no supported sensors are detected
CAMERA_DISABLE_OUT_CLOCK();
ESP_LOGE(TAG, "Detected camera not supported.");
return ESP_ERR_NOT_SUPPORTED;
}
ESP_LOGI(TAG, "Camera PID=0x%02x VER=0x%02x MIDL=0x%02x MIDH=0x%02x",
id->PID, id->VER, id->MIDH, id->MIDL);
ESP_LOGD(TAG, "Doing SW reset of sensor");
vTaskDelay(10 / portTICK_PERIOD_MS);
s_state->sensor.reset(&s_state->sensor);
return ESP_OK;
}
esp_err_t esp_camera_init(const camera_config_t *config)
{
esp_err_t err;
err = cam_init(config);
if (err != ESP_OK) {
ESP_LOGE(TAG, "Camera init failed with error 0x%x", err);
return err;
}
camera_model_t camera_model = CAMERA_NONE;
err = camera_probe(config, &camera_model);
if (err != ESP_OK) {
ESP_LOGE(TAG, "Camera probe failed with error 0x%x(%s)", err, esp_err_to_name(err));
goto fail;
}
framesize_t frame_size = (framesize_t) config->frame_size;
pixformat_t pix_format = (pixformat_t) config->pixel_format;
if (PIXFORMAT_JPEG == pix_format && (!camera_sensor[camera_model].support_jpeg)) {
ESP_LOGE(TAG, "JPEG format is not supported on this sensor");
err = ESP_ERR_NOT_SUPPORTED;
goto fail;
}
if (frame_size > camera_sensor[camera_model].max_size) {
ESP_LOGW(TAG, "The frame size exceeds the maximum for this sensor, it will be forced to the maximum possible value");
frame_size = camera_sensor[camera_model].max_size;
}
err = cam_config(config, frame_size, s_state->sensor.id.PID);
if (err != ESP_OK) {
ESP_LOGE(TAG, "Camera config failed with error 0x%x", err);
goto fail;
}
s_state->sensor.status.framesize = frame_size;
s_state->sensor.pixformat = pix_format;
ESP_LOGD(TAG, "Setting frame size to %dx%d", resolution[frame_size].width, resolution[frame_size].height);
if (s_state->sensor.set_framesize(&s_state->sensor, frame_size) != 0) {
ESP_LOGE(TAG, "Failed to set frame size");
err = ESP_ERR_CAMERA_FAILED_TO_SET_FRAME_SIZE;
goto fail;
}
s_state->sensor.set_pixformat(&s_state->sensor, pix_format);
if (s_state->sensor.id.PID == OV2640_PID) {
s_state->sensor.set_gainceiling(&s_state->sensor, GAINCEILING_2X);
s_state->sensor.set_bpc(&s_state->sensor, false);
s_state->sensor.set_wpc(&s_state->sensor, true);
s_state->sensor.set_lenc(&s_state->sensor, true);
}
if (pix_format == PIXFORMAT_JPEG) {
s_state->sensor.set_quality(&s_state->sensor, config->jpeg_quality);
}
s_state->sensor.init_status(&s_state->sensor);
cam_start();
return ESP_OK;
fail:
esp_camera_deinit();
return err;
}
esp_err_t esp_camera_deinit()
{
esp_err_t ret = cam_deinit();
CAMERA_DISABLE_OUT_CLOCK();
if (s_state) {
SCCB_Deinit();
free(s_state);
s_state = NULL;
}
return ret;
}
#define FB_GET_TIMEOUT (4000 / portTICK_PERIOD_MS)
camera_fb_t *esp_camera_fb_get()
{
if (s_state == NULL) {
return NULL;
}
camera_fb_t *fb = cam_take(FB_GET_TIMEOUT);
//set the frame properties
if (fb) {
fb->width = resolution[s_state->sensor.status.framesize].width;
fb->height = resolution[s_state->sensor.status.framesize].height;
fb->format = s_state->sensor.pixformat;
}
return fb;
}
void esp_camera_fb_return(camera_fb_t *fb)
{
if (s_state == NULL) {
return;
}
cam_give(fb);
}
sensor_t *esp_camera_sensor_get()
{
if (s_state == NULL) {
return NULL;
}
return &s_state->sensor;
}
esp_err_t esp_camera_save_to_nvs(const char *key)
{
#if ESP_IDF_VERSION_MAJOR > 3
nvs_handle_t handle;
#else
nvs_handle handle;
#endif
esp_err_t ret = nvs_open(key, NVS_READWRITE, &handle);
if (ret == ESP_OK) {
sensor_t *s = esp_camera_sensor_get();
if (s != NULL) {
ret = nvs_set_blob(handle, CAMERA_SENSOR_NVS_KEY, &s->status, sizeof(camera_status_t));
if (ret == ESP_OK) {
uint8_t pf = s->pixformat;
ret = nvs_set_u8(handle, CAMERA_PIXFORMAT_NVS_KEY, pf);
}
return ret;
} else {
return ESP_ERR_CAMERA_NOT_DETECTED;
}
nvs_close(handle);
return ret;
} else {
return ret;
}
}
esp_err_t esp_camera_load_from_nvs(const char *key)
{
#if ESP_IDF_VERSION_MAJOR > 3
nvs_handle_t handle;
#else
nvs_handle handle;
#endif
uint8_t pf;
esp_err_t ret = nvs_open(key, NVS_READWRITE, &handle);
if (ret == ESP_OK) {
sensor_t *s = esp_camera_sensor_get();
camera_status_t st;
if (s != NULL) {
size_t size = sizeof(camera_status_t);
ret = nvs_get_blob(handle, CAMERA_SENSOR_NVS_KEY, &st, &size);
if (ret == ESP_OK) {
s->set_ae_level(s, st.ae_level);
s->set_aec2(s, st.aec2);
s->set_aec_value(s, st.aec_value);
s->set_agc_gain(s, st.agc_gain);
s->set_awb_gain(s, st.awb_gain);
s->set_bpc(s, st.bpc);
s->set_brightness(s, st.brightness);
s->set_colorbar(s, st.colorbar);
s->set_contrast(s, st.contrast);
s->set_dcw(s, st.dcw);
s->set_denoise(s, st.denoise);
s->set_exposure_ctrl(s, st.aec);
s->set_framesize(s, st.framesize);
s->set_gain_ctrl(s, st.agc);
s->set_gainceiling(s, st.gainceiling);
s->set_hmirror(s, st.hmirror);
s->set_lenc(s, st.lenc);
s->set_quality(s, st.quality);
s->set_raw_gma(s, st.raw_gma);
s->set_saturation(s, st.saturation);
s->set_sharpness(s, st.sharpness);
s->set_special_effect(s, st.special_effect);
s->set_vflip(s, st.vflip);
s->set_wb_mode(s, st.wb_mode);
s->set_whitebal(s, st.awb);
s->set_wpc(s, st.wpc);
}
ret = nvs_get_u8(handle, CAMERA_PIXFORMAT_NVS_KEY, &pf);
if (ret == ESP_OK) {
s->set_pixformat(s, pf);
}
} else {
return ESP_ERR_CAMERA_NOT_DETECTED;
}
nvs_close(handle);
return ret;
} else {
ESP_LOGW(TAG, "Error (%d) opening nvs key \"%s\"", ret, key);
return ret;
}
}

View File

@@ -38,7 +38,8 @@
.pixel_format = PIXFORMAT_JPEG,
.frame_size = FRAMESIZE_SVGA,
.jpeg_quality = 10,
.fb_count = 2
.fb_count = 2,
.grab_mode = CAMERA_GRAB_WHEN_EMPTY
};
esp_err_t camera_example_init(){
@@ -74,6 +75,22 @@
extern "C" {
#endif
/**
* @brief Configuration structure for camera initialization
*/
typedef enum {
CAMERA_GRAB_WHEN_EMPTY, /*!< Fills buffers when they are empty. Less resources but first 'fb_count' frames might be old */
CAMERA_GRAB_LATEST /*!< Except when 1 frame buffer is used, queue will always contain the last 'fb_count' frames */
} camera_grab_mode_t;
/**
* @brief Camera frame buffer location
*/
typedef enum {
CAMERA_FB_IN_PSRAM, /*!< Frame buffer is placed in external PSRAM */
CAMERA_FB_IN_DRAM /*!< Frame buffer is placed in internal DRAM */
} camera_fb_location_t;
/**
* @brief Configuration structure for camera initialization
*/
@@ -95,7 +112,7 @@ typedef struct {
int pin_href; /*!< GPIO pin for camera HREF line */
int pin_pclk; /*!< GPIO pin for camera PCLK line */
int xclk_freq_hz; /*!< Frequency of XCLK signal, in Hz. Either 20KHz or 10KHz for OV2640 double FPS (Experimental) */
int xclk_freq_hz; /*!< Frequency of XCLK signal, in Hz. EXPERIMENTAL: Set to 16MHz on ESP32-S2 or ESP32-S3 to enable EDMA mode */
ledc_timer_t ledc_timer; /*!< LEDC timer to be used for generating XCLK */
ledc_channel_t ledc_channel; /*!< LEDC channel to be used for generating XCLK */
@@ -105,6 +122,8 @@ typedef struct {
int jpeg_quality; /*!< Quality of JPEG output. 0-63 lower means higher quality */
size_t fb_count; /*!< Number of frame buffers to be allocated. If more than one, then each frame will be acquired (double speed) */
camera_fb_location_t fb_location; /*!< The location where the frame buffer will be allocated */
camera_grab_mode_t grab_mode; /*!< When buffers should be filled */
} camera_config_t;
/**

View File

@@ -11,13 +11,51 @@
#include <stdint.h>
#include <stdbool.h>
#define NT99141_PID (0x14)
#define OV9650_PID (0x96)
#define OV7725_PID (0x77)
#define OV2640_PID (0x26)
#define OV3660_PID (0x36)
#define OV5640_PID (0x56)
#define OV7670_PID (0x76)
#ifdef __cplusplus
extern "C" {
#endif
typedef enum {
OV9650_PID = 0x96,
OV7725_PID = 0x77,
OV2640_PID = 0x26,
OV3660_PID = 0x3660,
OV5640_PID = 0x5640,
OV7670_PID = 0x76,
NT99141_PID = 0x1410,
GC2145_PID = 0x2145,
GC032A_PID = 0x232a,
GC0308_PID = 0x9b,
BF3005_PID = 0x30,
} camera_pid_t;
typedef enum {
CAMERA_OV7725,
CAMERA_OV2640,
CAMERA_OV3660,
CAMERA_OV5640,
CAMERA_OV7670,
CAMERA_NT99141,
CAMERA_GC2145,
CAMERA_GC032A,
CAMERA_GC0308,
CAMERA_BF3005,
CAMERA_MODEL_MAX,
CAMERA_NONE,
} camera_model_t;
typedef enum {
OV2640_SCCB_ADDR = 0x30,// 0x60 >> 1
OV5640_SCCB_ADDR = 0x3C,// 0x78 >> 1
OV3660_SCCB_ADDR = 0x3C,// 0x78 >> 1
OV7725_SCCB_ADDR = 0x21,// 0x42 >> 1
OV7670_SCCB_ADDR = 0x21,// 0x42 >> 1
NT99141_SCCB_ADDR = 0x2A,// 0x54 >> 1
GC2145_SCCB_ADDR = 0x3C,// 0x78 >> 1
GC032A_SCCB_ADDR = 0x21,// 0x42 >> 1
GC0308_SCCB_ADDR = 0x21,// 0x42 >> 1
BF3005_SCCB_ADDR = 0x6E,
} camera_sccb_addr_t;
typedef enum {
PIXFORMAT_RGB565, // 2BPP/RGB565
@@ -58,6 +96,15 @@ typedef enum {
FRAMESIZE_INVALID
} framesize_t;
typedef struct {
const camera_model_t model;
const char *name;
const camera_sccb_addr_t sccb_addr;
const camera_pid_t pid;
const framesize_t max_size;
const bool support_jpeg;
} camera_sensor_info_t;
typedef enum {
ASPECT_RATIO_4X3,
ASPECT_RATIO_3X2,
@@ -101,11 +148,13 @@ typedef struct {
// Resolution table (in sensor.c)
extern const resolution_info_t resolution[];
// camera sensor table (in sensor.c)
extern const camera_sensor_info_t camera_sensor[];
typedef struct {
uint8_t MIDH;
uint8_t MIDL;
uint8_t PID;
uint16_t PID;
uint8_t VER;
} sensor_id_t;
@@ -190,4 +239,10 @@ typedef struct _sensor {
int (*set_xclk) (sensor_t *sensor, int timer, int xclk);
} sensor_t;
camera_sensor_info_t *esp_camera_sensor_get_info(sensor_id_t *id);
#ifdef __cplusplus
}
#endif
#endif /* __SENSOR_H__ */

View File

@@ -0,0 +1,60 @@
// Copyright 2010-2020 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#pragma once
#include "esp_camera.h"
#ifdef __cplusplus
extern "C" {
#endif
/**
* @brief Uninitialize the lcd_cam module
*
* @param handle Provide handle pointer to release resources
*
* @return
* - ESP_OK Success
* - ESP_FAIL Uninitialize fail
*/
esp_err_t cam_deinit(void);
/**
* @brief Initialize the lcd_cam module
*
* @param config Configurations - see lcd_cam_config_t struct
*
* @return
* - ESP_OK Success
* - ESP_ERR_INVALID_ARG Parameter error
* - ESP_ERR_NO_MEM No memory to initialize lcd_cam
* - ESP_FAIL Initialize fail
*/
esp_err_t cam_init(const camera_config_t *config);
esp_err_t cam_config(const camera_config_t *config, framesize_t frame_size, uint16_t sensor_pid);
void cam_stop(void);
void cam_start(void);
camera_fb_t *cam_take(TickType_t timeout);
void cam_give(camera_fb_t *dma_buffer);
#ifdef __cplusplus
}
#endif

View File

@@ -1,49 +0,0 @@
#pragma once
#include <stdint.h>
#include <stddef.h>
#include <stdbool.h>
#include "esp_err.h"
#include "esp_intr_alloc.h"
#include "freertos/FreeRTOS.h"
#include "freertos/semphr.h"
#include "freertos/task.h"
#include "esp_camera.h"
#include "sensor.h"
#include "esp_system.h"
#if ESP_IDF_VERSION_MAJOR >= 4 // IDF 4+
#if CONFIG_IDF_TARGET_ESP32 // ESP32/PICO-D4
#include "esp32/rom/lldesc.h"
#else
#error Target CONFIG_IDF_TARGET is not supported
#endif
#else // ESP32 Before IDF 4.0
#include "rom/lldesc.h"
#endif
typedef union {
struct {
uint8_t sample2;
uint8_t unused2;
uint8_t sample1;
uint8_t unused1;
};
uint32_t val;
} dma_elem_t;
typedef enum {
/* camera sends byte sequence: s1, s2, s3, s4, ...
* fifo receives: 00 s1 00 s2, 00 s2 00 s3, 00 s3 00 s4, ...
*/
SM_0A0B_0B0C = 0,
/* camera sends byte sequence: s1, s2, s3, s4, ...
* fifo receives: 00 s1 00 s2, 00 s3 00 s4, ...
*/
SM_0A0B_0C0D = 1,
/* camera sends byte sequence: s1, s2, s3, s4, ...
* fifo receives: 00 s1 00 00, 00 s2 00 00, 00 s3 00 00, ...
*/
SM_0A00_0B00 = 3,
} i2s_sampling_mode_t;

View File

@@ -10,6 +10,7 @@
#define __SCCB_H__
#include <stdint.h>
int SCCB_Init(int pin_sda, int pin_scl);
int SCCB_Deinit(void);
uint8_t SCCB_Probe();
uint8_t SCCB_Read(uint8_t slv_addr, uint8_t reg);
uint8_t SCCB_Write(uint8_t slv_addr, uint8_t reg, uint8_t data);

View File

@@ -1,6 +1,8 @@
#pragma once
#include "camera_common.h"
#include "esp_system.h"
esp_err_t xclk_timer_conf(int ledc_timer, int xclk_freq_hz);
esp_err_t camera_enable_out_clock();

View File

@@ -11,6 +11,7 @@
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>
#include "sccb.h"
#include "sensor.h"
#include <stdio.h>
#include "sdkconfig.h"
#if defined(ARDUINO_ARCH_ESP32) && defined(CONFIG_ARDUHAL_ESP_LOG)
@@ -24,24 +25,22 @@ static const char* TAG = "sccb";
#include "driver/i2c.h"
#define SCCB_FREQ 100000 /*!< I2C master frequency*/
#define WRITE_BIT I2C_MASTER_WRITE /*!< I2C master write */
#define READ_BIT I2C_MASTER_READ /*!< I2C master read */
#define ACK_CHECK_EN 0x1 /*!< I2C master will check ack from slave*/
#define ACK_CHECK_DIS 0x0 /*!< I2C master will not check ack from slave */
#define ACK_VAL 0x0 /*!< I2C ack value */
#define NACK_VAL 0x1 /*!< I2C nack value */
#define SCCB_FREQ CONFIG_SCCB_CLK_FREQ /*!< I2C master frequency*/
#define WRITE_BIT I2C_MASTER_WRITE /*!< I2C master write */
#define READ_BIT I2C_MASTER_READ /*!< I2C master read */
#define ACK_CHECK_EN 0x1 /*!< I2C master will check ack from slave*/
#define ACK_CHECK_DIS 0x0 /*!< I2C master will not check ack from slave */
#define ACK_VAL 0x0 /*!< I2C ack value */
#define NACK_VAL 0x1 /*!< I2C nack value */
#if CONFIG_SCCB_HARDWARE_I2C_PORT1
const int SCCB_I2C_PORT = 1;
#else
const int SCCB_I2C_PORT = 0;
#endif
static uint8_t ESP_SLAVE_ADDR = 0x3c;
int SCCB_Init(int pin_sda, int pin_scl)
{
ESP_LOGI(TAG, "pin_sda %d pin_scl %d\n", pin_sda, pin_scl);
//log_i("SCCB_Init start");
ESP_LOGI(TAG, "pin_sda %d pin_scl %d", pin_sda, pin_scl);
i2c_config_t conf;
memset(&conf, 0, sizeof(i2c_config_t));
conf.mode = I2C_MODE_MASTER;
@@ -56,10 +55,30 @@ int SCCB_Init(int pin_sda, int pin_scl)
return 0;
}
uint8_t SCCB_Probe()
int SCCB_Deinit(void)
{
return i2c_driver_delete(SCCB_I2C_PORT);
}
uint8_t SCCB_Probe(void)
{
uint8_t slave_addr = 0x0;
while(slave_addr < 0x7f) {
// for (size_t i = 1; i < 0x80; i++) {
// i2c_cmd_handle_t cmd = i2c_cmd_link_create();
// i2c_master_start(cmd);
// i2c_master_write_byte(cmd, ( i << 1 ) | WRITE_BIT, ACK_CHECK_EN);
// i2c_master_stop(cmd);
// esp_err_t ret = i2c_master_cmd_begin(SCCB_I2C_PORT, cmd, 1000 / portTICK_RATE_MS);
// i2c_cmd_link_delete(cmd);
// if( ret == ESP_OK) {
// ESP_LOGW(TAG, "Found I2C Device at 0x%02X", i);
// }
// }
for (size_t i = 0; i < CAMERA_MODEL_MAX; i++) {
if (slave_addr == camera_sensor[i].sccb_addr) {
continue;
}
slave_addr = camera_sensor[i].sccb_addr;
i2c_cmd_handle_t cmd = i2c_cmd_link_create();
i2c_master_start(cmd);
i2c_master_write_byte(cmd, ( slave_addr << 1 ) | WRITE_BIT, ACK_CHECK_EN);
@@ -67,12 +86,10 @@ uint8_t SCCB_Probe()
esp_err_t ret = i2c_master_cmd_begin(SCCB_I2C_PORT, cmd, 1000 / portTICK_RATE_MS);
i2c_cmd_link_delete(cmd);
if( ret == ESP_OK) {
ESP_SLAVE_ADDR = slave_addr;
return ESP_SLAVE_ADDR;
return slave_addr;
}
slave_addr++;
}
return ESP_SLAVE_ADDR;
return 0;
}
uint8_t SCCB_Read(uint8_t slv_addr, uint8_t reg)

View File

@@ -1,5 +1,20 @@
#include <stdio.h>
#include "sensor.h"
const camera_sensor_info_t camera_sensor[CAMERA_MODEL_MAX] = {
// The sequence must be consistent with camera_model_t
{CAMERA_OV7725, "OV7725", OV7725_SCCB_ADDR, OV7725_PID, FRAMESIZE_VGA, false},
{CAMERA_OV2640, "OV2640", OV2640_SCCB_ADDR, OV2640_PID, FRAMESIZE_UXGA, true},
{CAMERA_OV3660, "OV3660", OV3660_SCCB_ADDR, OV3660_PID, FRAMESIZE_QXGA, true},
{CAMERA_OV5640, "OV5640", OV5640_SCCB_ADDR, OV5640_PID, FRAMESIZE_QSXGA, true},
{CAMERA_OV7670, "OV7670", OV7670_SCCB_ADDR, OV7670_PID, FRAMESIZE_VGA, false},
{CAMERA_NT99141, "NT99141", NT99141_SCCB_ADDR, NT99141_PID, FRAMESIZE_HD, true},
{CAMERA_GC2145, "GC2145", GC2145_SCCB_ADDR, GC2145_PID, FRAMESIZE_UXGA, false},
{CAMERA_GC032A, "GC032A", GC032A_SCCB_ADDR, GC032A_PID, FRAMESIZE_VGA, false},
{CAMERA_GC0308, "GC0308", GC0308_SCCB_ADDR, GC0308_PID, FRAMESIZE_VGA, false},
{CAMERA_BF3005, "BF3005", BF3005_SCCB_ADDR, BF3005_PID, FRAMESIZE_VGA, false},
};
const resolution_info_t resolution[FRAMESIZE_INVALID] = {
{ 96, 96, ASPECT_RATIO_1X1 }, /* 96x96 */
{ 160, 120, ASPECT_RATIO_4X3 }, /* QQVGA */
@@ -26,3 +41,13 @@ const resolution_info_t resolution[FRAMESIZE_INVALID] = {
{ 1088, 1920, ASPECT_RATIO_9X16 }, /* Portrait FHD */
{ 2560, 1920, ASPECT_RATIO_4X3 }, /* QSXGA */
};
camera_sensor_info_t *esp_camera_sensor_get_info(sensor_id_t *id)
{
for (int i = 0; i < CAMERA_MODEL_MAX; i++) {
if (id->PID == camera_sensor[i].pid) {
return (camera_sensor_info_t *)&camera_sensor[i];
}
}
return NULL;
}

View File

@@ -0,0 +1,9 @@
# The following lines of boilerplate have to be in your project's
# CMakeLists in this exact order for cmake to work correctly
cmake_minimum_required(VERSION 3.5)
set(EXTRA_COMPONENT_DIRS "../")
add_compile_options(-fdiagnostics-color=always)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(camera_example)

View File

@@ -0,0 +1,3 @@
set(COMPONENT_SRCS take_picture.c)
set(COMPONENT_ADD_INCLUDEDIRS .)
register_component()

View File

@@ -0,0 +1,5 @@
#
# "main" pseudo-component makefile.
#
# (Uses default behaviour of compiling all source files in directory, adding 'include' to include path.)

View File

@@ -29,7 +29,6 @@
// ================================ CODE ======================================
#include <esp_event.h>
#include <esp_log.h>
#include <esp_system.h>
#include <nvs_flash.h>
@@ -41,6 +40,8 @@
#include "esp_camera.h"
#define BOARD_WROVER_KIT 1
// WROVER-KIT PIN Map
#ifdef BOARD_WROVER_KIT
@@ -113,11 +114,12 @@ static camera_config_t camera_config = {
.ledc_timer = LEDC_TIMER_0,
.ledc_channel = LEDC_CHANNEL_0,
.pixel_format = PIXFORMAT_JPEG, //YUV422,GRAYSCALE,RGB565,JPEG
.frame_size = FRAMESIZE_VGA, //QQVGA-UXGA Do not use sizes above QVGA when not JPEG
.pixel_format = PIXFORMAT_RGB565, //YUV422,GRAYSCALE,RGB565,JPEG
.frame_size = FRAMESIZE_QVGA, //QQVGA-UXGA Do not use sizes above QVGA when not JPEG
.jpeg_quality = 12, //0-63 lower number means higher quality
.fb_count = 1 //if more than one, i2s runs in continuous mode. Use only with JPEG
.fb_count = 1, //if more than one, i2s runs in continuous mode. Use only with JPEG
.grab_mode = CAMERA_GRAB_WHEN_EMPTY,
};
static esp_err_t init_camera()
@@ -135,7 +137,9 @@ static esp_err_t init_camera()
void app_main()
{
init_camera();
if(ESP_OK != init_camera()) {
return;
}
while (1)
{
@@ -144,7 +148,8 @@ void app_main()
// use pic->buf to access the image
ESP_LOGI(TAG, "Picture taken! Its size was: %zu bytes", pic->len);
esp_camera_fb_return(pic);
vTaskDelay(5000 / portTICK_RATE_MS);
}
}
}

View File

@@ -0,0 +1,17 @@
CONFIG_ESP32_DEFAULT_CPU_FREQ_240=y
CONFIG_ESP32S2_DEFAULT_CPU_FREQ_240=y
CONFIG_ESP32S3_DEFAULT_CPU_FREQ_240=y
CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y
CONFIG_PARTITION_TABLE_OFFSET=0x10000
CONFIG_FREERTOS_HZ=1000
CONFIG_ESPTOOLPY_FLASHFREQ_80M=y
CONFIG_ESPTOOLPY_FLASHMODE_QIO=y
CONFIG_SPIRAM_SUPPORT=y
CONFIG_ESP32_SPIRAM_SUPPORT=y
CONFIG_ESP32S2_SPIRAM_SUPPORT=y
CONFIG_ESP32S3_SPIRAM_SUPPORT=y
CONFIG_SPIRAM_SPEED_80M=y

View File

@@ -1,3 +1,5 @@
version: "1.0.0"
description: This package hosts ESP32 compatible driver for OV2640 image sensors. Additionally it provides a few tools, which allow converting the captured frame data to the more common BMP and JPEG formats.
url: https://github.com/espressif/esp32-camera
description: ESP32 compatible driver for OV2640, OV3660, OV5640, OV7670 and OV7725 image sensors.
targets:
- esp32
- esp32s2
- esp32s3

View File

@@ -1,6 +1,6 @@
{
"name": "esp32-camera",
"version": "1.0.0",
"version": "2.0.0",
"keywords": "esp32, camera, espressif, esp32-cam",
"description": "ESP32 compatible driver for OV2640, OV3660, OV5640, OV7670 and OV7725 image sensors.",
"repository": {
@@ -16,6 +16,7 @@
"-Idriver/private_include",
"-Iconversions/private_include",
"-Isensors/private_include",
"-Itarget/private_include",
"-fno-rtti"
],
"includeDir": ".",

View File

@@ -0,0 +1,541 @@
/*
* This file is part of the OpenMV project.
* Copyright (c) 2013/2014 Ibrahim Abdelkader <i.abdalkader@gmail.com>
* This work is licensed under the MIT license, see the file LICENSE for details.
*
* BF3005 driver.
*
* Copyright 2015-2021 Espressif Systems (Shanghai) PTE LTD
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include "sccb.h"
#include "xclk.h"
#include "bf3005.h"
#include "bf3005_regs.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#if defined(ARDUINO_ARCH_ESP32) && defined(CONFIG_ARDUHAL_ESP_LOG)
#include "esp32-hal-log.h"
#else
#include "esp_log.h"
static const char* TAG = "bf3005";
#endif
static const uint8_t default_regs[][2] = {
{0x12, 0x40}, //soft reset
{0xff, 0xff}, //delay
{0xff, 0xff}, //delay
{0xff, 0xff}, //delay
{0xff, 0xff}, //delay
{0x13, 0x10},
{0x8c, 0x00},
{0x8d, 0x64},
{0x87, 0x10},
{0x13, 0x17},
{0x00, 0x20},
{0x01, 0x1a},
{0x02, 0x22},
{0x09, 0x03},
{0x0c, 0x80},
{0x0d, 0x24},
{0x0e, 0x21},
{0x0f, 0x28},
{0x11, 0x08},
{0x15, 0x10}, // 0X10
{0x16, 0x03},
{0x1e, 0x30},
{0x20, 0x8a},
{0x21, 0x03},
{0x23, 0x55},
{0x24, 0x68},
{0x25, 0x78},
{0x2a, 0x00},
{0x2b, 0x00},
{0x2d, 0x4f},
{0x2e, 0x98},
{0x2f, 0x04},
{0x30, 0xad},
{0x31, 0x17},
{0x32, 0x6e},
{0x33, 0x20},
{0x35, 0xa6},
{0x3b, 0x00},
{0x3e, 0x00},
{0x3f, 0xA8},
{0x40, 0x38},
{0x41, 0x32},
{0x42, 0x2b},
{0x43, 0x26},
{0x44, 0x1a},
{0x45, 0x16},
{0x46, 0x10},
{0x47, 0x0f},
{0x48, 0x0c},
{0x49, 0x0a},
{0x4b, 0x09},
{0x4c, 0x08},
{0x4d, 0x3c},
{0x4e, 0x06},
{0x4f, 0x05},
{0x50, 0x03},
{0x51, 0x25},
{0x52, 0x88},
{0x53, 0x03},
{0x63, 0x20},
{0x64, 0x02},
{0x65, 0xa6},
{0x66, 0xb6},
{0x69, 0x00},
{0x70, 0xFF},
{0x71, 0xa6},
{0x72, 0x2f},
{0x73, 0x2f},
{0x74, 0x2F},
{0x75, 0x0e},
{0x76, 0x1e},
{0x77, 0x00},
{0x78, 0x1e},
{0x79, 0x8a},
{0x7d, 0xe2},
{0x80, 0x44},
{0x81, 0x00},
{0x82, 0x18},
{0x83, 0x1b},
{0x84, 0x24},
{0x85, 0x2a},
{0x86, 0x4f},
{0x89, 0x82}, //0x82
{0x8b, 0x02},
{0x8e, 0x03},
{0x8f, 0xFC},
{0x9d, 0x4d},
{0x9e, 0x41},
{0xa1, 0x21},
{0xa2, 0x12},
{0xa3, 0x32},
{0xa4, 0x05},
{0xa5, 0x32},
{0xa6, 0x04},
{0xa7, 0x7f},
{0xa8, 0x7f},
{0xa9, 0x21},
{0xaa, 0x21},
{0xab, 0x21},
{0xac, 0x0a},
{0xad, 0xf0},
{0xae, 0xff},
{0xaf, 0x1d},
{0xb0, 0x94},
{0xb1, 0xc0},
{0xb2, 0xc0},
{0xd2, 0x30},
{0xe0, 0x0d},
{0xe1, 0x44},
{0xe7, 0x7c},
{0xe8, 0x89},
{0xe9, 0x01},
{0xea, 0x01},
{0xf0, 0x01},
{0xf3, 0x49},
{0xf4, 0xff},
{0xf5, 0x01},
{0xf6, 0xf2},
{0xf7, 0x6f},
{0x1b, 0x80},
{0x00, 0x00},
};
static int get_reg(sensor_t *sensor, int reg, int mask)
{
int ret = SCCB_Read(sensor->slv_addr, reg & 0xFF);
if(ret > 0){
ret &= mask;
}
return ret;
}
static int set_reg(sensor_t *sensor, int reg, int mask, int value)
{
int ret = 0;
ret = SCCB_Read(sensor->slv_addr, reg & 0xFF);
if(ret < 0){
return ret;
}
value = (ret & ~mask) | (value & mask);
ret = SCCB_Write(sensor->slv_addr, reg & 0xFF, value);
return ret;
}
static int set_reg_bits(sensor_t *sensor, uint8_t reg, uint8_t offset, uint8_t length, uint8_t value)
{
int ret = 0;
ret = SCCB_Read(sensor->slv_addr, reg);
if(ret < 0){
return ret;
}
uint8_t mask = ((1 << length) - 1) << offset;
value = (ret & ~mask) | ((value << offset) & mask);
ret = SCCB_Write(sensor->slv_addr, reg & 0xFF, value);
return ret;
}
static int get_reg_bits(sensor_t *sensor, uint8_t reg, uint8_t offset, uint8_t length)
{
int ret = 0;
ret = SCCB_Read(sensor->slv_addr, reg);
if(ret < 0){
return ret;
}
uint8_t mask = ((1 << length) - 1) << offset;
return (ret & mask) >> offset;
}
static int reset(sensor_t *sensor)
{
int i=0;
const uint8_t (*regs)[2];
// Write default regsiters
for (i=0, regs = default_regs; regs[i][0]; i++) {
SCCB_Write(sensor->slv_addr, regs[i][0], regs[i][1]);
}
// Delay
vTaskDelay(50 / portTICK_PERIOD_MS);
return 0;
}
static int set_pixformat(sensor_t *sensor, pixformat_t pixformat)
{
int ret=0;
sensor->pixformat = pixformat;
switch (pixformat) {
case PIXFORMAT_RGB565:
set_reg_bits(sensor, 0x12, 2, 1, 1);
break;
case PIXFORMAT_RAW:
set_reg_bits(sensor, 0x12, 0, 3, 0x4);
break;
case PIXFORMAT_YUV422:
case PIXFORMAT_GRAYSCALE:
set_reg_bits(sensor, 0x12, 2, 1, 0);
break;
default:
return -1;
}
// Delay
vTaskDelay(30 / portTICK_PERIOD_MS);
return ret;
}
static int set_framesize(sensor_t *sensor, framesize_t framesize)
{
int ret=0;
if (framesize > FRAMESIZE_VGA) {
return -1;
}
uint16_t w = resolution[framesize].width;
uint16_t h = resolution[framesize].height;
// uint8_t reg = SCCB_Read(sensor->slv_addr, COM7);
sensor->status.framesize = framesize;
// Write MSBs
ret |= SCCB_Write(sensor->slv_addr, 0x17, 0);
ret |= SCCB_Write(sensor->slv_addr, 0x18, w>>2);
ret |= SCCB_Write(sensor->slv_addr, 0x19, 0);
ret |= SCCB_Write(sensor->slv_addr, 0x1a, h>>2);
// Write LSBs
ret |= SCCB_Write(sensor->slv_addr, 0x03, 0);
printf("%s %d\r\n", __func__, __LINE__);
if((w<=320)&&(h<=240))
{
printf("%s %d\r\n", __func__, __LINE__);
// Enable auto-scaling/zooming factors
//ret |= SCCB_Write(sensor->slv_addr, 0x12, 0x50);
set_reg_bits(sensor, 0x12, 4, 1, 1);
ret |= SCCB_Write(sensor->slv_addr, 0x17, (80-w/4));
ret |= SCCB_Write(sensor->slv_addr, 0x18, (80+w/4));
ret |= SCCB_Write(sensor->slv_addr, 0x19, (60-h/4));
ret |= SCCB_Write(sensor->slv_addr, 0x1a, (60+h/4));
ret |= SCCB_Write(sensor->slv_addr, 0x03, 0);
} else if((w<=640)&&(h<=480))
{
// Enable auto-scaling/zooming factors
//ret |= SCCB_Write(sensor->slv_addr, 0x12, 0x40);
set_reg_bits(sensor, 0x12, 4, 1, 0);
ret |= SCCB_Write(sensor->slv_addr, 0x17, (80-w/8));
ret |= SCCB_Write(sensor->slv_addr, 0x18, (80+w/8));
ret |= SCCB_Write(sensor->slv_addr, 0x19, (60-h/8));
ret |= SCCB_Write(sensor->slv_addr, 0x1a, (60+h/8));
ret |= SCCB_Write(sensor->slv_addr, 0x03, 0);
}
// Delay
vTaskDelay(30 / portTICK_PERIOD_MS);
return ret;
}
static int set_colorbar(sensor_t *sensor, int value)
{
int ret=0;
sensor->status.colorbar = value;
ret |= SCCB_Write(sensor->slv_addr, 0xb9, value);
return ret;
}
static int set_whitebal(sensor_t *sensor, int enable)
{
if(set_reg_bits(sensor, 0x13, 1, 1, enable) >= 0){
sensor->status.awb = !!enable;
}
return sensor->status.awb;
}
static int set_gain_ctrl(sensor_t *sensor, int enable)
{
if(set_reg_bits(sensor, 0x13, 2, 1, enable) >= 0){
sensor->status.agc = !!enable;
}
return sensor->status.agc;
}
static int set_exposure_ctrl(sensor_t *sensor, int enable)
{
if(set_reg_bits(sensor, 0x13, 0, 1, enable) >= 0){
sensor->status.aec = !!enable;
}
return sensor->status.aec;
}
static int set_hmirror(sensor_t *sensor, int enable)
{
if(set_reg_bits(sensor, 0x1e, 5, 1, enable) >= 0){
sensor->status.hmirror = !!enable;
}
return sensor->status.hmirror;
}
static int set_vflip(sensor_t *sensor, int enable)
{
if(set_reg_bits(sensor, 0x1e, 4, 1, enable) >= 0){
sensor->status.vflip = !!enable;
}
return sensor->status.vflip;
}
static int set_raw_gma_dsp(sensor_t *sensor, int enable)
{
int ret = 0;
ret = set_reg_bits(sensor, 0xf1, 1, 1, !enable);
if (ret == 0) {
ESP_LOGD(TAG, "Set raw_gma to: %d", !enable);
sensor->status.raw_gma = !enable;
}
return ret;
}
static int set_lenc_dsp(sensor_t *sensor, int enable)
{
int ret = 0;
ret = set_reg_bits(sensor, 0xf1, 0, 1, !enable);
if (ret == 0) {
ESP_LOGD(TAG, "Set lenc to: %d", !enable);
sensor->status.lenc = !enable;
}
return ret;
}
static int set_agc_gain(sensor_t *sensor, int option)
{
int ret = 0;
ret = set_reg_bits(sensor, 0x13, 4, 1, !!option);
if (ret == 0) {
ESP_LOGD(TAG, "Set gain to: %d", !!option);
sensor->status.agc_gain = !!option;
}
return ret;
}
static int set_awb_gain_dsp(sensor_t *sensor, int value)
{
int ret = 0;
ret = SCCB_Write(sensor->slv_addr, 0xa6, value);
if (ret == 0) {
ESP_LOGD(TAG, "Set awb gain threthold to: %d", value);
sensor->status.awb_gain = value;
}
return ret;
}
static int set_brightness(sensor_t *sensor, int level)
{
int ret = 0;
ret = SCCB_Write(sensor->slv_addr, 0x55, level);
if (ret == 0) {
ESP_LOGD(TAG, "Set brightness to: %d", level);
sensor->status.brightness = level;
}
return ret;
}
static int set_contrast(sensor_t *sensor, int level)
{
int ret = 0;
ret = SCCB_Write(sensor->slv_addr, 0x56, level);
if (ret == 0) {
ESP_LOGD(TAG, "Set contrast to: %d", level);
sensor->status.contrast = level;
}
return ret;
}
static int set_sharpness(sensor_t *sensor, int level)
{
int ret = 0;
ret = SCCB_Write(sensor->slv_addr, 0x70, level);
if (ret == 0) {
ESP_LOGD(TAG, "Set sharpness to: %d", level);
sensor->status.sharpness = level;
}
return ret;
}
static int init_status(sensor_t *sensor)
{
sensor->status.brightness = SCCB_Read(sensor->slv_addr, 0x55);
sensor->status.contrast = SCCB_Read(sensor->slv_addr, 0x56);
sensor->status.saturation = 0;
sensor->status.ae_level = 0;
sensor->status.gainceiling = SCCB_Read(sensor->slv_addr, 0x87);
sensor->status.awb = get_reg_bits(sensor, 0x13, 1, 1);
sensor->status.awb_gain = SCCB_Read(sensor->slv_addr, 0xa6);
sensor->status.aec = get_reg_bits(sensor, 0x13, 0, 1);
sensor->status.agc = get_reg_bits(sensor, 0x13, 2, 1);
sensor->status.raw_gma = get_reg_bits(sensor, 0xf1, 1, 1);
sensor->status.lenc = get_reg_bits(sensor, 0xf1, 0, 1);
sensor->status.hmirror = get_reg_bits(sensor, 0x1e, 5, 1);
sensor->status.vflip = get_reg_bits(sensor, 0x1e, 4, 1);
sensor->status.colorbar = SCCB_Read(sensor->slv_addr, 0xb9);
sensor->status.sharpness = SCCB_Read(sensor->slv_addr, 0x70);
return 0;
}
static int set_dummy(sensor_t *sensor, int val){ return -1; }
static int set_gainceiling_dummy(sensor_t *sensor, gainceiling_t val){ return -1; }
static int set_res_raw(sensor_t *sensor, int startX, int startY, int endX, int endY, int offsetX, int offsetY, int totalX, int totalY, int outputX, int outputY, bool scale, bool binning){return -1;}
static int _set_pll(sensor_t *sensor, int bypass, int multiplier, int sys_div, int root_2x, int pre_div, int seld5, int pclk_manual, int pclk_div){return -1;}
static int set_xclk(sensor_t *sensor, int timer, int xclk)
{
int ret = 0;
sensor->xclk_freq_hz = xclk * 1000000U;
ret = xclk_timer_conf(timer, sensor->xclk_freq_hz);
return ret;
}
int bf3005_detect(int slv_addr, sensor_id_t *id)
{
if (BF3005_SCCB_ADDR == slv_addr) {
uint16_t PID = SCCB_Read(slv_addr, 0xFC);
if (BF3005_PID == PID) {
id->PID = PID;
id->VER = SCCB_Read(slv_addr, 0xFD);
id->MIDL = SCCB_Read(slv_addr, 0xFC);
id->MIDH = SCCB_Read(slv_addr, 0xFD);
return PID;
} else {
ESP_LOGI(TAG, "Mismatch PID=0x%x", PID);
}
}
return 0;
}
int bf3005_init(sensor_t *sensor)
{
// Set function pointers
sensor->reset = reset;
sensor->init_status = init_status;
sensor->set_pixformat = set_pixformat;
sensor->set_framesize = set_framesize;
sensor->set_brightness = set_brightness;
sensor->set_contrast = set_contrast;
sensor->set_colorbar = set_colorbar;
sensor->set_gain_ctrl = set_gain_ctrl;
sensor->set_exposure_ctrl = set_exposure_ctrl;
sensor->set_hmirror = set_hmirror;
sensor->set_vflip = set_vflip;
sensor->set_whitebal = set_whitebal;
sensor->set_awb_gain = set_awb_gain_dsp;
sensor->set_agc_gain = set_agc_gain;
sensor->set_raw_gma = set_raw_gma_dsp;
sensor->set_lenc = set_lenc_dsp;
sensor->set_sharpness = set_sharpness;
//not supported
sensor->set_saturation= set_dummy;
sensor->set_denoise = set_dummy;
sensor->set_quality = set_dummy;
sensor->set_special_effect = set_dummy;
sensor->set_wb_mode = set_dummy;
sensor->set_ae_level = set_dummy;
sensor->set_gainceiling = set_gainceiling_dummy;
sensor->get_reg = get_reg;
sensor->set_reg = set_reg;
sensor->set_res_raw = set_res_raw;
sensor->set_pll = _set_pll;
sensor->set_xclk = set_xclk;
ESP_LOGD(TAG, "BF3005 Attached");
return 0;
}

View File

@@ -0,0 +1,467 @@
// Copyright 2015-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "sccb.h"
#include "gc0308.h"
#include "gc0308_regs.h"
#include "gc0308_settings.h"
#if defined(ARDUINO_ARCH_ESP32) && defined(CONFIG_ARDUHAL_ESP_LOG)
#include "esp32-hal-log.h"
#else
#include "esp_log.h"
static const char *TAG = "gc0308";
#endif
#define H8(v) ((v)>>8)
#define L8(v) ((v)&0xff)
//#define REG_DEBUG_ON
static int read_reg(uint8_t slv_addr, const uint16_t reg)
{
int ret = SCCB_Read(slv_addr, reg);
#ifdef REG_DEBUG_ON
if (ret < 0) {
ESP_LOGE(TAG, "READ REG 0x%04x FAILED: %d", reg, ret);
}
#endif
return ret;
}
static int write_reg(uint8_t slv_addr, const uint16_t reg, uint8_t value)
{
int ret = 0;
#ifndef REG_DEBUG_ON
ret = SCCB_Write(slv_addr, reg, value);
#else
int old_value = read_reg(slv_addr, reg);
if (old_value < 0) {
return old_value;
}
if ((uint8_t)old_value != value) {
ESP_LOGI(TAG, "NEW REG 0x%04x: 0x%02x to 0x%02x", reg, (uint8_t)old_value, value);
ret = SCCB_Write(slv_addr, reg, value);
} else {
ESP_LOGD(TAG, "OLD REG 0x%04x: 0x%02x", reg, (uint8_t)old_value);
ret = SCCB_Write(slv_addr, reg, value);//maybe not?
}
if (ret < 0) {
ESP_LOGE(TAG, "WRITE REG 0x%04x FAILED: %d", reg, ret);
}
#endif
return ret;
}
static int check_reg_mask(uint8_t slv_addr, uint16_t reg, uint8_t mask)
{
return (read_reg(slv_addr, reg) & mask) == mask;
}
static int set_reg_bits(uint8_t slv_addr, uint16_t reg, uint8_t offset, uint8_t mask, uint8_t value)
{
int ret = 0;
uint8_t c_value, new_value;
ret = read_reg(slv_addr, reg);
if (ret < 0) {
return ret;
}
c_value = ret;
new_value = (c_value & ~(mask << offset)) | ((value & mask) << offset);
ret = write_reg(slv_addr, reg, new_value);
return ret;
}
static int write_regs(uint8_t slv_addr, const uint16_t (*regs)[2])
{
int i = 0, ret = 0;
while (!ret && regs[i][0] != REGLIST_TAIL) {
if (regs[i][0] == REG_DLY) {
vTaskDelay(regs[i][1] / portTICK_PERIOD_MS);
} else {
ret = write_reg(slv_addr, regs[i][0], regs[i][1]);
}
i++;
}
return ret;
}
static void print_regs(uint8_t slv_addr)
{
#ifdef DEBUG_PRINT_REG
ESP_LOGI(TAG, "REG list look ======================");
for (size_t i = 0xf0; i <= 0xfe; i++) {
ESP_LOGI(TAG, "reg[0x%02x] = 0x%02x", i, read_reg(slv_addr, i));
}
ESP_LOGI(TAG, "\npage 0 ===");
write_reg(slv_addr, 0xfe, 0x00); // page 0
for (size_t i = 0x03; i <= 0xa2; i++) {
ESP_LOGI(TAG, "p0 reg[0x%02x] = 0x%02x", i, read_reg(slv_addr, i));
}
ESP_LOGI(TAG, "\npage 3 ===");
write_reg(slv_addr, 0xfe, 0x03); // page 3
for (size_t i = 0x01; i <= 0x43; i++) {
ESP_LOGI(TAG, "p3 reg[0x%02x] = 0x%02x", i, read_reg(slv_addr, i));
}
#endif
}
static int reset(sensor_t *sensor)
{
int ret = 0;
// Software Reset: clear all registers and reset them to their default values
ret = write_reg(sensor->slv_addr, RESET_RELATED, 0xf0);
if (ret) {
ESP_LOGE(TAG, "Software Reset FAILED!");
return ret;
}
vTaskDelay(100 / portTICK_PERIOD_MS);
ret = write_regs(sensor->slv_addr, gc0308_sensor_default_regs);
if (ret == 0) {
ESP_LOGD(TAG, "Camera defaults loaded");
vTaskDelay(100 / portTICK_PERIOD_MS);
write_reg(sensor->slv_addr, 0xfe, 0x00);
#ifdef CONFIG_IDF_TARGET_ESP32
set_reg_bits(sensor->slv_addr, 0x28, 4, 0x07, 1); //frequency division for esp32, ensure pclk <= 15MHz
#endif
}
return ret;
}
static int set_pixformat(sensor_t *sensor, pixformat_t pixformat)
{
int ret = 0;
switch (pixformat) {
case PIXFORMAT_RGB565:
write_reg(sensor->slv_addr, 0xfe, 0x00);
ret = set_reg_bits(sensor->slv_addr, 0x24, 0, 0x0f, 6); //RGB565
break;
case PIXFORMAT_YUV422:
write_reg(sensor->slv_addr, 0xfe, 0x00);
ret = set_reg_bits(sensor->slv_addr, 0x24, 0, 0x0f, 2); //yuv422 Y Cb Y Cr
break;
default:
ESP_LOGW(TAG, "unsupport format");
ret = -1;
break;
}
if (ret == 0) {
sensor->pixformat = pixformat;
ESP_LOGD(TAG, "Set pixformat to: %u", pixformat);
}
return ret;
}
static int set_framesize(sensor_t *sensor, framesize_t framesize)
{
int ret = 0;
if (framesize > FRAMESIZE_VGA) {
ESP_LOGW(TAG, "Invalid framesize: %u", framesize);
framesize = FRAMESIZE_VGA;
}
sensor->status.framesize = framesize;
uint16_t w = resolution[framesize].width;
uint16_t h = resolution[framesize].height;
uint16_t row_s = (resolution[FRAMESIZE_VGA].height - h) / 2;
uint16_t col_s = (resolution[FRAMESIZE_VGA].width - w) / 2;
(void)row_s;
(void)col_s;
#if CONFIG_GC_SENSOR_SUBSAMPLE_MODE
struct subsample_cfg {
uint16_t ratio_numerator;
uint16_t ratio_denominator;
uint8_t reg0x54;
uint8_t reg0x56;
uint8_t reg0x57;
uint8_t reg0x58;
uint8_t reg0x59;
};
const struct subsample_cfg subsample_cfgs[] = { // define some subsample ratio
{84, 420, 0x55, 0x00, 0x00, 0x00, 0x00}, //1/5
{105, 420, 0x44, 0x00, 0x00, 0x00, 0x00},//1/4
{140, 420, 0x33, 0x00, 0x00, 0x00, 0x00},//1/3
{210, 420, 0x22, 0x00, 0x00, 0x00, 0x00},//1/2
{240, 420, 0x77, 0x02, 0x46, 0x02, 0x46},//4/7
{252, 420, 0x55, 0x02, 0x04, 0x02, 0x04},//3/5
{280, 420, 0x33, 0x02, 0x00, 0x02, 0x00},//2/3
{420, 420, 0x11, 0x00, 0x00, 0x00, 0x00},//1/1
};
uint16_t win_w = 640;
uint16_t win_h = 480;
const struct subsample_cfg *cfg = NULL;
/**
* Strategy: try to keep the maximum perspective
*/
for (size_t i = 0; i < sizeof(subsample_cfgs) / sizeof(struct subsample_cfg); i++) {
cfg = &subsample_cfgs[i];
if ((win_w * cfg->ratio_numerator / cfg->ratio_denominator >= w) && (win_h * cfg->ratio_numerator / cfg->ratio_denominator >= h)) {
win_w = w * cfg->ratio_denominator / cfg->ratio_numerator;
win_h = h * cfg->ratio_denominator / cfg->ratio_numerator;
row_s = (resolution[FRAMESIZE_VGA].height - win_h) / 2;
col_s = (resolution[FRAMESIZE_VGA].width - win_w) / 2;
ESP_LOGI(TAG, "subsample win:%dx%d, ratio:%f", win_w, win_h, (float)cfg->ratio_numerator / (float)cfg->ratio_denominator);
break;
}
}
write_reg(sensor->slv_addr, 0xfe, 0x00);
write_reg(sensor->slv_addr, 0x05, H8(row_s));
write_reg(sensor->slv_addr, 0x06, L8(row_s));
write_reg(sensor->slv_addr, 0x07, H8(col_s));
write_reg(sensor->slv_addr, 0x08, L8(col_s));
write_reg(sensor->slv_addr, 0x09, H8(win_h + 8));
write_reg(sensor->slv_addr, 0x0a, L8(win_h + 8));
write_reg(sensor->slv_addr, 0x0b, H8(win_w + 8));
write_reg(sensor->slv_addr, 0x0c, L8(win_w + 8));
write_reg(sensor->slv_addr, 0xfe, 0x01);
set_reg_bits(sensor->slv_addr, 0x53, 7, 0x01, 1);
set_reg_bits(sensor->slv_addr, 0x55, 0, 0x01, 1);
write_reg(sensor->slv_addr, 0x54, cfg->reg0x54);
write_reg(sensor->slv_addr, 0x56, cfg->reg0x56);
write_reg(sensor->slv_addr, 0x57, cfg->reg0x57);
write_reg(sensor->slv_addr, 0x58, cfg->reg0x58);
write_reg(sensor->slv_addr, 0x59, cfg->reg0x59);
write_reg(sensor->slv_addr, 0xfe, 0x00);
#elif CONFIG_GC_SENSOR_WINDOWING_MODE
write_reg(sensor->slv_addr, 0xfe, 0x00);
write_reg(sensor->slv_addr, 0xf7, col_s / 4);
write_reg(sensor->slv_addr, 0xf8, row_s / 4);
write_reg(sensor->slv_addr, 0xf9, (col_s + h) / 4);
write_reg(sensor->slv_addr, 0xfa, (row_s + w) / 4);
write_reg(sensor->slv_addr, 0x05, H8(row_s));
write_reg(sensor->slv_addr, 0x06, L8(row_s));
write_reg(sensor->slv_addr, 0x07, H8(col_s));
write_reg(sensor->slv_addr, 0x08, L8(col_s));
write_reg(sensor->slv_addr, 0x09, H8(h + 8));
write_reg(sensor->slv_addr, 0x0a, L8(h + 8));
write_reg(sensor->slv_addr, 0x0b, H8(w + 8));
write_reg(sensor->slv_addr, 0x0c, L8(w + 8));
#endif
if (ret == 0) {
ESP_LOGD(TAG, "Set framesize to: %ux%u", w, h);
}
return 0;
}
static int set_contrast(sensor_t *sensor, int contrast)
{
if (contrast != 0) {
write_reg(sensor->slv_addr, 0xfe, 0x00);
write_reg(sensor->slv_addr, 0xb3, contrast);
}
return 0;
}
static int set_global_gain(sensor_t *sensor, int gain_level)
{
if (gain_level != 0) {
write_reg(sensor->slv_addr, 0xfe, 0x00);
write_reg(sensor->slv_addr, 0x50, gain_level);
}
return 0;
}
static int set_hmirror(sensor_t *sensor, int enable)
{
int ret = 0;
sensor->status.hmirror = enable;
ret = write_reg(sensor->slv_addr, 0xfe, 0x00);
ret |= set_reg_bits(sensor->slv_addr, 0x14, 0, 0x01, enable != 0);
if (ret == 0) {
ESP_LOGD(TAG, "Set h-mirror to: %d", enable);
}
return ret;
}
static int set_vflip(sensor_t *sensor, int enable)
{
int ret = 0;
sensor->status.vflip = enable;
ret = write_reg(sensor->slv_addr, 0xfe, 0x00);
ret |= set_reg_bits(sensor->slv_addr, 0x14, 1, 0x01, enable != 0);
if (ret == 0) {
ESP_LOGD(TAG, "Set v-flip to: %d", enable);
}
return ret;
}
static int set_colorbar(sensor_t *sensor, int enable)
{
int ret = 0;
ret = write_reg(sensor->slv_addr, 0xfe, 0x00);
ret |= set_reg_bits(sensor->slv_addr, 0x2e, 0, 0x01, enable);
if (ret == 0) {
sensor->status.colorbar = enable;
ESP_LOGD(TAG, "Set colorbar to: %d", enable);
}
return ret;
}
static int get_reg(sensor_t *sensor, int reg, int mask)
{
int ret = 0;
if (mask > 0xFF) {
ESP_LOGE(TAG, "mask should not more than 0xff");
} else {
ret = read_reg(sensor->slv_addr, reg);
}
if (ret > 0) {
ret &= mask;
}
return ret;
}
static int set_reg(sensor_t *sensor, int reg, int mask, int value)
{
int ret = 0;
if (mask > 0xFF) {
ESP_LOGE(TAG, "mask should not more than 0xff");
} else {
ret = read_reg(sensor->slv_addr, reg);
}
if (ret < 0) {
return ret;
}
value = (ret & ~mask) | (value & mask);
if (mask > 0xFF) {
} else {
ret = write_reg(sensor->slv_addr, reg, value);
}
return ret;
}
static int init_status(sensor_t *sensor)
{
write_reg(sensor->slv_addr, 0xfe, 0x00);
sensor->status.brightness = 0;
sensor->status.contrast = 0;
sensor->status.saturation = 0;
sensor->status.sharpness = 0;
sensor->status.denoise = 0;
sensor->status.ae_level = 0;
sensor->status.gainceiling = 0;
sensor->status.awb = 0;
sensor->status.dcw = 0;
sensor->status.agc = 0;
sensor->status.aec = 0;
sensor->status.hmirror = check_reg_mask(sensor->slv_addr, 0x14, 0x01);
sensor->status.vflip = check_reg_mask(sensor->slv_addr, 0x14, 0x02);
sensor->status.colorbar = 0;
sensor->status.bpc = 0;
sensor->status.wpc = 0;
sensor->status.raw_gma = 0;
sensor->status.lenc = 0;
sensor->status.quality = 0;
sensor->status.special_effect = 0;
sensor->status.wb_mode = 0;
sensor->status.awb_gain = 0;
sensor->status.agc_gain = 0;
sensor->status.aec_value = 0;
sensor->status.aec2 = 0;
print_regs(sensor->slv_addr);
return 0;
}
static int set_dummy(sensor_t *sensor, int val)
{
ESP_LOGW(TAG, "Unsupported");
return -1;
}
static int set_gainceiling_dummy(sensor_t *sensor, gainceiling_t val)
{
ESP_LOGW(TAG, "Unsupported");
return -1;
}
int gc0308_detect(int slv_addr, sensor_id_t *id)
{
if (GC0308_SCCB_ADDR == slv_addr) {
write_reg(slv_addr, 0xfe, 0x00);
uint8_t PID = SCCB_Read(slv_addr, 0x00);
if (GC0308_PID == PID) {
id->PID = PID;
return PID;
} else {
ESP_LOGI(TAG, "Mismatch PID=0x%x", PID);
}
}
return 0;
}
int gc0308_init(sensor_t *sensor)
{
sensor->init_status = init_status;
sensor->reset = reset;
sensor->set_pixformat = set_pixformat;
sensor->set_framesize = set_framesize;
sensor->set_contrast = set_contrast;
sensor->set_brightness = set_dummy;
sensor->set_saturation = set_dummy;
sensor->set_sharpness = set_dummy;
sensor->set_denoise = set_dummy;
sensor->set_gainceiling = set_gainceiling_dummy;
sensor->set_quality = set_dummy;
sensor->set_colorbar = set_colorbar;
sensor->set_whitebal = set_dummy;
sensor->set_gain_ctrl = set_global_gain;
sensor->set_exposure_ctrl = set_dummy;
sensor->set_hmirror = set_hmirror;
sensor->set_vflip = set_vflip;
sensor->set_aec2 = set_dummy;
sensor->set_awb_gain = set_dummy;
sensor->set_agc_gain = set_dummy;
sensor->set_aec_value = set_dummy;
sensor->set_special_effect = set_dummy;
sensor->set_wb_mode = set_dummy;
sensor->set_ae_level = set_dummy;
sensor->set_dcw = set_dummy;
sensor->set_bpc = set_dummy;
sensor->set_wpc = set_dummy;
sensor->set_raw_gma = set_dummy;
sensor->set_lenc = set_dummy;
sensor->get_reg = get_reg;
sensor->set_reg = set_reg;
sensor->set_res_raw = NULL;
sensor->set_pll = NULL;
sensor->set_xclk = NULL;
ESP_LOGD(TAG, "GC0308 Attached");
return 0;
}

View File

@@ -0,0 +1,391 @@
// Copyright 2015-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "sccb.h"
#include "gc032a.h"
#include "gc032a_regs.h"
#include "gc032a_settings.h"
#if defined(ARDUINO_ARCH_ESP32) && defined(CONFIG_ARDUHAL_ESP_LOG)
#include "esp32-hal-log.h"
#else
#include "esp_log.h"
static const char *TAG = "gc032a";
#endif
#define H8(v) ((v)>>8)
#define L8(v) ((v)&0xff)
//#define REG_DEBUG_ON
static int read_reg(uint8_t slv_addr, const uint16_t reg)
{
int ret = SCCB_Read(slv_addr, reg);
#ifdef REG_DEBUG_ON
if (ret < 0) {
ESP_LOGE(TAG, "READ REG 0x%04x FAILED: %d", reg, ret);
}
#endif
return ret;
}
static int write_reg(uint8_t slv_addr, const uint16_t reg, uint8_t value)
{
int ret = 0;
#ifndef REG_DEBUG_ON
ret = SCCB_Write(slv_addr, reg, value);
#else
int old_value = read_reg(slv_addr, reg);
if (old_value < 0) {
return old_value;
}
if ((uint8_t)old_value != value) {
ESP_LOGI(TAG, "NEW REG 0x%04x: 0x%02x to 0x%02x", reg, (uint8_t)old_value, value);
ret = SCCB_Write(slv_addr, reg, value);
} else {
ESP_LOGD(TAG, "OLD REG 0x%04x: 0x%02x", reg, (uint8_t)old_value);
ret = SCCB_Write(slv_addr, reg, value);//maybe not?
}
if (ret < 0) {
ESP_LOGE(TAG, "WRITE REG 0x%04x FAILED: %d", reg, ret);
}
#endif
return ret;
}
static int check_reg_mask(uint8_t slv_addr, uint16_t reg, uint8_t mask)
{
return (read_reg(slv_addr, reg) & mask) == mask;
}
static void print_regs(uint8_t slv_addr)
{
#ifdef DEBUG_PRINT_REG
vTaskDelay(pdMS_TO_TICKS(100));
ESP_LOGI(TAG, "REG list look ======================");
for (size_t i = 0xf0; i <= 0xfe; i++) {
ESP_LOGI(TAG, "reg[0x%02x] = 0x%02x", i, read_reg(slv_addr, i));
}
ESP_LOGI(TAG, "\npage 0 ===");
write_reg(slv_addr, 0xfe, 0x00); // page 0
for (size_t i = 0x03; i <= 0x24; i++) {
ESP_LOGI(TAG, "p0 reg[0x%02x] = 0x%02x", i, read_reg(slv_addr, i));
}
for (size_t i = 0x40; i <= 0x95; i++) {
ESP_LOGI(TAG, "p0 reg[0x%02x] = 0x%02x", i, read_reg(slv_addr, i));
}
ESP_LOGI(TAG, "\npage 3 ===");
write_reg(slv_addr, 0xfe, 0x03); // page 3
for (size_t i = 0x01; i <= 0x43; i++) {
ESP_LOGI(TAG, "p3 reg[0x%02x] = 0x%02x", i, read_reg(slv_addr, i));
}
#endif
}
static int set_reg_bits(uint8_t slv_addr, uint16_t reg, uint8_t offset, uint8_t mask, uint8_t value)
{
int ret = 0;
uint8_t c_value, new_value;
ret = read_reg(slv_addr, reg);
if (ret < 0) {
return ret;
}
c_value = ret;
new_value = (c_value & ~(mask << offset)) | ((value & mask) << offset);
ret = write_reg(slv_addr, reg, new_value);
return ret;
}
static int write_regs(uint8_t slv_addr, const uint16_t (*regs)[2])
{
int i = 0, ret = 0;
while (!ret && regs[i][0] != REGLIST_TAIL) {
if (regs[i][0] == REG_DLY) {
vTaskDelay(regs[i][1] / portTICK_PERIOD_MS);
} else {
ret = write_reg(slv_addr, regs[i][0], regs[i][1]);
}
i++;
}
return ret;
}
static int reset(sensor_t *sensor)
{
int ret;
// Software Reset: clear all registers and reset them to their default values
ret = write_reg(sensor->slv_addr, RESET_RELATED, 0xf0);
if (ret) {
ESP_LOGE(TAG, "Software Reset FAILED!");
return ret;
}
vTaskDelay(100 / portTICK_PERIOD_MS);
ret = write_regs(sensor->slv_addr, gc032a_default_regs);
if (ret == 0) {
ESP_LOGD(TAG, "Camera defaults loaded");
vTaskDelay(100 / portTICK_PERIOD_MS);
write_reg(sensor->slv_addr, 0xfe, 0x00);
set_reg_bits(sensor->slv_addr, 0xf7, 1, 0x01, 1); // PLL_mode1:div2en
set_reg_bits(sensor->slv_addr, 0xf7, 7, 0x01, 1); // PLL_mode1:dvp mode
set_reg_bits(sensor->slv_addr, 0xf8, 0, 0x3f, 8); //PLL_mode2 :divx4
set_reg_bits(sensor->slv_addr, 0xfa, 4, 0x0f, 2); //vlk div mode :divide_by
}
return ret;
}
static int set_pixformat(sensor_t *sensor, pixformat_t pixformat)
{
int ret = 0;
switch (pixformat) {
case PIXFORMAT_RGB565:
write_reg(sensor->slv_addr, 0xfe, 0x00);
ret = set_reg_bits(sensor->slv_addr, 0x44, 0, 0x1f, 6); //RGB565
break;
case PIXFORMAT_YUV422:
write_reg(sensor->slv_addr, 0xfe, 0x00);
ret = set_reg_bits(sensor->slv_addr, 0x44, 0, 0x1f, 3);
break;
default:
ESP_LOGW(TAG, "unsupport format");
ret = -1;
break;
}
if (ret == 0) {
sensor->pixformat = pixformat;
ESP_LOGD(TAG, "Set pixformat to: %u", pixformat);
}
return ret;
}
static int set_framesize(sensor_t *sensor, framesize_t framesize)
{
ESP_LOGI(TAG, "set_framesize");
int ret = 0;
if (framesize > FRAMESIZE_VGA) {
ESP_LOGW(TAG, "Invalid framesize: %u", framesize);
framesize = FRAMESIZE_VGA;
}
sensor->status.framesize = framesize;
uint16_t w = resolution[framesize].width;
uint16_t h = resolution[framesize].height;
uint16_t row_s = (resolution[FRAMESIZE_VGA].height - h) / 2;
uint16_t col_s = (resolution[FRAMESIZE_VGA].width - w) / 2;
write_reg(sensor->slv_addr, 0xfe, 0x00);
write_reg(sensor->slv_addr, P0_ROW_START_HIGH, H8(row_s)); // Row_start[8]
write_reg(sensor->slv_addr, P0_ROW_START_LOW, L8(row_s)); // Row_start[7:0]
write_reg(sensor->slv_addr, P0_COLUMN_START_HIGH, H8(col_s)); // Column_start[9:8]
write_reg(sensor->slv_addr, P0_COLUMN_START_LOW, L8(col_s)); // Column_start[7:0]
write_reg(sensor->slv_addr, P0_WINDOW_HEIGHT_HIGH, H8(h + 8)); //window_height [8]
write_reg(sensor->slv_addr, P0_WINDOW_HEIGHT_LOW, L8(h + 8)); //window_height [7:0]
write_reg(sensor->slv_addr, P0_WINDOW_WIDTH_HIGH, H8(w + 8)); //window_width [9:8]
write_reg(sensor->slv_addr, P0_WINDOW_WIDTH_LOW, L8(w + 8)); //window_width [7:0]
write_reg(sensor->slv_addr, P0_WIN_MODE, 0x01);
write_reg(sensor->slv_addr, P0_OUT_WIN_HEIGHT_HIGH, H8(h));
write_reg(sensor->slv_addr, P0_OUT_WIN_HEIGHT_LOW, L8(h));
write_reg(sensor->slv_addr, P0_OUT_WIN_WIDTH_HIGH, H8(w));
write_reg(sensor->slv_addr, P0_OUT_WIN_WIDTH_LOW, L8(w));
if (ret == 0) {
ESP_LOGD(TAG, "Set framesize to: %ux%u", w, h);
}
print_regs(sensor->slv_addr);
return ret;
}
static int set_hmirror(sensor_t *sensor, int enable)
{
int ret = 0;
sensor->status.hmirror = enable;
ret = write_reg(sensor->slv_addr, 0xfe, 0x00);
ret |= set_reg_bits(sensor->slv_addr, P0_CISCTL_MODE1, 0, 0x01, enable);
if (ret == 0) {
ESP_LOGD(TAG, "Set h-mirror to: %d", enable);
}
return ret;
}
static int set_vflip(sensor_t *sensor, int enable)
{
int ret = 0;
sensor->status.vflip = enable;
ret = write_reg(sensor->slv_addr, 0xfe, 0x00);
ret |= set_reg_bits(sensor->slv_addr, P0_CISCTL_MODE1, 1, 0x01, enable);
if (ret == 0) {
ESP_LOGD(TAG, "Set v-flip to: %d", enable);
}
return ret;
}
static int set_colorbar(sensor_t *sensor, int enable)
{
int ret = 0;
ret = write_reg(sensor->slv_addr, 0xfe, 0x00);
ret |= set_reg_bits(sensor->slv_addr, P0_DEBUG_MODE2, 3, 0x01, enable);
if (ret == 0) {
sensor->status.colorbar = enable;
ESP_LOGD(TAG, "Set colorbar to: %d", enable);
}
return ret;
}
static int get_reg(sensor_t *sensor, int reg, int mask)
{
int ret = 0;
if (mask > 0xFF) {
ESP_LOGE(TAG, "mask should not more than 0xff");
} else {
ret = read_reg(sensor->slv_addr, reg);
}
if (ret > 0) {
ret &= mask;
}
return ret;
}
static int set_reg(sensor_t *sensor, int reg, int mask, int value)
{
int ret = 0;
if (mask > 0xFF) {
ESP_LOGE(TAG, "mask should not more than 0xff");
} else {
ret = read_reg(sensor->slv_addr, reg);
}
if (ret < 0) {
return ret;
}
value = (ret & ~mask) | (value & mask);
if (mask > 0xFF) {
} else {
ret = write_reg(sensor->slv_addr, reg, value);
}
return ret;
}
static int init_status(sensor_t *sensor)
{
write_reg(sensor->slv_addr, 0xfe, 0x00);
sensor->status.brightness = 0;
sensor->status.contrast = 0;
sensor->status.saturation = 0;
sensor->status.sharpness = 0;
sensor->status.denoise = 0;
sensor->status.ae_level = 0;
sensor->status.gainceiling = 0;
sensor->status.awb = 0;
sensor->status.dcw = 0;
sensor->status.agc = 0;
sensor->status.aec = 0;
sensor->status.hmirror = check_reg_mask(sensor->slv_addr, P0_CISCTL_MODE1, 0x01);
sensor->status.vflip = check_reg_mask(sensor->slv_addr, P0_CISCTL_MODE1, 0x02);
sensor->status.colorbar = 0;
sensor->status.bpc = 0;
sensor->status.wpc = 0;
sensor->status.raw_gma = 0;
sensor->status.lenc = 0;
sensor->status.quality = 0;
sensor->status.special_effect = 0;
sensor->status.wb_mode = 0;
sensor->status.awb_gain = 0;
sensor->status.agc_gain = 0;
sensor->status.aec_value = 0;
sensor->status.aec2 = 0;
return 0;
}
static int set_dummy(sensor_t *sensor, int val)
{
ESP_LOGW(TAG, "Unsupported");
return -1;
}
static int set_gainceiling_dummy(sensor_t *sensor, gainceiling_t val)
{
ESP_LOGW(TAG, "Unsupported");
return -1;
}
int gc032a_detect(int slv_addr, sensor_id_t *id)
{
if (GC032A_SCCB_ADDR == slv_addr) {
uint8_t MIDL = SCCB_Read(slv_addr, SENSOR_ID_LOW);
uint8_t MIDH = SCCB_Read(slv_addr, SENSOR_ID_HIGH);
uint16_t PID = MIDH << 8 | MIDL;
if (GC032A_PID == PID) {
id->PID = PID;
return PID;
} else {
ESP_LOGI(TAG, "Mismatch PID=0x%x", PID);
}
}
return 0;
}
int gc032a_init(sensor_t *sensor)
{
sensor->init_status = init_status;
sensor->reset = reset;
sensor->set_pixformat = set_pixformat;
sensor->set_framesize = set_framesize;
sensor->set_contrast = set_dummy;
sensor->set_brightness = set_dummy;
sensor->set_saturation = set_dummy;
sensor->set_sharpness = set_dummy;
sensor->set_denoise = set_dummy;
sensor->set_gainceiling = set_gainceiling_dummy;
sensor->set_quality = set_dummy;
sensor->set_colorbar = set_colorbar;
sensor->set_whitebal = set_dummy;
sensor->set_gain_ctrl = set_dummy;
sensor->set_exposure_ctrl = set_dummy;
sensor->set_hmirror = set_hmirror;
sensor->set_vflip = set_vflip;
sensor->set_aec2 = set_dummy;
sensor->set_awb_gain = set_dummy;
sensor->set_agc_gain = set_dummy;
sensor->set_aec_value = set_dummy;
sensor->set_special_effect = set_dummy;
sensor->set_wb_mode = set_dummy;
sensor->set_ae_level = set_dummy;
sensor->set_dcw = set_dummy;
sensor->set_bpc = set_dummy;
sensor->set_wpc = set_dummy;
sensor->set_raw_gma = set_dummy;
sensor->set_lenc = set_dummy;
sensor->get_reg = get_reg;
sensor->set_reg = set_reg;
sensor->set_res_raw = NULL;
sensor->set_pll = NULL;
sensor->set_xclk = NULL;
ESP_LOGD(TAG, "GC032A Attached");
return 0;
}

View File

@@ -0,0 +1,477 @@
// Copyright 2015-2021 Espressif Systems (Shanghai) PTE LTD
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "sccb.h"
#include "gc2145.h"
#include "gc2145_regs.h"
#include "gc2145_settings.h"
#if defined(ARDUINO_ARCH_ESP32) && defined(CONFIG_ARDUHAL_ESP_LOG)
#include "esp32-hal-log.h"
#else
#include "esp_log.h"
static const char *TAG = "gc2145";
#endif
#define H8(v) ((v)>>8)
#define L8(v) ((v)&0xff)
//#define REG_DEBUG_ON
static int read_reg(uint8_t slv_addr, const uint16_t reg)
{
int ret = SCCB_Read(slv_addr, reg);
#ifdef REG_DEBUG_ON
if (ret < 0) {
ESP_LOGE(TAG, "READ REG 0x%04x FAILED: %d", reg, ret);
}
#endif
return ret;
}
static int write_reg(uint8_t slv_addr, const uint16_t reg, uint8_t value)
{
int ret = 0;
#ifndef REG_DEBUG_ON
ret = SCCB_Write(slv_addr, reg, value);
#else
int old_value = read_reg(slv_addr, reg);
if (old_value < 0) {
return old_value;
}
if ((uint8_t)old_value != value) {
ESP_LOGI(TAG, "NEW REG 0x%04x: 0x%02x to 0x%02x", reg, (uint8_t)old_value, value);
ret = SCCB_Write(slv_addr, reg, value);
} else {
ESP_LOGD(TAG, "OLD REG 0x%04x: 0x%02x", reg, (uint8_t)old_value);
ret = SCCB_Write(slv_addr, reg, value);//maybe not?
}
if (ret < 0) {
ESP_LOGE(TAG, "WRITE REG 0x%04x FAILED: %d", reg, ret);
}
#endif
return ret;
}
static int check_reg_mask(uint8_t slv_addr, uint16_t reg, uint8_t mask)
{
return (read_reg(slv_addr, reg) & mask) == mask;
}
static int set_reg_bits(uint8_t slv_addr, uint16_t reg, uint8_t offset, uint8_t mask, uint8_t value)
{
int ret = 0;
uint8_t c_value, new_value;
ret = read_reg(slv_addr, reg);
if (ret < 0) {
return ret;
}
c_value = ret;
new_value = (c_value & ~(mask << offset)) | ((value & mask) << offset);
ret = write_reg(slv_addr, reg, new_value);
return ret;
}
static int write_regs(uint8_t slv_addr, const uint16_t (*regs)[2])
{
int i = 0, ret = 0;
while (!ret && regs[i][0] != REGLIST_TAIL) {
if (regs[i][0] == REG_DLY) {
vTaskDelay(regs[i][1] / portTICK_PERIOD_MS);
} else {
ret = write_reg(slv_addr, regs[i][0], regs[i][1]);
}
i++;
}
return ret;
}
static void print_regs(uint8_t slv_addr)
{
#ifdef DEBUG_PRINT_REG
vTaskDelay(pdMS_TO_TICKS(100));
ESP_LOGI(TAG, "REG list look ======================");
for (size_t i = 0xf0; i <= 0xfe; i++) {
ESP_LOGI(TAG, "reg[0x%02x] = 0x%02x", i, read_reg(slv_addr, i));
}
ESP_LOGI(TAG, "\npage 0 ===");
write_reg(slv_addr, 0xfe, 0x00); // page 0
for (size_t i = 0x03; i <= 0x24; i++) {
ESP_LOGI(TAG, "p0 reg[0x%02x] = 0x%02x", i, read_reg(slv_addr, i));
}
for (size_t i = 0x80; i <= 0xa2; i++) {
ESP_LOGI(TAG, "p0 reg[0x%02x] = 0x%02x", i, read_reg(slv_addr, i));
}
ESP_LOGI(TAG, "\npage 3 ===");
write_reg(slv_addr, 0xfe, 0x03); // page 3
for (size_t i = 0x01; i <= 0x43; i++) {
ESP_LOGI(TAG, "p3 reg[0x%02x] = 0x%02x", i, read_reg(slv_addr, i));
}
#endif
}
static int reset(sensor_t *sensor)
{
int ret = 0;
// Software Reset: clear all registers and reset them to their default values
ret = write_reg(sensor->slv_addr, RESET_RELATED, 0xe0);
if (ret) {
ESP_LOGE(TAG, "Software Reset FAILED!");
return ret;
}
vTaskDelay(100 / portTICK_PERIOD_MS);
ret = write_regs(sensor->slv_addr, gc2145_default_init_regs);
if (ret == 0) {
ESP_LOGD(TAG, "Camera defaults loaded");
vTaskDelay(100 / portTICK_PERIOD_MS);
#ifdef CONFIG_IDF_TARGET_ESP32
write_reg(sensor->slv_addr, 0xfe, 0x00);
//ensure pclk <= 15MHz for esp32
set_reg_bits(sensor->slv_addr, 0xf8, 0, 0x3f, 2); // divx4
set_reg_bits(sensor->slv_addr, 0xfa, 4, 0x0f, 2); // divide_by
#endif
}
return ret;
}
static int set_pixformat(sensor_t *sensor, pixformat_t pixformat)
{
int ret = 0;
switch (pixformat) {
case PIXFORMAT_RGB565:
write_reg(sensor->slv_addr, 0xfe, 0x00);
ret = set_reg_bits(sensor->slv_addr, P0_OUTPUT_FORMAT, 0, 0x1f, 6); //RGB565
break;
case PIXFORMAT_YUV422:
write_reg(sensor->slv_addr, 0xfe, 0x00);
ret = set_reg_bits(sensor->slv_addr, P0_OUTPUT_FORMAT, 0, 0x1f, 2); //yuv422
break;
default:
ESP_LOGW(TAG, "unsupport format");
ret = -1;
break;
}
if (ret == 0) {
sensor->pixformat = pixformat;
ESP_LOGD(TAG, "Set pixformat to: %u", pixformat);
}
return ret;
}
static int set_framesize(sensor_t *sensor, framesize_t framesize)
{
int ret = 0;
if (framesize > FRAMESIZE_UXGA) {
ESP_LOGW(TAG, "Invalid framesize: %u", framesize);
framesize = FRAMESIZE_UXGA;
}
sensor->status.framesize = framesize;
uint16_t w = resolution[framesize].width;
uint16_t h = resolution[framesize].height;
uint16_t row_s = (resolution[FRAMESIZE_UXGA].height - h) / 2;
uint16_t col_s = (resolution[FRAMESIZE_UXGA].width - w) / 2;
(void)row_s;
(void)col_s;
#if CONFIG_GC_SENSOR_SUBSAMPLE_MODE
struct subsample_cfg {
uint16_t ratio_numerator;
uint16_t ratio_denominator;
uint8_t reg0x99;
uint8_t reg0x9b;
uint8_t reg0x9c;
uint8_t reg0x9d;
uint8_t reg0x9e;
uint8_t reg0x9f;
uint8_t reg0xa0;
uint8_t reg0xa1;
uint8_t reg0xa2;
};
const struct subsample_cfg subsample_cfgs[] = { // define some subsample ratio
// {60, 420, 0x77, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}, //1/7 // A smaller ratio brings a larger view, but it reduces the frame rate
// {84, 420, 0x55, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}, //1/5
// {105, 420, 0x44, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00},//1/4
{140, 420, 0x33, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00},//1/3
{210, 420, 0x22, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00},//1/2
{240, 420, 0x77, 0x02, 0x46, 0x02, 0x46, 0x02, 0x46, 0x02, 0x46},//4/7
{252, 420, 0x55, 0x02, 0x04, 0x02, 0x04, 0x02, 0x04, 0x02, 0x04},//3/5
{280, 420, 0x33, 0x00, 0x02, 0x00, 0x00, 0x00, 0x02, 0x00, 0x00},//2/3
{420, 420, 0x11, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00},//1/1
};
uint16_t win_w = resolution[FRAMESIZE_UXGA].width;
uint16_t win_h = resolution[FRAMESIZE_UXGA].height;
const struct subsample_cfg *cfg = NULL;
/**
* Strategy: try to keep the maximum perspective
*/
uint8_t i = 0;
if (framesize >= FRAMESIZE_QVGA) {
i = 1;
}
for (; i < sizeof(subsample_cfgs) / sizeof(struct subsample_cfg); i++) {
cfg = &subsample_cfgs[i];
if ((win_w * cfg->ratio_numerator / cfg->ratio_denominator >= w) && (win_h * cfg->ratio_numerator / cfg->ratio_denominator >= h)) {
win_w = w * cfg->ratio_denominator / cfg->ratio_numerator;
win_h = h * cfg->ratio_denominator / cfg->ratio_numerator;
row_s = (resolution[FRAMESIZE_UXGA].height - win_h) / 2;
col_s = (resolution[FRAMESIZE_UXGA].width - win_w) / 2;
ESP_LOGI(TAG, "subsample win:%dx%d, ratio:%f", win_w, win_h, (float)cfg->ratio_numerator / (float)cfg->ratio_denominator);
break;
}
}
write_reg(sensor->slv_addr, 0xfe, 0x00);
write_reg(sensor->slv_addr, P0_CROP_ENABLE, 0x01);
write_reg(sensor->slv_addr, 0x09, H8(row_s));
write_reg(sensor->slv_addr, 0x0a, L8(row_s));
write_reg(sensor->slv_addr, 0x0b, H8(col_s));
write_reg(sensor->slv_addr, 0x0c, L8(col_s));
write_reg(sensor->slv_addr, 0x0d, H8(win_h + 8));
write_reg(sensor->slv_addr, 0x0e, L8(win_h + 8));
write_reg(sensor->slv_addr, 0x0f, H8(win_w + 16));
write_reg(sensor->slv_addr, 0x10, L8(win_w + 16));
write_reg(sensor->slv_addr, 0x99, cfg->reg0x99);
write_reg(sensor->slv_addr, 0x9b, cfg->reg0x9b);
write_reg(sensor->slv_addr, 0x9c, cfg->reg0x9c);
write_reg(sensor->slv_addr, 0x9d, cfg->reg0x9d);
write_reg(sensor->slv_addr, 0x9e, cfg->reg0x9e);
write_reg(sensor->slv_addr, 0x9f, cfg->reg0x9f);
write_reg(sensor->slv_addr, 0xa0, cfg->reg0xa0);
write_reg(sensor->slv_addr, 0xa1, cfg->reg0xa1);
write_reg(sensor->slv_addr, 0xa2, cfg->reg0xa2);
write_reg(sensor->slv_addr, 0x95, H8(h));
write_reg(sensor->slv_addr, 0x96, L8(h));
write_reg(sensor->slv_addr, 0x97, H8(w));
write_reg(sensor->slv_addr, 0x98, L8(w));
#elif CONFIG_GC_SENSOR_WINDOWING_MODE
write_reg(sensor->slv_addr, 0xfe, 0x00);
write_reg(sensor->slv_addr, P0_CROP_ENABLE, 0x01);
// write_reg(sensor->slv_addr, 0xec, col_s / 8); //measure window
// write_reg(sensor->slv_addr, 0xed, row_s / 8);
// write_reg(sensor->slv_addr, 0xee, (col_s + h) / 8);
// write_reg(sensor->slv_addr, 0xef, (row_s + w) / 8);
write_reg(sensor->slv_addr, 0x09, H8(row_s));
write_reg(sensor->slv_addr, 0x0a, L8(row_s));
write_reg(sensor->slv_addr, 0x0b, H8(col_s));
write_reg(sensor->slv_addr, 0x0c, L8(col_s));
write_reg(sensor->slv_addr, 0x0d, H8(h + 8));
write_reg(sensor->slv_addr, 0x0e, L8(h + 8));
write_reg(sensor->slv_addr, 0x0f, H8(w + 8));
write_reg(sensor->slv_addr, 0x10, L8(w + 8));
write_reg(sensor->slv_addr, 0x95, H8(h));
write_reg(sensor->slv_addr, 0x96, L8(h));
write_reg(sensor->slv_addr, 0x97, H8(w));
write_reg(sensor->slv_addr, 0x98, L8(w));
#endif
if (ret == 0) {
ESP_LOGD(TAG, "Set framesize to: %ux%u", w, h);
}
return ret;
}
static int set_hmirror(sensor_t *sensor, int enable)
{
int ret = 0;
sensor->status.hmirror = enable;
ret = write_reg(sensor->slv_addr, 0xfe, 0x00);
ret |= set_reg_bits(sensor->slv_addr, P0_ANALOG_MODE1, 0, 0x01, enable != 0);
if (ret == 0) {
ESP_LOGD(TAG, "Set h-mirror to: %d", enable);
}
return ret;
}
static int set_vflip(sensor_t *sensor, int enable)
{
int ret = 0;
sensor->status.vflip = enable;
ret = write_reg(sensor->slv_addr, 0xfe, 0x00);
ret |= set_reg_bits(sensor->slv_addr, P0_ANALOG_MODE1, 1, 0x01, enable != 0);
if (ret == 0) {
ESP_LOGD(TAG, "Set v-flip to: %d", enable);
}
return ret;
}
static int set_colorbar(sensor_t *sensor, int enable)
{
int ret = 0;
// ret = write_reg(sensor->slv_addr, 0xfe, 0x00);
// ret |= set_reg_bits(sensor->slv_addr, P0_DEBUG_MODE3, 3, 0x01, enable);
if (ret == 0) {
sensor->status.colorbar = enable;
ESP_LOGD(TAG, "Set colorbar to: %d", enable);
}
return ret;
}
static int get_reg(sensor_t *sensor, int reg, int mask)
{
int ret = 0;
if (mask > 0xFF) {
ESP_LOGE(TAG, "mask should not more than 0xff");
} else {
ret = read_reg(sensor->slv_addr, reg);
}
if (ret > 0) {
ret &= mask;
}
return ret;
}
static int set_reg(sensor_t *sensor, int reg, int mask, int value)
{
int ret = 0;
if (mask > 0xFF) {
ESP_LOGE(TAG, "mask should not more than 0xff");
} else {
ret = read_reg(sensor->slv_addr, reg);
}
if (ret < 0) {
return ret;
}
value = (ret & ~mask) | (value & mask);
if (mask > 0xFF) {
} else {
ret = write_reg(sensor->slv_addr, reg, value);
}
return ret;
}
static int init_status(sensor_t *sensor)
{
write_reg(sensor->slv_addr, 0xfe, 0x00);
sensor->status.brightness = 0;
sensor->status.contrast = 0;
sensor->status.saturation = 0;
sensor->status.sharpness = 0;
sensor->status.denoise = 0;
sensor->status.ae_level = 0;
sensor->status.gainceiling = 0;
sensor->status.awb = 0;
sensor->status.dcw = 0;
sensor->status.agc = 0;
sensor->status.aec = 0;
sensor->status.hmirror = check_reg_mask(sensor->slv_addr, P0_ANALOG_MODE1, 0x01);
sensor->status.vflip = check_reg_mask(sensor->slv_addr, P0_ANALOG_MODE1, 0x02);
sensor->status.colorbar = 0;
sensor->status.bpc = 0;
sensor->status.wpc = 0;
sensor->status.raw_gma = 0;
sensor->status.lenc = 0;
sensor->status.quality = 0;
sensor->status.special_effect = 0;
sensor->status.wb_mode = 0;
sensor->status.awb_gain = 0;
sensor->status.agc_gain = 0;
sensor->status.aec_value = 0;
sensor->status.aec2 = 0;
print_regs(sensor->slv_addr);
return 0;
}
static int set_dummy(sensor_t *sensor, int val)
{
ESP_LOGW(TAG, "Unsupported");
return -1;
}
static int set_gainceiling_dummy(sensor_t *sensor, gainceiling_t val)
{
ESP_LOGW(TAG, "Unsupported");
return -1;
}
int gc2145_detect(int slv_addr, sensor_id_t *id)
{
if (GC2145_SCCB_ADDR == slv_addr) {
uint8_t MIDL = SCCB_Read(slv_addr, CHIP_ID_LOW);
uint8_t MIDH = SCCB_Read(slv_addr, CHIP_ID_HIGH);
uint16_t PID = MIDH << 8 | MIDL;
if (GC2145_PID == PID) {
id->PID = PID;
return PID;
} else {
ESP_LOGI(TAG, "Mismatch PID=0x%x", PID);
}
}
return 0;
}
int gc2145_init(sensor_t *sensor)
{
sensor->init_status = init_status;
sensor->reset = reset;
sensor->set_pixformat = set_pixformat;
sensor->set_framesize = set_framesize;
sensor->set_contrast = set_dummy;
sensor->set_brightness = set_dummy;
sensor->set_saturation = set_dummy;
sensor->set_sharpness = set_dummy;
sensor->set_denoise = set_dummy;
sensor->set_gainceiling = set_gainceiling_dummy;
sensor->set_quality = set_dummy;
sensor->set_colorbar = set_colorbar;
sensor->set_whitebal = set_dummy;
sensor->set_gain_ctrl = set_dummy;
sensor->set_exposure_ctrl = set_dummy;
sensor->set_hmirror = set_hmirror;
sensor->set_vflip = set_vflip;
sensor->set_aec2 = set_dummy;
sensor->set_awb_gain = set_dummy;
sensor->set_agc_gain = set_dummy;
sensor->set_aec_value = set_dummy;
sensor->set_special_effect = set_dummy;
sensor->set_wb_mode = set_dummy;
sensor->set_ae_level = set_dummy;
sensor->set_dcw = set_dummy;
sensor->set_bpc = set_dummy;
sensor->set_wpc = set_dummy;
sensor->set_raw_gma = set_dummy;
sensor->set_lenc = set_dummy;
sensor->get_reg = get_reg;
sensor->set_reg = set_reg;
sensor->set_res_raw = NULL;
sensor->set_pll = NULL;
sensor->set_xclk = NULL;
ESP_LOGD(TAG, "GC2145 Attached");
return 0;
}

View File

@@ -10,6 +10,7 @@
#include <stdlib.h>
#include <string.h>
#include "sccb.h"
#include "xclk.h"
#include "nt99141.h"
#include "nt99141_regs.h"
#include "nt99141_settings.h"
@@ -144,28 +145,6 @@ static int write_addr_reg(uint8_t slv_addr, const uint16_t reg, uint16_t x_value
#define write_reg_bits(slv_addr, reg, mask, enable) set_reg_bits(slv_addr, reg, 0, mask, enable?mask:0)
static int calc_sysclk(int xclk, bool pll_bypass, int pll_multiplier, int pll_sys_div, int pll_pre_div, bool pll_root_2x, int pll_seld5, bool pclk_manual, int pclk_div)
{
const int pll_pre_div2x_map[] = { 2, 3, 4, 6 };//values are multiplied by two to avoid floats
const int pll_seld52x_map[] = { 2, 2, 4, 5 };
if (!pll_sys_div) {
pll_sys_div = 1;
}
int pll_pre_div2x = pll_pre_div2x_map[pll_pre_div];
int pll_root_div = pll_root_2x ? 2 : 1;
int pll_seld52x = pll_seld52x_map[pll_seld5];
int VCO = (xclk / 1000) * pll_multiplier * pll_root_div * 2 / pll_pre_div2x;
int PLLCLK = pll_bypass ? (xclk) : (VCO * 1000 * 2 / pll_sys_div / pll_seld52x);
int PCLK = PLLCLK / 2 / ((pclk_manual && pclk_div) ? pclk_div : 1);
int SYSCLK = PLLCLK / 4;
ESP_LOGD(TAG, "Calculated VCO: %d Hz, PLLCLK: %d Hz, SYSCLK: %d Hz, PCLK: %d Hz", VCO * 1000, PLLCLK, SYSCLK, PCLK);
return SYSCLK;
}
static int set_pll(sensor_t *sensor, bool bypass, uint8_t multiplier, uint8_t sys_div, uint8_t pre_div, bool root_2x, uint8_t seld5, bool pclk_manual, uint8_t pclk_div)
{
return -1;
@@ -309,7 +288,7 @@ static int set_framesize(sensor_t *sensor, framesize_t framesize)
ret = write_regs(sensor->slv_addr, sensor_framesize_VGA);
}
return 0;
return ret;
}
static int set_hmirror(sensor_t *sensor, int enable)
@@ -682,7 +661,6 @@ static int set_brightness(sensor_t *sensor, int level)
{
int ret = 0;
uint8_t value = 0;
bool negative = false;
switch (level) {
case 3:
@@ -699,17 +677,14 @@ static int set_brightness(sensor_t *sensor, int level)
case -1:
value = 0x78;
negative = true;
break;
case -2:
value = 0x70;
negative = true;
break;
case -3:
value = 0x60;
negative = true;
break;
default: // 0
@@ -730,7 +705,6 @@ static int set_contrast(sensor_t *sensor, int level)
{
int ret = 0;
uint8_t value1 = 0, value2 = 0 ;
bool negative = false;
switch (level) {
case 3:
@@ -947,7 +921,6 @@ static int _set_pll(sensor_t *sensor, int bypass, int multiplier, int sys_div, i
return set_pll(sensor, bypass > 0, multiplier, sys_div, pre_div, root_2x > 0, seld5, pclk_manual > 0, pclk_div);
}
esp_err_t xclk_timer_conf(int ledc_timer, int xclk_freq_hz);
static int set_xclk(sensor_t *sensor, int timer, int xclk)
{
int ret = 0;
@@ -961,6 +934,23 @@ static int set_xclk(sensor_t *sensor, int timer, int xclk)
return ret;
}
int nt99141_detect(int slv_addr, sensor_id_t *id)
{
if (NT99141_SCCB_ADDR == slv_addr) {
SCCB_Write16(slv_addr, 0x3008, 0x01);//bank sensor
uint16_t h = SCCB_Read16(slv_addr, 0x3000);
uint16_t l = SCCB_Read16(slv_addr, 0x3001);
uint16_t PID = (h<<8) | l;
if (NT99141_PID == PID) {
id->PID = PID;
return PID;
} else {
ESP_LOGI(TAG, "Mismatch PID=0x%x", PID);
}
}
return 0;
}
static int init_status(sensor_t *sensor)
{
sensor->status.brightness = 0;
@@ -991,7 +981,7 @@ static int init_status(sensor_t *sensor)
return 0;
}
int NT99141_init(sensor_t *sensor)
int nt99141_init(sensor_t *sensor)
{
sensor->reset = reset;
sensor->set_pixformat = set_pixformat;

View File

@@ -10,6 +10,7 @@
#include <stdlib.h>
#include <string.h>
#include "sccb.h"
#include "xclk.h"
#include "ov2640.h"
#include "ov2640_regs.h"
#include "ov2640_settings.h"
@@ -149,7 +150,7 @@ static int set_window(sensor_t *sensor, ov2640_sensor_mode_t mode, int offset_x,
{VSIZE, max_y & 0xFF},
{XOFFL, offset_x & 0xFF},
{YOFFL, offset_y & 0xFF},
{VHYX, ((max_y >> 1) & 0X80) | ((offset_y >> 4) & 0X70) | ((max_x >> 5) & 0X08) | ((offset_y >> 8) & 0X07)},
{VHYX, ((max_y >> 1) & 0X80) | ((offset_y >> 4) & 0X70) | ((max_x >> 5) & 0X08) | ((offset_x >> 8) & 0X07)},
{TEST, (max_x >> 2) & 0X80},
{ZMOW, (w)&0xFF},
{ZMOH, (h)&0xFF},
@@ -157,26 +158,40 @@ static int set_window(sensor_t *sensor, ov2640_sensor_mode_t mode, int offset_x,
{0, 0}
};
c.pclk_auto = 0;
c.pclk_div = 8;
c.clk_2x = 0;
c.clk_div = 0;
if(sensor->pixformat != PIXFORMAT_JPEG){
c.pclk_auto = 1;
if (sensor->pixformat == PIXFORMAT_JPEG) {
c.clk_2x = 0;
c.clk_div = 0;
c.pclk_auto = 0;
c.pclk_div = 8;
if(mode == OV2640_MODE_UXGA) {
c.pclk_div = 12;
}
// if (sensor->xclk_freq_hz == 16000000) {
// c.pclk_div = c.pclk_div / 2;
// }
} else {
#if CONFIG_IDF_TARGET_ESP32
c.clk_2x = 0;
#else
c.clk_2x = 1;
#endif
c.clk_div = 7;
c.pclk_auto = 1;
c.pclk_div = 8;
if (mode == OV2640_MODE_CIF) {
c.clk_div = 3;
} else if(mode == OV2640_MODE_UXGA) {
c.pclk_div = 12;
}
}
ESP_LOGI(TAG, "Set PLL: clk_2x: %u, clk_div: %u, pclk_auto: %u, pclk_div: %u", c.clk_2x, c.clk_div, c.pclk_auto, c.pclk_div);
if (mode == OV2640_MODE_CIF) {
regs = ov2640_settings_to_cif;
if(sensor->pixformat != PIXFORMAT_JPEG){
c.clk_div = 3;
}
} else if (mode == OV2640_MODE_SVGA) {
regs = ov2640_settings_to_svga;
} else {
regs = ov2640_settings_to_uxga;
c.pclk_div = 12;
}
WRITE_REG_OR_RETURN(BANK_DSP, R_BYPASS, R_BYPASS_DSP_BYPAS);
@@ -480,7 +495,6 @@ static int _set_pll(sensor_t *sensor, int bypass, int multiplier, int sys_div, i
return -1;
}
esp_err_t xclk_timer_conf(int ledc_timer, int xclk_freq_hz);
static int set_xclk(sensor_t *sensor, int timer, int xclk)
{
int ret = 0;
@@ -531,6 +545,24 @@ static int init_status(sensor_t *sensor){
return 0;
}
int ov2640_detect(int slv_addr, sensor_id_t *id)
{
if (OV2640_SCCB_ADDR == slv_addr) {
SCCB_Write(slv_addr, 0xFF, 0x01);//bank sensor
uint16_t PID = SCCB_Read(slv_addr, 0x0A);
if (OV2640_PID == PID) {
id->PID = PID;
id->VER = SCCB_Read(slv_addr, REG_VER);
id->MIDL = SCCB_Read(slv_addr, REG_MIDL);
id->MIDH = SCCB_Read(slv_addr, REG_MIDH);
return PID;
} else {
ESP_LOGI(TAG, "Mismatch PID=0x%x", PID);
}
}
return 0;
}
int ov2640_init(sensor_t *sensor)
{
sensor->reset = reset;

View File

@@ -10,6 +10,7 @@
#include <stdlib.h>
#include <string.h>
#include "sccb.h"
#include "xclk.h"
#include "ov3660.h"
#include "ov3660_regs.h"
#include "ov3660_settings.h"
@@ -142,7 +143,7 @@ static int calc_sysclk(int xclk, bool pll_bypass, int pll_multiplier, int pll_sy
int PCLK = PLLCLK / 2 / ((pclk_manual && pclk_div)?pclk_div:1);
int SYSCLK = PLLCLK / 4;
ESP_LOGD(TAG, "Calculated VCO: %d Hz, PLLCLK: %d Hz, SYSCLK: %d Hz, PCLK: %d Hz", VCO*1000, PLLCLK, SYSCLK, PCLK);
ESP_LOGI(TAG, "Calculated VCO: %d Hz, PLLCLK: %d Hz, SYSCLK: %d Hz, PCLK: %d Hz", VCO*1000, PLLCLK, SYSCLK, PCLK);
return SYSCLK;
}
@@ -310,13 +311,13 @@ static int set_image_options(sensor_t *sensor)
static int set_framesize(sensor_t *sensor, framesize_t framesize)
{
int ret = 0;
framesize_t old_framesize = sensor->status.framesize;
sensor->status.framesize = framesize;
if(framesize > FRAMESIZE_QXGA){
ESP_LOGE(TAG, "Invalid framesize: %u", framesize);
return -1;
ESP_LOGW(TAG, "Invalid framesize: %u", framesize);
framesize = FRAMESIZE_QXGA;
}
framesize_t old_framesize = sensor->status.framesize;
sensor->status.framesize = framesize;
uint16_t w = resolution[framesize].width;
uint16_t h = resolution[framesize].height;
aspect_ratio_t ratio = resolution[sensor->status.framesize].aspect_ratio;
@@ -355,7 +356,7 @@ static int set_framesize(sensor_t *sensor, framesize_t framesize)
}
if (sensor->pixformat == PIXFORMAT_JPEG) {
if (framesize == FRAMESIZE_QXGA) {
if (framesize == FRAMESIZE_QXGA || sensor->xclk_freq_hz == 16000000) {
//40MHz SYSCLK and 10MHz PCLK
ret = set_pll(sensor, false, 24, 1, 3, false, 0, true, 8);
} else {
@@ -363,12 +364,16 @@ static int set_framesize(sensor_t *sensor, framesize_t framesize)
ret = set_pll(sensor, false, 30, 1, 3, false, 0, true, 10);
}
} else {
if (framesize > FRAMESIZE_CIF) {
//10MHz SYSCLK and 10MHz PCLK (6.19 FPS)
ret = set_pll(sensor, false, 2, 1, 0, false, 0, true, 2);
//tuned for 16MHz XCLK and 8MHz PCLK
if (framesize > FRAMESIZE_HVGA) {
//8MHz SYSCLK and 8MHz PCLK (4.44 FPS)
ret = set_pll(sensor, false, 4, 1, 0, false, 2, true, 2);
} else if (framesize >= FRAMESIZE_QVGA) {
//16MHz SYSCLK and 8MHz PCLK (10.25 FPS)
ret = set_pll(sensor, false, 8, 1, 0, false, 2, true, 4);
} else {
//25MHz SYSCLK and 10MHz PCLK (15.45 FPS)
ret = set_pll(sensor, false, 5, 1, 0, false, 0, true, 5);
//32MHz SYSCLK and 8MHz PCLK (17.77 FPS)
ret = set_pll(sensor, false, 8, 1, 0, false, 0, true, 8);
}
}
@@ -953,7 +958,6 @@ static int _set_pll(sensor_t *sensor, int bypass, int multiplier, int sys_div, i
return set_pll(sensor, bypass > 0, multiplier, sys_div, pre_div, root_2x > 0, seld5, pclk_manual > 0, pclk_div);
}
esp_err_t xclk_timer_conf(int ledc_timer, int xclk_freq_hz);
static int set_xclk(sensor_t *sensor, int timer, int xclk)
{
int ret = 0;
@@ -992,6 +996,22 @@ static int init_status(sensor_t *sensor)
return 0;
}
int ov3660_detect(int slv_addr, sensor_id_t *id)
{
if (OV3660_SCCB_ADDR == slv_addr) {
uint8_t h = SCCB_Read16(slv_addr, 0x300A);
uint8_t l = SCCB_Read16(slv_addr, 0x300B);
uint16_t PID = (h<<8) | l;
if (OV3660_PID == PID) {
id->PID = PID;
return PID;
} else {
ESP_LOGI(TAG, "Mismatch PID=0x%x", PID);
}
}
return 0;
}
int ov3660_init(sensor_t *sensor)
{
sensor->reset = reset;

View File

@@ -10,6 +10,7 @@
#include <stdlib.h>
#include <string.h>
#include "sccb.h"
#include "xclk.h"
#include "ov5640.h"
#include "ov5640_regs.h"
#include "ov5640_settings.h"
@@ -196,7 +197,7 @@ static int calc_sysclk(int xclk, bool pll_bypass, int pll_multiplier, int pll_sy
unsigned int SYSCLK = PLL_CLK / 4;
ESP_LOGD(TAG, "Calculated XVCLK: %d Hz, REFIN: %u Hz, VCO: %u Hz, PLL_CLK: %u Hz, SYSCLK: %u Hz, PCLK: %u Hz", xclk, REFIN, VCO, PLL_CLK, SYSCLK, PCLK);
ESP_LOGI(TAG, "Calculated XVCLK: %d Hz, REFIN: %u Hz, VCO: %u Hz, PLL_CLK: %u Hz, SYSCLK: %u Hz, PCLK: %u Hz", xclk, REFIN, VCO, PLL_CLK, SYSCLK, PCLK);
return SYSCLK;
}
@@ -209,6 +210,7 @@ static int set_pll(sensor_t *sensor, bool bypass, uint8_t multiplier, uint8_t sy
if(multiplier > 127){
multiplier &= 0xFE;//only even integers above 127
}
ESP_LOGI(TAG, "Set PLL: bypass: %u, multiplier: %u, sys_div: %u, pre_div: %u, root_2x: %u, pclk_root_div: %u, pclk_manual: %u, pclk_div: %u", bypass, multiplier, sys_div, pre_div, root_2x, pclk_root_div, pclk_manual, pclk_div);
calc_sysclk(sensor->xclk_freq_hz, bypass, multiplier, sys_div, pre_div, root_2x, pclk_root_div, pclk_manual, pclk_div);
@@ -432,14 +434,22 @@ static int set_framesize(sensor_t *sensor, framesize_t framesize)
if (sensor->pixformat == PIXFORMAT_JPEG) {
//10MHz PCLK
uint8_t sys_mul = 200;
if(framesize < FRAMESIZE_QVGA){
if(framesize < FRAMESIZE_QVGA || sensor->xclk_freq_hz == 16000000){
sys_mul = 160;
} else if(framesize < FRAMESIZE_XGA){
sys_mul = 180;
}
ret = set_pll(sensor, false, sys_mul, 4, 2, false, 2, true, 4);
//Set PLL: bypass: 0, multiplier: sys_mul, sys_div: 4, pre_div: 2, root_2x: 0, pclk_root_div: 2, pclk_manual: 1, pclk_div: 4
} else {
ret = set_pll(sensor, false, 10, 1, 1, false, 1, true, 4);
//ret = set_pll(sensor, false, 8, 1, 1, false, 1, true, 4);
if (framesize > FRAMESIZE_HVGA) {
ret = set_pll(sensor, false, 10, 1, 2, false, 1, true, 2);
} else if (framesize >= FRAMESIZE_QVGA) {
ret = set_pll(sensor, false, 8, 1, 1, false, 1, true, 4);
} else {
ret = set_pll(sensor, false, 20, 1, 1, false, 1, true, 8);
}
}
if (ret == 0) {
@@ -1025,7 +1035,6 @@ static int _set_pll(sensor_t *sensor, int bypass, int multiplier, int sys_div, i
return ret;
}
esp_err_t xclk_timer_conf(int ledc_timer, int xclk_freq_hz);
static int set_xclk(sensor_t *sensor, int timer, int xclk)
{
int ret = 0;
@@ -1064,6 +1073,22 @@ static int init_status(sensor_t *sensor)
return 0;
}
int ov5640_detect(int slv_addr, sensor_id_t *id)
{
if (OV5640_SCCB_ADDR == slv_addr) {
uint8_t h = SCCB_Read16(slv_addr, 0x300A);
uint8_t l = SCCB_Read16(slv_addr, 0x300B);
uint16_t PID = (h<<8) | l;
if (OV5640_PID == PID) {
id->PID = PID;
return PID;
} else {
ESP_LOGI(TAG, "Mismatch PID=0x%x", PID);
}
}
return 0;
}
int ov5640_init(sensor_t *sensor)
{
sensor->reset = reset;

View File

@@ -45,7 +45,7 @@ static struct regval_list ov7670_default_regs[] = {
{CLKRC, 0x00},
{DBLV, 0x4A},
{COM10, COM10_VSYNC_NEG | COM10_PCLK_MASK},
{COM10, COM10_VSYNC_NEG | COM10_PCLK_FREE},
/* Improve white balance */
{COM4, 0x40},
@@ -393,6 +393,24 @@ static int init_status(sensor_t *sensor)
static int set_dummy(sensor_t *sensor, int val){ return -1; }
static int set_gainceiling_dummy(sensor_t *sensor, gainceiling_t val){ return -1; }
int ov7670_detect(int slv_addr, sensor_id_t *id)
{
if (OV7670_SCCB_ADDR == slv_addr) {
SCCB_Write(slv_addr, 0xFF, 0x01);//bank sensor
uint16_t PID = SCCB_Read(slv_addr, 0x0A);
if (OV7670_PID == PID) {
id->PID = PID;
id->VER = SCCB_Read(slv_addr, REG_VER);
id->MIDL = SCCB_Read(slv_addr, REG_MIDL);
id->MIDH = SCCB_Read(slv_addr, REG_MIDH);
return PID;
} else {
ESP_LOGI(TAG, "Mismatch PID=0x%x", PID);
}
}
return 0;
}
int ov7670_init(sensor_t *sensor)
{
// Set function pointers

View File

@@ -11,6 +11,7 @@
#include <string.h>
#include <stdio.h>
#include "sccb.h"
#include "xclk.h"
#include "ov7725.h"
#include "ov7725_regs.h"
#include "freertos/FreeRTOS.h"
@@ -58,10 +59,10 @@ static const uint8_t default_regs[][2] = {
{COM8, 0xF0},
{COM6, 0xC5},
{COM9, 0x11},
{COM10, COM10_VSYNC_NEG | COM10_PCLK_MASK}, //Invert VSYNC and MASK PCLK
{COM10, COM10_VSYNC_NEG | COM10_PCLK_FREE}, //Invert VSYNC and MASK PCLK
{BDBASE, 0x7F},
{DBSTEP, 0x03},
{AEW, 0x96},
{AEW, 0x75},
{AEB, 0x64},
{VPT, 0xA1},
{EXHCL, 0x00},
@@ -493,7 +494,6 @@ static int set_gainceiling_dummy(sensor_t *sensor, gainceiling_t val){ return -1
static int set_res_raw(sensor_t *sensor, int startX, int startY, int endX, int endY, int offsetX, int offsetY, int totalX, int totalY, int outputX, int outputY, bool scale, bool binning){return -1;}
static int _set_pll(sensor_t *sensor, int bypass, int multiplier, int sys_div, int root_2x, int pre_div, int seld5, int pclk_manual, int pclk_div){return -1;}
esp_err_t xclk_timer_conf(int ledc_timer, int xclk_freq_hz);
static int set_xclk(sensor_t *sensor, int timer, int xclk)
{
int ret = 0;
@@ -502,6 +502,24 @@ static int set_xclk(sensor_t *sensor, int timer, int xclk)
return ret;
}
int ov7725_detect(int slv_addr, sensor_id_t *id)
{
if (OV7725_SCCB_ADDR == slv_addr) {
SCCB_Write(slv_addr, 0xFF, 0x01);//bank sensor
uint16_t PID = SCCB_Read(slv_addr, 0x0A);
if (OV7725_PID == PID) {
id->PID = PID;
id->VER = SCCB_Read(slv_addr, REG_VER);
id->MIDL = SCCB_Read(slv_addr, REG_MIDL);
id->MIDH = SCCB_Read(slv_addr, REG_MIDH);
return PID;
} else {
ESP_LOGI(TAG, "Mismatch PID=0x%x", PID);
}
}
return 0;
}
int ov7725_init(sensor_t *sensor)
{
// Set function pointers

View File

@@ -0,0 +1,33 @@
/*
* This file is part of the OpenMV project.
* Copyright (c) 2013/2014 Ibrahim Abdelkader <i.abdalkader@gmail.com>
* This work is licensed under the MIT license, see the file LICENSE for details.
*
* BF3005 driver.
*
*/
#ifndef __BF3005_H__
#define __BF3005_H__
#include "sensor.h"
/**
* @brief Detect sensor pid
*
* @param slv_addr SCCB address
* @param id Detection result
* @return
* 0: Can't detect this sensor
* Nonzero: This sensor has been detected
*/
int bf3005_detect(int slv_addr, sensor_id_t *id);
/**
* @brief initialize sensor function pointers
*
* @param sensor pointer of sensor
* @return
* Always 0
*/
int bf3005_init(sensor_t *sensor);
#endif // __BF3005_H__

View File

@@ -0,0 +1,337 @@
/*
* This file is part of the OpenMV project.
* Copyright (c) 2013/2014 Ibrahim Abdelkader <i.abdalkader@gmail.com>
* This work is licensed under the MIT license, see the file LICENSE for details.
*
* BF3005 register definitions.
*/
#ifndef __REG_REGS_H__
#define __REG_REGS_H__
#if 0
#define GAIN 0x00 /* AGC <20>C Gain control gain setting */
#define BLUE 0x01 /* AWB <20>C Blue channel gain setting */
#define RED 0x02 /* AWB <20>C Red channel gain setting */
#define GREEN 0x03 /* AWB <20>C Green channel gain setting */
#define BAVG 0x05 /* U/B Average Level */
#define GAVG 0x06 /* Y/Gb Average Level */
#define RAVG 0x07 /* V/R Average Level */
#define AECH 0x08 /* Exposure Value <20>C AEC MSBs */
#define COM2 0x09 /* Common Control 2 */
#define COM2_SOFT_SLEEP 0x10 /* Soft sleep mode */
#define COM2_OUT_DRIVE_1x 0x00 /* Output drive capability 1x */
#define COM2_OUT_DRIVE_2x 0x01 /* Output drive capability 2x */
#define COM2_OUT_DRIVE_3x 0x02 /* Output drive capability 3x */
#define COM2_OUT_DRIVE_4x 0x03 /* Output drive capability 4x */
#define REG_PID 0x0A /* Product ID Number MSB */
#define REG_VER 0x0B /* Product ID Number LSB */
#define COM3 0x0C /* Common Control 3 */
#define COM3_VFLIP 0x80 /* Vertical flip image ON/OFF selection */
#define COM3_MIRROR 0x40 /* Horizontal mirror image ON/OFF selection */
#define COM3_SWAP_BR 0x20 /* Swap B/R output sequence in RGB output mode */
#define COM3_SWAP_YUV 0x10 /* Swap Y/UV output sequence in YUV output mode */
#define COM3_SWAP_MSB 0x08 /* Swap output MSB/LSB */
#define COM3_TRI_CLOCK 0x04 /* Tri-state option for output clock at power-down period */
#define COM3_TRI_DATA 0x02 /* Tri-state option for output data at power-down period */
#define COM3_COLOR_BAR 0x01 /* Sensor color bar test pattern output enable */
#define COM3_SET_CBAR(r, x) ((r&0xFE)|((x&1)<<0))
#define COM3_SET_MIRROR(r, x) ((r&0xBF)|((x&1)<<6))
#define COM3_SET_FLIP(r, x) ((r&0x7F)|((x&1)<<7))
#define COM4 0x0D /* Common Control 4 */
#define COM4_PLL_BYPASS 0x00 /* Bypass PLL */
#define COM4_PLL_4x 0x40 /* PLL frequency 4x */
#define COM4_PLL_6x 0x80 /* PLL frequency 6x */
#define COM4_PLL_8x 0xc0 /* PLL frequency 8x */
#define COM4_AEC_FULL 0x00 /* AEC evaluate full window */
#define COM4_AEC_1_2 0x10 /* AEC evaluate 1/2 window */
#define COM4_AEC_1_4 0x20 /* AEC evaluate 1/4 window */
#define COM4_AEC_2_3 0x30 /* AEC evaluate 2/3 window */
#define COM5 0x0E /* Common Control 5 */
#define COM5_AFR 0x80 /* Auto frame rate control ON/OFF selection (night mode) */
#define COM5_AFR_SPEED 0x40 /* Auto frame rate control speed selection */
#define COM5_AFR_0 0x00 /* No reduction of frame rate */
#define COM5_AFR_1_2 0x10 /* Max reduction to 1/2 frame rate */
#define COM5_AFR_1_4 0x20 /* Max reduction to 1/4 frame rate */
#define COM5_AFR_1_8 0x30 /* Max reduction to 1/8 frame rate */
#define COM5_AFR_4x 0x04 /* Add frame when AGC reaches 4x gain */
#define COM5_AFR_8x 0x08 /* Add frame when AGC reaches 8x gain */
#define COM5_AFR_16x 0x0c /* Add frame when AGC reaches 16x gain */
#define COM5_AEC_NO_LIMIT 0x01 /* No limit to AEC increase step */
#define COM6 0x0F /* Common Control 6 */
#define COM6_AUTO_WINDOW 0x01 /* Auto window setting ON/OFF selection when format changes */
#define AEC 0x10 /* AEC[7:0] (see register AECH for AEC[15:8]) */
#define CLKRC 0x11 /* Internal Clock */
#define COM7 0x12 /* Common Control 7 */
#define COM7_RESET 0x80 /* SCCB Register Reset */
#define COM7_RES_VGA 0x00 /* Resolution VGA */
#define COM7_RES_QVGA 0x40 /* Resolution QVGA */
#define COM7_BT656 0x20 /* BT.656 protocol ON/OFF */
#define COM7_SENSOR_RAW 0x10 /* Sensor RAW */
#define COM7_FMT_GBR422 0x00 /* RGB output format GBR422 */
#define COM7_FMT_RGB565 0x04 /* RGB output format RGB565 */
#define COM7_FMT_RGB555 0x08 /* RGB output format RGB555 */
#define COM7_FMT_RGB444 0x0C /* RGB output format RGB444 */
#define COM7_FMT_YUV 0x00 /* Output format YUV */
#define COM7_FMT_P_BAYER 0x01 /* Output format Processed Bayer RAW */
#define COM7_FMT_RGB 0x02 /* Output format RGB */
#define COM7_FMT_R_BAYER 0x03 /* Output format Bayer RAW */
#define COM7_SET_FMT(r, x) ((r&0xFC)|((x&0x3)<<0))
#define COM7_SET_RGB(r, x) ((r&0xF0)|(x&0x0C)|COM7_FMT_RGB)
#define COM8 0x13 /* Common Control 8 */
#define COM8_FAST_AUTO 0x80 /* Enable fast AGC/AEC algorithm */
#define COM8_STEP_VSYNC 0x00 /* AEC - Step size limited to vertical blank */
#define COM8_STEP_UNLIMIT 0x40 /* AEC - Step size unlimited step size */
#define COM8_BANDF_EN 0x20 /* Banding filter ON/OFF */
#define COM8_AEC_BANDF 0x10 /* Enable AEC below banding value */
#define COM8_AEC_FINE_EN 0x08 /* Fine AEC ON/OFF control */
#define COM8_AGC_EN 0x04 /* AGC Enable */
#define COM8_AWB_EN 0x02 /* AWB Enable */
#define COM8_AEC_EN 0x01 /* AEC Enable */
#define COM8_SET_AGC(r, x) ((r&0xFB)|((x&0x1)<<2))
#define COM8_SET_AWB(r, x) ((r&0xFD)|((x&0x1)<<1))
#define COM8_SET_AEC(r, x) ((r&0xFE)|((x&0x1)<<0))
#define COM9 0x14 /* Common Control 9 */
#define COM9_HISTO_AVG 0x80 /* Histogram or average based AEC/AGC selection */
#define COM9_AGC_GAIN_2x 0x00 /* Automatic Gain Ceiling 2x */
#define COM9_AGC_GAIN_4x 0x10 /* Automatic Gain Ceiling 4x */
#define COM9_AGC_GAIN_8x 0x20 /* Automatic Gain Ceiling 8x */
#define COM9_AGC_GAIN_16x 0x30 /* Automatic Gain Ceiling 16x */
#define COM9_AGC_GAIN_32x 0x40 /* Automatic Gain Ceiling 32x */
#define COM9_DROP_VSYNC 0x04 /* Drop VSYNC output of corrupt frame */
#define COM9_DROP_HREF 0x02 /* Drop HREF output of corrupt frame */
#define COM9_SET_AGC(r, x) ((r&0x8F)|((x&0x07)<<4))
#define COM10 0x15 /* Common Control 10 */
#define COM10_NEGATIVE 0x80 /* Output negative data */
#define COM10_HSYNC_EN 0x40 /* HREF changes to HSYNC */
#define COM10_PCLK_FREE 0x00 /* PCLK output option: free running PCLK */
#define COM10_PCLK_MASK 0x20 /* PCLK output option: masked during horizontal blank */
#define COM10_PCLK_REV 0x10 /* PCLK reverse */
#define COM10_HREF_REV 0x08 /* HREF reverse */
#define COM10_VSYNC_FALLING 0x00 /* VSYNC changes on falling edge of PCLK */
#define COM10_VSYNC_RISING 0x04 /* VSYNC changes on rising edge of PCLK */
#define COM10_VSYNC_NEG 0x02 /* VSYNC negative */
#define COM10_OUT_RANGE_8 0x01 /* Output data range: Full range */
#define COM10_OUT_RANGE_10 0x00 /* Output data range: Data from [10] to [F0] (8 MSBs) */
#define REG16 0x16 /* Register 16 */
#define REG16_BIT_SHIFT 0x80 /* Bit shift test pattern options */
#define HSTART 0x17 /* Horizontal Frame (HREF column) Start 8 MSBs (2 LSBs are at HREF[5:4]) */
#define HSIZE 0x18 /* Horizontal Sensor Size (2 LSBs are at HREF[1:0]) */
#define VSTART 0x19 /* Vertical Frame (row) Start 8 MSBs (1 LSB is at HREF[6]) */
#define VSIZE 0x1A /* Vertical Sensor Size (1 LSB is at HREF[2]) */
#define PSHFT 0x1B /* Data Format - Pixel Delay Select */
#define REG_MIDH 0x1C /* Manufacturer ID Byte <20>C High */
#define REG_MIDL 0x1D /* Manufacturer ID Byte <20>C Low */
#define LAEC 0x1F /* Fine AEC Value - defines exposure value less than one row period */
#define COM11 0x20 /* Common Control 11 */
#define COM11_SNGL_FRAME_EN 0x02 /* Single frame ON/OFF selection */
#define COM11_SNGL_XFR_TRIG 0x01 /* Single frame transfer trigger */
#define BDBASE 0x22 /* Banding Filter Minimum AEC Value */
#define DBSTEP 0x23 /* Banding Filter Maximum Step */
#define AEW 0x24 /* AGC/AEC - Stable Operating Region (Upper Limit) */
#define AEB 0x25 /* AGC/AEC - Stable Operating Region (Lower Limit) */
#define VPT 0x26 /* AGC/AEC Fast Mode Operating Region */
#define REG28 0x28 /* Selection on the number of dummy rows, N */
#define HOUTSIZE 0x29 /* Horizontal Data Output Size MSBs (2 LSBs at register EXHCH[1:0]) */
#define EXHCH 0x2A /* Dummy Pixel Insert MSB */
#define EXHCL 0x2B /* Dummy Pixel Insert LSB */
#define VOUTSIZE 0x2C /* Vertical Data Output Size MSBs (LSB at register EXHCH[2]) */
#define ADVFL 0x2D /* LSB of Insert Dummy Rows in Vertical Sync (1 bit equals 1 row) */
#define ADVFH 0x2E /* MSB of Insert Dummy Rows in Vertical Sync */
#define YAVE 0x2F /* Y/G Channel Average Value */
#define LUMHTH 0x30 /* Histogram AEC/AGC Luminance High Level Threshold */
#define LUMLTH 0x31 /* Histogram AEC/AGC Luminance Low Level Threshold */
#define HREF 0x32 /* Image Start and Size Control */
#define DM_LNL 0x33 /* Dummy Row Low 8 Bits */
#define DM_LNH 0x34 /* Dummy Row High 8 Bits */
#define ADOFF_B 0x35 /* AD Offset Compensation Value for B Channel */
#define ADOFF_R 0x36 /* AD Offset Compensation Value for R Channel */
#define ADOFF_GB 0x37 /* AD Offset Compensation Value for GB Channel */
#define ADOFF_GR 0x38 /* AD Offset Compensation Value for GR Channel */
#define OFF_B 0x39 /* AD Offset Compensation Value for B Channel */
#define OFF_R 0x3A /* AD Offset Compensation Value for R Channel */
#define OFF_GB 0x3B /* AD Offset Compensation Value for GB Channel */
#define OFF_GR 0x3C /* AD Offset Compensation Value for GR Channel */
#define COM12 0x3D /* DC offset compensation for analog process */
#define COM13 0x3E /* Common Control 13 */
#define COM13_BLC_EN 0x80 /* BLC enable */
#define COM13_ADC_EN 0x40 /* ADC channel BLC ON/OFF control */
#define COM13_ANALOG_BLC 0x20 /* Analog processing channel BLC ON/OFF control */
#define COM13_ABLC_GAIN_EN 0x04 /* ABLC gain trigger enable */
#define COM14 0x3F /* Common Control 14 */
#define COM15 0x40 /* Common Control 15 */
#define COM16 0x41 /* Common Control 16 */
#define TGT_B 0x42 /* BLC Blue Channel Target Value */
#define TGT_R 0x43 /* BLC Red Channel Target Value */
#define TGT_GB 0x44 /* BLC Gb Channel Target Value */
#define TGT_GR 0x45 /* BLC Gr Channel Target Value */
#define LC_CTR 0x46 /* Lens Correction Control */
#define LC_CTR_RGB_COMP_1 0x00 /* R, G, and B channel compensation coefficient is set by LC_COEF (0x49) */
#define LC_CTR_RGB_COMP_3 0x04 /* R, G, and B channel compensation coefficient is set by registers
LC_COEFB (0x4B), LC_COEF (0x49), and LC_COEFR (0x4C), respectively */
#define LC_CTR_EN 0x01 /* Lens correction enable */
#define LC_XC 0x47 /* X Coordinate of Lens Correction Center Relative to Array Center */
#define LC_YC 0x48 /* Y Coordinate of Lens Correction Center Relative to Array Center */
#define LC_COEF 0x49 /* Lens Correction Coefficient */
#define LC_RADI 0x4A /* Lens Correction Radius */
#define LC_COEFB 0x4B /* Lens Correction B Channel Compensation Coefficient */
#define LC_COEFR 0x4C /* Lens Correction R Channel Compensation Coefficient */
#define FIXGAIN 0x4D /* Analog Fix Gain Amplifier */
#define AREF0 0x4E /* Sensor Reference Control */
#define AREF1 0x4F /* Sensor Reference Current Control */
#define AREF2 0x50 /* Analog Reference Control */
#define AREF3 0x51 /* ADC Reference Control */
#define AREF4 0x52 /* ADC Reference Control */
#define AREF5 0x53 /* ADC Reference Control */
#define AREF6 0x54 /* Analog Reference Control */
#define AREF7 0x55 /* Analog Reference Control */
#define UFIX 0x60 /* U Channel Fixed Value Output */
#define VFIX 0x61 /* V Channel Fixed Value Output */
#define AWBB_BLK 0x62 /* AWB Option for Advanced AWB */
#define AWB_CTRL0 0x63 /* AWB Control Byte 0 */
#define AWB_CTRL0_GAIN_EN 0x80 /* AWB gain enable */
#define AWB_CTRL0_CALC_EN 0x40 /* AWB calculate enable */
#define AWB_CTRL0_WBC_MASK 0x0F /* WBC threshold 2 */
#define DSP_CTRL1 0x64 /* DSP Control Byte 1 */
#define DSP_CTRL1_FIFO_EN 0x80 /* FIFO enable/disable selection */
#define DSP_CTRL1_UV_EN 0x40 /* UV adjust function ON/OFF selection */
#define DSP_CTRL1_SDE_EN 0x20 /* SDE enable */
#define DSP_CTRL1_MTRX_EN 0x10 /* Color matrix ON/OFF selection */
#define DSP_CTRL1_INTRP_EN 0x08 /* Interpolation ON/OFF selection */
#define DSP_CTRL1_GAMMA_EN 0x04 /* Gamma function ON/OFF selection */
#define DSP_CTRL1_BLACK_EN 0x02 /* Black defect auto correction ON/OFF */
#define DSP_CTRL1_WHITE_EN 0x01 /* White defect auto correction ON/OFF */
#define DSP_CTRL2 0x65 /* DSP Control Byte 2 */
#define DSP_CTRL2_VDCW_EN 0x08 /* Vertical DCW enable */
#define DSP_CTRL2_HDCW_EN 0x04 /* Horizontal DCW enable */
#define DSP_CTRL2_VZOOM_EN 0x02 /* Vertical zoom out enable */
#define DSP_CTRL2_HZOOM_EN 0x01 /* Horizontal zoom out enable */
#define DSP_CTRL3 0x66 /* DSP Control Byte 3 */
#define DSP_CTRL3_UV_EN 0x80 /* UV output sequence option */
#define DSP_CTRL3_CBAR_EN 0x20 /* DSP color bar ON/OFF selection */
#define DSP_CTRL3_FIFO_EN 0x08 /* FIFO power down ON/OFF selection */
#define DSP_CTRL3_SCAL1_PWDN 0x04 /* Scaling module power down control 1 */
#define DSP_CTRL3_SCAL2_PWDN 0x02 /* Scaling module power down control 2 */
#define DSP_CTRL3_INTRP_PWDN 0x01 /* Interpolation module power down control */
#define DSP_CTRL3_SET_CBAR(r, x) ((r&0xDF)|((x&1)<<5))
#define DSP_CTRL4 0x67 /* DSP Control Byte 4 */
#define DSP_CTRL4_YUV_RGB 0x00 /* Output selection YUV or RGB */
#define DSP_CTRL4_RAW8 0x02 /* Output selection RAW8 */
#define DSP_CTRL4_RAW10 0x03 /* Output selection RAW10 */
#define AWB_BIAS 0x68 /* AWB BLC Level Clip */
#define AWB_CTRL1 0x69 /* AWB Control 1 */
#define AWB_CTRL2 0x6A /* AWB Control 2 */
#define AWB_CTRL3 0x6B /* AWB Control 3 */
#define AWB_CTRL3_ADVANCED 0x80 /* AWB mode select - Advanced AWB */
#define AWB_CTRL3_SIMPLE 0x00 /* AWB mode select - Simple AWB */
#define AWB_CTRL4 0x6C /* AWB Control 4 */
#define AWB_CTRL5 0x6D /* AWB Control 5 */
#define AWB_CTRL6 0x6E /* AWB Control 6 */
#define AWB_CTRL7 0x6F /* AWB Control 7 */
#define AWB_CTRL8 0x70 /* AWB Control 8 */
#define AWB_CTRL9 0x71 /* AWB Control 9 */
#define AWB_CTRL10 0x72 /* AWB Control 10 */
#define AWB_CTRL11 0x73 /* AWB Control 11 */
#define AWB_CTRL12 0x74 /* AWB Control 12 */
#define AWB_CTRL13 0x75 /* AWB Control 13 */
#define AWB_CTRL14 0x76 /* AWB Control 14 */
#define AWB_CTRL15 0x77 /* AWB Control 15 */
#define AWB_CTRL16 0x78 /* AWB Control 16 */
#define AWB_CTRL17 0x79 /* AWB Control 17 */
#define AWB_CTRL18 0x7A /* AWB Control 18 */
#define AWB_CTRL19 0x7B /* AWB Control 19 */
#define AWB_CTRL20 0x7C /* AWB Control 20 */
#define AWB_CTRL21 0x7D /* AWB Control 21 */
#define GAM1 0x7E /* Gamma Curve 1st Segment Input End Point 0x04 Output Value */
#define GAM2 0x7F /* Gamma Curve 2nd Segment Input End Point 0x08 Output Value */
#define GAM3 0x80 /* Gamma Curve 3rd Segment Input End Point 0x10 Output Value */
#define GAM4 0x81 /* Gamma Curve 4th Segment Input End Point 0x20 Output Value */
#define GAM5 0x82 /* Gamma Curve 5th Segment Input End Point 0x28 Output Value */
#define GAM6 0x83 /* Gamma Curve 6th Segment Input End Point 0x30 Output Value */
#define GAM7 0x84 /* Gamma Curve 7th Segment Input End Point 0x38 Output Value */
#define GAM8 0x85 /* Gamma Curve 8th Segment Input End Point 0x40 Output Value */
#define GAM9 0x86 /* Gamma Curve 9th Segment Input End Point 0x48 Output Value */
#define GAM10 0x87 /* Gamma Curve 10th Segment Input End Point 0x50 Output Value */
#define GAM11 0x88 /* Gamma Curve 11th Segment Input End Point 0x60 Output Value */
#define GAM12 0x89 /* Gamma Curve 12th Segment Input End Point 0x70 Output Value */
#define GAM13 0x8A /* Gamma Curve 13th Segment Input End Point 0x90 Output Value */
#define GAM14 0x8B /* Gamma Curve 14th Segment Input End Point 0xB0 Output Value */
#define GAM15 0x8C /* Gamma Curve 15th Segment Input End Point 0xD0 Output Value */
#define SLOP 0x8D /* Gamma Curve Highest Segment Slope */
#define DNSTH 0x8E /* De-noise Threshold */
#define EDGE0 0x8F /* Edge Enhancement Strength Control */
#define EDGE1 0x90 /* Edge Enhancement Threshold Control */
#define DNSOFF 0x91 /* Auto De-noise Threshold Control */
#define EDGE2 0x92 /* Edge Enhancement Strength Upper Limit */
#define EDGE3 0x93 /* Edge Enhancement Strength Upper Limit */
#define MTX1 0x94 /* Matrix Coefficient 1 */
#define MTX2 0x95 /* Matrix Coefficient 2 */
#define MTX3 0x96 /* Matrix Coefficient 3 */
#define MTX4 0x97 /* Matrix Coefficient 4 */
#define MTX5 0x98 /* Matrix Coefficient 5 */
#define MTX6 0x99 /* Matrix Coefficient 6 */
#define MTX_CTRL 0x9A /* Matrix Control */
#define MTX_CTRL_DBL_EN 0x80 /* Matrix double ON/OFF selection */
#define BRIGHTNESS 0x9B /* Brightness Control */
#define CONTRAST 0x9C /* Contrast Gain */
#define UVADJ0 0x9E /* Auto UV Adjust Control 0 */
#define UVADJ1 0x9F /* Auto UV Adjust Control 1 */
#define SCAL0 0xA0 /* DCW Ratio Control */
#define SCAL1 0xA1 /* Horizontal Zoom Out Control */
#define SCAL2 0xA2 /* Vertical Zoom Out Control */
#define FIFODLYM 0xA3 /* FIFO Manual Mode Delay Control */
#define FIFODLYA 0xA4 /* FIFO Auto Mode Delay Control */
#define SDE 0xA6 /* Special Digital Effect Control */
#define SDE_NEGATIVE_EN 0x40 /* Negative image enable */
#define SDE_GRAYSCALE_EN 0x20 /* Gray scale image enable */
#define SDE_V_FIXED_EN 0x10 /* V fixed value enable */
#define SDE_U_FIXED_EN 0x08 /* U fixed value enable */
#define SDE_CONT_BRIGHT_EN 0x04 /* Contrast/Brightness enable */
#define SDE_SATURATION_EN 0x02 /* Saturation enable */
#define SDE_HUE_EN 0x01 /* Hue enable */
#define USAT 0xA7 /* U Component Saturation Gain */
#define VSAT 0xA8 /* V Component Saturation Gain */
#define HUECOS 0xA9 /* Cosine value <20><> 0x80 */
#define HUESIN 0xAA /* Sine value <20><> 0x80 */
#define SIGN_BIT 0xAB /* Sign Bit for Hue and Brightness */
#define DSPAUTO 0xAC /* DSP Auto Function ON/OFF Control */
#define DSPAUTO_AWB_EN 0x80 /* AWB auto threshold control */
#define DSPAUTO_DENOISE_EN 0x40 /* De-noise auto threshold control */
#define DSPAUTO_EDGE_EN 0x20 /* Sharpness (edge enhancement) auto strength control */
#define DSPAUTO_UV_EN 0x10 /* UV adjust auto slope control */
#define DSPAUTO_SCAL0_EN 0x08 /* Auto scaling factor control (register SCAL0 (0xA0)) */
#define DSPAUTO_SCAL1_EN 0x04 /* Auto scaling factor control (registers SCAL1 (0xA1 and SCAL2 (0xA2))*/
#define SET_REG(reg, x) (##reg_DEFAULT|x)
#endif //__REG_REGS_H__
#endif

View File

@@ -0,0 +1,31 @@
#pragma once
#include "sensor.h"
#ifdef __cplusplus
extern "C" {
#endif
/**
* @brief Detect sensor pid
*
* @param slv_addr SCCB address
* @param id Detection result
* @return
* 0: Can't detect this sensor
* Nonzero: This sensor has been detected
*/
int gc0308_detect(int slv_addr, sensor_id_t *id);
/**
* @brief initialize sensor function pointers
*
* @param sensor pointer of sensor
* @return
* Always 0
*/
int gc0308_init(sensor_t *sensor);
#ifdef __cplusplus
}
#endif

View File

@@ -0,0 +1,25 @@
/*
* GC0308 register definitions.
*/
#ifndef __GC0308_REG_REGS_H__
#define __GC0308_REG_REGS_H__
#define RESET_RELATED 0xfe // Bit[7]: Software reset
// Bit[6:5]: NA
// Bit[4]: CISCTL_restart_n
// Bit[3:1]: NA
// Bit[0]: page select
// 0:page0
// 1:page1
// page0:
/**
* @brief register value
*/
#endif // __GC0308_REG_REGS_H__

View File

@@ -0,0 +1,245 @@
#ifndef _GC0308_SETTINGS_H_
#define _GC0308_SETTINGS_H_
#include <stdint.h>
#define REG_DLY 0xffff
#define REGLIST_TAIL 0x0000 /* Array end token */
static const uint16_t gc0308_sensor_default_regs[][2] = {
{0xfe, 0x00},
{0xec, 0x20},
{0x05, 0x00},
{0x06, 0x00},
{0x07, 0x00},
{0x08, 0x00},
{0x09, 0x01},
{0x0a, 0xe8},
{0x0b, 0x02},
{0x0c, 0x88},
{0x0d, 0x02},
{0x0e, 0x02},
{0x10, 0x26},
{0x11, 0x0d},
{0x12, 0x2a},
{0x13, 0x00},
{0x14, 0x11},
{0x15, 0x0a},
{0x16, 0x05},
{0x17, 0x01},
{0x18, 0x44},
{0x19, 0x44},
{0x1a, 0x2a},
{0x1b, 0x00},
{0x1c, 0x49},
{0x1d, 0x9a},
{0x1e, 0x61},
{0x1f, 0x00}, //pad drv <=24MHz, use 0x00 is ok
{0x20, 0x7f},
{0x21, 0xfa},
{0x22, 0x57},
{0x24, 0xa2}, //YCbYCr
{0x25, 0x0f},
{0x26, 0x03}, // 0x01
{0x28, 0x00},
{0x2d, 0x0a},
{0x2f, 0x01},
{0x30, 0xf7},
{0x31, 0x50},
{0x32, 0x00},
{0x33, 0x28},
{0x34, 0x2a},
{0x35, 0x28},
{0x39, 0x04},
{0x3a, 0x20},
{0x3b, 0x20},
{0x3c, 0x00},
{0x3d, 0x00},
{0x3e, 0x00},
{0x3f, 0x00},
{0x50, 0x14}, // 0x14
{0x52, 0x41},
{0x53, 0x80},
{0x54, 0x80},
{0x55, 0x80},
{0x56, 0x80},
{0x8b, 0x20},
{0x8c, 0x20},
{0x8d, 0x20},
{0x8e, 0x14},
{0x8f, 0x10},
{0x90, 0x14},
{0x91, 0x3c},
{0x92, 0x50},
//{0x8b,0x10},
//{0x8c,0x10},
//{0x8d,0x10},
//{0x8e,0x10},
//{0x8f,0x10},
//{0x90,0x10},
//{0x91,0x3c},
//{0x92,0x50},
{0x5d, 0x12},
{0x5e, 0x1a},
{0x5f, 0x24},
{0x60, 0x07},
{0x61, 0x15},
{0x62, 0x08}, // 0x08
{0x64, 0x03}, // 0x03
{0x66, 0xe8},
{0x67, 0x86},
{0x68, 0x82},
{0x69, 0x18},
{0x6a, 0x0f},
{0x6b, 0x00},
{0x6c, 0x5f},
{0x6d, 0x8f},
{0x6e, 0x55},
{0x6f, 0x38},
{0x70, 0x15},
{0x71, 0x33},
{0x72, 0xdc},
{0x73, 0x00},
{0x74, 0x02},
{0x75, 0x3f},
{0x76, 0x02},
{0x77, 0x38}, // 0x47
{0x78, 0x88},
{0x79, 0x81},
{0x7a, 0x81},
{0x7b, 0x22},
{0x7c, 0xff},
{0x93, 0x48}, //color matrix default
{0x94, 0x02},
{0x95, 0x07},
{0x96, 0xe0},
{0x97, 0x40},
{0x98, 0xf0},
{0xb1, 0x40},
{0xb2, 0x40},
{0xb3, 0x40}, //0x40
{0xb6, 0xe0},
{0xbd, 0x38},
{0xbe, 0x36},
{0xd0, 0xCB},
{0xd1, 0x10},
{0xd2, 0x90},
{0xd3, 0x48},
{0xd5, 0xF2},
{0xd6, 0x16},
{0xdb, 0x92},
{0xdc, 0xA5},
{0xdf, 0x23},
{0xd9, 0x00},
{0xda, 0x00},
{0xe0, 0x09},
{0xed, 0x04},
{0xee, 0xa0},
{0xef, 0x40},
{0x80, 0x03},
{0x9F, 0x10},
{0xA0, 0x20},
{0xA1, 0x38},
{0xA2, 0x4e},
{0xA3, 0x63},
{0xA4, 0x76},
{0xA5, 0x87},
{0xA6, 0xa2},
{0xA7, 0xb8},
{0xA8, 0xca},
{0xA9, 0xd8},
{0xAA, 0xe3},
{0xAB, 0xeb},
{0xAC, 0xf0},
{0xAD, 0xF8},
{0xAE, 0xFd},
{0xAF, 0xFF},
{0xc0, 0x00},
{0xc1, 0x10},
{0xc2, 0x1c},
{0xc3, 0x30},
{0xc4, 0x43},
{0xc5, 0x54},
{0xc6, 0x65},
{0xc7, 0x75},
{0xc8, 0x93},
{0xc9, 0xB0},
{0xca, 0xCB},
{0xcb, 0xE6},
{0xcc, 0xFF},
{0xf0, 0x02},
{0xf1, 0x01},
{0xf2, 0x02},
{0xf3, 0x30},
{0xf7, 0x04},
{0xf8, 0x02},
{0xf9, 0x9f},
{0xfa, 0x78},
{0xfe, 0x01},
{0x00, 0xf5},
{0x02, 0x20},
{0x04, 0x10},
{0x05, 0x08},
{0x06, 0x20},
{0x08, 0x0a},
{0x0a, 0xa0},
{0x0b, 0x60},
{0x0c, 0x08},
{0x0e, 0x44},
{0x0f, 0x32},
{0x10, 0x41},
{0x11, 0x37},
{0x12, 0x22},
{0x13, 0x19},
{0x14, 0x44},
{0x15, 0x44},
{0x16, 0xc2},
{0x17, 0xA8},
{0x18, 0x18},
{0x19, 0x50},
{0x1a, 0xd8},
{0x1b, 0xf5},
{0x70, 0x40},
{0x71, 0x58},
{0x72, 0x30},
{0x73, 0x48},
{0x74, 0x20},
{0x75, 0x60},
{0x77, 0x20},
{0x78, 0x32},
{0x30, 0x03},
{0x31, 0x40},
{0x32, 0x10},
{0x33, 0xe0},
{0x34, 0xe0},
{0x35, 0x00},
{0x36, 0x80},
{0x37, 0x00},
{0x38, 0x04},
{0x39, 0x09},
{0x3a, 0x12},
{0x3b, 0x1C},
{0x3c, 0x28},
{0x3d, 0x31},
{0x3e, 0x44},
{0x3f, 0x57},
{0x40, 0x6C},
{0x41, 0x81},
{0x42, 0x94},
{0x43, 0xA7},
{0x44, 0xB8},
{0x45, 0xD6},
{0x46, 0xEE},
{0x47, 0x0d},
{0x62, 0xf7},
{0x63, 0x68},
{0x64, 0xd3},
{0x65, 0xd3},
{0x66, 0x60},
{0xfe, 0x00},
{REGLIST_TAIL, 0x00},
};
#endif

View File

@@ -0,0 +1,31 @@
/*
*
* GC032A driver.
*
*/
#ifndef __GC032A_H__
#define __GC032A_H__
#include "sensor.h"
/**
* @brief Detect sensor pid
*
* @param slv_addr SCCB address
* @param id Detection result
* @return
* 0: Can't detect this sensor
* Nonzero: This sensor has been detected
*/
int gc032a_detect(int slv_addr, sensor_id_t *id);
/**
* @brief initialize sensor function pointers
*
* @param sensor pointer of sensor
* @return
* Always 0
*/
int gc032a_init(sensor_t *sensor);
#endif // __GC032A_H__

View File

@@ -0,0 +1,82 @@
/*
* GC032A register definitions.
*/
#ifndef __GC032A_REG_REGS_H__
#define __GC032A_REG_REGS_H__
#define SENSOR_ID_HIGH 0XF0
#define SENSOR_ID_LOW 0XF1
#define PAD_VB_HIZ_MODE 0XF2
#define SYNC_OUTPUT 0XF3
#define I2C_CONFIG 0XF4
#define PLL_MODE1 0XF7
#define PLL_MODE2 0XF8
#define CM_MODE 0XF9
#define ISP_DIV_MODE 0XFA
#define I2C_DEVICE_ID 0XFB
#define ANALOG_PWC 0XFC
#define ISP_DIV_MODE2 0XFD
#define RESET_RELATED 0XFE // Bit[7]: Software reset
// Bit[6]: cm reset
// Bit[5]: spi reset
// Bit[4]: CISCTL_restart_n
// Bit[3]: PLL_rst
// Bit[2:0]: page select
// 000:page0
// 001:page1
// 010:page2
// 011:page3
//----page0-----------------------------
#define P0_EXPOSURE_HIGH 0X03
#define P0_EXPOSURE_LOW 0X04
#define P0_HB_HIGH 0X05
#define P0_HB_LOW 0X06
#define P0_VB_HIGH 0X07
#define P0_VB_LOW 0X08
#define P0_ROW_START_HIGH 0X09
#define P0_ROW_START_LOW 0X0A
#define P0_COLUMN_START_HIGH 0X0B
#define P0_COLUMN_START_LOW 0X0C
#define P0_WINDOW_HEIGHT_HIGH 0X0D
#define P0_WINDOW_HEIGHT_LOW 0X0E
#define P0_WINDOW_WIDTH_HIGH 0X0F
#define P0_WINDOW_WIDTH_LOW 0X10
#define P0_SH_DELAY 0X11
#define P0_VS_ST 0X12
#define P0_VS_ET 0X13
#define P0_CISCTL_MODE1 0X17
#define P0_BLOCK_ENABLE_1 0X40
#define P0_AAAA_ENABLE 0X42
#define P0_SPECIAL_EFFECT 0X43
#define P0_SYNC_MODE 0X46
#define P0_GAIN_CODE 0X48
#define P0_DEBUG_MODE2 0X4C
#define P0_WIN_MODE 0X50
#define P0_OUT_WIN_Y1_HIGH 0X51
#define P0_OUT_WIN_Y1_LOW 0X52
#define P0_OUT_WIN_X1_HIGH 0X53
#define P0_OUT_WIN_X1_LOW 0X54
#define P0_OUT_WIN_HEIGHT_HIGH 0X55
#define P0_OUT_WIN_HEIGHT_LOW 0X56
#define P0_OUT_WIN_WIDTH_HIGH 0X57
#define P0_OUT_WIN_WIDTH_LOW 0X58
#define P0_GLOBAL_SATURATION 0XD0
#define P0_SATURATION_CB 0XD1
#define P0_SATURATION_CR 0XD2
#define P0_LUMA_CONTRAST 0XD3
#define P0_CONTRAST_CENTER 0XD4
#define P0_LUMA_OFFSET 0XD5
#define P0_FIXED_CB 0XDA
#define P0_FIXED_CR 0XDB
//----page3-----------------------------
#define P3_IMAGE_WIDTH_LOW 0X5B
#define P3_IMAGE_WIDTH_HIGH 0X5C
#define P3_IMAGE_HEIGHT_LOW 0X5D
#define P3_IMAGE_HEIGHT_HIGH 0X5E
#endif //__GC032A_REG_REGS_H__

View File

@@ -0,0 +1,401 @@
#ifndef _GC032A_SETTINGS_H_
#define _GC032A_SETTINGS_H_
#include <stdint.h>
#include <stdbool.h>
#include "esp_attr.h"
#include "gc032a_regs.h"
#define REG_DLY 0xffff
#define REGLIST_TAIL 0x0000
/*
* The default register settings, as obtained from OmniVision. There
* is really no making sense of most of these - lots of "reserved" values
* and such.
*
*/
static const uint16_t gc032a_default_regs[][2] = {
/*System*/
{0xf3, 0xff},
{0xf5, 0x06},
{0xf7, 0x01},
{0xf8, 0x03},
{0xf9, 0xce},
{0xfa, 0x00},
{0xfc, 0x02},
{0xfe, 0x02},
{0x81, 0x03},
{0xfe, 0x00},
{0x77, 0x64},
{0x78, 0x40},
{0x79, 0x60},
/*ANALOG & CISCTL*/
{0xfe, 0x00},
{0x03, 0x01},
{0x04, 0xce},
{0x05, 0x01},
{0x06, 0xad},
{0x07, 0x00},
{0x08, 0x10},
{0x0a, 0x00},
{0x0c, 0x00},
{0x0d, 0x01},
{0x0e, 0xe8}, // height 488
{0x0f, 0x02},
{0x10, 0x88}, // width 648
{0x17, 0x54},
{0x19, 0x08},
{0x1a, 0x0a},
{0x1f, 0x40},
{0x20, 0x30},
{0x2e, 0x80},
{0x2f, 0x2b},
{0x30, 0x1a},
{0xfe, 0x02},
{0x03, 0x02},
{0x05, 0xd7},
{0x06, 0x60},
{0x08, 0x80},
{0x12, 0x89},
/*blk*/
{0xfe, 0x00},
{0x18, 0x02},
{0xfe, 0x02},
{0x40, 0x22},
{0x45, 0x00},
{0x46, 0x00},
{0x49, 0x20},
{0x4b, 0x3c},
{0x50, 0x20},
{0x42, 0x10},
/*isp*/
{0xfe, 0x01},
{0x0a, 0xc5},
{0x45, 0x00},
{0xfe, 0x00},
{0x40, 0xff},
{0x41, 0x25},
{0x42, 0xcf},
{0x43, 0x10},
{0x44, 0x83},
{0x46, 0x23},
{0x49, 0x03},
{0x52, 0x02},
{0x54, 0x00},
{0xfe, 0x02},
{0x22, 0xf6},
/*Shading*/
{0xfe, 0x01},
{0xc1, 0x38},
{0xc2, 0x4c},
{0xc3, 0x00},
{0xc4, 0x32},
{0xc5, 0x24},
{0xc6, 0x16},
{0xc7, 0x08},
{0xc8, 0x08},
{0xc9, 0x00},
{0xca, 0x20},
{0xdc, 0x8a},
{0xdd, 0xa0},
{0xde, 0xa6},
{0xdf, 0x75},
/*AWB*/
{0xfe, 0x01},
{0x7c, 0x09},
{0x65, 0x06},
{0x7c, 0x08},
{0x56, 0xf4},
{0x66, 0x0f},
{0x67, 0x84},
{0x6b, 0x80},
{0x6d, 0x12},
{0x6e, 0xb0},
{0x86, 0x00},
{0x87, 0x00},
{0x88, 0x00},
{0x89, 0x00},
{0x8a, 0x00},
{0x8b, 0x00},
{0x8c, 0x00},
{0x8d, 0x00},
{0x8e, 0x00},
{0x8f, 0x00},
{0x90, 0x00},
{0x91, 0x00},
{0x92, 0xf4},
{0x93, 0xd5},
{0x94, 0x50},
{0x95, 0x0f},
{0x96, 0xf4},
{0x97, 0x2d},
{0x98, 0x0f},
{0x99, 0xa6},
{0x9a, 0x2d},
{0x9b, 0x0f},
{0x9c, 0x59},
{0x9d, 0x2d},
{0x9e, 0xaa},
{0x9f, 0x67},
{0xa0, 0x59},
{0xa1, 0x00},
{0xa2, 0x00},
{0xa3, 0x0a},
{0xa4, 0x00},
{0xa5, 0x00},
{0xa6, 0xd4},
{0xa7, 0x9f},
{0xa8, 0x55},
{0xa9, 0xd4},
{0xaa, 0x9f},
{0xab, 0xac},
{0xac, 0x9f},
{0xad, 0x55},
{0xae, 0xd4},
{0xaf, 0xac},
{0xb0, 0xd4},
{0xb1, 0xa3},
{0xb2, 0x55},
{0xb3, 0xd4},
{0xb4, 0xac},
{0xb5, 0x00},
{0xb6, 0x00},
{0xb7, 0x05},
{0xb8, 0xd6},
{0xb9, 0x8c},
/*CC*/
{0xfe, 0x01},
{0xd0, 0x40},
{0xd1, 0xf8},
{0xd2, 0x00},
{0xd3, 0xfa},
{0xd4, 0x45},
{0xd5, 0x02},
{0xd6, 0x30},
{0xd7, 0xfa},
{0xd8, 0x08},
{0xd9, 0x08},
{0xda, 0x58},
{0xdb, 0x02},
{0xfe, 0x00},
/*Gamma*/
{0xfe, 0x00},
{0xba, 0x00},
{0xbb, 0x04},
{0xbc, 0x0a},
{0xbd, 0x0e},
{0xbe, 0x22},
{0xbf, 0x30},
{0xc0, 0x3d},
{0xc1, 0x4a},
{0xc2, 0x5d},
{0xc3, 0x6b},
{0xc4, 0x7a},
{0xc5, 0x85},
{0xc6, 0x90},
{0xc7, 0xa5},
{0xc8, 0xb5},
{0xc9, 0xc2},
{0xca, 0xcc},
{0xcb, 0xd5},
{0xcc, 0xde},
{0xcd, 0xea},
{0xce, 0xf5},
{0xcf, 0xff},
/*Auto Gamma*/
{0xfe, 0x00},
{0x5a, 0x08},
{0x5b, 0x0f},
{0x5c, 0x15},
{0x5d, 0x1c},
{0x5e, 0x28},
{0x5f, 0x36},
{0x60, 0x45},
{0x61, 0x51},
{0x62, 0x6a},
{0x63, 0x7d},
{0x64, 0x8d},
{0x65, 0x98},
{0x66, 0xa2},
{0x67, 0xb5},
{0x68, 0xc3},
{0x69, 0xcd},
{0x6a, 0xd4},
{0x6b, 0xdc},
{0x6c, 0xe3},
{0x6d, 0xf0},
{0x6e, 0xf9},
{0x6f, 0xff},
/*Gain*/
{0xfe, 0x00},
{0x70, 0x50},
/*AEC*/
{0xfe, 0x00},
{0x4f, 0x01},
{0xfe, 0x01},
{0x0d, 0x00},
{0x12, 0xa0},
{0x13, 0x3a},
{0x44, 0x04},
{0x1f, 0x30},
{0x20, 0x40},
{0x26, 0x9a},
{0x3e, 0x20},
{0x3f, 0x2d},
{0x40, 0x40},
{0x41, 0x5b},
{0x42, 0x82},
{0x43, 0xb7},
{0x04, 0x0a},
{0x02, 0x79},
{0x03, 0xc0},
/*measure window*/
{0xfe, 0x01},
{0xcc, 0x08},
{0xcd, 0x08},
{0xce, 0xa4},
{0xcf, 0xec},
/*DNDD*/
{0xfe, 0x00},
{0x81, 0xb8},
{0x82, 0x12},
{0x83, 0x0a},
{0x84, 0x01},
{0x86, 0x50},
{0x87, 0x18},
{0x88, 0x10},
{0x89, 0x70},
{0x8a, 0x20},
{0x8b, 0x10},
{0x8c, 0x08},
{0x8d, 0x0a},
/*Intpee*/
{0xfe, 0x00},
{0x8f, 0xaa},
{0x90, 0x9c},
{0x91, 0x52},
{0x92, 0x03},
{0x93, 0x03},
{0x94, 0x08},
{0x95, 0x44},
{0x97, 0x00},
{0x98, 0x00},
/*ASDE*/
{0xfe, 0x00},
{0xa1, 0x30},
{0xa2, 0x41},
{0xa4, 0x30},
{0xa5, 0x20},
{0xaa, 0x30},
{0xac, 0x32},
/*YCP*/
{0xfe, 0x00},
{0xd1, 0x3c},
{0xd2, 0x3c},
{0xd3, 0x38},
{0xd6, 0xf4},
{0xd7, 0x1d},
{0xdd, 0x73},
{0xde, 0x84},
/*Banding*/
{0xfe, 0x00},
{0x05, 0x01},
{0x06, 0xad},
{0x07, 0x00},
{0x08, 0x10},
{0xfe, 0x01},
{0x25, 0x00},
{0x26, 0x9a},
{0x27, 0x01},
{0x28, 0xce},
{0x29, 0x02},
{0x2a, 0x68},
{0x2b, 0x02},
{0x2c, 0x68},
{0x2d, 0x07},
{0x2e, 0xd2},
{0x2f, 0x0b},
{0x30, 0x6e},
{0x31, 0x0e},
{0x32, 0x70},
{0x33, 0x12},
{0x34, 0x0c},
{0x3c, 0x30},
/*Analog&Cisctl*/
{0xfe, 0x00},
{0x05, 0x01},
{0x06, 0xa0},
{0x07, 0x00},
{0x08, 0x20},
{0x0a, 0x78},
{0x0c, 0xa0},
{0x0d, 0x00}, //window_height [8]
{0x0e, 0xf8}, //window_height [7:0] 248
{0x0f, 0x01}, //window_width [9:8]
{0x10, 0x48}, //window_width [7:0] 328
{0x55, 0x00},
{0x56, 0xf0}, // 240
{0x57, 0x01},
{0x58, 0x40}, // 320
/*SPI*/
{0xfe, 0x03},
{0x5b, 0x40},
{0x5c, 0x01},
{0x5d, 0xf0},
{0x5e, 0x00},
/*AEC*/
{0xfe, 0x01},
{0x25, 0x00}, //step
{0x26, 0x63},
{0x27, 0x01},
{0x28, 0x29},
{0x29, 0x01},
{0x2a, 0x29},
{0x2b, 0x01},
{0x2c, 0x29},
{0x2d, 0x01},
{0x2e, 0x29},
{0x2f, 0x01},
{0x30, 0x29},
{0x31, 0x01},
{0x32, 0x29},
{0x33, 0x01},
{0x34, 0x29},
{0x3c, 0x00},
/*measure window*/
{0xfe, 0x01},
{0xcc, 0x04},
{0xcd, 0x04},
{0xce, 0x72},
{0xcf, 0x52},
{REGLIST_TAIL, 0x00},
};
#endif

View File

@@ -0,0 +1,27 @@
#ifndef __GC2145_H__
#define __GC2145_H__
#include "sensor.h"
/**
* @brief Detect sensor pid
*
* @param slv_addr SCCB address
* @param id Detection result
* @return
* 0: Can't detect this sensor
* Nonzero: This sensor has been detected
*/
int gc2145_detect(int slv_addr, sensor_id_t *id);
/**
* @brief initialize sensor function pointers
*
* @param sensor pointer of sensor
* @return
* Always 0
*/
int gc2145_init(sensor_t *sensor);
#endif // __GC2145_H__

Some files were not shown because too many files have changed in this diff Show More