Automatic tone mapping (#996)

* Define structures

* More structures

* Add UI for automatic tone mapping settings

* Better explanation

* Add automatic tone mapping config handler

* Add communication to video processor

* Implementation of automatic tone mapping control

* Automatic tone mapping control: add timeout option for turning tone mapping off

* Update language file's (#998)

* Update changelog

---------

Co-authored-by: Roman Astafev <39379875+AstaRom@users.noreply.github.com>

CHANGELOG.md

@@ -1,3 +1,39 @@
+- Automatic tone mapping (#998), HyperHDR v21
+- Add Ubuntu 24.10 and Fedora 41 (#1001), HyperHDR v21
+- Add support for "unicam image" RPi HDMI capture card and UYVY format (#889), HyperHDR v21
+- Breaking change: remove LUT from installer and create it dynamically (#994), HyperHDR v21
+- Fix macOS 15 dark theme menu icons (#988), HyperHDR v21
+- Add P010 HDR10 video format support (2024-11-24) (#968), HyperHDR v21
+- DirectX multi-monitor support (#966), HyperHDR v21
+- macOS 15 sequoia: migrate to ScreenCaptureKit (#984), HyperHDR v21
+- Add Skydimo "support" (#985), HyperHDR v21
+- New LUT calibration based on mp4 test videos (part I) (#896), HyperHDR v21
+- feat: updated rpi_ws281x submodule, thanks @andreasvikke (#974), HyperHDR v21
+- DX grabber: stick to user specified device selection (#961), HyperHDR v21
+- Fix LED colors adjustments by configuration name (#956), HyperHDR v21
+- Add autoresume feature for the macOS software grabber (#879), HyperHDR v21
+- Add NV12 image format support for flatbuffers (#920), HyperHDR v21
+- Update language file's Thanks @AstaRom (#883, #923, #998), HyperHDR v21
+- Update flatbuffers to v24.3.25 (#875), HyperHDR v21
+- Add cache_cleaner for Github Action (#910), HyperHDR v21
+- New build.sh script for the HyperHDR build process (#904), HyperHDR v21
+- Remove Alsa libs from CMake recipe, HyperHDR v21
+- Remove more unnecessary libraries from installers, HyperHDR v21
+- Remove libasound from installers, HyperHDR v21
+- Fix restoring calibration settings (#874), HyperHDR v21
+- Reorganize HyperHDR libs (#887), HyperHDR v21
+- Fix macOS image alignment (#838), HyperHDR v21
+- Remove hyperhdr-remote app. Reason: #856, HyperHDR v21
+- Refactoring of the FlatBuffers client and server (#880), HyperHDR v21
+- Remove mbedtls. Use OpenSSL (#877), HyperHDR v21
+- Removal of QT SQL. Migrate to the SQLite library. (#872), HyperHDR v21
+- New smoothing option for sound effects, fix ArchLinux installer, refactoring of the smoothing module and music effects (#871), HyperHDR v21
+- Removal of QT D-Bus, switch to sdbus-cpp (#864), HyperHDR v21
+- Dependency reductions: removal of heavy QT Gui lib (#861), HyperHDR v21
+- Fix DirectX capturing after new Systray changes, HyperHDR v21
+- New independed systray. Dependency reductions thanks to removal of QWidget (#852), HyperHDR v21
+- Fix: lut calibration for grabberless builds (#840), HyperHDR v21
+- Fix: close LED device on correct Thread (#803), HyperHDR v21
 - Fedora uses xz as the package name, not xz-utils #771 Thanks @hsmalley (v20 beta2 🆕)
 - Fix LUT loading bug to reduce memory usage by 96MB #766 (v20 beta2 🆕)
 - Fix background music effect #761 (v20 beta2 🆕)

README.md

@@ -17,14 +17,15 @@ Open source ambient lighting implementation for television and music sets based
 * Support for USB grabbers under Linux, Windows 10, macOS
 * Pipewire/Portal hardware-accelerated screen capturer for Linux/Wayland
 * DirectX screen grabber with pixel and vertex shader processing acceleration for Windows 10/11
-* DirectX screen grabber supports native Windows HDR modes like DXGI_FORMAT_R16G16B16A16_FLOAT
+* DirectX screen grabber supports native Windows HDR modes like DXGI_FORMAT_R16G16B16A16_FLOAT and multi-monitor ( :new: HyperHDR v21)
 * Dynamic video cache buffers. Now Rpi can process even 1080p120 NV12 stream without any size decimation
 * Built-in audio visualization effects using **spectrum analysis**
 * MQTT support for IoT
 * Entertainment API v2: per-channel support for Philips Gradient Strip and others
-* Automatic LUT calibration detects grabber model specific properties for the best quality of HDR/SDR
+* Automatic LUT calibration detects grabber model specific properties for the best quality of HDR/SDR using MP4 test files ( :new: HyperHDR v21)
 * Optimized multi-instances. You can use for example your TV LED strip and multiple WLED or Philips Hue light sources.
 * Built-in latency benchmark for USB grabbers
+* support for high-quality P010 video format ( :new: HyperHDR v21)
 * easy LED strip geometry editing process, automatic or manual with mouse and context menu per single LED
 * Automatic signal detection with smart learning capability for USB grabbers
 * SK6812 RGBW: advanced calibration of the white color channel for [HyperSerialEsp8266](https://github.com/awawa-dev/HyperSerialEsp8266), [HyperSerialESP32](https://github.com/awawa-dev/HyperSerialESP32), [HyperSPI](https://github.com/awawa-dev/HyperSPI), [HyperSerialPico](https://github.com/awawa-dev/HyperSerialPico)

cmake/packages.cmake

@@ -151,7 +151,7 @@ endif()
 # OSX dmg generator
 if ( APPLE )
 	SET ( CPACK_GENERATOR "DragNDrop")
-	SET ( CPACK_DMG_FORMAT "UDBZ" )
+	SET ( CPACK_DMG_FORMAT "ULMO" )
 
 	unset(CPACK_PACKAGE_ICON)
 	set(CPACK_PACKAGE_ICON "${CMAKE_CURRENT_SOURCE_DIR}/cmake/osx/Hyperhdr.icns")

include/base/AutomaticToneMapping.h

@@ -0,0 +1,83 @@
+/* AutomaticToneMapping.h
+*
+*  MIT License
+*
+*  Copyright (c) 2020-2024 awawa-dev
+*
+*  Project homesite: https://github.com/awawa-dev/HyperHDR
+*
+*  Permission is hereby granted, free of charge, to any person obtaining a copy
+*  of this software and associated documentation files (the "Software"), to deal
+*  in the Software without restriction, including without limitation the rights
+*  to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+*  copies of the Software, and to permit persons to whom the Software is
+*  furnished to do so, subject to the following conditions:
+*
+*  The above copyright notice and this permission notice shall be included in all
+*  copies or substantial portions of the Software.
+
+*  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+*  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+*  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+*  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+*  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+*  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+*  SOFTWARE.
+ */
+
+#pragma once
+
+#ifndef PCH_ENABLED
+#endif
+
+#include <utils/Logger.h>
+
+class AutomaticToneMapping
+{	
+public:
+	struct ToneMappingThresholds;
+
+	AutomaticToneMapping();
+
+	AutomaticToneMapping* prepare();
+	void finilize();
+	void setConfig(bool enabled, const ToneMappingThresholds& newConfig, int timeInSec, int timeToDisableInMSec);
+	void setToneMapping(bool enabled);
+
+	constexpr uint8_t checkY(uint8_t y)
+	{
+		if (y > _running.y) _running.y = y;
+		return y;
+	}
+
+	constexpr uint8_t checkU(uint8_t u)
+	{
+		if (u > _running.u) _running.u = u;
+		return u;
+	}
+
+	constexpr uint8_t checkV(uint8_t v)
+	{
+		if (v > _running.v) _running.v = v;
+		return v;
+	}
+
+	struct ToneMappingThresholds {
+		uint8_t y;
+		uint8_t u;
+		uint8_t v;
+	};
+
+private:
+	bool _enabled;
+	int _timeInSec;
+	int _timeToDisableInMSec;
+
+	ToneMappingThresholds _config, _running;
+
+	bool _modeSDR;
+	long _startedTime;
+	long _endingTime;
+	Logger* _log;
+
+};

include/base/Grabber.h

@@ -20,6 +20,7 @@
 #include <utils/LutLoader.h>
 #include <base/DetectionManual.h>
 #include <base/DetectionAutomatic.h>
+#include <base/AutomaticToneMapping.h>
 #include <performance-counters/PerformanceCounters.h>
 
 #if  defined(_WIN32) || defined(WIN32)
@@ -125,6 +126,9 @@ class Grabber : public DetectionAutomatic, public DetectionManual, protected Lut
 
 	QString	getConfigurationPath();
 
+	void setAutomaticToneMappingConfig(bool enabled, const AutomaticToneMapping::ToneMappingThresholds& newConfig, int timeInSec, int timeToDisableInMSec);
+	void setAutoToneMappingCurrentStateEnabled(bool enabled);
+
 	struct DevicePropertiesItem
 	{
 		int		x, y, fps, fps_a, fps_b, input;
@@ -272,6 +276,7 @@ public slots:
 	bool		_signalDetectionEnabled;
 	bool		_signalAutoDetectionEnabled;
 	QSemaphore  _synchro;
+	AutomaticToneMapping _automaticToneMapping;
 };
 
 bool sortDevicePropertiesItem(const Grabber::DevicePropertiesItem& v1, const Grabber::DevicePropertiesItem& v2);

include/grabber/linux/v4l2/V4L2Worker.h

@@ -10,6 +10,7 @@
 #include <utils/PixelFormat.h>
 #include <base/Grabber.h>
 #include <utils/Components.h>
+#include <base/AutomaticToneMapping.h>
 
 #include <turbojpeg.h>
 
@@ -32,7 +33,7 @@ class V4L2Worker : public  QThread
 		unsigned	__cropBottom, unsigned __cropRight,
 		quint64		__currentFrame, qint64 __frameBegin,
 		int			__hdrToneMappingEnabled, uint8_t* __lutBuffer,
-		bool		__qframe, bool __directAccess, QString __deviceName);
+		bool		__qframe, bool __directAccess, QString __deviceName, AutomaticToneMapping* __automaticToneMapping);
 
 	void startOnThisThread();
 	void run() override;
@@ -76,6 +77,7 @@ class V4L2Worker : public  QThread
 	bool		_qframe;
 	bool		_directAccess;
 	QString		_deviceName;
+	AutomaticToneMapping* _automaticToneMapping;
 };
 
 class V4L2WorkerManager : public  QObject

include/grabber/osx/AVF/AVFWorker.h

@@ -17,6 +17,7 @@
 #include <utils/PixelFormat.h>
 #include <base/Grabber.h>
 #include <utils/Components.h>
+#include <base/AutomaticToneMapping.h>
 
 
 
@@ -38,7 +39,7 @@ class AVFWorker : public  QThread
 		unsigned	__cropBottom, unsigned __cropRight,
 		quint64		__currentFrame, qint64 __frameBegin,
 		int			__hdrToneMappingEnabled, uint8_t* __lutBuffer, bool __qframe,
-		bool		__directAccess, QString __deviceName);
+		bool		__directAccess, QString __deviceName, AutomaticToneMapping* __automaticToneMapping);
 
 	void startOnThisThread();
 	void run() override;
@@ -78,6 +79,7 @@ class AVFWorker : public  QThread
 	bool		_qframe;
 	bool		_directAccess;
 	QString		_deviceName;
+	AutomaticToneMapping* _automaticToneMapping;
 };
 
 class AVFWorkerManager : public  QObject

include/grabber/windows/MF/MFWorker.h

@@ -7,6 +7,7 @@
 #include <utils/PixelFormat.h>
 #include <base/Grabber.h>
 #include <utils/Components.h>
+#include <base/AutomaticToneMapping.h>
 
 #include <turbojpeg.h>
 
@@ -27,7 +28,7 @@ class MFWorker : public  QThread
 		unsigned	__cropBottom, unsigned __cropRight,
 		quint64		__currentFrame, qint64 __frameBegin,
 		int			__hdrToneMappingEnabled, uint8_t* __lutBuffer, bool __qframe,
-		bool		__directAccess, QString __deviceName);
+		bool		__directAccess, QString __deviceName, AutomaticToneMapping* __automaticToneMapping);
 
 	void startOnThisThread();
 	void run() override;
@@ -69,6 +70,7 @@ class MFWorker : public  QThread
 	bool		_qframe;
 	bool		_directAccess;
 	QString		_deviceName;
+	AutomaticToneMapping* _automaticToneMapping;
 };
 
 class MFWorkerManager : public  QObject

include/lut-calibrator/BestResult.h

@@ -80,6 +80,7 @@ struct BestResult
 		double upYLimit = 0;
 		double downYLimit = 0;
 		double yShift = 0;
+		byte3  yuvRange = {};
 		bool isSourceP010 = false;
 	} signal;
 
@@ -124,6 +125,7 @@ struct BestResult
 		out << "bestResult.signal.yShift = " << std::to_string(signal.yShift) << ";" << std::endl;
 		out << "bestResult.signal.isSourceP010 = " << std::to_string(signal.isSourceP010) << ";" << std::endl;
 		out << "bestResult.minError = " << std::to_string(std::round(minError * 100.0) / 30000.0) << ";" << std::endl;
+		out << "bestResult.signal.yuvRange = byte3{ " << std::to_string(signal.yuvRange[0]) << ", " << std::to_string(signal.yuvRange[1]) << ", " << std::to_string(signal.yuvRange[2]) << "};" << std::endl;
 		out << "*/" << std::endl;
 	}
 };

include/lut-calibrator/BoardUtils.h

@@ -84,6 +84,7 @@ namespace BoardUtils
 		double _yShift = 0;
 		double _downYLimit = 0;
 		double _upYLimit = 0;
+		byte3  _rangeYUV = {};
 
 	public:
 		CapturedColors() = default;
@@ -96,7 +97,7 @@ namespace BoardUtils
 		bool areAllCaptured();
 		void finilizeBoard();
 		static void correctYRange(double3& yuv, double yRange, double upYLimit, double downYLimit, double yShift);
-		void getSignalParams(double& yRange, double& upYLimit, double& downYLimit, double& yShift);
+		void getSignalParams(double& yRange, double& upYLimit, double& downYLimit, double& yShift, byte3& rangeYUV);
 		void setCaptured(int index);
 		void setRange(YuvConverter::COLOR_RANGE range);
 		YuvConverter::COLOR_RANGE getRange() const;

include/utils/FrameDecoder.h

@@ -4,6 +4,7 @@
 #include <image/ColorRgb.h>
 #include <image/Image.h>
 #include <utils/PixelFormat.h>
+#include <base/AutomaticToneMapping.h>
 
 // some stuff for HDR tone mapping
 #define LUT_INDEX(y,u,v) ((y + (u<<8) + (v<<16))*3)
@@ -24,7 +25,7 @@ class FrameDecoder
 
 	static void processQImage(
 		const uint8_t* data, const uint8_t* dataUV, int width, int height, int lineLength,
-		const PixelFormat pixelFormat, const uint8_t* lutBuffer, Image<ColorRgb>& outputImage, bool toneMapping = true);
+		const PixelFormat pixelFormat, const uint8_t* lutBuffer, Image<ColorRgb>& outputImage, bool toneMapping = true, AutomaticToneMapping* automaticToneMapping = nullptr);
 
 	static void processSystemImageBGRA(Image<ColorRgb>& image, int targetSizeX, int targetSizeY,
 									   int startX, int startY,

include/utils/settings.h

@@ -33,6 +33,7 @@ namespace settings {
 		RAWUDPSERVER,
 		PROTOSERVER,
 		MQTT,
+		AUTOTONEMAPPING,
 		INVALID
 	};
 
@@ -65,6 +66,7 @@ namespace settings {
 		case type::RAWUDPSERVER:  return "rawUdpServer";
 		case type::PROTOSERVER:   return "protoServer";
 		case type::MQTT:          return "mqtt";
+		case type::AUTOTONEMAPPING: return "automaticToneMapping";
 		default:                  return "invalid";
 		}
 	}
@@ -96,6 +98,7 @@ namespace settings {
 		else if (type == "rawUdpServer")         return type::RAWUDPSERVER;
 		else if (type == "protoServer")          return type::PROTOSERVER;
 		else if (type == "mqtt")                 return type::MQTT;
+		else if (type == "automaticToneMapping") return type::AUTOTONEMAPPING;
 		else                                     return type::INVALID;
 	}
 }