芯片资讯
你的位置：Megawin(笙泉科技)MCU-IC芯片全系列-亿配芯城 > 芯片资讯 > Camera算法集成实现指南
Camera算法集成实现指南
- 发布日期：2024-01-26 08:23 点击次数：67
算法概览
为了给用户提供更好的成像效果，现在的手机都会接入一些第三方的图像处理算法。MTK平台的HAL3也在P2这一层提供接入的plugin。按图像处理算法需要的帧数和摄像头数量，大体可以分为三类：
单帧算法：
常见的单帧算法有：美颜算法(瘦脸、磨皮、大眼)、广角镜头畸变校正算法、附加表情算法、单摄背景虚化算法(伪双摄算法)等等，仅需单帧图像输入的算法都属于单帧算法。一般情况下，输入一帧图像，算法处理完输出一帧处理后的图像。
多帧算法：
常见的多帧算法有：MFNR（多帧降噪）、HDR（高动态范围）等等，需要连续多帧图像输入的算法都属于多帧算法。一般情况下，输入连续多帧图像，算法处理完输出一帧处理后的图像。
双摄算法：
最常见的双摄算法是双摄景深算法或者叫双摄背景虚化算法，除此之外，也有彩色+黑白用于增强夜拍效果的双摄算法。单帧算法和多帧算法仅需要获取一个摄像头的图像。而双摄算法需要获取主、辅两个摄像头的图像，并且一般还会要求主、辅摄像头同步。分别获取主、辅摄像头的两帧同步图像，处理后输出一帧主摄图像，用户也仅能看到主摄图像。
根据这个大体上的分类，MTK HAL算法集成系列文章共三篇：
MTK HAL算法集成之单帧算法
MTK HAL算法集成之多帧算法
MTK HAL算法集成之双摄算法
本文是其中的第一篇。这个系列文章均基于Android 9.0，MT6763平台，HAL版本是HAL3。
一、算法集成前的准备
在开展集成工作之前，首先要对算法有一个基本的评估，并且对于集成也应有一定的要求。
1. 1 算法要求及评估
处理效果好，不能比竞品差，超过竞品更佳。（这条和camera调试的主观效果一样，主观性较强，往往一厢情愿，具体看项目要求吧）
各个场景及压力测试下效果稳定。
处理后照片无色差、锐度和饱和度无损失，或者损失在可接受范围。
达到可接受的分辨率，最好可达到摄像头的最大分辨率。
处理时间越快越好，不超过竞品时间、不超过项目和产品的目标时间。
无内存泄露，占用内存少。
提供必要的集成说明文档，包括算法类型、输入及输出图像要求、输入参数要求等等。
注意：如果有条件，处理时间、内存占用、分辨率等等可量化的指标可要求算法提供方给出具体的参考数据，以便集成完后测试验证。
1.2 算法集成要求
编译时可根据项目控制是否集成算法。
运行时可以用参数控制是否启用算法。
集成算法库正常运行、压力测试下效果稳定、无内存泄露。
1.3 算法集成的步骤
(1).根据算法选择feature类型，如果与MTK提供的feature不能对号入座，则需要添加自定义feature。
(2).将算法对应的feature类型添加到scenario配置表。
(3).根据算法选择plugin类型，编写CPP文件实现plugin，挂载算法。
(4).如果算法不能复用Android和MTK提供的metadata，则还需要为算法配置自定义的metadata以便APP控制是否启用算法。
首先，我准备了一个libwatermark.so，它仅仅实现了一个添加水印的功能，用它来模拟第三方的单帧算法库。如果想了解添加水印的实现代码，可以参考我另外一篇文章：Android 实现图片加水印或logo。接下来，我们就按照集成步骤，逐步详细讲解。
二、 为算法选择feature
2.1 MTK提供的feature
MTK在mtk_feature_type.h和customer_feature_type.h已经提供了一些feature。 vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/mtk/mtk_feature_type.h：
NO_FEATURE_NORMAL = 0ULL，
// MTK (bit 0-31)
MTK_FEATURE_MFNR = 1ULL << 0，
MTK_FEATURE_HDR = 1ULL << 1，
MTK_FEATURE_REMOSAIC = 1ULL << 2，
MTK_FEATURE_ABF = 1ULL << 3，
MTK_FEATURE_NR = 1ULL << 4，
MTK_FEATURE_FB = 1ULL << 5，
MTK_FEATURE_CZ = 1ULL << 6，
MTK_FEATURE_DRE = 1ULL << 7，
MTK_FEATURE_DEPTH = 1ULL << 8，
MTK_FEATURE_BOKEH = 1ULL << 9，
MTK_FEATURE_VSDOF = (MTK_FEATURE_DEPTH|MTK_FEATURE_BOKEH)，
MTK_FEATURE_FSC = 1ULL << 10，
MTK_FEATURE_3DNR = 1ULL << 11，
MTK_FEATURE_EIS = 1ULL << 12，
MTK_FEATURE_AINR = 1ULL << 13，
MTK_FEATURE_DUAL_YUV = 1ULL << 14，
MTK_FEATURE_DUAL_HWDEPTH = 1ULL << 15，
MTK_FEATURE_AIS = 1ULL << 16，
MTK_FEATURE_HFG = 1ULL << 17，
MTK_FEATURE_DCE = 1ULL << 18，
vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/customer/customer_feature_type.h：
// ThirdParty (bit 32-63)
TP_FEATURE_HDR = 1ULL << 32，
TP_FEATURE_MFNR = 1ULL << 33，
TP_FEATURE_EIS = 1ULL << 34，
TP_FEATURE_FB = 1ULL << 35，
TP_FEATURE_FILTER = 1ULL << 36，
TP_FEATURE_DEPTH = 1ULL << 37，
TP_FEATURE_BOKEH = 1ULL << 38，
TP_FEATURE_VSDOF = (TP_FEATURE_DEPTH|TP_FEATURE_BOKEH)，
TP_FEATURE_FUSION = 1ULL << 39，
TP_FEATURE_HDR_DC = 1ULL << 40， // used by DualCam
TP_FEATURE_DUAL_YUV = 1ULL << 41，
TP_FEATURE_DUAL_HWDEPTH = 1ULL << 42，
TP_FEATURE_PUREBOKEH = 1ULL << 43，
TP_FEATURE_RAW_HDR = 1ULL << 44，
TP_FEATURE_RELIGHTING = 1ULL << 45，
MTK提供的这些feature可以满足绝大多数算法的集成，在可以对号入座的情况下，我们直接使用已有feature即可。如果不能够满足我们的要求，可以参考下节内容添加新的feature。
2.2 添加自定义feature
本来单帧算法对应的feature可以选择MTK提供的MTK_FEATURE_FB和TP_FEATURE_FB，但是为了讲解如何添加新feature，我们选择添加一个自定义feature：TP_FEATURE_WATERMARK。
vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/customer/customer_feature_type.h：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/customer/customer_feature_type.h b/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/customer/customer_feature_type.h
old mode 100644
new mode 100755
index a41fd864f5..17bc35eea8
--- a/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/customer/customer_feature_type.h
+++ b/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/customer/customer_feature_type.h
& -59，6 +59，7 & enum eFeatureIndexCustomer {
TP_FEATURE_PUREBOKEH = 1ULL << 43，
TP_FEATURE_RAW_HDR = 1ULL << 44，
TP_FEATURE_RELIGHTING = 1ULL << 45，
+ TP_FEATURE_WATERMARK = 1ULL << 46，
// TODO: reserve for customer feature index (bit 32-63)
};
vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/CaptureFeature_Common.cpp：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/CaptureFeature_Common.cpp b/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/CaptureFeature_Common.cpp
old mode 100644
new mode 100755
index e32f80a609..47273b01c7
--- a/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/CaptureFeature_Common.cpp
+++ b/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/CaptureFeature_Common.cpp
& -599，6 +599，7 & const char* FeatID2Name(FeatureID_T fid)
case FID_FUSION_3RD_PARTY: return "fusion_3rd_party";
case FID_PUREBOKEH_3RD_PARTY: return "purebokeh_3rd_party";
case FID_RELIGHTING_3RD_PARTY: return "relighting_3rd_party";
+ case FID_WATERMARK_3RD_PARTY: return "watermark_3rd_party";
default: return "unknown";
};
vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/YUVNode.cpp：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/YUVNode.cpp b/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/YUVNode.cpp
index 8bb794ba02..d4343aaccf 100755
--- a/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/YUVNode.cpp
+++ b/vendor/mediatek/proprietary/hardware/mtkcam3/feature/core/featurePipe/capture/nodes/YUVNode.cpp
& -779，7 +779，8 & MBOOL YUVNode::onInit()
featId = FID_FB_3RD_PARTY;
else if (rProperty.mFeatures & TP_FEATURE_RELIGHTING)
featId = FID_RELIGHTING_3RD_PARTY;
-
+ else if (rProperty.mFeatures & TP_FEATURE_WATERMARK)
+ featId = FID_WATERMARK_3RD_PARTY;
if (featId != NULL_FEATURE) {
MY_LOGD_IF(mLogLevel， "%s finds plugin:%s， priority:%d"，
vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/feature/featurePipe/ICaptureFeaturePipe.h：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/feature/featurePipe/ICaptureFeaturePipe.h b/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/feature/featurePipe/ICaptureFeaturePipe.h
old mode 100644
new mode 100755
index 2f1ad8a665..ab47aae456
--- a/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/feature/featurePipe/ICaptureFeaturePipe.h
+++ b/vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/feature/featurePipe/ICaptureFeaturePipe.h
& -172，6 +172，7 & enum CaptureFeatureFeatureID {
FID_FUSION_3RD_PARTY，
FID_PUREBOKEH_3RD_PARTY，
FID_RELIGHTING_3RD_PARTY，
+ FID_WATERMARK_3RD_PARTY，
NUM_OF_FEATURE，
NULL_FEATURE = 0xFF，
};
vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/hwnode/p2/P2_CaptureProcessor.cpp：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/hwnode/p2/P2_CaptureProcessor.cpp b/vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/hwnode/p2/P2_CaptureProcessor.cpp old mode 100644 new mode 100755 index cc1dc549fd..00559cbc30 --- a/vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/hwnode/p2/P2_CaptureProcessor.cpp +++ b/vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/hwnode/p2/P2_CaptureProcessor.cpp & -428，6 +428，9 & MBOOL CaptureProcessor::onEnque(const sp&pP2Frame) pCapRequest->addFeature(FID_HFG); if (feature & MTK_FEATURE_DCE) pCapRequest->addFeature(FID_DCE); + if (feature & TP_FEATURE_WATERMARK) + pCapRequest->addFeature(FID_WATERMARK_3RD_PARTY); + } }
三、 将算法对应的feature添加到scenario配置表
在我们打开camera进行预览和拍照的时候，MTK HAL3会执行vendor/mediatek/proprietary/hardware/mtkcam3/pipeline/policy/FeatureSettingPolicy.cpp的代码，会分别调用 vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/scenario_mgr.cpp的 get_streaming_scenario函数和get_capture_scenario函数。它们会读取一个scenario的feature配置表，遍历所有的feature，决定哪些feature会被执行。这个配置表中有许多的scenario，一个scenario可能对应多个feature。因此添加自定义feature后，还需将自定义的feature添加到配置表中。MTK feature 对应的配置表是 gMtkScenarioFeaturesMaps，customer feature 对应的配置表是 gCustomerScenarioFeaturesMaps。
vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/customer_scenario_mgr.cpp：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/customer_scenario_mgr.cpp b/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/customer_scenario_mgr.cpp old mode 100644 new mode 100755 index f8d081e433..577f85797e --- a/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/customer_scenario_mgr.cpp +++ b/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/customer_scenario_mgr.cpp & -93，30 +93，30 & using namespace NSCam::v3::pipeline::policy::scenariomgr; // #define (key feature | post-processing features | ...) // // single cam capture feature combination -#define TP_FEATURE_COMBINATION_SINGLE (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB) -#define TP_FEATURE_COMBINATION_HDR (TP_FEATURE_HDR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB) -#define TP_FEATURE_COMBINATION_AINR (MTK_FEATURE_AINR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB) -#define TP_FEATURE_COMBINATION_MFNR (MTK_FEATURE_MFNR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB) -#define TP_FEATURE_COMBINATION_REMOSAIC (MTK_FEATURE_REMOSAIC| MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB) +#define TP_FEATURE_COMBINATION_SINGLE (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_WATERMARK) +#define TP_FEATURE_COMBINATION_HDR (TP_FEATURE_HDR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_WATERMARK) +#define TP_FEATURE_COMBINATION_AINR (MTK_FEATURE_AINR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_WATERMARK) +#define TP_FEATURE_COMBINATION_MFNR (MTK_FEATURE_MFNR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_WATERMARK) +#define TP_FEATURE_COMBINATION_REMOSAIC (MTK_FEATURE_REMOSAIC| MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_WATERMARK) #define TP_FEATURE_COMBINATION_CSHOT (NO_FEATURE_NORMAL | MTK_FEATURE_CZ| MTK_FEATURE_HFG) -#define TP_FEATURE_COMBINATION_YUV_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| TP_FEATURE_FB) -#define TP_FEATURE_COMBINATION_RAW_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| TP_FEATURE_FB) +#define TP_FEATURE_COMBINATION_YUV_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| TP_FEATURE_FB| TP_FEATURE_WATERMARK) +#define TP_FEATURE_COMBINATION_RAW_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| TP_FEATURE_FB| TP_FEATURE_WATERMARK) #define TP_FEATURE_COMBINATION_PRO (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE) -#define TP_FEATURE_COMBINATION_SUPER_NIGHT_RAW_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| TP_FEATURE_FB) +#define TP_FEATURE_COMBINATION_SUPER_NIGHT_RAW_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| TP_FEATURE_FB| TP_FEATURE_WATERMARK) // dual cam capture feature combination // the VSDOF means the combination of Bokeh feature and Depth feature -#define TP_FEATURE_COMBINATION_TP_VSDOF (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_VSDOF) -#define TP_FEATURE_COMBINATION_TP_VSDOF_HDR (TP_FEATURE_HDR_DC | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_VSDOF) -#define TP_FEATURE_COMBINATION_TP_VSDOF_MFNR (MTK_FEATURE_MFNR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_VSDOF) -#define TP_FEATURE_COMBINATION_TP_FUSION (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_FUSION) -#define TP_FEATURE_COMBINATION_TP_PUREBOKEH (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_PUREBOKEH) +#define TP_FEATURE_COMBINATION_TP_VSDOF (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_VSDOF| TP_FEATURE_WATERMARK) +#define TP_FEATURE_COMBINATION_TP_VSDOF_HDR (TP_FEATURE_HDR_DC | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_VSDOF| TP_FEATURE_WATERMARK) +#define TP_FEATURE_COMBINATION_TP_VSDOF_MFNR (MTK_FEATURE_MFNR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_VSDOF| TP_FEATURE_WATERMARK) +#define TP_FEATURE_COMBINATION_TP_FUSION (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_FUSION| TP_FEATURE_WATERMARK) +#define TP_FEATURE_COMBINATION_TP_PUREBOKEH (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE| TP_FEATURE_FB| TP_FEATURE_PUREBOKEH| TP_FEATURE_WATERMARK) // streaming feature combination (TODO: it should be refined by streaming scenario feature) -#define TP_FEATURE_COMBINATION_VIDEO_NORMAL (MTK_FEATURE_FB|TP_FEATURE_FB) -#define TP_FEATURE_COMBINATION_VIDEO_DUAL_YUV (MTK_FEATURE_FB|MTK_FEATURE_DUAL_YUV|TP_FEATURE_FB|TP_FEATURE_DUAL_YUV) -#define TP_FEATURE_COMBINATION_VIDEO_DUAL_HWDEPTH (MTK_FEATURE_FB|MTK_FEATURE_DUAL_HWDEPTH|TP_FEATURE_FB|TP_FEATURE_DUAL_HWDEPTH) -#define TP_FEATURE_COMBINATION_VIDEO_DUAL_HWVSDOF (MTK_FEATURE_FB|TP_FEATURE_FB) +#define TP_FEATURE_COMBINATION_VIDEO_NORMAL (MTK_FEATURE_FB|TP_FEATURE_FB|TP_FEATURE_WATERMARK) +#define TP_FEATURE_COMBINATION_VIDEO_DUAL_YUV (MTK_FEATURE_FB|MTK_FEATURE_DUAL_YUV|TP_FEATURE_FB|TP_FEATURE_DUAL_YUV|TP_FEATURE_WATERMARK) +#define TP_FEATURE_COMBINATION_VIDEO_DUAL_HWDEPTH (MTK_FEATURE_FB|MTK_FEATURE_DUAL_HWDEPTH|TP_FEATURE_FB|TP_FEATURE_DUAL_HWDEPTH|TP_FEATURE_WATERMARK) +#define TP_FEATURE_COMBINATION_VIDEO_DUAL_HWVSDOF (MTK_FEATURE_FB|TP_FEATURE_FB|TP_FEATURE_WATERMARK) // ====================================================================================================== // /******************************************************************************
vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/mtk_scenario_mgr.cpp：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/mtk_scenario_mgr.cpp b/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/mtk_scenario_mgr.cpp old mode 100644 new mode 100755 index 011f551354..f14ff8a6e2 --- a/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/mtk_scenario_mgr.cpp +++ b/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/mtk_scenario_mgr.cpp & -89，29 +89，29 & using namespace NSCam::v3::pipeline::policy::scenariomgr; // #define (key feature | post-processing features | ...) // // single cam capture feature combination -#define MTK_FEATURE_COMBINATION_SINGLE (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB) -#define MTK_FEATURE_COMBINATION_HDR (TP_FEATURE_HDR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB) -#define MTK_FEATURE_COMBINATION_AINR (MTK_FEATURE_AINR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB) -#define MTK_FEATURE_COMBINATION_MFNR (MTK_FEATURE_MFNR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB) -#define MTK_FEATURE_COMBINATION_REMOSAIC (MTK_FEATURE_REMOSAIC| MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB) +#define MTK_FEATURE_COMBINATION_SINGLE (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_WATERMARK) +#define MTK_FEATURE_COMBINATION_HDR (TP_FEATURE_HDR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_WATERMARK) +#define MTK_FEATURE_COMBINATION_AINR (MTK_FEATURE_AINR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_WATERMARK) +#define MTK_FEATURE_COMBINATION_MFNR (MTK_FEATURE_MFNR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_WATERMARK) +#define MTK_FEATURE_COMBINATION_REMOSAIC (MTK_FEATURE_REMOSAIC| MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_WATERMARK) #define MTK_FEATURE_COMBINATION_CSHOT (NO_FEATURE_NORMAL | MTK_FEATURE_CZ| MTK_FEATURE_HFG) -#define MTK_FEATURE_COMBINATION_YUV_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_FB) -#define MTK_FEATURE_COMBINATION_RAW_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_FB) -#define MTK_FEATURE_COMBINATION_SUPER_NIGHT_RAW_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_FB) +#define MTK_FEATURE_COMBINATION_YUV_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_FB| TP_FEATURE_WATERMARK) +#define MTK_FEATURE_COMBINATION_RAW_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_FB| TP_FEATURE_WATERMARK) +#define MTK_FEATURE_COMBINATION_SUPER_NIGHT_RAW_REPROCESS (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_FB| TP_FEATURE_WATERMARK) // dual cam capture feature combination // the VSDOF means the combination of Bokeh feature and Depth feature -#define MTK_FEATURE_COMBINATION_TP_VSDOF (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_VSDOF) -#define MTK_FEATURE_COMBINATION_TP_VSDOF_HDR (TP_FEATURE_HDR_DC | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_VSDOF) -#define MTK_FEATURE_COMBINATION_TP_VSDOF_MFNR (MTK_FEATURE_MFNR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_VSDOF) -#define MTK_FEATURE_COMBINATION_TP_FUSION (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_FUSION) -#define MTK_FEATURE_COMBINATION_TP_PUREBOKEH (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_PUREBOKEH) +#define MTK_FEATURE_COMBINATION_TP_VSDOF (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_VSDOF| TP_FEATURE_WATERMARK) +#define MTK_FEATURE_COMBINATION_TP_VSDOF_HDR (TP_FEATURE_HDR_DC | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_VSDOF| TP_FEATURE_WATERMARK) +#define MTK_FEATURE_COMBINATION_TP_VSDOF_MFNR (MTK_FEATURE_MFNR | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_VSDOF| TP_FEATURE_WATERMARK) +#define MTK_FEATURE_COMBINATION_TP_FUSION (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_FUSION| TP_FEATURE_WATERMARK) +#define MTK_FEATURE_COMBINATION_TP_PUREBOKEH (NO_FEATURE_NORMAL | MTK_FEATURE_NR| MTK_FEATURE_ABF| MTK_FEATURE_CZ| MTK_FEATURE_DRE| MTK_FEATURE_HFG| MTK_FEATURE_DCE | MTK_FEATURE_FB| TP_FEATURE_PUREBOKEH| TP_FEATURE_WATERMARK) // streaming feature combination (TODO: it should be refined by streaming scenario feature) -#define MTK_FEATURE_COMBINATION_VIDEO_NORMAL (MTK_FEATURE_FB|TP_FEATURE_FB) -#define MTK_FEATURE_COMBINATION_VIDEO_DUAL_YUV (MTK_FEATURE_FB|MTK_FEATURE_DUAL_YUV|TP_FEATURE_FB|TP_FEATURE_DUAL_YUV) -#define MTK_FEATURE_COMBINATION_VIDEO_DUAL_HWDEPTH (MTK_FEATURE_FB|MTK_FEATURE_DUAL_HWDEPTH|TP_FEATURE_FB|TP_FEATURE_DUAL_HWDEPTH) -#define MTK_FEATURE_COMBINATION_VIDEO_DUAL_HWVSDOF (MTK_FEATURE_FB|TP_FEATURE_FB) +#define MTK_FEATURE_COMBINATION_VIDEO_NORMAL (MTK_FEATURE_FB|TP_FEATURE_FB|TP_FEATURE_WATERMARK) +#define MTK_FEATURE_COMBINATION_VIDEO_DUAL_YUV (MTK_FEATURE_FB|MTK_FEATURE_DUAL_YUV|TP_FEATURE_FB|TP_FEATURE_DUAL_YUV|TP_FEATURE_WATERMARK) +#define MTK_FEATURE_COMBINATION_VIDEO_DUAL_HWDEPTH (MTK_FEATURE_FB|MTK_FEATURE_DUAL_HWDEPTH|TP_FEATURE_FB|TP_FEATURE_DUAL_HWDEPTH|TP_FEATURE_WATERMARK) +#define MTK_FEATURE_COMBINATION_VIDEO_DUAL_HWVSDOF (MTK_FEATURE_FB|TP_FEATURE_FB|TP_FEATURE_WATERMARK) // ====================================================================================================== // /******************************************************************************
注意： MTK在Android Q（10.0）上优化了scenario配置表的客制化，Android Q及更高版本，scenario需要在： vendor/mediatek/proprietary/custom/[platform]/hal/camera/camera_custom_feature_table.cpp中配置，[platform]是诸如mt6580，mt6763之类的。
将自定义feature添加到scenario配置表时，不可贪多，只要添加到合适的scenario就行，多了可能多个算法会有冲突。如果仅在简单场景，添加到MTK_FEATURE_COMBINATION_SINGLE和TP_FEATURE_COMBINATION_SINGLE就可以满足绝大多数需求。(2021-02-02更新)
四、挂载算法
4.1 为算法选择plugin
MTK HAL3在vendor/mediatek/proprietary/hardware/mtkcam3/include/mtkcam3/3rdparty/plugin/PipelinePluginType.h 中将三方算法的挂载点大致分为以下几类：
BokehPlugin： Bokeh算法挂载点，双摄景深算法的虚化部分。
DepthPlugin： Depth算法挂载点，双摄景深算法的计算深度部分。
FusionPlugin： Depth和Bokeh放在1个算法中，即合并的双摄景深算法挂载点。
JoinPlugin： Streaming相关算法挂载点，预览算法都挂载在这里。
MultiFramePlugin： 多帧算法挂载点，包括YUV与RAW，例如MFNR/HDR
RawPlugin： RAW算法挂载点，例如remosaic
YuvPlugin： Yuv单帧算法挂载点，例如美颜、广角镜头畸变校正等
对号入座，将要集成的算法选择相应的plugin。这里是单帧算法，所以预览我们选择JoinPlugin，拍照选择YuvPlugin。
4.2 编写算法集成文件
参照FBImpl.cpp和sample_streaming_fb.cpp中分别实现拍照和预览。目录结构如下： vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/tp_watermark/ ├── Android.mk ├── include │ └── watermark.h ├── lib │ ├── arm64-v8a │ │ └── libwatermark.so │ └── armeabi-v7a │ └── libwatermark.so ├── res │ └── watermark.rgba ├── WatermarkCapture.cpp └── WatermarkPreview.cpp
文件说明：
Android.mk中配置算法库、头文件、集成的源代码CPP文件编译成库libmtkcam.plugin.tp_watermark，供libmtkcam_3rdparty.customer依赖调用。
集成的源代码CPP文件，WatermarkCapture.cpp用于拍照，WatermarkPreview.cpp用于预览。
libwatermark.so实现了添加水印的功能，libwatermark.so用来模拟需要接入的第三方算法库。watermark.h是头文件。
watermark.rgba是对应的水印文件。
4.2.1 添加全局宏控
为了能控制某个项目是否集成此算法，我们在device/mediateksample/k63v2_64_bsp/ProjectConfig.mk中添加一个宏，用于控制新接入算法的编译：
QXT_WATERMARK_SUPPORT = yes
当某个项目不需要新接入的算法时，将device/mediateksample/[platform]/ProjectConfig.mk的QXT_WA_SUPPORT的值设为 no 就可以了。
4.2.2 mtkcam3/3rdparty/customer/tp_watermark/Android.mk
ifeq ($(QXT_WATERMARK_SUPPORT)，yes) LOCAL_PATH := $(call my-dir) include $(CLEAR_VARS) LOCAL_MODULE := libwatermark LOCAL_SRC_FILES_32 := lib/armeabi-v7a/libwatermark.so LOCAL_SRC_FILES_64 := lib/arm64-v8a/libwatermark.so LOCAL_MODULE_TAGS := optional LOCAL_MODULE_CLASS := SHARED_LIBRARIES LOCAL_MODULE_SUFFIX := .so LOCAL_PROPRIETARY_MODULE := true LOCAL_MULTILIB := both include $(BUILD_PREBUILT) ################################################################################ include $(CLEAR_VARS) #----------------------------------------------------------- include $(TOP)/$(MTK_PATH_SOURCE)/hardware/mtkcam/mtkcam.mk #----------------------------------------------------------- LOCAL_SRC_FILES += WatermarkCapture.cpp LOCAL_SRC_FILES += WatermarkPreview.cpp #----------------------------------------------------------- LOCAL_C_INCLUDES += $(MTKCAM_C_INCLUDES) LOCAL_C_INCLUDES += $(TOP)/$(MTK_PATH_SOURCE)/hardware/mtkcam3/include LOCAL_C_INCLUDES += $(TOP)/$(MTK_PATH_SOURCE)/hardware/mtkcam/include # LOCAL_C_INCLUDES += system/media/camera/include LOCAL_C_INCLUDES += $(TOP)/external/libyuv/files/include/ LOCAL_C_INCLUDES += $(TOP)/$(MTK_PATH_SOURCE)/hardware/mtkcam3/3rdparty/customer/tp_watermark/include #----------------------------------------------------------- LOCAL_CFLAGS += $(MTKCAM_CFLAGS) # #----------------------------------------------------------- LOCAL_STATIC_LIBRARIES += # LOCAL_WHOLE_STATIC_LIBRARIES += #----------------------------------------------------------- LOCAL_SHARED_LIBRARIES += liblog LOCAL_SHARED_LIBRARIES += libutils LOCAL_SHARED_LIBRARIES += libcutils LOCAL_SHARED_LIBRARIES += libmtkcam_modulehelper LOCAL_SHARED_LIBRARIES += libmtkcam_stdutils LOCAL_SHARED_LIBRARIES += libmtkcam_pipeline LOCAL_SHARED_LIBRARIES += libmtkcam_metadata LOCAL_SHARED_LIBRARIES += libmtkcam_metastore LOCAL_SHARED_LIBRARIES += libmtkcam_streamutils LOCAL_SHARED_LIBRARIES += libmtkcam_imgbuf LOCAL_SHARED_LIBRARIES += libyuv.vendor #----------------------------------------------------------- LOCAL_HEADER_LIBRARIES := libutils_headers liblog_headers libhardware_headers #----------------------------------------------------------- LOCAL_MODULE := libmtkcam.plugin.tp_watermark LOCAL_PROPRIETARY_MODULE := true LOCAL_MODULE_OWNER := mtk LOCAL_MODULE_TAGS := optional include $(MTK_STATIC_LIBRARY) ################################################################################ include $(call all-makefiles-under，$(LOCAL_PATH)) endif
4.2.3 mtkcam3/3rdparty/customer/tp_watermark/WatermarkCapture.cpp
主要函数介绍：
在property函数中feature类型设置我们在第三步中添加的TP_FEATURE_WATERMARK，并设置名称、优先级等等属性。
在negotiate函数中配置算法需要的输入、输出图像的格式、尺寸。
在negotiate函数或者process函数中获取上层传下来的metadata参数，根据参数决定算法是否运行，或者将参数传给算法。
在process函数中接入算法。
注意：
MTK原文： negotiate函数设置格式时，一个挂载点如果挂载多个同类型的plugin，则只有第一个 plugin 中的 negotiate 中的 input buffer 设定有效。 在YUVNode 下挂载单帧 YUV plugin时，一定要确保 MTK 平台的SWNR plugin 的 negotiate 直接返回不OK，不做任何 accepted format 等的设定。否则，可能会出现因 SWNR plugin和三方plugin negotiate时设定的 accepted format 不一致而导致的三方 plugin 拿不到它想要的 format 的buffer。
vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/swnr/SWNRImpl.cpp：
diff --git a/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/swnr/SWNRImpl.cpp b/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/swnr/SWNRImpl.cpp
old mode 100644
new mode 100755
index 0ae951cc83..c4819068f7
--- a/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/swnr/SWNRImpl.cpp
+++ b/vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/mtk/swnr/SWNRImpl.cpp
& -340，7 +340，7 & negotiate(Selection& sel)
sel.mOMetadataApp.setRequired(false);
sel.mOMetadataHal.setRequired(true);
- return OK;
+ return -EINVAL;//OK;
}
vendor/mediatek/proprietary/hardware/mtkcam3/3rdparty/customer/tp_watermark/WatermarkCapture.cpp：
#define LOG_TAG "WatermarkCapture"
//
#include
//
#include
#include
#include
#include
#include
//
#include
#include
//
//
#include
//
#include
#include
//
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
//
using namespace NSCam;
using namespace android;
using namespace std;
using namespace NSCam::NSPipelinePlugin;
/******************************************************************************
*
******************************************************************************/
#define MY_LOGV(fmt， arg...) CAM_LOGV("(%d)[%s] " fmt， ::gettid()， __FUNCTION__， ##arg)
#define MY_LOGD(fmt， arg...) CAM_LOGD("(%d)[%s] " fmt， ::gettid()， __FUNCTION__， ##arg)
#define MY_LOGI(fmt， arg...) CAM_LOGI("(%d)[%s] " fmt， ::gettid()， __FUNCTION__， ##arg)
#define MY_LOGW(fmt， arg...) CAM_LOGW("(%d)[%s] " fmt， ::gettid()， __FUNCTION__， ##arg)
#define MY_LOGE(fmt， arg...) CAM_LOGE("(%d)[%s] " fmt， ::gettid()， __FUNCTION__， ##arg)
//
#define FUNCTION_IN MY_LOGD("%s +"， __FUNCTION__)
#define FUNCTION_OUT MY_LOGD("%s -"， __FUNCTION__)
//systrace
#if 1
#ifndef ATRACE_TAG
#define ATRACE_TAG ATRACE_TAG_CAMERA
#endif
#include
#define WATERMARK_TRACE_CALL() ATRACE_CALL()
#define WATERMARK_TRACE_NAME(name) ATRACE_NAME(name)
#define WATERMARK_TRACE_BEGIN(name) ATRACE_BEGIN(name)
#define WATERMARK_TRACE_END() ATRACE_END()
#else
#define WATERMARK_TRACE_CALL()
#define WATERMARK_TRACE_NAME(name)
#define WATERMARK_TRACE_BEGIN(name)
#define WATERMARK_TRACE_END()
#endif
template
inline bool
tryGetMetadata(IMetadata const *pMetadata， MUINT32 tag， T& rVal)
{
if(pMetadata == nullptr) return MFALSE;
IMetadata::IEntry entry = pMetadata->entryFor(tag);
if(!entry.isEmpty())
{
rVal = entry.itemAt(0，Type2Type());
return true;
}
else
{
#define var(v) #v
#define type(t) #t
MY_LOGW("no metadata %s in %s"， var(tag)， type(pMetadata));
#undef type
#undef var
}
return false;
}
/******************************************************************************
*
******************************************************************************/
class WatermarkCapture : public YuvPlugin::IProvider {
public:
typedef YuvPlugin::Property Property;
typedef YuvPlugin::Selection Selection;
typedef YuvPlugin::Request::Ptr RequestPtr;
typedef YuvPlugin::RequestCallback::Ptr RequestCallbackPtr;
private:
int mOpenid;
MBOOL mEnable = 1;
MBOOL mDump = 0;
unsigned char *mSrcRGBA = nullptr;
unsigned char *mWatermarkRGBA = nullptr;
int mWatermarkWidth = 0;
int mWatermarkHeight = 0;
public:
WatermarkCapture();
~WatermarkCapture();
void init();
void uninit();
void abort(vector &pRequests);
void set(MINT32 iOpenId， MINT32 iOpenId2);
const Property &property();
MERROR negotiate(Selection &sel);
MERROR process(RequestPtr pRequest， RequestCallbackPtr pCallback);
};
WatermarkCapture::WatermarkCapture() : mOpenid(-1) {
FUNCTION_IN;
mEnable = property_get_bool("vendor.debug.camera.watermark.capture.enable"， 1);
mDump = property_get_bool("vendor.debug.camera.watermark.capture.dump"， 0);
FUNCTION_OUT;
}
WatermarkCapture::~WatermarkCapture() {
FUNCTION_IN;
FUNCTION_OUT;
}
void WatermarkCapture::init() {
FUNCTION_IN;
mWatermarkWidth = 180;
mWatermarkHeight = 640;
int watermarkSize = mWatermarkWidth * mWatermarkHeight * 4;
mWatermarkRGBA = (unsigned char *) malloc(watermarkSize);
FILE *fp;
char path[256];
snprintf(path， sizeof(path)， "/vendor/res/images/watermark.rgba");
if ((fp = fopen(path， 芯片采购平台 "r")) == NULL) {
MY_LOGE("Failed to open /vendor/res/images/watermark.rgba");
}
fread(mWatermarkRGBA， 1， watermarkSize， fp);
fclose(fp);
FUNCTION_OUT;
}
void WatermarkCapture::uninit() {
FUNCTION_IN;
free(mWatermarkRGBA);
FUNCTION_OUT;
}
void WatermarkCapture::abort(vector &pRequests) {
FUNCTION_IN;
(void)pRequests;
FUNCTION_OUT;
}
void WatermarkCapture::set(MINT32 iOpenId， MINT32 iOpenId2) {
FUNCTION_IN;
MY_LOGD("set openId:%d openId2:%d"， iOpenId， iOpenId2);
mOpenid = iOpenId;
FUNCTION_OUT;
}
const WatermarkCapture::Property &WatermarkCapture::property() {
FUNCTION_IN;
static Property prop;
static bool inited;
if (!inited) {
prop.mName = "TP_WATERMARK";
prop.mFeatures = TP_FEATURE_WATERMARK;
prop.mInPlace = MTRUE;
prop.mFaceData = eFD_Current;
prop.mPosition = 0;
inited = true;
}
FUNCTION_OUT;
return prop;
}
MERROR WatermarkCapture::negotiate(Selection &sel) {
FUNCTION_IN;
if (!mEnable) {
MY_LOGD("Force off TP_WATERMARK");
FUNCTION_OUT;
return -EINVAL;
}
sel.mIBufferFull
.setRequired(MTRUE)
.addAcceptedFormat(eImgFmt_I420)
.addAcceptedSize(eImgSize_Full);
sel.mIMetadataDynamic.setRequired(MTRUE);
sel.mIMetadataApp.setRequired(MTRUE);
sel.mIMetadataHal.setRequired(MTRUE);
sel.mOMetadataApp.setRequired(MTRUE);
sel.mOMetadataHal.setRequired(MTRUE);
FUNCTION_OUT;
return OK;
}
MERROR WatermarkCapture::process(RequestPtr pRequest，
RequestCallbackPtr pCallback = nullptr) {
FUNCTION_IN;
WATERMARK_TRACE_CALL();
MBOOL needRun = MFALSE;
if (pRequest->mIBufferFull != nullptr && pRequest->mOBufferFull != nullptr) {
IImageBuffer *pIBufferFull = pRequest->mIBufferFull->acquire();
IImageBuffer *pOBufferFull = pRequest->mOBufferFull->acquire();
if (pRequest->mIMetadataDynamic != nullptr) {
IMetadata *meta = pRequest->mIMetadataDynamic->acquire();
if (meta != NULL)
MY_LOGD("[IN] Dynamic metadata count: %d"， meta->count());
else
MY_LOGD("[IN] Dynamic metadata empty");
}
int frameNo = 0， requestNo = 0;
if (pRequest->mIMetadataHal != nullptr) {
IMetadata *pIMetataHAL = pRequest->mIMetadataHal->acquire();
if (pIMetataHAL != NULL) {
MY_LOGD("[IN] HAL metadata count: %d"， pIMetataHAL->count());
if (!tryGetMetadata(pIMetataHAL， MTK_PIPELINE_FRAME_NUMBER， frameNo)) {
frameNo = 0;
}
if (!tryGetMetadata(pIMetataHAL， MTK_PIPELINE_REQUEST_NUMBER， requestNo)) {
requestNo = 0;
}
MY_LOGD("frameNo: %d， requestNo: %d"， frameNo， requestNo);
} else {
MY_LOGD("[IN] HAL metadata empty");
}
}
if (pRequest->mIMetadataApp != nullptr) {
IMetadata *pIMetadataApp = pRequest->mIMetadataApp->acquire();
MINT32 mode = 0;
if (!tryGetMetadata(pIMetadataApp， QXT_FEATURE_WATERMARK， mode)) {
mode = 0;
}
needRun = mode == 1 ? 1 : 0;
}
MY_LOGD("needRun: %d"， needRun);
int width = pIBufferFull->getImgSize().w;
int height = pIBufferFull->getImgSize().h;
MINT inFormat = pIBufferFull->getImgFormat();
if (needRun && inFormat == NSCam::eImgFmt_I420) {
uint32_t currentTime = (NSCam::Utils::TimeTool::getReadableTime()) % 1000;
time_t timep;
time (&timep);
char currentDate[20];
strftime(currentDate， sizeof(currentDate)， "%Y%m%d_%H%M%S"， localtime(&timep));
//dump input I420
if (mDump) {
char path[256];
snprintf(path， sizeof(path)， "/data/vendor/camera_dump/capture_in_frame%d_%dx%d_%s_%d.i420"，
frameNo， width， height， currentDate， currentTime);
pIBufferFull->saveToFile(path);
}
nsecs_t t1 = systemTime(CLOCK_MONOTONIC);
if (mSrcRGBA == NULL) {
mSrcRGBA = (unsigned char *) malloc(width * height * 4);
}
//convert I420 to RGBA
libyuv::I420ToABGR((unsigned char *) (pIBufferFull->getBufVA(0))， width，
(unsigned char *) (pIBufferFull->getBufVA(1))， width >> 1，
(unsigned char *) (pIBufferFull->getBufVA(2))， width >> 1，
mSrcRGBA， width * 4，
width， height);
nsecs_t t2 = systemTime(CLOCK_MONOTONIC);
MY_LOGD("Prepare src cost ld ms"， ns2ms(t2 - t1));
Watermark::add(mSrcRGBA， width， height， mWatermarkRGBA， mWatermarkWidth， mWatermarkHeight， (width - mWatermarkWidth) / 2， (height - mWatermarkHeight) / 2);
nsecs_t t3 = systemTime(CLOCK_MONOTONIC);
MY_LOGD("Add watermark cost ld ms"， ns2ms(t3 - t2));
//convert RGBA to I420
libyuv::ABGRToI420(mSrcRGBA， width * 4，
(unsigned char *) (pOBufferFull->getBufVA(0))， width，
(unsigned char *) (pOBufferFull->getBufVA(1))， width >> 1，
(unsigned char *) (pOBufferFull->getBufVA(2))， width >> 1，
width， height);
nsecs_t t4 = systemTime(CLOCK_MONOTONIC);
MY_LOGD("Copy in to out cost ld ms"， ns2ms(t4 - t3));
//dump output I420
if (mDump) {
char path[256];
snprintf(path， sizeof(path)， "/data/vendor/camera_dump/capture_out_frame%d_%dx%d_%s_%d.i420"，
frameNo， width， height， currentDate， currentTime);
pOBufferFull->saveToFile(path);
}
free(mSrcRGBA);
} else {
if (!needRun) {
MY_LOGE("No need run， skip add watermark for capture.");
} else if (inFormat != NSCam::eImgFmt_YV12) {
MY_LOGE("Unsupported format， skip add watermark for capture.");
} else {
MY_LOGE("Unknown exception， skip add watermark for capture.");
}
memcpy((unsigned char *) (pOBufferFull->getBufVA(0))，
(unsigned char *) (pIBufferFull->getBufVA(0))，
pIBufferFull->getBufSizeInBytes(0));
memcpy((unsigned char *) (pOBufferFull->getBufVA(1))，
(unsigned char *) (pIBufferFull->getBufVA(1))，
pIBufferFull->getBufSizeInBytes(1));
memcpy((unsigned char *) (pOBufferFull->getBufVA(2))，
(unsigned char *) (pIBufferFull->getBufVA(2))，
pIBufferFull->getBufSizeInBytes(2));
}
pRequest->mIBufferFull->release();
pRequest->mOBufferFull->release();
if (pRequest->mIMetadataDynamic != nullptr) {
pRequest->mIMetadataDynamic->release();
}
if (pRequest->mIMetadataHal != nullptr) {
pRequest->mIMetadataHal->release();
}
if (pRequest->mIMetadataApp != nullptr) {
pRequest->mIMetadataApp->release();
}
}
if (pCallback != nullptr) {
MY_LOGD("callback request");
pCallback->onCompleted(pRequest， 0);
}
FUNCTION_OUT;
return OK;
}
REGISTER_PLUGIN_PROVIDER(Yuv， WatermarkCapture);
4.2.4 mtkcam3/3rdparty/customer/tp_watermark/WatermarkPreview.cpp
#include
#include
#include
#include
#include
#include
#include
#include
#include
using NSCam::NSPipelinePlugin::Interceptor;
using NSCam::NSPipelinePlugin::PipelinePlugin;
using NSCam::NSPipelinePlugin::PluginRegister;
using NSCam::NSPipelinePlugin::Join;
using NSCam::NSPipelinePlugin::JoinPlugin;
using namespace NSCam::NSPipelinePlugin;
using NSCam::MSize;
using NSCam::MERROR;
using NSCam::IImageBuffer;
using NSCam::IMetadata;
using NSCam::Type2Type;
#ifdef LOG_TAG
#undef LOG_TAG
#endif // LOG_TAG
#define LOG_TAG "WatermarkPreview"
#include
#include
#define MY_LOGI(fmt， arg...) ALOGI("[%s] " fmt， __FUNCTION__， ##arg)
#define MY_LOGD(fmt， arg...) ALOGD("[%s] " fmt， __FUNCTION__， ##arg)
#define MY_LOGW(fmt， arg...) ALOGW("[%s] " fmt， __FUNCTION__， ##arg)
#define MY_LOGE(fmt， arg...) ALOGE("[%s] " fmt， __FUNCTION__， ##arg)
#define FUNCTION_IN MY_LOGD("%s +"， __FUNCTION__)
#define FUNCTION_OUT MY_LOGD("%s -"， __FUNCTION__)
template
inline bool
tryGetMetadata(IMetadata const *pMetadata， MUINT32 tag， T& rVal)
{
if(pMetadata == nullptr) return MFALSE;
IMetadata::IEntry entry = pMetadata->entryFor(tag);
if(!entry.isEmpty())
{
rVal = entry.itemAt(0，Type2Type());
return true;
}
else
{
#define var(v) #v
#define type(t) #t
MY_LOGW("no metadata %s in %s"， var(tag)， type(pMetadata));
#undef type
#undef var
}
return false;
}
class WatermarkPreview : public JoinPlugin::IProvider {
public:
typedef JoinPlugin::Property Property;
typedef JoinPlugin::Selection Selection;
typedef JoinPlugin::Request::Ptr RequestPtr;
typedef JoinPlugin::RequestCallback::Ptr RequestCallbackPtr;
private:
bool mDisponly = false;
bool mInplace = false;
int mOpenID1 = 0;
int mOpenID2 = 0;
MBOOL mEnable = 1;
MBOOL mDump = 0;
unsigned char *mSrcRGBA = nullptr;
unsigned char *mWatermarkRGBA = nullptr;
int mWatermarkWidth = 0;
int mWatermarkHeight = 0;
public:
WatermarkPreview();
~WatermarkPreview();
void init();
void uninit();
void abort(std::vector &pRequests);
void set(MINT32 openID1， MINT32 openID2);
const Property &property();
MERROR negotiate(Selection &sel);
MERROR process(RequestPtr pRequest， RequestCallbackPtr pCallback);
private:
MERROR getConfigSetting(Selection &sel);
MERROR getP1Setting(Selection &sel);
MERROR getP2Setting(Selection &sel);
};
WatermarkPreview::WatermarkPreview() {
FUNCTION_IN;
mEnable = property_get_bool("vendor.debug.camera.watermark.preview.enable"， 1);
mDump = property_get_bool("vendor.debug.camera.watermark.preview.dump"， 0);
FUNCTION_OUT;
}
WatermarkPreview::~WatermarkPreview() {
FUNCTION_IN;
FUNCTION_OUT;
}
void WatermarkPreview::init() {
FUNCTION_IN;
mWatermarkWidth = 180;
mWatermarkHeight = 640;
int watermarkSize = mWatermarkWidth * mWatermarkHeight * 4;
mWatermarkRGBA = (unsigned char *) malloc(watermarkSize);
FILE *fp;
char path[256];
snprintf(path， sizeof(path)， "/vendor/res/images/watermark.rgba");
if ((fp = fopen(path， "r")) == NULL) {
MY_LOGE("Failed to open /vendor/res/images/watermark.rgba");
}
fread(mWatermarkRGBA， 1， watermarkSize， fp);
fclose(fp);
FUNCTION_OUT;
}
void WatermarkPreview::uninit() {
FUNCTION_IN;
free(mSrcRGBA);
free(mWatermarkRGBA);
FUNCTION_OUT;
}
void WatermarkPreview::abort(std::vector &pRequests) {
FUNCTION_IN;
(void)pRequests;
FUNCTION_OUT;
}
void WatermarkPreview::set(MINT32 openID1， MINT32 openID2) {
FUNCTION_IN;
MY_LOGD("set openID1:%d openID2:%d"， openID1， openID2);
mOpenID1 = openID1;
mOpenID2 = openID2;
FUNCTION_OUT;
}
const WatermarkPreview::Property &WatermarkPreview::property() {
FUNCTION_IN;
static Property prop;
static bool inited;
if (!inited) {
prop.mName = "TP_WATERMARK";
prop.mFeatures = TP_FEATURE_WATERMARK;
//prop.mInPlace = MTRUE;
//prop.mFaceData = eFD_Current;
//prop.mPosition = 0;
inited = true;
}
FUNCTION_OUT;
return prop;
}
MERROR WatermarkPreview::negotiate(Selection &sel) {
FUNCTION_IN;
MERROR ret = OK;
if (sel.mSelStage == eSelStage_CFG) {
ret = getConfigSetting(sel);
} else if (sel.mSelStage == eSelStage_P1) {
ret = getP1Setting(sel);
} else if (sel.mSelStage == eSelStage_P2) {
ret = getP2Setting(sel);
}
FUNCTION_OUT;
return ret;
}
MERROR WatermarkPreview::process(RequestPtr pRequest， RequestCallbackPtr pCallback) {
FUNCTION_IN;
(void) pCallback;
MERROR ret = -EINVAL;
MBOOL needRun = MFALSE;
IImageBuffer *in = NULL， *out = NULL;
if (pRequest->mIBufferMain1 != NULL && pRequest->mOBufferMain1 != NULL) {
in = pRequest->mIBufferMain1->acquire();
out = pRequest->mOBufferMain1->acquire();
int frameNo = 0， requestNo = 0;
if (pRequest->mIMetadataHal1 != nullptr) {
IMetadata *pIMetataHAL1 = pRequest->mIMetadataHal1->acquire();
if (pIMetataHAL1 != NULL) {
if (!tryGetMetadata(pIMetataHAL1， MTK_PIPELINE_FRAME_NUMBER， frameNo)) {
frameNo = 0;
}
if (!tryGetMetadata(pIMetataHAL1， MTK_PIPELINE_REQUEST_NUMBER， requestNo)) {
requestNo = 0;
}
pRequest->mIMetadataHal1->release();
MY_LOGD("frameNo: %d， requestNo: %d"， frameNo， requestNo);
} else {
MY_LOGD("HAL metadata empty");
}
}
MY_LOGD("in[%d](%dx%d)=%p out[%d](%dx%d)=%p"，
in->getPlaneCount()， in->getImgSize().w， in->getImgSize().h， in，
out->getPlaneCount()， out->getImgSize().w， out->getImgSize().h， out);
if (pRequest->mIMetadataApp != nullptr) {
IMetadata *pIMetadataApp = pRequest->mIMetadataApp->acquire();
MINT32 mode = 0;
if (!tryGetMetadata(pIMetadataApp， QXT_FEATURE_WATERMARK， mode)) {
mode = 0;
}
needRun = mode == 1 ? 1 : 0;
pRequest->mIMetadataApp->release();
}
MY_LOGD("needRun: %d"， needRun);
int width = in->getImgSize().w;
int height = in->getImgSize().h;
MINT inFormat = in->getImgFormat();
if (needRun && inFormat == NSCam::eImgFmt_YV12) {
uint32_t currentTime = (NSCam::Utils::TimeTool::getReadableTime()) % 1000;
time_t timep;
time (&timep);
char currentDate[20];
strftime(currentDate， sizeof(currentDate)， "%Y%m%d_%H%M%S"， localtime(&timep));
//dump input YV12
if (mDump) {
char path[256];
snprintf(path， sizeof(path)， "/data/vendor/camera_dump/preview_in_frame%d_%dx%d_%s_%d.yv12"，
frameNo， width， height， currentDate， currentTime);
in->saveToFile(path);
}
nsecs_t t1 = systemTime(CLOCK_MONOTONIC);
if (mSrcRGBA == NULL) {
mSrcRGBA = (unsigned char *) malloc(width * height * 4);
}
//convert YV12 to RGBA
libyuv::I420ToABGR((unsigned char *)(in->getBufVA(0))， width，
(unsigned char *)(in->getBufVA(2))， width >> 1，
(unsigned char *)(in->getBufVA(1))， width >> 1，
mSrcRGBA， width * 4，
width， height);
nsecs_t t2 = systemTime(CLOCK_MONOTONIC);
MY_LOGD("Prepare src cost ld ms"， ns2ms(t2 - t1));
Watermark::add(mSrcRGBA， width， height， mWatermarkRGBA， mWatermarkWidth， mWatermarkHeight， (width - mWatermarkWidth) / 2， (height - mWatermarkHeight) / 2);
nsecs_t t3 = systemTime(CLOCK_MONOTONIC);
MY_LOGD("Add watermark cost ld ms"， ns2ms(t3 - t2));
//convert RGBA to YV12
libyuv::ABGRToI420(mSrcRGBA， width * 4，
(unsigned char *)(out->getBufVA(0))， width，
(unsigned char *)(out->getBufVA(2))， width >> 1，
(unsigned char *)(out->getBufVA(1))， width >> 1，
width， height);
nsecs_t t4 = systemTime(CLOCK_MONOTONIC);
MY_LOGD("Copy in to out cost ld ms"， ns2ms(t4 - t3));
//dump output YV12
if (mDump) {
char path[256];
snprintf(path， sizeof(path)， "/data/vendor/camera_dump/preview_out_frame%d_%dx%d_%s_%d.yv12"，
frameNo， width， height， currentDate， currentTime);
out->saveToFile(path);
}
} else {
if (!needRun) {
MY_LOGE("No need run， skip add watermark for preview.");
} else if (inFormat != NSCam::eImgFmt_YV12) {
MY_LOGE("Unsupported format， skip add watermark for preview.");
} else {
MY_LOGE("Unknown exception， skip add watermark for preview.");
}
memcpy((unsigned char *) (out->getBufVA(0))，
(unsigned char *)(in->getBufVA(0))，
in->getBufSizeInBytes(0));
memcpy((unsigned char *) (out->getBufVA(1))，
(unsigned char *)(in->getBufVA(1))，
in->getBufSizeInBytes(1));
memcpy((unsigned char *) (out->getBufVA(2))，
(unsigned char *)(in->getBufVA(2))，
in->getBufSizeInBytes(2));
}
pRequest->mIBufferMain1->release();
pRequest->mOBufferMain1->release();
ret = OK;
}
FUNCTION_OUT;
return ret;
}
MERROR WatermarkPreview::getConfigSetting(Selection &sel) {
MY_LOGI("max out size(%dx%d)"，
sel.mCfgInfo.mMaxOutSize.w， sel.mCfgInfo.mMaxOutSize.h);
mDisponly = property_get_bool("vendor.debug.tpi.s.fb.disponly"， 0);
mInplace = mDisponly
相关资讯