feat: Frame Cryptor (aes gcm/cbc). #54

cloudwebrtc · 2022-11-28T07:08:28Z

iOS/macOS
Android
Windows/Linux feat: add frame encryptor/decryptor interface. libwebrtc#46

api/crypto/frame_crypto_transformer.h

theomonnom

This is 🔥
Just curious, why you didn't use webrtc::FrameEncryptorInterface (I guess you have not enough flexibility with it?)

sdk/android/src/jni/pc/frame_cryptor_key_manager.h

sdk/objc/api/peerconnection/RTCFrameCryptorKeyManager.mm

sdk/android/src/jni/pc/frame_cryptor_key_manager.h

cloudwebrtc · 2023-02-17T06:48:51Z

This is 🔥 Just curious, why you didn't use webrtc::FrameEncryptorInterface (I guess you have not enough flexibility with it?)

Yes, FrameEncryptorInterface is missing the key information of FrameCrypto, codec type (VP8, H264 use different encryption methods), frame type, ssrc and timestamp (for building IV)

theomonnom

lgtm!

sdk/objc/api/peerconnection/RTCFrameCryptorKeyManager.mm

sdk/android/src/jni/pc/frame_cryptor_key_manager.h

Co-authored-by: Théo Monnom <theo.monnom@outlook.com>

…com/webrtc-sdk/webrtc into feat/built-in-aes-gcm-frame-cryptor

* feat: add frame crypto transformer using aes-128/256-gcm. * update. * feat: add aes-cbc. * update. * fix compile. * update. * chore: add getIvSize(). * rename. * update. * Add ObjC API for framecryptor. * add FrameCryptor interfaces for android. * chore: add jni native code. * update. * update * fix compile for iOS. * update. * fix. * update. * Add cryptor_id. * update. * update. * update. * update. * chore: Experimental encryption for h.264. * chore: export class FrameCryptorTransformer. * aes frame crypto for h.264. * chore: add decrypto failed callback. * fix: Fix FrameTransformer work with simulcast. * revert changes. * Fix h264 screen freezing in frame encryption. * chore: Improve the code and add FrameCryptionError. * feat: Add RTCFrameCryptorDelegate for darwin. * WIP observer for android. * observer for android. * Add kNew/kOk state. * [objc] fix bug for state callack. * fix issue for re-RegisterFrameTransformer. * dispose. * Update sdk/objc/api/peerconnection/RTCFrameCryptorKeyManager.mm Co-authored-by: Théo Monnom <theo.monnom@outlook.com> * Update sdk/android/src/jni/pc/frame_cryptor_key_manager.h Co-authored-by: Théo Monnom <theo.monnom@outlook.com> * Update sdk/objc/api/peerconnection/RTCFrameCryptorKeyManager.mm Co-authored-by: Théo Monnom <theo.monnom@outlook.com> * update. * fix compile. --------- Co-authored-by: Théo Monnom <theo.monnom@outlook.com>

feat: Frame Cryptor (aes gcm/cbc). (#54) feat: key ratchet/derive. (#66) Co-authored-by: Théo Monnom <theo.monnom@outlook.com>

feat: Frame Cryptor (aes gcm/cbc). (#54) feat: key ratchet/derive. (#66) fix: skip invalid key when decryption failed. (#81) Co-authored-by: Théo Monnom <theo.monnom@outlook.com>

feat: Frame Cryptor (aes gcm/cbc). (#54) feat: key ratchet/derive. (#66) fix: skip invalid key when decryption failed. (#81) Improve e2ee, add setSharedKey to KeyProvider. (#88) add failure tolerance for framecryptor. (#91) fix h264 freeze. (#93) Fix/send frame cryptor events from signaling thread (#95) more improvements for E2EE. (#96) remove too verbose logs (#107) Add key ring size to keyProviderOptions. (#109) Co-authored-by: Théo Monnom <theo.monnom@outlook.com> Co-authored-by: Hiroshi Horie <548776+hiroshihorie@users.noreply.github.com> Co-authored-by: davidliu <davidliu@deviange.net>

feat: Frame Cryptor (aes gcm/cbc). (#54) feat: key ratchet/derive. (#66) fix: skip invalid key when decryption failed. (#81) Improve e2ee, add setSharedKey to KeyProvider. (#88) add failure tolerance for framecryptor. (#91) fix h264 freeze. (#93) Fix/send frame cryptor events from signaling thread (#95) more improvements for E2EE. (#96) remove too verbose logs (#107) Add key ring size to keyProviderOptions. (#109) Improvements to RTCFrameCryptor (#123) Co-authored-by: Théo Monnom <theo.monnom@outlook.com> Co-authored-by: Hiroshi Horie <548776+hiroshihorie@users.noreply.github.com> Co-authored-by: davidliu <davidliu@deviange.net>

Use M125 as the latest version and migrate historical patches to m125 Patches Group: ## 1. Update README.md b6c65fc * Add Apache-2.0 license and some note to README.md. (#9) * Updated readme detailing changes from original (#42) * Adding membrane framework (#51) * Updated readme (#83) ## 2. Audio Device Optimization 7454824 * allow listen-only mode in AudioUnit, adjust when category changes (#2) * release mic when category changes (#5) * Change defaults to iOS defaults (#7) * Sync audio session config (#8) * feat: support bypass voice processing for iOS. (#15) * Remove MacBookPro audio pan right code (#22) * fix: Fix can't open mic alone when built-in AEC is enabled. (#29) * feat: add audio device changes detect for windows. (#41) * fix Linux compile (#47) * AudioUnit: Don't rely on category switch for mic indicator to turn off (#52) * Stop recording on mute (turn off mic indicator) (#55) * Cherry pick audio selection from m97 release (#35) * [Mac] Allow audio device selection (#21) * RTCAudioDeviceModule.outputDevice / inputDevice getter and setter (#80) * Allow custom audio processing by exposing AudioProcessingModule (#85) * Expose audio sample buffers for Android (#89) * feat: add external audio processor for android. (#103) * android: make audio output attributes modifiable (#118) * Fix external audio processor sample rate calculation (#108) * Expose remote audio sample buffers on RTCAudioTrack (#84) * Fix memory leak when creating audio CMSampleBuffer #86 ## 3. Simulcast/SVC support for iOS/Android. b0b9fe9 - Simulcast support for iOS SDK (#4) - Support for simulcast in Android SDK (#3) - include simulcast headers for mac also (#10) - Fix simulcast using hardware encoder on Android (#48) - Add scalabilityMode support for AV1/VP9. (#90) ## 4. Android improvements. 9aaaab5 - Start/Stop receiving stream method for VideoTrack (#25) - Properly remove observer upon deconstruction (#26) - feat: Expose setCodecPreferences/getCapabilities for android. (#61) - fix: add WrappedVideoDecoderFactory.java. (#74) ## 5. Darwin improvements a13ea17 - [Mac/iOS] feat: Add RTCYUVHelper for darwin. (#28) - Cross-platform `RTCMTLVideoView` for both iOS / macOS (#40) - rotationOverride should not be assign (#44) - [ObjC] Expose properties / methods required for AV1 codec support (#60) - Workaround: Render PixelBuffer in RTCMTLVideoView (#58) - Improve iOS/macOS H264 encoder (#70) - fix: fix video encoder not resuming correctly upon foregrounding (#75). - add PrivacyInfo.xcprivacy to darwin frameworks. (#112) - Add NSPrivacyCollectedDataTypes key to xcprivacy file (#114) - Thread-safe `RTCInitFieldTrialDictionary` (#116) - Set RTCCameraVideoCapturer initial zoom factor (#121) - Unlock configuration before starting capture session (#122) ## 6. Desktop Capture for macOS. 841d78f - [Mac] feat: Support screen capture for macOS. (#24) (#36) - fix: Get thumbnails asynchronously. (#37) - fix: Use CVPixelBuffer to build DesktopCapture Frame, fix the crash caused by non-CVPixelBuffer frame in RTCVideoEncoderH264 that cannot be cropped. (#63) - Fix the crash when setting the fps of the virtual camera. (#62) ## 7. Frame Cryptor Support. fc08745 - feat: Frame Cryptor (aes gcm/cbc). (#54) - feat: key ratchet/derive. (#66) - fix: skip invalid key when decryption failed. (#81) - Improve e2ee, add setSharedKey to KeyProvider. (#88) - add failure tolerance for framecryptor. (#91) - fix h264 freeze. (#93) - Fix/send frame cryptor events from signaling thread (#95) - more improvements for E2EE. (#96) - remove too verbose logs (#107) - Add key ring size to keyProviderOptions. (#109) ## 8. Other improvements. eed6c8a - Added yuv_helper (#57) - ABGRToI420, ARGBToI420 & ARGBToRGB24 (#65) - more yuv wrappers (#87) - Fix naming for yuv helper (#113) - Fix missing `RTC_OBJC_TYPE` macros (#100) --------- Co-authored-by: Hiroshi Horie <548776+hiroshihorie@users.noreply.github.com> Co-authored-by: David Zhao <dz@livekit.io> Co-authored-by: davidliu <davidliu@deviange.net> Co-authored-by: Angelika Serwa <angelika.serwa@gmail.com> Co-authored-by: Théo Monnom <theo.monnom@outlook.com>

feat: Frame Cryptor (aes gcm/cbc). (webrtc-sdk#54) feat: key ratchet/derive. (webrtc-sdk#66) fix: skip invalid key when decryption failed. (webrtc-sdk#81) Co-authored-by: Théo Monnom <theo.monnom@outlook.com>

cloudwebrtc added 12 commits November 28, 2022 14:48

feat: add frame crypto transformer using aes-128/256-gcm.

a6bbeb0

update.

c6f0f9c

feat: add aes-cbc.

f546e28

update.

e9769ff

fix compile.

b2e337a

update.

1a11d05

chore: add getIvSize().

331870a

rename.

fa34239

update.

88de0b8

Add ObjC API for framecryptor.

ae8643f

add FrameCryptor interfaces for android.

38033f5

chore: add jni native code.

f673a8f

cloudwebrtc changed the title ~~feat: add frame crypto transformer using aes-128/256-gcm.~~ feat: Frame Cryptor (aes gcm/cbc). Dec 11, 2022

cloudwebrtc added 4 commits December 11, 2022 04:39

update.

24cf7e2

update

bf99b60

fix compile for iOS.

18fb9f8

update.

8138a27

cloudwebrtc marked this pull request as ready for review December 11, 2022 12:50

cloudwebrtc added 6 commits December 13, 2022 13:14

fix.

31e714a

update.

c3a6714

Add cryptor_id.

c940756

update.

c870fbe

update.

1394a60

update.

151e866

theomonnom reviewed Dec 20, 2022

View reviewed changes

api/crypto/frame_crypto_transformer.h Outdated Show resolved Hide resolved

cloudwebrtc added 5 commits December 20, 2022 19:50

update.

ad329f1

chore: Experimental encryption for h.264.

e2e8c03

chore: export class FrameCryptorTransformer.

9f1ebc2

aes frame crypto for h.264.

35c5d9b

chore: add decrypto failed callback.

4a53ac3

cloudwebrtc added 2 commits February 7, 2023 20:38

fix issue for re-RegisterFrameTransformer.

60d0db0

dispose.

b6c80e4

theomonnom reviewed Feb 17, 2023

View reviewed changes

sdk/android/src/jni/pc/frame_cryptor_key_manager.h Outdated Show resolved Hide resolved

sdk/objc/api/peerconnection/RTCFrameCryptorKeyManager.mm Outdated Show resolved Hide resolved

sdk/android/src/jni/pc/frame_cryptor_key_manager.h Outdated Show resolved Hide resolved

theomonnom approved these changes Feb 17, 2023

View reviewed changes

sdk/objc/api/peerconnection/RTCFrameCryptorKeyManager.mm Outdated Show resolved Hide resolved

sdk/objc/api/peerconnection/RTCFrameCryptorKeyManager.mm Outdated Show resolved Hide resolved

sdk/android/src/jni/pc/frame_cryptor_key_manager.h Outdated Show resolved Hide resolved

theomonnom reviewed Feb 17, 2023

View reviewed changes

sdk/android/src/jni/pc/frame_cryptor_key_manager.h Outdated Show resolved Hide resolved

cloudwebrtc and others added 6 commits February 20, 2023 13:56

Update sdk/objc/api/peerconnection/RTCFrameCryptorKeyManager.mm

643a0f2

Co-authored-by: Théo Monnom <theo.monnom@outlook.com>

Update sdk/android/src/jni/pc/frame_cryptor_key_manager.h

916139c

Co-authored-by: Théo Monnom <theo.monnom@outlook.com>

Update sdk/objc/api/peerconnection/RTCFrameCryptorKeyManager.mm

921ef89

Co-authored-by: Théo Monnom <theo.monnom@outlook.com>

update.

72f554b

Merge branch 'feat/built-in-aes-gcm-frame-cryptor' of https://github.…

a1ace5d

…com/webrtc-sdk/webrtc into feat/built-in-aes-gcm-frame-cryptor

fix compile.

69aa67c

cloudwebrtc merged commit 49058cc into m104_release Feb 20, 2023

cloudwebrtc deleted the feat/built-in-aes-gcm-frame-cryptor branch February 20, 2023 06:16

cloudwebrtc added a commit that referenced this pull request Jun 12, 2023

Frame Cryptor Support.

5b56ea1

feat: Frame Cryptor (aes gcm/cbc). (#54) feat: key ratchet/derive. (#66) Co-authored-by: Théo Monnom <theo.monnom@outlook.com>

cloudwebrtc mentioned this pull request Jun 12, 2023

update to m114 #76

Merged

cloudwebrtc added a commit that referenced this pull request Jun 12, 2023

Frame Cryptor Support.

b629056

feat: Frame Cryptor (aes gcm/cbc). (#54) feat: key ratchet/derive. (#66) Co-authored-by: Théo Monnom <theo.monnom@outlook.com>

cloudwebrtc added a commit that referenced this pull request Jun 12, 2023

Frame Cryptor Support.

4dd33c6

feat: Frame Cryptor (aes gcm/cbc). (#54) feat: key ratchet/derive. (#66) Co-authored-by: Théo Monnom <theo.monnom@outlook.com>

cloudwebrtc added a commit that referenced this pull request Jul 12, 2023

Frame Cryptor Support.

24a6a7a

feat: Frame Cryptor (aes gcm/cbc). (#54) feat: key ratchet/derive. (#66) fix: skip invalid key when decryption failed. (#81) Co-authored-by: Théo Monnom <theo.monnom@outlook.com>

cloudwebrtc added a commit that referenced this pull request Jul 12, 2023

Frame Cryptor Support.

eaa0969

feat: Frame Cryptor (aes gcm/cbc). (#54) feat: key ratchet/derive. (#66) fix: skip invalid key when decryption failed. (#81) Co-authored-by: Théo Monnom <theo.monnom@outlook.com>

cloudwebrtc added a commit that referenced this pull request Jul 13, 2023

Frame Cryptor Support.

981631a

feat: Frame Cryptor (aes gcm/cbc). (#54) feat: key ratchet/derive. (#66) fix: skip invalid key when decryption failed. (#81) Co-authored-by: Théo Monnom <theo.monnom@outlook.com>

cloudwebrtc added a commit that referenced this pull request Jul 13, 2023

Frame Cryptor Support.

3a2c008

feat: Frame Cryptor (aes gcm/cbc). (#54) feat: key ratchet/derive. (#66) fix: skip invalid key when decryption failed. (#81) Co-authored-by: Théo Monnom <theo.monnom@outlook.com>

cloudwebrtc mentioned this pull request May 22, 2024

Update to m125. #119

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Frame Cryptor (aes gcm/cbc). #54

feat: Frame Cryptor (aes gcm/cbc). #54

cloudwebrtc commented Nov 28, 2022 •

edited

Loading

theomonnom left a comment

cloudwebrtc commented Feb 17, 2023

theomonnom left a comment

feat: Frame Cryptor (aes gcm/cbc). #54

feat: Frame Cryptor (aes gcm/cbc). #54

Conversation

cloudwebrtc commented Nov 28, 2022 • edited Loading

theomonnom left a comment

Choose a reason for hiding this comment

cloudwebrtc commented Feb 17, 2023

theomonnom left a comment

Choose a reason for hiding this comment

cloudwebrtc commented Nov 28, 2022 •

edited

Loading