Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Voice messages #4433

Merged
merged 65 commits into from
Jul 22, 2021
Merged

Voice messages #4433

merged 65 commits into from
Jul 22, 2021

Conversation

stefanceriu
Copy link
Member

@stefanceriu stefanceriu commented Jun 15, 2021 •
edited by gileluard
Loading

fix #4090, fix #4091, fix #4092, fix #4094, fix #4095, fix #4096

Added initial voice messages implementation:

  • custom audio recorder based on AVAudioRecorder
  • new recording toolbar: long press to record, swipe left to cancel, release to send, duration label and red dot animation
  • custom audio player based on AVPlayer
  • audio message bubble cells which still supports all interaction types (reply, react etc.)
  • custom audio waveformView with sampling done through DSWaveformImage's WaveformAnalyzer
  • custom audio playback view

Notes:

  • currently recording and seding MPEG4AAC .m4a files, we need to replace that with opus ogg
  • missing locked mode recording UI
  • feature flag unde BuildSettings.voiceMessagesEnabled

@stefanceriu stefanceriu changed the title Element 4090 Voice messages Jun 15, 2021
@stefanceriu stefanceriu force-pushed the element_4090 branch 2 times, most recently from 3e52d2f to da7a136 Compare June 18, 2021 14:59
gileluard and others added 12 commits June 22, 2021 09:42
@gileluard @stefanceriu
Voice Messages - Hold and send
93fd5cd 
- Added voice message button
- voice recording UI state
@stefanceriu
#4090 - New voice recording toolbar ui and swipe to cancel interaction.
3624d2e
@stefanceriu
#4090 - Corrected dark theme appearance.
3015cba
@stefanceriu
#4090 - Corrected dark theme appearance.
b507097
@stefanceriu
#4090 - Add voice message controller, audio recorder and toolbar view…
8d54e03 
… links. Working audio file sending and cancellation.
@stefanceriu
#4090 - Added recording duration label and permissions checking.
58c1be0
@stefanceriu
#4096 - Added voice message decoding, timeline cell and playback UI.
ff3170e
@stefanceriu
#4090 - Added voice messages feature flag.
1294c27
@stefanceriu
#4096 - Voice message playback view theme support.
53307f9
@stefanceriu
#4096 - Various small UI tweaks and bug fixes.
3c1bc56
@stefanceriu
#4096 - Extracted logic from the VoiceMessagePlaybackView. Exposed po…
99eb2d1 
…wer levels from audio recorder.
@stefanceriu
#4096 - Added locked mode transition and animations, locked recording…
3130fd5 
… mode and real time waveform.
SBiOSoftWhare
SBiOSoftWhare reviewed Jun 22, 2021
View reviewed changes
@stefanceriu stefanceriu requested a review from gaelledel July 14, 2021 12:59
@stefanceriu
#4094 - Redrawing waveforms on bound changes.
521c713
@stefanceriu
#4094 - Reintroduced serial attachment loading processing queue and f…
9a3d26e 
…ixed completionCallback storage so they take the requestedNumberOfSamples into account.
@stefanceriu
#4090 - Switched the sendVoiceMessage method duration parameter to an…
511569a 
… integer.
@stefanceriu
Revert "#4094 - Reintroduced serial attachment loading processing que…
cf884c6 
…ue and fixed completionCallback storage so they take the requestedNumberOfSamples into account."

This reverts commit 9a3d26e.
@stefanceriu
#4090 - Changed FFMpeg-Kit to the LTS version and moved the project b…
488338f 
…ack to deployment target 11.0.
@stefanceriu
Revert "Revert "#4094 - Reintroduced serial attachment loading proces…
427e77e 
…sing queue and fixed completionCallback storage so they take the requestedNumberOfSamples into account.""

This reverts commit cf884c6.
@stefanceriu
#4090 - Fixed the AttachmentCacheManager's serial nature.
ec33ec8
pixlwave
pixlwave reviewed Jul 16, 2021
View reviewed changes
pixlwave
pixlwave reviewed Jul 19, 2021
View reviewed changes
Riot/Modules/Room/VoiceMessages/VoiceMessageAudioRecorder.swift

func recordWithOuputURL(_ url: URL) {

let settings = [AVFormatIDKey: Int(kAudioFormatMPEG4AAC),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably not an issue, but as an audio nerd it bothers me that this records to a lossy format rather than lossless when it's going to be converted to another lossy format immediately before sending. 🤓

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Talked about this privatelly and decided to try to only switch the container format when converting, using opus here.

@stefanceriu
#4090 - Various tweaks and fixes following code review. Switched back…
0ead60f 
… to DateFormatters for formatting durations, sanitising audio player durations and current times.
@manuroe manuroe merged commit b3cca16 into develop Jul 22, 2021
@manuroe manuroe deleted the element_4090 branch July 22, 2021 13:13
@MacLemon
Copy link

Will there be a possibility for Matrix servers to refuse that message type to conserve storage space?

I'm honestly not keen on receiving any audio from people who are too lazy to type or unwilling to use voice recognition to accommodate for their laziness.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SBiOSoftWhare SBiOSoftWhare SBiOSoftWhare left review comments

@ismailgulek ismailgulek ismailgulek left review comments

@pixlwave pixlwave pixlwave left review comments

@gaelledel gaelledel gaelledel approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
10 participants
@stefanceriu @philip-rehorst @ismailgulek @gileluard @Derad6709 @MacLemon @SBiOSoftWhare @pixlwave @gaelledel @manuroe