-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
smoother waveform #589
smoother waveform #589
Conversation
ComfyFluffy
commented
Jun 11, 2024
•
edited
Loading
edited
- The waveform now looks smoother and more stable.
- Performance is improved by using cache when showing waveform in recording.
let sum = 0 | ||
for (let j = 0; j < blockSize; j++) { | ||
sum += Math.abs(channel[i * blockSize + j]) | ||
try { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
如语音沟通,这里有两个已知的缺陷:
-
录制(
recording: true
)与文件(recording: false
)两种情况下,绘制结果的平滑程度不同;虽然都走了这里的renderFunction
这个差异应该是
peaks
的密度差异带来的,即,wavesurfer-record
中renderMicStream
处理与 wavesurfer 对文件处理逻辑的差异带来的 -
blockSize 不会随着音频长度增加持续增加,这导致在录制过程中,录制越久,波形越不平滑
这是因为 blockSize 越大,累计单个 block 需要的时间越久,波形更新越不频繁,给人“卡顿”的感觉;可能的处理方式是:不等到累计满一个 block 才去更新波形,不满的 block 也可以被绘制出来,这样在录制越久时,虽然波形变更越慢,但更新频率是固定的,不会显得“卡顿”
可以看下好不好处理,不好处理的话在代码中先注释说明下
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
我希望在之后可以重写整个逻辑去掉wavesurfer,这样之后也方便解决渲染逻辑不一样导致的密度不统一的问题。
问题2我看了下,如果说要绘制不满的 block,波形图“向左推”的速度依然会越来越慢,同样会导致视觉上的卡顿。“绘制不满的 block”不能改变“向左推”的速度,而我们预期应该是波形图在视觉上持续地收缩。可能的解决方案是非线性的波形图渲染,但这样可能更加麻烦。我目前倾向于设置一个上限让波形图看起来不至于卡顿。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
我希望在之后可以重写整个逻辑去掉wavesurfer,这样之后也方便解决渲染逻辑不一样导致的密度不统一的问题。
这个我没问题
问题2我看了下,如果说要绘制不满的 block,波形图“向左推”的速度依然会越来越慢,同样会导致视觉上的卡顿。
慢跟卡是不一样的,如果更新是高频率的,即使变更很小,也不应该会感觉到卡
“绘制不满的 block”不能改变“向左推”的速度,而我们预期应该是波形图在视觉上持续地收缩。
绘制不满的 block 不是为了改变“向左推”的速度,而是让眼睛看到的“推”更频繁(虽然每次推的距离在缩小)
可能的解决方案是非线性的波形图渲染,但这样可能更加麻烦。
这个必要性不大
我目前倾向于设置一个上限让波形图看起来不至于卡顿。
如上面提到的,卡是问题,但慢不应该是问题
如果是说 blockSize 频繁变更(对应均值计算结果的缓存失效)导致卡顿(大量的点重新计算均值),这个我能理解,不过那是另一个问题了,涉及的应该是 blockSize 的阶梯,而不是上限
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
绘制不满的 block 不是为了改变“向左推”的速度,而是让眼睛看到的“推”更频繁(虽然每次推的距离在缩小)
懂了。我想了一下,按目前的实现要做到这种效果会比较麻烦。我先加个注释,之后要再重构的时候一起想办法实现
The PR environment is ready, please check the PR environment [Attention]: This environment will be automatically cleaned up after a certain period of time., please make sure to test it in time. If you have any questions, please contact the builder team. |
* dropdown with tooltip (#594) * feat: UIDropdownWithTooltip * fix: missing import for UILoading * fix: logic * lint: enable vue/no-undef-components (#595) * lint: enable vue/no-undef-components * add comments * fix: delay on click to stop SoundPlayer (#596) * smoother waveform (#589) * smoothed waveform * update comments * do not reload audio on record end * adjust waveform scale to match scratch * add comments * load audio after recording (#598) * Product doc for animation support (#597) * product doc for animation * update doc for animation * refactor: sound editor & recorder (#607) * refactor: sound editor & recorder * better animation & cache * fix: export with gain * fix: not emitting stop on end * reset progress on stop * cleanup & comments * refactor: * refactor: * move to a seperate dir * uninstall wavesurfer * add comments * Fix last-select issue when project switched (#612) * fix last-select issue when project switched * avoid effectScope * Model `Animation` (#609) * model animation * prettier * typo * Costume pre-processing: split-sprite-sheet (#614) * costume preprocessing * details * typo * Remove right-click menu for backdrop (#616) fix #606 * fix normalizeAssetName (#619) * Save text files directly to the DB for performance (#617) Updates #369 * update model animation (#620) * component `AnimationSettings` (#621) * component AnimationSettings * use UIIcon instead of inline svg * fix svg icon * fix cancelled warning in console (#625) * fix(spx-gui): save to cloud after sharing project and remove login requirement for making signed web URLs (#631) Fixes #627 * refactor(spx-gui): remove "codeFile" concept and optimize project saving (#630) - Removed the "codeFile" concept from sprite and stage models introduced in #508. Since #617, lazy-loading for plain text files is no longer needed. - Avoided redundant `parseProjectData` in `@/models/common/cloud.save`. Fixes #369 * Animation Editor UI (#626) * feat: modal * feat: * modals * remove checkerboard files * feat: sound playing * image preload * refactor * hide onTouched (#632) * Update code-editor for animation (#629) * update code-editor for animation * update costume tools * fix costume add * Preload frames & audio / removal modal (#633) * feat: remove costume & preload frames * fix: use ref for muted * animation details (#634) * Update docs (#636) * update docs * add screenshot * Use center point as sprite pivot (#615) * use center point as sprite pivot * upgrade spx * refactor preload / modify translation (#635) * refactor preload / modify translation * do not update on zero duration * refactor: disposer * refactor: use single watch hook * fix: start timing / remove deep watch * disposer -> disposable * fix: use old sound on switch * `RotationStyle` support (#639) * Debug animation sound on project load (#640) * add preview for `GroupCostumesModal` (#646) * Product doc for Widget (#644) * add en docs * doc for widget * Support remove-background (#650) * support remove-background * debug details * api for aigc * details * add comment for TODO * exclude image url targeting private network * optimize details * spx-gui: implement automatic project save to cloud (#649) - Added functionality to automatically save projects to the cloud. - Implemented retry mechanism for failed save attempts with `5s` intervals. - Ensured debounce handling for save operations with a `1.5s` delay to prevent frequent saves. - Monitored changes in exported game files to trigger auto-save. - Saved to local cache if auto-save to cloud fails to prevent data loss. - Cleared local cache after successful cloud save. - Switched between saving to cloud and local cache based on connectivity. - Save to cloud when online. - Save to local cache when offline. - Replaced the `Save` button in the `TopNav` with an auto-save status icon. - Prevented the user from leaving the current project (navigating away, refreshing, or closing the page) when there are unsynced changes. Fixes #541 * Uniform item style (#641) * refactor: editor items * undo changes for library AssetItem * refactor: library item * fix type error * update project item design * refactor: use UIBlockItem * fix: ScratchItemContainer * refactor: UISpriteItem * sound & backdrop * remove editor item * refactor: * refactor & update design * update slot name * refactor: small editor sound/sprite * style cleanup * refactors * uiBlockItem default color to primary * move components to a folder * use visible instead of v-if * img loading mask * remove noLoadingMask option * Migrate to ghcr.io (#654) * dockerfile: utilize pre-built images for Go+ and Nginx (#660) - Replaced manual installation with pre-built images for Go+ and Nginx. - Modified CMD to enable log output. * dockerfile: fix nginx.conf copy path (#661) * spx-gui: avoid calling `next` within `router.beforeEach` (#664) See https://github.com/vuejs/rfcs/blob/master/active-rfcs/0037-router-return-guards.md#motivation * spx-gui: remove unnecessary `defineProps` and `defineEmits` imports (#666) * adapt animation for spx API change (#647) * pr-preview: use mirrors for Go and Node modules downloads during `docker build` (#667) - Updated `Dockerfile`: - Introduced `ARG`s for base images and custom mirror configs. - Modified the `CMD` instruction to use `exec` for proper signal handling. - Updated `scripts/pr-preview.sh`: - Refined `docker build` command to utilize the newly introduced `ARG`s. * spx-gui: implement batched object URL processing and optimize lazy loading (#668) - Extended `util.makeObjectUrls` with batching to queue and merge object URL requests. - Updated `cloud.universalUrlToWebUrl` to use batched processing. - Refactored `cloud.getFiles` and file creation process to support lazy URL resolution, addressing potential URL expiry issues. Fixes #653 * spx-gui: delay input URL processing in RemoveBackground component (#669) * fix required prop warnings (#673) * feat: add build script to support windows (#675) * spx-gui: prevent event propagation on UICornerIcon click (#674) * Fix animation sound loss during undo/redo (#718) * fix warning * fix animation sound during export & load * keep order of sprites & sounds (#720) * configurable AIGC feature (#726) * Self-host monaco instead of `cdn.jsdelivr.net` (#729) * spx-gui: upload files when adding resource (#732) - Added loading message type for global loading-feedback. - Integrated file upload process into `PreprocessModal`'s confirmation for files requiring preprocessing. - Implemented loading message display for file uploads without preprocessing. - Ensured resource addition only completes after successful file upload. - Fixed filename change issue in `PreprocessModal` caused by `RemoveBackground` application. Fixes #638 * feat: custom sprite transformer (#723) * konva flip demo * feat: sprite transformer * update button color * small refactors * update vue-konva * ui changes & flip in Transformer * fix type error * cursor behavior * fix cursor when not dragging * fix update on spriteRotationStyleChange * move & refactor * update cursor style * not allowed cursor * fix: centeredScaling * add comment for flip button * spx-gui: auto save unsaved changes on route change (#737) We block route changes when there are unsaved changes using `UIConfirmDialog`. For better UX, an automatic save is triggered when the dialog is mounted, reducing the need for users to manually click the `Save` button. If the save failes, users can simply retry by clicking `Save`. Fixes #665 * spx-gui(UIDropdown): fix `NPopover.setShow` not triggering `on-update:show` (#738) * update favicon (#747) * Automatically select the first animation on sprite switch (#749) * Automatically select the first animation on sprite switch * fix: not auto select on delete * update casdoor config (#762) * update casdoor config * update backend env * Widget Monitor (#676) * widget monitor * placeholder for empty widget list * fix detail * debug with spx --------- Co-authored-by: ComfyFluffy <24245520+ComfyFluffy@users.noreply.github.com> Co-authored-by: Aofei Sheng <aofei@aofeisheng.com> Co-authored-by: molinla <61045398+molinla@users.noreply.github.com>