Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

smoother waveform #589

Merged
merged 5 commits into from
Jun 14, 2024
Merged

smoother waveform #589

merged 5 commits into from
Jun 14, 2024

Conversation

ComfyFluffy
Copy link
Contributor

@ComfyFluffy ComfyFluffy commented Jun 11, 2024

  • The waveform now looks smoother and more stable.
  • Performance is improved by using cache when showing waveform in recording.
image image

@ComfyFluffy ComfyFluffy changed the title smoothed waveform smoother waveform Jun 11, 2024
@ComfyFluffy ComfyFluffy linked an issue Jun 13, 2024 that may be closed by this pull request
let sum = 0
for (let j = 0; j < blockSize; j++) {
sum += Math.abs(channel[i * blockSize + j])
try {
Copy link
Collaborator

@nighca nighca Jun 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

如语音沟通,这里有两个已知的缺陷:

  1. 录制(recording: true)与文件(recording: false)两种情况下,绘制结果的平滑程度不同;虽然都走了这里的 renderFunction

    这个差异应该是 peaks 的密度差异带来的,即,wavesurfer-recordrenderMicStream 处理与 wavesurfer 对文件处理逻辑的差异带来的

  2. blockSize 不会随着音频长度增加持续增加,这导致在录制过程中,录制越久,波形越不平滑

    这是因为 blockSize 越大,累计单个 block 需要的时间越久,波形更新越不频繁,给人“卡顿”的感觉;可能的处理方式是:不等到累计满一个 block 才去更新波形,不满的 block 也可以被绘制出来,这样在录制越久时,虽然波形变更越慢,但更新频率是固定的,不会显得“卡顿”

可以看下好不好处理,不好处理的话在代码中先注释说明下

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我希望在之后可以重写整个逻辑去掉wavesurfer,这样之后也方便解决渲染逻辑不一样导致的密度不统一的问题。

问题2我看了下,如果说要绘制不满的 block,波形图“向左推”的速度依然会越来越慢,同样会导致视觉上的卡顿。“绘制不满的 block”不能改变“向左推”的速度,而我们预期应该是波形图在视觉上持续地收缩。可能的解决方案是非线性的波形图渲染,但这样可能更加麻烦。我目前倾向于设置一个上限让波形图看起来不至于卡顿。

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我希望在之后可以重写整个逻辑去掉wavesurfer,这样之后也方便解决渲染逻辑不一样导致的密度不统一的问题。

这个我没问题

问题2我看了下,如果说要绘制不满的 block,波形图“向左推”的速度依然会越来越慢,同样会导致视觉上的卡顿。

慢跟卡是不一样的,如果更新是高频率的,即使变更很小,也不应该会感觉到卡

“绘制不满的 block”不能改变“向左推”的速度,而我们预期应该是波形图在视觉上持续地收缩。

绘制不满的 block 不是为了改变“向左推”的速度,而是让眼睛看到的“推”更频繁(虽然每次推的距离在缩小)

可能的解决方案是非线性的波形图渲染,但这样可能更加麻烦。

这个必要性不大

我目前倾向于设置一个上限让波形图看起来不至于卡顿。

如上面提到的,卡是问题,但慢不应该是问题

如果是说 blockSize 频繁变更(对应均值计算结果的缓存失效)导致卡顿(大量的点重新计算均值),这个我能理解,不过那是另一个问题了,涉及的应该是 blockSize 的阶梯,而不是上限

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

绘制不满的 block 不是为了改变“向左推”的速度,而是让眼睛看到的“推”更频繁(虽然每次推的距离在缩小)

懂了。我想了一下,按目前的实现要做到这种效果会比较麻烦。我先加个注释,之后要再重构的时候一起想办法实现

@qiniu-ci
Copy link

The PR environment is ready, please check the PR environment

[Attention]: This environment will be automatically cleaned up after a certain period of time., please make sure to test it in time. If you have any questions, please contact the builder team.

@nighca nighca merged commit d286987 into goplus:dev Jun 14, 2024
4 checks passed
@nighca nighca mentioned this pull request Jun 17, 2024
nighca added a commit that referenced this pull request Aug 16, 2024
* dropdown with tooltip (#594)

* feat: UIDropdownWithTooltip

* fix: missing import for UILoading

* fix: logic

* lint: enable vue/no-undef-components (#595)

* lint: enable vue/no-undef-components

* add comments

* fix: delay on click to stop SoundPlayer (#596)

* smoother waveform (#589)

* smoothed waveform

* update comments

* do not reload audio on record end

* adjust waveform scale to match scratch

* add comments

* load audio after recording (#598)

* Product doc for animation support (#597)

* product doc for animation

* update doc for animation

* refactor: sound editor & recorder (#607)

* refactor: sound editor & recorder

* better animation & cache

* fix: export with gain

* fix: not emitting stop on end

* reset progress on stop

* cleanup & comments

* refactor:

* refactor:

* move to a seperate dir

* uninstall wavesurfer

* add comments

* Fix last-select issue when project switched (#612)

* fix last-select issue when project switched

* avoid effectScope

* Model `Animation` (#609)

* model animation

* prettier

* typo

* Costume pre-processing: split-sprite-sheet (#614)

* costume preprocessing

* details

* typo

* Remove right-click menu for backdrop (#616)

fix #606

* fix normalizeAssetName (#619)

* Save text files directly to the DB for performance (#617)

Updates #369

* update model animation (#620)

* component `AnimationSettings` (#621)

* component AnimationSettings

* use UIIcon instead of inline svg

* fix svg icon

* fix cancelled warning in console (#625)

* fix(spx-gui): save to cloud after sharing project and remove login requirement for making signed web URLs (#631)

Fixes #627

* refactor(spx-gui): remove "codeFile" concept and optimize project saving (#630)

- Removed the "codeFile" concept from sprite and stage models introduced
  in #508. Since #617, lazy-loading for plain text files is no longer
  needed.
- Avoided redundant `parseProjectData` in `@/models/common/cloud.save`.

Fixes #369

* Animation Editor UI (#626)

* feat: modal

* feat:

* modals

* remove checkerboard files

* feat: sound playing

* image preload

* refactor

* hide onTouched (#632)

* Update code-editor for animation (#629)

* update code-editor for animation

* update costume tools

* fix costume add

* Preload frames & audio / removal modal (#633)

* feat: remove costume & preload frames

* fix: use ref for muted

* animation details (#634)

* Update docs (#636)

* update docs

* add screenshot

* Use center point as sprite pivot (#615)

* use center point as sprite pivot

* upgrade spx

* refactor preload / modify translation (#635)

* refactor preload / modify translation

* do not update on zero duration

* refactor: disposer

* refactor: use single watch hook

* fix: start timing / remove deep watch

* disposer -> disposable

* fix: use old sound on switch

* `RotationStyle` support (#639)

* Debug animation sound on project load (#640)

* add preview for `GroupCostumesModal` (#646)

* Product doc for Widget (#644)

* add en docs

* doc for widget

* Support remove-background (#650)

* support remove-background

* debug details

* api for aigc

* details

* add comment for TODO

* exclude image url targeting private network

* optimize details

* spx-gui: implement automatic project save to cloud (#649)

- Added functionality to automatically save projects to the cloud.
- Implemented retry mechanism for failed save attempts with `5s`
  intervals.
- Ensured debounce handling for save operations with a `1.5s` delay to
  prevent frequent saves.
- Monitored changes in exported game files to trigger auto-save.
- Saved to local cache if auto-save to cloud fails to prevent data loss.
- Cleared local cache after successful cloud save.
- Switched between saving to cloud and local cache based on
  connectivity.
  - Save to cloud when online.
  - Save to local cache when offline.
- Replaced the `Save` button in the `TopNav` with an auto-save status
  icon.
- Prevented the user from leaving the current project (navigating away,
  refreshing, or closing the page) when there are unsynced changes.

Fixes #541

* Uniform item style (#641)

* refactor: editor items

* undo changes for library AssetItem

* refactor: library item

* fix type error

* update project item design

* refactor: use UIBlockItem

* fix: ScratchItemContainer

* refactor: UISpriteItem

* sound & backdrop

* remove editor item

* refactor:

* refactor & update design

* update slot name

* refactor: small editor sound/sprite

* style cleanup

* refactors

* uiBlockItem default color to primary

* move components to a folder

* use visible instead of v-if

* img loading mask

* remove noLoadingMask option

* Migrate to ghcr.io (#654)

* dockerfile: utilize pre-built images for Go+ and Nginx (#660)

- Replaced manual installation with pre-built images for Go+ and Nginx.
- Modified CMD to enable log output.

* dockerfile: fix nginx.conf copy path (#661)

* spx-gui: avoid calling `next` within `router.beforeEach` (#664)

See https://github.com/vuejs/rfcs/blob/master/active-rfcs/0037-router-return-guards.md#motivation

* spx-gui: remove unnecessary `defineProps` and `defineEmits` imports (#666)

* adapt animation for spx API change (#647)

* pr-preview: use mirrors for Go and Node modules downloads during `docker build` (#667)

- Updated `Dockerfile`:
  - Introduced `ARG`s for base images and custom mirror configs.
  - Modified the `CMD` instruction to use `exec` for proper signal
    handling.
- Updated `scripts/pr-preview.sh`:
  - Refined `docker build` command to utilize the newly introduced
    `ARG`s.

* spx-gui: implement batched object URL processing and optimize lazy loading (#668)

- Extended `util.makeObjectUrls` with batching to queue and merge object
  URL requests.
- Updated `cloud.universalUrlToWebUrl` to use batched processing.
- Refactored `cloud.getFiles` and file creation process to support lazy
  URL resolution, addressing potential URL expiry issues.

Fixes #653

* spx-gui: delay input URL processing in RemoveBackground component (#669)

* fix required prop warnings (#673)

* feat: add build script to support windows (#675)

* spx-gui: prevent event propagation on UICornerIcon click (#674)

* Fix animation sound loss during undo/redo (#718)

* fix warning

* fix animation sound during export & load

* keep order of sprites & sounds (#720)

* configurable AIGC feature (#726)

* Self-host monaco instead of `cdn.jsdelivr.net` (#729)

* spx-gui: upload files when adding resource (#732)

- Added loading message type for global loading-feedback.
- Integrated file upload process into `PreprocessModal`'s confirmation
  for files requiring preprocessing.
- Implemented loading message display for file uploads without
  preprocessing.
- Ensured resource addition only completes after successful file upload.
- Fixed filename change issue in `PreprocessModal` caused by
  `RemoveBackground` application.

Fixes #638

* feat: custom sprite transformer (#723)

* konva flip demo

* feat: sprite transformer

* update button color

* small refactors

* update vue-konva

* ui changes & flip in Transformer

* fix type error

* cursor behavior

* fix cursor when not dragging

* fix update on spriteRotationStyleChange

* move & refactor

* update cursor style

* not allowed cursor

* fix: centeredScaling

* add comment for flip button

* spx-gui: auto save unsaved changes on route change (#737)

We block route changes when there are unsaved changes using
`UIConfirmDialog`. For better UX, an automatic save is triggered when
the dialog is mounted, reducing the need for users to manually click the
`Save` button. If the save failes, users can simply retry by clicking
`Save`.

Fixes #665

* spx-gui(UIDropdown): fix `NPopover.setShow` not triggering `on-update:show` (#738)

* update favicon (#747)

* Automatically select the first animation on sprite switch (#749)

* Automatically select the first animation on sprite switch

* fix: not auto select on delete

* update casdoor config (#762)

* update casdoor config

* update backend env

* Widget Monitor (#676)

* widget monitor

* placeholder for empty widget list

* fix detail

* debug with spx

---------

Co-authored-by: ComfyFluffy <24245520+ComfyFluffy@users.noreply.github.com>
Co-authored-by: Aofei Sheng <aofei@aofeisheng.com>
Co-authored-by: molinla <61045398+molinla@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SoundEditor UI details
3 participants