Audio waveform widget

[dumblob]

After using Gaupol for some time I must say it's a great tool, but I lack one thing. It's pretty difficult to make the timing right if there is just the audio feedback and the speed the reaction of your fingers. I think way better would be to have a simple audio frequency visualization with the possibility to select range with mouse. One could then add a new subtitle record, write some text and then select beginning with mouse left click and select end with mouse right click - both on a stripe visualizing the audio frequency.

The selection on the stripe shall be synchronized with selected subtitle record, so that one can scroll in the list of subtitle records, click on one and the visualization of audio frequency will immediately select the corresponding range.

What do you think?

[otsaloma]

I agree, it would be useful. An audio waveform display was on the TODO-list for a long time, until I concluded that it sounds like a lot of work and I don't have the time to do it in the foreseeable future. So, I'm unlikely to do it, but I would accept contributions and provide any help needed.

[cirosantilli]

I think https://github.com/Aegisub/Aegisub is the way to go nowadays.

[milahu]

I think https://github.com/Aegisub/Aegisub is the way to go nowadays.

active fork https://github.com/TypesettingTools/Aegisub

the beauty of a python tool: no need to compile, so its easier to hack

[mvaranda]

I felt the same. I am adding a wave widget in my clone and will create a PR once I have something featured and clean.

[mvaranda]

WIP:
https://rumble.com/v2jzbv2-april-23-2023.html

[otsaloma]

Looks good so far.

I'm here and on Gitter, whichever you prefer, if you have questions. (Gitter seems to have gone full Matrix, I'm not up to speed on that, might not have notifications set up properly, but maybe I'll figure it out.)

[mvaranda]

Thanks, I would prefer to avoid creating accounts in another forum sites (Gitter). We can communicate here, Linkedin or Signal. Here may be better as others can also learn. As I explained in my video I did not understand how your Signal/emit implementation works. I tried creating methods starting with "on_". Connect requires self/self/signal-name or self/"<method_name>"/signal-name". I could not make the dispatcher happy. Not sure which attribute it is looking for. the self/self/signal-name" seems to have no point as I want to send signals cross objects (not to itself).

Creating a signal in my Observable derived class seem to not become global as emitting it from other objects do nothing (handler never called). The code under "mv" has the class "SignalPoster" with a hack as I was not able to emit signal without getting reference from the object "owning" the signal. Giving parent access to children is bad architecture and I want to remove that hack.

I also tried to add emit from "page" instance to my Waveviewer (or maybe SignalPoster) to get notification when the subtitles object has changed. I tried different things and I was not able to figure out the way to go.

My suggestion: when you have a chance please write down a few lines showing how two instances from different classes can emit signals to/from each other. That would be helpful.

BTW... this project is very well implemented. I believe that by just adding a few shortcuts combined with mouse ( dragging) would increase the productivity for the users. For example, in Blender you can click/hold/drag any numeric field to increase/decrease values. That would speed-up changing start/end time. "Shift" key decreases the sensibility while dragging when precision is desired. This is just one example; typing to adjust time is not productive.

Thanks a lot,
Marcelo

[otsaloma]

Does this help?

#!/usr/bin/env python3

import aeidon

class Child(aeidon.Observable):

    signals = ("stuff-done",)

    def do_stuff(self):
        # Do something here,
        # once done emit:
        self.emit("stuff-done")

class Parent:

    def __init__(self):
        self.child = Child()
        self.child.connect("stuff-done", self._on_child_stuff_done)

    def _on_child_stuff_done(self, child):
        print("Callback: _on_child_stuff_done!")

parent = Parent()
parent.child.do_stuff()

Note that many parts use the helper function aeidon.util.connect as it allows shorter code, but as a tradeoff, it's probably just too much magic under the hood and unclear as a result. In here it'd be

aeidon.util.connect(self, "child", "stuff-done")

but it's probably better to avoid that helper.

[otsaloma]

Also note that GTK widgets are GObjects and they have the signal mechanism from GObject. GObject and Observable and different implementations of the same thing and you can't mix them in a single class. Using custom signals in a GTK class is complicated, badly documented and at worst liable to break with new GTK/GObject versions. I'd recommend avoiding it. For an example of that, see CellTextView in https://github.com/otsaloma/gaupol/blob/master/gaupol/renderers/multiline.py#L34

[mvaranda]

That helps. I fixed signal for SignalPoster class. But still not clear how classes without relationship (parenthood) can communicate. Please check the other (ugly) code under:
https://github.com/Varanda-Labs/gaupol/blob/mv/gaupol/page.py
line 147

Forces exposing the singleton line 87 of:
https://github.com/Varanda-Labs/gaupol/blob/mv/gaupol/waveview.py

Do you have a suggestion for a better communication? In Qt and I guess also for GObject we can connect class/instance to other class/instance without relationship as far as I remember.

Thanks in advance,
MV

[otsaloma]

The way signals work in GTK (and which I have copied for the Observable class) is that objects emit signals with no regard to who might be interested. It's up to other objects then to "subscribe" to signals with the connect method.

So, here, if I understand correctly

1. Waveview or GraphicArea should emit "request-seek" etc. which Application (probably in VideoAgent) or Page can then connect to, to do the corresponding video player or subtitle view updates.
2. Making the GraphicArea respond to subtitle changes gets more complicated, but maybe you can reuse the same subtitle cache mechanism that already exists for the video player. It's perhaps not elegant, but it's already there and works. See

When the subtitle cache is updated:

gaupol/gaupol/agents/video.py

Lines 52 to 57 in c75b4fb

	def _init_cache_updates(self):
	"""Initialize cache updates on application signals."""
	self.connect("page-added", self._update_subtitle_cache)
	self.connect("page-changed", self._update_subtitle_cache)
	self.connect("page-closed", self._update_subtitle_cache)
	self.connect("page-switched", self._update_subtitle_cache)

The cache itself (start, end, text):

gaupol/gaupol/agents/video.py

Lines 374 to 380 in c75b4fb

	def _update_subtitle_cache(self, *args, **kwargs):
	"""Update subtitle position and text cache."""
	page = self.get_current_page()
	if self.player is None or page is None:
	return self._clear_subtitle_cache()
	self._cache = [(x.start_seconds, x.end_seconds, x.main_text)
	for x in page.project.subtitles]

And how it's used to update subtitles in the video player:

gaupol/gaupol/agents/video.py

Lines 238 to 253 in c75b4fb

	def _on_player_update_subtitle(self, data=None):
	"""Update subtitle overlay from video position."""
	if not self.player.ready:
	return True # to be called again.
	pos = self.player.get_position(aeidon.modes.SECONDS)
	if pos is None:
	return True # to be called again.
	subtitles = list(filter(lambda x: x[0] <= pos <= x[1], self._cache))
	if subtitles:
	text = subtitles[-1][2]
	if text != self.player.subtitle_text_raw:
	self.player.subtitle_text = text
	else:
	if self.player.subtitle_text:
	self.player.subtitle_text = ""
	return True # to be called again.

The cache is basically the current texts of the current page. It gets updated in the video player on a 10 ms continuous loop.

Also, I don't think using singletons makes sense here. Application-level widgets should be instance attributes of the application, see the player widgets for comparison. Application-level widgets will be instantiated only once per application, so it's practically a singleton, but not technically.

[mvaranda]

Thanks a lot. I will check/digest these info once I have some time.

[mvaranda]

I did some more work this weekend:

https://rumble.com/v2l9ypa-gaupol-with-wave-widget-update-1.html

I may be done for now about features and integration. I may add a context menu for wave settings and will create a PR.
Thanks for your help,
MV

[otsaloma]

Looks good again so far.

One thing I notice is that you seem to be bypassing the undo-redo system, which is a problem. I'd recommend making the modifications on drag end with Project.set_start and Project.set_end. That way they're registered in the undo-redo system and users will have a consistent undo history. If I remember correctly, page.view should also automatically update via the signals if you use the proper set methods.

There are probably some other issues but I don't see anything major, those can be discussed in a PR line-by-line comments.

[mvaranda]

Good call, I believe that I fixed it now. You can double check, mv-progress-demo branch is updated.
Next weekend I will try to wrap it up.

[mvaranda]

Hello Osmo,

Latest demo:
https://rumble.com/v2meb2e-gaupol-with-wave-widget-update-2.html

I believe that I reached the completion of what I intend to do. There will be better ways to integrate my code with the overall application for complying with the architecture and designed. I would suggest you to make those changes as everything in your code is very organized. I may not have time or motivation to fully comply with the original design as it may lead to many back and forth interactions during PR review. Therefore, I would suggest the following:

• Create an integration branch in your repo and I will create a PR for the new branch
• We can have a couple iterations for bug fixing and once it is bug-free you can do the merge.
• You take your own time to smooth out the changes to match a better integration according to our design.

Missing:

• I did not update the Language files. There are a few new UI labels that need entries in those PO files. I guess you have used some script or tool to auto generate those files.

Thanks for your help and for creating this nice app.
Cheers,
MV

[otsaloma]

OK, thanks. There's now an "audio-waveform" branch you can do the PR against.

You might have noticed that I'm not interested in spending a lot of time on Gaupol anymore (#204), so I'm not going to do much work on this either. The audio waveform doesn't need to be in any way complete for the merge, but things that are implemented should work and fit right. I don't see any major issues, but probably a lot of smaller ones.

Language files are not an issue, I update those prior to releases. The only thing you need to do there is from aeidon.i18n import _ and then wrap UI strings in _("...").

[mvaranda]

Are you planning to do a new release?

[otsaloma]

Are you planning to do a new release?

There isn't much to release, just some translation commits, but those translations have been broken for maybe five years and no one complained apart from a translator, so it's probably no big issue. Is it the translations you're interested in or something else?