-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC: Proposed reorganization for automatic signal formatting and implementation of Sounds
API
#44
Conversation
…tting. This is a draft proposal to really mix things up. All the existing tests pass locally, but I have yet to write any new tests. It defines the basis of a new interface which could be used to build the functionality present in the `Sounds` package. The formatting is largely handled through a new trait called SignalFormat. This is the basis of a function, `promote_signals` which can be used to ensure all signals passed are an appropriate format.
One thing I realized I forgot to mention about My thinking was that, in this approach, to design a new kind of sample-buffer-like object with different behaviors you simply wrap some |
Another point that has occurred to me about this is that the resulting That said, having thought about the operators more, I kind of like having |
Hey there, just wanted to check in on this. If you are simply busy and need time to look at this, that's cool. However it would be nice to know if you would no longer like to integrate this functionality, or something like it, into |
Hello. Big apologies for ghosting on this. I haven't been able to go through in detail, but here're some initial thoughts:
Is this equivalent to
I don't really like
Yeah, this is something I've actually been meaning to do as well. Originally I did it this way to make it possible to define zero-allocating
I had it this way in an earlier iteration rather than separate types, but I think that dealing with dispatching on the type parameters was a hassle. That might not apply anymore given the other architectural changes you've made, so I'm definitely open to this.
I think that there's value to having these types more focused on time and frequency-domain signals, and not have to worry about generalized units. It lets us make more assumptions about use-cases.
I'm a little worried that That's all I can get into for now, but I'll think on this a little more and try to find some time to get more familiar with the code. Thanks for putting all the work into this! We'll get it sorted eventually... |
thinking about it a little more, I think your |
Great! I totally understand taking some time to get to this: it's a lot of material. Here are my responses to your initial thoughts:
I think the closest analogy would actually just be
I've thought about this as well; it is definitely very appealing. I ran into two problems in thinking through this possibility:
I see your point here. I definitely go back and forth on whether it makes sense. I think in either case the interface should be consistent: either
Cool, sounds like we're on the same page here.
I think we're also in sync here. And certainly it would be easy to make
I kind of like that idea, it has a nice consistency to it. Seems like a reasonable approach.
But, yeah, it might be worth thinking about another reasonable name that doesn't conflict with these libraries. |
More coming... just wanted to post my thoughts so far. |
Oh! I just realized, I think there is a way to implement This still leaves the issue of whether that's the right approach given that some axes are padded (time) but others aren't (channel), but maybe that is okay, because it is a specific type, where padding is well defined for this specific dimension. |
So I think there are a few orthogonal issues, which maybe can be separated into separate pull requests. Now that I've worked on all of the parts I think I have a better idea of how to best separate these out.
This would mean losing some functionality along the way, as we figure out how each part works, but it seems like a good way of breaking down what needs to be worked out into smaller parts.
|
Ha! Sorry, I keep thinking of new things. I think it is worth trying to extend broadcast, to see if it works: once the other pieces are in place. One can define two new broadcast styles: one for buffers alone and another that includes at least one stream. Then, by overriding |
Hey there, just checking in on this again. I'm just reflecting on this, and I definitely see some confusing pieces that could do with some cleaning up. I think it might be good for me to clean this up more, and try out the broadcasting idea. I think there's a big question of whether broadcasting will work well: if it does I think that addresses some of the naming issues, and will probably help clarify the code somewhat. Sound good? |
Hi @ssfrr, just checking in on this again, I haven't committed any time to this because it wasn't clear whether this is was the direction you wanted to see |
Yeah, that's probably the smart move. I'm trying to limit how much time I spend on things that are not shortest-path to finishing my dissertation, and this big reorganization isn't something that I have much time for. As an aside, I've been doing a little reworking of Ogg.jl and Opus.jl to handle streaming, and have had pretty good results with an |
Aye, yes, by all means, focus on the dissertation! Best wishes! I've wondered about an I think what I'll do for know then is refine an approach for the |
yes, |
If it's sample-by-sample my point about flexibility is moot. I would thinking you iterated block-by-block (in which case, the |
One point that occurs to me is that the two interfaces are inter-operable: you should be able to implement the one with the other. So that might mean that if I work on the general functions I'd like to have (e.g. |
Just an FYI: I've implemented my various operations as proposed in this PR in the package SignalOperators, which should interoperate fairly well with both |
Whew! Okay... this one is large. Quite a few things here—it should eventually
be broken up—but this was in a place where I wanted to actually discuss what
I've been thinking through.
If this is what SampledSignals ends up looking like I'll have some clean-up and
more testing to do and probably some fixes to make, but I'm pretty convinced
the basic approach is do-able and practical approach at this point. And
all of the original tests that make sense for the new approach (all but
a few) pass locally. This needs a few small updates to LibSndFile
so the tests won't pass on CI.
So the main question is whether this is the right way to go. It would
involve some fairly major changes:
mapsignals
which applies a function to each sample ofmultiple signals but automatically pads the end of signals and can operate
over streams, buffers, numbers and any abstract array, automatically
coercing the objects as necssary. This can be used to implement the most of
the methods in
Sounds
. This is really the motivating force behind many ofthe below changes and I think the goal should be to have something that
makes this work well. You can think of the rest of the changes as really a
proposal for how to make
mapsignals
work.The functions
promote_signals(xs...)
andpromote_signal(x,by=y)
inspecttheir objects and call
tosamplerate
andtochannels
to format the signalsappropriately. Signals define the latter two functions along with the
SignalFormat
trait to participate in the generic interface for signalpromotion.
rate and bit rate. A spectral signal has a sample rate in time, a temporal
signal a sample rate in frequency. The bit rate is represented separately
because it needs to be handled somewhat differently for buffers and streams.
I tried combining them but it was a bit of a mess and there were lots of
exceptions
comes in handy when trying to operate over ranges, for instance. It makes
the generic code to operate over signals with
mapsignals
easier to write.and length it simply takes an abstract array. One can limit the range by
using a
view
over a larger array. The fact that it's abstract also meansit can handle channel mixing lazily because I've defined an object that
mixes the channels on demand.
bit rates automatically, so most of the logic for same and different bit
rate code is exactly the same, and converting it before passing it is slow.
So slow, uncessary work. If necessary it's easy enough to convert the bit
rate within an implementation that can't be more generic.
and improve my understanding of the code.
take to do it—to create signals from functions. As a consequence
one no longer really neads
SinSource
, as it can be defined as, e.g.signal(sin,1kHz). A missing behavior, that would need to be implemented
is to combine signals into multiple channels, e.g. something like
channels(signal(sin,1kHz,phase=0),signal(sin,1kHz,phase=pi))
.NOTE: I'm not sure
mapsignals
can easily be separated into anotherpackage. We were originally thinking
Sounds
functions formanipulating sounds would be in another package. However,
mapsignals
has to somehow generate asignal which, to be non-specific to this package, would require
recreating the broadcast machinery ala
broadcast_similar
and friendsand defining an interface for reading and writing to signals that looks
a lot like the interface for
SampleSource
andSampleSink
... sowhy not just use those, and other packages that want to implement
signals that work with this machinery can just ensure their objects are
coercable to sources or sinks. I already have defined
tosource
tocoerce to sources, and it would be easy to also have
tosink
as well.