-
Notifications
You must be signed in to change notification settings - Fork 85
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Performance file handling #581
Performance file handling #581
Conversation
…digit number in the filename indicate a performance "voice number" in MiniDexed. The external filename numbers will now match any Program Change messages using the common MIDI concept of user selecting 1..128 whilst internally they are treated as 0..127. Note: in the case of performances, performance 1 (index 0) is the Default "performance.ini" file for backwards compatibility. Also note that in this version, new performances, when saved, cannot occupy free slots between other performances - they are added to the end. Even though the filename standard gives 6 digit numbers, the actual number of performances is still limited to 256.
Build for testing: |
The version from December 5th. works on my RPi4. |
@Banana71 please see #580 for details. This is the beginning of @diyelectromusic's plan to implement switching between folders = banks of performances. That part is not implemented yet, however. |
The key change in this first version is to align the performance filename numbers with the performance numbers themselves, so that if there is a gap, the numbers still correspond to the right performances. So 000002_Perf2 Will be selectable using MIDI PC voice instruments 2,3,5,8 (which "on the wire" is 1,2,4,7). Previously, they were loaded and sorted in numerical order, but without leaving gaps, so the above would have been 2,3,4,5 - meaning any gaps in numbering would get "out of sync" with the filenames. I considered sorting this out a pretty extensive change in terms of changing how performance files work (see the files changed), but need this done to be able to do the rest. I wanted to know this was done and working ok before changing anything else. (There is more on the problem here: https://diyelectromusic.wordpress.com/2023/11/22/a-dx7-usb-dongle/) Kevin |
There's nothing really to be testing yet, I'm just snapshotting a useful point for me to ensure everything is backed up online as things progress. I'll keep tinkering on and off over the next week or so and shout when there is something worth looking at :) Kevin |
Okay, good to know. |
Latest "in development" build (for testing, not for using) now does the following:
Limitations:
Changes in behaviour compared to existing releases:
What needs testing if anyone fancies helping.... Check file/directory handling:
Check MIDI handling when PCCH=0:
Check MIDI handling when PCCH = valid MIDI channel:
General performance file handling:
General MiniDexed use to see how it fares. I think this is largely doing what I'm expecting but it would be good to get some proper verification going on before moving on to the last main phase - the UI stuff. Report any errors, ideally letting me know what test you're doing, any relevant context (filenames, structure, MIDI msgs, screen output, etc) and what happened! Especially any assets going off! Thanks, |
Build for testing: For the sake of consistency and simplicity we should probably drop the legacy stuff? |
I almost proposed that, but I don't think we need to and it does allow someone to run it with a fairly minimal configuration. Kevin |
Thank you @diyelectromusic Kevin for your hard work. I tested with my RPi4, SSD1306, GY-PCM5102, classic MIDI input. General performance file handling:
Check MIDI handling when PCCH=0:
I use my Keystep37 to control MIDI CC 32 (LSB Controller) and MIDI CC 0 (Bank Select). I'm not sure if I'm doing something wrong, the program change doesn't work as expected pCCH=1, MIDI CC 0=0: then I can choose between the standard performance bank and the sub performance folder 02_Name with MIDI CC 32 =0 or 1. I would have expected to choose the performance with MIDI CC 32 and not the bank. Which MIDI CC commands do I have to use exactly to select the performances in miniDexed? Or what tools do you use to choose the bank and performance? |
With the Hydrasynth I can choose the banks and performances. Every now and then, graining occurs in the sound after a program change, but after switching back and forth several times, the graining disappears again. |
Banks are selected with a combination of the LSB and MSB for BankSelect, so MIDI CC 0 (MSB) and 32 (LSB). But as there are only 64 banks supported, actually we don't need to use the MSB, but I think the MIDI spec says to always send them. The performances within a bank are selected using Program Change messages. If you use something like MidiOx then it will always send the combination of Bank MSB, Bank LSB, PC for any instrument selection, which again is what I think the MIDI spec suggests if using banks. So the Banks will be numbered as follows: I think we can get away with only sending the LSB and PC though to actually change the bank, but I don't think this is technically valid within the spec! It also means it would apply whatever the last MSB sent was, so isn't recommended :) MIDI Spec added below... Kevin |
I think you made a mistake there |
Oops! Sorry, updated :) Kevin |
Latest update: Performance bank selection now included in the performance menu. Notes:
So outstanding questions now:
As before, this needs some proper testing (see my previous list of ideas, which really need repeating!). Also need to verify the operation of SelectPerformanceToLoad being 0 and 1 and make sure performance bank/voice loading from UI works as expected. I still need to remove some debug strings, but they are quite useful atm for seeing what is going on, so they are still in for now. Please let me know if you manage to give this a go. I'm not sure how much time I'll have over the next week now myself. Also if anyone feels like actually reviewing the code changes for any obvious things I've missed that would be cool too :) I'm currently expecting a final update (to tidy up/remove debug strings/etc) after Christmas probably now. Kevin |
Hi Kevin, thanks for your work. Here are my 2 cents:
Yes, I think that's ok. I don't assume that anyone will want to do more complicated "performance management" on the MiniDexed device itself.
Something to consider for later in case it turns out to be a commonly requested feature.
Again, I'd say let's not worry about that unless it turns out to be a commonly requested feature. (We could always use e.g., a .factory suffix-suffix on all kinds of files and directories and prevent those from being deleted.)
Yes. The question is, when does the performance switching occur. As soon as you switch to another bank, or only once you select a performance in the bank? (Ideally the behavior between voices and performances could be made consistent in this regard. What do real instruments do?)
If you ask me, "unlimited" is the correct number of banks. (Not all of those will be addressable over MIDI, though.) |
As I say, I think we ended up being inconclusive (more here: #425). But where we differ is that we might have a gap where an instrument is in another bank, so what happens then? Anyway, for now it switches to the first performance in the bank on bank select. People can try it and see how that feels. I tend to like the behaviour that changing banks changes to the same selected voice in the new bank. But I think that comes from a keyboard (I forget which) where the same patch numbers across different banks have variations of a similar voice...
Well as we're not really using any kind of dynamic memory management in MiniDexed all structures are pre-allocated, so we do need to pick a size and go with it :) |
That's concerning me. Which memory structure are we talking about? Can't we read the filesystem as-needed? |
Why? Pretty much every structure in MiniDexed is pre-allocated - that's why we set number of voices, number of banks, number of tone generators... if we have several of something, the number we handle is in a header file somewhere. There are some cases where the object oriented nature of C++ is used for more dynamic creation of objects, but generally speaking this mostly happens on startup from what I've seen. Kevin |
My comment is not specifically about this PR but is about how we handle files and directories in general. Whenever it comes to interacting with the filesystem, I think we should not hardcode things like the maximum number of files in a directory. Also, I think we should load files just-in-time when we need it (no sooner). If we did this, then the number of files on the SD card should not influence the boot time in any significant way. Does this make sense? (Again, not specific to this PR.) |
That's easy to say if we were running with a proper, full operating system, but with an embedded 'bare metal' OS, we have to think about such things a lot more up front. That's just embedded systems. In practical terms, we already know that having too many files in a directory causes everything to go really slowly due to the nature of FAT support in circle. But ultimately whilst there are no limits to the numbers of files and directories on the filesystem, by the nature of the system we only pay attention to a fixed number of them. And yes, "load at point of use" is pretty good when you can do it, but it is quite complicated to do. And I do that (pretty much) for the performance banks anyway (a lot more so that we do with voices), but we still need some kind of "up front" knowledge about how many banks we know about so that we don't drag the performance of the UI to a halt - we don't want it parsing the filesystem for every click of the encoder... really, we don't :) Ultimately, we need to remember what MiniDexed was for - an optimal, 'bare metal', embedded, performant system. If someone wants an open-ended, "all we can throw at a Raspberry Pi with large SD cards and lots of memory" type setup, then they are better off with a Zynthian or Dexed on Raspbian or something in my view. Kevin |
Possibly :) |
Without reading the other answers first, I'll give you my unbiased opinion.
It's okay, but the optimal solution would be to freely choose the storage space. You could choose a free storage space or overwrite an existing performance. The cherry on the cake would be if you could also change banks when saving. This way you could easily create your own favorites bank directly on the device.
Yes, everyone should be able to do everything. The best solution might be to be able to set write protection on a folder (bank) yourself.
Absolutely yes. And it's also ok that the last performance remains active.
I think 128 performances per bank makes sense. Anything beyond that cannot be accessed via Midi Program Change. If you want to quickly scroll between performances using the encoder, you probably won't want to have more than 64 performances in a bank. A limit of 256 performances is okay, but I wouldn't use it myself. I think you all know that I really like the miniDexed and I can work well with it as it is. Any improvement you make is fantastic. Thanks |
Actually, I probably agree with you. I'll probably change it to 128. We will need to renumber the default set though as some of those are currently numbered more than 128.
Pretty quick in tests, but I've not had many performances in each bank. I'm no longer sorting as the number is now significant, so the order is derived directly from the filenames in performances (and banks) and there might be gaps and that is fine.
I don't think so. I have to scan the bank names/numbers on power up, that is all. It only scans the directory itself when changing banks.
With MSB/LSB the MIDI limit is actually 16384 possible banks. I've not tested it to the full numbers yet. Also, it will be slower in this version as it is printing a lot of debug info out to the console. Kevin |
Let's use that then 👍 |
Or ping me an email at diyelectromusic at gmail :) I'm good to go with this one and will just have to bug fix if more things turn up. Kevin |
Merged. @diyelectromusic please feel free to edit the wiki accordingly. You can reach me at probono at puredarwin dot org, but the quickest way is usually via GitHub. Thanks! |
Phew! :) Will do, re wiki, but it might have to wait another day or so! Kevin |
Some notes now added to the wiki, plus a bit of general tidying up of the performance section. https://github.com/probonopd/MiniDexed/wiki/Files#performanceini Kevin |
Wow - what a fantastic implementation! Like clockwork! Using five Pi Zero2 modules MIDI-Thru-ed together. Having the option to load the Performances from MIDI is awesome! Much thanks for all the time put into organizing and building this!!! |
Are these your eurorack modules? I bet they sound amazing!! :) Kevin |
It should do, although it isn't permanent - is that what you mean? Basically on boot, the PCCH minidexed.ini value should set the default, but if you change PCCH via the menu then that should take effect instead until next power off/on... Are you asking for a means of allowing the menu PCCH to be saved back to minidexed.ini? We don't have any settings that are saved back like that atm apart from the contents of the performance.ini itself. I keep thinking we ought to have some kind of system.ini, but we don't yet. Kevin |
OK, I thought it was as you say, Including not being permanant - I get that too. Then is it possible that these Performance change parameters are Not being sent to MIDI thru (MIDI thru is pre-Performance-change code)? Here's what happens: IMG_2948-480.mov |
System level MIDI messages should be passed on via THRU, so that shouldn't be an issue. I'm not entirely sure, if I'm honest, what you're describing... But do you have an overlap between PCCH MIDI channels and channels in the performances? Could that be an issue? Try PCCH channels 2-6 maybe and leave channel 1 for playing? @Banana71 are all the default performances configuring all TGs for channel 1?
I don't quite follow what you mean here - when you say change MIDI channel on the MPC do you mean changing what channel it is transmitting? When you say "downstream get changed too" - what their MIDI channels? I don't think that is possible via MIDI...? Kevin |
What I'm trying to do is have the MPC sequencer load Performances for each MIDI-track. I have five tracks of MIDI transmitting respectively on MIDI channels 1-5 from the MPC to Module 1 on the left, the other modules are chained from the previous modules' thru. The PCCH channels are set 1-5 from left to right. In the video I'm transmitting from the MPC, changing the channel, then the program change parameter. I'm not comprehending when transmitting a prog change on let's say channel 3, that modules 4 & 5 change TG voices when their PCCH's are set 1-5 respectively & modules 1 & 2 don't change, only module 3 should change. |
Ha! At the moment - you and me both!! :) I've been reading through the code again and can't see a way that will do that atm... you might need to enable the MIDI debug log on say 4 and 5 (to keep it simple) and show me what is being received by each module (sorry, it will go to the display, so I know that will be a bit of a pain!) - but that might give a clue! I'll try to cobble together some boards here and give it a go. What are the MIDI channels of the performances you're using - just the defaults? I think they're all set for channel 1... or have you changed the channels in the performances on 2-5 too? Kevin |
I didn't change MIDI channels in any of the Performances, so all the modules should have Performances at their defaults which is typically channel 1 (unless it's a split type of performance ie 1 & 2). I'll do a screenshot of the MIDI debug log - I'm curious what module 1 will say compared to 5... ...gotta go dig for a second mini-HDMI cable first! |
Ok - so this is what might be happening. When a performance is selected, it does a sequence of internal "program changes" for each TG, but if the setting: ... so as all TGs are listening on MIDI 1, this gets sent out on MIDI channel 1 and received by any active TG. So I don't think it is passing the performance selection via the THRU but the TG SysEx on MIDI channel 1 is being passed to all TGs. So the performance isn't being changed, but the "live" DX voice is being set, via SysEx, to whatever voice is chosen as part of the selected performance... This is why you see several voice names flashing past - it is sequencing through all the voices used in the performance - now all sent as SysEx on channel 1! You might notice if you then try to change the voice using the UI that the voice name isn't correct for the voice number (just turn once left then right again to reselect the same number - the name will be different!). Hopefully, setting Aside: this will also happen when selecting a performance via the menu too... sigh... I don't like this feature tbh - and think we should default to off, but it is legacy behaviour that someone coded in at one time and I'm reluctant to change the default atm... But this seems like a strong reason to change it! :) Kevin. |
BINGO!!!! Works as expected now! Stupid boolean, ate my entire day. This makes all your combined efforts on this Performance-use implementation worthwhile! Thanks again for your time on this and agree it would make a great change. Check out this dump before the change, on a sixth module I added: That was from turning the encoder one click on the first module's TG voice change! |
Great that it works for you @Lonolo now. 👍
Anyone who works with more than one miniDexed at the same time or who wants to work with a MIDI channel other than 1 has the problem that the MIDI channel in the "default Performances" is set to "1" for the active TGs. I haven't come up with a really good solution yet. |
There is definitely something odd going on there - F7 should be a valid end of SysEx marker - looks like its missed the F0 start marker - maybe it got spammed part way through another MIDI message and screwed up the parsing... Anyway - from both these comments that settles it - that should be off by default :) Kevin |
A solution for multiple miniDexed:
If the Midi channel in the performance is set to 17, the Midi channel is adopted with the value of PCCH. So I could set the MIDI channel to 17 in all default performances. The performances would then also be compatible with older systems. |
Thanks @Banana71 that makes sense, So I'll write a script to modify all my Performance files changing MIDIChannel#=1 to MIDIChannel#=17, that would be ideal for a rack of MIDI selectable and controlled Performances played back. |
Unfortunately I have no idea about C++ performanceconfig.cpp
I have no idea if this is the right place in the program but I hope you understand what I mean. 😅 |
I'm afraid this doesn't exist - this is a proposal by @Banana71 I need to think about it properly and see how it might interact with everything else! :) Kevin |
Doh! I see. Then I second the motion! Let me know how I can help and or test. |
I plan to use a Raspberry Pi Zero 2 with miniDexed as a drum, bass and FX player. 16 TGs would be ideal for this, but the polyphony could be low. Kevin @diyelectromusic, can you compile for testing a version in which I can freely set the parameters for the number of TGs and the polyphony? To save resources, the reverb could be deactivated. |
Build for testing.
For details please see #580.
Not ready for merging yet, this PR only exists so that we have builds for testing.