Serato cue import: Risk of loosing data

[daschuer]

Bug Description

Mixxx is able to read the Cue points stored in the file metadata. However the data does not allow a lossless round-trip with Mixxx. We have recently a added a pull request that works around of some of these issues, but depending on the use case the results are not appropriated. See: #11466.

The main issue is that Serato cues are always imported when choosing "Metadata -> import from file Track". One use case of this feature is to sync Artist and Title after editing it with a third party tool. When you do it with a couple of files all your Mixxx cue points are gone and replaced with the Serato cue points from the file.

2.4 has an experimental feature to replace the Serato data in the file with the cues in Mixxx. This is however not a solution, because Serato has a different number schema and no color info for loop cues.

We can think of different solutions around this problem, with pro and cons. So let*s collect the requirements and use cases and than find the best solution with a good coverage that does not wipe user data.

Version

2.3

OS

All

[daschuer]

I think the use cases:

1. Easy migration from Serato to Mixxxx
2. Sync back Cue adjustments to Serato (round-tip)
3. Don't loose data in Mixxx
4. Backup Mixxx cues in File Metadata
5. See Serato Loop cues in Mixxx
6. See Mixxx Loop cues in Serato

I think we should focus to 1., 2. and 3. at the cost of 4. 5. and 6.

What do you think? @Swiftb0y @Holzhaus

[JoergAtGithub]

I doubt, that 2.3.5 is the right target here. This requires a behavioral change, which would need a lot of practical testing, to not break a use case of somebody else.

[JoergAtGithub]

One typical use case is that somebody get tracks from DJ download pools. These are often prepared with Serato Cues, to allow a DJ to use them immediately after download without preparation.
e.g. downloading a wish track during a gig and jump immediately to the right cue

[daschuer]

@JoergAtGithub Which behavior change do you have in mind?

I can confirm your use case. What is your expectation with loop cues, that are on the loop bank in Serato, that Mixxx not has. Currently they are inserted in the hot-cues 1-8 if possible, or moved to higher invisible Indexes if the place is occupied.

[Swiftb0y]

I think the use cases:

1. Easy migration from Serato to Mixxxx

2. Sync back Cue adjustments to Serato (round-tip)

3. Don't loose data in Mixxx

4. Backup Mixxx cues in File Metadata

5. See Serato Loop cues in Mixxx

6. See Mixxx Loop cues in Serato

I think we should focus to 1., 2. and 3. at the cost of 4. 5. and 6.

What do you think? @Swiftb0y @Holzhaus

I generally agree with the focus, though I simply don't think think a lossless roundtrip is particularly possible considering syncing in either direction has to be lossy.

In general, I think mapping serato loopcues to hotcues 8-16 and mapping the loopcues section on serato controllers to those cues is a good solution. I also think that tweaking skins to show 16 hotcues is quite possible, there is more than enough screen-realestate on 1080p screens to do that.

Unfortunately, I don't see any way to make the roundtrip work with that approach since we'd need to differentiate between loopcues coming from mixxx (so they can be reassembled interspersed like they have been created) and loopcues coming from serato (to reassemble with the +8 offset).

[daschuer]

... don't think think a lossless roundtrip is particularly ...

Yes, that is a technical boundary, the underlying issue of this bug.

mapping serato loopcues to hotcues 8-16

Yes I think that is a working solution. Just move the mode of the integration from "keep" all to move to the bare minimum.

1. Export Mixxx Hotcues and Loops 1-8 to Serato Hotcues 1-8, loosing the loop nature.
2. Export Mixxx Hotcues and Loops 9-16 to Serato loops 1-8, hotcues become zero length loops.
3. Don't export other numbers.

Instead of changing type we can consider to not export them.

The same during Import:

1. Import Serato Hotcues 1-8, as Mixxx Hotcues 1-8
2. Don't Import other Hotcues
3. Import Serato Loops 1-8, as Mixxx Loops 9-16
4. Don't Import other Loops

Do the import only if it is a new Track or there are no hot-cues set. This protects Mixxx Cues form overwriting when Reload Track Metadata.

What does in mean for the initial use-cases:
1 . Easy migration from Serato to Mixxxx

Works, drawback: Loop Cues are not visible in default skins

2. Sync back Cue adjustments to Serato (round-tip)

Works for Serato Cues. Mixxx cues are changing type. Steep learning curve.

3. Don't loose data in Mixxx

Mixxx cues are protected, because cues are not re-imported.

4. Backup Mixxx cues in File Metadata

Possible if the user only users Serato compatible cues.

5. See Serato Loop cues in Mixxx

Only possible with a custom skin

6. See Mixxx Loop cues in Serato

Mixxx loop cues 1...8 are converted to HotCues,

[daschuer]

What do you think about that?

[Swiftb0y]

5. See Serato Loop cues in Mixxx

Only possible with a custom skin

As I said, we could probably allow for more cuepoints in mixxx. Latenight can toggle between 4 and 8, so 16 should also be an option. but thats unimportant right now.

5. Don't loose data in Mixxx

Mixxx cues are protected, because cues are not re-imported.

If that's the solution we opt for (which I'm totally fine with) then I feel like we could simplify the export scheme a lot. I don't think zero-length loop cues are a good idea for example because calling them will basically pause or even break Serato. IMO the scheme for importing is fine. For exporting, rather than having lossy/janky conversions, I'd prefer if those cues would just get skipped.

[Swiftb0y]

Export Mixxx Hotcues and Loops 1-8 to Serato Hotcues 1-8, loosing the loop nature.
Export Mixxx Hotcues and Loops 9-16 to Serato loops 1-8, hotcues become zero length loops.

My problem with that is any loopcues in 1-8 just get lost during export if they have been created in mixxx.

[daschuer]

My problem with that is any loopcues in 1-8 just get lost during export if they have been created in mixxx.

Yes, that is the cost of the "predictable" solution. If a users wants to use Searto and Mixxx side by side they simply need to move the Loop cues to the range of 8..16.

[daschuer]

@Holzhaus: Does this work for you?

[Swiftb0y]

Yes, that is the cost of the "predictable" solution. If a users wants to use Searto and Mixxx side by side they simply need to move the Loop cues to the range of 8..16.

Well, the alternative would be partition the cues before export. The issue with that is that the cues would get moved around a lot more (relative distances between cues change and the gaps will be filled from the partitioning), but then no information is really lost, nor do we add redundant or janky information. With this strategy, the Serato->Mixxx->Serato roundtrip would work, but Mixxx->Serato->Mixxx wouldn't (though that's already the case with your proposed export scheme as well since its a limitation of the serato format).

they simply need to move the Loop cues to the range of 8..16.

Since skins currently don't show these, nor do we have any tools to change the indices of cues in the GUI, simply moving them manually is not an option IMO.

[Holzhaus]

I doubt, that 2.3.5 is the right target here. This requires a behavioral change, which would need a lot of practical testing, to not break a use case of somebody else.

@JoergAtGithub To elaborate, the behavior change already happened in 2.3.5 due to the merge of #11466 (commit 383f682). Before 2.3.5, Hotcues from Serato were imported as Mixxx Hotcues 1-8 and Serato Loops were imported as Mixxx Loop Cues 9-16. Since PR #11466, Mixxx now reorders cues and looks for "gaps" in hotcues 1-8 and moves loops from 9-16 there.

I propose to restore the original, pre-#11466 import behavior. This is roughly what @daschuer described:

1. Import Serato Hotcues 1-8, as Mixxx Hotcues 1-8
2. Don't Import other Hotcues
3. Import Serato Loops 1-8, as Mixxx Loops 9-16
4. Don't Import other Loops

Note that Serato only has 8 hotcues and 8 loops, so cases 2 and 4 are not really relevant, but since we should not trust user data it makes sense to discard such invalid cues during import.

Regarding export I'd propose the following:

1. Export Mixxx Hotcues 1-8 as Serato Hotcues (i.e. only those Mixxx cues that are jump cues, loops are ignored)
2. Check if there are any loop cues in range 1-8
   If yes: Export Mixxx Loops 1-8 as Serato Loops (i.e. only those Mixxx cues that are loops, jump cues are ignored)
   If no: Export Mixxx Loops 9-16 as Serato Loops (i.e. only those Mixxx cues that are loops, jump cues are ignored)

[Swiftb0y]

Can you define what you mean by "cues are ignored"? Eg should it result in a gap or should the cues following that fill the gap?

[daschuer]

Cues are identified by there index resp. the position in the grid. This may match the position in a track or a position schema of the user. Serato users will likely use the same controller for Serato and Mixxx and there is a good chance that the bank select for the clue pads is part of the mapping.

This means that any moving and shifting will cause confusion, because a cue is moved into a place that is normally used for something else. The other issue is that all magic shifting code will suffer from corner cases where it fails, which likely makes the code complex and the behavior hard to predict

We need currently a quick solution for our 2.3.5 release. I think for that, I will just skip the cues that cannot be expressed without shifting in the Serato tag. When a user uses the Hotcues in Mixxx in the same way he is used to do in Serato, a full round-trip works without any daw backs. If not, what can he expect. I think it is understandable that loop cues in the 1 ... 8 bank are not show up in Serato because Serato does not allow it.

If it turns out that this is a real draw back reported by Serato users we can come up with a solution that matches the not yet know expectations.

[Holzhaus]

Can you define what you mean by "cues are ignored"? Eg should it result in a gap or should the cues following that fill the gap?

Result in a gap.

Here's an example:

1. Hotcue A
2. Hotcue B
3. Loop A
5. Hotcue C
6. Loop B
7. Loop C
8. Hotcue D

Would result in the following Serato cues:

Hotcues:

1. Hotcue A
2. Hotcue B
5. Hotcue C
8. Hotcue D

Loops:

3. Loop A
6. Loop B
7. Loop C

[Holzhaus]

If you only have jump cues in 1-8 and loop cues in 9-16, the proposed export scheme would be able to export such cues without information loss to Serato, and also matches the proposed import mechanism. So the round-trip Serato -> Mixxx -> Serato should work fine.

Cues are identified by there index resp. the position in the grid. This may match the position in a track or a position schema of the user.

Yes this is why I think we should try to preserve cue order and gaps.

[Swiftb0y]

Cues are identified by there index resp. the position in the grid. This may match the position in a track or a position schema of the user.

This convinced me as well. I'm in favor of using the export behavior described by Jan. #11530 (comment)