Music descriptors and algorithms wishlist (post your suggestions)

[dbogdanov]

This issue summarizes our wishlist of music descriptors and algorithms to be added in Essenita. You can post your suggestions in comments.

Essentia 2.1

• Implement standard mode and finish testing EBU R128 loudness descriptors (implement EBU R128 loudness measure #94)
• Finish testing PitchYin (unit tests) (PitchYIN error on stationary signals  #230)
• SuperFlux (final clean implementation, review documentation, add unit tests Add unittests for SuperFlux algorithm #226)
• Improved PredominantMelody algorithm (Incorrect outputs for predominant melody algorithm using essentia #42, Predominant Melody - Contours gets drifted to high ranges and some contours missed #62)

Long-term future

• Beat tracker with downbeat detection (Determining first beat in a measure #253)
• Real-time beat tracker
• Algorithm for music segmentation (+ test current algorithm)
• More envelope descriptors (attack start/end time, temporal centroid, temporal increase/decrease, ADSR) (@ffont)
• Constant-Q transform (ConstantQ Transform? #136)
• Histogram streaming algorithm (map input stream of values into a configurable histogram) (Implement generic Histogram algorithm #529)
• Rhythm Patterns, Rhythm Histogram, and SSD (Implement Rhythm Patterns, Rhythm Histogram and Statistical Spectrum Descriptor #532)
• Beatogram and Meter features (currently marked as experimental algorithms) (Meter and Beatogram #486) (@ffont)
• Graham Percival's BPM [1] estimation method (@ffont)
• Audio fingerprinting (Echoprint, AcoustID) (Implement Chromaprint and Echoprint fingerprinting algorithm #474)
• Algorithms for estimation of time signature, bars, downbeat and offbeat (related to Beatogram)
• Melody extraction using source filter model (Improved predominant melody extraction with a Source-Filter model (SIMM) #469) (@juanjobosch)

[1] Graham Percival and George Tzanetakis. Streamlined tempo estimation based on autocorrelation and cross-correlation with pulses. IEEE/ACM Transac- tions on Audio, Speech, and Language Processing, 22(12):1765–1776, 2014.

[andimarafioti]

Hi! I'm working on an 'acoustical instruments detection' algorithm as a final project for my engineering degree. I know Ferdinand Fuhrmann worked a lot in this field for his phD in the Universitat Pompeu Fabra but I don't see any implementations of his work for/in Essentia. I think an implementation of an instrument recognition algorithm using the IRMAS dataset would be a great add-on to show just how complete the library is.

[andimarafioti]

On another subject, it would be cool to see more acoustics algorithms. For example, a toolkit for Room Response Measurement using Angelo Farina's method. This would mean a logsinesweep generator, maybe a convolution algorithm, a kirkeby inverse filter generator and some tools to check the output like the THD, the reverberation time (RT20, RT30), the early decay time.
It could be said that these are out of the scope of the library, but this type of information could be useful when trying to retrieve information from a music signal.

[AnasGhrab]

What about implementing the Praat algorithm for Pitch detection (http://www.ucl.ac.uk/~ucjt465/tutorials/praatpitch.html), besides PitchYinFFT and PredominentMelody ?

[dbogdanov]

@AnasGhrab Does Praat algorithm have any advantage over Yin?

[sertansenturk]

I was just going to add a general histogram implementation, but the entry "Histogram streaming algorithm (map input stream of values into a configurable histogram)" seems to include it. Am I right?

[dbogdanov]

@sertansenturk yes, you are
I've created a separate issue for that: #529

[dbogdanov]

@perfe

[regorxxx]

Please provide a proper guide to build Essentia + gaia since it's currently impossible on any OS, no matter what you do.

There are literally tenths of issues about this since years.