MIR with AudioMuse-AI-SAE [P]

查看原文

GitHub模型发布模型访问开源代码

热度趋势

趋势数据积累中

百分比基于当前可用热度信号，而非评论数或独立用户人数。

推荐理由

GitHub 相关模型动态已经出现，适合跟踪能力变化、生态影响和后续可用性。

AI 摘要

一位开发者最近讨论了一篇关于通过开放词汇概念发现来引导密集音乐检索的论文，作者是Julien Guinot、Alain Riou、Elio Quinton和Gyorgy Fazekas。该开发者还提及了他们之前的工作，即LAION CLAP的精炼版本，名为DCLAP。这个DCLAP模型拥有大约700万个参数，并能高效地在CPU上运行。开发者提供了DCLAP项目在GitHub上的链接，并欢迎任何反馈、建议或想法。

Hi all, I recently read this paper: Julien Guinot, Alain Riou, Elio Quinton, Gyorgy Fazekas. Steering dense music retrieval with open-vocabulary concept discovery. https://arxiv.org/abs/2608.08757

There is multiple model where you can get embedding from Song and Text so that you can search song by text. The challenges is when you get a result that don’t really respect your query, for example you search:

- POP viola with female vocalist

And as a result you usually get POP song with female vocalist because they are usually very common in you library, instead the Viola tend to disappear being uncommon. But the power of MIR isn’t exactly search to specific song even if uncommon ?

Here the idea of the paper is to get the compressed embbeding layer, make it sparse again and be able to identify for some common words exactly which neuron are activated try to avoiding overlapping with other neuron. Then you can improve the value of let’s say the Viola neurons and then bring back to the compressed embbeding space.

This is not mine, is only explanation, what I did is in the past a Distilled version of LAION CLAP called **DCLAP** that is around 7 million parameters and can efficiently run on CPU:
 -  https://github.com/NeptuneHub/AudioMuse-AI-DCLAP

The following the paper I also trained a **SAE for DCLAP**:
 -  https://github.com/NeptuneHub/AudioMuse-AI-SAE

So if you’re interested you can take a look at let me know. Are all free and opensource.

Finally if you want to see both of them in action I used both the model for **AudioMuse-AI**, a software dedicated to sonic analysis and automatic playlist creation (with various model):
 -  https://github.com/NeptuneHub/AudioMuse-AI

If you have any feedback, suggestion, idea, please let me know !