Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 4 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,7 @@ WaveGenSynth is a unique synthesizer that leverages diffusion techniques to crea

## [Text-to-Playable-Sound: synthesizers based on latent diffusion technology](https://github.com/suckrowPierre/BachelorThesis)

In this study, the integration and applicability of generative artificial intelligence in the field of music
production was analyzed through the introduction of a digital instrument. Using selected diffusion models,
users can define sounds through textual descriptions and play and manipulate them with standardized music
production tools. The diffusion models used were evaluated for their suitability in the given context and
modified for integration into a digital instrument. Using certain frameworks, the digital instrument was
created, resulting in a user interface. This allows users to edit model and instrument-specific parameters. The
analysis showed that the models used do not always adequately respond to the context of music production,
resulting in unexpected sound patterns or abstract artifacts. Currently, available text-to-audio models do not
provide high-quality reproduction of familiar sounds but offer opportunities for experimental applications.
The implementation of a prototype of the digital instrument allows for such experiments and the exploration
of innovative sound synthesis methods. However, functions are currently missing to reproduce selected areas
of the generated sounds or to play them indefinitely. Nevertheless, fascinating and unusual soundscapes can
already be produced, which could potentially find application in musical compositions.
In this work, the integration and applicability of generative artificial intelligence in the field of music production was analyzed through the introduction of a digital instrument. Using selected diffusion models, users can define sounds through textual descriptions and play and manipulate them with standardized music production tools. The diffusion models used were evaluated for their suitability in the given context and modified for integration into a digital instrument. Using certain frameworks, the digital instrument was created, resulting in a user interface. This allows users to edit model and instrument-specific parameters. The analysis showed that the models used do not always adequately respond to the context of music production, resulting in unexpected sound patterns or abstract artifacts. Currently, available text-to-audio models do not provide high-quality reproduction of familiar sounds but offer opportunities for experimental applications. The implementation of a prototype of the digital instrument allows for such experiments and the exploration of innovative sound synthesis methods. However, functions are currently missing to reproduce selected areas of the generated sounds or to play them indefinitely. Nevertheless, interesting and unusual soundscapes can already be produced, which could find application in musical compositions.

## Table of Contents
- [Installation](#installation)
Expand All @@ -31,6 +19,8 @@ already be produced, which could potentially find application in musical composi

## Installation

!!! Installer and Server-Application files are currently unsigned for mac. To open right click and click on open. Double click won't work

### Binary Installations (VST/AU/Standalone)
Download the latest stable release from the [releases page](https://github.com/suckrowPierre/WaveGenSynth/releases), and follow the instructions for your specific platform.

Expand Down Expand Up @@ -71,7 +61,7 @@ Operating WaveGenSynth is straightforward:
2. Initialize a model. This step may take some time during the first run, causing the plugin to freeze momentarily. Monitor the server console until the download and setup are complete. If a timeout occurs in the plugin, hit refresh after the setup is complete. (Select 'cuda' if running on hardware with a Nvidia GPU, 'mps' if running on Apple Silicon, and 'cpu' as a fallback)
3. Enter a prompt in the designated field.
4. Click the "Generate" button.
5. Select the appropriate MIDI input device.
5. Select the appropriate MIDI input device. If using as a standalone click option and then select your MIDI device
6. Enjoy creating and playing sounds!

## Modelevaluation
Expand Down