Code Text Copy to Drive
Notebook

Gemini

GANSynth Demo

This notebook is a demo GANSynth, which generates audio with Generative Adversarial Networks. GANSynth learns to produce individual instrument notes like the NSynth Dataset. With pitch provided as a conditional attribute, the generator learns to use its latent space to represent different instrument timbres. This allows us to synthesize performances from MIDI files, either keeping the timbre constant, or interpolating between instruments over time.

GANSynth figure


Gemini

1: Environment Setup

This notebook synthesizes audio from uploaded MIDI files. There are two different flavors:

  • Interpolate between random instruments
  • Interpolate between two chosen instruments

Have fun! And please feel free to hack this notebook to make your own creative interactions.

Instructions for running:

  • Make sure to use a GPU runtime, click: Runtime >> Change Runtime Type >> GPU
  • Then press the Play button on the left of each of the cells
  • Double-click any of the cells to view the code

Gemini

Install magenta, define some helper functions, and download the model. This transfers a lot of data and should take a minute or two.

Show code

Gemini

2(a): Random Interpolation

These cells take the MIDI for a full song and interpolate between several random latent vectors (equally spaced in time) over the whole song. The result sounds like instruments that slowly and smoothly morph between each other.


Gemini

Upload a MIDI file (.mid, single instrument) for audio synthesis, or use the provided default. You can find lots of free MIDI files online.

midi_file:
Bach Prelude (Default)Upload your own
Show code

Gemini

Select the number of seconds to take in interpolating between each random instrument. Larger numbers will have slower and smoother interpolations.

seconds_per_instrument:

This cell plays and displays a Constant-Q spectrogram of the synthesized audio.

Show code

Gemini

Get the .wav file (optional)

Show code

Gemini

2(b): You Choose the Interpolation

These cells allow you to choose two latent vectors and interpolate between them over a MIDI clip.


Gemini

Upload a MIDI file (.mid, single instrument) for audio synthesis, or use the provided default.

midi_file:
Arpeggio (Default)Upload your own
Show code

Gemini
number_of_random_instruments:
Show code

Gemini

Using the instrument numbers from the cell above, create a list of instruments to interpolate between. You can repeat instruments.

instruments:

For each instrument in the list above, place it at a relative time [0-1.0] in the clip. The list of times must always increase and start at 0 and end at 1.0.

times:

This cell plays and displays a Constant-Q spectrogram of the synthesized audio.

Show code

Gemini

Get the .wav file (optional)

Show code