Soundtoy Prototype 2

Download required (Wwise not supported in WebGL); Mac and Windows application files available at bottom

Controls:

space = create new shape
dlt/bksp = delete most recent shape
click and drag shapes to move them
ctrl/cmd + drag shape = rotate and transform
alt + drag shape = change color
esc = close application

This project is essentially a basic, one-bar MIDI sequencer at 90bpm. When hit by a "playhead" (the large flat rectangular prisms moving across the screen), each shape plays a note on a synthesizer, the parameters of which are determined by its roundness (mixes between two different oscillator types) and color (increments the amount of frequency modulation). The note value is determined based on the black or white "key" the shape is currently on when it is hit, and increments up through the G major pentatonic scale from bottom to top.

Acknowledgements:

• The base procedural mesh generation for the shapes, which I adapted in order to be able to change from a sphere to a cube at runtime, was accomplished by following this Catlike Coding tutorial.
• The gratuitous skybox is certainly not my own work - this was taken from the asset pack Polyverse Skies.

Wwise implementation notes:

When a shape is hit, it posts an event to Wwise, telling it to play a blend container of 2 Wwise SynthOne objects. Roundness and color are both set as RTPCs with a value between 0 and 1; roundness changes the mix of this blend container ("sphere" including both sine tone oscillators, "square" using a square wave as the top oscillator), and color changes its frequency modulation value. The note value is determined based on the black or white "key" the shape is currently on, and then sent to Wwise in a very non-elegant way - it maps this as an RTPC, which then sets a switch group, which is then used to select between a range of 13 different MIDI notes (which were each exported as .mid files from Logic) within a music switch container. (I was unable to find a way to send an explicit note value with a NoteOn message within Unity, but I would very much welcome any information on how to do that if possible!)

Things in the original scope I may revisit for the midterm or otherwise:

• Implementing the "playhead" generation as an ocean wave (represented as a flat mesh with vertices being manipulated with a water shader)
  • Floating mechanics for shapes on ocean mesh
  • Replacement of keyboard with subtle shading differences on sections of ocean
• Addition of moon object in upper right of screen
  • Ability to drag moon closer/further to change "tempo" of playhead/waves (this isn't how ocean wave speed actually works, but it's hopefully at least thematically coherent)
  • Ability to change moon's phase to change the scale used by the keyboard
• Ability to change size/base glow amount of shapes to change note velocity, reverb amount
• General synth editing to add complexity to the sounds used, effects of parameter changes
• Flexible, selection-based shape deletion
• Probably some other things, I drastically overscoped this