Baked in support for midi-in, audio-in and audio devices/channels?

Strudel
1

Hello. Firstly I would like to congratulate all the devs on Strudel for creating such an amazing piece of software. I've been using Sonic Pi for some time now and while I find it great, recently I've been playing with Strudel a lot because of its great amount of built in synths and samples and the ease of using 3rd party ones.

As a sequencer strudel feels perfect to me already :slight_smile:

There are three things that I am missing from Sonic Pi:

  1. Ability to easily use midi inputs to play live music:

I want to use Strudel and patterns for drums but I want to be also able to be able to use a midi keyboard to trigger/control synths/samples "live". This is actually easy enough to accomplish already, here's an example of metronome done with Strudel's sound running next to a midi-controlled piano that uses synths/samples from Strudel (supports velocity and press duration):

await WebMidi.enable()
const myInput = WebMidi.inputs[0];
const noteControls = Array.from(Array(128)).map(x => null)
myInput.removeListener('noteon');
myInput.removeListener('noteoff');
myInput.addListener('noteon', e => {
  const [n, vel] = e.dataBytes;
  superdough({ note: n, s: 'piano', postgain: vel / 127 }, 0.05, 10).then(function (x) {
    noteControls[n] = x;
  });
});
myInput.addListener('noteoff', e => {
  const [note] = e.dataBytes;
  const { mute } = noteControls[note]
  mute(0.1);
  noteControls[note] = null
})

sound("metal casio casio casio")

This won't work in latest Strudel, but it required only one line of code to be changed to make it happen:

Midi-in could also be useful for control over some things like swing etc, so maybe it deserves a wrapper that makes the code above easier to use and reduces boilerplate (and possibly preserves midi state across live code reloads)?

  1. Ability to explicitly control audio channels (and possibly devices):

I've already figured out that this is possible with channels (ch).

I want to be able to redirect some synths to audio channels different than default. Exact use case is - direct some output generated by notes or sound to headphones device instead of default speakers device so I can try out some melodies before I switch the mix to "default" and play so that everyone can hear it. What I described would require support for managing media devices, but to simplify things for start, I could just use a 4-in-channel, 4-out-channel sound card and have the ability to use channels 0,1 for "default" and channels 2,3 for things I'd like to try out. Then I could just revert everything to go to channels 0,1. There is already support for panning in stereo, so I guess this can't be too difficult to implement.

3*. Support for live audio in.

This is a little cherry on top and I don't need it very badly, but I would love to have support for live audio in. In the simplest form I just want to mix music from other sources with Strudel output, which can be accomplished just with using an external mixer, but... If the audio could be handled inside the editor I could use the mixing facilities from point 2 to do the mix and also it would open up the possibility to use the already-present effects like reverb or delay.

Some feedback I'd love to have back:

  • Does the Strudel/Tidal community even want/need that stuff? Maybe Strudel/Tidal should just be used for its awesome patterns as a sequencer, not as a general music live coding tool?
  • How to approach coding this? Should I push for merging it into Strudel core? Should I build it as an esm "plugin", so that you import it and do something like "initMidiIn" or "initAudioIn" (similar to initHydra).

All feedback appreciated.

2 Likes
2

I would definitely love to have live audio input to Strudel. My personal use case is to chop up live input and use it as samples to do (weird things). I finally used Tidal Cycles with Tidal Looper to achieve the same, but having such support in Strudel would be awesome. I checked a bit how to implement it but I didn't find a solution at the time.

Your other two suggestions seem totally reasonable, and even though I don't need it personally yet, I might in the future. I would be in favor of having it in the core even though I have no call in the decision :slight_smile:
You might find people to discuss this in more details on the Tidal/Strudel Discord in the Strudel channels: Meet the community | Tidal Cycles

3

You could use tidal looper with strudel, via [the superdirt bridge(MIDI & OSC 🌀 Strudel).

4

for midi, there is already the midi package which also already has a midin function that currently only supports cc in. It is not yet documented, but you can find out more in the PR. Note in would be a good addition to it. There is also this issue for midi input, maybe we can talk about the details there (or preferably on the discord?).

Audio input is also very desirable for me, for which there is is this issue. I think there are 2 stages for audio input: 1. record samples into the sounds tab 2. record samples from code and loop them instantly (like tidal-looper).

5

Hello ! I've implemented a live audio in looper which you can try and re-use:

2 Likes