Using VSTPlugin with Tidal

[mattia.paterna]

Background

I am using VSTPlugin library in SuperCollider to load VSTi as Ugens and have them played from within Tidal using MIDI. Here is a minimal definition of this workaround:

(
SynthDef(\VSTi, {|out|
	var sound = VSTPlugin.ar(nil, ~dirt.numChannels, id: \SynthName);
	Out.ar(out, sound);
}).add;
)

~synth = Synth(\VSTi);
~instrument = VSTPluginController(~synth, \SynthName).open("/path/to/vst/plugin", editor: true);
// add MIDI device to SuperDirt
~dirt.soundLibrary.addMIDI(\vst, ~instrument.midi);

The library has a nice feature, that is it is possible to change the synth parameter using set messages such as
~instrument.set(param, value);.
There is also another alternative, i.e. playing an Event such as
(vst: ~instrument, type: \vst_set, params: [param], param: value).play;

I though of having this automated through Tidal and therefore I made a function that does nothing but calling set method for the instrument. Here is a minimal definition:

(
SynthDef(\VSTiParams, { |param, value|
	~instrument.set(param, value);
}).add;
)

I then have a loopback into Tidal so I can apply SuperDirt effects to the VSTi output:

(
SynthDef(\VSTIn, { |vstOut, out, pan|
	var sound;
	sound = In.ar(vstOut);
	Out.ar(out,
		DirtPan.ar(sound, ~dirt.numChannels, pan)
	)
}).add
);

and here is how I call everything in a Tidal session:

do
    d1
        $ slow 8 $ n (scale "<minor scriabin>" ( "0 ~ 2 3 [5 ~ 7] -1"))
        # "vst"
    d2
        $ sound "VSTIn"
    d3
        $ s "VSTiParams" # pI "param" "25" >| pF "value" "0 0.5"

---

Problem
The VSTi seems unresponsive although I can see that Dirt events are received from Tidal.
I haven't been lucky in identifying the problem, does anyone know if it is possible to call object methods in SuperCollider from Tidal using an ad-hoc SynthDef? Or is it a better way?

[mrreason]

Hello @mattia.paterna,
VSTPlugins with Tidal/SuperDirt would be a nice thing!

To help you with this, here are a few things you should consider. SynthDefs are in my understanding a static (kind of?) object and normally only UGens are virtually "patched" inside. This has to do with the server side representation of Synths. And controlling a Synth in another SynthDef or triggering events are dynamic actions. I believe you want to trigger a non dirt event with a dirt event. Look to the similarity between

(vst: ~instrument, type: \vst_set, params: [param], param: value).play;

and

(type: \dirt, dirt: ~dirt, \s: "808").play;

I guess you should start a non dirt event outside a Snythdef. If you want to use it inside s, than you should implement it as an function. Maybe it could be useful as well to use the VSTPlugin has an effect, so you can specify the fx chain inside SuperDirt (here you find a summary about the meaning of SynthDefs, functions and effects in the SuperDirt context: Custom Synthdef not working on Windows - #2 by mrreason)

I think you could do something like this (not tested):

~dirt.soundLibrary.addSynth(\vst, (play: { 
    ~instrument.set("param", ~myparam);
    //Alternative
    //(vst: ~instrument, type: \vst_set, param: ~myparam).play;
}));

And use it in TidalCycles like

d1 $ s "vst" # myparam "<2 3 4>"

Notice, that you can use every global SuperCollider variable with TidalCycles if you defined the pattern function for it.

[mattia.paterna]

Thanks @mrreason,

@mrreason

I believe you want to trigger a non dirt event with a dirt event

That is correct, I want to send patterns of parameterNumber, parameterValue couples to the VSTPluginController.

I am trying two approaches now but I have miserably failed so far.

---

Approach 1: function
I am looking into function hacks so I can play the event from within a function and call the function using s as you said. I don't know for what reason but if I substitute to the original function below

(
~diversions[\tlalak]  = {
	Pfindur(~sustain ? 1,
		Pbind(
			\dur, 0.04,
			\note, Pbrown(-15, 15 ! 5, 0.3, inf) + 16,
			\out, ~out, \pan, ~pan, \amp, ~amp * 0.1
		)
	).play
};
)

this new function where I do parameter modulation

(
~diversions[\tlalak]  = {
	Pbind(
		\type, \vst_set,
		\vst, ~instrument, // the VSTPluginController instance
		\params, [25, 26], // parameter indices/names
		25, Pwhite(0.1, 0.9), // value for parameter 25
		26, Pwhite(0.3, 0.7), // value for parameter 26
		\dur, Prand([0.25, 0.5, 1], 16), // how many?
	).play
};
)

when I test it with (type:\dirt, s:\tlalak, dirt: ~dirt, out: 0, sustain:0).play; I see no effect at all.
The interesting thing is that if I execute the Pbind alone, it does have an effect on the synth parameters 25 and 26.
If I then wrap the function into a synth definition such as

(
~dirt.soundLibrary.addSynth(\tlk,
	(play: {(type:\dirt, s:\tlalak, dirt: ~dirt).play});
);
)

I get the same behaviour: it works with the original snippet, it does not work with mine.

---

Approach 2: effect
The problem with the approach above is that I don't know exactly how to give parameter no. and parameter value as input argument (probably you explained it before and I did not get it). The idea with this is to use it as effect applied to the vst loopback i.e.

let param = pI "param"
    value = pF "value"

d1 $ s "vstLoopback" # param "25 26" >| value "0 0.5 1 0.5"

I am following the adding effect hack section and inside the function defined at l. 81 I put the event to play. If I use the example provided it works, but when I change it I do not see any event triggered.

I am bit confused

[mattia.paterna]

One more thing: since the parameter modulation works in parameter index/value pair how could I rewrite this function

~dirt.soundLibrary.addSynth(\vst, (play: { 
    ~instrument.set("param", ~myparam);
    //Alternative
    //(vst: ~instrument, type: \vst_set, param: ~myparam).play;
}));

so that it takes two parameters?

[mrreason]

I try to create a minimal working example, but I have problems to get the VSTPlugin quarks to work under macOS...Do you have any ideas?

In SuperCollider I get the message exception in GraphDef_Recv: UGen 'VSTPlugin' not installed.
But I linked the UGen in SuperCollider and can see it in the quarks list.

[mattia.paterna]

Have you checked this post?
Apparently VSTPlugin is not a Quark. I hope this fixes it for you, I am running on Mac OS and I had no problem using VSTPlugin.

[mrreason]

All right, now it can be used in SC. I came across the post but didn't read it carefully yesterday. I'll see if I can get a look at it this evening.

[mattia.paterna]

Thomas, I also wrote in the SuperCollider Forum, you can find the question here. I hope we can work this out because it would be so cool and personally a game changer.

[mrreason]

Allright! As you can see here I try to get VSTPlugin to work and asked the developer for help. But I had no success so far. Something is really strange in my environment.

I guess with my knowledge I should be able to suggest an implementation, but it definitely can't hurt if someone in the SC forum posts a possible solution. Maybe the SC forum is the better place to discuss an implementation anyway?

But I will let you know when I make progress.

[mattia.paterna]

@mrreason

Something is really strange in my environment.

I read the thread, I am sorry I can't really help. Everything is working fine on my end, but I do have a quite old version of OS X (10.13.6).
I'll keep you posted whenever I glimpse any answer.

[mrreason]

Hey @mattia.paterna
I managed to get the VSTPlugin working. I just couldn't use it because I had a small bug in my startup script...

The best news is that I have taken a first step towards using VST plugins with Tidal.
The SC code for this looks like this:

// For macOS 10.15+ because of gatekeeper
// ~/Library/Application Support/SuperCollider/Extensions/VSTPlugin/plugins
// xattr -rd com.apple.quarantine *.scx

// /Library/Audio/Plug-Ins/VST/
// spctl --add --label "ApprovedVSTs" *.vst
// spctl --enable --label "ApprovedVSTs"

// You should not use
// ~dirt.soundLibrary.addMIDI(\vst, ~instrument.midi);
// (because then you are not able to use set)

(
SynthDef("VST", { |out|
	var signal = In.ar(out, ~dirt.numChannels);
	signal = VSTPlugin.ar(signal, ~dirt.numChannels, id: \vk); // This is a vst instrument
	// Here you could define effects or more instruments(?)
	ReplaceOut.ar(out, signal)
}).add;
)

(
var instrumentSynths= ();
var instruments = Dictionary[
	\vk -> VSTPluginController(Synth("VST", [id: \vk]), id: \vk).open("VK-1", editor: true, verbose: true),
	//... Define your vst instruments here
];

// Just to make instrument accessible from outside
// to use ~instrument.editor
~instruments = instruments;

instruments.keysValuesDo { |key, value|
	instrumentSynths[key] = {
		if (~param.notNil && ~value.notNil, {
			instruments[key].set(~param, ~value);
		});
		(vst: instruments[key], type: \vst_midi, midinote: ~n+60).play;
	};
};

instrumentSynths.keysValuesDo{ |key, func|  ~dirt.soundLibrary.addSynth( key, (play: func)) };

)

// Call this to manually change parameter for debugging
~instruments[\vk].editor

And in TidalCycles you can use it like this:

d1 $ n (scale "<minor scriabin>" ( "0 ~ 2 3 [5 ~ 7] -1")) 
   # s "vk" # pI "param" "2" # pF "value" "<0 0.5 1>"

You are able to set the parameters and playing notes in one statement and switch between vst synths. Unfortunately I have not yet been ablet to get VST effects into the fx chain of SuperDirt.

Maybe we should rename this thread or make another thread explicitly to discuss the "Using VST plugins with TidalCycles"?

I think I will open a github repo for this in the mid-term, because the VST Tidal project could get pretty big.

[mattia.paterna]

Okay that sounds interesting! I have couple of follow-up comments / questions.

@mrreason

// You should not use
// ~dirt.soundLibrary.addMIDI(\vst, ~instrument.midi);
// (because then you are not able to use set)

Do you know the reason behind this, or is it just something triggered by the VSTPlugin/SuperDirt combination?

@mrreason

instruments.keysValuesDo { |key, value|
	instrumentSynths[key] = {
		if (~param.notNil && ~value.notNil, {
			instruments[key].set(~param, ~value);
		});
		(vst: instruments[key], type: \vst_midi, midinote: ~n+60).play;
	};
};

Do I understand correctly if I say that you are using the diversion concept here e.g. you are using SuperDirt to trigger functions?

---

Comments

• I see that I am not able to call any effect e.g. d1 $ n (run 4) # s "vk" # crush 3 has no sound -- it seems only # gainworks somehow
• events are overlapping e.g. if I have 4 events per cycle each event does not last 1/4 but 1/2
• I am able to hear some periodic glitches, could that be due to SuperDirt own envelope?

---

Having said that, great input @mrreason! I thank you a lot for your effort so far. I see comments coming from the other thread in the SC Forum and I will try to summarise them here.
I agree with changing the topic name, I would say let's wait for the new thread.

[mattia.paterna]

Yesterday we went through this in the SC meetup in Copenhagen and we found two possible ways to achieve the desirable.
Big shout out to @elgiano who runs these meetups and literally saw things that I could never see. Many thanks to Claudio Cabral as well (he is not in the Tidal club).

---

APPROACH 1
Send exclusively MIDI messages to VSTPlugin

Since we are already able to send midinote messages from Tidal to VSTPlugin, we can just send ccn and ccv as explained in the SuperDirt MIDI Tutorial.
The issue with this is that we must go through MIDI Learn assignment for every parameter we want to control (at least in the synths I use). This is not particularly handy.

What we can do instead is to create one more MIDIOut from SuperCollider and have SuperCollider connect to it with MIDIIn. A MIDIDef is then created, which responds to MIDI control messages setting the VST parameter.

-- in SuperCollider
MIDIClient.init;
~midiOut = MIDIOut.newByName("LoopMIDI", "IAC Bus 1"); // change it accordingly
// add MIDI device to SuperDirt for parameter automation
~dirt.soundLibrary.addMIDI(\vstparam, ~midiOut);
// add MIDI device to SuperDirt to playing midinotes
~dirt.soundLibrary.addMIDI(\vstkeys, ~vsti.midi);
MIDIIn.connectAll // TODO: we should just connect to ~midiOut

// define response to CC from Tidal
(
MIDIdef.cc(\vstParamAutomation, {
	| val, cc, channel |
	//[ val, cc, channel ].postln;
	~vsti.set(cc,  val/127);
});
)

and then call it from Tidal

-- in Tidal
do
    d1
        $ slow 4
        $ n (scale "<minor scriabin>" ( "0 ~ 2 3 [5 ~ 7] -1"))
        # s "vst"
    d2
        $ stack [
            ccv (segment 16 $ range 0 127 $ slow 2 $ isaw) # ccn 25,
            ccv (segment 16 $ range 0 64 $ slow 6 $ sine) # ccn 26
        ] # s "vstparam"

PS: MIDIIn.connectAll should be replaced as now SuperCollider listens and responds to any CC message coming from anywhere.

---

APPROACH 2
Send midinote messages and do parameter automation using diversions

The idea of having SuperDirt execute a custom function still holds and proved to be really effective with two minor tweaks:

-- in SuperCollider
(
var diversions = ();
~diversions = diversions;
~d1.defaultParentEvent[\diversion] = { |dirtEvent| diversions[~s].value(dirtEvent.event) };
)

(
var vsti = ~vsti;
~diversions[\vstparam] = {
	|event|
	var ccn = event.ccn;
	var ccv = event.ccv;
	event.postln; // just for inspection
	vsti.set(ccn, ccv);
}
)

The tweaks:

• I was always getting an error message such as Message vsti not understood.
  What Gianluca spotted is that we have to take the environment variable ~vsti and make it global in the scope of the diversion (I still have to understand why though).
• you would like to pass input arguments to the diversion via Dirt events, therefore we should pass the event when calling the function in the diversion.

We can then use the same Tidal code as above, where vst is responsible to playing the synth.

---

Epilogue

We also tried to have both notes and parameters in one single orbit:

-- in SuperCollider
(
var vsti = ~vsti;
~dirt.soundLibrary.addSynth(\vst, (play: {
	(vst: vsti, type: \vst_midi, midinote: ~n+60).play;
	vsti.set(~ccn, ~ccv);
}));
)


-- in Tidal
d1 
  $ n "0" 
  # s "Arp" 
  # pI "ccn" 25 >| pF "ccv" (segment 16 $ range 0 64 $ slow 6 $ sine)

and this works indeed (it is basically a less refined version of yours @mrreason).
Though I do not find it powerful as the vst_midi event gets re-triggered every time there is a new ccv or ccn to set.
I would rather prefer keep the two actions distinct as in my point of view, the melorhythmic contour should be independent of changes in the synth.

---

An interesting discussion could be which approach to prefer and why.

It could also be nice to have all these options documented somewhere so they do not get hidden here (@yaxu any thought on how to structure it?).

[Robin_Hase]

hmm while i was able to get @mrreason's example to work i'm currently struggling with enormous timing jitter when sending notes to vst plugins via tidal...

when sent from the same orbit superdirt and regular synthdefs are bang-on while vstplugin sounds like someone randomly turning the swing amount on an mpc.

triggering vst instruments via supercollider pbinds is 100% timing accurate though, so there must be some issue in the communication between superdirt and vstplugin...

edit: increasing oLatency in BootTidal.hs did not help at all

[mrreason]

That sounds really strange. I'll take a look over the weekend and see if I can reproduce that. I was anyway planning to invest more time in this topic in the next few weeks.

Can you please tell me what operating system you are using and are you using Tidal 1.7?

[Robin_Hase]

tidal 1.7 on windows 10 - the problem persists across multiple audio interfaces and asio drivers as well, and again it's not an issue with superdirt or other synths like the ones in the scl2ork quark...

i'm now playing around with variations of

~dirt.soundLibrary.addMIDI(\vital, ~vital.midi);

and while in practice that's more stable than the method you posted it still has the occasional dropouts and flakiness at high tempos that i've also found when sending MIDI to vst instruments hosted in DAWs or Max/MSP...

i guess that's what got me so excited about hosting vst instruments directly within SC in the first place - seeing a light at the end of the tunnel on my eternal quest for accurate timing

what really confuses me is how rock solid the timing is when patterning midi messages via Pbinds directly in SC as in the documentation for vstplugin - this at least proves that it's not a technical limitation of supercollider or the vstplugin package

edit: and by rock solid i mean i can get consistent karplus strong like overtones when sending 1000 notes per second to my vst via Pbinds - with MIDI via superdirt it struggles to play a 24th note arpeggio without dropouts :S

[mrreason]

I also have to admit that the code here is only a proof of concept so far, and it's all still a work in progress. Therefore your experiences are very valuable

I don't know what's going wrong yet, but may need to look at whether the incoming OSC messages are not being processed properly by the server latency.

However, I must also confess that I have not yet done a real stress test with up to 1000 notes per second, and it cannot be excluded that this is a separate issue.

[Robin_Hase]

oh i mean i'm not actually planning to use 1000 notes a second with vst instruments
the timing jitter is within the range of up to 200 milliseconds - and without shifting the pattern relative to superdirt patterns even the 1 in 10 notes that happen on time would be delayed by what seems to be a fixed amount - your point about the server latency not being communicated properly seems like a good direction to investigate!

from what i could find so far it looks like tidal is including timing delta values in its OSC messages, but i don't know how the actual scheduling happens within superdirt - and while i was able to add standard arguments such as ~sustain in the code you posted (to allow for automatically adjusted note lenghts determined by the note pattern and #legato) i honestly wouldn't know which method to pass these arguments to in supercollider...

i'm moving to a new place next week so i'll be rather busy but i'll try investigating further in that direction, thanks for the quick answer (:

[Robin_Hase]

further findings -

vstplugin defines two new event types (as found here: VSTPluginController.sc)
these are called \vst_midi for note messages and \vst_set for parameter automation.

superdirt has its own event type called \dirt (DirtEventTypes.sc)

i'm still not familliar enough with Supercollider to grasp all this properly, but could it be that when tidal dispatches an OSC message about a \dirt event to be scheduled by superdirt, which then calls vstplugincontroller at the "correct" time, then vstplugin itself dispatches a \vst_midi or \vst_set message, which in turn takes additional (and probably undefined) time to resolve?

what i'm assuming is that we have two places where we're scheduling events where we want to do it only once, and also that relevant information for proper scheduling such as latency and delta information only get passed to superdirt but not to vstplugincontroller

[mrreason]

@Robin_Hase

but could it be that when tidal dispatches an OSC message about a \dirt event to be scheduled by superdirt, which then calls vstplugincontroller at the "correct" time, then vstplugin itself dispatches a \vst_midi or \vst_set message, which in turn takes additional (and probably undefined) time to resolve?

This is a good point and in fact cannot be excluded too. TidalCycles schedules its osc messages. But TidalCycles osc message are send over udp. And then the Dirt events gets scheduled too.

My idea for the "big picture" currently is that you should control the VSTPluginController without SuperDirt but directly via TidalCycles OSC messages. TidalCycles is able to send its control messages to multiple addresses and ports in parallel.

Meanwhile, since Tidal 1.7 there are control buses that you could use to manipulate the automation parameters while you playing long notes. This way the \vst_midi and \vst_set events could be controlled separately via the same orbit.

However, all of this still requires a deep dive into how it all fits together. But IMHO, this should be the right direction at the moment.