How Would I Go About Programmatically Interacting with VSt(I) Plugins to Synthesize Audio

How would I go about programmatically interacting with VST(i) Plugins to synthesize audio?

Well, since you asked, the ideal language for a project like this is going to be C++. Although there are wrappers for higher-level languages such as Java & .NET for the VST SDK, I couldn't find one for Ruby (though I did find this rather cool project which lets you program VST plugins in Ruby). So you will be stuck doing some degree of C/C++ integration on your own.

That said, you have basically two options here:

1. Write a VST Host in C++, and launch it as a separate process from within Ruby.
2. Integrate your Ruby code directly to the VST SDK, and load the plugin DLL's/Bundles directly from your code. This is probably the cleaner but harder way to accomplish your goal.

I wrote up a VST host programming tutorial on my blog awhile back which you may find useful in either case. It details how you open and communicate with VST plugins on both Mac OSX and Windows. Once you have gotten your host to load up the plugins, you need to be able to either send MIDI events directly to the plugin, either by reading them from file or some type of communication between your Ruby code and the VST host (ie, a named pipe, socket, file, etc.). If you are unfamiliar with the MIDI protocol, check out these links:

• The MIDI technical fanatic's brainwashing center (silly name, serious resource)
• The Sonic Spot's MIDI file specification (in case you need to read MIDI files)

As you might have already figured out, VST is fundamentally a block-based protocol. You request small blocks of audio data from the plugin, and you send along any MIDI events to the plugin right before it processes that respective block. Be sure not to ignore the MIDI delta field; this will ensure that the plugin starts processing the MIDI event directly on the desired sample. Otherwise, the plugin will sound a bit off-tempo, especially in the case of instruments.

The VST SDK is also based around floating-point blocks, so any data you get back will contain individual samples in the range { -1.0 .. 1.0 }. Depending on your desired output format, you may need to convert these to some other format. Fortunately, there seems to be a Ruby binding for the audiofile library, so you may be able to send your output into that in order to generate a proper AIFF/WAV file.

In all, it'll be a fair amount of work to get to your desired end goal, but it's not impossible by any means. Good luck!

How to call audio plugins from within Python?

...Two years later, here's an answer:

Igor Gadelha wrote a GitHub repo dpm that includes his vstRender class, which he wrote in JUCE. Currently it only works for mono plugins.
I wrote some simple code to illustrate how to use vstRender, which Igor included in
his "contrib" section: run_plugin.py.

Web Audio player with VST effect plugins?

This would require a LOT of custom C++ piping, and wouldn't likely ever be supported in-browser. We have an issue on VST support in web audio, but VST the way it is is very unlikely.

How to send blocks of audio to be processed by synthesizer -- without discontinuities

As I wrote the final paragraph, I realised:

fluid_synth_process() provides no mechanism for specifying timing information or sample offset. Yet we observe that time advances nevertheless (each block is different), so the simplest explanation is: the FluidSynth instance begins at time 0, and advances by numSamples*sampleRate seconds every time fluid_synth_process() is invoked.

This leads to the revelation: since fluid_synth_process() has side-effects upon the FluidSynth instance's timing: it is dangerous for multiple voices to run this upon the same synth instance.

I tried reducing const int numVoices = 8; to const int numVoices = 1;. So only one agent would invoke fluid_synth_process() per block.

This fixed the problem; it produced a perfect waveform, and revealed the source of the discontinuity.

So, I'm left now with a much easier problem of "what's the best way to synthesize a plurality of voices in FluidSynth". This is a much nicer problem to have. That's outside of the scope of this question, and I'll investigate it separately. Thanks for your time!

EDIT: fixed the multiple voices. I did this by making SynthesiserVoice::renderNextBlock() a no-op, and moving its fluid_synth_process() into AudioProcessor::processBlock() instead — because it should be invoked once per block (not once per voice per block).

I want to do a project on sound programming, but I don't know where to start

VST plugins are normally written with C++. But it's possible to use other languages as well. Building a VST plugin with C++ is quite an involved project. Without pre-existing experience it's probably too much for a school project. Additionally you'll probably spend as much or more time implementing the VST part and GUI as you will actual sound programming.

However there are a other ways to create a VST plugin and still get your hands dirty writing code.

SynthEdit is a modular environment for creating synthesizers and effects. Modules such as oscillators, envelopes etc are connected visually with wires. Patches can be exported as VST plugins. SynthEdit allows additional modules to be programmed with C++.

SynthMaker is another option similar to SynthEdit. It features a code module which allows you write DSP code without the difficultly of compiling modules in another language.

The advantage of using SynthEdit or SynthMaker in your situation is that you can focus your programming efforts on a particular area, such as creating a filter or oscillator module. SynthEdit/SynthMaker can then handle the other areas (GUI, voice logic, etc).

Using SynthEdit or SynthMaker will also allow you to prototype ideas quicker. So you will have more time to experiment with different synthesizer or effect architectures before settling on something to complete for your school project.

DSP theory can be quite involved and maths heavy but that alone shouldn't put you off. Depending on what you want to do, a basic understanding of digital audio principles and algebra level maths may be enough to take you a surprising long way.

---

Related Topics