Introduction

Impropmtu has been designed to provide a highly dynamic real-time environment for crafting multimedia algorithms. Impromptu places a heavy emphasis on 'time based programming', which is the ability to accurately schedule code blocks for future execution. Another key emphasis in Impromptu is the ability to create and modify code blocks in real-time. Impromptu's Scheme implementation provides a small and elegant language for developing multimedia algorithms 'on-the-fly'.

This guide provides an introduction to some of the primary concepts behind Impromptu.

Architecture Overview

Impromptu is an amalgam of five primary components real-time scheduler, a Scheme language interpreter, an Integrated Development Environment (IDE), an audio plugin architecture and a vector based graphics engine. In this section we'll take a brief look at each of these components.

Scheduler
At its most fundamental level, Impromptu's real-time scheduler is a priority queue whose tasks are linked to C++ instance methods. At this level of operation, a method of any C++ class instance can be registered with a task and scheduled on the priority queue. This is the basis for all precisely timed activity in the Impromptu system. Scheduling in Impromptu is sample based and attempts to guarantee sample accuracy (more on this later).

AudioUnits
The scheduling engine is driven by the currently selected audio device which is automatically started when Impromptu initialises. Any default audio device settings, including samplerate and channel configurations will remain active for the duration of the Impromptu session **Currently applying changes to the default audio device or it's parameters midway through an Impromptu session requires a restart**. Impromptu uses Apples CoreAudio engine, and more precisely Apple's AudioUnit plugin architecture for all audio processing. Apple's AudioUnit library provides a plugin architecture allowing individual AudioUnit's to be connected together to form signal processing graphs of arbitrary complexity. The AudioUnit standard is well supported and provides Impromptu with an enormous array of commercial and non-commercial virtual instruments, filters, delays, reverbs, mixers, generators etc..

Once Impromptu is initialized the AudioUnit graph is started with a single default output unit. An impromptu preference specifies whether this should be a stereo or surround sound output. Once started the AudioUnit graph will run for the duration of the session, pulling audio data directly from the default output unit, which in turn pulls data from any units that are attached to it. **see Apples developer documentation for more information on the AudioUnit graph**. AudioUnit's can be instantiated during the session and freely attached or detached from the Graph at run-time. The AudioUnit graph runs in a real-time thread and is responsible for audio production, updating Impromptu's internal sample clock and notifying the scheduling thread each buffer cycle. The scheduler is run in an independent thread to ensure that the execution of scheduled tasks takes a lower priority to audio processing. Impromptu's internal clock is incremented in buffer size increments where the buffer size is an Impromptu parameter that defaults to 128 samples. At it's worst this means that scheduled events can be out by up to the buffer size. However, the AudioUnit specification includes an event offset that allows AudioUnit calls to be offset from buffer boundaries. This results in guaranteed sample accuracy for most audio events (note-no's, note-off's, contol changes, program changes, AudioUnit parameter changes etc..).

Scheme Interpreter
During Impromptu's initialisation a default Scheme interpreter is started. Impromptu uses a modified version of the TinyScheme interpreter, a light weight Scheme interpreter written in the C programming language. The interpreter runs as a service allowing local or remote connections. This provides a flexible architecture whereby multiple remote users can share a single interpreter or a single user can run multiple local interpreters. Each new interpreter started on a single host shares the same AudioUnit Graph and internal clock but runs within in it's own Mach kernel thread. A shared clock ensures that multiple local interpreters can be temporally synchronised. Impromptu also provides the ability to set the internal clock, allowing remote interpreters to synchronise across the network **The author has successfully used OSC messages to synchronise multiple hosts**.

A large library of native calls provide the Impromptu scheme interpreter with direct access to the Impromptu scheduling engine, audio engine and graphics engine. Of primary importance among these is the CALL-CC function which allows Scheme to schedule C++ methods for later execution. All C++ methods available from within the Scheme environment are defined as global constants when the interpreter is initialised. The CALL-CC function uses these constants to build tasks to send to the scheduling engine. Along with the C++ method a Scheduled task must also contain any arguments that the method may require. C++ methods can access Scheme values directly from the interpreters heap. The interpreters garbage collector has been modified to allow this to occur in a thread safe manner. The Scheme interpreter also ensures that any Scheme values currently stored as arguments on the Scheduling queue are "eprotected"e from garbage collection. This is required so that Scheme values passed as arguments to C++ methods are not garbage collected until after the scheduler executes the method call. The delay between the initial Scheme call and the scheduled C++ call could be samples, minutes, hours or days. Impromptu users are hidden from this layer of complexity and should never need to call CALL-CC directly. Instead, Scheme wrapper functions are used to hide this scaffolding.

Integrated Development Environment
In order to evaluate Scheme expressions 'on-the-fly' Impromptu includes a dynamic code editor that is designed to assist real-time, dynamic programming. As well as supporting standard features such as colour-syntax highlighting, bracket matching, auto-indenting and code markers, the editor also supports the ability to send code to the Scheme interpreter for evaluation. This is achieved by pressing the evaluate button (or equivalent keyboard shortcut) while the cursor is located within a valid Scheme expression. The editor will pass the expression to the currently selected Scheme interpreter (this is usually the default interpreter) across a TCP connection (remember that interpreters can be local or remote). The Scheme interpreter will then evaluate the expression and display the result in the editors result bar. The Impromptu IDE also supports custom AudioUnit User Interfaces, allowing users to modify AudioUnit parameters in real-time via their User Interface as well as programatically through Impromptu library functions.

Graphics
The Impromptu IDE also incorporates CoreGraphics canvas's. Impromptu supports bezier paths, font rendering, image manipulation and QuickTime movie support. The complete set of CoreImage filters are supported along with UI event capture including mouse down, mouse up, mouse drag, QWERTY key up and down etc.. Impromptu's real-time scheduling provides interesting opportunities for animators who are interested in breaking free from the traditional timeline. Animators can schedule drawing commands and implement multiple time independent animations using temporal recursion.

Time Based Programming

One of the primary differences between Impromptu and most other general purpose programming environments is an emphasis on precisely scheduled code execution. But what is the destinction we are making here, all programming languages are temporal.

Impromptu incorporates a scheduling engine directly into the programming environment allowing tasks to be scheduled for execution at precise times into the future. Unlike using thread constructs such as sleep(), wait() etc. that do not provide any strong guarantees about temporal accuracy the scheduling engine is guaranteed to execute its tasks at the requested time (the number of audio samples since Impromptu was started). This temporal guarantee is significant for time critical domains such as music.

		; Schedule three notes to play in sequence
	        (play-note (now) piano 60 80 0.5)
		(play-note (+ (now) 22050) piano 64 80 0.5)
		(play-note (+ (now) 44100) piano 67 80 1.0)
	      

Impromptu uses a schedule and forget, or asynchronous, style of programming, and a design pattern called termporal recursion. This is different from languages such as ChucK that use a synchronous approach. What's the difference? Synchronous timing works by holding up code until some specified time in the future. This is basically the same concept as using sleep() although strongly timed languages like ChucK guarantee the length of the sleep. Impromptu works by scheduling tasks to be executed at some time in the future. Once a task has been scheduled thread execution moves immediately onto the next expression. A seudo code example may help to illustrate.

	      //synchrous timing
	      play-note(now)
	      time = (now) + 44100
	      play-note(now)

	      //asynchronous timing
	      play-note(now)
	      play-note(now + 44100)
	    

Strongly timed code holds up thread execution until the global time is equal to the value 'time'. The asynchronous code example schedules tasks into the future and immediately continues execution.

Scheduling events is a fairly common programming technique and there wouldn't be much else to say if Impromptu wasn't a dynamic language. Impromptu allows us to create and schedule code for future execution as well as data events such as notes and graphics objects. The ability to schedule blocks of code for precise future execution is a very useful technique for time based programming.

There is a common design pattern in Impromptu programming called temporal recursion. By writing a function that schedules itself as its final action a termporally recursive callback loop can be established. Here is an example demonstrating a foo function that will play a note and then schedule itself to be called back in one second. This loop will continue indefinitly.

	      (define (foo)
	          (play-note (now) piano 60 80 1.0)
	          (callback (+ (now) 44100) 'foo))
	    

You can create as many temporal recursions as you like. You can even create temporal recursions inside temporal recursions. A termportal recursion need not run at a constant rate. By adjusting the time increment on each cycle the callback rate (control rate) can be constantly adjusted. Here is an extension to the previous example that will randomize the note length (NOTE: that the each callback is now scheduled at (now) + the duration of the note).

	      (define (foo)
	          (let ((note-length (random '(0.25 0.5 1.0 2.0))))
                     (play-note (now) piano 60 80 note-length)
                     (callback (+ (now) (* note-length *second*)) 'foo)))
            

One off ananymous functions can also be scheduled for future evalatuion. The code example below shows a one off anonymous function scheduled for evaluation in one minute from now.

	      (callback (+ (now) *minute*)
	                (lambda ()
	                    (play-note (now) piano 60 80 1.0)))
	    

There are a couple of gotcha's to keep in mind when doing schedule and forget programming. The first thing to keep in mind is that (now) is an airy fairy thing. In the example below the two notes may be scheduled to play on the same sample but they may not.

	      (play-note (now) piano 60 80 1.0)
	      (play-note (now) piano 72 80 1.0)
	    

Often this lack of precision is fine but where absolute accuracy is required a time variable should be used.

	      (let ((time (now)))
	         (play-note time piano 60 80 1.0)
	         (play-note time piano 72 80 1.0))
	    

This inaccuracy becomes more evident when amplified over time such as using (now) inside a recursive callback loop. We can avoid the problem by precisely incrementing a time value between each recursive callback (note that any arguments required by the function being called back too must also be passed to callback).

	      ; This is bad
	      (define (loop)
	         (play-note (now) piano 60 80 1.0)
                 (callback (+ (now) 44100) 'loop))

	      ; This is good
	      (define (loop time)
	         (play-note time piano 60 80 1.0)
	         (callback (+ time 44100) 'loop (+ time 44100)))
	    

The second major gotcha in recursive callback loops is that (now) is now. Code requires some time to execute. If you are executing a call to evaluate a note (now) by the time the code is evaluated it will already be late. You should always try to schedule your code execution ahead of the scheduled time of your tasks.

	      ; This is bad
	      (define (loop time)
	         (play-note time piano 60 80 1.0)
	         (callback (+ time 44100) 'loop (+ time 44100)))

	      ; This is good
	      (define (loop time)
	         (play-note time piano 60 80 1.0)
	         (callback (+ time 40000) 'loop (+ time 44100)))
	    

In the first definition of loop the time sent as an argument to loop is exactly the same time as the scheduled callback time. The problem with this is that the next note needs to be scheduled at exactly the same time that the function is called. The note will always be late. The second function schedules the callback just ahead of the time that we want to schedule the note. This gives us 4100 samples to execute the code to schedule the note before the note is required to sound.

...

Live Programming

Impromptu is a dynamic, interactive programming environment designed for use in live performance. Here I refer to 'live' programming in two senses. Impromptu was designed for use in 'live' performance, but Impromptu is also a 'live' environment. Everything is 'live'. Impromptu is an environment that encourages change, from defining and re-defining code on the fly to code that writes itself, Impromptu is designed for constant change.

...

Local And Remote Processes

Impromptu can talk to multiple processes simultaneously. These processes can be running locally and/or remotely. You can send evaluations from the editor window to any active process by selecting that process from the drop down list at top of the editor window. The default process is the 'primary process' and is the only process running when Impromptu first starts up.

Impromptu supports multiple processes for two primary reasons. Firstly, the ability to connect to remote processes allows multiple users to collaborate in a shared runtime process. Secondly, each process is computationally independant, which stops code in one process from interfering with code running in another process. This is useful in situations where you need to execute functions that will take a long time to complete and would normally delay the evaluation of timed callback functions.

Impromptu does not currently support shared process memory so functions defined in one process will not be available in another process. However, you can use IPC-DEFINE to trasfer values from one interpreter into another interpreter.

	      ; define my-print in default process
	      (define my-print
	         (lambda (s)
	            (print s)))

	      ; bind the value associated with 'my-print
	      ; in process2
	      (ipc-define "eprocess2"e 'my-print)
	    

All local processes share the same scheduling engine and so time is synchronized across all local processes. All local processes also share the same AudioUnit graph and graphics canvases which would allow you to schedule play-notes from any process for example.

...

Generative Programming

...