Interactive Animation in Python

This sample package demonstrates interactive live generation of bridge animations using popular open-source Python tools. The scripts output image streams over the network to either a local previewer or the bridge server for live testing.

The general intent of the package is to provide a simple interactive toolkit as a programmatic alternative to Pharos Designer. The visual content is specified as data and code but can also be controlled via keyboard interaction. This approach supports a hybrid compositional approach which blends generative programming with improvisational exploration.

The code requires a working installation of Python 3 with OpenCV, NumPy, and several other third-party packages. For suggestions on setting up your system please see Python 3 Installation.

The code can be browsed on the course site under streaming/, downloaded as streaming.zip.

Generator Classes

The following classes represent different building blocks for synthesizing image streams. The visual instrument makes extensive use of the Python iterator protocol and generator functions to define finite and infinite image sequences. This allows using standard libraries such as itertools to define compositions.

class streaming.pb_instrument.PianoRoll(config)

Collection of images which can be iterated to emit each row as a video frame. Implements the iterator protocol to return generator objects which produce a sequence of smoothly interpolated image frames at a constant tempo. The semantics of PianoRoll are that of a collection; each iteration of the collection is independent and starts at the beginning.

class streaming.pb_instrument.Waves(config)

Iterable object to simulate 1D waves. Implements the iterator protocol to generate image frames. Note that all iterators returned use the same object state, so evaluating multiple iterations simultaneously may produce unexpected results.

class streaming.pb_instrument.ConwayLife(config)

Iterable object to simulate Conway’s Life. Implements the iterator protocol to return generator objects which produce interpolated image frames. Typically simulates a binary world somewhat larger than the frame size. Produces a smoothly interpolated image at the frame rate while the simulation runs at a slower tempo. Note that all iterators returned use the same object state, so evaluating multiple iterations simultaneously may produce unexpected results.

class streaming.pb_instrument.Diffusion(config, color_table)

Iterable object which produces an animation of blurring impulses. Note that all iterators returned use the same object state, so evaluating multiple iterations simultaneously may produce unexpected results.

class streaming.pb_instrument.Colors(config)

Color and palette tables. Can produce several different image generators with static color fields or simple generated patterns.

Utility Classes

The following classes represent other useful building blocks within the visual instrument.

class streaming.pb_instrument.VideoWriter(config, path=None)

Video file writer to transcode image frames into a video file with the given path. The default codec is lossless PNG image frames in an AVI container.

class streaming.pb_instrument.BridgeInstrument(config)

Real-time multi-function image generator for the Pausch Bridge. Implemented as an infinite iterator, i.e. responds to __next__() to produce image frames, responds to __iter__() to return self.

class streaming.pb_instrument.MainApp(config, instrument)

Main application object implementing the user input processing, frame timing, and networking I/O.

Interactive Instrument

pb_instrument.py

Pausch Bridge visual light instrument for real-time performance and improvisation. Sends image output in real time as OSC messages to a previewer and/or the bridge.

class streaming.pb_instrument.BridgeInstrument(config)

Real-time multi-function image generator for the Pausch Bridge. Implemented as an infinite iterator, i.e. responds to __next__() to produce image frames, responds to __iter__() to return self.

reset_graph()

Safety function to reset the operator graph if the sequence ends.

transition(next_iterable)

Start a transition to the next effect.

class streaming.pb_instrument.Colors(config)

Color and palette tables. Can produce several different image generators with static color fields or simple generated patterns.

color_by_name(name, duration=None)

Create a color field generator which produces a floating point frame of the named color. The default is an infinite sequence, or a duration in frames can be specified.

iterate_image(frame, duration=None)

Image sequence generator which produces a stream of the same image. The default is an infinite sequence, or a duration in frames can be specified.

noise(duration=None, tempo=300)

Create a noise field generator. The default is an infinite sequence, or a duration in frames can be specified. Image are generated at a tempo slower than frame rate and interpolated.

palette_by_name(name, duration=None)

Create a palette color field generator. The default is an infinite sequence, or a duration in frames can be specified.

rainbow(duration=None)

Create a rainbow field generator. The default is an infinite sequence, or a duration in frames can be specified.

class streaming.pb_instrument.ConwayLife(config)

Iterable object to simulate Conway’s Life. Implements the iterator protocol to return generator objects which produce interpolated image frames. Typically simulates a binary world somewhat larger than the frame size. Produces a smoothly interpolated image at the frame rate while the simulation runs at a slower tempo. Note that all iterators returned use the same object state, so evaluating multiple iterations simultaneously may produce unexpected results.

random_reset()

Flood the world with random cells.

class streaming.pb_instrument.Diffusion(config, color_table)

Iterable object which produces an animation of blurring impulses. Note that all iterators returned use the same object state, so evaluating multiple iterations simultaneously may produce unexpected results.

class streaming.pb_instrument.MainApp(config, instrument)

Main application object implementing the user input processing, frame timing, and networking I/O.

process_user_input()

Logic for interpreting keystrokes and triggering animation events, intended to be customized for a particular show.

run()

Infinite event loop to interpret user input and stream out image frames.

sigint_handler(signal_number, stack_frame)

Handler for user keyboard interrupts (e.g. control-C), to be customized with close or shutdown handlers.

class streaming.pb_instrument.PianoRoll(config)

Collection of images which can be iterated to emit each row as a video frame. Implements the iterator protocol to return generator objects which produce a sequence of smoothly interpolated image frames at a constant tempo. The semantics of PianoRoll are that of a collection; each iteration of the collection is independent and starts at the beginning.

class streaming.pb_instrument.VideoWriter(config, path=None)

Video file writer to transcode image frames into a video file with the given path. The default codec is lossless PNG image frames in an AVI container.

close()

Finalize the stream and close the output file.

write(frame)

Write an RGB image to the video file.

class streaming.pb_instrument.Waves(config)

Iterable object to simulate 1D waves. Implements the iterator protocol to generate image frames. Note that all iterators returned use the same object state, so evaluating multiple iterations simultaneously may produce unexpected results.

streaming.pb_instrument.create_defaults()

Create a configparser object with default values for all configurable settings. Note that this includes values for several different objects.

streaming.pb_instrument.crossfade(source1, source2, duration)

Generator function to produce successive frames of a finite video sequence which fades from one source to another. Yields a video image frame. Exits once the fade sequence is complete. Being a generator, it returns a function which follows the iterator protocol, e.g. which can be evaluated using next() to produce images.

Parameters:

• source1 – iterable of images fading out

• source2 – iterable of images fading in

• duration – cross-fade duration in frames

streaming.pb_instrument.cyclefade(source1, source2, period)

Generator function to produce successive frames of a video sequence which fades back and forth from one source to another. Yields a video image frame. Does not exit, so infinite sources produce infinite sequences. Being a generator, it returns a function which follows the iterator protocol, e.g. which can be evaluated using next() to produce images.

Parameters:

• source1 – iterable of images

• source2 – iterable of images

• period – cross-fade cycle duration in frames

streaming.pb_instrument.keyframe_interpolator(keyframes, tempo=60, frame_rate=30.0)

Generator function to produce successive frames of a video sequence by linear interpolation between keyframes at a constant tempo. Yields a video image frame. Exits once the sequence is complete. Being a generator, it returns a function which follows the iterator protocol, e.g. which can be evaluated using next() to produce images.

Parameters:

• keyframes – iterable of images, typically a generator function

• tempo – keyframe rate in beats per minute

streaming.pb_instrument.main(args)

Main script entry point. Reads configuration files, sets up logging, creates the main application objecs, then enters the run loop.

Previewer (PyQt6)

pb_previewer.py

Graphical viewer application for showing a representation of the Pausch Bridge lights. Uses python-osc to receive an image stream, uses PyQt6 to display a bitmap.

class streaming.pb_previewer.AppWindow(*args: Any, **kwargs: Any)

A custom main window which provides all GUI controls. This generally follows a model-view-controller convention in which this window provides the views, passing events to the application controller via callbacks.

class streaming.pb_previewer.MainApp(args, config)

Main application object holding any non-GUI related state.

unknown_message(msgaddr, *args)

Default handler for unrecognized OSC messages.

class streaming.pb_previewer.QtFrameBuffer(width=400, height=400, pixel='rgba')

Object to hold an image which can be used either as a drawing surface with QPainter or updated directly using numpy operations. Storage is shared between the QImage and ndarray.

Parameters:

• width – frame buffer width in pixels, default is 400

• height – frame buffer height in pixels, default is 400

• pixel – string token identifying pixel format (e.g. ‘mono’)

get_default_painter()

Return a QPainter which can draw on the frame buffer. The painter has default image coordinates with +X to the right, +Y down, origin in the upper left, and drawing units in pixels.

get_np_frame_buffer()

Return a reference to the frame buffer as a numpy matrix. This matrix shares storage with the QImage serving as the backing store.

get_qt_frame_buffer()

Return a reference to the frame buffer as a QImage. This object shares storage with a numpy matrix.

class streaming.pb_previewer.QtImageWidget(*args: Any, **kwargs: Any)

Widget to display an image. This keeps a fixed-size frame buffer as a backing store so the display is always able to be repainted, e.g. during resize operations. The buffer can be used as a drawing surface by QPainter or updated directly using numpy operations. The buffer image is scaled to the current widget size when repainting the display, so the pixel data may be higher or lower resolution than the currently visible display.

Parameters:

• width – frame buffer width in pixels, default is 400

• height – frame buffer height in pixels, default is 400

• pixel – string token identifying pixel format, default is ‘bgr’

get_frame_buffer_painter()

Convenience function for drawing access to the frame buffer painter, returns a QPainter object. The end() method of the returned object should never be called.

height()

Return the frame buffer height.

paintEvent(e)

QtWidget callback to draw the content. This paints the frame buffer onto the screen, scaled to fit within the current bounds.

widget_to_image_pos(widget_x, widget_y)

Translate a widget pixel position into the frame buffer pixel coordinates. This may return a position outside the frame buffer image. Returns a tuple (x, y).

width()

Return the frame buffer width.

class streaming.pb_previewer.QtPauschBridgeWidget(*args: Any, **kwargs: Any)

Subclass the frame buffer display widget to draw the Pausch Bridge cartoon and manage user input.