chARm

Project title: chARm:

Jenna Martinez, Paris Kampel

Advisor: Keith O’Hara

Most mixed-reality exhibits rely on the use of multiple stationary projectors to cover many different surfaces. Our goal was to provide an expansive mixed-reality interface with one mobile projector-camera system rather than using many projectors. We attempted to address this question with the development of chARm, an augmented-reality aviary that projects keystone-corrected, user-created 2D birds onto every wall within a room. Our mobile projector-camera system consists of a stationary projector pointed upwards at a mobile mirror that can pan and tilt. 

These pan and tilt signals are sent to the mobile mirror via a dmx connection that is constantly updated in our software. As the mirror moves, so does the projection, allowing us to create the illusion of a flock of birds flying throughout the room. While our hardware handles the physical movement of the projection, our software decides how and when our projection actually moves. We use a combination of python libraries to create a streamlined process of determining bird flight behavior. To determine where the birds are allowed to fly within a room, we calibrate many “perch points” by using the camera to inform our program the spatiality of these points within the room. These locations are then translated into pan and tilt values for the mirror and into homography matrices that we then use to geometrically correct our flock. This information then gets saved into a graph, where the nodes represent “perch points” and the edges represent travelable paths, which are calculated by a shortest path algorithm. 

As the birds move between nodes our program interpolates between node values to create smooth transitions between calibration points. Furthermore, our graph is used to stop the flock from traveling to certain nodes. For example, if our camera detects that a person is blocking a path, that path then becomes untravelable for our flock as to prevent our projection beam from interfering with observers and to mimic real bird behavior. Viewers of our exhibit are not only encouraged to observe the flock, but are also encouraged to contribute their own customized bird designs by using a touch-screen interface. When a user creates a bird, it is automatically added to the visible flock, and is saved in our bird database. The birds in our database are regularly cycled into the visible flock so that viewers can return to the exhibit and possibly see their creation again. Please visit chARm in Martin Hall to join our flock!