Research at Design Tech

Research in 2009

The best is yet to come!

Research in 2008

Skimmer: Agent-based Visualization of Streaming Text

With student Riley Benson, PI Watson furthered DesignTech’s previous Feedviz project to produce Skimmer, an agent-based visualization of streaming text. Each agent represents a significant word, and visualizes it by displaying the word itself, centered in a circle sized by the frequency of word occurrence. Agents move to minimize the difference between displayed agent-to-agent distances, and an input matrix of ideal distances. The ideal distance matrix is derived from word co-occurrence, mapping higher co-occurrence to lower distance. To depict co-occurrence in its textual context, the ratio of intersection to circle area approximates the ratio of word co-occurrence to frequency. A networked backend process gathers articles from news feeds, blogs, Digg or Twitter, exploiting online search APIs to focus on user-chosen topics. Resulting visuals reveal the primary topics in text streams as clusters, which change dynamically as stream content changes.

Skimmer was recently presented as a poster and interactive demonstration at IEEE Information Visualization ’08 in Columbus, Ohio. Skimmer is being developed further for full conference publication.

Meaningful Evaluation of Adaptive Computer Graphics Rendering

Computer graphics rendering creates the imagery widely used in computer games such as Halo and films such as Finding Nemo. These images can be quite expensive to compute, especially when they must be computed in real time for computer games.

Adaptive rendering seeks to expend this computational effort only where it is truly needed. There are a number of adaptive techniques today, but deciding which is best is still difficult. Our project seeks both to develop new adaptive renderers, and new techniques for evaluating adaptive renderers.

With student Mathew Schwartz and collaborators at the University of Utah and NVIDIA Research, PI Watson is creating several short computer animations they will use for adaptive rendering comparison. A "gold standard" animation will form the ideal case, the bar against which they will compare various adaptive renderers. They are also generating several other animations using existing adaptive renders. They will use their new evaluation methods to compare these adaptively rendered animations to an animation created with their new adaptive renderer.

The image above shows four renderings of the same animation with varying quality. Upper right, the lowest quality, one sample for every four pixels, created in 6.5 minutes. Upper left, one sample for every single pixel, created in 14 minutes. Lower left, the highest quality, sixteen samples for every single pixel, created in 170 minutes. Lower right, an adaptive rendering that varies quality throughout the image to maximize quality, while holding render time down to 10 minutes.

All of these animations depict the same scene, in which a car races through an alley. This scene includes dynamic and reflective elements that stress the various adaptive rendering techniques they examine.

Results should enable them to perform more meaningful rendering comparisons and create more effective adaptive renderings, ultimately resulting in better and more convincing imagery in both film and games.

Totem Pole: Visualization of Friends on Facebook

Undergraduates Britney Joyner and Wray Bowling built a Facebook application with PI Healey called Totem Pole that visualizes a user’s social friend network in a manner that is as accessible as possible to everyday Facebook users. Figure 3 shows some of the results, visualizing friend profile text, gender and years until graduation. Building Totem Pole involved working with the design programming language Processing (based on Java), MySQL, PHP, Photoshop, and web 2.0.

Multimedia Content Delivery on Mobile Devices

Undergraduate Luke Downey worked with PI Young to demonstrate the utility of mobile devices as platforms for the delivery and control of interactively generated multimedia content. First, using the newly available iPhone SDK, he developed a client application that is location-aware and persistently connected to a content delivery server.

Through the Unreal engine, the server renders a map of the NCSU campus and exports video, which is then streamed to the client for playback. The system developed for this research could be used for a wide range of applications; for example, an interactive tour guide in a museum. Second, he explored the possibilities of using the iPhone as a wireless multi-axis control device. The iPhone's motion sensors and wireless connectivity make it conceivable that dynamically generated 3D environments might be controlled through movements and multi-touch gestures of the iPhone.

Non-Linear Narrative Models in Virtual Environments

Student Adam Tramposh worked with PI Young to realize systems for studying the effectiveness of non-linear, immersive virtual narratives for learning. Unlike conventional linear stories, non-linear narratives offer many alternative paths through stories, and can present underlying “contingent” tasks that supplement the more evident tasks pertaining to a particular event. Contingent tasks afford learners the opportunity to re-evaluate their initial assumptions, stimulating independent reasoning and problem-solving abilities.

Adam diagrammed non-linear narrative and associated tasks, and designed interfaces for narrative virtual environments. The diagram is shown in the image above and was based on a network intrusion scenario. At the top, a timeline of narrative events. Below that, a flow chart of tasks to accomplish in response. Below, the pre- and post-conditions of actions, and the tree of decisions that the player may make.

GoMap 2.0

Undergraduates Josh Andrassy, Lauren Smith and Harold Mallette worked with PI Fitzgerald on two projects. GoMap 2.0 is a system that displays interactively generated maps onto a table. These maps may be both schematic and immersive. The maps may be manipulated by hand gestures sensed by an infrared camera. Sensing is multi-touch, allowing multiple people to interact with the map simultaneously. GoMap 2.0 is intended for communal use in museums or research labs, and is built using Adobe Flash 3.0.

The above images show a top-down schematic view of the NC State campus and a first person view of a particular building, the College of Design’s Leazer Hall.

FlipIx

FlipIx is a new media format and iPhone application prototype, allowing iPhone users to combine images and text in a rapidly changing frame-by-frame format inspired by RSVP (rapid serial visual presentation) systems designed by reading researchers. FlipIx includes components and tools for both content creation and display. The image above show how users can select images while using FlipIx

Research in 2007

FeedViz -- Visualizing Streaming Data

With undergraduates Phil Lafleur, Riley Benson and Daniel Kuchta, Watson and Healey continued experimenting with solutions for visualization of streaming, constantly changing information.The Feedzor prototype from the previous summer made it clear that such prototypes were not especially difficult to build, but how could they become more effective? Work in the past year focused on StreamViz, which improved on Feedzor in a number of ways:

Deeper textual analysis: FeedViz performs a keyword analysis of online content to extract the salient concepts being discussed, and how they are related. Feedzor simply grabbed headlines and put them on the display as is, without any summarization at all.

More meaningful visualization: where Feedzor chose position, color and direction of motion randomly, StreamViz derives these visual properties from stream content.

Public display: Feedzor was a lab-only desktop prototype, while FeedViz is already visualizing online news and blog content constantly on a display in the main Computer Science hallway.

Innovative camera-based interface: users could only interact with Feedzor through a desktop PC mouse, while FeedViz is matched with a camera- and voice-tracked interface that permits passersby in the hallway to rearrange the visualization, request additional detail on certain topics, and change the topic being visualized.

Quilts -- A New Depiction For Large Layered Graphics

With undergraduate Matt Rakow, Renaissance Computing Institute director Theresa-Marie Rhyne and the SAS Institute, Watson and CSC faculty member Matthias Stallman developed quilts, a new depiction for large layered graphs. These graphs appear in flow charts, structure charts, genealogy diagrams and a wide range of business applications, including activity-based management.

Matrices have always been recognized as an alternative to node-link depictions of graphs. Quilts concatenate several such matrices together, one for each adjacent pair of layers in the graph. Nodes in layers appear as cells in levels between matrices, while links between layers become dots in the matrices. Links that skip layers cannot be represented within a matrix, and therefore appear as special skip links that indicate their destination link using color. Just as matrices are easier to understand than node-link depictions when graphs are large, we believe quilts are easier to understand than node-link depictions when layered graphs are large. We have also developed quilts further, describing how summarize them, an alternative representation that eliminates the path discontinuities associated with skip links, and a method for integrating them with OLAP hierarchies.

We published our work on quilts as a poster at the IEEE Information Visualization 2007 conference, and plan to submit a full paper on quilts to the 2008 event. The SAS Institute has expressed strong interest in quilts, and is currently negotiating with NC State for licensing rights to the depiction.

Vision-Based Interfaces and Location-Aware Communication

With undergraduates Josh Andrassy, Michael Laut, and Lauren Smith, Fitzgerald of Design worked on the Interactive Window Project, which developed a vision-based user interface for large, back- projected displays. To track user hand movements, the system uses a single infrared (IR) web camera mounted over the display and pointed down, with matching IR illumination. Users can point, select and push displayed media. The system communicates with Flash, the standard for animation and video online and in the design community.

The interactive window is currently being used with GoMap 2.0, a large-scale interactive map of the NC State campus featuring live RSS and Flickr data streams, as well as 3D flythroughs. The system links a public kiosk on campus to mobile phones, and supports informational and social networking by students in the local spatial context.

Serious Game Interfaces For Education

With students Smith Newnam and Joseph France, Young worked on the HI-FIVES project, an NSF-funded project that is creating Virtuoso, an easy-to-use tool for middle and high school teachers and students to build 3D interactive computer games. Teachers and students build games based on science and math curricula covered in their classrooms, with the goal of engaging students more directly with the content of their courses. Student work includes coded implementation of Virtuoso functionality, development of its 3D and 2D art content, and design of its user interface.

Experiments in Natural Language Generation For Games

With student David Tredwell, Young worked on the project examining automated generation of natural language warnings and explanations needed to guide users following task instructions in a game environment. Tredwell was the sole developer of the complex 3D model used in human subject experiment, and integrated the 3D model into the experimental gaming environment.

Research in 2006

Feedzor -- Visualizing Streaming Data

Feedzor visualizes RSS feeds as moving headlines, with the goal of enabling peripheral monitoring (secondary to a primary task) of a large number of information streams. Design students Horne and Sams prototyped several alternatives, including headlines that arrive as gumballs falling out of a vending machine, leaves that fall from a tree, and as drops in a pond. Computer Science Crenshaw implemented a working prototype with a simple visualization: linearly moving headlines. The headlines either move across the screen, or scroll up it. Clicking on any headline reveals more feed detail. We created a specialized version of Feedzor that uses Google News feeds to enable users to monitor net-wide feed content on any search topic they enter. We plan to experiment with several more visual front-ends for this prototype.

3D Mall -- Visual Streaming Data

3DMall is an experiment in adding 3D spatiality to the online shopping experience. Here students Mohr and Levesque use RSS feeds provided by eBay to find information about merchandise for sale. They display these in windows much like a shopping mall. Users can navigate between product categories by taking elevators between hallways, and examine a product category on a single floor. By clicking on a product, users can learn more about it.

3DMall aggregates shopping RSS feeds and places them in a 3D environment, providing something of the old-school shopping experience. CDs are available at left, cars at right.

Serious Game Interfaces

Four students began working with Young in the fall of 2006 to create tools that enable novices to create games.

The HI-FIVES project involves two design students and one computer science student. This NSF-funded project creates an easy-to-use tool for middle and high school teachers and students to build 3D interactive computer games. Teachers and students use the HI-FIVES software to build games based on science and math curricula covered in their classrooms, with the goal of engaging students more directly with the content of their courses. Design students Bennett and Newnam are working to design and to build the user interface for the HI-FIVES software, as well as to manage key aspects of the design of the user experience central to the program's effectiveness. Computer Science undergraduate Computer Science undergraduate Ricky Patel is creating an expression builder, which enables the specification of the behaviors of characters and other objects within the games being built by HI-FIVES users.

The Longboard project involves Computer Science undergraduate Macic in a Microsoft research effort based on technologies produced by an on-going NSF CAREER award. Longboard enables automatic creation of cinematic videos based on sketches of storyboards drawn by people using tablet PCs. Macik is developing all aspects of the tablet-side user interface, including the management of story elements, cast of characters, setting specifications, screenplay editing and storyboard management.

Location-based information and communication

Fitzgerald hired Design students Andrassy and Laut and Computer Science student Drake to work on a large-scale interactive map featuring live data RSS and Flickr data streams, as well as 3D fly-throughs of NC State’s campus. The system will eventually be set up in a public location on campus and be used by students as a new form of informational and social networking.