Projects
Here are some projects I have worked on for school, work, or fun.
Exploring alternatives is a key part of interface design, yet processes for creating alternatives are mostly manual. We created Scout, a system to help designers explore alternatives rapidly through mixed-initiative interaction with high-level constraints and design feedback. We formalized design concepts combined with high-level constraints into low-level spatial constraints to enable rapidly generating layout alternatives through constraint solving and synthesis.
Tapping is an immensely important gesture in mobile interfaces, yet people still frequently are required to learn which elements are tappable through trial and error. We created TapShoe, an approach for modeling the tappability of mobile interfaces at scale. We collected a crowdsourced dataset of over 20k tappability labels and built a deep learning model to evaluate tappability automatically which interface designers can use to evaluate this key aspect of usability without needing to collect any data.
Interface designers frequently adapt screenshot examples into designs, yet, images are unstructured and difficult to edit. We created Rewire, an interactive system that helps designers leverage example screenshots by automatically inferring a vector representation of a screenshot where UI components have editble shape and style properties. We demonstrate that Rewire can help designers reconstruct and edit example designs more efficiently compared to a baseline tool.
Most web applications are designed as one-size-fits-all, despite considerable variation in people’s expertise, physical abilities, and other factors that impact interaction. We created Genie, a system that reverse engineers an abstract model of the underlying commands in a web application, then enables interaction with that functionality through alternative interfaces and other input modalities (e.g., speech, keyboard, or command line input).
Interface designers use human performance models to compare designs to legacy systems or competitors’ interfaces, yet these models require manual creation of storyboards in human performance modeling tools (i.e., CogTool). We created a system to automatically generate storyboards and predictive models for desktop interfaces to help UI designers estimate human task performance (CHI 2012), and detect human performance regressions in interfaces (ICSE 2013).
I conducted interviews with web users, and developed an algorithm and a web plugin to detect change in web applications over time and and highlight them to help people see what has changed since they last visited a webpage. This was done as part of a class project for INSC 570 - Research Design.
End users with little software background develop software like spreadsheets, web mashups, and macros. We investigated whether versioning capabilities could help end users develop, understand, and reuse web mashups. Our results showed that versioning allowed mashup creators to perform mashup creation tasks more correctly in less time than users without versioning, while also improving reusability.
We created a reformulation strategy for solving multi-dimensional constraint satisfaction problems (CSPs), through the use of the game of SET as an illustration. We also built a basic constraint solver to find all solutions of an instance of the game of SET, and deployed it in a web interface for playing SET that provided automatic hints.