I would like to convey two views on user interface research, one relating to the process, and the other relating to research agendas. Both of these views are personal ones. Finally I will briefly describe some of Microsoft's user interface research interests.
Publishing should not be considered the ultimate goal at either universities or corporate research labs. Although it is important to publish user interface papers to explain how things are done architecturally, it is just as important and often more so, to widely distribute research prototypes. Without trying out a user interface idea, it is very difficult to analyze its true worth. Usability tests do not suffice, because typically they only involve a small, select group of individuals, and the tests tend to be limited in both scope and duration. It is absolutely critical for large numbers of people try to use experimental interfaces or new interface architectures to get their jobs done.
I cannot stress this more strongly: university and corporate researchers must release the systems that they build to the general populace. The Web and Internet are tremendous tools for enabling this. Newsgroups and mailing lists need to be set up for people to comment on the prototypes. After a sufficient time has past, researchers should publish "experience" papers to summarize what worked and what did not. Frankly, I do not think funding sources should approve any research contracts, without first having a statement indicating how the work will be made available to the computing public. No Ph.D. dissertation should be accepted without the fruits of the research being distributed on the Web for all to try.
It should be pointed out that occasionally university (and corporate) research does get broad distribution and testing. Brad Myers' Garnet system and Randy Pausch's SUIT are both excellent examples of this. But right now these are the exception rather than the rule.
Too many user interface researchers pursue research on topics that are primarily of interest to user interface researchers. Instead, more research should focus on helping people get their tasks done. We need to focus less on "cool technology" and more on what people really like to use in practice.
An example of this is much of the 3D virtual reality research that has become trendy recently. Some of this is excellent research, but much of it focuses on esoteric aspects of creating such systems, without really focusing on when these systems are actually better than more conventional interfaces. There seems to be an underlying assumption that people should be using 3D virtual reality systems for many of their tasks, when this is still unproven. Instead, I would much prefer to see research that addresses when virtual reality is worthwhile, and when it can help people get their jobs done more effectively.
Similarly, a large number of user interface research projects study constraints. Constraints research has been under way for over 30 years, and hardly any commercial user interfaces rely on constraint technology. In spite of that, papers continue to be published on abstruse technical details that will not result in constraints being used more broadly. Personally, I believe that constraints still show plenty of promise. It is a great technology, but people need to focus on what can be done to make constraint technology practical. There is an incredible amount of valuable research that can be done on this.
Microsoft is particularly interested in UI research that makes products easier to use, and more compelling. Obviously Microsoft wants more people to buy software, and there is the feeling here that our research should make computers appeal more to the average person. People here are exploring several approaches to this.
3D graphics accelerators are becoming extremely cheap for the PC platform. Some researchers here believe that now is the time to explore how interfaces can be improved through the use of 3D, because new PCs will enable far more people to experience interactive 3D graphics than ever before. One such project is "Peedy the Parrot", a 3D agent-based interface for controlling a CD jukebox. Our experience was that people found the interface fun and amusing, but only if they knew precisely what to say. Peedy's speech and language understanding capabilities were not sufficiently flexible to make the interface easier to use than a traditional graphical user interface. However, the 3D component is very robust and can be applied to other interface animation problems. Researchers at Microsoft are interested in other ways to use 3D in the interface, including extending the desktop metaphor to 3D, and 3D visualization.
A group within Microsoft Research is exploring novel interfaces for social computing. This includes Web-based interfaces and virtual communities. One of our current projects is to develop a graphical chat system based on a comic strip representation. The program, called Comic Chat, automatically generates a comic strip to represent a chat session. It determines which characters to put in a panel, where to place them, what gestures and expressions to give each character, how the word balloons should look, where word balloons should be placed, and when to move on to a new panel. There are a number of graphical chat systems available, but Comic Chat is the first such system with a graphical history and a dynamically varying visual presentation. Currently, Comic Chat is freely available on the Internet for beta testing, and thousands of people have already downloaded it. Among other things, we are interested in learning what people like and don't like about 2D virtual spaces, such as the one presented in Comic Chat, as compared to 3D virtual environments.
Audio for non-interactive linear media is created using an iterative post-production process. Our goal for audio in interactive media is to achieve a level of quality comparable to that of non-interactive media, but we are challenged by having to do much of the compositing, mixing, and even creation at run-time, on available hardware. We are trying to meet those challenges by creating run-time environments that can embed some of the experience of audio engineers. Also, we are investigating new ways of making audio an indispensable component of human computer interfaces.
Another project under way here is studying how to make interfaces predictive, so that less interaction is required. One group here is applying decision theory to create more intelligent user interfaces. Two other groups are building speech recognition and language processing technology that might be used in future interfaces.