Finlandsgade 10 8200 Århus N, Denmark
http://www.cetrea.com thomasr@cetrea.com
ABSTRACT
Interacting with large touch displays requires different interaction techniques compared to desktop systems. In this paper, the problems and challenges around using drag-and-drop on touch displays are introduced. To accommodate these problems the ClickDrop interaction technique is introduced. The ClickDrop technique introduces a special mode in which objects can easily be picked up and dropped using only two touches. The paper briefly discusses how the technique is being used in two commercial applications and some of the strengths and challenges with the proposed interaction technique.
Keywords
Touch displays, interaction techniques, ClickDrop, large displays, drag and drop, multiple displays.
INTRODUCTION
One of the fundamental interaction techniques behind direct manipulation is drag and drop. Drag and drop is used to combining elements or to combine a command with an element. Drag and drop is typically carried out with the use of a mouse. However, on large touch displays drag-and-drop is less suited for interaction purposes.
First, drag-and-drop on a touch display requires the user to hold down the finger during the entire drag. Missing the contact with the surface for just a short while will either cancel the drag or result in the element being dragged to the wrong location. Losing touch with the surface can both be due to the user losing touch with the surface or inaccuracy in the tracking hardware. In both cases the interaction is either canceled or error prone.
Second, dragging on large touch displays results in having to move the finger over longer distances while keeping touch with the surface. Depending on the surface of the touch screen, the friction between the surface and the finger will eventually result in skin burns and smaller abrasions.
Third, dragging on a large display will often take seconds making it a relatively slow way of interacting with the display compared to other interaction techniques. The challenge is further strengthen if the application spans multiple displays.
Combined, these drawbacks make drag-and-drop a less
obvious choice for applications that uses touch displays.
However, touch applications still need a way to quickly combining elements or pair a command with an object.
In this paper we present a technique called ClickDrop1 for combining elements on large touch displays. The technique has been used in several of Cetrea’s products and based on the initial experiences with the technique the technique is briefly discussed.
RELATED WORK
Pick and drop is an interaction technique introduced by Rekimoto and it is used to move item between devices by picking up an item on one device and dropping it on another [5]. The technique uses a pen to pick up the objects from either a display or a PDA. A similar technique for moving objects between displays is take-and-put where digital objects can be associated with physical objects and move to other displays [4]. Both techniques are directed at distributed display environment where objects need to be moved between displays and devices, but they do not solve the problem about how to use drag-and-drop on large touch displays.
Another type of interaction techniques are directed large displays and deal with different ways of overcoming distances. One approach is to use a throwing metaphor for allowing the user to select distant targets by providing a mini-map of the large display and using this mini map to direct the throw [3]. Another technique called drag-and-pop draws distant target closer to the icon you are currently dragging [1]. Finally, Collomb et al combines the two techniques in a technique called push-and-pop and they show how all three techniques are faster than drag-and-drop on large wall displays [2].
The ClickDrop interaction technique extends the above presented techniques by allowing a fast way to move objects while keeping the element on the display in place.
THE CLICK-DROP INTERACTION TECHNIQUE
The idea behind ClickDrop is to introduce a special mode called ClickDrop mode. While the display is in ClickDrop mode the application switches between two states. In
1 ClickDrop® is a registered trademark for Cetrea A/S.
picking up state the application waits for the user to select an object. Selecting an object will switch the display into dropping state. In dropping state the current selected object and all objects that are able to receive the current object is highlight. Clicking on the selected object will drop the object and return the application to the picking up state.
Clicking on a highlighted target will issue the associated command and move the display back into the picking up state. Figure 1 shows how objects can be selected in the picking up state and how the selected object and the potential targets are highlighted.
Figure 1. ClickDrop states
The technique is pretty simple, but highly effective. It allows objects to be combined by clicking just two times on the display (One for picking an object up and one for dropping an object).
Currently, the technique is, similar to drag-and-drop, used for three different type of interaction. The first is moving an object from one location to another (moving). The second use it to combine one type of object with another (combining). Finally, the third usage is to clone an object (cloning). At this time, the interaction context determines which action the ClickDrop interaction accommodates.
USING THE CLICK-DROP TECHNIQUE
The ClickDrop technique is implemented in two commercial products: Cetrea Surgical and Cetrea Emergency. Cetrea Surgical is a system for supporting real-time coordination in the operating department at hospitals and Cetrea Emergency supports real-time coordination in the emergency department.
Figure 2. The coordinating central in an operation ward using Cetrea Surgical
In Cetrea Surgical the ClickDrop technique is used to move surgeries between operating room and to move surgeries in time in the calendar (moving), to associate personnel with a surgery (combining) and associate personnel with an operation room (cloning). The ClickDrop interaction technique is implemented in an updated version of the product and replaced a system where menus were used for providing the same functionality. Without having performed detailed performance studies of the two techniques the ClickDrop technique seems to be substantially faster, especially when objects are moved on the same display. Figure 2 shows a picture of Cetrea Surgical running at a surgical ward.
Figure 3. A coordinating room in the Emergency Department running Cetrea Emergency
The technique is also used in Cetrea Emergency for moving patients between beds (moving), associating nurses and doctors with the different treatment tasks around the patient (combining) and for selecting the role of different key personnel (cloning). Also in this application the technique performs substantially faster than the menus previously applied. However, a detailed performance study is future work. Figure 3 shows Cetrea Emergency running on three large displays in the emergency department.
Both applications are currently used by a number of hospitals. In some of the installation the application run on up to four large displays.
To facilitate ClickDrop interaction between multiple displays a new concept called a SwapBoard is introduced.
The SwapBoard is shared between all four displays and the SwapBoard is able to hold a reference to all the different objects being picked up. Hence, to ClickDrop an object between displays the user will first have to click on an object and dropping it on the SwapBoard. As soon as an object is dropped on the SwapBoard the object becomes visible on the other displays SwapBoard. The user can following on another display pick the object up from the SwapBoard and drop it at the correct target. While it takes four clicks to move an object between displays, using the SwapBoard has the advantage of allowing other users to use the other displays while the ClickDrop technique is used on one of the displays.
DISCUSSION
The ClickDrop Technique has proven to be an intuitive and fast way of moving objects, but is not without its drawbacks.
First, while the technique is really fast for moving objects on one display, it is slower when the user has to move objects between several large displays. This is partly due to the extra clicks associated with the SwapBoard and partly due to the user having to move physically to another display to drop the object. This might be overcome by using some of the techniques for throwing or dragging distant target closer (see related work). However, these techniques still have drawbacks when being used on many large displays and are more complex to use.
Second, the technique relies on the interface being in a ClickDrop mode. The problem with introducing modes is that the user manually has to switch into the ClickDrop mode and switch back again after the interaction has been completed. This is especially a drawback if objects often are moved one at a time. However, if several objects are moved shortly after each other, the time used on switching modes is reduced.
Despite these challenges the technique in the current phase has proven to be really intuitive and fast to use. However, future work will look at how the technique can be used without having to enter a specific mode.
CONCLUSION
Using large touch displays requires different interaction techniques compared to desktop computers using keyboard and mouse. One challenge is to provide similar functionality to drag-and-drop without having to deal with the problems of dragging with the finger on a touch displays.
In this paper the ClickDrop technique is introduced and the initial experiences with using the technique points to the
technique providing an intuitive and fast way of interacting with touch displays. We are currently working on improving the technique and quantifying some of the performance issues gained by using this technique.
ACKNOWLEDGEMENT
I would like to thank the talented people at Cetrea who have worked on developing and refining the presented technique and bringing it into commercial products.
REFERENCES
[1] Baudisch, P., Cutrell, E., Robbins, D., Czerwinski, M., Tandler, P. Bederson, B., and Zierlinger, A. Drag-and-Pop and Drag-and-Pick: Techniques for Accessing Remote Screen Content on Touch and Pen-operated Systems. In Proc Interact'03, pp. 57—64.
[2] Collomb, M., Hascoët, M., Baudisch, P., and Lee, B.
2005. Improving drag-and-drop on wall-size displays.
In Proceedings of Graphics interface 2005 (Victoria, British Columbia, May 09 - 11, 2005). GI, vol. 112.
Canadian Human-Computer Communications Society, School of Computer Science, University of Waterloo, Waterloo, Ontario, 25-32.
[3] Hascoët, M. (2003). Throwing models for large displays. In Proc. HCI'03, pp. 73–77.
[4] Prante, T., Streitz, N., and Tandler, P. 2004.
Roomware: Computers Disappear and Interaction Evolves. Computer 37, 12 (Dec. 2004), 47-54. DOI=
http://dx.doi.org/10.1109/MC.2004.255
[5] Rekimoto, J. 1997. Pick-and-drop: a direct manipulation technique for multiple computer environments. In Proceedings of the 10th Annual ACM Symposium on User interface Software and Technology (Banff, Alberta, Canada, October 14 - 17, 1997). UIST '97. ACM, New York, NY, 31-39. DOI=
http://doi.acm.org/10.1145/263407.263505