Sensing Activity in Video Images
Alison Lee
NYNEX Science & Technology
White Plains, NY 10604, USA
+1 914 644-2322
alee@nynexst.com
Kevin Schlueter
University of Toronto
Toronto, ON M5S 1A4, Canada
+1 416 978-0835
kevins@dgp.toronto.edu
Andreas Girgensohn
FX Palo Alto Laboratory
Palo Alto, CA 94304, USA
+1 415 813-7244
andreasg@pal.xerox.com
ABSTRACT
Video-based awareness tools increase familiarity among remote group
members and provide pre-communication information. Low-cost iconic
indicators provide less but more succinct information than video
images while preserving privacy. Observations of and feedback from
users of our video awareness tool suggest that an activity sensing
feature along with a variety of privacy options combines advantages of
both the video images and iconic indicator approaches. We introduced
the activity sensing feature in response to user requests. It derives
activity information from video images and provides options to control
privacy and improves the usability of video-based awareness tools.
Keywords
Activity sensing, awareness, differences, privacy, usability.
INTRODUCTION
A classic problem in CSCW is supporting group and collaboration
awareness in distributed work groups. Awareness is thought to increase
familiarity among co-workers as well as provide important
pre-communication information on presence, interruptibility and
availability. This problem was first solved by allowing group members
to see video images of each other. Xerox Portholes [2], Telepresence
Postcards, and NYNEX Portholes use periodically refreshed video
snapshots while Montage [5] uses brief reciprocal glances.
Recently, a number of techniques have been proposed which use low-cost
iconic indicators as an alternative to video images [1, 3, 6]. These
indicators sacrifice a great deal of the information available in
video awareness tools but provide more succinct information about
presence, and availability while, in some cases, ensuring more
privacy.
Observations of and feedback from users of our video awareness tool
suggest that it is unnecessary to sacrifice the rich information
provided by video images in order to achieve the advantages of the
indicator approach. To support our point, we describe a capability in
NYNEX Portholes that provides activity sensing and increased user
control over privacy. We developed this capability in response to our
user requests. We believe that this capability provides an improvement
over both the traditional video-based awareness tools and iconic
indicators because it meets three crucial user requirements. This
paper describes the activity sensing feature, the three user
requirements, and how the activity sensing feature meets these
requirements.
Figure 1: Portholes Viewer
 |
WHAT: ACTIVITY SENSING
In our Portholes system, a video image for each user is snapped once
every five minutes. The images for the last hour (12 of them) are
kept. The activity sensing capability uses a image difference
mechanism to derive a measure for the amount of change (1 <=
change <= 100) between two successive video images. This
measure is represented as a red horizontal bar at the bottom of the
video image and can optionally be used to order the images in the
Viewer (Figure 1). The red bars have the added benefit of making
images with lots of activity "jump out" so that users can easily do a
visual scan for activity. The "sort by amount of activity" feature
displays images with lots of activity earlier in the
matrix. Therefore, users can use a smaller Portholes window and still
see all colleagues who are in their offices.
WHY: THREE USER REQUIREMENTS
Video images are used in our awareness tool because the visual channel
provides many of the pieces of information that people find valuable
for awareness. Experiences with Xerox Portholes [2] indicate that
sightings, presence and availability are some information that users
remember and seek out. In addition, video provides as much information
as possible so that we could get maximal feedback about what is useful
and what needs to change. After more than two years of use at three
NYNEX sites and three distant universities, we have accumulated many
user requirements. We present the three that relate to the activity
sensing feature.
Increased Activity Awareness
Figure 2: Activity Bar Chart
 |
Our users require an indication or a notification of the activity or
changes in a co-worker's work area. However, different users seek
different kinds of activity information (e.g., presence, absence,
availability, interruptibility). Displaying a matrix of the current
video images of one's co-workers is not always sufficient (e.g., to
determine whether a co-worker has just left, the user has to examine
the co-worker's last 12 images). Thus, to make activity or changes
accessible, we developed the activity sensing feature illustrated
in Figure 2.
User Controls on Privacy and Information Disclosure
Although our users are willing to disclose awareness information to
their co-workers, the rich information in the video images raises
concerns about privacy, the amount of information disclosed and
cameras in the work place. Some users do not want to provide specific
activity information but only activity changes. To address these
concerns, we use activity sensing to provide a range of privacy
options. Users wanting maximal privacy or minimal disclosure can
simply show an activity bar chart which gives a single integrative
view of the changes over the last hour. Users willing to disclose more
but who are uncomfortable with detailed video images can provide
activity bars along with images blurred at one of two levels (e.g.,
"black" image in Figure 3). Some users have mentioned that they are
more comfortable with using the activity sensing feature with a
blurred image because unflattering images are imperceptible.
Figure 3: Privacy options ranging, from left to right,'black', 'gray', 'foggy', 'normal', and 'sharp'.
 |
 |
 |
 |
 |
Usability of Portholes Viewer
The Portholes Viewer (which displays the video images matrix) requires
screen space even when used as a backdrop. However, when a Portholes
community is large (many video images), a usability problem is created
because not all images are visible in the Viewer and some may be
obscured by other windows on the desktop. The activity-based ordering
of images combined with setting a cut-off for images to display
addresses this problem. The combination of features turns the
Portholes Viewer into a visibility filter which preserves valuable
screen real estate and optimizes scanning and peripheral
monitoring. We have observed that some users are further exploiting
this combination by using a reduced size Viewer window that is
positioned at an unobstructed corner of their desktop.
HOW: IMAGE DIFFERENCING
Our image differencing algorithm, which is the foundation for activity
sensing, differs somewhat from those used elsewhere. Telepresence
Postcards, for example, has a Sense option that allows a user to be
alerted when a change in the current image exceeds a set
threshold. While Postcards also uses an image differencing algorithm
to support this option, our algorithm is more complex because it
attempts to ignore changes that are simply due to shifts in
illumination and video digitization noise. To accomplish both, we
gamma correct and blur the image slightly. Next, we compute two
different image difference metrics:
- The percentage of the image pixels that have changed more than a
certain threshold.
- The variance in the change in corresponding pixels.
In our experience, the first metric is very good at detecting changes
in images due to people moving as well as people entering and leaving
the frame. However, it is quite sensitive to changes in image
brightness due to actual illumination changes or the automatic
brightness in some cameras. The second metric is quite insensitive to
changes in image brightness. By taking the smaller of the two values,
we get a measure that is less sensitive to illumination changes.
FUTURE DIRECTIONS AND CONCLUSION
We are presently exploring two approaches to improving the activity
sensing. The first identifies other candidate statistics for the
difference metrics that describe the distributions for the differences
between corresponding pixels. The second, inspired by [4], determines
the presence or absence information using reference video images.
NYNEX Portholes is a vehicle to explore, through use, the value of
video awareness tools for distributed work groups and the rich set of
awareness information available in video images. Our users have told
us that they are willing to provide certain information but they want
to exercise different degrees of control over the amount and form of
the information disclosed. Activity sensing permits this while also
enhancing usability. Our experiences strongly suggest that this is a
worthy addition to video-based awareness tools.
ACKNOWLEDGEMENT
The research was performed while all the authors were at NYNEX.
REFERENCES
- M. Ackerman and B. Starr. Social Activity Indicators: Interface
Components for CSCW Systems. In UIST'95 Conference Proceeding,
ACM Press, pp. 159-168, 1995.
- P. Dourish and S. Bly. Supporting Awareness in a Distributed
Work Group. In CHI'92 Conference Proceeding, ACM Press,
pp. 541-547, 1992.
- S. Greenberg. Peepholes: Low Cost Awareness of One's
Community. In CHI'96 Conference Companion, ACM, pp. 206-207,
1996.
- S.E. Hudson and I. Smith. Techniques for Addressing Fundamental
Privacy and Disruption Tradeoffs in Awareness Support Systems. In
CSCW'96 Proceeding, ACM Press, pp. 248-257, 1996.
- J.C. Tang and M. Rua. Montage: Providing Teleproximity for
Distributed Groups. In CHI'94 Conference Proceedings, ACM
Press, pp. 37-43, 1994.
- T. Wax. Red Light, Green Light. In CSCW'96 Conference Companion,
ACM, pp. 1-2, 1996.