System and method that facilitates computer desktop use via scaling of displayed objects with shifts to the periphery Number:7,386,801 from the United States Patent and Trademark Office (PTO) owispatent The present invention relates to a system that facilitates multi-tasking in a computing environment. A focus area component defines a focus area within a display space--the focus area occupying a subset area of the display space area. A scaling component scales display objects as a function of proximity to the focus area, and a behavior modification component modifies respective behavior of the display objects as a function their location of the display space. Thus, and more particularly the subject invention provides for interaction technique(s) and user interface(s) in connection with managing display objects on a display surface. One aspect of the invention defines a central focus area where the display objects are displayed and behave as usual, and a periphery outside the focus area where the display objects are reduced in size based on their location, getting smaller as they near an edge of the display surface so that many more objects can remain visible. In addition or alternatively, the objects can fade as they move toward an edge, fading increasing as a function of distance from the focus area and/or use of the object and/or priority of the object. Objects in the periphery can also be modified to have different interaction behavior (e.g., lower refresh rate, fading, reconfigured to display sub-objects based on relevance and/or visibility, static, etc.) as they may be too small for standard rendering. The methods can provide a flexible, scalable surface when coupled with automated policies for moving objects into the periphery, in response to the introduction of new objects or the resizing of pre-existing objects by a user or autonomous process.<H facilitates, periphery, System, and, meth, 7386801.html, Patent, pto, 7386801

Title: System and method that facilitates computer desktop use via scaling of displayed objects with shifts to the periphery

Abstract: The present invention relates to a system that facilitates multi-tasking in a computing environment. A focus area component defines a focus area within a display space--the focus area occupying a subset area of the display space area. A scaling component scales display objects as a function of proximity to the focus area, and a behavior modification component modifies respective behavior of the display objects as a function their location of the display space. Thus, and more particularly the subject invention provides for interaction technique(s) and user interface(s) in connection with managing display objects on a display surface. One aspect of the invention defines a central focus area where the display objects are displayed and behave as usual, and a periphery outside the focus area where the display objects are reduced in size based on their location, getting smaller as they near an edge of the display surface so that many more objects can remain visible. In addition or alternatively, the objects can fade as they move toward an edge, fading increasing as a function of distance from the focus area and/or use of the object and/or priority of the object. Objects in the periphery can also be modified to have different interaction behavior (e.g., lower refresh rate, fading, reconfigured to display sub-objects based on relevance and/or visibility, static, etc.) as they may be too small for standard rendering. The methods can provide a flexible, scalable surface when coupled with automated policies for moving objects into the periphery, in response to the introduction of new objects or the resizing of pre-existing objects by a user or autonomous process.

Patent Number: 7,386,801 Issued on 06/10/2008 to Horvitz, et al.

Inventors: Horvitz; Eric J. (Kirkland, WA), Robertson; George G. (Seattle, WA), Czerwinski; Mary P. (Woodinville, WA), Meyers; Brian R. (Issaquah, WA), Smith; Gregory R. (Bellevue, WA)
Assignee: Microsoft Corporation (Redmond, WA)

Appl. No.: 10/851,928

Filed: May 21, 2004

Related U.S. Patent Documents


Application Number	Filing Date	Patent Number	Issue Date
10374351	Feb., 2003

Current U.S. Class: 715/764 ; 715/765; 715/767; 715/778; 715/805; 715/815

Field of Search: 715/763-765,769,778,779,781,788,805,815

References Cited [Referenced By]

U.S. Patent Documents


5305435	April 1994	Bronson
5381523	January 1995	Hayashi
5423043	June 1995	Fitzpatrick et al.
5493692	February 1996	Theimer et al.
5495567	February 1996	Iizawa et al.
5513342	April 1996	Leong et al.
5544321	August 1996	Theimer et al.
5555376	September 1996	Theimer et al.
5603054	February 1997	Theimer et al.
5611050	March 1997	Theimer et al.
5652850	July 1997	Hollander
5799292	August 1998	Hekmatpour
5812128	September 1998	Sterling, IV
5812865	September 1998	Theimer et al.
5838318	November 1998	Porter et al.
5838326	November 1998	Card et al.
5844559	December 1998	Guha
5874962	February 1999	De Judicibus et al.
5880733	March 1999	Horvitz et al.
5920315	July 1999	Santos et al.
6016145	January 2000	Horvitz et al.
6025841	February 2000	Finkelstein et al.
6043817	March 2000	Bolnick et al.
6104359	August 2000	Endres et al.
6134566	October 2000	Berman et al.
6160553	December 2000	Robertson et al.
6188405	February 2001	Czerwinski et al.
6256648	July 2001	Hill et al.
6263352	July 2001	Cohen
6266064	July 2001	Snyder
6286043	September 2001	Cuomo et al.
6369827	April 2002	Pan et al.
6369837	April 2002	Schirmer
6374200	April 2002	Nakagawa
6411292	June 2002	Cook et al.
6414677	July 2002	Robertson et al.
6462759	October 2002	Kurtzberg et al.
6466232	October 2002	Newell et al.
6466237	October 2002	Miyao et al.
6513046	January 2003	Abbott, III et al.
6549218	April 2003	Gershony et al.
6549915	April 2003	Abbott, III et al.
6570590	May 2003	Dubrow et al.
6590593	July 2003	Robertson et al.
6629097	September 2003	Keith
6684177	January 2004	Mishra et al.
6747675	June 2004	Abbott et al.
6791580	September 2004	Abbott et al.
6801223	October 2004	Abbott et al.
6812937	November 2004	Abbott et al.
6826727	November 2004	Mohr et al.
6842877	January 2005	Robarts et al.
6868441	March 2005	Greene et al.
6909443	June 2005	Robertson et al.
6959425	October 2005	Krauklis
6964025	November 2005	Angiulo et al.
6999955	February 2006	Horvitz
7015910	March 2006	Card et al.
7051291	May 2006	Sciammarella et al.
7111231	September 2006	Huck et al.
2001/0040590	November 2001	Abbott et al.
2001/0040591	November 2001	Abbott et al.
2001/0043231	November 2001	Abbott et al.
2001/0043232	November 2001	Abbott et al.
2002/0032689	March 2002	Abbott, III et al.
2002/0033779	March 2002	Nakai et al.
2002/0044152	April 2002	Abbott, III et al.
2002/0052930	May 2002	Abbott et al.
2002/0052963	May 2002	Abbott et al.
2002/0054130	May 2002	Abbott, III et al.
2002/0054174	May 2002	Abbott et al.
2002/0077156	June 2002	Smethers
2002/0078204	June 2002	Newell et al.
2002/0080155	June 2002	Abbott et al.
2002/0080156	June 2002	Abbott et al.
2002/0083025	June 2002	Robarts et al.
2002/0083158	June 2002	Abbott et al.
2002/0087525	July 2002	Abbott et al.
2002/0099817	July 2002	Abbott et al.
2002/0101418	August 2002	Frederic et al.
2002/0184233	December 2002	Schneider
2003/0046401	March 2003	Abbott et al.
2003/0069892	April 2003	Hind et al.
2003/0154476	August 2003	Abbott, III et al.
2003/0156124	August 2003	Good et al.
2003/0179236	September 2003	Good et al.
2003/0220895	November 2003	Vailaya
2003/0225853	December 2003	Wang et al.
2004/0109006	June 2004	Matthews
2004/0123238	June 2004	Hefetz et al.
2004/0143590	July 2004	Wong et al.
2004/0148573	July 2004	Buice et al.
2004/0225986	November 2004	Lin et al.
2005/0034066	February 2005	Ahlert et al.
2005/0034078	February 2005	Abbott et al.
2005/0085239	April 2005	Swann et al.

Foreign Patent Documents


1253510	Oct., 2002	EP
9800787	Jan., 1998	WO

Other References

Mario Ruggier, Guide for using the Software Documentation Layout Template, Jul. 5, 1999. cited by examiner .
Tim Pattison, et al., "Information Visualization Using Component Layouts and Visual Sets," Copyright 2001, Conference in Research and Practice in Information Technology, vol. 9, Commonwealth of Australia. cited by other .
Jonathan Grudin, "Partitioning Digital Worlds: Focal and Peripheral Awareness in Multiple Monitor Use," vol. 3, No. 3, Issue No. 1, ACM 2001. cited by other .
K.S. Booth, et al., "On the Parameters of Human Visual Performance an Investigation of the Benefits of Antialiasing", Computer Graphics Laboratory, University of Waterloo, Waterloo, Ontario, Canada, ACM 2001. cited by other .
European Search Report for European Application No. EP04004253, dated Oct. 26, 2006. 3 pages. cited by other .
XDesk Software, About Virtual Desktop Managers, Available at: http://www.virtual-desktop.info/, last accessed on May 17, 2006, 3 pages. cited by other .
Andy Harter, et al., A Distributed Location System for the Active Office, IEEE Network, 1994, pp. 62-70. cited by other .
Guanling Chen, et al., A Survey of Context-Aware Mobile Computing Research, Dartmouth Computer Science Technical Report, 2000, 16 pages. cited by other .
William Noah Schilt, A System Architecture for Context-Aware Mobile Computing, Columbia University, 1995, 153 pages. cited by other .
Mike Spreitzer, et al., Providing Location Information in a Ubiquitous Computing Environment, SIGOPS '93, 1993, pp. 270-283. cited by other .
Marvin Theimer, et al., Operating System Issues for PDAs, In Fourth Workshop on Workstation Operating Systems, 1993, 7 pages. cited by other .
Roy Want, Active Badges and Personal Interactive Computing Objects, IEEE Transactions on Consumer Electronics, 1992, 11 pages, vol. 38--No. 1. cited by other .
Bill N. Schilit, et al., The ParcTab Mobile Computing System, IEEE WWOS-IV, 1993, 4 pages. cited by other .
Bill Schilit, et al., Context-Aware Computing Applications, In Proceedings of the Workshop on Mobile Computing Systems and Applications, Dec. 1994. pp. 85-90. cited by other .
Bill N. Schilit, et al., Customizing Mobile Applications, Proceedings USENIX Symposium on Mobile and Location Independent Computing, Aug. 1993, 9 pages. cited by other .
Mike Spreitzer, et al., Architectural Considerations for Scalable, Secure, Mobile Computing with Location Information, In The 14th International Conference on Distributed Computing Systems, Jun. 1994, pp. 29-38. cited by other .
Mike Spreitzer et al., Scalable, Secure, Mobile Computing with Location Information, Communications of the ACM, Jul. 1993, 1 page, vol. 36--No. 7. cited by other .
Roy Want, et al., The Active Badge Location System, ACM Transactions on Information Systems, Jan. 1992, pp. 91-102, vol. 10--No. 1. cited by other .
Mark Weiser, Some Computer Science Issues in Ubiquitous Computing, Communications of the ACM, Jul. 1993, pp. 75-84, vol. 36--No. 7. cited by other .
M. Billinghurst, et al., An Evaluation of Wearable Information Spaces, Proceedings of the Virtual Reality Annual International Symposium, 1998, 8 pages. cited by other .
Bradley J. Rhodes, Remembrance Agent: A continuously running automated information retrieval system, The Proceedings of The First International Conference on The Practical Application Of Intelligent Agents and Multi Agent Technology, 1996, pp. 487-495. cited by other .
Eric Horvitz, et al., In Pursuit of Effective Handsfree Decision Support: Coupling Bayesian Inference, Speech Understanding, and User Models, 1995, 8 pages. cited by other .
Bradley J. Rhodes, The Wearable Remembrance Agent: A System for Augmented Theory, The Proceedings of The First International Symposium on Wearable Computers, Oct. 1997, pp. 123-128. cited by other .
Eric Horvitz, et al., Attention-Sensitive Alerting in Computing Systems, Microsoft Research, Aug. 1999. cited by other .
Bill N. Schilit, et al., Disseminationg Active Map Information to Mobile Hosts, IEEE Network, 1994, pp. 22-32, vol. 8--No. 5. cited by other .
Mark Billinghurst, et al., Wearable Devices: New Ways to Manage Information, IEEE Computer Society, Jan. 1999, pp. 57-64. cited by other .
Thad Eugene Starner, Wearable Computing and Contextual Awareness, Massachusetts Institute of Technology, Jun. 1999, 248 pages. cited by other .
Bradley J. Rhodes, The Wearable Remembrance Agent: A System for Augmented Memory, Personal Technologies Journal Special Issue on Wearable Computing, 1997, 12 pages. cited by other .
Workshop on Wearable Computing Systems, Aug. 19-21, 1996. cited by other .
Mark Billinghurst, Research Directions in Wearable Computing, University of Washington, May 1998, 48 pages. cited by other .
Mark Weiser, The Computer for the 21st Century, Scientific American, Sep. 1991, 8 pages. cited by other .
T. Joachims, Text categorization with support vector machines: learning with many relevant features, Machine Learning, European Conference on Machine Learning, Apr. 21, 1998, pp. 137-142. cited by other .
International Search Report dated Sep. 29, 2003 for PCT Application Serial No. 00/20685, 3 pages. cited by other .
Robert M. Losee, Jr., Minimizing information overload: the ranking of electronic messages, Journal of Information Science 15, Elsevier Science Publishers B.V., 1989, pp. 179-189. cited by other .
Liam Bannon, et al., Evaluation and Analysis of User's Activity Organization, Proceedings of the CHI '83, Dec. 1983, pp. 54-57. cited by other .
Patrick Baudisch, et al., Focus Plus Context Visualization Screens: Combining Display Technology with Visualization Techniques, Proceedings of UIST '01, ACM 2002, pp. 31-40. cited by other .
Benjamin B. Bederson, et al., Pad++: A Zooming Graphical Interface for Exploring Alternative Interface Physics, Proceedings of UIST '94, Nov. 2-4, 1994, pp. 17-26. cited by other .
Blaine A. Bell, et al., Dynamic Space Management for User Interfaces, Proceedings of UIST '00, ACM Symposium of User Interface Software and Technology, San Diego, California, Nov. 5-8, 2000, pp. 238-248. cited by other .
Sara A. Bly, et al., A Comparison of Tiled and Overlapping Windows, Proceedings of CHI '86, Apr. 1986, pp. 101-106. cited by other .
Stuart K. Card, et al., A Multiple, Virtual-Workspace Interface to Support User Task Switching. In Proc. CHI+GI 1987, pp. 53-59. cited by other .
Edward Cutrell, et al., Notification, Disruption, and Memory: Effects of Messaging Interruptions on Memory and Performance, Human-Computer Interaction--Interact '01, IOS Press, 2001, pp. 263-269. cited by other .
Mary Czerwinski, et al., Instant Messaging and Interruption: Influence of Task Type on Performance, Proceedings of OZCHI 2000, pp. 356-361. cited by other .
Mary Czerwinski, et al., Instant Messaging: Effects of Relevance and Time, Proceedings of HCI 2000, British Computer Society, 2000, vol. 2, pp. 71-76. cited by other .
Mary Czerwinski, et al., An Investigation of Memory for Daily Computing Events, Proceedings of HCI 2002, pp. 230-245. cited by other .
Mary Czerwinski, et al., A Diary Study of Task Switching and Interruptions, Proceedings of the Conference on Human Factors in Computing Systems (CHI 2004), Vienna, Austria, Apr. 24-29, 2004, vol. 6, No. 1, pp. 175-182. cited by other .
A. Goldberg, Smalltalk-80: Chapter 17-Views, New York: Addison-Wesley, 1983, pp. 292-297. cited by other .
D. Austin Henderson, Jr. et al., Rooms: The Use of Multiple Virtual Workspaces to Reduce Space Contention in a Window-Based Graphical User Interface, ACM Transactions on Graphics, vol. 5, No. 3, Jul. 1986, pp. 211-243. cited by other .
Eser Kandogan, et al., Elastic Windows: Evaluation of Multi-Window Operations, Proceedings of CHI 1997, pp. 250-257. cited by other .
Victor Kaptelinin, Umea: Translating Interaction Histories into Project Contexts, Proceedings of CHI 2003, Ft. Lauderdale, Florida, Apr. 5-10, 2003, vol. No. 5, Issue No. 1, pp. 353-360. cited by other .
Blaire Macintyre, et al., Support for Multitasking and Background Awareness Using Interactive Peripheral Displays, Proceedings of UIST 2001, Orlando, Florida, Nov. 11-14, 2001, ACM 2001, vol. 3, No. 2, pp. 41-50. cited by other .
Thomas W. Malone, How Do People Organize Their Desks? Implications for the Design of Office Information Systems, ACM Transactions on Office Information Systems, Jan. 1983, vol. 1, No. 1, pp. 99-112. cited by other .
Brad A. Myers, Window Interfaces: A Taxonomy of Window Manager User Interfaces, IEEE Computer Graphics and Applications, Sep. 1988, vol. 8, No. 5, pp. 65-84. cited by other .
Jun Rekimoto, Time-Machine Computing: A Time-centric Approach for the Information Environment, Proceedings of UIST '99, pp. 45-54. cited by other .
George Robertson, et al., The Task Gallery: A 3D Window Manager, Proceedings of CHI '00, Apr. 1-6, 2000, vol. 2, Issue 1, pp. 494-501. cited by other .
Greg Smith, et al., GroupBar: The TaskBar Evolved, Proceedings of OZCHI '03, 2003, 10 pages. cited by other .
Warren Teitelman, Ten Years of Window Systems--A Retrospective View, in F. Hopgood, et al. (Eds.), Methodology of Window Management, Berlin: Springer-Verlag, 1986, pp. 35-46. cited by other .
Desney S. Tan, et al., WinCuts: Manipulating Arbitrary Window Regions for More Effective Use of Screen Space, CHI 2004, Apr. 2004, pp. 1525-1528, ACM, Vienna, Austria. cited by other .
Dugald Hutching, et al., Revisiting Display Space Management: Understanding Current Practice to Inform Next-generation Design, Graphics Interface 2004, May 19, 2004, pp. 127-134, Canadian Human-Computer Communications Society, Ontario. cited by other .
Nicolas Roussel, Ametista: A Mini-Toolkit for Exploring New Window Management Techniques, Latin American Conference on Human-Computer Interaction, Aug. 2003, pp. 117-124, ACM, Rio de Janeiro, Brazil. cited by other .
George Robertson, et al., The Task Gallery: A 3D Window Manager, CHI 2000, Apr. 2000, pp. 494-501, V. 2.1, ACM, The Hague, Amsterdam. cited by other .
Eser Kandogan, et al., Elastic Windows: Evaluation of Multi-Window Operations, CHI 97, 1997, pp. 250-257, ACM, Atlanta, GA. cited by other .
George Robertson, et al., Scalable Fabric: Flexible Task Management, AVI '04, May 2004, pp. 85-89, ACM, Gallipoli, Italy. cited by other .
Francois Guimbretiere, Maureen Stone, and Terry Winograd. Fluid Interaction with High-resolution Wall-size Displays. Symposium on User Interface Software and Technology '01. Nov. 2001. 10 pages. cited by other .
Mary P. Czerwinski, Maarten Van Dantzich, George Robertson, and Hunter Hoffman. The Contribution of Thumbnail Image, Mouse-over Text and Spatial Location Memory to Web Page Retrieval in 3D. Interact '99, Edinburgh, UK. cited by other .
George Robertson, Mary Czerwinski, Kevin Larson, Daniel C. Robbins, David Thiel, and Maarten Van Dantzich. Data Mountain: Using Spatial Memory for Document Management. Symposium on User Interface Software. cited by other .
Elizabeth D. Mynatt, Takeo Igarashi, W. Keith Edwards, and Anthony Lamarca. Flatland: New Dimensions in Office Whiteboards. Conference on Human Factors and Computing Systems '99. pp. 346-356. May 1999. cited by other .
Mary Czerwinski, Susan Dumais, George Robertson, Susan Dziadosz, Scott Tiernan, and Maarten Van Dantzich. Visualizing Implicit Queries For Information Management Retrieval. Conference on Human Factors and Computing. cited by other .
Robert St. Amant, Michael D. Dinardo, and Nickie Buckner. Balancing Efficiency and Interpretability in an Interactive Statistical Assistant. International Conference on Intelligent User Interfaces'03, pp. 181-188, Jan. 2003. cited by other.

Primary Examiner: Hailu; Tadeese
Attorney, Agent or Firm: Amin, Turocy & Calvin, LLP

Parent Case Text

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation in part application of U.S. Ser. No. 10/374,351, entitled SYSTEM AND METHOD THAT FACILITATES COMPUTER DESKTOP USE VIA SCALING OF DISPLAYED OBJECTS WITH SHIFTS TO THE PERIPHERY, and filed on Feb. 25, 2003, the entirety of which is incorporated herein by reference.

Claims

What is claimed is:

1. A computer-implemented method that facilitates organizing data within a display, comprising: identifying items to be or already are displayed on a window of the display; automatically organizing the items via employment of predefined templates that define layout for the items within the window; automatically providing a prototypical structure for the window; and providing a template component that provides gestures such that separate components of the prototypical structure are selectively expanded to focus, the entire prototypical structure is selectively expanded, both selectively maintaining a cleaned up structure, isomorphic in layout to a minimized structure, and selectively returned to a last state.

2. The method of claim 1 further comprising identifying type of the items and associating a most relevant template to apply to the items.

3. The method of claim 1 further comprising inferring a most appropriate template to apply to a set of items.

4. The method of claim 1 further comprising learning a layout of items displayed on the display and applying the learned layout to a new set of items.

5. The method of claim 1 further comprising organizing and displaying the items as a function of user context.

6. The method of claim 1 further comprising organizing and displaying the items as a function of item content.

7. The method of claim 1 further comprising organizing and displaying the items as a function of determined and/or inferred priority of the respective items.

8. The method of claim 1, further comprising automatically setting behaviors and/or interaction modes for the items.

9. The method of claim 1, further comprising employing a user model in connection with automatically organizing and displaying a subset of the items.

10. The method of claim 9, wherein the user model comprises at least one of rules, historical data, preferences, Bayesian networks, neural networks, and non-linear training systems.

11. The method of claim 9, further comprising automatically configuring the items via employment of the user model.

12. The method of claim 1, further comprising automatically organizing the items as a function of at least one of: data types of the items, determined and/or inferred priority of the item, utility given user context and/or data content, and display space.

13. A computer readable storage medium having stored thereon computer-executable instructions for performing the method of claim 1.

14. A system that facilitates presenting data, comprising: means for defining how data is to be rendered on a display; means for automatically rendering the data on the display as a function of determined and/or inferred user desire for the data rendering; means for providing a glanceable conveyance of information associated with the data; and means for providing gestures such that separate components of a prototypical structure are selectively expanded to focus, the entire prototypical structure is selectively expanded, both selectively maintaining a cleaned up structure, isomorphic in layout to a minimized structure, and selectively returned to a last state.

15. A computer-implemented system that facilitates organizing items within one or more displays, comprising: a data store that stores information relating to organizing and displaying items within one or more windows of the display(s); and a template component that employs the information to automatically organize and display a set of items within the one or more windows, wherein the items comprise e-mail messages and the template component automatically organizes the messages so that the messages are resized and arranged in accordance with a template that provides for a desired organization of the e-mails, wherein the template component provides gestures such that separate components of a prototypical structure are selectively expanded to focus, the entire prototypical structure is selectively expanded, both selectively maintaining a cleaned up structure isomorphic in layout to a minimized structure, and selectively returned to a last state.

16. The system of claim 15, wherein the set of items are tasks, and the template component automatically configures and sets a layout of the tasks within the display(s).

17. The system of claim 15, wherein the items comprise at least one of clusters of windows, windows, and tasks.

18. The system of claim 15, further comprising an artificial intelligence component that infers user desired configuration of the items.

19. The system of claim 15, further comprising a task profile that stores encodings of representations of tasks, layouts, and/or behaviors on gestures.

20. The system of claim 19, wherein the task profile provides for defining global defaults and/or defining specialized canonical tasks.

21. The system of claim 15, wherein the items comprise windows and the template component automatically provides a prototypical structure for the window.

22. The system of claim 21, wherein the template component provides gestures that allow for separate components of the prototypical structure to be selectively expanded to focus, or to have the entire prototypical structure expanded, either maintaining a cleaned up structure, isomorphic in layout to a minimized structure, or returned to a last state.

23. The system of claim 15, wherein the organization is based upon at least one of: pre-defined rules, an explicitly and/or implicitly trained machine learning system, a utility-based analysis, probabilistic-based analysis, statistical-based analysis, and confidence-based analysis.

24. The system of claim 15, wherein the template component facilitates glanceable conveyance of information associated with the items.

25. The system of claim 15, wherein the template component employs a subset of a plurality of pre-defined templates and/or dynamically generated templates to facilitate organization and presentation of the items.

26. The system of claim 15, wherein the template component automatically learns a configuration for a data type and thereafter applies such learned configuration to a new set of data.

27. The system of claim 26, the template component organizes items associated with the new data in a manner inferred to be most consistent with the learned layout and/or consistent with inferred user desire for the manner in which the items should be presented.

Description

TECHNICAL FIELD

The present invention relates generally to user interfaces, and more particularly to a graphical user interface, data structure and method to facilitate management of data.

BACKGROUND OF THE INVENTION

Various graphical user interfaces have been developed to provide a rich experience for computer users. Computer programs typically provide a graphical user interface (GUI) to facilitate data entry, to enable viewing output on a display screen, as well as to manipulate or rearrange data. A graphical user interface can be associated with an application program or operating system shell, which may be running on a user's local machine and/or remotely, such as in a distributing computing system or over the Internet. In view of continuing technological developments and increasing use of the Internet, people are using computers to access information to an ever-increasing extent. Such information can reside locally on the person's computer or within a local network or be global in scope, such as over the Internet.

Users of window-based graphical user interfaces face difficult problems when they employ a same machine for multiple tasks or activities. They often have a large number of windows to manage, with one or more windows for each task. Switching among tasks is difficult because the windows often may be scattered around the display surface. Moreover, if windows are minimized while not in use, they are typically not organized together. If not minimized, a user can be faced with a difficult task of locating all relevant obscured windows and bringing them to a top of a display.

When users begin employing large display configurations (e.g., multiple monitors), managing windows and tasks becomes an ever more difficult problem, because minimized windows are kept in a location that may be significantly distant from where they will be used. Managing many display objects on small displays (e.g., PDA's) is also difficult--in such case, oftentimes sufficient screen space is not available to display objects of interest.

Although various attempts have been made via conventional user interface schemes to address some of the aforementioned concerns, there is still a substantial unmet need for a system and/or methodology that facilitates efficient use of valuable computer desktop real estate in a multi-task working environment.

SUMMARY OF THE INVENTION

The following presents a simplified summary of the invention in order to provide a basic understanding of some aspects of the invention. This summary is not an extensive overview of the invention. It is not intended to identify key/critical elements of the invention or to delineate the scope of the invention. Its sole purpose is to present some concepts of the invention in a simplified form as a prelude to the more detailed description that is presented later.

The subject invention relates to a system and/or method that provides for interaction technique(s) and user interface(s) in connection with managing display objects on a display surface. The inventors of the subject invention have found that as displays become larger, users leave more windows open for easy multitasking. A larger number of windows, however, may increase the time that users spend arranging and switching between tasks. The subject invention relates to a task management system that mitigates problems associated with proliferation of open windows on a personal computer (PC) desktop. The invention couples a flexible visual representation with window management to provide a focus-plus-context solution to desktop complexity. Users interact with windows in a central focus region of a display in a normal manner, but when a user moves window(s) into a periphery region, the window(s) shrinks down in size, getting smaller as they near an edge of the display. The Window "minimize" action is redefined to return the window to its preferred location in the periphery, allowing windows to remain visible when not in use. Windows in the periphery can be grouped together into named tasks, and task switching can be accomplished with a single mouse click. The spatial arrangement of tasks in accordance with the present invention leverages human spatial memory to make task switching easier.

The subject invention enhances windows and task management by affording for the handling of large numbers of concurrent windows, potentially clustered by task, a fundamentally more natural and effective experience. More particularly, a windows management system/methodology in accordance with the invention assists users in managing tasks for example on a Windows.RTM. desktop, allocating screen real estate in accordance with a user's attention, using a focus-plus-context display. The periphery of the screen is used to hold scaled down live windows rather than hiding them with traditional windows minimization. In order to facilitate task switching, the invention allows users to group collections of windows that are used together (e.g., tasks refer to groups of windows that are used together).

Although conventional virtual desktop managers typically impose strict separation between tasks, the subject invention provides for users to simultaneously display substantially any subset of windows, even if they should be assigned to different tasks. The invention makes use of the periphery of the display for spatial layout of tasks, in addition to leveraging users' efficient visual recognition memory for images. Moreover, the invention allows users to leave windows and clusters of windows open and visible at all times via a process of scaling down and moving the windows and clusters to the periphery.

One aspect of the invention defines one or more central focus area(s) where the display objects are displayed and behave as usual, and a periphery outside the focus area(s) where the display objects are reduced in size based on their location, getting smaller as they near an edge of the display surface so that many more objects can remain visible. In addition or alternatively, the objects can be diminished along dimensions defined by other properties of visual salience such opacity, luminosity, and saturation as they move away from a focus region. Thus objects may fade and/or become increasingly transparent as they move toward an edge--e.g., fading increasing as a function of distance from the focus area and/or use of the object and/or priority of the object. Objects in the periphery can also be modified to have different interaction behavior (e.g., lower refresh rate, fading, reconfigured to display sub-objects based on relevance and/or visibility, static, etc.) since they may be too small for standard techniques.

The invention can be implemented in connection with any suitable display area (e.g., large display surfaces, standard display surfaces and small, for example PDA, displays as well). The invention has wide applicability to window and task management in a multi-task computing environment, for example. In such cases, display objects are typically windows although they can be objects (e.g., icons, documents, presentations, media, pictures, audio files, video files etc.) as well. In the focus area, the windows can have a standard appearance and behavior. When the display objects cross into a periphery of the focus area, the objects reduce in size and the behavior can change to a much simpler set of interactions. For example, refresh rate of the windows can be made a function of size and/or distance from the focus area. In addition or alternatively, the windows outside of the periphery can become static until repositioned into the focus area.

By grouping windows in the periphery into clusters, and supporting cluster operations (e.g., move all cluster windows into the focus area, or move windows in the focus area back to their peripheral location), the invention significantly facilitates task management, allowing user(s) to easily swap from one task (or activity) to another.

Another aspect of the invention provides for employing computer-based intelligence (e.g., inference, probabilistic determination, statistical determination, machine learning etc.) that can move the display objects as a function of activity and/or priority associated therewith. For example, display objects that have not been used for a predetermined period of time can be made to drift from the focus area to the edge of the display space--alternatively, windows requiring notification attention can move toward the focus region. Likewise, size of the display object can also be automatically adjusted as a function of various pre-defined and/or learned metrics. Moreover, based on computer-based inference as to a user desired display of object(s), object(s) can automatically be moved to the focus area for interaction therewith based at least upon predetermined and/or learned metrics associated with user use and inferred intentions. Finally, notifications or other objects determined based on a computer-based inference as to be related to objects in the focus or peripheral areas, could be automatically clustered with similar items in the appropriate spatial location.

It is to be appreciated that the subject invention is not limited to window and/or task management, but can work for any suitable kind of display objects. For example, the invention can be employed in connection with photo management.

One particular aspect of the invention relates to a notion of automated, patterned, canonical display object (e.g., window) motion given enlargement or opening of indicia at a center of focus. A concept behind such notion is the idea that the invention can include flexible policies for effecting shifts of multiple display objects so as to automatically move older, less active objects into a periphery.

Another particular aspect of the invention relates to a notion of fundamental transitions or phases of transition that can be discontinuous, also to maintain, in a substantially continuous manner scaling properties associated therewith. In such case, a minimum density or resolution of an object can be established as a threshold and the object can be rendered into a different, potentially less visually accessible representation--and the object(s) can be re-rendered when other object(s) are closed. Thus, the subject invention can provide for several multilevel transitions, including continuous and potentially discontinuous transitions.

Yet another aspect of the invention provides for continuously adding more objects and a mechanism for automatically shifting and moving off and on objects, in a manner that keeps the respective objects on a screen usable (e.g., above a particular resolution). The invention also provides a means for implicitly returning to similar state(s) as newer objects are closed.

Another aspect of the invention relates to display object occlusion avoidance within a display area (e.g., focus area, periphery or progressive shrink area)--such aspect of the invention mitigates objects from obscuring one another. An extension of this aspect relates to cluster occlusion avoidance which mitigates clusters from obscuring other clusters as well as mitigating merging (e.g., since clustering can be proximity based) of clusters as a result of moving clusters.

Thus, the subject invention mitigates many of the aforementioned windows and task management problems by keeping windows open but making them much smaller when they are not in use. By doing so, it becomes possible to maintain a large number of windows around the periphery of the display. Task management is supported by placing windows into clusters that represent tasks and adding some simple mechanisms for task switching. Human spatial memory can be used to facilitate locating a desired window or task. The invention also mitigates a display object management problem for small displays, because the objects in the periphery take so little space.

To the accomplishment of the foregoing and related ends, certain illustrative aspects of the invention are described herein in connection with the following description and the annexed drawings. These aspects are indicative, however, of but a few of the various ways in which the principles of the invention may be employed and the present invention is intended to include all such aspects and their equivalents. Other advantages and novel features of the invention may become apparent from the following detailed description of the invention when considered in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a general block diagram of a display system in accordance with the present invention.

FIGS. 2-4 illustrate a display space having a stable region (e.g., focus area) and a progressive shrink region in accordance with the subject invention.

FIGS. 5-13 illustrate a display object changing behavior in accordance with the subject invention.

FIGS. 14-17 illustrate a web page window object changing behavior in accordance with the subject invention.

FIG. 18 illustrates a plurality of display objects and behavior thereof in accordance with the subject invention.

FIGS. 19-20 illustrate a plurality of display objects and behavior thereof in connection with a small device (e.g., PDA) display in accordance with the subject invention.

FIGS. 21-37 illustrate various aspects of the invention in connection with a template component.

FIGS. 38-42 illustrate various aspects of the invention related to markers for windows and/or clusters, and schemes for differentiating and/or distinguishing windows and/or clusters.

FIGS. 43-45 illustrate various aspects of the invention in connection with navigating among and/or within clusters via keys of a computing device.

FIGS. 46-52 illustrate various aspects of the invention relating to cluster avoidance techniques.

FIGS. 53-60 illustrate various aspects of the invention regarding employment of gradients to facilitate differentiating windows and/or clusters.

FIGS. 61-63 illustrate various aspects of the invention in connection with window detection and clustering.

FIGS. 64-82 illustrate an optional aspect of transition of display objects off of screen (e.g., to side bar) to another iconic form in accordance with the subject invention.

FIGS. 83-90 illustrate a focus region and/or progressive shrink region be configurable in accordance with the subject invention.

FIGS. 91-111 illustrate various aspects of the invention relating to modifying behavior of certain secondary objects as a function of modification to a primary object, and various embodiments for modifying behavior of display objects to facilitate a user experience in accordance with the subject invention.

FIG. 112 illustrates a block diagram of a computer operable to execute the disclosed architecture.

DETAILED DESCRIPTION OF THE INVENTION

The present invention is now described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It may be evident, however, that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to facilitate describing the present invention.

As used in this application, the terms "component" and "system" are intended to refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a server and the server can be a component. One or more components may reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers.

The subject invention can incorporate various inference schemes and/or techniques in connection with scaling and/or mo