Title: Systems and methods for rendering active stereo graphical data as passive stereo
Abstract: A preferred method includes the steps of receiving the active stereo video data containing the right channel pixel data and the left channel pixel data corresponding to the image to be rendered, re-sequencing the right channel pixel data and the left channel pixel data, and simultaneously outputting corresponding frames of the right channel pixel data and the left channel pixel data for displaying the image to be rendered in passive stereo. Devices also are provided.
Patent Number: 6,985,162 Issued on 01/10/2006 to Schinnerer, et al.
| Inventors:
|
Schinnerer; James A (Fort Collins, CO);
Alcorn; Byron A (Ft Collins, CO)
|
| Assignee:
|
Hewlett-Packard Development Company, L.P. (Houston, TX)
|
| Appl. No.:
|
715600 |
| Filed:
|
November 17, 2000 |
| Current U.S. Class: |
345/630; 345/629; 345/589; 345/679; 345/680 |
| Current Intern'l Class: |
G09G 5/00 (20060101) |
| Field of Search: |
345/630,629,589,679,680
|
References Cited [Referenced By]
U.S. Patent Documents
| 5408602 | Apr., 1995 | Giokas et al.
| |
| 5510832 | Apr., 1996 | Garcia.
| |
| 5523886 | Jun., 1996 | Johnson-Williams et al.
| |
| 5742333 | Apr., 1998 | Faris.
| |
| 5757321 | May., 1998 | Billyard.
| |
| 5790086 | Aug., 1998 | Zelitt.
| |
| 5844553 | Dec., 1998 | Hao et al.
| |
| 5923791 | Jul., 1999 | Hanna et al.
| |
| 5949428 | Sep., 1999 | Toelle et al.
| |
| 6034653 | Mar., 2000 | Robertson et al.
| |
| 6067098 | May., 2000 | Dye.
| |
| 6075917 | Jun., 2000 | Kim.
| |
| 6088036 | Jul., 2000 | Morris.
| |
| 6188385 | Feb., 2001 | Hill et al.
| |
| 6229553 | May., 2001 | Duluk et al.
| |
| 6351280 | Feb., 2002 | Benton.
| |
| 6400361 | Jun., 2002 | Toffolo.
| |
| 6400374 | Jun., 2002 | Lanier.
| |
| 6466205 | Oct., 2002 | Simpson et al.
| |
| 6501480 | Dec., 2002 | MacInnis et al.
| |
Other References
Alcorn et al., "Systems and Methods for Rendering Active Stereo Graphical Data
as Passive Stereo," U.S. Appl. No. 09/715,600, filed Nov. 17, 2000.
Lefebvre et al., "System and Method for Efficiently Rendering Graphical Data,"
U.S. Appl. No. 09/715,335, filed Nov. 17, 2000.
Lefebvre et al., "System and Method for Efficiently Rendering a Jitter Enhanced
Graphical Image," U.S. Appl. No. 09/715,253, filed Nov. 17, 2000.
"Understanding X Features: Distributed Single Logical Screen" http://www.hp.com/xwindow/sharedInfo/Whitepapers/Slsd/slsd.html,
1998, pp. 1-10.
Lefebvre, Kevin "Hewlett-Packard's Large Screen Multi-Display Technology: An
Exploration of the Architecture Behind HP's New Immersive Visualization Solutions"
http://www.hp.com;xwindow /sharedInfo/Whitepapers/Sls3d/sls—3d.html;
1998, pp. 1-9.
"Understanding X Features: Multiple Display Technologies" http://www.hp.com/xwindow/sharedInfo/Whitepapers/Sls/sls.html,
1997, pp. 1-13.
Alcorn et al., "Systems for Compositing Graphical Data," U.S. Appl. No. 09/715,892,
filed Nov. 17, 2000.
Robertus et al., "Systems and Methods for Compositing Graphical Data," U.S. Appl.
No. 09/715,232, filed Nov. 17, 2000.
Alcorn et al., "Systems and Methods for Rendering Graphical Data," U.S. Appl.
No. 09/715,882, filed Nov. 17, 2000.
|
Primary Examiner: Bella; Matthew C.
Assistant Examiner: Rahmjoo; Manucher
Claims
Now, therefore, the following is claimed:
1. A method for providing passive stereo images comprising:
receiving active stereo video data containing right channel pixel data and left
channel pixel data corresponding to an image to be rendered, the active stereo
video data being configured to enable alternate output of corresponding frames
of the right channel pixel data and the left channel pixel data for displaying
an image to be rendered in active stereo
converting the active stereo video data into passive stereo video data; and
displaying the image to be rendered in passive stereo using the passive stereo
video data.
2. The method of claim 1, displaying the image to be rendered in passive stereo
comprises displaying the image to be rendered at a frame rate that is approximately
one half of a frame rate of the pixel data of the active stereo video data.
3. The method of claim 1, wherein receiving the active stereo video data comprises
receiving the active stereo video data from multiple digital video data streams,
each of the multiple digital video data streams being provided by a graphics pipeline,
each graphics pipeline being configured to process pixel data corresponding to
at least a portion of the image to be rendered.
4. The method of claim 3, wherein:
receiving the active stereo video data from multiple digital video data streams comprises:
receiving a first of the multiple digital video data streams containing three-dimensional
pixel data corresponding to the image to be rendered; and
receiving a second of the multiple digital video data streams containing two-dimensional
pixel data corresponding to the image to be rendered; and
converting the active stereo video data into passive stereo video data comprises
combining the two-dimensional pixel data and the three-dimensional pixel data.
5. The method of claim 4, wherein combining the two-dimensional pixel data and
the three-dimensional pixel data comprises replacing at least a portion of the
pixel data provided by the second of the multiple digital video data streams with
at least a portion of the pixel data provided by the first of the multiple digital
video data streams.
6. The method of claim 1, further comprising:
providing a first left channel frame buffer, a second left channel frame buffer,
a first right channel frame buffer, and a second right channel frame buffer; and
wherein converting the active stereo video data into passive stereo video data
comprises allocating the right channel pixel data and the left channel pixel data
to the first left channel frame buffer, the second left channel frame buffer, the
first right channel frame buffer, and the second right channel frame buffer.
7. The method of claim 6, wherein allocating the right channel pixel data and
the left channel pixel data comprises utilizing chroma-key values for allocating
the right channel pixel data and the left channel pixel data.
8. The method of claim 6, wherein allocating the right channel pixel data and
the left channel pixel data comprises utilizing overscanned information contained
in the active stereo video data for allocating the right channel pixel data and
the left channel pixel data.
9. The method of claim 8, wherein utilizing overscanned information comprises
utilizing overscanned information contained in the active stereo video data for
allocating the right channel pixel data to one of various right channel buffers,
and the left channel pixel data to one of various left channel buffers.
10. The method of claim 1, wherein converting the active stereo video data into
passive stereo video data comprises:
buffering a first frame of the right channel pixel data;
buffering a first frame of the left channel pixel data;
simultaneously providing the first frame of the right channel pixel data and
the first frame of the left channel pixel data for displaying the image to be rendered;
determining whether a second frame of the right channel pixel data and a second
frame of left channel pixel data are ready for simultaneously providing; and
if the second frame of the right channel pixel data and the second frame of left
channel pixel data are not ready for simultaneously providing, again simultaneously
providing the first frame of the right channel pixel data and the first frame of
the left channel pixel data.
11. A device for providing passive stereo images comprising:
means for receiving active stereo video data containing right channel pixel data
and left channel pixel data corresponding to an image to be rendered, the active
stereo video data being configured to enable alternate output of corresponding
frames of the right channel pixel data and the left channel pixel data for displaying
an image to be rendered in active stereo; and
means for converting the active stereo video data into passive stereo video data.
12. The device of claim 11, wherein a frame rate of the pixel data for displaying
the image to be rendered in passive stereo is approximately one half of a frame
rate of the pixel data of the active stereo video data.
13. The device of claim 11, further comprising:
means for frame buffering the left channel pixel data;
means for frame buffering the right channel pixel data; and
means for allocating the right channel pixel data and the left channel pixel
data to said means for frame buffering the left channel pixel data and said means
for frame buffering the right channel.
14. The device of claim 11, further comprising:
means for determining whether a second frame of the right channel pixel data
and a second frame of left channel pixel data are ready for simultaneously providing; and
means for simultaneously re-providing a first frame of the right channel pixel
data and a first frame of the left channel pixel data if the second frame of the
right channel pixel data and the second frame of left channel pixel data are not
ready for simultaneously providing.
15. The device of claim 11, further comprising means for displaying the image
to be rendered in passive stereo.
16. A device for providing passive stereo images comprising:
logic configured to convert active stereo video data into passive stereo video
data, the active stereo video data containing right channel pixel data and left
channel pixel data, the active stereo video data being configured to enable alternate
output of corresponding frames of the right channel pixel data and the left channel
pixel data for displaying an image to be rendered in active stereo.
17. The device of claim 16, further comprising:
logic configured to determine whether a second frame of the right channel pixel
data and a second frame of left channel pixel data are ready for simultaneously
providing; and
logic configured to simultaneously re-provide a first frame of the right channel
pixel data and a first frame of the left channel pixel data if the second frame
of the right channel pixel data and the second frame of left channel pixel data
are not ready for simultaneously providing.
18. A device for providing passive stereo images comprising:
a graphics processor operative to convert active stereo video data into a passive
stereo video data, said graphics processor comprising:
an input mechanism configured to receive the active stereo video data, the active
stereo video data containing right channel pixel data and left channel pixel data,
the active stereo video data being configured to enable alternate output of corresponding
frames of the right channel pixel data and the left channel pixel data for displaying
an image to be rendered in active stereo, the active stereo video data being provided
as multiple digital video data streams containing the right channel pixel data
and the left channel pixel data; and
an output mechanism electrically communicating with said input mechanism, said
output mechanism being configured to receive the right channel pixel data and the
left channel pixel data and selectively provide the pixel data as a passive stereo
video data stream.
19. The device of claim 18, wherein said input mechanism is configured to provide
a frame of data corresponding to the image to be rendered, and insert pixel data
from the multiple digital video data streams into said frame of data such that,
in response to receiving a first of the multiple digital video data streams, said
input mechanism provides said frame of data and inserts the pixel data from the
first of the multiple digital video data streams into a corresponding portion of
said frame of data.
20. The device of claim 18, wherein said output mechanism has a first left channel
frame buffer, a second left channel frame buffer, a first right channel frame buffer,
and a second right channel frame buffer, said output mechanism being selectively
configured to provide said passive stereo video data stream by receiving pixel
data from said input mechanism, allocating said pixel data to said first left channel
frame buffer, said second left channel frame buffer, said first right channel frame
buffer, and said second right channel frame buffer, and simultaneously outputting
pixel data from one of said left channel frame buffers and one of said right channel
frame buffers.
21. The device of claim 18, wherein said output mechanism is further configured
to selectively provide the pixel data as an active stereo video data stream.
Description
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention generally relates to techniques for rendering graphical
data and, in particular, to systems and methods rendering active stereo graphical
data as passive stereo.
2. Related Art
Computer graphical display systems are commonly used for displaying graphical
representations of two-dimensional and/or three-dimensional objects on a two-dimensional
display device, such as a cathode ray tube, for example. Current computer graphical
display systems provide detailed visual representations of objects and are used
in a variety of applications.
FIG. 1 depicts an exemplary embodiment of a conventional computer graphical
display system
15. A graphics application
17 stored on a computer
21 defines, in data, an object to be rendered by the system
15. To
render the object, the application
17 transmits graphical data defining
the object to graphics pipeline
23, which may be implemented in hardware,
software, or a combination thereof. The graphics pipeline
23, through well
known techniques, processes the graphical data received from the application
17
and stores the graphical data in a frame buffer
26. The frame buffer
26
stores the graphical data necessary to define the image to be displayed by a display
device
29. In this regard, the frame buffer
26 includes a set of
data for each pixel displayed by the display device
29. Each set of data
is correlated with the coordinate values that identify one of the pixels displayed
by the display device
29, and each set of data includes the color value
of the identified pixel as well as any additional information needed to appropriately
color or shade the identified pixel. Normally, the frame buffer
26 transmits
the graphical data stored therein to the display device
29 via a scanning
process such that each line of pixels defining the image displayed by the display
device
29 is consecutively updated.
When large images are to be displayed, multiple display devices may be used
to display a single image, in which each display device displays a portion of the
single image. In such an embodiment, the multiple display devices are treated as
a single logical screen (SLS), and different portions of an object may be rendered
by different display devices. FIG. 2 depicts an exemplary embodiment of a computer
graphics system
41 capable of utilizing a plurality of display devices
31-
34
to render a single logical screen. In this embodiment, a client computer
42
stores the application
17 that defines, in data, an image to be displayed.
Each of the display devices
31-
34 may be used to display a portion
of an object such that the display devices
31-
34, as a group, display
a single large image of the object.
To render the object, graphical data defining the object is transmitted to an
SLS server
45. The SLS server
45 routes the graphical data to each
of the graphics pipelines
36-
39 for processing and rendering. For
example, assume that the object is to be positioned such that each of the display
devices
31-
34 displays a portion of the object. Each of the pipelines
36-
39 renders the graphical data into a form that can be written
into one of the frame buffers
46-
49. Once the data has been rendered
by the pipelines
36-
39 to the point that the graphical data is in
a form suitable for storage into frame buffers
46-
49, each of the
pipelines
36-
39 performs a clipping process before transmitting the
data to frame buffers
46-
49.
In the clipping process, each pipeline
36-
39 discards the graphical
data defining the portions of the object that are not to be displayed by the pipeline's
associated display device
31-
34 (i.e., the display device
31-
34
coupled to the pipeline
36-
39 through one of the frame buffers
46-
49).
In other words, each graphics pipeline
36-
39 discards the graphical
data defining the portions of the object displayed by the display devices
31-
34
that are not coupled to the pipeline
36-
39 through one of the frame
buffers
46-
49. For example, pipeline
36 discards the graphical
data defining the portions of the object that are displayed by display devices
32-
34, and pipeline
37 discards the graphical data defining
the portions of the object that are displayed by display devices
31,
33,
and
34.
Thus, each frame buffer
46-
49 should only store the graphical
data defining the portion of the object displayed by the display device
31-
34
that is coupled to the frame buffer
46-
49. At least one solution
for providing SLS functionality in an X Window System environment is taught by
Jeffrey J. Walls, Ian A. Elliott, and John Marks in U.S. Pat. No. 6,088,005, filed
Jan. 10, 1996, and entitled "Design and Method for a Large, Virtual Workspace,"
which is incorporated herein by reference.
A plurality of networked computer systems are often employed in implementing
SLS
technology. For example, in the embodiment shown by FIG. 2, the client
42,
the SLS server
45, and the individual graphics pipelines
36-
39
may each be implemented via a single computer system interconnected with the other
computer systems within the system
41 via a computer network, such a local
area network (LAN), for example. The X Window System is a standard for implementing
window-based user interfaces in a networked computer environment, and it may be
desirable to utilize X Protocol in rendering graphical data in the system
41.
For a more detailed discussion of the X Window System and the X Protocol that defines
it, see Adrian Nye,
X Protocol Reference Manual Volume Zero (O'Riley & Associates 1990).
U.S. patent application Ser. No. 09/138,456, filed on Aug. 21, 1998, and entitled
"3D Graphics in a Single Logical Screen Display Using Multiple Remote Computer
Systems," which is incorporated herein by reference, describes an SLS system of
networked computer stations that may be used to render two-dimensional (2D) and
three-dimensional (3D) graphical data. In the embodiments described by the foregoing
patent application, X Protocol is generally utilized to render 2D graphical data,
and OpenGL Protocol (OGL) is generally used to render 3D graphical data.
Although it is possible to render 2D and/or 3D data in conventional computer
graphical display systems, including SLS environments, there exists limitations
that restrict the performance and/or image quality exhibited by the conventional
computer graphical display systems. More specifically, high quality images, particularly
3D images, are typically defined by a large amount of graphical data, and the speed
at which conventional graphics pipelines
36-
39 can process the graphical
data defining an object is limited. Thus, a trade-off often exists between increasing
the quality of the image rendered by a computer graphical display system and the
speed at which the image can be rendered, and there exists a need in the industry
for better techniques and systems for rendering graphical data.
SUMMARY OF THE INVENTION
Briefly described, the present invention relates to techniques for rendering
graphical data. In this regard, embodiments of the present invention may be construed
as providing methods for converting active stereo video data into passive stereo
video data. Active stereo video data contains right channel pixel data and left
channel pixel data, and is configured to enable alternate output of corresponding
frames of the right channel pixel data and the left channel pixel data for displaying
an image to be rendered in active stereo. In this regard, a preferred method includes
the steps of receiving the active stereo video data containing the right channel
pixel data and the left channel pixel data corresponding to the image to be rendered,
re-sequencing the right channel pixel data and the left channel pixel data, and
simultaneously outputting corresponding frames of the right channel pixel data
and the left channel pixel data for displaying the image to be rendered in passive stereo.
Other embodiments of the present invention may be construed as providing devices
for converting active stereo video data into passive stereo video data. In this
regard, a preferred device includes an input mechanism and an output mechanism.
The input mechanism is configured to receive the active stereo video data, which
may be provided as multiple digital video data streams containing the right channel
pixel data and the left channel pixel data. The output mechanism electrically communicates
with the input mechanism, and is configured to receive the right channel pixel
data and the left channel pixel data. Additionally, the output mechanism is configured
to selectively provide the pixel data as either a passive stereo video data stream
or an active stereo video data stream.
An alternative embodiment of the device includes means for receiving the active
stereo video data containing the right channel pixel data and the left channel
pixel data corresponding to the image to be rendered, means for re-sequencing the
right channel pixel data and the left channel pixel data, and means for simultaneously
outputting corresponding frames of the right channel pixel data and the left channel
pixel data for displaying the image to be rendered in passive stereo.
Another embodiment of the device includes logic configured to receive the
active stereo video data containing the right channel pixel data and the left channel
pixel data corresponding to the image to be rendered, logic configured to re-sequence
the right channel pixel data and the left channel pixel data, and logic configured
to simultaneously output corresponding frames of the right channel pixel data and
the left channel pixel data for displaying the image to be rendered in passive stereo.
Other features and advantages of the present invention will become apparent
to one skilled in the art upon examination of the following detailed description,
when read in conjunction with the accompanying drawings. It is intended that all
such features and advantages be included herein within the scope of the present
invention and protected by the claims.
BRIEF DESCRIPTION OF THE DRAWINGS
The invention can be better understood with reference to the following drawings.
The elements of the drawings are not necessarily to scale relative to each other,
emphasis instead being placed upon clearly illustrating the principles of the invention.
Furthermore, like reference numerals designate corresponding parts throughout the
several views.
FIG. 1 is a block diagram illustrating a conventional graphical display system.
FIG. 2 is a block diagram illustrating a conventional single logical screen
(SLS) graphical display system.
FIG. 3 is a block diagram illustrating a graphical display system in accordance
with the present invention.
FIG. 4 is a block diagram illustrating a more detailed view of a client depicted
in FIG. 3.
FIG. 5 is a block diagram illustrating a more detailed view of a master pipeline
depicted in FIG. 3.
FIG. 6 is a block diagram illustrating a more detailed view of a slave pipeline
depicted in FIG. 3.
FIG. 7 is a diagram illustrating a more detailed view of a display device depicted
in FIG. 3. The display device of FIG. 7 is displaying an exemplary X window having
a center region for displaying three-dimensional objects.
FIG. 8 is a diagram illustrating the display device depicted in FIG. 7 with
the center region partitioned according to one embodiment of the present invention.
FIG. 9 is a diagram illustrating the display device depicted in FIG. 7 with
the center region partitioned according to another embodiment of the present invention.
FIG. 10 is a diagram illustrating the display device depicted in FIG. 8 with
a three-dimensional object displayed within the center region.
FIG. 11 is a diagram illustrating the display device depicted in FIG. 7 when
super sampled data residing in one of the frame buffers interfaced with one of
the slave pipelines is displayed within the center region of the display device.
FIG. 12 is a diagram illustrating the display device depicted in FIG. 11 when
super sampled data residing in another of the frame buffers interfaced with another
of the slave pipelines is displayed within the center region of the display device.
FIG. 13 is a block diagram illustrating another embodiment of the graphical
display system depicted in FIG. 3.
FIG. 14 is a single logical screen (SLS) graphical display system that utilizes
a graphical acceleration unit depicted in FIG. 3 or FIG. 13.
FIG. 15 is a diagram illustrating a more detailed view of display devices that
are depicted in FIG. 14.
FIG. 16 is a block diagram illustrating a graphical display system in accordance
with the present invention.
FIG. 17 is a flowchart illustrating functionality of a preferred embodiment
of the compositor of the present invention.
FIG. 18 is a flowchart illustrating functionality of a preferred compositor
of the present invention.
FIG. 19 is a block diagram illustrating a portion of a preferred embodiment
of the compositor of the present invention.
FIG. 20A is a schematic diagram illustrating a representative active stereo
system which may be implemented by a preferred embodiment of the present invention.
FIG. 20B is a schematic diagram illustrating the representative active stereo
system of FIG. 20A.
FIG. 21 is a block diagram illustrating a portion of a preferred embodiment
of the compositor of the present invention.
FIG. 22 is a diagram illustrating a representative frame buffer sequence in
relation to an image sequence corresponding to an active stereo implementation
of the present invention.
FIG. 23 is a schematic diagram illustrating a representative passive stereo
system which may be implemented a preferred embodiment of the present invention.
FIG. 24 is a diagram illustrating a representative frame buffer sequence in
relation to an image sequence corresponding to a passive stereo implementation
of the present invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
In general, the present invention pertains to a computer graphical display system
employing a plurality of graphics pipelines that process graphical data in parallel.
The graphics pipelines provide the graphical data to a compositor that combines
or composites the graphical data into a single data stream that can be rendered
via a single display device. The parallel processing of graphical data according
to the present invention enables improved performance and/or improved image quality
over conventional computer graphical display systems.
FIG. 3 depicts a computer graphical display system
50 in accordance with
the present invention. As shown by FIG. 3, the system
50 includes a client
52, a master graphics pipeline
55, and one or more slave graphics
pipelines
56-
59. The client
52 and pipelines
55-
59
may be implemented via hardware, software or any combination thereof. It should
be noted that the embodiment shown by FIG. 3 depicts four slave pipelines
56-
59
for illustrative purposes only, and any number of slave pipelines
56-
59
may be employed to implement the present invention in other embodiments. As shown
by FIG. 3, the pipelines
55-
59, frame buffers
65-
69,
and compositor
76 that render graphical data to a single display device
83 are collectively referred to herein as a graphical accelerations unit
95.
The master pipeline
55 receives graphical data from the application
17
stored in the client
52. The master pipeline
55 preferably renders
two-dimensional (2D) graphical data to frame buffer
65 and routes three-dimensional
(3D) graphical data to slave pipelines
56-
59, which render the 3D
graphical data to frame buffers
66-
69, respectively. Except as otherwise
described herein, the client
52 and the pipelines
55-
59 may
be configured similar to pipelines described in U.S. patent application Ser. No.
09/138,456. The client
52 and the pipelines
55-
59 will be
described in more detail hereinafter.
Each frame buffer
65-
69 outputs a stream of graphical data to
the compositor
76. The compositor
76 is configured to combine or
composite each of the data streams from frame buffers
65-
69 into
a single data stream that is provided to display device
83, which may be
a display device (e.g., cathode ray tube) for displaying an image. The graphical
data provided to the display device
83 by the compositor
76 defines
the image to be displayed by the display device
83 and is based on the graphical
data received from frame buffers
65-
69. The compositor
76
will be further described in more detail hereinafter. Note that each data stream
depicted in FIG. 3 may be either a serial data stream or a parallel data stream.
In the preferred embodiment, the client
52 and each of the pipelines
55-
59
are respectively implemented via stand alone computer systems, commonly referred
to as a "computer workstations." Thus, the system
50 shown by FIG. 3 may
be implemented via six computer workstations (i.e., one computer workstation for
the client
52 and one computer workstation for each of the pipelines
55-
59).
However, it is possible to implement the client
52 and pipelines
55-
59
via other configurations, including other numbers of computer workstations or no
computer workstations. As an example, the client
52 and the master pipeline
55 may be implemented via a single computer workstation. Any computer workstation
used to implement the client
52 and/or pipelines
55-
59 may
be utilized to perform other desired functionality when the workstation is not
being used to render graphical data.
Furthermore, as shown by FIG. 3, the client
52 and the pipelines
55-
59 may be interconnected via a local area network (LAN)
62.
However, it is possible to utilize other types of interconnection circuitry without
departing from the principles of the present invention.
FIG. 4 depicts a more detailed view of the client
52. As can be seen
by referring to FIG. 4, the client
52 preferably stores the graphics application
17 in memory
102. Through conventional techniques, the application
17 is executed by an operating system
105 and one or more conventional
processing elements
111, such as a central processing unit (CPU), for example.
The operating system
105 performs functionality similar to conventional
operating systems. More specifically, the operating system
105 controls
the resources of the client
52 through conventional techniques and interfaces
the instructions of the application
17 with the processing element
111
as necessary to enable the application
17 to run properly.
The processing element
111 communicates to and drives the other elements
within the client
52 via a local interface
113, which can include
one or more buses. Furthermore, an input device
115, for example, a keyboard
or a mouse, can be used to input data from a user of the client
52, and
an output device
117, for example, a display device or a printer, can be
used to output data to the user. A disk storage mechanism
122 can be connected
to the local interface
113 to transfer data to and from a nonvolatile disk
(e.g., magnetic, optical, etc.). The client
52 is preferably connected to
a LAN interface
126 that allows the client
52 to exchange data with
the LAN
62.
In the preferred embodiment, X Protocol is generally utilized to render 2D graphical
data, and OpenGL Protocol (OGL) is generally utilized to render 3D graphical data,
although other types of protocols may be utilized in other embodiments. By way
of background, OpenGL Protocol is a standard application programmer's interface
(API) to hardware that accelerates 3D graphics operations. Although OpenGL Protocol
is designed to be window system independent, it is often used with window systems,
such as the X Window System, for example. In order that OpenGL Protocol may be
used in an X Window System environment, an extension of the X Window System has
been developed called GLX. For more complete information on the GLX extension to
the X Window System and on how OpenGL Protocol can be integrated with the X Window
System, see for example Mark J. Kilgard,
OpenGL Programming for the X Window
System (Addison-Wesley Developers Press 1996), which is incorporated herein
by reference.
When the application
17 issues a graphical command, a client side GLX
layer
131 of the client
52 transmits the command over LAN
62
to master pipeline
55. FIG. 5 depicts a more detailed view of the master
pipeline
55. Similar to client
52, the master pipeline
55
includes one or more processing elements
141 that communicate to and drive
the other elements within the master pipeline
55 via a local interface
143,
which can include one or more buses. Furthermore, an input device
145, for
example, a keyboard or a mouse, can be used to input data from a user of the pipeline
55, and an output device
147, for example, a display device or a
printer, can be used to output data to the user. A disk storage mechanism
152
can be connected to the local interface
143 to transfer data to and from
a nonvolatile disk (e.g., magnetic, optical, etc.). The pipeline
55 may
be connected to a LAN interface
156 that allows the pipeline
55 to
exchange data with the LAN
62.
The pipeline
55 also includes an X server
162. The X server
162
may be implemented in software, hardware, or a combination thereof, and in the
embodiment shown by FIG. 5, the X server
162 is implemented in software
and stored in memory
164. In the preferred embodiment, the X server
162
renders 2D X window commands, such as commands to create or move an X window. In
this regard, an X server dispatch layer
173 is designed to route received
commands to a device independent layer (DIX)
175 or to a GLX layer
177.
An X window command that does not include 3D data is interfaced with DIX, whereas
an X window command that does include 3D data (e.g., an X command having embedded
OGL protocol, such as a command to create or change the state of a 3D image within
an X window) is routed to GLX layer
177. A command interfaced with the DIX
175 is executed by the DIX
175 and potentially by a device dependent
layer (DDX)
179, which drives graphical data associated with the executed
command through pipeline hardware
166 to frame buffer
65. A command
interfaced with GLX layer
177 is transmitted by the GLX layer
177
across LAN
62 to slave pipelines
56-
59. One or more of the
pipelines
56-
59 executes the command and drives graphical data associated
with the command to one or more frame buffers
66-
69.
In the preferred embodiment, each of slave pipelines
56-
59 is configured
according to FIG. 6, although other configurations of pipelines
56-
59
in other embodiments are possible. As shown by FIG. 6, each slave pipeline
56-
59
includes an X server
202, similar to the X server
162 previously
described, and an OGL daemon
205. The X server
202 and OGL daemon
205 may be implemented in software, hardware, or a combination thereof,
and in the embodiment shown by FIG. 6, the X server
202 and OGL daemon
205
are implemented in software and stored in memory
206. Similar to client
52 and master pipeline
55, each of the slave pipelines
56-
59
includes one or more processing elements
181 that communicate to and drive
the other elements within the pipeline
56-
59 via a local interface
183, which can include one or more buses. Furthermore, an input device
185,
for example, a keyboard or a mouse, can be used to input data from a user of the
pipeline
56-
59, and an output device
187, for example, a display
device or a printer, can be used to output data to the user. A disk storage mechanism
192 can be connected to the local interface
183 to transfer data
to and from a nonvolatile disk (e.g., magnetic, optical, etc.). Each pipeline
56-
59
is preferably connected to a LAN interface
196 that allows the pipeline
56-
59 to exchange data with the LAN
62.
Similar to X server
162, the X server
202 includes an X server
dispatch layer
208, a GLX layer
211, a DIX layer
214, and
a DDX layer
216. In the preferred embodiment, each command received by the
slave pipelines
56-
59 includes 3D graphical data, since the X server
162 of master pipeline
55 executes each X window command that does
not include 3D graphical data. The X server dispatch layer
208 interfaces
the 2D data of any received commands with DIX layer
214 and interfaces the
3D data of any received commands with GLX layer
211. The DIX and DDX layers
214 and
216 are configured to process or accelerate the 2D data and
to drive the 2D data through pipeline hardware
166 to one of the frame buffers
66-
69 (FIG. 3).
The GLX layer
211 interfaces the 3D data with the OGL dispatch layer
223
of the OGL daemon
205. The OGL dispatch layer
223 interfaces this
data with the OGL DI layer
225. The OGL DI layer
225 and DD layer
227 are configured to process the 3D data and to accelerate or drive the
3D data through pipeline hardware
199 to one of the frame buffers
66-
69
(FIG. 3). Thus, the 2D graphical data of a received command is processed or accelerated
by the X server
202, and the 3D graphical data of the received command is
processed or accelerated by the OGL daemon
205. For a more detailed description
of the foregoing process of accelerating 2D data via an X server
202 and
of accelerating 3D data via an OGL daemon
205, refer to U.S. patent application
Ser. No. 09/138,456.
Preferably, the slave pipelines
56-
59, based on inputs
from the master pipeline
55, are configured to render 3D images based on
the graphical data from master pipeline
55 according to one of three modes
of operation: the optimization mode, the super sampling mode, and the jitter mode.
In the optimization mode, each of the slave pipelines
56-
59 renders
a different portion of a 3D image such that the overall process of rendering the
3D image is faster. In the super sampling mode, each portion of a 3D image rendered
by one or more of the slave pipelines
56-
59 is super-sampled in order
to increase quality of the 3D image via anti-aliasing. Furthermore, in the jitter
mode, each of the slave pipelines
56-
59 renders the same 3D image
but slightly offsets each rendered 3D image with a different offset value. Then,
the compositor
76 averages the pixel data of each pixel for the 3D images
rendered by the pipelines
56-
59 in order to produce a single 3D image
of increased image quality. Each of the foregoing modes will be described in more
detail hereafter.
Optimization Mode
Referring to FIG. 3, the operation and interaction of the client
52,
the pipelines
55-
59, and the compositor
76 will now be described
in more detail according to a preferred embodiment of the present invention while
the system
50 is operating in the optimization mode. In such an embodiment,
the master pipeline
55, in addition to controlling the operation of the
slave pipelines
56-
59 as described hereinafter, is used to create
and manipulate an X window to be displayed by the display device
83. Furthermore,
each of the slave pipelines
56-
59 is used to render 3D graphical
data within a portion of the foregoing X window.
For the purposes of illustrating the aforementioned embodiment, assume that the
application
17 issues a function call (i.e., the client
52 via processing
element
111 (FIG. 4) executes a function call within the application
17)
for creating an X window having a 3D image displayed within the X window. FIG.
7 depicts a more detailed view of the display device
83 displaying such
a window
245 on a display device screen
247. In the example shown
by FIG. 7, the screen
247 is 2000 pixels by 2000 pixels ("2K×2K"),
and the X window
245 is 1000 pixels by 1000 pixels ("1K×1K"). The window
245 is offset from each edge of the screen
247 by 500 pixels. Assume
that 3D graphical data is to be rendered in a center region
249 of the X
window
245. This center region
249 is offset from each edge of the
window
245 by 200 pixels in the embodiment shown by FIG. 7.
In response to execution of the function call by client
52, the application
17 transmits to the master pipeline
55 a command to render the X
window
245 and a command to render a 3D image within portion
249
of the X window
245. The command for rendering the X window
245 should
include 2D graphical data defining the X window
245, and the command for
rendering the 3D image within the X window
245 should include 3D graphical
data defining the 3D image to be displayed within region
249. Preferably,
the master pipeline
55 renders 2D graphical data from the former command
(i.e., the command for rendering the X window
245) to frame buffer
65
(FIG. 3) via X server
162 (FIG. 6).
The graphical data rendered by any of the pipelines
55-
59 includes
sets of values that respectively define a plurality of pixels. Each set of values
includes at least a color value and a plurality of coordinate values associated
with the pixel being defined by the set of values. The coordinate values define
the pixel's position relative to the other pixels defined by the graphical data,
and the color value indicates how the pixel should be colored. While the coordinate
values indicate the pixel's position relative to the other pixels defined by the
graphical data, the coordinate values produced by the application
17 are
not the same coordinate values assigned by the display device
83 to each
pixel of the screen
247. Thus, the pipelines
55-
59 should
translate the coordinate values of each pixel rendered by the pipelines
55-
59
to the coordinate values used by the display device
83 to display images.
Sometimes the coordinate values produced by the application
17 are said
to be "window relative," and the aforementioned coordinate values translated from
the window relative coordinates are said to be "screen relative." The concept of
translating window relative coordinates to screen relative coordinates is well
known, and techniques for translating window relative coordinates to screen relative
coordinates are employed by most conventional graphical display systems.
In addition to translating coordinates of the 2D data rendered by the master
pipeline
55 from window relative to screen relative, the master pipeline
55
in each mode of operation also assigns a particular color value, referred to hereafter
as the "chroma-key," to each pixel within the region
249. The chroma-key
indicates which pixels within the X window
245 may be assigned a color value
of a 3D image that is generated by slave pipelines
56-
59. In this
regard, each pixel assigned the chroma-key as the color value by master server
55 is within region
249 and, therefore, may be assigned a color of
a 3D object rendered by slave pipelines
56-
59, as will be described
in further detail hereafter. In the example shown by FIG. 7, the graphical data
rendered by master pipeline
55 and associated with screen relative coordinate
values ranging from (700, 700) to (1300, 1300) are assigned the chroma-key as their
color value by the master pipeline
55, since the region
249 is the
portion of X window
245 that is to be used for displaying 3D images.
As shown by FIG. 5, the master pipeline
55 includes a slave controller
261 that is configured to provide inputs to each slave pipeline
56-
59
over the LAN
62. The slave controller
261 may be implemented in software,
hardware, or a combination thereof, and in the embodiment shown by FIG. 5, the
slave controller
261 is implemented in software and stored in memory
206.
The inputs from the slave controller
261 inform the slaves
56-
59
of which mode each slave
56-
59 should presently operate. In the present
example, the slave controller
261 transmits inputs to each slave
56-
59
indicating that each slave
56-
59 should be in the optimization mode
of operation. The inputs from slave controller
261 also indicate which portion
of region
249 (FIG. 7) that is each slave's responsibility. For example,
assume for illustrative purposes, that each slave
56-
59 is responsible
for rendering the graphical data displayed in one of the portions
266-
269
shown by FIG. 8.
In this regard, assume that: (1) slave pipeline
56 is responsible for
rendering
graphical data defining the image displayed in portion
266 (i.e., screen
relative coordinates (700, 1000) to (1000, 1300), (2) slave pipeline
57
is responsible for rendering graphical data defining the image displayed in portion
267 (i.e., screen relative coordinates (1000, 1000) to (1300, 1300), (3)
slave pipeline
58 is responsible for rendering graphical data defining the
image displayed in portion
268 (i.e., screen relative coordinates (700,
700) to (1000, 1000), and (4) slave pipeline
59 is responsible for rendering
graphical data defining the image displayed in portion
269 (i.e., screen
relative coordinates (1000, 700) to (1300, 1000). The inputs transmitted by the
slave controller
261 to the slave pipelines
56-
59 preferably
indicate the range of screen coordinate values that each slave pipeline
56-
59
is responsible for rendering.
Note that the partition of the region
249 can be divided among the pipelines
56-
59 via other configurations, and it is not necessary for each
pipeline
56-
59 to be responsible for an equally sized area of the
region
249. For example, FIG. 9 shows an embodiment where each portion
266-
269
represents a different sized horizontal area of the region
249.
Each slave pipeline
56-
59 is configured to receive from master
pipeline
55 the graphical data of the command for rendering the 3D image
to be displayed in region
249 and to render this data to frame buffers
66-
69,
respectively. In this regard, each pipeline
56-
59 renders graphical
data defining a 2D X window that displays a 3D image within the window. More specifically,
slave pipeline
56 renders graphical data to frame buffer
66 that
defines an X window displaying a 3D image within portion
266 (FIG. 8). The
X server
202 within slave pipeline
56 renders the data that defines
the foregoing X window, and the OGL daemon
205 within the slave pipeline
56 renders the data that defines the 3D image displayed within the foregoing
X window. Furthermore, slave pipeline
57 renders graphical data to frame
buffer
67 that defines an X window displaying a 3D image within portion
267 (FIG. 8). The X server
202 within slave pipeline
57 renders
the data that defines the foregoing X window, and the OGL daemon
205 within
the slave pipeline
57 renders the data that defines the 3D image displayed
within the foregoing X window. Similarly, slave pipelines
58 and
59
render graphical data to frame buffers
68 and
69, respectively, via
the X server
202 and the OGL daemon
205 within the pipelines
58
and
59.
Note that the graphical data rendered by each pipeline
56-
59 defines
a portion of the overall image to be displayed within region
249. Thus,
it is not necessary for each pipeline
56-
59 to render all of the
graphical data defining the entire 3D image to be displayed in region
249.
Indeed, in the preferred embodiment, each slave pipeline
56-
59 preferably
discards the graphical data that defines a portion of the image that is outside
of the pipeline's responsibility. In this regard, each pipeline
56-
59
receives from master pipeline
55 the graphical data that defines the 3D
image to be displayed in region
249. Each pipeline
56-
59,
based on the aforementioned inputs received from slave controller
261 then
determines which portion of this graphical data is within pipeline's responsibility
and discards the graphical data outside of this portion.
For example, as described previously, slave pipeline
56 is responsible
for rendering the graphical data defining the image to be displayed within portion
266 of FIG. 8. This portion
266 includes graphical data associated
with screen relative coordinates (700, 1000) to (1000, 1300). Thus, any graphical
data having screen relative coordinates outside of this range is discarded by the
pipeline
56, and only graphical data having screen relative coordinates
within the foregoing range is rendered to frame buffer
66.
Furthermore, slave pipeline
57 is responsible for rendering the
graphical data defining the image to be displayed within portion
267 of
FIG. 8. This portion
267 includes graphical data associated with screen
relative coordinates (1000, 1000) to (1300, 1300). Thus, any graphical data having
screen relative coordinates outside of this range is discarded by the pipeline
57, and only graphical data having screen relative coordinates within the
foregoing range is rendered to frame buffer
67.
In addition, slave pipeline
58 is responsible for rendering the graphical
data defining the image to be displayed within portion
268 of FIG. 8. This
portion
268 includes graphical data associated with screen relative coordinates
(700, 700) to (1000, 1000). Thus, any graphical data having screen relative coordinates
outside of this range is discarded by the pipeline
58, and only graphical
data having screen relative coordinates within the foregoing range is rendered
to frame buffer
68.
Also, slave pipeline
59 is responsible for rendering the graphical data
defining the image to be displayed within portion
269 of FIG. 8. This portion
269 includes graphical data associated with screen relative coordinates
(1000, 700) to (1300, 1000). Thus, any graphical data having screen relative coordinates
outside of this range is discarded by the pipeline
59, and only graphical
data having screen relative coordinates within the foregoing range is rendered
to frame buffer
69.
To increase the efficiency of the system
50, each slave pipeline
56-
59
preferably discards the graphical data outside of the pipeline's responsibility
before significantly processing any of the data to be discarded. Bounding box techniques
may be employed to enable each pipeline
56-
59 to quickly discard
a large amount of graphical data outside of the pipeline's responsibility before
significantly processing such graphical data.
In this regard, each set of graphical data transmitted to pipelines
56-
59
may be associated with a particular set of bounding box data. The bounding box
data defines a graphical bounding box that contains at least each pixel included
in the graphical data that is associated with the bounding box data. The bounding
box data can be quickly processed and analyzed to determine whether a pipeline
56-
59 is responsible for rendering any of the pixels included within
the bounding box. If the pipeline
56-
59 is responsible for rendering
any of the pixels included within the bounding box, then the pipeline
56-
59
renders the received graphical data that is associated with the bounding box. However,
if the pipeline
56-
59 is not responsible for rendering any of the
pixels included within the bounding box, then the pipeline
56-
59
discards the received graphical data that is associated with the bounding box,
and the pipeline
56-
59 does not attempt to render the discarded graphical
data. Thus, processing-power is not wasted in rendering any graphical data that
defines an object outside of the pipeline's responsibility and that can be discarded
vi
