USOO6999.

093B1

(12) United States Patent (10) Patent No.: US 6,999,093 B1

Wetzel et al. (45) Date of Patent: Feb. 14, 2006

(54) DYNAMIC TIME-OF-DAYSKY BOX (56) References Cited

LIGHTING
U.S. PATENT DOCUMENTS
(75) Inventors: Mikey S. Wetzel, Redmond, WA (US); 5,714,997 A 2/1998 Anderson .................... 348/39
David Scott McCoy, Bellevue, WA 6,577.351 B1 * 6/2003 Yu et al................ ... 348/624
(US) 6.634,947 B1 * 10/2003 Miyamoto et al. ............ 463/36
6,661,918 B1 * 12/2003 Gordon et al. ........ ... 382/173
(73) Assignee: Microsoft Corporation, Redmond, WA 2003/019411.0 A1 * 10/2003 Brodsky ........... ... 382/103
(US) 2004/O080507 A1 4/2004 Von Prittwitz. .............. 345/419

(*) Notice: Subject to any disclaimer, the term of this * cited by examiner
patent is extended or adjusted under 35 Primary Examiner Matthew C. Bella
U.S.C. 154(b) by 434 days. Assistant Examiner Wesner Sajous
(74) Attorney, Agent, or Firm-Ronald M. Anderson
(21) Appl. No.: 10/338,940
(57) ABSTRACT
(22) Filed: Jan. 8, 2003
A background (sky box) in a video game or other virtual
(51) Int. Cl. reality environment dynamically changes as a function of a
G06T I7/00 (2006.01) parameter, producing a more realistic display. Luminance
G09G 5/00 (2006.01) only texture and color texture data are employed to create
H04N 9/66 (2006.01) the background. The color texture data are provided as a
H04N 9/65 (2006.01) lookup table in which one axis of the lookup table corre
H04N 9/74 (2006.01) sponds to time or another variable, and the other axis
indicates the colors of pixels in the display as a function of
(52) U.S. Cl. ....................... 345/581; 345/428; 34.5/582; the altitude or azimuth of the pixels. The luminance-only
34.5/586; 34.5/589; 34.5/630; 34.5/633; 348/586; texture data are thus modulated by the chrominance values
348/638; 348/642 from the color texture data for pixels in the background.
(58) Field of Classification Search ................. 345/473, Optionally, one of a plurality of two-dimensional color
345/474,629–630, 633-634, 636, 638, 589, textures can be Selected as a function of the variable and
345/586, 596-597, 601-602, 643, 581–583, applied to the luminance-only texture data.
345/418-428,587; 348/553,557, 563-566,
348/577–578,584,586, 599, 638, 642, 136, 21 Claims, 9 Drawing Sheets
348/671, 708, 624; 382/162-167, 169,171,
382/172-173, 103: 463/31-36
See application file for complete Search history. (4 of 9 Drawing Sheet(s) Filed in Color)

StArt

LOAO SKY BOX 322

GEOMETRY

OAO GRAYSCALE SKY 324

BOXTEXTUREDATA

LOAD COLOR 328

TEXTUREDATA

DETERMINE 328
WIEWPOINT

SIM WArAble to 330

RANGEFROMOTO

SETTEXTURE OTOE 332

GRAYSCALE

SETTEXURE TO BE 334
COLORTEXTUREDATA

DETERMINE CURRENT 336

TIME WARIABLE

RNDERSKYBOX 338
GEOMRYBASED ON
TIME WARIABLE

32O
U.S. Patent Feb. 14, 2006 Sheet 1 of 9 US 6,999,093 B1

E_LOWRH

"IOHSETACJWN
U.S. Patent Feb. 14, 2006 Sheet 2 of 9 US 6,999,093 B1

(DISPLAY'S VIDEO
GAME IMAGES)

1 O6

108

142B

14.4B

FIG. 2
U.S. Patent Feb. 14, 2006 Sheet 3 of 9 US 6,999,093 B1

O NEE PIXEL WIE

GRAY SCALETEXRE DATA

FIG. 3
All DE

2O6

TIME

3SODEGREES
APPLICATION OF COLOR
EXTURE DATA A ME "" O
GRAY SCALE TEXTURE DATA
FIG. 5
U.S. Patent Feb. 14, 2006 Sheet 4 of 9 US 6,999,093 B1

START 3O2

GENERATE 3D TERRAN
FOR SKY BOX

306
CREATE BACKGROUND ON EACH
SIDE OF SKY BOX (i.E., EACH
SIDE OF CUBE) USING GRAPHICS
PROGRAM

STRIP COLOR DATA FROM

BACKGROUND, YELDING GRAY
SCALE IMAGES FOR SKY BOX
(I.E., EACHFACE OF CUBE)

310
GENERATE MESH WITH
VERTICES OF SKY BOX (I.E., AT
EACH CORNER OF CUBE)

312
GENERATE COLORTEXTURE
LOOK UP TABLE FOR SKY BOX,
AS FUNCTION OF TIME (OR
OTHER PARAMETER)

300 DONE

FIG. 6
U.S. Patent Feb. 14, 2006 Sheet 5 of 9 US 6,999,093 B1

LOAD SKY BOX N-322

GEOMETRY

LOAD GRAYSCALE SKY 324

BOXTEXTURE DATA

326
LOAD COLOR
TEXTURE DATA

DETERMINE 328
VIEWPOINT

SET TIME WARIABLE TO 330

RANGE FROM O TO 1

SET TEXTURE O TO BE 332

GRAYSCALE

334
SET TEXTURE 1 TO BE
COLORTEXTURE DATA

DETERMINE CURRENT 336

TIME VARIABLE

RENDER SKY BOX 338

GEOMETRY BASED ON
TIME WARIABLE

32O
FIG. 7
U.S. Patent Feb. 14, 2006 Sheet 6 of 9 US 6,999,093 B1

HARDWARE
START 350 RASTERIZES 368
(RENDERING) POLYGONS INTO
SCREEN PIXELS
370

BREAK GEOMETRY 352 FOREACH PIXEL

NTO INDIVIDUAL
VERTICES
372
354 FETCHTEXEL
FROM TEXTURE O
FOREACH AT COORDNATES
VERTEX TEXO.UV
356
374
FETCHTEXEL
TRANSFORM VERTEX
FROM TEXTURE 1
FROM 3D VIEW SPACE
AT COORDNATES
TO 2D SCREEN SPACE
EX 1. UV
(USINGVIEWPOINT) 376
358
MODULATE BOTH
USE TEXTURE TEXTURESTOGETHER 8.
COORDINATES FOR OUTPUT PIXEL
VERTEX FROM TEXTURE O PXEL = TEXELO * TEXEL 1
TEX O.U = WTXU
TEX O.W = WXV
360
GENERATE TEXTURE
COORDINATES
TEX 1U = TIME WARIABLE
SET EARLIER
TEX 1.V = VTXV

362 364

NEXT
NO VFEx
YES 366 FIG. 8
HARDWARE
ASSEMBLES VERTICES
N POLYGONS
U.S. Patent Feb. 14, 2006 Sheet 7 of 9 US 6,999,093 B1

COUMNS
40a 40 400c 40:

398

FIG. 9
U.S. Patent Feb. 14, 2006 Sheet 8 of 9 US 6,999,093 B1

4O42d

4034)2a

FIG. It
U.S. Patent Feb. 14, 2006 Sheet 9 of 9 US 6,999,093 B1

41)

Coor textures ate nifferee since title of

day color effects are syfainericai.
Birroring erasures texture
edges match and saves memory.
4) 40

42
 US 6,999,093 B1
1 2
DYNAMIC TIME-OF-DAY SKY BOX SUMMARY OF THE INVENTION
LIGHTING One aspect of the present invention is directed to a method
for displaying a background in a virtual environment So that
FIELD OF THE INVENTION the background varies dynamically. The Virtual environment
The present invention generally relates to background might be an electronic game or other form of a virtual or
imaginary display in which Sky box lighting is typically
lighting for use in graphically rendering a virtual used. The program that implements the virtual reality will
environment, and more Specifically, to dynamically varying include geometry data that define the background, gray Scale
the lighting on the background in a Video game or virtual texture data for the background (i.e., luminance-only texture
environment as a function of time or of Some other Specified data), and color texture data for the background. The color
variable.
texture data include chrominance values for the background
BACKGROUND OF THE INVENTION that vary as a function of time (or as a function of another
Video games and Virtual environments in which Sky, parameter) in the virtual environment. As time (or the value
mountains, and other objects in the general background of a 15
of the other parameter) changes in the Virtual environment,
Scene must be portrayed are typically represented with appropriate corresponding chrominance values are loaded
geometry and textures that are Static and unchanging and from the color texture data for pixels comprising the back
thus often appear flat and unimpressive. The background of ground or sky box. The gray Scale texture data is then
a virtual environment is often referred to as a “sky box,” multiplied by the chrominance values for the pixels in the
because it usually includes at least Some portion of the Sky background to render the background of the virtual envi
and in three dimensions, Surrounds a virtual viewpoint like ronment on a display. As time (or another Selected
a cube or box. Objects Such as clouds may be Scrolled acroSS parameter) changes, the background in the virtual environ
the background to provide Some improvement in realism, ment dynamically changes in a generally realistic manner.
but the result can Still appear very unrealistic. For example, When time is the parameter that determines the dynami
if the Virtual reality is Supposed to represent the environment 25
cally changing background lighting, the color texture data
over a period of time during the day, an observer in the preferably comprise a color texture lookup table having a
Virtual environment would expect the lighting to change in first axis that varies with the passing of time in the Virtual
a realistic manner corresponding to that period of time. For environment, and a Second axis that is orthogonal to the first
example, there should be a Sunrise in the morning, which axis and along which the chrominance values define the
might appear as a yellow band along the horizon, with red color of pixels in either Successive rows or columns of the
mid bands, fading through a gradient into a dark blue Sky background.
above and gray to black in the foreground. At midday, the Since the game or other application using the present
Sky might appear a light blue color with darker blue above, invention may only be displaying a portion of the
and as evening approaches, the Sunset might be represented background, the method further includes the Step of deter
as a band of reddish orange fading into dark blue above. 35 mining the portion of the background being viewed, So that
Furthermore, the transition between these lighting condi the portion is rendered for viewing.
tions should change dynamically as time passes during the To provide the gray Scale texture data, it is preferable to
day. Since time can be compressed in a Video game or virtual Simply Strip out chrominance components from an image
environment, the change in lighting conditions in a Sky box depicting the background. The chrominance values that vary
will typically be readily apparent to a viewer, So that the 40 with the parameter can either be manually created or created
added realism can provide a much more Satisfying experi with the aid of an appropriate graphic tool or application
ence than the typical Static lighting background provided in program.
most games or virtual environment. In one embodiment, the color texture data are provided as
Currently, to achieve different light effects in a sky box, a plurality of two-dimensional (2D) color texture lookup
it is necessary to provide a bit map of the background for 45 tables, each corresponding to a different value of time or
each different lighting condition, which can quickly increase another parameter. The chrominance values defined along
the amount of Storage for the game files required on a user's one axis of each of the color texture lookup tables is applied
hard drive. Also, if only two or three different bit maps are to corresponding pixels in the gray Scale texture data as a
used, rendering each new bitmap does not provide a gradu function of a disposition of the pixels within the virtual
ally changing lighting in the sky box, Since as each bit map 50 environment, e.g., relative to the azimuth in the display, So
is rendered, there is a Sudden change in the appearance of the that the chrominance values applied to each pixel are
background as the lighting conditions and colors in one determined as a function of three-dimensional (3D) coordi
bitmap replace those of another. Such a Sudden change in the nates associated with each pixel. The 3D coordinates include
lighting will appear unrealistic. Alternatively, one bit map a value for the predefined parameter used to determine the
can be made to “cross fade' into another, but the result is 55 Specific color texture lookup table that will be used, and
again not very realistic, unless the changes between Succes coordinates for the disposition of the pixel relative to
Sive bit maps is made relatively slight, So that the transitions orthogonal axes in the gray Scale texture data and in the
are not apparent. However, as the number of bitmaps used color texture lookup table that is currently being used. In this
increases, the requirements for Storage Space for the bitmap embodiment, the color texture data are not necessarily
files also increases. In consideration of these problems, it 60 uniform as the colors extend around the background, but can
will be apparent that there is a need for a more efficient and instead include corresponding areas of the background that
more realistic method of providing sky box lighting that differ between the various color texture lookup tables. For
changes dynamically. It should not be necessary to provide example, the Sun can be shown in different positions at
different bitmaps of the background for each different time different times during the day, by depicting the Sun in the
of day that will be displayed. The approach employed should 65 color texture lookup tables at the different positions.
also be applicable to display backgrounds that are dynami Another aspect of the present invention is directed to a
cally varying in response to parameters other than time. memory medium having machine instructions Stored
 US 6,999,093 B1
3 4
thereon for carrying out the Steps of the method noted above. computer executable instructions, Such as program modules,
Yet another aspect of the present invention is directed to a which are executed by a personal computer (PC) or a gaming
System for providing a background for a virtual console. Generally, program modules include application
environment, where the background is dynamically varied programs, Such as computer Simulations, routines, objects,
as a function of a predefined parameter. The System includes components, functions, data structures, etc. that perform
a display, and a memory Stores a plurality of machine particular tasks or implement particular abstract data types.
instructions, gray Scale texture data for the background, and In addition, those skilled in the art will appreciate that this
color texture data for the background. The machine instruc invention may be practiced with other computer System
tions are executed by a processor, causing it to carry out configurations, particularly in distributed computing
Steps that are generally consistent with the Steps of the environments, where tasks are performed by remote pro
method discussed above. cessing devices that are linked through a communications
network. In a distributed computing environment, program
BRIEF DESCRIPTION OF THE DRAWING modules may be located in either or both local and remote
FIGURES memory Storage devices.
The file of this patent contains at least one drawing 15 With reference to FIG. 1, an exemplary system includes a
executed in color. Copies of this patent with color drawings general purpose computing device in the form of a conven
will be provided by the Patent and Trademark Office upon tional PC 20. PC 20 is provided with a processing unit 21,
request and payment of the necessary fee. a System memory 22, and a System buS 23. The System buS
The foregoing aspects and many of the attendant advan couples various System components, including the System
tages of this invention will become more readily appreciated memory, to processing unit 21 and may be any of Several
as the Same becomes better understood by reference to the types of bus structures, including a memory bus or memory
following detailed description, when taken in conjunction controller, a peripheral bus, and a local bus using any of a
with the accompanying drawings, wherein: variety of bus architectures. The System memory includes
FIG. 1 is a Schematic block diagram of an exemplary
read only memory (ROM) 24 and random access memory
computing environment Suitable for implementing the
25 (RAM) 25. Abasic input/output system 26 (BIOS), contain
present invention; ing the basic routines that help to transfer information
between elements within PC 20, such as during start up, is
FIG. 2 is an isometric view of an exemplary client Stored in ROM 24. PC 20 further includes a hard disk drive
electronic gaming System that includes a game console and 27 for reading from and writing to a hard disk (not shown)
Support for up to four user input devices, and may include a magnetic disk drive 28 for reading from
FIG. 3 is an exemplary luminance-only sky box texture or writing to a removable magnetic disc 29, and an optical
image; disk drive 30 for reading from or writing to a removable
FIG. 4 (color) is an exemplary color texture for use with optical disc 31, Such as a compact disc-read only memory
the luminance-only sky box texture image of FIG. 3, in (CD-ROM) or other optical media. Hard disk drive 27,
accord with the present invention; 35 magnetic disk drive 28, and optical disk drive 30 are
FIG. 5 (color) is a resulting sky box lighting using a connected to system bus 23 by a hard disk drive interface 32,
selected column within the color lookup texture of FIG. 4, a magnetic disk drive interface 33, and an optical disk drive
applied to the luminance-only Skybox texture image of FIG. interface 34, respectively. The drives and their associated
3; computer readable media provide nonvolatile Storage of
FIG. 6 is a flow chart showing the logical Steps imple
40 computer readable machine instructions, data structures,
mented to create a luminance-only Sky box texture and program modules, and other data for PC 20. Although the
corresponding color texture, for use in accord with the exemplary environment described herein includes a hard
present invention; disk, removable magnetic disc 29, and removable optical
disc 31, it will be appreciated by those skilled in the art that
FIG. 7 is a flow chart showing the logical steps imple 45 other types of computer readable media, which can Store
mented to dynamically vary the Skybox lighting in response data that are accessible by a computer, Such as magnetic
to a variable; cassettes, flash memory cards, digital versatile discs
FIG. 8 is a flow chart showing the logic for rendering the (DVDs), Bernoulli cartridges, RAMs, ROMs, and the like,
sky box geometry, in accord with the present invention; may also be used in the exemplary operating environment.
FIG. 9 (color) is an array of a portion of a plurality of 2D 50 A number of program modules may be stored on the hard
time of day color textures for use with a luminance-only Sky disk, magnetic disc 29, optical disc 31, ROM 24 or RAM 25,
box texture to produce sky box lighting that varies in 3D including an operating System 35, one or more application
aspect and changes based upon the time of day; programs 36, other program modules 37, and program data
FIG. 10 (color) is a more complete set of the 2D time of 38. A user may enter commands and information into PC 20
day color textures, illustrated in 3D aspect to Show how each 55 through input devices Such as a keyboard 40 and a pointing
axis of the color textures are mapped; and device 42. Pointing device 42 may include a mouse, Stylus,
FIG. 11 (color) illustrates how a 2D color texture is wireleSS remote control, or other pointer. Other input devices
mirrored and Stretches around a horizon. (not shown) may include a joystick, game pad, wheel, pedal,
microphone, Satellite dish, Scanner, digital camera, digital
DESCRIPTION OF THE PREFERRED 60 Video recorder, or the like. These and other input/output
EMBODIMENT (I/O) devices are often connected to processing unit 21
Exemplary Computing Environment through an I/O interface 46 that is coupled to the system bus
FIG. 1 and the following discussion are intended to 23. The term I/O interface is intended to encompass each
provide a brief, general description of a Suitable computing interface Specifically used for a Serial port, a parallel port, a
environment in which Software implementing the present 65 game port, a keyboard port, and/or a universal Serial bus
invention may be implemented. Although not required, the (USB). A monitor 47 or other type of display device is also
present invention will be described in the general context of connected to System buS 23 via an appropriate interface,
 US 6,999,093 B1
S 6
Such as a Video adapter 48, and is usable to display appli extraction of storage disc 108, so that the digital data on the
cation programs, Web pages, a simulated environment in disc can be read for use by the game console.
which the present invention is implemented, and/or other Game console 102 connects to a television 90 or other
information. In addition to the monitor, PCs are often display monitor or Screen via audio/visual (A/V) interface
coupled to other peripheral output devices (not shown), Such cables 120. A power cable plug 122 conveys electrical power
as speakers (through a Sound card or other audio interface to the game console when connected to a conventional
(not shown)), and printers. alternating current line Source (not shown). Game console
In a networked environment, PC 20 can be logically 102 includes an Ethernet data connector 124 to transfer and
connected to one or more other computers, Such as a remote receive data over a network (e.g., through a peer-to-peer link
computer 49. Remote computer 49 may be another PC, a to another game console or through a connection to a hub or
Server, a router, a network PC, a peer device, a Satellite, or a Switch (not shown)), or over the Internet, for example,
other common network node comprising a client computer. through a connection to an xDSL interface, a cable modem,
Remote computer 49 also be a game console, like that or other broadband interface (not shown). Other types of
described in greater detail below, and may include many or game consoles may be coupled together in communication
all of the elements described above in connection with PC 15 using a conventional telephone modem.
20, or may include typical elements of other electronic Each controller 104a and 104b is coupled to game console
computing devices. 102 via a lead (or alternatively, through a wireless interface).
So as not to make FIG. 1 unnecessarily complex, remote In the illustrated implementation, the controllers are univer
computer 49 is shown with only an external memory Storage Sal Serial port (USB) compatible and are connected to game
device 50. The logical connections depicted in FIG. 1 console 102 via USB cables 130. Game console 102 may be
include a local area network (LAN) 51 and a wide area equipped with any of a wide variety of user devices for
network (WAN) 52, which may encompass the Internet. interacting with and controlling the game Software. AS
Such networking environments are common in offices, illustrated in FIG. 2, each controller 104a and 104b is
enterprise wide computer networks, intranets, and in regard equipped with two thumbsticks 132a and 132b, a D-pad 134,
to computing devices coupled to the Internet. 25 buttons 136, and two triggers 138. These controllers are
When used in a LAN networking environment, PC 20 is merely representative, and other gaming input and control
connected to LAN 51 through a network interface or adapter devices may be substituted for or added to those shown in
53. When used in a WAN networking environment, PC 20 FIG. 2 for use with game console 102.
typically includes a modem 54, or other means Such as a A removable function unit 140 can optionally be inserted
cable modem, Digital Subscriber Line (DSL) interface, or an into controller 104 to provide additional features and func
Integrated Service Digital Network (ISDN) interface, for tions. For example, a portable memory unit (MU) enables
establishing communications over WAN 52. One type of users to Store game parameters and port them for play on
WAN commonly used for communication between remote other game consoles, by inserting the portable MU into a
computing devices is the Internet. Modem 54, which may be controller connected to the other game console. Another
internal or external, is connected to the System buS 23 or 35 removable functional unit comprises a voice communication
coupled to the bus via I/O device interface 46, i.e., through unit that enables a user to Verbally communicate with other
a Serial port. In a networked environment, program modules users locally and/or over a network. Connected to the Voice
depicted relative to PC 20, or portions thereof, may be stored communication unit is a headset 142, which includes a boom
in the remote memory Storage device. It will be appreciated microphone 144. In the described implementation, each
that the network connections shown are exemplary and other 40 controller is configured to accommodate two removable
means of establishing a communications link between the function units, although more or fewer than two removable
computerS may be used, Such as wireleSS communication function units or modules may instead be employed.
and wide band network links. Gaming System 100 is capable of playing, for example,
Exemplary Gaming System games, music, and Videos. It is contemplated that other
FIG. 2 illustrates an exemplary electronic gaming System 45 functions can be implemented using digital data Stored on
100, which includes a game console 102 and support for up the hard disk drive or read from optical storage disc 108 in
to four user input devices, Such as controllers 104a and drive 106, or using digital data obtained from an online
104b. Of course game consoles Supporting only a Single user Source, or from the MU. For example, gaming system 100
input device, or additional user input devices, can also be is capable of playing:
employed. Game console 102 is equipped with an internal 50 Game titles stored on CD and DVD discs, on the hard disk
hard disk drive (not shown) and includes a portable media drive, or downloaded from an online Source;
drive 106 that supports various forms of portable optical Digital music stored on a CD in portable media drive 106,
Storage media, as represented by an optical Storage disc 108. in a file on the hard disk drive (e.g., WINDOWS
Examples of suitable portable storage media include DVD MEDIA AUDIOTM (WMA) format), or derived from
discS and CD-ROM discs. In this gaming System, game 55 online Streaming Sources on the Internet or other net
programs are preferably distributed for use with the game work; and
console on DVD discs, but it is also contemplated that other Digital AV data such as movies that are stored on a DVD
Storage media might instead be used on this or other types disc in portable media drive 106, or in a file on the hard
of clients in connection with the present invention. disk drive (e.g., in an Active Streaming Format), or
On a front face of game console 102 are four ports 110 for 60 from online Streaming Sources on the Internet or other
connection to Supported controllers, although the number network.
and arrangement of these ports may be modified. A power Exemplary Illustration of a Sky Box with Dynamic Lighting
button 112, and an eject button 114 are also disposed on the To enable a reader to better understand the present
front face of game console 102. Power button 112 controls invention, it should be helpful to illustrate how the invention
application of electrical power to the game console, and 65 is used to create dynamically changing lighting for an
eject button 114 alternately opens and closes a tray (not exemplary sky box. Accordingly, in FIG. 3, a gray Scale
shown) of portable media drive 106 to enable insertion and texture or luminance-only texture 200 is illustrated that is
 US 6,999,093 B1
7 8
Suitable for use in carrying out the present invention. along that one dimension of the background change as a
Typically, an artist will manually prepare a full color back function of a Specified parameter. Typically, the parameter
ground for a virtual environment, Such as those used in an will be time, as shown in an exemplary color texture table
electronic game with the aid of a conventional graphic art 202 in FIG. 4. In this exemplary color texture table, it is
application, such as Microsoft Corporation's PAINT assumed that the color texture that will applied to the Sky
BRUSHTM program. It will be understood that graphic art box in a direction corresponding to altitude varies dynami
applications are available from other Sources that can also be cally with the passage of time (along the horizontal axis of
used for preparing a graphic image Suitable for use as a the color texture table), during a predefined period. For
background or sky box in a virtual environment. AS an example, the period could be an entire 24-hour day, So that
alternative, the background for a virtual environment can be the color texture table includes color textures that change
captured using one or more digital cameras to produce dynamically over the time from full darkneSS during the very
images of a Scene that is either artificially created, or which early morning, through Sunrise, midday, Sunset, and full
exists at an actual site. Further, the background need not be darkness late at night. However, the time period might only
developed for all directions around a viewpoint, if the be for a specified number of hours during the day or during
application for which the sky box is to be used limits the 15 the night. Also, other parameters beside time can be used to
Viewpoint to less than all directions. Traditionally, the back control the color texture data applied to the Sky box. For
ground for a virtual environment is developed for each side example, the color texture table could include a plurality of
of a six-sided cube Surrounding a viewpoint in the Virtual different color textures corresponding to different weather
environment. This approach has led to the term “sky box” conditions in the background, including very overcast,
being used for the background, Since the background is Stormy conditions, bright Sunny Sky conditions, partially
represented by the internal images applied to Six Sides of cloudy conditions, etc.
Such a cube Surrounding a viewpoint. However, the back In regard to FIG. 4, Substantially greater resolution in the
ground can instead be represented as an image on the inner color texture applied at a given time of the day is provided
Surface of a sphere Surrounding the viewpoint within the than may be necessary. In Some instances, the number of
Virtual environment. Again, if the application does not 25 variations in the color texture as a function of the parameter
require that a background be provided for the entire inner can be relatively few in number. However, in the example of
Surface of a sphere Surrounding the viewpoint, it may be FIG. 4, the color texture applied to dynamically vary the sky
necessary to develop the background image for only a box lighting is resolved to a column 204 that is only one
portion of the inner Surface of the Sphere. pixel wide. Column 204 changes with the value of the
In most applications, the background on one side of a parameter, which in FIG. 4 is the time “T” in the virtual
cube can be replicated for the other Sides of the cube, Since environment. Thus, at a time T, column 204 indicates the
there will typically not be any specific landmark in the color values that are applied to the pixels as a function of
background, which would appear Strange if replicated in altitude to luminance-only texture 200 in FIG. 3. Sky box
each of the four sides of the cube. For example, if the lighting 206 in FIG. 5 is the result of applying the color
background includes mountains that are in the distance, it 35 values in column 204 at time T to luminance-only texture
would not be unusual for Similar mountains to Surround the 200 in FIG. 3.
Viewpoint on all four Sides. But, if the background includes It will be apparent that in the example shown for this
a predominant peak or other distinctive landmark on one embodiment, the color at each Successively higher altitude
Side, it would be unrealistic to include the Same distinctive in color texture table 202 extends horizontally across the
landmark on all four Sides. Also, Since the background is 40 corresponding gray Scale luminance-only texture. However,
often hidden by haze, details of the background are generally application of the same color to portions of the Sky and the
not critical, and the lack of Such details do not cause an mountains does not cause the background to be unrealistic,
observer to conclude that the background is any leSS realistic Since this effect is often observed in nature due to haze in the
because it does not include Such details. distant background. Indeed, the relatively low resolution
Since haze often obscures the background, the lighting 45 required for a virtual environment background lends itself to
conditions applied to the background can generally be this technique for dynamically varying the color applied to
uniform, even where the same light color extends acroSS the sky box as a function of a parameter Such as time. Since
portions of the Sky and portions of the background Such as an observer in a game tends to be viewing the background
mountains. An observer would generally expect that the over only a portion of its full extent, use of a Sunrise color
haze in front of mountains would cause a purple sky in the 50 that extends in a band 360 degrees, completely around the
background to cause the mountains to appear purple also. horizon, compared to a more realistic Sunrise that is limited
Since the background is not normally the Subject of close to only a portion of the horizon on the East side of the sky
observation by a player in a game, the player's concentration box will not be viewed as unrealistic to the observer.
on action that is in the foreground makes details in the Moreover, another embodiment of the present invention
background much less important. 55 addresses this issue, as explained below. Also, for greater
Once the background for the virtual environment has been realism, other objects (e.g., moving sprites) can be Super
created, the chrominance data is removed from the imposed over the background, Such as clouds, aircraft, birds,
background, leaving the luminance-only texture as shown in etc., to provide a more realistic impression of a virtual
the example of FIG. 3. Next, an illustrator will prepare a environment.
color texture table for a location within the virtual environ 60 In FIG. 6, a flow chart 300 illustrates the steps employed
ment. In a game, it is possible that a number of Scenes at to produce the gray Scale or luminance-only texture, and the
different locations will be required, and an appropriate color texture lookup table for use with the present invention.
background image and color texture table will be required From a start block 302, the logic proceeds to generating a 3D
for each different Scene. The color texture table has an axis terrain for the sky box in a step 304. As noted above, this
along which is depicted the color applied to pixels of the 65 Step involves an artist using appropriate tools for generating
background relative to one dimension of the background, the 3D terrain image for the background of the virtual
and an axis along which the colors applied to the background environment, whether for a virtual environment on the earth,
 US 6,999,093 B1
9 10
or in Some other location, Such as Outer Space. In a Step 306, location of the corner vertices of the cube, in regard to the
using Such a tool, the artist creates a background for each luminance-only texture. In a step 354, for each vertex, Steps
Side of a sky box cube. Alternatively, the artist may produce 356,358, 360,362, and 364 are carried out, until all vertices
a spherical background, as noted above. In a step 308, using have been processed. In Step 356, the program transforms a
a Suitable computer graphics program, the color data are current vertex from 3D view Space into 2D Screen Space
Stripped from the background image, yielding the relative to the viewpoint of the observer. Next, step 358 uses
luminance-only texture for each Side of the sky box or each the texture coordinates for the current vertex from texture
face of the cube Surrounding an observer Viewpoint. The Zero (the luminance-only texture), Setting the variable TEX
luminance-only texture data represent the original in Shades 0.U=to VTX.U and TEX 0.V=to VTX.V. A step 360 gener
of gray, without any chrominance information. ates the texture coordinates as a function of the variable that
Next, in a step 310, the user generates a mesh having was Set earlier. In this case, it is assumed that the variable is
vertices of the sky box. For example, the vertices for the time, so TEX 1.U equals the time variable set earlier and
corners of the skybox cube may be included in the mesh to TEX 1.V equals VTX.V. A decision step 362 determines if
provide a reference for the different sides of the sky box the last vertex has been processed, and if not, a step 364
when rendering the sky box in the Virtual environment. 15 moves to the next vertex, enabling the process to loop back
Finally, in a step 312, the color texture lookup table data are to step 354.
generated for the sky box wherein the different colors Once all of the Vertices have been processed by Setting up
applied along an axis, Such as in regard to the altitude of the the luminance-only texture data for the vertices, the logic
background image, vary as a function of a Selected proceeds to a step 366, wherein the hardware (e.g., a video
parameter., e.g., time. In non-terrestrial virtual adapter card-not shown) assembles the vertices into poly
environments, it may be preferable to have color vary as a gons. A Step 368 provides for using the hardware to rasterize
function of angular displacement around the observer, the polygons into Screen pixels for use in displaying the
resulting in color bands that extend Vertically in contrast to background on a display monitor.
the examples shown in FIGS. 4 and 5, where the color bands As indicated in a block 370, steps 372,374,376,378, and
extend horizontally. In most terrestrial backgrounds, the 25 380 are carried out for each pixel in the display. Step 372
color bands will extend horizontally, and different colors provides for fetching a texel from texture 0 (the gray Scale
will be arranged as a function of altitude (i.e., vertical texture) at coordinates TEX 0.UV. A “texel” is the Smallest
position) in columns that are selected as a function of the addressable unit of a texture map. In the examples shown in
value of the predefined parameter. FIGS. 3-5, each texel corresponds to a pixel. However, it is
Once the gray Scale texture and color texture data have also contemplated that in many instances, less resolution
been developed as described above, they are made available will be required for the color texture so that a texel may
within a virtual environment application program such as a correspond to a group of pixels in the display.
game So that the sky box lighting changes dynamically as a Step 374 fetches the texel from texture 1 (the color
function of the value of the Selected parameter. texture) at coordinates TEX 1. UV so that the texels from
FIG. 7 illustrates a flow chart 320 in which the steps 35 texture 0 correspond to those from texture 1. Step 376 then
applied for using the gray Scale texture and color texture data modulates the textures together to produce an output pixel
in accordance with the present invention are illustrated. equal to a product of texel 0 and texel 1. Decision step 378
After the start box, a step 322 provides for loading the sky determines if the last pixel in the display being rendered has
box geometry, e.g., as defined by the Vertices of the cube been processed, and if not, the logic proceeds to a step 380
corners. Next, a step 324 loads the gray Scale or luminance 40 to process the next pixel. After all pixels have been
only texture data that were created as described above. In a processed, the logic required for rendering the Sky box is
Step 326, the color texture data that will be used for applying completed.
chrominance values to the luminance-only texture are 3D Texture Map Embodiment
loaded. FIG. 9 illustrates a plurality of color textures that are used
To determine the side or sides of the sky box that will be 45 for a different embodiment of the present invention in which
visible to the observer in a virtual environment, a step 328 the color applied to the luminance-only texture can differ
determines the viewpoint of the observer. Furthermore, to two-dimensionally within the image as well as in response
determine the portion of the color texture data that will be to the value of a parameter such as time. Color textures 398
used, a variable must be set and normalized to range from a represent only a few of the color textures that might be used
value of Zero to one, as indicated in a step 330. A step 332 50 for a particular Sky box of this type, which are arranged in
provides for Setting texture 0 to be equal to the gray Scale or columns 400a, 400b, 400c, and 400dand in rows 402a, 402b,
luminance-only texture data that were loaded in Step 324. 402c, and 402d. By displaying the color textures in this
Similarly, in step 334, texture 1 is set equal to the color manner, it is easy to see the differences between them.
texture data loaded in step 326. In a step 336, the program In FIG. 10, more of the color textures are illustrated in a
determines the current value for time or the value of the 55 stacked array 406 showing the relationship of the color
another variable that will determine the specific portion of textures in 3D. The dimensions in FIG. 10 include altitude,
the color texture data that will be applied to the luminance azimuth, and time of day (or another parameter). Thus, in
only texture. Finally, in a step 338, the program renders the this embodiment, instead of Selecting a single column of
sky box geometry based on the time or other variable that pixels corresponding to a parameter Such as the time of day,
was Selected in Step 336, using the corresponding color 60 a different color texture from the stack 406 is selected as a
texture data for that time. The result is dynamically variable function of the value of the time of day or other parameter.
sky box lighting for the Virtual environment, for example, Furthermore, as shown in FIG. 9, the color textures differ at
like that shown in FIG. 5. different times of the day, for example, to show a Sunrise at
Details of step 338 are illustrated in FIG.8. Following a row 402a column 400d, and row 402b column 400a.
start box 350 for the rendering process, a step 352 provides 65 Similarly, a Sunset is most evident at a row 402c column
for breaking the geometry into individual vertices. In this 400c. This embodiment addresses the problem noted above,
Step, the Surfaces of the cubes are delineated based upon the wherein changes in lighting that are unrealistic, because they
 US 6,999,093 B1
11 12
extend completely around the observer instead of being 2. The method of claim 1, wherein the color texture data
disposed at an expected azimuth location. Using the present comprise a color texture lookup table having a first axis
embodiment, Sky box lighting can be more realistically along which the value of the predefined variable varies, and
portrayed. a Second axis that is orthogonal to the first axis and along
FIG. 11 provides an example wherein the color texture which the chrominance values define the colors of pixels
410 (corresponding to color texture at row 402b, column extending acroSS one dimension of the background.
400a in FIG. 9) is mirrored, producing a color texture 410. 3. The method of claim 1, further comprising the step of
Since time of day color effects are generally Symmetrical determining a viewpoint within the Virtual environment to
about a center line 412, mirroring the color textures in this determine a portion of the background to display.
way ensures that the edges match while Saving memory, 4. The method of claim 1, wherein the step of providing
since only color texture 410 needs to be saved in memory. the gray Scale texture data comprises the Step of Stripping out
chrominance components from an image depicting the
In the example shown in FIG. 11, a selected color texture (at background, producing the gray Scale texture data.
Sunrise) is rendered with the luminance-only texture data 5. The method of claim 1, wherein the predefined param
providing a ring 414 that extends around a viewpoint (not eter comprises time, and wherein the Step of Supplying the
shown) in the virtual environment. Thus, in this example, the 15 value for the predefined parameter comprises the Step of
color is not uniform along bands that extend horizontally. indicating the time varying during a time interval.
Instead, the observer Sees the Sunrise in the east as would be 6. The method of claim 1, wherein the color texture data
expected rather than extending a full 360 degrees around the define bands of color in the background, where Said bands
observer. of color change as a function of the value of the predefined
It will be understood that parameters other than time can parameter.
be applied in this embodiment So that a corresponding color 7. The method of claim 1, wherein the color texture data
texture can be used to represent each different value of the comprise a plurality of color texture lookup tables, each
parameter. For example, different color textures could be color texture lookup table corresponding to a different value
employed to represent various Stages of a forest fire or other of the predefined parameter, further comprising the Step of
occurrence Visible in the background in a virtual environ 25 applying the chrominance values defined along one axis of
ment. Similarly, different conditions of weather can be each of the color texture lookup tables to corresponding
represented with color textures. Thus, one color texture may pixels in the gray Scale texture data as a function of an
include a lightening flash that is visible in a certain portion angular disposition of the pixels within the Virtual
of the virtual environment background relative to azimuth environment, So that the chrominance values applied to each
and altitude. The present embodiment thus provides consid pixel are determined as a function of three-dimensional
erably more versatility and enables the background to be coordinates associated with each pixel, Said three
dimensional coordinates including a value for the predefined
depicted in a more realistic manner than the first embodi parameter that determines the color texture lookup table that
ment.
will be used and a disposition of the pixel relative to
Although the present invention has been described in orthogonal axes in the gray Scale texture data and in the
connection with the preferred form of practicing it, those of 35 color texture lookup table that is currently being used.
ordinary skill in the art will understand that many modifi 8. A memory medium having machine instructions Stored
cations can be made thereto within the Scope of the claims thereon for carrying out the Steps of claim 1.
that follow. Accordingly, it is not intended that the Scope of 9. A method for displaying a background in a virtual
the invention in any way be limited by the above description, environment So that the background varies dynamically as a
but instead be determined entirely by reference to the claims 40 function of time, comprising the Steps of:
that follow. (a) providing:
The invention in which an exclusive right is claimed is (i) geometry data that define the background;
defined by the following: (ii) gray Scale texture data for the background;
1. A method for creating a background for display in a (iii) color texture data for the background, said color
Virtual environment, where the background is dynamically 45 texture data defining chrominance values that vary as
variable, comprising the Steps of: a function of time in the virtual environment;
(a) providing: (b) as time passes in the virtual environment, determining
(i) geometry data that define the background; corresponding chrominance values from the color tex
(ii) gray Scale texture data for the background; ture data, for pixels comprising the background in the
(iii) color texture data for the background, said color 50 Virtual environment;
texture data defining chrominance values that vary as (c) multiplying the gray Scale texture data by the chromi
a function of a predefined parameter; nance values to determine data defining pixels in the
(b) Supplying a value for the predefined parameter, said background; and
value dynamically changing, (d) using data for the pixels for rendering the background
(c) using the value Supplied for the predefined parameter
55 in the virtual environment, So that the background
to determine chrominance values from the color texture
dynamically changes as time passes in the virtual
environment.
data for pixels comprising the background, where spe 10. The method of claim 9, wherein the color texture data
cific pixels correspond to specific portions of the color comprise a color texture lookup table having a first axis that
texture data; 60 varies with the passing of time in the virtual environment,
(d) modulating the gray Scale texture data for Successive and a Second axis that is orthogonal to the first axis and along
pixels with the chrominance values determined for the which the chrominance values define the color of pixels in
pixels, and one of rows and columns within the background.
(e) using results from the step of modulating, Successively 11. The method of claim 9, further comprising the step of
rendering the background in the virtual environment, So 65 determining a portion of the background being viewed in the
that the background dynamically changes as a function Virtual environment, So that Said portion is rendered for
of the value for the predefined parameter. Viewing.
 US 6,999,093 B1
13 14
12. The method of claim 9, wherein the step of providing applying the chrominance values defined along one axis of
the gray Scale texture data comprises the Step of Stripping each of the color texture lookup tables to corresponding
away chrominance components from an image depicting the pixels in the gray Scale texture data as a function of an
background, producing the gray Scale texture data. angular disposition of the pixels within the Virtual
13. A memory medium having machine instructions environment, So that the chrominance values applied to each
Stored thereon for carrying out the Steps of claim 9. pixel are determined as a function of three-dimensional
14. A System for providing a background for a virtual coordinates associated with each pixel, Said three
environment, where the background is dynamically varied dimensional coordinates including a value for the predefined
as a function of a predefined parameter, comprising: parameter that determines the color texture lookup table that
(a) a display; will be used and a disposition of the pixel relative to
(b) a memory in which are stored: orthogonal axes in the gray Scale texture data and in the
(i) a plurality of machine instructions; color texture lookup table that is currently being used.
(ii) gray Scale texture data for the background; and 19. A System for displaying a background in a virtual
(iii) color texture data for the background, said color environment So that the background varies dynamically as a
texture data defining chrominance values that vary as 15 function of time, comprising:
a function of the predefined parameter; and (a) a display;
(c) a processor coupled to the memory and to the display, (b) a memory in which are stored;
Said processor executing the machine instructions to (i) a plurality of machine instructions;
carry out a plurality of functions, including: (ii) gray Scale texture data for the background; and
(i) using the value for the predefined parameter, deter (iii) color texture data for the background, said color
mining chrominance values from the color texture texture data defining chrominance values that vary as
data, for corresponding pixels comprising the back a function of time passing in the virtual environment;
ground; and
(ii) modulating the gray Scale texture data for pixels 25 (c) a processor coupled to the memory and to the display,
with the chrominance values corresponding to the Said processor executing the machine instructions to
pixels, and carry out a plurality of functions, including:
(iii) using results from the step of modulating, Succes (i) as time passes in the virtual environment, determin
Sively rendering the background in the virtual envi ing corresponding chrominance values from the
ronment on the display as the value of the predefined color texture data, for pixels extending acroSS the
parameter changes, So that the background dynami background in the Virtual environment;
cally changes as a function of the value of the (ii) multiplying the gray Scale texture data by the
predefined parameter. chrominance values to determine pixels in the back
15. The system of claim 14, wherein the color texture data ground; and
comprise a color texture lookup table having a first axis that 35 (iii) using the pixels for rendering the background in
varies with the value of the predefined variable, and a Second the Virtual environment, So that the background
axis that is orthogonal to the first axis and along which the dynamically changes on the display as time passes in
chrominance values define the colors of pixels comprising the virtual environment.
the background. 20. The system of claim 19, wherein the color texture data
16. The system of claim 14, wherein the predefined 40 comprise a color texture lookup table having a first axis that
parameter comprises time within the Virtual environment, So varies with time in the virtual environment, and a Second
that the chrominance values applied to the gray Scale texture axis that is orthogonal to the first axis and along which the
data change with passing time. chrominance values define the color of pixels in one of rows
17. The system of claim 14, wherein the color texture data and columns within the background.
define bands of color in the background that change on the 45 21. The system of claim 19, wherein the machine instruc
display as the value of the predefined parameter changes. tions further cause the processor to determine a portion of
18. The system of claim 14, wherein the color texture data the background being viewed in the virtual environment, So
comprise a plurality of color texture lookup tables, each that Said portion is rendered on the display for viewing.
color texture lookup table corresponding to a different value
of the predefined parameter, further comprising the Step of k k k k k