Appendix A. Using OpenGL

Rendering

Expo needs an OpenGL renderer that meets the following requirements: it must provide hardware acceleration, it must support full-screen double-buffered displays, and it must provide at least 8 bits per pixel in each of the R,G,B and Alpha channels. It must also provide at least three hardware units for multi-texturing. Recent ATI and Nvidia graphics cards meet these requirements. When you configure a display through Environment-> Monitors Expo will indicate for each monitor whether or not it is supported by an adequate renderer (OpenGL: Yes). If the video card does not support an adequate renderer, Expo will not allow you to select it.

Expo moves images, or manipulates image contrast and color, by redrawing in the frame buffer. It uses lookup tables only for linearizing the display. Rendering generally imposes a negligible load on Expo, and Expo will run comfortably at high frame rates. Images can be constructed and redrawn at rates of at least 120Hz (1024 x 768).

Texture Fragment Shaders

When Expo uses a video card (ATI Radeon 8500 and above; Nvidia Geforce FX and above) that supports OpenGL extensions for programmable fragment shading (ARB_fragment_program or ATI_text_fragment_shader), Expo uses fragment shading programs to control the contrast and opacity of objects it displays. This greatly speeds up rendering, and also provides capabilities not available otherwise. When you configure a display through Environment-> Monitors Expo indicates whether it supports fragment shading.

Several display capabilities require fragment shaders:

• When you overlay one semi-transparent object on another, and lack fragment shaders, the surfaces are not blended correctly if the surface shape is non-uniform (e.g., a Gaussian or Raised Cosine).
• The Surface Overlay, Surface Matrix, Dot Field and RGB Texture routines require fragment-shading hardware.
• The Opacity parameter of the Image Texture and Movie Texture routines works only with fragment-shading hardware.

Multithreaded OpenGL

On computers with multiple processors, the OpenGL engine may be able to use more than one concurrently. If the hardware supports this, you can enable multithreading through Environment-> Monitors. Running OpenGL in multithreaded mode is likely to be beneficial only if you are drawing many objects. It can harm performance if you are drawing relatively few objects (an 'object' for this purpose is a Surface, a Dot Field, or a Surface Matrix). You should check that enabling multithreading actually improves performance.

Frame Synchronization

Expo uses OpenGL in double-buffered mode, drawing into one (invisible) image buffer while displaying the contents of one previously used. Buffers are exchanged on the frame tick, which can also provide the timebase for Expo operations while a program is running. Without double-buffering (for example if Expo simply drew directly to the frame buffer) display artifacts could (do) arise from simultaneously reading from and writing to the framebuffer. Without synchronization, an exchange of buffers could occur during the middle of a frame sweep, giving rise to what is called "image tear" (part of the screen shows the old frame, and part the new one). Synchronized buffering avoids these problems, but at the cost of a one-tick latency between any change that Expo makes to images and the update on the screen. You need to take account of this if you are concerned with the precise timing of events.

Calls to OpenGL to draw images and swap buffers are automatically synchronized to the video refresh. Although OpenGL is fast, some operations might take longer than a frame tick. If this happens Expo will not update the display on each frame. Expo automatically detects dropped frames, and if any occur displays a "TIMEBASE OVERRUN" warning in the running program's Status window. You can set Expo (via Expo->Preferences) to abort a running program if it drops a frame. For more on the precision of timing see Thread Timing.

The speed with which Expo can generate textures with controllable color/contrast and opacity depends greatly on the video hardware. If the hardware supports either the ARB_fragment_program or ATI_text_fragment_shader extensions for programmable texture fragment shading, rendering takes negligible time, but if the video card does not support programmable shading, colorable textures generated by the routines DKL Texture or LMS Texture must be (re)constructed on-the-fly whenever the chromatic signature of the texture is first established or changes (a texture is not reconstructed for a change in spatial frequency). On a 500MHz G4 without fragment shading, the generation of a 2D texture takes about 0.175 µsec per texture element. This means that the largest texture array that can comfortably be handled within a frame tick is 128 x 128 elements. You can freely use textures made from larger arrays, but you should not change their chromatic characteristics while a state is running. Applying many cycles of a 2-D texture to a surface can also take a long time, and may provoke a timebase overrun.

Blending Surfaces

When you define a texture with Opacity less than 1, Expo uses OpenGL blending to combine the current texture with whatever underlies it. To blend a pixel in the source texture with what lies underneath it (a pixel in the destination image already in the framebuffer), the blending function modifies separately the source pixel and the destination pixel then adds them, replacing the destination with the result.

The rule used to modify the source is to multiply each R,G,B,A value by the value of the Alpha channel. The rule used to modify the destination is to multiply each RGBA value by 1-Alphasrc, where Alphasrc is the value of Alpha channel in the source pixel. This rule makes it possible to generate compound textures, such as plaids (two overlaid gratings whose local luminances are summed).

Using Arbitrary Textures and Forms

Expo creates textures from images and movies; it also creates shape masks (used by the routines Surface, Surface Element, Surface Matrix and Dot Field) from images. Some standard images and masks are built-in; additional images can be used if image files are placed in Expo's Resources folder. Expo provides no built-in movies from which textures can be derived—all movies must be in files in the Resources folder.

Image Formats and Requirements

Expo distinguishes three kinds of bitmap images and uses them differently:

• Greyscale images that contain no alpha (transparency) channel are used to form colorable textures made by the routines DKL Texture and LMS Texture.
• Full color RGB images that contain no alpha (transparency) channel are used to form the textures made by the routine Image Texture.
• Images that contain an alpha channel are used to make shape masks used by the routines Surface, Surface Element, Surface Matrix and Dot Field. These can either be four-channel RGBA images or two channel greyscale+alpha images. The alpha component is used to define the outline shape of the image (e.g., a circle or wedge); the RGB or greyscale component is used to define the contrast profile (e.g., Gaussian) of the remaining visible surface; larger RGB or greyscale values mean greater contrast.

Expo automatically distinguishes images of the different types and makes the appropriate ones available where needed in routines.

Expo can make textures from images of the following types: PNG, TIFF, JPEG, BMP, PSD. PNG provides compact images that use lossless compression.

OpenGL textures must have X- and Y- extents (in pixels) that are integral powers of two, though X- and Y-extents need not be the same. You should ensure that your images satisfy this requirement. If Expo finds an image file that does not meet this requirement it ignores the file and logs a message on the console. For images that will be used to generate periodic patterns of the kind likely to be used with DKL Texture or LMS Texture it will seldom be necessary to have X and Y extents greater than 128 or 256 elements. Mask images (used in Surface, Surface Element and Dot Field) should be 256 x 256 pixels.

Movie Formats and Requirements

Movies used by the routine Movie Texture can be contained in .mpg or .mov or .avi files.

When building a texture from a movie frame, Expo automatically reduces the image size, if necessary, to ensure that the number of pixels on the larger dimension of the resulting texture is a power of 2; the scaled larger dimension is never larger than 512 elements. The aspect ratio is preserved during scaling.