This chapter discusses advanced topics in volumetric appearance. For an understanding of basic appearance concepts, see “Decoupling Geometry and Appearance”. For an understanding of the API associated with these concepts, see “Step 2: Define Appearance”.
This chapter discusses the following topics:
Voxel data sets often exceed the size of texture memory. To handle such large data sets, volumes are subdivided into a number of chunks, each of which are small enough to fit into texture memory. These chunks are called bricks.
A voBrick is a hexahedral (box-like), three-dimensional textures that you use to approximate volumes. A voBrick is the same as a voTexture3D object except that a voBrick has a defined position; voTexture3D does not.
 | Note: A voBrick can also represent a two-dimensional texture when you set one of the voBrick's dimensions to one. |
A voBrick can hold voxels represented by various data types. OpenGL Volumizer supports the following types:
These data types map directly onto types supported by OpenGL (consult documentation on glDrawPixels(3G) or glTexImage2D(3G) for details). For example, a data type of BYTE indicates that every voxel holds a single, 8-bit value represented as an unsigned char. Similarly, INT_8_8_8_8_EXT indicates that four 8-bit values, for example, RGBA components, are packed into a single voxel of type unsigned int.
The choice of data type is driven by convenience, performance, and memory usage. More compact types consume less memory and may download associated textures faster. Some types are faster to convert to the native format of the graphics accelerator.
The data format refers to the number of channels encoded in the voxel information, for example, a one-channel data format specifies the INTENSITY of the voxel; a four-channel data format might specify the red, green, blue, and alpha (RGBA) values of the voxel.
In Volumizer, data can be used in a variety of formats at different processing stages, as shown in Figure 7-1.
As voxel data is processed at various stages of the visualization pipeline, the data can be stored in various formats. In Volumizer, there are three places where the data can be stored in different formats:
Native format—is the data format that is used to store data on the disk.
External format—is the data format used by the application.
Internal format—is the format that the underlying graphics API uses to save the voxel data in texture memory.
We refer to the transitions between these stages as “texture read” and “texture download” respectively.
For example, your data may be stored on disk as a single value, INTENSITY, per voxel. You may choose to use a LUMINANCE_ALPHA external format for your application-side processing. This choice requires that each voxel value be duplicated after a brick is read. Finally, you could use the RGBA internal format as the storage format in texture memory. The voxel data may need to be converted as it goes from one processing stage to another.
You specify each of these formats in the voTexture3D constructor.
Every time a brick is downloaded from the host memory to texture memory, the formats are converted. Format choices can be driven by convenience, performance, and memory usage considerations.
Because native and external formats are conceptually the same, they can both be any of the following values:
enum voExternalFormatType{
DEFAULT,
INTENSITY,
LUMINANCE_ALPHA,
LUMINANCE,
RGBA,
ABGR_EXT
};
|
The following internal formats, on the other hand, are more sensitive to the internal memory layout:
enum voInternalFormatType{
DEFAULT,
INTENSITY8_EXT,
LUMINANCE8_EXT,
LUMINANCE8_ALPHA8_EXT,
RGBA8_EXT,
RGB8_EXT,
RGBA4_EXT,
QUAD_LUMINANCE8_SGIS,
DUAL_LUMINANCE_ALPHA8_SGIS
};
|
The conversion from the native format to the external format is typically only done once per-brick during the initialization phase. The conversion from external to internal format has to be done every time a brick is downloaded from the host to texture memory.
There may be a performance penalty associated with specific formats depending on the underlying hardware. Therefore, it may be advantageous to keep all the formats as similar as possible.
There is typically an optimal format for any given platform. For example, on Impact graphics hardware, LUMINANCE_ALPHA and LUMINANCE8_ALPHA8_EXT produce optimal performance for grayscale rendering. However, it results in a times-two memory bloat because the same piece of information, the voxel's intensity, is replicated in the ALPHA channel.
On the other hand, using a “tighter” external format reduces your application's memory requirements. So an application can request INTENSITY as the external format and LUMINANCE8_ALPHA8_EXT as internal and minimize its memory usage at the expense of run-time performance penalty during texture download.
In situations where the disk format differs from the external format, voxel data can be suitably converted with help of voAppearanceActions::dataConvert() after it was read:
static int dataConvert(voTexture3D* aTexture3D, void* data, voExternalFormatType diskFormat); |
In this method the voxel data, aTexture3D, is converted to the specified format, diskFormat, and the result is pointed to by data.
data can point at the brick's data storage, in which case the conversion is done in place. This may be a little faster (and certainly more economical in terms of memory usage and code size) than using an additional I/O buffer.
Some data values have to be scaled into the range expected by the graphics API. For example, if the voxel data type is float, it has to fit in the range of <0.0, 1.0>. Similarly, unsigned short voxels need to be scaled to <0, 65535>.
voAppearanceActions::dataScaleRange() expands the values of data to span the entire dynamic range of the underlying data type.
Example 7-0 initializes the voxel data by loading, converting, and expanding it.
voBrickSetIterator aBrickSetIter(aBrickSet);
for(voBrick *brick; brick = aBrickSetIter(); ) {
unsigned char *vdata = (unsigned char *)
voTexture3DActions::dataAlloc(brick);
myReadBrickIfl(fileName, vdata, xBrickOrigin, yBrickOrigin,
zBrickOrigin, xBrickSize, yBrickSize, zBrickSize);
// convert to the desired externalFormat
Texture3DActions::dataConvert(brick,vdata,INTENSITY);
// expand the values to span the whole dymanic range
voAppearanceActions::dataScaleRange(brick, loValue, hiValue);
}
|
Consult InitAppearance.cxx for sample source code.
On certain architectures, texel values are internally stored and manipulated as RGBA quadruples regardless of their original format. Therefore, even if the external, application side voxel format is INTENSITY, this single value is replicated 4 times during the download and stored internally as 1111. This means that a 4 MB texture memory can hold at most 1 MVoxel of intensity data. Other forms of packing texels are also possible. On some machines it is possible to increase the effective size of the texture memory and improve the download performance by brick interleaving.
When two bricks are interleaved, their voxels are combined together: the first voxel of the first brick is followed by the first voxel of the second, and so on. This way two values (one from each brick) in the INTENSITY_ALPHA format can be packed on the host into a single RGBA value, one in each channel pair. This 2-way interleaved texture can be transferred into the texture memory with a single command (reducing download effort 2-fold) and either one of the two original textures can be selected.
To determine if two bricks are interleaved, use voTexture3DActions::cliqueTest(brick1, brick2). Only 2-way interleaving is currently supported in Volumizer.
Interleaving bricks provides two important advantages:
Transfer of appearance information is dramatically increased.
If, for example, two LUMINANACE_ALPHA voxels are stored in a single RGBA value, the texture transfer rate is increased by 200%.
Memory is conserved.
If, for example, two LUMINANACE_ALPHA voxels are stored in a single RGBA value, the number of voxels that can be held in memory is increased by 200%.
To perform two-way interleaving of bricks manually, use one of the following methods, respectively:
static int textureInterleave (
voTexture3D* aBrick1, voTexture3D* aBrick2);
|
In order for two bricks to be suitable for interleaving they both have to be in compatible format. That is, their sizes, with a call to data types, and external formats have to be identical.
The TEXTURE_OBJECTS flag takes advantage of the Texture Manager in OpenGL by creating texture objects out of textures thus relieving the application from explicit texture memory management. Without texture objects, the application itself has to keep track of the texture memory usage. For example, if the whole volume fits into the texture memory, the application should make sure not to repeatedly re-load the texture.
With texture objects, the texture manager takes over this bookkeeping task. If a texture object is bound repeatedly, no explicit downloads occur. For more information about texture objects, see OpenGL Programming Guide.
| Note: Texture object creation requires a valid graphics context. |
If a texture object is used for only one or two frames and then discarded, textures should not be converted into texture objects because the advantage of single-time processing is lost. Performance might be harmed because the creation of a new texture object for every frame is more costly than processing a texture for each new frame.
For example, if your application displays a beating heart, texture object creation should not be enabled because the texture changes in each frame.
Voxel values do not have to represent the intensity or color, but can describe other values, such as stencils, also called tags. Tags are discussed in “Tagged Voxels”.
Similarly, data in the voxel may be subject to interpretation by applications. For example, an RGB value may contain gradient components rather than colors.
You might tag individual voxels with class identifiers to, for example:
Render different parts of a volume with different parameters.
For example, in clinical applications, rendering a variety of tissues with different color/transparency parameters.
Clip textures to arbitrary surfaces in situations where the surface is too fine to be tessellated into individual tetrahedra.
For example, when interpreting seismic data, you volumetrically render only the sub-volume that falls between two horizontal planes. Defining the sub-volume as WIDTH x HEIGHT number of hexahedra may not be a viable alternative so a tagged volume should be used instead.
The functionality provided in the current release of the API allows application programmers to render tagged volumes using a simple multi-pass algorithm. Namely, the application can maintain an additional volume of tags and use stencil planes to select individual classes for rendering.
Just as you can map two dimensional texture coordinates onto a polygon, you can map voxel coordinates onto volumetric geometry (tetrasets), as shown in Figure 7-2.
In the OpenGL Volumizer API, the geometry and voxel reference frames coincide: the voxel coordinates span [0,0,0] to [xVolumeSize-1, yVolumeSize-1, zVolumeSize-1]. When the size of the volumetric geometry matches the size of the voxel data array, the voxel and vertex coordinates are identical. Since this is a very common occurrence, applications do not have to explicitly specify texture coordinates in this situation.
However, there are situations where there may not be a 1-to-1 mapping from coordinates to voxel coordinates. For example, that may be required for modeling reasons: the volumetric object may have to exist in the world space that has dimensions of <1.0,1.0,1.0> even though the voxel array has sizes of <256.0,256.0,256.0>. In a more involved scenario, the volume may be deformed. In such situations, the application is required to explicitly specify both voxel and vertex coordinate triples.
The voxel reference frame is different from the texture reference frame. For a single brick volume, there is a simple linear mapping that takes the voxel coordinates and scales them back to <0.0, 1.0> range so that they can be used as texture coordinates. In the general case of a multi-brick volume, there is a series of such piecewise linear mappings that map a voxel coordinate into texture coordinates (s, t, r) within each individual brick by applying a suitable scale and bias.
During polygonization, the voxel coordinates, if specified, are sampled and clipped with the vertex coordinates. To draw the resulting polygons, however, these clipped voxel coordinates have to be transformed into the brick texture's reference frame such that the coordinate system, (s, t, r), spans [0,0,0] to [1,1,1] for each brick.
voAppearanceActions::xfmVox2TexCoords(), defined as follows, performs this transformation for each voFaceSet within a brick. It must be used before any draw calls.
static void xfmVox2TexCoords(voBrick* brick,
voIndexedFaceSet* aFaceSet,
voInterleavedArrayType interleavedArrayFormat,
voPlaneOrientation orientation);
static void xfmVox2TexCoords(voBrickSet* aBrickSet,
int samplesNumber, voIndexedFaceSet*** aFaceSet,
voInterleavedArrayType interleavedArrayFormat);
|
In the case where the voxel and vertex coordinates match, it is unnecessary to specify the voxel coordinates. Instead you use voAppearanceActions::texgenEnable() once per drawing loop to generate the texture coordinates, and voAppearanceActions::texgenSetEquation() for each brick before drawing any face sets, as follows:
if( interleavedArrayFormat == _V3F )
voAppearanceActions::texgenEnable();
else
voAppearanceActions::xfmVox2TexCoords(
aBrickSet,
samplesNumber,
aPolygonSetArray,
interleavedArrayFormat);
// iterate over all bricks
for(brickNo=0;brickNo < BrickCount;brickNo++)
{
...
// update texgen equation for the current brick
if( interleavedArrayFormat == _V3F )
voAppearanceActions::texgenSetEquation(aBrick);
// iterate over all sampling planes
for(int binNo=0;binNo < samplesNumber;binNo++)
voGeometryActions::draw(
aPolygonSetArray[brickSortedNo][binNo],GL_FILL,interleavedArrayFormat);
}
if( interleavedArrayFormat == _V3F )
voAppearanceActions::texgenDisable();
|
Generating voxel coordinates with voAppearanceActions::texgenEnable() is convenient and it may increase performance because fewer data items are transmitted down the graphics pipeline. In some situations, however, voxel coordinate generation can harm performance because the coordinates cannot be cached and the generation relies on hardware acceleration, which a machine may not have.
Instead of using xfmVox2TexCoords() to explicitly transform the voxel coordinates into the brick texture reference frame, you can use a texture matrix to apply scale and bias to brick coordinates on the fly. Use voAppearanceActions::textureMakeMatrix(), defined as follows, to construct a suitable matrix and push it onto the texture matrix stack with standard OpenGL calls.
static void textureMakeMatrix (voBrick* aBrick, float* matrix); |
You can set and return the size of a voBrick using the following voBrick methods:
These methods use, as arguments, the coordinates of kitty-corner corners of a hexahedral brick to set or return its size. The size values are relative to the min values rather than the coordinate-axis origin. For example, if the min values are (1, 1, 1), the size values could also be (1, 1, 1), in which case, the kitty-corner coordinates are (2, 2, 2).
| Note: The brick origin can be an arbitrary number, but the brick sizes are typically (but not necessarily) powers of two. |
Selecting optimal brick sizes for a shape is tedious. Volumizer, therefore, provides the following helper routine to select the optimum brick size:
For more information about voAppearanceActions::getBestParameters(), see “Step 2: Define Appearance”.
To see sample code that determines brick size, refer to InitAppearance.cxx.
Typically, it takes more than one brick to represent a volume. A number of such adjacent, possibly-overlapping, bricks constitute a voBrickSet. Applications can use voBrickSet to access volume size, requested brick sizes, handles to individual voBrick objects in the set, and the set's orientation.
For more information about implementing brick sets, see “Allocate Storage for Bricks”.
Some volumes must be represented by multiple voBrickSets. For example, machines that do not support three dimensional texture mapping require maintaining three separate copies of a brickset: one for each major axis, as shown in Figure 2-18 on page 24. In this case, each copy is represented by a separate voBrickSet and the entire volume is represented by the collection of the three voBrickSets.
voBrickSetCollection works like a switch that allows applications to select one of the voBrickSets in the collection. For example, the following call selects the voBrickSet within the collection sliced in planes that are the most perpendicular to the line of sight:
aBrickSetCollection->setCurrentBrickSet( findClosestAxisIndex(modelMatrix,projMatrix,AXIS_ALIGNED) ); |
For multiple voBrickSets, it is convenient to know the maximum number of bricks in a collection. For example, the maximum number of bricks in a 256x256x128 volume represented by three stacks of two dimensional images is 256. this value can be used to allocate a single buffer for transient geometry that will be large enough to hold the results of polygonization regardless of the volume's orientation.
Creating a voBrick does not allocate voxel data storage. Applications must call the voAppearanceActions method, dataAlloc(), defined as follows, before operating on any data.
static void* dataAlloc(voTexture3D* aTexture3D); static void* dataAlloc(voBrickSet* brickSet); |
The following code allocates data for all of the bricks in a voBrickSet:
voBrickSetIterator aBrickSetIter(aBrickSet);
for(voBrick *brick; brick = aBrickSetIter(); )
(void)voTexture3DActions::dataAlloc(brick);
|
dataAlloc() returns a pointer to the voxel data storage. This pointer can be used by applications to directly reference voxel data.
voTexture3D::getDataPtr() returns the same pointer but only after the brick has been created.
An application can use this pointer as a destination for any application-specific, disk I/O routines and thereby avoid allocation of additional buffer space, for example,
voBrickSetIterator aBrickSetIter(aBrickSet);
for(voBrick *brick; brick = aBrickSetIter(); )
{
unsigned char *vdata = (unsigned char *) voAppearanceActions::dataAlloc(brick);
myReadBrickIfl(fileName, vdata, xBrickOrigin, yBrickOrigin, zBrickOrigin, xBrickSize, yBrickSize, zBrickSize);
}
|
myReadBrickIfl() is a utility function provided with the API that reads a a block of data from (xBrickOrigin, yBrickOrigin, zBrickOrigin) to (xBrickOrigin+xBrickSize-1, xBrickOrigin+xBrickSize-1, xBrickOrigin+xBrickSize-1) from a three dimensional TIFF file using the Image Formal Library (IFL).
Applications need to load the voxel data into the texture memory before the volume that is associated with this data can be drawn. For multi-brick volumes, each brick has to be loaded in turn once per frame. However, for single-brick volumes it is enough to load the voxel data once. Applications can keep track of such situations and use voAppearanceActions::textureLoad() to explicitly transfer the voxel data from the host to texture memory.
Once the volume is optimized with TEXTURE_OBJECTS as a parameter, the application can use voAppearanceActions::textureBind(). This action uses the OpenGL Texture Manager to determine if the requested brick data already resides in texture memory. If it does, textureBind() does nothing; otherwise, textureLoad() is called, which downloads the texture from host to texture memory.
static int voAppearanceActions::textureBind(voTexture3D* aTexture3D); |
Using textureBind() instead of textureLoad() results in a significant performance boost when the brick data is already in texture memory.
Applications must call textureBind() or textureLoad() before drawing any shapes.
voAppearanceActions::textureLoad(), invoked as follows, forces a download of texture data associated with a brick from the host memory into the texture memory regardless of whether or not the texture is already resident in memory.
static int voAppearanceActions::textureLoad(voTexture3D* aTexture3D); |
File I/O is not a part of the OpenGL Volumizer API. The sample applications in this section illustrate how to read voxel data from three-dimensional TIFF files using the IFL library, and from files that are stored as raw, two-dimensional images.
If your voxel data set is in a format that can be loaded by one of the demonstration OpenGL Volumizer programs, you can use the utilities in this section to load your two- or three-dimensional data set, as demonstrated in “Read Brick Data from Disk”.
To load three-dimensional, TIFF files and to determine the size of the volumes they contain, use the following two methods in InitAppearance.cxx:
int myGetVolumeSizesIfl(
char *fileName, int &xSize, int &ySize, int &zSize,
voExternalFormatType & diskDataFormat,
voDataType & dataType);
int myReadBrickIfl(char *fileName, void *data,
int xBrickOrigin, int yBrickOrigin, int zBrickOrigin,
int xBrickSize, int yBrickSize, int zBrickSize,
int xVolumeSize, int yVolumeSize, int zVolumeSize);
|
myGetVolumeSizesIfl() determines the volume sizes, format (for example, INTENSITY or RGBA) and data type (for example, ubyte or float).
myReadBrickIfl() reads a brick of voxels with their origin at (xBrickOrigin, yBrickOrigin, zBrickOrigin) and sizes (xBrickSize, yBrickSize, zBrickSize). See myBrickIO.cxx for the complete source code.
If your data is stored on disk as a sequence of 2D raw images, use the following method:
int myReadBrickRaw(char **fileNames, void *data,
int xBrickOrigin, int yBrickOrigin, int zBrickOrigin,
int xBrickSize, int yBrickSize, int zBrickSize,
int xVolumeSize, int yVolumeSize, int zVolumeSize,
int headeLength, int bytesPerVoxel);
|
By a “raw” image we understand a verbatim voxel stream in row-major order possibly proceeded by a fixed size header.
The file information, header length, dimensions, format, and data types, should be provided by the application, for example, read from the command line or read from the proprietary header.
Applications that prefer to perform their own file I/O must provide their own API equivalents for myReadBrickIfl() or myReadBrickRaw() that read an arbitrary block of voxels from (xBrickOrigin, yBrickOrigin, zBrickOrigin) to (xBrickOrigin+xBrickSize-1, xBrickOrigin+xBrickSize-1, xBrickOrigin+xBrickSize-1) into a contiguous memory area. should replace.
OpenGL Volumizer provides a variety of voxel manipulation utilities in the util directory.
iflfrombin.cxx illustrates how to convert a raw image into a two-dimensional TIFF file (or any other two dimensional image recognizable by IFL). The syntax is:
% iflfrombin hdrlen xsize ysize csize bpp infile outimage |
hdrlen is the length of the image file's header in bytes.
xsize and ysize are file dimensions, csize is the number of channels (1 for INTENSITY, 4 for RGBA data).
bpp, the data type, is uchar, ubyte, or float.
infile and outimage are the input file and the TIFF file, respectively. The TIFF file should have an extension of .tif by convention.
To facilitate a simple and repeatable testing environment, a utility for generating a volumetric test pattern is provided in Volumizer/util/mkcubes.cxx. Invoking this program with the following parameters produces a volume in a three-dimensional TIFF format:
mkcubes xVolumeSize yVolumeSize zVolumeSize cubeSize fileName.tif |
The volume has dimensions specified on the command line and it contains a pattern of small cubes of size, cubeSize, with intensity varying linearly from their centers to their surface. For example, use:
mkcubes 256 256 256 128 out.tiff |
to produce a 2563 volume filled with 8 cubes each having 128 voxels on the side.
The latest version of Volumizer provides interfaces for reading and modifying voxels after they have been loaded into memory. The following five methods are provided to facilitate this functionality. Each function is implemented both for voBrickSets and voBrickSetCollections:
void initSubVolume(
int *origin, int *size, void *value);
void setSubVolume(
int *origin, int *size, void *values);
void getSubVolume(
int *origin, int *size, void *values);
void setVoxel(
int x, int y, int z, void *value);
void *getVoxelAddr(
int x, int y, int z);
|
initSubVolume() sets a region of voxel data to the single value pointed to by value. This function is the equivalent of the memset operation for volume data. origin and size specify the origin and size of the initialization in global voxel coordinates. The function marks the affected bricks as dirty so that they will be reloaded into texture memory. value should be formatted according to the external format of the data.
setSubVolume() replaces a hexahedral region of voxel data with new texture pointed to by values.This function is an equivalent of the memcpy operation for volume data. origin and size specify the origin and size of the copy in global voxel coordinates. The function marks the affected bricks as dirty so that they will be reloaded into texture memory. values must be non-interleaved but, otherwise, should be formatted according to the external format of the data.
getSubVolume() retrieves a hexahedral region of voxel data. It copies the data into the location pointed to by values. This function is an equivalent of the memcpy operation for volume data. origin and size specify the origin and size of the copy in global voxel coordinates. The data returned in values will be non-interleaved but, otherwise, will be formatted according to the external format of the data.
setVoxel() sets the voxel at voxel coordinates (x,y,z) to the value specified. value should be formatted according to the external format of the data.
getVoxelAddr() returns a memory address that points to the voxel at coordinates (x,y,z). Since bricks may overlap, it is possible for a single voxel to be contained in up to eight separate memory locations. This routine is guaranteed to return one of these locations. Applications should cast the pointer according to the external format of their data before using it.