XIL is a general-purpose imaging library that provides low-level functions. These low-level functions are designed to be used as building blocks to create higher-level functions.
Functions in higher-level libraries can minimize the number of times that an image is transferred to and from memory by grouping operations that work on the same image. Higher-level functions also can minimize the number of temporary images that need to be created and destroyed during an operation.
Instead of providing directly callable high-level functions, XIL provides a deferred execution facility that automatically recognizes certain sequences of XIL atomic functions and executes the sequences as a single high-performance molecule. An example is a sequence of XIL functions that scales (implicitly capturing) and compresses an image. XIL defers execution of the scale function to see if a compression function follows. If it does, the scale and compression functions are executed together as a high-performance molecule.
XIL defines a set of general-purpose molecules that perform sequences of operations such as color conversion and decompression. There are a few general rules you must follow to execute XIL molecules (see the chapter "Acceleration in XIL Programs" in the XIL Programmer's Guide). The SunVideo subsystem further accelerates some capture-based molecules by executing the molecules on the card. These capture-based molecules include sequences of functions that scale and compress images that are captured from the SunVideo card.
The definition of some SunVideo molecules (molecules that can be executed on the SunVideo card) have stricter requirements than their XIL counterparts. For example, SunVideo molecules must perform specific kinds of scaling or dithering. SunVideo compression molecules must have specific scale factors or XIL attributes.
Figure 8-1 shows an example of how SunVideo molecules work. The sequence of the second and third functions meet the requirements of a SunVideo molecule. The two functions are combined and executed on the SunVideo card.
If a molecule does not meet the SunVideo requirements, it can still be accelerated on the host as an XIL molecule. The molecule made up of the sixth, seventh and eighth functions is an example.
Figure 8-1 XIL and SunVideo Acceleration
The SunVideo subsystem tries to accelerate as much of a molecule as possible. Therefore, in some situations, a molecule can execute partly on the SunVideo card and partly on the host. An example is a molecule consisting of an XIL scale function and an XIL compression function. The scale function can execute on the SunVideo card, even if the compression function executes on the host.
Two other important points to remember about molecules are that:
For more information about deferred execution and molecules, refer to the "Acceleration in XIL Programs" chapter of the XIL Programmer's Guide.
The following code fragment is from the rtvc_display example program, when displaying to an 8-bit display. The highlighted functions are executed on the SunVideo card as a molecule that:
--------------------------------------------------------------
xil_rescale(rtvc_image, rescaled_image, rescale_values, offset_values); if (doscale) { xil_scale(rescaled_image, scaled_image, "nearest", scale_factor, scale_factor); xil_ordered_dither(scaled_image, display, colorcube, mask); xil_toss(scaled_image); } else { xil_ordered_dither(rescaled_image, display, colorcube, mask); } --------------------------------------------------------------
The requirements for this particular SunVideo molecule are that:
The results of using molecules with a device image, such as a SunVideo card, can be different than the ones you expect. For example, you may want your application to capture, compress, and display every frame from a video stream, but instead it displays every other frame and compresses the frames it does not display. Let's look at how this can happen.
When an atomic operation that affects the state of an image or a compressed image sequence (CIS) is called, the operation is not executed immediately. Instead, information about the operation, such as the name of the function called and its arguments, is stored by the XIL library. This information continues to be stored until the library must produce a CIS or particular destination image (for example, because that image is to be displayed). When a CIS or destination image must be produced, the XIL library searches a list of molecules to see if all or part of the sequence of operations can be replaced by a molecule. If the library finds a molecule that can perform the entire sequence of operations, the sequence executes as a molecule and the program proceeds. The molecule performs the work of all of the operations in the sequence; the individual operations are not executed.
Consider what happens when the following operations appear in an application program.
-------------------------------------------------------------------
xil_scale(src_image, dst_image, interpolation, xscale, yscale); xil_compress(dst_image, cis); xil_ordered_dither(dst_image, display_image, cmap, dithermask); -------------------------------------------------------------------
This sequence of operations actually represents four operations because of the implied capture operation associated with the xil_scale() operation. Therefore, the four operations are:
In normal operation, the XIL library executes two molecules to perform the above operations:
The first molecule captures, scales and stores an image in a CIS. The second molecule captures, scales, dithers and displays a second image (Figure 8-2). The reason two captures are needed is that the first molecule does not save the raw captured image (the xil_scale() function does not compute the value of the image). Therefore, the xil_ordered_dither() function does not have an image to dither and display. In order for xil_ordered_dither() to execute, another image must be captured by the second molecule.
Figure 8-2 Operations Executed as Molecules
To display the same image that is compressed, the xil_scale() function must execute synchronously (so that it will compute the value of the image) (Figure 8-3). To get the xil_scale() function to execute synchronously, you must place the xil_sync() function after the xil_scale() function. The xil_sync() forces the library to compute the value of the image, which provides to the xil_ordered_dither()function an image to dither and display.
---------------------------------------------------------------------
xil_scale(src_image, dst_image, interpolation, xscale, yscale); xil_sync(dst_image); xil_compress(dst_image, cis); xil_ordered_dither(dst_image, display_image, cmap, dithermask); ---------------------------------------------------------------------
Figure 8-3 Scale Executed Synchronously
Note -
For both of the sequences shown in Figures 8-2 and 8-3, the video will
not be processed at full frame rates. In the first case, the SunVideo compression
engine software is reloaded for each frame, alternating between the
capture ¤ scale ¤ compress and the capture ¤ scale ¤ dither software. In the
second case, while the capture ¤ scale operation is accelerated, the compress
and dither operations are not.
This section explains the syntax that we use to describe the SunVideo molecules, provides the general rules that apply to the molecules, then lists the sequences of functions that are defined as SunVideo molecules.
To save space in the molecule definitions, we use a shorthand notation. For example:
capture ¤ [scale8nearest ¤ ] compress_CellB
The implicit capture operation is shown for completeness. The "scale8nearest" notation is shorthand for the xil_scale() function, operating on an image with datatype XIL_BYTE, using the "nearest" argument. The "compress_CellB" notation is shorthand for the CellB xil_compress() function.
In the molecule definitions, functions in square brackets are optional. For example, the scale8nearest function is optional in the previous molecule. This sequence will be executed as a molecule with all three operations (capture, scale8nearest, and compress_CellB) or just the capture and compress_CellB operations.
The following code fragment is an example of some code that implements this molecule.
---------------------------------------------------------------
XilImage rtvc_image, scaled_image; XilSystemState state; XilCis cis; . . cis = xil_cis_create(state, "CellB"); xil_scale(rtvc_image, scaled_image, "nearest", 0.5, 0.5); xil_compress(scaled_image, cis); ---------------------------------------------------------------
Table 8-1 describes the shorthand notation used in the SunVideo molecules' syntax statements. We use this notation because it appears when you use the XIL "show action" facility to determine if functions are executing within a molecule. (The "show action" facility is described in the section "Determining if Molecules are Executing" on page 84.)
-------------------------------------------------------------------------------
Notation Meaning -------------------------------------------------------------------------------
capture An implicit capture operation from a SunVideo card.
scale8nearest The xil_scale() function, operating on an image with
datatype XIL_BYTE, using nearest neighbor interpolation.
colorconvert The xil_color_convert() function.
rescale8 The xil_rescale() function, operating on an image
with datatype XIL_BYTE.
ordereddither8_8 The xil_ordered_dither() function, using an image
with datatype XIL_BYTE as both source and destination.
compress_CellB The xil_compress() function, operating on a CIS of
type CellB.
compress_Jpeg The xil_compress() function, operating on a CIS of
type Jpeg.
compress_Mpeg1 The xil_compress() function, operating on a CIS of
type Mpeg1.
compress_UYVY The xil_compress() function, operating on a CIS of
type UYVY.
decompress_UYVY The xil_decompress() function, operating on a CIS of type
UYVY.
threshold8 The xil_threshold() function, operating on an image
with datatype XIL_BYTE.
addconst8 The xil_add_const() function, operating on an image
with datatype XIL_BYTE.
display_ioSUNWcg14 An implicit display operation to a cg14 (SX) frame buffer.
-------------------------------------------------------------------------------
The SunVideo molecules can be divided into three categories: color, grayscale, and UYVY decompression. The color molecules use a YUV image from the SunVideo card as their source. The grayscale molecules use a Y-only image, that is a single-banded child image from the SunVideo card with a band offset of 0. The UYVY molecules decompress and color convert UYVY CIS's.
The next three subsections list the definitions and requirements for the SunVideo color molecules, grayscale molecules, and UYVY decompression molecules.
capture ¤ scale8nearest
Use: To use the SunVideo card as a YUV frame grabber, meaning to take a snapshot of incoming video frames.
Requirements: Scale factor must be in the form 1/x, where x is a positive, non- zero integer. x must be greater than 1 for the X-axis scale factor if the video format is PAL.
capture ¤ [scale8nearest ¤ ] colorconvert [ ¤ display_ioSUNWcg14]
Use: This molecule is known as a "punch-through" molecule, meaning to take the video that is coming into the SunVideo card and display it using no compression. This molecule is for 24-bit displays.
Requirements: Scale factor must be in the form 1/x, where x is a positive, non- zero integer greater than 1. The colorconvert operation must use a YUV image as its source (XIL colorspace "ycc601") and a RGB image as its destination (XIL colorspace "rgb709").
Code example: display.c
capture ¤ [rescale8 ¤ ] [scale8nearest ¤ ] ordereddither8_8
capture ¤ scale8nearest ¤ rescale8 ¤ ordereddither8_8
Use: These two molecules are known as a "punch-through" molecules, which means to take the video that is coming into the SunVideo card and display it using no compression. These molecules are for 8-bit displays.
Requirements: Scale factor must be in the form 1/x, where x is a positive, non- zero integer. x must be greater than 1 for the X-axis scale factor if the video format is PAL.
The ordered dither operation must use a 4x4 dither mask; its source must be a 3-band, 8-bit per band image and the destination must be a 1-band, 8-bit image.
Code example: display.c
capture ¤ [scale8nearest ¤ ] compress_CellB
Use: To use the SunVideo card as a capture and compression board to produce a CellB bitstream.
Requirements: X-axis scale factor must be 1, 1/2, 1/4, 1/8, 1/16, or 1/32.
Y-axis scale factor must be 1/x, where x is a positive non-zero integer.
Code example: capture_movie.c
capture ¤ scale8nearest ¤ compress_Jpeg
Use: To use the SunVideo card as a capture and compression board to produce a JPEG bitstream.
Requirements: Scale factor must be 1/2.
If a child image is used, it must be the same size as the original device image.
The XIL JPEG compression attributes must be set as follows:
The following XIL JPEG attributes must have their default value (shown in parenthesis):
if (CQ != 50)
inverse_cq = 101 - CQ;
else
inverse_cq = 50;
scaled_quant = (orig_quant * inverse_cq) / 50;
For more information about these XIL JPEG attributes, refer to the "JPEG Baseline Sequential Codec" chapter of the XIL Programmer's Guide.
Code example: capture_movie.c
capture ¤ scale8nearest ¤ compress_Mpeg1
Use: To use the SunVideo card as a capture and compression board to produce an MPEG-1 bitstream.
Requirements: Scale factor must be 1/2.
If a child image is used, it must be the same size as the original device image.
The XIL MPEG-1 attributes must be as follows:
The following XIL MPEG-1 attributes must have their default value (shown in parenthesis):
(XIL does not have a SPARC-based MPEG-1 compressor. Therefore, the MPEG- 1 compression operation will generate an XIL error message if the MPEG-1 molecule cannot execute on the SunVideo card.)
For more information about these XIL MPEG attributes, refer to the "MPEG I Codec" chapter of the XIL Programmer's Guide. For information about SunVideo's implementation of two MPEG-1 compressors, see Appendix B, "Video Compression."
Code example: capture_movie.c
capture ¤ [scale8nearest ¤ ] compress_UYVY
Use: To use the SunVideo card as a way to produce 4:2:2 data.
Requirements: Scale factor must be in the form 1/x, where x is a positive, non- zero integer.
Code example: xilbroadcast.c
capture ¤ scale8nearest
capture ¤ [scale8nearest ¤ ] [ [addconst 8 ¤ ] addconst8 ¤ ] addconst8
capture ¤ [scale8nearest ¤ ] [threshold8 ¤ ] threshold8
capture ¤ [scale8nearest ¤ ] [threshold8 ¤ ] threshold8 ¤ addconst8
Use: To use the SunVideo card as a luminance-only frame grabber, meaning to take a snapshot (in grayscale) of incoming video frames. The molecules that contain addconst8 also provide a way for you to brighten or darken the overall luminance of an image. The molecules that contain threshold8 also provide a way for you to create an image that uses a specific range of grays (for example, shades of gray in the range 100-200).
Requirements: The scale operation must use as its source a single-banded child image from the SunVideo card with a band offset of 0 (a Y-only image). Scale factor must be in the form 1/x, where x is a positive, non-zero integer. If the video format is PAL, x must be greater than 1 for the X-axis scale factor.
Also, the xil_threshold() arguments must meet the following conditions:
low = 0, map = high or high + 1
or
high = 255, map = low or low - 1
Code example: display.c
decompress_UYVY ¤ colorconvert
Use: To display UYVY data on a 24-bit display.
Requirements: The colorconvert operation must use a YUV image as its source (XIL colorspace "ycc601") and an RGB image as its destination (XIL colorspace "rgb709"). The destination image must not have any ROIs, must be of the same dimensions as the CIS, and must have the default image origin (0.0, 0.0).
Code example: window.c
decompress_UYVY ¤ [rescale8 ¤] [scale8nearest ¤] ordereddither8_8
Use: To display UYVY data on an 8-bit display.
Requirements: Scale factor must be 2.
The destination image must not have any ROIs, must be of the same dimensions as the CIS, and must have the default image origin (0.0, 0.0).
The ordered dither operation must use a 4x4 dither mask.
Code example: window.c
To determine if XIL functions are executing within molecules, set the XIL_DEBUG environment variable to show_action. For example:
% setenv XIL_DEBUG show_action
This causes the XIL library to print a message to stderr whenever an operation that affects an XIL image or compressed image sequence is executed.
Here is sample "show action" output from running the rtvc_video_conference example program with different scale factors. The first sample specifies a scale factor of 1/2 (the -s 2 argument). This enables the capture ¤ [scale8nearest ¤] compress_Jpeg molecule to execute completely on the SunVideo card.
In the second sample, the scale factor is 1/4 (the -s 4 argument). Because the scale requirement for the JPEG compression molecule is 1/2, the entire molecule cannot execute on the SunVideo card and the debug output displays XIL_ACTION:FAILED. However, the scale function (with its implicit capture) still executes (as a smaller molecule) on the SunVideo card. This is shown by the line that begins with XIL_ACTION[XilDeviceComputeRtvc]. The JPEG compression function is executed in host software, as shown by the line that begins with XIL_ACTION[XilDeviceCompJpegMemory].
---------------------------------------------------------------------------------------------------
% ./rtvc_video_conference -f 1 -C Jpeg -s 2 XIL_ACTION[XilDeviceCompJpegRtvc]:compress_Jpeg(scale8nearest(capture_SUNWrtvc())) XIL_ACTION[XilDeviceCompJpegGX]:display_ioSUNWgx(ordereddither8_8(rescale8(decompress_Jpeg()))) 1 frames in 0.314387 seconds, 3.18079 frames/second % % % ./rtvc_video_conference -f 1 -C Jpeg -s 4 XIL_ACTION[XilDeviceCompJpegRtvc]:compress_Jpeg(scale8nearest(capture_SUNWrtvc())) XIL_ACTION:FAILED XIL_ACTION[XilDeviceComputeRtvc]:scale8nearest(capture_SUNWrtvc()) XIL_ACTION[XilDeviceCompJpegMemory]:compress_Jpeg() XIL_ACTION[XilDeviceCompJpegGX]:display_ioSUNWgx(ordereddither8_8(rescale8(decompress_Jpeg()))) 1 frames in 0.158051 seconds, 6.32707 frames/second ---------------------------------------------------------------------------------------------------
