The SunVideo subsystem provides support for:
These four compression techniques are highlighted in the following sections.
The non-proprietary Cell image compression technology [6]) provides high quality, low bit-rate image compression at low computational cost. Applications where Cell compression can be used include video conferencing, media distributions on CD-ROM, and multimedia mail applications.
There are two versions of the Cell image compression technology: Cell and CellB. The SunVideo card supports CellB compression. The SunVideo software includes a video-bitstream conversion tool, mpeg_to_cell, which converts an MPEG-1 bitstream to a Cell bitstream.
Cell compression, which is designed for use with movies, is computationally asymmetric; decompression requires less overhead than compression. To provide high-quality images, Cell supports Adaptive Colormap Selection, which enables an encoder to change colormaps dynamically. The bytestream of images that are compressed with Cell can be decompressed in software on any SPARCstation. Depending on compression ratios, NTSC 640x480-resolution movies can be played back at 30 frames/second (fps).
CellB, which is similar to Cell, is designed for use in video conferencing applications. To reduce computational overhead and meet the timing demands of video conferencing, CellB compression is more computationally symmetric than Cell. CellB uses two fixed codebooks and vector quantization techniques. The bytestream of a CellB image is simpler (and more compact) than that of Cell, which reduces requirements on network bandwidth.
Cell works with 3-band RGB images, with no subsampling, and requires that the width and height of the images be divisible by four. CellB takes 3-band YUV images as input, with the width and height divisible by four.
A cell encoder breaks the video into cells. A cell is 16 pixels, arranged in a 4x4 group (Figure B-1). Cells are encoded into the bytestream in scanline order, from left to right and from top to bottom.
Figure B-1 A Cell
The basic encoding scheme used in both versions of Cell is based on an image coding method called Block Truncation Coding (BTC). The 16 pixels in a cell are represented by a 16-bit mask and two intensities or colors. These values specify which intensity to place at each of the pixel positions. The mask and intensities can be chosen to maintain certain statistics of the cell, or they can be chosen to reduce contouring in a manner similar to ordered dither.
The primary advantage of BTC is that its decoding process is similar to the operation of character fonting in a color framebuffer. The character display process for a framebuffer takes as input a foreground color, a background color, and a mask that specifies whether to use the foreground or background color at each pixel. Because this function is so important to the window system, it is often implemented as a display primitive in graphics accelerators. The Cell compression technique leverages these existing primitives to provide full-motion video decoding without special hardware or modifications to the window system.
The basic component of the Cell and CellB bytestreams is the four-byte cell code. The first two bytes of the cell code are a bitmask (Figure B-2).
Figure B-2 Encoding a Cell Code
Each bit in the mask represents a pixel in a 16-bit cell. The bitmask is normalized so that the most significant bit is 0. The figure shows the relationship of the bits in the mask to the location of the pixels in a cell. A value of 0 in a mask bit means that the pixel is rendered in the background color (color 0). A value of 1 means that the pixel is rendered in the foreground color (color 1).
The last two bytes of a cell code establish a pixel's color. Cell and CellB differ in the way that pixel colors are derived. In Cell, the values in the color 0 and color 1 bytes are indexes into a colormap. In CellB, the values are pointers into Y/Y and U/V vector quantization tables. Refer to the "Cell and CellB Bytestream Definitions" appendix in the XIL Programmer's Guide for more information about the contents of a CellB bytestream.
The Joint Photographic Experts Group (JPEG) is a joint ISO/CCITT technical committee. JPEG has developed a general purpose international standard for the compression of continuous tone (grayscale or color) images.
JPEG was defined as a still-image compression technique for the archiving of digital image data. However, the introduction of high performance hardware capable of coding and decoding JPEG images in real-time has enabled the development of full-motion video applications based on JPEG.
You can obtain the ISO specifications for JPEG (ISO CD 10918) from the
American National Standards Institute in New York, NY USA (telephone
212-642-4900).
In 1988, the International Organization for Standardization (ISO) undertook an effort to develop a standard for video and associated audio on digital storage media. The digital storage mediums included CD-ROM, DAT, tape drives, hard disks, and writable optical disks. This effort is known by the name of the group responsible for defining the standard, the Moving Picture Experts Group (MPEG).
The motion estimation techniques used in MPEG-1 allows for interpolative (noncausal) coding techniques as well as the traditional predictive (causal) techniques. The motion estimation is done on a 16 x 16 pixel block basis.
MPEG-1 works with out-of-order processing, which occurs on both the encoding and decoding sides and can introduce considerable latencies. Therefore, it is generally believed that MPEG-1 is not a good choice for video telephony and video conferencing applications.
You can obtain the ISO specifications for MPEG-1 (ISO CD 11172) from the
American National Standards Institute in New York, NY USA (telephone
212-642-4900).
The SunVideo subsystem implements two MPEG-1 compressors: variable bit rate and constant bit rate. This section discusses the conditions that must be met for the SunVideo subsystem's MPEG-1 compressors to be used. If any of the conditions described are not met, the XIL compress molecule will fail and an atomic compress will be attempted. If there is not an XIL software MPEG-1 compressor, the atomic compress also will fail.
The variable bit rate MPEG-1 compressor can operate only on patterns composed of I frames. The constant bit rate MPEG-1 compressor can compress patterns that also can include P and B frames.
XIL defines an XilMpeg1Pattern as shown in the following declaration.
--------------------------------------
typedef struct {
char* pattern;
Xil_unsigned32 repeat_count;
} XilMpeg1Pattern;
--------------------------------------
The repeat_count member of the XilMpeg1Pattern structure defines how often pattern repeats before another group of pictures (GOP) starts.
The pattern string of the XilMpeg1Pattern structure is defined by two variables: intraframe distance and reference-frame distance (Table B-1).
Table B-1 Three Variables of an XilMpeg1Pattern
---------------------------------------------------------------------------
Variable Definition ---------------------------------------------------------------------------
Intraframe distance The number of frames until the next I frame
Reference-frame distance The number of frames until the next I or P frame
---------------------------------------------------------------------------
The constant bit rate MPEG-1 compressor imposes the following rules on the pattern string.
No restriction is placed on repeat_count.
Table B-2 lists examples of supported patterns.
----------------------------------------------------------
Pattern Intraframe Reference- Compresses as Frame Distance Distance ----------------------------------------------------------
I 1 1 IIIIII. . .
IPP 3 1 IPPIPPIPP. . .
IBIB 2 2 IBIBIBIB. . .
IBB 3 3 IBBIBBIBB. . .
IBPBPBPB 8 2 IBPBPBPBIBPBPB. . .
IBBPBBPBB 9 3 IBBPBBPBBIBBPBB. . .
----------------------------------------------------------
Table B-3 lists examples of unsupported patterns.
Table B-3 Unsupported Patterns
------------------------------------
Pattern Intraframe Reference- frame Distance Distance ------------------------------------
IBPBB 5 ?
IPIPP ? 1
IBIBBP ? ?
IBIBPBI ? ?
IBBBPBBB 8 4 (3)
------------------------------------
Because of the pipelined nature of the constant bit rate MPEG-1 compressor, an extra subgroup of pictures is added to the start of the first GOP in a video sequence. For example, the compressor adds an extra IB to the start of the pattern IBPB in the first group of pictures in the video sequence. Therefore, what actually gets compressed is IBIBPBIBPBIBPB. . .. For a pattern such as IBBRBB, where R is either an I or P picture, the added subgroup is IBB.
The quality of a constant bit rate MPEG-1 bitstream depends on the number of bits allocated to each frame (picture). In general, the quality for a given picture increases as the number of bits allocated to it increases. The number of bits allocated to a picture is based on the following:
With all else constant, the frame rate and bit rate effectively increase and decrease by the same percentage the number of bits allocated to each frame. For example, twice as many bits are allocated to a given frame at 15 frames per second (fps) than at 30 fps. Thus, the quality of a 15 fps bitstream will generally be better than a 30 fps bitstream at the same bit rate. Likewise, to achieve the same quality at 15 fps as at 30 fps, the bit rate at 15 fps can be set to half of the specified bit rate for 30 fps. Increasing the bit rate improves quality up to a point, at which further increases simply add padding to the size of the bitstream without improving quality. For a given compression pattern, there is also a lower bound to the bit rate at which the compressor cannot effectively go below.
The bit rate is determined by the XIL MPEG-1 CIS attribute COMPRESSOR_BITS_PER_SECOND (see the Mpeg1(3) and xil_cis_get_attribute(3) man pages).
The frame rate is derived from the SunVideo device attribute IMAGE_SKIP and the format (NTSC/PAL) of the source video. It is not determined by the rate at which video is retrieved from the SunVideo card. Therefore, a measurement of bit rate is not accurate unless the application is retrieving compressed frames from the SunVideo card at the rate specified by IMAGE_SKIP. See Chapter 9, "XIL Attributes for the SunVideo Subsystem," for a complete definition of IMAGE_SKIP.
The best allocation of bits among picture types depends on the scene content. The work of the International Organization for Standardization (ISO) MPEG-1 committee has suggested that in general, for natural scenes, P pictures should be allocated two to five times as many bits as B pictures and that I pictures should be allocated up to three times as many bits as P pictures. If there is little motion, more bits should be allocated to I pictures. If there is a lot of motion, the proportion of bits allocated to I pictures should be decreased and most of the savings given to P pictures. The SunVideo device attribute MPEG_WEIGHTS enables you to set the weights of pictures. See Chapter 9, "XIL Attributes for the SunVideo Subsystem," for a complete definition of MPEG_WEIGHTS.
For a complete discussion of the XIL MPEG-1 attribute COMPRESSOR_INSERT_VIDEO_SEQUENCE_END, see the "MPEG-1 Codec" chapter of the XIL Programmer's Guide.
For the SunVideo subsystem, the COMPRESSOR_INSERT_VIDEO_SEQUENCE_END attribute should always be set to TRUE.
The bit rate and the image size are not allowed to be changed within an MPEG- 1 video sequence. If they are to change, the current video sequence must end and a new one started. To end a video sequence, an MPEG-1 "sequence_end_code" must be added to the end of the bitstream. For this code to be added, the XIL MPEG-1 CIS attribute COMPRESSOR_INSERT_VIDEO_SEQUENCE_END must be set to TRUE.
------------------------------------------------------------------------
xil_cis_set_attribute(cis, "COMPRESSOR_INSERT_VIDEO_SEQUENCE_END", (void*) TRUE); ------------------------------------------------------------------------
To add the "sequence_end_code," you call xil_cis_flush() on the MPEG-1 CIS. This call causes any pending write operations for the CIS to complete. For SunVideo, any buffered frames are thrown away and "dummy" P frames are written to the CIS instead. [7]) The "dummy" P frames have all forward motion vectors set so that the last reference frame is repeated in the bitstream.
If an application is to change the bit rate or the image size of the bitstream, it first must call xil_cis_flush() and have COMPRESSOR_INSERT_VIDEO_SEQUENCE_END set to TRUE. An application also must call xil_cis_flush() when it has finished compressing. Calling xil_cis_flush() completes the bitstream with a "sequence_end_code" and instructs the XIL MPEG-1 CIS to release the last set of frames to the application for retrieval and for use by one of the xil_cis_get_bits functions. [8]) If an application does not add a "sequence_end_code" but the bit rate or image size of the video sequence changes, the following error results:
There are times when a new video sequence starts but the new one is not different in terms of bit rate or image size than the last one. This may happen on SunVideo, for example, if the SunVideo card was flushed because the FLUSH_BUFFERS attribute was set to TRUE. The SunVideo card will also be flushed if the weights or compression patterns of a video sequence are changed. When the constant bit rate MPEG-1 compressor is flushed, any buffered frames are thrown away and "dummy" P frames are written to the CIS instead. The compressor then starts a new video sequence.
The UYVY compression type provides access to the raw 4:2:2 data produced by the SunVideo card. This compression type is useful if you want to pass the raw data through the SunVideo card and use your own software compression on the raw data.
The UYVY compression type is supplied as part of the SunVideo packages and is not part of the normal XIL delivery. Therefore, you must have the SunVideo software (SUNWrtvcu package) installed to decompress a CIS created with the UYVY compressor. The SUNWrtvcu package includes molecules to decompress and display UYVY CIS's (see Chapter 8, "Acceleration in XIL and the SunVideo Subsystem," for information about these molecules).
UYVY data is word aligned and two pixels are stored in each UYVY word. Each of the data values (U, Y0, V, and Y1) in a word are 8 bit values.The values of U and V are the same for each pixel in a UYVY word. The Y value of a pixel depends upon whether the x-location of the pixel is even or odd. If the x- location of the pixel is even (for example, x=0), the Y value is Y0. If the x- location of the pixel is odd (for example, x=1), the Y value is Y1. Figure B-3 shows the layout of UYVY data.
Figure B-3 The Layout of UYVY Data
To access UYVY data, use the xil_cis_get_bits_ptr() function. This function returns a pointer to the beginning of the data. The following example illustrates accessing UYVY data.
----------------------------------------------------------------------------------
void
UYVY_get_pixel(XilCis cis, int x, int y)
/* Example routine to print the YUV values of a single pixel from a UYVY CIS. */
{
Xil_unsigned32 *data;
int nframes;
int nbytes;
int width;
Xil_unsigned32 uyvy;
Xil_unsigned8 uval, yval, vval;
/* get a 32 bit pointer to the raw 4:2:2 data */
data = (Xil_unsigned32 *) xil_cis_get_bits_ptr(cis, &nbytes, &nframes);
/* figure out the width of the images in the CIS */
width = xil_imagetype_get_width(xil_cis_get_output_type(cis));
/* There are two pixels stored in each UYVY word */
/* Note that the UYVY data is guaranteed to be word aligned so it is not */
/* necessary to read it a byte at a time */
uyvy = data[(y * width + x) / 2];
uval = uyvy > 24;
vval = (uyvy > 8) & 0xff;
/* if the x-location of the pixel is odd, use the second y value; */
/* otherwise, use the first y value */
yval = (x & 0x1) ? (uyvy & 0xff) : ((uyvy > 16) & 0xff);
printf("YUV = (%x,%x,%x)\n", yval, uval, vval);
/* return the read frame to where it was at the beginning of this routine */
xil_cis_seek(cis, -nframes, 1);
}
----------------------------------------------------------------------------------
Before you can use the UYVY compressor to compress images, you must create a UYVY CIS. You do this by passing the compressor name UYVY to the function xil_cis_create(). See the following code example.
-------------------------------------------
XilCis cis; XilSystemState state; cis = xil_cis_create(state, "UYVY"); -------------------------------------------
There are two UYVY-specific attributes: WIDTH and HEIGHT. These attributes are "set-only." You set these attributes using the function xil_cis_set_attribute().
If you have put compressed data into your CIS using xil_cis_put_bits() or xil_cis_put_bits_ptr(), you must set the values of these attributes to the width and height in pixels of the images to be decompressed. If you do not set these attributes, their values are 0, and an error occurs if you attempt to decompress a CIS that does not implicitly have WIDTH and HEIGHT set through a compress.
A valid value for HEIGHT is any positive integer. For WIDTH, a valid value is any positive multiple of 2. The following code example shows the WIDTH attribute set to 320 and the HEIGHT attribute set to 240.
-----------------------------------------------------------
XilCis cis; int width = 320; int height = 240; xil_cis_set_attribute(cis, "WIDTH", (void *)width); xil_cis_set_attribute(cis, "HEIGHT", (void *)height); -----------------------------------------------------------
