1.4 GeoRaster Physical Storage

GeoRaster optimizes the physical storage of metadata and data.

As mentioned in GeoRaster Data Model, GeoRaster data consists of a multidimensional matrix of cells and the GeoRaster metadata. Most metadata is stored as an XML document using the Oracle XMLType data type. The metadata is defined according to the GeoRaster metadata XML schema, which is described in GeoRaster Metadata XML Schema. The spatial extent (footprint) of a GeoRaster object is part of the metadata, but it is stored separately as an attribute of the GeoRaster object. This approach allows GeoRaster to take advantage of the spatial geometry type and related capabilities, such as using R-tree indexing on GeoRaster objects. The spatial extent is described in spatialExtent Attribute.

The GeoRaster metadata is stored using either the CLOB storage option or the binary XML storage option. The binary XML storage option for the GeoRaster metadata is the default, which saves disk space and improves performance. You can specify or change the storage option when you create a GeoRaster table.

The multidimensional matrix of cells is blocked into small subsets for large-scale GeoRaster object storage and optimal retrieval and processing. Each block is stored in a table as a binary large object (BLOB), and a geometry object (of type SDO_GEOMETRY) is used to define the precise extent of the block. Each row of the table stores only one block and the blocking information related to that block. (This blocking scheme applies to pyramids also.)

The dimension sizes (along row, column, and band dimensions) may not be evenly divided by their respective block sizes. GeoRaster adds padding to the boundary blocks that do not have enough original cells to be completely filled. The boundary blocks are the end blocks along the positive direction of each dimension. The padding cells have the same cell depth as other cells and have values equal to zero. Padding makes each block have the same BLOB size. Padding mainly applies to row and column blocks, but for multiband and hyperspectral imagery, padding can be applied to the band dimension also. For example, assume the following specification: band interleaved by line, blocking as (64,64,3), and 8 bands, each with 64 rows and 64 columns. In this case:

1. Bands 0, 1, and 2 are stored interleaved by line in the first block.

2. Bands 3, 4, and 5 are stored interleaved by line in the second block.

3. The third block holds the following in this order: line 1 of band 6, line 1 of band 7, 64 column values that are padding, line 2 of band 6, line 2 of band 7, 64 column values that are padding, and so on, until all 64 rows are stored.

However, the top-level pyramids are not padded if both the row and column dimension sizes of the pyramid level are less than or equal to one-half the row block size and column block size, respectively. See Pyramids for information about the physical storage of pyramids.

Each GeoRaster block has the same size. The dimension sizes of the blocks do not need to be a power of 2. They can be random integer values. The block sizes can be optimized automatically based on the dimension sizes of the GeoRaster object, so that each GeoRaster object uses only minimum padding space. See Table 1-1 in Storage Parameters for more information.

The raster blocks (BLOBs) contain the binary representation of the raster cell values. Specifically, floating-point cell values are represented in the IEEE 754 standard formats on supported platforms. If the cell depth is greater than 8 bits, GeoRaster cell data is stored in big-endian format in raster blocks. If the cell depth is less than 8 bits, each byte in the raster blocks contains two or more cells, so that the bits of a byte are fully filled with cell data. The cells are always filled into the byte from left to right. For example, if the cell depth is 4 bits, one byte contains two cells: the first four bits of the byte contain the value of a cell, and the second four bits contain the value of its following cell, which is determined by the interleaving type.

Based on this physical storage model, two object types are provided: SDO_GEORASTER for the raster data set and related metadata, and SDO_RASTER for each block in a raster image.

• The SDO_GEORASTER object contains a spatial extent geometry (footprint or coverage extent) and relevant metadata. A table containing one or more columns of this object type is called a GeoRaster table.

• The SDO_RASTER object contains information about a block (tile) of a GeoRaster object, and it uses a BLOB object to store the raster cell data for the block. An object table of this object type, or a relational table containing the same columns as the attributes of this object type, is called a raster data table (RDT).

The SDO_GEORASTER object stores and refers to an image or a raster data set. The SDO_RASTER object is an internal object for GeoRaster. The SDO_GEORASTER object fully encapsulates the raster data set's metadata and raster cell data, that is, a collection of SDO_RASTER objects. The relationship between the SDO_GEORASTER object and its SDO_RASTER objects is maintained by GeoRaster automatically. All interfaces of GeoRaster functions and procedures deal with the SDO_GEORASTER objects only; the SDO_RASTER objects of a SDO_GEORASTER object are internally handled automatically. The SDO_GEORASTER object is the major interface for users to build and manage a GeoRaster database; you only need to use the SDO_RASTER object to create raster data tables (RDTs).

Each SDO_GEORASTER object has a pair of attributes (rasterDataTable, rasterID) that uniquely identify the RDT and the rows within the RDT that are used to store the raster cell data for the GeoRaster object.

Figure 1-3 shows the storage of GeoRaster objects, using as an example an image of Boston, Massachusetts in a table that contains rows with images of various cities.

Figure 1-3 Physical Storage of GeoRaster Data

Description of "Figure 1-3 Physical Storage of GeoRaster Data"

As shown in Figure 1-3:

• Each row in the table of city images contains information about the image for a specific city (such as Boston), including an SDO_GEORASTER object.

• The SDO_GEORASTER object includes the spatial extent geometry covering the entire area of the image, the metadata, the raster ID, and the name of the raster data table associated with this image.

• Each row in the raster data table contains information about a block (or tile) of the image, including the block's minimum bounding rectangle (MBR) and image data (stored as a BLOB). The raster data table is described in Raster Data Table.

The SDO_GEORASTER and SDO_RASTER object types are described in detail in GeoRaster Data Types and Related Structures.

Figure 1-4 shows the physical storage of GeoRaster data and several related objects in a database.

Figure 1-4 GeoRaster Data in an Oracle Database

Description of "Figure 1-4 GeoRaster Data in an Oracle Database"

In Figure 1-4:

• Each GeoRaster object in the GeoRaster table has an associated raster data table, which has an entry for each block of the raster image.

• The BLOB with image data for each raster image block is stored separately from the raster table data. You can specify storage parameters (described in Storage Parameters) for the BLOBs.

• Each GeoRaster object has a raster data table associated with it. However, a raster data table can store blocks of multiple GeoRaster objects, and GeoRaster objects in a GeoRaster table can be associated with one or multiple raster data tables.

• GeoRaster system data (described in GeoRaster System Data Views (xxx_SDO_GEOR_SYSDATA)) maintains the relationship between the GeoRaster tables and the raster data tables.

• Indexes (standard and spatial) can be built on the GeoRaster table and raster data tables. For information about indexing GeoRaster data, see Indexing GeoRaster Objects.

• Additional information, such as ground control points (GCPs) and value attribute tables (VATs), can be related to the GeoRaster objects.

You generally maintain a one-to-many relationship between a GeoRaster table and its associated raster data tables, even though they could have a many-to-many relationship. That is, let a raster data table only contain cell data of GeoRaster objects that belong to the same GeoRaster table. A GeoRaster table can contain a large number (potentially unlimited) of GeoRaster objects. An RDT should be used to contain the raster blocks of a limited number of GeoRaster objects, depending on the size of the rasters.

The following considerations apply to table and column names that are stored in any Oracle Spatial metadata views. For example, these considerations apply to geometry tables, GeoRaster tables, raster data tables, and geometry and GeoRaster columns.

• The name must contain only letters, numbers, and underscores. For example, the name cannot contain a space ( ), an apostrophe ('), a quotation mark ("), or a comma (,).

• All letters in the names are converted to uppercase before the names are stored in geometry metadata views or GeoRaster system data (xxx_SDO_GEOR_SYSDATA) views or before the tables are accessed.

Note:

For schema names, some different considerations apply. The schema name can be in mixed case and with special characters such as space ( ) and apostrophe ('). For more information about naming rules for schemas and other objects, see "Database Object Naming Rules" in Database SQL Language Reference.

For more information about raster data tables, see Raster Data Table.

• Storage Parameters
  
• Raster Data Table
  
• Blank and Empty GeoRaster Objects
  
• Empty Raster Blocks
  
• Cross-Schema Support with GeoRaster
  

1.4.1 Storage Parameters

Several GeoRaster operations let you specify or change aspects of the storage. The relevant subprograms contain a parameter named storageParam, which is a quoted string of keywords and their values. The storageParam parameter keywords apply to characteristics of the raster data (see Table 1-1).

Note:

For any numbers in string (VARCHAR2) parameters to GeoRaster subprograms, the period (.) must be used for any decimal points regardless of the locale.

Table 1-1 storageParam Keywords for Raster Data

Keyword	Explanation
bitmapmask	Specifies whether or not bitmap masks are considered. TRUE specifies to consider any associated bitmap masks; FALSE specifies not to consider the bitmap masks. The default is TRUE for SDO_GEOR.copy, SDO_GEOR.changeFormatCopy, SDO_GEOR.mergeLayers, SDO_GEOR.scaleCopy, and SDO_GEOR.subset; the default is FALSE for SDO_GEOR.mosaic (A value of TRUE is invalid and is ignored for SDO_GEOR.mosaic.)
blocking	Specifies whether or not raster data is blocked. TRUE causes raster data to be blocked using the blocks of the specified or default blockSize value; OPTIMALPADDING is the same as TRUE except that the specified blockSize value will be adjusted to an optimal value to reduce padding space; FALSE causes raster data not to be blocked (that is, only one block will be used for the entire image). Specifying OPTIMALPADDING causes GeoRaster to call the SDO_GEOR_UTL.calcOptimizedBlockSize procedure internally. The default value for blocking is TRUE if you specify the blockSize keyword. If you specify blocking=TRUE but do not specify the blockSize keyword, the default blockSize is (512,512,B), where B is the number of bands in the output GeoRaster object. If you specify neither blocking nor blockSize, default values are derived from the source GeoRaster object: that is, if the original data is not blocked, the data in the output GeoRaster object is by default not blocked; and if the original data is blocked, the data in the output GeoRaster object is blocked with the same blocking scheme.
blockSize	Specifies the block size, that is, the number of cells per block. You must specify a value for each dimension of the output GeoRaster object. For example, blocksize=(512,512,3) specifies 512 for the row dimension, 512 for the column dimension, and 3 for the band dimension; and blocksize=(512,512) specifies row and column block sizes of 512 for a GeoRaster object that has no band dimension. The values must be non-negative integers. If a value is 0, it means the block size is the corresponding dimension size. If a value is greater than the corresponding dimension size, padding is applied. See also the explanation of the blocking keyword in this table and of the SDO_GEOR_UTL.calcOptimizedBlockSize procedure. Only regular blocking is supported; that is, all blocks must be the same size and be aligned with each other, except for some top-level pyramids. However, the dimension sizes of the blocks do not need to be a power of 2. They can be random integer values. For example, the blockSize value can be (589,1236,7). The physical storage size of a raster block must be less than or equal to 4GB.
cellDepth	Specifies the cell depth of the raster data set, which indicates the number of bits and the sign for the data type of all cells. Note, however, that changing the cell depth can cause loss of data and a reduction in precision and image quality. Must be one of the following values (_U indicating unsigned and _S indicating signed): 1BIT, 2BIT, 4BIT, 8BIT_U, 8BIT_S, 16BIT_U, 16BIT_S, 32BIT_U, 32BIT_S, 32BIT_REAL, or 64BIT_REAL. (Complex cellDepth types are not supported.) If cellDepth is not specified, the value from the source GeoRaster object is used by default. Example: celldepth=16BIT_U
compression	Specifies the compression type to be applied to the GeoRaster object. Must be one of the following values: JPEG-F, DEFLATE, or NONE. (You can use NONE to decompress a compressed GeoRaster object.) If compression is not specified, the compression type of the source GeoRaster object is used. For more information about compression and decompression, see Compression and Decompression. Example: compression=JPEG-F If the source GeoRaster object is blank, the compression keyword is ignored, except for the SDO_GEOR.getRasterSubset and SDO_GEOR.getRasterData functions. (Blank GeoRaster objects are explained in Blank and Empty GeoRaster Objects.)
interleaving	Specifies the interleaving type. (Interleaving is explained in Bands_ Layers_ and Metadata.) Must be one of the following values: BSQ (band sequential), BIL (band interleaved by line), or BIP (band interleaved by pixel). Example: interleaving=BSQ
parallel	Specifies the degree of parallelism for the compression operation. (This parameter is ignored when a subprogram call specifies the parallelParam parameter.) If specified, must be in the form parallel=n, where n is greater than 1. Must be used with the compression storage parameter. Parallelism is supported for the following compression operations: From NONE to JPEG-F From NONE to DEFLATE From JPEG-F to NONE From DEFLATE to NONE Parallelism is not supported for the following compression operations: From JPEG-F to DEFLATE From DEFLATE to JPEG-F
pyramid	TRUE specifies to keep the original pyramid data; FALSE specifies not to keep the original pyramid data. The default value depends on the specific procedure: the default is TRUE for SDO_GEOR.copy and SDO_GEOR.changeFormatCopy; the default is FALSE for SDO_GEOR.scaleCopy, SDO_GEOR.mosaic, and SDO_GEOR.subset. (A value of TRUE is invalid and is ignored for SDO_GEOR.scaleCopy or SDO_GEOR.subset.) You cannot generate pyramid data through the use of storage parameters; instead, you must use the SDO_GEOR.generatePyramid procedure after creating the GeoRaster object.
quality	Specifies the JPEG compression quality, which is the degree of lossiness caused by the compression. Must be an integer from 0 (lowest quality) through 100 (highest quality) to be applied to the GeoRaster object. The default value is 75. For more information about compression quality, see JPEG Compression of GeoRaster Objects. Example: quality=80

Example 1-1 shows a GeoRaster object being copied, with its block size changed and any pyramid data from the original object not copied.

Example 1-1 Using storageParam Keywords

DECLARE
   gr1 sdo_georaster;
   gr2 sdo_georaster;
BEGIN
INSERT INTO georaster_table (georid, georaster) VALUES (2, sdo_geor.init('RDT_1'))
       RETURNING georaster INTO gr2;
SELECT georaster INTO gr1 FROM georaster_table WHERE georid=1;
sdo_geor.changeFormatCopy(gr1, 'blocking=OPTIMALPADDING
   blocksize=(512,512) pyramid=FALSE', gr2);
UPDATE georaster_table SET georaster=gr2 WHERE georid=2;
COMMIT;
END;
/

In Example 1-1, the raster data table for GeoRaster object gr2 is RDT_1. If raster data is to be written into table RDT_1, that table must exist before the PL/SQL block is run; otherwise, an error is generated by the SDO_GEOR.changeFormatCopy procedure.

Note:

If you insert, update, or delete GeoRaster cell data or metadata, update the GeoRaster object before committing the transaction, as shown in Example 1-1 and as explained in Updating GeoRaster Objects Before Committing.

Example 1-1 and many examples in SDO_GEOR Package Reference refer to a table named GEORASTER_TABLE, which has the following definition:

CREATE TABLE georaster_table
( georid    NUMBER PRIMARY KEY,
  name      VARCHAR2(32),
  georaster SDO_GEORASTER );

1.4.2 Raster Data Table

A raster data table (RDT) must be an object table of SDO_RASTER type, or a relational table with the following column definitions:

  rasterID          NUMBER,
  pyramidLevel      NUMBER,
  bandBlockNumber   NUMBER,
  rowBlockNumber    NUMBER,
  columnBlockNumber NUMBER,
  blockMBR          SDO_GEOMETRY,
  rasterBlock       BLOB

The RDT, whether an object table or a relational table, must have the primary key defined on the columns (rasterID, pyramidLevel, bandBlockNumber, rowBlockNumber, columnBlockNumber).

Each RDT name must be or equivalent to a valid nonquoted identifier, and it will be stored in the GeoRaster sysdata views and in the SDO_GEORASTER objects in all uppercase characters, without any schema prefix. (Each GeoRaster column name must be or equivalent to a valid nonquoted identifier, and it is stored in the GeoRaster sysdata views in all uppercase characters.)

Note, the RDTs used or referenced by the GeoRaster objects in a GeoRaster table must be in the same schema as that of the GeoRaster table. The name of each RDT must be unique and the pair of (rasterDataTable, rasterID) of a GeoRaster object must be unique in the database in order to support cross-schema manipulation and ensure data integrity of GeoRaster objects. When you initiate a GeoRaster object using the SDO_GEOR.init and SDO_GEOR.createBlank functions without specifying the RDT name and RID number, those functions will automatically generate a rasterDataTable name and a rasterID number for the new GeoRaster object to ensure the uniqueness requirements. You can use the SDO_GEOR_ADMIN.isRDTNameUnique function to check if an RDT name is already used by GeoRaster before you create it. You can use the SDO_GEOR_UTL.renameRDT procedure to rename an existing RDT already used by GeoRaster to a different name. To resolve any duplication in raster data table names, you can use the SDO_GEOR_ADMIN.maintainSysdataEntries function. For transferring GeoRaster data between databases, see Transferring GeoRaster Data Between Databases for information on how to resolve conflicts.

Creating a raster data table enables you to control the placement and storage characteristics of the RDT (for example, if the table should be partitioned for better performance). For a large GeoRaster object, consider putting its raster data in a separate raster data table and partitioning the raster data table by pyramid level or block numbers, or both; however, always consider sharing an RDT for a certain number of smaller GeoRaster objects to avoid creating too many RDTs. Do not use the SYSTEM tablespace for storing GeoRaster tables and raster data tables. Instead, create separate locally managed (the default) tablespaces for GeoRaster tables.

Never insert or delete any rows directly in a raster data table. The rows in the appropriate RDTs are automatically inserted or deleted when GeoRaster objects are created with raster data or deleted from a GeoRaster table.

In choosing block sizes for raster data, consider the following:

• The maximum length of a raster block is 4 GB; therefore, do not specify a block size greater than 4 GB.

• Consider the cellDepth value of the GeoRaster object when you calculate the desired size for a raster block.

• Choosing an appropriate block size is a trade-off between the size of a raster block and the number of blocks needed for a GeoRaster object. For raster data of a large size, Oracle recommends at least 512 by 512 for the row and column dimension sizes. A blocking size value that results in a raster block smaller than or close to 4 KB (such as 64 by 64) is usually a bad choice, because 4 KB is the threshold for storing an Oracle BLOB out-of-line.

For information about creating object or relational raster data tables, see Creating Raster Data Tables.

1.4.3 Blank and Empty GeoRaster Objects

A blank GeoRaster object is a special type of GeoRaster object in which all cells have the same value. There is no need to store its cells in any SDO_RASTER block; instead, the cell value is registered in the metadata in the blankCellValue element. Otherwise, blank GeoRaster objects are treated in the same way as other GeoRaster objects. Use the SDO_GEOR.createBlank function to create a blank GeoRaster object, the SDO_GEOR.isBlank function to check if a GeoRaster object is a blank GeoRaster object, and the SDO_GEOR.getBlankCellValue function to return the value of the cells in a blank GeoRaster object.

An empty GeoRaster object contains only a rasterDataTable name and a rasterID. To create an empty GeoRaster object, use the SDO_GEOR.init function. You must create an empty GeoRaster object before you perform an action that outputs a new GeoRaster object, so that the output can be stored in the previously initialized empty GeoRaster object.

1.4.4 Empty Raster Blocks

GeoRaster supports empty raster blocks to save storage space with large mosaic objects and to improve raster processing speed. Empty raster blocks are used when there is no raster data available for a specific raster block of a large GeoRaster object. Such GeoRaster data is of a special sparse data type. There is still an entry in the raster data table for each empty raster block, but the length of the BLOB is zero (indicating empty).

When a GeoRaster operation (for example, SDO_GEOR.changeCellValue, SDO_GEOR.changeFormatCopy, SDO_GEOR.generatePyramid, SDO_GEOR.getRasterData, SDO_GEOR.getRasterSubset, SDO_GEOR.mergeLayers, SDO_GEOR.mosaic, SDO_GEOR.scaleCopy, SDO_GEOR.subset, or SDO_GEOR.updateRaster) is applied to a source GeoRaster object with empty raster blocks, it may lead to empty or partially empty result raster blocks.

A resulting raster block is empty if all the cells in it are derived from empty source raster blocks. A resulting raster block is partially empty if only some of the cells in it are derived from empty source raster blocks. Any cells in a partially empty result raster block that are derived from an empty source raster block are either set to certain background values (as specified in the bgValues parameter) or set to 0 (if the bgValues parameter is not specified). Once this is done, a partially empty raster block becomes just like a normal non-empty raster block; and after the operation is finished, each raster block in the resulting GeoRaster object is either empty or non-empty.

Because the filling of partially empty raster blocks changes the raster data permanently, you should carefully choose consistent background values when manipulating a GeoRaster object. The NODATA values stored in the GeoRaster metadata, if present, are good choices for background values, although you can also select other background values as long as they are used consistently.

If a GeoRaster object has empty raster blocks, its pyramid data may not contain any empty raster blocks at all because partially empty raster blocks are filled with background values or 0 during the SDO_GEOR.generatePyramid operation. When you call this function to generate the pyramid, be careful in choosing a consistent background value, as explained in this section.

A bitmap mask (see Bitmap Masks) can also have empty raster blocks, with the missing cell values indicating 0. If filling is required, the missing cells are always filled with the value 0.

1.4.5 Cross-Schema Support with GeoRaster

A GeoRaster table and its associated raster data table or tables must have the same owner. However, users with appropriate privileges can create GeoRaster tables and associated raster data tables owned by other schemas, and they can also create, query, update, and delete GeoRaster objects owned by other schemas. For cross-schema query of GeoRaster objects, you must have the SELECT or READ privilege on the GeoRaster tables and their associated raster data tables. For cross-schema update of GeoRaster objects, you must have the SELECT or READ privilege and the INSERT, UPDATE, and DELETE privileges on the GeoRaster tables and their associated raster data tables.

The ALL_SDO_GEOR_SYSDATA view (described in GeoRaster System Data Views (xxx_SDO_GEOR_SYSDATA)) contains information about all GeoRaster objects accessible to the current user. For each object listed, the GeoRaster table must be accessible by the current user. If the current user also needs to access the raster data, that user must also have the appropriate privileges on the associated raster data table.

All SDO_GEOR subprograms can work on GeoRaster objects defined in schemas other than the current connection schema.

For examples of cross-schema GeoRaster operations, see Performing Cross-Schema Operations.