3.5 Loading Raster Data

To load and export imagery or raster data, you can consider third-party ETL tools (see the note in GeoRaster Tools: Viewer_ Loader_ Exporter). For example, you can use the gdal_translate command line and other GDAL utilities, which fully support GeoRaster through the Oracle Spatial GeoRaster driver.

You can also use features in GeoRaster to load raster data, With GeoRaster, you have the following options:

• Use the GDAL based ETL tool for concurrent batch loading and exporting of large amount of images in GDAL supported formats. This tool is described in GDAL-Based ETL Wizard for Concurrent Batch Loading and Exporting.

• In PL/SQL, call the SDO_GEOR_GDAL.translate procedure (for non-autonomous databases only) to load images into GeoRaster objects.

• Use the GeoRaster JAI-based loader tool or viewer tool, which are described in JAI-Based Viewer_ Loader_ and Exporter.

It is recommended to use GDAL, the GDAL-Based ETL, and SDO_GEOR_GDAL.translate to load and export image and raster files. With the last option (JAI-based tool), you can do the following:

• Compress raster data and store the data in JPEG-compressed or DEFLATE-compressed GeoRaster objects.

• Load an ESRI world file or a Digital Globe RPC text file (.rpb) into an existing GeoRaster object, and georeference the raster data without reloading it. You can also specify an SRID with the world file and generate the spatial extent of the data.

• Load a GeoTIFF format file with georeferencing, with or without raster data. To load and export the georeferencing information of GeoTIFF images, the GeoTIFF libraries are required. See Georeferencing GeoRaster Objects for instructions.

After loading raster data into a GeoRaster object, you must ensure that the object is valid by calling the SDO_GEOR.validateGeoRaster function, as explained in Validating GeoRaster Objects.

Because an ESRI world file or .rpb file does not contain coordinate system information, you can specify the SRID value of a coordinate reference system for the load operation. However, if you do not specify an SRID, the model SRID of the GeoRaster objects is set to 0 (zero) by the loader, which means that the GeoRaster object is invalid, and therefore you must use the SDO_GEOR.setModelSRID procedure to specify a valid model space for this object. If you do not yet know the coordinate system of the model space, you can specify the SRID value as 999999, which means that the coordinate reference system is unknown. (Specifically, SRID 999999 is associated with a coordinate reference system named unknown CRS.) Later, when you know the actual coordinate reference system of the model space, you can reset the SRID value accordingly.

For more information about the unknown CRS (SRID 999999) coordinate reference system, see Oracle Spatial Developer's Guide.

• Loading with Blocking and Optimal Padding
  
• Loading JPEG and JPEG 2000 Images Without Decompression
  
• Reformatting the Source Raster Before Loading
  

3.5.1 Loading with Blocking and Optimal Padding

Unless you want to load JPEG or JPEG2000 images and store them without any change, when you load an image or raster file into GeoRaster object, always consider and apply appropriate blocking of the data, because file formats might have very different blocking schemes. In general, blocking sizes should be 512x512 or larger. There is no absolute rule for the blocking sizes, but the larger the raster, the larger the blocking sizes you might use. For regular rasters, 512x512 to 2048x2048 is appropriate. For very small images (less than 1024x1024x3), no blocking may be a good choice. Avoid blocking sizes that are either too small (such as 64x64 and 128x128) or too large, and avoid extreme blocking sizes such as 0.5 (one-half), 1, or 8 rows of pixels per block. Generally, the rectangular shape of blocks should be a square or close to a square. For different applications, you might tune the blocking to balance efficient storage with optimal performance.

You should also always apply optimal padding during loading. In other words, specify blocking=OPTIMALPADDING in addition to specifying blocksize. GeoRaster applies padding to the right column and bottom row of blocks to make them the same size as other blocks. If the block size is not optimal for a specific raster, the default resulting padding would waste some storage space. When you specifyblocking=OPTIMALPADDING, all GeoRaster procedures and the ETL tools automatically adjust the GeoRaster dimension size array so that it will be optimal for reducing the amount of padding in GeoRaster object storage. The adjustment is always made around the user-specified values. See the explanation of the blocking keyword in the table in the Usage Notes for the SDO_GEOR_UTL.calcOptimizedBlockSizeprocedure.

For how to apply optimal padding when using the GDAL command line, see the following example:

gdal_translate -of georaster /images/image_1.tif \
      georaster:georaster/georaster@my_db, image_table, raster \
      -co "insert=(id,label,raster) values (1, 'image_1', sdo_geor.init('rdt_table', 1)" \
      -co blockxsize= 512 \
      -co blockysize=512 \
      -co blockbsize=3 \
      -co blocking=optimalpadding \
      -co interleave=BIP

3.5.2 Loading JPEG and JPEG 2000 Images Without Decompression

GeoRaster supports JPEG compression, in which the GeoRaster blocks are stored as JPEG files. GeoRaster also supports JPEG 2000 compression, in which the GeoRaster has a single block stored as a JPEG 2000 file. There are some special cases where you can load and export JPEG or JPEG 2000 images without decompressing and recompressing, thus improving performance significantly.

For JPEG, you can use the JAI-based GeoRaster loader to load the image directly without decompression and recompression if the image file is a JPEG file and has 1 or 3 bands, the GeoRaster object's compression type is specified as JPEG-F and no blocking is specified for the GeoRaster object's storage (that is, the GeoRaster object has only one block).

For JPEG 2000, you can use GDAL or the GDAL-based GeoRaster ETL tool to load the image directly without decompression and recompression – if the image file is a JPEG2000 file and if no parameters in use require any change to the internal structure of the JPEG 2000 file. For example, the following script loads the JPEG 2000 file directly without decompression.

gdal_translate -of georaster /images/image_3.jp2 \
      georaster:georaster/georaster@my_db,image_table,raster \
      -co "insert=(id,label,raster) values (3, 'image_3', sdo_geor.init('rdt_table', 3)" \
      -co compress=jp2-f

However, if any of the parameter in use require changing the internal structure of the JPEG 2000 data, direct loading will not be possible. The following example requires decompression and recompression, resulting in a substantial increase of the loading time.

gdal_translate -of georaster /images/image_4.jp2 \
      georaster:georaster/georaster@my_db,image_table,raster \
      -co "insert=(id,label,raster) values (4, 'image_4', sdo_geor.init('rdt_table', 4)" \
      -co compress=jp2-f \
      -co blockxsize=1024 \
      -co blockysize=1024 \
      -srcwin 100 200 1000 1000 \
      -outsize 50% 50%

3.5.3 Reformatting the Source Raster Before Loading

The GeoRaster JAI-based loader does not support source raster files in BSQ interleaving, and it might raise an "insufficient memory" error if the files are too big, and it might have other restrictions. To avoid such problems, you can reformat and reblock the source files so that they can be properly loaded. However, it is always recommended that you use GDAL or the GDAL-based ETL loader, which generally does not have such issues and requirements, before you consider the following approach.

As an example, one way to do this is to use GDAL, an Open Source raster transformation library available from http://www.gdal.org, to reformat or reblock the image or raster file so that JAI (Java Advanced Imaging) can handle it. GDAL supports GeoRaster natively and can import and export GeoRaster objects directly, and can also process GeoRaster objects; for more information, see http://www.oracle.com/technetwork/database/enterprise-edition/getting-started-with-gdal-133874.pdf. You can also use GDAL to generate TFW files. For example, execute commands such as the following two (each command on a single line) using the GDAL command line or (for batch conversion) shell:

gdal_translate -of GTiff -co "TFW=YES" -co "INTERLEAVE=PIXEL" -co "TILED=YES" D:\my_image.tif D:\my_new_image.tif
 
gdal_translate -of GTiff -co "TILED=YES" -co "TFW=YES" D:\my_image.ecw D:\my_new_image.tif

In the preceding example, the first command generates a TFW file, changes the interleaving to BIP (which is supported by JAI), and reblocks the image to 256x256. The second command converts ECW to TIFF, generates TFW, and reblocks the image.

Then use the GeoRaster loader tool (described in GeoRaster Tools: Viewer_ Loader_ Exporter) , specifying reblocking so that the image can be loaded successfully and later retrieved from the database efficiently, as in the following example (a single command):

java -Xmx1024m oracle.spatial.georaster.tools.GeoRasterLoader mymachine db11 6521 georaster georaster thin 32 T globe image "blocking=true, blocksize=(512,512,3)" "D:my_image.tif,2,RDT_15, D:\my_image.tfw,82213"

For extremely large images, you can also use GDAL to tile the image into multiple smaller image files with sizes that JAI can handle, or you use GDAL to load and export the images directly.