KCMS Test Suite User's Guide

Chapter 2 Running KCMS Test Scripts

In This Chapter

This chapter explains the basic information you need to run the KCMS test scripts. It describes the file hierarchy of the testing environment, introduces you to the script commands, and shows the basic script command format. Finally it provides two methods of running the test scripts: one using the kcmstest command and a second, using automated script files.

Getting Started

Packaging

To run the KCMS test suite, you first must install the Solaris operating system. It includes the KCMS Software Development Kit (SDK) package, which contains the KCMS "C" API functions.

The KCMS test suite is a packaged in the KCMS Driver Development Kit (DDK). When you package add the DDK, the test suite files are installed in the /opt/SUNWddk/kcms/kcmstest directory.

Environment Variables

To run the scripts, you need to know about two environment variables: KCMSROOT and KCMS_PROFILES.

KCMSROOT specifies the path to the top of the kcmstest directory.

KCMS_PROFILES specifies the path to the kcmstest/profiles directory. See Figure 2-1.

Prior to running test scripts using the kcmstest command, you set these variables from the command line, for example

%setenv KCMSROOTpath

where path is the path to the kcmstest directory.

Alternately, if you run the automated script files, you set the variables at the time you run the scripts. See "Using Automated Script Files To Run Test Scripts" for details.

Required File Hierarchy

Figure 2-1 shows the required directory structure you need to run test scripts. When you package add the test suite, the kcmstest directory contains the structure shown in the figure.

Figure 2-1 `kcmstest` File Hierarchy

`kcmstest` Directory

The kcmstest directory is at the top of the test suite hierarchy in Figure 2-1. It contains the executables necessary to run the test suite.

The initialization file icc.ini in this directory lists the all the default test scripts that are packaged with the test suite. See "Initialization File" for details on the contents of this file.

Significant Directories

Four directories shown in Figure 2-1 are of particular significance. These are

script
profiles
data
log

The script directory contains the test scripts to be executed. By default, this directory includes all the test scripts listed in icc.ini. You can run a subset of the scripts, or specify an alternate initialization file when you run the kcmstest command. See "Using kcmstest To Run Test Scripts" for details. If you have written customized versions of scripts to test your CMM, you must install them in this directory.

The profiles directory contains a default set of profiles used with the default test scripts. You can install the profiles used by your CMM into this directory. Note that this is a separate installation from the one you do to make your CMM profiles available to the KCMS framework. For details, see Chapter 6, Putting It All Together.

The data directory contains measurement and calibration data.

The log directory contains output. This directory initially is empty. It holds the results of running test scripts.

Images

The images directory contains images resulting from running the test suite and test TIFF images.

Initialization File

The default initialization file icc.ini is shown in Example 2-1.

Example 2-1 Initialization file `icc.ini`

[Verbose] 
[ProfilePath]
profiles/ 
[ImagePath] 
images/ 
[DataPath] 
data/ 
[NumberOfTests] 
30 
[Tests]
IC_lhints.scr 
IC_conerr.scr 
IC_lana.scr 
IC_eval.scr 
IC_lmany.scr
IC_optspeed.scr 
IC_connect.scr 
IC_evalmany.scr 
IC_attr1.scr 
IC_attr2.scr
IC_layouts.scr 
IC_conmany.scr 
IC_optsize.scr 
IC_evalerr.scr 
IC_update1.scr
IC_update2.scr 
IC_xprofile.scr 
IC_xprofilehost.scr 
IC_xprofilesav.scr
IC_xprofilesavremote.scr 
IC_xwindow.scr 
IC_xwindowerr.scr 
IC_xdisplay.scr
IC_evalplus.scr 
IC_pacbug.scr 
IC_loadsol.scr 
IC_sun_update.scr 
IC_gray.scr
IC_gamut.scr 
IC_lut.scr

Note -

The icc.ini file does not include the tests, IC_xprofilesavroot.scr and IC_updatewin.scr, which must be run as root. To run IC_xprofilesavremote.scr, you need to change the DISPLAY environment variable. See the comments in the automated test scripts (auto-kcmstest and auto-kcmstest-root) for details.

The icc.ini file contains the path to the profiles, images, and data required to run the test scripts. In addition, it lists the number of test scripts following the [NumberOfTests] field, and it lists the filename of each test script.

If your CMM requires a different set of test scripts, you can create an alternate initialization file. Say, for example, you edited several of the scripts to test special features of your CMM. In such a case you need to install the scripts you plan to test with in the script directory. To add to the existing initialization file, you also must create an alternate file that reflects test script changes. See "Creating An Alternate Initialization File" for details.

Creating An Alternate Initialization File

You can create an alternate initialization file if, for example, you customized scripts for your CMM.

To create the file (see Example 2-1),

Use a text editor to save a copy of icc.ini under a new filename, for example alternate.ini.

Add (or remove) test script name(s) in the file list.

Change the value immediately following the [NumberOfTests] field to update the number of tests.

KCMS Test Script Commands

In general, a KCMS test script command corresponds to each of the KCMS "C" API functions. Additionally, there are some commands that are necessary to facilitate scripting and reading the test results log. See Chapter 3, KCMS Test Suite Commands for a detailed description of each command. Table 2-1 lists each of the script commands and the KCMS "C" API function to which it corresponds.

Table 2-1 Test Script Commands and "C" API Functions


Test Script Command	KCMS "C" API Function
`CONNECT:`	`KcsConnectProfile()`
`CREATE:`	`KcsCreateProfile()`
`EVAL:`	`KcsEvaluate()`
`FREE:`	`KcsFreeProfile()`
`GETATTR:`	`KcsGetAttribute()`
`LOAD:`	`KcsLoadProfile()`
`LOG:`	No specific function. It writes to a log file.
`MODIFYLH:`	`KcsModifyLoadHints()`
`OPTIMIZE:`	`KcsOptimizeProfile()`
`SAVE:`	`KcsSaveProfile()`
`SETATTR:`	`KcsSetAttribute()`
`UPDATE:`	`KcsUpdateProfile()`

Script Command Format

A single script command consists of the command name (including the colon), followed by one or more keyword/value pairs. A keyword is separated from its value by an equal sign (=). Each keyword/value pair ends with a semicolon (;).

The basic script command format is shown below:

COMMAND_NAME:keyword=value; keyword=value;

You can free-format test scripts. That is, you can insert any whitespace character into any script command.

Example 2-2 shows an actual test script that demonstrates some of the script commands and their associated keywords and values.

Example 2-2 Sample Test Script Showing Commands

LOAD:Reference=scanner; Profile=mtk600zs.inp; Handling=File; 
LoadHint=AllNow; 
LOAD:Reference=monitor; Profile=sony16.mon; Handling=File; 
LoadHint=AllNow; 
CONNECT:NAME=scan-mon;    
    Count=2;
    Reference=scanner;
    Reference=monitor;     
    Operation=FORWARD; 
EVAL:Reference=scan-mon;     
    SourcePixLayout=RGBInterLeaved;
    DestPixLayout=RGBInterLeaved;     
    Callbacks=;
    ImageIn=rhg_mtek600;
    ImageOut=rhg_mon.tst;     
    Operation=Forward; 
FREE:Reference=scanner; 
FREE:Reference=monitor;
FREE:Reference=scan-mon;

Using `kcmstest` To Run Test Scripts

The kcmstest command is a test script interpreter that reads test scripts and performs the KCMS "C" API function calls based upon the commands in the test script.

To run test scripts with this command, you use the procedure described below. For details on kcmstest, see the manual page.

Starting the `kcmstest` Command

Note -

Be sure to set the KCMSROOT environment variable before using the kcmstest command. See "Environment Variables" for details.

The simplest way to start kcmstest is to type the following from a command shell and press Return.

%kcmstest

You are prompted with the following message:

Enter the script name to be executed or "quit" to exit 
Script Name(s)?

You can enter the name of a script, for example IC_attrl.scr. Alternately, you can enter all, which executes all the scripts listed in icc.ini.

Note -

You must perform a few tasks manually to be able to run all the test scripts when you enter all. See the contents of the auto-kcmstest script for details.

When you run individual test scripts, an output log file is generated for each script. When you run all the scripts listed in icc.ini, a single log file is generated. See "Recording Test Script Results To a Log File" for details.

Note -

Use the test script auto-kcmstest to run the entire icc.ini test list. (See "Using Automated Script Files To Run Test Scripts".) The script creates certain setup files automatically.

Command Line Options -i, -h, -s

From the command line, you can enter various options to the kcmstest command. Three frequently used options are -i, -h, and -s.

To specify your own initialization file, you can enter its name on the command line preceded by the -i option, for example

%kcmstest -i
optional.ini

See "Creating An Alternate Initialization File" for details on alternate initialization files.

You can use the -s option to specify a script name (or all) and the -h option, to specify an alternate legal remote host name for scripts that test remote host access. The -h option attempts to pull a profile from the default directories on the remote host. Be sure that host has these directories and profiles.

The following example specifies the alternate initialization file alternate.ini, the script IC_attrl.scr, and the alternate host name dusk:

%kcmstest -i
alternate.ini -s IC_attrl.scr -h dusk

The example below defaults to using the icc.ini file:

%kcmstest -s
all

In this example, if any of the scripts in the icc.ini file access a remote host, the host name will be NULL and the scripts will fail.

Script Display

As each of the test script commands is executed, information about the command that is currently being interpreted is displayed to the command shell window as well as written to a log file in the kcmstest/log directory.

Recording Test Script Results To a Log File

For each script file executed, results are recorded in a log file. All the log files can be found in the kcmstest/log directory. The log file name is the name of the script file, with the .scr file extension replaced by the .log extension. If, for example, the test script name is IC_eval.scr, the log file name is IC_eval.log.

One exception to this naming scheme is if you enter all as the test script name. See "Starting the kcmstest Command" for details on this entry. In this case, the log file name is testall.log.

Two versions of a log file may exist at any given time: the current and the previous version. The previous version has its extension changed to .bak.

Example 2-3 is the log file output created from the test script shown in Example 2-2.

Example 2-3 Log File Output

Parsing a KcsLoadProfile Command
         Profile Reference = scanner
         Profile File Name = kcmsEKls3510.inp
         Profile Handling = By File
         Profile Load Hint = LoadWhenNeeded;    
         Profile Load Hint = UnLoadwhenNeeded;   
         Profile Operation Hint = Image;    
         Load Hint =   2024000   
         Thu Jul 25 08:16:07 1996
         
Completed KcsLoadProfile command, status =    0
         Thu Jul 25 08:16:07 1996  
         
Parsing a KcsLoadProfile Command         
         Profile Reference = printer 
         Profile File Name = kcmsEKsunnws.out        
         Profile Handling = By File 
         Profile Load Hint = LoadWhenNeeded;  
         Profile Load Hint = UnLoadwhenNeeded; 
         Profile Operation Hint = Image; 
         Load Hint =   2024000         
         Thu Jul 25 08:16:07 1996  
         
Completed KcsLoadProfile command, status =    0
         Thu Jul 25 08:16:07 1996  

Parsing a KcsConnectProfiles Command
         Profile Reference = scan-print   
         Number of Profiles in Connect = 2         
         Profile Reference = scanner    
         Profile Reference = printer         
         Operation Hint =     20001          
         Thu Jul 25 08:16:07 1996  

Completed KcsConnectProfiles command, status =    0     
         Thu Jul 25 08:16:08 1996    
         
Parsing a KcsEvaluate Command
         Profile Reference = scan-print
         Source Layout = RGBInterLeaved;
         Destination Layout = RGBInterLeaved;
         Input Image Name = macbeth_1550.tif
         Output Image Name = None
         Operation Hint = 20001         
         Thu Jul 25 08:16:08 1996  
         
Completed KcsEvaluate command, status =    0
         364800.000000 pixels processed in 0.621338 seconds.         
         The processing rate = 587120.062500 pixels/second.

Parsing a Free Profile command
          Profile reference =scanner  
          
Completed KcsFreeProfile command, status =    0

Parsing a Free Profile command
          Profile reference = printer  
          
Completed KcsFreeProfile command, status =    0 

Parsing a Free Profile command
          Profile reference = scan-print  
          
Completed KcsFreeProfile command, status =    0

Status Codes

If at any time during script execution, a KCMS framework API function call returns with an unexpected status code, the test is immediately aborted. For a list of all the status codes strings and their values, see Appendix A, Status Codes.

Note -

It may be your intention to have a status code returned that indicates an error because you deliberately set up a script to test an error condition. The script commands provide the optional keyword XStatus, which allows you to do this. For details, see the script command descriptions in Chapter 3, KCMS Test Suite Commands . Also see "Checking Status Codes".

Using Automated Script Files To Run Test Scripts

The kcmstest directory includes two automated scripts: auto-kcmstest and auto-kcmstest-root.

Note -

See "Tips on Running the Automated Test Scripts" before using this testing method.

Using `auto-kcmstest`

The auto-kcmstest script allows you to run the complete test suite in icc.ini, including scripts in the icc.ini file list that access a remote host. This script is located in the kcmstest directory.

Note -

You may need to edit the script to change path information.

To run this script, do not set the environment variable KCMSROOT with the setenv command. Instead, provide two arguments: the KCMSROOT environment variable as the first argument and the remote host name as the second, for example

%auto-kcmstest
/opt/SUNWddk/kcms dusk

In this example, /opt/SUNWddk/kcms is the KCMSROOT environment variable and dusk is the remote host name. Note that if you are in the directory where auto-kcmstest() is located, only the host argument is needed, for example

%auto-kcmstest
dusk

Using `auto-kcmstest-root`

Certain test scripts require that you be root to run them. You would use these tests if, for example, you wanted to create an X Window System profile in a root-owned directory. To run these scripts, a second automated script called auto-kcmstest-root is provided.

To run the auto-kcmstest-root script,

Become superuser.
```
%su
```

Provide one argument: the KCMSROOT environment variable, for example

#./auto-kcmstest-root /opt/SUNWddk/kcms

Note that if you are in the directory containing auto-kcmstest-root, no argument is required.

Getting a Failure and Performance Report

After you have run the complete test suite using auto-kcmstest and auto-kcmstest-root, you can get an automated failure and performance report by running the kcms-testreport command. This command takes two arguments: the name of the test log and the report title. Very likely, you would redirect output to a file of the same name as the report title, for example

%kcms-testreport
log/testall.log my_test_1 > my_test_2

In this example, my_test_1 is the report title and my_test_2 is the output filename.

Tips on Running the Automated Test Scripts

The following is a suggested sequence for running a complete test suite using the automated script files:

Run the auto-kcmstest script, for example
```
%auto-kcmstest
/opt/SUNWddk/kcms dusk
```

Become root, for example
```
%su
```

Run the auto-kcmstest-root script, for example
```
#./auto-kcmstest-root
/opt/SUNWddk/kcms 
```

Run kcms-testreport and redirect output to a file, for example
```
#kcms_testreport
log/testall.log my_test_1 > my_test_2 
```
Note -
auto-kcmstest-root must be run after auto-kcmstest because it appends its resulting logs to the auto-kcmstest log file.

You may want to redirect the automated-test-script output to a file, as it is quite lengthy.

Chapter 2 Running KCMS Test Scripts

In This Chapter

Getting Started

Packaging

Environment Variables

Required File Hierarchy

Figure 2-1 kcmstest File Hierarchy

kcmstest Directory

Significant Directories

Images

Initialization File

Example 2-1 Initialization file icc.ini

Creating An Alternate Initialization File

KCMS Test Script Commands

Script Command Format

Example 2-2 Sample Test Script Showing Commands

Using kcmstest To Run Test Scripts

Starting the kcmstest Command

Command Line Options -i, -h, -s

Script Display

Recording Test Script Results To a Log File

Example 2-3 Log File Output

Status Codes

Using Automated Script Files To Run Test Scripts

Using auto-kcmstest

Using auto-kcmstest-root

Getting a Failure and Performance Report

Tips on Running the Automated Test Scripts

Figure 2-1 `kcmstest` File Hierarchy

`kcmstest` Directory

Example 2-1 Initialization file `icc.ini`

Using `kcmstest` To Run Test Scripts

Starting the `kcmstest` Command

Using `auto-kcmstest`

Using `auto-kcmstest-root`