KCMS Test Suite User's Guide

Chapter 2 Running KCMS Test Scripts

In This Chapter

This chapter explains the basic information you need to run the KCMS test scripts. It describes the file hierarchy of the testing environment, introduces you to the script commands, and shows the basic script command format. Finally it provides two methods of running the test scripts: one using the kcmstest command and a second, using automated script files.

Getting Started

Packaging

To run the KCMS test suite, you first must install the Solaris operating system. It includes the KCMS Software Development Kit (SDK) package, which contains the KCMS "C" API functions.

The KCMS test suite is a packaged in the KCMS Driver Development Kit (DDK). When you package add the DDK, the test suite files are installed in the /opt/SUNWddk/kcms/kcmstest directory.

Environment Variables

To run the scripts, you need to know about two environment variables: KCMSROOT and KCMS_PROFILES.

KCMSROOT specifies the path to the top of the kcmstest directory.

KCMS_PROFILES specifies the path to the kcmstest/profiles directory. See Figure 2-1 .

Prior to running test scripts using the kcmstest command, you set these variables from the command line, for example

%setenv KCMSROOTpath

where path is the path to the kcmstest directory.

Alternately, if you run the automated script files, you set the variables at the time you run the scripts. See "Using Automated Script Files To Run Test Scripts" for details.

Required File Hierarchy

Figure 2-1 shows the required directory structure you need to run test scripts. When you package add the test suite, the kcmstest directory contains the structure shown in the figure.

Figure 2-1 `kcmstest` File Hierarchy

`kcmstest` Directory

The kcmstest directory is at the top of the test suite hierarchy in Figure 2-1 . It contains the executables necessary to run the test suite.

The initialization file icc.ini in this directory lists the all the default test scripts that are packaged with the test suite. See "Initialization File" for details on the contents of this file.

Significant Directories

Four directories shown in Figure 2-1 are of particular significance. These are

script
profiles
data
log

The script directory contains the test scripts to be executed. By default, this directory includes all the test scripts listed in icc.ini. You can run a subset of the scripts, or specify an alternate initialization file when you run the kcmstest command. See "Using kcmstest To Run Test Scripts" for details. If you have written customized versions of scripts to test your CMM, you must install them in this directory.

The profiles directory contains a default set of profiles used with the default test scripts. You can install the profiles used by your CMM into this directory. Note that this is a separate installation from the one you do to make your CMM profiles available to the KCMS framework. For details, see Chapter 6, Putting It All Together .

The data directory contains measurement and calibration data.

The log directory contains output. This directory initially is empty. It holds the results of running test scripts.

Images

The images directory contains images resulting from running the test suite and test TIFF images.

Initialization File

The default initialization file icc.ini is shown in Example 2-1 .

Example 2-1 Initialization file `icc.ini`

[Verbose] [ProfilePath]
profiles/ [ImagePath] images/ [DataPath] data/ [NumberOfTests] 30 [Tests]

IC_lhints.scr IC_conerr.scr IC_lana.scr IC_eval.scr IC_lmany.scr

IC_optspeed.scr IC_connect.scr IC_evalmany.scr IC_attr1.scr IC_attr2.scr

IC_layouts.scr IC_conmany.scr IC_optsize.scr IC_evalerr.scr IC_update1.scr

IC_update2.scr IC_xprofile.scr IC_xprofilehost.scr IC_xprofilesav.scr

IC_xprofilesavremote.scr IC_xwindow.scr IC_xwindowerr.scr IC_xdisplay.scr

IC_evalplus.scr IC_pacbug.scr IC_loadsol.scr IC_sun_update.scr IC_gray.scr

IC_gamut.scr IC_lut.scr

Note -

The icc.ini file does not include the tests, IC_xprofilesavroot.scr and IC_updatewin.scr, which must be run as root. To run IC_xprofilesavremote.scr, you need to change the DISPLAY environment variable. See the comments in the automated test scripts (auto-kcmstest and auto-kcmstest-root) for details.

The icc.ini file contains the path to the profiles, images, and data required to run the test scripts. In addition, it lists the number of test scripts following the [NumberOfTests] field, and it lists the filename of each test script.

If your CMM requires a different set of test scripts, you can create an alternate initialization file. Say, for example, you edited several of the scripts to test special features of your CMM. In such a case you need to install the scripts you plan to test with in the script directory. To add to the existing initialization file, you also must create an alternate file that reflects test script changes. See "Creating An Alternate Initialization File" ," for details.

Creating An Alternate Initialization File

You can create an alternate initialization file if, for example, you customized scripts for your CMM.

To create the file (see Example 2-1 ),

Use a text editor to save a copy of icc.ini under a new filename, for example alternate.ini.

Add (or remove) test script name(s) in the file list.

Change the value immediately following the [NumberOfTests] field to update the number of tests.

KCMS Test Script Commands

In general, a KCMS test script command corresponds to each of the KCMS "C" API functions. Additionally, there are some commands that are necessary to facilitate scripting and reading the test results log. See Chapter 3, KCMS Test Suite Commands ," for a detailed description of each command. Table 2-1 lists each of the script commands and the KCMS "C" API function to which it corresponds.

Table 2-1 Test Script Commands and "C" API Functions


Test Script Command	KCMS "C" API Function
`CONNECT:`	`KcsConnectProfile()()`
`CREATE:`	`KcsCreateProfile()()`
`EVAL:`	`KcsEvaluate()()`
`FREE:`	`KcsFreeProfile()()`
`GETATTR:`	`KcsGetAttribute()()`
`LOAD:`	`KcsLoadProfile()()`
`LOG:`	No specific function. It writes to a log file.
`MODIFYLH:`	`KcsModifyLoadHints()()`
`OPTIMIZE:`	`KcsOptimizeProfile()()`
`SAVE:`	`KcsSaveProfile()()`
`SETATTR:`	`KcsSetAttribute()()`
`UPDATE:`	`KcsUpdateProfile()()`

Script Command Format

A single script command consists of the command name (including the colon), followed by one or more keyword/value pairs. A keyword is separated from its value by an equal sign (=). Each keyword/value pair ends with a semicolon(;).

The basic script command format is shown below:

COMMAND_NAME:keyword=value; keyword=value;

You can free-format test scripts. That is, you can insert any whitespace character into any script command.

Example 2-2 shows an actual test script that demonstrates some of the script commands and their associated keywords and values.

Example 2-2 Sample Test Script Showing Commands

LOAD:Reference=scanner;
Profile=mtk600zs.inp; Handling=File; LoadHint=AllNow; LOAD:Reference=monitor;

Profile=sony16.mon; Handling=File; LoadHint=AllNow; CONNECT:NAME=scan-mon;    

 Count=2;      Reference=scanner;      Reference=monitor;     

Operation=FORWARD; EVAL:Reference=scan-mon;     

SourcePixLayout=RGBInterLeaved;      DestPixLayout=RGBInterLeaved;     

Callbacks=;      ImageIn=rhg_mtek600;      ImageOut=rhg_mon.tst;     

Operation=Forward; FREE:Reference=scanner; FREE:Reference=monitor;

FREE:Reference=scan-mon;

Using `kcmstest` To Run Test Scripts

The kcmstest command is a test script interpreter that reads test scripts and performs the KCMS "C" API function calls based upon the commands in the test script.

To run test scripts with this command, you use the procedure described below. For details on kcmstest, see the manual page.

Starting the `kcmstest` Command

Note -

Be sure to set the KCMSROOT environment variable before using the kcmstest command. See "Environment Variables" for details.

The simplest way to start kcmstest is to type the following from a command shell and press Return.

%kcmstest

You are prompted with the following message:

Enter
the script name to be executed or "quit" to exit Script

Name(s)?

You can enter the name of a script, for example IC_attrl.scr. Alternately, you can enter all, which executes all the scripts listed in icc.ini.

Note -

You must perform a few tasks manually to be able to run all the test scripts when you enter all. See the contents of the auto-kcmstest script for details.

When you run individual test scripts, an output log file is generated for each script. When you run all the scripts listed in icc.ini, a single log file is generated. See "Recording Test Script Results To a Log File" for details.

Note -

Use the test script auto-kcmstest to run the entire icc.ini test list. (See "Using Automated Script Files To Run Test Scripts" .) The script creates certain setup files automatically.

Command Line Options -i, -h, -s

From the command line, you can enter various options to the kcmstest command. Three frequently used options are -i, -h, and -s.

To specify your own initialization file, you can enter its name on the command line preceded by the -i option, for example

%kcmstest -i
optional.ini

See "Creating An Alternate Initialization File" for details on alternate initialization files.

You can use the -s option to specify a script name (or all) and the -h option, to specify an alternate legal remote host name for scripts that test remote host access. The -h option attempts to pull a profile from the default directories on the remote host. Be sure that host has these directories and profiles.

The following example specifies the alternate initialization file alternate.ini, the script IC_attrl.scr, and the alternate host name dusk:

%kcmstest -i
alternate.ini -s IC_attrl.scr -h dusk

The example below defaults to using the icc.ini file:

%kcmstest -s
all

In this example, if any of the scripts in the icc.ini file access a remote host, the host name will be NULL and the scripts will fail.

Script Display

As each of the test script commands is executed, information about the command that is currently being interpreted is displayed to the command shell window as well as written to a log file in the kcmstest/log directory.

Recording Test Script Results To a Log File

For each script file executed, results are recorded in a log file. All the log files can be found in the kcmstest/log directory. The log file name is the name of the script file, with the .scr file extension replaced by the .log extension. If, for example, the test script name is IC_eval.scr, the log file name is IC_eval.log.

One exception to this naming scheme is if you enter all as the test script name. See "Starting the kcmstest Command" for details on this entry. In this case, the log file name is testall.log.

Two versions of a log file may exist at any given time: the current and the previous version. The previous version has its extension changed to .bak.

Example 2-3 is the log file output created from the test script shown in Example 2-2 .

Example 2-3 Log File Output

Parsing a KcsLoadProfile
Command         Profile Reference = scanner         Profile File Name =

kcmsEKls3510.inp         Profile Handling = By File         Profile Load Hint

= LoadWhenNeeded;         Profile Load Hint = UnLoadwhenNeeded;        

Profile Operation Hint = Image;         Load Hint =   2024000         Thu Jul

25 08:16:07 1996  Completed KcsLoadProfile command, status =    0         Thu

Jul 25 08:16:07 1996    Parsing a KcsLoadProfile Command         Profile

Reference = printer         Profile File Name = kcmsEKsunnws.out        

Profile Handling = By File         Profile Load Hint = LoadWhenNeeded;        

Profile Load Hint = UnLoadwhenNeeded;         Profile Operation Hint = Image; 

       Load Hint =   2024000         Thu Jul 25 08:16:07 1996  Completed

KcsLoadProfile command, status =    0         Thu Jul 25 08:16:07 1996  

Parsing a KcsConnectProfiles Command         Profile Reference = scan-print   

     Number of Profiles in Connect = 2         Profile Reference = scanner    

    Profile Reference = printer         Operation Hint =     20001         Thu

Jul 25 08:16:07 1996  Completed KcsConnectProfiles command, status =    0     

   Thu Jul 25 08:16:08 1996    Parsing a KcsEvaluate Command         Profile

Reference = scan-print         Source Layout = RGBInterLeaved;        

Destination Layout = RGBInterLeaved;         Input Image Name =

macbeth_1550.tif         Output Image Name = None         Operation Hint =    

20001         Thu Jul 25 08:16:08 1996  Completed KcsEvaluate command, status

=    0         364800.000000 pixels processed in 0.621338 seconds.         

The processing rate = 587120.062500 pixels/second.   Parsing a Free Profile

command          Profile reference =scanner  Completed KcsFreeProfile command,

status =    0  Parsing a Free Profile command          Profile reference

=printer  Completed KcsFreeProfile command, status =    0  Parsing a Free

Profile command          Profile reference =scan-print  Completed

KcsFreeProfile command, status =    0

Status Codes

If at any time during script execution, a KCMS framework API function call returns with an unexpected status code, the test is immediately aborted. For a list of all the status codes strings and their values, see Appendix A, Status Codes .

Note -

It may be your intention to have a status code returned that indicates an error because you deliberately set up a script to test an error condition. The script commands provide the optional keyword XStatus, which allows you to do this. For details, see the script command descriptions in Chapter 3, KCMS Test Suite Commands . Also see "Checking Status Codes" .

Using Automated Script Files To Run Test Scripts

The kcmstest directory includes two automated scripts: auto-kcmstest and auto-kcmstest-root.

Note -

See "Tips on Running the Automated Test Scripts" before using this testing method.

Using `auto-kcmstest`

The auto-kcmstest script allows you to run the complete test suite in icc.ini, including scripts in the icc.ini file list that access a remote host. This script is located in the kcmstest directory.

Note -

You may need to edit the script to change path information.

To run this script, do not set the environment variable KCMSROOT with the setenv command. Instead, provide two arguments: the KCMSROOT environment variable as the first argument and the remote host name as the second, for example

%auto-kcmstest
/opt/SUNWddk/kcms dusk

In this example, /opt/SUNWddk/kcms is the KCMSROOT environment variable and dusk is the remote host name. Note that if you are in the directory where auto-kcmstest() is located, only the host argument is needed, for example

%auto-kcmstest
dusk

Using `auto-kcmstest-root`

Certain test scripts require that you be root to run them. You would use these tests if, for example, you wanted to create an X Window System profile in a root-owned directory. To run these scripts, a second automated script called auto-kcmstest-root is provided.

To run the auto-kcmstest-root script,

Become superuser.
```
%su
```

Provide one argument: the KCMSROOT environment variable, for example

#./auto-kcmstest-root /opt/SUNWddk/kcms

Note that if you are in the directory containing auto-kcmstest-root, no argument is required.

Getting a Failure and Performance Report

After you have run the complete test suite using auto-kcmstest and auto-kcmstest-root, you can get an automated failure and performance report by running the kcms-testreport command. This command takes two arguments: the name of the test log and the report title. Very likely, you would redirect output to a file of the same name as the report title, for example

%kcms-testreport
log/testall.log my_test_1 > my_test_2

In this example, my_test_1 is the report title and my_test_2 is the output filename.

Tips on Running the Automated Test Scripts

The following is a suggested sequence for running a complete test suite using the automated script files:

Run the auto-kcmstest script, for example
```
%auto-kcmstest
/opt/SUNWddk/kcms dusk
```

Become root, for example
```
%su
```

Run the auto-kcmstest-root script, for example
```
#./auto-kcmstest-root
/opt/SUNWddk/kcms 
```

Run kcms-testreport and redirect output to a file, for example
```
#kcms_testreport
log/testall.log my_test_1 > my_test_2 
```
Note -
auto-kcmstest-root must be run after auto-kcmstest because it appends its resulting logs to the auto-kcmstest log file.

You may want to redirect the automated-test-script output to a file, as it is quite lengthy.

Chapter 2 Running KCMS Test Scripts

In This Chapter

Getting Started

Packaging

Environment Variables

Required File Hierarchy

Figure 2-1 kcmstest File Hierarchy

kcmstest Directory

Significant Directories

Images

Initialization File

Example 2-1 Initialization file icc.ini

Creating An Alternate Initialization File

KCMS Test Script Commands

Script Command Format

Example 2-2 Sample Test Script Showing Commands

Using kcmstest To Run Test Scripts

Starting the kcmstest Command

Command Line Options -i, -h, -s

Script Display

Recording Test Script Results To a Log File

Example 2-3 Log File Output

Status Codes

Using Automated Script Files To Run Test Scripts

Using auto-kcmstest

Using auto-kcmstest-root

Getting a Failure and Performance Report

Tips on Running the Automated Test Scripts

Figure 2-1 `kcmstest` File Hierarchy

`kcmstest` Directory

Example 2-1 Initialization file `icc.ini`

Using `kcmstest` To Run Test Scripts

Starting the `kcmstest` Command

Using `auto-kcmstest`

Using `auto-kcmstest-root`