test

ChorusOS 4.0 Introduction

Part III Programming Overview

Chapter 4 Programming Overview

This chapter introduces the steps involved in developing applications, also called actors, that run on the ChorusOS operating system. It includes the following sections:

System development and advanced programming topics are not covered.

Note -

The source code for many of the examples shown in this book is provided in the examples directory. By default this directory is /opt/SUNWconn/SEW/4.0/chorus-<target>/src/opt/examples.

ChorusOS Applications

The ChorusOS operating system provides an environment for applications running on a network of target machines, controlled by a remote host.

Programming Conventions

Services provided by the ChorusOS operating system are accessed as C routines. C header files provide the required constants, types and prototypes definitions. As the ChorusOS operating system is highly modular, header files reflect this modularity. However, in the following examples a global header file, named chorus.h, which collects most of the required header files, has been used for simplicity. Please refer to the man pages to get the actual minimum header file required for each service.

Most ChorusOS operating system constants start with K_. ChorusOS operating system error codes start with K_E. Constants and error codes are all written in uppercase.

Most specific data types are prefixed by Kn. When type names are composed of several lexems, the first letter of each lexem is written in uppercase while other letters are in lowercase, as in KnRgnDesc (region descriptor).

General Principles

In order to compile and link an application, the following information is needed:

Program Entry Point

In order to initialize the libraries correctly before starting the execution of the application code, the program entry point must be set to _start. After the initialization of libraries is completed, _start calls the _main routine which initializes variables in C++ programs. The main() routine is then called.

The _main routine manages any double calling at program initialization; some C++ compilers force a call to _main at the beginning of main().

Depending on the development system, it may be necessary to use specific linker directives to force the linker to extract the _start and _main routines from the libraries.

Libraries

In order to choose which ChorusOS operating system libraries to use, the following points need to be considered:

Supervisor Actor Binaries

As supervisor actors share the same supervisor address space, they are built as relocatable binaries, leaving the choice of the final link addresses to either the system configuration utility building the system image (for the basic environment) or the Actor Manager (for the extended environment).

Care must be taken when programming supervisor actors: no memory protection is provided between supervisor actors. A badly written supervisor actor accessing addresses outside its own address space can corrupt any supervisor region and cause unexpected behavior such as a system crash or reboot.

User Actor Binaries

User actors are also built as relocatable binaries, even though they use private address spaces. The link address of the user actors and the size of the user address space are board dependent. For a given board, all user actors are linked at the same address.

The final link is done by the Actor Manager when actors are loaded dynamically on the target.

Application Programming Interfaces

This section provides an overview of all programming interfaces available for applications developed for the ChorusOS operating system. The programming interface may differ from one program to another depending on:

Naming Conventions

Library names in the ChorusOS operating system use the following conventions with regard to their suffixes:

.u.a These libraries can only be used to build actors that will be loaded in a user address space.
.s.a These libraries can only be used to build actors that will be loaded in the supervisor address space.
.a These libraries can be used to build any type of actor.

Note -

When a library has both a user and supervisor version, it will be referred to using the .a suffix only.

All header file and library pathnames listed in the next subsections are related to the installation path of your ChorusOS delivery, typically /opt/SUNWconn/SEW/4.0/chorus-<target>.

Basic Environment APIs

The programming environment of basic actors consists of the following interfaces:

All routines implementing these APIs have been grouped into two libraries:

kernel/lib/embedded/libebd.u.a for user actors
kernel/lib/embedded/libebd.s.a  for supervisor actors

ChorusOS actors using the Basic Environment API are called embedded actors.

Extended Environment API

The programming environment of extended actors consists of the following interfaces:

All routines implementing these APIs have been grouped into one library:

os/lib/classix/libcx.a for user and supervisor actors

Note -

An extended supervisor actor should not use the svExcHandler() call as an extended actor inherits the Actor Manager exception handler.

Other APIs

Other APIs are provided with the ChorusOS operating system. Depending on their nature, they may be available to both basic and extended environments or restricted to a single environment. The following subsections give a description of the libraries implementing these APIs.

POSIX Micro Real-time Profile API

Routines implementing the MRTP (Micro Real-time Profile) API are included within the libcx and libebd libraries. They are available to both basic and extended actors.

Mathematical API

Routines implementing the Mathematical API are packaged in an independent library kernel/lib/libm/libm.a. This library is available to both basic and extended actors.

Sun RPC API

Routines implementing the Sun RPC API are packaged in an independent library os/lib/classix/librpc.a which is not thread-safe. This API is restricted to extended actors.

GNU 2.7.1 C++ API

The C++ library os/lib/CC/libC.a provides support for C++ applications with a complete and thread-safe library package. Every service offered by libC.a ensures that shared data is only accessed after signaling the relevant synchronization objects.

To allow atomic manipulation of any stream class (iostream or fstream for example), the API of libC.a has been extended with the following two services:

The ios::lock()service is used to lock any stream class object. The ios::unlock() service is used to unlock any stream class object. All services called upon a given stream object StrObj which are preceded by StrObj.lock() and followed by StrObj.unlock() are executed in an atomic way. It is guaranteed that no other thread can access StrObj as long as the lock is on.

An I/O stream object can be locked in two ways. For example, if cout is an I/O stream object: 

cout.lock();  
cout << "atomic " << "output";  
... (any other operation on cout)  
cout.unlock();
In this case the member function ios::lock() is called.

The following syntax could also be used:

cout << lock << "atomic " << "output" << unlock;

Embedded C++ actors can be linked with os/lib/CC/libC.a if they do not make use of the iostream and fstream packages.

Multithreading

The libebd.a, libcx.a, libm.a and libC.a libraries have been made thread-safe in order to support multithreaded actors. This is managed by the library in the following way:

Defining errno as one global variable for the actor is not suitable for multithreaded actors as situations can arise where a thread, examining errno on return from a failed system call, concludes that the call failed for the wrong reason because the global errno was changed by another system call in another thread in the meantime. Some programs also test errno rather than system call return values to detect errors.

To avoid this, the header file errno.h, exported by the extended environment, should be included in any source file using errno. This will result in a separate value for errno for each thread.

Header Files

The ChorusOS operating system header files are packaged in five different directories:

Typical ChorusOS operating system applications use header files of the include/chorus and include/posix directories (and also include/CC for applications using the GNU 2.7.1 C++ API).

Developing personality servers (such as servers implementing a UNIX personality) on a ChorusOS operating system needs extra care in order to avoid conflicts between data types declared by ChorusOS operating system header files, and data types declared by the server's header files. These servers should be restricted to header files of the include/chorus and include/stdc directories and use the _CHO_POSIX_TYPES_NOTDEF compile option.

Developing ChorusOS Applications

This section explains how to build a component to be included in a ChorusOS operating system. The component could be an application, a device driver, or a BSP. To build a ChorusOS component, you use the make and imake tools. All development tools are provided in the tools directory of your delivery.

make Environment

The make environment is defined by a file containing variable definitions and rules. Rules for compiling C, C++, and assembly language are provided. The rules are specific to the compiler you use, and the name of the file indicates the compiler. For example, if you are using the gcc compiler, the make environment file is called tgt-make/gcc-devsys.mk. The file contains the variables and rules required for building the component. The following variables are defined:

The make environment includes the following commands: cc, ld, as, and mkactors.

imake Environment

The ChorusOS imake environment extends the make environment by providing template rules for common ChorusOS build operations through generic names. When using those predefined imake rules, you do not need to know which libraries, crt files, or entry points you should use to build an application, as they are automatically selected for you.

Instead of creating Makefiles you must create Imakefiles, as imake will generate Makefiles from them.

The imake environment is defined by four files containing sets of variables and rules, located in the tools/imake directory. The rules are independent of the compiler you use.

imake Variable Definitions

The file Imake.tmpl contains the following definitions:

imake Build Rules

The file Imake.rules contains macros known as Imake build rules. Their name and function are described in Table 4-2.

Table 4-2 Imake build rules
 Macro name Function
MakeDir(dir)Creates the directory named dir.
LibraryTarget(lib, objs)Adds the objects indicated by objs into the library lib.
Depend(srcs)Computes the dependencies of srcs and adds them to the dependency list in the Makefile (using makedepend).
ActorTarget(prog, objs, options, crt0, libs)Uses objs to create a C actor called prog, and passes options, crt0 and libs to the linker.
UserActorTarget(prog, objs, libs) Creates a user C actor.
SupActorTarget(prog, objs, libs) Creates a supervisor C actor.
EmbeddedUserActorTarget(prog, objs, libs) Creates an embedded user C actor.
EmbeddedSupActorTarget(prog, objs, libs) Creates an embedded supervisor C actor.
BuiltinDriver(prog, objs, libs) Creates a ChorusOS operating system driver.
BspProgtarget(prog, entry, objs, libs) Creates a BSP program.
CXXActorTarget(prog, objs, options, crt0, libs)Uses objs to create a C++ actor called prog, and passes options, crt0 and libs to the linker.
CXXUserActorTarget(prog, objs, libs) Creates a user C++ actor.
CXXSupActorTarget(prog, objs, libs) Creates a supervisor C++ actor.
CXXEmbeddedUserActorTarget(prog, objs, libs) Creates an embedded user C++ actor.
CXXEmbeddedSupActorTarget(prog, objs, libs) Creates an embedded supervisor C++ actor.
DynamicUserTarget(prog, objs, libs, dynamicLibs, dlDeps, options) Creates a dynamic user C actor.
DynamicSupTarget(prog, objs, libs, dynamicLibs, dlDeps, options) Creates a dynamic user C actor.
DynamicCXXUserTarget(prog, objs, libs, dynamicLibs, dlDeps, options) Creates a dynamic user C actor.
DynamicCXXSupTarget(prog, objs, libs, dynamicLibs, dlDeps, options) Creates a dynamic user C actor.
DynamicLibraryTarget(dlib, objs, staticLibs, dynamicLibs, dlDeps, options) Creates a dynamic library.

Rules used to build actors use the following common arguments:

The rules used to build dynamic actors are described in more detail in "Building a Dynamic Program".

imake Packaging Rules

The file Package.rules contains macros known as Imake packaging rules for building a binary distribution. Their name and function are described in Table 4-3.

Table 4-3 Imake packaging rules
 Macro name Function
DistLibrary(lib, dir)Creates the directory dir and copies the library lib into it.
DistActor(actor, dir)Creates the directory dir and copies the actor actor into it.
DistFile(file, dir)Creates the directory dir and copies the file file into it.
DistRenFile(file, nFile, dir)Creates the directory dir, copies file into it, changing the name of file to nFile.
DistProgram(program, dir)Creates the directory dir and copies program into it.

Examples

Simple imake Example

The application in this example is composed of a single C source file, for example myprog.c in the directory myprog. Writing an Imakefile is quite straightforward. First, you must set the SRCS variable to the list of source files (in this case only one).

SRCS = myprog.c

Then, you must specify how to build the executable. The macro you use depends on the type of binary you want. If you want to build a user-mode binary (for example myprog_u), use the UserActorTarget() macro, as illustrated below. The first argument is the name of the executable. The second argument lists the object files. The third argument allows you to specify which libraries your program depends on. In this example there is no library, hence the empty argument (you could also pass NullParameter).

UserActorTarget(myprog_u,myprog.o,)

If you want to build a supervisor-mode binary (for example, myprog_s.r), use the SupActorTarget() as shown below. The arguments are the same as for UserActorTarget(). 

SupActorTarget(myprog_s.r,myprog.o,)

Finally, use the Depend() macro to generate the Makefile dependencies. 

Depend($(SRCS))

The Imakefile is complete. It looks like this: 

SRCS = myprog.c
UserActorTarget(myprog_u,myprog.o,)
SupActorTarget(myprog_s.r,myprog.o,)
Depend($(SRCS))

Next, generate the Makefile with the ChorusOSMkMf tool (see the ChorusOSMkMf(1CC) manpage for details). In the myprog directory, type: 


% ChorusOSMkMf build_dir

Where build_dir is the directory where you have built a ChorusOS system image on which your application will run.

Next, generate the make dependencies by typing the following command: 


% make depend

Finally, compile and link the program by typing: 


% make

The program is now ready to be executed, and can be run on your target by following the steps in "Running the "Hello World" Example".

imake with Multiple Source Files

If an application used source files located in several subdirectories, you need to create a root Imakefile in the root directory, containing only the following: 

#define IHaveSubdirs
SUBDIRS = subdir1 subdir2 ...

where subdir1, subdir2, ... are the subdirectories containing the source files (or other intermediate root Imakefile files). Next, create an Imakefile in each subdirectory containing source files. To generate the first Makefile, go to the root directory and type: 


% ChorusOSMkMf build_dir

Next, populate the tree with Makefile files, generate dependencies and finally compile the programs by typing the make Makefiles, make depend, and then make commands.


% make Makefiles   
% make depend   
% make

The program is now ready to be executed.

Note -

Examples of Imakefiles which you can modify and use to build your own applications are provided in /opt/SUNWconn/SEW/4.0/chorus-<target>/src/opt/examples.

Using Dynamic Libraries

There are two types of library in the ChorusOS operating system:

Relocatable code can be contained in two types of executable:

An actor that uses dynamic libraries is called a dynamic actor. A relocatable actor uses only static libraries.

Static and Dynamic Linking

The following table summarizes the actions performed by the static linker, which runs on the development host, and by the runtime linker, which runs on the target.

 Link Relocatable Executable Dynamic executable
 Static Linker.a Static linker adds necessary objects (.o) of a static library (.a) to the executable. .a Static linker adds necessary objects (.o) of a static library (.a) to the executable. .so Static linker adds the library to the list of libraries to load at actor start-up (afexec).
Runtime Linker (afexec) -.so At actor start-up, libraries are loaded and linked by the runtime linker. Libraries to load are defined either at static link, or in the LD_PRELOAD environment variable. The runtime linker uses a library search path to find dynamic and shared libraries.
Runtime Linker (dlopen) -.so Application explicitly asks the runtime linker to dynamically load and link a dynamic library, using the dlopen() function of the dynamic linking API.

Dynamic linking of libraries applies recursively to library dependencies: when a library is loaded, all the libraries it uses are also loaded.

Building a Dynamic Library

This section describes how to build dynamic and shared programs using the imake tool. See "Developing ChorusOS Applications" for more general information about using imake.

The following imake macro builds dynamic libraries: 

DynamicLibraryTarget(dlib, objs, staticLibs, dynamicLibs, dlDeps, options)

The following example builds a dynamic library named libfoo.so from the binary objects files a.o and b.o. When this library is loaded dynamically, the runtime linker will also load the dynamic library libdyn.so, which must be in its search path.

DynamicLibraryTarget(
        libfoo.so, 
        a.o  b.o, ,
        libdyn.so, , )

Building a Dynamic Program

The following imake macros build dynamic executables:

DynamicUserTarget(prog, objs, staticLibs, 
        dynamicLibs, dlDeps, options)
DynamicSupTarget(prog, objs, staticLibs, 
        dynamicLibs, dlDeps, options)
DynamicCXXUserTarget(prog, objs, staticLibs, 
        dynamicLibs, dlDeps, options)
DynamicCXXSupTarget(prog, objs, staticLibs, 
        dynamicLibs, dlDeps, options)
DynamicLibraryTarget(prog, objs, staticLibs, 
        dynamicLibs, dlDeps, options)

The prog argument is the name of the resulting program. Other arguments are the same as the DynamicLibraryTarget() macro. For the options argument, the following options are particularly useful:

The following example builds a dynamic program named prog from the binary object files a.o and b.o. The program is statically linked with the static ChorusOS operating system library. When this program is started, the runtime linker will load the dynamic library libdyn.so. In the target file system, this library can be located in the /libraries directory, as this directory is added to the search path of the runtime linker. 

DynamicUserTarget(
        prog, 
        a.o  b.o, ,
        libdyn.so, ,
        -Xlinker -rpath -Xlinker /libraries)

Dynamic Programming

The previous sections described how to use dynamic objects (dynamic and shared libraries) that are loaded by the runtime linker at actor start-up. In addition to this mechanism, an actor can also bind a dynamic or shared library explicitly during its execution. This on-demand object binding has several advantages:

Typically, an application performs the following sequence to access an additional dynamic object, using the dynamic library API:

Runtime Linker

This section describes the functions performed by the runtime linker, as well as the features it supports for dynamic applications.

Dynamic applications consist of one or more dynamic objects. They are typically a dynamic executable and its dynamic object dependencies. As part of the initialization of a dynamic application, the runtime linker completes the binding of the application to its dynamic object dependencies.

In addition to initializing an application, the runtime linker provides services that allow the application to extend its address space by mapping additional dynamic objects and binding to symbols within them.

The runtime linker performs the following functions:

The runtime linker uses a prescribed search path for locating the dynamic dependencies of an object. The default search paths are the runpath recorded in the object, followed by /usr/lib. The runpath is specified when the dynamic object is constructed using the -rpath option of the linker. The environment variable LD_LIBRARY_PATH can be used to indicate directories to be searched before the default directories.

Note -

The runtime linker needs a file system to load dynamic objects. This file system can be on a host and accessed through NFS from the target. In embedded systems without a network connection, a bank of the system image can be used. For example: /image/sys_bank.

Environment Variables

The following environment variables are used by the runtime linker:

Supported Features

Examples

This section contains two examples of dynamic programs. In all these examples, it is assumed that a standard development environment has been set up: system build tree, search path, boot and initialization of target machine. It also assumes that the chorus_root_directory is the path of the target root directory on the NFS host (for example /home/chorus/root), the name of the target is jericho, and the environment variable WORK refers to the directory used for building these examples

Dynamic Link at Actor Start-up

The following dynamic program uses a custom dynamic library which will be loaded and linked at actor start-up. It uses a function foo() which is defined in the dynamic library. This function calls a function bar() defined in the main program.

This is the dynamic program progdyn.c:

#include <chorus.h> 

extern void foo();

main() {
    foo();              /* calling foo defined in the library */
}

void bar() {
    printf ("bar called\n");
}

This is the dynamic library libdyn.c:

#include <chorus.h> 

extern void bar();

void foo() {
    printf ("Calling bar\n");
    bar();              /* calling bar defined in the main program */
}

Building the Dynamic Library

Create a directory libdyndir in $WORK, containing libdyn.c and the following Imakefile: 

SRCS = libdyn.c
DynamicLibraryTarget (libdyn.so, libdyn.o, , , ,-Xlinker -soname=libdyn.so )
Depend(libdyn.c)

In the libdyndir directory, build the dynamic library libdyn.so using the ChorusOSMkMf, make depend, and make commands.

Building the Dynamic Program

Create a directory progdyndir in $WORK, containing progdyn.c and the following Imakefile: 

SRCS = progdyn.c
DynamicUserTarget (progdyn, progdyn.o, , 
        $(WORK)/libdyndir/libdyn.so,
        $(WORK)/libdyndir/libdyn.so, )
Depend()

In the progdyndir directory, build the dynamic program progdyn using the ChorusOSMkMf, make depend and make commands.


% ChorusOSMkMf $WORK
% make depend
% make

Running the Dynamic Program

Copy the dynamic program into the /bin subdirectory of the chorus_root_directory directory: 


% cp $WORK/progdyndir/progdyn chorus_root_directory/bin

Copy the dynamic library into the /lib subdirectory of the chorus_root_directory directory: 


% cp $WORK/libdyndir/libdyn.so chorus_root_directory/lib

Then, the following command will tell the runtime linker where to find the libdyn.so dynamic library: 


% rsh jericho setenv LD_LIBRARY_PATH /lib

Alternatively, set the runpath to /lib in the ldopts argument of the program macro (-rpath /lib).

Finally, the following command will start the program and dynamically load the libdyn.so library: 


% rsh jericho arun /bin/progdyn

Explicit Link Using dlopen

The following program explicitly loads a dynamic library at runtime, using the function dlopen(). It searches for the address of the dynfunc() function defined in the library and calls this function.

This is the dynamic program progdyn2.c: 

#include <chorus.h>
#include <cx/dlfcn.h>

int main()
{
    void    (*funcptr)();       /* pointer to function to search */
    void    *handle;            /* handle to the dynamic library */

        /* finding the library */
    handle = dlopen ("libdyn2.so", RTLD_NOW);
    if !(handle) { printf ("Cannot find library libdyn2.so\n"); exit(1); } 

        /* finding the function in the library */
    funcptr = (void (*)()) dlsym (handle, "dynfunc");
    if !(funcptr) { printf ("Cannot find function dynfunc\n"); exit(1); }

        /* calling library function */
    (*funcptr)();
}

This is the dynamic library libdyn2.c:

#include <chorus.h> 

void dynfunc() {
    printf ("Calling dynfunc\n");
}

Building the Program and the Library

The program and library above are built in the same way as in the previous example, using two Imakefiles:

Create a directory libdyn2dir in $WORK, containing libdyn2.c and the following Imakefile: 

SRCS = libdyn2.c
DynamicLibraryTarget (libdyn2.so, libdyn2.o, , , , )
DependTarget(libdyn2.c)

Create a directory progdyn2dir in $WORK, containing progdyn2.c and the following Imakefile: 

SRCS = progdyn2.c
DynamicUserTarget (progdyn2, progdyn2.o, , , , )
Depend(progdyn2.c)

Running the Dynamic Program

Copy the dynamic program into the /bin subdirectory of the chorus_root_directory directory: 


% cp $WORK/progdyn2dir/progdyn2 chorus_root_directory/bin

Copy the dynamic library into the /lib subdirectory of the chorus_root_directory directory: 


% cp $WORK/libdyn2dir/libdyn2.so chorus_root_directory/lib

Then, the following command will tell the runtime linker where to find the libdyn2.so dynamic library: 


% rsh jericho setenv LD_LIBRARY_PATH /lib

Finally, the following command will start the program: 


% rsh jericho arun /bin/progdyn2

At program start-up, the runtime linker will only load the executable progdyn2. The libdyn2.so library will be loaded when the dlopen() function is called.

Chapter 5 Using Actors

This chapter explains the role of actors in a ChorusOS operating system application. It contains the following sections:

Actor Definition

An actor is the unit of loading for an application. It serves also as the encapsulation unit to associate all system resources used by the application and the threads running within the actor. Threads, memory regions and communication end-points are some examples of these resources. They will be covered in more detail throughout this chapter. All system resources used by an actor are freed upon actor termination.

Some resources, known as anonymous resources, are not bound to a given actor. They must be freed explicitly when they are no longer required. Examples of anonymous resources are physical memory, reserved ranges of virtual memory, and interrupt vectors.

The ChorusOS operating system is dedicated to the development and execution of applications in a host-target environment where applications are developed, compiled, linked and stored on a host system and then executed on a target machine where the ChorusOS operating system is running. When properly configured, the ChorusOS operating system offers convenient support for writing and running distributed applications.

Within the ChorusOS operating system environment, an application is a program or a set of programs usually written in C or C++. In order to run, an application must be loaded on the ChorusOS runtime system. The normal unit of loading is called an actor and is loaded from a binary file located on the host machine. As with any program written in C or C++, an actor has a standard entry point:

int main()
 {
     /* A rather familiar starting point, isn't it? */
 }

The code of this type of application will be executed by a main thread which is automatically created at load time by the system. The ChorusOS operating system provides means to dynamically create and run more than one thread in an actor. It also offers services which enable these actors, whether single-threaded or multithreaded, to cooperate, synchronize, exchange data either locally or remotely, or get control of hardware events, for example. These topics will be covered step by step throughout this chapter.

An actor may be of two types: it may be either a supervisor actor or a user actor. This information defines the nature of the actor address space. User actors have separate and protected address spaces so that they cannot overwrite each other's address spaces. Supervisor actors use a common but partitioned address space. Depending on the underlying hardware, a supervisor actor can execute privileged hardware instructions (such as initiating an I/O), while a user actor cannot.

Note -

In flat memory, supervisor and user actors share the same address space and there is no address protection mechanism.

Binary files from which actors are loaded may also be of two kinds: either absolute or relocatable. An absolute binary is a binary where all addresses have been resolved and computed from a well-known and fixed basis which may not be changed. A relocatable file is a binary which may be loaded or relocated at any address.

Both user and supervisor actors can be loaded either from absolute or relocatable binary files. However, common practice is to load them from relocatable files to avoid a static partitioning of the common supervisor address space, and to allow the loading of user actors into this space in the flat memory model. This is covered in more detail in "User and Supervisor Actors".

Naming Actors

Every actor, whether it is a boot actor or a dynamically loaded actor, is uniquely identified by an actor capability. When several ChorusOS operating systems are cooperating together over a network in a distributed system, these capabilities are always unique through space and time. An actor may identify itself with a predefined capability:

K_MYACTOR.

In addition, an actor created from the POSIX personality is identified by a local actor identifier. This actor identifier is displayed on the console as the result of the arun command. It may be used from the console as a parameter of the akill command. 


% rsh target arun hello
Started aid = 13

%

In this example, target is the name of your target.

User and Supervisor Actors

There are two main kinds of actors which may be run within the ChorusOS operating system environment: user actors and supervisor actors. A user actor runs in its own private address space so that if it attempts to reference a memory address which is not valid in its address space, it will encounter a fault and, by default, will be automatically deleted by the ChorusOS operating system.

Supervisor actors do not have their own fully contained private address space. Instead, they share a common supervisor address space, which means that an ill-behaved supervisor actor can access, and potentially corrupt, memory belonging to another supervisor actor. The common supervisor address space is partitioned between the ChorusOS operating system components and all supervisor actors.

As supervisor actors reside in the same address space, there is no memory context switch to perform when execution switches from one supervisor actor to another. Thus, supervisor actors provide a trade-off between protection and performance. Moreover, they allow execution of privileged hardware instructions and so enable device drivers, for example, to be loaded and run as supervisor actors.

On most platforms, the address space is split into two ranges: one reserved for user actors and one for supervisor actors (see Figure 5-1). As user actor address spaces are independent and overlap each other, the address where these actors run is usually the same, even if the actors are loaded from relocatable binaries. On the other hand, available address ranges in supervisor address space may vary depending on how many and which supervisor actors are currently running. Since the ChorusOS operating system is able to find a slot dynamically within the supervisor address space to load the actor, the user does not need to be aware of the partitioning of the supervisor address space: using relocatable binary files will suffice.

Figure 5-1 User and Supervisor Address Spaces

Graphic

The ChorusOS operating system offers a way to determine dynamically whether a program is currently running as a user or a supervisor actor:

#include <chorus.h> 

int actorPrivilege(KnCap* actorCap,   
        KnActorPrivilege* old,
        KnActorPrivilege* new);

If actorCap is set to the name of an actor, you can use this API to obtain the privilege of the named actor. If actorCap is set to the predefined value K_MYACTOR, you can obtain the privilege of the current actor. This call may also be used to dynamically change the privilege of an actor from user to supervisor or vice versa.

The following example illustrates a usage of the actorPrivilege() service. It is a small program that retrieves its privilege, without trying to modify it. It prints one message if the actor is running as a user actor, and another if it is running as a supervisor actor.

Example 5-1 Getting Actor Privilege

#include <stdio.h>
#include <chorus.h>

int main(int argc, char** argv, char** envp)
{
   KnActorPrivilege    actorP;
   int                 res;

       /* Get actor's privilege */
   res = actorPrivilege(K_MYACTOR, &actorP, NULL);
   if (res != K_OK) {
      printf("Cannot get the privilege of the actor, error %d\n", res);
      exit(1);
   }

   if (actorP == K_SUPACTOR) {
      printf("This actor is running as a supervisor actor\n");
   } else {
      printf("This actor is running as a user actor\n");
   }

   exit(0);
}

KnActorPrivilege is the type defined by the ChorusOS operating system to handle the type of an actor. The defined values for the actor type are:

Loading Actors

Actors may be loaded in two different ways: either at system boot time or dynamically.

The ChorusOS operating system is started from a bootable file, called the system image, which is loaded in memory either by a hardware boot or a primary boot, depending on the hardware. This bootable file contains the image of the system to be run on the target machine.

Boot Actors

The ChorusOS operating system environment provides tools to configure this system image with user provided actors, which may be user or supervisor actors. Once the system has performed its own initialization, it starts these actors automatically, creating a main thread in each of them. These actors are often referred to as boot actors.

Loading Actors Dynamically

In order to be able to dynamically load an application on a ChorusOS operating system, the system must have been configured with the ACTOR_EXTENDED_MNGT feature. In this type of configuration, the ChorusOS operating system is able to dynamically load binary files from the host system acting as an NFS server, from a local disk, or from the system image (/image/sys_bank). This host-target environment enables the user to load supervisor and user actors using a simple remote shell mechanism. To execute an application called hello on the target host moon, use the arun command, as follows: 


% rsh moon arun hello

The ChorusOS operating system uses the .r suffix to denote relocatable binary files.

A relocatable actor is executed as follows: 


% rsh moon arun mySupAppl.r

In this example, the .r suffix could be omitted, because the ChorusOS operating system looks first for the name as specified, mySupAppl, and then, if it does not find a file of that name, automatically looks for a file of that name with the suffix .r, mySupAppl.r.

Execution Environment of Actors

The execution environment of actors varies slightly depending on whether the actors have been loaded dynamically or at boot time.

An actor loaded at boot time does not have any arguments or environment. If it is linked with the embedded library, it may perform very simple input or output operations such as printing traces on the system console using the printf() C library routine. It may also read characters typed in from the keyboard of the system console through the scanf() C library routine. If it is linked with the libcx.a library, it must first open /dev/console three times in order to activate stdin, stdout, or stderr and allow printf() or scanf() operations on the system console. The main thread of an actor loaded at boot time will belong to the SCHED_FIFO scheduling policy, with an arbitrary priority depending on the rank of the actor within the system image. The size of the stack provided to the main thread of this type of actor is defined by a system-wide tunable parameter.

A dynamically loaded actor is started as a regular C program with arguments and environments: 

int main(int argc, char** argv, char** envp)
{
        /* Main routine of a dynamically loaded actor */
        /* regardless of whether the actor is a user  */
        /* or supervisor actor.                       */
}

The standard input, output and error files of an extended actor may be redirected so that the I/O operations performed by this actor occur either on the system console, on a regular file (accessed through NFS) or on a terminal window of the host system. The main thread of a dynamically loaded actor has its scheduling policy, priority and stack size set according to system-wide tunable parameters.

Actor Context

The precise context of an actor depends on how the system is configured. An extended actor has a file context similar to the file context of a UNIX process: it has a root directory as well as a current directory. It may also create, open, close, read and write files or sockets.

An extended actor runs on behalf of a user who is identified by means of a credentials structure. The actor credentials include: the identifier of the user, the identifier of the group of the user as well as a possibly empty list of identifiers of supplementary groups. Readers familiar with the concept of credentials in UNIX should note that the ChorusOS operating system concept of credentials is simpler than the UNIX one. ChorusOS 4.0 does not differentiate between real or effective user/group identification as it is not supported.

These actor credentials are used for file access. They are also used when the ChorusOS operating system runs in secured mode to check the validity of an operation. For example, in secured mode only the superuser, whose user identifier is 0, may load supervisor actors.

Standard Input/Output (I/O)

An extended actor may take advantage of the entire C library for dealing with I/O. In addition to the I/O interface provided by the C library, an extended actor may also use POSIX I/O services such as open, read, or write, as well as POSIX socket services such as socket, bind, and connect.

The following program may be run as an actor, and illustrates the way in which the C library might be used from an actor.

Example 5-2 Using the C Library from an Actor

#include <stdio.h
#include <stdlib.h>
#include <chorus/stat.h>

#define  BUF_SIZE 80

struct stat st;

int main(int argc, char** argv, char** envp)
{
   FILE* file;
   FILE* filew;
   char* buf;
   int res;

   if (argc != 2 && argc != 3) {
      fprintf(stderr, "Usage: %s filename\n", argv[0]);
      exit(1);
   }

   file = fopen(argv[1], "r");

   if (file == NULL) {
      fprintf(stderr, "Cannot open file %s\n", argv[1]);
      exit(1);
   }

   res = stat(argv[1], &st);
   if (res < 0) {
      fprintf(stderr, "Cannot stat file\n");
      exit(1);
   }

   printf("File size is %d mode 0x%x\n", st.st_size, st.st_mode);

   buf = (char*) malloc(BUF_SIZE);
   if (buf == NULL) {
      fprintf(stderr, "Cannot allocate buffer\n");
      exit(1);
   }

   bzero(buf, BUF_SIZE);
   res = read(fileno(file), buf, 80);
   if (res == -1) {
      fprintf(stderr, "Cannot read file\n");
      exit(1);
   }

   printf("%s\n", buf);

   if (argv[2] != NULL) {
      filew = fopen(argv[2], "w");

      if (filew == NULL) {
         fprintf(stderr, "Cannot open file %s\n", argv[2]);
         exit(1);
      }

      printf("Type any input you like: \n");

      do {
         scanf("%80s", buf);
         printf("buf=%s\n", buf);
         fprintf(filew, "%s", buf);
         printf("buf=%s\n", buf);
      } while (buf[0] != 'Q');
   }
   exit(0);
}

Note -

This example assumes that argv[0] is valid, and the actor is linked with the ~lib/classix/libcx.a library, since the library is common to both user and supervisor actors. Referencing argv[0] without checking if argc is greater than zero can cause the actor to make an exception and be deleted.

Allocating Memory

In any C program, memory can be dynamically allocated by means of the malloc() C library routine within actors, whether loaded at boot time or dynamically, and whether running as user or supervisor actors.

Example 5-2 shows a usage of the malloc() routine.

Terminating an Actor

As shown in the previous example, an actor may terminate by invoking the exit() routine, as with any typical C program. Invoking exit() ensures that all resources used by the actor are freed: I/O buffers will be flushed, all open files are closed, and all other system resources provided by features configured within the system are released automatically.

Spawning an Actor

So far, two ways of loading and running actors have been described: either the inclusion of the actor as part of the system image, or the usage of the arun mechanism. The ChorusOS operating system also enables an actor to dynamically spawn another actor from a binary file. This spawned actor may be either a supervisor or a user actor. This service is similar to the exec() UNIX system call:

#include <am/afexec.h> 

int afexecve (const char* path, 
              KnCap* actorCap, 
              const AcParam* param, 
              char* const* argv, 
              char* const* envp);

This service creates a new actor whose capability is returned by the system at the location pointed to by the actorCap argument. The actor created will execute the binary file stored in the file named path. The main thread of this actor will run the main routine of the program. This thread will have the same scheduling attributes and stack size as an actor loaded using the arun mechanism.

argv and envp are pointers to the array of arguments and environments that will be received by the newly created actor.

The afexec() service comes in several variants, similar to the UNIX exec() call variants. When successful, all afexec() routines return the actor identifier of the newly created actor. Otherwise, they return -1, and the error code is returned in the errno variable.

Most of the time, application writers will not need to use the afexec() service. They will use either the arun facility or include the actor as part of the system image. However, for convenience, some examples within this document do use this service.

Below is an example of use of the afexecve() service call.

Example 5-3 Spawning an Actor

#include <stdio.h>
#include <errno.h>
#include <am/afexec.h>

AcParam param;
char*   spawnedArgs[3];
char*   tagPtr = "Welcome newly created actor!";

main(int argc, char** argv, char**envp)
{
  KnCap        spawnedCap;
  int          res;

  if (argc == 1) {
        /*
         * This is the first actor (or spawning actor):
         *   Binary file used to load this actor is passed
         *   by "arun" as argv[0],
         *
         *   Set an argument in order to enable the second
         *   actor to know it is the second one.
         */
    param.acFlags = AFX_USER_SPACE;

    spawnedArgs[0] = argv[0];
    spawnedArgs[1] = tagPtr;
    spawnedArgs[2] = NULL;
        /*
         * Other fields are implicitly set to NULL, as 
         * param is allocated within the bss segment of 
         * the program.
         */
    res = afexecve(argv[0], &spawnedCap, &param , spawnedArgs, envp);

    if (res == -1) {
      printf("Cannot spawn second actor, error %d\n", errno);
      exit(1);
    }
    printf("I succeeded creating actor whose aid is %d\n", res);
  } else {
        /*
         * This is the spawned actor:
         *   Check the number of args,
         *   Print args,
         *   Exit
         */
    if ((argc == 2) && (strcmp(tagPtr, argv[1]) == 0)) {
            /*
             * This is really the spawned actor.
             */
        printf("My spawning actor passed me this argument: %s\n",
               argv[1]);
    } else {
        printf("You ran %s with an argument, you should not!\n", argv[0]);
        exit(1);
    }
  }
  exit(0);
}

Note -

This example assumes that argv[0] is valid, and the actor is linked with the ~lib/classix/libcx.a library, since the library is common to both user and supervisor actors. Referencing argv[0] without checking if argc is greater than zero can cause the actor to make an exception and be deleted.

Chapter 6 Multithreaded Programming with the ChorusOS Operating System

This chapter describes how to use ChorusOS operating system services to create a multithreaded actor. It contains the following sections:

Basic Multi-Thread Programming

Within an actor, whether user or supervisor, one or more threads may execute concurrently. A thread is the unit of execution in a ChorusOS operating system and represents a single flow of sequential execution of a program. A thread is characterized by a context corresponding to the state of the processor (registers, program counter, stack pointer or privilege level, for example). See Figure 6-1.

Threads may be created and deleted dynamically. A thread may be created in another actor than the one to which the creator thread belongs, provided they are both running on the same machine. The actor in which the thread was created is named the home actor or the owning actor. The home actor of a thread is constant during the life of the thread.

The system assigns decreasing priorities to boot actor threads, so that boot actor main threads are started in the order in which they were loaded into the system image. If a boot actor's main thread sleeps or is blocked, the next boot actor threads will be scheduled for running.

Although there are no relationships maintained by the ChorusOS operating system between the creator thread and the created thread, the creator thread is commonly called the parent thread, and the created thread is commonly called the child thread.

A thread is named by a local identifier referred to as a thread identifier. The scope of this type of identifier is the home actor. In order to name a thread of another actor, you must provide the actor capability and the thread identifier. It is possible for a thread to refer to itself by using the predefined constant: K_MYSELF.

All threads belonging to the same home actor share all the resources of that actor. In particular, they may access its memory regions, such as the code and data regions, freely. In order to facilitate this access, the ChorusOS operating system provides synchronization tools which are covered in a later section of this document.

Threads are scheduled by the kernel as independent entities; the scheduling policy used depends on the scheduling module configured within the system. In a first approach, assume that a thread may be either active or waiting. A waiting thread is blocked until the arrival of an event. An active thread may be running or ready to run.

Figure 6-1 A Multi-Threaded Actor

Graphic

Thread Handling

Getting a Thread Identifier

A thread may obtain its local identifier by means of the following ChorusOS operating system service:

#include <chorus.h>

int threadSelf();

An example of how this call can be used is provided in Example 6-1.

Creating a Thread

A thread may be created dynamically by means of the following ChorusOS operating system service:

#include <chorus.h>

int threadCreate(KnCap*         actorCap, 
                 KnThreadLid*   thLi, 
                 KnThreadStatus status,
                 void*          schedParam, 
                 void*          startInfo);

The actorCap parameter identifies the actor in which the new thread will be created. You can create the new thread in the current actor by passing K_MYACTOR as the actor capability. This is the usual case. Should this be successful, the local identifier of the newly created thread is returned at the location defined by the thLi parameter.

The schedParam parameter is used to define the scheduling properties of the thread to be created. If this parameter is set to 0, the created thread inherits the scheduling attributes of the creator thread.

The startInfo parameter is used to define the initial state of the thread, such as the initial program counter of the thread (the thread entry point), as well as the initial value of the stack pointer to be used by the created thread. You can also define whether the thread will run as a user thread or as a supervisor thread.

A thread needs a stack to run, in order to have room to store its local variables. When the thread is a user thread, the user must explicitly provide a stack to the thread. However, stacks for supervisor threads are implicitly allocated by the system. In fact, a system stack is allocated for all threads, even those running in user mode.

Note -

As the operating system does not prevent the user stack from overflowing, checks must be made every time a thread is created.

System stacks are not allowed to overflow as memory will become corrupted, resulting in unpredictable operating system behavior.

Example 6-1 is a simple program illustrating the creation of a thread by the main thread of an actor. The actor is loaded by the arun command. Its main thread is implicitly created by the system. The goal of the example is to:

This example will work without modification whether it is run as a user or as a supervisor actor. In the first case, a user thread must be created, while in the second case a supervisor thread must be created. Using the actorPrivilege() service call might be helpful for this purpose.

This example requires some kind of synchronization between the main thread and the created one. Execution of a thread can be suspended for a given delay:

#include <chorus.h>

int threadDelay(KnTimeVal* waitLimit);

This call suspends the execution of the invoking thread for a period specified by the KnTimeVal structure (see "Current Time " for more detail). There are two predefined values:

These values may be used instead of the pointer to the KnTimeVal data structure. There is also a predefined macro which sets such a structure from a delay expressed in milliseconds: K_MILLI_TO_TIMEVAL(KnTimeVal* waitLimit, int delay). For more information, see the threadCreate(2K), threadDelay(2K), and threadSelf(2K) man pages.

Example 6-1 Creating a Thread

(file: progov/thCreate.c)

#include <stdio.h>
#include <stdlib.h>
#include <chorus.h>

#define USER_STACK_SIZE (1024 * sizeof(long))

    int
childCreate(KnPc entry)
{
  KnActorPrivilege      actorP;
  KnDefaultStartInfo_f  startInfo;
  char*                 userStack;
  int                   childLid = -1;
  int                   res;

      /* Set defaults startInfo fields */
  startInfo.dsType            = K_DEFAULT_START_INFO;
  startInfo.dsSystemStackSize = K_DEFAULT_STACK_SIZE;

      /* Get actor's privilege */
  res = actorPrivilege(K_MYACTOR, &actorP, NULL);
  if (res != K_OK) {						      
    printf("Cannot get the privilege of the actor, error %d\n", res); 
    exit(1);							      
  }								      

      /* Set thread privilege */
  if (actorP == K_SUPACTOR) {
    startInfo.dsPrivilege = K_SUPTHREAD;
  } else {
    startInfo.dsPrivilege = K_USERTHREAD;
  }

      /* Allocate a stack for user threads */
  if (actorP != K_SUPACTOR) {
    userStack = malloc(USER_STACK_SIZE);
    if (userStack == NULL) {					      
      printf("Cannot allocate user stack\n");			      
      exit(1);							      
    }								      

    startInfo.dsUserStackPointer = userStack + USER_STACK_SIZE;
  } 

      /* Set entry point for the new thread */
  startInfo.dsEntry = entry;

      /* Create the thread in the active state */
  res = threadCreate(K_MYACTOR, &childLid, K_ACTIVE, 0, &startInfo);
  if (res != K_OK) {						      
    printf("Cannot create the thread, error %d\n", res);	      
    exit(1);							      
  }								      
								      
  return childLid;
}

    void
sampleThread()
{
  int myThreadLi;

  myThreadLi = threadSelf();

  printf("I am the new thread. My thread identifier is: %d\n", myThreadLi);

      /* Block itself for ever */
  threadDelay(K_NOTIMEOUT);
}


int main(int argc, char** argv, char**envp)
{

  int        myThreadLi;
  int        newThreadLi;
  int        res;						      
  KnTimeVal  wait;

  newThreadLi = childCreate((KnPc)sampleThread);

  myThreadLi = threadSelf();

      /* Initialize KnTimeVal structure */
  K_MILLI_TO_TIMEVAL(&wait, 10);

      /*
       * Suspend myself for 10 milliseconds to give the newly
       * created thread the opportunity to run before
       * the actor terminates.
       */
  res = threadDelay(&wait);

  printf("Parent thread identifier = %d, Child thread identifier = %d\n",
	 myThreadLi, newThreadLi);

  return 0;
}

Deleting a Thread

A thread may be dynamically deleted by itself or by another one using the following service:

#include <chorus.h>

int threadDelete(KnCap* actorCap,
                 KnThreadLid thLi);

This call enables a thread to delete another one inside the same actor, when actorCap is set to K_MYACTOR, by knowing the thread identifier of the thread to be deleted. It also enables a thread to delete another one inside another actor (provided they are both running on the same machine), as long as it provides both the actor capability and the target thread identifier. The predefined thread identifier K_MYSELF enables a thread to name itself without knowing its actual thread identifier.

Example 6-2 is a slightly different version of the previous program. The subroutine childCreate() is unchanged, but now the created thread kills itself, instead of going idle forever.

Note -

This does not solve the synchronization problem occurring in the previous example: the main thread still does not know exactly when to terminate the actor.

Refer to the threadDelete(2K) man page.

Example 6-2 Deleting a Thread

(file: progov/thDelete.c)

#include <stdio.h>
#include <stdlib.h>

#include <chorus.h>

#define USER_STACK_SIZE (1024 * sizeof(long))

    int
childCreate(KnPc entry)
{
  KnActorPrivilege      actorP;
  KnDefaultStartInfo_f  startInfo;
  char*                 userStack;
  int                   childLid = -1;
  int                   res;

  startInfo.dsType            = K_DEFAULT_START_INFO;
  startInfo.dsSystemStackSize = K_DEFAULT_STACK_SIZE;

  res = actorPrivilege(K_MYACTOR, &actorP, NULL);
  if (res != K_OK) {
    printf("Cannot get the privilege of the actor, error %d\n", res);
    exit(1);
  }

  if (actorP == K_SUPACTOR) {
    startInfo.dsPrivilege = K_SUPTHREAD;
  } else {
    startInfo.dsPrivilege = K_USERTHREAD;
  }

  if (actorP != K_SUPACTOR) {
    userStack = malloc(USER_STACK_SIZE);
    if (userStack == NULL) {
      printf("Cannot allocate user stack\n");
      exit(1);
    }
    startInfo.dsUserStackPointer = userStack + USER_STACK_SIZE;
  } 

  startInfo.dsEntry = entry;

  res = threadCreate(K_MYACTOR, &childLid, K_ACTIVE, 0, &startInfo);
  if (res != K_OK) {
    printf("Cannot create the thread, error %d\n", res);
    exit(1);
  }

  return childLid;
}

    void
sampleThread()
{
  int myThreadLi;

  myThreadLi = threadSelf();

  printf("I am the new thread. My thread identifier is: %d\n", myThreadLi);

      /* Suicide */
  threadDelete(K_MYACTOR, K_MYSELF);

      /* Should never reach this point! */
}

int main(int argc, char** argv, char**envp)
{
  int        myThreadLi;
  int        newThreadLi;
  int        res;						      
  KnTimeVal  wait;

  newThreadLi = childCreate((KnPc)sampleThread);

  myThreadLi = threadSelf();

      /* Initialize KnTimeVal structure */
  K_MILLI_TO_TIMEVAL(&wait, 10);

      /*
       * Suspend myself for 10 milliseconds to give the newly
       * created thread the opportunity to run before
       * the actor terminates.
       */
  res = threadDelay(&wait);

  printf("Parent thread identifier = %d, Child thread identifier = %d\n",
	 myThreadLi, newThreadLi);

  return 0;
}

Synchronizing Threads

The previous section explained the need for threads to be synchronized accurately, avoiding using delays which are difficult to tune and which depend on the load of the system. The ChorusOS operating system offers various tools for synchronizing threads:

Semaphores

A semaphore is an integer counter associated with a queue, possibly empty, of waiting threads. At initialization, the semaphore counter receives a user-defined positive or null value. Initialization is performed by invoking the following ChorusOS operating system service:

#include <chorus.h>

int semInit(KnSem*        semaphore,
            unsigned int  count);

The semaphore parameter is the location of the semaphore and count is the semaphore counter. The semaphore must have been previously allocated by the user: allocation is not performed by the ChorusOS operating system. This implies that semaphores may be freely allocated by the user where convenient for his applications. As data structures representing semaphores are allocated by the applications, the ChorusOS operating system does not impose any limit on the maximum number of semaphores which may be used within the system.

Two atomic operations, named P and V, are provided on these semaphores. 

#include <chorus.h>

int semP(KnSem*     semaphore,
         KnTimeVal* waitLimit);

semP() decrements the counter by one. If the counter reaches a negative value, the invoking thread is blocked and queued within the semaphore queue. Otherwise the thread continues its execution normally. The waitLimit parameter may be used to control how long the thread will stay queued. If waitLimit is set to K_NOTIMEOUT, the thread will stay blocked until the necessary V operation is performed. In the case of the thread being awakened due to the expiration of the period, a specific error code is returned as the result of the semP() invocation. In this case, the counter is incremented to compensate for the effect of the semP() operation.

#include <chorus.h>

int semV(KnSem* semaphore);

semV() increments the counter by one. If the counter is still lower than or equal to zero, one of the waiting threads is picked up from the queue and awakened. If the counter is strictly greater than zero, there should be no thread waiting in the queue.

Figure 6-2 shows an example of two threads synchronizing by means of a semaphore.

Figure 6-2 Two Threads Synchronizing with a Semaphore

Graphic

The following example is based on the previous one, but the two threads explicitly synchronize by means of a semaphore, so that the actor will eventually be destroyed when the created thread has done its job and as soon as it has done so. Refer to the semInit(2K) man page.

Example 6-3 Synchronizing Using Semaphores

(file: progov/semaphore.c)

#include <stdio.h>
#include <stdlib.h>
#include <chorus.h>

#define USER_STACK_SIZE (1024 * sizeof(long))			      
								      
KnSem   sampleSem; /* Semaphore allocated as global variable */

    int
childCreate(KnPc entry)
{
  KnActorPrivilege      actorP;
  KnDefaultStartInfo_f  startInfo;
  char*                 userStack;
  int                   childLid = -1;
  int                   res;

  startInfo.dsType            = K_DEFAULT_START_INFO;
  startInfo.dsSystemStackSize = K_DEFAULT_STACK_SIZE;

  res = actorPrivilege(K_MYACTOR, &actorP, NULL);
  if (res != K_OK) {
    printf("Cannot get the privilege of the actor, error %d\n", res);
    exit(1);
  }

  if (actorP == K_SUPACTOR) {
    startInfo.dsPrivilege = K_SUPTHREAD;
  } else {
    startInfo.dsPrivilege = K_USERTHREAD;
  }

  if (actorP != K_SUPACTOR) {
    userStack = malloc(USER_STACK_SIZE);
    if (userStack == NULL) {
      printf("Cannot allocate user stack\n");
      exit(1);
    }
    startInfo.dsUserStackPointer = userStack + USER_STACK_SIZE;
  } 

  startInfo.dsEntry = entry;

  res = threadCreate(K_MYACTOR, &childLid, K_ACTIVE, 0, &startInfo);
  if (res != K_OK) {
    printf("Cannot create the thread, error %d\n", res);
    exit(1);
  }

  return childLid;
}

    void
sampleThread()
{
  int myThreadLi;
  int res;

  myThreadLi = threadSelf();

  printf("I am the new thread. My thread identifier is: %d\n", myThreadLi);

  res  = semV(&sampleSem);
  if (res != K_OK){						      
    printf("Cannot perform the semV operation, error %d\n", res);     
    exit(1);							      
  }								      
      /* Suicide */
  res = threadDelete(K_MYACTOR, K_MYSELF);
  if (res != K_OK){						      
    printf("Cannot suicide, error %d\n", res);			      
    exit(1);							      
  }								      

      /* Should never reach this point! */
}


int main(int argc, char** argv, char**envp)
{
  int        myThreadLi;
  int        newThreadLi;
  int        res;

     /*
      * Initialize the semaphore to 0 so that
      * the first semP() operation blocks.
      */
  res = semInit(&sampleSem, 0);
  if (res != K_OK) {						      
    printf("Cannot initialize the semaphore, error %d\n", res);	      
    exit(1);							      
  }								      

  newThreadLi = childCreate((KnPc)sampleThread);

  myThreadLi = threadSelf();

  printf("Parent thread identifier = %d, Child thread identifier = %d\n",
	 myThreadLi, newThreadLi);

      /* 
       * Since semaphore has been initialized to 0
       * this semP will block until a semV is performed 
       * by the created thread, letting the main thread know
       * that created thread's job is done.
       */
  res = semP(&sampleSem, K_NOTIMEOUT);
  if (res != K_OK) {						      
    printf("Cannot perform the semP operation, error %d\n", res);     
    exit(1);							      
  }								      
     /*
      * Created thread has run and done all of its job.
      * It is time to safely exit.
      */
 return 0;
}

Mutexes

Assume that the two threads need to access one or more global variables in a consistent fashion. A simple example could be that each of the threads needs to add two numbers to a unique global counter. Whatever the scheduling may be, the unique global counter should always reflect the accurate sum of all numbers added by both threads.

This could be done using semaphores. However, the ChorusOS operating system provides mutexes which have been specifically designed and tuned for these types of needs.

A mutex is a binary flag associated with a queue, possibly empty, of waiting threads. The mutex can be locked or free. At initialization, the mutex is set to the free state. 

#include <chorus.h>

int mutexInit(KnMutex* mutex);

As for semaphores, the mutex must have been previously allocated by the user. This implies that mutexes may be allocated where convenient for the application, and that there is no limit imposed by the system on the maximum number of mutexes.

Three operations are provided on these mutexes.

The following example shows a small and simple library routine named sampleAdd() which receives two integer arguments and adds them to a global variable one after the other. The code of the previous semaphore example has been modified so that both the main thread and the created thread perform a number of calls to that library. When the job is done, the main thread prints the result and terminates the actor. Refer to the mutexInit(2K) man page.

Example 6-4 Protecting Shared Data Using Mutexes 

(file: progov/mutex.c)

#include <stdio.h>
#include <stdlib.h>
#include <chorus.h>

#define USER_STACK_SIZE (1024 * sizeof(long))			      
								      
KnSem   sampleSem;
KnMutex sampleMutex;
long    grandTotal;

    int
childCreate(KnPc entry)
{
  KnActorPrivilege      actorP;
  KnDefaultStartInfo_f  startInfo;
  char*                 userStack;
  int                   childLid = -1;
  int                   res;

  startInfo.dsType            = K_DEFAULT_START_INFO;
  startInfo.dsSystemStackSize = K_DEFAULT_STACK_SIZE;

  res = actorPrivilege(K_MYACTOR, &actorP, NULL);
  if (res != K_OK) {
    printf("Cannot get the privilege of the actor, error %d\n", res);
    exit(1);
  }

  if (actorP == K_SUPACTOR) {
    startInfo.dsPrivilege = K_SUPTHREAD;
  } else {
    startInfo.dsPrivilege = K_USERTHREAD;
  }

  if (actorP != K_SUPACTOR) {
    userStack = malloc(USER_STACK_SIZE);
    if (userStack == NULL) {
      printf("Cannot allocate user stack\n");
      exit(1);
    }
    startInfo.dsUserStackPointer = userStack + USER_STACK_SIZE;
  } 

  startInfo.dsEntry = entry;

  res = threadCreate(K_MYACTOR, &childLid, K_ACTIVE, 0, &startInfo);
  if (res != K_OK) {
    printf("Cannot create the thread, error %d\n", res);
    exit(1);
  }

  return childLid;
}

    void
sampleAdd(int a, int b)
{
  int   res;
 
  res = mutexGet(&sampleMutex);

  grandTotal += a;
  grandTotal += b;

  res = mutexRel(&sampleMutex);
}

    void
sampleThread()
{
  int res;
  int i;

  for(i = 0; i < 10; i++) {
    sampleAdd(threadSelf(), i);        /* Why not ??? */
  }

  res  = semV(&sampleSem);
  if (res != K_OK){						      
    printf("Cannot perform the semV operation, error %d\n", res);     
    exit(1);							      
  }								      
      /* Suicide */
  threadDelete(K_MYACTOR, K_MYSELF);
}

int main(int argc, char** argv, char**envp)
{
  int        i;
  int        newThreadLi;
  int        res;

  res = semInit(&sampleSem, 0);
  if (res != K_OK) {						      
    printf("Cannot initialize the semaphore, error %d\n", res);       
    exit(1);							      
  }								      
								      
  res = mutexInit(&sampleMutex);

  newThreadLi = childCreate((KnPc)sampleThread);

  for(i = 0; i < 20; i++){
    sampleAdd(threadSelf(), i);      /* Why not ??? */
  }

  res = semP(&sampleSem, K_NOTIMEOUT);
  if (res != K_OK) {						      
    printf("Cannot perform the semP operation, error %d\n", res);     
    exit(1);							      
  }								      
								      
  printf("grandTotal is %d\n", grandTotal);

  return 0;
}

Basic Scheduling Control

The ChorusOS operating system provides two alternative ways of scheduling threads. These two features are mutually exclusive:

The default FIFO scheduler defines a pure priority-based, preemptive, FIFO (first-in first-out) policy. Priority of threads may vary from K_FIFO_PRIOMAX (0 and highest priority) to K_FIFO_PRIOMIN (255 and lowest priority). Within this policy, a thread which becomes ready to run after being blocked is always inserted at the end of its priority ready queue. A running thread is preempted only if a thread of a strictly higher priority becomes ready to run. A preempted thread is placed at the head of its priority queue, so that it will be selected when no other ready thread has a greater priority.

The ROUND_ROBIN feature is a general framework supporting simultaneous multiple scheduling policies or classes. The main classes dealt with here are the CLASS_FIFO and the CLASS_RR policies.

The CLASS_FIFO reproduces the behavior of the default scheduler policy precisely.

The CLASS_RR implements a priority-based preemptive policy with round-robin time slicing. Priority of threads may vary from K_RR_PRIOMAX to K_RR_PRIOMIN. It is similar to the default scheduler policy, except that an elected thread is given a fixed time quantum. If the thread is still running at quantum expiration, it is de-scheduled and placed at the end of its priority queue, thus yielding the processor to other threads of equal priority (if any).

It is possible to set scheduling attributes of threads at thread creation time (using the void* schedParams parameter of threadCreate()). It is also possible to get and modify scheduling attributes of a thread dynamically through the following call. 

#include  <chorus.h>
#include  <sched/chFifo.h>
#include  <sched/chRr.h>
#include  <sched/chRt.h>
#include  <sched/chTs.h>
 
int threadScheduler(KnCap*      actorCap,
                    KnThreadLid thLi,
                    void*       oldParam,
                    void*       newParam);

This service enables you to get or set scheduling parameters of any thread of any actor, as long as both the actor capability and the thread identifier are known. threadScheduler() returns the current scheduling attributes of the target thread at the location defined by oldParam, if non-null. It will also set the attributes of the target thread according to the description provided at the location defined by newParam if non-null.

As the size, layout and semantics of scheduling parameters may vary depending on the scheduler configured in the system, or on the class of the ROUND_ROBIN framework, parameters are untyped in the generic interface definition. However, all scheduling parameter descriptions are similar, at least for the initial fields: 

struct KnFifoThParms {
    KnSchedClass     fifoClass;      /* Always set to K_SCHED_FIFO */
    KnFifoPriority   fifoPriority;
} KnFifoThParms;

struct KnRrThParms {
    KnSchedClass     rrClass;        /* Always set to K_SCHED_RR */
    KnRrPriority     rrPriority;
} KnRrThParms;

The first field defines the scheduling policy applied or to be applied to the thread. The second field defines the priority of the thread within the scheduling policy.

Example 6-5 is based on the semaphore example, with a modification to the childCreate() routine so that it can receive scheduling attributes of the thread to be created. The main thread invokes this modified routine, so that the created thread will start as soon as it is created, rather than waiting for the main thread to yield the processor. Thus, the created thread must be given a higher priority than the main thread.

Refer to the threadScheduler(2K) and threadCreate(2K) man pages.

Example 6-5 Changing Scheduling Attributes

(file: progov/thSched.c)

#include <stdio.h>
#include <stdlib.h>
#include <chorus.h>

#define USER_STACK_SIZE (1024 * sizeof(long))

KnSem   sampleSem;

    int
childSchedCreate(KnPc entry, void* schedParams)
{
  KnActorPrivilege      actorP;
  KnDefaultStartInfo_f  startInfo;
  char*                 userStack;
  int                   childLid = -1;
  int                   res;

      /* Set defaults startInfo fields */
  startInfo.dsType            = K_DEFAULT_START_INFO;
  startInfo.dsSystemStackSize = K_DEFAULT_STACK_SIZE;

      /* Get actor's privilege */
  res = actorPrivilege(K_MYACTOR, &actorP, NULL);
  if (res != K_OK) {						      
    printf("Cannot get the privilege of the actor, error %d\n", res); 
    exit(1);							      
  }								      

      /* Set thread privilege */
  if (actorP == K_SUPACTOR) {
    startInfo.dsPrivilege = K_SUPTHREAD;
  } else {
    startInfo.dsPrivilege = K_USERTHREAD;
  }

      /* Allocate a stack for user threads */
  if (actorP != K_SUPACTOR) {
    userStack = malloc(USER_STACK_SIZE);
    if (userStack == NULL) {					      
      printf("Cannot allocate user stack\n");			      
      exit(1);							      
    }								      

    startInfo.dsUserStackPointer = userStack + USER_STACK_SIZE;
  } 

      /* Set entry point for the new thread */
  startInfo.dsEntry = entry;

      /* Create the thread in the active state */
  res = threadCreate(K_MYACTOR, &childLid, K_ACTIVE, schedParams, &startInfo);
  if (res != K_OK) {						      
    printf("Cannot create the thread, error %d\n", res);	      
    exit(1);						      	      
  }								      
								      
  return childLid;
}

    void
sampleThread()
{
  int myThreadLi;
  int res;

  myThreadLi = threadSelf();

  printf("I am the new thread. My thread identifier is: %d\n", myThreadLi);

  res  = semV(&sampleSem);
  if (res != K_OK){						      
    printf("Cannot perform the semV operation, error %d\n", res);     
    exit(1);							      
  }								      
  threadDelete(K_MYACTOR, K_MYSELF);
}

int main(int argc, char** argv, char**envp)
{

  int                  myThreadLi;
  int                  newThreadLi;
  int                  res;
  KnThreadDefaultSched schedParams;

  res = semInit(&sampleSem, 0);
  if (res != K_OK) {						      
    printf("Cannot initialize the semaphore, error %d\n", res);       
    exit(1);							      
  }								      

      /* acquire my own scheduling attributes  */
  res = threadScheduler(K_MYACTOR, K_MYSELF, &schedParams, NULL);

      /* Increase priority of thread to be created */
  schedParams.tdPriority -= 1;

  newThreadLi = childSchedCreate((KnPc)sampleThread, &schedParams);

  myThreadLi = threadSelf();

  printf("Parent thread identifier = %d, Child thread identifier = %d\n",
	 myThreadLi, newThreadLi);

  res = semP(&sampleSem, K_NOTIMEOUT);
  if (res != K_OK) {						      
    printf("Cannot perform the semP operation, error %d\n", res);     
    exit(1);							      
  }								      

  return 0;
}

Managing Per-Thread Data

One of the most common issues in a multithreaded environment is how to manage per-thread data structures. This may become an important question for libraries. In a single-threaded process, managing these data as global variables is fine. In a multithreaded environment, it will no longer work.

The ChorusOS operating system provides a convenient way for threads to manage per-thread data. A piece of data which needs to be instantiated on a per-thread basis must be associated with a unique key. The key may be obtained from the system through a call to ptdKeyCreate(). This data may accessed using specific calls named ptdSet() and ptdGet(). 

#include <pd/chPd.h>

int ptdKeyCreate(PdKey*  key,
                 KnPdHdl destructor);

ptdKeyCreate() generates a unique key, which is opaque to the user. This key is stored at the location defined by the key argument. The user may, optionally, specify a routine as the destructor argument. This routine will be invoked at thread deletion time and will be passed the value associated with key. Upon return from ptdKeyCreate(), the value associated with key is 0. This type of key is visible to all threads of the actor, but each thread using a given key will have its own private copy of the data.

#include <pd/chPd.h>

int ptdSet(PdKey  key,
           void*  value);

ptdSet() enables a thread to associate the value value with the key key which has been generated previously by a call to ptdKeyCreate(). 

#include <pd/chPd.h>

int ptdGet(PdKey  key);

ptdGet() returns the last value associated with the key by this same thread.

Example 6-6 includes a small library that returns a pointer to the next word of a string. This is a simplified version of the strtok() C library routine. For simplicity, it is assumed that words are always separated by spaces in the string.

This library is callable simultaneously from different threads, each thread working on its own string. The routine that returns the pointer to the next word does not take any parameters.

These routines are called snw routines (where snw stands for String Next Word). There is a snwSet(char *str) routine which defines the string that will be looked up by the invoking thread, and a char* snwGet() returning a pointer to the next word.

The library is invoked from the main thread and the created thread on two different strings in order to count the number of words in each string. The results are printed and the threads are synchronized before terminating the actor.

Refer to the ptdKeyCreate(2K), ptdSet(2K), and ptdGet(2K) man pages.

Example 6-6 Managing Per-Thread Data

(file: progov/perThreadData.c)

#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <chorus.h>
#include <pd/chPd.h>

#define USER_STACK_SIZE (1024 * sizeof(long))			      
								      
KnSem   sampleSem;
PdKey   snwKey;

    int
childCreate(KnPc entry)
{
  KnActorPrivilege      actorP;
  KnDefaultStartInfo_f  startInfo;
  char*                 userStack;
  int                   childLid = -1;
  int                   res;

  startInfo.dsType            = K_DEFAULT_START_INFO;
  startInfo.dsSystemStackSize = K_DEFAULT_STACK_SIZE;

  res = actorPrivilege(K_MYACTOR, &actorP, NULL);
  if (res != K_OK) {
    printf("Cannot get the privilege of the actor, error %d\n", res);
    exit(1);
  }

  if (actorP == K_SUPACTOR) {
    startInfo.dsPrivilege = K_SUPTHREAD;
  } else {
    startInfo.dsPrivilege = K_USERTHREAD;
  }

  if (actorP != K_SUPACTOR) {
    userStack = malloc(USER_STACK_SIZE);
    if (userStack == NULL) {
      printf("Cannot allocate user stack\n");
      exit(1);
    }
    startInfo.dsUserStackPointer = userStack + USER_STACK_SIZE;
  } 

  startInfo.dsEntry = entry;

  res = threadCreate(K_MYACTOR, &childLid, K_ACTIVE, 0, &startInfo);
  if (res != K_OK) {
    printf("Cannot create the thread, error %d\n", res);
    exit(1);
  }

  return childLid;
}

    void
snwInit()
{
  int res;
      /* Just allocate a key for our "snw" library */
  res = ptdKeyCreate(&snwKey, NULL);
  if (res != K_OK) {						      
    printf("Cannot create a ptd key, error %d\n", res);		      
    exit(1);							      
  }								      
}

    void
snwSet(char* str)
{
   int res;

   res = ptdSet(snwKey, str);
   if (res != K_OK) {						      
     printf("Cannot set the ptd key, error %d\n", res);		      
     exit(1);							      
   }								      
}

    char*
snwGet()
{
  int    res;
  char*  p;
  char*  s;

  p = (char*)ptdGet(snwKey);
  if (p == NULL) return NULL;

  s = strchr(p, ' ');

  if (s != NULL) {
    s++;
  } else if (*p != '\0') {
      /* Last word might not have a following space */
    s = p + strlen(p);
  }

  res = ptdSet(snwKey, s);
  return s;
}

    void
sampleThread()
{
  char* ptr;
  int   words = 0;
  int   res;

  snwSet("This is the child thread!");

  for (ptr= snwGet(); ptr != NULL; ptr = snwGet()) {
    words++;
  }

  printf("Child thread found %d words.\n", words);

  res  = semV(&sampleSem);
  if (res != K_OK){						      
    printf("Cannot perform the semV operation, error %d\n", res);     
    exit(1);							      
  }								      
  threadDelete(K_MYACTOR, K_MYSELF);
}


int main(int argc, char** argv, char**envp)
{
  char*      ptr;
  int        words = 0;
  int        res;
  int        newThreadLi;

  res = semInit(&sampleSem, 0);
  if (res != K_OK) {						      
    printf("Cannot initialize the semaphore, error %d\n", res);	      
    exit(1);							      
  }								      

  snwInit();

  newThreadLi = childCreate((KnPc)sampleThread);

  snwSet("I am the main thread and counting words in this string!");

  for (ptr= snwGet(); ptr != NULL; ptr = snwGet()) {
    words++;
  }

  printf("Main thread found %d words.\n", words);

  res = semP(&sampleSem, K_NOTIMEOUT);
  if (res != K_OK) {						      
    printf("Cannot perform the semP operation, error %d\n", res);     
    exit(1);							      
  }								      

  return 0;
}

Threads and Libraries

As illustrated in the previous example, it is often the case that C and C++ libraries have been designed for UNIX processes which were initially mono-threaded entities. In order to allow C programmers to continue using the usual libraries within multithreaded actors, the ChorusOS operating environment provides a set of adapted C libraries which may be used from different threads of a given actor without encountering problems.

In the previous examples, some of these adapted libraries, such as printf(), fprintf(), fopen(), and malloc(), were already used. All of these C libraries have been adapted to work efficiently even within a multithreaded actor. Modifications are not visible to the programmer. They rely mainly on synchronization such as mutexes for protecting critical sections and on the per-thread data mechanism to store per-thread global data.

Some libraries did not require any modification and can work in a straightforward fashion within a multithreaded actor. These libraries, such as strtol() (string to lower case), work exclusively on local variables and do not access or generate any global states.

Chapter 7 Memory Management

Actors within the ChorusOS operating system environment may extend their address space using the malloc() library call as illustrated earlier in this document. However, this is a rather inflexible way of allocating memory, as there is no way to control the attributes of the allocated memory; that is, whether it is a read only or a read/write memory area. The malloc() routine uses the ChorusOS operating system services described in this section. These services may also be used to share a region of memory between two or more actors.

This chapter explains the recommended way of allocating memory for an actor. It contains the following sections:

The ChorusOS hot restart feature provides support for using persistent memory; memory which can extend beyond the lifetime of the runtime instance of an actor. Hot restart is not covered in this chapter. For information about using hot restart and the persistent memory services it provides, see the ChorusOS 4.0 Hot Restart Programmer's Guide.

Memory Region Descriptors

The ChorusOS operating system offers various services which enable an actor to extend its address space dynamically by allocating memory regions. An actor may also shrink its address space by freeing memory regions. An area of memory to be allocated or freed is described to the system through a region descriptor of the following type:

typedef struct {
    VmAddr    startAddr;
    VmSize    size;
    VmFlags   options;
    VmAddr    endAddr;
    void*     opaque1;
    VmFlags   opaque2;
} KnRgnDesc;

The startAddr field defines the starting address of the memory region. The size field defines its length expressed in bytes. The options field enables a low level control on the attributes of the memory region to be allocated. The opaque1 and opaque2 fields should be set to NULL if they are not being used by the application.

The options field is a combination of flags, of which the following are the most important:

On a ChorusOS operating system configured with the Virtual Memory feature, further options are available. The most important one allows control of the swapping policy to be applied to the pages of the created region:

Allocating and Freeing Memory Regions

A memory region is allocated through the following call:

#include <chorus.h>

int rgnAllocate(KnCap*     actorCap,
                KnRgnDesc* rgnDesc);

This call creates a memory region as described by the rgnDesc parameter within the address space of the actor defined by the actorCap parameter. Most applications set actorCap to K_MYACTOR to manage their own address space.

An unused part of an address space may be freed by the following call:

#include <chorus.h>

int rgnFree(KnCap*     actorCap,
            KnRgnDesc* rgnDesc);

This call frees a memory region as described by the rgnDesc parameter within the address space of the actor defined by the actorCap parameter.

Figure 7-1 Memory Region Allocation and Deallocation

Graphic

Example 7-1 does the following:

The main steps of the example are illustrated in Figure 7-1. Refer to the rgnAllocate(2K) and rgnFree(2K) man pages.

Example 7-1 Allocating a Memory Region

(file: progov/rgnAlloc.c)

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <chorus.h>

#define RGN_SIZE_1 (6 * vmPageSize())
#define RGN_SIZE_2 (3 * vmPageSize())
#define FREE_START (2 * vmPageSize())
#define FREE_SIZE  (4 * vmPageSize())
#define STILL_ALLOC_START (FREE_SIZE - (RGN_SIZE_2 - FREE_START))

int main(int argc, char** argv, char**envp)
{
  KnRgnDesc           rgnDesc;
  KnActorPrivilege    actorP;
  int                 res;
  VmFlags             rgnOpt = 0;
  char*               ptr1 = NULL;	/* Avoids "uninited" warning */
  char*               ptr2 = NULL;	/* Avoids "uninited" warning */

  printf("Starting rgnAllocate example\n");			      

  res = actorPrivilege(K_MYACTOR, &actorP, NULL);
  if (res != K_OK) {						      
    printf("Cannot get actor privilege, error %d\n", res);	      
    exit(1);							      
  }								      
								      
  if (actorP == K_SUPACTOR) {
    rgnOpt = K_SUPERVISOR;
  }

  rgnDesc.size = RGN_SIZE_1;
  rgnDesc.options = rgnOpt | K_WRITABLE | K_FILLZERO | K_ANYWHERE;
  rgnDesc.opaque1 = NULL;
  rgnDesc.opaque2 = NULL;

      /*
       * No need to set rgnDesc.startAddr
       * since we set the K_ANYWHERE flag
       */
  res = rgnAllocate(K_MYACTOR, &rgnDesc);

  if (res == K_OK) {
    printf("Successfully allocated memory starting at 0x%x\n", 
	   rgnDesc.startAddr);

    ptr1 = (char*) rgnDesc.startAddr;
  } else {
    printf("First rgnAllocate failed with error %d\n", res);
    exit(1);
  }

  strcpy(ptr1, "Fill the allocated memory with this string\n");

      /* 
       * Second allocate has a fixed address, such that
       * both memory areas will be contiguous. Hence
       * we do not want the K_ANYWHERE flag any more.
       */
  rgnDesc.size       = RGN_SIZE_2;
  rgnDesc.options   &= ~K_ANYWHERE;
  rgnDesc.startAddr -= RGN_SIZE_2;

  res = rgnAllocate(K_MYACTOR, &rgnDesc);

  if (res == K_OK) {
    printf("Successfully allocated memory starting at 0x%x)\n", 
	   rgnDesc.startAddr);
    ptr2 = (char*) rgnDesc.startAddr;
  } else {
    printf("Second rgnAllocate failed with error %d\n", res);
    exit(1);
  }

      /* Copy from first allocated area to second one */
  strcpy(ptr2, ptr1);

      /* 
       * Free a memory area spanning both areas 
       * previously created
       */
  rgnDesc.options = NULL;
  rgnDesc.startAddr = (VmAddr) (ptr2 + FREE_START);
  rgnDesc.size      = FREE_SIZE;

  res = rgnFree(K_MYACTOR, &rgnDesc);
								      
  if (res != K_OK) {						      
    printf("Cannot free memory, error %d\n", res);		      
    exit(1);							      
  }								      
								      
      /* 
       * Access to ptr2: beginning of secondly allocated area
       * is still valid.
       * Access to ptr1 is now invalid: memory has been freed.
       */
  printf("%s", ptr2);	

      /*
       * Access to "end" of first allocated area
       * is still valid
       */
  ptr1 += STILL_ALLOC_START;
  strcpy(ptr1, ptr2);
  
      /*
       * Remaining memory areas not yet freed will 
       * effectively be freed at actor termination time.
       */
  return 0;
}

Sharing Memory Between Two Actors

The ChorusOS operating system offers the possibility of sharing an area of memory between two or more actors, regardless of whether these actors are user or supervisor actors. The memory area does not need to be located at the same address within the address space of each actor.

This mechanism is based on the following service: 

#include <chorus.h>

int rgnMapFromActor(KnCap*    targetActor,
                    KnRgnDesc*targetRgnDesc,
                    KnCap*    sourceActor,
                    KnRgnDesc*sourceRgnDesc);

This call allows mapping of a memory area as defined by sourceActor and sourceRgnDesc in the address space of the actor defined by the targetActor parameter. The source memory area is explicitly defined by the startAddr and size fields of the sourceRgnDesc parameter. The region created within the address space of the target actor is defined by the targetRgnDesc parameter: the address may be fixed or undefined if the K_ANYWHERE flag is set. After the mapping has been established, both actors may freely use the shared memory area.

Figure 7-2 shows two actors before they share memory and after sharing has been established. Usually some synchronization mechanism is required in order to get a consistent view of the memory: semaphores may be used in shared memory areas to synchronize threads from the various actors.

Figure 7-2 Actors Sharing Memory

Graphic

Example 7-2 does the following:

This example uses a call which enables an actor to get its own capability in order to pass it to another actor.

#include <chorus.h>

int actorSelf(KnCap* myCap);

Refer to the rgnMapFromActor(2K) man page.

Example 7-2 Sharing a Memory Region

(file: progov/rgnMapFromActor.c)

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <chorus.h>
#include <am/afexec.h>

AcParam param;

#define RGN_SIZE  (3 * vmPageSize())
#define SHARED_RGN_SIZE (1 * vmPageSize())

typedef struct sampleSharedArea {
  KnSem    sem;
  char     data[1];
} sharea_t;

char capString[80];
char sharedAddr[20];

int main(int argc, char** argv, char**envp)
{
  KnRgnDesc    rgnDesc;
  sharea_t*    ptr;
  KnCap        spawningCap;
  KnCap        spawnedCap;
  int          res;
  VmFlags      rgnOpt = 0;
  KnActorPrivilege actorP;

  res = actorPrivilege(K_MYACTOR, &actorP, NULL);
  if (res != K_OK) {						      
    printf("Cannot get privilege, error %d\n", res);		      
    exit(1);							      
  }								      
								      
  if (actorP == K_SUPACTOR) {
    rgnOpt = K_SUPERVISOR;
  }

  if (argc == 1) {
        /*
         * This is the first actor (or spawning actor):
         *   Allocate a memory region
         *   Initialize a semaphore within the region
         *   Spawn the second actor
         *   Wait on the semaphore
         *   Get data written in shared mem by spawned actor
         *   Terminate
         */
    rgnDesc.size = RGN_SIZE;
    rgnDesc.options = rgnOpt | K_ANYWHERE | K_WRITABLE | K_FILLZERO;
    rgnDesc.opaque1 = NULL;
    rgnDesc.opaque2 = NULL;

    res = rgnAllocate(K_MYACTOR, &rgnDesc);
    if (res != K_OK) {						      
      printf("Cannot allocate memory, error %d\n", res);	      
      exit(1);							      
    }								      

    ptr = (sharea_t*) rgnDesc.startAddr;

    strcpy(&ptr->data[0], "First actor initializing the shared mem\n");

    res = semInit(&ptr->sem, 0);

        /*
         * Get my own capability and pass it as a string argument
         * to spawned actor, so that it may use it to share memory
         */
    actorSelf(&spawningCap);
    sprintf(capString, "0x%x 0x%x 0x%x 0x%x", spawningCap.ui.uiHead,
    spawningCap.ui.uiTail, spawningCap.key.keyHead,
    spawningCap.key.keyTail);

       /*
        * Pass address of memory to be shared as a string argument
        * to spawned actor.
        */
    sprintf(sharedAddr, "0x%x", ptr);

    param.acFlags = (actorP == K_SUPACTOR)? AFX_SUPERVISOR_SPACE : 
                                            AFX_USER_SPACE;
    res = afexeclp(argv[0], &spawnedCap, &param , argv[0], capString,
		   sharedAddr, NULL);
    if (res == -1) {						      
      printf("cannot spawn second actor, error %d\n", errno);	      
      exit(1);							      
    }								      

    semP(&ptr->sem, K_NOTIMEOUT);

    printf("%s", &ptr->data[0]);

  } else {
    
    KnRgnDesc srcRgn;
    KnRgnDesc tgtRgn;
    unsigned long uHead, uTail, kHead, kTail;
        /*
         * This is the spawned actor:
         *   Get arguments
         *   Set up the memory sharing 
         *   Write some string in shared memory
         *   Wake up spawning actor
         *   Terminate
         */
    sscanf(argv[1], "0x%x 0x%x 0x%x 0x%x", &uHead, &uTail, &kHead, &kTail);
    spawningCap.ui.uiHead = uHead;
    spawningCap.ui.uiTail = uTail;
    spawningCap.key.keyHead = kHead;
    spawningCap.key.keyTail = kTail;

    sscanf(argv[2], "0x%x", &srcRgn.startAddr);

    if (actorP != K_SUPACTOR) {

      srcRgn.size      = SHARED_RGN_SIZE;
      tgtRgn.startAddr = srcRgn.startAddr;
      tgtRgn.size      = SHARED_RGN_SIZE; 
      tgtRgn.options   = rgnOpt | K_WRITABLE;
      tgtRgn.opaque1   = NULL;
      tgtRgn.opaque2   = NULL;

      res = rgnMapFromActor(K_MYACTOR, &tgtRgn, &spawningCap, &srcRgn);

      if (res != K_OK) {
        	printf("Cannot share memory, error %d\n", res);
        	exit(1);
      }
      ptr = (sharea_t*) tgtRgn.startAddr;

    } else {
       /*
        * Both actors are running in supervisor space,
        * There is no need to perform the rgnMapFromActor.
        * One may use the received shared address.
        */
      ptr = (sharea_t*) srcRgn.startAddr;
    }

	/* Get data from spawning actor */
    printf("Spawning actor sent: %s", &ptr->data[0]);

	/* Modify contents of shared memory */
    sprintf(&ptr->data[0], "Spawned actor mapped shared memory at 0x%x\n",
	    ptr);

    res = semV(&ptr->sem);
    if (res != K_OK) {						      
      printf("Spawned actor failed on semV, error %d\n", res);	      
      exit(1);							      
    }								      
  }
  return 0;
}

Chapter 8 Inter-actor Communication

This chapter explains how actors can communicate. It contains the following sections:

Introduction

The ChorusOS operating system offers a set of services for communicating between actors. Sharing memory between two actors to enable them to communicate was described in "Sharing Memory Between Two Actors". Other communication mechanisms can be split into two categories:

Message Queues

This feature is designed to allow an application composed of one or multiple actors to create a shared communication environment, often referred to as message space, within which these actors can exchange messages efficiently. In particular, supervisor and user actors of the same application can use this feature to exchange messages. Furthermore, messages may be initially allocated and sent by interrupt handlers in order to be processed later by threads.

The feature is designed around the concept of message space which encapsulates within a single entity:

A message space is a temporary resource which must be explicitly created by one actor within the application. Once created, a message space may be opened by other actors within the application. Actors which have opened the same message space are said to share this message space. A message space is automatically deleted when its creating actor and all actors which opened it have exited.

A message pool is defined by two parameters (message size and number of messages) provided by the application when it creates the message space. The configuration of the set of message pools defined within a message space depends upon the needs of the application.

A message is an array of bytes which can be structured and used at application level through any appropriate convention. Messages are presented to actors as pointers within their address space.

Messages are posted to message queues belonging to the same message space. All actors sharing a message space can allocate messages from the message pools. In the same way, all actors sharing a message space have send and receive rights on each queue of the message space.

Even though most applications only need to create a single message space, the feature is designed to allow an application to create or open multiple message spaces. However, messages allocated from one message space cannot be sent to a queue of a different message space. A typical use of message spaces is as follows:

  1. The first actor, aware of the overall requirements of the application, creates the message space.

  2. Other actors of the application open the shared message space.

  3. An actor allocates a message from a message pool, and fills it with the data to be sent.

  4. The actor which allocated the message can then post it to the appropriate queue, and can assign a priority to the message.

  5. The destination actor can get the message from the queue. At this point, the message is removed from the queue.

  6. Once the destination actor has processed the message, it may free the message so that the application may allocate it again. Alternatively, the destination actor could, for example, modify the message and post it again to another queue.

In order to make the service as efficient as possible, physical memory is allocated for all messages and data structures of the message space at message space creation. At message space open time, this memory is transparently mapped by the system into the actor address space. Further operations such as posting and receiving a message are done without any copy involved.

Creating a message space is performed as follows: 

#include <mipc/chMipc.h>

int msgSpaceCreate (KnMsgSpaceId     spaceGid,
                    unsigned int     msgQueueNb,
                    unsigned int     msgPoolNb,
                    const KnMsgPool* msgPools);

The spaceGid parameter is a unique global identifier assigned by the application to the message space being created. This identifier is also used by other actors of the application to open the message space. Thus, this identifier serves to bind actors participating in the application to the same message space. The K_PRIVATEID predefined global identifier indicates that the message space created will be private to the invoking actor: its queues and message pools will only be accessible to threads executing within this actor. No other actor will be able to open that message space. The message space is described by the last three parameters:

Figure 8-1 shows an example of a message space recently created by an actor.

Figure 8-1 Creating a Message Space

Graphic

The created message space is assigned a local identifier which is returned to the invoking actor as the return value of the msgSpaceCreate() call. The scope of this local identifier is the invoking actor.

A message space may be opened by other actors through the following call:

#include <mipc/chmipc.h>

int msgSpaceOpen(KnMsgSpaceId spaceGid);

The message space assigned with the spaceGid unique global identifier must have been previously created by a call to msgSpaceCreate(). A local identifier is returned to the invoking actor. This message space local identifier can then be used to manipulate messages and queues within the message space. Figure 8-2 shows an example of a message space recently opened by a second actor.

Figure 8-2 Opening a Message Space

Graphic

A message may be allocated by the following call: 

#include <mipc/chmipc.h> 

int msgAllocate(int             spaceLid,
                unsigned int    poolIndex,
                unsigned int    msgSize,
                KnTimeVal*      waitLimit,
                char**          msgAddr);

msgAllocate() attempts to allocate a message from the appropriate pool of the message space identified by the spaceLid return value of a previous call to msgSpaceOpen() or msgSpaceCreate(). If poolIndex is not set to K_ANY_MSGPOOL, the allocated message will be the first free (not yet allocated) message of the pool defined by poolIndex, regardless of the value specified by the msgSize parameter. Otherwise, if poolIndex is set to K_ANY_MSGPOOL, the message will be allocated from the first pool for which the message size fits the requested msgSize. In this context, first pool means the one with the lowest index in the set of pools defined at message space creation time. If the pool is empty, no attempt will be made to allocate a message from another pool.

If the message pool is empty (all messages have been allocated and none has been freed yet), msgAllocate() will block, waiting for a message in the pool to be freed. The invoking thread is blocked until the wait condition defined by waitLimit expires.

If successful, the address of the allocated message is stored at the location defined by msgAddr. The returned address is the address of the message within the address space of the actor. Remember that a message space is mapped within the address space of the actors sharing it. However, message spaces and, as a consequence, messages themselves, may be mapped at different addresses in different actors. This is specially true for message spaces shared between supervisor and user actors.

Figure 8-3 illustrates two actors allocating two messages from two different pools of the same message space.

Figure 8-3 Allocating Messages from Pools

Graphic

Once it has been allocated and initialized by the application, a message may be posted to a message queue with:

#include <mipc/chmipc.h> 

int msgPut(int             spaceLid,
           unsigned int    queueIndex,
           char*           msg,
           unsigned int    prio);

msgPut() posts the message, the address of which is msg, to the message queue queueIndex within the message space, the local identifier of which is spaceLid. The message must have been previously allocated by a call to msgAllocate(). The message will be inserted into the queue according to its priority, prio. Messages with a high priority will be taken first from the queue.

Posting a message to a queue is done without any message copy, and may be done within an interrupt handler, or with preemption disabled.

Figure 8-4 illustrates the previous actors posting their messages to different queues.

Figure 8-4 Posting Messages to Queues

Graphic

Getting a message from a queue, if any, is achieved using:

#include <mipc/chmipc.h>

int msgGet(int             spaceLid,
           unsigned int    queueIndex,
           KnTimeVal*      waitLimit,
           char**          msgAddr,
           KnUniqueId*     srcActor);

msgGet() enables the invoking thread to get the first message with the highest priority pending behind the message queue queueIndex, within the message space whose local identifier is spaceLid. Messages with equal priority are posted and delivered in a first-in first-out order.

The address of the message delivered to the invoking thread is returned at the location defined by the msgAddr parameter. If no message is pending, the invoking thread is blocked until a message is sent to the message queue, or until the time-out, if any, defined by the waitLimit parameter expires.

The srcActor, if non-null, points to a location where the unique identifier of the actor (referred to as the source actor) which posted the message is to be stored.

No data copy is performed to deliver the message to the invoking thread. Multiple threads can be blocked, waiting in the same message queue. At present it is not possible for one thread to wait for message arrival on multiple message queues. This type of multiplexing mechanism could be implemented at the application level using the ChorusOS event flags facility.

Figure 8-5 illustrates previous actors receiving messages from queues.

Figure 8-5 Getting Messages from Queues

Graphic

A message which is of no further use to the application may be returned to its pool of messages available for further allocation with the following call:

#include <mipc/chMipc.h> 

int msgFree(int      spaceLid,
            char*    msg);

Example 8-1 illustrates a very simple use of the message queue facility.

Refer to the msgSpaceCreate(2K), msgSpaceOpen(2K), msgAllocate(2K), msgPut(2K), msgGet(2K), and msgFree(2K) man pages.

Example 8-1 Communicating Using Message Spaces

(file: progov/msgSpace.c)

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <chorus.h>
#include <mipc/chMipc.h>
#include <am/afexec.h>

AcParam param;

#define NB_MSG_POOLS 2
#define NB_MSG_QUEUES 3
#define SMALL_MSG_SZ  32
#define LARGE_MSG_SZ  256
#define NB_SMALL_MSG  13
#define NB_LARGE_MSG  4
#define SAMPLE_SPACE  1111
#define LARGE_POOL 0
#define SMALL_POOL 1
#define Q1 0
#define Q2 1
#define Q3 2

KnMsgPool samplePools[NB_MSG_POOLS];
char* tagPtr = "Spawned";

int main(int argc, char** argv, char**envp)
{
  int          res;
  int          msgSpaceLi;
  char*        smallMsg;
  char*        smallReply;
  char*        largeMsg;
  KnCap        spawnedCap;
  KnActorPrivilege actorP;

  res = actorPrivilege(K_MYACTOR, &actorP, NULL);
  if (res != K_OK) {						      
      printf("Cannot get actor privilege, error %d\n", res);	      
      exit(1);							      
  }								      

  if (argc == 1) {
  /*
   * This is the first actor (or spawning actor):
   *   Create a message space,
   *   Spawn another actor,
   *   Allocate, modify and post a small message on Q2
   *   Get a large Message from Q3, print its contents, free it
   *   Get reply of small message on Q1, print its contents, free it.
   */

    samplePools[LARGE_POOL].msgSize = LARGE_MSG_SZ;
    samplePools[LARGE_POOL].msgNumber = NB_LARGE_MSG;

    samplePools[SMALL_POOL].msgSize = SMALL_MSG_SZ;
    samplePools[SMALL_POOL].msgNumber = NB_SMALL_MSG;

    msgSpaceLi = msgSpaceCreate(SAMPLE_SPACE, NB_MSG_QUEUES, 
				NB_MSG_POOLS, samplePools);
    if (msgSpaceLi < 0) {					      
      printf("Cannot create the message space error %d\n",	      
        	msgSpaceLi);					      
      exit(1);						      
    }								      

	/*
	 * Message Space has been created, spawn the other actor,
	 * argv[1] set to "Spawned" to differentiate the 2 actors.
	 */
    param.acFlags = (actorP == K_SUPACTOR)? AFX_SUPERVISOR_SPACE : 
                                            AFX_USER_SPACE;
    res = afexeclp(argv[0], &spawnedCap, &param , argv[0], tagPtr,
		   NULL);
    if (res == -1) {						      
      printf("Cannot spawn second actor, error %d\n", errno);	      
      exit(1);							      
    }								      

  	/*
	    * Allocate a small message 
	    */
    res = msgAllocate(msgSpaceLi, SMALL_POOL, SMALL_MSG_SZ, 
		      K_NOTIMEOUT, &smallMsg);
    if (res != K_OK) {						      
      printf("Cannot allocate a small message, error %d\n", res);     
      exit(1);							      
    }								

      /*
       * Initialize the allocated message
       */
    strncpy(smallMsg, "Sending a small message\n", SMALL_MSG_SZ);

      /*
       * Post the allocated small message to Q2 with priority 2
       */
    res = msgPut(msgSpaceLi, Q2, smallMsg, 2);
    if (res != K_OK) {						      
      printf("Cannot post the small message to Q2, error %d\n", res); 
      exit(1);							      
    }								      
  
      /*
       * Get a large message from Q3 and print its contents
       */
    res = msgGet(msgSpaceLi, Q3, K_NOTIMEOUT, &largeMsg, NULL);
    if (res != K_OK) {						      
      printf("Cannot get the large message from  Q3, error %d\n",     
	     res);						      
      exit(1);							      
    }								      

    printf("Received large message contains:\n%s\n", largeMsg);

      /*
       * Free the received large message
       */
    res = msgFree(msgSpaceLi, largeMsg);
    if (res != K_OK) {						      
      printf("Cannot free the large message, error %d\n", res);	      
      exit(1);							      
    }								      

  /*
   * Get the reply to small message from Q1 and print its contents
   */
    res = msgGet(msgSpaceLi, Q1, K_NOTIMEOUT, &smallReply, NULL);
    if (res != K_OK) {						      
      printf("Cannot get the small message reply from  Q1, "	      
             "error %d\n", res);				      
      exit(1);							      
    }								      

    printf("Received small reply contains:\n%s\n", smallReply);

      /*
       * Free the received small reply
       */
    res = msgFree(msgSpaceLi, smallReply);
    if (res != K_OK) {						      
      printf("Cannot free the small reply message, error %d\n", res); 
      exit(1);							      
    }								      


  } else {
    
      /*
       * This is the spawned actor:
       *   Check we have effectively been spawned
       *   Open the message space
       *   Allocate, initialize and post a large message to Q3
       *   Get a small message from Q2, print its contents
       *   Modify it and repost it to Q1
       */

    int     l;

    if ((argc != 2) || (strcmp(argv[1], tagPtr) != 0)) {	      
      printf("%s does not take any argument!\n", argv[0]);	      
      exit(1);						      
    }								      
	    /*
	     * Open the message space, using the same global identifier
	     */
    msgSpaceLi = msgSpaceOpen(SAMPLE_SPACE);
    if (msgSpaceLi < 0) {					      
      printf("Cannot open the message space error %d\n",	      
      msgSpaceLi);					      
      exit(1);						      
    }								      

	     /*
	      * Allocate the large message
	      */
    res = msgAllocate(msgSpaceLi, K_ANY_MSGPOOL, LARGE_MSG_SZ, 
		      K_NOTIMEOUT, &largeMsg);
    if (res != K_OK) {						      
      printf("Cannot allocate a large message, error %d\n", res);     
      exit(1);							      
    }								      

    strcpy(largeMsg, "Sending a very large large large large large message\n");

      /*
       * Post the large message to Q3 with priority 0
       */
    res = msgPut(msgSpaceLi, Q3, largeMsg, 0);
    if (res != K_OK) {						      
      printf("Cannot post the large message to Q3, error %d\n", res); 
      exit(1);							      
    }								      

      /*
       * Get the small message from Q2
       */
    res = msgGet(msgSpaceLi, Q2, K_NOTIMEOUT, &smallMsg, NULL);
    if (res != K_OK) {						      
      printf("Cannot get the small message from Q2, error %d\n", res);
      exit(1);							      
    }								      
	
    printf("Spawned actor received small message containing:\n%s\n", smallMsg);
    
    for (l = 0; l < strlen(smallMsg); l++) {
      if ((smallMsg[l]>= 'a') && (smallMsg[l] <= 'z')) {
        smallMsg[l] = smallMsg[l] - 'a' + 'A';
      }
    }
    
      /*
       * Post the small message back to Q1, with priority 4
       */
    res = msgPut(msgSpaceLi, Q1, smallMsg, 4);
    if (res != K_OK) {						      
      printf("Cannot post the small message reply to Q1, error %d\n", 
	     res);						      
      exit(1);							      
    }	     
  }
  return 0;
}

Local Access Points

Local Access Points (LAPs) are a low overhead mechanism for calling service routines in supervisor actors on the local site by both user and supervisor actor calls.

The following is an example of the use of local access points. Refer to the lapInvoke(2K),svLapBind(2K), svLapCreate(2K), and svLapDelete(2K) man pages.

Example 8-2 Creating and Invoking LAPs 

(file: progov/lap.c)

#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <chorus.h>
#include <lap/chLap.h>
#include <am/afexec.h>
#include <exec/chExec.h>

AcParam param;

KnCap            actorCap;  
KnLapDesc        lapDesc;
char*            lapArgument;
KnTimeVal        timeval;

    void
lapHandler(char* message, char* cookie)
{
    int          res;

    res = actorSelf(&actorCap);
    if (res != K_OK) {                                                
        printf("actorSelf failed, returns %d\n",res);                 
        exit(1);                                                      
    }                                                                 

    printf("LAP handler is running \n");
    printf("     thread LI = %d, actor UI = 0x%x 0x%x\n", 
           threadSelf(), actorCap.ui.uiHead, actorCap.ui.uiTail);
    printf("     Argument = %s, cookie = %s\n",message, cookie);
}


int main(int argc, char** argv, char** envp)
{
    int              res;
    KnCap            clientCap;
    KnActorPrivilege actorP;
    char*            cookie  = "Chorus";

    res = actorPrivilege(K_MYACTOR, &actorP, NULL);
    if (res != K_OK) {                                                
        printf("Cannot get actor privilege, error %d\n", res);        
        exit(1);                                                      
    }                                                                 

    if (actorP != K_SUPACTOR) {
        printf("This program must be run in supervisor mode\n");
        exit(1);
    }

    if (argc == 1) {
        printf("Must be run with one argument: LAP name\n");
        exit(1);
    }

    if (argc == 2) {
            /*
             * This is the Server actor.
             *   connect a LAP handler,
             *   bind a symbolic name to this LAP (name given as argv[1])
             *   spawn a client actor (give the LAP symbolic name in argument)
             *   wait one minute for Lap invocation
             */

            /*
             * Spawn the client actor
             */
        param.acFlags = AFX_SUPERVISOR_SPACE;
        res = afexeclp(argv[0], &clientCap, &param , argv[0], argv[1],
                       "ARGH", NULL);
        if (res == -1) {                                              
            printf("Cannot spawn client actor, error %d\n", errno);   
            exit(1);                                                  
        }                                                             

            /*
             * Create the LAP
             */
        res = svLapCreate(K_MYACTOR, (KnLapHdl) lapHandler,
                          cookie, K_LAP_SETJMP, &lapDesc);
        if (res != K_OK) {                                            
            printf("svLapCreate failed, returns %d\n",res);           
            exit(1);                                                  
        }                                                             

            /*
             * Bind a symbolic name
             */
        res = svLapBind(&lapDesc, argv[1], 0);
        if (res != K_OK) {                                            
            printf("svLapBind failed, returns %d\n",res);             
            exit(1);                                                  
        }                                                             

            /*
             * Wait one minute
             * Other client actors can be run from the console:
             *     rsh target arun lap.r lap-name lap-argument
             * 
             */
        timeval.tmSec = 60;
        timeval.tmNSec = 0;
        threadDelay(&timeval);

            /*
             * Unbind the LAP name and Delete the LAP
             */

        res = svLapUnbind(argv[1]);
        if (res != K_OK) {                                            
            printf("svLapUnBind failed, returns %d\n",res);           
            exit(1);                                                  
        }                                                             

        res = svLapDelete(&lapDesc); 
        if (res != K_OK) {                                            
            printf("svLapDelete failed, returns %d\n",res);           
            exit(1);                                                  
        }
        printf("Server actor is leaving ...\n"); 

    } else {
        
            /*
             * This is the Client Actor:
             * argv[1] is the LAP name, argv[2] is the LAP argument.
             * get the LAP descriptor and invoke the LAP handler.
             */
        
        res = actorSelf(&actorCap);
        if (res != K_OK) {                                            
            printf("actorSelf failed ! Return code = %d\n",res);       
            exit(1);                                                  
        }                                                             
        
        printf("Client actor is running, thread li = %d, "
               "actor UI = 0x%x 0x%x\n",
               threadSelf(), actorCap.ui.uiHead, actorCap.ui.uiTail);
        
            /*
             * Get the LAP descriptor knowing its name
             */
        res = lapResolve(&lapDesc, argv[1], 0);
        if (res != K_OK) {                                            
            printf("lapResolve failed, returns %d\n", res);           
            exit(1);                                                  
        }                                                             
        
            /* 
             * Invoke the LAP handler
             */
        res = lapInvoke(&lapDesc, argv[2]);
        if (res != K_OK) {                                            
            printf("lapInvoke failed, returns %d\n", res);            
            exit(1);                                                  
        }                                                             
        
        printf("Client actor is leaving ...\n");
    }
    return 0;
}

IPC

IPC is a set of programming interfaces that allow you to create and manage individual program processes that can run concurrently in an operating system. This allows a program to handle many user requests at the same time. Since a single user request may result in multiple processes running in the operating system on the user's behalf, the processes need to communicate with other users. The IPC interfaces make this possible. Each IPC method has its own advantages and limitations so it is not unusual for a single program to use all of the IPC methods. IPC methods include:

The following is an example of the use of the IPC mechanisms provided in the ChorusOS operating system.

Refer to the grpAllocate(2K), grpPortInsert(2K), ipcReceive(2K), ipcSend(2K), ipcTarget(2K), portCreate(2K), and portDelete(2K) man pages.

Example 8-3 Communicating Using IPC

(file: progov/ipcSend.c)

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <chorus.h>
#include <am/afexec.h>
#include "ipc/chIpc.h"

#define ABORT_DELAY 1000               /* Delay for ipcReceive       */

static KnUniqueId      thePortUi;      /* Our port unique identifier */
static int             thePortLi;      /* .......  local identifier  */
static KnCap           groupCap;       /* Our group capability       */
static KnCap           clientCap;      /* Capability of the spawned  */
                                       /* actor                      */

/* The outgoing annex and body  */
static char sndAnnex[] = "Hello world from Chorus ...\n";
static char sndBody[]  = "The sea is calm, the tide is full ...\n";

/* The received annex and body  */
static char rcvAnnex[K_CMSGANNEXSIZE];
static char rcvBody[1000];

/* Parameter for actor spawning */
AcParam param;

/* Port group stamp             */
char*       stamp;
#define MYSTAMP   100
#define STAMPSIZE  10

int main(int argc, char** argv, char** envp)
{
    int         rslt;           /* Work */
    KnMsgDesc   smsg;           /* Descriptor for message being sent */
    KnMsgDesc   rmsg;           /* Descriptor for message being received */
    KnIpcDest   ipcDest;        /* IPC address */

    if (argc == 1) {
            /* 
             * Server actor
             * Create the destination port
             */
        thePortLi = portCreate(K_MYACTOR, &thePortUi);
        if (thePortLi < 0) {                                          
            printf("portCreate failed, returns %d\n", thePortLi);     
            exit(1);                                                  
        }                                                             
            /*
             * Allocate a port group and insert the port into it
             */
        rslt = grpAllocate(K_STATUSER, &groupCap, MYSTAMP);
        if (rslt < 0) {                                               
            printf("grpAllocate failed, returns %d\n", rslt);         
            exit(1);                                                  
        }                                                             
   
        rslt = grpPortInsert(&groupCap, &thePortUi);
        if (rslt < 0) {                                               
            printf("grpPortInsert failed, returns %d\n", rslt);       
            exit(1);                                                  
        }
            /*
             * Spawn the client actor
             * The group stamp is given in argument
             */
        stamp = malloc(STAMPSIZE);
        sprintf(stamp, "%d", MYSTAMP);

        rslt = afexeclp(argv[0], &clientCap, NULL , argv[0], stamp, NULL);
        if (rslt == -1) {                                             
            printf("Cannot spawn client actor, error %d\n", errno);   
            exit(1);                                                  
        }                                                             
            /*
             * Receive the message
             */
        rmsg.flags     = 0;
        rmsg.bodySize  = sizeof(rcvBody);
        rmsg.bodyAddr  = (VmAddr)rcvBody;
        rmsg.annexAddr = (VmAddr)rcvAnnex;

        rslt = ipcReceive(&rmsg, &thePortLi, ABORT_DELAY);
        if (rslt < 0) {                                               
            printf("ipcReceive failed, returns %d\n", rslt);          
            exit(1);                                                  
        }                                                             

        printf ("%s\n%s\n", rcvAnnex, rcvBody);
    
        rslt = portDelete(K_MYACTOR, thePortLi);
        if (rslt < 0) {                                               
            printf("portDelete failed, returns %d\n", rslt);          
            exit(1);                                                  
        }                                                             
    } else {
            /*
             * Get the port group capability giving the stamp.
             * Stamp has been received in argv[1]
             */
        rslt = grpAllocate(K_STATUSER, &groupCap, (int) atoi(argv[1]));
        if (rslt < 0) {                                               
            printf("grpAllocate failed, returns %d\n", rslt);         
            exit(1);                                                  
        }                                                             
            /*
             * Prepare the message descriptor for the message to send
             */
        smsg.flags     = 0;
        smsg.bodySize  = sizeof(sndBody);
        smsg.bodyAddr  = (VmAddr)sndBody;
        smsg.annexAddr = (VmAddr)sndAnnex;
            /* 
             * Prepare the IPC address for the message destination.
             * Send the message in broadcast mode.
             */
        ipcDest.target = groupCap.ui;
        rslt = ipcTarget(&ipcDest.target, K_BROADMODE);
        if (rslt < 0) {                                               
            printf("ipcTarget failed, returns %d\n", rslt);           
            exit(1);                                                  
        }                                                             

            /* Send from our DEFAULT port */
        rslt = ipcSend(&smsg, K_DEFAULTPORT, &ipcDest);
        if (rslt < 0) {                                               
            printf("ipcSend failed, returns %d\n", rslt);             
            exit(1);                                                  
        }                                                             
    }

    return 0;
}

Chapter 9 Time Management

The ChorusOS operating system offers five time management services:

The configuration of your ChorusOS operating system determines which services are available.

Time Management Services

The following time management services are available:

Table 9-1 shows which services are available for a given configuration:

Table 9-1 Time Management Service Availability
 Service Availability
  tick  always available
  date configured with DATE
  time-out  always available
  timer configured with TIMER
  virtual time configured with VTIMER

Current Time

An actor, whether user or supervisor, may get the time elapsed since the last reboot through the following system call: 

#include <exec/chTime.h> 

int sysTime(KnTimeVal* time);

This will fill in the time data structure which is built from two fields:

The resolution of the value depends on the platform on which the system is running, and may be obtained by a call to:

#include <chorus.h>

int sysTimeGetRes(KnTimeVal* resolution);

The time value returned at the location defined by the resolution parameter represents the smallest possible difference between two distinct values of the system time.

Timers

This feature provides timer services for both user and supervisor actors. One-shot as well as periodic timers are provided. Time-out notification is achieved through user-provided handler threads which are woken up in the application actor.

The timer facility uses the concept of a timer object within the actor. These timer objects may be created, deleted and set dynamically. Once created, they are addressed by local identifiers within the context of the actor and are deleted automatically when the actor terminates.

The application is expected to create one or more threads dedicated to timer notification handling, by declaring themselves ready to handle these types of events. The relationship between a timer object and a thread (or a set of threads) is established through a threadPool object which is used to block threads waiting for the expiration of a timer.

Thus, the basic mechanism for dealing with timers is:

  1. Allocate and initialize a threadPool object.

  2. Create one thread which will block on the threadPool object.

  3. Create a timer associated with the above threadPool object.

  4. Set the timer (effectively arm it).

The second and third steps may take place in any order. When timer expiration occurs, the dedicated thread will be unblocked so that it may now perform any operation which should be done due to timer expiration. For example, it may print a warning message, re-arm the timer (unless it was a periodic timer), and block itself again. As usual with the ChorusOS operating system data structures, these threadPool objects must be pre-allocated by the application.

A threadPool object is initialized as follows:

#include <etimer/chEtimer.h>

int timerThreadPoolInit(KnThreadPool* threadPool);

A timer may then be created as follows: 

#include <etimer/chEtimer.h>

int timerCreate(KnCap*        actorCap,
                int           clockType,
                KnThreadPool* threadPool,
                void*         cookie,
                int*          timerLi);

This creates a timer object in the actor defined by the actorCap parameter. Applications will usually use K_MYACTOR. When the timer is armed and reaches expiration, one of the threads blocked on the threadPool object will be selected and awakened. This thread will be passed the cookie parameter of the timerCreate() call. When successful, timerCreate() returns the local identifier of the created timer at the location defined by the timerLi parameter. The only clock type currently supported is K_CLOCK_REALTIME, and corresponds to the time returned by sysTime().

A thread may block itself on a threadPool object through the following call: 

#include <etimer/chEtimer.h>

int timerThreadPoolWait(KnThreadPool* threadPool,
                        void**        cookie,
                        int*          overrun,
                        KnTimeVal*    waitLimit);

The threadPool object must have been previously initialized. timerThreadPoolWait() blocks the invoking thread until a timer associated with threadPool expires or until the waitLimit condition is reached. Upon timer expiration, the thread will return from this call, and the cookie field will have been updated with the value associated with the timer.

The overrun counter is used to indicate to the thread that either the time-out notification has been delayed (in this case the overrun value is 1) or that a number of time-out notifications have been lost (in this case the overrun value is strictly greater than 1).

A timer may be armed with: 

#include <etimer/chEtimer.h>

int timerSet(KnCap*    actorCap,
             int       timerLi,
             int       flag,
             KnITimer* new,
             KnITimer* old);

This call arms the timer defined by the first two parameters where timerLi is the timer identifier as returned by timerCreate(). timerSet() allows the specification of the time-out using either a relative or an absolute time using the flag parameter. The time-out is specified using the new parameter which is a structure containing the following fields:

If the old parameter is non-null, the time remaining before timer expiration is returned at the location defined by old. If new is non-null and the timer has already been set, the current setting is cancelled and replaced with the new one. If the new time specified is 0, the current setting will simply be cancelled. If new is set to null, the current setting specification is left unchanged.

Refer to the timerThreadPoolInit(2K), timerCreate(2K), timerSet(2K), timerThreadPoolWait(2K), and sysTime(2K) man pages.

The following example illustrates the use of timer services for both user and supervisor actors.

Example 9-1 Using Timers

(file: progov/timers.c)

#include <stdio.h>
#include <stdlib.h>
#include <chorus.h>
#include <etimer/chEtimer.h>

KnThreadPool samplePool;
int          periodic;
int          oneShot;
int          periodicLid;
int          oneShotLid;

#define USER_STACK_SIZE (1024 * sizeof(long))			      
								      
KnSem   sampleSem; /* Semaphore allocated as global variable */

    int
childCreate(KnPc entry)
{
  KnActorPrivilege      actorP;
  KnDefaultStartInfo_f  startInfo;
  char*                 userStack;
  int                   childLid = -1;
  int                   res;

  startInfo.dsType            = K_DEFAULT_START_INFO;
  startInfo.dsSystemStackSize = K_DEFAULT_STACK_SIZE;

  res = actorPrivilege(K_MYACTOR, &actorP, NULL);
  if (res != K_OK) {
    printf("Cannot get the privilege of the actor, error %d\n", res);
    exit(1);
  }

  if (actorP == K_SUPACTOR) {
    startInfo.dsPrivilege = K_SUPTHREAD;
  } else {
    startInfo.dsPrivilege = K_USERTHREAD;
  }

  if (actorP != K_SUPACTOR) {
    userStack = malloc(USER_STACK_SIZE);
    if (userStack == NULL) {
      printf("Cannot allocate user stack\n");
      exit(1);
    }
    startInfo.dsUserStackPointer = userStack + USER_STACK_SIZE;
  } 

  startInfo.dsEntry = entry;

  res = threadCreate(K_MYACTOR, &childLid, K_ACTIVE, 0, &startInfo);
  if (res != K_OK) {
    printf("Cannot create the thread, error %d\n", res);
    exit(1);
  }

  return childLid;
}

    void
timerWait(int myThLi)
{
  /* do nothing */
}

    void
sampleThread()
{
  int      myThLi;
  int      res;
  void*    cookie;
  int      overrun;
  KnITimer periodicTimer;
  KnTimeVal tv;

  myThLi = threadSelf();
  printf("Thread %d started\n", myThLi);

  for(;;) {
      res = timerThreadPoolWait(&samplePool, &cookie, &overrun, K_NOTIMEOUT);
      if (res != K_OK) {					      
          printf("Cannot wait on thread pool, error %d\n", res);      
          exit(1);						      
      }								      
      if (overrun != 0) {
          printf("Thread %d. We were late! overrun set to : %d\n", 
                  myThLi, overrun);
      }
      if (cookie == &periodic) {
         printf("Thread %d. Time is flying away!\n", myThLi);
      } else if (cookie == &oneShot) {
         printf("Thread %d. Isn't it time to go home?\n", myThLi);
         periodicTimer.ITmValue.tmSec   = 0; /* seconds */
         periodicTimer.ITmValue.tmNSec  = 0; /* nanoseconds */
         periodicTimer.ITmReload.tmSec  = 0; /* seconds */
         periodicTimer.ITmReload.tmNSec = 0; /* nanoseconds */
         res = timerSet(K_MYACTOR, periodicLid, NULL, &periodicTimer, NULL);
         if (res != K_OK) {					      
            printf("Cannot cancel periodic timer, error %d\n", res);  
            exit(1);						      
         }							      
         /* 
          * Periodic timer is cancelled 
          * Get current time,
          * Wait for a short while (3 seconds) and quit
          */
         res = sysTime(&tv);
         if (res != K_OK) {					      
            printf("Cannot get system time, error %d\n", res);	      
            exit(1);						      
         }							      
         printf("Current system time is %d seconds\n", tv.tmSec);
         printf("No more periodic messages should be printed now!\n");
         K_MILLI_TO_TIMEVAL(&tv, 3000);
         threadDelay(&tv);
         /* We are all done ! */
         exit(0);
      } else {
         printf("Spurious timer!\n");
      }
  } /* for() */
}

int main(int argc, char** argv, char** envp) 
{
   int       res;
   KnTimeVal tv;
   int       thLi1;
   int       thLi2;
   KnITimer  periodicTimer;
   KnITimer  oneShotTimer;
	
   res = timerThreadPoolInit(&samplePool);
   if (res != K_OK) {						      
      printf("Cannot initialize thread pool, error %d\n", res);	      
      exit(1);							      
   }								      

   res = timerCreate(K_MYACTOR, K_CLOCK_REALTIME, &samplePool,
			   &periodic, &periodicLid);
   if (res != K_OK) {						      
      printf("Cannot create periodic timer, error %d\n", res);	      
      exit(1);							      
   }								      

   res = timerCreate(K_MYACTOR, K_CLOCK_REALTIME, &samplePool,
			   &oneShot, &oneShotLid);
   if (res != K_OK) {						      
      printf("Cannot create one shot timer, error %d\n", res);	      
      exit(1);							      
   }								      

   thLi1 = childCreate((KnPc)sampleThread);
   thLi2 = childCreate((KnPc)sampleThread);
      
   res = sysTime(&tv);
   if (res != K_OK) {						      
      printf("Cannot get system time, error %d\n", res);	      
      exit(1);							      
   }								      
   printf("Current system time is %d seconds\n", tv.tmSec);

   periodicTimer.ITmValue.tmSec   = 1; /* seconds */
   periodicTimer.ITmValue.tmNSec  = 0; /* nanoseconds */
   periodicTimer.ITmReload.tmSec  = 1; /* seconds */
   periodicTimer.ITmReload.tmNSec = 0; /* nanoseconds */
   res = timerSet(K_MYACTOR, periodicLid, NULL, &periodicTimer, NULL);
   if (res != K_OK) {						      
      printf("Cannot arm periodic timer, error %d\n", res);	      
      exit(1);							      
   }								      
   
   oneShotTimer.ITmValue.tmSec   = tv.tmSec + 30; /* seconds */
   oneShotTimer.ITmValue.tmNSec  = 0; /* nanoseconds */
   oneShotTimer.ITmReload.tmSec  = 0; /* seconds */
   oneShotTimer.ITmReload.tmNSec = 0; /* nanoseconds */

   res = timerSet(K_MYACTOR, oneShotLid, K_TIMER_ABSOLUTE, 
	          &oneShotTimer, NULL);
   if (res != K_OK) {						      
      printf("Cannot arm one shot timer, error %d\n", res);	      
      exit(1);							      
   }								      

   res = threadDelete(K_MYACTOR, K_MYSELF);
   if (res != K_OK) {						      
      printf("Cannot suicide myself, error %d\n", res);		      
      exit(1);							      
   }								      

   return 0;
}