C H A P T E R 4 |
Configuration Information and Guidelines |
This chapter provides configuration information for both OpenSolaris or Solaris servers and Linux based Sun Shared Visualization 1.1 servers. Topics include:
To configure a client system, see the Sun Shared Visualization 1.1 Software Client Administration Guide.
Shared Visualization 1.1 dramatically eases graphics host configuration, compared to earlier releases. In this release, you take the following steps to configure a host with a graphics accelerator to be a graphics server:
1. Decide which options you will use to configure VirtualGL.
2. Enable X11 forwarding for ssh.
3. Shut down the window system.
4. Invoke vglserver_config, then select a configuration option and answer questions presented for that option.
6. Verify the host as a graphics server for VirtualGL.
7. If this host will also be a Sun Grid Engine execution host:
a. Configure Sun Grid Engine’s graphics resources.
b. Verify the host as a graphics execution host.
Tip - If you plan to use VirtualGL only with GLP, then you can skip this section. |
VirtualGL requires access to the application server’s 3D graphics accelerator so that VirtualGL can create off-screen pixel buffers (Pbuffers) and redirect the 3D rendering from applications into these Pbuffers. Accessing a 3D graphics accelerator on a Linux system, an OpenSolaris or Solaris x86 system, or a Solaris SPARC system without GLP requires going through an X server. On such systems, the only way to share the application server’s 3D graphics accelerator among multiple users is to grant those users access to the “3D X server” (the X server attached to the application server’s 3D graphics accelerator). Refer to the figures in Chapter 1.
Be aware that some security risks are associated with this sharing. Once a user has access to the 3D X server, nothing prevents that user from logging keystrokes or reading back images from the X display. Using xauth, one can obtain untrusted X authentication keys that prevent this behavior, but unfortunately, those untrusted keys also disallow access to the 3D hardware. Therefore, you must grant full trusted X access to any users needing to run VirtualGL. Unless you fully trust the users to whom you are granting this access, you should avoid logging in locally to the 3D X server, particularly as root, unless absolutely necessary.
The server’s SSH® daemon (ssh) should have the X11Forwarding option enabled. You configure this option in sshd_config. TABLE 4-1 lists the location of sshd_config, depending on your distribution of ssh.
When X11Forwarding is enabled, its line in sshd_config is:
The UseLogin option of SSH is incompatible, so that option must not be enabled in sshd_config. You can specifically disable UseLogin or simply not mention it in sshd_config.
Before running vglserver_config to configure a Solaris SPARC graphics server, decide which of the following options you will choose:
An OpenSolaris or Solaris x86 or x64 system can use only GLX.
When a user logs into a Solaris host locally, the system automatically assigns ownership of the framebuffer devices to that user and sets the permissions for those devices to those specified in /etc/logindevperm. The default permissions disallow anyone from using the graphics accelerator devices except the user that is logged in.
In order to run VirtualGL, however, a user must have write access to the graphics accelerator devices as a shared resource. vglserver_config disables the login device permissions mechanism for the graphics accelerator devices. This command also sets the owner and group for these devices such that any member of the vglusers group (or optionally, all users) can write to the devices.
A Solaris SPARC graphics server with Sun OpenGL 1.5 and an XVR-2500, XVR-1200, or XVR-600 graphics accelerator can be configured to use those devices through GLP without having to start a “3D X server” on the graphics accelerators. (See GLP Access on Solaris SPARC Servers for more details.) If you have a graphics server with those characteristics, you can use vglserver_config to configure the server to use GLP mode exclusively.
Note - Sun OpenGL 1.5 is available at: http://www.sun.com/software/graphics/opengl |
Unless absolutely needed, disable the XTEST extension. (For example, XTEST is disabled by Step 6 of To Configure an OpenSolaris or Solaris Server to Grant Access to the 3D X Server.)
To Configure a Solaris SPARC Server to Use VirtualGL With GLP and Without a 3D X Server |
Use this procedure if you determined in Using GLP on Solaris SPARC Servers that you will use GLP on your Solaris system.
A list of options is displayed.
3. Reply y or n to this question:
4. If you answered y in Step 3, edit the vglusers group in /etc/group.
If framebuffer device access will be restricted to the vglusers group, edit that entry to include root and any additional users. Any users that you add to vglusers at this time must log out and back in again before their new group permissions take effect.
5. Edit the /etc/dt/config/GraphicsDevices file as necessary.
This file contains a list of paths to 3D framebuffer devices that you want to use with VirtualGL. Each path is on a separate line. For example:
6. Verify that the system is ready to run VirtualGL.
b. Log back in to the server using SSH.
c. Execute the following command in the SSH session:
This command should output a list of visuals and complete with no errors.
7. If you want VirtualGL to use GLP by default, add these lines to /etc/profile:
These lines cause VirtualGL to use the first device specified in /etc/dt/config/GraphicsDevices as the default rendering device. A user can override this default by setting VGL_DISPLAY in a startup script (such as ~/.profile or ~/.login) or by passing vglrun an argument of -d device when invoking VirtualGL.
To Configure an OpenSolaris or Solaris Server to Grant Access to the 3D X Server |
Use this procedure if you determined in Using GLP on Solaris SPARC Servers that you will not use GLP on your Solaris SPARC system, or if your OpenSolaris or Solaris system does not have a SPARC processor. This procedure configures a VirtualGL server so that selected users can run VirtualGL, even if the server is currently at the login prompt.
1. Shut down the display manager.
2. Log in as root from the text console (or remotely using ssh) and enter:
A list of options is displayed.
4. Reply y or n to this question:
5. Reply y or n to this question:
6. Reply y or n to this question:
7. If you answered y in Step 4 or Step 5, edit the vglusers group in /etc/group.
If framebuffer device access will be restricted to the vglusers group, edit that entry to include root and any additional users. Any users that you add to vglusers at this time must log out and back in again before their new group permissions take effect.
8. Restart the display manager.
9. Verify that the system is ready to run VirtualGL.
b. Log back in to the server using SSH.
c. Execute one of the following command sequences in the SSH session:
Either command should output a list of visuals and complete with no errors. If xdpyinfo fails to run, then the permissions on display :0 are too restrictive.
d. If you chose to disable the XTEST extension, check the output of xdpyinfo to verify that XTEST is not included in the list of extensions.
This section explains how to configure a VirtualGL server such that select users can run VirtualGL, even if the server is currently at the login prompt.
To Grant VirtualGL Access to the 3D X server on a Linux Server |
1. If the server is configured to boot into run level 5 (graphical login), temporarily shut down the X server as root. Type:
2. Otherwise, log in as root from the text console.
A list of options is displayed.
5. Reply y or n to this question:
6. Reply y or n to this question:
7. Reply y or n to this question:
8. If you answered y in Step 5 or Step 6, edit the vglusers group in /etc/group.
If framebuffer device access will be restricted to the vglusers group, edit that entry to include root and any additional users. Any users that you add to vglusers at this time must log out and back in again before their new group permissions take effect.
9. As the root user, restart the X server. Type:
10. Verify that the system is ready to run VirtualGL.
b. Log back in to the server using SSH.
c. Execute one of the following command sequences in the SSH session:
Either command should output a list of visuals and complete with no errors. If xdpyinfo fails to run, then the permissions on display :0 are too restrictive.
d. If you chose to disable the XTEST extension, check the output of xdpyinfo to verify that XTEST is not included in the list of extensions.
This method calls vglgenkey from the display manager’s startup script. vglgenkey calls xauth to generate an authorization key for the 3D X server and stores this key under /etc/opt/VirtualGL. The VirtualGL launcher script (vglrun) then attempts to read this key and merge the key into the user’s .Xauthority file, granting the user access to the 3D X server. Using this method, you can control who has access to the 3D X server by controlling who has read access to the /etc/opt/VirtualGL directory.
This section describes how to add graphics resources to Sun Grid Engine. You must first install Sun Grid Engine and the Sun Shared Visualization 1.1 software before continuing with this procedure.
These steps are to be performed as the sgeadmin user on the queue master host, or on an administrative host that mounts $SGE_ROOT read-write.
To Set the Variables |
1. Set $SGE_ROOT and set PATH to include Sun Grid Engine directories:
where /gridware is the base directory of your $SGE_ROOT.
2. Assure that your DISPLAY environment variable is set and refers to the system whose X server keyboard you are using:
where myhost is the hostname of the X server, and :0.0 identifies the X screen and display.
If you access the server using ssh -X, ssh sets DISPLAY for you. However, the ssh tunnel is available only on that server host, not on all execution hosts in the grid.
To Add Graphics to Sun Grid Engine |
Note - If you are upgrading an existing Sun Shared Visualization 1.1 software installation, you only need to perform Step 1, Step 8, and Step 9. |
1. If the optional software was not already installed on the grid’s NFS server, then, as superuser, install that software.
Note - Ensure that your $SGE_ROOT value is your answer to the installation prompt, “Please enter your SGE_ROOT directory.” |
2. Set an administrative email for Sun Grid Engine so that all errors are reported through email.
This command starts your $EDITOR with a file containing configuration variables.
b. Add the email address for the administrator_mail configuration variable, then save and quit the file.
3. Add resource names to the Sun Grid Engine complex.
The complex is the vocabulary of variables that can be specified. Eight resources will be added in this step (in the 1.1.1 release): graphics, dedicated_graphics, graphics_alone, maximum_graphics, headnode, chromium, sc_cols, and sc_rows. (Some of those resources are used by Sun Scalable Visualization software.) TABLE 4-2 describes the four resources most important for Sun Shared Visualization.
Further details on some of these resources are provided in More Graphics Resource Allocation Information.
Add resource names by running the add_to_complex script. As a Sun Grid Engine administrator (sgeadmin), type:
The script adds the information in TABLE 4-2 to your Sun Grid Engine complex. The script reports if a resource already exists or is added. When the script is finished, it will prompt you to perform the next step in this procedure.
4. Define which hosts have how many graphics resources available.
This step determines the maximum number of simultaneous graphics jobs that Sun Grid Engine could start on that host. For example, if your host has two graphics boards and the boards can accommodate three jobs each, your resources would be 2 x 3, for a total of 6.
The easiest way to specify graphics resources is by using the config_gfx script.
a. As sgeadmin or root on each graphics execution host, type:
By default, the graphics server host is the host on which config_gfx is invoked. The default name of the queue to be configured is all.q. To use different values for either of these systems, type config_gfx with the -h (host) or -q (queue) option:
b. Respond to questions asked by the config_gfx script.
Your answers will determine values for some of the resources that were added in Step 3.
For GLP access to Solaris SPARC graphics devices, the count is of unique graphics boards, but for a host using GLX access, the count is the number of X screens. If a host is using Xinerama across multiple graphics devices, SGE cannot allocate those devices separately. See GLX for more information about Xinerama and about configuring the list of these unique graphics devices or X displays.
The number of graphics jobs to allow simultaneously is of key importance. If this number is too high, graphics accelerator memory could be exhausted and performance will suffer greatly (for example, when applications page or swap their data onto the accelerator). If the number is too low, applications will need to wait until another application exits, even if an application is not actively using the graphics accelerator (for example, when the user of that application is away from the desk).
A good starting point is two or three applications per graphics accelerator. Allow more simultaneous jobs if the load is typically light users and data sets. Limit the number of simultaneous jobs if there are heavy users and data sets.
The following example of running config_gfx is for a Sun X4600 M2 server with two Nvidia QuadroPlex model 4s (a total of four graphics devices), which are not expected to run Chromium:
5. Set the starter and epilog scripts for Sun Grid Engine’s all.q cluster queue.
These scripts are hooks supported by Sun Grid Engine to provide queue-specific activity before and after a job runs.
This action sets the starter and epilog scripts for all Sun Grid Engine queues.
6. (Optional) Copy the graphics/docs/README file to a more user accessible location.
Tip - The contents of the README file summarize Sun Grid Engine use. Edit the file to better describe your particular site, rename the file, and make the file available to users in $SGE_ROOT. |
7. Ensure that your DISPLAY environment variable refers to your 2D X server:
where myhost is the hostname of the X server.
8. Attempt to run a graphics job.
This example submits to any Shared Visualization 1.1 graphics server:
The following example names a graphics execution host:
% qrsh -b n -q all.q@hostname /opt/VirtualGL/bin/vglrun -c proxy -spoil \ $SGE_ROOT/graphics/RUN.glxspheres |
9. Start $SGE_ROOT/graphics/empty_jobs from /etc/init.d/sgeexecd.
/etc/init.d/sgeexecd is the Sun Grid Engine standard startup script. This script initiates shepherd processes. If these processes are shut down before the graphics jobs, you cannot reclaim the resources of those graphics jobs. To alleviate any possibility of this problem:
a. Edit the /etc/init.d/sgeexecd file and around line 245, find $bin_dir/sge_execd.
b. Insert the following text before that line:
Replace sgeadmin in this line if your site uses a different SGE administrative login.
Note - You need to repeat this step if the Sun Grid Engine software is upgraded. |
Note - The steps referenced in this section pertain to the procedure To Add Graphics to Sun Grid Engine |
You can control which graphics devices are used by Sun Grid Engine by creating or editing a world-readable local graphics configuration file /etc/dt/config/GraphicsDevices on any execution host. If the GraphicsDevices file is not present, only X server :0.0 is used by GLX.
To Create a GraphicsDevices File |
This procedures also creates a directory for the GraphicsDevices file.
As superuser, type these commands:
# mkdir -p /etc/dt/config # touch /etc/dt/config/GraphicsDevices # chmod 644 /etc/dt/config/GraphicsDevices |
On a SPARC Solaris graphics server, kfb (XVR-2500) devices and jfb (XVR-1200 and XVR-600) devices can be used by VirtualGL through GLP. There is no need for a 3D X server on the devices. See To Configure a Solaris SPARC Server to Use VirtualGL With GLP and Without a 3D X Server. On such a host, each line of the graphics server’s configuration file can be a device name followed by an optional maximum number of concurrent jobs for that device. (If no number is added, the default is that the device is used by only one job at a time.) For example:
In this example, the host’s jfb0 device can support only one Sun Grid Engine graphics job. The jfb1 device can only support up to two Sun Grid Engine graphics jobs. The jfb2 device is not used for any Sun Grid Engine graphics jobs. (This device might be used by a local console user.)
Any UNIX host can be configured to start a “3D X server” on each graphics device. The vglgenkey technique of vglserver_config (described in Configuration Information for Linux Servers) will enable access to that display for VirtualGL users. If there are several graphics accelerators, the local configuration file can control allocation of jobs to these X displays or screens (for example, an x86 host with two devices used by screens :0.0 and :0.1).
Xinerama is an extension to the X Window System that enables multiheaded X. When X is configured to use Xinerama, X can provide a user with one large virtual screen spread across two or more physical displays (also called heads). This configuration enables any application’s window to be moved from one physical display to another, or for one window to span multiple displays. A similar effect can be produced, without using Xinerama, by a single graphics accelerator that is able to offer a single desktop across multiple monitor outputs. Use Xinerama with Sun Scalable Visualization Software on a host with multiple graphics accelerators that drive any portion of a group of displays (also called a power wall).
When multiple graphics devices are used with Xinerama, X provides only a single large, virtual screen, which is typically :0.0. Therefore, applications do not generally control the head on which a window or dialog will appear. VirtualGL uses only this single virtual X screen to provide remote users with GLX access to the graphics accelerator devices that drive the multiple heads. All remote visualization users share the first graphics accelerator, which causes resources on subsequent graphics devices to be underutilized.
Without Xinerama, multiple graphics accelerators can be used as independent devices. In this case, the X desktop on each device is an independent desktop, but all desktops share the mouse and keyboard. Application windows started on one device must remain on that device. Those windows cannot be moved to a different device, nor can the windows span across multiple devices.
Because X treats each graphics device as a separate screen (typically, :0.0 and :0.1), VirtualGL can use any of these X screens to provide remote users with GLX access to a graphics accelerator device. For remote visualization, you normally want to configure a single 3D X server to use all the graphics devices without Xinerama. Then configure Sun Grid Engine to allocate all graphics accelerators to remote visualization jobs, a configuration that provides load balancing.
Consequently, the host’s /etc/dt/config/GraphicsDevices local configuration file might be:
The number of graphics resources for each execution host (Step 4 in To Add Graphics to Sun Grid Engine) is the maximum number of concurrent graphics jobs Sun Grid Engine schedules for that host. The total of the maximum number of jobs on all graphics devices in the local configuration file should be no less.
Similarly, the execution host should have at least as many total slots as the total number of maximum jobs for concurrent jobs, if you want the execution host to allow that many concurrent graphics jobs. The Sun Grid Engine default for slots is the number of CPUs (cores) in the system when Sun Grid Engine’s install_execd (Sun Grid Engine execution host configuration) script is run.
The configuration files are used by the graphics allocation script, $SGE_ROOT/graphics/alloc (or alloc.debug). You can study the script and comments, to adjust the script to suit your needs. Make a copy of your changes, so that your customizations to the script can be reintegrated in the event of a software upgrade.
A user might demand a certain number of graphics boards for a job. This is a hard limit. If not possible, the job does not run. The hard limit is specified (for example, Step 8 of To Add Graphics to Sun Grid Engine), with -l gfx=value.
A user might also request a desired (maximum) number of graphics boards, which Sun Grid Engine calls a soft limit. In this situation, a queued job waits for a time when more resources are available. An interactive job is immediate and fails. A soft limit needs the -soft introduction, and also must use a different resource, maximum_graphics (shorthand: maxgfx). This situation is due to Sun Grid Engine restrictions.
A user can start qsub or qrsh specifying both hard quantities of necessary resources and soft quantities of desired resources. For example:
In this example, the job requests four graphics devices, but demands at least one. If two devices are assigned, the VGL_DISPLAY value in the environment of the job contains two words. Each word could be a graphics device name (on a SPARC Solaris host using GLP, such as /dev/fbs/kfb0;) or an X display name (such as :0.0 or :0.1).
VirtualGL itself only uses the first device (or display) in the VGL_DISPLAY environment variable value. Allocating multiple devices is of value only if the job divides work among multiple processes, using one device or display value for each process.
The following tables provide more information about the Sun Grid Engine integers and environment variables that control allocation of graphics resources:
The graphics allocation called by the starter script on a graphics server host attempts logging of which users use how many graphics devices at what start and finish times, using the system logger. By default, these messages are local0.info messages and are discarded by the Solaris logger.
Follow this procedure to configure system logging to save the logging messages in /var/adm/messages.
1. Review the syslog.conf(4) man page.
2. Open the /etc/syslog.conf file in an editor.
3. Search for the /var/adm/messages line following mail.crit.
4. Add ;local0.info to that line.
When a job uses gfx=1 (or more) resources, Sun Grid Engine allocates one or more graphics accelerators. Sun Grid Engine also sets the graphics accelerator’s device name into the VGL_DISPLAY environment variable used by vglrun.
Such a job can interpose on an application by starting the job with vglrun or /opt/VirtualGL/bin/vglrun if not in the users’ $PATH.
If qsub starts vglrun directly, qsub requires the full path. For example:
vglrun can also start an executable. For example:
Also, a Sun Grid Engine job script can start vglrun when ready to run the application. The following example job script starts /opt/VirtualGL/bin/glxspheres on a Solaris or Linux graphics server. This script is a simplified version of $SGE_ROOT/graphics/RUN.glxspheres. Italicized text in this listing provides commentary, but is not part of the job script itself.
VirtualGL cannot use multiple graphics accelerators, so you cannot specify gfx any greater than 1, nor even configure maxgfx. To do so would consume resources without a performance improvement. You might want to specify gfx greater than 1 when your job requires several graphics accelerators concurrently, yet separately.
You might want Sun Grid Engine to allocate a graphics accelerator and start a TurboVNC server session to use the graphics accelerator. For example:
When a shell in the TurboVNC session is ready to start an application, start vglrun from within a terminal on the TurboVNC session. For example:
By writing a specialized script, Sun Grid Engine resources and VGL_ environment variables can be set at run time. You can see this situation in the $SGE_ROOT/graphics/RUN.vncserver script.
If you need the server to support quad-buffered stereographic display for remote VirtualGL clients, read the requirements in the "Verifying Advanced Feature Support" section of Appendix A in the Sun Shared Visualization 1.1 Software Client Administration Guide.
To Determine if a Server Has a Suitable Visual for Quad-buffered Stereographic Rendering |
1. Type one of the following on the VirtualGL server:
2. In the output, see if one or more of the visuals says stereo=1 and lists Pbuffer as one of the Drawable Types.
This output is an indicator that the server is suitable.
To Verify Client Visuals |
Now you need to determine whether the X display on the client has a suitable visual for quad-buffered stereographic rendering, transparent overlays, or Pseudocolor.
1. On the VirtualGL server, type:
2. In the output, look for the following:
You can use the vglserver_config script to restore the unshared, secure default settings that do not allow VirtualGL access. However, unconfiguring the server does not remove the vglusers group or the /etc/dt/config/GraphicsDevices file.
Both of the options described in the following procedure will restore the framebuffer device permissions to their default. The default is that the framebuffer devices can be accessed only by root or the user currently logged into the system locally.
To Unconfigure the VirtualGL Server |
1. Shut down the display manager.
2. Log in as root from the text console (or remotely using ssh) and type:
A list of options is displayed.
3. To unconfigure GLX mode, select this option:
This option removes any shared access to the 3D X server, preventing VirtualGL from accessing the 3D hardware in that manner. This option also reenables the XTEST extension on the 3D X server.
4. To unconfigure GLP mode, select this option:
5. Restart the display manager as described in Step 8, Restart the display manager..
Note - To resolve problems involving the proper use of vglconnect, refer to the Sun Shared Visualization 1.1 Software Client Administrator Guide. |
The vglclient program is normally started implicitly by vglconnect.
The vglrun script can be used to launch either binary executables or shell scripts. When you use vglrun to run a shell script, the VirtualGL faker library is preloaded into every executable that the script launches. If the script invokes any executables that are setuid or setgid, the operating system might refuse to load VirtualGL into those executables because you are attempting to preload a library (VirtualGL) that is not in a secure path. The error message depends on the operating system.
The Solaris OS allows preloading into a setuid or setgid process only if the library is in a directory recognized as containing only secure libraries.
Linux allows preloading a library into a setuid or setgid process only if the library is also in set-user-ID. Linux may also require that the a library is in a standard search directories such as /lib, /usr/lib, and /usr/lib64.
The following sections describe alternative ways to work with this restriction.
A system administrator can configure the operating system to consider the VirtualGL library to be secure Be aware of the security ramifications before you run these commands. You are essentially telling the operating systems that you trust the security and stability of the VirtualGL code as much as you trust the security and stability of the operating system. If you decide to use this alternative, the details depend on the operating system:
If the application is a script, a more secure method is to edit the application script and have it run vglrun only for the executables that you want to run in the VirtualGL environment. Here are various ways to do this:
Here is an example of using these options. The script calls a binary needing setuid that is a 32-bit executable. However, the graphics application is a 64-bit executable. In this situation, you can use vglrun -64 to launch the application script. The result is that the 32-bit setuid binary will not attempt to preload VirtualGL’s faker library.
Rather than running this with vglrun my_script, you can create a similar script (called my_vgl_script) as follows:
Invoke my_vgl_script directly (that is, do not enter vglrun my_vgl_script). The result is that this script does not attempt to preload VirtualGL into some_setuid_binary, but will preload VirtualGL into some_application_binary, as you wanted. Unfortunately, this method does not allow the user invoking my_vgl_script to provide vglrun options on the command line.
Copyright © 2008 Sun Microsystems, Inc. All Rights Reserved.