C H A P T E R 4 |
Configuration Information and Guidelines |
This chapter provides configuration information for both Solaris and Linux based Sun Shared Visualization 1.1 servers. Topics include:
Shared Visualization 1.1 dramatically eases graphics host configuration, compared to earlier releases. In this release, you take the following steps to configure a host with a graphics accelerator to be a graphics server:
1. Decide which options you will use to configure VirtualGL.
2. Enable X11 forwarding for ssh.
3. Shut down the window system.
4. Invoke vglserver_config, then select a configuration option and answer questions presented for that option.
6. Verify the host as a graphics server for VirtualGL.
7. If this host will also be a Sun Grid Engine execution host:
a. Configure Sun Grid Engine’s graphics resources.
b. Verify the host as a graphics execution host.
VirtualGL requires access to the server’s 3D graphics accelerator so that VirtualGL can create off-screen pixel buffers (Pbuffers) and redirect the 3D rendering from applications into these Pbuffers. Accessing a 3D graphics accelerator on a Linux system, a Solaris x86 system, or a Solaris SPARC system without GLP requires going through an X server. Sharing the server’s 3D graphics resources among multiple users means granting those users display access to the X server.
It is important to understand the security risks associated with this sharing. Once the X display access is granted to a user, nothing prevents that user from logging keystrokes or reading back images from the X display. Using xauth, you can obtain untrusted X authentication keys that prevent such exploits. However, those untrusted keys also disable access to the 3D hardware. Therefore, you must grant full trusted X access to any users needing to run VirtualGL.
Even if you fully trust the users to whom you are granting access, you should avoid logging in locally to the server’s X display as root. When root login is absolutely necessary, ensure that there are no suspicious processes running on the system prior to logging in.
The server’s SSH® daemon (ssh) should have the X11Forwarding option enabled. You configure this option in sshd_config. TABLE 4-1 lists the location of sshd_config, depending on your distribution of ssh.
When X11Forwarding is enabled, its line in sshd_config is:
The UseLogin option of SSH is incompatible, so that option must not be enabled in sshd_config. You can specifically disable UseLogin or simply not mention it in sshd_config.
Before running vglserver_config to configure a Solaris server, decide which of the following options you will choose:
When a user logs into a Solaris machine, the system automatically assigns ownership of the framebuffer devices to that user and sets the permissions for the graphics accelerator devices to those specified in /etc/logindevperm. The default permissions disable anyone from using the graphics accelerator devices except the user that is logged in.
In order to run VirtualGL, a user needs write access to the graphics accelerator devices as a shared resource. vglserver_config disables the login device permissions mechanism for the graphics accelerator devices. This variable also sets the owner and group for these devices such that any VirtualGL users (or optionally, all users) can write to the devices.
A Solaris SPARC graphics server with Sun OpenGL 1.5 and an XVR-2500, XVR-1200, or XVR-600 graphics accelerator can be configured to use those devices through GLP without having to start an X server on the graphics accelerators. (See GLP Access on Solaris SPARC Servers for more details.) If you have a graphics server with those characteristics, you can choose to use VirtualGL only in GLP mode when you run vglserver_config.
Note - Sun OpenGL 1.5 is available at: http://www.sun.com/software/graphics/opengl |
Unless absolutely needed, disable the XTEST extension. (For example, XTEST is disabled by Step 6 of To Configure a Solaris Server to Grant Access to the X Server.)
To Configure a Solaris SPARC Server to Use VirtualGL Without an X Server Through GLP |
Use this procedure if you determined in Using GLP Access on Solaris SPARC Servers that you will use GLP on your Solaris system.
A list of options is displayed.
3. Reply y or n to this question:
4. If you answered y in Step 3, edit the vglusers group in /etc/group.
If framebuffer device access will be restricted to the vglusers group, edit that entry to include root and any additional users. Any users that you add to vglusers at this time must log out and back in again before their new group permissions take effect.
5. Edit the /etc/dt/config/GraphicsDevices file as necessary.
This file contains a list of paths to 3D framebuffer devices that you want to use with VirtualGL. Each path is on a separate line. For example:
6. Verify that the system is ready to run VirtualGL.
b. Log back in to the server using SSH.
c. Execute the following command in the SSH session:
This command should output a list of visuals and complete with no errors.
7. If you want VirtualGL to use GLP by default, add these lines to /etc/profile:
These lines cause VirtualGL to use the first device specified in /etc/dt/config/GraphicsDevices as the default rendering device. A user can override this default by setting VGL_DISPLAY in a startup script (such as ~/.profile or ~/.login) or by passing vglrun an argument of -d device when invoking VirtualGL.
To Configure a Solaris Server to Grant Access to the X Server |
Use this procedure if you determined in Using GLP Access on Solaris SPARC Servers that you will not use GLP on your Solaris system. This procedure configures a VirtualGL server so that selected users can run VirtualGL, even if the server is currently at the login prompt.
1. Shut down the display manager.
2. Log in as root from the text console (or remotely using ssh) and enter:
A list of options is displayed.
4. Reply y or n to this question:
5. Reply y or n to this question:
6. Reply y or n to this question:
7. If you answered y in Step 4 or Step 5, edit the vglusers group in /etc/group.
If framebuffer device access will be restricted to the vglusers group, edit that entry to include root and any additional users. Any users that you add to vglusers at this time must log out and back in again before their new group permissions take effect.
8. Restart the display manager.
9. Verify that the system is ready to run VirtualGL.
b. Log back in to the server using SSH.
c. Execute one of the following command sequences in the SSH session:
Either command should output a list of visuals and complete with no errors. If xdpyinfo fails to run, then the permissions on display :0 are too restrictive.
d. If you chose to disable the XTEST extension, check the output of xdpyinfo to verify that XTEST is not included in the list of extensions.
The vglrun script can be used to launch either binary executables or shell scripts. When you use vglrun to run a shell script, the VirtualGL faker library is preloaded into every executable that the script launches. If the script calls any executables that are setuid or setgid, the operating system might refuse to load virtualGL into those executables, because you are attempting to preload a library (VirtualGL) that is not in a secure path.
The Solaris software has constraints on what goes into /usr/lib and /lib. By default, the Solaris software only permits libraries in those paths to be preloaded into an executable that is setuid or setgid. This situation means third-party packages are forbidden from installing anything into /usr/lib or /lib.
The following are alternative ways to work with this restriction:
Be aware of the security ramifications before you run the commands. You are essentially telling the Solaris software that you trust the security and stability of the VirtualGL code as much as you trust the security and stability of the operating system.
TABLE 4-2 lists two vglrun options relevant to launching scripts:
This section explains how to configure a VirtualGL server such that select users can run VirtualGL, even if the server is currently at the login prompt. The method is to call vglgenkey from the display manager’s startup script. vglgenkey calls xauth to generate an authorization key for the server’s X display, and stores this key under /etc/opt/VirtualGL. The VirtualGL launcher script (vglrun) then attempts to read this key and merge the key into the user’s .Xauthority file, granting the user access to the server’s X display. With this method, you can control who has access to the server’s X display by controlling who has read access to the /etc/opt/VirtualGL directory.
To Grant VirtualGL Access to the Server’s X Display on a Linux Server |
1. If the server is configured to boot into run level 5 (graphical login), temporarily shut down the X server as root. Type:
2. Otherwise, log in as root from the text console.
A list of options is displayed.
5. Reply y or n to this question:
6. Reply y or n to this question:
7. Reply y or n to this question:
8. If you answered y in Step 5 or Step 6, edit the vglusers group in /etc/group.
If framebuffer device access will be restricted to the vglusers group, edit that entry to include root and any additional users. Any users that you add to vglusers at this time must log out and back in again before their new group permissions take effect.
9. As the root user, restart the X server. Type:
10. Verify that the system is ready to run VirtualGL.
b. Log back in to the server using SSH.
c. Execute one of the following command sequences in the SSH session:
Either command should output a list of visuals and complete with no errors. If xdpyinfo fails to run, then the permissions on display :0 are too restrictive.
d. If you chose to disable the XTEST extension, check the output of xdpyinfo to verify that XTEST is not included in the list of extensions.
This section describes how to add graphics resources to Sun Grid Engine. You must first install Sun Grid Engine and the Sun Shared Visualization 1.1 software before continuing with this procedure.
These steps are to be performed as the sgeadmin user on the queue master host, or on an administrative host that mounts $SGE_ROOT read-write.
To Set the Variables |
1. Set $SGE_ROOT and set PATH to include Sun Grid Engine directories:
where /gridware is the base directory of your $SGE_ROOT.
2. Assure that your DISPLAY environment variable is set and refers to the system whose X server keyboard you are using:
where myhost is the hostname of the X server, and :0.0 identifies the X screen and display.
If you access the server using ssh -X, ssh sets DISPLAY for you. However, the ssh tunnel is available only on that server host, not on all execution hosts in the grid.
To Add Graphics to Sun Grid Engine |
Note - If you are upgrading an existing Sun Shared Visualization 1.1 software installation, you only need to perform Step 1, Step 8, and Step 9. |
1. If the optional software was not already installed on the grid’s NFS server, then, as superuser, install that software.
Note - Ensure that your $SGE_ROOT value is your answer to the installation prompt, “Please enter your SGE_ROOT directory.” |
2. Set an administrative email for Sun Grid Engine so that all errors are reported through email.
This command starts your $EDITOR with a file containing configuration variables.
b. Add the email address for the administrator_mail configuration variable, then save and quit the file.
3. Add resource names to the Sun Grid Engine complex.
The complex is the vocabulary of variables that can be specified. Seven resources will be added in this step: graphics, graphics_alone, maximum_graphics, headnode, chromium, sc_cols, and sc_rows. (Some of those resources are used by Sun Scalable Visualization software.) TABLE 4-3 describes the four resources most important for Sun Shared Visualization.
Further details on some of these resources are provided in More Graphics Resource Allocation Information.
Add resource names by running the add_to_complex script. As a Sun Grid Engine administrator (sgeadmin), type:
The script adds the information in TABLE 4-3 to your Sun Grid Engine complex. The script reports if a resource already exists or is added. When the script is finished, it will prompt you to perform the next step in this procedure.
4. Define which hosts have how many graphics resources available.
This step determines the maximum number of simultaneous graphics jobs that Sun Grid Engine could start on that host. For example, if your host has two graphics boards and the boards can accommodate three jobs each, your resources would be 2 x 3, for a total of 6.
The easiest way to determine graphics resources is by using the config_gfx script.
a. As sgeadmin or root on each graphics execution host, type:
By default, the graphics server host is the host on which config_gfx is invoked. The default name of the queue to be configured is all.q. To use different values for either of these systems, type config_gfx with the -h (host) or -q (queue) option:
b. Respond to questions asked by the config_gfx script.
Your answers will determine values for some of the resources that were added in Step 3.
The number of graphics jobs to allow simultaneously is of key importance. If this number is too high, graphics accelerator memory could be exhausted and performance will suffer greatly (for example, when applications page or swap their data onto the accelerator). If the number is too low, applications will need to wait until another application exits, even if an application is not actively using the graphics accelerator (for example, when the user of that application is away from the desk).
A good starting point is two or three applications per graphics accelerator. Allow more simultaneous jobs if the load is typically light users and data sets. Limit the number of simultaneous jobs if there are heavy users and data sets.
The following example of running config_gfx is for a Sun X4600 M2 server with two Nvidia QuadroPlex model 4s (a total of four graphics devices), which are not expected to run Chromium:
5. Set the starter and epilog scripts for Sun Grid Engine’s all.q cluster queue.
These scripts are hooks supported by Sun Grid Engine to provide queue-specific activity before and after a job runs.
This action sets the starter and epilog scripts for all Sun Grid Engine queues.
6. (Optional) Copy the graphics/docs/README file to a more user accessible location.
Tip - The contents of the README file summarize Sun Grid Engine use. Edit the file to better describe your particular site, rename the file, and make the file available to users in $SGE_ROOT. |
7. Ensure that your DISPLAY environment variable refers to your X server:
where myhost is the hostname of the X server.
8. Attempt to run a graphics job.
This example submits to any Shared Visualization 1.1 graphics server:
The following example names a graphics execution host:
% qrsh -b n -q all.q@hostname /opt/VirtualGL/bin/vglrun -c proxy -spoil \ $SGE_ROOT/graphics/RUN.glxspheres |
9. Start $SGE_ROOT/graphics/empty_jobs from /etc/init.d/sgeexecd.
/etc/init.d/sgeexecd is the Sun Grid Engine standard startup script. This script initiates shepherd processes. If these processes are shut down before the graphics jobs, you cannot reclaim the resources of those graphics jobs. To alleviate any possibility of this problem:
a. Edit the /etc/init.d/sgeexecd file and around line 245, find $bin_dir/sge_execd.
b. Insert the following text before that line:
Replace sgeadmin in this line if your site uses a different SGE administrative login.
Note - You need to repeat this step if the Sun Grid Engine software is upgraded. |
Note - The steps referenced in this section pertain to the procedure To Add Graphics to Sun Grid Engine |
You can control which graphics devices are used by Sun Grid Engine by creating or editing a world-readable local graphics configuration file /etc/dt/config/GraphicsDevices on any execution host. If the GraphicsDevices file is not present, only X server :0.0 is used by GLX.
To Create a GraphicsDevices File |
This procedures also creates a directory for the GraphicsDevices file.
As superuser, type these commands:
# mkdir -p /etc/dt/config # touch /etc/dt/config/GraphicsDevices # chmod 644 /etc/dt/config/GraphicsDevices |
On a SPARC Solaris graphics server, kfb (XVR-2500) devices and jfb (XVR-1200 and XVR-600) devices can be used by VirtualGL through GLP. There is no need for an X server on the devices. See To Configure a Solaris SPARC Server to Use VirtualGL Without an X Server Through GLP. On such a host, each line of the graphics server’s configuration file can be a device name followed by an optional maximum number of concurrent jobs for that device. (If no number is added, the default is that the device is used by only one job at a time.) For example:
In this example, the host’s jfb0 device can support only one Sun Grid Engine graphics job. The jfb1 device can only support up to two Sun Grid Engine graphics jobs. The jfb2 device is not used for any Sun Grid Engine graphics jobs. (This device might be used by a local console user.)
Any UNIX host can be configured to start an X server on each device. The vglgenkey technique of vglserver_config (described in Configuration Information for Linux Servers) will enable access to that display for VirtualGL users. If there are several graphics accelerators, the local configuration file can control allocation of jobs to these X displays or screens (for example, an x86 host with two devices used by screens :0.0 and :0.1).
Xinerama is an extension to the X Window System that enables multiheaded X. When X is configured to use Xinerama, X can provide a user with one large virtual screen spread across two or more physical displays (also called heads). This configuration enables any application’s window to be moved from one physical display to another, or for one window to span multiple displays. A similar effect can be produced, without using Xinerama, by a single graphics accelerator that is able to offer a single desktop across multiple monitor outputs. Use Xinerama with Sun Scalable Visualization Software on a host with multiple graphics accelerators that drive any portion of a group of displays (also called a power wall).
When multiple graphics devices are used with Xinerama, X provides only a single large, virtual screen, which is typically :0.0. Therefore, applications do not generally control the head on which a window or dialog will appear. VirtualGL uses only this single virtual X screen to provide remote users with GLX access to the graphics accelerator devices that drive the multiple heads. All remote visualization users share the first graphics accelerator, which causes resources on subsequent graphics devices to be underutilized.
Without Xinerama, multiple graphics accelerators can be used as independent devices. In this case, the X desktop on each device is an independent desktop, but all desktops share the mouse and keyboard. Application windows started on one device must remain on that device. Those windows cannot be moved to a different device, nor can the windows span across multiple devices.
Because X treats each graphics device as a separate screen (typically, :0.0 and :0.1), VirtualGL can use any of these X screens to provide remote users with GLX access to a graphics accelerator device. For remote visualization, you normally want to configure a single X server to use all the graphics devices without Xinerama. Then configure Sun Grid Engine to allocate all graphics accelerators to remote visualization jobs, a configuration that provides load balancing.
Consequently, the host’s /etc/dt/config/GraphicsDevices local configuration file might be:
The number of graphics resources for each execution host (Step 4 in To Add Graphics to Sun Grid Engine) is the maximum number of concurrent graphics jobs Sun Grid Engine schedules for that host. The total of the maximum number of jobs on all graphics devices in the local configuration file should be no less.
Similarly, the execution host should have at least as many total slots as the total number of maximum jobs for concurrent jobs, if you want the execution host to allow that many concurrent graphics jobs. The Sun Grid Engine default for slots is the number of CPUs (cores) in the system when Sun Grid Engine’s install_execd (Sun Grid Engine execution host configuration) script is run.
The configuration files are used by the graphics allocation script, $SGE_ROOT/graphics/alloc (or alloc.debug). You can study the script and comments, to adjust the script to suit your needs. Make a copy of your changes, so that your customizations to the script can be reintegrated in the event of a software upgrade.
A user might demand a certain number of graphics boards for a job. This is a hard limit. If not possible, the job does not run. The hard limit is specified (for example, Step 8 of To Add Graphics to Sun Grid Engine), with -l gfx=value.
A user might also request a desired (maximum) number of graphics boards, which Sun Grid Engine calls a soft limit. In this situation, a queued job waits for a time when more resources are available. An interactive job is immediate and fails. A soft limit needs the -soft introduction, and also must use a different resource, maximum_graphics (shorthand: maxgfx). This situation is due to Sun Grid Engine restrictions.
A user can start qsub or qrsh specifying both hard quantities of necessary resources and soft quantities of desired resources. For example:
In this example, the job requests four graphics devices, but demands at least one. If two devices are assigned, the VGL_DISPLAY value in the environment of the job contains two words. Each word could be a graphics device name (on a SPARC Solaris host using GLP, such as /dev/fbs/kfb0;) or an X display name (such as :0.0 or :0.1).
VirtualGL itself only uses the first device (or display) in the VGL_DISPLAY environment variable value. Allocating multiple devices is of value only if the job divides work among multiple processes, using one device or display value for each process.
The following tables provide more information about the Sun Grid Engine integers and environment variables that control allocation of graphics resources:
The graphics allocation called by the starter script on a graphics server host attempts logging of which users use how many graphics devices at what start and finish times, using the system logger. By default, these messages are local0.info messages and are discarded by the Solaris logger.
Follow this procedure to configure system logging to save the logging messages in /var/adm/messages.
1. Review the syslog.conf(4) man page.
2. Open the /etc/syslog.conf file in an editor.
3. Search for the /var/adm/messages line following mail.crit.
4. Add ;local0.info to that line.
When a job uses gfx=1 (or more) resources, Sun Grid Engine allocates one or more graphics accelerators. Sun Grid Engine also sets the graphics accelerator’s device name into the VGL_DISPLAY environment variable used by vglrun.
Such a job can interpose on an application by starting the job with vglrun or /opt/VirtualGL/bin/vglrun if not in the users’ $PATH.
If qsub starts vglrun directly, qsub requires the full path. For example:
vglrun can also start an executable. For example:
Also, a Sun Grid Engine job script can start vglrun when ready to run the application. The following example job script starts /opt/VirtualGL/bin/glxspheres on a Solaris or Linux graphics server. This script is a simplified version of $SGE_ROOT/graphics/RUN.glxspheres. Italicized text in this listing provides commentary, but is not part of the job script itself.
VirtualGL cannot use multiple graphics accelerators, so you cannot specify gfx any greater than 1, nor even configure maxgfx. To do so would consume resources without a performance improvement. You might want to specify gfx greater than 1 when your job requires several graphics accelerators concurrently, yet separately.
You might want Sun Grid Engine to allocate a graphics accelerator and start a TurboVNC server to use the graphics accelerator. For example:
When a shell in the TurboVNC server is ready to start an application, start vglrun from within a terminal on the TurboVNC server. For example:
By writing a specialized script, Sun Grid Engine resources and VGL_ environment variables can be set at runtime. You can see this situation in the $SGE_ROOT/graphics/RUN.vncserver script.
If you need the server to support quad-buffered stereographic display for remote VirtualGL clients, read the requirements in the "Verifying Advanced Feature Support" section of Appendix A in the Sun Shared Visualization 1.1 Software Client Administration Guide.
To Determine if a Server Has a Suitable Visual for Stereographic Rendering |
1. Type one of the following on the VirtualGL server:
2. In the output, see if one or more of the visuals says stereo=1 and lists Pbuffer as one of the Drawable Types.
This output is an indicator that the server is suitable.
To Verify Client Visuals |
Now you need to determine whether the X display on the client has a suitable visual for stereographic rendering, transparent overlays, or Pseudocolor.
1. On the VirtualGL server, type:
2. In the output, look for the following:
You can use the vglserver_config script to restore the unshared, secure default settings that do not allow VirtualGL access. However, unconfiguring the server does not remove the vglusers group or the /etc/dt/config/GraphicsDevices file.
Both of the options described in the following procedure will restore the framebuffer device permissions to their default. The default is that the framebuffer devices can be accessed only by root or the user currently logged into the system locally.
To Unconfigure the VirtualGL Server |
1. Shut down the display manager.
2. Log in as root from the text console (or remotely using ssh) and type:
A list of options is displayed.
3. To unconfigure GLX mode, select this option:
This option removes any shared access to the server’s X display, preventing VirtualGL from accessing the 3D hardware in that manner. This option also reenables the XTEST extension on the server’s X display.
4. To unconfigure GLP mode, select this option:
5. If you selected the GLX option in Step 3, you must restart the display manager before the unconfiguration changes will take effect.
Note - To resolve problems involving the proper use of vglconnect, refer to the Sun Shared Visualization 1.1 Software Client Administrator Guide. |
The vglclient program is normally started implicitly by vglconnect.
Copyright © 2008, Sun Microsystems, Inc. All Rights Reserved.