E Troubleshooting Oracle HTTP Server

You can get help to troubleshoot some of the common problems that you might encounter when using Oracle HTTP Server.

Oracle HTTP Server Fails to Start Due to Port Conflict

If Oracle HTTP Server cannot start due to a port conflict, a message containing the string [VirtualHost: main] (98)Address already in use is generated. This error condition occurs if the listen port configured for Oracle HTTP Server is the same as the one in use by another process.

The generated message may look like the following:

[VirtualHost: main] (98)Address already in use: make_sock: could not bind to address [::]:7777

Solution

Determine what process is already using that port, and then either change the IP:port address of Oracle HTTP Server or the port of the conflicting process.

Note:

If the Oracle HTTP Server instance was created with the config Wizard, there is no automated port management. It is possible to create multiple instances using the same Listen port.

System Overloaded by Number of httpd Processes

When the system is overloaded by too many httpd processes, there are insufficient resources for normal processing. This slows down the response time. You can lower the value of MaxRequestWorkers to a value the machine can accommodate.

When too many httpd processes run on a system, the response time degrades because there are insufficient resources for normal processing.

Solution

Lower the value of MaxRequestWorkers to a value the machine can accommodate.

Permission Denied When Starting Oracle HTTP Server On a Port Below 1024

If you try to start Oracle HTTP Server on a port below 1024, a message containing the string [VirtualHost: main] (13)Permission denied: make_sock: could not bind to address [::]:443 is generated. This error condition occurs because root privileges are needed to bind these ports.

The generated message may look like the following:

[VirtualHost: main] (13)Permission denied: make_sock: could not bind to address [::]:443

Oracle HTTP Server will not start on ports below 1024 because root privileges are needed to bind these ports.

Solution

Follow the steps in Starting Oracle HTTP Server Instances on a Privileged Port (UNIX Only) to start Oracle HTTP Server on a Privileged Port.

Using Log Files to Locate Errors

There are three types of log files that help you locate errors, namely, rewrite, script, and error.

The log files are explained in the following sections:

Rewrite Log

This log file is necessary for debugging when mod_rewrite is used. The log file produces a detailed analysis of how the rewriting engine transforms requests. The value of the LogLevel directive controls the level of detail.

Script Log

This log file enables you to record the input to and output from the CGI scripts. This should only be used in testing, and not for production servers.

See Also:

ScriptLog in the Apache HTTP Server documentation at:

http://httpd.apache.org/docs/current/mod/mod_cgi.html#scriptlog

Error Log

This log file records overall server problems. Refer to Managing Oracle HTTP Server Logs for details on configuring and viewing error logs.

Recovering an Oracle HTTP Server Instance on a Remote Host

To recover an Oracle HTTP Server instance on a remote host, you must use tar and untar; pack.sh and unpack.sh do not work in this scenario.

If you need to recover an Oracle HTTP Server instance that is installed on a remote host (that is, a host with just managed servers but no Administration Server), you must use tar and untar; pack.sh and unpack.sh do not work in this scenario.

Oracle HTTP Server Performance Issues

You might encounter performance issues when running Oracle HTTP Server. The documentation includes several topics to explain such performance related problems.

Special Runtime Files Reside on a Network File System

Oracle HTTP Server uses locks for its internal processing, which in turn use lock files. These files are created dynamically when the lock is created and are accessed every time the lock is taken or released. If these files reside on a slower file system (for example, network file system), then there could be severe performance degradation. To counter this issue:

On Linux:

In httpd.conf, change Mutex fnctl:fileloc default to Mutex sysvsem default where fileloc is the value of the directive LockFile (two places).

On Solaris:

In httpd.conf, change Mutex fnctl:fileloc default to Mutex pthread default where fileloc is the value of the directive LockFile (two places).

UNIX Sockets on a Network File System

The mod_cgid module is not enabled by default. If enabled, this module uses UNIX sockets internally. If UNIX sockets reside on a slower file system (for example, network file system), a severe performance degradation could be observed. You can set the following directive to avoid the issue:

  • If mod_cgid is enabled, use the ScriptSock directive to place mod_cgid's UNIX socket on a local filesystem.

DocumentRoot on a Slow File System

If you are using mod_wl_ohs to route the requests to back-end WLS server/cluster, and the DocumentRoot is on a slower file system (for example, network file system), then every request that mod_wl_ohs routes to the backend server can experience performance issues. This can be overcome by setting WLSRequest to ON instead of SetHandler weblogic-handler.

Instances Created on Shared File Systems

If you encounter functional or performance issues when creating an Oracle HTTP Server instance on a shared file system, including NFS (Network File System), it might be due to file system accesses in the default configuration. In this case, you must update the httpd.conf file specific to your operating systems. See Updating Oracle HTTP Server Component Configurations on a Shared File System.

Out of DMS Shared Memory

When there is an incorrect calculation of the required shared memory for Oracle HTTP Server DMS, error logs are displayed. These problems can be resolved by setting the DMS shared memory directive to a value larger than the default value of 4096 or continuing to set the directive 50% higher until the problem is resolved.

An error log containing the string dms_fail_shm_expansion: out of DMS shared memory in pid XXX, disabling DMS; increase DMSProcSharedMem directive from YYY is displayed when an incorrect calculation of required shared memory for Oracle HTTP Server DMS. This can be resolved by setting DMSProcSharedMem to a larger value than the default value of 4096. In some extreme configurations, you might see the following message in the Oracle HTTP Server error log:

dms_fail_shm_expansion: out of DMS shared memory in pid XXX, disabling DMS; increase DMSProcSharedMem directive from YYY

This is because of an incorrect calculation of required shared memory for Oracle HTTP Server DMS. This can be resolved by setting DMSProcSharedMem to a larger value than the default of 4096. Continue setting DMSProcSharedMem 50% higher until the problem is resolved. The minimum value for DMSProcSharedMem is 256 and the maximum value is 65536.

In a configuration with a very large number of virtual hosts (hundreds or thousands), if the above workaround does not work, you can instead, set the environment variable OHS_DMS_BLOCKSIZE to a large enough value that Oracle HTTP Server starts without error. The value of this variable is in kilobytes and a value of 524288 is a good starting point. If the error persists, continue to increase the value by 50% until Oracle HTTP Server starts without error.

Oracle HTTP Server Fails to Start When mod_security is Enabled on RHEL or Oracle Linux 7

If mod_security is configured in Oracle HTTP Server in Red Hat Enterprise Linux (RHEL) or Oracle Linux (OL) 7, Oracle HTTP Server fails to start. This error condition occurs because there is no symbolic link /lib64/liblzma.so.0

The generated error looks like the following:

iblzma.so.0: cannot open shared object file: No such file or directory
Solution
  1. Log in as a root user.

  2. To create a symbolic link, /lib64/liblzma.so.0, run the following command:

    cd /lib64 
    ln -s liblzma.so.5.0.99 liblzma.so.0 
  3. Verify the symlink as follows:
    ls -al *liblzma*
  4. Exit root.

  5. Start Oracle HTTP Server.

    For example, startComponent.sh ohs1, where ohs1 is the Oracle HTTP Server instance you want to start.

Oracle HTTP Server Fails to Start due to Certificates Signed Using the MD5 Algorithm

If Oracle HTTP Server cannot start due to the server wallet containing a certificate signed with the Message Digest 5 (MD5) algorithm, you can replace the MD5 certificate with a Secure Hash Algorithm 2 (SHA-2) certificate.

Oracle HTTP Server fails to start if the Oracle HTTP Server wallet contains a certificate or certificate request that is signed with the Message Digest 5 (MD5) algorithm.

  • Solution: Replace the MD5 certificate with a Secure Hash Algorithm 2 (SHA-2) certificate.

  • Workaround: To enable MD5 supported certificate, set the ORACLE_SSL_ALLOW_MD5_CERT_SIGNATURES environment variable in the ohs.plugins.nodemanager.properties file to 1.

To set the environment variable in Oracle HTTP Server, see Environment Variable Configuration Properties.

Node Manager Logs Don't Show Clear Message When a Component Fails to Start

When an Oracle HTTP Server (OHS) component fails to start, the following errors are seen in ORACLE_INSTANCE/servers/COMPONENT_NAME/logs/COMPONENT_NAME.log:

[OHS] [INCIDENT_ERROR:20] [AH00480] [mpm_event] [host_id: xxx] [host_addr: xxx] [pid: xxx] 
[tid:xxxx] [user: xxx] [VirtualHost: main] (11)Resource
temporarily unavailable: AH00480: apr_thread_create: unable to create worker thread

This can be caused due to lack of Virtual Memory or a limit has been placed on the OHS for the number of processes it can run.

Solution

  1. Check and increase the Virtual memory on the host or check the local process limits and open file descriptor limit for the user.

    For example, on Linux, check the user limits using the command ulimit -a and also check the /etc/security/limits.conf file for any system-wide user limits.

    To increase process limit, use the following command on Linux:

    $ ulimit -u xxxx

    To increase open files : file descriptors limit, use the following command on Linux:

    $ ulimit -n xxxxx
  2. Kill the processes that are not required and start the OHS.

SSL Handshake Fails Due to Certificate Chain

Certain browsers, such as Internet Explorer require that the entire certificate chain be imported to the browsers for the SSL handshake to work. If your certificate was issued by an intermediate CA, you will need to ensure that the complete chain of certificates is available on the browser or the handshake will fail. If an intermediate certificate in the chain expires, it must be renewed along with all the certificates (such as OHS server) in the chain.

Solution

When you configure SSL for Oracle HTTP Server, you may need to import the entire certificate chain (rootCA, Intermediate CA’s and so on).