This section describes how to troubleshoot problems with the Nova driver.
The Oracle VM Server for SPARC OpenStack Nova driver provides different levels of detail based on the debug and verbose property values in nova.conf.
It is best to have verbose=true. You can also set debug=true to assist in troubleshooting an issue.
When in debug mode, the Oracle VM Server for SPARC OpenStack Nova driver provides trace messages to help identify precisely which method is being run at any given time. These messages can help you identify the cause of an issue that you encounter. To find a method entry, search for method_run: in the nova-compute service (SMF) log. To find a method return, search for method_return: in the nova-compute service log.
View other log messages from the driver. The driver also provides other log output, which begins with DEBUG:, WARNING:, ERROR:, EXCEPTION:, or INFO:.
Note that the Nova debug driver log entries start with a line such as the following:
2016-07-07 15:14:51.404 29186 DEBUG nova.virt.ldoms.driver
The nova-compute service itself provides TRACE messages if an unhandled exception has occurred. Search for TRACE messages to identify the root cause of a problem.
Ensure that the driver started as expected. When you start the nova-compute service, ensure that it starts properly. When the driver starts or restarts, you should see lines in the log that look similar to the following:
2016-07-07 15:20:14.011 1098 DEBUG nova.service [req-d8973f20-af93-4de6-9732-9e4a0ab6c61a - - - - -] Creating RPC server for service compute start /usr/lib/python2.7/vendor-packages/nova/service.py:188 2016-07-07 15:20:14.013 1098 INFO oslo_messaging._drivers.impl_rabbit [req-d8973f20-af93-4de6-9732-9e4a0ab6c61a - - - - -] Connecting to AMQP server on 10.0.68.21:5672 2016-07-07 15:20:14.026 1098 INFO oslo_messaging._drivers.impl_rabbit [req-d8973f20-af93-4de6-9732-9e4a0ab6c61a - - - - -] Connected to AMQP server on 10.0.68.21:5672 2016-07-07 15:20:14.033 1098 DEBUG nova.service [req-d8973f20-af93-4de6-9732-9e4a0ab6c61a - - - - -] Join ServiceGroup membership for this service compute start /usr/lib/python2.7/vendor-packages/nova/service.py:206 2016-07-07 15:20:14.033 1098 DEBUG nova.servicegroup.drivers.db [req-d8973f20-af93-4de6-9732-9e4a0ab6c61a - - - - -] DB_Driver: join new ServiceGroup member 10.0.68.22 to the compute group, service = <nova.service.Service object at 0xea967cf0> join /usr/lib/python2.7/vendor-packages/nova/servicegroup/drivers/db.py:59
If you do not see a connection to AMQP and the DB_Driver: join new ServiceGroup member message, ensure that the driver has not stopped because of a configuration issue. If the problem is a configuration issue, you will see an exception starting with EXCEPTION or ERROR in the early startup log.
If the driver does not detect configuration issues, ensure that your MTU for the management network is the same everywhere, that NTP is configured, and name resolution is working.
When in debug mode (debug=true), DEBUG messages from the driver are written to the Nova compute manager logs. The Nova driver has run_method and method_return traces. You can follow these traces for all methods that the driver runs. These traces help you to locate precisely where a problem occurs and shows all preceding steps that led to the problem.
You can use the following commands to simplify the debugging process when in debug mode or with a debug log file:
nova# tail -f `svcs -L nova-compute` | egrep 'DEBUG: run_method:|TRACE|EXCEPTION'
Or, run a trace that includes method_return with the return value:
nova# tail -f `svcs -L nova-compute` \ | egrep 'DEBUG: run_method:|DEBUG: method_return:|TRACE|EXCEPTION'
The following command performs a trace and excludes DLM and other PERIODIC tasks:
nova# tail -f `svcs -L nova-compute` | egrep 'DEBUG: run_method:|TRACE|EXCEPTION' \ | egrep -v 'PERIODIC|dlm'