In order to allow concurrent operations, BDB XML provides the locking subsystem. This subsystem provides inter- and intra- process concurrency mechanisms. It is extensively used by BDB XML concurrent applications, but it can also be generally used for non-BDB XML resources.
This section describes the locking subsystem as it is used to protect BDB XML resources. In particular, issues on configuration are examined here. For information on using the locking subsystem to manage non-BDB XML resources, see the Berkeley DB Programmer's Reference Guide.
You initialize the locking subsystem by specifying
DB_INIT_LOCK
to the
DB_ENV->open()
method.
Before opening your environment, you can configure various values for your locking subsystem. Note that these limits can only be configured before the environment is opened. Also, these methods configure the entire environment, not just a specific environment handle.
Finally, each bullet below identifies the
DB_CONFIG
file parameter that can be used
to specify the specific locking limit. If used, these
DB_CONFIG
file parameters override any
value that you might specify using the environment handle.
The limits that you can configure are as follows:
The number of lockers supported by the environment. This value is used by the environment when it is opened to estimate the amount of space that it should allocate for various internal data structures. By default, 1,000 lockers are supported.
To configure this value, use the
DB_ENV->set_memory_init()
method to configure the DB_MEM_LOCKER
structure.
As an alternative to this method, you can configure this
value using the DB_CONFIG
file's
set_lk_max_lockers
parameter.
The number of locks supported by the environment. By default, 1,000 locks are supported.
To configure this value, use the
DB_ENV->set_memory_init()
method to configure the DB_MEM_LOCK
structure.
As an alternative to this method, you can configure this
value using the DB_CONFIG
file's
set_lk_max_locks
parameter.
The number of locked objects supported by the environment. By default, 1,000 objects can be locked.
To configure this value, use the
DB_ENV->set_memory_init()
method to configure the DB_MEM_LOCKOBJECT
structure.
As an alternative to this method, you can configure this
value using the DB_CONFIG
file's
set_lk_max_objects
parameter.
For a definition of lockers, locks, and locked objects, see Lock Resources.
For example, to configure the number of locks that your environment can use:
#include "DbXml.hpp" ... using namespace DbXml; int main(void) { u_int32_t env_flags = DB_CREATE | // If the environment does not // exist, create it. DB_INIT_LOCK | // Initialize locking DB_INIT_LOG | // Initialize logging DB_INIT_MPOOL | // Initialize the cache DB_THREAD | // Free-thread the env handle DB_INIT_TXN; // Initialize transactions char *envHome = "/export1/testEnv"; DB_ENV *myEnv = NULL; int dberr; XmlManager *myManager = NULL; dberr = db_env_create(&myEnv, 0); if (dberr) { std::cout << "Unable to create environment: " << db_strerror(dberr) << std::endl; if (myEnv) myEnv->close(myEnv, 0); return (EXIT_FAILURE); } // Configure max locks myEnv->set_lk_max_locks(envp, 5000); myEnv->open(myEnv, envHome, env_flags, 0); myManager = new XmlManager(myEnv, 0); // Do work here. Clean up when all done. if (myManager != NULL) { delete myManager; } myEnv->close(myEnv, 0); return (EXIT_SUCCESS); }
In order for BDB XML to know that a deadlock has occurred, some mechanism must be used to perform deadlock detection. There are three ways that deadlock detection can occur:
Allow BDB XML to internally detect deadlocks as they occur.
To do this, you use
DB_ENV->set_lk_detect()
.
This method causes BDB XML to walk its internal lock table
looking for a deadlock whenever a lock request
is blocked. This method also identifies how BDB XML decides which lock
requests are rejected when deadlocks are detected. For example,
BDB XML can decide to reject the lock request for the transaction
that has the most number of locks, the least number of locks,
holds the oldest lock, holds the most number of write locks, and
so forth (see the API reference documentation for a complete
list of the lock detection policies).
You can call this method at any time during your application's lifetime, but typically it is used before you open your environment.
Note that how you want BDB XML to decide which thread of control should break a deadlock is extremely dependent on the nature of your application. It is not unusual for some performance testing to be required in order to make this determination. That said, a transaction that is holding the most number of locks is usually indicative of the transaction that has performed the most amount of work. Frequently you will not want a transaction that has performed a lot of work to abandon its efforts and start all over again. It is not therefore uncommon for application developers to initially select the transaction with the minimum number of write locks to break the deadlock.
Using this mechanism for deadlock detection means that your application will never have to wait on a lock before discovering that a deadlock has occurred. However, walking the lock table every time a lock request is blocked can be expensive from a performance perspective.
Use a dedicated thread or external process to perform deadlock detection. Note that this thread must be performing no other container operations beyond deadlock detection.
To externally perform lock detection, you can use
either the
DB_ENV->lock_detect()
method, or use the
db_deadlock command line
utility. This method (or command) causes BDB XML to walk the
lock table looking for deadlocks.
Note that like
DB_ENV->set_lk_detect()
,
you also use this method (or command line utility)
to identify which lock requests are rejected in the
event that a deadlock is detected.
Applications that perform deadlock detection in this way typically run deadlock detection between every few seconds and a minute. This means that your application may have to wait to be notified of a deadlock, but you also save the overhead of walking the lock table every time a lock request is blocked.
Lock timeouts.
You can configure your locking subsystem such that
it times out any lock that is not released within a
specified amount of time. To do this, use the
DB_ENV->set_timeout()
method.
Note that lock timeouts are only checked when a
lock request is blocked or when deadlock
detection is otherwise performed. Therefore, a lock can have timed out and still be held for
some length of time until BDB XML has a reason to examine its locking tables.
Be aware that extremely long-lived transactions, or operations that hold locks for a long time, may be inappropriately timed out before the transaction or operation has a chance to complete. You should therefore use this mechanism only if you know your application will hold locks for very short periods of time.
For example, to configure your application such that BDB XML checks the lock table for deadlocks every time a lock request is blocked:
#include "DbXml.hpp" ... using namespace DbXml; int main(void) { u_int32_t env_flags = DB_CREATE | // If the environment does not // exist, create it. DB_INIT_LOCK | // Initialize locking DB_INIT_LOG | // Initialize logging DB_INIT_MPOOL | // Initialize the cache DB_THREAD | // Free-thread the env handle DB_INIT_TXN; // Initialize transactions DB_ENV *myEnv = NULL; XmlManager *myManager = NULL; char *envHome = "/export1/testEnv"; int dberr; dberr = db_env_create(&myEnv, 0); if (dberr) { std::cout << "Unable to create environment: " << db_strerror(dberr) << std::endl; if (myEnv) myEnv->close(myEnv, 0); return (EXIT_FAILURE); } // Configure db to perform deadlock detection internally, and to // choose the transaction that has performed the least amount // of writing to break the deadlock in the event that one // is detected. myEnv->set_lk_detect(DB_LOCK_MINWRITE); myEnv->open(myEnv, envHome, env_flags, 0); myManager = new XmlManager(myEnv, 0); // From here, you open your containers, proceed with your // container operations, and respond to deadlocks as // is normal (omitted for brevity). ...
Finally, the following command line call causes
deadlock detection to be run against the
environment contained in /export/dbenv
. The
transaction with the youngest lock is chosen to break the
deadlock:
> /usr/local/db_install/bin/db_deadlock -h /export/dbenv -a y
For more information, see the
db_deadlock
reference documentation.
When BDB XML determines that a deadlock has occurred, it will
select a thread of control to resolve the deadlock and then
throws XmlException
in that
thread. You must then test this exception to see if it is caused by a deadlock situation.
Do this by
checking whether XmlException.getDbErrno()
is equal to DB_LOCK_DEADLOCK
.
If a deadlock is detected, the thread must:
Cease all read and write operations.
Abort the transaction.
Optionally retry the operation. If your application retries deadlocked operations, the new attempt must be made using a new transaction.
If a thread has deadlocked, it may not make any additional container calls using the handle that has deadlocked.
For example:
// Environment, Manager and Container opens are omitted for brevity XmlTransaction txn; // retry_count is a counter used to identify how many times we've // retried this operation. To avoid the potential // for endless looping, we won't retry more than // MAX_DEADLOCK_RETRIES times. while (retry_count < MAX_DEADLOCK_RETRIES) { try { txn = myManager.createTransaction(); // Need an update context for the put. XmlUpdateContext theContext = myManager.createUpdateContext(); // Get the first input stream. XmlInputStream *theStream = myManager.createLocalFileInputStream("/export/file.xml"); // Put the first document myContainer.putDocument(txn, // the transaction object "file.xml", // The document's name theStream, // The actual document. theContext, // The update context // (required). 0); txn.commit(); return (EXIT_SUCCESS); } catch (XmlException &e) { if (e.getDbErrno() == DB_LOCK_DEADLOCK) { try { // Abort the transaction and increment the // retry counter txn.abort(); // Increment the retry count retry_count++; // If we've retried too many times, log it // and exit if (retry_count >= MAX_DEADLOCK_RETRIES) { std::cerr << "Exceeded retry limit. Giving up." << std::endl; return (EXIT_FAILURE); } } catch (DbException &ae) { envp->err(ae.getErrorCode(), "txn abort failed."); return (EXIT_FAILURE); } } else { try { // For a generic error, log it and abort. std::cerr << "Error in transaction: " << e.what() << "\n" << "Aborting." << std::endl; txn.abort(); } catch (DbException &ae) { envp->err(ae.get_errno(), "txn abort failed."); return (EXIT_FAILURE); } } } }
Normally when a thread of control must be selected to resolve a deadlock, BDB XML decides which thread will perform the resolution; you have no way of knowing in advance which thread will be selected to resolve the deadlock.
However, there may be situations where you know it is better for one thread to resolve a deadlock over another thread. As an example, if you have a background thread running data management activities, and another thread responding to user requests, you might want deadlock resolution to occur in the background thread because you can better afford the throughput costs there. Under these circumstances, you can identify which thread of control will be selected for resolved deadlocks by setting a transaction priorities.
When two transactions are deadlocked, BDB XML will abort the transaction with the lowest priority. By default, every transaction is given a priority of 100. However, you can set a different priority on a transaction-by-transaction basis by using the method.
When two or more transactions are tied for the lowest priority, the tie is broken based on the policy provided to
A transaction's priority can be changed at any time after the transaction handle has been created and before the transaction has been resolved (committed or aborted). For example: