There are numerous techniques the developer can use in order to ensure that Kodo JDO operates in the fastest and most efficient manner. Following are some guidelines. These hints contain information about what impact they will have on performance and scalability. Note that general guidelines regarding performance or scalability issues are just that -- guidelines. Depending on the particular characteristics of your application, the optimal settings may be considerably different than what is outlined below.
In the following table, each row is labeled with a list of italicized keywords. These keywords identify what characteristics the row in question may improve upon.
Many of the rows are marked with one or both of the performance and scalability labels. It is important to bear in mind the differences between performance and scalability (for the most part, we are referring to system-wide scalability, and not necessarily only scalability within a single JVM). The performance-related hints will probably improve the performance of your application for a given user load, whereas the scalability-related hints will probably increase the total number of users that your application can service. Sometimes, increasing performance will decrease scalability, and vice versa. Typically, options that reduce the amount of work done on the database server will improve scalability, whereas those that push more work onto the server (for example, flushing before queries when changes have been detected and IgnoreCache is set to false) will have a negative impact on scalability.
Table 16.1. Optimization Techniques
| Optimize database indexes performance, scalability |
The default set of indexes created by Kodo JDO's mapping
tool may not always be the most appropriate for your
application. Using Kodo's
jdbc-indexed metadata extension
or manually manipulating indexes to include
frequently-queried fields (as well as dropping indexes on
rarely-queried fields) can yield significant performance
benefits.
A database must do extra work on insert, update, and delete to maintain an index. This extra work will benefit selects with WHERE clauses, which will execute much faster when the terms in the WHERE clause are appropriately indexed. So, for a read-mostly application, appropriate indexing will slow down updates (which are rare) but greatly accelerate reads. This means that the system as a whole will be faster, and also that the database will experience less load, meaning that the system will be more scalable. Bear in mind that over-indexing is a bad thing, both for scalability and performance, especially for applications that perform lots of inserts, updates, or deletes. |
| Use the best JDBC driver performance, scalability, reliability | The JDBC driver provided by the database vendor is not always the fastest and most efficient. Some JDBC drivers do not support features like batched statements, the lack of which can significantly slow down Kodo JDO's data access and increase load on the database, reducing system performance and scalability. |
| JVM optimizations performance, reliability | Manipulating various parameters of the Java Virtual Machine (such as hotspot compilation modes and the maximum memory) can result in performance improvements. For more details about optimizing the JVM execution environment, please see http://java.sun.com/docs/hotspot/PerformanceFAQ.html. |
| Use the data cache performance, scalability | Using Kodo's data caching and query caching features (available in Kodo JDO Performance Pack and Enterprise Edition) can often result in a dramatic improvement in performance. Additionally, these caches can significantly reduce the amount of load on the database, increasing the scalability characteristics of your application. |
| Disable logging, performance
tracking performance | Developer options such as verbose logging and the JDBC performance tracker can result in serious performance hits for your application. Before evaluating any Kodo JDO's performance, these options should all be disabled. |
| Set IgnoreCache to true, or set
FlushBeforeQueries to true performance vs. scalability |
When both the
javax.jdo.option.IgnoreCache and
kodo.FlushBeforeQueries properties are set
to false, Kodo needs to evaluate in-memory dirty instances
against the datastore values that are returned from a query.
This can sometimes result in Kodo needing to evaluate the
entire extent of objects in order to return the correct
query results, which can have drastic performance
consequences. If it is appropriate for your application,
configuring FlushBeforeQueries
to automatically flush queries will ensure that this never
happens. Setting IgnoreCache to
false will result in a small performance hit even if
FlushBeforeQueries is true, as
incremental flushing is not as efficient overall as
delaying all flushing to a single operation during commit.
This is because incrementally flushing decreases Kodo's
ability to maximize statement batching, and increases
resource utilization.
Note that the default setting of FlushBeforeQueries is with-connection, which means that data will be flushed only if a dedicated connection is already in use by the persistence manager. So, the default value may not be appropriate for you. Setting IgnoreCache to true will help performance, since the persistence manager cache can be ignored for queries, meaning that incremental flushing or client-side processing is not necessary. It will also improve scalability, since overall database server usage is diminished. On the other hand, setting IgnoreCache to false will have a negative impact on scalability, even when using automatic flushing before queries, since more operations will be performed on the database server. |
| Configure
kodo.ConnectionRetainMode appropriately performance vs. scalability |
The
ConnectionRetainMode configuration option
controls when Kodo will obtain a connection, and how long
it will hold that connection. The optimal settings for this
option will vary considerably depending on the particular
behavior of your application. You may even benefit from
using different retain modes for different parts of your
application.
The default setting of on-demand minimizes the amount of time that Kodo holds onto a datastore connection. This is generally the best option from a scalability standpoind, as database resources are held for a minimal amount of time. However, if your connection pool is overly small relative to the number of concurrent persistence managers that need access to the database, or if your DataSource is not efficient at managing its pool, then this default value could cause undesirable pool contention. |
| Ensure that batch updates are
available performance, scalability | When performing bulk inserts, updates, or deletes, Kodo JDO will use batched statements. If this feature is not available in your JDBC driver, then Kodo JDO will need to issue multiple SQL statements instead of a single batch statement. |
| Use
single-table inheritance performance, scalability vs. disk space |
Using a single-table (flat) inheritance model is faster
for most operations than a multi-table (vertical)
inheritance model or horizontal mappings. If it is
appropriate for your application, you should use the
single-table inheritance model whenever possible.
However, single-table inheritance will require more disk space on the database side. Disk space is relatively inexpensive, but if your object model is particularly large, this can become a factor. |
| High increment in the sequence
factory performance, scalability | For applications that perform large bulk inserts, the retrieval of sequence numbers can be a bottleneck. Increasing the value of the Increment property of the kodo.jdbc.SequenceFactory plugin can reduce or eliminate this bottleneck. In some cases, implementing your own sequence factory can further optimize sequence number retrieval. |
| Use optimistic transactions performance, scalability |
Using datastore transactions translates into pessimistic
database row locking, which can be a performance hit
(depending on the database). If appropriate for your
application, optimistic transactions are typically faster
than datastore transactions.
Optimistic transactions provide the same transactional guarantees as datastore transactions, except that you must handle a potential optimistic verification exception at the end of a transaction instead of assuming that a transaction will successfully complete. In many applications, it is unlikely that different concurrent transactions will operate on the same set of data at the same time, so optimistic verification increases the concurrency, and therefore both the performance and scalability characteristics, of the application. A common approach to handling optimistic verification exceptions is to simply present the end user with the fact that concurrent modifications happened, and require that the user redo any work. |
| Use query aggregates and projections
performance, scalability | Using aggregates to compute reporting data on the database server can drastically speed up queries. Similarly, using projections when you are interested in specific object fields or relations rather than the entire object state can reduce the amount of data Kodo must transfer from the database to your application. |
| Perform nontransactional data reads
outside datastore (pessimistic) transactions performance, scalability | When using optimistic transactions, there is very little overhead involved in starting a transaction, so this does not help out very much in those situations. |
| Always close persistence managers,
extent iterators, and query results scalability |
Under certain settings, these objects may be backed by
resources in the database. For example, if you have
configured Kodo to use scrollable cursors and lazy object
instantiation by default, each query result will hold open
a ResultSet object, which, in turn,
will hold open a Statement object
(preventing it from being re-used). Garbage collection
will clean up these resources, so it is never necessary to
explicitly close them, but it is always faster if it is
done at the application level.
Example 16.1. Explicitly Closing Resources
public void giveRaise (String jdoql, double amnt)
{
PersistenceManagerFactory factory = ...;
PersistenceManager pm = factory.getPersistenceManager ();
Query query = null;
try
{
query = pm.newQuery (Employee.class, jdoql);
Collection res = (Collection) query.execute ();
for (Iterator itr = res.iterator (); itr.hasNext ();)
{
Employee emp = (Employee) itr.next ();
emp.setSalary (emp.getSalary () * (1 + amnt));
}
}
finally
{
if (query != null)
query.closeAll ();
pm.close ();
}
}
|
| Optimize connection pool
settings performance, scalability | Kodo JDO's built-in connection pool's default settings may not be optimal for all applications. For applications that instantiate and close many PersistenceManagers (such as a web application), increasing the size of the connection pool will reduce the overhead of waiting on free connections or opening new connections. You may want to tune the prepared statement pool size with the connection pool size. |
| Utilize the persistence manager
cache performance, scalability | When possible and appropriate, re-using persistence managers and setting the RetainValues configuration option to true may result in significant performance gains, since the persistence manager's built-in object cache will be used. |
| Enable multithreaded operation only
when necessary performance | Kodo JDO respects the javax.jdo.option.Multithreaded option in that it does not impose synchronization overhead for applications that set this value to false. If your application is guaranteed to only access a given persistence manager or related objects (extent, query) from a single thread, setting this option to false will result in the elimination of synchronization overhead, and may result in a modest performance increase. |
| Enable large data set
handling performance, scalability | If you execute queries that return large numbers of objects or have relations (collections or maps) that are large, and if you often only access parts of these data sets, enabling large result set handling where appropriate can dramatically speed up your application, since Kodo will bring the data sets into memory from the database only as necessary. |
| Disable large data set handling
performance, scalability | If you have enabled scrollable result sets and on-demand loading but do you not require it, consider disabling it again. Some JDBC drivers and databases (SQLServer for example) are much slower when used with scrolling result sets. |
| Develop a custom class indicator,
use the metadata-value indicator with short symbolic
constants, or do not use class indicators performance, scalability |
Kodo JDO's default class indicator is quite robust, in
that it can handle any class and needs no configuration,
but the downside of this robustness is that it puts a
relatively lengthy string into each row of the database.
With the metadata-value
indicator and a little application-specific
configuration, you could easily reduce this to a single
character or integer. This can result in significant
performance gains when dealing with many small objects,
since the subclass indicator data can become a significant
proportion of the data transferred between the JVM and
the database.
Alternately, if certain persistent classes in your application do not make use of inheritance, then you can disable the class indicator for these classes altogether.
Example 16.2. Disabling the Class Indicator
<jdo>
<package name="com.xyz">
<class name="NoSubclasses">
<extension vendor-name="kodo" key="jdbc-class-ind-name" value="none"/>
<!-- rest of class metadata -->
</class>
</package>
</jdo>
If you use some sort of mapping of symbolic constants to subclasses, bear in mind that changes to your class structure will require a bit more care, since you must take care to maintain the extra indirection from class indicator value to actual value. |
| Use the
DynamicSchemaFactory performance, validation | Kodo JDO's default schema factory reflects on the database schema to validate that object-relational mapping information is valid when a persistent class is first used. This can be a slow process on some databases. Though the database reflection is only performed once for each class, switching the kodo.jdbc.SchemaFactory configuration property to dynamic can reduce the warm-up time for your application. Note, however, that the dynamic schema factory does not perform any validation and cannot detect foreign key constraints (and is therefore not able to re-order statements to satisfy any non-deferrable non-nullable foreign keys). |
| Do not use XA transactions performance, scalability | XA transactions
can be orders of magnitude slower than standard
transactions. Unless distributed transaction functionality
is required by your application, use standard transactions.
Recall that XA transactions are distinct from managed transactions -- managed transaction services such as that provided by EJB declarative transactions can be used both with XA and non-XA transactions. XA transactions should only be used when a given business transaction involves multiple different transactional resources (an Oracle database and an IBM transactional message queue, for example). |
| Use Sets
instead of List/Collections
performance, scalability | There is a small amount of extra overhead for Kodo to maintain collections where each element is not guaranteed to be unique. If your application does not require duplicates for a collection, you should always declare your fields to be of type Set, SortedSet, HashSet, or TreeSet. |
| Use JDOQL parameters instead of
encoding search data in filter strings performance |
If your queries depend on parameter data only known at
runtime, you should use JDOQL parameters rather than
dynamically building different query filters. Kodo
performs aggressive caching of both query compilation
data and PreparedStatements, and the
effectiveness of both of these caches are diminished if
multiple query filters are used where a single one could
have been used.
Example 16.3. Appropriate use of JDOQL parameters
public Person findPerson (String firstName, String lastName)
{
PersistenceManager pm = factory.getPersistenceManager ();
try
{
// good -- the query uses parameters
Query query = pm.newQuery (Person.class);
query.setFilter ("firstName == fname && lastName == lname");
query.declareParameters ("String fname, String lname");
Collection res = (Collection) query.execute (firstName, lastName);
Iterator itr = res.iterator ();
return (itr.hasNext ()) ? (Person) itr.next () : null;
}
finally
{
if (query != null)
query.closeAll ();
pm.close ();
}
}
Example 16.4. Inappropriate use of JDOQL parameters
public Person findPerson (String firstName, String lastName)
{
PersistenceManager pm = factory.getPersistenceManager ();
Query query = null;
try
{
// bad -- the query encodes parameters directly in filter
query = pm.newQuery (Person.class);
query.setFilter ("firstName == \"" + firstName
+ "\" && lastName == \"" + lastName + "\"");
query.declareParameters ("String fname, String lname");
Collection res = (Collection) query.execute ();
Iterator itr = res.iterator ();
return (itr.hasNext ()) ? (Person) itr.next () : null;
}
finally
{
if (query != null)
query.closeAll ();
pm.close ();
}
}
|
| Tune your fetch groups
appropriately performance, scalability |
The fetch groups
used when loading an object control how much data is
eagerly loaded, and by extension, which fields must be
lazily loaded at a future time. The ideal fetch group
configuration loads all the data that is needed in one
fetch, and no extra fields -- this minimizes both the
amount of data transferred from the database, and the
number of trips to the database.
If extra fields are specified in the fetch groups (in particular, large fields such as binary data, or relations to other persistence-capable objects), then network overhead (for the extra data) and database processing (for any necessary additional joins) will hurt your application's performance. If too few fields are specified in the fetch groups, then Kodo will have to make additional trips to the database to load additional fields as necessary. |
| Use eager fetching performance, scalability | Using eager fetching when traversing relations for each instance in a large collection of results can be sped up by considerably by employing eager fetching (available in Kodo JDO Performance Pack). |