Basic Directory Server Sizing Example: Disk and Memory Requirements

This section provides an example that shows initial steps in sizing Directory Server disk and memory requirements for deployment. The system used for this example was selected by chance and because it had sufficient processing power and memory to complete the sizing tasks quickly. It does not necessarily represent a recommended system for production use. You can it however to gain insight into how much memory and disk space might be required for production systems.

System Characteristics

The following system information was observed using the Solaris Management Console (smc).

• 2 AMD64 CPUs (2.2 gigahertz)

• Solaris 10 Operating System

• 4 gigabytes physical memory

• 40 gigabytes swap

• Physical memory in use before Directory Server installation: 700 megabytes

• Physical memory free before Directory Server installation: 3400 megabytes

• CPU usage: 1%

• Local disk: one partition formatted UFS with logging

For this example, the system was dedicated to Directory Server. No other user was logged in, and only the default system processes were running.

Preparing a Directory Server Instance

Unpack the zip distribution to installDirectory Server Enterprise Edition software on local disk space.

For detailed information, see Installing Directory Server Enterprise Edition Using Zip Distribution in Oracle Directory Server Enterprise Edition Installation Guide.

For convenience set environment variables as shown.

$ export PATH=/local/dsee7/bin:${PATH}
$ export DIRSERV_PORT=1389
$ export LDAP_ADMIN_PWF=~/.pwd

After installing the software and setting environment variables, create a Directory Server instance using default ports for LDAP and LDAPS, respectively.

$ dsadm create -p 1389 -P 1636 /local/dsInst
Choose the Directory Manager password:
Confirm the Directory Manager password:
$ du -hs /local/dsInst
610K   /local/dsInst

Until you create a suffix, the Directory Server instance uses less than one megabyte of disk space.

$ dsadm start /local/dsInst
Server started: pid=8046
$ dsconf create-suffix dc=example,dc=com
Certificate "CN=hostname, CN=1636, CN=Directory Server,
 O=Sun Microsystems" presented by the server is not trusted.
Type "Y" to accept, "y" to accept just once, "n" to refuse, "d" for more
 details: Y
$ du -hs /local/dsInst
53M   /local/dsInst

For this example, make no additional changes to the default Directory Server configuration except those shown explicitly.

Populating the Suffix With 10,000 Sample Directory Entries

Using the makeldif command with the example files, you can create sample LDIF files from one kilobyte to one megabyte in size. See To Load Sample Data in Directory Server Instance in Oracle Directory Server Enterprise Edition Administration Guide for an example showing how to use the makeldif command.

The entries in these files are smaller than you would expect in a real deployment.

$ du -h /var/tmp/*
 57M   /var/tmp/100k.ldif
 5.7M   /var/tmp/10k.ldif
 573M   /var/tmp/1M.ldif

An example entry from these files is shown in the following LDIF.

dn: uid=Aartjan.Aalders,ou=People,dc=example,dc=com
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
givenName: Aartjan
sn: Aalders
cn: Aartjan Aalders
initials: AA
uid: Aartjan.Aalders
mail: Aartjan.Aalders@example.com
userPassword: trj49xeq
telephoneNumber: 935-748-6699
homePhone: 347-586-0252
pager: 906-399-8417
mobile: 452-898-9034
employeeNumber: 1000004
street: 64197 Broadway Street
l: Lawton
st: IN
postalCode: 57924
postalAddress: Aartjan Aalders$64197 Broadway Street$Lawton, IN  57924
description: This is the description for Aartjan Aalders.

Begin sizing by importing the content of 10k.ldif, which occupies 5.7 megabytes on disk.

$ dsadm stop /local/dsInst
Server stopped
$ dsadm import -i /local/dsInst /var/tmp/10k.ldif dc=example,dc=com
…

With default indexing the content of 10k.ldif increases the size of the instance files by 72 megabytes - 53 megabytes, or 19 megabytes.

$ du -hs /local/dsInst
 72M   /local/dsInst

If you index five more attributes, size increases by about seven megabytes.

$ dsconf create-index dc=example,dc=com employeeNumber street st \ postalCode description
$ dsconf reindex dc=example,dc=com
…
## example: Finished indexing.

Task completed (slapd exit code: 0).
$ du -hs /local/dsInst
 79M   /local/dsInst

Observing memory size with the default cache settings, and nothing loaded from the suffix into entry cache yet, the server process occupies approximately 170 megabytes of memory with a heap size of about 56 megabytes.

$ dsadm start /local/dsInst
Server started: pid=8482
$ pmap -x 8482
…
         Address     Kbytes        RSS       Anon     Locked Mode   Mapped File
0000000000437000      61348      55632      55380          - rw---    [ heap ]
…
---------------- ---------- ---------- ---------- ----------
        total Kb     178444     172604      76532          -

As you then prime the cache and examine output from the pmap command again, the heap grows by about 10 megabytes, and so does the total size of the process.

$ ldapsearch -D cn=Directory\ Manager -w - -p 1389 -b dc=example,dc=com \ objectclass=\* > /dev/null
Enter bind password:
$ pmap -x 8482
…
         Address     Kbytes        RSS       Anon     Locked Mode   Mapped File
…
0000000000437000      70564      65268      65024          - rw---    [ heap ]
…
---------------- ---------- ---------- ---------- ----------
        total Kb     187692     182272      86224          -

The numbers are comparable for default indexing.

$ dsconf delete-index dc=example,dc=com employeeNumber street st \ postalCode description
$ dsconf reindex dc=example,dc=com
…
## example: Finished indexing.

Task completed (slapd exit code: 0).
$ dsadm stop /local/dsInst
 Server stopped
$ dsadm start /local/dsInst
 Server started: pid=8541
$ ldapsearch -D cn=Directory\ Manager -w - -p 1389 -b dc=example,dc=com \ objectclass=\* > /dev/null
Enter bind password:
$ pmap -x 8541
…
         Address     Kbytes        RSS       Anon     Locked Mode   Mapped File
…
0000000000437000      70564      65248      65004          - rw---    [ heap ]
…
---------------- ---------- ---------- ---------- ----------
        total Kb     187680     182240      86192          -

For only 10,000 entries, do not change the default cache sizes.

$ dsconf get-server-prop | grep cache
db-cache-size                      :  32M
import-cache-size                  :  64M
$ dsconf get-suffix-prop dc=example,dc=com | grep entry-cache-size
entry-cache-size                   :  10M

The small default entry cache was no doubt filled completely after priming, even with only 10,000 entries. To see the size for a full entry cache, set a large entry cache size, import the data again, and prime the cache.

$ dsconf set-suffix-prop dc=example,dc=com entry-cache-size:2G
$ dsadm stop /local/dsInst
Server stopped
$ dsadm import -i /local/dsInst /var/tmp/10k.ldif dc=example,dc=com
…
$ dsadm start /local/dsInst
Server started: pid=8806
$ ldapsearch -D cn=Directory\ Manager -w - -p 1389 -b dc=example,dc=com \ objectclass=\* > /dev/null
Enter bind password:
$ pmap -x 8806
…
         Address     Kbytes        RSS       Anon     Locked Mode   Mapped File
…
0000000000437000     116644     109996     109780          - rw---    [ heap ]

Here 10,000 entries occupy approximately 55 megabytes of entry cache memory (110 - 55).

Populating the Suffix With 100,000 Sample Directory Entries

As you move to 100,000 entries, you have more directory data to fit into database and entry caches. Initially, import 100,000 entries and examine the size required on disk for this volume of directory data.

$ dsadm import -i /local/dsInst /var/tmp/100k.ldif dc=example,dc=com
…
$ du -hs /local/dsInst
 196M   /local/dsInst

Directory data contained in the database for our example suffix, dc=example,dc=com, now occupy about 142 megabytes.

$ du -hs /local/dsInst/db/example/
 142M   /local/dsInst/db/example

You can increase the size of the database cache to hold this content. If you expect the volume of directory data to grow over time, you can set the database cache larger than currently necessary. You can also set the entry cache size larger than necessary. Entry cache grows as the server responds to client requests, unlike the database cache, which is allocated at startup.

$ dsconf set-server-prop db-cache-size:200M
$ dsconf set-suffix-prop dc=example,dc=com entry-cache-size:2G

$ dsadm stop /local/dsInst
 Server stopped
$ dsadm start /local/dsInst
 Server started: pid=8640
$ pmap -x 8640
…
         Address     Kbytes        RSS       Anon     Locked Mode   Mapped File
…
0000000000437000      61348      55404      55148          - rw---    [ heap ]
…
---------------- ---------- ---------- ---------- ----------
        total Kb     491984     485736     174620          -

This shows the server instance has a relatively small heap at startup, but that the database cache memory has been allocated. Process size is nearing half a gigabyte.

$ ldapsearch -D cn=Directory\ Manager -w - -p 1389 -b dc=example,dc=com \ objectclass=\* > /dev/null
Enter bind password:
$ pmap -x 8640
…
         Address     Kbytes        RSS       Anon     Locked Mode   Mapped File
…
0000000000437000     610212     604064     603840          - rw---    [ heap ]
…
---------------- ---------- ---------- ---------- ----------
        total Kb    1040880    1034428     723360          -

Heap size now reflects the entry cache being filled. It has increased by roughly 550 megabytes for 100,000 small directory entries, whose LDIF occupied 57 megabytes on disk.

With five extra indexes, the process size is about the same. The database cache size has not changed.

$ dsconf create-index dc=example,dc=com employeeNumber street st \ postalCode description
$ dsadm stop /local/dsInst
 Server stopped
$ dsadm import -i /local/dsInst /var/tmp/100k.ldif dc=example,dc=com
…
$ dsadm start /local/dsInst
 Server started: pid=8762
$ ldapsearch -D cn=Directory\ Manager -w - -p 1389 -b dc=example,dc=com \ objectclass=\* > /dev/null
Enter bind password:
$ pmap -x 8762
…
         Address     Kbytes        RSS       Anon     Locked Mode   Mapped File
…
0000000000437000     610212     603832     603612          - rw---    [ heap ]
…
---------------- ---------- ---------- ---------- ----------
        total Kb    1040876    1034192     723128          -

The database is somewhat larger, however. The additional indexes increased the size of the database from 142 megabytes to 163 megabytes.

$ du -hs /local/dsInst/db/example/
 163M   /local/dsInst/db/example

Populating the Suffix With 1,000,000 Sample Directory Entries

As you move from 100,000 entries to 1,000,000 entries, you no longer have enough space on a system with 4 gigabytes of physical memory to include all entries in the entry cache. You can begin by importing the data and examining the size it occupies on disk.

$ dsadm import -i /local/dsInst /var/tmp/1M.ldif dc=example,dc=com
…
$ du -hs /local/dsInst/db/example/
 1.3G   /local/dsInst/db/example

Assuming you expect approximately 25% growth in directory data size during the lifetime of the instance, set the database cache size to 1700 megabytes.

$ dsadm start /local/dsInst
Server started: pid=9060
$ dsconf set-server-prop db-cache-size:1700M
$ dsadm stop /local/dsInst
Server stopped
$ dsadm start /local/dsInst
Server started: pid=9118
$ pmap -x 9118
…
         Address     Kbytes        RSS       Anon     Locked Mode   Mapped File
…
0000000000437000      65508      55700      55452          - rw---    [ heap ]
…
---------------- ---------- ---------- ---------- ----------
        total Kb    1882448    1034180      76616          -

Given a database cache this large and only 4 gigabytes of physical memory, you cannot fit more than a fraction of entries into the entry cache for the suffix. Here, set entry cache size to one gigabyte, and then prime the cache to see the change in the process heap size.

$ dsconf set-suffix-prop dc=example,dc=com entry-cache-size:1G
$ ldapsearch -D cn=Directory\ Manager -w - -p 1389 -b dc=example,dc=com \ objectclass=\* > /dev/null
Enter bind password:
$ pmap -x 9118
…
         Address     Kbytes        RSS       Anon     Locked Mode   Mapped File
…
0000000000437000    1016868    1009852    1009612          - rw---    [ heap ]
…
---------------- ---------- ---------- ---------- ----------
        total Kb    2883268    2477064    1080076          -

Total process size is over 2.8 gigabytes.

$ prstat -p 9118
   PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU PROCESS/NLWP
  9118 myuser   2816M 2374M sleep   59    0   0:03:26 0.5% ns-slapd/42

Extrapolating from earlier entry cache sizes, you can expect to use 5.5 or 6 gigabytes for entry cache alone if you had enough physical memory.

Examining the directory database size with five additional indexes, you find adding indexes has increased the size of the database by about 200 megabytes.

$ dsconf create-index dc=example,dc=com employeeNumber street st \ postalCode description
$ dsadm stop /local/dsInst
Server stopped
$ dsadm import -i /local/dsInst /var/tmp/1M.ldif dc=example,dc=com
…
$ du -hs /local/dsInst/db/example
 1.5G   /local/dsInst/db/example

Summary of Observations

Table 6-3 records what was observed in this example. It includes neither server process size, nor default database cache file size.

Table 6-3 Sizing Summary

¹The suffix has been created, but is empty.

In an actual deployment, you may have significantly larger entries and more indexes. Do your own empirical testing and tuning before ordering hardware.