Go to main content

Oracle® VM Server for SPARC 3.6 Administration Guide

Exit Print View

Updated: September 2019
 
 

Using Memory Dynamic Reconfiguration

Memory dynamic reconfiguration (DR) is capacity-based and enables you to add an arbitrary amount of memory to or remove it from an active logical domain.

    The requirements and restrictions for using the memory DR feature are as follows:

  • You can perform memory DR operations on any domain. However, only a single memory DR operation can be in progress on a domain at a given time.

  • The memory DR feature enforces 256-Mbyte alignment on the address and size of the memory involved in a given operation. See Memory Alignment.

If the memory of a domain cannot be reconfigured by using a memory DR operation, the domain must be stopped before the memory can be reconfigured. If the domain is the control domain, you must first initiate a delayed reconfiguration.

Under certain circumstances, the Logical Domains Manager rounds up the requested memory allocation to either the next largest 8-Kbyte or 4-Mbyte multiple. The following example shows sample output of the ldm list-domain -l command, where the constraint value is smaller than the actual allocated size:

Memory:
          Constraints: 1965 M
          raddr          paddr5          size
          0x1000000      0x291000000     1968M

Adding Memory

If a domain is active, you can use the ldm add-memory command to dynamically add memory to the domain. The ldm set-memory command can also dynamically add memory if the specified memory size is greater than the current memory size of the domain.

Removing Memory

If a domain is active, you can use the ldm remove-memory command to dynamically remove memory from the domain. The ldm set-memory command can also dynamically remove memory if the specified memory size is smaller than the current memory size of the domain.

Memory removal can be a long-running operation. You can track the progress of an ldm remove-memory command by running the ldm list -l command for the specified domain.

You can cancel a removal request that is in progress by interrupting the ldm remove-memory command (by pressing Control-C) or by issuing the ldm cancel-operation memdr command. If you cancel a memory removal request, only the outstanding portion of the removal request is affected; namely, the amount of memory still to be removed from the domain.

Partial Memory DR Requests

A request to dynamically add memory to or remove memory from a domain might only be partially fulfilled. This result depends on the availability of suitable memory to add or remove, respectively.


Note - Memory is cleared after it is removed from a domain and before it is added to another domain.

Memory Reconfiguration of the Control Domain

You can use the memory DR feature to reconfigure the memory of the control domain. If a memory DR request cannot be performed on the control domain, you must first initiate a delayed reconfiguration.

Using memory DR might not be appropriate for removing large amounts of memory from an active domain because memory DR operations might be long running. In particular, during the initial configuration of the system, you should use delayed reconfiguration to decrease the memory in the control domain.

Decrease the Control Domain's Memory

Use a delayed reconfiguration instead of a memory DR to decrease the control domain's memory from an initial factory-default configuration. In such a case, the control domain owns all of the host system's memory. The memory DR feature is not well suited for this purpose because an active domain is not guaranteed to add, or more typically give up, all of the requested memory. Rather, the OS running in that domain makes a best effort to fulfill the request. In addition, memory removal can be a long-running operation. These issues are amplified when large memory operations are involved, as is the case for the initial decrease of the control domain's memory.

For these reasons, use a delayed reconfiguration by following these steps:

  1. Use the ldm start-reconf primary command to put the control domain in delayed reconfiguration mode.

  2. Partition the host system's resources that are owned by the control domain, as necessary.

  3. Use the ldm cancel-reconf command to undo the operations in Step 2, if necessary, and start over.

  4. Reboot the control domain to make the reconfiguration changes take effect.

Dynamic and Delayed Reconfiguration

If a delayed reconfiguration is pending in the control domain, a memory reconfiguration request is rejected for any other domain. If a delayed reconfiguration is not pending in the control domain, a memory reconfiguration request is rejected for any domain that does not support memory DR. For those domains, the request is converted to a delayed reconfiguration request.

Memory Alignment

  • Dynamic addition and removal. The address and size of a memory block are 256-Mbyte-aligned for dynamic addition and dynamic removal. The minimum operation size is 256 Mbytes.

    A nonaligned request or a removal request that is larger than the bound size is rejected.

      Use the following commands to adjust memory allocations:

    • ldm add-memory. If you specify the –-auto-adj option with this command, the amount of memory to be added is 256-Mbyte-aligned, which might increase the amount of memory actually added to the domain.

    • ldm remove-memory. If you specify the –-auto-adj option with this command, the amount of memory to be removed is 256-Mbyte-aligned, which might decrease the amount of memory actually removed from the domain.

    • ldm set-memory. This command is treated as an addition or a removal operation. If you specify the –-auto-adj option, the amount of memory to be added or removed is 256-Mbyte-aligned as previously described. Note that this alignment might increase the resulting memory size of the domain.

  • Delayed reconfiguration. The address and size of a memory block are 256-Mbyte-aligned. If you make a nonaligned request, the request is rounded up to be 256-Mbyte-aligned.

Memory DR Examples

The following examples show how to perform memory DR operations. For information about the related CLI commands, see the ldm(8) man page.

Example 71  Memory DR Operations on Active Domains

This example shows how to dynamically add memory to and remove it from an active domain, ldom1.

The ldm list output shows the memory for each domain in the Memory field.

primary# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active     -n-cv-  SP      4     27392M   0.4%  1d 22h 53m
ldom1            active     -n----  5000    2     2G       0.4%  1d 1h 23m
ldom2            bound      ------  5001    2     200M

The following ldm add-mem command exits with an error because you must specify memory in multiples of 256 Mbytes. The next ldm add-mem command uses the –-auto-adj option so that even though you specify 200M as the amount of memory to add, the amount is rounded up to 256 Mbytes.

primary# ldm add-mem 200M ldom1
The size of memory must be a multiple of 256MB.

primary# ldm add-mem --auto-adj 200M ldom1
Adjusting request size to 256M.
The ldom1 domain has been allocated 56M more memory
than requested because of memory alignment constraints.

primary# ldm list
NAME            STATE   FLAGS   CONS    VCPU    MEMORY  UTIL    UPTIME
primary         active  -n-cv-  SP      4       27392M  5.0%    8m
ldom1           active  -n----  5000    2       2304M   0.5%    1m
ldom2           bound   ------  5001    2       200M

The ldm remove-mem command exits with an error because you must specify memory in multiples of 256 Mbytes. When you add the –-auto-adj option to the same command, the memory removal succeeds because the amount of memory is rounded down to the next 256-Mbyte boundary.

primary# ldm remove-mem --auto-adj 300M ldom1
Adjusting requested size to 256M.
The ldom1 domain has been allocated 44M more memory
than requested because of memory alignment constraints.

primary# ldm list
NAME            STATE   FLAGS   CONS    VCPU    MEMORY  UTIL    UPTIME
primary         active  -n-cv-  SP      4       27392M  0.3%    8m
ldom1           active  -n----  5000    2       2G      0.2%    2m
ldom2           bound   ------  5001    2       200M
Example 72  Memory DR Operations on Bound Domains

This example shows how to add memory to and remove it from a bound domain, ldom2.

The ldm list output shows the memory for each domain in the Memory field. The first ldm add-mem command adds 100 Mbytes of memory to the ldom2 domain. The next ldm add-mem command specifies the –-auto-adj option, which causes an additional 112 Mbytes of memory to be dynamically added to ldom2.

The ldm remove-mem command dynamically removes 100 Mbytes from the ldom2 domain. If you specify the –-auto-adj option to the same command to remove 300 Mbytes of memory, the amount of memory is rounded down to the next 256-Mbyte boundary.

primary# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active     -n-cv-  SP      4     27392M   0.4%  1d 22h 53m
ldom1            active     -n----  5000    2     2G       0.4%  1d 1h 23m
ldom2            bound      ------  5001    2     200M          

primary# ldm add-mem 100M ldom2

primary# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active     -n-cv-  SP      4     27392M   0.5%  1d 22h 54m
ldom1            active     -n----  5000    2     2G       0.2%  1d 1h 25m
ldom2            bound      ------  5001    2     300M       

primary# ldm add-mem --auto-adj 100M ldom2
Adjusting request size to 256M.
The ldom2 domain has been allocated 112M more memory
than requested because of memory alignment constraints.

primary# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active     -n-cv-  SP      4     27392M   0.4%  1d 22h 55m
ldom1            active     -n----  5000    2     2G       0.5%  1d 1h 25m
ldom2            bound      ------  5001    2     512M          

primary# ldm remove-mem 100M ldom2
primary# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active     -n-cv-  SP      4     27392M   3.3%  1d 22h 55m
ldom1            active     -n----  5000    2     2G       0.2%  1d 1h 25m
ldom2            bound      ------  5001    2     412M          

primary# ldm remove-mem --auto-adj 300M ldom2
Adjusting request size to 256M.
The ldom2 domain has been allocated 144M more memory
than requested because of memory alignment constraints.

primary# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active     -n-cv-  SP      4     27392M   0.5%  1d 22h 55m
ldom1            active     -n----  5000    2     2G       0.2%  1d 1h 26m
ldom2            bound      ------  5001    2     256M
Example 73  Setting Domain Memory Sizes

This example shows how to use the ldm set-memory command to add memory to and remove it from a domain.

The ldm list output shows the memory for each domain in the Memory field.

primary# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active     -n-cv-  SP      4     27392M   0.5%  1d 22h 55m
ldom1            active     -n----  5000    2     2G       0.2%  1d 1h 26m
ldom2            bound      ------  5001    2     256M

The following ldm set-mem command attempts to set the primary domain's size to 3400 Mbytes. The resulting error states that the specified value is not on a 256-Mbyte boundary. Adding the –-auto-adj option to the same command enables you to successfully remove some memory and stay on the 256-Mbyte boundary. This command also issues a warning to state that not all of the requested memory could be removed as the domain is using that memory.

primary# ldm set-mem 3400M primary
An ldm set-mem 3400M command would remove 23992MB, which is not a multiple
of 256MB. Instead, run ldm rm-mem 23808MB to ensure a 256MB alignment.

primary# ldm set-mem --auto-adj 3400M primary
Adjusting request size to 3.4G.
The primary domain has been allocated 184M more memory
than requested because of memory alignment constraints.
Only 9472M of memory could be removed from the primary domain
because the rest of the memory is in use.

The next ldm set-mem command sets the memory size of the ldom2 domain, which is in the bound state, to 690 Mbytes. If you add the –-auto-adj option to the same command, an additional 78 Mbytes of memory is dynamically added to ldom2 to stay on a 256-Mbyte boundary.

primary# ldm set-mem 690M ldom2
primary# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active     -n-cv-  SP      4     17920M   0.5%  1d 22h 56m
ldom1            active     -n----  5000    2     2G       0.6%  1d 1h 27m
ldom2            bound      ------  5001    2     690M          

primary# ldm set-mem --auto-adj 690M ldom2
Adjusting request size to 256M.
The ldom2 domain has been allocated 78M more memory
than requested because of memory alignment constraints.

primary# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active     -n-cv-  SP      4     17920M   2.1%  1d 22h 57m
ldom1            active     -n----  5000    2     2G       0.2%  1d 1h 27m
ldom2            bound      ------  5001    2     768M