Memory dynamic reconfiguration (DR) is capacity-based and enables you to add an arbitrary amount of memory to or remove it from an active logical domain.
The requirements and restrictions for using the memory DR feature are as follows:
You can perform memory DR operations on any domain. However, only a single memory DR operation can be in progress on a domain at a given time.
The memory DR feature enforces 256-Mbyte alignment on the address and size of the memory involved in a given operation. See Memory Alignment.
Unaligned memory in the free memory pool cannot be assigned to a domain by using the memory DR feature. See Adding Unaligned Memory.
If the memory of a domain cannot be reconfigured by using a memory DR operation, the domain must be stopped before the memory can be reconfigured. If the domain is the control domain, you must first initiate a delayed reconfiguration.
Under certain circumstances, the Logical Domains Manager rounds up the requested memory allocation to either the next largest 8-Kbyte or 4-Mbyte multiple. The following example shows sample output of the ldm list-domain -l command, where the constraint value is smaller than the actual allocated size:
Memory: Constraints: 1965 M raddr paddr5 size 0x1000000 0x291000000 1968M
If a domain is active, you can use the ldm add-memory command to dynamically add memory to the domain. The ldm set-memory command can also dynamically add memory if the specified memory size is greater than the current memory size of the domain.
If a domain is active, you can use the ldm remove-memory command to dynamically remove memory from the domain. The ldm set-memory command can also dynamically remove memory if the specified memory size is smaller than the current memory size of the domain.
Memory removal can be a long-running operation. You can track the progress of an ldm remove-memory command by running the ldm list -l command for the specified domain.
You can cancel a removal request that is in progress by interrupting the ldm remove-memory command (by pressing Control-C) or by issuing the ldm cancel-operation memdr command. If you cancel a memory removal request, only the outstanding portion of the removal request is affected; namely, the amount of memory still to be removed from the domain.
A request to dynamically add memory to or remove memory from a domain might only be partially fulfilled. This result depends on the availability of suitable memory to add or remove, respectively.
You can use the memory DR feature to reconfigure the memory of the control domain. If a memory DR request cannot be performed on the control domain, you must first initiate a delayed reconfiguration.
Using memory DR might not be appropriate for removing large amounts of memory from an active domain because memory DR operations might be long running. In particular, during the initial configuration of the system, you should use delayed reconfiguration to decrease the memory in the control domain.
Use a delayed reconfiguration instead of a memory DR to decrease the control domain's memory from an initial factory default configuration. In such a case, the control domain owns all of the host system's memory. The memory DR feature is not well suited for this purpose because an active domain is not guaranteed to add, or more typically give up, all of the requested memory. Rather, the OS running in that domain makes a best effort to fulfill the request. In addition, memory removal can be a long-running operation. These issues are amplified when large memory operations are involved, as is the case for the initial decrease of the control domain's memory.
For these reasons, use a delayed reconfiguration by following these steps:
Use the ldm start-reconf primary command to put the control domain in delayed reconfiguration mode.
Partition the host system's resources that are owned by the control domain, as necessary.
Use the ldm cancel-reconf command to undo the operations in Step 2, if necessary, and start over.
Reboot the control domain to make the reconfiguration changes take effect.
If a delayed reconfiguration is pending in the control domain, a memory reconfiguration request is rejected for any other domain. If a delayed reconfiguration is not pending in the control domain, a memory reconfiguration request is rejected for any domain that does not support memory DR. For those domains, the request is converted to a delayed reconfiguration request.
Memory reconfiguration requests have different alignment requirements that depend on the state of the domain to which the request is applied.
Dynamic addition and removal. The address and size of a memory block are 256-Mbyte-aligned for dynamic addition and dynamic removal. The minimum operation size is 256 Mbytes.
A nonaligned request or a removal request that is larger than the bound size is rejected.
Use the following commands to adjust memory allocations:
ldm add-memory. If you specify the –-auto-adj option with this command, the amount of memory to be added is 256-Mbyte-aligned, which might increase the amount of memory actually added to the domain.
ldm remove-memory. If you specify the –-auto-adj option with this command, the amount of memory to be removed is 256-Mbyte-aligned, which might decrease the amount of memory actually removed from the domain.
ldm set-memory. This command is treated as an addition or a removal operation. If you specify the –-auto-adj option, the amount of memory to be added or removed is 256-Mbyte-aligned as previously described. Note that this alignment might increase the resulting memory size of the domain.
Delayed reconfiguration. The address and size of a memory block are 4-Mbyte-aligned. If you make a nonaligned request, the request is rounded up to be 4-Mbyte-aligned.
The address and size of a memory block are 4-Mbyte-aligned for bound domains. If you make a nonaligned request, the request is rounded up to be 4-Mbyte-aligned. Therefore, the resulting memory size of the domain might be more than you specified.
For the ldm add-memory, ldm set-memory, and ldm remove-memory commands, the –-auto-adj option rounds up the size of the resulting memory to be 256-Mbyte-aligned. Therefore, the resulting memory might be more than you specified.
For the ldm add-memory, ldm set-memory, and ldm remove-memory commands, the –-auto-adj option rounds up the size of the resulting memory to be 256-Mbyte-aligned. There is no alignment requirement for an inactive domain. The restrictions described in Memory Alignment for Bound Domains take effect after such a domain is bound.
The memory DR feature enforces 256-Mbyte memory alignment on the address and size of the memory that is dynamically added to or removed from an active domain. Therefore, any unaligned memory in an active domain cannot be removed by using memory DR.
Also, any unaligned memory in the free memory pool cannot be added to an active domain by using memory DR.
After all the aligned memory has been allocated, you can use the ldm add-memory command to add the remaining unaligned memory to a bound or inactive domain. You can also use this command to add the remaining unaligned memory to the control domain by means of a delayed reconfiguration operation.
The following example shows how to add the two remaining 128-Mbyte memory blocks to the primary and ldom1 domains. The ldom1 domain is in the bound state.
The following command initiates a delayed reconfiguration operation on the control domain.
primary# ldm start-reconf primary Initiating a delayed reconfiguration operation on the primary domain. All configuration changes for other domains are disabled until the primary domain reboots, at which time the new configuration for the primary domain also takes effect.
The following command adds one of the 128-Mbyte memory blocks to the control domain.
primary# ldm add-memory 128M primary ------------------------------------------------------------------------------ Notice: The primary domain is in the process of a delayed reconfiguration. Any changes made to the primary domain will only take effect after it reboots. ------------------------------------------------------------------------------ primary# ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -ndcv- SP 8 2688M 0.1% 23d 8h 8m primary# ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 8 2560M 0.5% 23d 8h 9m ldom1 bound ------ 5000 1 524M
The following command adds the other 128-Mbyte memory block to the ldom1 domain.
primary# ldm add-mem 128M ldom1 primary# ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 8 2560M 0.1% 23d 8h 9m ldom1 bound ------ 5000 1 652M
The following examples show how to perform memory DR operations. For information about the related CLI commands, see the ldm(1M) man page.
Example 60 Memory DR Operations on Active DomainsThis example shows how to dynamically add memory to and remove it from an active domain, ldom1.
The ldm list output shows the memory for each domain in the Memory field.
primary# ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 4 27392M 0.4% 1d 22h 53m ldom1 active -n---- 5000 2 2G 0.4% 1d 1h 23m ldom2 bound ------ 5001 2 200M
The following ldm add-mem command exits with an error because you must specify memory in multiples of 256 Mbytes. The next ldm add-mem command uses the –-auto-adj option so that even though you specify 200M as the amount of memory to add, the amount is rounded up to 256 Mbytes.
primary# ldm add-mem 200M ldom1 The size of memory must be a multiple of 256MB. primary# ldm add-mem --auto-adj 200M ldom1 Adjusting request size to 256M. The ldom1 domain has been allocated 56M more memory than requested because of memory alignment constraints. primary# ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 4 27392M 5.0% 8m ldom1 active -n---- 5000 2 2304M 0.5% 1m ldom2 bound ------ 5001 2 200M
The ldm rm-mem command exits with an error because you must specify memory in multiples of 256 Mbytes. When you add the –-auto-adj option to the same command, the memory removal succeeds because the amount of memory is rounded down to the next 256-Mbyte boundary.
primary# ldm rm-mem --auto-adj 300M ldom1 Adjusting requested size to 256M. The ldom1 domain has been allocated 44M more memory than requested because of memory alignment constraints. primary# ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 4 27392M 0.3% 8m ldom1 active -n---- 5000 2 2G 0.2% 2m ldom2 bound ------ 5001 2 200MExample 61 Memory DR Operations on Bound Domains
This example shows how to add memory to and remove it from a bound domain, ldom2.
The ldm list output shows the memory for each domain in the Memory field. The first ldm add-mem command adds 100 Mbytes of memory to the ldom2 domain. The next ldm add-mem command specifies the –-auto-adj option, which causes an additional 112 Mbytes of memory to be dynamically added to ldom2.
The ldm rm-mem command dynamically removes 100 Mbytes from the ldom2 domain. If you specify the –-auto-adj option to the same command to remove 300 Mbytes of memory, the amount of memory is rounded down to the next 256-Mbyte boundary.
primary# ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 4 27392M 0.4% 1d 22h 53m ldom1 active -n---- 5000 2 2G 0.4% 1d 1h 23m ldom2 bound ------ 5001 2 200M primary# ldm add-mem 100M ldom2 primary# ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 4 27392M 0.5% 1d 22h 54m ldom1 active -n---- 5000 2 2G 0.2% 1d 1h 25m ldom2 bound ------ 5001 2 300M primary# ldm add-mem --auto-adj 100M ldom2 Adjusting request size to 256M. The ldom2 domain has been allocated 112M more memory than requested because of memory alignment constraints. primary# ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 4 27392M 0.4% 1d 22h 55m ldom1 active -n---- 5000 2 2G 0.5% 1d 1h 25m ldom2 bound ------ 5001 2 512M primary# ldm rm-mem 100M ldom2 primary# ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 4 27392M 3.3% 1d 22h 55m ldom1 active -n---- 5000 2 2G 0.2% 1d 1h 25m ldom2 bound ------ 5001 2 412M primary# ldm rm-mem --auto-adj 300M ldom2 Adjusting request size to 256M. The ldom2 domain has been allocated 144M more memory than requested because of memory alignment constraints. primary# ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 4 27392M 0.5% 1d 22h 55m ldom1 active -n---- 5000 2 2G 0.2% 1d 1h 26m ldom2 bound ------ 5001 2 256MExample 62 Setting Domain Memory Sizes
This example shows how to use the ldm set-memory command to add memory to and remove it from a domain.
The ldm list output shows the memory for each domain in the Memory field.
primary# ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 4 27392M 0.5% 1d 22h 55m ldom1 active -n---- 5000 2 2G 0.2% 1d 1h 26m ldom2 bound ------ 5001 2 256M
The following ldm set-mem command attempts to set the primary domain's size to 3400 Mbytes. The resulting error states that the specified value is not on a 256-Mbyte boundary. Adding the –-auto-adj option to the same command enables you to successfully remove some memory and stay on the 256-Mbyte boundary. This command also issues a warning to state that not all of the requested memory could be removed as the domain is using that memory.
primary# ldm set-mem 3400M primary An ldm set-mem 3400M command would remove 23992MB, which is not a multiple of 256MB. Instead, run ldm rm-mem 23808MB to ensure a 256MB alignment. primary# ldm set-mem --auto-adj 3400M primary Adjusting request size to 3.4G. The primary domain has been allocated 184M more memory than requested because of memory alignment constraints. Only 9472M of memory could be removed from the primary domain because the rest of the memory is in use.
The next ldm set-mem command sets the memory size of the ldom2 domain, which is in the bound state, to 690 Mbytes. If you add the –-auto-adj option to the same command, an additional 78 Mbytes of memory is dynamically added to ldom2 to stay on a 256-Mbyte boundary.
primary# ldm set-mem 690M ldom2 primary# ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 4 17920M 0.5% 1d 22h 56m ldom1 active -n---- 5000 2 2G 0.6% 1d 1h 27m ldom2 bound ------ 5001 2 690M primary# ldm set-mem --auto-adj 690M ldom2 Adjusting request size to 256M. The ldom2 domain has been allocated 78M more memory than requested because of memory alignment constraints. primary# ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv- SP 4 17920M 2.1% 1d 22h 57m ldom1 active -n---- 5000 2 2G 0.2% 1d 1h 27m ldom2 bound ------ 5001 2 768M