VMware ESXi 5.x supports a maximum of 2 TB of system memory. ESXi might not boot if more than 2 TB is installed in the system. ESXi will begin to boot, but then report that only a very small amount of memory is available.
There is no workaround for this issue. Do not install more than 2 TB of system memory when using VMware ESXi 5.x.