VMware ESXi 5.x supports a maximum of 2 TB of system memory. ESXi might not boot if more than 2 TB is installed in the system. ESXi begins to boot, but then reports that only a very small amount of memory is available.
There is no workaround for this issue. Do not install more than 2 TB of system memory when using VMware ESXi 5.x.