4816837

o

o

System hangs when executing parallel hot-plug operation with SP DR in suspend phase.

There is no workaround.

6459540

o

o

The DAT72 internal tape drive connected to M4000/M5000/ M8000/M9000 servers might time out during tape operations.

The device might also be identified by the system as a QIC drive.

Add the following definition to /kernel/drv/st.confs:

tape-config-list= "SEAGATE DAT   DAT72-000", "SEAGATE_DAT   DAT72-000", "SEAGATE_DAT   DAT72-000"; SEAGATE_DAT   DAT72-000=1,0x34,0,0x9639,4,0x00,0x8c,0x8c 0x8c,3;

There are four spaces between SEAGATE DAT and DAT72-000.

6522017

o

o

Domains using the Oracle ZFS file system cannot use DR.

Set the maximum size of the ZFS ARC lower. For detailed assistance, contact your authorized service representative.

6531036

o

o

o

The error message network initialization failed appears repeatedly after a boot net installation.

There is no workaround.

6532215

o

o

o

volfs or dscp services might fail when a domain is booted.

Restart the service. To avoid the problem, issue the following commands.

# svccfg -s dscp setprop \

start/timeout_seconds=count: 300

# svccfg -s volfs setprop \start/timeout_seconds=count: 300

# svcadm refresh dscp

# svcadm refresh volfs

6588650

o

o

On occasion, a M4000/M5000/M8000/M9000 server is unable to DR after an XSCF failover to or from backup XSCF.

There is no workaround.

6589644

o

When XSCF switchover happens on an M8000/M9000 server after the system board has been added using the addboard(8) command, the console is no longer available.

Press CTRL-q to recover.

6592302

o

o

Unsuccessful DR operation leaves memory partially configured.

It might be possible to recover by adding the board back to the domain with an addboard -d command. Otherwise try deleteboard(8) again.

6611966

o

o

DR deleteboard(8) and moveboard(8) operations might fail. Example for messages on domain:

drmach: WARNING: Device driver failure: /pci dcs: <xxxx>

config_change_state:

Hardware specific failure:

unconfigure SB1: Device driver failure: /pci

Try DR operations again.

6660168

o

o

o

See CR 6660168. It was removed from this table due to the length of the description.

6674266

o

o

This CR is a duplicate of CR 6611966.

6745410

o

o

o

Boot program ignores the Kadb option which causes the system not to boot.

None.

7009469

o

Creating a RAID configuration using the raidctl(1M) command generates a warning message on initial process. Subsequent RAID configurations do not.

None.

7135497

o

On systems with the RAID-capable IOUA card and running Oracle Solaris 11, after a RAID volume is deleted using raidctl -d, the format utility does not display the disk.

Execute cfgadm(1M): cfgadm -c unconfigure cn where cn is the controller number.

6794630

o

o

o

An attempt to use the GUI to install Oracle Solaris in a domain larger than 2TB might fail.

Use the command-line interface to install the Oracle Solaris OS.

6888928

o

o

o

The IPMP interface fails since probe packets are not sent through that interface. Problem occurs with M3000/M4000/M5000/M8000/ M9000. Seen on servers running the Oracle Solaris 10 10/09 OS and IPMP, or any Oracle Solaris release running IPMP with Patch 141444-09 installed.

Disable probe-based failure detection. See IPMP Link-based Only Failure Detection with Solaris 10 Operating System (OS) (Doc ID 1008064.1)

6668237

o

o

o

After DIMMs are replaced, the corresponding DIMM faults are not cleared on the domain.

Use the command fmadm repair fmri|uuid to record the repair. Then use the command fmadm rotate to clear out any leftover events.

6872501

o

o

o

Cores are not offlined when requested by the XSCF. This CR affects only Oracle Solaris 10 5/09 and Oracle Solaris 10 10/09 releases.

Use fmdump(1M) with its -v option on the Service Processor to identify the faulty core. Once identified, use psradm(8) on the domain to offline the core.

6572827

o

o

o

The prtdiag -v command reports PCI bus types incorrectly. It reports “PCI” for PCI-X leaf devices and “UNKN” for legacy PCI devices.

There is no workaround.

6724307

o

Scheduler decisions are occasionally unbalanced.

Sometimes two threads will be on one core (causing both to run at about half speed) while another core is idle. For many OpenMP and similar parallel applications, the application performance is limited by the speed of the slowest thread.

Uneven scheduling is not common, perhaps 1 in 50 or 1 in 100 decisions. But if there are 128 threads running, then the application might have at least one uneven schedule event.

Use processor sets to prevent uneven threads to core assignment.

6800734

o

o

deleteboard hangs in a domain

There is no workaround.

6816913

o

o

The XSCF showdevices(8) command displays the incorrect processor cache size for fractional processor cache sizes, such as displaying “5MB” when the correct display would be “5.5MB.”

Use the Oracle Solaris command prtdiag(1M) on the domain to report processor information.

6821108

o

o

DR and showdevices(8) don't work after XSCF reboot.

Reboot the XSCF service processor twice. Half the SAs (security associations) are deleted the first time, half are deleted the second time, so the second addition succeeds and IPsec communication is reestablished.

6827340

o

o

o

DR and Memory patrol might fail due to SCF command error.

There is no workaround.

6588555

o

o

Resetting the XSCF during a DR operation on permanent memory might cause domain panic.

Do not start an XSCF reset while a DR operation is underway. Wait for the DR operation to complete before starting the reset.

6623226

o

o

o

The Oracle Solaris command lockstat(1M) or the dtrace lockstat provider might cause a system panic.

Do not use the Oracle Solaris lockstat(1M) command or the dtrace lockstat provider.

6680733

o

o

o

Sun Quad-port Gigabit Ethernet Adapter UTP (QGC) & Sun Dual 10 GigE Fiber XFP Low Profile Adapter (XGF) NICs might panic under high load conditions.

If possible, use the card in x8 slot. Otherwise, there is no workaround.

6689757

o

o

o

Sun Dual 10 GigE Fiber XFP Low Profile adapter (XGF) with a single or improperly installed XFP optical transceivers might cause the following error to show on the console:

The XFP optical transceiver is broken or missing.

Verify that both XFP optical transceivers are firmly seated in the housing.

Do not mix INTEL and Sun XFP optical transceivers in the same adapter.

Do NOT plumb a port with the Oracle Solaris command ifconfig(1M) if the port does not contain an XFP optical transceiver or it contains one but the transceiver is not in use.

6725885

o

The XCP command cfgadm(8) displays non-existent M3000 system boards (SB1 to SB15).

The cfgadm output for SB1-SB15 can be ignored.

6511374

o

o

Memory translation warning messages might appear during boot if memory banks were disabled due to excessive errors.

After the system is rebooted, the fmadm repair command can be used to prevent a recurrence of the problem on the next boot.

6533686

o

o

When XSCF is low on system resources, DR deleteboard or moveboard operations that relocate permanent memory might fail with one or more of these errors:

SCF busy

DR parallel copy timeout

This applies only to Quad-XSB configured System Boards hosting multiple domains.

Retry the DR operation at a later time.

6535018

o

In Oracle Solaris domains that include SPARC64 VII processors, workloads that make heavy use of the Oracle Solaris kernel might not scale as expected when you increase the thread count to a value greater than 256.

For Oracle Solaris domains that include SPARC64 VII processors, limit domains to a maximum of 256 threads.

6556742

o

o

o

The system panics when DiskSuite cannot read the metadb during DR. This bug affects the following cards:

• SG-XPCIE2FC-QF4, 4-Gigabit PCI-e Dual-Port Fiber Channel HBA

• SG-XPCIE1FC-QF4, 4-Gigabit PCI-e Single-Port Fiber Channel HBA

• SG-XPCI2FC-QF4, 4-Gigabit PCI-X Dual-Port Fiber Channel HBA

• SG-XPCI1FC-QF4, 4-Gigabit PCI-X Single-Port Fiber Channel HBA

Panic can be avoided when a duplicated copy of the metadb is accessible via another Host Bus Adapter.

6589833

o

o

The DR addboard command might cause a system hang if you are adding a Sun StorageTek Enterprise Class 4-Gigabit Dual-Port Fiber Channel PCI-E HBA card (SG-XPCIE2FC-QF4) at the same time that an SAP process is attempting to access storage devices attached to this card. The chance of a system hang is increased if the following cards are used for heavy network traffic:

• X4447A-Z, PCI-e Quad-port Gigabit Ethernet Adapter UTP

• X1027A-Z1, PCI-e Dual 10 Gigabit Ethernet Fiber XFP Low profile Adapter

There is no workaround.

6608404

o

o

Hot-plug of the X4447A-Z, PCI-e Quad-port Gigabit Ethernet Adapter UTP card in slot 1 might cause other network devices to fail.

To avoid the defect, do not install this card in slot 1.

6614737

o

o

The DR deleteboard(8) and moveboard(8) operations might hang if any of the following conditions exist:

• A DIMM has been degraded.

• The domain contains system boards with different memory size.

Avoid performing DR operations if any of the following conditions exist:

• Degraded memory – To determine whether the system contains degraded memory, use the XSCF command showstatus(8).

• Differing memory sizes – To determine whether the domain contains system boards with different memory sizes, display the list of memory sizes using the XSCF command showdevices(8) or the Oracle Solaris command prtdiag(1M) on the domain.

  If a DR command hangs, reboot the domain to recover.

6619224

o

For Oracle Solaris domains that include SPARC64 VII processors, a single domain of 256 threads or more might hang for an extended period of time under certain unusual situations. Upon recovery, the uptime command will show extremely high load averages.

For Oracle Solaris domains that include SPARC64 VII processors, do not exceed a domain size of 256 virtual processors in a single Oracle Solaris domain. This means a maximum of 32 CPUs in a single domain configuration (maximum configuration for an M8000 server).

6632549

o

o

fmd service on domain might fail to go into maintenance mode after DR operations.

Issue the following command on the domain:

# svcadm clear fmd

6660197

o

o

DR might cause the domain to hang if either of the following conditions exist:

• A domain contains 256 or more CPUs.

• Memory error occurred and the DIMM has been degraded.

1. Set the following parameter in the system specification file (/etc/system):set):           set drmach:drmach_disable_mcopy=1

2. Reboot the domain.

6679370

o

o

The following message might be output on the console during system boot, addition of the External I/O Expansion Unit using hotplug, or an FMEMA operation by DR:

SUNW-MSG-ID: SUN4-8000-75, TYPE:

Fault, VER: 1, SEVERITY: Critical ... DESC:

A problem was detected in the PCIExpress subsystem.

Add the following to the /etc/system file, then reboot the domain.

set pcie_expected_ce_mask = 0x2001

6720261

o

o

o

If your domain is running Oracle Solaris 10 5/08 OS, the system might panic/trap during normal operation.

Set the following parameter in the system specification file (/etc/system):

set heaplp_use_stlb=0

Then reboot the domain.

6737039

o

WAN boot of an M3000 server fails intermittently with a panic early in the boot process. Sample output:

ERROR: Last Trap: Fast Data Access MMU Miss %TL:1 %TT:68 %TPC:13aacc %TnPC:13aad0 %TSTATE:1605 %PSTATE:16 (IE:1 PRIV:1 PEF:1) DSFSR:4280804b ( FV:1 OW:1 PR:1 E:1 TM:1 ASI:80 NC:1 BERR:1 ) DSFAR:fda6f000 DSFPAR:401020827000 D- TAG:6365206f66206000

Power the chassis off then on, then retry the operation.

5076574

o

A PCIe error can lead to an invalid fault diagnosis on a large M8000/M9000 domain.

Create a file /etc/fm/fmd/fmd.conf containing the following lines:

setprop client.buflim 40msetprop client.memlim 40m

6348554

o

o

Using the cfgadm -c disconnect command on the following cards might hang the command:

• SG-XPCIE2FC-QF – Sun StorageTek Enterprise Class 4-Gigabit Dual-Port Fiber Channel PCI-E HBA

• SG-XPCIE1FC-QF4 – Sun StorageTek Enterprise Class 4-Gigabit Single-Port Fiber Channel PCI-E HBA

• SG-XPCI2FC-QF4 – Sun StorageTek Enterprise Class 4-Gigabit Dual-Port Fiber Channel PCI-X HBA

• SG-XPCI1FC-QF4 – Sun StorageTek Enterprise Class 4-Gigabit Single-Port Fiber Channel PCI-X HBA

Do not perform the cfgadm -c disconnect operation on the affected cards.

6402328

o

If more than six IOUA (Base I/O Card) cards are used in a single domain, a panic might occur under high I/O stress.

Limit the maximum number of IOUAs in a single domain to 6.

6472153

o

o

If you create a Oracle Solaris Flash archive on a sun4u server other than an M4000/M5000/M8000/M9000 server, then install it on one of these servers, the console's TTY flags will not be set correctly. This can cause the console to lose characters during stress

Just after installing Oracle Solaris OS from a Oracle Solaris Flash archive, telnet into the M4000/M5000/ M8000/M9000 server to reset the console's TTY flags as follows:

# sttydefs -r console

# sttydefs -a console -i "9600 hupcl opost onlcr crtscts" -f "9600"

This procedure is required only once.

6505921

o

Correctable error on the system PCIe bus controller generates an invalid fault.

Create a file /etc/fm/fmd/fmd.conf containing the following lines:

setprop client.buflim 40m

setprop client.memlim 40m

6522433

o

o

The incorrect motherboard might be identified by fmdump for CPU faults after reboot.

Check system status on XSCF.

6527811

o

o

The showhardconf(8) command on the XSCF cannot display PCI card information that is installed in the External I/O Expansion Unit, if the External I/O Expansion Unit is configured using PCI hot-plug.

There is no workaround. When each PCI card in the External I/O Expansion Unit is configured using PCI hot-plug, the PCI card information is displayed correctly.

6536564

o

o

showlogs(8) and showstatus(8) command might report wrong I/O component.

To avoid this problem, issue the following commands on the domain.

# cd /usr/platform/SUNW,SPARCEnterprise/lib/fm/topo/plugins

# mv ioboard.so ioboard.so.orig

# svcadm restart fmd

Contact a service engineer if the following messages are displayed:

SUNW-MSG-ID: SUNOS-8000-1L, TYPE: Defect, VER: 1, SEVERITY: Minor, EVENT-TIME: Sun May 6 18:22:24 PDT 2007 PLATFORM: SUNW,SPARC-Enterprise, CSN: BE80601007, HOSTNAME: sparc

6545143

o

o

There is a low probability that a system panic can occur during trap processing of a TLB miss for a user stack address. The problem can occur if the user stack is unmapped concurrently with the user process executing a flush windows trap (ta 3). The panic message will contain the following string:

bad kernel MMU trap at TL 2

There is no workaround.

6545685

o

o

If the system has detected correctable memory errors (CE) at power-on self-test (POST), the domains might incorrectly degrade 4 or 8 DIMMs.

Increase the memory patrol timeout values used via the following setting in /etc/system and reboot the system:

set mc-opl:mc_max_rewrite_loop =20000

6546188

o

o

The system panics when running hot-plug (cfgadm) and DR operations (addboard and deleteboard) on the following cards:

• X4447A-Z, PCI-e Quad-port Gigabit Ethernet Adapter UTP

• X1027A-Z1, PCI-e Dual 10 Gigabit Ethernet Fiber XFP Low profile Adapter

There is no workaround.

6551356

o

o

The system panics when running hot-plug (cfgadm) to configure a previously unconfigured card. The message “WARNING: PCI Expansion ROM is not accessible” will be seen on the console shortly before the system panic. The following cards are affected by this defect:

• X4447A-Z, PCI-e Quad-port Gigabit Ethernet Adapter UTP

• X1027A-Z1, PCI-e Dual 10 Gigabit Ethernet Fiber XFP Low profile Adapter

Use cfgadm -c disconnect to completely remove the card. After waiting at least 10 seconds, the card might be configured back into the domain using the cfgadm -c configure command.

6559504

o

o

Messages of the form nxge: NOTICE: nxge_ipp_eccue_valid_check: rd_ptr = nnn wr_ptr = nnn will be observed on the console with the following cards:

• X4447A-Z, PCI-e Quad-port Gigabit Ethernet Adapter UTP

• X1027A-Z1, PCI-e Dual 10 Gigabit Ethernet Fiber XFP Low profile Adapter

These messages can be safely ignored.

6563785

o

o

Hot-plug operation with the following cards might fail if a card is disconnected and then immediately reconnected:

• SX-PCIE2SCSIU320Z – Sun StorageTek PCI-E Dual-Port Ultra320 SCSI HBA

• SGXPCI2SCSILM320-Z – Sun StorageTek PCI Dual-Port Ultra 320 SCSI HBA

After disconnecting a card, wait for a few seconds before re-connecting.

6564934

o

o

Performing a DR deleteboard operation on a board that includes Permanent Memory when using the following network cards results in broken connections:

• X4447A-Z, PCI-e Quad-port Gigabit Ethernet Adapter UTP

• X1027A-Z1, PCI-e Dual 10 Gigabit Ethernet Fiber XFP Low profile Adapter

Reconfigure the affected network interfaces after the completion of the DR operation. For basic network configuration procedures, refer to the ifconfig man page for more information.

6568417

o

o

After a successful CPU DR deleteboard operation, the system panics when the following network interfaces are in use:

• X4447A-Z, PCI-e Quad-port Gigabit Ethernet Adapter UTP

• X1027A-Z1, PCI-e Dual 10-Gigabit Ethernet Fiber XFP Low profile Adapter

Add the following line to /etc/system and reboot the system:

set ip:ip_soft_rings_cnt=0

6571370

o

o

Use of the following cards have been observed to cause data corruption in stress test under laboratory conditions:

• X4447A-Z, PCI-e Quad-port Gigabit Ethernet Adapter UTP

• X1027A-Z1, PCI-e Dual 10-Gigabit Ethernet Fiber XFP Low profile Adapter

Add the following line in /etc/system and reboot the system:

set nxge:nxge_rx_threshold_hi=0

6584984

o

The busstat(1M) command with -w option might cause M8000/M9000 server domains to reboot.

There is no workaround. Do not use the busstat -w command on pcmu_p.

6589546

o

o

prtdiag does not show all IO devices of the following cards:

• SG-XPCIE2FC-EM4 Sun StorageTek Enterprise Class 4-Gigabit Dual-Port Fiber Channel PCI-E HBA

• SG-XPCIE1FC-EM4 Sun StorageTek Enterprise Class 4-Gigabit Single-Port Fiber Channel PCI-E HBA

Use prtdiag -v for full output.

6663570

o

o

DR operations involving the lowest numbered CPU might cause the domain to panic.

Do not use DR to remove the system board that hosts the CPU with the lowest CPU ID. Use the Oracle Solaris prtdiag command to identify the CPU with the lowest CPU ID.

6303418

o

M9000 server with a single domain and 11 or more fully populated system boards might hang under heavy stress.

Do not exceed 170 CPU threads.

Limit the number of CPU threads to one per CPU core by using the Oracle Solaris psradm command to disable the excess CPU threads. For example, disable all odd-numbered CPU threads.

6416224

o

o

System performance can degrade using a single NIC card with more than 5,000 connections.

Use multiple NIC cards to split network connections.

6441349

o

o

I/O error can hang the system.

There is no workaround.

6485555

o

o

On-board Gigabit Ethernet NVRAM corruption could occur due to a race condition. The window of opportunity for this race condition is very small.

There is no workaround.

6496337

o

o

The “cpumem-diagnosis” module might fail to load after uncorrectable error (UE) panic. Systems will function correctly, but events normally automatically diagnosed by FMA using this module will require manual diagnosis.

Example:

SUNW-MSG-ID: FMD-8000-2K, TYPE:

Defect, VER: 1, SEVERITY: Minor

EVENT-TIME: Thu Feb 15 15:46:57 JST 2007 PLATFORM: SUNW,SPARC-Enterprise, CSN: BE80601007,

HOSTNAME: col2-ffem7-d0

If the problem has already occurred:

1. Remove the cpumemdiagnosis file

   # rm /var/fm/fmd/ckpt\ /cpumemdiagnosis/\ cpumem-diagnosis

2. Restart fmd service:

   # svcadm restart fmd

To avoid this problem in advance, add the following line in the file: /lib/svc/method/svc-dumpadm:

# savedev=none

rm -f

/var/fm/fmd/ckpt/cpumemdiagnosis /cpumem-diagnosis

#

6495303

o

o

The use of a PCIe Dual-Port Ultra320 SCSI controller card (SG-(X)PCIE2SCSIU320Z) in IOU Slot 1 on a SPARC Enterprise M4000/M5000 server might result in a system panic.

Do not use this card in IOU Slot 1.

6498283

o

o

Using the DR deleteboard command while psradm operations are running on a domain might cause a system panic.

There is no workaround.

6499304

o

o

Unexpected message is displayed on console and CPU isn't offlined when numerous correctable errors (CEs) occur.

Example:

SUNW-MSG-ID: FMD-8000-11, TYPE: Defect, VER: 1, SEVERITY: Minor EVENT-TIME: Fri Feb 2 18:31:07 JST 2007, PLATFORM: SPARC-Enterprise, CSN: BE80601035, HOSTNAME: FF2-35-0

Check CPU status on XSCF.

6502204

o

o

Unexpected error messages might be displayed on console on booting after CPU UE panic.

Example:

SUNW-MSG-ID: FMD-8000-11, TYPE: Defect, VER: 1, SEVERITY: Minor EVENT-TIME: Tue Jan 9 20:45:08 JST 2007 PLATFORM: SUNW,SPARC-Enterprise, CSN: 2030636002, HOSTNAME: P2-DC1- 16-d0

If you see unexpected messages, use the showdomainstatus(8) command to check system status on XSCF.

650275

o

o

Inserted or removed hot-plugged PCI card might not output notification message.

There is no workaround.

6508432

o

o

A large number of spurious PCIe correctable errors can be recorded in the FMA error log.

To mask these errors, add the following entry to /etc/system and reboot the system:

set pcie:pcie_aer_ce_mask = 0x2001

6508434

o

A domain might panic when an additional PCI-X card is installed or a PCI-X card is replaced using PCI hot-plug.

Do not insert a different type of PCI-X card in the same PCI slot by using PCI hot-plug.

6510861

o

o

When using the PCIe Dual-Port Ultra320 SCSI controller card (SG-(X)PCIE2SCSIU320Z), a PCIe correctable error causes a Oracle Solaris panic.

Add the following entry to /etc/system to prevent the problem:

set pcie:pcie_aer_ce_mask = 0x31c1

6520990

o

o

When a domain reboots, SCF might not be able to service other domains that share the same physical board. DR operation can exceed the default timeout period and panic can occur.

Increase the DR timeout period by setting the following statement in /etc/system and reboot your system:

set drmach:fmem_timeout = 30

6527781

o

The cfgadm command fails while moving the DVD/DAT drive between two domains.

There is no workaround. To reconfigure DVD/Tape drive, execute reboot -r from the domain exhibiting the problem.

6530178

o

o

The DR addboard command can hang. Once the problem is observed, further DR operations are blocked. Recovery requires reboot of the domain.

There is no workaround.

6530288

o

o

The cfgadm(1M) command might not correctly show Ap_Id format.

There is no workaround.

6534471

o

o

Systems might panic/trap during normal operation.

If a patch is not available, disable the kernel large page sTLB programming. In the file /etc/system, change the heaplp_use_stlb variable to 0:

set heaplp_use_stlb=0

6535564

o

o

PCI hot-plug to PCI slot #0, #1 or External I/O Expansion Unit might fail on XSB added by DR.

Use DR instead of PCI hot plug if you need to add or remove PCI card on the XSB.

6539084

o

o

There is a low probability of a domain panic during reboot when the Sun Quad GbE UTP x8 PCIe (X4447A-Z) card is present in a domain.

There is no workaround.

6539909

o

o

Do not use the following I/O cards for network access when you are using the boot net install command to install the Oracle Solaris OS:

• X4447A-Z/X4447A-Z, PCIe Quad-port Gigabit Ethernet Adapter UTP

• X1027A-Z/X1027A-Z, PCIe Dual 10-Gigabit Ethernet Fiber XFP

Use an alternative type of network card or onboard network device to install the Oracle Solaris OS via the network.

6542632

o

o

Memory leak in PCIe module if driver attach fails.

There is no workaround.