E.2 InfiniBand Network Cabling Tables

This section contains the tables for the InfiniBand network cabling. The Sun Network QDR InfiniBand Gateway Switch is located in located in rack units 26, 24, 23, or 16 (depending on the rack configuration).

This section contains the following topics:

E.2.1 Exalogic Machine Full Rack

This section contains the following tables:

Table E-5 lists the location, ports and cables for the InfiniBand connections for an Exalogic machine full rack.

Table E-5 InfiniBand Network Cabling for Exalogic Machine Full Rack

From Rack Unit (Sun Network QDR InfiniBand Gateway Switch) Port To Rack Unit Type of Equipment Port Description

U26

9A

U42

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U26

9B

U41

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U26

10A

U40

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U26

10B

U39

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U26

11A

U38

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U26

11B

U37

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U26

12A

U36

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U26

12B

U35

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U26

13A

U34

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U26

13B

U33

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U26

14A

U32

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U26

14B

U31

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U26

15A

U30

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U26

15B

U29

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

 

U24

9A

U42

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U24

9B

U41

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U24

10A

U40

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U24

10B

U39

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U24

11A

U38

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U24

11B

U37

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U24

12A

U36

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U24

12B

U35

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U24

13A

U34

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U24

13B

U33

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U24

14A

U32

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U24

14B

U31

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U24

15A

U30

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U24

15B

U29

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

 

U23

7A

U28

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U23

7B

U27

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

 

U23

8A

U22

Storage appliance server head

PCIe 0, P2

2 meter QDR InfiniBand cable

U23

8B

U21

Storage appliance server head

PCIe 0, P2

2 meter QDR InfiniBand cable

 

U23

9A

U15

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U23

9B

U14

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U23

10A

U13

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U23

10B

U12

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U23

11A

U11

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U23

11B

U10

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U23

12A

U9

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

12B

U8

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

13A

U7

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

13B

U6

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

14A

U5

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

14B

U4

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

15A

U3

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

15B

U2

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

 

U16

7A

U28

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U16

7B

U27

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

 

U16

8A

U22

Storage appliance server head

PCIe 0, P1

3 meter QDR InfiniBand cable

U16

8B

U21

Storage appliance server head

PCIe 0, P1

3 meter QDR InfiniBand cable

 

U16

9A

U15

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U16

9B

U14

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U16

10A

U13

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U16

10B

U12

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U16

11A

U11

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U16

11B

U10

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U16

12A

U9

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U16

12B

U8

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U16

13A

U7

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U16

13B

U6

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U16

14A

U5

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U16

14B

U4

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U16

15A

U3

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U16

15B

U2

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

Table E-6 lists the InfiniBand gateway switch inter-connections for an Exalogic machine full rack.

Table E-6 InfiniBand Gateway Switch Inter-Connections for Exalogic Machine Full Rack

From Rack Unit (Sun Network QDR InfiniBand Gateway Switch) Port To Rack Unit Type of Equipment Port Description

U16

4A

U23

Sun Network QDR InfiniBand Gateway Switch

4A

2 meter QDR InfiniBand cable

U16

4B

U23

Sun Network QDR InfiniBand Gateway Switch

4B

2 meter QDR InfiniBand cable

U16

5A

U23

Sun Network QDR InfiniBand Gateway Switch

5A

2 meter QDR InfiniBand cable

U16

5B

U23

Sun Network QDR InfiniBand Gateway Switch

5B

2 meter QDR InfiniBand cable

U16

6A

U23

Sun Network QDR InfiniBand Gateway Switch

6A

2 meter QDR InfiniBand cable

U16

6B

U23

Sun Network QDR InfiniBand Gateway Switch

6B

2 meter QDR InfiniBand cable

 

U24

4A

U26

Sun Network QDR InfiniBand Gateway Switch

4A

1 meter QDR InfiniBand cable

U24

4B

U26

Sun Network QDR InfiniBand Gateway Switch

4B

1 meter QDR InfiniBand cable

U24

5A

U26

Sun Network QDR InfiniBand Gateway Switch

5A

1 meter QDR InfiniBand cable

U24

5B

U26

Sun Network QDR InfiniBand Gateway Switch

5B

1 meter QDR InfiniBand cable

U24

6A

U26

Sun Network QDR InfiniBand Gateway Switch

6A

1 meter QDR InfiniBand cable

U24

6B

U26

Sun Network QDR InfiniBand Gateway Switch

6B

1 meter QDR InfiniBand cable

Table E-7 lists the InfiniBand gateway switch cross-connections for an Exalogic machine full rack.

Table E-7 InfiniBand Gateway Switch Cross-Connections for Exalogic Machine Full Rack

From Rack Unit (Sun Network QDR InfiniBand Gateway Switch) Port To Rack Unit Type of Equipment Port Description

U16

0A

U26

Sun Network QDR InfiniBand Gateway Switch

0A

2 meter QDR InfiniBand cable

U16

0B

U26

Sun Network QDR InfiniBand Gateway Switch

0B

2 meter QDR InfiniBand cable

U16

1A

U26

Sun Network QDR InfiniBand Gateway Switch

1A

2 meter QDR InfiniBand cable

U16

1B

U26

Sun Network QDR InfiniBand Gateway Switch

1B

2 meter QDR InfiniBand cable

 

U16

2A

U24

Sun Network QDR InfiniBand Gateway Switch

2A

2 meter QDR InfiniBand cable

U16

2B

U24

Sun Network QDR InfiniBand Gateway Switch

2B

2 meter QDR InfiniBand cable

U16

3A

U24

Sun Network QDR InfiniBand Gateway Switch

3A

2 meter QDR InfiniBand cable

U16

3B

U24

Sun Network QDR InfiniBand Gateway Switch

3B

2 meter QDR InfiniBand cable

 

U23

0A

U24

Sun Network QDR InfiniBand Gateway Switch

0A

1 meter QDR InfiniBand cable

U23

0B

U24

Sun Network QDR InfiniBand Gateway Switch

0B

1 meter QDR InfiniBand cable

U23

1A

U24

Sun Network QDR InfiniBand Gateway Switch

1A

1 meter QDR InfiniBand cable

U23

1B

U24

Sun Network QDR InfiniBand Gateway Switch

1B

1 meter QDR InfiniBand cable

 

U23

2A

U26

Sun Network QDR InfiniBand Gateway Switch

2A

1 meter QDR InfiniBand cable

U23

2B

U26

Sun Network QDR InfiniBand Gateway Switch

2B

1 meter QDR InfiniBand cable

U23

3A

U26

Sun Network QDR InfiniBand Gateway Switch

3A

1 meter QDR InfiniBand cable

U23

3B

U26

Sun Network QDR InfiniBand Gateway Switch

3B

1 meter QDR InfiniBand cable

E.2.2 Exalogic Machine Half Rack

This section contains the following tables:

Table E-8 lists the location, ports and cables for the InfiniBand connections for an Exalogic machine half rack.

Table E-8 InfiniBand Network Cabling for Exalogic Machine Half Rack

From Rack Unit (Sun Network QDR InfiniBand Gateway Switch) Port To Rack Unit Type of Equipment Port Description

U23

7A

U28

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U23

7B

U27

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

 

U23

8A

U22

Storage appliance server head

PCIe 0, P2

2 meter QDR InfiniBand cable

U23

8B

U21

Storage appliance server head

PCIe 0, P2

2 meter QDR InfiniBand cable

 

U23

9A

U15

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U23

9B

U14

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U23

10A

U13

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U23

10B

U12

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U23

11A

U11

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U23

11B

U10

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U23

12A

U9

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

12B

U8

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

13A

U7

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

13B

U6

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

14A

U5

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

14B

U4

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

15A

U3

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

15B

U2

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

 

U16

7A

U28

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U16

7B

U27

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

 

U16

8A

U22

Storage appliance server head

PCIe 0, P1

3 meter QDR InfiniBand cable

U16

8B

U21

Storage appliance server head

PCIe 0, P1

3 meter QDR InfiniBand cable

 

U16

9A

U15

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U16

9B

U14

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U16

10A

U13

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U16

10B

U12

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U16

11A

U11

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U16

11B

U10

Compute node

PCIe 2, P2

2 meter QDR InfiniBand cable

U16

12A

U9

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U16

12B

U8

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U16

13A

U7

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U16

13B

U6

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U16

14A

U5

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U16

14B

U4

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U16

15A

U3

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U16

15B

U2

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

Table E-9 lists the InfiniBand gateway switch inter-connections for an Exalogic machine half rack.

Table E-9 InfiniBand Gateway Switch Inter-Connections for Exalogic Machine Half Rack

From Rack Unit (Sun Network QDR InfiniBand Gateway Switch) Port To Rack Unit Type of Equipment Port Description

U16

4A

U23

Sun Network QDR InfiniBand Gateway Switch

4A

2 meter QDR InfiniBand cable

U16

4B

U23

Sun Network QDR InfiniBand Gateway Switch

4B

2 meter QDR InfiniBand cable

U16

5A

U23

Sun Network QDR InfiniBand Gateway Switch

5A

2 meter QDR InfiniBand cable

U16

5B

U23

Sun Network QDR InfiniBand Gateway Switch

5B

2 meter QDR InfiniBand cable

U16

6A

U23

Sun Network QDR InfiniBand Gateway Switch

6A

2 meter QDR InfiniBand cable

U16

6B

U23

Sun Network QDR InfiniBand Gateway Switch

6B

2 meter QDR InfiniBand cable

E.2.3 Exalogic Machine Quarter Rack

This section contains the following tables:

Table E-10 lists the location, ports and cables for the InfiniBand connections for an Exalogic machine quarter rack.

Table E-10 InfiniBand Network Cabling for Exalogic Machine Quarter Rack

From Rack Unit (Sun Network QDR InfiniBand Gateway Switch) Port To Rack Unit Type of Equipment Port Description

U23

8A

U22

Storage appliance server head

PCIe 0, P2

2 meter QDR InfiniBand cable

U23

8B

U21

Storage appliance server head

PCIe 0, P2

2 meter QDR InfiniBand cable

 

U23

12A

U9

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

12B

U8

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

13A

U7

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

13B

U6

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

14A

U5

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

14B

U4

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

15A

U3

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

15B

U2

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

 

U16

8A

U22

Storage appliance server head

PCIe 0, P1

3 meter QDR InfiniBand cable

U16

8B

U21

Storage appliance server head

PCIe 0, P1

3 meter QDR InfiniBand cable

 

U16

12A

U9

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U16

12B

U8

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U16

13A

U7

Compute node

PCIe 2, P1

2 meter QDR InfiniBand cable

U16

13B

U6

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U16

14A

U5

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U16

14B

U4

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U16

15A

U3

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U16

15B

U2

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

Table E-11 lists the InfiniBand gateway switch inter-connections for an Exalogic machine quarter rack.

Table E-11 InfiniBand Gateway Switch Inter-Connections for Exalogic Machine Quarter Rack

From Rack Unit (Sun Network QDR InfiniBand Gateway Switch) Port To Rack Unit Type of Equipment Port Description

U16

4A

U23

Sun Network QDR InfiniBand Gateway Switch

4A

2 meter QDR InfiniBand cable

U16

4B

U23

Sun Network QDR InfiniBand Gateway Switch

4B

2 meter QDR InfiniBand cable

U16

5A

U23

Sun Network QDR InfiniBand Gateway Switch

5A

2 meter QDR InfiniBand cable

U16

5B

U23

Sun Network QDR InfiniBand Gateway Switch

5B

2 meter QDR InfiniBand cable

U16

6A

U23

Sun Network QDR InfiniBand Gateway Switch

6A

2 meter QDR InfiniBand cable

U16

6B

U23

Sun Network QDR InfiniBand Gateway Switch

6B

2 meter QDR InfiniBand cable

E.2.4 Exalogic Machine Eighth Rack

This section contains the following tables:

Table E-12 lists the location, ports and cables for the InfiniBand connections for an Exalogic machine eighth rack.

Table E-12 InfiniBand Network Cabling for Exalogic Machine Eighth Rack

From Rack Unit (Sun Network QDR InfiniBand Gateway Switch) Port To Rack Unit Type of Equipment Port Description

U23

8A

U22

Storage appliance server head

PCIe 0, P2

2 meter QDR InfiniBand cable

U23

8B

U21

Storage appliance server head

PCIe 0, P2

2 meter QDR InfiniBand cable

 

U23

14A

U5

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

14B

U4

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

15A

U3

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

U23

15B

U2

Compute node

PCIe 2, P2

3 meter QDR InfiniBand cable

 

U16

8A

U22

Storage appliance server head

PCIe 0, P1

3 meter QDR InfiniBand cable

U16

8B

U21

Storage appliance server head

PCIe 0, P1

3 meter QDR InfiniBand cable

 

U16

14A

U5

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U16

14B

U4

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U16

15A

U3

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

U16

15B

U2

Compute node

PCIe 2, P1

3 meter QDR InfiniBand cable

Table E-13 lists the InfiniBand gateway switch inter-connections for an Exalogic machine eighth rack.

Table E-13 InfiniBand Gateway Switch Inter-Connections for Exalogic Machine Eighth Rack

From Rack Unit (Sun Network QDR InfiniBand Gateway Switch) Port To Rack Unit Type of Equipment Port Description

U16

4A

U23

Sun Network QDR InfiniBand Gateway Switch

4A

2 meter QDR InfiniBand cable

U16

4B

U23

Sun Network QDR InfiniBand Gateway Switch

4B

2 meter QDR InfiniBand cable

U16

5A

U23

Sun Network QDR InfiniBand Gateway Switch

5A

2 meter QDR InfiniBand cable

U16

5B

U23

Sun Network QDR InfiniBand Gateway Switch

5B

2 meter QDR InfiniBand cable

U16

6A

U23

Sun Network QDR InfiniBand Gateway Switch

6A

2 meter QDR InfiniBand cable

U16

6B

U23

Sun Network QDR InfiniBand Gateway Switch

6B

2 meter QDR InfiniBand cable

U16

USB Serial

Not applicable

Open ended

Not applicable

USB Adapter

U16

USB Serial

Not applicable

Open ended

Not applicable

USB Adapter