Q-Logic IB6054601-00 D manual Adapter Required Component Optional Memory Footprint Comment

Q

2 – InfiniPath Cluster Administration Memory Footprint

on system configuration. OpenFabrics support is under development and has not been fully characterized. This table summarizes the guidelines.

Table 2-1. Memory Footprint of the InfiniPath Adapter on Linux x86_64 Systems

Adapter	Required/
component	optional	Memory Footprint	Comment
InfiniPath Driver	Required	9 MB	Includes accelerated IP
			support. Includes tables
			space to support up to
			1000 node systems.
			Clusters larger than 1000
			nodes can also be
			configured.
MPI	Optional	71 MB per process with	Several of these
		default parameters: 60 MB	parameters (sendbufs,
		+ 512*2172 (sendbufs) +	recvbufs and size of the
		4096*2176 (recvbufs) +	shared memory region)
		1024*1K (misc. allocations)	are tunable if reduced
		+ 32 MB per node when	memory footprint is
		multiple processes	desired.
		communicate via shared
		memory
		+ 264 Bytes per MPI node
		on the subnet
OpenFabrics	Optional	1~6 MB	This not been fully
		+ ~500 bytes per QP	characterized as of this
		+ TBD bytes per MR	writing.
		+ ~500 bytes per EE
		Context
		+ OpenFabrics stack from
		openfabrics.org (size not
		included in these
		guidelines)

Here is an example for a 1024 processor system:

■1024 cores over 256 nodes (each node has 2 sockets with dual-core processors)

■1 adapter per node

■Each core runs an MPI process, with the 4 processes per node communicating via shared memory.

■Each core uses OpenFabrics to connect with storage and file system targets using 50 QPs and 50 EECs per core.

IB6054601-00 D 2-3