NIC Data Caching Scheme
Traditionally, the NIC memory is used for maintaining connection information
and address translation, and serving as a staging area for sending and receiving packets.
The NIC memory is essential to facilitate the intra cluster communication. However, in
most of cases, only part of the NIC memory is used for this purpose. For example, our
Myrinet NIC has 8MBytes of on chip memory, out of which only 2MBytes are used for
the NIC firmware and temporary communication buffer. The remaining 6MBytes can
be used for other purposes .
According to the IBA specification , an HCA can maintain up to 64,000 con
nections, so recent vendors provide large NIC memories to handle connections. Recent
Host Channel Adapters (HCAs) from Mellanox and other vendors have 64 to 512MBytes
of NIC memory [50, 73, 65]. Even if we support a large number of connections, not all of
the memory will be used for handling communication. Thus, the rest of the NIC memory
can be used for user applications [77, 41, 78].
While Web servers typically use only the main memory for data caching, the
abundant NIC memory can be used as an extended cache for application programs. To
design a NIC caching scheme, two fundamental issues need to be addressed are cache
placement and cache replacement.
For cache placement, there are two possible designs (shown in Figure 5.3). The
first goal is to increase the total data cache size and thus raise the cache hit ratio. When
a request arrives, the Web server typically tries to answer the request from the cache
in the main memory. If the requested data is not cached, it will be retrieved from the