Multiple Core Performance Hint Option

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶

This Internet-Draft will expire on 27 November 2022.¶

Copyright Notice

Copyright (c) 2022 IETF Trust and the persons identified as the document authors. All rights reserved.¶

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶

Table of Contents

1. Introduction

The Internet protocol allows datagrams to be re-ordered. Protocols which require datagrams to be ordered must retain out of order datagams until preceding datagrams have been received. While this works, the effect of out of order datagrams on network performance is highly detrimental: Out of order packets at first appear to be packet loss from the receivers point of view. The perceived packegt loss can trigger unneeded retransmission and delays from TCP and any other protocol which uses packet loss to implement congestion control.¶

With the advent of 10Gbit transmission speeds, it is not possible for a single CPU core to keep up with the incoming data running at full line speed. Hardware vendors have implemented mechanisms to distribute incoming datagrams to multiple CPU cores. If they did this on a random or round-robin basis, the different latencies between the multiple cores would result in datagram re-ordering, which can severly impact performance. Hardware solves this problem by distributing the data deterministically between CPU cores: This is done using a hash of the source and destination IP addresses and the source and destination port numbers. Using just the source and destination IP addresses is not sufficient, because the resulting traffic will often go to a single CPU core.¶

A performance problem arises when handling IPSec traffic: The port numbers are encrypted and can no longer be read by the hardware.¶

The performance problem also occurs with fragmented datagrams: The port numbers are only in the first fragment.¶

This standard defines IPv4 and IPv6 options to provide differentiation that can be used to distribute incoming datagrams to multiple CPU cores.¶

2. IPv4 Option Format

A host transmitting an IPv4 datagram MAY add an MCPHINT option to the IPv4 header under any of the following circumstances:¶

• The datagram contains an AH or ESP header.¶
• The datagram is fragmented.¶
• The datagram is to be transmitted beyond the current subnet and the don't fragment bit is not set.¶

The MCPHINT option provides 2 bytes of differentiation data. If present, the MCPHINT option MUST occur first - at offset 20 from the beginning of the IPv4 header.¶

The MCPHINT option MUST NOT be used with upper layer protocols which do not have unique identifiers beyond the IPv4 source and destination address.¶

The datagram MUST NOT be for the ICMP protocol.¶

The format of the IPv4 MCPHINT options is:¶

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Type        |  Length = 4   |  Differentiation Data         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    Type = TBD_IP4OPT_MCPHINT

Refer to RFC0791 [RFC0791] for more information about IP options.¶

If there is a mechanism by which an application can provide IPv4 options for transmission and that mechanism is used to provide an MCPHINT option, the value provided by the application MUST be used.¶

The macro OPT_MCPHINT MAY be added to netinet/in.h defined as TBD_IP4OPT_MCPHINT.¶

3. IPv6 Option Format

A host transmitting an IPv6 datagram MAY add an MCPHINT option under any of the following circumstances:¶

• The datagram contains an AH or ESP header.¶
• The datagram will be fragmented.¶

The MCPHINT option MUST be added to a destination options header. The MCPHINT option provides 2 bytes of differentiation data. The Destination options header is defined in section 4.6 of RFC8200 [RFC8200].¶

If present, the MCPHINT option MUST occur first in the first destination options header - normally at offset 42 from the beginning of the IPv6 header.¶

Note that RFC8200 [RFC8200] requires that per fragment destination headers to be followed by a routing header. If one applies this hint to a packet containing an IPv6 fragmentation header, a routing header must be included. RFC8200 [RFC8200] explicitly states that a routing header with zero "Segments Left" is always ignored; so, this is possible.¶

The MCPHINT option MUST NOT be used with upper layer protocols which do not have unique identifiers beyond the IPv6 source and destination address.¶

The datagram MUST NOT be for the ICMP6 protocol.¶

The format of the IPv6 MCPHINT options is:¶

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Type        |  Data Len = 2 |   Differentiation Data        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    Type = TBD_IP6OPT_MCPHINT

If there is a mechanism by which an application can provide destination options for transmission and that mechanism is used to provide an MCPHINT option, the value provided by the application MUST be used.¶

The macro IP6OPT_MCPHINT MAY be added to netinet/ip6.h defined as TBD_IP6OPT_MCPHINT.¶

4. Differentiation Data

For both IPv4 and IPv6, there is two bytes of differentiation data. The differentiation data MUST NOT be zero. The differentation data MUST be the same for all datagrams in a logical stream. The actual value chosen for differentiation data is left to the implementation.¶

A preferable mechanism would be to generate two bytes of random data when a socket is created and to use that data for the life of the socket. The random data could be updated every time a connection is specified.¶

Alternatively, exclusive or'ing the source and destination ports is an acceptable method for generating the differentiation data.¶

5. Forwarding

Forwarding is already defined to pass through unknown options.¶

6. Tunneling

Tunneling implementations MAY copy the MCPHINT option from the datagrams being tunneled to the outer headers.¶

7. Parsing Input Datagrams

 o IHL MUST be greater than or equal to 6
 o The byte at offset 20 MUST be TBD_IP4OPT_MCPHINT

 o Next Header (offset 6) MUST be 60 (for destination options).
 o The byte at offset 42 MUST be TBD_IP6OPT_MCPHINT

8. Future Considerations

A future revision of this standard could allow the differentation data to be longer as long as the first two bytes are generated the same way.¶

A future revision of this standard could add fields to this option.¶

9. Security Considerations

The MCPHINT option provides some minimal insight to internal network configurations that wouldn't otherwise be discernable for IPSec tunnels.¶

Xor'ing the port numbers to obtain differentiation data provides slightly more information than using random data.¶

The implementation MUST provide an adminitrative mechanism to disable the use of MCPHINT options.¶

If the implementation implements both random generation of differentiation data AND uses the Xor'ing ports method, there MUST be separate administrative mechanisms for each method.¶

10. IANA Considerations

IANA is asked to assign a value for TBD_IP4OPT_MCPHINT under "Internet Protocol Version 4 (IPv4) Parameters", "IP Option Numbers", Refer to RFC2780 [RFC2780] and RFC0791 [RFC0791].¶

The Copy bit MUST be 1 and the class bits MUST be 00.¶

IANA is asked to assign a value for TBD_IP6OPT_MCPHINT under "Internet Protocol Version 6 (IPv6) Parameters", "Destination Options and Hop-by-Hop Options", Refer to RFC2780 [RFC2780] and RFC8200 [RFC8200].¶

The act bits MUST 00 and the chg bit MUST be 0¶

11. Appendix A - Design Considerations

This is done as an option so it may be added without affecting implementations that don't implement it.¶

Use with ICMP and ICMPv6 is prohibited because there is no reason to optimize them and, given that correct IP layer behavior depends on thier transmission, it is best to avoid anything that might interfere with correct operation..¶

One should note that when using this option with IPSec, the same security association is likely to be processed on multiple CPU cores. This requires a good locking design to acheive the desired performance improvement. It also requires much larger replay windows.¶

 IPv6 Header
 IPv6 Headar, Hop-by-Hop Header
 IPv6 Header, Destination Options Header, Routing Header
 IPv6 Header, Hop-by-Hop Header, Dest Options Header, Routing Header

Author's Address