Internet-Draft | Intent-aware Routing using Color | July 2022 |
Hegde, et al. | Expires 16 January 2023 | [Page] |
This draft describes the scope, set of use-cases and requirements for a distributed routing based solution to establish end-to-end intent-aware paths spanning multi-domain packet networks. The document focuses on BGP given its predominant use in inter-domain routing deployments, however the requirements may also apply to other solutions.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 16 January 2023.¶
Copyright (c) 2022 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
Evolving trends in wireless access technology, cloud applications, virtualization, and network consolidation all contribute to the increasing demands being placed on a common packet network. In order to meet these demands, a given network will need to scale horizontally in terms of its bandwidth, absolute number of nodes, and geographical extent. The same network will need to extend vertically in terms of the different services and variety of intent that it needs to simultaneously support.¶
In order to operate networks with large numbers of devices, network operators organize networks into multiple smaller network domains. Each network domain typically runs an IGP which has complete visibility within its own domain, but limited visibility outside of its domain. Network operators will continue to use multiple domains to scale horizontally. In MPLS based networks BGP-LU (RFC8277) has been widely deployed for providing reachability across multiple domains.¶
The evolving network requirements (e.g. 5G, native cloud) in such a multi-domain network requires the establishment of paths that span multiple domains or AS's while maintaining specific transport characteristics or intent (e.g. bandwidth, latency). There is also a need to provide flexible, scalable, and reliable end-to-end connectivity for multiple services across the network domains.¶
This document describes requirements for scalable, intent-aware reachability across multiple domains.¶
The base problem that it focuses on is the BGP-based delivery of an intent across several transport domains, however the requirements may also apply to other distributed solutions.¶
The problem space is then widened to include any intent (including Network Function Virtualization (NFV) chains and their location), any data plane and the application of intent-based routing to the Service/VPN routes.¶
It is intended that the requirements enable the design of technology and protocol extensions that address the widest application, while ensuring consistency and compatibility with existing deployed solutions.¶
This section describes a few typical deployment scenarios that involve large-scale multi-domain network designs and use of various topology, IGP and BGP routing models. While the examples use specific types of deployments for illustration, neither the use-cases nor the network designs are limited to any particular provider deployment.¶
Service Provider networks can contain many nodes distributed over a large geographic area. 5G networks can include as many as one million nodes, with the majority of those being radio access nodes. Radio and access nodes may be constrained by their memory and processing capabilities.¶
Such transport networks use multiple domains to support scalability. For this analysis, we consider a representative network design with four level of hierarchy: access domains, pre-aggregation domains, aggregation domains and a core. (See Figure 1). The separation of domains internal to the service provider can be performed by using either IGP or BGP.¶
5G networks support a variety of service use cases that may require end to-end network slicing. In certain cases, the end-to-end connectivity requires the ability to forward over intent-aware paths, such as paths delivering low-delay. The inter-domain routing solution should support the establishment of end to end paths that address specific intent requirements, as well as support multiple such paths to address slicing requirements.¶
Networks built for providing delivery of content are geographically distributed by design to provide connectivity in multiple regions and sharing of data across regions.¶
As these WAN networks grow beyond several thousand nodes, they are divided into multiple IGP domains for scale and reliability. An illustration is provided in in Figure 2.¶
These large WAN networks often cross national boundaries. In order to meet data sovereignty requirements, operators need to maintain strict control over end-to-end traffic-engineered (TE) paths. A distributed inter-domain solution should be able to create highly constrained inter domain TE paths in a scalable manner.¶
Some deployments may use a controller to acquire the topologies of multiple domains and build end-to-end constrained paths. This approach can be scaled with hierarchical controllers. However, there is still a risk of a loss of network connectivity to one or more controllers, which could lead to a failure to satisfy the strict requirements of data sovereignty. The network should be able to have pre-established TE paths end-to-end that don't rely on controllers, to address these failure scenarios.¶
Distributed data centers are playing an increasingly important role in providing access to information and applications. Geographically diverse data centers are usually connected via a high speed, reliable and secure DC WAN core network.¶
One variation of a DCI topology is shown in .Figure 3.¶
In many DC WAN deployments, applications require end-to-end path diversity and end-to-end low latency paths.¶
Another consideration in DC WAN deployments is the choice of encapsulation technologies. Some deployments use the same tunneling mechanism within the DC and DCI networks, while other deployments use different mechanisms in each. It is important for a solution to provide flexibility in choice of tunneling mechanisms across domains.¶
The use cases for inter-domain intent-based packet transport described in this section are intended to provide motivation for the requirements that follow. They apply to all the different deployment scenarios described above.¶
Figure 4 depicts an example of a WAN with multiple ASes, where each AS serves a continent. Certain traffic from PE1 (in AS1) to PE3 (in AS3) must not traverse country Z in AS2. However, all paths from AS1 to AS3 traverse AS 2. The inter-domain solution should provide end-to-end path creation that traverses AS 2 but avoids country Z.¶
In other networks, the domain to avoid may encompass an entire AS.¶
Service provider networks running L2 and L3VPNs carry traffic for particular VPNs on low-latency paths that traverse multiple domains.¶
RFC7665 defines service function chaining as an ordered set of service functions and automated steering of traffic through this set of service functions. There could be a variety of service functions such as firewalls, parental control, CGNAT etc. In 5G networks these functions may be completely virtualized or could be a mix of virtualized functions and physical appliances. It is required that the inter-domain solution caters to the service function chaining requirements. The service functions may be virtualized and spread across different data centers attached to different domains.¶
Multicast services such as IPTV and multicast VPN also need to be supported across a multi-domain service provider network.¶
Figure 5 shows a simplified multi-domain network supporting multicast. Multicast sources S1 and S2 are in a different domain from the receivers R1 and R2. The solution should support establishment of intent-aware multicast distribution trees (P tunnels) across the domains and steer customer multicast streams on it. It should maintain the scaling properties of a multi-domain architecture by avoiding leaking of RPF routing state into the IGP domains.¶
In diagram Figure 5 above, AS1 and AS2 may be operating as closely coordinated but independent administrative domains, and still require end-to-end paths across the two ASes to deliver services. This scenario could be a result of a merger. It is possible that AS1 and AS2 may have assigned different values for the same intent.¶
In some cases, organizations may continue to use option A or option B [RFC4364] style interconnectivity in which case the inter-domain solution should satisfy intent of the path on inter-domain links for the service prefixes. In other cases, organizations may prefer to use option C style connectivity from PE1 to PE2.¶
An inter-domain solution should provide effective mechanisms to translate intent across domains without requiring renumbering of the intent mapping.¶
This section describes the basic concepts, terminologies and architectural principles that define intent-aware routing and the protocols and technologies that currently support it. The goal of this section is to establish the requirement for consistency with existing deployed solutions and describe the framework for it.¶
The figure below is used as reference.¶
Intent in routing may be any combination of the following behaviors:¶
An intent-aware routed path may be within a single network domain or across multiple domains.¶
Color is a 32-bit numerical value that is associated with an intent, as defined in [I-D.ietf-spring-segment-routing-policy]¶
An Egress PE E2 colors a BGP service (e.g. VPN) route V/v to indicate the particular intent that E2 requests for the traffic bound to V/v. The color (C) is encoded as a BGP Color Extended community [I-D.ietf-idr-tunnel-encaps].¶
(C, E2) represents a intent-aware route to E2 which satisfies the intent associated with color C.¶
Multiple technologies already provide intent-aware paths in solutions that are widely deployed.¶
In the context of large-scale SR-MPLS networks, SR Policy is applicable to both intra-domain and inter-domain deployments; whereas IGP Flex-Algo is better suited to intra-domain scenarios.¶
An ingress PE E1 automatically steers V-destined packets onto a intent-aware path bound to (C, E2). If several such paths exist, a preference scheme is used to select the best path: E.g. IGP Flex-Algo first, then SR Policy.¶
If E1 and E2 are in different domains, E1 may request an SR-PCE in its domain for a path to (C, E2). The SR-PCE (or a set of them) computes the end-to-end path and installs it at E1 as an SR Policy. The end-to-end intent-aware path may seamlessly cross multiple domains.¶
While the following requirements may be covered with an SR Policy solution, an operator may prefer a BGP-based solution due to:¶
A BGP Intent-Aware Routing solution signals intent-aware routes to reach a given destination (e.g. E2). (C, E2) represents a BGP hop-by-hop distributed route that builds an inter-domain intent-aware path to E2 for color C.¶
As seen above, multiple technologies exist that provide intent aware routing in a network. A BGP based solution must be compliant with the existing principles that apply to them.¶
A deployment model that provides consistency is as follows:¶
Service routes are colored using BGP Color Extended-Community to request intent [I.D.ietf-spring-segment-routing-policy]¶
Colored service routes are automatically steered on an appropriate intent-aware path using color¶
Intent-aware routes may resolve recursively via other intent-aware routes¶
Here is a brief example that illustrates these principles.¶
In the figure above, all the nodes are part of an inter-domain network under a single authority and with a consistent color-to-intent mapping:¶
Color C1 is mapped to "low delay"¶
Color C2 is mapped to "low delay and avoid resource R"¶
E1 receives two BGP colored service routes from E2:¶
E1 has the following inter-domain intent-aware paths using color:¶
(E2, C1) provided by BGP which recursively resolves via intra-domain intent-aware paths:¶
E1 automatically steers the received colored service routes as follows:¶
The example illustrates the benefits provided by leveraging the architectural principles:¶
Seamless co-existence of multiple intent-aware technologies, e.g. BGP and SR Policy¶
Seamless and complementary interworking between different intent-aware technologies¶
The BGP Intent-Aware routing solution must support the following intents bound to a color:¶
Minimization of a cost metric vs a latency metric¶
Inclusion of one or several virtual network function chains¶
Localization of the virtual network function chains¶
Subsequent sections elaborate on these requirements.¶
The requirements described in this document are mostly applicable to network under a single administrative domain that are organized into multiple network domains. The requirements are also applicable to multi-AS networks with closely cooperating administration.¶
The network diagram below illustrates the reference network topology used in this section¶
The following network design assumptions apply to the reference topology above, as an example:¶
Intent-aware inter-domain routing information to end point E with intent C is represented using (C,E). The notation used is a representation of the intent-aware route using color, and does not indicate a specific protocol encoding.¶
The following sections illustrate requirements and provide detailed examples for several intent types.¶
Various metric types can be advertised within an IGP domain and minimum metric paths can be computed within IGP domain, with Flex-Algo [I-D.ietf-lsr-flex-algo] for instance.¶
The BGP solution should allow the establishment of inter-domain intent-aware paths with low values of a metric type, accumulated over the end-to-end path.¶
In the reference topology of Figure 9¶
Cost Optimized end-to-end path¶
Example: PE11 learns (C1, PE31) intent-aware route via several equal paths:¶
Latency Optimized End-to-end path¶
The Intent-aware BGP routing solution should allow the establishment of inter-domain paths that satisfy link affinity inclusion/exclusion constraints. The link affinity constraints should also be satisfied for inter-domain links, such as those between ASBRs.¶
Using the reference topology of Figure 7 for the example below:¶
Example: PE11 learns (C3, PE31) intent-aware route via 2 paths.¶
Support creating an inter-domain path that includes or excludes a certain set of nodes in each domain.¶
Mechanisms used to achieve the node inclusion/exclusion constraints within different domains should be independent.¶
For example, an RSVP-based domain may use link affinities to achieve node exclusion constraints, while an SR-based domain may use Flex-Algo, which natively supports excluding nodes.¶
The example below describes the details for Figure 9¶
Color C4 - Intent to Minimize cost metric and avoid nodes¶
Example: PE11 learns (C4, PE31) intent-aware route via 1 path.¶
Support the creation of node- and link-diverse inter-domain paths.¶
The intra-domain portion of the end-to-end paths should make use of existing mechanisms for computing and instantiating diverse paths within a domain.¶
Inter-domain links (such as those connecting ASBRs) should also be taken into account for diverse inter-domain paths.¶
Support creation of inter-domain diverse paths that avoid shared risk links.¶
The example below describes the details for Figure 8¶
Color C5 and C6 - Intent to create diverse paths avoiding common node, link and shared risk¶
Shared risk among inter-domain links is as described in the topology description¶
The path is through PE11,121-211 (bottom link), 231-321 (bottom link), PE31¶
Support creation of paths with certain intents applicable to only a subset of domains.¶
No constraint specific state on internal nodes where intent is not applicable.¶
Color C4 - Avoid sending selected traffic via Domain 3¶
E11 and E21 MAY be involved in inter-domain signalling in order to send service traffic towards PEs in remote domains. Different functions may be collocated at the same network node. (For example, PE functionality and NFV attachment functionality may be collocated.)¶
This section describes requirements and reference use-cases for extending intent-aware routing to the VPN (Service) layer. Details for this section will be added in the next revision.¶
Traffic arriving at an ingress PE for a colored service route gets steered into an intent-aware path to the egress PE. Section 5.1.9 illustrates the automated steering mechanism, driven through Color Extended Community in the service route.¶
Flexible traffic steering is required, with support for different types:¶
When no path that fulfills the desired intent is available:¶
An option of ordered fallback should be supported¶
Fallback scheme per service route should be supported¶
The solution must support the representative deployment designs and associated deployment requirements described in the following sub sections.¶
This section describes four different ways that multi-domain networks could be organized. This is a representation of most common deployments and not an exhaustive coverage.¶
The above diagram shows three different IGP domains, Domain1, Domain2 and Domain3 inter-connected at the ABRs 121,122,231,232.¶
This single-AS network uses I-BGP sessions, with ABRs acting as inline route reflectors to PEs.¶
Note that the IGP design included here and in other models below is illustrative. In practice, there may be multiple areas/levels or multiple IGP instances.¶
The above diagram shows a single AS1 with three different IGP domains, Domain1, Domain2, and Domain3. 121,122,211,212,231,232,321,322 are border nodes for the IGP domains and they participate in only one IGP domain.¶
In this design, domain inter-connect is via iBGP peering links between Area border nodes. ABRs act as inline route reflectors to PEs.¶
The above diagram shows three different ASes (AS1, AS2 and AS3.) 121,122, 211, 212, 231,232, 321,322 are border nodes between the ASes.¶
In this design, domain inter-connect is via eBGP peering links between AS border nodes. The ASBR also runs I-BGP sessions with other ASBRs or RRs in the same AS.¶
121,122,231,232 belong to AS2 only. AS1 and AS2 domains may run multi-instance IGP or different levels/areas.¶
This topology uses I-BGP sessions to some clients and E-BGP sessions to other nodes. When an RR is used between PEs in AS1 and ABRs in AS2, it will have iBGP sessions to clients in same AS and e-BGP sessions to nodes in other AS.¶
BGP confederations [RFC 5065] allows the division of a public AS into multiple sub-ASes, usually with private identifiers. The solution should support BGP based intent-aware paths within the sub-AS or across the sub-ASes of the confederation, in any of the network designs described in sections 5.4.1.1 to section 5.4.1.4.¶
The solution must support the following:¶
A routing solution for end-to-end intent-aware paths should support multicast as well as unicast. This section will be updated in the next revision of the document.¶
The BGP intent-aware routing solution MUST be compliant with the intent-aware routing framework described in Section 5.1.9. Specifically,¶
BGP-LU [RFC3107] is widely deployed to provide inter-domain best-effort connectivity across different domains. The BGP intent-aware routing solution should support:¶
All domains in a network may not support the same number and granular definition of colors. However, the maximum granularity of colors should be provided for end to end paths that are set up for steering of a colored service route, with mapping from a more granular color to a less granular color where needed.¶
Section 5 describes co-existence and interworking of the BGP intent aware routing solution with other existing intent-aware solutions.¶
Controller based approaches or other distributed TE solutions can also address the use-cases in this document.¶
The intent-aware routing solution should coexist with such alternative solutions.¶
Support a massive scaled transport network¶
Constraints that need to be addressed:¶
To address these constraints:¶
Support ability to abstract the topology and network events from remote domains - for scale, stability and faster convergence.¶
PE nodes may be devices with limited CPU and memory. The state on a PE should be restricted to transport endpoints that it needs for service steering.¶
BGP Signaling is natively a PUSH model.¶
For comparison, the SR-PCE solution natively supports a PULL model: when PE1 installs a VPN route V/v via (C, PE2), PE1 requests its serving SR-PCE to compute the SR Policy to (C, PE2). I.e. PE1 does not learn unneeded SR policies.¶
Emulated-PULL refers to the ability for a BGP node PE1 to "subscribe" to (C, PE2) route such that only paths for (C, PE2) are signaled to PE1.¶
The requirements for an Emulated-PULL solution are as follows:¶
For transport routes, this means¶
Automation of the subscription/filter route¶
Efficient propagation and processing of subscription/filter routes.¶
This section will be updated in the future revision of the document.¶
In the Seamless-MPLS reference topology in section 5.4.1.1 :¶
In the Inter-AS Option C VPN reference topology in Section 5.4.1.3:¶
This section summarizes the key protocol requirements that should be addressed by the intent-aware BGP routing solution. While the context for several requirements has been discussed earlier in the document, this section emphasizes aspects pertinent to the protocol design.¶
The solution should support the following:¶
Signaling and distribution of different Intent-aware routes to reach a participating node, e.g. a PE. Intent must be indicated by the notion of a Color as defined in [I-D.ietf-spring-segment-routing-policy]¶
Path selection for Intent-aware routes¶
Validation of received paths¶
Next-hop resolution for BGP Intent-aware route¶
Flexibility to use different intra-domain and inter-domain mechanisms, both intent-aware and traditional¶
Flexible, efficient, extensible protocol definition¶
Hence, the protocol definition should¶
Separation of transport and VPN service semantics¶
This section will be updated in the future revision of the document.¶
This section will be updated in the future revision of the document.¶
This section will be updated in the future version of the document.¶
This section will be updated in the future version of the document.¶
This section will be updated in the future version of the document.¶
The authors would especially like to thank Joel Halpern for his guidance on the collaboration work that has produced this document and feedback on many aspects of the problem statement.¶
We would like to thank Daniel Voyer, Robert Raszuk, Kireeti Kompella, Ron Bonica, Krzysztof Szarkowicz, Julian Lucek, Ram Santhanakrishnan, Stephane Litkowski for discussions and inputs.¶
We also express our appreciation to Hannes Gredler Simon Spraggs, Jose Liste and Jiri Chaloupka for discussions that have helped provide input to the problem statement.¶
Many thanks to Colby Barth, John Scudder, Kamran Raza, Kris Michelson, Huaimo Chen for their review and valuable suggestions.¶
1.Kaliraj Vairavakkalai¶
Juniper Networks¶
kaliraj@juniper.net¶
2. Jeffrey Zhang¶
Juniper Networks¶
zzhang@juniper.net¶