view Side-By-Side changes
Congestion and Pre-Congestion Philip. Eardley (Editor) NotificationNetwork Working Group P. Eardley, Ed. Request for Comments: 5559 BTInternet-Draft April 7, 2009 Intended status:Category: InformationalExpires: October 9,June 2009 Pre-Congestion Notification (PCN) Architecturedraft-ietf-pcn-architecture-11Status ofthisThis Memo ThisInternet-Draftmemo provides information for the Internet community. It does not specify an Internet standard of any kind. Distribution of this memo is unlimited. Copyright Notice Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved. This document issubmittedsubject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents infull conformance witheffect on theprovisionsdate ofBCP 78publication of this document (http://trustee.ietf.org/license-info). Please review these documents carefully, as they describe your rights andBCP 79.restrictions with respect to this document. This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English.Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on October 9, 2009. Copyright Notice Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved. Eardley (Editor) Expires October 9, 2009 [Page 1] Internet-Draft PCN Architecture April 2009 This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents in effect on the date of publication of this document (http://trustee.ietf.org/license-info). Please review these documents carefully, as they describe your rights and restrictions with respect to this document.Abstract This document describes a general architecture for flow admission and termination based on pre-congestion information in order to protect the quality of service ofestablishedestablished, inelastic flows within a single Diffserv domain.StatusEardley Informational [Page 1] RFC 5559 PCN Architecture June 2009 Table of Contents 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . 4....................................................3 1.1. Overview of PCN. . . . . . . . . . . . . . . . . . . . . 4............................................3 1.2. Exampleuse caseUse Case for PCN. . . . . . . . . . . . . . . . 4...................................4 1.3. Applicability of PCN. . . . . . . . . . . . . . . . . . 8.......................................7 1.4. Documents about PCN. . . . . . . . . . . . . . . . . . . 9........................................8 2. Terminology. . . . . . . . . . . . . . . . . . . . . . . . . 10.....................................................9 3.High-level functional architecture . . . . . . . . . . . . . . 12High-Level Functional Architecture .............................11 3.1. Flowadmission . . . . . . . . . . . . . . . . . . . . . 14Admission ............................................13 3.2. Flowtermination . . . . . . . . . . . . . . . . . . . . 15Termination ..........................................14 3.3. FlowadmissionAdmission and/orflow termination when there are only twoFlow Termination When There Are Only Two PCNencoding states . . . . . . . . . . . . . . 16Encoding States ...................................15 3.4. Informationtransport . . . . . . . . . . . . . . . . . . 17Transport .....................................16 3.5.PCN-traffic . . . . . . . . . . . . . . . . . . . . . . . 17PCN-Traffic ...............................................16 3.6. Backwardscompatibility . . . . . . . . . . . . . . . . . 18Compatibility ...................................17 4. Detailed Functionalarchitecture . . . . . . . . . . . . . . . 19Architecture ...............................18 4.1.PCN-interior-node functions . . . . . . . . . . . . . . . 20PCN-Interior-Node Functions ...............................19 4.2.PCN-ingress-node functions . . . . . . . . . . . . . . . 21PCN-Ingress-Node Functions ................................19 4.3.PCN-egress-node functions . . . . . . . . . . . . . . . . 22PCN-Egress-Node Functions .................................20 4.4. Admissioncontrol functions . . . . . . . . . . . . . . . 22Control Functions ...............................21 4.5. Flowtermination functions . . . . . . . . . . . . . . . 23Termination Functions ................................22 4.6. Addressing. . . . . . . . . . . . . . . . . . . . . . . 24................................................22 4.7. Tunnelling. . . . . . . . . . . . . . . . . . . . . . . 24................................................23 4.8. Faulthandling . . . . . . . . . . . . . . . . . . . . . 26Handling ............................................25 5. Operations and Management. . . . . . . . . . . . . . . . . . 26......................................25 5.1.ConfigurationFault Operations and Management. . . . . . . . . 27 5.1.1. System options . . . . . . . . . . . . . . . . . . . . 27 5.1.2. Parameters . . . . . . . . . . . . . . . . . . . . . . 28...........................25 5.2.Performance & ProvisioningConfiguration Operations and Management. . 30...................26 5.2.1. System Options .....................................27 5.2.2. Parameters .........................................28 5.3. Accounting Operations and Management. . . . . . . . . . 31......................30 5.4.FaultPerformance and Provisioning Operations and Management. . . . . . . . . . . . . 31 Eardley (Editor) Expires October 9, 2009 [Page 2] Internet-Draft PCN Architecture April 2009....30 5.5. Security Operations and Management. . . . . . . . . . . 32........................31 6. Applicability of PCN. . . . . . . . . . . . . . . . . . . . . 33...........................................32 6.1. Benefits. . . . . . . . . . . . . . . . . . . . . . . . 33..................................................32 6.2. Deploymentscenarios . . . . . . . . . . . . . . . . . . 35Scenarios ......................................33 6.3. Assumptions andconstraintsConstraints onscope . . . . . . . . . . 36Scope ......................35 6.3.1. Assumption 1: Trust andsupportSupport of PCN -controlled environment . . . . . . . . . . . . . . . . 37Controlled Environment .............................36 6.3.2. Assumption 2:Real-time applications . . . . . . . . . 37Real-Time Applications ...............36 6.3.3. Assumption 3: ManyflowsFlows andadditional load . . . . . 38Additional Load .......37 6.3.4. Assumption 4: Emergencyuse outUse Out ofscope . . . . . . . 38Scope ...........37 6.4. Challenges. . . . . . . . . . . . . . . . . . . . . . . 39................................................37 7.IANASecurity Considerations. . . . . . . . . . . . . . . . . . . . . 41........................................40 8.Security considerations . . . . . . . . . . . . . . . . . . . 41 9.Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 42 10.....................................................41 9. Acknowledgements. . . . . . . . . . . . . . . . . . . . . . . 42 11. Comments Solicited (to be removed by RFC Editor) . . . . . . . 43 12. Changes (to be removed by...............................................41 Eardley Informational [Page 2] RFCEditor) . . . . . . . . . . . . 43 12.1. Changes from -10 to -11 . . . . . . . . . . . . . . . . . 43 12.2. Changes from -09 to -10 . . . . . . . . . . . . . . . . . 44 12.3. Changes from -08 to -09 . . . . . . . . . . . . . . . . . 44 12.4. Changes from -07 to -08 . . . . . . . . . . . . . . . . . 44 12.5. Changes from -06 to -07 . . . . . . . . . . . . . . . . . 45 12.6. Changes from -05 to -06 . . . . . . . . . . . . . . . . . 45 12.7. Changes from -04 to -05 . . . . . . . . . . . . . . . . . 46 12.8. Changes from -03 to -04 . . . . . . . . . . . . . . . . . 46 12.9. Changes from -02 to -03 . . . . . . . . . . . . . . . . . 47 12.10. Changes from -01 to -02 . . . . . . . . . . . . . . . . . 48 12.11. Changes from -00 to -01 . . . . . . . . . . . . . . . . . 49 13.5559 PCN Architecture June 2009 10. References. . . . . . . . . . . . . . . . . . . . . . . . . . 51 13.1.....................................................42 10.1. Normative References. . . . . . . . . . . . . . . . . . 51 13.2......................................42 10.2. Informative References. . . . . . . . . . . . . . . . . 51...................................42 Appendix A. Possiblefuture work items . . . . . . . . . . . . . 55Future Work Items ...........................48 A.1. Probing. . . . . . . . . . . . . . . . . . . . . . . . . 57.................................................50 A.1.1. Introduction. . . . . . . . . . . . . . . . . . . . . 57....................................50 A.1.2. Probingfunctions . . . . . . . . . . . . . . . . . . 58Functions ...............................50 A.1.3. Discussion ofrationaleRationale forprobing, its downsidesProbing, Its Downsides andopen issues . . . . . . . . . . . . . . . . . . . 58 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 61 Eardley (Editor) Expires October 9, 2009 [Page 3] Internet-Draft PCN Architecture April 2009Open Issues .......................51 1. Introduction 1.1. Overview of PCN The objective of Pre-Congestion Notification (PCN) is to protect the quality of service (QoS) of inelastic flows within a Diffservdomain,domain in a simple,scalablescalable, and robust fashion. Two mechanisms are used: admission control, to decide whether to admit or block a new flow request, and (in abnormal circumstances) flowterminationtermination, to decide whether to terminate some of the existing flows. To achieve this, the overall rate ofPCN trafficPCN-traffic is metered on every link in the domain, and PCN packets are appropriately marked when certain configured rates are exceeded. These configured rates are below the rate of thelinklink, thus providing notification to boundary nodes about overloads before any congestion occurs(hence "pre-congestion notification").(hence, "Pre-Congestion Notification"). The level of marking allows boundary nodes to make decisions about whether to admit or terminate. Within a PCN-domain, PCN-traffic is forwarded in a prioritised Diffserv traffic class. Every link in the PCN-domain is configured with two rates (PCN-threshold-rate and PCN-excess-rate). If the overall rate of PCN-traffic on a link exceeds a configured rate, then a PCN-interior-node marks PCN-packets appropriately. The PCN-egress- nodes use this information to make admission control and flow termination decisions. Flow admission control determines whether a new flow can be admitted without any impact, in normal circumstances, on the QoS of existing PCN-flows. However, in abnormalcircumstances, for instancecircumstances (for instance, a disaster affecting multiple nodes and causing trafficre-routes, thenre-routes), the QoS on existing PCN-flows may degrade even though care was exercised when admitting those flows. The flow termination mechanism removes sufficient traffic in order to protect the QoS of the remaining PCN-flows. All PCN-boundary-nodes andPCN-interior-nodesPCN- interior-nodes are PCN-enabled and are trusted for correct PCN operation. PCN-ingress-nodes police arriving packets to check that they are part of an admitted PCN-flow that keeps within its agreed flowspec, and hence they maintainper flowper-flow state.PCN- interior-nodesPCN-interior-nodes meter allPCN traffic,PCN-traffic, and hence do not need to maintain anyper flowper-flow Eardley Informational [Page 3] RFC 5559 PCN Architecture June 2009 state. Decisions about flow admission and termination are made for a particular pair of PCN-boundary-nodes, and hence PCN-egress-nodes must be able to identify whichPCN-ingress- nodePCN-ingress-node sent each PCN-packet. 1.2. Exampleuse caseUse Case for PCN This section outlines an end-to-end QoS scenario that uses the PCN mechanisms within one domain. The parts outside the PCN-domain are out of scope for PCN, but are included to help clarify how PCN could be used. Note thatthethis section is only an example--- inparticular Eardley (Editor) Expires October 9, 2009 [Page 4] Internet-Draft PCN Architecture April 2009particular, there are other possibilities (seelater)Section 3) for how thePCN-boundary- nodesPCN- boundary-nodes perform admission control and flow termination. As a fundamental building block, each link of the PCN-domain operatesa [PCN08-2] (Figure 1):the following. Please refer to [Eardley09] and Figure 1. oThresholdA threshold meter and marker, which marks all PCN-packets if thePCN trafficrate of PCN-traffic is greater than a first configured rate, thePCN- threshold-rate.PCN-threshold-rate. The admission control mechanism limits thePCN- trafficPCN-traffic on each link to *roughly* its PCN-threshold-rate. oExcess trafficAn excess-traffic meter and marker, which marks a proportion ofPCN- packets,PCN-packets such that the amount marked equals the traffic rate in excess of a second configured rate, the PCN-excess-rate. The flow termination mechanism limits the PCN-traffic on each link to *roughly* its PCN-excess-rate.OverallOverall, the aim is to give an "early warning" of potential congestion before there is any significant build-up of PCN-packets in the queue on the link; we term this"pre-congestion notification""Pre-Congestion Notification" by analogy with ECN (Explicit Congestion Notification, [RFC3168]). Note that the link only meters the bulk PCN-traffic (and not per flow). Eardley(Editor) Expires October 9, 2009Informational [Page5] Internet-Draft4] RFC 5559 PCN ArchitectureAprilJune 2009 == Metering & == ==Marking behaviour== ==PCN mechanisms== ^ Rate of ^ PCN-traffic on | bottleneck link | | | Some pkts Terminate some | excess-traffic-marked admitted flows | & & | Rest of pkts Block new flows | threshold-marked | PCN-excess-rate -|------------------------------------------------ (=PCN-supportable-rate)| | All pkts Block new flows | threshold-marked | PCN-threshold-rate -|------------------------------------------------ (=PCN-admissible-rate)| | No pkts Admit new flows | PCN-marked | Figure 1: Example of how the PCN admission control and flow termination mechanisms operate as the rate of PCN-traffic increases. The two forms of PCN-marking are indicated by settingofthe ECN and DSCP (Differentiated Services Codepoint [RFC2474]) fields to known values, which are configured for the domain.ThusThus, the PCN-egress- nodes can monitor the PCN-markings in order to measure the severity of pre-congestion. In addition, the PCN-ingress-nodes need to set the ECN and DSCP fields to that configured for an unmarked PCN- packet, and the PCN-egress-nodes need to revert to values appropriate outside the PCN-domain. For admission control, we assume end-to-end RSVPsignalling(Resource Reservation Protocol) [RFC2205]) signalling in this example. The PCN-domain is a single RSVP hop. The PCN-domain operates Diffserv, and we assume that PCN-traffic is scheduled with the expedited forwarding (EF)per- hop behaviour,per-hop behaviour [RFC3246].HenceHence, the overall solution is in line with the "IntServ over Diffserv" framework defined in [RFC2998], as shown in Figure 2. Eardley(Editor) Expires October 9, 2009Informational [Page6] Internet-Draft5] RFC 5559 PCN ArchitectureAprilJune 2009 ___ ___ _______________________________________ ____ ___ | | | | | PCN- PCN- PCN- | | | | | | | | | |ingress interior egress| | | | | | | | | | -node -nodes -node | | | | | | | | | |-------+ +-------+ +-------+ +------| | | | | | | | | | | | PCN | | PCN | | | | | | | | |..| |..|Ingress|..|meter &|..|meter &|..|Egress|..| |..| | | |..| |..|Policer|..|marker |..|marker |..|Meter |..| |..| | | | | | |-------+ +-------+ +-------+ +------| | | | | | | | | | \ / | | | | | | | | | | \ / | | | | | | | | | | \ PCN-feedback-information / | | | | | | | | | | \ (for admission control) / | | | | | | | | | | --<-----<----<----<-----<-- | | | | | | | | | | PCN-feedback-information | | | | | | | | | | (for flow termination) | | | | | |___| |___| |_______________________________________| |____| |___| Sx Access PCN-domain Access Rx End Network Network End Host Host <---- signalling across PCN-domain---> (for admission control & flow termination) <-------------------end-to-end QoS signalling protocol---------------> Figure 2: Example of possible overall QoSarchitecturearchitecture. A source wanting to start a new QoS flow sends an RSVP PATH message. Normal hop-by-hop IntServ [RFC1633] is used outside the PCN-domain (we assume successfully). The PATH message travels across the PCN- domain; the PCN-egress-node reads the PHOP (previous RSVP hop) object to discover the specific PCN-ingress-node for this flow. The RESV message travels back from the receiver, and triggers thePCN-egress-nodePCN-egress- node to check what fraction of thePCN-traffic,PCN-traffic from the relevantPCN-ingress-node,PCN- ingress-node is currently being threshold-marked. It adds an object with this information onto the RESV message, and hence thePCN-ingress-nodePCN- ingress-node learns about the level of pre-congestion on the path. If this level is below some threshold, then the PCN-ingress-node admits the new flow into the PCN-domain. The RSVP message triggers thePCN-ingress- nodePCN-ingress-node to install two normal IntServ items: five-tuple information, so that it can subsequently identify data packets that are part of a previously admittedPCN-flow;PCN-flow, and a traffic profile, so that it can police the flow to within itscontract.reservation. Similarly, the RSVP message triggers the PCN-egress-node to install five-tuple and PHOPinformation,information so that it can identify packets as part of a flow from a specific PCN-ingress-node. Eardley(Editor) Expires October 9, 2009Informational [Page7] Internet-Draft6] RFC 5559 PCN ArchitectureAprilJune 2009 The flow termination mechanism may happen when some abnormalcircumstancescircumstance causes a link to become so pre-congested that itexcess-traffic-marksexcess- traffic-marks (and perhaps also drops) PCN-packets. In this example, when a PCN-egress-node observes such apacketpacket, it then, with some probability, terminates this PCN-flow; the probability is configured low enough to avoidover-terminationover termination and high enough to ensure rapid termination of enough flows. It also informs the relevantPCN-ingress-node,PCN- ingress-node so that it can block any further traffic on the terminated flow. 1.3. Applicability of PCN Compared with alternative QoS mechanisms, PCN has certain advantages and disadvantages that will make it appropriate in particular scenarios. For example, compared with hop-by-hop IntServ [RFC1633], PCN only requiresper flowper-flow state at the PCN-ingress-nodes. Compared with the Diffserv architecture [RFC2475], an operator needs to be less accurate and/or conservative in its prediction of the traffic matrix. The Diffserv architecture'straffic conditioningtraffic-conditioning agreements are static and coarse; they are defined at subscriptiontime,time andtheyare used (for instance) to limit the total traffic at each ingress of thedomaindomain, regardless of the egress for the traffic. On the other hand, PCN firstly uses admission control based on measurements of the current conditions between the specific pair of PCN-boundary-nodes, and secondly, in case of a disaster, PCN protects the QoS of most flows by terminating a few selected ones. PCN's admission control is a measurement-based mechanism.HenceHence, it assumes that the present is a reasonable prediction of the future: the network conditions are measured at the time of a new flow request, but the actual network performance must be acceptable during the call some time later.HenceHence, PCN is unsuitable in several circumstances: o If the source adapts its bit rate dependent on the level of pre- congestion, because then the aggregate traffic might become unstable. The assumption in this document is that PCN-packets come fromreal timereal-time applications generating inelastic traffic, such as the Controlled LoadService,Service [RFC2211]. o If a potential bottleneck link has capacity for only a few flows, because then a new flow can move a link directly from no pre- congestion to being so overloaded that it has to drop packets. The assumption in this document is that this isn't a problem. o If there is the danger of a "flashcrowd"crowd", in which many admission requests arrive within the reaction time of PCN's admission mechanism, because then they all might get admitted and so Eardley(Editor) Expires October 9, 2009Informational [Page8] Internet-Draft7] RFC 5559 PCN ArchitectureAprilJune 2009 overload the network. The assumption in this document is that, if it is necessary, then flash crowds are limited in some fashion beyond the scope of this document, for instance byrate limitingrate-limiting QoS requests. The applicability of PCN is discussed further in Section 6. 1.4. Documents about PCN The purpose of this document is to describe a general architecture for flow admission and termination based on(pre-) congestion(pre-)congestion information in order to protect the quality of service of flows within a Diffserv domain. This document describes the PCN architecture at a high level (Section 3) and in more detail (Section 4). It also defines some terminology, and provides considerations aboutoperations andoperations, management, and security. Section 6 considers the applicability of PCN in more detail, covering its benefits, deployment scenarios,assumptionsassumptions, and potential challenges. The Appendix covers some potential future work items. Aspects of PCN are also documented elsewhere: o Metering and marking:[PCN08-2][Eardley09] standardises threshold metering andmarking,marking andexcess trafficexcess-traffic metering and marking. APCN- packetPCN-packet may be marked, depending on the metering results. o Encoding: the "baseline" encoding is described in[PCN08-1],[Moncaster09-1], which standardises two PCN encoding states (PCN-marked and notPCN- marked),PCN-marked), whilst (experimental) extensions to the baseline encoding can provide three encoding states (threshold-marked,excess- traffic-marked, not PCN-marked,excess-traffic-marked, orperhapsnot PCN-marked), for instance, see [Moncaster09-2]. (There may be further encoding states as suggested in[Westberg08]).[Westberg08].) Section 3.6 considers the backwardscompatabilitycompatibility of PCN encoding with ECN. o PCN-boundary-node behaviour: how the PCN-boundary-nodes convert the PCN-markings into decisions about flow admission and flow termination, as described in Informationaldocuments.documents such as [Taylor09] and [Charny07-2]. The concept is that the standardised metering and marking by PCN-nodes allows several possiblePCN-boundary-nodePCN- boundary-node behaviours. A number of possibilities are outlined in this document; detailed descriptions and comparisons are in [Charny07-1] and[Menth08-3].[Menth09-2]. o Signalling between PCN-boundary-nodes:Signallingsignalling is needed to transport PCN-feedback-information between the PCN-boundary-nodes (in the example above, this is the fraction of traffic, between the pair of PCN-boundary-nodes, that is PCN-marked). The exact Eardley Informational [Page 8] RFC 5559 PCN Architecture June 2009 details vary for different PCN-boundary-node behaviours, and so should be described in those documents. It may require anEardley (Editor) Expires October 9, 2009 [Page 9] Internet-Draft PCN Architecture April 2009extension to the signalling protocol--- standardisation is out of scope of the PCN WG. o The interface by which the PCN-boundary-nodes learn identification information about the admitted flows: the exact requirements vary for different PCN-boundary-node behaviours and for different signalling protocols, and so should be described in those documents. They will be similar to those described in the example above--- a PCN-ingress-node needs to be able to identify that a packet is part of a previously admitted flow (typically from its five-tuple) and each PCN-boundary-node needs to be able to identify the other PCN-boundary-node for the flow. 2. Terminology o PCN-domain: a PCN-capable domain; a contiguous set of PCN-enabled nodes that perform Diffserv scheduling [RFC2474]; the complete set of PCN-nodes that in principle can, through PCN-marking packets, influence decisions about flow admission and termination for the PCN-domain;the PCN-domainincludes the PCN-egress-nodes, which measure these PCN-marks, and the PCN-ingress-nodes. o PCN-boundary-node: a PCN-node that connects one PCN-domain to a node either in another PCN-domain or in anon PCN-domain.non-PCN-domain. o PCN-interior-node: a node in a PCN-domain that is not a PCN- boundary-node. o PCN-node: a PCN-boundary-node or aPCN-interior-nodePCN-interior-node. o PCN-egress-node: a PCN-boundary-node in its role in handling traffic as it leaves a PCN-domain. o PCN-ingress-node: a PCN-boundary-node in its role in handling traffic as it enters a PCN-domain. o PCN-traffic, PCN-packets, PCN-BA: a PCN-domain carries traffic of different Diffserv behaviour aggregates (BAs) [RFC2474]. The PCN-BA uses the PCN mechanisms to carryPCN-trafficPCN-traffic, and the corresponding packets are PCN-packets. The same network will carry traffic of other Diffserv BAs. The PCN-BA is distinguished by a combination of the Diffserv codepoint (DSCP) and ECN fields. Eardley Informational [Page 9] RFC 5559 PCN Architecture June 2009 o PCN-flow: the unit of PCN-traffic that the PCN-boundary-node admits (or terminates); the unit could be a single microflow (as defined in [RFC2474]) or some identifiable collection of microflows.Eardley (Editor) Expires October 9, 2009 [Page 10] Internet-Draft PCN Architecture April 2009o Pre-congestion: a condition of a link within a PCN-domain such that the PCN-node performs PCN-marking, in order to provide an "early warning" of potential congestion before there is any significant build-up of PCN-packets in the real queue. (Hence, by analogy withECNECN, we call our mechanism Pre-Congestion Notification.) o PCN-marking: the process of setting the header in a PCN-packet based on defined rules, in reaction to pre-congestion; either threshold-marking or excess-traffic-marking.o PCN-threshold-rate: a reference rate configured for each link in the PCN-domain, which is lower than the PCN-excess-rate. It is used by a metering behaviour that determines whetherSuch a packetshould be PCN-marked with a first encoding, "threshold-marked".is then called PCN-marked. o Threshold-metering: a metering behaviour that, if the PCN-traffic exceeds the PCN-threshold-rate, indicates that all PCN-traffic is to be threshold-marked. o PCN-threshold-rate: the reference rate of a threshold-meter, which is configured for each link in the PCN-domain and which is lower than the PCN-excess-rate. o Threshold-marking: the setting of the header in a PCN-packet to a specific encoding, based on indications from the threshold-meter.o PCN-excess-rate: a reference rate configured for each link in the PCN-domain, which is higher than the PCN-threshold-rate. It is used by a metering behaviour that determines whetherSuch a packetshould be PCN-marked with a second encoding, "excess-traffic- marked".is then called threshold-marked. o Excess-traffic-metering: a metering behaviour that, if the PCN- traffic exceeds the PCN-excess-rate, indicates that the amount of PCN-traffic to bePCN-markedexcess-traffic-marked is equal to the amount in excess of the PCN-excess-rate. o PCN-excess-rate: the reference rate of an excess-traffic-meter, which is a configured for each link in the PCN-domain and which is higher than the PCN-threshold-rate. o Excess-traffic-marking: the setting of the header in a PCN-packet to a specific encoding, based on indications from the excess- traffic-meter. Such a packet is then called excess-traffic- marked. o PCN-colouring: the process of setting the header in a PCN-packet by a PCN-boundary-node; performed by a PCN-ingress-node so that PCN-nodes can easily identify PCN-packets; performed by a PCN- egress-node so that the header is appropriate for nodes beyond the PCN-domain. Eardley Informational [Page 10] RFC 5559 PCN Architecture June 2009 o Ingress-egress-aggregate: The collection of PCN-packets from all PCN-flows that travel in one direction between a specific pair of PCN-boundary-nodes.Eardley (Editor) Expires October 9, 2009 [Page 11] Internet-Draft PCN Architecture April 2009o PCN-feedback-information: information signalled by a PCN-egress- node to a PCN-ingress-node (or a central control node), which is needed for the flow admission and flow termination mechanisms. o PCN-admissible-rate: the rate of PCN-traffic on a link up to which PCN admission control should accept new PCN-flows. o PCN-supportable-rate: the rate of PCN-traffic on a link down to which PCN flow termination should, if necessary, terminate already admitted PCN-flows. 3.High-level functional architectureHigh-Level Functional Architecture The high-level approach is to split functionality between: o PCN-interior-nodes'inside'"inside" the PCN-domain, which monitor their own state of pre-congestion and mark PCN-packets as appropriate. They are not flow-aware, nor are they aware ofingress-egress-aggregates.ingress-egress- aggregates. The functionality is also done by PCN-ingress-nodes for their outgoing interfaces(ie(ie, those'inside'"inside" the PCN-domain). o PCN-boundary-nodes at the edge of the PCN-domain, which control admission of new PCN-flows and termination of existing PCN-flows, based on information from PCN-interior-nodes. This information is in the form of the PCN-marked data packets (which are intercepted by the PCN-egress-nodes) and is not in signalling messages.GenerallyGenerally, PCN-ingress-nodes are flow-aware. The aim of this split is to keep the bulk of the network simple,scalablescalable, and robust, whilst confining policy,application-levelapplication-level, and security interactions to the edge of the PCN-domain. Forexampleexample, the lack of flow awareness means that the PCN-interior-nodes don't care about the flow information associated with PCN-packets, nor do the PCN-boundary-nodes care about which PCN-interior-nodes itsingress- egress-aggregatesingress-egress-aggregates traverse. In order to generate information about the current state of the PCN- domain, each PCN-node PCN-marks packets if it is "pre-congested". Exactly when a PCN-node decides if it is "pre-congested" (the algorithm) and exactly how packets are "PCN-marked" (the encoding) will be defined in separatestandards-trackStandards Track documents, but at a high level it is as follows: Eardley Informational [Page 11] RFC 5559 PCN Architecture June 2009 o the algorithms: a PCN-node meters the amount of PCN-traffic on each one of its outgoing (or incoming) links. The measurement is made as an aggregate of all PCN-packets,andnot per flow. There are twoalgorithms,algorithms: one for threshold-metering and one for excess-Eardley (Editor) Expires October 9, 2009 [Page 12] Internet-Draft PCN Architecture April 2009traffic-metering. The meters trigger PCN-marking as necessary. o the encoding(s): a PCN-node PCN-marks a PCN-packet by modifying a combination of the DSCP and ECN fields. In the "baseline" encoding[PCN08-1],[Moncaster09-1], the ECN field is set to 11 and the DSCP is not altered. Extension encodings may be defined that, at most, use a second DSCP(eg(eg, as in[Moncaster08])[Moncaster09-2]) and/or set the ECN field to values other than 11(eg(eg, as in [Menth08-2]). In aPCN-domainPCN-domain, the operator may have two or three encoding states available. The baseline encoding provides two encoding states (notPCN-marked,PCN-marked and PCN-marked), whilst extended encodings can provide three encoding states (not PCN-marked, threshold-marked,excess-traffic- marked).and excess- traffic-marked). An operator may choose to deploy either admission control or flow termination or both. Although designed to work together, they are independent mechanisms, and the use of one does not require or prevent the use of the other. Three encoding states naturally allows both flow admission and flow termination. If there are only two encoding states, then there are several options--- see Section 3.3. The PCN-boundary-nodes monitor the PCN-marked packets in order to extract information about the current state of the PCN-domain. Based on this monitoring, a distributed decision is made about whether to admit a prospective new flow orwhether toterminate existing flow(s). Sections 4.4 and 4.5 mention various possibilities for how the functionality could be distributed. PCN-metering and PCN-markingneedsneed to be configured on all (potentially pre-congested) links in the PCN-domain to ensure that the PCN mechanisms protect all links. The actual functionality can be configured on the outgoing or incoming interfaces of PCN-nodes--- or one algorithm could be configured on the outgoing interface and the other on the incoming interface. The important point is that a consistent choice is made across the PCN-domain to ensure that the PCN mechanisms protect all links. See[PCN08-2][Eardley09] for further discussion. The objective of threshold-marking, astriggerdtriggered by the threshold- metering algorithm, is to threshold-mark all PCN-packets whenever the bit rate of PCN-packets is greater than some configured rate, thePCN- threshold-rate.PCN-threshold-rate. The objective of excess-traffic-metering, as triggered by the excess-traffic-marking algorithm, is to excess- Eardley Informational [Page 12] RFC 5559 PCN Architecture June 2009 traffic-mark PCN-packets at a rate equal to the difference between the bit rate of PCN-packets and some configured rate, the PCN-excess- rate. Note that this description reflects the overall intent of the algorithms rather than their instantaneous behaviour, since the rateEardley (Editor) Expires October 9, 2009 [Page 13] Internet-Draft PCN Architecture April 2009measured at a particular moment depends on the detailed algorithm, its implementation, and the traffic's variance as well as its rate(eg(eg, marking may well continue after a recentoverloadoverload, even after the instantaneous rate has dropped). The algorithms are specified in[PCN08-2].[Eardley09]. Admission and termination approaches are detailed and compared in [Charny07-1] and[Menth08-3].[Menth09-2]. The discussion below is just a brief summary. Sections 3.1 and 3.2 assume there are three encoding states available, whilst Section 3.3 assumes there are two encoding states available. From the perspective of the outside world, a PCN-domain essentially looks like a Diffserv domain, but without the Diffserv architecture'straffic conditioningtraffic-conditioning agreements. PCN-traffic is either transported across it transparently or policed at the PCN-ingress-node(ie(ie, dropped or carried at a lower QoS). One difference is that PCN- traffic has better QoS guarantees than normal Diffservtraffic,traffic because the PCN mechanisms better protect the QoS of admitted flows. Another difference may occur in the rare circumstance when there is a failure: on the onehandhand, some PCN-flows may getterminated, butterminated but, on the otherhandhand, other flows will get their QoS restored.Non PCN-Non-PCN- traffic is treated transparently,ieie, the PCN-domain is a normal Diffserv domain. 3.1. FlowadmissionAdmission The objective of PCN's flow admission control mechanism is to limit the PCN-traffic on each link in the PCN-domain to *roughly* its PCN-admissible-rate,admissible-rate by admitting or blocking prospective new flows, in order to protect the QoS of existing PCN-flows. With three encoding states available, the PCN-threshold-rate is configured by the operator as equal to the PCN-admissible-rate on each link. It is set lower than the traffic rate at which the link becomes congested and the node drops packets. Exactly how the admission control decision is made will be defined separately ininformationalInformational documents. This document describes two approaches (others might be possible): otheThe PCN-egress-node measures (possibly as a moving average) the fraction of the PCN-traffic that is threshold-marked. The fraction is measured for a specific ingress-egress-aggregate. If the fraction is below a thresholdvaluevalue, then the new flow isadmitted, andEardley Informational [Page 13] RFC 5559 PCN Architecture June 2009 admitted; if the fraction is above the thresholdvaluevalue, then it is blocked. The fraction could be measured as an EWMA (exponentially weighted moving average), which has sometimes been called the "congestion level estimate".Eardley (Editor) Expires October 9, 2009 [Page 14] Internet-Draft PCN Architecture April 2009otheThe PCN-egress-node monitors PCN-traffic and if it receives one (or several) threshold-marked packets, then the new flow isblocked, otherwiseblocked; otherwise, it is admitted. One possibility may be to react to the marking state of an initialflow set-upflow-setup packet(eg(eg, RSVP PATH). Another is that after one (or several) threshold-marks thenmarks, all flows are blocked until after a specific period of no congestion. Note that the admission control decision is made for a particular pair of PCN-boundary-nodes. So it is quite possible for a new flow to be admitted between one pair of PCN-boundary-nodes, whilst at the same time another admission request is blocked between a different pair of PCN-boundary-nodes. 3.2. FlowterminationTermination The objective of PCN's flow termination mechanism is to limit the PCN-traffic on each link to *roughly* its PCN-supportable-rate, by terminating some existing PCN-flows, in order to protect the QoS of the remaining PCN-flows. With three encoding states available, the PCN-excess-rate is configured by the operator as equal to the PCN- supportable-rate on each link. It may be set lower than the traffic rate at which the link becomes congested and at which the node drops packets. Exactly how the flow termination decision is made will be defined separately ininformationalInformational documents. This document describes several approaches (others might be possible): o In oneapproachapproach, the PCN-egress-node measures the rate of PCN- traffic that is not excess-traffic-marked, which is the amount of PCN-traffic that can actually be supported, and communicates this to the PCN-ingress-node.AlsoAlso, the PCN-ingress-node measures the rate of PCN-traffic that is destined for this specific PCN-egress- node. The difference represents the excess amount that should be terminated. o Another approach instead measures the rate of excess-traffic- marked traffic and terminates this amount of traffic. This terminates less traffic than the previousbulletapproach, if some nodes are dropping PCN-traffic. Eardley Informational [Page 14] RFC 5559 PCN Architecture June 2009 o Another approach monitors PCN-packets and terminates some of the PCN-flows that have an excess-traffic-marked packet. (If all such flows were terminated, far too much traffic would be terminated, so a random selection needs to be made from those with an excess- traffic-markedpacket,packet [Menth08-1].) Since flow termination is designed for "abnormal" circumstances, itEardley (Editor) Expires October 9, 2009 [Page 15] Internet-Draft PCN Architecture April 2009is quite likely that some PCN-nodes are congestedand henceand, hence, that packets are being dropped and/or significantly queued. The flow termination mechanism must accommodate this. Note also that the termination control decision is made for a particular pair of PCN-boundary-nodes. So it is quite possible for PCN-flows to be terminated between one pair of PCN-boundary-nodes, whilst at the same time none are terminated between a different pair of PCN-boundary-nodes. 3.3. FlowadmissionAdmission and/orflow termination when there are only twoFlow Termination When There Are Only Two PCNencoding statesEncoding States If a PCN-domain has only two encoding states available (PCN-marked and not PCN-marked),ieie, it is using the baseline encoding[PCN08-1],[Moncaster09-1], then an operator has three options (others might be possible): o admission control only: PCN-marking means threshold-marking,ieie, only the threshold-metering algorithm triggers PCN-marking. Only PCN admission control is available. o flow termination only: PCN-marking means excess-traffic-marking,ieie, only the excess-traffic-metering algorithm triggers PCN- marking. Only PCN termination control is available. o both admission control and flow termination: only the excess- traffic-metering algorithm triggersPCN-marking, howeverPCN-marking; however, the configured rate (PCN-excess-rate) is set equal to the PCN- admissible-rate, as shown in Figure 3. [Charny07-2] describes how both admission control and flow termination can be triggered in this case and also gives someof thepros and cons of this approach. The main downside is that admission control is less accurate. Eardley(Editor) Expires October 9, 2009Informational [Page16] Internet-Draft15] RFC 5559 PCN ArchitectureAprilJune 2009 == Metering & == ==Marking behaviour== ==PCN mechanisms== ^ Rate of ^ PCN-traffic on | bottleneck link | Terminate some | admitted flows | & | Block new flows | | Some pkts U*PCN-excess-rate -| excess-traffic-marked ----------------- (=PCN-supportable-rate)| | Block new flows | | PCN-excess-rate -|------------------------------------------------ (=PCN-admissible-rate)| | No pkts Admit new flows | PCN-marked | Figure 3: Schematic of how the PCN admission control and flow termination mechanisms operate as the rate of PCN-traffic increases, for a PCN-domain with two encoding states and using the approach of [Charny07-2]. Note: U is a global parameter for all links in the PCN-domain. 3.4. InformationtransportTransport The transport of pre-congestion information from a PCN-node to a PCN- egress-node is through PCN-markings in data packet headers,ieie, "in-band":band"; no signalling protocol messaging is needed. Signalling is needed to transportPCN-feedback-information,PCN-feedback-information -- forexampleexample, to convey the fraction of PCN-marked traffic from a PCN-egress-node to the relevant PCN-ingress-node. Exactly what information needs to be transported will be described in future documents about possible boundary mechanisms. The signalling could be done by an extension of RSVP orNSIS,NSIS (Next Steps in Signalling), for instance; [Lefaucheur06] describes the extensions needed for RSVP. 3.5.PCN-trafficPCN-Traffic The following are some high-level points about how PCN works: o There needs to be a way for a PCN-node to distinguish PCN-traffic from other traffic. This is through a combination of the DSCP field and/or ECN field. Eardley(Editor) Expires October 9, 2009Informational [Page17] Internet-Draft16] RFC 5559 PCN ArchitectureAprilJune 2009 o It is not advised to havenon PCN-traffic that competes for the same capacity as PCN-trafficcompeting-non-PCN-traffic but, if there is such traffic, there needs to be a mechanism to limit it."Capacity" means the forwarding bandwidth on a link; "competes""Competing-non-PCN-traffic" means traffic thatnon PCN- packets will delay PCN-packets in the queueshares a link with PCN-traffic and competes forthe link. Henceits forwarding bandwidth. Hence, morenon PCN-trafficcompeting-non-PCN-traffic results in poorer QoS for PCN. Further, the unpredictable amount ofnon PCN-trafficcompeting-non-PCN-traffic makes the PCN mechanisms less accurate and so reduces PCN's ability to protect the QoS of admittedPCN-flowsPCN-flows. o Two examples of suchnon PCN-traffic (ie that competes for the same capacity as PCN-traffic)competing-non-PCN-traffic are: 1. traffic that is priority scheduled over PCN (perhaps a particular application or an operator's controlmessages).messages); 2. traffic that is scheduled at the same priority as PCN (forexampleexample, if the Voice-Admit codepoint is used for PCN-traffic[PCN08-1][Moncaster09-1] and there isnon-PCNnon-PCN, voice-admit traffic in thePCN- domain).PCN-domain). o If there is suchnon PCN-traffic (ie that competes for the same capacity as PCN-traffic),competing-non-PCN-traffic, then PCN's mechanisms should take account of it, in order to improve the accuracy of the decision about whether to admit (or terminate) a PCN-flow. For example, one mechanism is that suchnon PCN-trafficcompeting-non-PCN-traffic contributes to thePCN meters (iePCN-meters (ie, is metered by thethreshold-marking and excess-traffic-threshold- marking and excess-traffic-marking algorithms). o There will benon PCN-trafficother non-PCN-traffic that doesn't compete for the samecapacityforwarding bandwidth as PCN-traffic, because it is forwarded at lower priority.HenceHence, it shouldn't contribute to thePCNPCN- meters. Examples arebest effortbest-effort andassured forwardingassured-forwarding traffic. However, a PCN-node should dedicate some capacity tolowerlower- priority traffic so that it isn't starved. oTheThis document assumes that the PCN mechanisms are applied to a single behaviour aggregate in the PCN-domain. However, it would also be possible to apply them independently to more than one behaviour aggregate, which are distinguished by DSCP. 3.6. BackwardscompatibilityCompatibility PCN specifies semantics for the ECN field that differ from the default semantics of [RFC3168]. A particular PCN encoding scheme needs to describe how it meets the guidelines of BCP 124 [RFC4774] for specifying alternative semantics for the ECN field. Insummarysummary, the approach is to:Eardley (Editor) Expires October 9, 2009 [Page 18] Internet-Draft PCN Architecture April 2009o use a DSCP to allow PCN-nodes to distinguish PCN-traffic that uses the alternative ECN semantics; Eardley Informational [Page 17] RFC 5559 PCN Architecture June 2009 o define these semantics for use within a controlled region, the PCN-domain; o take appropriate action ifECN capable, non-PCN trafficECN-capable, non-PCN-traffic arrives at a PCN-ingress-node with the DSCP used by PCN. For the baseline encoding[PCN08-1],[Moncaster09-1], the'appropriate action'"appropriate action" is to block ECN-capable traffic that uses the same DSCP as PCN from entering the PCN-domain directly.Blocking"Blocking" means it is dropped or downgraded to alower prioritylower-priority behaviour aggregate, or alternatively such traffic may be tunnelled through the PCN-domain. The reason that'appropriate action'"appropriate action" is needed is that the PCN-egress-node clears the ECN field to 00. Extended encoding schemes may need to take different'appropriate action'."appropriate action". 4. Detailed FunctionalarchitectureArchitecture This section is intended to provide a systematic summary of the new functional architecture in the PCN-domain.FirstFirst, it describes functions needed at the three specific types of PCN-node; these are data plane functions and are in addition totheirthe normal routerfunctions. Thenfunctions for PCN-nodes. Then, it describes the further functionality needed for both flow admission control and flow termination; these are signalling and decision-making functions, and there are various possibilities for where the functions are physically located. The section is split into: 1. functions needed at PCN-interior-nodes 2. functions needed at PCN-ingress-nodes 3. functions needed at PCN-egress-nodes 4. other functions needed for flow admission control 5. other functions needed for flow termination control Note: Probing is covered in the Appendix. The section then discusses some other detailed topics:Eardley (Editor) Expires October 9, 2009 [Page 19] Internet-Draft PCN Architecture April 20091. addressing 2. tunnelling 3. fault handling Eardley Informational [Page 18] RFC 5559 PCN Architecture June 2009 4.1.PCN-interior-node functionsPCN-Interior-Node Functions Each link of the PCN-domain is configured with the following functionality: o Behaviour aggregate classification - determine whether or not an incoming packet is aPCN-packet or not.PCN-packet. o PCN-meter - measure the'amount"amount ofPCN-traffic'.PCN-traffic". The measurement is made on the overall PCN-traffic,andnot per flow. Algorithms determine whether to indicate to the PCN-marking functionality that packets should be PCN-marked. o PCN-mark - as triggered by indications from the PCN-meterfunctionality,functionality; ifnecessarynecessary, PCN-mark packetswthwith theappropiateappropriate encoding. o Drop - if the queueoverflowsoverflows, then naturally packets are dropped. In addition, the link may be configured with a maximum rate for PCN-traffic (below the physical link rate), above which PCN- packets are dropped. The functions are defined in[PCN08-2][Eardley09] and the baseline encoding in[PCN08-1][Moncaster09-1] (extended encodings are to be defined in other documents).Eardley (Editor) Expires October 9, 2009 [Page 20] Internet-Draft PCN Architecture April 2009+---------+ Result +->|Threshold|-------+ | | Meter | | | +---------+ V +----------+ +- - - - -+ | +------+ | BA | | | | | | Marked Packet =>|Classifier|==>| Dropper |==?===============>|Marker|==> Packet Stream | | | | | | | Stream +----------+ +- - - - -+ | +------+ | +---------+ ^ | | Excess | | +->| Traffic |-------+ | Meter | Result +---------+ Figure 4: Schematic of PCN-interior-nodefunctionalityfunctionality. 4.2.PCN-ingress-node functionsPCN-Ingress-Node Functions Each ingress link of the PCN-domain is configured with the following functionality: Eardley Informational [Page 19] RFC 5559 PCN Architecture June 2009 o Packet classification - determine whether an incoming packet is part of a previously admittedflow,flow by using a filter spec(eg(eg, DSCP, source and destination addresses, port numbers, and protocol). o Police - police, bydropping,dropping any packets received with a DSCP indicating PCN transport that do not belong to an admitted flow. (A prospective PCN-flow that is rejected could be blocked or admitted into alower prioritylower-priority behaviour aggregate.) Similarly, police packets that are part of a previously admitted flow, to check that the flow keeps to the agreed rate or flowspec(eg(eg, see [RFC1633] for a microflow and its NSIS equivalent). o PCN-colour - set the DSCP and ECN fields appropriately for the PCN-domain, forexampleexample, as in[PCN08-1].[Moncaster09-1]. o Meter - some approaches to flow termination require the PCN- ingress-node to measure the (aggregate) rate of PCN-traffic towards a particular PCN-egress-node. The first two are policing functions, needed to make sure that PCN- packets admitted into the PCN-domain belong to a flow that has been admitted and to ensure that the flow keeps to the flowspec agreed(eg(eg, doesn't exceed an agreed maximum rate and is inelastic traffic). Installing the filter spec will typically be done by the signalling protocol, as will re-installing the filter, forexampleexample, after a re-Eardley (Editor) Expires October 9, 2009 [Page 21] Internet-Draft PCN Architecture April 2009route that changes the PCN-ingress-node (see [Briscoe06] for an example using RSVP). PCN-colouring allows the rest of the PCN-domain to recognise PCN-packets. 4.3.PCN-egress-node functionsPCN-Egress-Node Functions Each egress link of the PCN-domain is configured with the following functionality: o Packet classify - determine which PCN-ingress-node a PCN-packet has come from. o Meter - "measure PCN-traffic" or "monitor PCN-marks". o PCN-colour - for PCN-packets, set the DSCP and ECN fields to the appropriate values for use outside the PCN-domain. The meteringfunctionalityfunctionality, ofcoursecourse, depends on whether it is targeted at admission control or flow termination. Alternatives involve the PCN-egress-node"measuring""measuring", as an aggregate(ie(ie, not perflow)flow), all PCN-packets from a particular PCN-ingress-node, or "monitoring" the PCN-traffic and reacting to one (or several) PCN- Eardley Informational [Page 20] RFC 5559 PCN Architecture June 2009 marked packets. For PCN-colouring,[PCN08-1][Moncaster09-1] specifies that thePCN- egress-node re-setsPCN-egress-node resets the ECN field to 00; other encodings may define different behaviour. 4.4. Admissioncontrol functionsControl Functions As well as the functions covered above, other specific admission control functions need to be performed (others might be possible): o Make decision about admission - based on the output of the PCN- egress-node's meter function. In the case where it "measures PCN- traffic", the measured traffic on the ingress-egress-aggregate is compared with some reference level. In the case where it "monitors PCN-marks",thenthe decision is based on whether or not one (or several) packetsis (are)are PCN-markedor not (eg(eg, the RSVP PATH message). In either case, the admission decision also takes account of policy andapplication layerapplication-layer requirements [RFC2753]. o Communicate decision about admission - signal the decision to the node making the admission control request (which may be outside thePCN-domain),PCN-domain) and to the policer (PCN-ingress-node function) for enforcement of the decision. There are various possibilities for how the functionality could be distributed (we assume the operatorwouldwill configure which is used):Eardley (Editor) Expires October 9, 2009 [Page 22] Internet-Draft PCN Architecture April 2009o The decision is made at the PCN-egress-node and the decision (admit or block) is signalled to the PCN-ingress-node. o The decision is recommended by the PCN-egress-node (admit orblock)block), but the decision is definitively made by the PCN-ingress- node. The rationale is that the PCN-egress-node naturally has the necessary information about the amount of PCN-marks on the ingress-egress-aggregate,butwhereas the PCN-ingress-node is the policy enforcement point[RFC2753], which[RFC2753] that polices incoming traffic to ensure it is part of an admitted PCN-flow. o The decision is made at the PCN-ingress-node, which requires that the PCN-egress-node signals PCN-feedback-information to the PCN- ingress-node. For example, it could signal the current fraction of PCN-traffic that is PCN-marked. o The decision is made at a centralised node (see Appendix). Note: Admission control functionality is not performed by normal PCN- interior-nodes. Eardley Informational [Page 21] RFC 5559 PCN Architecture June 2009 4.5. Flowtermination functionsTermination Functions As well as the functions covered above, other specific termination control functions need to be performed (others might be possible): o PCN-meter at PCN-egress-node - similarly to flow admission, there are two types of possibilities: to "measure PCN-traffic" on the ingress-egress-aggregate,andor to "monitor PCN-marks" and react to one (or several) PCN-marks. o (if required) PCN-meter at PCN-ingress-node - make "measurements of PCN-traffic" being sent towards a particular PCN-egress-node; again, this is done for the ingress-egress-aggregate and not per flow. o (if required) Communicate PCN-feedback-information to the node that makes the flow terminationdecision. Fordecision - for example, as in [Briscoe06], communicate the PCN-egress-node's measurements to the PCN-ingress-node. o Make decision about flow termination - use the information from the PCN-meter(s) to decide which PCN-flow or PCN-flows to terminate. The decision takes account of policy andapplicationapplication- layer requirements [RFC2753]. o Communicate decision about flow termination - signal the decision to the node that is able to terminate the flow (which may beEardley (Editor) Expires October 9, 2009 [Page 23] Internet-Draft PCN Architecture April 2009outside thePCN-domain),PCN-domain) and to the policer (PCN-ingress-node function) for enforcement of the decision. There are various possibilities for how the functionality could be distributed, similar to those discussed above inthe Admission control section.Section 4.4. Note: Flow termination functionality is not performed by normal PCN- interior-nodes. 4.6. Addressing PCN-nodes may need to know the address of other PCN-nodes.Note: in all casesNote that PCN-interior-nodes don't need to know the address ofanyother PCN-nodes (exceptas normaltheirnext hop neighbours,next-hop neighbours for routing purposes).TheAt a minimum, the PCN-egress-node needs to know the address of the PCN-ingress-node associated with aflow, at a minimumflow so that the PCN-ingress-node can be informedto enforceof the admission decision (and any flow termination decision) and enforce it through policing. There are various Eardley Informational [Page 22] RFC 5559 PCN Architecture June 2009 possibilities for how the PCN-egress-node can do this,ieie, associate the received packet to the correct ingress-egress-aggregate. It is not the intention of this document to mandate a particular mechanism. o The addressing information can be gathered fromsignalling. Forsignalling -- for example, through the regular processing of an RSVP PATH message, as thePCN- ingress-nodePCN-ingress-node is the previous RSVP hop (PHOP) ([Lefaucheur06]).OrAnother option is that the PCN-ingress-node could signal its address to thePCN-egress- node.PCN-egress-node. o Always tunnel PCN-traffic across the PCN-domain. Then the PCN- ingress-node's address is simply the source address of the outer packet header. The PCN-ingress-node needs to learn the address of the PCN-egress-node, either by manual configuration or by one of the automated tunnel endpoint discovery mechanisms (such as signalling or probing over the data route, interrogatingroutingrouting, or using a centralised broker). 4.7. Tunnelling Tunnels may originate and/or terminate within a PCN-domain(eg(eg, IP over IP, IP over MPLS). It is important that the PCN-marking of any packet can potentially influence PCN's flow admission control and termination--- it shouldn't matter whether the packet happens to be tunnelled at the PCN-node that PCN-marks the packet, or indeed whether it's decapsulated or encapsulated by a subsequent PCN-node. This suggests that the "uniform conceptual model" described inEardley (Editor) Expires October 9, 2009 [Page 24] Internet-Draft PCN Architecture April 2009[RFC2983] should be re-applied in the PCN context. In line with both this and the approach of [RFC4303] and[Briscoe08-2],[Briscoe09], the following rule is applied if encapsulation is done within the PCN-domain: oanyAny PCN-marking is copied into the outerheaderheader. Note: A tunnel will not provide this behaviour if it complies with [RFC3168] tunnelling in either mode, but it will if it complies with [RFC4301]IPSecIPsec tunnelling. Similarly, in line with the "uniform conceptual model" of [RFC2983], with the "full-functionality option" of [RFC3168], and with [RFC4301], the following rule is applied if decapsulation is done within thePCN- domain:PCN-domain: oifIf the outer header's marking state is moreseveresevere, then it is copied onto the inner header.Note:Note that the order of increasing severity is: notPCN-marked; threshold- marked;PCN-marked, threshold-marked, and excess-traffic-marked. Eardley Informational [Page 23] RFC 5559 PCN Architecture June 2009 An operator may wish to tunnel PCN-traffic from PCN-ingress-nodes to PCN-egress-nodes. The PCN-marks shouldn't be visible outside the PCN-domain, which can be achieved by the PCN-egress-node doing the PCN-colouring function (Section 4.3) after all the other (PCN and tunnelling) functions. The potential reasons for doing such tunnelling are: the PCN-egress-node then automatically knows the address of the relevant PCN-ingress-node for aflow;flow, and, even if ECMP (Equal Cost Multi-Path) is running, all PCN-packets on a particular ingress-egress-aggregate follow the samepath. (ECMP: Equal Cost Multi-Path,path (for more on ECMP, see Section6.4.)6.4). Butitsuch tunnelling also has drawbacks, forexampleexample, the additional overhead in terms of bandwidth andprocessing, andprocessing as well as the cost of setting up a mesh of tunnels betweenPCN-boundary-nodesPCN- boundary-nodes (there is an N^2 scaling issue). Potential issues arise for a "partially PCN-capable tunnel",ieie, where only one tunnel endpoint is in thePCN domain:PCN-domain: 1. The tunnel originates outside a PCN-domain and ends inside it. If the packet arrives at the tunnel ingress with the same encoding as used within the PCN-domain to indicate PCN-marking, then this could lead the PCN-egress-node to falsely measure pre- congestion. 2. The tunnel originates inside a PCN-domain and ends outside it. If the packet arrives at the tunnel ingress already PCN-marked, then it will still have the same encoding when it'sdecapsulateddecapsulated, which could potentially confuse nodes beyond the tunnel egress.Eardley (Editor) Expires October 9, 2009 [Page 25] Internet-Draft PCN Architecture April 2009In line with the solution for partially capable Diffserv tunnels in [RFC2983], the following rules are applied: o For case (1), the tunnel egress node clears any PCN-marking on the inner header. This rule is applied before the'copy"copy ondecapsulation'decapsulation" rule above. o For case (2), the tunnel ingress node clears any PCN-marking on the inner header. This rule is applied after the'copy"copy onencapsulation'encapsulation" rule above. Note that the above implies that one has to know, or determine, the characteristics of the other end of the tunnel as part of establishing it. Tunnelling constraints were a major factor in the choice of the baseline encoding. As explained in[PCN08-1],[Moncaster09-1], with current tunnellingendpointsendpoints, only the 11 codepoint of the ECN field survives decapsulation, and hence the baseline encoding only uses the 11 codepoint to indicate PCN-marking. Extended encoding schemes need to Eardley Informational [Page 24] RFC 5559 PCN Architecture June 2009 explain their interactions with (or assumptions about) tunnelling. A lengthy discussion of all the issues associated with layered encapsulation of congestion notification (for ECN as well as PCN) is in[Briscoe08-2].[Briscoe09]. 4.8. FaulthandlingHandling If a PCN-interior-node (or one of its links) fails, thenlower layerlower-layer protection mechanisms or the regular IP routing protocol will eventually re-route around it. If the new route can carry all the admitted traffic, flows will gracefully continue. If instead this causes early warning of pre-congestion on the new route, then admission control based onpre-congestion notificationPre-Congestion Notification will ensure that new flows will not be admitted until enough existing flows have departed. Re-routing may result in heavy (pre-)congestion,whenwhich will cause the flow termination mechanismwillto kick in. If a PCN-boundary-nodefailsfails, then we would like the regular QoS signalling protocol to be responsible for taking appropriate action. As anexample [Briscoe08-2]example, [Briscoe09] considers what happens if RSVP is the QoS signalling protocol. 5. Operations and Management ThisSectionsection considers operations and management issues, under the FCAPS headings:the Operations and Management ofFaults, Configuration, Accounting,PerformancePerformance, and Security. Provisioning isEardley (Editor) Expires October 9, 2009 [Page 26] Internet-Draft PCN Architecture April 2009discussed with performance. 5.1.ConfigurationFault Operations and ManagementThreshold-metering and -marking and excess-traffic-meteringFault Operations and-marking are standardised in [PCN08-2]. However, more diversity in PCN-boundary-node behaviours is expected, in order to interface with diverse industry architectures. It may be possible to have different PCN-boundary-node behaviours for different ingress-egress-aggregates within the same PCN-domain. PCN metering behaviourManagement isenabled on eitherabout preventing faults, telling theegress ormanagement system (or manual operator) that theingress interfaces of PCN-nodes. A consistent choice mustsystem has recovered (or not) from a failure, and about maintaining information to aid fault diagnosis. Admission blocking and, particularly, flow termination mechanisms should rarely bemade across the PCN-domainneeded in practice. It would be unfortunate if they didn't work after an option had been accidentally disabled. Therefore, it will be necessary toensureregularly test that thePCN mechanisms protect all links. PCN configuration control variables fall into the following categories: olive systemoptions (enabling or disabling behaviours) o parameters (setting levels, addresses etc) One possibilityworks as intended (devising a meaningful test isthat all configurable variables sit withinleft as anSNMP management framework [RFC3411], being structured within a defined management information base (MIB) on each node, and being remotely readable and settable via a suitably secure management protocol (SNMPv3). Some configuration options and parameters haveexercise for the operator). Section 4 describes how the PCN architecture has been designed tobe set onceensure admitted flows continue gracefully after recovering automatically from link or node failures. The need to'globally' control the whole PCN-domain. Where possible, these are identified below. This may affect operational complexityrecord and monitor re-routing events affecting signalling is unchanged by thechancesEardley Informational [Page 25] RFC 5559 PCN Architecture June 2009 addition ofinteroperability problems between equipment from different vendors. It may be possible for an operatorPCN toconfigure some PCN-interior- nodes so that they don't runa Diffserv domain. Similarly, re-routing events within thePCN mechanisms, if it knows that these links will never become (pre-)congested. 5.1.1. System options On PCN-interior-nodes therePCN-domain will bevery few system options: o Whether two PCN-markings (threshold-markedrecorded andexcess-traffic- marked) are enabled or only one. Typically all nodes throughout a PCN-domain willmonitored just as they would beconfiguredwithout PCN. PCN-marking does make it possible to record "near-misses". For instance, at thesame in this respect. However, exceptionsPCN-egress-node a "reporting threshold" could bemade. For example, if most PCN-nodes used both markings, but some legacy hardware was incapable of running two algorithms, an operator might be willingset toconfigure these Eardley (Editor) Expires October 9, 2009 [Page 27] Internet-Draft PCN Architecture April 2009 legacy nodes solelymonitor how often -- and forexcess-traffic-markinghow long -- the system comes close toenabletriggering flow blocking without actually doing so. Similarly, bursts of flow terminationas a back-stop. It wouldmarking could besensible to place such nodes whererecorded even if they are not sufficiently sustained to trigger flow termination. Such statistics could beprovisionedcorrelated witha greater leeway over expected traffic levels. o In the case where only one PCN-marking is enabled, all nodes must be configuredper-queue counts of marking volume (Section 5.2) togenerate PCN-marks from the same meter (ie either the threshold meterupgrade resources in danger of causing service degradation orthe excess traffic meter). PCN-boundary-nodes (ingress and egress) willto trigger manual tracing of intermittent incipient errors that would otherwise havemore system options: o Whichgone unnoticed. Finally, ofadmission and flow terminationcourse, many faults areenabled. If any PCN- interior-node iscaused by failings in the management process ("human error"): a wrongly configuredto generateaddress in amarking, all PCN- boundary-nodes must be able to interpret that marking (which includes understanding,node, a wrong address given in aPCN-domain that uses only one type of PCN-marking, whether they are generated by PCN-interior-nodes' threshold meters or the excess traffic meters). Therefore all PCN-boundary-nodes must besignalling protocol, a wrongly configuredthe sameparameter in a queueing algorithm, a node set into a different mode from other nodes, and so on. Generally, a clean design with few configurable options ensures thisrespect. o Where flow admissionclass of faults can be traced more easily andtermination decisions are made: at PCN- ingress-nodes or at PCN-egress-nodes (orprevented more often. Sound management practice at run-time also helps. For instance, acentralised node, see Appendix). Theoretically, this configuration choice couldmanagement system should benegotiated for each pair of PCN-boundary-nodes, but we cannot imagine why such complexity wouldused that constrains configuration changes within system rules (eg, preventing an option setting inconsistent with other nodes), configuration options should berequired, except perhapsrecorded infuture inter-domain scenarios. o How PCN-markings are translated into admission controlan offline database, andflow termination decisions (see Section 3.1regular automatic consistency checks between live systems andSection 3.2). PCN-egress-nodes will have further system options: o Howthemappingdatabase should beestablished between each packetperformed. PCN adds nothing specific to this class of problems. 5.2. Configuration Operations andits aggregate, eg by MPLS label, by IP packet filter spec;Management Threshold-metering andhow-marking and excess-traffic-metering and -marking are standardised in [Eardley09]. However, more diversity in PCN-boundary-node behaviours is expected, in order totake account of ECMP. o If an equipment vendor provides a choice, thereinterface with diverse industry architectures. It may beoptions to select which smoothing algorithmpossible tousehave different PCN-boundary-node behaviours formeasurements. 5.1.2. Parameters Like any Diffserv domain, every nodedifferent ingress-egress-aggregates withina PCN-domain will need to be configured withtheDSCP(s) used to identify PCN-packets. On each interior linksame PCN-domain. PCN-metering behaviour is enabled on either themain configuration parameters areegress or thePCN- threshold-rate and PCN-excess-rate.ingress interfaces of PCN-nodes. Alarger PCN-threshold-rate enables more PCN-traffic toconsistent choice must beadmitted on a link, hence improving capacity utilisation. A PCN-excess-rate set further abovemade across thePCN- threshold-rate allows greater increases in traffic (whether duePCN-domain to ensure that the PCN mechanisms protect all links. Eardley(Editor) Expires October 9, 2009Informational [Page28] Internet-Draft26] RFC 5559 PCN ArchitectureAprilJune 2009natural fluctuationsPCN configuration control variables fall into the following categories: o system options (enabling orsome unexpected event) before any flowsdisabling behaviours) o parameters (setting levels, addresses, etc.) One possibility is that all configurable variables sit within an SNMP (Simple Network Management Protocol) management framework [RFC3411], being structured within a defined management information base (MIB) on each node, and being remotely readable and settable via a suitably secure management protocol (such as SNMPv3). Some configuration options and parameters have to be set once to "globally" control the whole PCN-domain. Where possible, these areterminated, ie minimisesidentified below. This may affect operational complexity and the chances ofunnecessarily triggering the termination mechanism. For instance,interoperability problems between equipment from different vendors. It may be possible for an operatormay wanttodesign their networkconfigure some PCN-interior- nodes so that they don't run the PCN mechanisms, if itcan cope with a failure of any single PCN- node without terminating any flows. Settingknows that theserates on first deployment of PCNlinks willbe very similar to the traditional process for sizing an admission controlled network, depending on: the operator's requirements for minimising flow blocking (grade of service), the expected PCN traffic load on each linknever become (pre-)congested. 5.2.1. System Options On PCN-interior-nodes there will be very few system options: o Whether two PCN-markings (threshold-marked andits statistical characteristics (the traffic matrix), contingency for re-routing the PCN traffic matrix in the event of singleexcess-traffic- marked) are enabled ormultiple failures, and the expected load from other classes relative to link capacities [Menth07]. But once a domain is in operation,only one. Typically, all nodes throughout aPCN design goal is toPCN-domain will beable to determine growth in theseconfiguredrates much more simply, by monitoring PCN-marking rates from actual rather than expected traffic (see Section 5.2 on Performance & Provisioning). Operators may also wish to configure a rate greater than the PCN- excess-rate that istheabsolute maximum rate that a link allows for PCN-traffic. This may simplysame in this respect. However, exceptions could bethe physical link rate,made. For example, if most PCN-nodes used both markings but someoperators may wishlegacy hardware was incapable of running two algorithms, an operator might be willing to configurea logical limitthese legacy nodes solely for excess-traffic-marking toprevent starvation of other traffic classes during any brief period after PCN-traffic exceeds the PCN-excess-rate but beforeenable flow terminationbrings it back below this rate. Threshold-metering requiresas athreshold token bucket depthback-stop. It would be sensible to place such nodes where they could beconfigured, excess-traffic-metering needs a value for the MTU (maximum size ofprovisioned with aPCN-packet ongreater leeway over expected traffic levels. o In thelink) and both require setting a maximum size of their token buckets. It will be preferable for there tocase where only one PCN-marking is enabled, all nodes must berules to set defaults for these parameters, but then allow operatorsconfigured tochange them, for instance if average traffic characteristics change over time. The PCN-egress-node may allow configuration ofgenerate PCN-marks from thefollowing:same meter (ie, either the threshold meter or the excess-traffic meter). PCN-boundary-nodes (ingress and egress) will have more system options: ohow it smooths meteringWhich ofPCN-markings (eg EWMA parameters) Whichever node makesadmission and flow terminationdecisions will contain algorithms for converting PCN-marking levels into admission or flow termination decisions. These will also require configurable parameters, for instance: o an admission control algorithm thatare enabled. If any PCN- interior-node isbased on the fraction of marked packets will at least requireconfigured to generate amarking threshold setting above which it denies admissionmarking, all PCN- boundary-nodes must be able tonew flows;interpret that marking (which Eardley(Editor) Expires October 9, 2009Informational [Page29] Internet-Draft27] RFC 5559 PCN ArchitectureAprilJune 2009 includes understanding, in a PCN-domain that uses only one type of PCN-marking, whether they are generated by PCN-interior-nodes' threshold meters or their excess-traffic meters). Therefore, all PCN-boundary-nodes must be configured the same in this respect. o Where flow admission and terminationalgorithms will probably requiredecisions are made: at PCN- ingress-nodes or at PCN-egress-nodes (or at aparameter to delay terminationcentralised node, see Appendix). Theoretically, this configuration choice could be negotiated for each pair ofany flows until it is more certain that an anomalous event is not transient;PCN-boundary-nodes, but we cannot imagine why such complexity would be required, except perhaps in future inter-domain scenarios. oa parameter toHow PCN-markings are translated into admission control and flow termination decisions (see Sections 3.1 and 3.2). PCN-egress-nodes will have further system options: o How thetrade-offmapping should be established betweenhow quickly excess flows are terminated,each packet andover-termination. One particular approach, [Charny07-2] would require a global parameterits aggregate (eg, by MPLS label and by IP packet filter spec) and how to take account of ECMP. o If an equipment vendor provides a choice, there may bedefined on all PCN-nodes, but only needs one PCN marking rateoptions for selecting which smoothing algorithm to use for measurements. 5.2.2. Parameters Like any Diffserv domain, every node within a PCN-domain will need to be configuredonwith the DSCP(s) used to identify PCN-packets. On eachlink. The global parameter is a scaling factor between admissioninterior link, the main configuration parameters are the PCN- threshold-rate andtermination (thePCN-excess-rate. A larger PCN-threshold-rate enables more PCN-trafficrateto be admitted on alink uplink, hence improving capacity utilisation. A PCN-excess-rate set further above the PCN- threshold-rate allows greater increases in traffic (whether due towhichnatural fluctuations or some unexpected event) before any flows areadmitted vsterminated, ie, minimises therate above which flows are terminated). [Charny07-2] discusses in fullchances of unnecessarily triggering theimpacttermination mechanism. For instance, an operator may want to design their network so that it can cope with a failure ofthis particular approachany single PCN- node without terminating any flows. Setting these rates on theoperationfirst deployment ofPCN. 5.2. Performance & Provisioning Operations and Management Monitoring of performance factors measurable from *outside* thePCNdomainwill beno different with PCN than with any other packet-based flow admission control system, both atvery similar to the traditional process for sizing an admission-controlled network, depending on: the operator's requirements for minimising flowlevel (blocking probability, etc)blocking (grade of service), the expected PCN-traffic load on each link and its statistical characteristics (the traffic matrix), contingency for re-routing thepacket level (jitter [RFC3393], [Y.1541], loss rate [RFC4656], mean opinion score [P.800], etc). The difference is that PCN is intentionally designed to indicate *internally* which exact resource(s) arePCN-traffic matrix in thecauseevent ofperformance problemssingle or multiple failures, andby how much. Even better,the expected load from other classes relative to link capacities [Menth09-1]. But, once a domain is in operation, a PCNindicates which resources will probably cause problems if they are not upgraded soon. This candesign goal is to beachievedable to determine growth in Eardley Informational [Page 28] RFC 5559 PCN Architecture June 2009 these configured rates much more simply, bythe management systemmonitoring PCN-marking rates from actual rather than expected traffic (see Section 5.4 on Performance and Provisioning). Operators may also wish to configure a rate greater than thetotal amount (in bytes) ofPCN-marking generated by each queue over a period. Given possible long provisioning lead times, pre-congestion volumeexcess-rate that is thebest metricabsolute maximum rate that a link allows for PCN-traffic. This may simply be the physical link rate, but some operators may wish toreveal whether sufficient persistent demand has occurredconfigure a logical limit towarrant an upgrade. Because, even before utilisation becomes problematic, the statistical variabilityprevent starvation of other trafficwill cause occasional bursts of pre-congestion. This 'early warning system' decouples the process of adding customers from the provisioning process. This should cutclasses during any brief period after PCN-traffic exceeds thetime to add a customer when compared against admission control provided over native Diffserv [RFC2998], becausePCN-excess-rate but before flow termination brings itsaves havingback below this rate. Threshold-metering requires a threshold token bucket depth toverify the capacity planning process before adding each customer. Alternatively, before triggering an upgrade, the long term pre- congestion volume on each link canbeused to balance traffic load across the PCN-domain by adjustingconfigured, excess-traffic-metering requires a value for thelink weightsMTU (maximum size of a PCN-packet on therouting system. When an upgrade tolink), and both require setting alink's configured PCN-ratesmaximum size of their token buckets. It isrequired, it may also be necessarypreferable toupgrade the physical capacity availablehave rules that set defaults for these parameters but toother classes. But usually therethen allow operators to change them -- for instance, if average traffic characteristics change over time. The PCN-egress-node may allow configuration of: o how it smooths metering of PCN-markings (eg, EWMA parameters) Whichever node makes admission and flow termination decisions willbe sufficient physical capacitycontain algorithms for converting PCN-marking levels into admission or flow termination decisions. These will also require configurable parameters, for instance: o An admission control algorithm that is based on theupgrade to go ahead asfraction of marked packets will at least require asimple configuration change. Alternatively, [Songhurst06] describes an Eardley (Editor) Expires October 9, 2009 [Page 30] Internet-Draft PCN Architecture April 2009 adaptive rather than preconfigured system, where the configured PCN- threshold-rate is replaced with a high and low water mark and themarkingalgorithm automatically optimises how physical capacitythreshold setting above which it denies admission to new flows. o Flow termination algorithms will probably require a parameter to delay termination of any flows until it isshared usingmore certain that an anomalous event is not transient. o A parameter to control therelative loads from PCNtrade-off between how quickly excess flows are terminated andother traffic classes. All the above processesover-termination. One particular approach [Charny07-2] would requirejust three extra counters associated with each PCN queue: threshold-markings, excess-traffic-markings and drop. Every timeaPCN packet is marked or dropped its size in bytes shouldglobal parameter to beaddeddefined on all PCN-nodes, but would only need one PCN-marking rate tothe appropriate counter. Then the management system can read the counters at any timebe configured on each link. The global parameter is a scaling factor between admission andsubtracttermination (the rate of PCN- traffic on aprevious readinglink up toestablishwhich flows are admitted vs. theincremental volumerate above which flows are terminated). [Charny07-2] discusses in full the impact ofeach typethis particular approach on the operation of(pre-)congestion. Readings should be taken frequently, so that anomalous events (eg re-routes) can be distinguished from regular fluctuating demand if required.PCN. Eardley Informational [Page 29] RFC 5559 PCN Architecture June 2009 5.3. Accounting Operations and Management Accounting is only done at trust boundaries so it is out of scope of this document, which is confined to intra-domain issues. Use of PCN internal to a domain makes no difference to the flow signalling events crossing trust boundaries outside the PCN-domain, which are typically used for accounting. 5.4.Fault OperationsPerformance andManagement FaultProvisioning Operations and Managementis about preventing faults, tellingMonitoring of performance factors measurable from *outside* themanagement system (or manual operator) that the system has recovered (or not) from a failure, and about maintaining information to aid fault diagnosis. Admission blocking and particularly flow termination mechanisms should rarely be needed in practice. It would be unfortunate if they didn't work after an option had been accidentally disabled. Therefore itPCN- domain will benecessary to regularly test that the live system works as intended (devising a meaningful test is left as an exercise forno different with PCN than with any other packet- based, flow admission control system, both at theoperator). Section 4 describes howflow level (blocking probability, etc.) and the packet level (jitter [RFC3393], [Y.1541], loss rate [RFC4656], mean opinion score [P.800], etc.). The difference is that PCNarchitecture has beenis intentionally designed toensure admitted flows continue gracefully after recovering automatically from link or node failures. The need to record and monitor re-routing events affecting signalling is unchanged byindicate *internally* which exact resource(s) are theadditioncause of performance problems and by how much. Even better, PCNto a Diffserv domain. Similarly, re-routing events within the PCN-domainindicates which resources willbe recorded and monitored just asprobably cause problems if theywouldare not upgraded soon. This can bewithout PCN. PCN-marking does make it possible to record 'near-misses'. For instance, atachieved by thePCN-egress-nodemanagement system monitoring the total amount (in bytes) of PCN- marking generated by each queue over a'reporting threshold' could be set to monitor how often - and for howperiod. Given possible long-provisioning lead times, pre-congestion volume is thesystem comes closebest metric toEardley (Editor) Expires October 9, 2009 [Page 31] Internet-Draft PCN Architecture April 2009 triggering flow blocking without actually doing so. Similarly, bursts of flow termination marking could be recorded even if they are not sufficiently sustained to trigger flow termination. Such statistics could be correlated with per-queue counts of marking volume (Section 5.2) to upgrade resources in danger of causing service degradation, or to trigger manual tracing of intermittent incipient errors that would otherwise have gone unnoticed. Finally, of course, many faults are caused by failings in the management process ('human error'): a wrongly configured address in a node, a wrong address given in a signalling protocol, a wrongly configured parameter in a queueing algorithm, a node set into a different mode from other nodes, and so on. Generally, a clean design with few configurable options ensures this class of faults can be traced more easily and prevented more often. Sound management practice at run-time also helps. For instance: a management system should be used that constrains configuration changes within system rules (eg preventing an option setting inconsistent with other nodes); configuration options should also be recorded in an offline database; and regular automatic consistency checks between live systems and the database should be performed. PCN adds nothing specific to this class of problems. 5.5. Security Operations and Management Security Operations and Management is about using secure operational practices as well as being able to track security breaches or near- misses at run-time. PCN adds few specifics to the general good practice required in this field [RFC4778], other than those below. The correct functions of the system should be monitored (Section 5.2) in multiple independent ways and correlated to detect possible security breaches. Persistent (pre-)congestion marking should raise an alarm (both on the node doing the marking and on the PCN-egress- node metering it). Similarly, persistently poor external QoS metrics (such as jitter or mean opinion score) should raise an alarm. The following are examples of symptoms that may be the result of innocent faults, rather than attacks, but until diagnosed they should be logged and trigger a security alarm: o Anomalous patterns of non-conforming incoming signals and packets rejected at the PCN-ingress-nodes (eg packets already marked PCN- capable, or traffic persistently starving token bucket policers). o PCN-capable packets arriving at a PCN-egress-node with no associated state for mapping them to a valid ingress-egress- aggregate. Eardley (Editor) Expires October 9, 2009 [Page 32] Internet-Draft PCN Architecture April 2009 o A PCN-ingress-node receiving feedback signals about the pre- congestion level on a non-existent aggregate, or that are inconsistent with other signals (eg unexpected sequence numbers, inconsistent addressing, conflicting reports of the pre-congestion level, etc). o Pre-congestion marking arriving at a PCN-egress-node with (pre-)congestion markings focused on particular flows, rather than randomly distributed throughout the aggregate. 6. Applicability of PCN 6.1. Benefits The key benefits of the PCN mechanisms are that they are simple, scalable, and robust because: o Per flow state is only required at the PCN-ingress-nodes ("stateless core"). This is required for policing purposes (to prevent non-admitted PCN traffic from entering the PCN-domain) and so on. It is not generally required that other network entities are aware of individual flows (although they may be in particular deployment scenarios). o Admission control is resilient: with PCN QoS is decoupled from the routing system. Hence in general admitted flows can survive capacity, routing or topology changes without additional signalling. The PCN-admissible-rate on each link can be chosen small enough that admitted traffic can still be carried after a rerouting in most failure cases [Menth07]. This is an important feature as QoS violations in core networks due to link failures are more likely than QoS violations due to increased traffic volume [Iyer03]. o The PCN-metering behaviours only operate on the overall PCN- traffic on the link, not per flow. o The information of these measurements is signalled to the PCN- egress-nodes by the PCN-marks in the packet headers, ie [Style] "in-band". No additional signalling protocol is required for transporting the PCN-marks. Therefore no secure binding is required between data packets and separate congestion messages. o The PCN-egress-nodes make separate measurements, operating on the aggregate PCN-traffic from each PCN-ingress-node, ie not per flow. Similarly, signalling by the PCN-egress-node of PCN-feedback- information (which is used for flow admission and termination Eardley (Editor) Expires October 9, 2009 [Page 33] Internet-Draft PCN Architecture April 2009 decisions) is at the granularity of the ingress-egress-aggregate. An alternative approach is that the PCN-egress-nodes monitor the PCN-traffic and signal PCN-feedback-information (which is used for flow admission and termination decisions) at the granularity of one (or a few) PCN-marks. o The admitted PCN-load is controlled dynamically. Therefore it adapts as the traffic matrix changes, and also if the network topology changes (eg after a link failure). Hence an operator can be less conservative when deploying network capacity, and less accurate in their prediction of the PCN-traffic matrix. o The termination mechanism complements admission control. It allows the network to recover from sudden unexpected surges of PCN-traffic on some links, thus restoring QoS to the remaining flows. Such scenarios are expected to be rare but not impossible. They can be caused by large network failures that redirect lots of admitted PCN-traffic to other links, or by malfunction of the measurement-based admission control in the presence of admitted flows that send for a while with an atypically low rate and then increase their rates in a correlated way. o Flow termination can also enable an operator to be less conservative when deploying network capacity. It is an alternative to running links at low utilisation in order to protect against link or node failures. This is especially the case with SRLGs (shared risk link groups, which are links that share a resource, such as a fibre, whose failure affects all those links [RFC4216]). Fully protecting traffic against a single SRLG failure requires low utilisation (~10%) of the link bandwidth on some links before failure [Charny08]. o The PCN-supportable-rate may be set below the maximum rate that PCN-traffic can be transmitted on a link, in order to trigger termination of some PCN-flows before loss (or excessive delay) of PCN-packets occurs, or to keep the maximum PCN-load on a link below a level configured by the operator. o Provisioning of the network is decoupled from the process of adding new customers. By contrast, with the Diffserv architecture [RFC2475] operators rely on subscription-time Service Level Agreements, which statically define the parameters of the traffic that will be accepted from a customer, and so the operator has to verify provision is sufficient each time a new customer is added to check that the Service Level Agreement can be fulfilled. A PCN-domain doesn't need such traffic conditioning. Eardley (Editor) Expires October 9, 2009 [Page 34] Internet-Draft PCN Architecture April 2009 6.2. Deployment scenarios Operators of networks will want to use the PCN mechanisms in various arrangements, for instance depending on how they are performing admission control outside the PCN-domain (users after all are concerned about QoS end-to-end), what their particular goals and assumptions are, how many PCN encoding states are available, and so on. A PCN-domain may have three encoding states (or pedantically, an operator may choose to use up three encoding states for PCN): not PCN-marked, threshold-marked, excess-traffic-marked. Then both PCN admission control and flow termination can be supported. As illustrated in Figure 1, admission control accepts new flows until the PCN-traffic rate on the bottleneck link rises above the PCN- threshold-rate, whilst if necessary the flow termination mechanism terminates flows down to the PCN-excess-rate on the bottleneck link. On the other hand, a PCN-domain may have two encoding states (as in [PCN08-1]) (or pedantically, an operator may choose to use up two encoding states for PCN): not PCN-marked, PCN-marked. Then there are three possibilities, as discussed in the following paragraphs (see also Section 3.3). First, an operator could just use PCN's admission control, solving heavy congestion (caused by re-routing) by 'just waiting' - as sessions end, PCN-traffic naturally reduces, and meanwhile the admission control mechanism will prevent admission of new flows that use the affected links. So the PCN-domain will naturally return to normal operation, but with reduced capacity. The drawback of this approach would be that, until sufficient sessions have ended to relieve the congestion, all PCN-flows as well as lower priority services will be adversely affected. Second, an operator could just rely for admission control on statically provisioned capacity per PCN-ingress-node (regardless of the PCN-egress-node of a flow), as is typical in the hose model of the Diffserv architecture [RFC2475]. Such traffic conditioning agreements can lead to focused overload: many flows happen to focus on a particular link and then all flows through the congested link fail catastrophically. PCN's flow termination mechanism could then be used to counteract such a problem. Third, both admission control and flow termination can be triggered from the single type of PCN-marking; the main downside is that admission control is less accurate [Charny07-2]. This possibility is illustrated in Figure 3. Eardley (Editor) Expires October 9, 2009 [Page 35] Internet-Draft PCN Architecture April 2009 Within the PCN-domain there is some flexibility about how the decision making functionality is distributed. These possibilities are outlined in Section 4.4 and also discussed elsewhere, such as in [Menth08-3]. The flow admission and termination decisions need to be enforced through per flow policing by the PCN-ingress-nodes. If there are several PCN-domains on the end-to-end path, then each needs to police at its PCN-ingress-nodes. One exception is if the operator runs both the access network (not a PCN-domain) and the core network (a PCN- domain); per flow policing could be devolved to the access network and not done at the PCN-ingress-node. Note: to aid readability, the rest of this draft assumes that policing is done by the PCN-ingress- nodes. PCN admission control has to fit with the overall approach to admission control. For instance [Briscoe06] describes the case where RSVP signalling runs end-to-end. The PCN-domain is a single RSVP hop, ie only the PCN-boundary-nodes process RSVP messages, with RSVP messages processed on each hop outside the PCN-domain, as in IntServ over Diffserv [RFC2998]. It would also be possible for the RSVP signalling to be originated and/or terminated by proxies, with application-layer signalling between the end user andreveal whether sufficient persistent demand has occurred to warrant an upgrade because, even before utilisation becomes problematic, theproxy (eg SIP signalling with a home hub). A similar example would use NSIS signalling insteadstatistical variability ofRSVP. (NSIS: Next Steps in Signalling, [RFC3726].) It is possible that a user wants its inelastictrafficto use the PCN mechanisms but also react to ECN marking outside the PCN-domain [Sarker08]. Two possible ways to do this are to tunnel all PCN- packets acrosswill cause occasional bursts of pre-congestion. This "early warning system" decouples thePCN-domain, so thatprocess of adding customers from theECN marks are carried transparently acrossprovisioning process. This should cut thePCN-domain, ortime touse an encoding like [Moncaster08]. Tunnelling is discussed further in Section 4.7. Some further possible deployment models are outlined in the Appendix. 6.3. Assumptions and constraints on scope The scope is restricted by the following assumptions: 1. these components are deployed inadd asinglecustomer when compared against admission control that is provided over native Diffservdomain, within which all PCN-nodes are PCN-enabled and are trusted for truthful PCN-marking and transport 2. all flows handled by these mechanisms are inelastic and constrained[RFC2998] because it saves having toa known peak rate through policing or shaping Eardley (Editor) Expires October 9, 2009 [Page 36] Internet-Draft PCN Architecture April 2009 3.verify thenumber of PCN-flows across any potential bottleneck link is sufficiently large that stateless, statistical mechanisms can be effective. To put it another way,capacity-planning process before adding each customer. Alternatively, before triggering an upgrade, theaggregate bit rate of PCN- traffic across any potential bottlenecklong-term pre- congestion volume on each linkneeds tocan besufficiently large relativeused to balance traffic load across themaximum additional bit rate addedPCN-domain byone flow. This is the basic assumption of measurement- based admission control. 4. PCN-flows may have different precedence, but the applicability ofadjusting thePCN mechanisms for emergency use (911, GETS, WPS, MLPP, etc.) is out of scope. 6.3.1. Assumption 1: Trust and supportlink weights ofPCN - controlled environment It is assumed thatthePCN-domain is a controlled environment, ie all the nodes in a PCN-domain run PCN and are trusted. There are several reasons this assumption: o The PCN-domain hasrouting system. When an upgrade tobe encircled by a ring of PCN-boundary- nodes, otherwise traffic could enteraPCN-BA without being subjectlink's configured PCN-rates is required, it may also be necessary toadmission control, which would potentially degradeupgrade theQoS of existing PCN-flows. o Similarly, a PCN-boundary-node hasphysical capacity available totrust that all the PCN-nodes mark PCN-traffic consistently. A node not performing PCN-marking wouldn'tother classes. However, there will usually beable to alert when it suffered pre-congestion, which potentially would leadsufficient physical capacity for the upgrade totoo many PCN-flows being admitted (or too few being terminated). Worse, a rogue node could perform various attacks,go ahead asdiscussed ina simple configuration change. Alternatively, [Songhurst06] describes an adaptive rather than preconfigured system, where theSecurity Considerations section. One way of assuringconfigured PCN- threshold-rate is replaced with a high and low water mark and the marking algorithm automatically optimises how physical capacity is shared, using the relative loads from PCN and other traffic classes. Eardley Informational [Page 30] RFC 5559 PCN Architecture June 2009 All the abovetwo points is that the entire PCN- domain is run byprocesses require just three extra counters associated with each PCN queue: threshold-markings, excess-traffic-markings, and drops. Every time asingle operator. Another possibilityPCN-packet isthat there are several operators that trust each othermarked or dropped, its size intheir handling of PCN-traffic. Note: All PCN-nodes need tobytes should betrustworthy. However if it is known that an interface cannot become pre-congested then it is not strictly necessary for itadded tobe capable of PCN-marking. But this must be known even in unusual circumstances, eg afterthefailure of some links. 6.3.2. Assumption 2: Real-time applications It is assumed thatappropriate counter. Then the management system can read the counters at anyvariation of source bit rate is independent oftime and subtract a previous reading to establish thelevelincremental volume ofpre-congestion. We assumeeach type of (pre-)congestion. Readings should be taken frequently so thatPCN-packets comeanomalous events (eg, re-routes) can be distinguished fromreal time applications generating inelastic traffic, ie sending packetsregular fluctuating demand, if required. 5.5. Security Operations and Management Security Operations and Management is about using secure operational practices as well as being able to track security breaches or near- misses atthe rate the codec produces them, regardless of the Eardley (Editor) Expires October 9, 2009 [Page 37] Internet-Draftrun-time. PCNArchitecture April 2009 availabilityadds few specifics to the general good practice required in this field [RFC4778]. The correct functions ofcapacity [RFC4594]. For example, voice and video requiring low delay, jitter and packet loss,theControlled Load Service, [RFC2211],system should be monitored (Section 5.4) in multiple independent ways andthe Telephony service class, [RFC4594]. This assumption iscorrelated tohelp focusdetect possible security breaches. Persistent (pre-)congestion marking should raise an alarm (both on theeffort where it looks like PCN would be most useful, ienode doing thesorts of applications where per flow QoS is a known requirement. In other words we focusmarking and onPCN providing a benefit to inelastic traffic (PCN maythe PCN-egress-node metering it). Similarly, persistently poor external QoS metrics (such as jitter or mean opinion score) should raise an alarm. The following are examples of symptoms that maynot provide a benefit to other typesbe the result oftraffic). Asinnocent faults, rather than attacks; however, until diagnosed, they should be logged and should trigger aconsequence, it is assumed that PCN-meteringsecurity alarm: o Anomalous patterns of non-conforming incoming signals andPCN-marking is being applied to traffic scheduled withpackets rejected at theexpedited forwarding per- hop behaviour, [RFC3246],PCN-ingress-nodes (eg, packets already marked PCN- capable or traffic persistently starving token bucket policers). o PCN-capable packets arriving at aper-hop behaviourPCN-egress-node withsimilar characteristics. 6.3.3. Assumption 3: Many flows and additional load It is assumedno associated state for mapping them to a valid ingress-egress- aggregate. o A PCN-ingress-node receiving feedback signals thattherearemany PCN-flowsabout the pre-congestion level onany bottleneck link ina non-existent aggregate or that are inconsistent with other signals (eg, unexpected sequence numbers, inconsistent addressing, conflicting reports of thePCN-domain (or, to put it another way,pre-congestion level, etc.). o Pre-congestion marking arriving at a PCN-egress-node with (pre-)congestion markings focused on particular flows, rather than randomly distributed throughout theaggregate bit rateaggregate. Eardley Informational [Page 31] RFC 5559 PCN Architecture June 2009 6. Applicability of PCN 6.1. Benefits The key benefits ofPCN-traffic across any potential bottleneck link is sufficiently large relative tothemaximum additional bit rate added by one PCN- flow). Measurement-based admission control assumesPCN mechanisms are thatthe presentthey are simple, scalable, and robust, because: o Per-flow state isa reasonable prediction ofonly required at thefuture:PCN-ingress-nodes ("stateless core"). This is required for policing purposes (to prevent non-admitted PCN-traffic from entering the PCN-domain) and so on. It is not generally required that other networkconditionsentities aremeasured at the timeaware ofa new flow request, however the actual network performance mustindividual flows (although they may beacceptable during the call some time later. One issuein particular deployment scenarios). o Admission control isthat if there are only a few variable rate flows, thenresilient: with PCN, QoS is decoupled from theaggregaterouting system. Hence, in general, admitted flows can survive capacity, routing, or topology changes without additional signalling. The PCN-admissible-rate on each link can be chosen to be small enough that admitted trafficlevel may varycan still be carried after alot, perhaps enough to cause some packetsre-routing in most failure cases [Menth09-1]. This is an important feature, as QoS violations in core networks due toget dropped. If therelink failures aremany flows then the aggregatemore likely than QoS violations due to increased trafficlevel should be statistically smoothed. How many flows is enough dependsvolume [Iyer03]. o The PCN-metering behaviours only operate ona number of factors such asthevariation in each flow's rate,overall PCN- traffic on thetotal ratelink, not per flow. o The information ofPCN-traffic, andthese measurements is signalled to thesize ofPCN- egress-nodes by the"safety margin" betweenPCN-marks in thetraffic level at which we start admission-marking and at which packets are dropped or significantly delayed.packet headers, ie, "in- band". Noexplicit assumptions are made about how many PCN-flows are in each ingress-egress-aggregate. Performance evaluation work may clarify whether itadditional signalling protocol isnecessary torequired for transporting the PCN-marks. Therefore, no secure binding is required between data packets and separate congestion messages. o The PCN-egress-nodes makeany additional assumptionseparate measurements, operating onaggregation attheingress-egress-aggregate level. 6.3.4. Assumption 4: Emergency use out of scope PCN-flows may have different precedence, butaggregate PCN-traffic from each PCN-ingress-node, ie, not per flow. Similarly, signalling by theapplicabilityPCN-egress-node ofthe PCN mechanismsPCN- feedback-information (which is used foremergency use (911, GETS, WPS, MLPP, etc)flow admission and termination decisions) isout of scopeat the granularity ofthis document. Eardley (Editor) Expires October 9, 2009 [Page 38] Internet-Draft PCN Architecture April 2009 6.4. Challenges Prior work on PCNthe ingress- egress-aggregate. An alternative approach is that the PCN-egress- nodes monitor the PCN-traffic andsimilar mechanisms has thrown up a number of considerations about PCN's design goals (things PCN should be good at)signal PCN-feedback-information (which is used for flow admission andsome issues that have been hard to solve in a fully satisfactory manner. Taken astermination decisions) at the granularity of one (or awholefew) PCN-marks. o The admitted PCN-load is controlled dynamically. Therefore, itrepresentsadapts as the traffic matrix changes. It also adapts if the network topology changes (eg, after alist of trade- offs (it is unlikely that theylink failure). Hence, an operator canallbe100% achieved)less conservative when deploying network capacity andperhaps as evaluation criteria to help an operator (orless accurate in their prediction of theIETF) decide between options.PCN-traffic matrix. Eardley Informational [Page 32] RFC 5559 PCN Architecture June 2009 o Thefollowing are open issues. They are mainly takentermination mechanism complements admission control. It allows the network to recover from[Briscoe06], which also describes some possible solutions. Note thatsudden unexpected surges of PCN-traffic on somemay be considered unimportant in general or in specific deploymentlinks, thus restoring QoS to the remaining flows. Such scenarios are expected to be rare but not impossible. They can be caused by large network failures that redirect lots of admitted PCN-traffic to other links or bysome operators. NOTE: Potential solutions are outthe malfunction ofscope for this document. o ECMP (Equal Cost Multi-Path) Routing: The levelmeasurement-based admission control in the presence ofpre-congestion is measured onadmitted flows that send for aspecific ingress-egress-aggregate. However, if the PCN-domain runs ECMP,while with an atypically low rate and thentraffic on this ingress-egress- aggregate may follow several different paths - some of the paths could be pre-congested whilst others are not. There are three potential problems: 1. over-admission:increase their rates in anew flowcorrelated way. o Flow termination can also enable an operator to be less conservative when deploying network capacity. It is an alternative to running links at low utilisation in order to protect against link or node failures. This isadmitted (because the pre- congestion level measured byespecially thePCN-egress-node is sufficiently diluted by unmarked packets from non-congested pathscase with SRLGs (shared risk link groups), which are links that share anew flow is admitted), but its packets travel throughresource, such as apre-congested PCN-node. 2. under-admission:fibre, whose failure affects all links in that group [RFC4216]). Fully protecting traffic against anew flow is blocked (becausesingle SRLG failure requires low utilisation (~10%) of thepre- congestion level measured bylink bandwidth on some links before failure [Charny08]. o The PCN-supportable-rate may be set below thePCN-egress-node is sufficiently increased by PCN-marked packets from pre- congested pathsmaximum rate that PCN-traffic can be transmitted on anew flow is blocked), but its packets travel along an uncongested path. 3. ineffective termination: a flow is terminated, but its path doesn't travel throughlink in order to trigger the(pre-)congested router(s). Since flowterminationis a 'last resort', which protects the network should over-admission occur, this problem is probably more importantof some PCN-flows before loss (or excessive delay) of PCN-packets occurs, or tosolve thankeep theother two. o ECMP and signalling: It is possible that, inmaximum PCN-load on aPCN-domain running ECMP, the signalling packets (eg RSVP, NSIS) followlink below adifferent path thanlevel configured by thedata packets, which could matter ifoperator. o Provisioning of thesignalling packets are used as probes. Whether thisnetwork isan issue dependsdecoupled from the process of adding new customers. By contrast, with the Diffserv architecture [RFC2475], operators rely on subscription-time Service Level Agreements, whichfieldsstatically define the parameters of theECMP algorithm uses; iftraffic that will be accepted from a customer. This way, theECMP algorithmoperator has to verify that provision isrestrictedsufficient each time a new customer is added to check that thesource and destination IP addresses, then it Eardley (Editor) Expires October 9, 2009 [Page 39] Internet-Draft PCN Architecture April 2009 will notService Level Agreement can bean issue. ECMP and signalling interactions are a specific instancefulfilled. A PCN-domain doesn't need such traffic conditioning. 6.2. Deployment Scenarios Operators ofa general issue for non-traditional routing combined with resource management along a path [Hancock02]. o Tunnelling: There are scenarios where tunnelling makes it difficultnetworks will want todetermineuse thepathPCN mechanisms in various arrangements depending, for instance, on how they are performing admission control outside thePCN-domain. The problem, its impact,PCN-domain (users after all are concerned about QoS end-to-end), what their particular goals andthe potential solutionsassumptions are, how many PCN encoding states aresimilaravailable, and so on. A PCN-domain may have three encoding states (or pedantically, an operator may choose tothoseuse up three encoding states forECMP. o Scenarios with only one tunnel endpointPCN): not PCN-marked, threshold-marked, and excess-traffic-marked. This way, both PCN admission control and flow termination can be supported. As Eardley Informational [Page 33] RFC 5559 PCN Architecture June 2009 illustrated in Figure 1, admission control accepts new flows until thePCN domain may make it harder forPCN-traffic rate on thePCN-egress-node to gather frombottleneck link rises above thesignalling messages (eg RSVP, NSIS)PCN- threshold-rate, whilst, if necessary, theidentity offlow termination mechanism terminates flows down to the PCN-excess-rate on thePCN-ingress-node. o Bi-Directional Sessions: Many applicationsbottleneck link. On the other hand, a PCN-domain may havebi-directional sessions - hence there aretwomicroflows that should be admittedencoding states (as in [Moncaster09-1]) (orterminated) as a pair -pedantically, an operator may choose to use up two encoding states forinstance a bi-directional voice call only makes sense if microflows in both directionsPCN): not PCN-marked and PCN-marked. This way, there areadmitted. However,three possibilities, as discussed in thePCN mechanisms concernfollowing paragraphs (see also Section 3.3). First, an operator could just use PCN's admissionand termination of a single flow, and coordination ofcontrol, solving heavy congestion (caused by re-routing) by "just waiting" -- as sessions end, PCN-traffic naturally reduces; meanwhile, thedecision for bothadmission control mechanism will prevent admission of new flowsis a matter forthat use thesignalling protocol and out of scopeaffected links. So, the PCN-domain will naturally return to normal operation, but with reduced capacity. The drawback ofPCN. One possible example would use SIP pre-conditions. However, there are others. o Global Coordination: PCN makes its admission decision based on PCN-markings on a particular ingress-egress-aggregate. Decisions about flows through a different ingress-egress-aggregate are made independently. However, one can imagine network topologies and traffic matrices where, from a global perspective, itthis approach would bebetterthat, until sufficient sessions have ended tomake a coordinated decision acrossrelieve the congestion, all PCN-flows as well as lower-priority services will be adversely affected. Second, an operator could just rely on statically provisioned capacity per PCN-ingress-node (regardless of theingress- egress-aggregatesPCN-egress-node of a flow) for admission control, as is typical in thewhole PCN-domain. For example,hose model of the Diffserv architecture [Kumar01]. Such traffic-conditioning agreements can lead toblock (or even terminate)focused overload: many flows happen to focus onone ingress-egress-aggregate so that more importanta particular link and then all flows througha different ingress-egress-aggregatethe congested link fail catastrophically. PCN's flow termination mechanism could then beadmitted. The problem may wellused to counteract such a problem. Third, both admission control and flow termination can berelatively insignificant. o Aggregate Traffic Characteristics: Even whentriggered from thenumbersingle type offlows is stable, the traffic level throughPCN-marking; thePCN-domain will vary becausemain downside here is that admission control is less accurate [Charny07-2]. This possibility is illustrated in Figure 3. Within thesources vary their traffic rates. PCN works best whenPCN-domain, there isnot too much variability insome flexibility about how thetotal traffic level at a PCN-node's interface (iedecision-making functionality is distributed. These possibilities are outlined inthe aggregate traffic from all sources). Too much variation means that a node may (at one moment) not be doing any PCN-markingSection 4.4 andthen (at another moment) drop packets because it is overloaded. This makes it hardare also discussed elsewhere, such as in [Menth09-2]. The flow admission and termination decisions need totunebe enforced through per-flow policing by theadmission control schemePCN-ingress-nodes. If there are several PCN-domains on the end-to-end path, then each needs tostop admitting new flowspolice at its PCN-ingress-nodes. One exception is if theright time. Thereforeoperator runs both theproblem is more likely with fewer, burstier flows.access network (not a PCN-domain) and the core network (a PCN- domain); per-flow policing could be devolved to the access network Eardley(Editor) Expires October 9, 2009Informational [Page40] Internet-Draft34] RFC 5559 PCN ArchitectureAprilJune 2009o Flash crowdsandSpeednot be done at the PCN-ingress-node. Note that, to aid readability, the rest ofReaction: PCN is a measurement-based mechanism and so therethis document assumes that policing isan inherent delay between packet markingdone byPCN-interior-nodes and anythe PCN-ingress-nodes. PCN admission controlreaction at PCN- boundary-nodes.has to fit with the overall approach to admission control. Forexample, potentially ifinstance, [Briscoe06] describes the case where RSVP signalling runs end-to-end. The PCN-domain is abig burst of admission requests occurssingle RSVP hop, ie, only the PCN-boundary-nodes process RSVP messages, with RSVP messages processed on each hop outside the PCN-domain, as ina very short space of time (eg prompted by a televote), they could all get admitted before enough PCN-marks are seen to block new flows. In other words, any additional load offered withinIntServ over Diffserv [RFC2998]. It would also be possible for thereaction time ofRSVP signalling to be originated and/or terminated by proxies, with application-layer signalling between themechanism must not moveend user and thePCN-domain directly fromproxy (eg, SIP signalling with ano congestion statehome hub). A similar example would use NSIS (Next Steps in Signalling) [RFC3726] instead of RSVP. It is possible that a user wants its inelastic traffic tooverload. This 'vulnerability period' may have an impact atuse thesignalling level, for instance QoS requests should be rate limitedPCN mechanisms but also react toboundECN markings outside thenumber of requests ablePCN-domain [Sarker08]. Two possible ways toarrive withindo this are to tunnel all PCN- packets across thevulnerability period. o Silent at start: after a successful admission requestPCN-domain, so that thesource may wait some time before sending data (eg waiting forECN marks are carried transparently across thecalled partyPCN-domain, or toanswer). Then the riskuse an encoding like [Moncaster09-2]. Tunnelling isthat,discussed further in Section 4.7. Some further possible deployment models are outlined insome circumstances, PCN's measurements underestimate what the pre-congestion level will be whenthesource does start sending data. 7. IANA Considerations This memo includes no request to IANA. 8. Security considerations Security considerations essentially come fromAppendix. 6.3. Assumptions and Constraints on Scope The scope of this document is restricted by theTrust Assumption (Section 6.3.1), ie thatfollowing assumptions: 1. These components are deployed in a single Diffserv domain, within which all PCN-nodes are PCN-enabled and are trusted for truthfulPCN-metering and PCN-marking. PCN splits functionality between PCN-interior-nodes and PCN-boundary-nodes,PCN-marking andthe security considerations are somewhat different for each, mainly because PCN-boundary-nodestransport. 2. All flows handled by these mechanisms areflow-awareinelastic andPCN-interior-nodes are not. o Because the PCN-boundary-nodes are flow-aware, they are trustedconstrained touse that awareness correctly.a known peak rate through policing or shaping. 3. Thedegree of trust required depends on the kindsnumber ofdecisions they have to make andPCN-flows across any potential bottleneck link is sufficiently large that stateless, statistical mechanisms can be effective. To put it another way, thekindsaggregate bit rate ofinformation they need to make them. There is nothing specific to PCN. o The PCN-ingress-nodes police packetsPCN- traffic across any potential bottleneck link needs toensure a PCN-flow sticks within its agreed limit, andbe sufficiently large, relative toensure that only PCN-flows that have been admitted contribute PCN-traffic intothePCN-domain. The policer must drop (or perhaps downgrade to a different DSCP) any PCN-packets received that are outside this remit.maximum additional bit rate added by one flow. This issimilar to the existing IntServ behaviour. Between themthePCN-basic assumption of measurement- based admission control. Eardley(Editor) Expires October 9, 2009Informational [Page41] Internet-Draft35] RFC 5559 PCN ArchitectureAprilJune 2009boundary-nodes must encircle4. PCN-flows may have different precedence, but thePCN-domain, otherwise PCN-packets could enterapplicability of the PCN mechanisms for emergency use (911, GETS (Government Telecommunications Service), WPS (Wireless Priority Service), MLPP (Multilevel Precedence and Premption), etc.) is out of scope. 6.3.1. Assumption 1: Trust and Support of PCN - Controlled Environment It is assumed that the PCN-domain is a controlled environment, ie, all the nodes in a PCN-domain run PCN and are trusted. There are several reasons for this assumption: o The PCN-domain has to be encircled by a ring of PCN-boundary- nodes; otherwise, traffic could enter a PCN-BA without being subject to admission control, which would potentiallydestroydegrade the QoS of existingflows.PCN-flows. oPCN-interior-nodes areSimilarly, a PCN-boundary-node has to trust that all the PCN-nodes mark PCN-traffic consistently. A node notflow-aware. This prevents some security attacks whereperforming PCN-marking wouldn't be able to send anattacker targets specific flowsalert when it suffered pre-congestion, which potentially would lead to too many PCN-flows being admitted (or too few being terminated). Worse, a rogue node could perform various attacks, as discussed in Section 7. One way of assuring thedata plane - for instance for DoS or eavesdropping. o The PCN-boundary-nodes rely on correct PCN-marking byabove two points are in effect is to have thePCN- interior-nodes. For instanceentire PCN-domain run by arogue PCN-interior-node could PCN- mark all packets so that no flows were admitted.single operator. Anotherpossibilityway is to have several operators thatit doesn't PCN-mark any packets, even whentrust each other in their handling of PCN- traffic. Note: All PCN-nodes need to be trustworthy. However, if it ispre-congested. More subtly, the rogue PCN-interior-node could perform these attacks selectively on particular flows, orknown that an interface cannot become pre-congested, then itcould PCN-mark the correct fraction overall, but carefully choose which flowsis not strictly necessary for itmarked. o The PCN-boundary-nodes should be abletodeal with DoS attacks and state exhaustion attacks based on fast changes in per flow signalling. o The signalling between the PCN-boundary-nodesbe capable of PCN-marking, but this must beprotected from attacks. For exampleknown even in unusual circumstances, eg, after therecipient needs to validatefailure of some links. 6.3.2. Assumption 2: Real-Time Applications It is assumed thatthe messageany variation of source bit rate isindeed fromindependent of thenodelevel of pre-congestion. We assume thatclaims to have sent it. Possible measuresPCN-packets come from real-time applications generating inelastic traffic, ie, sending packets at the rate the codec produces them, regardless of the availability of capacity [RFC4594]. Examples of such real-time applications includedigest authenticationvoice andprotection against replayvideo requiring low delay, jitter, andman-in-the-middle attacks. Forpacket loss, thespecific protocol RSVP, hop-by-hop authentication is in [RFC2747], and [Behringer07] may also be useful. Operational security advice is given in Section 5.5. 9. Conclusions The document describes a general architecture for flow admission and termination based on pre-congestion information in order to protectControlled Load Service [RFC2211], and thequality ofTelephony serviceof established inelastic flows within a single Diffserv domain. The main topicclass [RFC4594]. This assumption is to help focus thefunctional architecture. It also mentions other topicseffort where it looks like PCN would be most useful, ie, theassumptions and open issues. 10. Acknowledgements This document is a revised versionsorts ofan earlier individual draft authored by: P. Eardley, J. Babiarz, K. Chan, A. Charny, R. Geib, G. Karagiannis, M. Menth, T. Tsou. They are therefore contributors to this document.Eardley(Editor) Expires October 9, 2009Informational [Page42] Internet-Draft36] RFC 5559 PCN ArchitectureAprilJune 2009Thanks to those who have made commentsapplications where per-flow QoS is a known requirement. In other words, we focus onthis document: Lachlan Andrew, Joe Babiarz, Fred Baker, David Black, Steven Blake, Ron Bonica, Scott Bradner, Bob Briscoe, Ross Callon, Jason Canon, Ken Carlberg, Anna Charny, Joachim Charzinski, Andras Csaszar, Francis Dupont, Lars Eggert, Pasi Eronen, Adrian Farrel, Ruediger Geib, Wei Gengyu, Robert Hancock, Fortune Huang, Christian Hublet, Cullen Jennings, Ingemar Johansson, Georgios Karagiannis, Hein Mekkes, Michael Menth, Toby Moncaster, Dimitri Papadimitriou, Dan Romascanu, Daisuke Satoh, Ben Strulo, Tom Taylor, Hannes Tschofenig, Tina Tsou, David Ward, Lars Westberg, Magnus Westerlund, Delei Yu. ThanksPCN providing a benefit toBob Briscoe who extensively revised the Operationsinelastic traffic (PCN may or may not provide a benefit to other types of traffic). As a consequence, it is assumed that PCN-metering andManagement section. This documentPCN-marking is being applied to traffic scheduled with an expedited forwarding per- hop behaviour [RFC3246] or with a per-hop behaviour with similar characteristics. 6.3.3. Assumption 3: Many Flows and Additional Load It isthe result of discussionsassumed that there are many PCN-flows on any bottleneck link in thePCN WG and forerunner activity inPCN-domain (or, to put it another way, theTSVWG. A numberaggregate bit rate ofprevious drafts were presentedPCN-traffic across any potential bottleneck link is sufficiently large, relative toTSVWG; their authors were: B, Briscoe, P. Eardley, D. Songhurst, F. Le Faucheur, A. Charny, J. Babiarz, K. Chan, S. Dudley, G. Karagiannis, A. Bader, L. Westberg, J. Zhang, V. Liatsos, X-G. Liu, A. Bhargava. 11. Comments Solicited (to be removedthe maximum additional bit rate added byRFC Editor) Comments and questionsone PCN- flow). Measurement-based admission control assumes that the present is a reasonable prediction of the future: the network conditions areencouraged and very welcome. They can be addressed tomeasured at theIETF PCN working group mailing list <pcn@ietf.org>. 12. Changes (totime of a new flow request, but the actual network performance must beremoved by RFC Editor) 12.1. Changes from -10 to -11 Changes to deal with IESG comments from routing area review: o Small clarifications to Introduction oacceptable during theterm "marking" now only used to refercall some time later. One issue is that if there are onlyto setting the codepoint (not asashorthand for 'metering and setting the codepoint') o Added Figure 4 (Schematic of PCN-interior-node functionality) (from [PCN08-2] o Appendix A brought back intofew variable rate flows, then themain body. o Other minor clarifications Eardley (Editor) Expires October 9, 2009 [Page 43] Internet-Draft PCN Architecture April 2009 12.2. Changes from -09 to -10 Changesaggregate traffic level may vary a lot, perhaps enough todeal with IESG comments: o New introductioncause some packets toprovide gentler introduction forget dropped. If there are many flows, then thePCN novice: quick summary of PCN's applicability; quick exampleaggregate traffic level should be statistically smoothed. How many flows is enough depends on a number ofhow it all hangs togetherfactors, such as the variation inone end-to-end qos scenario; quick summaryeach flow's rate, the total rate ofPCN "documentation" o OAM changed to OperationsPCN-traffic, andManagement o Processed somethe size of theminor suggestions in"safety margin" between theGen-ART Review by Francis Dupont o Two wording tweakstraffic level at which we start admission-marking and at which packets are dropped or significantly delayed. No explicit assumptions are made about how many PCN-flows are inSections 3.2 & 3.4 (as agreed on mailing list) o Updated boilerplate. this drafteach ingress-egress-aggregate. Performance-evaluation work mayinclude material pre- Nov 10 2008 blah. 12.3. Changes from -08 to -09 Small changes to deal with WG Chair comments: o tweak language in various places to makeclarify whether itmore RFC-like and less that of a scholarly work, for instance from "we propose" to "this document describes" o tweak language in various placesis necessary to makeit a stand alone architecture document rather than a discussionany additional assumptions on aggregation at the ingress-egress-aggregate level. 6.3.4. Assumption 4: Emergency Use Out of Scope PCN-flows may have different precedence, but thePCN WG. Now only mentions WG at startapplicability ofAnnex. o References: IDs are no longer referenced to bythedraft name o References: removed somePCN mechanisms for emergency use (911, GETS, WPS, MLPP, etc.) is out ofless important references to IDs 12.4. Changes from -07 to -08 Small changes from second WG last call: o Section 2: added definitionscope forPCN-admissible-ratethis document. 6.4. Challenges Prior work on PCN andPCN- supportable-rate. Small changes to use these terms as follows: Section 3, bullets 2 & 9; S6.1 para 1; S6.2 para1; S6.3 bullet 3; addedsimilar mechanisms has led toFigs 1 & 2. o added the phrase "(others mighta number of considerations about PCN's design goals (things PCN should bepossible") before thegood at) and some issues that have been hard to solve in a fully satisfactory manner. Taken as a whole, PCN represents a list ofapproaches in Section 6.3, 7.4 & 7.5.Eardley(Editor) Expires October 9, 2009Informational [Page44] Internet-Draft37] RFC 5559 PCN ArchitectureAprilJune 2009o added references to RFC2753 (A framework for policy-based admission control) in S7.4 & S7.5. o throughout, updated references nowtrade-offs (it is unlikely thatmarking behaviour & baseline encoding are WG drafts. othey can all be 100% achieved) and perhaps afew typos corrected 12.5. Changes from -06 to -07 References re-formatted to pass ID nits. No other changes. 12.6. Changes from -05list of evaluation criteria to-06 Minor clarifications throughout,help an operator (or theleast insignificantIETF) decide between options. The following areas follows: o Section 1: added to the list of encoding statesopen issues. They are mainly taken from [Briscoe06], which also describes some possible solutions. Note that some may be considered unimportant inan 'extended' scheme: "or perhaps further encoding states as suggestedgeneral or indraft-westberg-pcn-load-control" o Section 2: added definitionspecific deployment scenarios, or by some operators. Note: Potential solutions are out of scope forPCN-colouring (to clarify that the term is used consistently differently from 'PCN-marking')this document. oSection 6.1 and 6.2: added "(others might be possible)" beforeECMP (Equal Cost Multi-Path) Routing: The level of pre-congestion is measured on a specific ingress-egress-aggregate. However, if thelistPCN-domain runs ECMP, then traffic on this ingress-egress- aggregate may follow several different paths -- some ofhighthe paths could be pre-congested whilst others are not. There are three potential problems: 1. over-admission: a new flow is admitted (because the pre- congestion levelapproaches for makingmeasured by the PCN-egress-node is sufficiently diluted by unmarked packets from non-congested paths that a new flowadmission (termination) decisions. o Section 6.2: correctedis admitted), but its packets travel through asignificant typo in 2nd bullet (more -> less) o Section 6.3: correctedpre-congested PCN-node. 2. under-admission: a new flow is blocked (because the pre- congestion level measured by the PCN-egress-node is sufficiently increased by PCN-marked packets from pre- congested paths that acouple of significant typos in Figure 2 o Section 6.5 (PCN-traffic) re-written for clarity. Non PCN-traffic contributing to PCN metersnew flow isnow given asblocked), but its packets travel along anexample (there may be cases where don't need to meter it). o Section 7.7: added to the text about encapsulation being done within the PCN-domain: "Note: A tunnel will not provide this behaviour if it complies with [RFC3168] tunnelling in either mode,uncongested path. 3. ineffective termination: a flow is terminated butit will if it complies with [RFC4301] IPSec tunnelling." o Section 7.7: added mention of [RFC4301] to the text about decapsulation being done within the PCN-domain. o Section 8: deletedits path doesn't travel through thetext about design goals, since this(pre-)congested router(s). Since flow termination isalready covered adequately earlier eg in S3. Eardley (Editor) Expires October 9, 2009 [Page 45] Internet-Draft PCN Architecture April 2009 o Section 11: replaceda "last resort", which protects thelast sentence of bullet 1 by "Therenetwork should over-admission occur, this problem isnothing specificprobably more important toPCN."solve than the other two. oAppendix: added to open issues: possibility of automaticallyECMP andperiodically probing. o References: Split out Normative references (RFC2474 & RFC3246). 12.7. Changes from -04 to -05 Minor nits removed as follows: o Further minor changes to reflect that baseline encodingSignalling: It isconsensus, standards track document, whilst there can be (experimental track) encoding extensions o Traffic conditioning updated to reflect discussions in Dublin, mainly that PCN-interior-nodes don't police PCN-traffic (so deleted bulletpossible that, inS7.1) and that it is not advised to have non PCN-traffic that sharesa PCN-domain running ECMP, thesame capacity (onsignalling packets (eg, RSVP, NSIS) follow alink) as PCN- traffic (so added bullet in S6.5) o Probing moved into Appendix A and deleteddifferent path than the'third viewpoint' (admission control baseddata packets, which could matter if the signalling packets are used as probes. Whether this is an issue depends on which fields themarking of a single packet like an RSVP PATH message) - since this isn't really probing, and in any caseECMP algorithm uses; if the ECMP algorithm isalready mentioned in S6.1. o Minor changesrestricted toS9 Operationsthe source and destination IP addresses, then it will not be an issue. ECMP and signalling interactions are a specific instance of a general issue for non-traditional routing combined with resource management- mainly to reflect that consensus on marking behaviour has simplified things so eg therealong a path [Hancock02]. Eardley Informational [Page 38] RFC 5559 PCN Architecture June 2009 o Tunnelling: There arefewer parametersscenarios where tunnelling makes it difficult toconfigure. o A few terminology-related errors expunged,determine the path in the PCN-domain. The problem, its impact, andtwo pictures addedthe potential solutions are similar tohelp.those for ECMP. oRe-phrasedScenarios with only one tunnel endpoint in theclaim aboutPCN-domain: Such scenarios may make it harder for thenatural decision point in S7.4 o Clarified that extended encoding schemes needPCN-egress-node toexplain their interactions with (or assumptions about) tunnelling (S7.7) and how they meetgather from theguidelinessignalling messages (eg, RSVP, NSIS) the identity ofBCP124 (S6.6) o Correctedthethird bullet in S6.2 (to reflect consensus about PCN-marking) 12.8. Changes from -03 to -04PCN-ingress-node. oMinor changes throughout to reflect the consensusBi-Directional Sessions: Many applications have bi-directional sessions -- hence, there are two microflows that should be admitted (or terminated) as a pair -- for instance, a bi- directional voice callabout PCN- marking (as reflectedonly makes sense if microflows in[PCN08-2]). Eardley (Editor) Expires October 9, 2009 [Page 46] Internet-Draftboth directions are admitted. However, the PCNArchitecture April 2009 o Minor changes throughout to reflectmechanisms concern admission and termination of a single flow, and coordination of the decision for both flows is a matter for thecurrent decisions about encoding (as reflected in [PCN08-1] and [Moncaster08]). o Introduction: re-structured to create new sections on Benefits, Deployment scenariossignalling protocol andAssumptions.out of scope for PCN. One possible example would use SIP pre-conditions. However, there are others. oIntroduction: Added pointers to otherGlobal Coordination: PCNdocuments. o Terminology: changed PCN-lower-rate to PCN-threshold-ratemakes its admission decision based on PCN-markings on a particular ingress-egress-aggregate. Decisions about flows through a different ingress-egress-aggregate are made independently. However, one can imagine network topologies andPCN- upper-ratetraffic matrices where, from a global perspective, it would be better toPCN-excess-rate; excess-rate-markingmake a coordinated decision across all the ingress- egress-aggregates for the whole PCN-domain. For example, toexcess- traffic-marking. o Benefits: added bullet about SRLGs.block (or even terminate) flows on one ingress-egress-aggregate so that more important flows through a different ingress-egress-aggregate could be admitted. The problem may well be relatively insignificant. oDeployment scenarios: new section combining material from various places withinAggregate Traffic Characteristics: Even when thedocument. o S6 (highnumber of flows is stable, the traffic levelfunctional architecture): re-structured and edited to improve clarity, and reflectthrough thelatest PCN-marking and encoding drafts. o S6.4: added claim thatPCN-domain will vary because themost natural place to make an admission decisionsources vary their traffic rates. PCN works best when there is not too much variability in the total traffic level at aPCN-egress-node. o S6.5: updatedPCN-node's interface (ie, in thebullet about non-PCN-trafficaggregate traffic from all sources). Too much variation means thatuses the same DSCP as PCN-traffic. o S6.6: addedasection about backwards compatibility with respect to [RFC4774]. o Appendix A: added bullet about end-to-end PCN. o Probing: moved to Appendix B. o Other minor clarifications, typos etc. 12.9. Changes from -02node may (at one moment) not be doing any PCN-marking and then (at another moment) drop packets because it is overloaded. This makes it hard to-03 o Abstract: Clarified by removingtune theterm 'aggregated'. Follow-up clarifications later in draft: S1: expanded PCN-egress-nodes bulletadmission control scheme tomention case wherestop admitting new flows at thePCN-feedback-informationright time. Therefore, the problem isabout one (or a few) PCN-marks, rather than aggregated information; S3 clarified PCN-meter; S5 minor changes; conclusion.more likely with fewer, burstier flows. oS1: addedFlash crowds and Speed of Reaction: PCN is aparagraph about how the PCN-domain looks to the outside world (essentially it looks likemeasurement-based mechanism and so there is an inherent delay between packet marking by PCN-interior-nodes and any admission control reaction at PCN- boundary-nodes. For example, if aDiffserv domain).big burst of admission requests Eardley(Editor) Expires October 9, 2009Informational [Page47] Internet-Draft39] RFC 5559 PCN ArchitectureAprilJune 2009o S2: tweaked the PCN-traffic terminology bullet: changed PCN traffic classes to PCN behaviour aggregates, to be more in line with traditional Diffserv jargon (-> follow-up changes laterpotentially occurs indraft); includedadefinitionvery short space ofPCN-flows (and correctedtime (eg, prompted by acoupletelevote), they could all get admitted before enough PCN-marks are seen to block new flows. In other words, any additional load offered within the reaction time of'PCN microflows'the mechanism must not move the PCN-domain directly from a no congestion state to'PCN-flows' later in draft) o S3.5: added possibilityoverload. This "vulnerability period" may have an impact at the signalling level, for instance, QoS requests should be rate-limited to bound the number ofdowngradingrequests able tobest effort, where PCN- packetsarriveat PCN-ingress-node already ECN marked (CE or ECN nonce)within the vulnerability period. oS4: added note about whether talk about PCN operating on an interface or onSilent at Start: After alink. In S8.1 (OAM) mentioned that PCN functionality needssuccessful admission request, the source may wait some time before sending data (eg, waiting for the called party to answer). Then the risk is that, in some circumstances, PCN's measurements underestimate what the pre-congestion level will beconfigured consistently on eitherwhen theingress orsource does start sending data. 7. Security Considerations Security considerations essentially come from theegress interface ofTrust Assumption Section 6.3.1, ie, that all PCN-nodesin a PCN-domain.are PCN-enabled and are trusted for truthful PCN-metering and PCN-marking. PCN splits functionality between PCN-interior-nodes and PCN-boundary-nodes, and the security considerations are somewhat different for each, mainly because PCN- boundary-nodes are flow-aware and PCN-interior-nodes are not. oS5.2: clarifiedBecause PCN-boundary-nodes are flow-aware, they are trusted to use thatsignalling protocol installs flow filter spec at PCN-ingress-node (& updates after possible re-route) o S5.6: addressing: clarified o S5.7: added tunnelling issueawareness correctly. The degree ofN^2 scaling if you set up a meshtrust required depends on the kinds oftunnels between PCN-boundary-nodes o S7.3: Clarifieddecisions they have to make and the"third viewpoint"kinds ofprobing (always probe).information they need to make them. There is nothing specific to PCN. oS8.1: clarifiedThe PCN-ingress-nodes police packets to ensure a PCN-flow sticks within its agreed limit, and to ensure thatSNMP isonlyan example; added notePCN-flows thatan operator may be ablehave been admitted contribute PCN-traffic into the PCN-domain. The policer must drop (or perhaps downgrade tonot run PCN on some PCN-interior-nodes, if it knows that these links will never become (pre-)congested; added notea different DSCP) any PCN-packets received thatit may be possibleare outside this remit. This is similar tohave different PCN-boundary-node behaviours for different ingress-egress-aggregates withinthesame PCN-domain. o Appendix: Created an Appendix about "Possible work items beyondexisting IntServ behaviour. Between them, thescope ofPCN- boundary-nodes must encircle the PCN-domain; otherwise, PCN- packets could enter the PCN-domain without being subject to admission control, which would potentially destroy thecurrent PCN WG Charter". Material moved from near startQoS ofS3 and elsewhere throughout draft. Moved text about centralised decision node to Appendix. o Other minor clarifications. 12.10. Changes from -01 to -02existing flows. oS1: Benefits: provisioning bullet extended to stress that PCN doesPCN-interior-nodes are notuse RFC2475-style traffic conditioning. o S1: Deployment models: mentioned, as variant of PCN-domain extending to end nodes, that may extend to LAN edge switch.flow-aware. This prevents some security attacks where an attacker targets specific flows in the data plane -- for instance, for DoS or eavesdropping. Eardley(Editor) Expires October 9, 2009Informational [Page48] Internet-Draft40] RFC 5559 PCN ArchitectureAprilJune 2009 oS3.1: Trust Assumption: added note about not needingThe PCN-boundary-nodes rely on correct PCN-markingcapability if knownby the PCN- interior-nodes. For instance, a rogue PCN-interior-node could PCN-mark all packets so thatan interface cannot becomeno flows were admitted. Another possibility is that it doesn't PCN-mark any packets, even when it is pre-congested.o S4: now divided into sub-sections o S4.1: Admission control: added second proposed method for how to decide to block newMore subtly, the rogue PCN-interior-node could perform these attacks selectively on particular flows, or it could PCN-mark the correct fraction overall but carefully choose which flows(PCN-egress-node receives one (or several) PCN-marked packets).it marked. oS5: Probing sub-section removed. Material nowThe PCN-boundary-nodes should be able to deal with DoS attacks and state exhaustion attacks based on fast changes innew S7. o S5.6: Addressing: clarified how PCN-ingress-node can discover address of PCN-egress-nodeper-flow signalling. oS5.6: Addressing: centralised node case, addedThe signalling between the PCN-boundary-nodes must be protected from attacks. For example, the recipient needs to validate thatPCN-ingress-the message is indeed from the node that claims to have sent it. Possible measures include digest authentication and protection against replay and man-in-the-middle attacks. For the RSVP protocol specifically, hop-by-hop authentication is in [RFC2747], and [Behringer09] mayneed to know address of PCN-egress-node o S5.8: Tunnelling: added case of "partially PCN-capable tunnel"also be useful. Operational security advice is given in Section 5.5. 8. Conclusions This document describes a general architecture for flow admission anddegraded bullettermination based onthispre-congestion information, inS6 (Open Issues) o S7: Probing: new section. Much more comprehensive than old S5.5. o S8: Operations and Management: substantially revised. o other minor changes not affecting semantics 12.11. Changes from -00 to -01 In additionorder toclarifications and nit squashing,protect themain changes are: o S1: Benefits: added one about provisioning (and contrast withquality of service of established, inelastic flows within a single DiffservSLAs) o S1: Benefits: clarified that the objectivedomain. The main topic is the functional architecture. This document alsoto stop PCN- packets being significantly delayed (previously only mentioned not dropping packets) o S1: Deployment models: added one where policingmentions other topics like the assumptions and open issues associated with the PCN architecture. 9. Acknowledgements This document isdone at ingressa revised version ofaccess networkan earlier individual working draft authored by: P. Eardley, J. Babiarz, K. Chan, A. Charny, R. Geib, G. Karagiannis, M. Menth, andnot at ingress of PCN-domain (assume trust between networks) o S1: Deployment models: corrected MPLS-TET. Tsou. They are therefore contributors toMPLS o S2: Terminology: adjusted definition of PCN-domain o S3.5: Other assumptions: corrected, so that two assumptions (PCN- nodes not performing ECN and PCN-ingress-node discarding arrivingthis document. Thanks to those who have made comments on this document: Lachlan Andrew, Joe Babiarz, Fred Baker, David Black, Steven Blake, Ron Bonica, Scott Bradner, Bob Briscoe, Ross Callon, Jason Canon, Ken Carlberg, Anna Charny, Joachim Charzinski, Andras Csaszar, Francis Dupont, Lars Eggert, Pasi Eronen, Adrian Farrel, Ruediger Geib, Wei Gengyu, Robert Hancock, Fortune Huang, Christian Hublet, Cullen Jennings, Ingemar Johansson, Georgios Karagiannis, Hein Mekkes, Michael Menth, Toby Moncaster, Dimitri Papadimitriou, Dan Romascanu, Daisuke Satoh, Ben Strulo, Tom Taylor, Hannes Tschofenig, Tina Tsou, Eardley(Editor) Expires October 9, 2009Informational [Page49] Internet-Draft41] RFC 5559 PCN ArchitectureAprilJune 2009CE packet) only apply if the PCN WG decidesDavid Ward, Lars Westberg, Magnus Westerlund, and Delei Yu. Thanks toencode PCN-marking inBob Briscoe who extensively revised theECN-field. o S4 & S5: changed PCN-marking algorithm to marking behaviour o S4: clarified that PCN-interior-node functionality applies for each outgoing interface,Operations andadded clarification: "The functionality is also done by PCN-ingress-nodes for their outgoing interfaces (ie those 'inside' the PCN-domain)." o S4 (near end): altered to say that a PCN-node "should" dedicate some capacity to lower priority traffic so that it isn't starved (was "may") o S5: clarified to say that PCN functionalityManagement section. This document isdone on an 'interface' (rather than on a 'link') o S5.2: deleted erroneous mention of service level agreement o S5.5: Probing: re-written, especially to distinguish probing to testtheingress-egress-aggregate from probing to test a particular ECMP path. o S5.7: Addressing: added mentionresult ofprobing; added thatdiscussions in thecase where traffic is always tunnelled across the PCN-domain, add a note that he PCN-ingress-node needs to know the address of the PCN-egress-node. o S5.8: Tunnelling: re-written, especially to provide a clearer description of copying on tunnel entry/exit, by adding explanation (keeping tunnel encaps/decapsPCN WG andPCN-marking orthogonal), deleting one bullet ("ifforerunner activity in theinner header's marking state is more sever then it is preserved" - shouldn't happen), and better referencing of other IETF documents. o S6: Open issues: stressed that "NOTE: Potential solutions are outTSVWG. A number ofscope for this document"previous drafts were presented to TSVWG; their authors were: B. Briscoe, P. Eardley, D. Songhurst, F. Le Faucheur, A. Charny, J. Babiarz, K. Chan, S. Dudley, G. Karagiannis, A. Bader, L. Westberg, J. Zhang, V. Liatsos, X-G. Liu, andedited a couple of sentences that were close to solution space. o S6: Open issues: added one about scenarios with only one tunnel endpointA. Bhargava. The admission control mechanism evolved from the work led by Martin Karsten on the Guaranteed Stream Provider developed in thePCN domain . o S6: Open issues: ECMP: added under-admission as another potential risk o S6: Open issues: added one about "Silent at start" Eardley (Editor) Expires October 9, 2009 [Page 50] Internet-Draft PCN Architecture April 2009 o S10: Conclusions: a small conclusions section added 13.M3I project [Karsten02] [M3I], which in turn was based on the theoretical work of Gibbens and Kelly [Gibbens99]. 10. References13.1.10.1. Normative References [RFC2474] Nichols, K., Blake, S., Baker, F., and D. Black, "Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 Headers", RFC 2474, December 1998. [RFC3246] Davie, B., Charny, A., Bennet, J., Benson, K., Le Boudec, J., Courtney, W., Davari, S., Firoiu, V., and D. Stiliadis, "An Expedited Forwarding PHB(Per-Hop(Per- Hop Behavior)", RFC 3246, March 2002.13.2.10.2. Informative References [RFC1633] Braden, B., Clark, D., and S. Shenker, "Integrated Services in the Internet Architecture: an Overview", RFC 1633, June 1994. [RFC2205] Braden, B., Zhang, L., Berson, S., Herzog, S., and S. Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 Functional Specification", RFC 2205, September 1997. [RFC2211] Wroclawski, J., "Specification of theControlled-LoadControlled- Load Network Element Service", RFC 2211, September 1997. [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., and W. Weiss, "An Architecture for Differentiated Services", RFC 2475, December 1998. Eardley Informational [Page 42] RFC 5559 PCN Architecture June 2009 [RFC2747] Baker, F., Lindell, B., and M. Talwar, "RSVP Cryptographic Authentication", RFC 2747, January 2000. [RFC2753] Yavatkar, R., Pendarakis, D., and R. Guerin, "A Framework for Policy-based Admission Control", RFC 2753, January 2000. [RFC2983] Black, D., "Differentiated Services and Tunnels", RFC 2983, October 2000. [RFC2998] Bernet, Y., Ford, P., Yavatkar, R., Baker, F., Zhang, L., Speer, M., Braden, R., Davie, B., Wroclawski, J., and E. Felstaine, "A Framework for Integrated Services Operation over Diffserv Networks", RFC 2998, November 2000.Eardley (Editor) Expires October 9, 2009 [Page 51] Internet-Draft PCN Architecture April 2009[RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition of Explicit Congestion Notification (ECN) to IP", RFC 3168, September 2001. [RFC3270] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, P., Krishnan, R., Cheval, P., and J. Heinanen,"Multi- Protocol"Multi-Protocol Label Switching (MPLS) Support of Differentiated Services", RFC 3270, May 2002. [RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation Metric for IP Performance Metrics (IPPM)", RFC 3393, November 2002. [RFC3411] Harrington, D., Presuhn, R., and B. Wijnen, "An Architecture for Describing Simple Network Management Protocol (SNMP) Management Frameworks", STD 62, RFC 3411, December 2002. [RFC3726] Brunner, M., "Requirements for Signaling Protocols", RFC 3726, April 2004. [RFC4216] Zhang, R. and J. Vasseur, "MPLS Inter-Autonomous System (AS) Traffic Engineering (TE) Requirements", RFC 4216, November 2005. [RFC4301] Kent, S. and K. Seo, "Security Architecture for the Internet Protocol", RFC 4301, December 2005. [RFC4303] Kent, S., "IP Encapsulating Security Payload (ESP)", RFC 4303, December 2005. Eardley Informational [Page 43] RFC 5559 PCN Architecture June 2009 [RFC4594] Babiarz, J., Chan, K., and F. Baker, "Configuration Guidelines for DiffServ Service Classes", RFC 4594, August 2006. [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. Zekauskas, "A One-way Active Measurement Protocol (OWAMP)", RFC 4656, September 2006. [RFC4774] Floyd, S., "Specifying Alternate Semantics for the Explicit Congestion Notification (ECN) Field", BCP 124, RFC 4774, November 2006. [RFC4778] Kaeo, M., "Operational Security Current Practices in Internet Service Provider Environments", RFC 4778, January 2007. [RFC5129] Davie, B., Briscoe, B., and J. Tay, "Explicit CongestionEardley (Editor) Expires October 9, 2009 [Page 52] Internet-Draft PCN Architecture April 2009Marking in MPLS", RFC 5129, January 2008. [RFC5462] Andersson, L. and R. Asati, "Multiprotocol Label Switching (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic Class" Field", RFC 5462, February 2009. [P.800] "Methods for subjective determination of transmission quality", ITU-T Recommendation P.800, August 1996. [Y.1541] "Network Performance Objectives for IP-based Services", ITU-T Recommendation Y.1541, February 2006.[PCN08-1] "Baseline Encoding and Transport of Pre-Congestion Information (work in progress)", Oct 2008. [PCN08-2] "Metering and marking behaviour of PCN-nodes (work in progress)", Oct 2008. [PWE3-08] "Pseudowire Congestion Control Framework (work in progress)", May 2008.[Babiarz06] Babiarz, J., Chan, K., Karagiannis, G., and P. Eardley, "SIP Controlled Admission andPreemption (workPreemption", Work inprogress)", OctProgress, October 2006.[Behringer07][Behringer09] Behringer, M. and F. Le Faucheur, "Applicability of Keying Methods for RSVPSecurity (workSecurity", Work inprogress)", Nov 2007.Progress, March 2009. [Briscoe06] Briscoe, B., Eardley, P., Songhurst, D., Le Faucheur, F., Charny, A., Babiarz, J., Chan, K., Dudley, S., Karagiannis, G., Bader, A., and L. Westberg, "An edge-to-edge Deployment Model forPre-CongestionPre- Congestion Notification: Admission Control over a DiffservRegion (workRegion", Work inprogress)",Progress, October 2006.[Briscoe08-1]Eardley Informational [Page 44] RFC 5559 PCN Architecture June 2009 [Briscoe08] Briscoe, B., "Emulating Border Flow Policing using Re-PCN on BulkData (workData", Work inprogress)", SeptProgress, September 2008.[Briscoe08-2][Briscoe09] Briscoe, B., "Tunnelling of Explicit CongestionNotification (workNotification", Work inprogress)", JulyProgress, March 2009. [Bryant08] Bryant, S., Davie, B., Martini, L., and E. Rosen, "Pseudowire Congestion Control Framework", Work in Progress, May 2008. [Charny07-1] Charny, A., Babiarz, J., Menth, M., and X. Zhang, "Comparison of Proposed PCNApproaches (workApproaches", Work inprogress)",Progress, November 2007. [Charny07-2] Charny, A., Zhang, X., Le Faucheur, F., and V. Liatsos, "Pre-Congestion Notification Using Single Marking forEardley (Editor) Expires October 9, 2009 [Page 53] Internet-Draft PCN Architecture April 2009Admission andTermination (workTermination", Work inprogress)",Progress, November 2007. [Charny07-3] Charny, A., "Email to PCN WG mailing list", November 2007,<http:// www1.ietf.org/mail-archive/web/pcn/current/msg00871.html>.<http://www1.ietf.org/mail-archive/ web/pcn/current/msg00871.html>. [Charny08] Charny, A., "Email to PCN WG mailing list", March 2008,<http:// www1.ietf.org/mail-archive/web/pcn/current/msg01359.html>.<http://www1.ietf.org/mail-archive/web/ pcn/current/msg01359.html>. [Eardley07] Eardley, P., "Email to PCN WG mailing list", October 2007,<http:// www1.ietf.org/mail-archive/web/pcn/current/msg00831.html>.<http://www1.ietf.org/mail-archive/ web/pcn/current/msg00831.html>. [Eardley09] Eardley, P., "Metering and marking behaviour of PCN- nodes", Work in Progress, May 2009. [Gibbens99] Gibbens, R. and F. Kelly, "Distributed connection acceptance control for a connectionless network", Proceedings International Teletraffic Congress (ITC16), Edinburgh, pp. 941-952, 1999. [Hancock02] Hancock, R. and E. Hepworth, "Slide 14 of 'NSIS: An Outline Framework for QoS Signalling'", May 2002,<http://www-nrc.nokia.com/sua/ nsis/interim/nsis-framework-outline.ppt>.<h ttp://www-nrc.nokia.com/sua/nsis/interim/ nsis-framework-outline.ppt>. Eardley Informational [Page 45] RFC 5559 PCN Architecture June 2009 [Iyer03] Iyer, S., Bhattacharyya, S., Taft, N., and C. Diot, "An approach to alleviate link overload as observed on an IP backbone", IEEEINFOCOM ,INFOCOM, 2003, <http://www.ieee-infocom.org/2003/papers/10_04.pdf>. [Karsten02] Karsten, M. and J. Schmitt, "Admission Control Based on Packet Marking and Feedback Signalling -- Mechanisms, Implementation and Experiments", TU- Darmstadt Technical Report TR-KOM-2002-03, May 2002, <http://www.kom.e-technik.tu-darmstadt.de/ publications/abstracts/KS02-5.html>. [Kumar01] Kumar, A., Rastogi, R., Silberschatz, A., and B. Yener, "Algorithms for Provisioning Virtual Private Networks in the Hose Model", Proceedings ACM SIGCOMM (ITC16), , 2001. [Lefaucheur06] Le Faucheur, F., Charny, A., Briscoe, B., Eardley, P., Babiarz, J., and K. Chan, "RSVP Extensions for Admission Control over Diffserv using Pre-congestion Notification(PCN) (work(PCN)", Work inprogress)",Progress, June 2006.[Menth07] "PCN-Based Resilient Network Admission Control: The Impact of a Single Bit"", Technical Report , 2007, <http:// www3.informatik.uni-wuerzburg.de/staff/menth/Publications/ Menth07-PCN-Config.pdf>.[M3I] "M3I - Market Managed Multiservice Internet", <http://www.m3iproject.org/>. [Menth08-1] Menth, M., Lehrieder, F., Eardley, P., Charny, A., and J. Babiarz, "Edge-Assisted Marked FlowTermination (workTermination", Work inprogress)",Progress, February 2008. [Menth08-2] Menth, M., Babiarz, J., Moncaster, T., and B. Briscoe, "PCN Encoding for Packet-Specific Dual Marking(PSDM) (work(PSDM)", Work inprogress)",Progress, July 2008.[Menth08-3] "PCN-Based[Menth09-1] Menth, M. and M. Hartmann, "Threshold Configuration and Routing Optimization for PCN-Based Resilient Admission Control", Computer Networks, 2009, <http://dx.doi.org/10.1016/j.comnet.2009.01.013>. [Menth09-2] Menth, M., Lehrieder, F., Briscoe, B., Eardley, P., Moncaster, T., Babiarz, J., Chan, K., Charny, A., Karagiannis, G., Zhang, X., Taylor, T., Satoh, D., and R. Geib, "A Survey of PCN-Based Admission Control and Flow Termination",2008, <http://www3.informatik.uni-wuerzburg.de/staff/menth/ Publications/Menth08-PCN-Comparison.pdf>. [Moncaster08]IEEE Communications Surveys and Tutorials, <http:// www3.informatik.uni-wuerzburg.de/staff/menth/ Publications/papers/Menth08-PCN-Overview.pdf>>. Eardley(Editor) Expires October 9, 2009Informational [Page54] Internet-Draft46] RFC 5559 PCN ArchitectureAprilJune 2009 [Moncaster09-1] Moncaster, T., Briscoe, B., and M. Menth, "Baseline Encoding and Transport of Pre-Congestion Information", Work in Progress, May 2009. [Moncaster09-2] Moncaster, T., Briscoe, B., and M. Menth, "Athree state extendedPCN encodingscheme (workusing 2 DSCPs to provide 3 or more states", Work inprogress)", June 2008.Progress, April 2009. [Sarker08] Sarker, Z. and I. Johansson, "Usecases and Benefits of end to end ECN support in PCNDomains (workDomains", Work inprogress)",Progress, November 2008. [Songhurst06] Songhurst, DJ., Eardley, P., Briscoe, B., Di Cairano Gilfedder, C., and J. Tay, "Guaranteed QoS Synthesis for Admission Control with Shared Capacity", BT Technical Report TR-CXR9-2006-001, Feburary 2006,<http://www.cs.ucl.ac.uk/staff/B.Briscoe/ projects/ipe2eqos/gqs/papers/GQS_shared_tr.pdf>. [Style] "Guardian Style", Note: This document uses the abbreviations 'ie' and 'eg' (not 'i.e.'<http://www.cs.ucl.ac.uk/staff/ B.Briscoe/projects/ipe2eqos/gqs/papers/ GQS_shared_tr.pdf>. [Taylor09] Charny, A., Huang, F., Menth, M., and'e.g.'), asT. Taylor, "PCN Boundary Node Behaviour for the Controlled Load (CL) Mode of Operation", Work inmany style guides, eg, 2007, <http://www.guardian.co.uk/styleguide/>.Progress, March 2009. [Tsou08] Tsou, T., Huang, F., and T. Taylor, "Applicability Statement for the Use of Pre-Congestion Notification in a Resource-ControlledNetwork (workNetwork", Work inprogress)",Progress, November 2008. [Westberg08] Westberg, L., Bhargava, A., Bader, A., Karagiannis, G., and H. Mekkes, "LC-PCN: The Load Control PCNSolution (workSolution", Work inprogress)",Progress, November 2008. Eardley Informational [Page 47] RFC 5559 PCN Architecture June 2009 Appendix A. Possiblefuture work itemsFuture Work Items This section mentions some topics that are outside the PCN WG's currentcharter,charter butwhichthat have been mentioned as areas of interest. They might be work itemsfor:for the PCN WG after a futurere- chartering;re-chartering, some other IETFWG;WG, another standardsbody;body, or anoperator- specificoperator-specific usage that is not standardised.NOTE: itNote: It should be crystal clear that this section discusses possibilities only. The first set of possibilities relate to the restrictions described in Section 6.3: oaA single PCN-domain encompasses several autonomous systems that do not trust eachother, perhaps by usingother. A possible solution is a mechanism likere-PCN, [Briscoe08-1].re- PCN [Briscoe08]. onotNot all the nodes run PCN. For example, the PCN-domain is a multi-site enterprise network. The sites are connected by a VPN tunnel; although PCN doesn't operate inside the tunnel, the PCNEardley (Editor) Expires October 9, 2009 [Page 55] Internet-Draft PCN Architecture April 2009mechanisms still work properly because of the good QoS on the virtual link (the tunnel). Another example is that PCN is deployed on the general Internet(ie(ie, widely but not universally deployed). oapplyingApplying the PCN mechanisms to other types of traffic,ieie, beyond inelastictraffic. Fortraffic -- for instance, applying the PCN mechanisms to traffic scheduled with the Assured Forwarding per-hop behaviour. One example could be flow-rate adaptation by elastic applications that adapt according to the pre-congestion information. otheThe aggregation assumption doesn't hold, because the link capacity is too low. Measurement-based admission control is less accurate, with a greater risk of over-admission for instance. otheThe applicability of PCN mechanisms for emergency use (911, GETS, WPS, MLPP,etc.)etc.). Other possibilities include: o Probing. This is discussed inSectionAppendix A.1 below. o The PCN-domain extends to the end users.TheThis scenario is described in [Babiarz06]. The end users need to be trusted to do their own policing. If there is sufficient traffic, then the aggregation assumption may hold. A variant is that the PCN-domain extends out as far as the LAN edge switch. Eardley Informational [Page 48] RFC 5559 PCN Architecture June 2009 oindicatingIndicating pre-congestion through signalling messages rather than in-band (in the form of PCN-markedpackets)packets). otheThe decision-making functionality is at a centralised node rather than at the PCN-boundary-nodes. This requires that the PCN- egress-node signals PCN-feedback-information to the centralised node, and that the centralised node signals to the PCN-ingress- node the decision about admission (or termination).ItSuch possibility may need the centralised node and thePCN-boundary-nodesPCN-boundary- nodes to be configured with each other's addresses. The centralised case is described further in [Tsou08]. o Signalling extensions for specific protocols(eg RSVP, NSIS). For example:(eg, RSVP and NSIS) -- for example, the details of how the signalling protocol installs the flowspec at the PCN-ingress-node for an admittedPCN-flow;PCN- flow, and how the signalling protocol carries thePCN-feedback-information.PCN-feedback- information. Perhaps also for other functions suchas:as for coping with failure of a PCN-boundary-node ([Briscoe06] considers what happens if RSVP is the QoS signallingprotocol);protocol) and for establishing a tunnel across the PCN-domain if it is necessary to carry ECN marks transparently.Eardley (Editor) Expires October 9, 2009 [Page 56] Internet-Draft PCN Architecture April 2009o Policing by the PCN-ingress-node may not be needed if the PCN- domain can trust that the upstream network has already policed the traffic on its behalf. o PCN forPseudowire:Pseudowire. PCN may be used as a congestion avoidance mechanism foredge to edgeedge-to-edge pseudowire emulations[PWE3-08].[Bryant08]. o PCN forMPLS:MPLS. [RFC3270] defines how to support the Diffserv architecture in MPLSnetworks (Multi-protocol label switching).(Multiprotocol Label Switching) networks. [RFC5129] describes how to add PCN for admission control of microflows into a set of MPLS aggregates. PCN-marking is done in MPLS's EXP field (which [RFC5462] re-names the Class of Service (CoS) field). o PCN forEthernet:Ethernet. Similarly, it may be possible to extend PCN into Ethernet networks, where PCN-marking is done in the Ethernet header.NOTE:Note: Specific consideration of this extension is outside of the IETF's remit. Eardley Informational [Page 49] RFC 5559 PCN Architecture June 2009 A.1. Probing A.1.1. Introduction Probing is a potential mechanism to assist admission control. PCN's admission control, as described so far, is essentially a reactive mechanism where the PCN-egress-node monitors the pre- congestion level for traffic from each PCN-ingress-node; if the levelrisesrises, then it blocks new flows on that ingress-egress-aggregate. However, it's possible that an ingress-egress-aggregate carries no traffic, and so the PCN-egress-node can't make an admission decision using the usual method described earlier. One approach is to be "optimistic" and simply admit the new flow.HoweverHowever, it's possible to envisage a scenario where the traffic levels on other ingress-egress-aggregates are already so high that they're blocking new PCN-flows, and admitting a new flow onto this'empty'"empty" ingress-egress-aggregate adds extra traffic onto a link that is alreadypre-congested - whichpre-congested. This may 'tip the balance' so that PCN's flow termination mechanism is activated or some packets are dropped. This risk could be lessened byconfiguringconfiguring, on eachlinklink, a sufficient 'safety margin' above the PCN-threshold-rate. An alternative approach is to make PCN a more proactive mechanism. The PCN-ingress-node explicitly determines, before admitting the prospective new flow, whether the ingress-egress-aggregate can support it. This can be seen as a "pessimistic" approach, in contrast to the "optimism" of the approach above. It involvesEardley (Editor) Expires October 9, 2009 [Page 57] Internet-Draft PCN Architecture April 2009probing: a PCN-ingress-node generates and sends probe packets in order to test the pre-congestion level that the flow would experience. One possibility is that a probe packet is just a dummy data packet, generated by the PCN-ingress-node and addressed to the PCN-egress- node. A.1.2. ProbingfunctionsFunctions The probing functions are: o Make the decision that probing is needed. As described above, this is when the ingress-egress-aggregate (or the ECMP path--- see Section 6.4) carries no PCN-traffic. An alternative isalwaysto always probe,ieie, probe before admittingeveryany PCN-flow. Eardley Informational [Page 50] RFC 5559 PCN Architecture June 2009 o (if required) Communicate the request that probing isneeded -needed; the PCN-egress-node signals to the PCN-ingress-node that probing isneededneeded. o (if required) Generate probetraffic -traffic; the PCN-ingress-node generates the probe traffic. The appropriate number (or rate) of probe packets will depend on the PCN-metering algorithm; forexampleexample, an excess-traffic-metering algorithm triggers fewer PCN- marks than a threshold-metering algorithm, and so will need more probe packets. o Forward probepackets -packets; as far as PCN-interior-nodes are concerned, probe packets are handled the same as (ordinary data)PCN-packets,PCN-packets in terms of routing,schedulingscheduling, and PCN-marking. o Consume probepackets -packets; the PCN-egress-node consumes probe packets to ensure that they don't travel beyond the PCN-domain. A.1.3. Discussion ofrationaleRationale forprobing, its downsidesProbing, Its Downsides andopen issuesOpen Issues It is an unresolved question whether probing is really needed, but two viewpoints have been put forward as to why it is useful. The first is perhaps the most obvious: there is no PCN-traffic on the ingress-egress-aggregate. The second assumes that multipath routingECMP(eg, ECMP) is running in the PCN-domain. We now consider each in turn. The first viewpoint assumes the following:Eardley (Editor) Expires October 9, 2009 [Page 58] Internet-Draft PCN Architecture April 2009o There is no PCN-traffic on the ingress-egress-aggregate (so a normal admission decision cannot be made). o Simply admitting the new flow has a significant risk of leading to overload: packets dropped or flows terminated. On the former bullet, [Eardley07] suggests that, during the future busy hour of a national network with about 100 PCN-boundary-nodes, there are likely to be significant numbers of aggregates with very few flows under nearly all circumstances. The latter bullet could occur if new flows start on many of the empty ingress-egress-aggregates, which together overload a link in the PCN- domain. To be aproblemproblem, this would probably have to happen in a short time period (flash crowd) because, after the reaction time of the system, other (non-empty) ingress-egress-aggregates that pass through the link will measure pre-congestion and so block new flows. Also, flows naturally end anyway. Eardley Informational [Page 51] RFC 5559 PCN Architecture June 2009 The downsides of probing for this viewpoint are: o Probing adds delay to the admission control process. o Sufficient probing traffic has to be generated to test the pre- congestion level of the ingress-egress-aggregate. But the probing traffic itself may cause pre-congestion, causing other PCN-flows to be blocked or even terminated- and-- and, in the flash crowdscenarioscenario, there will be probing on many ingress-egress-aggregates. The second viewpoint applies in the case where there is multipath routing(ECMP)(eg, ECMP) in the PCN-domain. Note that ECMP is often used on core networks. There are two possibilities: (1) If admission control is based on measurements of the ingress- egress-aggregate, then the viewpoint that probing is useful assumes:o there's* There's a significant chance that the traffic is unevenly balanced across the ECMPpaths, and hencepaths and, hence, there's a significant risk of admitting a flow that should be blocked (because it follows an ECMP path that is pre-congested) or of blocking a flow that should be admitted.oNote: [Charny07-3] suggests unbalanced traffic is quite possible, even with quite a large number of flows on a PCN-link(eg 1000)(eg, 1000), when Assumption 3 (aggregation) is likely to be satisfied. (2) If admission control is based on measurements of pre-congestion on specific ECMP paths, then the viewpoint that probing is usefulEardley (Editor) Expires October 9, 2009 [Page 59] Internet-Draft PCN Architecture April 2009assumes:o* There is no PCN-traffic on the ECMP path on which to base an admission decision.o* Simply admitting the new flow has a significant risk of leading to overload.o* The PCN-egress-node can match a packet to an ECMP path.oNote: This is similar to the first viewpoint andso similarlyso, similarly, could occur in a flash crowd if a new flow startsmore-or-lessmore or less simultaneously on many of the empty ECMP paths. Because there are several(sometimes many)ECMP paths between each pair ofPCN- boundary-nodes,PCN-boundary-nodes, it's presumably more likely that an ECMP path is'empty'"empty" than an ingress-egress-aggregate is. To constrain the number of ECMP paths, a few tunnels could beset-upset up between each pair ofPCN-boundary-nodes.PCN- Eardley Informational [Page 52] RFC 5559 PCN Architecture June 2009 boundary-nodes. Tunnelling also solves the issue in thebulletpoint immediately above (which is otherwise hard to solve because an ECMP routing decision is made independently on each node). The downsides of probing for this viewpoint are: o Probing adds delay to the admission control process. o Sufficient probing traffic has to be generated to test the pre- congestion level of the ECMP path. But there's the risk that the probing traffic itself may cause pre-congestion, causing other PCN-flows to be blocked or even terminated. o The PCN-egress-node needs to consume the probe packets to ensure they don't travel beyond the PCN-domain, since they might confuse the destination end node. This is non-trivial, since probe packets are addressed to the destination endnode,node in order to test the relevant ECMP path(ie(ie, they are not addressed to the PCN- egress-node, unlike the first viewpoint above). The open issues associated withthis viewpointthese viewpoints include: o What rate and pattern of probe packets does the PCN-ingress-node need togenerate,generate so that there's enough traffic to make the admission decision? o What difficulty does the delay (whilst probing is done), and possible packet drops, cause applications? o Can the delay be alleviated by automatically and periodically probing on the ingress-egress-aggregate? Or does this add tooEardley (Editor) Expires October 9, 2009 [Page 60] Internet-Draft PCN Architecture April 2009much overhead? o Are there other ways of dealing with the flash crowd scenario? For instance, by limiting the rate at which new flows areadmitted;admitted, or perhaps by a PCN-egress-node blocking new flows on its empty ingress-egress-aggregates when its non-empty ones are pre-congested. o (Second viewpoint only) How does the PCN-egress-node disambiguate probe packets from data packets (so it can consume the former)? The PCN-egress-node must match the characteristic setting of particular bits in the probe packet's header orbody -body, but these bits must not be used by any PCN-interior-node's ECMP algorithm. In the generalcasecase, this isn't possible, but it should be possible for a typical ECMP algorithm (whichexamines:examines the source and destination IP addresses and port numbers, the protocol ID, and the DSCP). Eardley Informational [Page 53] RFC 5559 PCN Architecture June 2009 Author's Address Philip Eardley (editor) BT B54/77, Sirius House Adastral Park Martlesham Heath Ipswich, Suffolk IP5 3RE United KingdomEmail:EMail: philip.eardley@bt.com Eardley(Editor) Expires October 9, 2009Informational [Page61]54] ----