Benchmarking Methodology (bmwg) Internet Drafts
| Terminology for Benchmarking IPsec Devices | |||||||||||||||||
|
This purpose of this document is to define terminology specific to measuring the performance of IPsec devices. It builds upon the tenets set forth in [RFC1242], [RFC2544], [RFC2285] and other IETF Benchmarking Methodology Working Group (BMWG) documents used for benchmarking routers and switches. This document seeks to extend these efforts specific to the IPsec paradigm. The BMWG produces two major classes of documents: Benchmarking Terminology documents and Benchmarking Methodology documents. The Terminology documents present the benchmarks and other related terms. The Methodology documents define the procedures required to collect the benchmarks cited in the corresponding Terminology documents. | ||||||||||||||||
| Benchmarking Methodology for Link-State IGP Data Plane Route Convergence | |||||||||||||||||
|
This document describes the methodology for benchmarking Interior Gateway Protocol (IGP) Route Convergence. The methodology is to be used for benchmarking IGP convergence time through externally observable (black box) data plane measurements. The methodology can be applied to any link-state IGP, such as ISIS and OSPF. Link-State IGP Data Plane Route Convergence | ||||||||||||||||
| Terminology for Benchmarking Link-State IGP Data Plane Route Convergence | |||||||||||||||||
|
This document describes the terminology for benchmarking Interior Gateway Protocol (IGP) Route Convergence. The terminology is to be used for benchmarking IGP convergence time through externally observable (black box) data plane measurements. The terminology can be applied to any link-state IGP, such as ISIS and OSPF. Link-State IGP Data Plane Route Convergence | ||||||||||||||||
| Considerations for Benchmarking Link-State IGP Data Plane Route Convergence | |||||||||||||||||
|
This document discusses considerations for benchmarking Interior Gateway Protocol (IGP) Route Convergence for any link-state IGP, such as Intermediate System-Intermediate System (ISIS) and Open-Shorted Path first (OSPF). A companion methodology document is to be used for benchmarking IGP convergence time through externally observable (black box) data plane measurements. A companion | ||||||||||||||||
| Terminology for Accelerated Stress Benchmarking | |||||||||||||||||
|
This document provides the Terminology for performing Accelerated Stress Benchmarking of networking devices. The three phases of the Stress Test: Startup, Instability and Recovery are defined along with the benchmarks and configuration terms associated with the each phase. Also defined are the Benchmark Planes fundamental to stress testing configuration, setup and measurement. The | ||||||||||||||||
| Methodology Guidelines for Accelerated Stress Benchmarking | |||||||||||||||||
|
Routers in an operational network are configured with multiple protocols and security policies while simultaneously forwarding traffic and being managed. To accurately benchmark a router for deployment it is necessary to test the router in a lab environment under accelerated conditions, which is known as Stress Testing. This document provides the Methodology Guidelines for performing Accelerated Stress Benchmarking of networking devices. The methodology is to be used with the companion terminology document [4]. These guidelines can be used as the basis for additional methodology documents that benchmark stress conditions for specific network technologies. for Accelerated Stress Benchmarking | ||||||||||||||||
| Methodology for Benchmarking IPsec Devices | |||||||||||||||||
|
The purpose of this draft is to describe methodology specific to the benchmarking of IPsec IP forwarding devices. It builds upon the tenets set forth in [RFC2544], [RFC2432] and other IETF Benchmarking Methodology Working Group (BMWG) efforts. This document seeks to extend these efforts to the IPsec paradigm. The BMWG produces two major classes of documents: Benchmarking Terminology documents and Benchmarking Methodology documents. The Terminology documents present the benchmarks and other related terms. The Methodology documents define the procedures required to collect the benchmarks cited in the corresponding Terminology documents. | ||||||||||||||||
| Benchmarking Terminology for Protection Performance | |||||||||||||||||
|
This document provides common terminology and metrics for benchmarking the performance of sub-IP layer protection mechanisms. The performance benchmarks are measured at the IP-Layer, so avoid dependence on specific sub-IP protection mechanisms. The benchmarks and terminology can be applied in methodology documents for different sub-IP layer protection mechanisms such as Automatic Protection Switching (APS), Virtual Router Redundancy Protocol (VRRP), Stateful High Availability (HA), and Multi-Protocol Label Switching Fast Reroute (MPLS-FRR). | ||||||||||||||||
| Methodology for benchmarking MPLS Protection mechanisms | |||||||||||||||||
|
This draft describes the methodology for benchmarking MPLS Protection mechanisms for link and node protection as defined in [MPLS-FRR-EXT]. The benchmarking and terminology [TERM-ID] are to be used for benchmarking MPLS based protection mechanisms [MPLS-FRR-EXT]. This document provides test methodologies and test-bed setup for measuring failover times while considering all dependencies that might impact faster recovery of real time services riding on MPLS based primary tunnel. The terms used in the procedures included in this document are defined in [TERM-ID]. | ||||||||||||||||
 |
 |
 |
 |
|
 |
|
 |
|
 |
|
 |
Benchmarking Methodology (bmwg)
In addition to this official charter maintained by the IETF Secretariat, there is additional information about this working group on the Web at:
Additional BMWG Page
Last Modified: 2008-04-23
Additional information is available at tools.ietf.org/wg/bmwg
Chair(s):
Operations and Management Area Director(s):
Operations and Management Area Advisor:
Mailing Lists:
General Discussion: bmwg@ietf.orgTo Subscribe: bmwg-request@ietf.org
In Body: subscribe your_email_address
Archive: http://www.ietf.org/mail-archive/web/bmwg/index.html
Description of Working Group:
The major goal of the Benchmarking Methodology Working Group is to makea series of recommendations concerning the measurement of the
performance characteristics of various internetworking technologies;
further, these recommendations may focus on the systems or services
that are built from these technologies.
Each recommendation will describe the class of equipment, system, or
service being addressed; discuss the performance characteristics that
are pertinent to that class; clearly identify a set of metrics that aid
in the description of those characteristics; specify the methodologies
required to collect said metrics; and lastly, present the requirements
for the common, unambiguous reporting of benchmarking results.
To better distinguish the BMWG from other measurement initiatives in
the IETF, the scope of the BMWG is limited to technology
characterization using simulated stimuli in a laboratory environment.
Said differently, the BMWG does not attempt to produce benchmarks for
live, operational networks. Moreover, the benchmarks produced by this
WG shall strive to be vendor independent or otherwise have universal
applicability to a given technology class.
Because the demands of a particular technology may vary from
deployment to deployment, a specific non-goal of the Working Group is
to define acceptance criteria or performance requirements.
An ongoing task is to provide a forum for discussion regarding the
advancement of measurements designed to provide insight on the
operation internetworking technologies.
Goals and Milestones:
| Done | Expand the current Ethernet switch benchmarking methodology draft to define the metrics and methodologies particular to the general class of connectionless, LAN switches. | |
| Done | Edit the LAN switch draft to reflect the input from BMWG. Issue a new version of document for comment. If appropriate, ascertain consensus on whether to recommend the draft for consideration as an RFC. | |
| Done | Take controversial components of multicast draft to mailing list for discussion. Incorporate changes to draft and reissue appropriately. | |
| Done | Submit workplan for initiating work on Benchmarking Methodology for LAN Switching Devices. | |
| Done | Submit workplan for continuing work on the Terminology for Cell/Call Benchmarking draft. | |
| Done | Submit initial draft of Benchmarking Methodology for LAN Switches. | |
| Done | Submit Terminology for IP Multicast Benchmarking draft for AD Review. | |
| Done | Submit Benchmarking Terminology for Firewall Performance for AD review | |
| Done | Progress ATM benchmarking terminology draft to AD review. | |
| Done | Submit Benchmarking Methodology for LAN Switching Devices draft for AD review. | |
| Done | Submit first draft of Firewall Benchmarking Methodology. | |
| Done | First Draft of Terminology for FIB related Router Performance Benchmarking. | |
| Done | First Draft of Router Benchmarking Framework | |
| Done | Progress Frame Relay benchmarking terminology draft to AD review. | |
| Done | Methodology for ATM Benchmarking for AD review. | |
| Done | Terminology for ATM ABR Benchmarking for AD review. | |
| Done | Terminology for FIB related Router Performance Benchmarking to AD review. | |
| Done | Firewall Benchmarking Methodology to AD Review | |
| Done | First Draft of Methodology for FIB related Router Performance Benchmarking. | |
| Done | First draft Net Traffic Control Benchmarking Methodology. | |
| Done | Methodology for IP Multicast Benchmarking to AD Review. | |
| Done | Resource Reservation Benchmarking Terminology to AD Review | |
| Done | First I-D on IPsec Device Benchmarking Terminology | |
| Done | EGP Convergence Benchmarking Terminology to AD Review | |
| Done | Resource Reservation Benchmarking Methodology to AD Review | |
| Done | Net Traffic Control Benchmarking Terminology to AD Review | |
| Done | IGP/Data-Plane Terminology I-D to AD Review | |
| Done | IGP/Data-Plane Methodology and Considerations I-Ds to AD Review | |
| Done | Hash and Stuffing I-D to AD Review | |
| Mar 2007 | IPv6 Benchmarking Methodology to AD Review | |
| Dec 2007 | Net Traffic Control Benchmarking Methodology to AD Review. | |
| Dec 2007 | IPsec Device Benchmarking Terminology to AD Review | |
| Dec 2007 | Router Accelerated Test Terminology to AD Review | |
| Dec 2007 | Router Accelerated Test Methodology to AD Review | |
| Dec 2007 | IPsec Device Benchmarking Methodology to AD Review | |
| Dec 2007 | Router Accelerated Test Method for EBGP to AD Review | |
| Dec 2007 | Router Accelerated Test Method for Operational Security to AD Review | |
| Feb 2008 | Terminology For Protection Benchmarking to AD Review | |
| Feb 2008 | Methodology For Protection Benchmarking to AD Review | |
| Jul 2009 | Basic BGP Convergence Benchmarking Methodology to AD Review. |
Internet-Drafts:
Terminology for Benchmarking IPsec Devices (97352 bytes)Benchmarking Methodology for Link-State IGP Data Plane Route Convergence (48516 bytes)
Terminology for Benchmarking Link-State IGP Data Plane Route Convergence (41039 bytes)
Considerations for Benchmarking Link-State IGP Data Plane Route Convergence (15194 bytes)
Terminology for Accelerated Stress Benchmarking (52858 bytes)
Methodology Guidelines for Accelerated Stress Benchmarking (31277 bytes)
Methodology for Benchmarking IPsec Devices (93113 bytes)
Benchmarking Terminology for Protection Performance (60392 bytes)
Methodology for benchmarking MPLS Protection mechanisms (70163 bytes)
Request For Comments:
Benchmarking Terminology for Network Interconnection Devices (RFC 1242) (22143 bytes)Benchmarking Methodology for Network Interconnect Devices (RFC 1944) (66061 bytes) obsoleted by RFC 2544
Benchmarking Terminology for LAN Switching Devices (RFC 2285) (43130 bytes)
Terminology for IP Multicast Benchmarking (RFC 2432) (29758 bytes)
Benchmarking Methodology for Network Interconnect Devices (RFC 2544) (66688 bytes) obsoletes RFC 1944
Benchmarking Terminology for Firewall Performance (RFC 2647) (45374 bytes)
Terminology for ATM Benchmarking (RFC 2761) (61219 bytes)
Benchmarking Methodology for LAN Switching Devices (RFC 2889) (73251 bytes)
Methodology for ATM Benchmarking (RFC 3116) (294857 bytes)
Terminology for Frame Relay Benchmarking (RFC 3133) (44182 bytes)
Terminology for ATM ABR Benchmarking (RFC 3134) (29542 bytes)
Terminology for Forwarding Information Base (FIB) based Router Performance (RFC 3222) (25483 bytes)
Benchmarking Methodology for Firewall Performance (RFC 3511) (67916 bytes)
Methodology for IP Multicast Benchmarking (RFC 3918) (64652 bytes)
Terminology for Benchmarking BGP Device Convergence in the Control Plane (RFC 4098) (66845 bytes)
Benchmarking Basic OSPF Single Router Control Plane Convergence (RFC 4061) (32706 bytes)
OSPF Benchmarking Terminology and Concepts (RFC 4062) (15784 bytes)
Considerations When Using Basic OSPF Convergence Benchmarks (RFC 4063) (23401 bytes)
Terminology for Benchmarking Network-layer Traffic Control Mechanisms (RFC 4689) (62369 bytes)
Hash and Stuffing: Overlooked Factors in Network Device Benchmarking (RFC 4814) (59272 bytes)
Benchmarking Terminology for Resource Reservation Capable Routers (RFC 4883) (54205 bytes)
IPv6 Benchmarking Methodology for Network Interconnect Devices (RFC 5180) (41712 bytes)
IETF Secretariat - Please send questions, comments, and/or suggestions to ietf-web@ietf.org.
Return to working group directory. 
