draft-ietf-sigtran-mdtp-04.txt  -->   draft-ietf-sigtran-mdtp-05.txt

view Side-By-Side changes

INTERNET-DRAFT                                                   Q. Xie
                                                               Motorola
                                                                 T. Bova
                                                               S
	                                                     S. Hussain
                                                           T Krivoruchka
                                                                R. Revis
                                                               C. Sharp
	                                                          Cisco
                                                     H. J. Schwarzbauer
                                                                Siemens
                                                              T. Taylor
                                                        Nortel Networks
                                                              I. Rytina
                                                               Ericsson

expires in six months                                      April 19                                      June 2, 1999



           MULTI_NETWORK DATAGRAM TRANSMISSION PROTOCOL
                <draft-ietf-sigtran-mdtp-04.txt>
                <draft-ietf-sigtran-mdtp-05.txt>


Status of This Memo

This document is an Internet-Draft and is in full conformance with 
all provisions of Section 10 of RFC2026. Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas,
and its working groups.  Note that other groups may also distribute
working documents as Internet-Drafts.

The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt

The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.



Abstract

This Internet Draft discusses an experimental call control signaling
transport a new protocol, namely the Multi-network
Datagram Transmission Protocol (MDTP), that is intended to provide
fault-tolerant reliable data transfer between communicating entities
over IP networks [1].

MDTP is proposed as an application-level protocol which that is designed
with a high emphasis on supporting to
support redundant networks and transparent fault management. MDTP also gives the user a great degree of
provides timing control and configuration flexibilities in order to meet the
stringent
time constraints timing requirements often found in telephony signaling
protocols. The motivation of developing MDTP is to establish a framework for
supporting support
Internet-based high reliability real-time commercial applications such as signaling and
call control for Internet telephony.




Stewart, et al                                                  [Page  1]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999


                        TABLE OF CONTENTS

1.  Introduction  Introduction.......................................................3
  1.1 Terminology......................................................3
  1.2 Design Requirements of MDTP
     1.2 Interfaces MDTP......................................4
  1.3 Interface to MDTP MDTP................................................5
2.  MDTP Datagram Format Format...............................................5
  2.1 MDTP Common Header Field Descriptions Descriptions............................6
  2.2 MDTP Control Parameter Part Definitions..........................7
  2.3 MDTP Data Field Part Definitions......................................11
3.  Transmission Initialization
     3.1  Endpoint Association Initialization
       3.1.1 Choice of Initialization...............................12
  3.1 Initiation Message and Tag Value Lock.................................12
  3.2 Data Field Format Tag Unlock and TSN Initialization...............................13
  3.3 Datagram Processing during Tag Lock ............................14
  3.4 An Example of Association Initialization .......................14
  3.5 Other Initiation Datagrams
     3.3 Issues.........................................15
    3.5.1 Selection of Tag Value......................................15
    3.5.2 Initiation from behind a NAT................................15
    3.5.3 Initialization Collision
     3.4 Collision....................................16
    3.5.4 Association Re-initialization Re-initialization...............................16
4.  Reliable  Transfer of Datagrams User Datagram............................................16
  4.1 Timer Management Rules Rules..........................................17
    4.1.1 Link Rotation T3-send Timer Adjustment with RTT...........................18
  4.2 Gap Acknowledgment for Missing Datagrams Multihoming Rotation............................................18
    4.2.1 Remote Multihoming Rotation.................................18
    4.2.2 Local Multihoming Rotation..................................19
  4.3 Stream Sequence Number..........................................19
  4.4 Ordered and Un-ordered Delivery.................................19
  4.5 Report Missing Datagrams........................................20
  4.6 Range Check on TSN .............................................21
  4.7 Advisory Ack Request............................................21
5   Congestion Control
       4.3.1 Sending Controls...............................................22
  5.1 Send with Window Control
       4.3.2 Control........................................22
    5.1.1 Window Length Adjustment
       4.3.3 Flow Control using In-Queue Information
       4.3.4 T3-send Adjustment....................................23
  5.2 Send Timer Adjustment with RTT
     4.4 Sequence Number Reset
     4.5 Datagram Re-transmission
       4.5.1 Re-transmission on Back-off at Re-transmission..........................24
6.  Network Management................................................25
  6.1 Failure Detection in Redundant networks
     4.6 RTT Measurement
       4.6.1 Networks.........................25
  6.2 RTT Datagram Header Format
       4.6.2 Measure RTT
     4.7 Link Measurement.................................................26
  6.3 Network Heart Beat
     4.8 Advisory Acknowledgment
     4.9 .............................................26
7.  Termination of an Association
     4.10 Draining Association........................................27
  7.1 Graceful Shutdown of an Association
5. Interface with upper level protocols
6. Suggested MDTP Protocol Parameter Values
7. Acknowledgments Association.............................28
8. Author's Addresses
9. References
Appendix A: Stream-based Reliable and Ordered Delivery
     A.1 Stream Initiation
     A.2 Stream Termination
     A.3 Stream Datagram Transfer
       A.3.1 Header Format in  Stream Datagrams with User Data
       A.3.2 Transmission of Operations.................................................29
  8.1 Stream Datagrams
       A.3.3 Extended Initiation...............................................29
  8.2 Stream Ack
     A.4 Termination..............................................29
  8.3 Other Issues with Stream Transfer
Appendix B: Bundled Message Transfer
     B.1 Format of Bundled Datagram
     B.2 Bundled Datagram Transfer
Appendix C: Fragmented Message Transfer
Appendix D: Multicast Datagram Transfer
     D.1 Multicast Operations.............................30
9.  Interface with Upper Layer........................................30
10. Suggested MDTP Timer and Protocol Parameter Values................34
11. Acknowledgments...................................................34
12. Authors' Addresses................................................34
13. References........................................................35



Stewart, et al                                                  [Page  2]

Internet Draft   Multi-network Datagram Header Format
     D.2 Transmission of Multicast Datagrams
Appendix E: Unreliable Delivery
     E.1 Ordered Unreliable Delivery Protocol   June 1999



1.  Introduction

This Internet Draft discusses an experimental a new protocol, namely the Multi-network
Datagram Transmission Protocol (MDTP). The intention of developing
MDTP is to provide a fault-tolerant, real-time reliable data transfer
mechanism between communicating endpoints over IP networks [1].

MDTP is proposed as an application-level protocol which that is designed
with a high emphasis on supporting to
support redundant networks and transparent fault management. MDTP also gives the user a great degree of
provides timing control and configuration flexibilities in order to meet the
stringent
time constraints timing requirements often found in telephony signaling
protocols. The motivation of developing MDTP is to establish a framework for
supporting support
Internet-based high reliability real-time commercial applications such as signaling and
call control for Internet telephony.

MDTP is also designed to be scalable in order to support different
signaling transport requirements for different interfaces in to a
telephony network.

For example, the transportation of signaling protocols such as PRI ISDN
PRI may not require redundant links, networks, and hence only a subset of
MDTP will need to be implemented.  On the other hand, redundant
networks may be mandated when transporting SS7 signaling messages
amongst different components in a carrier-grade telephony core
network.  In such cases, the transparent support for redundant
networks, load sharing, and fault management defined in MDTP become essential and
likely need to be fully supported in an implementation.
essential.

Many of the fundamental concepts that have made TCP such a  useful
protocol are reused in MDTP, and some of the advantages of UDP are
also merged into the design. This has lead


1.1 Terminology

The following terms are defined and used in this document: 

- Redundant networks:

  An endpoint may be able to a highly effective,
robust protocol for fault tolerant data communications. transmit or receive on more than one IP
  address/UDP port. RFC 1122 refers to this as multihoming. This document describes
  constitutes a redundant local network (for MDTP) relative to the functional interface
  endpoint. MDTP makes no attempt to assure routing diversity within
  the internet connecting two endpoints. Each endpoint attempts to
  send to its peer endpoint using all the IP addresses and UDP ports
  its peer has open (within the details
necessary for implementing MDTP. The main body constraints of this document
contains the minimal set any application
  specified restrictions). The choice of functionalities which local socket to send
  upon is an implementation detail (it is possible only one socket is
  available and bound to all of the local networks to which the machine is
  connected). The O/S also will play a role in the multihoming/redundancy. 
  MDTP attempts a best effort at spreading the traffic across a 

Stewart, et al                                                  [Page  3]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999

  destination's available interfaces. It is assumed by MDTP that must be
implemented. In the Appendices, 
  network (if fault tolerance is desired) is engineered for diversity 
  and MDTP's best effort will play only a set small role in that diversity.

- Endpoint:

  Representation of additional the logical point where MDTP functions,
such as reliable stream, multicast, message bundling, message
fragmentation, are defined. Those additional functionalities are
optional datagrams can be sent
  to implementation.

1.1 Design Requirements of or received from. Moreover, an MDTP

The following are some of the design requirements endpoint shall be defined as
  a set of MDTP, IP address/port combinations in order to
make MDTP capable of supporting real-time call control environments
which potentially may employ support redundant networks:

A) High communication fan-out:
  networks. For example, an endpoint may need to be in
   simultaneous communication on a multi-homed host connected 
  with hundreds or thousands of endpoints
   performing various call processing N IP networks can be represented as:

     [IP addr1/port1,
       ...
      IP addrN/portN]

  where the port numbers or IP addresses may not be unique, but their
  combinations shall be guaranteed to be unique by the underneath IP
  networks. 

- Association:

  Representation of an ongoing communication channel between two MDTP
  endpoints. 

- Stream:

  Defines a sub-channel within an association. Datagrams sent through
  a stream shall be reliably transmitted and delivered independent to
  datagrams from other streams. 

  Each stream shall be identified by a stream number that is unique
  within the association. Stream 0xffff is reserved and shall not be
  used. 


1.2 Design Requirements of MDTP

The following are some of the design requirements of MDTP to
make MDTP capable of supporting real-time call control environments
that may employ redundant networks:

A) High communication fan-out: an endpoint may need to be in
   simultaneous communication with hundreds or thousands of endpoints
   performing various call processing functions. These endpoints may
   be codec converters, SS7 to IP translation applications, or, in the
   case of mobile networks, data selector and combiner applications.

B) Stringent timer control: an endpoint needs to have a very fine
   control over the timing for delivering a datagram. The timing
   should be easily adjusted depending on the message type and the
   destination. For example, after a few seconds of non-delivery the
   call which the message is about may not exist anymore.

Stewart, et al                                                  [Page  4]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999


C) Support redundant links: multiple network paths: an endpoint communicating with a peer
   should be able to take advantage of the redundant networks multiple network paths and
   multihoming in a transparent way. Therefore, the protocol must 
   be able to take advantage of local multi-homed hosts and remote 
   multi-homed hosts to provide resilient data delivery. This means 
   that the application or upper layer protocols need not to be involved 
   in the network fault management. Instead, when network failure occurs 
   MDTP should be able to automatically re-route the transmit out-bound datagram datagrams to the an
   alternate destination network interface (if one exists) without
   intervention from the application.

D) Orderly delivery: Reliable transport: datagrams may arrive out of order, might be lost or discarded while
   traveling in the IP network towards the destination. The protocol
   must handle the re-transmission of lost messages in an autonomous
   way without any intervention from the upper layer. Also, sometimes
   datagrams may arrive in duplicate copies. This is especially true if redundant networks
   are used. copies, in such cases MDTP should must
   be strong enough able to properly handle both
   situations with little intervention from detect and remove the upper layer protocols
   or applications.

F) duplicates automatically.

E) Support stream sequencing: on both ordered and un-ordered delivery: MDTP must support
   both ordered and un-ordered delivery. In the demand case of ordered
   delivery, the upper layer
   protocols or applications, MDTP should receiver shall detect out-of-order datagrams and
   re-order them before dispatching them to the upper layer. In the
   un-ordered case, received datagrams shall be dispatched without any
   effort of re-ordering.

F) Support stream sequencing: on the demand of the upper layer
   protocols or applications, MDTP should be able to support sequenced
   delivery with regard to each individual stream, i.e., the delay caused
   by the loss and retransmission of a datagram should be isolated to
   only the stream to which the datagram belongs. This is particularly
   important in some call control applications, where a loss of a 
   message should only affect the call whom the message belongs to.

1.2 Interfaces


1.3 Interface to MDTP

The application programs or upper layer protocols interface with MDTP
through a set of primitives (see section 5. for details). 9).

Towards the IP networks, it is assumed that a UDP-like data transport
protocol will provide the interface between MDTP and UDP is used for the operating
system.
transport layer. No special interfaces or changes are assumed within
UDP or at the
operating system, all UDP/MDTP interface.  MDTP maintains its own queuing and
endpoint association information are
maintained inside association.  When MDTP layer. runs on a router or on a
gateway-enabled host, it will place no special constraints on the
lower layer protocol implementations other than those described in the
Router Requirements and Host Requirements RFCs.


2.  MDTP Datagram Format

A MDTP inserts the following protocol header at the beginning datagram consists of every
user datagram. The integer fields shall be transmitted in network byte
order. a common header and possibly a control
parameter part, a data part, or both.


Stewart, et al                                                  [Page  5]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999

                      MDTP Header Datagram Format
                                    
    0                   1                   2                   3   
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  MDTP Protocol Identifier             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Version     |              Flags            |   In Queue    |
   |               |N N W I F R D A M S W R R F G U|               |
   |               |O O I S I T A C U H N E T L A N|               |
   |               |M B N B R M T K L U R 1 C O R R|               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                Acknowledgment Number (Seen) Vers  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |C/D| Msg Type  |                   Sequence Number (Send)                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+   Reserved    |         Data Size             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   \                                                               \
   /                    Control Parameter Part       |      Of       |                     /
   \                                                               \
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   \                                                               \
   /                             data                          Data Part                            /
   \                                                               \
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


Note: integers in the header of MDTP datagrams MUST be transmitted in
network byte-order. 

Note: when both the control part and data part are present in an MDTP
datagram, the control part MUST be processed first. 


2.1 MDTP Common Header Field Descriptions

    MDTP Protocol Identifier: 32 28 bits

      This shall be a fixed long value of 0xf7873072. 0xf787307. The receiver shall
      always verify this Protocol Identifier before it proceeds any
      further in interpreting the header fields. 

    Version: 8 4 bits 

      This field represents the version number of the MDTP protocol
      (value TBD).

    Flags: 16 bits

      NOM - protocol,
      and shall be set to 1 (reserved for fragmentation, see
      Appendix C)

      NOB 0x3. 

    C/D Bits: 2 bits

      This field indicates whether the Message Type and Data Size fields
      are filled in the present datagram:

         00 - reserved, shall not be used. The receiver shall silently
              discard any datagram with C/D bits set to 1 (reserved for bundling, see Appendix B)

      WIN 00.
         01 - Window Up. Data Size only
         10 - Message Type only
         11 - Message Type and Data Size

    Message Type: 6 bits
    
      This bit is set by the sender of this datagram
      to shall indicate that the sender needs type of control message. Its value is valid
      only when the receiver C/D bits are set to acknowledge on
      previously received datagrams before either "10" or "11". Otherwise it can send more datagrams.

      ISB -

Stewart, et al                                                  [Page  6]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999

      shall be set to 0 (reserved for bundling, see Appendix B)

      FIR - First Datagram. This flag 0x0 and ignored by the receiver. 

      Message Type determines whether the control part is set to indicate that this present in
      the current datagram.

      The value of Message Type is a defined as the follows:

      0x0  - reserved and shall not be used

      0x1  - Initiation datagram.

      RTM
      0x2  - normally set Initiation Ack
      0x3  - Extended Data Ack
      0x4  - Advisory Ack Request
      0x5  - Window-up
      0x6  - Window-up Ack
      0x7  - RTT-request
      0x8  - RTT-ack
      0x9  - Abort
      0xa  - Graceful Shutdown
      0xb  - Graceful Shutdown Ack
      0xc  - Stream Initiation
      0xd  - Stream Initiation Ack
      0xe  - Stream Termination
      0xf  - Stream Termination Ack

      0x10 to 0 (used for Link Heart Beat 0x3f - reserved and RTT
      measurement, see sections 4.6 shall not be used

    Reserved: 8 bits

      These bits are reserved for future use. The sender shall always
      set these bits to '0', and 4.7)

      DAT - the receiver shall ignore there
      values. 

    Data Present. Size: 16 bits

      This bit is set to indicate that, following
      this header, application value represents, in number of octets, the size of the user
      data is present in this the Data Part of the current datagram.

      ACK - Acknowledge. This bit Its value
      is only valid when C/D bits are set to indicate that the sender is
      acknowledging the reception of the specified Acknowledgment Number.

      MUL - either "01" or "11". 
      Otherwise it shall be set to 0 (reserved for multicast, see Appendix D)

      SHU - Shutdown. This bit is set when the sender initiates its
      closing procedure 0x0 and indicates to the receiver that ignored by the sender
      is no longer receiver.


2.2 MDTP Control Parameter Part Definitions

This section defines whether a valid destination. If the UNR bit control parameter part is set in
      conjunction with the SHU bit, an incomplete shutdown present for
each message type, and its format if a control parameter part is
      specified. After an incomplete shutdown,
present. 

2.2.1 Initiation (0x1) and Initiation Ack (0x2):

    The parameter field of the receiver can still
      re-establish Initiation and Initiation Ack messages 
    shall carry two initiation Tags, the communication with maximum window length and the sender by re-initiating
      with 
    sender's local network information. Note that the sender (see 4.7).

      WNR - Window Up Response. This bit is set in endpoint MAY 
    be multi-homed. 


Stewart, et al                                                  [Page  7]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999

    The following defines the acknowledgment
      reply to a Window Up flag.

      RE1 - normally set to 0 (used for advisory ACK, see section 4.8)

      RTC - normally set to 0, (used parameter format for RTT, see section 4.6)

      FLO - shall carrying N IPv4
    Network addresses (other network address formats can be set to carried by
    setting the size and type fields accordingly):

     0 (reserved for reliable stream, see
      Appendix A)

      GAR - shall be set to                   1 (reserved for unreliable mode, see
      Appendix E)

      UNR - shall be set to                   2                   3   
     0 (reserved for unreliable mode see
      Appendix E)

    In Queue: 1 2 3 4 5 6 7 8 bits

      This field contains the number 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                       Tag Value 1 (Seen)                      |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                       Tag Value 2 (Send)                      |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                       Max Window Length                       |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                    Number of messages the sender has on its
      incoming queue, waiting to be read by the application. This
      gives the receiver an indication Networks = N                     |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |       Size of the flow control conditions
      within the sender.

    Acknowledgment Number (or Seen): 32 bits

      If the flag ACK is set this value is the last sequence number
      that the sender address=8       |    Type of this datagram received from the
      receiver Address=2          |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                   IP Address of this datagram.

    Sequence Number (or Send): 32 bits Network 1                     |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |         Port # 1              |      Padding = 0              |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    /                                                               /
    \                              ...                              \
    /                                                               /
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |       Size of address=8       |    Type of Address=2          |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                   IP Address of Network N                     |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |         Port # N              |      Padding = 0              |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

    If DAT flag there is set, this value represents the sequence number of
      the current any implementation-specific data unit following this header. Otherwise, this
      value will needed to be
    exchanged at the sequence number setup of the next data unit that
      will association, it should be sent.

    Data Size: 16 bits

      This value represents, in number of octets, appended
    to the size end of the above data
      field that follows this header in the current datagram.

    Part: 8 bits

      shall have value '0' (reserved for fragmentation, see Appendix C)

    Of: 8 bits

      shall have value '1' (reserved for fragmentation, see Appendix C)

2.2 Data Field

When the DAT flag is set to 1, the MDTP datagram header will be
followed by a data field. An implementation may choose to pad some
'0's at the end structure. The format of the
    implementation-specific data field so should follow "Size/Type/Data Field"
    format as to align with certain memory
boundaries. However, defined above. In case an endpoint does not support the padded '0' octets, if there are any,
    implementation-specific data received, it shall
not be counted in ignore the
    additional fields. 


2.2.2 Extended Data Size. Ack (0x3):

    The maximal Data Size for a single MDTP datagram is the MTU size of
the underlying transport protocol (e.g., UDP) minus parameter field contains 0 or more gap reports and the MDTP header
size.

3.  Transmission Initialization

3.1 Endpoint Association Initialization

Before the first data
    highest transmission can take place from one endpoint
("A") to another endpoint ("Z"), the two endpoints will need to
complete an initialization process in order to set up an association
between them.

The initialization procedure should be made transparent to the upper
layer protocol, i.e., it should take place automatically whenever the
upper layer tries to send a datagram to an endpoint which has never
been sent to before. sequence number (TSN) received.










Stewart, et al                                                  [Page  8]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999


     0                   1                   2                   3   
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                        Number of Gaps = N                     |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                         Gap #1 Start TSN                      |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                          Gap #1 End TSN                       |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    /                                                               /
    \                              ...                              \
    /                                                               /
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                         Gap #N Start TSN                      |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                          Gap #N End TSN                       |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                        Last TSN Seen                          |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


2.2.3 Advisory Ack Request (0x4):

    No parameter field.

2.2.4 Window-up (0x5): 

    No parameter field.

2.2.5 Window-up Ack (0x6): 

    Same as that of Extended Data Ack.

2.2.6 RTT-request (0x7) and RTT-ack (0x8):

    The user datagram parameter field shall be withheld by MDTP from
transmission till the completion of the initialization.

A tag-and-lock mechanism is employed during contain the initialization in
order to guard against erroneous or stale datagrams (this time value that is
especially true if redundant networks are deployed).

The initialization process consists of the following steps (assuming
the upper layer at "A" tries to send data to "Z" used for the first time):

A) "A" first sends
    RTT calculation (see section 6.2), and optionally an Initiation (FIR) to "Z", with
    acknowledgment Seen field set
   to value.

     0 and Send field set to Tag_A, and then enters the Tag-lock mode
   (see below).

B) "Z" responds immediately with an Initiation Ack (FIR|ACK), with                   1                   2                   3   
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                         Time Value 1                          |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                         Time Value 2                          |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                       0x0 or TSN Seen set to Tag_A and Send set to Tag_Z, and then enters                         |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

2.2.7 Abort (0x9):

    The Abort message shall carry the
   Tag-lock mode, too (see below).

Note that no user data should initiation Tag of the
    destination endpoint as a measure of security.

Stewart, et al                                                  [Page  9]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999


     0                   1                   2                   3   
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                        Init-Tag                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

2.2.8 Graceful Shutdown (0xa):

    The destination endpoint initiation Tag shall be carried in the Initiation or
Initiation as a
    measure of security. 

     0                   1                   2                   3   
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                            Init-Tag                           |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                            TSN Seen                           |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

2.2.9 Graceful Shutdown Ack datagram.

At this point "Z" is ready to send user data to "A". And upon (0xb):

    Same as that of Abort.

2.2.10 Stream Initiation (0xc):

    The parameter field shall contain the
receipt initiation Tag of the above Initiation Ack from "Z", "A" can also start
sending user data to "Z".

However,
    destination endpoint (see section 3.1), the first datagram with Stream Identifier,
    and the Initial Sequence Number of this stream. Also, there shall
    be a "Size of Stream Info" and "Stream Information" fields that
    may contain an opaque user data transmitted by "A" structure specific to "Z"
shall have the Seen value set stream
    being opened. The "Stream Information" field should be padded with
    '0's to Tag_Z, which is obtained from the 32 bit word boundary before transmission. 

     0                   1                   2                   3   
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                           Init-Tag                            |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |         Stream Identifier     |       Reserved (set to 0)     |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                 Size of Stream Info = N                       |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    /                                                               /
    \                Stream Information (N octets)                  \
    /                                                               /
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

2.2.11 Stream Initiation Ack. And similarly, Ack (0xd):

    The parameter field shall contain the first datagram with user data
transmitted by "Z" Stream Identifier.




Stewart, et al                                                  [Page 10]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999


     0                   1                   2                   3   
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |         Stream Identifier   |       Reserved (set to "A" 0)       |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

2.2.12 Stream Termination (0xe):

    The parameter field shall have contain the Seen initiation Tag value set to Tag_A,
which comes from the Initiation datagram.

In the Tag-lock mode, each side will silently discard any datagrams
with user data from the other side until it receives the first
datagram with user data (see
    section 3.1) and with a Seen value that matches its own
Tag. Once that datagram is received, that endpoint will leave the
Tag-lock mode and immediately send back a data acknowledgment, and
start using the sequence numbers to filter out missing and duplicate
datagrams.

If another Initiation from "A" is received by "Z" after it sent out
the Initiation Ack, "Z" will acknowledge this Initiation by re-sending
the Initiation Ack only when the Send field of this new Initiation has
the same tag as that of the original Initiation.  Otherwise, "Z" will
send an Initiation of its own with Send field set to Tag_Z back to "A"
to elicit an Initiation Ack from "A".

In the following example, "A" initiates the association first and then
sends a datagram with user data to "Z":

   Endpoint A                                          Endpoint Z

   {first app message to Z}
   [Header Flags=FIR
             & other options
           Seen=0,Send=Tag_A] ----------------------->
   (Start T1-init timer)
   (Enter Tag_A-lock mode)
                                              [Header Flags=FIR|ACK
                                                        & other options
                                   /---------- Seen=Tag_A,Send=Tag_Z]
                                  /           (Enter Tag_Z-lock mode)
   (Cancel T1-init timer)<-------/

   [Header Flags=ACK|DAT
             & other options
           Seen=Tag_Z,Send=1]
           [data field]   -----------\
   (Start T3-send timer)              \
                                       \----> (Leave Tag_Z-lock mode)

If T1-init timer expires at "A" after the Initiation sent, the same
Initiation datagram with the same Tag_A value will be retransmitted
and the timer restarted. This will be repeated Max.Init.Retransmit
times before "A" considers "Z" unreachable and optionally reports the
failure.

3.1.1 Choice of Tag Value

Tag values should be selected from the range of 0x80000000 to
0xffffffff.

3.2 Data Field Format of Initiation Datagrams

If redundant networks exist between two endpoints, the data field of
the Initiation and Initiation Ack datagrams will carry the redundant
network information.

The following shows the data field format carrying N IPv4 redundant
network information: Stream Identification

     0                   1                   2                   3   
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                     Number of Networks = N                        Init-Tag                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |       Size of address=8       |    Type of Address=AF_INET (2)|
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+         Stream Identifier   |             IP Address of Network 1       Reserved (set to 0)       |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |         Port #

2.2.13 Stream Termination Ack (0xf):

    Same has that of Stream Initiation Ack.


2.3 MDTP Data Part Definitions

The following format shall be used for MDTP datagram Data Part:

    0                   1              |      Padding =                   2                   3   
    0              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   /                                                               /
   \                              ...                              \
   /                                                               / 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       Size of address=8                           TSN Seen                            |    Type of Address=AF_INET (2)|
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             IP Address of Network N                           TSN Send                            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |         Port #      Stream Identifier N      |      Padding = 0    Sequence Number n          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Additional implementation-specific data
   \                                                               \
   /                 User Data (seq n of Stream N)                 /
   \                                                               \
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


    TSN Seen: 32 bits

      This is allowed after a piggy-backed acknowledgment, indicating the redundant
network information. No user data, however, is allowed reception
      of datagrams up to be
transported this TSN.

    TSN Send: 32 bits

      This value represents the TSN of the user data carried in Initiation or Initiation Ack datagrams.

3.3 Initialization Collision

If two endpoints attempt to initialize an association with each other
at about this
      datagram.


Stewart, et al                                                  [Page 11]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999

    Stream Number: 16 bits

      Identify the same instance, a collision will occur, i.e., each side
will receive an Initiation datagram from stream to which the other side after it
transmitted its own. In such a case, both sides shall acknowledge following user date belongs.

    Sequence Number: 16 bits

      This value presents the
Initiation datagram sequence number of the other side in following user
      data within the normal procedure stream. 

      Sequence number 0x0 indicates that the following user data shall
      be treated as
described above.

3.4 Association Re-initialization

An endpoint un-ordered, and shall be allowed dispatched to re-initialize an established
association with another endpoint.

In such a case, the endpoint that initiates the re-initialization
(i.e, the initiator) shall use a tag different from upper
      layer by the one used in receiver without any attempt of re-ordering.

    User Data: variable length

      This is the previous initialization. And payload user data. The size of the initiator user data shall follow the normal
initialization procedure as stated
      be specified in section 3.1.

Once left the Tag-lock mode of the current association initialization,
an endpoint shall treat any new incoming Initiation from its peer as a
re-initialization event. Upon Data Size field. The implementation may
      optionally have some '0' padded at the arrival end of User Data field.


3. Endpoint Association Initialization

Before the new Initiation
datagram first data transmission can take place from the peer, the receiving one endpoint shall also follow
("A") to another endpoint ("Z"), the
procedure stated two endpoints must complete an
initialization process in section 3.1 order to respond.

4.  Reliable Transfer of Datagrams

Reliable transfer is indicated if the datagram being transferred has
GAR bit set up an association between them.

The upper layer may explicitly request MDTP to 1 and initialize an
association to an endpoint, or implicitly open the UNR bit set association by
sending the first datagram to that endpoint on stream 0. The receiver of a
reliable datagram shall always acknowledgment

Once the sender.

Normally, delayed acknowledgment association is used, and established, the acknowledgment global stream, i.e., stream
0, is automatically open and ready for datagram transmission. Other
streams must be explicitly opened before data transmission can
either occur.

A tag-and-lock mechanism must be sent separately or piggy-backed on a datagram traveling
in employed during the opposite direction. initialization
in order to guard against security attacks as well as erroneous
datagrams. 


3.1 Initiation Message and Tag Lock

The initialization process consists of the following example illustrates both separate and piggy-backed
acknowledgments with both ends transmitting in reliable mode:

Endpoint A                                      Endpoint Z
{App sends 3 messages}
[Header Flags=DAT|ACK|GAR
        Part=0,Of=1
        Seen=0,Send=1,Size=100]-------------> (Start T2-receive timer)
(Start T3-send timer)

[Header Flags=DAT|ACK|GAR
        Part=0,Of=1
        Seen=0,Send=2,Size=100]----------->
(Restart T3-send timer)

[Header Flags=DAT|ACK|GAR
        Part=0,Of=1
        Seen=0,Send=3,Size=100]----------->
(Restart T3-send timer)
                                              ...
                                              {Timer T2 expires}
                                 /----------- [Header Flags=ACK
                                /              Part=0,Of=0
                               /               Seen=3,Send=1]
                              /
(cancel T3-send timer) <------
...
...
{App sends 1 message}
[Header Flags=DAT|ACK|GAR
        Part=0,Of=1
        Seen=1,Send=4,Size=100]-----------> (Start T2-receive timer)
(Start T3-send timer)
                                              ...
                                              {App sends 1 message}
                                              (cancel T2-receive timer)
                                 /----------- [Header Flags=DAT|ACK|GAR
                                /              Part=0,Of=1
                               /               Seen=4,Send=1,Size=45]
                              /               (Start T3-send timer)
(cancel T3-send timer) <------
(Start T2-receive timer)
..
{Timer T2 Expires}
[Header Flags=ACK
        Part=0,Of=0
        Seen=1,Send=5]------------------> (cancel T3-send timer)

Note steps (assuming
that if the datagrams previously received from the same sending MDTP endpoint was transmitted in Unreliable transfer mode (see Appendix E
for details on Unreliable transfer), the receiving "A" tries to set up an association with MDTP
endpoint must
reset its "Z"):  

A) "A" shall first send an Initiation message to "Z", with Tag Seen counter
   field set to the value of the 0x0 and Tag Send field in the current
reliable datagram.

4.1 Timer Management Rules

The the following rules set to Tag_A, where Tag_A shall
   be used to manage a random number in the timers during
normal Reliable transfer, unless otherwise stated range of 0x80000000 to 0xffffffff (see
   3.1.4 for some special
cases:

A) When a reliable datagram Tag value selection), and enter the Tag-lock mode.

B) "Z" shall respond immediately with user data (i.e., an Initiation Ack message, with DAT flag set) is
   received,
   Seen set to Tag_A and Send set to Tag_Z (same range as Tag_A), and
   enter the endpoint shall start a T2-receive timer if no other
   timer Tag-lock-new mode.


Stewart, et al                                                  [Page 12]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999

   At this point, "Z" is running, and ready to send user datagrams to "A" in stream
   0. And upon the expiration reception of the T2-receive timer,
   the endpoint shall ack above Initiation Ack from "Z", "A"
   also becomes ready to the sender all the un-acked send user datagrams
   it has received.

B) When a reliable datagram with to "Z" in stream 0. 

   Note: user data is in other streams can not be sent out, until the sending
   endpoint
   respective streams are opened.

C) "Z" shall start leave Tag-lock-new mode and enter Tag-lock mode only if a T3-send timer. If
   user datagram has been sent out from "Z" to "A". 

   Note: to guard against "man in the T3-send timer middle" attacks, a limit should
   be imposed on the number of associations in the Tag-lock-new mode
   at any given endpoint; whenever that limit is
   already running, reached, any further
   association Initiations received by the endpoint shall first stop the old T3 timer
   and then start be silently
   discarded. Also, a new one. If the T2-receive timer is running, the
   endpoint shall first stop the T2 timer, piggyback an Ack unto be used on each association that is
   in the
   out-bound datagram, and then start a T3-send timer. Upon Tag-lock-new mode; at the expiration of the T3-send that timer, the endpoint that
   association shall follow be shutdown by the rules
   described endpoint. 

Note: no user data shall be carried in 4.5 for possible re-transmission of the un-acked
   datagrams. Whenever both the T3-send timer is started Initiation and
Initiation Ack messages, i.e., the RTT estimate
   last calculated for that network should C/D bits must be added set to 10. 

Note: both side must exchange their local network information and
their maximal window length in the Initiation and Initiation Ack
messages. 


3.2 Tag Unlock and TSN Initialization

The first user datagram transmitted by "A" to "Z" shall have the base
   T3-send timer value (if a RTT TSN
Seen value is measured, see section 4.6).

C) When all outstanding datagrams are acknowledged, set to Tag_Z in the T3-send timer
   shall be stopped if one is still running.

The following example shows Data Part (see 2.3).

Similarly, the use of various timers.

Endpoint A                                         Endpoint Z
{App sends 2 messages}
[Header Flags=DAT|ACK|GAR
        Part=0,Of=1
        Seen=1,Send=6,Size=100]-----------> (Start T2-receive timer)
(Start T3-send timer)

[Header Flags=DAT|ACK|GAR
        Part=0,Of=1                           {App sends 1 message}
        Seen=1,Send=7,Size=100]---\      /--- (cancel T2-receive timer)
(Restart T3-send timer)            \    /     [Header Flags=DAT|ACK|GAR
                                    \  /       Part=0,Of=1
                                     \/        Seen=6,Send=2,Size=100]
                                     /\       (Start T3-send timer)
                                    /  \
                              <----/    ---->
...
...
{T3-send timer expires}
(re-transmit 2nd datagram)
[Header Flags=DAT|ACK|GAR
        Part=0,Of=1
        Seen=2,Send=7,Size=100]---------> (Cancel T3-send timer)
(Restart T3-send timer)                       (Start T2-receive timer)

                                              ..
                                              {Timer T2 expires}
(Cancel T3-send timer)        <-------------- [Header Flags=ACK
                                               Part=0,Of=0
                                               Seen=7,Send=3]

4.1.1 Link Rotation

When multiple networks exist between two communicating endpoints,
every time the application transmits a datagram, first user datagram transmitted by "Z" to "A" shall
have the MDTP
implementation MUST keep track TSN Seen value set to Tag_A.

The reception of which network this first datagram with user data and with the transmission was
sent on (if more than one network exists)
correct Tag value in the MDTP protocol variable
'last.sent.intf'. If TSN Seen field from its peer shall unlock the user does not specifically override rotation,
each send should be rotated in a round robin fashion amongst all
available networks
Tag and cause the protocol variable 'last.sent.intf' should
be updated endpoint to indicate which interface was used last. leave the Tag-lock or Tag-lock-new mode.

The MDTP implementation MUST allow a user receiver shall immediately send back an Extended Data Ack to override
acknowledge the reception of this rotation
defeating MDTP's rotation upon each send. first user datagram.

The implementation must also
provide a interface TSN Send value carried in this first datagram with user data shall
be used to add and remove a link from rotation eligibility.

4.2 Gap Acknowledgment for Missing Datagrams

If reliable datagrams become missing during a series establish the initial TSN of transmissions,
a special type this peer, i.e., the sender of acknowledgment known as
this datagram. 

To strengthen the Gap Ack will security, this initial TSN shall be sent
back to inform the sender to re-transmit randomly
selected from the missing datagrams.

The following example shows range between 0x1 and 0x7fffffff by the use of Gap Ack.

Endpoint A                                    Endpoint Z
{App sends 3 messages}
[Header Flags=DAT|ACK|GAR
        Part=0,Of=1
        Seen=3,Send=8,Size=100]-----------> (Start T2-receive timer)
(Start T3-send timer)

[Header Flags=DAT|ACK|GAR
        Part=0,Of=1
        Seen=3,Send=9,Size=100]-----X (lost)
(Restart T3-send timer)

[Header Flags=DAT|ACK|GAR
        Part=0,Of=1
        Seen=3,Send=10,Size=100]-----------> (A gap detected sender, by
means such as those suggested in data)
(Restart T3-send timer)
                                             ..
                                             {T2-receive timer expires}
                                    /------- [Header Flags=ACK
                                   /          Seen=9,Send=3,
                                  /           Part=1,Of=1
                                 /            data=(long integer)10]
(Prepare retransmit)   <--------/

In this example, RFC 1750 [9].

Note: if there exists any un-acked datagram(s) when "Z" receives the third an endpoint is to
send its first user datagram from "A" it
realizes that a gap exists in the received data.  At to its peer, the expiration of
T2-receive timer, "Z" sends a Gap Ack, in place of endpoint MUST send a normal Ack, to
"A"
stand-alone Extended Data Ack to indicate acknowledge the missing un-acked datagram(s)
it has received from that peer before it sends out its first user
datagram.

In This is because the Gap Ack, TSN Seen field in the Part and Of fields are both set to '1', as opposed
to '0' first out-bound

Stewart, et al                                                  [Page 13]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999

user datagram can not be used as in a normal Ack. The data field of the Gap Ack TSN ack, instead it is a four (4)
octet long integer containing used to
carried the sequence number of peer's Tag. 


3.3 Datagram Processing during Tag Lock 

In Tag-lock or Tag-lock-new mode, an endpoint shall silently discard
any user datagrams from the next datagram
after peer endpoint that does not carried the Gap (which
correct Tag value.

However, if there is 10 in this example).  The Seen field a control part present in
the Gap Ack will contain the sequence number of the a discarded user
datagram of (i.e., C/D = 0x11), the
gap.  Using these two values, "A" should be able to calculate endpoint shall always process the
control part even when the missing datagram numbers (which is 9 in this
example) and thus determine which datagrams will need to be
retransmitted.

Note that Gap Acks cannot be piggy-backed with user data; if there is
user data to be sent when a gap part is detected, the Gap Ack must be being discarded.

If another Initiation from "A" is received by "Z" after "Z" sent out first before the datagram carrying user data can be sent.

4.3 Flow and Congestion Controls

Several different mechanisms shall be used jointly to achieve
flow and congestion controls in MDTP.

4.3.1 Sending with Window Control

The sending endpoint
its Initiation Ack, "Z" shall use a transmission window to control the
number of outstanding datagrams, i.e., datagrams that have been sent,
but yet respond to be acknowledged. The length of this second Initiation by
re-sending the window is defined as Initiation Ack if the
maximal number Tag Send field of outstanding datagrams a sending endpoint can
allow. This length is adjusted dynamically, depending on this second
Initiation has the current
number of successful transmissions as well same value as the number of lost
datagrams.

When the number that of outstanding datagrams reaches the current window
length, the endpoint original Initiation.
Otherwise, "Z" shall still accept send requests from respond by sending an Initiation of its upper
layer, but shall transmit no more datagrams until own, with
Tag Send field set to Tag_Z, so as to elicit an Initiation Ack is received.

Moreover, when the window length is reached, the next send request from
"A".


3.4 An Example of Association Initialization 

In the upper layer will trigger following example, "A" initiates the sending endpoint to transmit association first and then
sends a
special Window Up message. Upon receiving this Window Up (WIN|ACK) the
receiver must respond with a Window Up Response (WNR|ACK), as
illustrated by the following example (assuming current window length
is 3): user datagram to "Z", then "Z" sends two user datagrams
sometimes later:

Endpoint A                                          Endpoint Z
{App sends 3 messages}
[Header Flags=DAT|GAR|ACK
        Part=0,Of=1
        Seen=0,Send=11,Size=100]----------->

{app sets association with Z}
Initiation(C/D = 10)
   [Tag Seen=0,Tag Send=Tag_A
    & net addr info] --------\
(Start T2-recv T1-init timer)         \
(Enter Tag_A-lock mode)	       \---->Initiation Ack(C/D = 10)
                                       [Tag Seen=Tag_A,Tag Send=Tag_Z
                                /----   & net addr info]
                               /     (Enter Tag_Z-lock-new mode)
(Cancel T1-init timer)<-------/

{app sends 1st user data; strm 0}
U-Data(C/D = 01)
   [Seen=Tag_Z,Send=init TSN-A
    Strm=0,Seq=1,
    & user data]      -------\               
(Start T3-send timer)

[Header Flags=DAT|GAR|ACK
        Part=0,Of=1
        Seen=0,Send=12,Size=100]----------->
(Restart T3-send timer)

[Header Flags=DAT|GAR|ACK
        Part=0,Of=1
        Seen=0,Send=13,Size=100]----------->
(Restart         \
                               \---->(Leave Tag_Z-lock-new mode)
                               ------Ext Data Ack(C/D = 10)
                              /        [Gap=0,TSN Seen=init TSN-A]
(Cancel T3-send timer)

{App <-----/
..


Stewart, et al                                                  [Page 14]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999

                                     ..
                                     {app sends a new message}
(queue new message and send Win Up)
[Header Flags=WIN|ACK
        Seen=0,Send=14]--------------------> (cancel T2-recv timer)
                                      /----- [Header Flags=WNR|ACK
                                     /        Part=0,Of=0 2 datagrams;strm 0}
                               /---- U-data(C/D = 01)
                              /         Seen=14,Send=0]
[Header Flags=DAT|GAR|ACK <--------/
        Part=0,Of=1
        Seen=0,Send=15,Size=100]----------->        [Seen=Tag_A,Send=init TSN-Z
(Leave Tag_A-lock mode) <----/          Strm=0,Seq=1,
(Start T2-recv timer)
(Restart T3-send T2-receive timer)

In the above example,                & user data 1]
                               /---- U-data(C/D = 01)
                              /        [Seen=init TSN-A,
                             /          Send=init TSN-Z +1,
                        <---/           Strm=0,Seq=1,
                                        & user data 2]



If T1-init timer expires at "A" after the transmission of Initiation is sent, the first three
datagrams, "A" reached its window length. The next same
Initiation message from with the
user triggered a Window Up that was sent to "Z". The Window Up same Tag_A value shall
contain no user data. In response, "Z" cancelled be retransmitted and
the timer T2 restarted. This shall be repeated Max.Init.Retransmit times
before "A" considers "Z" unreachable and
immediately sent a Window Up Response. The arrival optionally reports the
failure.


3.5 Other Initiation Issues

3.5.1 Selection of this Window Up
Response effectively resolved all Tag Value

Tag values should be selected from the outstanding datagrams at "A",
thus allowed "A" range of 0x80000000 to send out
0xffffffff. It is very important that the next datagram.

4.3.2 Window Length Adjustment

The window length shall Tag value be initially set randomized to 2, and shall then be
dynamically adjusted based on
guard against "man in the datagram loss middle" and acknowledgment
conditions of the underlying network.

When 4 consecutive outstanding datagrams are acknowledged at once by
the receiver, the sender's window length will "sequence number" attacks. It is
suggested that RFC 1750 [9] be raised by 1 until it
reaches used for the protocol parameter 'Max.Outstanding.dg' (which should be Tag randomization.


3.5.2 Initiation from behind a
user configurable parameter).

If the current window length is less than 4, every time when the
number of consecutively outstanding datagrams acknowledged in NAT

When a single
Ack NAT is equal to or greater than present between two endpoints, the current window length, endpoint that is
behind the
sender's window length NAT, i.e., one that does not have a publicly available
network address, shall be raised by 1, until it reaches
'Max.Outstanding.dg'.

In take one of the following circumstances, the sender's window length shall be
decreased. However, when the window length reaches 2 options:

A) Indicate that it shall not be
decreased any further.

If between 1 to 3 consecutive datagrams are lost, the window length
will be decreased has only one network by 1. If between 4 to 7 datagrams are lost, setting the
window length will be decreased by 2. If 8 or more datagrams are lost, 'Number of
   networks' field in the window length will be decreased by 4.

Moreover, any time a Window Up is sent Initiation message to 0. This will make the receiving
   endpoint that receives this Initiation message to consider the
sender's window length will sender
   as only having that one address. This method can be decreased by 1. Also, if a timeout
forces used for a retransmission dynamic
   NAT, but any multihoming configuration at the sender's window length endpoint that is behind
   the NAT will not be reduced visible to half its peer, and thus not be taken
   advantage of.

B) Indicate all of its currently value.

The following table summarizes these rules:
- -----------------------------------------------------------------------
  Duplicate Ack received by sender  | Adjust down by 4
- -----------------------------------------------------------------------
  Greater than 8 datagrams lost     | Adjust down by 4
- -----------------------------------------------------------------------
  Greater than 4 datagrams lost     | Adjust down by 2
- -----------------------------------------------------------------------
  Greater than 0 datagrams lost     | Adjust down by 1
- -----------------------------------------------------------------------
  Timeout forces retransmission     | Adjust down by 1/2 of networks in the current
                                    | window.
- -----------------------------------------------------------------------
  Window Up sent                    | Adjust down by 1
- -----------------------------------------------------------------------
  4 or more consecutive datagrams   | Adjust up by 1
  acknowledged (window length > 4)  |
- -----------------------------------------------------------------------
  1/2 Window length or more acked   | Adjust up Initiation by 1
  (window length <=4)               |
- -----------------------------------------------------------------------

4.3.3 Flow Control using In-Queue Information

By using specifying all
   the In Queue field in actual IP addresses and ports that the MDTP header, NAT will substitute for the sender can inform
   endpoint. This method requires that the receiver endpoint behind the number NAT must
   have pre-knowledge of pending datagrams which all the sender has
received, but yet to deliver to its application. The following example
shows how IP addresses and ports that the NAT will
   assign.





Stewart, et al                                                  [Page 15]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999

3.5.3 Initialization Collision

If two endpoints use In Queue value attempt to accomplish Flow control.

Assume that Endpoint A has sent Endpoint Z 20 datagrams, and when
Endpoint Z sends initialize an Ack on the reception of these 20 datagrams, only
the first one of them has been delivered to the upper layer association with each other
at
Endpoint Z.

In about the Ack sent by Endpoint Z, same instance, a collision will occur. As a result, each
side will receive an Initiation datagram from the other side after it
transmitted its own. In Queue field would then have such a
value of 19, indicating the number of datagrams pending for delivery case, both sides shall send an
Initiation Ack datagram to the other side using the procedure
described above.


3.5.4 Association Re-initialization

An endpoint shall be allowed to re-initialize an established
association with the other endpoint.

Once an endpoint has left the Tag-lock or Tag-lock-new mode of the
previous association initialization process, it shall treat any new
Initiation message from its upper layer. This value would peer as a re-initialization event.

During a re-initialization, both endpoint shall follow the same
procedure as defined in section 3.1. And a new Init-Tag must be checked used
by Endpoint A before the endpoint that receives the Initiation message if it sent has already
left the next previous Tag-lock or Tag-lock-new mode.


4.  Transfer User Datagram

The receiver of a user datagram to Endpoint Z. If this value was found to be
greater than its current window length, Endpoint A would not send shall always acknowledge the
next datagram. Instead, Endpoint A would start its T3-send timer and
send a Window Up message reception
to Endpoint Z at the expiration sender of the datagram. Normally, delayed acknowledgment shall
be used. The delay shall be controlled by a T2-receive timer.
This would force Endpoint Z to send another Ack with an updated In
Queue value. If

At the new In Queue value was still greater than its
window length, Endpoint A would re-start its T3-send expiration of T2-receive timer, and repeat
this procedure until if there is out-bound user data,
the In Queue value of Endpoint Z dropped below ack should be piggy-backed on the current window length data part of Endpoint A.  Then, the transmission at
Endpoint A would resume.

4.3.4 T3-send Timer Adjustment with RTT

If out-bound user
datagram, occupying the RTT measurement is available on TSN Seen field (see section 2.3). Otherwise, a specific network, the sender
stand-alone Extended Data Ack shall adjust be used to carry the T3-send timer each time when sending datagram using
this network. The calculation and adjustment of the timer should
follow
acknowledgment.

When Extended Data Ack is used, the method described in [4]. RTT measurement sender shall be tracked
for each network if redundant networks are in use.

MDTP defines two optional methods fill the Last TSN
Seen field to obtain RTT measurements, see
sections 4.6 and 4.7.

4.4 Sequence Number Reset

When indicate the datagram sequence highest TSN Send number reaches it has received
from the value 0x7fffffff peer. Any detected gaps must also be reported 
(see section 4.5).

The following example illustrates both stand-alone and piggy-backed
acknowledgments:

Endpoint A                                      Endpoint Z
{App sends 3 messages in strm 0}
U-Data(C/D = 01)
   [Seen=5,Send=7,Strm=0,Seq=3]--------> (Start T2-receive timer)
(Start T3-send timer)

U-Data(C/D = 01)
   [Seen=5,Send=8,Strm=0,Seq=4]--------> 


Stewart, et al                                                  [Page 16]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999


U-Data(C/D = 01)
   [Seen=5,Send=9,Strm=0,Seq=5]--------> 
                                         ...
                                         {Timer T2 expires}
                              /--------- Extended Data Ack(C/D=10)  
                             /             [Gap=0,Seen=9]
(cancel T3-send timer) <----/
...
...
{App sends 1 message; strm 0}
U-Data(C/D = 01)
   [Seen=5,Send=10,Strm=0,Seq=6]-------> (Start T2-receive timer)
(Start T3-send timer)
                                         ...
                                         {App sends 1 message; strm 1}
                                         (cancel T2-receive timer)
                                 /------ U-Data (C/D=01)
                                /          [Seen=10,Send=6,Strm=1,Seq=2]
                               /         (Start T3-send timer)
(cancel T3-send timer) <------/
(Start T2-receive timer)
..
{Timer T2 Expires}
Extended Data Ack(C/D=10)
   [Gap=0,Seen=6]----------------------> (cancel T3-send timer)


4.1 Timer Management Rules

The the
next sequence number following rules shall be set used to 1.

4.5 Datagram Re-transmission

Whenever manage the timers during
normal datagram transfer, unless otherwise stated for some special
cases: 

A) When a T3-send user datagram is received, the endpoint shall start a
   T2-receive timer expires, if no T2-receive timer is currently running. Upon
   the expiration of the T2-receive timer, the endpoint shall re-transmit
   acknowledge to the sender all the un-acked datagram that user datagrams it has the lowest Send value, unless:

A) If the current window length is reached,
   received. 

B) When a Window Up message will
   be user datagram is sent out (see 4.3 Congestion Control), or

B) If out, the current window length is not reached and there is still
   user data pending for transmission, a new datagram with user data sending endpoint shall be sent out and start
   a T3-send timer shall be restarted.

When a if no T3-send timer is started at currently running. 

   If the T2-receive timer is running, the endpoint shall first stop
   the T2 timer, piggy-back an ack (or Extended Data Ack) unto the 
   out-bound datagram, and then start a re-transmission, T3-send timer. 

   If the length T3-send timer expires, the endpoint shall follow the rules
   described in 4.6 for possible re-transmission of the next un-acked
   datagrams.

   Moreover, whenever the T3-send timer for this destination should be doubled and is started the
last estimated RTT value estimate
   last calculated for that remote network address should be added to
   the timer.

4.5.1 Re-transmission on Redundant networks base T3-send timer value (see sections 6.2 and 6.3 for RTT). 

Stewart, et al                                                  [Page 17]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999


C) When redundant networks exist between two communicating endpoints, all outstanding datagrams are acknowledged, the
re-transmission T3-send timer
   shall be attempted on the network specified in the
MDTP protocol variable 'last.good.intf'. The value of 'last.good.intf' stopped if one is always updated to refer to the network on which the last datagram
from the peer still running.

D) If an endpoint arrived.

Moreover, the number of consecutive re-transmissions is also recorded
in a variable 'retran.count' for each network. Every time has a datagram
is received on T3-send timer running and receives a network, the corresponding 'retran.count' shall be
reset to 0.

If the value in the 'retran.count' partial
   acknowledgment (one that acknowledges some of the current network exceeds
half of outstanding
   datagrams) then the value of endpoint shall restart the protocol parameter 'Max.Retransmit', T3-send timer.

The following example shows the
'last.good.intf' will be changed, so as to force the next
re-transmission to be directed to an alternate network and
optionally report a failure condition.

The total number use of consecutive re-transmissions across all the
networks in an association is also recorded. various timers.

Endpoint A                                         Endpoint Z
{App sends 2 messages; strm 0}
U-Data (C/D=01)
  [Seen=5,Send=7,Strm=0,Seq=3] ---------> (Start T2-receive timer)
(Start T3-send timer)

U-Data (C/D=01)                           {App sends 1 message; strm 1}
  [Seen=5,Send=8,Strm=0,Seq=4] -\     /-- (cancel T2-receive timer)
                                 \   /    U-Data (C/D=01)
                                  \ /       [Seen=7,Send=6,Strm=1,Seq=2]
                                   \      (Start T3-send timer)
                                  / \
(Re-start T3-send timer) <-------/   \
(Start T2-receive timer)              \
...                                    -> (Start T2-receive timer)
...
{T2-receive timer expires}
Extended Data Ack(C/D=10)
  [Gap=0,Seen=6] -----------------------> (Cancel T3-send timer)
                                          ..
                                          {T2-receive timer expires}
(Cancel T3-send timer)  <---------------- Extended Data Ack(C/D=10)
                                            [Gap=0,Seen=8]

4.1.1 T3-send Timer Adjustment with RTT

If this value exceeds the
limit defined by 'Max.Retransmit', RTT measurement is available to a remote IP address, the sending endpoint sender
shall consider adjust the peer endpoint unreachable and stop transmitting data T3-send timer each time when sending datagrams to it,
that IP address. The calculation and
optionally report adjustment of the failure.

4.6 RTT Measurement

This defines timer should
follow the mechanism for round-trip-time (RTT) measurement method described in
MDTP.

On occasions either side of an association may need to perform an [4]. RTT measurement of the network (or one of shall be tracked
for each destination IP address if the redundant networks) between
them.

4.6.1 remote host is multi-homed.

MDTP defines three methods to obtain RTT Datagram Header Format

The following shows the header format measurements, see sections
4.7, 6.2, and 6.3.


4.2 Multihoming Rotation

4.2.1 Remote Multihoming Rotation

When an endpoint is transmitting to a remote multi-homed endpoint, the
transmitting endpoint shall use for RTT
measurement: rotate between destination IP addresses.
Every time the application transmits a datagram, MDTP Header Format - RTT measurement

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  MDTP Protocol Identifier                     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Version     |           Flags               |   In Queue    |
   |               |N N W I F R D A M S W R R F G U|               |
   |               |O O I S I T A C U H N E T L A N|               |
   |               |M B N B R M T K L U R 1 C O R R|               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                 Acknowledgment Number (Seen)                  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   Sequence Number (Send)                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        Data Size              |    Part       |      Of       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  Transparent Time Int-1                       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  Transparent Time Int-2                       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Two long integers are used in the data field to carry the time value.
The RTT datagram is identified by setting MUST keep track
of the RTC or RTM bit remote IP address to 1.

4.6.2 Measure RTT

AT which it sent the request of its upper layer, an endpoint shall initiate an RTT
measurement by sending an RTT datagram with GAR, ACK, and RTC bits set
to 1 (to a specific network if redundant networks exist). No
user data shall be carried. The sender shall also place in Time Int-1
and Time Int-2 the value of the current time of day MDTP

Stewart, et al                                                  [Page 18]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999

protocol variable 'last.sent.intf'. MDTP should rotate each send in seconds and
microseconds.

Upon the reception of this RTT datagram, a
round robin fashion amongst all available destination IP addresses on
the recipient shall
immediately return remote multi-homed host and should update the datagram protocol variable
'last.sent.intf' to indicate which destination IP address it last
used.

If possible, acks should be transmitted to the sender (over the same network
on IP address from
which the datagram arrives if redundant networks exist), with acked messages were received.  When acknowledging multiple
messages, this may not be possible.  In the
RTM and ACK bits set to 1.

Upon latter case, MDTP SHOULD
rotate the reception transmission of acknowledgments to all remote IP addresses.

The MDTP implementation MUST allow an application to override this reply, the sender shall use the Time Int-1
and Time Int-2 in
rotation by specifying the reply datagram destination IP address to calculate the RTT (of the
specific network if redundant networks exist).

Endpoint A                                      Endpoint Z
RTT - Request Now=x.y
[Header Flags=ACK|GAR|RTC
        Part=0,Of=1
        Seen=1,Send=31,Size=0
        Time-Int1=x
        Time-Int2=y] ----------------------->
                                      ------ [Header Flags=ACK|RTM
                                     /        Part=0,Of=0
                                    /         Seen=31,Send=1
                                   /          Time-Int1=x
                                  /           Time-Int2=y]
                                 /
(Endpoint A uses     <-----------
 current time subtracted from
 x.y which to calculate RTT)

4.7 Link Heart Beat

This defines the mechanism for activating send a
datagram.  The implementation must also provide an interface to add
and transmitting of link
heart beats remove a remote IP address from rotation eligibility.


4.2.2 Local Multihoming Rotation

As discussed in MDTP.

At request by its upper layer, section 3.3.4 of RFC 1122, an endpoint shall enable heart beat on MAY rotate
transmitted messages amongst all local network interfaces by
specifying the local IP address and UDP port or it may allow the
networking protocol to decide which local IP address (and network
interface) to use to transmit a specific peer with datagram..

If possible, acks should be transmitted from the same IP address over
which it has an established association in the
Reliable transfer mode. acked messages were received. When acknowledging multiple
messages, this may not be possible. In the latter case, MDTP SHOULD
rotate the transmission of acknowledgments from all configured IP
address/port pairs.


4.3 Stream Sequence Number

The RTT datagram defined in section 4.6.1 stream sequence number shall always be used as set to 1 when the Heart
Beat.

After having heart beat enabled,
stream is opened.

Also, when the endpoint stream sequence number reaches the value 0xffff the
next sequence number shall transmit a Heart
Beat be set to that specific peer 1. Sequence number '0' has
special meaning (see section 4.4) and start a T5-heartBeat timer. The peer shall immediately respond to the Heart Beat not be used in the same manner as an
RTT as described in section 4.6.  This response normal
sequence number rotation..


4.4 Ordered and Un-ordered Delivery

Normally, the receiver shall be stored by ensure the
first endpoint (also can user datagrams within any
given stream be used delivered to update its RTT measurement).

When the T5-heartBeat timer expires, upper layer according to the endpoint shall first check if order of
their stream sequence number. If there are datagram arrived out of
order of their stream sequence number, the previous heart beat has been responded (on receiver must hold the same network it was
sent in
received datagrams from delivery until they are re-ordered.

However, a sender can set the case stream sequence number of redundant network). If not, the network a user
datagram to 0, to indicate that the
last Heart Beat was sent upon no ordering shall be counted as a transmission
failure, and be handled following performed on that
datagram within that stream. Upon the rules described in section 4.5.
Then, reception of the endpoint shall send another Heart Beat and re-start datagram, the
T5-heartBeat timer.

In

Stewart, et al                                                  [Page 19]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999

receiver must by-pass the case where redundant networks exist, ordering mechanism and immediately delivery
the sending of Heart beats
shall follow datagram to the link rotation rules outlined upper layer.

This provides an effective way to transmit "out-of-band" data in section 4.1.1.

If, before any
given stream. Also, a stream can be used as an "un-ordered" stream by
simply setting the expiration stream sequence number of T5-heartBeat timer, a each out-bound user
datagram is
transmitted or received by to 0.


4.5 Report Missing Datagrams

MDTP uses a receiver-based retransmission policy, where the endpoint, sender
attempts to elicit from the T5-heartBeat timer shall
be stopped and receiver information on the appropriate T2-T4 timer shall be started. In other
words, missing
datagrams before the T5-heartBeat timer has retransmission.

If a receiver detects holes in the lowest precedence.

When no received user datagram to send and no other timers are running, the
T5-heartBeat timer sequence (by
examining TSN Send numbers), an Extended Data Ack with gap reports
shall be start and the above procedure shall
continue.

The suggested interval for T5-heartBeat timer is 4000 ms.

4.8 Advisory Acknowledgment

This defines sent back to inform the mechanism for sending and handling of sender so that the Advisory
Acknowledgments in MDTP.

An endpoint may use Advisory Acks to increase bandwidth utilization
when transmitting over a reliable association.

An Advisory Ack shall missing datagrams
can be re-transmitted.

Multiple gaps can be indicated by setting RE1 flag to 1 in one single Extended Data Ack.

If there is out-bound user data, the
datagram.

The endpoint shall send an Advisory piggy-back the
Extended Data Ack to its peer when it reaches
half of its current window length, and also when it detects that with the
next send will reach user data in the full window length.

Upon same MDTP datagram, by
setting the reception of an Advisory Ack, C/D bits to '11'. And the peer endpoint TSN Seen field in the data part
shall
immediately acknowledge all not be used, i.e., the datagrams it has received but yet
acked upon, sender shall set the field to 0x0 and then cancel the T2-recv timer if one is still
running.
receiver shall ignore it.

The following shows an example shows the use of using Advisory Ack: gap report in an Extended Data
Ack. 

Endpoint A                                    Endpoint Z
{App sends 3 messages}
[Header Flags=DAT|GAR|ACK
        Part=0,Of=1
        Seen=0,Send=1,Size=100]-------------> messages; strm 0}
U-Data (C/D=01)
   [Seen=3,Send=6,Strm=0,Seq=2]-------> (Start T2-recv T2-receive timer)
(Start T3-send timer)

[Header Flags=DAT|GAR|ACK
        Part=0,Of=1
        Seen=0,Send=2,Size=100]----------->
(Restart T3-send timer)
{detects window half full, use Advisory Ack}
[Header Flags=DAT|GAR|ACK|RE1
        Part=0,Of=1
        Seen=0,Send=3,Size=100]------\
(Stop and restart T3-send timer)      \
                                       \----> (cancel T2-receive timer)
                      <---------------------- [Header Flags=ACK
                                               Part=0,Of=0
                                               Seen=3,Send=1]

4.9 Termination of an Association

When an endpoint terminates, it shall send a Shutdown datagram
(FIR|SHU) to each of the peer endpoints

U-Data (C/D=01)
   [Seen=3,Send=7,Strm=0,Seq=3]-----X (lost)

U-Data (C/D=01)
   [Seen=3,Send=8,Strm=0,Seq=4]-------> (A gap detected in all its existing
associations.  The Shutdown data)
                                        ..
                                        {T2-receive timer expires} 
                                /------ Extended Data Ack (C/D = 10)
                               /          [Gap=1,Strt=7,End=7,Seen=8]
(Prepare retransmission) <----/


In this example, when "Z" receives the third datagram itself is sent from "A" it
realizes that a gap exists in unreliable
transfer mode and thus needs not to be acknowledged.

When the received data. At the expiration of
T2-receive timer, "Z" sends an Extended Data Ack with a peer endpoint receives gap report to
"A" to indicate the Shutdown, it will remove missing datagram. Note that the sender
from its record, Start and optionally End
fields in the gap report specify the termination edges of the sender gap, i.e., the TSN

Stewart, et al                                                  [Page 20]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999

numbers between Start and End are missing.

When the peer endpoint is multi-homed, the Extended Data Ack should be
sent out to the upper layer.

The following shows an example of destination IP address specified in the termination MDTP protocol
variable 'last.good.intf'. The value of Endpoint A:

Endpoint A
{App indicates termination}
[Header Flags=FIR|SHU
        Seen=3,Send=14,    ------------------------> to Endpoint X

[Header Flags=FIR|SHU
        Seen=1496,Send=101,------------------------> to Endpoint Y

[Header Flags=FIR|SHU
        Seen=14,Send=2    -------------------------> 'last.good.intf' is always
updated to Endpoint Z

4.10 Draining of an Association

An endpoint in a association may decide point to "drain" the association
without completely shutting it down. By draining an association, both
endpoints will remove any record and pending datagrams associated with the association.  Further communications between source IP address from which the two endpoints can
be resumed by going through a re-initialization procedure (see
section 3).

In such a case, a Drain last datagram (FIR|SHU|UNR) is sent to
from the peer endpoint arrived.


4.6 Range Check on TSN

For security reasons, the receiver must check the range of the association, and no Ack TSN
Send value in each received user datagrams.

Assume that the highest TSN received from a peer is required.

The following sequence shows an example T and the maximal
window length of Draining:

Endpoint A
{App indicates draining}
[Header Flags=FIR|SHU|UNR
        Seen=146,Send=1301]------------------------> to Endpoint X

5. Interface with upper level protocols

The upper layer protocols (ULP) the same peer is W (exchanged during association
initiation, see section 3.1). When the next user datagram arrives from
this peer, the receiver shall request for services by passing
primitives silently discard the datagram if the TSN
Send value carried in the datagram is greater than T+W (calculation
rounds up at 0x7fffffff to MDTP and shall receive notifications from MDTP for
various events. 0x1).


4.7 Advisory Ack Request

An endpoint may use Advisory Ack Requests to improve bandwidth
utilization.

The primitives and notifications described in this section endpoint should be
used as a guideline for implementing MDTP.

A) Init.MDTP primitive

This primitive allows MDTP send an Advisory Ack Request to initialize its internal data structures
and allocate necessary resources for setting up peer when it
reaches half of its operation
environment. Note current window length, and also when it detects
that once MDTP is initialized, ULP can communicate
directly with any other endpoints without re-invoking this primitive.

Mandatory attributes:

None.

Optional attributes:

The following types of attributes may be passed along with the primitive:

 o Timer selection and its operation syntax -- to indicate to MDTP
   an alternative timer next send will reach the MDTP should use full window length (see section 5.1
for its operation.
 o Initial MDTP operation mode;
 o IP port number, if ULP wants window control).

Upon the reception of an Advisory Ack Request, when it to be specified;

B) Send.Data primitive

This is not under
flow control condition the main method to send datagrams via MDTP.

Mandatory attributes:

 o data - This is peer endpoint should immediately
acknowledge all the payload ULP wants to transmit;
 o size - The size of the payload in number of octets;
 o to-address - The IP address datagrams it has received but not yet
acknowledged, and port number of the intended
   receiver. In case of redundant networks, to-address can be any one
   of the multiple IP addresses of the receiver. The network which the
   datagram will actually be sent through will be determined by MDTP due
   to the link rotation, unless the current mode prohibits MDTP link
   rotation; in such case the datagram will be sent through the network
   specified by to-address (see section 4.5).

Optional attributes:

 o mode-flags - This indicates a new MDTP operation mode, taking effect
   immediately including the current datagram send;

 o context - optional information that will be carried in the
   Send.Failure notification to the ULP if the transportation of
   this datagram fails.

C) Receive.Data primitive

This primitive shall return the first datagram in then cancel the MDTP in-queue to
ULP, T2-receive timer if there is one available. It may, depending on the specific
implementation, also return other informations such as the sender's
address, whether there are more datagrams available for retrieval,
etc. The behavior is undefined if no datagram is available when this
primitive is invoked.

Mandatory attributes:

 o buffer - the memory location indicated by the ULP to store still
running. Otherwise, the
   received datagram and other information.

Optional attributes:

   None.

D) Data.Arrive notification

MDTP peer endpoint shall invoke this notification on the ULP when a datagram is
successfully received take no action and ready for retrieval.

E) Send.Failure notification

If a datagram can not be delivered MDTP shall invoke this notification
on ignore
the ULP.

The following may be optionally passed with the notification:

 o data - the location ULP can find the un-delivered datagram.
 o context - optional information associated with this datagram (see
   13.2).

F) Link.Status.Change notification

When a link is marked down (e.g., when MDTP detects a link failure),
or marked up (e.g., when MDTP detects a link recovery), MDTP shall
invoke this notification on the ULP.

The following shall be passed with the notification:

 o link-address - This indicates the IP address of the affected link;
 o new-status - This indicates the new status of the link;

G) Communication.Up notification

This notification is used when MDTP becomes ready to send or receive
datagrams, or when a lost communication to an endpoint is restored.

The following shall be passed with the notification:

 o status - This indicates what type of event that has occurred;
 o endpoint-id - The IP address and port number to identify the
   endpoint;

H) Communication.Lost notification

When MDTP loses communication to an endpoint completely or detects
that the endpoint has performed a shut-down operation, it shall invoke
this notification on the ULP.

The following shall be passed with the notification:

 o status - This indicates what type of event that has occurred;
 o endpoint-id - The IP address and port number to identify the
   endpoint;
 o packets-enqueue - The number and location of un-sent datagrams
   still holding by MDTP;
 o last-acked - the sequence number last acked by that peer endpoint;
 o last-sent - the sequence number last sent to that peer endpoint;

I) Change.Link.Rotation primitive

When the upper layer wants to inform MDTP to make a specific network
eligible or ineligible for in link rotation, the upper layer will send
this primitive to MDTP.

Mandatory attributes:

 o  action - This indicates if the network is to be made eligible or
             ineligible for link rotation.
 o  network-id - This is the IP address and port of the network to be
    added or removed from link rotation consideration.

J) Open.Stream primitive

This shall be used by the upper layer to open a new stream.

Mandatory attributes:

 o endpoint-id - The IP address and port number to identify the
   peer endpoint to which the stream is to be opened. An association
   must have existed at the time of stream open.

Returned attributes:

 o The stream number that is opened.

K) Close.Stream primitive

This shall be used by the upper layer to request to close a stream.

Mandatory attributes:

 o endpoint-id - The IP address and port number to identify the
   peer endpoint to which the stream is to be closed.

 o stream number - The stream number to identify the stream to be
   closed (this should be the number returned by the Stream.Open
   primitive on this stream).

6. Suggested MDTP Protocol Parameter Values

The following are suggested timer values for MDTP:

T1-init Timer    -  160 ms
T2-receive Timer -   20 ms
T3-send Timer    -  160 ms + Last calculated RTT for that network. Advisory Ack Request.

The following protocol parameters are recommended:

Max.Outstanding.dg      - 20 messages
Max.Retransmit          - 10 attempts
Max.Init.Retransmit     - 8  attempts
Min.Mcast.Time.To.Reset - 5 seconds
Num.Of.Mcast.Reset.Msg  - 5 messages

7. Acknowledgments

The authors wish to thank Brian Wyld, A. Sankar, Henry Houh, Gary
Lehecka, Ken Morneault, Lyndon Ong, and others for their very valuable
comments.

8.  Author's Addresses

Randall R. Stewart                          Tel: +1-847-632-7438
Cellular Infrastructure Group               EMail: stewrtrs@cig.mot.com
Motorola, Inc.
1475 W. Shure Drive, #2C-6
Arlington Heights, IL 60004
USA

Qiaobing Xie                                Tel: +1-847-632-3028
Cellular Infrastructure Group               EMail: xieqb@cig.mot.com
Motorola, Inc.
1501 W. Shure Drive, #2309
Arlington Heights, IL 60004
USA

Tom Bova                                    Tel: +1-703-484-3331
Cisco Systems Inc.                          EMail: tbova@cisco.com
13615 Dulles Technology Drive
Herndon, VA  20171

Suheel Hussain                              Tel: +1-919-472-2312
Cisco Systems Inc.                          EMail:ssh@cisco.com
7025 Kit Creek Road
Research Triangle Park, NC  27709

Ted Krivoruchka                             Tel: +1-703-484-3331
Cisco Systems Inc.                          EMail: tedk@cisco.com
13615 Dulles Technology Drive
Herndon, VA  20171

Renee Revis                                 Tel: +1-703-472-5681
Cisco Systems Inc.                          EMail: drrevis@cisco.com
7025 Kit Creek Road
Research Triangle Park, NC  27709

9. References

[1] Postel, J. (ed.), "Internet Protocol - DARPA shows an example of using Advisory Ack Request:

Endpoint A                                      Endpoint Z
{App sends 3 messages; strm 0}
U-Data(C/D = 01)
   [Seen=5,Send=7,Strm=0,Seq=3]-------------> (Start T2-recv timer)
(Start T3-send timer)

U-Data(C/D = 01)
   [Seen=5,Send=8,Strm=0,Seq=4]----------->






Stewart, et al                                                  [Page 21]

Internet Program
Protocol Specification", RFC 791, USC/Information Sciences Institute,
September 1981.

[2] Postel, J., "User Draft   Multi-network Datagram Protocol", RFC 768, USC/Information Sciences
Institute, August 1980.

[3] Postel, J. (ed.), "Transmission Control Protocol", RFC 793, USC/
Information Sciences Institute, September 1981.

[4] Jacobson V., "Congestion Avoidance and Control", Proceedings of
SIGCOMM '88, pp 314-329, August, 1988.

[5] Seth, T., etc. "Performance Requirements Transmission Protocol   June 1999


{detects window half full, use Advisory Ack Req}
Adv Ack Request(C/D=11)
   [Seen=5,Send=9,Strm=0,Seq=5]------\
                                      \
                                       \----> (cancel T2-receive timer)
                            <---------------- Extended Data Ack(C/D=10)
                                                 [Gap=0,Seen=9]

An endpoint sending an Advisory Ack Request may also use this request
for Signaling in Internet
Telephony", Internet-Draft <draft-seth-sigtran-req-00.txt>, May, 1999.

Appendix A: Stream-based Reliable its RTT calculation. The sending endpoint may note the time mark
when sending the datagram with the Advisory Ack Request.  When the
peer endpoint responds with an Extended Data Ack, the sender of the
Advisory Ack Request may use the time mark of the arriving Extend Data
Ack and Ordered Delivery

This defines a reliable the stored time mark to calculate the RTT as defined in
[4]. However, the sender of the Advisory Ack Request shall abandon the
RTT calculation if more datagrams are sent to its peer and ordered stream mechanism for MDTP. It no Extended
Data Ack is
optional for implementation.

A stream received.


5   Congestion Controls

Several different mechanisms shall be used jointly to achieve
congestion control in MDTP MDTP. These mechanisms are always used in regard
to the association, not a individual stream.


5.1 Send with Window Control

The sending endpoint shall use a transmission window to control the
number of outstanding datagrams, i.e., datagrams that have been sent,
but yet to be acknowledged. The length of the window is defined as the
maximal number of outstanding datagrams a sequence sending endpoint can
allow. This length is adjusted dynamically, depending on the current
number of successful transmissions as well as the number of lost
datagrams or retransmissions.

When the number of outstanding datagrams reaches the current window
length, the endpoint shall still accept send requests from its upper
layer, but shall transmit no more datagrams until some or all of user the
outstanding datagrams that needs are acknowledged. The endpoint may also elect
to be reliably delivered with sequence preservation queue only a specified number of its own. In
other words, datagram when the delivery window is full.
When this maximal number of a stream queued datagrams is reached the endpoint
shall not return an error to its upper layer.

Moreover, when the window length is reached, the next send request
from the upper layer will trigger a Window-up message to be delayed because of
transmitted. Upon receiving this Window-up the losses or re-transmissions occurred in other streams within receiver must respond
with a Window-up Ack, as illustrated by the
same MDTP association. This capability following example
(assuming current window length is 3):





Stewart, et al                                                  [Page 22]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999


Endpoint A                                      Endpoint Z
{App sends 3 messages, strm 0}
U-Data(C/D = 01)
   [Seen=5,Send=7,Strm=0,Seq=3]--------> (Start T2-receive timer)
(Start T3-send timer)

U-Data(C/D = 01)
   [Seen=5,Send=8,Strm=0,Seq=4]--------> 

U-Data(C/D = 01)
   [Seen=5,Send=9,Strm=0,Seq=5]--------> 

{App sends a critical requirement new message, strm 1}
(queue new message and send Win-up)
Window-up(C/D = 10)     ---------------> (cancel T2-recv timer)
                                   /---- Window-up Ack(C/D = 10)
                                  /         [Gap=0,Seen=9]
(Cancel T3-send timer)  <--------/
U-Data(C/D = 01) 
   [Seen=5,Send=10,Strm=1,Seq=2]-------> (Start T2-receive timer)
(Start T3-send timer)

In the above example, after the transmission of
some telephony call signaling protocols [5].

Stream the first three
datagrams, "A" reached its window length. The next message from the
user triggered a Window-up that was sent to "Z". The Window-up shall
contain no user data. In response, "Z" cancelled timer T2 and
immediately sent a Window-up Ack. The arrival of this Window-up Ack
effectively resolved all the outstanding datagrams are identified by setting FLO bit at "A", thus
allowing "A" to 1.

A.1 Stream Initiation

First, an MDTP association between send out the two endpoints must be initiated
before any stream operation.

A stream next datagram.


5.1.1 Window Length Adjustment

The window length shall be initiated (opened) by the sender before datagrams
can initially set to 2, and shall then be sent in the stream,
dynamically adjusted based on datagram loss and after acknowledgment.

If the stream current window length is complete it shall
be terminated (closed) by less than or equal to 4, every time
the user. Also, both sides number of the
association shall be able consecutive outstanding datagrams acknowledged in a
single ack is equal to initiate or terminate streams
independently.

The sender initiates a stream by sending a Stream Initiation
(NOB|UNR), using the following header format:

                          Stream Initiation

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                 MDTP Protocol Identifier                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Version     |              Flags            |   In Queue    |
   |               |N N W I F R D A M S W R R F G U|               |
   |               |O O I S I T A C U H N E T L A N|               |
   |               |M B N B R M T K L U R 1 C O R R|               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         Seen = 0x0 (or Tag)                   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           Send = 0x0                          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        Data Size              |    Part       |      Of       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        New Stream Number      |              0x0              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Note that in greater than half the Stream Initiation, current window length,
the Seen and Send sender's window length shall be set to 0,
and raised by 1, until it reaches
'Max.Outstanding.dg'(which should be a user configurable parameter).

If the current window length is greater than 4, every time the number
of consecutive outstanding datagrams acknowledged in a single ack is
equal to or greater than 4, the new stream being initiated sender's window length shall be indicated
in raised
by 1, until it reaches 'Max.Outstanding.dg'.

In the first two octets of following circumstances, the data field. sender's window length shall be
decreased. However, if this is when the first window length reaches 2 it shall not be
decreased any further.

The peer endpoint may report reception gaps which may correspond to
multiple datagram sent out after receiving the
Initiation losses (indicated by an Extended Data Ack from the peer (see section 3.1), or

Stewart, et al                                                  [Page 23]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999

Window-up Ack). If between 1 to 3 datagrams are lost, the Seen field of
above Stream Initiation window
length shall be set decreased by 1. If between 4 to 7 datagrams are lost,
the Tag value carried in the
Initiation Ack.

Upon window length shall be decreased by 2. If 8 or more datagrams are
lost, the reception of window length shall be decreased by 4.

Any time a Window Up is sent to the Stream Initiation, receiving endpoint the peer sender's
window length shall respond
immediately with be decreased by 1. Also, if a Stream Initiation Ack (NOB|UNR|ACK), using timeout forces a
retransmission the sender's window length shall be reduced to half of
its currently value.

The following header format:

                        Stream Initiation Ack

    0                   1                   2                   3
    0 1 2 3 table summarizes these rules:

-----------------------------------------------------------------
  duplicate ack received by sender  | Adjust down by 4 5 6 7 		 
-----------------------------------------------------------------
  8 9 0 1 2 3 or more datagrams lost          | Adjust down by 4 5 6 7 8 9 0 1 2 3		 
-----------------------------------------------------------------
  4 5 6 to 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                 MDTP Protocol Identifier                      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Version     |              Flags            |   In Queue    |
   |               |N N W I F R D A M S W R R F G U|               |
   |               |O O I S I T A C U H N E T L A N|               |
   |               |M B N B R M T K L U R 1 C O R R| datagrams lost             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Adjust down by 2		 
-----------------------------------------------------------------
  1 to 3 datagrams lost             |                      Seen = Stream Number Adjust down by 1		 
-----------------------------------------------------------------
  Timeout forced retransmission     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Adjust down by 1/2 of the 
                                    |                           Send = 0x0 current window.		 
-----------------------------------------------------------------
  Window up sent                    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Adjust down by 1		 
-----------------------------------------------------------------
  4 or more consecutive datagrams   |        Data Size Adjust up by 1		 
  acknowledged (window length > 4)  |    Part				 
-----------------------------------------------------------------
  1/2 Window length or more acked   |      Of Adjust up by 1		 
  (window length <=4)               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The following example shows				 
-----------------------------------------------------------------


5.2 Send Timer Back-off at Re-transmission

Whenever a T3-send timer expires, the endpoint shall re-transmit the
un-acked datagram that has the highest TSN Send value in that and
re-start the opening of stream 5 by "A":

Endpoint A                                      Endpoint Z
{App Initiates stream 5}
[Header Flags=FLO|UNR
        Part=0,Of=1
        Seen=0,Send=0,Size=0,
        Stream=5 ]--------------------------->
(Start T3-send timer)
(Cancel timer, unless:

A) If the current window length is reached, a Window-up message shall
   be sent out (see section 5.1), or

B) If the current window length is not reached and there is still user
   data pending for transmission, a new datagram with user data shall
   be sent out and T3-send timer) <--------------------- [Header Flags=FLO|UNR|ACK
                                               Mode=UNR
                                               Part=0,Of=1
                                               Seen=5,Send=0]

A.2 Stream Termination

For an existing stream, either side timer shall be allowed to terminate restarted. 

When the
stream by sending T3-send timer is re-started at a Stream Termination (FLO|UNR|SHU) to retransmission, the other side.

Besides flag RES, The Stream Termination length
of the timer shall use be doubled from its previous value. Also, the same header
format as
latest estimated RTT value for that used in Stream Initiation datagram (see A.2)

A Stream Termination Ack (FLO|UNR|SHU|ACK) shall network should be sent by added to the peer
endpoint in response. new
timer value. The following example shows the termination calculation of stream 5 by "A":

Endpoint A                                      Endpoint Z
{App terminates stream 5}
[Header Flags=FLO|UNR|SHU
        Part=0,Of=1
        Seen=0,Send=0,Size=0,
        Stream=5 ]--------------------->
(Start T3-send timer s-5)
(Cancel timer
value, where 'TL3-default' is a configurable protocol parameter.



Stewart, et al                                                  [Page 24]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999

<at normal transmission>
  1. TL3-value = 'TL3-default'
  2. T3-send = TL3-value + RTT

<at re-transmissions>
  1. TL3-value = TL3-value * 2
  2. T3-send = TL3-value + RTT

The T3-send timer s-5) <------------ [Header Flags=FLO|UNR|SHU|ACK
                                          Part=0,Of=1
                                          Seen=5,Send=0]

Datagrams associated at the sender shall be restored to its default value
when a terminated stream received by either side
should be silently discarded. It datagram is up to the side which terminates received from the stream peer endpoint.

The total number of consecutive re-transmissions to assure that all outstanding user datagrams destination IP
addresses in an association shall be recorded. If this value exceeds
the stream
are acknowledged before the termination.

A.3 Stream Datagram Transfer

A.3.1 Header Format in Stream Datagrams with User Data

The MDTP header limit defined in a stream datagram with user data 'Max.Retransmit', the sending endpoint shall have
consider the
following format:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  MDTP Protocol Identifier                     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Version     |              Flags            |   In Queue    |
   |               |N N W I F R D A M S W R R F G U|               |
   |               |O O I S I T A C U H N E T L A N|               |
   |               |M B N B R M T K L U R 1 C O R R|               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             Seen                              |
   |         Stream Number         |    Sequence Number            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             Send                              |
   |         Stream Number         |    Sequence Number            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        Data Size              |    Part       |      Of       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   \                                                               \
   / peer endpoint unreachable and shall stop transmitting any
more data                              /
   \                                                               \
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ to it. The stream number and sequence sending endpoint MAY report the failure to the
upper layer, including all datagrams in its out-bound buffer which
have not been acknowledged. Whenever a datagram is received from a
peer endpoint the total number of retransmissions shall be cleared.


6.  Network Management


6.1 Failure Detection in Redundant Networks

When the Send field shall peer endpoint is multi-homed, the re-transmission of a
datagram should be used
by attempted to the sender destination IP address specified
in the MDTP protocol variable 'last.good.intf'. The value of
'last.good.intf' is always updated to point to identify the current stream datagram. And, source IP address
from which the
stream number and sequence last datagram from the peer endpoint arrived.

The number of consecutive T3-send timeout events is also recorded in
a variable 'retran.count' for each destination IP address. This count
should be incremented when a T3-send time-out event occurs for that
destination IP address. Every time a datagram is received from a peer
endpoint, the Seen field receiving endpoint shall be used
by reset to 0 the sender 'retran.count'
corresponding to acknowledgment the source IP address .

If the value in 'retran.count' exceeds half of stream datagrams it has received.

Stream number 0 the value of the
protocol parameter 'Max.Retransmit', the destination IP address shall
be reported to the upper layer as out-of-service and sequence number 0 are reserved shall be removed
from eligibility for special
purposes rotation.  When re-transmitting a datagram, the
re-transmission should use 'last.good.intf' as the preferred
destination IP address to which to re-transmit, unless 'last.good.intf'
points to the destination IP address on which the original T3-send
time-out event occurred.

In the event that a datagram is received from an IP address that has
been reported as out-of-service, the 'retran.count' shall be cleared
as specified above, the destination IP address shall be reported as
in-service to the upper layer, and are not the destination IP address shall be
considered valid stream number or sequence number.

A.3.2 for rotation.


Stewart, et al                                                  [Page 25]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999


6.2 RTT Measurement

On occasions an endpoint of Stream Datagrams

The rules an association may need to perform an RTT
measurement of using the Seen Sequence Number and Send Sequence Number
are similar to those defined for normal MDTP non-stream datagram
transmissions (see section 4), except that for stream transfer network (or one of the
sequence numbers shall roll-over to 1 after 0xFFFF.

Moreover, each stream maintains redundant networks) between
itself and its individual T3-send timer, but only
one global T2-receive timer is maintained for all existing streams.

Acknowledgment to a stream datagram peer.

RTT-request and RTT-ack messages shall either be sent separately
or be piggy-backed with a stream datagram (not necessarily belonging used to perform the same stream) traveling in RTT
measurement. In the opposite direction. For a
separate Stream Ack, messages, two 32 bit long opaque integers are used
in the Send control parameter field will be set to 0000:0000.

The following shows an example carry the time value.

At the request of transmitting a stream datagram
(FLO|REL|DAT) and its upper layer, an endpoint shall initiate an RTT
measurement by sending an RTT-request (to a separate Stream Ack (FLO|REL|ACK):

Endpoint A                                      Endpoint Z
{App sends first data on stream 5}
[Header Flags=FLO|REL|DAT
        Part=0,Of=1
        Seen=0-0,Send=5-1,Size=20]----\
(Start T3-send timer-s5)               \--->(Start T2-recv)
                                            ...
                                            {T2-recv Timer Expires}
(Cancel T3-send timer-s5)   <--------------- [Header Flags=FLO|REL|ACK
                                             Part=0,Of=1
                                             Seen=5-1,Send=0-0,Size=0] specific network if
redundant networks exist). The following example shows sender shall also place in Time value 1
and Time value 2 the use value of a piggy-backed Stream Ack.

{App sends new data on stream 5}
[Header Flags=FLO|REL|DAT
        Part=0,Of=1
        Seen=0-0,Send=5-4,Size=20]--------->(Start T2-recv)
(Start T3-send timer-s5)                    ...
                                            {App sends data on stream 11}
                                            (cancel T2-recv Timer)
                                     /----- [Header Flags=FLO|REL|DAT|ACK
                                    /        Part=0,Of=1
                                   /         Seen=5-4,Send=11-8,Size=10]
                                  /         (Start T3-send timer-s11)
(Cancel T3-send timer-s5)  <-----/
(Start T2-recv timer)
...
{T2-recv Timer Expires}
[Header Flags=FLO|REL|ACK
        Part=0,Of=1
        Seen=11-8,Send=0-0,Size=0]--------->(Cancel T3-send-s11)

Note that when piggy-back a Stream Ack with an out-bound stream
datagram when more than one streams have un-acked datagrams, the
endpoint current time mark.

Upon the reception of this RTT-request message, the recipient shall choose one stream and piggy-back
immediately respond with a Stream Ack RTT-ack to the sender (over the same
network on one of which the datagrams, and shall leave RTT-request arrives if the T2-recv timer running.

A.3.3 Extended Stream Ack recipient is
multi-homed), with the time mark carried in the original RTT-request
copied into its own Time value fields.

Upon the expiration reception of T2-recv timer, if there are more than one
stream datagrams received but yet acked upon by this reply, the endpoint, an
Extended Stream Ack sender shall be used.

The following defines use the header format of time mark
in the Extended Stream Ack
that acknowledges N stream datagrams received:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  MDTP Protocol Identifier                     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Version     |              Flags            |   In Queue    |
   |               |N N W I F R D A M S W R R F G U|               |
   |               |O O I S I T A C U H N E T L reply RTT-ack to calculate the RTT (to the specific destination
IP address if redundant networks exist) as defined in [4].

Endpoint A N|               |
   |               |M B N B R M T K L U R 1 C O R R|               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             Seen                              |
   |         Stream Number #0      |    Sequence Number #0         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  Number of Extra Acks = N-1                   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        Data Size              |    Part       |      Of       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |         Stream Number #1      |    Sequence Number #1         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   \                                      Endpoint Z
{RTT - Request Now=x.y}
RTT-request (C/D=10)
   [Time-value1=x,
    Time-value2=y,
    Seen=81]       ----------------------->
                                    /------- RTT-ack (C/D=10)
                                   /            [Time-value1=x,
                                  /                                                               \
   \              Time-value2=y,
                                 /
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |         Stream Number #N-1    |    Sequence Number #N-1       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Note that an Extended Stream Ack is identified by setting the Seen
field               Seen=3]
(Endpoint A uses     <----------/
 x.y to calculate RTT)


6.3 Network Heart Beat 


At the number request of extra acks carried in its data field, upper layer, an endpoint shall enable heart beat
to a specific peer with which it has an established association.

The RTT-request message defined in section 2.2 shall be used as shown
above. Also, Extended Stream Acks
the heart beat while the RTT-ack shall not be piggy-backed.

The following example shows used as the using of an Extended Stream Ack
(NOB|REL|ACK) by "Z":

Endpoint A                                      Endpoint Z
{App sends data on stream 5}
[Header Flags=FLO|REL|DAT
        Part=0,Of=1
        Seen=0-0,Send=5-2,Size=20]----------> (Start T2-recv)
(Start T3-send timer-s5)
{App sends data on stream 9}
[Header Flags=FLO|REL|DAT
        Part=0,Of=1
        Seen=0-0,Send=9-4,Size=20]---------->
(Start T3-send timer-s9)
{App sends more data on stream 5}
[Header Flags=FLO|REL|DAT
        Part=0,Of=1
        Seen=0-0,Send=5-3,Size=20]---------->
(Restart T3-send timer-s5)
{App sends data on stream 7}
[Header Flags=FLO|REL|DAT
        Part=0,Of=1
        Seen=0-0,Send=7-11,Size=20]--------->
(Start T3-send timer-s7)
                                              ...
                                              {T2-recv Timer Expires}
(Cancel T3-send timer-s5)     <-------------- [Header Flags=FLO|REL|ACK
(Cancel T3-send timer-s7)                      Part=0,Of=1
(Cancel T3-send timer-s9)                      Seen=5-3,NumExtAck=2,
                                               Size=0,
                                               ext[0]=9-4,
                                               ext[1]=7-11]

A.4 Other Issues with Stream Transfer

- -- Congestion control, including heart beat
response.

After having heart beat enabled, the rules for timer management endpoint shall transmit a heart
beat to that specific peer and window
management, start a T5-heartBeat timer. The peer
shall apply immediately respond to Stream Transfer the heart beat in the same way as it does to
non-Stream based transfer, manner as defined the
RTT measurement procedure described in section 4.3.

- -- When an association is re-initialized (see section 3.4), all existing
stream within that association will 6.2. This response, as
well as the new RTT measurement, shall be automatically terminated.

- -- The receiver stored by the endpoint.

Stewart, et al                                                  [Page 26]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999


When the T5-heartBeat timer expires, the endpoint shall silently discard any datagrams associated
with a stream which has not been initiated or first check if
the previous heart beat has already been
terminated.

- -- The responded to (on the same re-transmission network it
was sent in the case of multi-homed hosts). If not, the destination IP
address to which the last heart beat was sent shall have the
'retran.count' incremented and link rotation checked following the rules as defined described
in section 4 6.1. Then, the endpoint shall apply to Stream Transfer.

- -- Bundled Message (see Appendix B) may be allowed in Stream Transfer,
but fragmentation (see Appendix C) send another heart beat and
re-start the T5-heartBeat timer.

In the case where one or both endpoints are multi-homed, the sending of
Heart beats shall not be allowed.

Appendix B: Bundled Message Transfer

This defines follow the mechanism for bundled datagram transport network rotation rules outlined in MDTP. It
is optional for implementation.

Bundling
section 4.2.

If, before the expiration of T5-heartBeat timer, a datagram is sometimes desired
received by the user when transferring small
datagrams, as a way of improving network utilization. endpoint, the T5-heartBeat timer shall be stopped and
the appropriate T2-receive timer shall be started. In bundled transfer, MDTP allows an endpoint other words, the
T5-heartBeat timer has the lower precedence than the T2-receive timer.

When there are no datagrams to bundle small
application messages into one single datagram for transmission. This
bundled mode can send and no other timers are running,
the T5-heartBeat timer shall be applied to both reliable started and unreliable datagrams
(see Appendix E the above procedure shall
continue.

The suggested interval for Unreliable Delivery).

Note that T5-heartBeat timer is 4000 ms, and may be
dynamically adjusted by adding the current RTT measurement if it is
available.


7.  Termination of Association

Before an endpoint terminates itself, it shall never send bundled messages an Abort message
to a each of its peer if
that endpoints in all existing associations. The Abort
shall be sent without requiring an acknowledgment from the peer
endpoint. However, the sender of the Abort message MUST fill in the
peer's Init-Tag.

When the peer endpoint set NOB bit to 1 during their association
initialization (see section 3).

B.1 Format receives the Abort, after verifying the Tag,
the peer shall remove the sender from its record, and optionally
report the termination of Bundled Datagram

The ISB bit in the flag field is set sender to indicate its upper layer. However if
the current datagram Tag sent with the Abort message is bundled, i.e., it contains multiple messages. incorrect, the peer must
silently discard the Abort message.

The format following shows an example of a
bundled datagram is defined as follows:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  MDTP Protocol Identifier                     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Version     |              Flags            |   In Queue    |
   |               |N N W I F R D A M S W R R F G U|               |
   |               |O O I S I T A C U H N E T L the termination of Endpoint A:

Endpoint A N|               |
   |               |M B N B R M T K L U R 1 C O R R|               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                 Acknowledgment Number (Seen)                  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    Sequence Number (Send)                     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        Data Size              |    Part       |      Of       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Total Number Of Messages=N   |   Message #1 Size                                     
{App indicates termination}
Abort (C/D = B1        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   |                     B1 octets of data                         |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Message #2 Size 10)
    [Tag-X]   --------------------------------> to Endpoint X

Abort (C/D = B2        |                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               |
   |                     B2 octets of data                         |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   \                                                               \
   /                                                               /
   \                                                               \
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Message #N Size 10)
    [Tag-Y]   --------------------------------> to Endpoint Y

Abort (C/D = BN        |                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               |
   |                     BN octets 10)
    [Tag-Z]   --------------------------------> to Endpoint Z


Stewart, et al                                                  [Page 27]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999


7.1 Graceful Shutdown of data                         |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Data_Size an Association

An endpoint in an association may decide to "graceful shutdown" the
association without completely closing it down. With graceful
shutdown, both endpoints shall remove any record and pending datagrams
associated with the association. Further communications between the
two endpoints can be resumed by going through a bundled datagram indicates re-initialization
procedure (see section 3.5.4).

A Graceful Shutdown message shall be sent to the actually size peer endpoint of the
data field
association, and the peer shall send back an acknowledgment.  Note
that it shall be the responsibility of the datagram, including both endpoint that sends the bundling overhead and
Graceful Shutdown message to assure that all the actually user data. Since no fragmentation is allowed in a bundled
datagram, outstanding datagrams
from its side have been resolved before it initiates the Part field will always be '0' and graceful
shutdown procedure.

In the Of field always be
'1'.

The first two octets Graceful Shutdown message, the sender shall indicate the
highest TSN Seen it has received from the peer, as well as the
Init-Tag of the data field is a 16 bit integer indicating peer.

Upon the number reception of messages bundled the Graceful Shutdown, the peer shall first
verify that Tag value contained in the current datagram. This is
followed immediately by a list of bundled messages. Each bundled Graceful Shutdown message starts with an integer of two octets indicating is
valid. If the size of Tag is invalid, the data in message must be silently discarded.

The peer then shall verify, by checking the Seen numbers from the
Graceful Shutdown message, followed by that all the data itself.

All integers in out-bound datagrams have
reached the datagram should be transmitted in destination.  Otherwise, the network byte
order.

B.2 Bundled Datagram Transfer

The T4-bundling timer and two protocol parameters, namely peer shall re-transmit all
lost datagrams.

After sending the Graceful Shutdown, if the
Min.Bundle endpoint receives any new
user datagram it shall immediately respond with an Extended Data Ack
and Max.Bundle, re-start its T3-send timer.

The peer shall send a Graceful Shutdown Ack when all the outstanding
datagrams are used to control acknowledged, then start a T4-shutdown timer. The
endpoint, after receiving the Graceful Shutdown Ack, must also
validate the Tag value contained in the message. If it does not match
the bundling of user
datagrams.

The endpoint will withhold Tag value that unlocked the datagram from transmission and start
T4-bundle timer, if association, the combined size message should be
silently discarded.

The following sequence shows an example of all user Graceful Shutdown:

    Endpoint A                                  Endpoint X
{App indicates graceful shutdown}
Graceful Shutdown (C/D=10)
   [Tag-X, Seen=10] ---------------------> (all datagrams currently
pending for transmission in resolved) 
(start T3-send timer)            /-------- Graceful Shutdown Ack (C/D=10)
                                /             [Tag-A]
                               /           (start T4-shutdown timer)  
(cancel T3-send timer) <------/            ...
(clean-up the out-bound buffer is smaller than
'Min.Bundle'.

Each time a association)                 (T4-shutdown expires)
                                           (clean-up the association)


Stewart, et al                                                  [Page 28]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999

Both endpoints shall reject any new out-bound user data becomes available for
transmission, request from their upper layers
while the endpoint will attempt to bundle graceful shutdown procedure is in progress. 


8.  Stream Operations


8.1 Stream Initiation

An MDTP association between the new data with two endpoints must be established
before any stream operation.

Except for the current withheld datagram global stream (i.e, stream id 0), a stream shall be
initiated (opened) by using the following rules:

A) If sender before any datagrams can be sent in
that stream. When a stream is no longer used, it shall be terminated
(closed) by the size user. Moreover, both sides of the new data is greater than association shall be
able to initiate or equal terminate streams independently.

The sender initiates a stream by sending a Stream Initiation. In
addition to
   'Min.Bundle', specifying the current withheld datagram will be transmitted and
   T4-bundle timer will be canceled. Then, Stream Identifier, the new data will be
   transmitted in a separate datagram.

B) If sender must set the size
Init-Tag field of the new data is less than 'Min.Bundle', but Stream Initiation to the
   combined size Tag value of the current datagram and peer
endpoint.

The sender shall also attach the new data is greater
   than or equal to 'Max.Bundle', stream-specific data, if any (usually
provided by the current datagram will be sent and upper layer), with the new data will Stream Initiation. Otherwise,
the Size of Stream Info shall be withheld as set to 0x0.

Then, the new current datagram.

C) sender shall start a T3-send timer. If the size of T3-send timer
expires, the new data is less than 'Min.Bundle', and sender shall re-transmit the
   combined size Stream Initiation.

Upon the reception of the current datagram and Stream Initiation, the new data peer must first
verify that the correct Tag value is greater
   than 'Min.Bundle', but less than 'Max.Bundle', carried in the new data will be
   bundled into Init-Tag field of
the current datagram and Stream Initiation. If so, the bundled datagram will be peer shall respond immediately transmitted. and T4-bundle timer will be canceled.

D) If with
a Stream Initiation Ack. Otherwise, the size of peer must silently discard the new data is less than 'Min.Bundle', and
Stream Initiation.

The following example shows the
   combined size opening of stream 5 by "A":

    Endpoint A                                   Endpoint Z
{App Initiates stream 5}
Stream Initiation (C/D=10)
   [Tag=Tag-Z,Strm=5]   ----------------->
(Start T3-send timer)
(Cancel T3-send timer)  <----------------- Stream Initiation Ack
                                             (C/D=10) [Strm=5]


8.2 Stream Termination

An endpoint shall be allowed to terminate one of its streams by
sending a Stream Termination to the current datagram and the new data is less than
   Min.Bundle, the new data will other side. 

The same Tag verification process used for stream initiation shall
be bundled into applied to stream termination.

Stewart, et al                                                  [Page 29]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999


The peer shall send a Stream Termination Ack in response to the current
   datagram. And Stream
Termination. 

The following example shows the T4-bundle timer will termination of stream 5 by "A":

    Endpoint A                                  Endpoint Z
{App closes stream 5}
Stream Termination (C/D=10)
   [Tag=Tag-Z,Strm=5] ------------------->
(Start T3-send timer)
(Cancel T3-send timer) <------------------ Stream Termination Ack
                                             (C/D=10) [Strm=5]

Received datagrams associated with a terminated stream shall be restarted.

E) If T4-bundle timer expires,
silently discarded. It is up to the endpoint to assure that all
outstanding user datagrams in the current datagram stream are acknowledged before the
stream termination.


8.3 Other Issues with Stream Operations

When an association is re-initialized (see section 3.5.4), all existing
streams within that association will be sent
   immediately.

F) When a T2-receive timer expires, automatically terminated.

The receiver shall silently discard any bundled data waiting datagrams associated with a
stream which has not yet been opened or has already been terminated.


9.  Interface with Upper Layer

The upper layer protocols (ULP) shall request for services by passing
primitives to be
   transmitted MDTP and shall receive notifications from MDTP for
various events. 

The primitives and notifications described in this section should be sent immediately with
used as a piggy-backed Ack guideline for implementing MDTP.

A) Init.MDTP primitive

This primitive allows MDTP to
   acknowledge all un-acked initialize its internal data previously received.

G) If a T4-bundle timer is running structures
and data arrives, the T2-receive
   timer should not be started.

H) A T4-bundle timer should never be canceled unless it allocate necessary resources for setting up its operation
environment. Note that once MDTP is being
   supplanted by a T3-send timer.

When a bundled datagram arrives at initialized, ULP can communicate
directly with any other endpoints without re-invoking this primitive.

Mandatory attributes:

None.

Optional attributes:

The following types of attributes may be passed along with 
the receiving endpoint, each
message is unbundled primitive:


Stewart, et al                                                  [Page 30]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999

 o Timer selection and delivered separately its operation syntax -- to indicate to MDTP
   an alternative timer the upper layer.

The following are the suggested protocol parameter values MDTP should use for bundled
datagram transfer:

T4-bundle Timer  -   40 ms
Min.Bundle       - 1000 octets
Max.Bundle       - 1432 octets

Appendix C: Fragmented Message Transfer its operation.
 o Initial MDTP operation mode;
 o IP port number, if ULP wants it to be specified;

B) Init.Association

This defines the mechanism for fragmented datagram transport in
MDTP. It is optional for implementation.

When primitive allows the size of upper layer to initiate an out-bound user message exceeds the value defined
in the protocol parameter Max.Bundle, the endpoint shall fragment the
message into smaller pieces of size equal association to or smaller than
'Max.Bundle' and send each piece out in a separate datagram.

The "Part" and "Of" fields are used to disassemble and reassemble the
fragmented message.
specific peer endpoint. The combination peer endpoint shall be specified by one of
the maximal 'Of' value, IP address/port pairs which
is 255, and define the maximal Data Size endpoint (see section 2.2) will determined 1.1).

Mandatory attributes:

 o associationID - specified as one of the maximal size IP address/port pairs of a single user message that
   the MDTP can send or
receive in fragmented message transfer mode.

However, an endoint shall never send fragmented datagrams peer endpoint with which the association is to be established.

Optional attributes:

 o eligibleNetList - a peer if list of destination IP address/port pairs that peer set
   the NOM bit peer endpoint is allowed to 1 during their association
initialization.

The following example shows use in its network rotation. By
   default, all destination IP address/port pairs on the transmission peer are
   available. 

C) Term.Association

Terminating an association.

Mandatory attributes:

 o associationID - specified as one of a fragmented message
(assuming Max.Bundle=1432, Min.Bundle=1000):

Endpoint A                                      Endpoint Z
{App sends message size=3300 octets}
[Header Flags=DAT|ACK|GAR
        Part=0,Of=3
        Seen=3,Send=16,Size=1432]-------> (Start T2-receive timer)
[Header Flags=DAT|ACK|GAR
        Part=1,Of=3
        Seen=3,Send=17,Size=1432]------->
[Header Flags=DAT|ACK|GAR
        Part=2,Of=3
        Seen=3,Send=18,Size=436]-------->
(Start T3-send timer)
                                              ..
                                              {Timer T2 Expires}
                                 /----------- [Header Flags=ACK
                                /              Mode=0
                               /               Part=0,Of=0
(cancel timer T3) <-----------/                Seen=18,Send=4]

Notice that "A" is using the reliable transfer mode to send IP address/port pairs of
   the
fragmented message, therefore "Z" will hold peer endpoint with which the fragments and request
retransmission if a fragment association is found missing, i.e., if a gap to be terminated.

Optional attributes:

None.

D) Send.Data primitive

This is found
in the received main method to send datagrams via MDTP.

Mandatory attributes:

 o data (see ). When all - This is the payload ULP wants to transmit;
 o size - The size of the parts payload in number of octets;
 o associationID - One of the fragmented
message are received, IP address/port pair of the receiving endpoint will re-assemble peer endpoint.
   Note that the
message and dispatch it actual destination address sent to the upper layer.

It is also allowed in will be determined
   by MDTP due to send fragmented message using Unreliable
Transfer the network rotation, unless the current mode (see section 4.5). However,
   prohibits MDTP network rotation; in unreliable mode, each
fragment such a case the datagram will
   be dispatch sent to the application upon its arrival, and no
retransmission will be requested even if IP address/port specified by associationID.

Optional attributes:

 o mode-flags - This indicates a fragment is found missing.

Bundling is prohibited if new MDTP operation mode, taking effect
   immediately including the current datagram contains a fragment of
a fragmented message.

Appendix D: Multicast send;

Stewart, et al                                                  [Page 31]

Internet Draft   Multi-network Datagram Transfer

This defines Transmission Protocol   June 1999


 o context - optional information that will be carried in the
   Send.Failure notification to the ULP if the mechanism for unreliable transportation of multicast
datagrams
   this datagram fails. 

 o streamID - to indicate which stream to send the data on. By
   default, the global stream will be used.

E) Receive.Data primitive

This primitive shall return the first datagram in MDTP. It the MDTP in-queue to
ULP, if there is optional for implementation.

D.1 Multicast Datagram Header Format

Multicast datagrams one available. It may, depending on the specific
implementation, also return other informations such as the sender's
address, whether there are identified more datagrams available for retrieval,
etc. The behavior is undefined if no datagram is available when this
primitive is invoked.

Mandatory attributes:

 o buffer - the memory location indicated by setting MUL, UNR, and DAT bits
to 1.

Two new fields are added the ULP to store the standard 
   received datagram and other information.

Optional attributes:

   None.

F) Data.Arrive notification

MDTP shall invoke this notification on the ULP when a datagram header to
support multicast:

Multicast To Transmit address - This is
successfully received and ready for retrieval.

G) Send.Failure notification

If a datagram can not be delivered MDTP shall invoke this notification
on the multicast address, in
network byte order, that the sender transmitted ULP.

The following may be optionally be passed with the notification:

 o data to. The
receiver can use this information for internal tracking purposes.

Multicast From - This is the network address (or the IP Address of
Network 1 as described in 3.2, if redundant networks exist) of location ULP can find the
sender, in network byte order.

                 MDTP Header Format un-delivered datagram.
 o context - Multicast Format

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | optional information associated with this datagram (see
   D).

H) Network.Status.Change notification

When a endpoint-id is marked down (e.g., when MDTP Protocol Identifier                     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Version     |              Flags            |   In Queue    |
   |               |N N W I F R D A M S W R R F G U|               |
   |               |O O I S I T A C U H N E T L A N|               |
   |               |M B N B R M T K L U R 1 C O R R|               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                 Acknowledgment Number (Seen)                  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    Sequence Number (Send)                     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |        Data Size              |    Part       |      Of       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  Multicast To Transmit address                |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             Multicast From - senders base address             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   \                                                               \
   /                             data                              /
   \                                                               \
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

For multicast datagrams, the value in detects a failure),
or marked up (e.g., when MDTP detects a recovery), MDTP shall
invoke this notification on the Send field ULP. 

The following shall indicate be passed with the sequence number notification:

 o endpoint-id - This indicates the IP address/port of multicast datagrams transmitted the
   peer endpoint affected by the
sender. change;
 o new-status - This information helps indicates the receiver of new status.


Stewart, et al                                                  [Page 32]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999

I) Communication.Up notification

This notification is used when MDTP becomes ready to send or receive
datagrams, or when a lost communication to an endpoint is restored. 

The following shall be passed with the multicast notification:

 o status - This indicates what type of event that has occurred;
 o associationID - An IP address/port to detect
duplicated multicast datagrams and also identify the peer endpoint;

J) Communication.Lost notification

When MDTP loses communication to detect lost multicast
datagrams from an endpoint completely or detects
that the same sender. endpoint has performed a abort or graceful shutdown
operation, it shall invoke this notification on the ULP.

The Seen field following shall normally be
set passed with the notification:

 o status - This indicates what type of event that has occurred;
 o associationID - An IP address/port number to 0, unless in some special cases stated below.

Bundling identify the peer
   endpoint;
 o packets-enqueue - The number and fragmentation are not allowed in either multicast location of un-sent datagrams
   still holding by MDTP;
 o last-acked - the sequence number last acked by that peer endpoint;
 o last-sent - the sequence number last sent to that peer endpoint;

K) Change.Network.Rotation primitive

When the upper layer wants to inform MDTP to make a specific network
eligible or
broadcast datagrams.

No initiation shall ineligible for in network rotation, the upper layer will send
this primitive to MDTP.

Mandatory attributes:

 o  action - This indicates if the network is to be needed made eligible or
             ineligible for an network rotation.
 o  network-id - This is the IP address/port of the peer endpoint to transmit
    be added or removed from network rotation consideration.

L) Open.Stream primitive

This should be used by the upper layer to open a
multicast address.

D.2 Transmission new stream.

Mandatory attributes:

 o associationID - One of Multicast Datagrams

The following example illustrates multicast transmissions between two
endpoints.

Endpoint A                                               Endpoint Z
{App multicasts a message}
[Header Flags=MUL|UNR|DAT
        Part=0,Of=1
        Seen=0,Send=5,Size=250]--------------> (no Ack necessary)

...
{App multicasts a message}
[Header Flags=MUL|UNR|DAT
        Part=0,Of=1
        Seen=0,Send=6,Size=500]--------------> (no Ack necessary)


Notice that the values IP address/port to identify the peer
   endpoint of the Send field in association to which the multicast datagrams
(which are 5 and 6, respectively). They represent stream is to be opened. An
   association must have existed at the sequence numbers time of the multicast datagrams "A" has sent out. Endpoint Z stream open.

Optional attributes:

   streamInfo - The upper layer should use this value field to detect missing or duplicate datagrams.

Duplicate datagrams will be discarded and no effort will be made pass any
   stream-specific data to
retransmit lost multicast datagrams.

D.3 Reset of the Multicast Datagram Sequence Number

If the Seen field other endpoint of a received multicast datagram equals to '1', this
indicates that the sender has reset its multicast datagram sequence
number. association.


Stewart, et al                                                  [Page 33]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999

Returned attributes:

 o The receiving endpoint, upon detecting this reset indicator in
the incoming multicast datagram, should start a procedure to adopt the
new sequence stream number for error detection. However, caution
should that is opened.

M) Close.Stream primitive

This shall be taken used by the upper layer to prevent false resets due request to duplicated datagrams
with reset indicator propagating through multiple networks.

To guarantee that all receivers close a stream.

Mandatory attributes:

 o associationID - One of the multicast group adopt the new
sequence number, IP address/port to identify the reset indicator should be repeated within peer
   endpoint of the
first N multicast datagrams sent out after association to which the reset. N stream is predefined
by the protocol parameter 'Num.Of.Mcast.Reset.Msg'.

At the receiving endpoint, when to be closed.

 o stream number - The stream number to identify the reset indicator is detected stream to be
   closed (this should be the
new sequence number will be adopted. However, if two reset events are
detected within a predefined time interval (Min.Mcast.Time.To.Reset), returned by the second reset indicator will be ignored. Stream.Open
   primitive on this stream).


10. Suggested MDTP Timer and Protocol Parameter Values

The following are suggested timer values for these two MDTP:

T1-init Timer       -   160 ms
T2-receive Timer    -    20 ms
T3-send Timer       -   160 ms (TL3-default)
T4-shutdown Timer   -   300 ms
T5-heartBeat timer  -  4000 ms (TL5-default)

The following protocol parameters are: are recommended:

Max.Outstanding.dg	- 20 messages
Max.Retransmit          - 10 attempts
Max.Init.Retransmit     -  8 attempts
Min.Mcast.Time.To.Reset -  5 seconds
Num.Of.Mcast.Reset.Msg  -  5 messages

Appendix E: Unreliable Delivery

This defines the support


11. Acknowledgments

The authors wish to thank Brian Wyld, A. Sankar, Henry Houh, Gary
Lehecka, Ken Morneault, Lyndon Ong, Greg Sidebottom and others for
their very valuable comments.


12.  Authors' Addresses

Randall R. Stewart                          Tel: +1-847-632-7438
Cellular Infrastructure Group               EMail: stewrtrs@cig.mot.com
Motorola, Inc.                              
1475 W. Shure Drive, #2C-6                  
Arlington Heights, IL 60004
USA



Stewart, et al                                                  [Page 34]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999


Qiaobing Xie                                Tel: +1-847-632-3028
Cellular Infrastructure Group               EMail: xieqb@cig.mot.com
Motorola, Inc.                              
1501 W. Shure Drive, #2309
Arlington Heights, IL 60004
USA

Suheel Hussain                              Tel: +1-919-472-2312
Cisco Systems Inc.                          EMail:ssh@cisco.com
7025 Kit Creek Road
Research Triangle Park, NC  27709

Chip Sharp                                  Tel: +1-919-851-2085
Cisco Systems Inc.                          EMail:chsharp@cisco.com
7025 Kit Creek Road
Research Triangle Park, NC  27709

Hanns Juergen Schwarzbauer                  Tel: +49-89-722-24236
SIEMENS AG
Hofmannstr. 51
81359 Munich,  Germany
EMail: HannsJuergen.Schwarzbauer@icn.siemens.de

Tom Taylor                                  Tel: +1-613-736-0961
Nortel Networks                             EMail:taylor@nortelnetworks.com
1852 Lorraine Ave.
Ottawa Ontario Canada
K1H6Z8

Ian Rytina                                  Tel:
Ericsson Australia                          EMail:ian.rytina@ericsson.com
37/360 Elizabeth Street
Melbourne, Victoria 3000, Australia


13. References

[1] Postel, J. (ed.), "Internet Protocol - DARPA Internet Program
Protocol Specification", RFC 791, USC/Information Sciences Institute,
September 1981. 

[2] Postel, J., "User Datagram Protocol", RFC 768, USC/Information Sciences 
Institute, August 1980. 

[3] Postel, J. (ed.), "Transmission Control Protocol", RFC 793, USC/ 
Information Sciences Institute, September 1981.

[4] Jacobson V., "Congestion Avoidance and Control", Proceedings of 
SIGCOMM '88, pp 314-329, August 1988.

[5] Seth, T., etc. "Performance Requirements for sending Unreliable datagrams Signaling in MDTP.  It
is optional Internet
Telephony", Internet-Draft <draft-seth-sigtran-req-00.txt>, May 1999.


Stewart, et al                                                  [Page 35]

Internet Draft   Multi-network Datagram Transmission Protocol   June 1999


[6] Rytina, I., "Framework for implementation.

The unreliable transfer mode allows two endpoints to send to each
other without acknowledging the receiving. This can usually achieve
higher data throughput than the reliable transfer mode. To indicate
the unreliable transfer mode the sender Generic Common Signaling Transport
Protocol", draft-rytina-sigtran-generic-framework-00.txt>, Feb. 1999.

[7] Ashworth, J., "The Naming of a datagram with user data
simply sets the UNR flag to 1. The following sequence illustrates
unreliable data transfer.

Endpoint A                                      Endpoint Z
{App sends 2 messages}
[Header Flags=UNR|DAT|ACK
        Part=0,Of=1
        Seen=0,Send=4,Size=100]-------->
[Header Flags=UNR|DAT|ACK
        Part=0,Of=1
        Seen=0,Send=5,Size=100]-------->

                                             {App sends 1 message}
                                   <------- [Header Flags=UNR|DAT|ACK
                                             Part=0,Of=1
                                             Seen=5,Send=1,Size=450]
...
{App sends 2 more messages}
[Header Flags=UNR|DAT|ACK
        Part=0,Of=1
        Seen=1,Send=6,Size=100]------>

[Header Flags=UNR|DAT|ACK
        Part=0,Of=1
        Seen=451,Send=7,Size=100]------>

Note that no timers shall be started by either end, Hosts", RFC 2100, April 1997.

[8] Braden, R., "Requirements for Internet hosts - Application and that even
though both ends are in Unreliable transfer mode, the ACK flag is
still set by the sender of the datagram. This means that the Seen
field in the datagram header is still valid to indicating the sequence
number of the last datagram received by the sender.  The upper layer
can use this information to help detecting missing or duplicated
datagrams. However, MDTP shall make no effort to detect or retransmit
missing data or to screen out duplicated datagrams.

E.1 Ordered Unreliable Delivery

In unreliable transfer, the sender should be allowed to request
ordered delivery by setting the RE1 flag to 1.

When Ordered Unreliable Delivery is indicated, the receiver shall
order the newly arrived datagram with any datagrams it has received
but yet passed to its upper layer.

If it receives a datagram which is older than the last datagram it has
passed to the upper layer, that datagram shall be silently discarded.
Support", RFC 1122, October 1989.

[9] Eastlake 3rd, D., Crocker, S., and Schiller, J., "Randomness
Recommendations for Security", December 1994.











      This Internet Draft expires in 6 months from April June 1999.































Stewart, et al                                                  [Page 36]
----