INTERIM_MEETING_REPORT_
Reported by Lou Berger/Bolt Beranek and Newman
Minutes of the Internet Stream Protocol V2 Working Group (ST2)
The ST-II Working Group held two days of meetings on 8 and 9 February.
The meetings were held at George Mason University (GMU) and were hosted
by Professor Mark Pullen.
Agenda
The goal of the meeting was to cover the following items:
o State diagrams
o ST-II service
o Elimination of HIDs
o Elimination of options/parameters
o Groups of streams
o Receiver-oriented connections
o RFC structure
The following items were also identified as needing attention but the
group did not feel that there would be enough time to cover them:
o Routing
o FlowSpec
o ST-II MIB
o ST-II over various networks (ATM, Ethernet, PPP, FDDI, etc.)
o Alternate data packet headers
o Sub-flows/heterogeneous flows
o Reason codes
State Diagram Review
Murali Rajagopal presented a series of state diagrams based on his
reading of RFC 1190. (His slides will be added to the ST-II Working
Group archive.) He presented a model that contained five state
machines:
1. FSM at origin
2. FSM for the origin-side of a router
3. FSM for the target-side of a router
4. FSM to coordinate/synchronize the router origin-side and
target-side FSMs
5. FSM at target
He also listed what he saw as unresolved issues in the RFC:
o When acknowledgements on connect and other messages should be sent
o When can data be sent (after connect issued or after accept
received)
o HID negation is complex and may be flawed
o The state transition out of the ``connecting'' state is unresolved
when a connect message goes unanswered.
o Is the complexity of full duplex needed
o The RFC should include state tables
o Restriction on possible state transitions
o It is possible to receive disconnects in the middle of stream
establishment
o It is possible to receive multiple connects at any point in time
o There are mistakes in the current RFC (uses wrong message type)
Some time was spent reviewing and discussing the state diagrams. The
group felt that there may be a problem in the presented synchronization
FSM with handling refuses. It was agreed that this was probably a
problem in the RFC that was propagated to the FSM diagram.
Murali presented tables listing which messages may be issued as requests
and which as responses.
Murali ended with a list of specific issues with the current RFC:
o HID-CHANGE and HID-CHANGE-REQUEST messages may not converge on an
HID
o DISCONNECT message appears to apply to intermediate agents
o Contradiction to the use of DISCONNECT message (page 114)
downstream machine sends a DISCONNECT instead of REFUSE,
contradicts page 79, page 110
o HID-REJECT and HID-APPROVE are valid responses to change?
o What is REROUTE? (REROUTE has now been removed)
o There are ``criss-cross'' states
o REFUSE used inconsistently
ST-II Service
Luca lead a discussion on the user's view of the service provided by
ST-II. This included a discussion on when a user can send data (i.e.
after sending connect, or after first accept) and states seen by the
user. The states were documented by Claudio Topolcic. A state diagram
will be included in the Internet-Draft.
This was followed by a proposal to limit the combination of messages
that can be sent at one time, and thereby to limit state transitions.
The following table represents the agreed upon allowed actions per
state.
_______________________________________________________________
|_State______________|Allowed_Transaction______________________|
| | |
| IDLE |CONNECT --> CONNECTION PENDING |
| |(initiated by API at origin) |
| | |
| CONNECTION PENDING |TIMEOUT --> DROP --> ESTABLISHED |
| |or CLOSE --> IDLE |
| |(IDLE when all refused) |
| |(ESTABLISHED when at least 1 and accept) |
| |ALL REFUSE --> IDLE |
| |ALL ACCEPT --> ESTABLISHED |
| |LAST RESP + at least 1 ACCEPT |
| |--> ESTABLISHED |
| |ACCEPT | REFUSE --> CONNECTION PENDING |
| |DATA can be sent at any time |
| | |
| ESTABLISHED |{ADD, DROP, CHANGE, DATA, CLOSE} |
| |DROP --> ESTABLISH |
| |CLOSE --> IDLE |
| |API ISSUES CHANGE --> CHANGE |
| |CHANGE-REQUEST translated by |
| |API to CHANGE or ignored |
| |REFUSED --> ESTABLISHED |
| |ALL REFUSED --> IDLE |
| CHANGE |DATA |
| |REFUSES may be received |
| | |
| ADD TARGET |DATA |
|____________________|REFUSES_may_be_received__________________|
The discussion on ST-II service closed with a review of open issues:
o Steve Jackowski volunteered to write a description of where ST-II
fits in from the application perspective.
o Should HID negation be eliminated (replaced)?
o Handling of heterogeneous receivers and interpretation of
heterogeneous FlowSpecs.
o Handling of additions of more targets than can be contained in an
MTU connect message. Options:
- SCMP fragmentation
- issue multiple connects
There was a proposal to implement a rudimentary fragmentation
capability for ST control packets only (not data). Even a simple
send/acknowledgement scheme could be considered.
o Full duplex complicates the state machine and when data can be
sent. It was voted at the last meeting to drop FDX, so it is
dropped!
o Handling of stream preemption.
o Can a router issue a CHANGE-REQUEST?
Receiver Initiated Connections
The issue of changing ST-II service to support receiver-initiated
connections was discussed next. The general consensus was that it
should be added, but only if it is easy. In order to evaluate
complexity, the group discussed some details.
A `join' message was defined as having three components:
1. ST connection name
2. Origin's IP address
3. Desired QOS
The way and order by which a receiver can initiate a connection was
defined as follows:
1. Origin creates connection to 0 or more targets. In doing so it is
assigned an ST name.
2. Target gets ST name out of band (maybe one RTT).
3. Target sends JOIN request, request propagates to origin.
4. First agent that is encountered that contains matching connection
sends CONNECT. If the connection previously had no targets, the
origin agent is the one that sends the connect.
Steps 3 and 4 are repeated for every receiver-initiated connection.
Several outstanding points were then discussed. 1) The first item was
rerouting after a failure. The issue is that in order for an ST agent
to attempt stream recovery it must have a full list of all targets.
This implies that if automatic stream recovery is desired, all upstream
agents must be notified of a successful join. 2) This leads to the
second point, that maintaining full target lists limits the maximum size
of an ST-II connection. This is due to message overhead and required
state table space requirements. This implies that upstream agents
should not be notified of a successful join. 3) The last point was that
without origin notification of successful joins, there is no limiting
who may join a connection. This lack of access control may be
unacceptable to some applications.
The group reviewed IBM's proposed solution to these issues. In its
paper on receiver-initiated connections, IBM proposed solving the listed
issues by defining per-connection classes of services. (The paper is in
ibminet.awdpa.ibm.com:pub/st/t2-recv.ps.) Each class defines how
receiver-initiated connections are handled. Four classes were defined
as follows:
1. Sender initiated: Nobody is allowed to join, all joins are
refused.
2. Ask application: Poll application prior to allowing each join.
The group felt that this should be done out of band and should not
be included.
3. Any with notification: Allow any agent to join a connection.
Upstream agents are notified upon successful join. This permits
each upstream agent to maintain a full target list.
4. Any, no notification: Allow any agent to join a connection.
Upstream agents are not notified of any joins. Full target lists
are not maintained.
The group then discussed if ST-II name registration took place via the
creation of a connection with no targets or via a special name
registration process. The issue is when is the ``established'' state
entered: is it after name registration or after the first join request?
The group agreed that this is a special case only effecting the origin.
General consensus was that there should be no special name registration
function, and to support this via connects with zero targets. After
such a connect, the state will be ``established.''
The issue of all agents maintaining full target lists was again raised.
It was pointed out that without full target lists, specific target
disconnects would not work. One of the GMU participants suggested that
this problem can be solved via flooding (forward the connect) to all
downstream next hop agents. This lead to more discussion on the value
of receiver initiated connections. The group decided to chart out the
implications of IBM's different classes of service. The two axes are
``access control'' and ``maintained state.'' Explanations are listed
below that table.
_______________________________________________
| NOBODY Ask Join Join |
|_State___________Origin__w/notify__w/o_notify_|
| Next Hop |
|_Only_______Y1_____Y2_______Y3_________Y7_____|
| full |
|_Target_____Y4_____Y5_______Y6_________NA_____|
Y1 Implies that individual drops cannot be handled (maybe by
flooding). Only refuses and closes can be handled. No recovery,
no individual QOS changes. The value of this is that it limits
state space in intermediate hops. Is this worth anything?
Y2 Implies that individual drops cannot be handled (maybe by
flooding). Only refuses and closes can be handled. No recovery,
no individual QOS changes. The value of this is that it limits
state space in intermediate hops. Is this worth anything?
Probably not (same as Y2).
Y3 Implies that individual drops cannot be handled (maybe by
flooding). Only refuses and closes can be handled. No recovery,
no individual QOS changes. The value of this is that it limits
state space in intermediate hops and shortens connect time. Is
this worth anything? Probably not.
Y4 Full target list is maintained at all agents so all requests and
recovery can be supported. Matches current ST-II definition.
Y5 Full target list is maintained at all agents so all requests and
recovery can be supported. Is similar to current ST-II
definition, but receivers initiate connection and origin can
control access prior to successful join.
Y6 Full target list is maintained at all agents so all requests and
recovery can be supported. Is similar to current ST-II
definition, but receivers initiate connection.
Y7 Implies that individual drops cannot be handled (maybe by
flooding). Only refuses and closes can be handled. No recovery,
no individual QOS changes. The value of this is that it limits
state space in intermediate hops.
Claudio Topolcic suggested that Y1 through Y3 have little value since
agents are involved in the connection but do not take any real actions.
The group agreed that if state was not going to be maintained, only case
Y7 makes sense.
Claudio suggested that Y4 through Y6 were the same. It was agreed that
Y6 differed in that connection time was shortened, and that this may
have value in some cases. This left Y4 and Y5. Some group members felt
that they were separate cases and should be included. Other members
felt that Y5 was not different than Y4 used in conjunction with an out
of band message sent requesting a connect by the receiver. The group
agreed with the latter.
Given the discussion, the above table was simplified into the following:
______________________________________
| Maintained |Attributes |
| State |Access-control |
|_____________|security_______________|
| Next hop |Join w/o notify |
|_only________|_______________________|
| Full Target |no Join w/ |
|_List________|join___________notify__|
The following message types were decided to be needed:
o JOIN-REQ
o JOIN-REJECT previous hop will issue CONNECTs and target will answer
with ACCEPTs.
o JOIN-NOTIFY looks like accepts, acts like an accept. The issue was
raised if a JOIN-NOTIFY message should be used or if an
`unsolicited' ACCEPT should be used. It was agreed to leave this
detail for the future.
The group then discussed mapping the processing of the new messages into
existing states:
o Notify may be processed in all states
o JOIN-REQ may only be processed when ESTABLISHED. (They are queued
when not in established and handled after transition into
established or idle.)
o When ESTABLISHED and ALL REFUSED when in ``join possible'' state go
to ESTABLISHED, null connection.
Mark Pullen then asked if the group should be spending time working on
receiver-initiated. The group felt that this was a minor modification
(given IBM's running implementation experience), and Luca felt this fit
in to the charter. The group also thought the issue may be revisited
based on prototype implementation experience.
The relationship to RSVP was also brought up. The group discussed this
for a bit and concluded that ST experience should not be lost. That ST
features should be moved into RSVP for the future, and that this working
group should be focused on interoperability, product delivery, and ease
of implementation.
This migrated into a general discussion on what revised ST-II should be
and the definition of interoperability. The group agreed that
interoperable means interoperability between different implementations
of new versions and not with current (RFC 1190) versions.
Elimination of HIDs/VLIDs
What should be done with VLIDs?
Steve Jackowski asked if we can be IP address independent? Lots of
discussion, should ST-II be protocol-family independent. This
discussion was tabled for the future.
Proposal: replace with 48-bit SIDs, 16-bit unique ID + IP address. How
does this affect performance? Most thought not a great deal.
Conclusion: replace ST stream name with SID.
Can all VLIDs be replaced with new 48-bit SID? Looks like VLIDs can be
replaced by the combination of the existing ST name (SID + time-stamp)
and existing originator (sender) field (page 77). Conclusion: replace
HIDs, ST name, and VLIDs with SID.
Is it possible for SCMP to setup resources for arbitrary data formats?
Digression on heterogeneous flows.
Do we want to include sender information in data packets for debugging?
o Now will know origin but may not know sender, is this enough?
- Can talk to origin but not sender
- Can generate an error but maybe not a refuse
o General feeling is origin is enough, especially considering:
- Keep alive mechanism
- Maybe add a new error message message to be sent to origin
Review of Inclusions and Exclusions
Eliminated:
o Time-stamp
o Target list parameter (use target lists)
o Reverse charge
o Full duplex** (Luca may want to reopen based on group meeting)
o Point to point
o Errored PDU - page 70
o HID negotiation
Modified:
o replaced HIDs, VLIDs, ST name with SID
Added:
o Join
Issues:
o Error reason codes - maybe add fatal bit
o Notify, status, hello messages
o FlowSpec
o IP Encapsulation
o Preemption
o Groups
(List from above)
o Routing
o Heterogeneous flows
Groups of Streams
Used to associate attributes across multiple connections. Typical
example, sharing bandwidth (e.g. multi-way video conferencing).
Relationship:
o resources (bandwidth, addresses)
o interdependent streams (preemption/fate sharing)
o routing (fate sharing/(in)dependence)
o open/closed groups
G = Streams + relationship + attributes (e.g. group access control).
May need new options parameter giving relationship + attributes.
When does a connection get added to a group? It is ``easier'' if it
happens at stream creation and persists for life of connection. Should
there be an enforced limit?
There was a lot of discussion on what is bandwidth sharing. Proposal:
allocate max stream size, or multiplier of it.
There was discussion on what is our objective. Define group mechanism.
Define relationships that can be identified (and are thought to be
usable).
Action: need to flush out relationships.
Proposed resource allocation sharing parameter: allocation is based on
per connection ``reserved bandwidth'' multiplied by the parameter.
Open/Closed groups: decided to only have open. Closed is ugly and
requires new mechanisms.
Why would ``routing dependency'' be used: fate sharing, similar
FlowSpec do we need this relation ship? Does not look like it.
Want shared resources (bandwidth, addresses).
Steve wants interdependent streams: (fate sharing) for full duplex
connections and preemption.
Is there a limit on how many groups may a stream be associated with?
Looks like multiple would be useful (shared bandwidth+fate sharing).
What exactly is being shared, do we share resources that are control via
a single agent or across the whole subnet? Lou wants across the whole
sub-net. Everyone thinks this is very hard, Luca says that IBM does
this by using a central manager (ala ATM ARP server). Lou considers
this important for the wide area.
Want parameter indicates to the sub-net layer to optimize use of
multicast addresses, and/or share bandwidth between different
``sources?''
Claudio suggests that if its not going to be implemented do not include
it in the specification. Do we want to drop groups?
Open discussions:
Discussion on sending data once first ACCEPT received, problem of what
to do when subsequent ACCEPTS come in with smaller FlowSpec.
Intermediate (or any) agents will not forward data until 1) resources
are allocated or 2) ACCEPT received.
Undocumented implementation issue: all implementors do not propagate
CONNECT until local resources allocated.
Discussion on handling of higher offer load than FlowSpec. General
discussion on requirement for definition of how resource enforcement is
done. Alternatives: when exceeds ! random traffic dropped (current
implementations)
Claudio says that this unspecified behavior is totally unacceptable.
o Proposal 1: FlowSpec will be met, excess traffic may be dropped.
If dropped, lower priority packets are dropped first.
o Proposal 2: Allow mechanism that permits a customized dropping
algorithm.
o Group decided that this needs resolution. Agree that will use ST
priority bits.
o Proposal 3: Dropping algorithm based on FlowSpec. Default
behavior with default FlowSpec is FlowSpec will be met, excess
traffic may be dropped or delivered outside FlowSpec. If dropped
(or delivered late), lower priority packets are dropped (or
delivered late) first. 0=drop first, 7 = drop last.
The conclusion was to do Proposal 3.
There was discussion of definition of priority. Issue: Do higher
priority packets get processed first? Luca thinks this is evil and
should be avoided and that processing order should not be affected and
only dropping should be affected. Question: Does this preclude
misordering. Claudio says no, but want nondeterministic misordering ok.
Need to think about regulation policy.
RFC Structure
1. Introduction
o Position technology (why ST-II exists)
o Components in reservation world
o Relationships
o ST-II's role (what it does)
o Co-requisite components (other modules required for ST-II to
operate)
o ST-II and applications
o ST-II and other protocols
2. Basic Concepts (High level definition)
o Terminology
o Streams - types, Connection process, FlowSpec
o Multicast
o Differences form other connection oriented protocols
o Out of band issues
3. High level functional description with examples
o Includes: high level services
4. SCMP functional description
5. State machines and tables
6. ST-II PDU formats
o Note no acknowledgements
7. FlowSpec
o Routing
o IP encapsulation
8. Index
Separate RFC ST-II over sub-nets specifics examples of nets (ether,
FDDI, ATM, PPP, etc.).
An issue is how to get ``small'' paper. Maybe put 1-3 in a separate
document which gives high level concepts. The proposal is to make 1-3 a
separate document from 4-6. This was agreed. It is possibe that 6
should be a separate document.
There was a lot of discussion on who does what.
Components:
o IP
o Routing
o Resource managers
- Admission control
- Allocation
- Scheduling
o Subnet interface (includes subnet resource manager)
o Resource enforcement
Heterogeneous Flows
Can be done via multiple streams or by heterogeneous flows.
Useful for hierarchical coding where lower priority packets are
discarded at network ``choke'' points.
Luca proposes heterogeneous connection with FlowSpec that describes
requirements of sub flows. Data packets has sub-flow identifier + drop
priority bits. A disadvantage is complicated FlowSpec. An advantage is
graceful handling of hierarchical flows.
Claudio proposes the use of a single FlowSpec and the use of a drop
priority field. A disadvantage is that it may waste bandwidth. An
advantage is that there are no extra mechanisms.
Should this be an option? Should there be any options? No options are
wanted. Maybe there should be an options capability but with none
defined.
Also need to consider implications of case where subsequent targets
(connects) are able to receive larger sub-set of flows.
o Reason Codes - each person should write up their favorites so all
can compare them.
o Tunneling - someone should write up a proposal.
o Different packet headers - this needs more study. First, someone
should ensure that we are not stepping on RSVP people's toes.
o Routing - we need to provide guidance to the routing module, such
as what ST-II expects from the routing module.
o FlowSpec - pass this one for today. Everybody should send in their
FlowSpec, plus the ones that are in the literature.
o MIB - in progress. Luca and Chip will publish theirs.
o State machines - Murali will write up modified state machines for
the simplified protocol that we came up with.
o ST2 over Foo subnet - Luca wants help because different people are
using different subnets. Each proponent should write up ``ST2 over
their favorite net.'' Will probably need some coordination up
front.
Claudio and Steve will each write up their proposals to support
heterogeneous receivers.
Attendees
Lou Berger lberger@bbn.com
Luca Delgrossi luca@ibmpa.awdpa.ibm.com
Abe Drier adrier@mason1.gmu.edu
Jeffrey Dunn dunn@neptune.nrl.navy.mil
Eric Herrman eherrman@mason1.gmu.edu
Steven Jackowski stevej@syzygycomm.com
Cynthia Martin martin@neptune.nrl.navy.mil
Mark Pullen mpullen@cs.gmu.edu
Murali Rajagopal murali@mitre.org
Claudio Topolcic topolcic@bbn.com
Ann Tso tsoa@cc.ims.disa.mil