CMCC PERFORMANCE MEASUREMENT MESSAGE FORMATS
IEN 157
David Flood Page
Bolt, Beranek and Newman Inc.
50 Moulton Street
Cambridge, Massachusetts 02238
(617)491-1850
26 September 1980
IEN 157 Bolt, Beranek and Newman Inc.
TABLE OF CONTENTS
Page
1. PREFACE 1
2. INTRODUCTION 2
3. MESSAGE FORMATS 3
3.1 CPU Idle Time - report type 8 4
3.2 Packet delay - report type 9 4
3.3 Gateway to gateway delay - report type 10 5
3.4 Bit Throughput - report type 11 7
3.5 Queue occupancy - trap type 3 8
i
IEN 157 Bolt, Beranek and Newman Inc.
1. PREFACE
This document describes the message formats for the
performance measurement reports and traps which are to be added
to the ARPA CMCC gateway monitoring messages. It is an addendum
to the Gateway Monitoring Protocol described in IEN 131, which
should be consulted first. Processing for the new reports and
traps will be added to the CMCC, and a document describing their
use will be issued later.
1
Bolt, Beranek and Newman Inc. IEN 157
2. INTRODUCTION
The message types described here will be used in two ways:
either experimentally, in conjunction with message generators
used to load the Catenet until something breaks, or regularly as
are the other report types, to show how the Catenet is behaving
at any time.
In evaluating Catenet performance, message generators will
be required to load the gateways with traffic until packets are
dropped, the delays start to rise steeply, or the throughput
saturates. These conditions indicate that the limit of some
resource has been reached. These message generators, whose
implementation is yet to be defined, can be located in either the
gateways, the CMCC program, or in other internet hosts. When
both the CMCC program and the gateways implement the new message
types, and message generators are defined and implemented, the
CMCC will be able to find out how much traffic the gateways were
processing, where the bottlenecks lie in the Catenet, and what
the accompanying delays were.
All the measures described here are cumulative from the time
the gateway started. The CMCC will, by keeping suitable
histories of the measures, be able to show shorter term values as
required.
2
IEN 157 Bolt, Beranek and Newman Inc.
3. MESSAGE FORMATS
All gateway monitoring messages consist of an Internet
header followed by a monitoring header, and then the message
data. A monitoring header, as described in IEN 131, has the
following format:
Bits Contents
0 0 - Report or trap.
1 - Negotiation message.
1 0 - Report.
1 - Trap.
2-3 For a negotiation message:
0 - DO
1 - DONT
2 - WILL
3 - WONT
For a report or trap: zero.
4-7 Reserved.
8-15 Report or trap type.
16-31 For a negotiation message: report identifier.
For a regular report: Sequence number.
For a trap: data depending on trap type, i.e.
this field is not part of the header
for a trap message.
The five new message types are:
o CPU idle time (which gives a measure of how heavily the
gateway is loaded).
o Packet delay across a gateway.
o Gateway to gateway delay (round trip time).
o Throughput (bits).
3
Bolt, Beranek and Newman Inc. IEN 157
o Queue traps (which signal when the occupancy of a queue
goes above or below a certain threshold value).
3.1 CPU Idle Time - report type 8
CPU idle time gives an idea of the amount of time the
gateway machine is not doing useful processing. The purpose of
this is to find out when the CPU becomes saturated, which will be
the case if the proportion of idle time becomes very small. The
report consists of two 32-bit counts following the monitoring
header:
1. The amount of CPU idle time since the gateway started,
in milliseconds.
2. The time since the gateway started, in seconds.
3.2 Packet delay - report type 9
Packet delay refers to the length of time a packet stays in
the gateway. The measurement of this delay and of gateway to
gateway delay are related: measurement of one begins where the
other ends. The model used here assumes that gateway processing
takes place in three parts: network I/O, queuing and routing.
Implementation considerations will affect just where the packets
can be timestamped on their way through the gateway, so that for
some gateways it may be possible to stamp a packet at the network
I/O level, while for others it may not be possible until the
4
IEN 157 Bolt, Beranek and Newman Inc.
packet enters the routing processing. This specification
therefore does not define where the boundary should lie, but it
is important that together the measures account for all the delay
that a packet will experience as far as the gateway is concerned.
It is recommended that the packet delay be made to refer to as
large a fraction as possible of the time the packet spends in the
gateway. The report consists of two 32-bit counts and two 16-bit
counts. All delays are in milliseconds. The format is:
1. The total number of packets processed since the gateway
started (32 bits).
2. The total delay for all packets processed (32 bits).
3. The minimum delay experienced by a single packet (16
bits).
4. The maximum delay experienced by a single packet (16
bits).
3.3 Gateway to gateway delay - report type 10
The measurement of this delay and of packet delay are
related: see the first paragraph in the previous section.
This report could be something of an interim measure,
provided the gateways obtain radio synchronizing equipment for
measuring the one-way delay directly. However, currently only
the round trip delay can be determined. The report assumes that
gateways will use some kind of tagged echo packets to find the
5
Bolt, Beranek and Newman Inc. IEN 157
round trip delay to each of their neighbors, the tagging being
used to identify specific packets.
The report format is a table ordered by internet address
considered as a 32-bit unsigned integer. Each table entry
consists of an internet address followed by two 32-bit counts and
two 16-bit counts. The internet address is the neighbor address
for which this delay applies. Of the 32-bit counts, the first is
the cumulative total of the echo packets returned by the neighbor
since this gateway started, and the second is the total delay
experienced by those returned packets, in milliseconds. The two
16-bit counts are the minimum and maximum delays, in
milliseconds, for a single packet sent to the neighbor. There
will be one table entry for each neighbor address, so that if a
gateway is a neighbor on two networks then it will have two table
entries. There will be an entry for each such address for each
neighbor that replies to the echoes, whether or not that neighbor
is a routing gateway. The table size may grow as new neighbors
come up while a gateway is running, but it may not shrink; the
entry for a gateway that stops replying will simply remain
unchanged.
6
IEN 157 Bolt, Beranek and Newman Inc.
The report format is therefore:
Internet address of first neighbor,
lowest network number
Total of echo packets returned by this neighbor
(32 bits)
Total delay experienced (32 bits)
Minimum delay to this neighbor (16 bits)
Maximum delay to this neighbor (16 bits)
.
.
Internet address of last neighbor,
highest network number
Total echo packets returned (32 bits)
Total delay (32 bits)
Minimum delay for this neighbor (16 bits)
Maximum delay for this neighbor (16 bits)
3.4 Bit Throughput - report type 11
In contrast with the packet throughput report type, which
has its emphasis on the number of packets a gateway can process,
the bit throughput report type focuses on how fast a gateway's
network connections can accept or deliver data. The report is a
table of pairs of 32-bit counts ordered by interface; the first
count in each pair is the cumulative total of bits processed
coming in at that interface, and the second is the output count.
Interfaces are ordered as in the gateway description message,
report type 0. There are two extra 32-bit counts at the end of
the message: the first is the total of bits dropped, and the
second is the time since the gateway started, in seconds. The
counts for the interfaces include all traffic at that interface,
including control traffic and messages originating at the
7
Bolt, Beranek and Newman Inc. IEN 157
gateway.
The format of the message is therefore:
Input count for first interface
Output count for first interface
.
.
Input count for nth interface
Output count for nth interface
Dropped count
Time since gateway up
3.5 Queue occupancy - trap type 3
This is a trap message which is sent by the gateway whenever
a queue length exceeds a threshold percentage specified in the
trap request message, or when the occupancy falls below that
threshold after having been above it for some time. If a queue
is loaded such that the threshold occupancy is continually being
passed in each direction, a large number of these traps would be
generated in a short time. To avoid this, there should be some
minimum time interval between successive trap messages. It is
left up to the individual gateway implementors to decide what
this time interval should be; experience with using this trap
type will probably suggest a reasonable value.
Note that this replaces the earlier Queue full trap
described in IEN 131. I believe that a percentage occupancy trap
is more useful because if a queue becomes full, the gateway is
8
IEN 157 Bolt, Beranek and Newman Inc.
probably already dropping packets and the message is not useful
as an early warning. In any case, a queue full trap is just a
100% percentage occupancy trap.
The DO TRAP message for this trap type requires an extra
piece of information: the percentage occupancy of the queue which
is to trigger the trap. This is expressed as an integer in a
single byte following the report id field in the DO TRAP message.
A gateway should only use one value of this threshold at a time,
so that a second DO TRAP message will supersede the previous one
if the threshold value is different.
The DO TRAP message for this trap type has the format:
Bits Contents
0 1
1 1
2-3 0
4-7 0
8-15 3
16-31 report identifier
32-39 occupancy threshold
The trap message has the following format:
Bits Contents
0-7 Interface number of Queue.
8-11 Input(0) or output(1) queue.
12-15 Above(0) or below(1) the specified occupancy.
16-23 The occupancy percentage used as a trigger.
9