[ih] Secret precedence schemes back then

[Jack Haverty]

On Mon, 2009-01-26 at 15:39 -0500, Matthias Bärwolff wrote:
> Hi,
> 
> I have come across two sources briefly pointing to the secret precedence
> scheme implemented with NSFNET's Fuzzball routers aimed at giving
> priority to telnet over other less time sensitive apps. (Mills:
> http://doi.acm.org/10.1145/52324.52337, and also: Bohn et al, 1994,
> Mitigating the Coming Internet Crunch: Multiple Service Levels Via
> Precedence)
> 
> Does anyone know of further literature sources dealing with those
> schemes and, specifically, how they developed in time and beyond NSFNET.
> What about the practices of private ISPs up until today, maybe there are
> some scholarly accounts out there about the general picture and history.
> 
> I appreciate your help.
> 
> Thanks
> Matthias
> 

Matthias et al,

I think it's important to note that, as far as I can remember, there was
never any mandatory specification of how a router (then called gateway)
was required to handle traffic.  The role of a router was to exert "best
efforts" to move the traffic along, given its knowledge of the state of
the world - info in the packet headers, info from the routers' own
measurements, and info about the characteristics of the particular
networks to which the router was attached.  

So we tried all sorts of things, some of which exhibited bizarre and
unexpected behavior.  Maybe most.

My group at BBN implemented and operated the "core gateways" - mainly
the ones that interconnected between Arpanet, Satnet, WidebandNet, and
other major installations.  Lots of different schemes were tried to see
what worked best.  This was mostly occurring through the very late 1970s
and 1980s.

Also, hardware was severely limited.  I remember one router, connecting
ArpaNet and Satnet, which at one point had only enough memory to buffer
exactly ONE packet.  So there wasn't much question of how to handle
priority in the "queue".  Sometimes it just saved the "next" packet and
discarded everything that came in afterwards until the output buffer was
free again.  Sometimes it discarded every incoming packet until the
buffer was free, and then allowed the next incoming packet to get
through.  So the queue-management approaches were severely affected by
the realities of the hardware, modem speeds, etc. at the time.

TOS was viewed as a piece of advisory information about the packet, just
like other info such as TTL.  Routers also could use knowledge of the
underlying network to which they were connected.  For example, the
Arpanet had a "RFNM" scheme that essentially applied flow control to
specific destinations.  So at any time it might be possible to send
packets out to that net to certain destinations, but not to other
destinations.

Such details were all fodder for the thinking about how to manage the
queue.  Sometimes, if TOS indicated that the packet needed fast service
(real-time voice), but the appropriate output queue was clogged, the
"high-priority" packet was discarded - the theory being that it wouldn't
get there in time to be useful anyway, so it was better to drop it early
and avoid wasting bandwidth downstream.  The same kind of argument
applied to TTL - "if this packet is so old it's likely not going to be
useful when it finally gets there, so let's drop it now."

Most of this kind of behavior and experimentation was unfortunately
never documented anywhere except maybe in the code and in email traffic.
I don't think it was particularly "secret" - it's just that back then
writing code took precedence over writing documentation or papers.  

Dave was particularly prolific in experimenting with all those
Fuzzballs, and wasn't particularly secretive it!  Whenever we saw weird
new behavior in the "core gateways", one of the first things to look at
was what the fuzzballs were sending our way.

While I was at BBN, we created the notion of "Exterior Gateway Protocol"
or EGP in response to Bob Kahn's request (while hanging on a subway
strap in some city at some Internet meeting, if I remember correctly).
This EGP was intended to provide a way to interface two gateways of
different provenance - different manufacturers, or different owners, who
were not entirely trusting of each other.  So that permitted the "core
gateways" and the "fuzzballs" to coexist peacefully.  Well, mostly...

In contrast to EGP, within a set of cooperating gateways/routers, an IGP
(Internal Gateway Protocol) was used, and it was up to the owners of
those particular systems to pick and configure an IGP as it suited their
own needs.  GGP became popular as an IGP, but it wasn't mandatory.  Any
router manufacturer could create whatever scheme that worked best for
their situation.  It was this architecture that enabled the coexistence
of different implementations and approaches to form the composite
Internet.  I don't know exactly what IGP was used within the Fuzzball
world - but that was exactly the point.  You didn't have to know in
order to interconnect.

So, the overall architecture had EGP as the interface between any number
of sub-internets, each possibly using it's own hardware, software, and
internal protocols and algorithms, implementing it's own ideas of
precedence, priority, queuing approaches, routing algorithms, and the
like.

All of the above occurred in the 80s - someone else is better equipped
to talk about the 90s and beyond (I left BBN and went "up the stack" at
Oracle).

But as far as I know, "private ISPs" still have the same environment -
when they connect "outside" they have to follow certain rules, but
inside their own system they can use whatever hardware and software they
like that has the behavior they desire.

Of course, all of this relates to purely technical constraints.  Legal
ones are a whole new issue.

I hope this has provided a bit of the "general history", at least from
the very early days.

/Jack Haverty
ex-BBN Chief Network Architect
ex-IAB (ICCB) member from way back when...
Point Arena, CA