[ih] ARPANET uncontrolled packets (was: UDP Length Field?)

[Jack Haverty]

Thanks Alex!

I wasn't an IMP programmer either, but I remember there were quite a few
people involved in evolving the ARPANET over the 80s timeframe, changing
the internal mechanisms in response to issues that surfaced during
operation.  So it's also possibly important to match the IMP behavior to
the time of any Internet experiments in order to interpret any results. 
That's probably very difficult to do decades after the fact.

I do recall that the Internet had a somewhat disruptive reputation
within the ARPANET engineering and operations staffs.  

One time in particular I remember is when the guy in charge of
operations at the time (I can see his face but not his name...) came in
to my office to ask what the Internet was doing to the ARPANET.  It was
in the early 80s IIRC.  ARPANET behavior had suddenly changed, and they
had traced the cause back to an unanticipated change in IP traffic.  
The Internet was screwing up the ARPANET.   Since I was in charge of the
Internet projects at BBN at the time, the ARPANET's problem became my
problem.

After investigating, we found that the change in behavior was caused by
a new release of the (IIRC) Berkeley OS (I think this happened before
the advent of Sun), which had recently been announced and was being
brought online by lots of sites around the ARPANET.  That OS release had
a new feature that "pinged" every gateway regularly, maintaining a
snapshot of the current state of the Internet (which machines were
up/down) that users could display on their screens.

This new feature was enabled by default.  So as each computer OS was
updated, it started continuously pinging all over the ARPANET.

This wreaked havoc on the ARPANET, which was designed with an
understanding that users generally operated in "sessions" which involved
connecting to some remote host, using it for a while, disconnecting, and
repeating that behavior with the next remote host.   Telnet, FTP, and
EMail all had such a traffic pattern, and the IMP code was tuned to
behave well in such situations.

The Berkeley OS' new feature involved sending one packet to a remote
computer, receiving one packet, and then going on to another remote
computer.  Continuously, at a rapid rate.

That "single-packet connection" behavior was causing the IMPs to thrash,
repeatedly and rapidly setting up and tearing down internal IMP-IMP
connections and associated resources.  It was bad, and getting worse
with every new computer that came online with the new OS release.

We "fixed" the problem for the short term by convincing the OS group to
issue an emergency new release, identical except with the "Internet
snapshot" mechanism defaulted to off.   The IMP crew also went off to
figure out what to do about this latest disruption caused by the
newgangled Internet crowd, but I don't know any detail about what they
might have changed in the IMP code to avoid future problems.

I think this kind of thing happened through the 80s, as Internet users
disrupted traditional traffic patterns on the ARPANET.   The details of
changes in ARPANET internal mechanisms are probably, as the old saying
goes, "documented in the code".   There may be some clues in the various
BBN reports from that timeframe too, if any historian is interested.

The OS "pinging" was a perfectly legal use of the ARPANET, and didn't
require "uncontrolled packets".  It was not subject to any requirement
for permission.   But such experiences must have reinforced the fear of
allowing Internet users access to uncontrolled packets.

/Jack Haverty



On 12/3/20 1:01 PM, Alex McKenzie via Internet-history wrote:
>  I was not an IMP programmer so I may have the details wrong, but as I recall an uncontrolled packet was treated just like any other packet, except:- if it ever arrived at an IMP that didn't have a buffer available it was ACKed but dropped, and- no RFNM was generated by the destination IMP.A host sending an uncontrolled packet could set the priority bit on or off; if priority was on the packet would go to the heads of queues just like any other packet.  It was retransmitted like a regular packet if it encountered errors on a circuit (to do otherwise would have been much more complicated).
> If there were an uncontrolled packet resident in an IMP buffer and then that IMP ran out of buffers I don't believe that the IMP had a way of finding and deleting the uncontrolled packet to free up a buffer.  As I recall, that was the principal fear in allowing uncontrolled packets.
> Cheers,Alex
>
>     On Thursday, December 3, 2020, 3:37:16 PM EST, Jack Haverty via Internet-history <internet-history at elists.isoc.org> wrote:  
>  
>  Thanks, Steve, I vaguely remember that there were some highly restricted
> experiments reluctantly authorized, but not anything widespread.   E.g.,
> an obvious experiment would have been to use uncontrolled mode for all
> the core gateways and other gateways attached to ARPANET, but AFAIK that
> was never authorized.
>
> Also, I don't remember ever knowing in detail exactly how "uncontrolled"
> packets were actually handled by the IMP code, or what was experienced
> in any experiments that were done.   E.g., did uncontrolled packets "go
> to the front of the queue" bypassing normal traffic?  Did they interfere
> with other traffic, e.g., by preempting buffer space, or just get
> discarded when resources were unavailable?  In experiments, what were
> latencies, drop rates, et al like for UDP datagrams?   How did the
> ARPANET behave during such experiments?   Etc.
>
> /Jack
>
> On 12/2/20 4:33 PM, Stephen Casner wrote:
>> On Wed, 2 Dec 2020, Jack Haverty via Internet-history wrote:
>>
>>> There was an "uncontrolled mode" of operation possible within the
>>> ARPANET, which wouldallow a user computer to send packets bypassing all
>>> of the reliability mechanisms.  That could be used to deliver datagram
>>> Internet service.  However, the ARPANET managers (at DCA and BBN) were
>>> extremely reluctant to permit hosts (e.g., gateways) to use that mode,
>>> for fear that the uncontrolled traffic would crash the ARPANET.  I
>>> recall being involved in several "discussions" about using ARPANET
>>> uncontrolled mode for Internet experiments, but I don't remember any
>>> permissions ever being granted.
>> We definitely did get permission to use this for packet voice
>> experiments between ISI and LL, for example.  What I recall is that
>> initially we were required to arrange with BBN for a specific
>> experiment time between specific hosts and the uncontrolled mode (Type
>> 0, Subtype 3 -- BBN 1822 p. 3-17) would be enabled only for those
>> hosts.  But later I believe our packet voice hosts were enabled all
>> the time.
>>
>>                                                         -- Steve