[ih] Arpanet raw messages, voice, and TCP

[Louis Mamakos]

On Nov 26, 2009, at 11:48 AM, Noel Chiappa wrote:
> 
> Why exactly fragmentation didn't work so well I don't recollect very well
> (if we ever knew for sure exactly why). I suspect that the network back
> then was 'lossier' (partly due to poor congestion control causing
> congestion drops, partly due to flakier transmission systems). Since
> end-end retransmission schemes don't work so well when loss rates go up
> (typically there's a 'knee' where performance goes over a cliff), with
> that many more packets involved for a given amount of data, we may have
> gone over the 'knee'.


We encountered a particular syndrome related to poorly designed 
network interface hardware; though this was in early Ethernet network
interfaces, and not 1822 LH/DH interfaces.  When generated, IP fragments
tended to have very small inter-packet gap times since the fragmentation
operation happend pretty "close" to the network interface, as compared
to sequentially generated data from the application.

One fairly painful manifestation of this that I experienced was in the
phase-2 NSFNET backbone, with T-1 trunks to upgrade from the 56kb/s
trunks between the "fuzzballs" on the phase-1 backbone.  The phase-2
NSFNET routers were built out of IBM RT hardware, and the Ethernet
interfaces to the co-located subscriber site used the early 3Com 3C501
PC-AT Ethernet controller.  Ah, what a wretched piece of hardware this
was!  It had minimal on-board buffering, using the same buffer for receiving
a frame as well as transmitting an outgoing packet.  A train of larger,
back-to-back packets on a busy interface was just about worst-case.  We
customers donated the later 3C503 cards to the cause to make this 
stop hurting as much.

There was a similar issue with NFS over UDP, with 8kbyte IP packets,
where the same retransmitted packet would never complete because
fragment 4 or so would never reliably be received because of the
small inter-packet gap.  In this case, reusing the same IP ID field on
the retransmitted packets was of no help at all.

Of course, these days the hardware is more powerful and not so performance
constrained, and the CPU in the end systems have more MIPS available to 
service the hardware.  The bottlenecks have moved elsewhere, now that
we've got multiple CPU's all trying help.  

Certainly in later years, all sorts of interesting bugs related to firewalls
and packet filtering attempts came to light..

Louis Mamakos