[ih] Separation of TCP and IP

[Jack Haverty]

Another thought about the how and why of the split of TCP into 
(TCP|UDP)/IP...

IIRC, there was interest well before the TCP era (i.e., pre-1974) in 
having a network that provided a "datagram" service.  I wasn't involved 
then except as a network user, but my impression is that the motivation 
was a desire to have a "low latency" capability that wasn't necessarily 
as reliable as a "connection-oriented" service. An obvious user of such 
a service would be *interactive* voice, but there were other uses - 
e.g., "process control" kinds of applications where timeliness was more 
important than getting every piece of data.   Delays due to buffering 
and flow control mechanisms would be avoided, but some datagrams might 
never be delivered. That was OK for some uses.

Within the ARPANET, there was such a mechanism that could be used with a 
"datagram" kind of behavior.   Such "uncontrolled messages" could be 
sent, and it would bypass the normal ARPANET mechanisms for flow control 
et al.

I was at BBN from 1977 through 1990, and I remember that, especially in 
the "ARPANET group", there was a strong fear that use of uncontrolled 
packets would be disastrous, likely to crash the entire ARPANET.  So 
"loss of datagrams" wasn't the primary concern; the perceived risk was 
that uncontrolled traffic would disrupt the entire network.

As a result, it was very difficult to get permission to use this 
"datagram" service.   IIRC it was rarely granted, and even then only for 
limited times and only between certain pairs of hosts.

Back in the early 80s, I was responsible for several "Internet 
projects", including running the "core gateways".  Since gateways (aka 
routers) handled only IP datagrams, they were an obvious match to the 
ARPANET "uncontrolled" service.  So we asked for permission to use the 
"uncontrolled" option for traffic passing between gateways.

We never got that permission.   Too risky.

So it was very difficult to get permission to use that datagram 
service.   Even the "gateway group", in the same part of BBN that ran 
the ARPANET, with offices just down the hall from the ARPANET NOC, was 
too risky.  For others seeking to experiment with datagrams I suspect it 
was all but impossible.

I have a theory.   More a speculation really, about how and why TCP/IP 
evolved (Noel's question).

At the time, there was a longstanding desire in the research community 
to try out ideas using some kind of "datagram" service. But the ARPANET 
was the only available long-haul network, and it would only provide a 
reliable byte-stream service.   You could send individual IP datagrams 
over the ARPANET, but they would always come out the other end intact 
and in order, delayed by whatever buffering and retransmissions was 
needed to accomplish that.   Although it was possible to change the IMP 
code, and a "datagram" service had actually been implemented, no one 
could use it.   And only BBN could change the code.

I can't remember where I saw it, but there was work done inside the 
ARPANET to enable the interconnection of multiple networks.  I.e., in 
some document, or perhaps code, I saw packet formats that included a 
"NETWORK" field, to allow packets to be addressed to different 
networks.   I'm not sure if that ever got implemented or used.   That 
"NETWORK" field may have been in one of the internal BBN documents (or 
the code) that defined packet formats used between IMPs.   If so, 
probably few people outside BBN ever saw it.

TCP was also motivated by the need to integrate multiple networks into a 
single communications system.  One way to accomplish that would have 
been to define mechanisms and implement them in every type of network 
that needed to be interconnected.   That would likely have been 
difficult, especially with the experience with the ARPANET's 
uncontrolled datagram mechanism.

The other approach was to put all the mechanism into the host computers 
that were attached to the various networks.  Host computer software was 
significantly easier to change, especially since one or a few 
"experimental" machines could be readily modified without fear of 
somehow crashing all the other similar machines being used, by simply 
not installing the experimental software (TCP) in "production" 
machines.  Such a technique was pragmatically difficult in a network.

So TCP (V1,2) put all of the mechanisms for flow control, reliability, 
et al into the host computers, replicating the mechanisms that already 
existed in the ARPANET IMPs. Experimentation could proceed inside that 
computer software, constrained by the fact that the ARPANET was 
providing superfluous mechanisms to create "virtual circuit" behavior.

Then we noticed that the Internet architecture was even more flexible 
than it seemed.   A "wire" connecting two gateways could be simply 
considered as a very simple type of "network", with only two addresses - 
"this end" and "the other end".  So gateways could be directly connected 
with just a wire (or a long-haul telephone circuit), and the Internet 
would still work.   The ARPANET was no longer needed.   And with 
gateways interconnecting via "wire networks", the core service of the 
Internet evolved toward a pure datagram service, where IP datagrams 
could be lost, reordered, mangled, duplicated, and in general treated 
badly -- in other words, what TCP was designed to handle.

With TCP in the host computers, which could be easily modified, it was 
now possible to design and actually implement a true "datagram" core 
service, and experiment with applications that used it (like packet 
voice), as well as the backlog of ideas about mechanisms for flow 
control, congestion control, routing, etc., etc.

So it was a natural step to "split out" that datagram service and make 
it separately accessible to "application" developers.  IIRC, UDP 
happened pretty much in conjunction with the TCP/IP split, essentially 
as a mechanism for making that raw "datagram service" accessible to 
application programs (by introducing the next level of addressing with 
ports).

I don't remember all of the "timing" of all these changes, except that 
they all happened in probably less than a year.  And I'm not sure if 
this patteren of events was actually orchestrated by anyone, or if it 
just happened naturally.  For example, did the TCP/IP split happen 
because of the pent-up demand to try out ideas, and the new freedom to 
implement that came from putting the software in the "hosts" rather than 
in the "switches".

So, it's possible that the TCP/IP split happened simply because, with 
the software now accessible in the host computers, it could, to satisfy 
that longstanding desire to experiment with datagrams.   Not just for 
uses like packet voice, but also for all the people with ideas about 
schemes for routing, window management, packetization, congestion 
control, etc. etc., etc.

Jack Haverty



On 6/23/22 00:15, Noel Chiappa via Internet-history wrote:
> I'm interested in finding out more about the process by which TCP and IP were
> separated: to begin with, how it came to be recognized that this separation
> was a good thing. (This split was what enabled the later creation of UDP, of
> course.) In particular, that the basic service model (of what later became
> the internet layer) should be directly usable by applications, and that the
> complete data network be accessible not _just_ only via TCP. I am also
> interested in who drove this change (if any players in particular stand out).
>
> I have poked around a bit in the early IEN's, but I didn't find much on this
> specific area - either why, or who. From comments in IEN-22 "Internet Meeting
> Notes - 1 February 1978" (in "Introduction and Objectives) it sounds like the
> formal decision to do the split was made at the TCP meeting the day before.
> The minutes from that meeting, IEN-67 "TCP Meeting Notes - 30 & 31 January
> 1978", don't provide much, though. IEN-66 "TCP Meeting Notes - 13 & 14
> October 1977" shows that there had been a drift in this direction for a
> while; it didn't seem to be present as of IEN-3, "Internet Meeting Notes - 15
> August 1977", though.
>
> I arrived on the scene shortly after this happened (my first meeting was the
> August 1978 one), but I retain some impressions (gained no doubt from
> discussions with people like Clark and Reed). These are the impressions that
> I retain: that Danny was _a_ significant force in making this happen, because
> of his voice work - for which timeliness was important, not correctness. (In
> IEN-67, "Arrangements - Cohen" Danny "complain[ed] about TCP-3 becoming all
> things to all people".) Is that correct? (If so, it's probably his most
> significant technical legacy.) For others, I think Dave Reed may have been in
> favour too (perhaps he'd already started to think of RPC-like things). And
> perhaps some of the other voice people - e.g. Forgie? And I'm sure the PARC
> guys were trying to throw a few clues our way. Am I missing anyone? Did
> anyone stand out as being a bigger influence than the rest?
>
> Maybe there's some significan paper that discusses the architectural benefit
> of making the basic unreliable data carriage substrate accessible to _some_
> applications, but the concept didn't seem to get much coverage in the IENs.
> Maybe it was so obviously the Right Thing that not much discussion was
> needed, and the only question was when/how to do it?
>
> 	Noel