[ih] Installed base momentum (was Re: Design choices in SMTP)

[Clem Cole]

below ... in line.

On Mon, Feb 13, 2023 at 3:28 PM Craig Partridge via Internet-history <
internet-history at elists.isoc.org> wrote:

> HI Scott:
>
> Small nit.
>
Craig - I'm going to pick a few nits to your nits ...

>
> DARPA funded Berkeley to port the BBN Unix to BSD

Right .... (more in a minute)




> -- and Bill Joy chose to reimplement and develop sockets.

I believe that this statement is actually backward - which I think is why
it is often misunderstood; and why I'm going to offer a little history/the
steps in the process - as it helps explain what happened and makes it
easier to understand the thinking at the time. History might have proven
some as good/bad, but this is what was going on at the time to the best
that know/remember - which may help explain to people -- particularly those
that did not live it - the bottom line - 4.2BSD, its socket API and
networking implementation was extremely economical to many people and fit a
need.

First an important piece of background that seems to get lost to history,
but turns out to be fundamental to prove the foundation/mechanism that
would allow other actions top occur.

In the 1960s UCB's EE Dept had set up the ILO - Industrial Laison Office,
to help coordinate the research it was doing with is sponsors like
Tektronix, HP, IBM, TI and the like.    One of the fruits of that work was
making freely available its "open source" SW tools [SPICE/SPLICE/MOTIS et
al.]  *to anyone* that asked for them and, in particular, its partners in
industry and academia.  Remember, CS is part of EE at UCB, and when UNIX
came to UCB, the ILO started freely making the local modifications (a.k.a
the Berkeley Software Distribution [BSD] for UNIX) available to its members
and eventually anyone with a proper UNIX license. *The point is that UCB
had the mechanism in the ILO to distribute things* (9-track mag tape) -
long before the Internet/websites *et al. * So having the ILO expand on
that mission/operation was fairly easy / made sense at the time.[1]

Also, please remember that BSD begets 2BSD [both PDP-11], which begets 3BSD
and later 4BSD and 4.1BSD for the VAX [these are basically available on
TUHS.org for anyone interested]. These distributions became popular in the
universities/research community diue to the AT&T 1956 concept degree and
licensing arrangement. Those sites are superset of a great deal of the ARPA
research community.

ARPA eventually chose to stop funding the use of 36-bit PDP-10s and switch
to the 32-bit VAX line [numerious battle ensues which I'm going to ignore
although I was there at the time], but in the end, the UCB's CSRG (Joy et
al.) was created and given a contract 'to support and enhance UNIX' for the
DARPA since AT&T was not going too. By the way, DARPA set up a committee to
try to "steer" that effort -- IIRC Al Nemeth may have chaired it [In my
mind, those earlier meetings I think of as the processors to the IETFs].

But as Craig has pointed out, BBN had the ARPA contract to develop IP/TCP
for several systems, *including UNIX for the VAX* [IIRC They did a
'portable' IP/TCP  stack that was used for the DG/Nova and the HP3000 and
as a side note - as I understand it, the original MBUF code came from that
implementation -- Rob needed a hunk of memory he controlled that was easy
to interface into number of different OSes - if 'portable C implementation
of IP' code is known to exist - again we would like to added to the
TUHS.org archives].

During this time, the CMU folks sends out the SPICE proposal [which I need
to put online somewhere - I have a copy in my archives] - which ended up
starting the Accent/Mach *et al* work.  Stanford did the SUN, V Kernel, and
the W windows system, MIT eventually created Athena, which would give us
X-Windows and Steve Ward's group at MIT gave us the Nu machine/its UNIX
ports and compilers.

If you look at the original BBN (Gurwitz) UNIX IP/TCP stack implementation,
Rob had used the traditional file API - open/close/read/write paradigm that
UNIX always had -- similar to the University of Ill / Rand NCP and the MIT
ChaosNet code base -- where a programmer could open a connection to a
remote system by opening a path to a special file: "/dev/tcp/host" or some
such. I've forgotten the complete syntax, although a copy of the
sources/distribution is in the BBN archive on TUHS.org for the curious.


Stanford V-Kernel, CMU's Accent (and later Mach) were all exploring
distributed computing paradigms and different APIs to enable things. Also
note that, the Accent PORT concept was considered quite elegant at the time
(there were other schemes such as what DEMOS and V used, but being in the
room as were, IMO Accent was the one that Joy seems to have a hang up).  As
part of Joy's work to support UNIX at the time ( in at least one UCB System
seminar which I was present), he expressed his belief that he base his
networking scheme using the traditional 'everything is a file idea but add
new functionality that does some of the same tricks ports could do (like
passing rights over an open file descriptor).  He also felt that the
open(path to special file) scheme was inelegant compared to "modern" OS
concepts of them. You also need to remember that while the ARPA (CS)
community was gungho about IP/TCP, other parts of the industry, including
other parts of the US Gov, tended to have more industrial alignment with
Boeing/Ford *et al.*, and were pushing ISO;  Xerox, of course, has PUP -
which begets XNS. So Bill created a new set of APIs for his enhanced UNIX
for DARPA that was supposed to be protocol stack independent but still
retains the traditional UNIX I/O semantics.

Thus the sockets API was born ...

Note at that time at UCB EECS dept, we already had the BBN code running on
our *4.1BSD*-based Vaxen - just like what BBN had targeted.  In fact, in
Cory Hall, Eric Cooper and I had installed it in the CAD systems which I
was associated at the time and IIRC Eric and Eric Allman brought it up on
IngVax which was the UCB APRAnet connection originally (replacing Ing70).
Sam, of course, had the BBN code base running on the Ethernet in Evans Hall
where all the VAX750s and we had 10 M coax between to the two buildings.
San would write the original routed(1) to keep our LANs updated (based on
the something similar he had seen at PARC for PUP).

The BBN code, as is, was not going to work in the new interface that Bill
was imagining and thus he believed had to be implemented. But notice the
new UCB IP/TCP creation inherits a lot of things (such as the MBUFS from
BBN code base). When he was adding his new sockets code to the new kernel,
Bill decided to rewrite pieces of the stack itself instead of trying to
move it.

My point here is that *Bill reimplemented BBN, but it was hardly a scratch
rewrite*. To be honest, if you look at his implementation - such as the
MBUF code, I probably would not have used it, if I were starting from
scratch.  In fact, I can say, when Stan and I wrote the original VMS IP
stack at Tektronix a few years earlier, most of the time and debugging was
around the memory code - as we learned how to call into Culter's memory
system (I always found it interesting that Bill wrote the BSD memory
system, and he still chose to use MBUFs).

*Thus I believe what happened is that Bill grabbed an idea/implementation
concept/maybe even a routine and then reformed it into what he needed; but
that works was done decidedly after sockets were already created.   *But
the point is that the first sockets API went into BSD4.1A and as wnj was
building them up and created an update/reimplemented stack.  How much was
new, how much was not was debatable.

FWIW: Gurwitz and team re-wrote their stack (which I call BBN2 and also
seems to be lost to time -- if anyone has a copy - please let me know
offline - as I said, TUHS would like it for the archives], that did
reimplement the BBN stack in the key of a later tickets [remember sockets
went through a couple of changes 4.1ABSD sockets != 4.1C sockets although
4.1c is fairly close to what eventually came out in the 4.2BSD release.[2]

What gets lost to time I think is that while the entire code in the stack
that implements IP/TCP for 4.2BSD != BBN.  You need to look at the sockets
code independently of the networking stack implementation.  Its true the #1
consumer would be IP/TCP and there is evidence [Marshall Rose's ISO stack
for example and Novell XNS Stack] that WNJ's interface was really only
reasonable for IP without more work, Bill did succeed in his idea as UNIX
Domain sockets of getting what he set out to build.  For getting the
influence of other conteporary OS efforts and their APIs such Accent,
DEMOS, V-Kernel and focussing only on the stack is likely to cause you to
miss some important things that were the cause of what was the result/not
ther other way around.

I too am of the 'less is more' school and always found the
open(special_file) call a tad similper to understand in terms of UNIX.
That said, using that API reveals a great example of how UNIX's ioctl(2)
ended up becoming the soft underbelly of many of those implementations.  So
there is a reasonable cause to believe that Bill may have been correct that
a new system interfaces were needed. But of course, elegance is not good
enough. The AT&T USG Streams implementation while in many ways more elegant
than 'pure-joy' sockets, it failed because the stacks and what were behind
them were not very good and the truth is sockets were good-enough. which as
I have said many times:  "Simple Economics always beats Sophisticated Design
"

Clem


[1] You might ask - why is the ILO important to the Internet story? Well
before the Internet, sharing SW in particular, was a bit more complicated -
you had to share magtapes via physical mail/UPS.   Yes, the small ARPANet
community had FTP [but remember UCB is really an major ARPANet player when
this all starts - but the ARPANET Researchers are primarily PDP-10s shops.
Places like UCB are running CDC, SDS and maybe IBM gear].  Long before UCB
was involved with UNIX, it already had a mechanism and a culture of making
its work available. The Berkeley (UNIX) SW Distribution was just a natural
result of an older practice and used that mechanism.    As another side
note, IMO the true 'father of open source software' [we did not have a name
for it then] was the late Prof. Donald O. Pederson (a.k.a. "dop" to his
students).  At a time where CMU, Stanford, MIT et al, were often making
licensing agreements for their work,  dop gave everything away.  His wisdom
was expressed best with the words: *"If I license our work, I walk in the
front door like any other salesman.   But if we give it away, we get to
walk in the backdoor.  We are invited into their labs and we are asked to
work with them becoming their partner."*

[2] In fact at Stellar, we added sockets to a SRV3 kernel, then used the
BBN2 stack not the BSD one, as we felt it was easier to all parallel
executing too, (particularly because of course we were using a SRV3 Memory
System, not a 4.2BSD one].