[ih] Installed base momentum (was Re: Design choices in SMTP)

[vinton cerf]

Clem,
wow thanks for this lengthy history. So many familiar names. I sure hope
this mailing list does get archived properly as it contains a wealth of
information it would be hard to re-create in the future.

The success of TCP/IP is a direct consequence of the many people who
devoted so much time and effort to making it work, improving its operation
and porting it to so many platforms.

v


On Tue, Feb 14, 2023 at 10:10 AM Clem Cole via Internet-history <
internet-history at elists.isoc.org> wrote:

> below ... in line.
>
> On Mon, Feb 13, 2023 at 3:28 PM Craig Partridge via Internet-history <
> internet-history at elists.isoc.org> wrote:
>
> > HI Scott:
> >
> > Small nit.
> >
> Craig - I'm going to pick a few nits to your nits ...
>
> >
> > DARPA funded Berkeley to port the BBN Unix to BSD
>
> Right .... (more in a minute)
>
>
>
>
> > -- and Bill Joy chose to reimplement and develop sockets.
>
> I believe that this statement is actually backward - which I think is why
> it is often misunderstood; and why I'm going to offer a little history/the
> steps in the process - as it helps explain what happened and makes it
> easier to understand the thinking at the time. History might have proven
> some as good/bad, but this is what was going on at the time to the best
> that know/remember - which may help explain to people -- particularly those
> that did not live it - the bottom line - 4.2BSD, its socket API and
> networking implementation was extremely economical to many people and fit a
> need.
>
> First an important piece of background that seems to get lost to history,
> but turns out to be fundamental to prove the foundation/mechanism that
> would allow other actions top occur.
>
> In the 1960s UCB's EE Dept had set up the ILO - Industrial Laison Office,
> to help coordinate the research it was doing with is sponsors like
> Tektronix, HP, IBM, TI and the like.    One of the fruits of that work was
> making freely available its "open source" SW tools [SPICE/SPLICE/MOTIS et
> al.]  *to anyone* that asked for them and, in particular, its partners in
> industry and academia.  Remember, CS is part of EE at UCB, and when UNIX
> came to UCB, the ILO started freely making the local modifications (a.k.a
> the Berkeley Software Distribution [BSD] for UNIX) available to its members
> and eventually anyone with a proper UNIX license. *The point is that UCB
> had the mechanism in the ILO to distribute things* (9-track mag tape) -
> long before the Internet/websites *et al. * So having the ILO expand on
> that mission/operation was fairly easy / made sense at the time.[1]
>
> Also, please remember that BSD begets 2BSD [both PDP-11], which begets 3BSD
> and later 4BSD and 4.1BSD for the VAX [these are basically available on
> TUHS.org for anyone interested]. These distributions became popular in the
> universities/research community diue to the AT&T 1956 concept degree and
> licensing arrangement. Those sites are superset of a great deal of the ARPA
> research community.
>
> ARPA eventually chose to stop funding the use of 36-bit PDP-10s and switch
> to the 32-bit VAX line [numerious battle ensues which I'm going to ignore
> although I was there at the time], but in the end, the UCB's CSRG (Joy et
> al.) was created and given a contract 'to support and enhance UNIX' for the
> DARPA since AT&T was not going too. By the way, DARPA set up a committee to
> try to "steer" that effort -- IIRC Al Nemeth may have chaired it [In my
> mind, those earlier meetings I think of as the processors to the IETFs].
>
> But as Craig has pointed out, BBN had the ARPA contract to develop IP/TCP
> for several systems, *including UNIX for the VAX* [IIRC They did a
> 'portable' IP/TCP  stack that was used for the DG/Nova and the HP3000 and
> as a side note - as I understand it, the original MBUF code came from that
> implementation -- Rob needed a hunk of memory he controlled that was easy
> to interface into number of different OSes - if 'portable C implementation
> of IP' code is known to exist - again we would like to added to the
> TUHS.org archives].
>
> During this time, the CMU folks sends out the SPICE proposal [which I need
> to put online somewhere - I have a copy in my archives] - which ended up
> starting the Accent/Mach *et al* work.  Stanford did the SUN, V Kernel, and
> the W windows system, MIT eventually created Athena, which would give us
> X-Windows and Steve Ward's group at MIT gave us the Nu machine/its UNIX
> ports and compilers.
>
> If you look at the original BBN (Gurwitz) UNIX IP/TCP stack implementation,
> Rob had used the traditional file API - open/close/read/write paradigm that
> UNIX always had -- similar to the University of Ill / Rand NCP and the MIT
> ChaosNet code base -- where a programmer could open a connection to a
> remote system by opening a path to a special file: "/dev/tcp/host" or some
> such. I've forgotten the complete syntax, although a copy of the
> sources/distribution is in the BBN archive on TUHS.org for the curious.
>
>
> Stanford V-Kernel, CMU's Accent (and later Mach) were all exploring
> distributed computing paradigms and different APIs to enable things. Also
> note that, the Accent PORT concept was considered quite elegant at the time
> (there were other schemes such as what DEMOS and V used, but being in the
> room as were, IMO Accent was the one that Joy seems to have a hang up).  As
> part of Joy's work to support UNIX at the time ( in at least one UCB System
> seminar which I was present), he expressed his belief that he base his
> networking scheme using the traditional 'everything is a file idea but add
> new functionality that does some of the same tricks ports could do (like
> passing rights over an open file descriptor).  He also felt that the
> open(path to special file) scheme was inelegant compared to "modern" OS
> concepts of them. You also need to remember that while the ARPA (CS)
> community was gungho about IP/TCP, other parts of the industry, including
> other parts of the US Gov, tended to have more industrial alignment with
> Boeing/Ford *et al.*, and were pushing ISO;  Xerox, of course, has PUP -
> which begets XNS. So Bill created a new set of APIs for his enhanced UNIX
> for DARPA that was supposed to be protocol stack independent but still
> retains the traditional UNIX I/O semantics.
>
> Thus the sockets API was born ...
>
> Note at that time at UCB EECS dept, we already had the BBN code running on
> our *4.1BSD*-based Vaxen - just like what BBN had targeted.  In fact, in
> Cory Hall, Eric Cooper and I had installed it in the CAD systems which I
> was associated at the time and IIRC Eric and Eric Allman brought it up on
> IngVax which was the UCB APRAnet connection originally (replacing Ing70).
> Sam, of course, had the BBN code base running on the Ethernet in Evans Hall
> where all the VAX750s and we had 10 M coax between to the two buildings.
> San would write the original routed(1) to keep our LANs updated (based on
> the something similar he had seen at PARC for PUP).
>
> The BBN code, as is, was not going to work in the new interface that Bill
> was imagining and thus he believed had to be implemented. But notice the
> new UCB IP/TCP creation inherits a lot of things (such as the MBUFS from
> BBN code base). When he was adding his new sockets code to the new kernel,
> Bill decided to rewrite pieces of the stack itself instead of trying to
> move it.
>
> My point here is that *Bill reimplemented BBN, but it was hardly a scratch
> rewrite*. To be honest, if you look at his implementation - such as the
> MBUF code, I probably would not have used it, if I were starting from
> scratch.  In fact, I can say, when Stan and I wrote the original VMS IP
> stack at Tektronix a few years earlier, most of the time and debugging was
> around the memory code - as we learned how to call into Culter's memory
> system (I always found it interesting that Bill wrote the BSD memory
> system, and he still chose to use MBUFs).
>
> *Thus I believe what happened is that Bill grabbed an idea/implementation
> concept/maybe even a routine and then reformed it into what he needed; but
> that works was done decidedly after sockets were already created.   *But
> the point is that the first sockets API went into BSD4.1A and as wnj was
> building them up and created an update/reimplemented stack.  How much was
> new, how much was not was debatable.
>
> FWIW: Gurwitz and team re-wrote their stack (which I call BBN2 and also
> seems to be lost to time -- if anyone has a copy - please let me know
> offline - as I said, TUHS would like it for the archives], that did
> reimplement the BBN stack in the key of a later tickets [remember sockets
> went through a couple of changes 4.1ABSD sockets != 4.1C sockets although
> 4.1c is fairly close to what eventually came out in the 4.2BSD release.[2]
>
> What gets lost to time I think is that while the entire code in the stack
> that implements IP/TCP for 4.2BSD != BBN.  You need to look at the sockets
> code independently of the networking stack implementation.  Its true the #1
> consumer would be IP/TCP and there is evidence [Marshall Rose's ISO stack
> for example and Novell XNS Stack] that WNJ's interface was really only
> reasonable for IP without more work, Bill did succeed in his idea as UNIX
> Domain sockets of getting what he set out to build.  For getting the
> influence of other conteporary OS efforts and their APIs such Accent,
> DEMOS, V-Kernel and focussing only on the stack is likely to cause you to
> miss some important things that were the cause of what was the result/not
> ther other way around.
>
> I too am of the 'less is more' school and always found the
> open(special_file) call a tad similper to understand in terms of UNIX.
> That said, using that API reveals a great example of how UNIX's ioctl(2)
> ended up becoming the soft underbelly of many of those implementations.  So
> there is a reasonable cause to believe that Bill may have been correct that
> a new system interfaces were needed. But of course, elegance is not good
> enough. The AT&T USG Streams implementation while in many ways more elegant
> than 'pure-joy' sockets, it failed because the stacks and what were behind
> them were not very good and the truth is sockets were good-enough. which as
> I have said many times:  "Simple Economics always beats Sophisticated
> Design
> "
>
> Clem
>
>
> [1] You might ask - why is the ILO important to the Internet story? Well
> before the Internet, sharing SW in particular, was a bit more complicated -
> you had to share magtapes via physical mail/UPS.   Yes, the small ARPANet
> community had FTP [but remember UCB is really an major ARPANet player when
> this all starts - but the ARPANET Researchers are primarily PDP-10s shops.
> Places like UCB are running CDC, SDS and maybe IBM gear].  Long before UCB
> was involved with UNIX, it already had a mechanism and a culture of making
> its work available. The Berkeley (UNIX) SW Distribution was just a natural
> result of an older practice and used that mechanism.    As another side
> note, IMO the true 'father of open source software' [we did not have a name
> for it then] was the late Prof. Donald O. Pederson (a.k.a. "dop" to his
> students).  At a time where CMU, Stanford, MIT et al, were often making
> licensing agreements for their work,  dop gave everything away.  His wisdom
> was expressed best with the words: *"If I license our work, I walk in the
> front door like any other salesman.   But if we give it away, we get to
> walk in the backdoor.  We are invited into their labs and we are asked to
> work with them becoming their partner."*
>
> [2] In fact at Stellar, we added sockets to a SRV3 kernel, then used the
> BBN2 stack not the BSD one, as we felt it was easier to all parallel
> executing too, (particularly because of course we were using a SRV3 Memory
> System, not a 4.2BSD one].
> --
> Internet-history mailing list
> Internet-history at elists.isoc.org
> https://elists.isoc.org/mailman/listinfo/internet-history
>