[ih] More topology
Jack Haverty
jack at 3kitty.org
Mon Aug 30 11:01:38 PDT 2021
Yes, but it was more complicated than that...a little more history:
ARPANET used RFNMs (Request For Next Message) as a means of flow
control. Every message (packet/datagram/whatever) sent by a host would
eventually cause a RFNM to be returned to the host. IIRC, hosts were
allowed to send up to 8 messages to any particular destination. So
there could be up to 8 pending RFNMs to come back to the host for
traffic to that destination. If the host tried to send a 9th message
to a particular destination, the IMP would block all transmissions from
the host until those RFNMs arrived, by shutting off the hardware
interface. So, if a host exceeded that limit of "8 in flight" to any
destination, the IMP would block it, at least temporarily, from sending
anything to any destination. That would probably be A Bad Thing.
Hosts could implement a simple algorithm and simply send one message,
and hold the next message until a RFNM came back. But to increase
throughput, it was advisable to implement some sort of "RFNM Counting"
where the host would keep track of how many messages were "in flight",
and avoid sending another message to a particular destination if that
message would exceed the 8-in-flight constraint, and thereby avoid
having the IMP shut off all of its traffic to all destinations. The
TCP/IP I implemented for Unix did that kind of RFNM Counting on the
ARPANET interface, but I'm not sure how other implementations handled
the RFNM issues.
Any "box" (such as a Port Expander) that was "spliced into" the
connection between a host and an IMP had to perform two related
functions. It had to act as a host itself in interacting with the IMP.
It also had to "look like an IMP" to the host(s) that were attached
to it. It had to essentially implement "timesharing" of the IMP's
interface.
The "1822 specifications" defined the interface between a Host and an
IMP. From it, engineers could build interfaces for their hosts to
connect them to the ARPANET. However (always a however...) the 1822
spec appeared to be symmetrical. But it wasn't. Interfaces that met
the 1822 specs could successfully interact with an IMP. Also, if you
plugged two such 1822 interfaces back-to-back (as was done in connecting
the 4 host to a Port Expander), it would often work apparently fine.
The "Host to IMP" specification wasn't quite the same as the
(internal-to-BBN) "IMP To Host" specification; it was easy for people
to treat it as if it was.
But in that early Internet, there were lots of "outages" to be
investigated. I remember doing a "deep dive" into one such
configuration where equipment was "spliced into" a Host/IMP 1822 cable
with unreliable results. It turned out to be a hardware issue, with
the root cause being the invalid assumption that any 1822-compliant
interface on a host could also successfully emulate the 1822 interface
on an IMP.
This was a sufficiently common problem that I wrote IEN 139 "Hosts As
IMPs" to explain the situation (see
https://www.rfc-editor.org/ien/scanned/ien139.pdf ), to warn anyone
trying to do such things. But that IEN only addressed the low-level
issues of hardware, signals, voltages, and noise., and warned that to do
such things might require more effort to actually behave as an IMP.
RFNMs, and RFNM counting, weren't specified in 1822, but to "look like
an IMP", a box such as a Port Expander faced design choices for
providing functionality such as RFNMs. I never knew how it did that,
and how successfully it "looked like an IMP" to all its attached
hosts. E.g., if all 4 hosts, thinking they were connected to their own
dedicated IMP port, did their own RFNM Counting, how did the PE make
that all work reliably? Maybe the situation just never came up often
enough in practice to motivate troubleshooting.
Not an issue now of course, but historically I wonder how much of the
early reliability issues in the Internet in the Fuzzy Peach era might
have been caused by such situations.
/Jack
PS - the same kind of thought has occurred to me with respect to NAT,
which seems to perform a similar "look like an Internet" function.
On 8/30/21 3:54 AM, Vint Cerf wrote:
> two tcp connections could multiplex on a given IMP-IMP link - one RFNM
> per IP packet regardless of the TCP layer "connection"
> v
>
>
> On Sun, Aug 29, 2021 at 10:30 PM Jack Haverty via Internet-history
> <internet-history at elists.isoc.org
> <mailto:internet-history at elists.isoc.org>> wrote:
>
> Thanks Barbara -- yes, the port Expander was one of the things I
> called
> "homegrown LANs". I never did learn how the PE handled RFNMs, in
> particular how it interacted with its associated NCP host that it was
> "stealing" RFNMs from.
> /jack
>
> On 8/29/21 2:38 PM, Barbara Denny wrote:
> > There was also SRI's port expander which increased the number of
> host
> > ports available on an IMP.
> >
> > You can find the SRI technical report (1080-140-1) on the web. The
> > title is "The Arpanet Imp Port Expander".
> >
> > barbara
> >
> > On Sunday, August 29, 2021, 12:54:39 PM PDT, Jack Haverty via
> > Internet-history <internet-history at elists.isoc.org
> <mailto:internet-history at elists.isoc.org>> wrote:
> >
> >
> > Thanks Steve. I guess I was focussed only on the longhaul
> hops. The
> > maps didn't show where host computers were attached. At the time
> > (1981) the ARPANET consisted of several clusters of nodes (DC,
> Boston,
> > LA, SF), almost like an early form of Metropolitan Area Network
> (MAN),
> > plus single nodes scattered around the US and a satellite circuit to
> > Europe. The "MAN" parts of the ARPANET were often richly
> connected, and
> > the circuits might have even been in the same room or building or
> > campus. So the long-haul circuits were in some sense more
> important in
> > their scarcity and higher risk of problems from events such as
> marauding
> > backhoes (we called such network outages "backhoe fade").
> >
> > While I still remember...here's a little Internet History.
> >
> > The Internet, at the time in late 70s and early 80s, was in what
> I used
> > to call the "Fuzzy Peach" stage of its development. In addition to
> > computers directly attached to an IMP, there were various kinds of
> > "local area networks", including things such as Packet Radio
> networks
> > and a few homegrown LANs, which provided connectivity in a small
> > geographical area. Each of those was attached to an ARPANET IMP
> > somewhere close by, and the ARPANET provided all of the long-haul
> > communications. The exception to that was the SATNET, which
> provided
> > connectivity across the Atlantic, with a US node (in West Virginia
> > IIRC), and a very active node in the UK. So the ARPANET was the
> > "peach" and all of the local networks and computers in the US
> were the
> > "fuzz", with SATNET attaching extending the Internet to Europe.
> >
> > That topology had some implications on the early Internet behavior.
> >
> > At the time, I was responsible for BBN's contract with ARPA in
> which one
> > of the tasks was "make the core Internet reliable 24x7". That
> > motivated quite frequent interactions with the ARPANET NOC,
> especially
> > since it was literally right down the hall.
> >
> > TCP/IP was in use at the time, but most of the long-haul traffic
> flows
> > were through the ARPANET. With directly-connected computers at each
> > end, such as the ARPA-TIP and a PDP-10 at ISI, TCP became the
> protocol
> > in use as the ARPANET TIPs became TACs.
> >
> > However... There's always a "however"... The ARPANET itself
> already
> > implemented a lot of the functionality that TCP provided. ARPANET
> > already provided reliable end-end byte streams, as well as flow
> control;
> > the IMPs would allow only 8 "messages" in transit between two
> endpoints,
> > and would physically block the computer from sending more than that.
> > So IP datagrams never got lost, or reordered, or duplicated, and
> never
> > had to be discarded or retransmitted. TCP/IP could do such
> things too,
> > but in the "fuzzy peach" situation, it didn't have to do so.
> >
> > The prominent exception to the "fuzzy peach" was transatlantic
> traffic,
> > which had to cross both the ARPANET and SATNET. The gateway
> > interconnecting those two had to discard IP datagrams when they
> came in
> > faster than they could go out. TCP would have to notice,
> retransmit,
> > and reorder things at the destination.
> >
> > Peter Kirstein's crew at UCL were quite active in experimenting
> with the
> > early Internet, and their TCP/IP traffic had to actually do all
> of the
> > functions that the Fuzzy Peach so successfully hid from those
> directly
> > attached to it. I think the experiences in that path motivated
> a lot
> > of the early thinking about algorithms for TCP behavior, as well as
> > gateway actions.
> >
> > Europe is 5+ hours ahead of Boston, so I learned to expect
> emails and/or
> > phone messages waiting for me every morning advising that "The
> Internet
> > Is Broken!", either from Europe directly or through ARPA. One
> of the
> > first troubleshooting steps, after making sure the gateway was
> running,
> > was to see what was going on in the Fuzzy Peach which was so
> important
> > to the operation of the Internet. Bob Hinden, Alan Sheltzer,
> and Mike
> > Brescia might remember more since they were usually on the front
> lines.
> >
> > Much of the experimentation at the time involved interactions
> between
> > the UK crowd and some machine at ISI. If the ARPANET was
> acting up,
> > the bandwidth and latency of those TCP/IP traffic flows could gyrate
> > wildly, and TCP/IP implementations didn't always respond well to
> such
> > things, especially since they didn't typically occur when you
> were just
> > using the Fuzzy Peach.
> >
> > Result - "The Internet Is Broken". That long-haul ARPA-ISI
> circuit was
> > an important part of the path from Europe to California. If it was
> > "down", the path became 3 or more additional hops (IMP hops, not
> IP),
> > and became further loaded by additional traffic routing around the
> > break. TCPs would timeout, retransmit, and make the problem worse
> > while their algorithms tried to adapt.
> >
> > So that's probably what I was doing in the NOC when I noticed the
> > importance of that ARPA<->USC ARPANET circuit.
> >
> > /Jack Haverty
> >
> >
> > On 8/29/21 10:09 AM, Stephen Casner wrote:
> > > Jack, that map shows one hop from ARPA to USC, but the PDP10s
> were at
> > > ISI which is 10 miles and 2 or 3 IMPs from USC.
> > >
> > > -- Steve
> > >
> > > On Sun, 29 Aug 2021, Jack Haverty via Internet-history wrote:
> > >
> > >> Actually July 1981 -- see
> > >> http://mercury.lcs.mit.edu/~jnc/tech/jpg/ARPANet/G81Jul.jpg
> <http://mercury.lcs.mit.edu/~jnc/tech/jpg/ARPANet/G81Jul.jpg>
> > <http://mercury.lcs.mit.edu/~jnc/tech/jpg/ARPANet/G81Jul.jpg
> <http://mercury.lcs.mit.edu/~jnc/tech/jpg/ARPANet/G81Jul.jpg>
> >(thanks,
> > Noel!)
> > >> The experience I recall was being in the ARPANET NOC for some
> > reason and
> > >> noticing the topology on the big map that covered one wall of
> the
> > NOC. There
> > >> were 2 ARPANET nodes at that time labelled ISI, but I'm not sure
> > where the
> > >> PDP-10s were attached. Still just historically curious how the
> > decision was
> > >> made to configure that topology....but we'll probably never
> know.
> > /Jack
> > >>
> > >>
> > >> On 8/29/21 8:02 AM, Alex McKenzie via Internet-history wrote:
> > >>> A look at some ARPAnet maps available on the web shows
> that in
> > 1982 it was
> > >>> four hops from ARPA to ISI, but by 1985 it was one hop.
> > >>> Alex McKenzie
> > >>>
> > >>> On Sunday, August 29, 2021, 10:04:05 AM EDT, Alex
> McKenzie via
> > >>> Internet-history <internet-history at elists.isoc.org
> <mailto:internet-history at elists.isoc.org>
> > <mailto:internet-history at elists.isoc.org
> <mailto:internet-history at elists.isoc.org>>> wrote:
> > >>> This is the second email from Jack mentioning a
> > point-to-point line
> > >>> between the ARPA TIP and the ISI site. I don't believe that
> is an
> > accurate
> > >>> statement of the ARPAnet topology. In January 1975 there
> were 5 hops
> > >>> between the 2 on the shortest path. In October 1975 there
> were 6.
> > I don't
> > >>> believe it was ever one or two hops, but perhaps someone can
> find
> > a network
> > >>> map that proves me wrong.
> > >>> Alex McKenzie
> > >>>
> > >>> On Saturday, August 28, 2021, 05:06:54 PM EDT, Jack
> Haverty via
> > >>> Internet-history <internet-history at elists.isoc.org
> <mailto:internet-history at elists.isoc.org>
> > <mailto:internet-history at elists.isoc.org
> <mailto:internet-history at elists.isoc.org>>> wrote:
> > >>> Sounds right. My experience was well after that early
> > experimental
> > >>> period. The ARPANET was much bigger (1980ish) and the
> topology had
> > >>> evolved over the years. There was a direct 56K line (IIRC
> between
> > >>> ARPA-TIP and ISI) at that time. Lots of other circuits too,
> but in
> > >>> normal conditions ARPA<->ISI traffic flowed directly over that
> > long-haul
> > >>> circuit. /Jack
> > >>>
> > >>> On 8/28/21 1:55 PM, Vint Cerf wrote:
> > >>>> Jack, the 4 node configuration had two paths between UCLA
> and SRI and
> > >>>> a two hop path to University of Utah.
> > >>>> We did a variety of tests to force alternate routing (by
> congesting
> > >>>> the first path).
> > >>>> I used traffic generators in the IMPs and in the UCLA
> Sigma-7 to get
> > >>>> this effect. Of course, we also crashed the Arpanet with
> these early
> > >>>> experiments.
> > >>>>
> > >>>> v
> > >>>>
> > >>>>
> > >>>> On Sat, Aug 28, 2021 at 4:15 PM Jack Haverty
> <jack at 3kitty.org <mailto:jack at 3kitty.org>
> > <mailto:jack at 3kitty.org <mailto:jack at 3kitty.org>>
> > >>>> <mailto:jack at 3kitty.org <mailto:jack at 3kitty.org>
> <mailto:jack at 3kitty.org <mailto:jack at 3kitty.org>>>> wrote:
> > >>>>
> > >>>> Thanks, Steve. I hadn't heard the details of why ISI was
> > >>>> selected. I can believe that economics was probably a
> > factor but
> > >>>> the people and organizational issues could have been the
> > dominant
> > >>>> factors.
> > >>>>
> > >>>> IMHO, the "internet community" seems to often ignore
> > non-technical
> > >>>> influences on historical events, preferring to view
> > everything in
> > >>>> terms of RFCs, protocols, and such. I think the other
> > influences
> > >>>> are an important part of the story - hence my
> "economic lens".
> > >>>> You just described a view through a manager's lens.
> > >>>>
> > >>>> /Jack
> > >>>>
> > >>>> PS - I always thought that the "ARPANET demo" aspect
> of that
> > >>>> ARPANET timeframe was suspect, especially after I noticed
> > that the
> > >>>> ARPANET had been configured with a leased circuit
> directly
> > between
> > >>>> the nearby IMPs to ISI and ARPA. So as a demo of "packet
> > >>>> switching", there wasn't much actual switching
> involved. The 2
> > >>>> IMPs were more like multiplexors.
> > >>>>
> > >>>> I never heard whether that configuration was mandated by
> > ARPA, or
> > >>>> BBN decided to put a line in as a way to keep the
> customer
> > happy,
> > >>>> or if it just happened naturally as a result of the
> ongoing
> > >>>> measurement of traffic flows and reconfiguration of
> the topology
> > >>>> to adapt as needed. Or something else. The
> interactivity
> > of the
> > >>>> service between a terminal at ARPA and a PDP-10 at ISI was
> > >>>> noticeably better than other users (e.g., me) experienced.
> > >>>>
> > >>>> On 8/28/21 11:51 AM, Steve Crocker wrote:
> > >>>>> Jack,
> > >>>>>
> > >>>>> You wrote:
> > >>>>>
> > >>>>> I recall many visits to ARPA on Wilson Blvd in
> > Arlington, VA.
> > >>>>> There were
> > >>>>> terminals all over the building, pretty much all
> connected
> > >>>>> through the
> > >>>>> ARPANET to a PDP-10 3000 miles away at USC in Marine
> > Del Rey,
> > >>>>> CA. The
> > >>>>> technology of Packet Switching made it possible
> to keep a
> > >>>>> PDP-10 busy
> > >>>>> servicing all those Users and minimize the costs of
> > everything,
> > >>>>> including those expensive communications circuits.
> > This was
> > >>>>> circa
> > >>>>> 1980. Users could efficiently share expensive
> > communications,
> > >>>>> and
> > >>>>> expensive and distant computers -- although I always
> > thought
> > >>>>> ARPA's
> > >>>>> choice to use a computer 3000 miles away was
> probably
> > more to
> > >>>>> demonstrate the viability of the ARPANET than
> because
> > it was
> > >>>>> cheaper
> > >>>>> than using a computer somewhere near DC.
> > >>>>>
> > >>>>>
> > >>>>> The choice of USC-ISI in Marina del Rey was due to other
> > >>>>> factors. In 1972, with ARPA/IPTO (Larry Roberts) strong
> > support,
> > >>>>> Keith Uncapher moved his research group out of RAND.
> Uncapher
> > >>>>> explored a couple of possibilities and found a
> comfortable
> > >>>>> institutional home with the University of Southern
> California
> > >>>>> (USC) with the proviso the institute would be off campus.
> > >>>>> Uncapher was solidly supportive of both ARPA/IPTO and
> of the
> > >>>>> Arpanet project. As the Arpanet grew, Roberts needed a
> > place to
> > >>>>> have multiple PDP-10s providing service on the Arpanet.
> > Not just
> > >>>>> for the staff at ARPA but for many others as well.
> > Uncapher was
> > >>>>> cooperative and the rest followed easily.
> > >>>>>
> > >>>>> The fact that it demonstrated the viability of
> packet-switching
> > >>>>> over that distance was perhaps a bonus, but the same
> would have
> > >>>>> been true almost anywhere in the continental U.S. at
> that time.
> > >>>>> The more important factor was the quality of the
> relationship.
> > >>>>> One could imagine setting up a small farm of machines at
> > various
> > >>>>> other universities, non-profits, or selected for profit
> > companies
> > >>>>> or even some military bases. For each of these, cost,
> > >>>>> contracting rules, the ambitions of the principal
> investigator,
> > >>>>> and staff skill sets would have been the dominant
> concerns.
> > >>>>>
> > >>>>> Steve
> > >>>>>
> > >>>>
> > >>>> --
> > >>>> Please send any postal/overnight deliveries to:
> > >>>> Vint Cerf
> > >>>> 1435 Woodhurst Blvd
> > >>>> McLean, VA 22102
> > >>>> 703-448-0965 <tel:(703)%20448-0965>
> > >>>>
> > >>>> until further notice
> >
> >
> > --
> > Internet-history mailing list
> > Internet-history at elists.isoc.org
> <mailto:Internet-history at elists.isoc.org>
> <mailto:Internet-history at elists.isoc.org
> <mailto:Internet-history at elists.isoc.org>>
> > https://elists.isoc.org/mailman/listinfo/internet-history
> <https://elists.isoc.org/mailman/listinfo/internet-history>
> > <https://elists.isoc.org/mailman/listinfo/internet-history
> <https://elists.isoc.org/mailman/listinfo/internet-history>>
>
> --
> Internet-history mailing list
> Internet-history at elists.isoc.org
> <mailto:Internet-history at elists.isoc.org>
> https://elists.isoc.org/mailman/listinfo/internet-history
> <https://elists.isoc.org/mailman/listinfo/internet-history>
>
>
>
> --
> Please send any postal/overnight deliveries to:
> Vint Cerf
> 1435 Woodhurst Blvd
> McLean, VA 22102
> 703-448-0965
>
> until further notice
>
>
>
More information about the Internet-history
mailing list