[ih] inter-network communication history

Fri Nov 8 11:16:28 PST 2019

Thanks, Craig - that fills in some detail from after I "lost track".  
But I'm still not sure if it's Simple Networks, Simple Management, or
Simple Protocol....

My use of the term "NMP" was probably misleading - it was more of a
concept than a Protocol.  At the time, whenever you started a new
project, one of the obvious first tasks was to define the necessary
protocol.  NMP might better expand to Network Management Principles -
the concepts, techniques, mechanisms, and tools that had developed over
the previous decade of ARPANET operations.

In ARPANET operations, I wasn't there until 1977 but there were lots of
horror stories from that first decade of operational experience that fed
into thinking about managing the neonatal Internet.  There had been
situations in the ARPANET such as some minor problem somewhere that
would flood the network with management traffic as every node that
noticed a problem had to yell "Fire! Fire!" strongly and continuously. 
The NMP (Principles) had evolved to preclude such events.

I still think of that ARPANET "NMP" experience as the ultimate source
leading to the later work on management in the Internet.  But that's
just what I saw from my viewpoint, so there were probably other
contemporaneous sources of inspiration as well.  Maybe someone will
write about those too.  E.G., how did Xerox do such things in their
internet universe...?  Or maybe Novell?

Some other factoids I recalled:

    - The original ARPANET management software at BBN was called "U",
for Utilities.  Later we developed a successor tool, cleverly called
"NU", for New Utilities.   I'm pretty sure that the Internet management
was integrated into NU when the NOC assumed 24x7 operations responsibility.

    - There was another possibly related project I ran at BBN in the
early 80s called "Remote Site Maintenance" or RSM.  This involved doing
SysAdmin tasks remotely, managing a bunch of Unix systems that had no
local expertise at their remote sites.   For extra flavor, those Unix
systems were all Navy machines in classified environments, which added a
lot of pragmatic complexity.   You couldn't just ssh in to the Unix
console as root....  That project is somewhat documented in the BBN
CCRs.  I suspect the experience fed into the subsequent work involving
"Host Management".

---------

While I'm thinking of it, here's a little more Internet history, early
1990s.  This time it's from my "outsider" perspective, i.e., as a mere
User, just trying to use this newfangled internet stuff.

When I joined Oracle in 1990, our group had responsibility for
development of Internet-related products, and also for
operating/managing the corporate internal internet (intranet?).  At the
time, it was pretty sizable, with nodes (most, maybe all, Cisco routers)
in ~100+ countries, growing rapidly.   There were virtually no
management tools available so operating our internet was like driving
blindfolded from fire to fire.   Of course we couldn't even look at the
router or hosts' TCP/IP code!   That's how users saw the Internet...

I dove back into the Internet world, from the "outside", to see what
kind of tools were around.  It was harder then since TimBL hadn't
invented the Web yet.   I found some of the old familiar tools, e.g.,
traceroute, ping, tcpdump, and flakeway but not much else.   There were
lots of documents (RFCs et al), but little code to be found.

Since Oracle's mission was to provide software on any computer you
liked, we had at least one of every kind of computer you can imagine. 
Our data center was kaleidoscopic rather than all blue or red or
whatever color the manufacturer picked.   I noticed that a mechanism
(MIB?) had been defined for getting information out of a TCP on a host
-- but didn't find a single computer that seemed to have implemented it.

Two of us sat down one day and started playing with the tools we found
in some Unix system (probably a Sun workstation) - mainly SNMP basics,
something like "snmpget", "snmpput" or such.  

We made up some shell scripts that collected information continuously
from all of our cisco routers scattered around the world, and simply
stuffed it all into a database.  A MIB in database lingo is akin to a
"schema" and the contents are just data, so it was a natural use for a
database.  A few more shell scripts collected things like ping data for
roundtrip times and it all went into the database.   Curiously, I don't
recall that we had to deal with ASN.1, but maybe I've suppressed such
memories.

Of course there were lots of tools developed over the years in the
database world to analyze, manipulate, compare, and display data, and to
create new data such as trends and predictions.  Data becomes Information. 

We also had lots of people who knew how to use those tools.   I think it
took a few days to put together a very useful NOC/NCC that included
things like real-time display of network behavior, traffic history and
trends, and the like.  Much of that time involved setting up more
workstations to make a wall of displays.  The software was mostly a few
pages of shell scripts and fragments of SQL.

That experience made me realize that much of the work of "managing
internets" is about manipulating data, and that there was lots of
off-the-shelf software available for making raw network data into useful
management information.  

I wish we had realized that 10 years earlier.  It might have greatly
simplified many things in addition to internet management. E.g., DNS
could readily be characterized as a distributed database.  Even the
routing mechanism of gateways can be viewed as a distributed database.

But I, at least, never had databases "on my radar", and I don't recall
anyone else in the Internet community ever mentioning databases as
possible components in any system, or offering any database-world
techniques for consideration (e.g., "two-phase commit").    We didn't
speak that language.  Perhaps that's because we never encountered them
in school so didn't know much about them.  Also, the Internet community
seemed inclined to shun technologies from the "Real World", except
sometimes for hardware and operating systems.

The History of The Internet is also about Roads Not Taken (with
apologies to Robert Frost...)

/Jack Haverty

On 11/8/19 6:56 AM, Craig Partridge wrote:
> Hi Jack:
>
> You're crossing network management streams.
>
> SNMP had nothing to do with NMP except, perhaps, some influences on
> Marty Schoffstall.  You are probably thinking of HMP (Host Management
> Protocol).
>
> Around 1983, network management work at BBN started going down two
> paths.  One was the stuff that Hinden's team was doing for the NOC. 
> The other was Jil Wescott's distributed network management system (I
> think called NOMS?) with Charlie Lynn and I think Ross Callon and
> Karen Seo and, briefly me, working on it.  Then DCA funded a large
> project (name I've forgotten) trying to build a better suite of tools
> (but not protocols as I recall) for the MILNET NOC. That work never
> made it out of BBN.
>
> In 1987, NSF realized the cupboard for multi-vendor network management
> protocols was bare.  Steve Wolff asked me to come up with a solution,
> which become HEMS and was based on my experience with NOMS and
> discussions at IETF and some great ideas from Glenn Trewitt (who had
> been dealing with Stanford's substantial internet).  Jeff Case and
> Marty Schoffstall (and others) felt HEMS was too complex to deploy
> ASAP (and ASAP was the need -- NSFNET was growing like topsy) and
> devised SGMP (Simple Gateway Management Protocol) which the IETF then
> evolved into SNMP.
>
> Thanks!
>
> Craig
>
>
> On Thu, Nov 7, 2019 at 4:33 PM Jack Haverty via Internet-history
> <internet-history at elists.isoc.org
> <mailto:internet-history at elists.isoc.org>> wrote:
>
>     On 11/7/19 1:35 PM, Vint Cerf wrote:
>
>     > Don't forget CMIP, HEMS AND SNMP
>
>     Hmmm.   Well, what I remember is:
>
>     SNMP was instantiated as a first step toward a comprehensive NMP
>     (Network Management Protocol).   As I recall, NMP was more of a
>     concept
>     than a protocol.  It was an umbrella covering a lot of pieces.
>
>     As we were struggling to make the Internet 24x7, we did a lot of
>     cogitating about what it meant to manage a network, which went well
>     beyond basic real-time monitoring and control.   In addition to the
>     Internet "core gateways", at BBN we were managing and/or operating a
>     variety of networks, so we had a fairly broad view of the issues that
>     crop as you operate and evolve networks over 5 or 10 years.
>
>     One dimension of that exploration was into life-cycle management. 
>     That
>     involved things like how to measure traffic statistics and patterns,
>     identify and predict trends, plan for topology alterations to handle
>     future traffic, plan for changes needed to add new users or
>     applications, etc.
>
>     It also included operational disturbances.  How do you deploy a new
>     major software release without disrupting users' activities?  How
>     do you
>     add a large number of new users with a new application without
>     disrupting current activity?   How do you debug problems, without
>     disrupting other users?
>
>     In the DDN context, a memorable one of those was the release of a
>     major
>     new IMP version.  To make that run smoothly, we had to create a lot of
>     mechanisms and processes even beyond those needed within the IMPs
>     themselves to smoothly convert to the new release.  E.G., we created a
>     service product called "TestNet" which allowed systems integrators to
>     test and debug their software in the new environment via a dial-up
>     link
>     and get it all running smoothly before going live in the users'
>     environment.
>
>     I recall one motivation for that was some Army system -- the one that
>     provided payroll services and issued the soldiers' checks.  If it
>     didn't
>     work the week after the software transition.... well, those guys have
>     some serious capability to protest such disrespect.
>
>     At one point an ARPANET clone was involved in the systems that are
>     used
>     by the officers at points of entry into the US.  When something was
>     wrong... well, it seemed like having a BBN identifier on your
>     paperwork
>     made you much more likely to be randomly selected for extra
>     scrutiny at
>     the airport.
>
>     Another dimension of network management was into management of
>     resources
>     - i.e., respecting the "boundaries" Dave mentioned.  This came up
>     in the
>     context of SATNET and the VAN Gateway (ARPANET connected to the public
>     X.25 world).   Whoever owned something, and paid for its operation,
>     wanted it used for their purposes only.   So ARPA research traffic
>     could
>     use SATNET, but other stuff should use the X.25 service (and
>     getting the
>     bill to go to the right place depended on quirky details like
>     which side
>     "dialed the phone" to establish the X.25 connection between
>     gateways.) 
>     In the technical environment, this requirement appeared as "policy
>     routing" and related technologies, protocols, etc. 
>
>     Resource management extended into the security world as well, i.e.,
>     making sure that only authorized "people" (humans, computers,
>     software,
>     whatever) were able to access only the appropriate resources.   It
>     also
>     impacted functionality like TOS - how do you make traffic go down the
>     most appropriate path for the kind of service it required.
>
>     All of that kind of "management" was folded under "Network
>     Management". 
>     It was, and is, a huge area.   It went far beyond the basic
>     monitoring/control of SNMP.  But to get something quickly for the
>     network we were trying to operate, SNMP was created.
>
>     SNMP was Simple Network Management Protocol.  I remember we had
>     discussions about exactly which noun the adjective "Simple" was
>     modifying.   My thought was that it really should have been Simple
>     Protocol for Simple Management of Simple Networks - i.e., SPSMSN or
>     something like that.
>
>     Anyway, SNMP was an "interim" solution while the larger issues were
>     sorted out.  I recall that Bob Kahn and I circa 1983 initiated a new
>     project area called "Automated Network Management" where we would
>     explore the more esoteric resource management ideas aligned with
>     ARPA's
>     charter.  We also put together (got funded) projects from several
>     of the
>     various "operational" clients we had, who were pretty desperate for
>     near-term pragmatic tools to help them manage their own stuff (those
>     research guys never write the Users' Manual!).   All of that combined
>     would make a foundation project for further NMP development.
>
>     Sometime in late 1983, I lost track of the "NMP" work.   BBN
>     reorganized
>     and those various projects and contracts ended up scattered around to
>     different divisions after the upheavals settled.  I think that
>     separation of the  "research" and "operational" worlds might have
>     had a
>     noticeable effect on the history of the Internet, by breaking the
>     "pipeline" from research to operations that Vint started when he moved
>     the gateways to become 24x7 services.
>
>     I don't know anything about HEMS.  It may be what later came out
>     of the
>     Automated Network Management project as a protocol part of NMP,
>     but I'm
>     not sure if it ever got deployed or transitioned and used in any
>     of the
>     operational network environments.
>
>     I know even less about CMIP.  I could never get my teeth into ISO
>     technology.  There wasn't any code to play with, and every time I
>     tried
>     to read those stacks of documents I'd fall asleep.
>
>     Fun times though,
>
>     /Jack Haverty
>
>
>
>     -- 
>     Internet-history mailing list
>     Internet-history at elists.isoc.org
>     <mailto:Internet-history at elists.isoc.org>
>     https://elists.isoc.org/mailman/listinfo/internet-history
>
>
>
> -- 
> *****
> Craig Partridge's email account for professional society activities
> and mailing lists.
-- 
Internet-history mailing list
Internet-history at elists.isoc.org
https://elists.isoc.org/mailman/listinfo/internet-history