[ih] MIBs and YANG [was GOSIP & compliance]

[Brian E Carpenter]

It's perhaps worth noting that the IETF hasn't published a MIB since
2018 (RFC 8502) but has published many YANG modules in recent years.

I don't know whether there are any statistics about deployment,
but it's clear that NETCONF/YANG has taken over from SNMP/MIB
as far as development work goes.

Regards
    Brian Carpenter

On 29-Mar-22 08:09, Bob Purvy via Internet-history wrote:
> I wasn't one of the *pioneers* of SNMP, but I led RFC 1697, implemented 
it
> on Oracle, and acquired Emanate for the Packeteer devices.
> 
> It's not true that there was NO commonality among devices. Everyone
> implemented MIB-II. HP OpenView was able to do a reasonable job of network
> discovery, using only or primarily that.
> 
> Configuration via SNMP: I know a lot of people did that. We never did. It
> wasn't suited for it, IMHO.
> 
> On Mon, Mar 28, 2022 at 11:19 AM John Day via Internet-history <
> internet-history at elists.isoc.org> wrote:
> 
>> Just to add to the comments,
>>
>>> On Mar 28, 2022, at 12:48, Craig Partridge via Internet-history <
>> internet-history at elists.isoc.org> wrote:
>>
>> 802 had already done a management protocol (84-85) very similar to SNMP
>> and had discovered its weaknesses. In particular that it generated a lot of
>> traffic to accomplish little. (I always called it the “Turning 
Machine
>> Syndrome.” It was so simple, it was too complex.  That lead to 
the adoption
>> of an object-oriented model with Request/Responses on a Transport Protocol
>> and Events on a connectionless protocol.
>>
>> It is very easy to generate important things to do that with SNMP take
>> 100s of request/responses that with HEMS or CMIP take 2 and considerably
>> fewer packets.
>>>
>>> Quick comments on Karl's comments (from this note and later).
>>>
>>> ASN.1 was in SNMP because it was in HEMS.  HEMS did it to allow us to
>>> return self-describing vendor specific data without requiring a
>>> supplemental MIB definition.  This was important because HEMS allowed 
one
>>> to ask for the full status of an aggregate object (such as an interface
>> or
>>> even an entire subjection of the MIB) and we wanted folks to be able to
>> add
>>> additional data they thought relevant to the aggregate.  ASN.1 was the
>> only
>>> self-describing data format of the time.
>>
>> ASN.1 also has the property that it makes the protocol invariant with
>> respect to syntax. By not having the ability to define the encoding rules
>> to be used, SNMP was locked into the overly verbose BER. Whereas, PER was
>> 70% more compact, and 80% less processing. PER was so processing efficient
>> that the plans for “Lightweight Encoding Rules” that was to be processing
>> efficient were scrapped.
>>>
>>> The UDP vs. TCP debate was pretty fierce and the experience of the time
>>> came down firmly on the UDP side. Recall this was the era of daily
>>> congestion collapse of the Internet between 1987 and 1990.
>>
>> Somehow this argument (which I know was intense at the time) is the most
>> absurd. All of the functions in TCP that are relevant are feedback
>> functions that only involve the source and destination. In between, the
>> handling of UDP and TCP packets by the routers is the same. If anything,
>> TCP packets with congestion control have a better chance of being received
>> and a TCP solution would have required fewer packets be generated in the
>> first place. (The last thing a management system should be doing when
>> things go bad is generating lots of traffic, but SNMP was good at that.)
>>
>> One of the misconceptions at the time (across the board) was that
>> connection-oriented of virtual-circuit and connection-oriented of transport
>> protocols were both the same ‘connection-oriented.’  They aren’t. Transport
>> protocols should not have been lumped into that argument, which speaking of
>> intense was *really* intense.
>>
>> (I know that someone will correct me on this.)
>>>
>>> Re: doing things in Python.  I'm not surprised.  The HEMS implementation
>>> proved reasonably small at the time.
>>
>> I was told by reliable sources at the time, that of the 3 protocols, SNMP
>> was the largest implementation. And Python didn’t exist yet.
>>>
>>> By the way, many of the features noted in CMIP were actually in HEMS
>> and/or
>>> SNMP first.  We were pretty open about playing with ideas at the time 
and
>>> the CMIP folks, who had an empty spec when SNMP and HEMS started, chose
>> to
>>> borrow liberally.  (Or, at least, that's my recollection).
>>
>> As hinted above, the CMIP work was based on the earlier experience in 802.
>> The big switch was moving to an OO model and including scope and filter,
>> which HEMS could have had and SNMP didn’t.
>>
>> OSI had the additional problem that IBM was advertising that OSI did data
>> transfer, not management, SNA did management, and was playing the usual
>> ‘keep the discussion going’ games to see to it that progress was not being
>> made. They were totally unprepared when complete management architecture
>> and protocol proposals came in from IEEE in 1985. (They worked like hell to
>> try to get it thrown out but couldn’t because there were too many companies
>> behind it). That broke the logjam and got things going.
>>
>> What is amusing is that when SNMP was approved, a major router vendor
>> played the same game IBM had arguing that SNMP would be okay for monitoring
>> but not configuration because it wasn’t secure. ;-) Of course, 
it wasn’t
>> secure, but it was a heck of a lot more secure using ASN.1 than the
>> vendor’s solution of opening a Telnet connection and sending passwords in
>> the clear. ;-)  Every laptop on the planet had a telnet program, but
>> exceedingly few had ASN.1 compilers. The vendor played the industry for
>> suckers and they fell for it.  The resulting debacle over SNMPv2 pretty
>> much sealed the fate of SNMP.
>>
>>> There was a network management project in the late 1980s, name now eludes
>>> me but led by Jil Wescott and DARPA funded, that sound similar in goals
>> to
>>> what Jack H. describes doing at Oracle.  I leaned on wisdom from those
>>> folks (esp. the late Charlie Lynn) as Glenn Trewitt and I sought to
>> figure
>>> out what HEMS should look like.
>>
>> The database issues were always at the forefront in the network management
>> development. Most everyone else blew it by trying to use relational
>> databases, which were totally unsuited for the problem. (Charlie Bachman
>> and I use to jokingly debate: He would say, you can’t do a bill of material
>> (parts explosion) structure in a relational database. I would counter that
>> you could but who the heck would want to!! It would be like writing a COBOL
>> compiler for a Turing Machine!) ;-)
>>
>> So when it came to network management, we immediately adopted an E-R
>> database. (Charlie always contended that every relational database had 
an
>> ER model under it for speed. I don’t know if he was right about ‘all’ but
>> it was true of a lot of them.) One of our MIT grads had been taught
>> relational was the only way. So we said do the performance comparison. 
We
>> hadn’t heard anything so we finally asked how it came out: In the best
>> case, the relational model was only 19 times slower than what we were
>> using.  HP, DEC, and others had to learn the hard way.
>>
>> We also recognized from the beginning that commonality across MIB
>> structures would the key element. Leveraging Chapter 5 of the OSI Reference
>> Model (not the part that describes the specific 7 layers) and augmenting
>> it, we were able to achieve far more commonality than OSI was. (The company
>> wouldn’t let us contribute what we had.) and of course with SNMP MIBs there
>> basically was none.  We had a common MIB structure that covered all 3 forms
>> of LANs, X.25, T1, IP, TCP, the OSI stuff and probably some things I have
>> forgotten.
>>
>> That commonality allowed our management system (fielded in 86) to at least
>> partially manage devices we had never seen and automatically conofigure
>> devices that we had: One just selected the objects on the network map to be
>> configured and pull down a menu and selected ‘configure’ 
and it was done.
>> Automatic configuration turned out to be straightforward.
>>
>> The processors at the time were a bit of a constraint but not
>> overwhelming. A lot of people were still operating under the influence 
from
>> when the constraints were even greater.
>>
>> There was a lot of learning going on during that period.
>>
>> John
>>
>>>
>>> As we do these assessments, it is worth remembering that the operational
>>> community of the time was struggling with the immediate challenge of
>>> managing networks that were flaky and ran on 68000 processors and where
>>> only a few 100K of memory was available for the management protocols.
>> The
>>> SNMP team found a way to shoe-horn the key features into that
>>> limited footprint and it promptly made a *huge* difference.
>>>
>>> Craig
>>>
>>> On Sat, Mar 26, 2022 at 2:22 AM Karl Auerbach via Internet-history <
>>> internet-history at elists.isoc.org> wrote:
>>>
>>>> ...
>>>> BTW, I thought the use of ASN.1/BER for SNMP was far from the best
>>>> choice (indeed, from an implementation quality and interoperability
>>>> point of view would be hard to find one that was worse.)  I preferred
>>>> the HEMS proposal as the most elegant, even if it did use XML.
>>>>
>>>> CMIP had some really good ideas (most particularly with regard to
>>>> selecting and filtering data at the server for highly efficient bulk
>>>> fetches.)  Marshall Rose's rehosting CMIP onto TCP (thus creating CMOT)
>>>> was very inventive and clever.  That kinda demonstrated the potential
>>>> viability, rather than the impossibility, of things like CMIP/T, even in
>>>> its bloated form.
>>>>
>>>> Diverting from the main points here:
>>>>
>>>> About a dozen years ago I decided to rework SNMP, throwing out ASN.1/BER
>>>> and using JSON, throwing out UDP and using TCP (optionally with TLS),
>>>> and adding in some of the filtering concepts from CMIP.  I preserved
>>>> most MIB names and instrumentation variable semantics (and thus
>>>> preserving a lot of existing instrumentation code in devices.)
>>>>
>>>> The resulting running code (in Python) is quite small - on par with the
>>>> 12kbytes (machine code) of the core of my Epilogue SNMP engine.  And 
it
>>>> runs several decimal orders of magnitude faster than SNMP (in terms of
>>>> compute cycles, network activity, and start-to-finish time.)  Plus I 
can
>>>> do things like "give me all data on all interfaces with a received
>>>> packet error rate greater than 0.1%".  I can even safely issue complex
>>>> control commands to devices, something that SNMP can't do very well. 
 I
>>>> considered doing commercial grade, perhaps open-source, version but it
>>>> could have ended up disturbing the then nascent Netconf effort.
>>>>
>>>>          --karl--
>>>>
>>>>
>>>> --
>>>> Internet-history mailing list
>>>> Internet-history at elists.isoc.org
>>>> https://elists.isoc.org/mailman/listinfo/internet-history
>>>>
>>>
>>>
>>> --
>>> *****
>>> Craig Partridge's email account for professional society activities and
>>> mailing lists.
>>> --
>>> Internet-history mailing list
>>> Internet-history at elists.isoc.org
>>> https://elists.isoc.org/mailman/listinfo/internet-history
>>
>> --
>> Internet-history mailing list
>> Internet-history at elists.isoc.org
>> https://elists.isoc.org/mailman/listinfo/internet-history
>>