[ih] Gateway Issue: Certification (was Re: booting linux on a 4004)

[Craig Partridge]

Hi Jack:

Complete agreement.  My poorly expressed point in the larger history was it
gets hard to certify what isn't specified and the Internet community had
struggled to make clear what was specified.
Craig

On Thu, Oct 3, 2024 at 11:43 AM Jack Haverty <jack at 3kitty.org> wrote:

> Hi Craig,
>
> Thanks for the history; it helped me fill in the gaps of what happened
> after I was directly involved.
>
> But... (there's always a but)
>
> Those RFCs are not part of the issue I was remembering.  That issue was
> "Certification", which is distinct from "Specification".  Specifications
> are documents which delineate what an implementation MUST, SHOULD, MAY, or
> MAY NOT do.  Certification is a process whereby a specific implementation
> is tested, often by an independent third party, to see if it actually does
> what the Specifications dictate.
>
> Around the time that TCP/IP became a DoD Standard in the 1980s, NIST/NBS
> also created a Certification methodology.   DoD altered its procurement
> regulations to require such Certifications for everything it purchased.
> I've never learned who specifically made either of those things happen.
> But someone did.  Perhaps Vint remembers more?
>
> The RFCs you mention are a follow-on to the Specifications that Jon
> orchestrated.  Bob Braden was also on the ICCB during that time, so it was
> probably natural for him to champion subsequent rounds of Specifications
> for NSF.
>
> But, AFAIK, no one continued the work that NIST/NBS had started, to
> further evolve Certification for the Internet.  I also never heard that
> DoD's procurement regulations were changed to require compliance with
> additional RFCs.   Maybe it happened, but I suspect they couldn't do that
> unless there was some well-defined way to Certify that a product met those
> Specifications.
>
> It's curious to me that such mechanisms have not been created for the
> Internet Industry.    Other computing technologies did develop such
> mechanisms.  For example, in the Database Industry where I worked in the
> 1990s, there were concepts like "Transactions", and testing procedures to
> see how a particular software/hardware combination actually worked in
> standard tests.  For example, vendors touted their particular hardware and
> software products as Certified to achieve some number of TPS (Transactions
> Per Second).
>
> Similarly, even today there are lots of "Benchmarks" that are used today
> to evaluate computers and their component software and hardware.
> Magazines and websites compare products and show graphs indicating how
> their test results compare, so that customers can make informed purchase
> decisions based on independent test results.
>
> Most devices we can now buy contain hardware and software that enables
> them to interact on the Internet.  But, other than raw speed, I've never
> seen any of such test results that even mention conformance with any RFC
> Specifications.
>
> Why not?
>
> IMHO, such testing and certification is more important in a networked
> environment than in a single computer.  In network environments, there are
> at least two, and probably many more computers involved in anything a user
> does.  Some of them are servers, some are clients, some are routers,
> modems, switches, etc. etc.  All of these affect the users' experience, as
> well as affecting the network and the experience of others using it.
>
> The ongoing discussions about source quench, congestion, queue management,
> et al made we wonder.  My home LAN has more than 50 "devices" attached to
> it, and contains a bunch of switches, modems, routers, cables, and other
> such stuff we all grew up with.
>
> How can I tell if they all implement <pick some acronym>?   Or if any of
> them do?
>
> Jack Haverty
>
>
> On 10/3/24 03:15, Craig Partridge wrote:
>
>
>
> On Wed, Oct 2, 2024 at 11:18 PM Jack Haverty via Internet-history <
> internet-history at elists.isoc.org> wrote:
>
>> [changed the subject to reflect the content...]
>>
>> No such tests or testing program was defined for Gateways, or anything
>> else AFAIK - such as DNS servers, mail, file, telnet servers and
>> clients, etc.  TCP and IP were subject to testing, but other important
>> technology, such as ICMP, was not.
>>
>> The need for such tests and certifications was noted on the ICCB list of
>> "things we still need to do", circa 1983.
>>
>>
> The issue was picked up again in the late 1980s as NSF was working to make
> NSFNET happen.
>
> The first realization was that there was not a list of "these are the RFCs
> [and their modifying RFCs/best practices/whatever]" that a router must
> implement.  So NSFNET participants had trouble specifying what their router
> needs were to vendors. Bob Braden and Jon Postel were tasked with creating
> a router/gateway profile, RFC 1009, which notably still uses the term
> router and gateway semi-interchangeably.
>
> RFC 1009 was a big step forward, but (quietly) a number of folks also
> reacted it was well short of what was required.  It was a tutorial of
> about 50 pages, rather than a firm specification of "do this", "don't do
> this".  It was an awkward document to use in a procurement.
>
> So when NSF encouraged the IETF to create a similar host requirements, a
> bunch of the not quite happy with RFC 1009 folks joined together to work
> with Bob Braden to try to do a better requirements document.  And mostly,
> in my biased view (I was a participant), did a pretty good job -- 200 pages
> of dense requirements split over two RFCs (1122 and 1123).  The group also
> developed the now familiar "MUST", "SHOULD", "MAY" terminology
> that defined conformance with the requirements.  Bob deserves huge credit
> for stewarding the effort.
>
> Based on the success of Host Requirements, folks turned around to look at
> router requirements again -- it took years until finally, RFC 1812 appeared
> (c. 175 pages).  And, I think (not sure), RFC 1812 was only that short
> because people went back and updated RFCs (a chunk of Host Requirements was
> text saying "oh, by the way, you MUST NOT do X and MUST do Y as documented
> in paper Z").
>
> Craig
> --
> *****
> Craig Partridge's email account for professional society activities and
> mailing lists.
>
>
>

-- 
*****
Craig Partridge's email account for professional society activities and
mailing lists.