[ih] There is no such thing as an "end-to-end failure"

[Dave Crocker]

Note that my primary comment was about terminology.

I'm going to venture that this exchange demonstrates how unproductive --
or even counterproductive -- language like "end-to-end failure" is.  It
provides a simplistic, and frankly arrogant, term that is readily
invoked, but has no precise technical meaning.  (Much like 'security' or
'privacy'...)


On 9/13/2020 9:58 AM, Joseph Touch via Internet-history wrote:
>> On Sep 13, 2020, at 9:34 AM, Dave Crocker via Internet-history
>> <internet-history at elists.isoc.org> wrote: ... On 9/13/2020 8:26 AM,
>> Joseph Touch via Internet-history wrote:
>>> On the one hand, that E2E is the composition of a set of
>>> applications into a single service that was never implemented as
>>> E2E at the mail delivery layer.
>> 
>> Oh?  Please clarify, with details.
> 
> I did; E2E would mean the entire chain of “send the mail to the
> server” and “server sent the mail out to the list and to the
> archive”.
> 
> My mail client never sends mail to your mail client. There are
> several steps involved there - SMTP and either IMAP or POP for
> unicast; SMTP to SMTP via Mailman to IMAP or POP for mailing lists.
> There is no single E2E signaling mechanism.

In formal architectural parlance, a basic mailing list sequence

   MUA -> MSA -> MTA -> ... MTA -> MDA -> MLM ->

       -> MSA -> MTA -> ... MTA -> MDA -> MUA

I say 'basic' because, of course, a message might go through multiple
mailing lists.

But for this basic sequence, since there are two MDA's noted, there are
two mail deliveries.  Hence my observation that mailing list activity is
a logical layer above basic email.  Hence "mail delivery" per se, is not
the issue, absent clarification to the contrary.

It's possible that the real desire here is for something that /is/ at
the level of basic mail transport and that something really is desired
at the time of normal mail delivery.  In that case, it has nothing to do
with mailing lists.  If it /does/ have to do with mailing lists, then it
has nothing to do with basic mail delivery, since that would be a layer
violation.

It's tempting to think that the issue might be to provide a mechanism
invoked during 'final' delivery, to the 'ultimate' recipient.  That
would presume knowing when that occurs.  cf, multiple mailing lists.


>>> If it were, your mailer would have let you know that you didn’t
>>> receive a copy of the mail

My "mailer" would let me know I didn't receive some mail?  Isn't that
akin to proving a negative?  I don't understand how my "mailer" can be
expected to know about mail I didn't receive.


> or that your post didn’t appear in the archive. So, at some level,
> one E2E failure happened in the design and implementation of ALL mail
> readers.

Sorry, but I do not understand the control mechanisms you are postulating.

They appear to involve quite a bit of mechanism that hasn't been
discussed much, over the 45+ years of email, or implemented for Internet
mail at all.

So while there might be some interesting opportunities for
enhancements, calling any of this a 'failure' is not helpful.


>> The operational reality of modern Internet mail is that perhaps 95%
>> of the traffic across the Internet is spam.  This has required
>> quite a bit of mitigation, of course, where one of the resulting
>> requirements is to permit essentially no automated feedback to the
>> author or originating operator.  Sorry.  That's unfortunate, but
>> it's not a 'failure'.
> 
> That’s an eventuality an E2E system could have accounted for, e.g.,
> via a timeout - as is done with TCP when a segment is silently
> dropped.

Rubbish.  Such hand-waves about difficult operational challenges that
develop over time, and at scale, are frequently made as convenient slams
but they never have the theoretical, experimental or operational basis
to be valid.

The basic stance of such rubbish is that all eventualities must be
anticipated before anything is put into the field.  That was the OSI
approach.  Besides creating infinite delay, it doesn't work.

Again, it's possible that there are useful enhancements to be made, but
calling these sorts of issues 'failures' serves more to polarize
discussion than to promote it.  (Hence my stooping to 'rubbish'.)


>> To the extent that feedback is essential, please offer a design
>> that is robust against abuse.  We do not currently have one. Return
>> Receipt still works, but it's not quite what you are looking for, I
>> suspect.
> 
> For a system that involves email distribution, which is another step,
> the check would have to occur by trying to match list posts to sent
> mail, not merely expect return receipts.

Sorry, but I don't know what "matching list posts to sent mail" is,
since I consider "posts" to equate to "sent".  Further, there is nothing
that identifies mail that is sent as specifically going to a list.


>> To the extent that someone wants to declare a failure at that
>> level, they need to point to the documentation involved and clarify
>> what, exactly has failed.
>> 
>> Typically what has actually failed is that their entirely
>> reasonable expectations, which were not documented anywhere, were
>> not met.
> 
> I agree; I’m claiming that expecting email to be E2E here is
> incorrect. It’s one hop in a chain for which there is no E2E system
> except the user to ensure oversight (at least at present).

This demonstrates my original point about language even more strongly. 
It appears that we agree on an essential architectural issue, though it 
also seems clear to me that there is work to be done to establish enough 
shared detail for discussion to be productive.  (And invoking 'failure' 
actually was counter-productive.)

So here's my suggestion, by way of a recitation:

The MUA-MTA model was developed around 1980 and was pretty much the only 
documented architectural model for mail for almost 30 years.  At the 
time, it was, in fact, an end-to-end model, though it's primary utility 
was in clarifying the distinction between user-level email functions and 
transport-level functions.

In the early 2000s, I noticed diligent, experienced email workers using 
wildly inconsistent references to email details and behaviors and 
decided that we needed a comprehensive email architecture document.  I 
blithely figured that I had enough background in the topic for this to 
be a tractable task to take on.

The task was to document the architecture of the /existing/ Internet 
Mail service, with no effort to propose changes.

It took 5 years of extensive interaction with the email community.  I 
constantly produced a draft that resolved all of the outstanding issues 
-- with 'rough consensus' sometimes producing a result that did not 
match my own views -- circulated the draft and would get still-more 
input that I couldn't legitimately ignore (no matter how much I wanted 
to.)  It was, by far, the hardest design work I've ever had to do.

Your certitude about what is needed for end-to-end mailing list behavior 
serves as self-nomination to produce something similar for mailing 
lists.  I look forward to reviewing your first draft.

d/


-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net