[ih] SMTP History

[Jack Haverty]

On 3/28/22 10:30, Dave Crocker via Internet-history wrote:
> 4. In  the very early 1970s, the Arpanet FTP protocol -- which has 
> mostly transferred to the Internet environment -- was under 
> development.  I only went to its final meeting and don't remember 
> whether email was in the version; from the RFC publication history, it 
> appears not. However Abhay Bhushan, who was the document editor, some 
> years ago told me it was always planned.

Much of the history of that early research has been lost in email 
exchanges which were apparently never archived.  Here's what I remember, 
perhaps clouded a bit by 50 years of elapsed time.   Lots of other 
people were involved too, so this is only what I personally can recall.

-----------------------------------

Abhay Bhushan's office was a few doors down the hall from mine while we 
were in Licklider's group at MIT.   I was working on "messaging", which 
was part of Lick's vision of office automation, which included more than 
just email - e.g., things like workflow of documents, approval processes 
for releasing documents, functions such as notarization, escrow (proof 
of sending), etc.   Abhay was working on FTP; we had a lot of 
discussions/arguments about how to get the "higher level" mechanisms of 
Lick's vision supported by the mechanisms of FTP.

There are artifacts in the RFCs capturing some of the early work. FTP 
began circa 1971 with RFC172.  At the same time, there was discussion of 
a "Mail Box Protocol" intended to enable functions like remote printing 
as a way of sending something to someone else over the ARPANET.   You 
just send it to their printer.   See RFCs 196, 221.

At first, FTP added a "MAIL <user>" command, which each machine 
receiving such MAIL could process as it saw fit.  Print it out.   Put it 
in a file somewhere.  Tell the <user> that mail is waiting.   Whatever...

Human users could send mail by simply opening an FTP connection to their 
addressee's computer, issuing a MAIL <user> command, and typing their 
message, indicating the end by a line containing just a period.  No 
"mail software" was needed, although people started building such things 
to send and receive mail on behalf of their users.   Such software 
especially solved the problem of a user wanting to send mail to another 
user, but the destination host was down at the time, which wasn't 
uncommon.   So a local "mail daemon" could keep the outgoing mail and 
periodically try to send it as needed.   I wrote one; so did lots of 
other people.

FTP was upgraded in late 1971, to include the command "MLFL".   RFCs 265 
and 278 show some of the details of how that worked.   I recall at the 
time badgering Abhay about the problems I was having with the "MAIL" 
command, and its use of <CR>.<CR> as an end-of-message indicator, which 
meant that every message had to be scanned before transmission to avoid 
premature end-of-message if the message's author happened to include 
that sequence in the message text.   I think he got tired of my 
complaining and put in the MLFL command which solved my <CR>.<CR> problem.

Ray's introduction of the @ convention was really a private protocol 
that only worked if both participants used the TENEX software.   It was 
more of a user-interface convention on how to format users' addresses in 
the <user>@<host> pattern, which would then be used by the TENEX Cpynet 
mechanisms.  So it provided email across the ARPANET, but only between 
consenting TENEX machines.   Other people (like me, Ken Pogran on 
Multics, Ken Harrenstien on MIT-AI, etc.)  used FTP to transfer 
"messages" from one machine to another across the ARPANET.

Using FTP's mechanisms, the sender would specify the target of a message 
by issuing the command "MAIL <user>" after establishing a connection to 
the proper destination host.   If the recipient server recognized the 
<user>, it would receive the message and put it where the <user> could 
see it - e.g., appending to a file in the user's home directory such as 
"DSK:JFH;JFH MAIL" (on my ITS system).   There was no particular 
standardization and various problems surfaced.  E.g., Multics was 
especially reluctant to allow processes receiving something from 
somewhere via the ARPANET to write into a user's file space without 
first logging in as that user.

Such messages didn't have any particular format.   It was just whatever 
text the sender submitted.   There were no "headers" unless the human 
created them.  It became popular to put some lines at the beginning of 
the message containing useful information, like who the message was 
from, when it was sent, etc.   There was no standard format for that, 
and people got very creative, which caused havoc in any program that 
tried to understand that information.     "User at Host" was popular, as 
well as Ray's "User at Host".   But creativity could result in headers such 
as "From the nimble fingers of Bill Smith at the Center of The Universe 
known as Host 987 on the ARPANET".   Just try writing software to 
understand such stuff!  I did, with the computer power of 50 years ago.  
Not good.....

RFC 475 describes how mail was handled in the 1973 timeframe, and 
suggests more additions to the FTP protocol to permit such "metadata" as 
the identity of the sender to be conveyed as part of the FTP protocol.   
There was also a *lot* of discussion on a mailing list called 
"HEADER-PEOPLE" (@MIT-AI IIRC, started by Ken Harrenstien) about how to 
define email headers that would be included at the beginning of each 
message, as an alternative to putting them into the FTP protocol.

At least two camps emerged in that discussion.  I was in the "Office 
Automation" camp, following Lick's vision, which motivated providing 
headers which could contain a lot of different kinds of such metadata 
about a message, but were very precisely defined to make them more 
easily readable by programs on the ends of the message transfers.   The 
other camp was interested in getting something workable and easy to 
implement, which was understandable since their "real work" was 
something other than implementing elaborate email systems.

The "elaborate" scheme was loosely called "MTP" for Message Transmission 
Protocol".   I documented one proposed format for the associated data 
transfer component called MSDTP - Message Services Data Transmission 
Protocol" in RFC 713.

MTP would be powerful but necessarily somewhat complex to implement, so 
an interim solution was developed to provide a much simpler initial step 
- the Simple Message Transmission Protocol, or SMTP.

All hosts had to implement SMTP in order to participate in email on the 
ARPANET, which was rather important since that's how ARPA insisted on 
interacting with its contractors.   So it was straightforward for 
everyone to find the time to implement SMTP.

MTP was only of interest to groups working on the broader realm of 
"Office Automation", but it did live on to some extent in projects such 
as the MME (Military Message Experiment), where the "office automation" 
aspects were pursued in the context of a military command and control 
"office".  There's a summary of MME in 
https://apps.dtic.mil/sti/pdfs/ADA098187.pdf   -- which illustrates some 
of the complex functionality of such "office automation" technology.   
Quite less "simple" than SMTP.

That's what I remember, IIRC of course.   I'm still waiting for MTP.......

Jack Haverty