[ih] TCP PUSH flag

[John Day]

Jack,

Wasn’t earlier than the 80s?  I remember it being mixed up with trying to optimize the character-at-a-time nature of TENEX Telnet.  Mixed up with Remote Control Transmission and Echo (RCTE).

John

> On May 16, 2025, at 14:10, Jack Haverty via Internet-history <internet-history at elists.isoc.org> wrote:
> 
> The "PUSH" mechanism was part of a larger issue that was discussed and debated in the early 1980s, while we had The Internet on the operating table to replace TCP V2 with TCP/IP V4.  I don't recall exactly what we called that issue - something like "Out-Of-Band Control" (OOBC).
> 
> OOBC was one of the items that the ICCB put on the "things we have to figure out" list, for more research to figure out exactly what to do and get it into the next release.
> 
> The basic issue of OOBC was the need for some mechanism to "interrupt" an ongoing interaction between the two end-users of a TCP connection.   When a connection was in progress, significant data might be buffered "in the pipe", and the apps communicating over TCP often were unwilling to accept more data from the pipe as they worked to complete a previous task.
> 
> This was a real practical problem.  One memorable example involved the simple use of printers, such as one that might be attached to a TAC port, printing a document arriving via TCP from some computer out on the 'net.
> 
> It was a common, and annoying, experience to accidentally send a document which wasn't a text file to be printed, such as some kind of graphics document, core dump, or other "data" file.  At the time, printers that could print diagrams or pictures were quite rare (the Xerox XGP being one example).   Most printers only printed characters, either very slowly (a mod 33 TTY or a TI700 terminal) or very quickly (any of the big high-speed printers attached to a PDP-10 or similar CPU).
> 
> If you accidentally sent a non-text file to such a printer, it tried to put it on paper.   But many of the bytes being "printed" would be control codes - e.g., octal 014 was "form feed".  That caused a printer to advance to the next page.   Octal 015 was "carriage-return".  Octal 012 was "line feed".
> 
> A graphics file with lots of 015s but few 012s would print lines of text on top of previous lines.  After a few such lines, the paper would shred at that line, and then create a huge mess inside the printer as it kept feeding paper.
> 
> Or a file with lots of 014 bytes could consume a thousand feet of paper in seconds.   Every 014 caused a new page to be rapidly ejected.
> 
> We were quite familiar with such disasters.  It had even been a problem at the demonstration of the ARPANET at the ICCC '72 conference.
> 
> So, the question was how to stop such chaos - what mechanisms would allow a user (or the user's app) to somehow stop such scenarios.
> 
> The PUSH mechanism provided one approach, telling the recipient TCP to stop waiting for more data.  It complemented the URGent mechanism which indicated that there was something important coming and the receiving app should do something about it.  Perhaps it was an ABORT command, which would stop that out-of-control printer.
> 
> For me at least, at the time I was remembering earlier experiences in the 70s implementing electronic mail, and even wrote up some "Thoughts on Interactions in Distributed Services" (see RFC722 if you're curious, especially the notion of "Request Reply Discipline").
> 
> The PUSH mechanism provided a way to tell a receiving TCP that it shouldn't wait for more data.   But that didn't mean the receiving app would be willing to accept it.   The URGent mechanism was intended to tell the app that it should accept more data as fast as possible, because there was something important coming.  But there was no way for the app to find out how that future data should be handled.    E.g., in that printer scenario, what would your app code do when it somehow got an URGent indication?   How would the human user stop a printout?
> 
> There were many discussions about how to provide some mechanisms for OOBC.  One idea was to simply advise apps to utilize a second TCP connection for "control" traffic (FTP did this on the ARPANET).   A Type-of-Service (TOS) setting of "Priority" might cause the underlying Internet to deliver such "control" traffic more quickly than the bulk data being transferred through the main TCP connection - once someone figured out how to implement TOS in the datagram infrastructure.
> 
> Using multiple connections would be of course more complicated to implement and coordinate in the app code, so a simpler solution was desired.  That needed further research, but the PUSH, URGent, and such mechanisms were a way to start.
> 
> The ARPANET was the main source of experience at the time. Although it was a packet-switched network, it had a lot of internal mechanisms that created a "virtual circuit" service to the computers attached to it.  Everything that went in at a source came out at the destination intact, complete and in sequence.  In effect, the functionality provided by TCP for creating a reliable byte-stream was already also present inside the ARPANET, with mechanisms in the source and destination IMPs for each traffic flow.  Internal messages, such as ALLOcate, travelled between IMPs to implement that mechanism.  Data flow was also regulated to achieve a typical maximum latency of 250 milliseconds - dictated by the desire to have characters typed by the human user quickly processed by the app consuming them.  Scenarios such as "buffer bloat" were not a problem.
> 
> Those internal mechanisms weren't well known at the time, except by the ARPANET implementers at BBN.  The algorithms and protocols used also changed as the traffic patterns changed over time.   Technology advances, such as the introduction of personal workstations and later PCs, altered the way people used the network.  The internal mechanisms similarly changed as needed.  I was never an ARPANET implementer, but that group was literally right down the hall so I was aware of a lot of the internal issues and mechanisms as I worked on TCP.
> 
> TCP made significant changes to how networks worked.   Some of these may not be obvious even now.
> 
> For example, the internal mechanisms in the IMPs were largely invisible to the computers attached to IMPS.   That included Internet-capable computers such as gateways and hosts running TCP. The IMP internal mechanisms could change without the need for TCP/IP code to change at all.   Mechanisms in TCP and gateway implementations needed "rough consensus and running code" within a much larger and geographically scattered community.
> 
> TCP moved the mechanisms for creating virtual circuit behavior into the users' computers.   That meant that the internal mechanisms operating in the ARPANET had to have some kind of analogous mechanism inside the users' computers' TCP implementations as well as the gateways.
> 
> Of course such mechanisms would have to be well documented so that all TCP implementations would behave correctly.   The ARPANET approach of having all implementers in close proximity was not practical for The Internet.
> 
> We didn't know how to do that.   It required further research. Meanwhile, PUSH and URGent and TOS and other such "placeholders" would hopefully be enlightening as The Internet Experiment continued.
> 
> It's 2025 now.  I still can't stop my printer when I accidentally send it the wrong document to print.
> 
> Hope this helps explain a bit of the History.
> 
> Jack Haverty
> 
> 
> 
> 
> On 5/15/25 06:15, David Finnigan via Internet-history wrote:
>> Following is something of a short essay that I researched and wrote in March. Posting here because it may invoke some comment or interesting discussion.
>> 
>> Correction and clarifications always welcome, too! :-)
>> 
>> ---------
>> 
>> What to do with the TCP PUSH flag?
>> 
>> Yesterday I spent a lot of time researching the TCP PUSH flag, its purpose, and whether it has any relevance today. The PUSH flag evolved out of the End of Letter (EOL) flag during the development of TCP in the late 1970s and early 1980s. RFC 793, dated September 1981 tells us that "A push causes the TCPs to promptly forward and deliver data up to that point to the receiver. The exact push point might not be visible to the receiving user and the push function does not supply a record boundary marker." And further summarizes that "The purpose of push function and the PUSH flag is to push data through from the sending user to the receiving user. It does not provide a record service."
>> 
>> Later on, RFC 793 describes the TCP Send function as containing a PUSH flag argument. When this argument is set, "the data must be transmitted promptly to the receiver, and the PUSH bit will be set in the last TCP segment created from the buffer.  If the PUSH flag is not set, the data may be combined with data from subsequent SENDs for transmission efficiency." In its description of a TCP Receive call, the document states that the PUSH flag may be returned as part of this call.
>> 
>> That was the specification in 1981, but the varying implementations differed. Some implementations of TCP would not return a buffer of received data to the reading application if the buffer were not completely full. The TCP would only send a partial buffer of received data if the PUSH flag accompanied it. The TCP and SMTP implementations on TOPS-20 were known to have this behavior in the early 1980s.
>> 
>> 
>> -- Berkeley's Implementation of PUSH --
>> 
>> The Berkeley TCP/IP implementation took a different approach. Received data that fits in the receive window is always made available to a reading socket, regardless of whether the PUSH flag is set or not. In fact, for many years, the BSD TCP did nothing at all with an incoming PUSH flag, other than to clear it when trimming a segment to fit the receive window.
>> 
>> The rationale, so far as I can surmise, is because Berkeley's TCP and sockets interface do not withhold any received, valid, in-window and in-order data from a receiving application.
>> 
>> Upon output, BSD only set the PSH bit in the segment header if it emptied the socket's send buffer. There was no user-accessible mechanism to set or clear the PUSH flag upon sending data, nor was the application provided any means to detect whether the PUSH flag had been set for incoming data.
>> 
>> 
>> -- RFC 1122 makes user-control of the PUSH flag optional --
>> 
>> RFC 1122, issued October 1989, clarified the PUSH flag in its section 4.2.2.2, stating that "A TCP MAY implement PUSH flags on SEND calls.  If PUSH flags are not implemented, then the sending TCP: (1) must not buffer data indefinitely, and (2) MUST set the PSH bit in the last buffered segment (i.e., when there is no more queued data to be sent)."
>> 
>> Just as RFC1122 made a user-accessible PUSH flag an optional part of the Send call, this RFC also made signaling a PUSH flag to a receiving application optional, saying "Passing a received PSH flag to the application layer is now OPTIONAL."
>> 
>> BSD's implementation and use of TCP's PUSH function was thus standardized.
>> 
>> Thirty years later, the PUSH flag is still largely ignored by BSD-derived TCP implementations. With just one exception of which I am aware. The TCP in Mac OS X has a congestion control module that uses the PUSH flag as part of a heuristic to control delayed ACKs. The subroutine tcp_cc_delay_ack() in file tcp_cc.c has a comment reading "If TH_PUSH is set, take this as a clue that we need to ACK with no delay. This helps higher level protocols who won't send us more data even if the window is open because their last "segment" hasn't been ACKed."  NetBSD has a similar ACK-on-PUSH option which will refrain from delaying an ACK to an incoming segment which has the PSH bit set.
>> 
>> Those, so far as I know, are the only practical uses of the PUSH flag on incoming TCP segments in BSD-derived TCP implementations.
>> 
>> So, for my implementation of TCP, what should I do with the PUSH flag? Is it, like the URGENT flag, effectively deprecated? Is it just a relic of the early 1980s? Or should I just model Berkeley TCP's use of it, setting the PSH bit automatically when transmitting segments?
>> 
>> 
>> ---------
>> 
>> -David Finnigan
> 
> -- 
> Internet-history mailing list
> Internet-history at elists.isoc.org
> https://elists.isoc.org/mailman/listinfo/internet-history
> -
> Unsubscribe: https://app.smartsheet.com/b/form/9b6ef0621638436ab0a9b23cb0668b0b?The%20list%20to%20be%20unsubscribed%20from=Internet-history