[ih] TCP PUSH flag

[Vint Cerf]

I would have thought this debate was over by 1978 or so.

v


On Fri, May 16, 2025 at 3:38 PM John Day via Internet-history <
internet-history at elists.isoc.org> wrote:

> Jack,
>
> Wasn’t earlier than the 80s?  I remember it being mixed up with trying to
> optimize the character-at-a-time nature of TENEX Telnet.  Mixed up with
> Remote Control Transmission and Echo (RCTE).
>
> John
>
> > On May 16, 2025, at 14:10, Jack Haverty via Internet-history <
> internet-history at elists.isoc.org> wrote:
> >
> > The "PUSH" mechanism was part of a larger issue that was discussed and
> debated in the early 1980s, while we had The Internet on the operating
> table to replace TCP V2 with TCP/IP V4.  I don't recall exactly what we
> called that issue - something like "Out-Of-Band Control" (OOBC).
> >
> > OOBC was one of the items that the ICCB put on the "things we have to
> figure out" list, for more research to figure out exactly what to do and
> get it into the next release.
> >
> > The basic issue of OOBC was the need for some mechanism to "interrupt"
> an ongoing interaction between the two end-users of a TCP connection.
>  When a connection was in progress, significant data might be buffered "in
> the pipe", and the apps communicating over TCP often were unwilling to
> accept more data from the pipe as they worked to complete a previous task.
> >
> > This was a real practical problem.  One memorable example involved the
> simple use of printers, such as one that might be attached to a TAC port,
> printing a document arriving via TCP from some computer out on the 'net.
> >
> > It was a common, and annoying, experience to accidentally send a
> document which wasn't a text file to be printed, such as some kind of
> graphics document, core dump, or other "data" file.  At the time, printers
> that could print diagrams or pictures were quite rare (the Xerox XGP being
> one example).   Most printers only printed characters, either very slowly
> (a mod 33 TTY or a TI700 terminal) or very quickly (any of the big
> high-speed printers attached to a PDP-10 or similar CPU).
> >
> > If you accidentally sent a non-text file to such a printer, it tried to
> put it on paper.   But many of the bytes being "printed" would be control
> codes - e.g., octal 014 was "form feed".  That caused a printer to advance
> to the next page.   Octal 015 was "carriage-return".  Octal 012 was "line
> feed".
> >
> > A graphics file with lots of 015s but few 012s would print lines of text
> on top of previous lines.  After a few such lines, the paper would shred at
> that line, and then create a huge mess inside the printer as it kept
> feeding paper.
> >
> > Or a file with lots of 014 bytes could consume a thousand feet of paper
> in seconds.   Every 014 caused a new page to be rapidly ejected.
> >
> > We were quite familiar with such disasters.  It had even been a problem
> at the demonstration of the ARPANET at the ICCC '72 conference.
> >
> > So, the question was how to stop such chaos - what mechanisms would
> allow a user (or the user's app) to somehow stop such scenarios.
> >
> > The PUSH mechanism provided one approach, telling the recipient TCP to
> stop waiting for more data.  It complemented the URGent mechanism which
> indicated that there was something important coming and the receiving app
> should do something about it.  Perhaps it was an ABORT command, which would
> stop that out-of-control printer.
> >
> > For me at least, at the time I was remembering earlier experiences in
> the 70s implementing electronic mail, and even wrote up some "Thoughts on
> Interactions in Distributed Services" (see RFC722 if you're curious,
> especially the notion of "Request Reply Discipline").
> >
> > The PUSH mechanism provided a way to tell a receiving TCP that it
> shouldn't wait for more data.   But that didn't mean the receiving app
> would be willing to accept it.   The URGent mechanism was intended to tell
> the app that it should accept more data as fast as possible, because there
> was something important coming.  But there was no way for the app to find
> out how that future data should be handled.    E.g., in that printer
> scenario, what would your app code do when it somehow got an URGent
> indication?   How would the human user stop a printout?
> >
> > There were many discussions about how to provide some mechanisms for
> OOBC.  One idea was to simply advise apps to utilize a second TCP
> connection for "control" traffic (FTP did this on the ARPANET).   A
> Type-of-Service (TOS) setting of "Priority" might cause the underlying
> Internet to deliver such "control" traffic more quickly than the bulk data
> being transferred through the main TCP connection - once someone figured
> out how to implement TOS in the datagram infrastructure.
> >
> > Using multiple connections would be of course more complicated to
> implement and coordinate in the app code, so a simpler solution was
> desired.  That needed further research, but the PUSH, URGent, and such
> mechanisms were a way to start.
> >
> > The ARPANET was the main source of experience at the time. Although it
> was a packet-switched network, it had a lot of internal mechanisms that
> created a "virtual circuit" service to the computers attached to it.
> Everything that went in at a source came out at the destination intact,
> complete and in sequence.  In effect, the functionality provided by TCP for
> creating a reliable byte-stream was already also present inside the
> ARPANET, with mechanisms in the source and destination IMPs for each
> traffic flow.  Internal messages, such as ALLOcate, travelled between IMPs
> to implement that mechanism.  Data flow was also regulated to achieve a
> typical maximum latency of 250 milliseconds - dictated by the desire to
> have characters typed by the human user quickly processed by the app
> consuming them.  Scenarios such as "buffer bloat" were not a problem.
> >
> > Those internal mechanisms weren't well known at the time, except by the
> ARPANET implementers at BBN.  The algorithms and protocols used also
> changed as the traffic patterns changed over time.   Technology advances,
> such as the introduction of personal workstations and later PCs, altered
> the way people used the network.  The internal mechanisms similarly changed
> as needed.  I was never an ARPANET implementer, but that group was
> literally right down the hall so I was aware of a lot of the internal
> issues and mechanisms as I worked on TCP.
> >
> > TCP made significant changes to how networks worked.   Some of these may
> not be obvious even now.
> >
> > For example, the internal mechanisms in the IMPs were largely invisible
> to the computers attached to IMPS.   That included Internet-capable
> computers such as gateways and hosts running TCP. The IMP internal
> mechanisms could change without the need for TCP/IP code to change at all.
>  Mechanisms in TCP and gateway implementations needed "rough consensus and
> running code" within a much larger and geographically scattered community.
> >
> > TCP moved the mechanisms for creating virtual circuit behavior into the
> users' computers.   That meant that the internal mechanisms operating in
> the ARPANET had to have some kind of analogous mechanism inside the users'
> computers' TCP implementations as well as the gateways.
> >
> > Of course such mechanisms would have to be well documented so that all
> TCP implementations would behave correctly.   The ARPANET approach of
> having all implementers in close proximity was not practical for The
> Internet.
> >
> > We didn't know how to do that.   It required further research.
> Meanwhile, PUSH and URGent and TOS and other such "placeholders" would
> hopefully be enlightening as The Internet Experiment continued.
> >
> > It's 2025 now.  I still can't stop my printer when I accidentally send
> it the wrong document to print.
> >
> > Hope this helps explain a bit of the History.
> >
> > Jack Haverty
> >
> >
> >
> >
> > On 5/15/25 06:15, David Finnigan via Internet-history wrote:
> >> Following is something of a short essay that I researched and wrote in
> March. Posting here because it may invoke some comment or interesting
> discussion.
> >>
> >> Correction and clarifications always welcome, too! :-)
> >>
> >> ---------
> >>
> >> What to do with the TCP PUSH flag?
> >>
> >> Yesterday I spent a lot of time researching the TCP PUSH flag, its
> purpose, and whether it has any relevance today. The PUSH flag evolved out
> of the End of Letter (EOL) flag during the development of TCP in the late
> 1970s and early 1980s. RFC 793, dated September 1981 tells us that "A push
> causes the TCPs to promptly forward and deliver data up to that point to
> the receiver. The exact push point might not be visible to the receiving
> user and the push function does not supply a record boundary marker." And
> further summarizes that "The purpose of push function and the PUSH flag is
> to push data through from the sending user to the receiving user. It does
> not provide a record service."
> >>
> >> Later on, RFC 793 describes the TCP Send function as containing a PUSH
> flag argument. When this argument is set, "the data must be transmitted
> promptly to the receiver, and the PUSH bit will be set in the last TCP
> segment created from the buffer.  If the PUSH flag is not set, the data may
> be combined with data from subsequent SENDs for transmission efficiency."
> In its description of a TCP Receive call, the document states that the PUSH
> flag may be returned as part of this call.
> >>
> >> That was the specification in 1981, but the varying implementations
> differed. Some implementations of TCP would not return a buffer of received
> data to the reading application if the buffer were not completely full. The
> TCP would only send a partial buffer of received data if the PUSH flag
> accompanied it. The TCP and SMTP implementations on TOPS-20 were known to
> have this behavior in the early 1980s.
> >>
> >>
> >> -- Berkeley's Implementation of PUSH --
> >>
> >> The Berkeley TCP/IP implementation took a different approach. Received
> data that fits in the receive window is always made available to a reading
> socket, regardless of whether the PUSH flag is set or not. In fact, for
> many years, the BSD TCP did nothing at all with an incoming PUSH flag,
> other than to clear it when trimming a segment to fit the receive window.
> >>
> >> The rationale, so far as I can surmise, is because Berkeley's TCP and
> sockets interface do not withhold any received, valid, in-window and
> in-order data from a receiving application.
> >>
> >> Upon output, BSD only set the PSH bit in the segment header if it
> emptied the socket's send buffer. There was no user-accessible mechanism to
> set or clear the PUSH flag upon sending data, nor was the application
> provided any means to detect whether the PUSH flag had been set for
> incoming data.
> >>
> >>
> >> -- RFC 1122 makes user-control of the PUSH flag optional --
> >>
> >> RFC 1122, issued October 1989, clarified the PUSH flag in its section
> 4.2.2.2, stating that "A TCP MAY implement PUSH flags on SEND calls.  If
> PUSH flags are not implemented, then the sending TCP: (1) must not buffer
> data indefinitely, and (2) MUST set the PSH bit in the last buffered
> segment (i.e., when there is no more queued data to be sent)."
> >>
> >> Just as RFC1122 made a user-accessible PUSH flag an optional part of
> the Send call, this RFC also made signaling a PUSH flag to a receiving
> application optional, saying "Passing a received PSH flag to the
> application layer is now OPTIONAL."
> >>
> >> BSD's implementation and use of TCP's PUSH function was thus
> standardized.
> >>
> >> Thirty years later, the PUSH flag is still largely ignored by
> BSD-derived TCP implementations. With just one exception of which I am
> aware. The TCP in Mac OS X has a congestion control module that uses the
> PUSH flag as part of a heuristic to control delayed ACKs. The subroutine
> tcp_cc_delay_ack() in file tcp_cc.c has a comment reading "If TH_PUSH is
> set, take this as a clue that we need to ACK with no delay. This helps
> higher level protocols who won't send us more data even if the window is
> open because their last "segment" hasn't been ACKed."  NetBSD has a similar
> ACK-on-PUSH option which will refrain from delaying an ACK to an incoming
> segment which has the PSH bit set.
> >>
> >> Those, so far as I know, are the only practical uses of the PUSH flag
> on incoming TCP segments in BSD-derived TCP implementations.
> >>
> >> So, for my implementation of TCP, what should I do with the PUSH flag?
> Is it, like the URGENT flag, effectively deprecated? Is it just a relic of
> the early 1980s? Or should I just model Berkeley TCP's use of it, setting
> the PSH bit automatically when transmitting segments?
> >>
> >>
> >> ---------
> >>
> >> -David Finnigan
> >
> > --
> > Internet-history mailing list
> > Internet-history at elists.isoc.org
> > https://elists.isoc.org/mailman/listinfo/internet-history
> > -
> > Unsubscribe:
> https://app.smartsheet.com/b/form/9b6ef0621638436ab0a9b23cb0668b0b?The%20list%20to%20be%20unsubscribed%20from=Internet-history
>
> --
> Internet-history mailing list
> Internet-history at elists.isoc.org
> https://elists.isoc.org/mailman/listinfo/internet-history
> -
> Unsubscribe:
> https://app.smartsheet.com/b/form/9b6ef0621638436ab0a9b23cb0668b0b?The%20list%20to%20be%20unsubscribed%20from=Internet-history
>


-- 
Please send any postal/overnight deliveries to:
Vint Cerf
Google, LLC
1900 Reston Metro Plaza, 16th Floor
Reston, VA 20190
+1 (571) 213 1346


until further notice