[ih] History of AI and Internet

[william yeager]

LOL ! Back propogation is extremely important to assure that as computation takes proceeds on successive layers, that the current layers predecessors are informed and correct what is learned about the current token in R^n at the current layer. N is called the dimension of the LLM, say, 4096. Extremely important. 

I personally think the neuron is a misnomer. It is a mathematically defined relationship between active tokens and the memory. But, an LLM does definitely use a training created, digital version of the human brain. 

I’d forget the differences between real neurons and an LLM’s. Just note that the human brain has an associative memory and a LLM does not. The later also operates on an average of 20 watts ! Now that is astounding. 

I think we also use tokens in the sense that if I want to remember a forgotten name, I just go a, b, c, …, z over and over and various names arrive, and with luck the desired one. I’m sure most of us have done this.

> On Jun 23, 2026, at 2:14 PM, Brian E Carpenter <brian.e.carpenter at gmail.com> wrote:
> 
> On 24-Jun-26 03:10, william yeager wrote:
>> Don’t want to spend too much time on this but apparently you’ve never read John McCarthy’s proposal for naming AI and what AI should do. Here it is:
>> 
>> We propose that a 2 month, 10 man study of artificial intelligence be carried out during the summer of 1956 at Dartmouth College in Hanover, New Hampshire
>> 
>> The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.
> Oh, I am quite familiar with that, and coincidentally I am just reading Matthew Cobb's new biography of Francis Crick, which brings out very clearly that Crick's work on brain function was very clearly contrasted with AI simulation. I suspect that Crick would have violently objected to McCarthy’s conjecture. (Crick wasn't interested in artificial neural networks, because real neurons don't have a backward propagation mechanism.) 
> 
> Regards/Ngā mihi
>    Brian Carpenter
>>> On Jun 22, 2026, at 11:10 PM, Brian E Carpenter <brian.e.carpenter at gmail.com <mailto:brian.e.carpenter at gmail.com>> wrote:
>>> 
>>> On 23-Jun-26 17:25, william yeager wrote:
>>>>> On Jun 22, 2026, at 7:54 PM, Brian E Carpenter via Internet-history <internet-history at elists.isoc.org <mailto:internet-history at elists.isoc.org> <mailto:internet-history at elists.isoc.org <mailto:internet-history at elists.isoc.org>>> wrote:
>>>>> 
>>>>> but expert systems were not real AI
>>>> Rubbish ! (Steven Hawking’s favorite denial). They WERE real AI at the time they existed. 
>>> 
>>> Fair enough, but all the expert systems I saw were basically rules created by human experts.
>> 
>> This is true and the Expert Systems did simulate human thinking. These were not simple rules. Ted Shortliffe’s MYCIN and EMYCIN incorporated rules for diagnosing and prescribing antibiotics. These systems did as well as 90% of physicians.
>> 
>> I personally worked on T-Helper that was used to remotely diagnose and treat AIDs patients and could be used by rural communities that lacked the expertise. The ideas weren’t mine. I helped a Phd student in medical informatics - ie - the student was an MD and wanted to add a Phd in the computational aspect of medicine - the underlying code misused TCP/IP - the latter was buried in the bowels of the large system and as I dug into it I also began to underside the medical side. Amazing software.
>> 
>> The work in Expert Systems covered so many bases and Stanford the a national resource for the work in experimentation in AI and Medicine in the 1970s-90s. 
>> 
>> Yes, Expert Systems did simulate human thinking. This was not about generating stochastic sentences like LLMs do. I really understand the internals of these systems. For me the math was easy since I did my Phd studies in math at U of Washington. I am amazed at the beauty in LLMs and with the arrival in 2017 of the Foundation that enabled deep learning to scale and compute on 20 thousand NVIDIA GPUs in parallel; multiplying token based, weighted matrices on the order of 4096x4096 or larger along with smaller queries, keys and values matrices all created during training, the the embedded matrix 80000x4096 tokens + a few more … the details are a thing of pure mathematical wonder to me. Do note that if one understands  and not just knows advanced multivariate calculus and linear algebra along with standard error calculations and a few other details, they ought to be able to figure out how LLMs work. WHY is more complicated and worth another long discussion.
>> 
>> The basic thing to understand about LLMs is that they are 90% training and 10% execution. They have a fixed memory of about 30 billion parameters and are becoming larger. The do not keep the vast amount of text that is scrapped off of the Internet; They have a fixed vocabulary of about 60 thousand words; and about 80 thousand tokens - the embedded array - on which all computation is based. Tokens are like syllables - pieces of words - have binary values that map to the text. So Bill might be “bi” and “ll”. One knows a token by the friends it keeps. They attract one another in millions of ways. Every query is reduced to a string of tokens and through massive computation and multiple feed back loops of say 50 or so layers, the next best token is produced stochastically from finding and converging to the parameters in memory that represent the best response to the query.
>> 
>> The above is rough … the curious should seek out the details.
>>> 
>>>> I worked in the Stanford Knowledge Systems Lab at Stanford as well as for the DENDRAL project - AI in chemistry. My area of interest was using organic chemistry along with signal processing to analyze GCMS data with peak analysis gradients to detect inherited rare diseases in children from the fractions of these children’s blood. It worked and these were not heuristics. 
>>> 
>>> But were they systems that learned, and that detected features that they hadn't been told about in advance?
>> 
>> I can’t answer the above for all expert systems. But that is not a requirement for AI. Also, LLMs do not KNOW anything. They build stochastic sentences from a fixed memory of parameters. They do NOT think. Rather they COMPUTE. There is NO ONE HOME. And they are explicitly trained to produce sentences that are pleasing to humans. One of the final phases of training is pumping through millions of queries and using           RLHF - Reinforced Learning with Human Feedback to produce grammatically pretty and pleasing sentences. RLHF uses mathematical algorithms. I forget their names. Super cool ! This works for all languages, eg, Chinese tokens are characters - like 你，她，他，们, 好，etc.
>> 
>> I’ll close with Yann LeCun’s description of LLMs and why he’s taking a divergent path to create models for AGI: LLMs regurgitate stochastic sentences ! Yann along with Geoffrey Hinton and Yoshua Benglo received the Turing prize for deep learning in 2018.
>> 
>> 
>>> 
>>>> The spectra that were produced were give to several Phd’s in chemistry to identify the compounds, they were stored in a library for future recognition. Is it AI now? No. Was it then. Yes. What we did was state-of-the-art AI between 1970 and 1990 or so.
>>>> Ed Feigenbaum who is a good friend received the Turing Prize along with Raj Redy for their work on expert systems in 1994.
>>> 
>>> Yes, and I didn't mean to insult all that work (so I should have chosen my words more carefully).
>>> 
>>>> I am currently writing a history of AI from 1955 to the present from the perspective that it was a Gestalt. Have looked into what was AI for each of the multiple versions over the years. I’ve been in computer science and math for 60 years, lived through these, epochs, and believe me there were battles noted in the press initiated by antagonists who were worried that expert systems were going to replace the experts.
>>> 
>>> I never thought that, but it's already happening to a considerable extent with LLMs.
>>> 
>> True - but better said - to help them work and do research more efficiently.
>>  
>> This is true in math too - which I carefully follow. 
>> 
>>>> AI evolved over the years to where we are now and definitely NOT as a SILO.
>>> 
>>> However, multi-layer perceptrons were basically ignored until a few years ago. (That includes me ignoring them when working on speech recognition in the late 1960s, because they were computationally infeasible.)
>>> 
>> 
>> Exactly. We knew about neural nets but they were just toys back then. Everything was mega-x.
>> 
>> Sorry for any of the typos above.
>> 
>> Bye for now. When work is play for someone, all is well and this someone is a happy person (-: As my granddaughter always said, Papi (me) let’s play play play.
>> 
>> Bill
>>> Peace
>>>   Brian
>>> 
>>>> Bill
>>>> ————打🎾————
>>>> 马年快乐🐎
>>>> <Happy year of the horse>
>>>> 保佑众生 🙏🐘🌲🦋🏃‍♀️🐳
>>>> <Bless & Protect all living things>
>> 
>> ————打🎾————
>> 马年快乐🐎
>> <Happy year of the horse>
>> 保佑众生 🙏🐘🌲🦋🏃‍♀️🐳
>> <Bless & Protect all living things>
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 

————打🎾————
马年快乐🐎
<Happy year of the horse>
保佑众生 🙏🐘🌲🦋🏃‍♀️🐳
<Bless & Protect all living things>