Do LLMs understand their output?

You will often read statements to the effect “LLMs do not understand what they produce as output”. My controversial view is that, in a way, we do not understand what we are saying. Consider this. When we speak words flow from us in a stream without ratiocination, that is, straight from our neural network. If stopped and asked to justify what we have said we are able to create a plausible answer. We believe we understand in the sense of always being able to justify our statements and answer questions about the topic and we can test if someone really understands a subject using a viva voce.
We believe we have a logical conceptual model of the world from which our statements flow but we actually have an impenetrable neural network. I maintain that our accessible mental models are a thin veneer of conceptual deep structures but the real work is done by an inaccessible neural network. This means that at some level our explanations and justifications are illusory, they are constructed post hoc. In the same way, LLMs can, and will become better at, explaining their output and they thus demonstrate their understanding. If an LLM can pass a doctorate level viva voce then can we plausibly argue that it does not really understand the subject?
I am not arguing that LLMs are a model of our neural structures, far from it. They are potentially much more powerful as we have many well-known limitations, our short-term memory is tiny, our ability to analyse deeply structured logic is very limited and our speed of thought is slow. I believe it is a mistake to argue that LLMs are not conscious, are not sentient and do not have feelings and emotions. All this may or may not be true but it is irrelevant to everyone except philosophers. The key questions is are they useful and beyond that can they perform at a level that exceeds any human. If so, we have a very useful tool, to say the least, that will amplify all that is good and all that is bad about our human species.

Improving Large Language Models

The rules of syntax are well known but the rules of semantics (beyond predicate calculus) are not and this limits the capability of Large Language Models (LLMs). My PhD thesis, “A Conversation Problem Solving System Represented as Procedures with Natural Language Conceptual Structure” (www.shafe.co.uk/wp-content/uploads/PhD-Thesis-A-Conversational-Problem-Solving-System.pdf) attempted to approach this problem. It was based on the early work of Roger Schank (en.wikipedia.org/wiki/Roger_Schank) on “Primitive Concepts Underlying Verbs of Thought” (apps.dtic.mil/sti/tr/pdf/AD0744634.pdf) explained more fully in Chapter 5 of “Computer Models of Thought and Language”(ed. R. Schank, 1973, W.H. Freeman).
The interesting aspect of his paper is that he reduced the number of verbs to about 10 (later increased to seventeen). This is profound as it results in a semantic grammar that represents the underlying meaning of a statement. Research into this area progressed slowly over the years and neural networks took centre stage. Now seems to be the time to combine the best of both approaches.
My idea is to process the input during tokenisation and produce a semantic set of tokens, à la Schank, with each verb expanded into its appropriate linear semantic structure. The neural network weights would then be adjusted based on the meaning rather than the text and the system could be said to understand what it was outputting.
Note that nouns can be handled using, for example, Word2Vec, but similar verbs are not close in a multi-dimensional space because they require a complex structure to represent their meaning. For example, “I took the money” and “I gave the money” are, according to Schank, something like “I TRANS money from place1 to myself” and “I TRANS money from myself to place2”. By explicating verbs we reveal their underlying structure and meaning and if we feed that into the neural network then it should be faster to learn and better ‘understand’ the data and so produce better outputs.
I asked ChatGPT, “Human understanding partly requires a knowledge of the way verbs change the world. If all verbs could be processed into a structure that represented such knowledge then LLMs would learn more quickly and have a better understanding of the world. Roger Schank proposed such a structure in his paper “Primitive Concepts Underlying Verbs of Thought”.”
It replied, “You’re right about the importance of understanding how verbs can change or reflect states in the world, and Roger Schank’s work in conceptual dependency theory addresses this aspect. In his paper “Primitive Concepts Underlying Verbs of Thought,” Schank proposes a structure for categorizing and understanding verbs, especially those related to cognitive processes, in a more systematic and meaningful way.
Schank’s approach involves breaking down verbs into primitive actions or concepts. This methodology aims to represent the essential meaning of verbs in a simplified, universal form. By doing so, it becomes easier for a system (like a language model or an AI) to understand and process the implications and nuances of different actions or states described by verbs.
Incorporating such structured knowledge into the training of Large Language Models (LLMs) could indeed enhance their understanding of language and the world. By having a more nuanced grasp of how verbs signify changes or states, LLMs could make more accurate inferences and generate more contextually appropriate responses.
However, implementing this approach in LLMs would require a significant amount of structured data and sophisticated algorithms to integrate this structured understanding of verbs with the model’s broader language understanding capabilities. It’s a challenging task but one that could lead to significant advancements in the field of natural language understanding.
Schank’s work is a pivotal contribution to the field of AI and language understanding, highlighting the importance of deep semantic processing in achieving more human-like understanding in machines.”
I believe the task is challenging but straightforward as the number of verbs in the LLM’s dictionary is finite and each one has fixed linear semantic structure that can replace it. The complication is that the structure will contain placeholders that must be filled by carrying out a syntactic analysis of the sentence.

Ancient Egyptian Art


The ancient Egyptian civilisation lasted for over 3,000 years. In fact, we are today nearer in time to Cleopatra than she was the the building of the pyramids. The enormous timescale can be subdivided into the:
  • Early Dynastic period (c. 3150-2613 BCE)
  • Old Kingdom (c. 2613-2181 BCE)
  • First Intermediate Period (2181-2040 BCE)
  • Middle Kingdom (2040-1782 BCE)
  • Second Intermediate Period (c. 1782-1570 BCE)
  • New Kingdom (c. 1570-1069 BCE)
  • Third Intermediate Period (c. 1069-525 BCE)
  • Late Period (525-332 BCE)
  • Ptolemaic Period (323-30 BCE)
The commemorative slab shown below is from the Middle Kingdom shows a son on the right bringing food and drink to his parents Khety and Henet on the left. The surprising thing to us is that this is tomb art. His parents are dead, so why is he bringing the food and drink including a massive calf’s leg that he is holding out to them. Most ancient Egyptian wall art is found in tombs and was never designed to be seen by the living. It is there to assist the dead to enter paradise, known as the Field of Reeds. When you consider that they thought a person consisted of multiple parts that had to be assisted into the world of the gods then the complexity is easy to understand. The Book of the Dead consisted of spells and procedures to assist the journey which involved the weighing of the heart by Anubis, the jackal-headed god. If the person had lived a decent life and their heart weighed less then the feather of Matt, the goddess of truth and justice, then they were worthy to live forever in paradise with Osiris.
For more information on this mysterious and intriguing period my YouTube video on ancient Egyptian art is here:

William Beechey, ‘Portrait of Sir Francis Ford’s Children Giving a Coin to a Beggar Boy’, exhibited 1793

Sir William Beechey (1753-1839), Portrait of Sir Francis Ford's Children Giving a Coin to a Beggar Boy, exhibited 1793

Sir William Beechey (1753-1839), Portrait of Sir Francis Ford’s Children Giving a Coin to a Beggar Boy, exhibited 1793

The 1790s were a time when society and the arts debated the relative merits of good sense and reason as opposed to sensibility. ‘Sensibility’ was a heightened awareness of beauty and deep feelings. Jane Austen’s Sense and Sensibility was published in 1811 and was set in the same decade as this painting. We see this heightened sense of feeling takes the form of a fashionably dressed girl and her brother handing a coin to a hunched over beggar. Such emotional scenes of charity had become fashionable at this time but there might be another justification for this scene.

Pro-Slavery Propaganda?

The artist, William Beechey, did not identify the children when he exhibited the work but they were recognised by Horace Walpole, son of the first British prime Minister, Robert Walpole. They are Francis Ford and his sister Mary, the children of Sir Francis Ford, who owned a sugar plantation in Barbados and a property in Ember Court, Thames Ditton, Surrey.

While William Wilberforce was fighting to stop the British slave trade there were many wealthy plantation owners, like Ford, who were arguing that their slaves enjoyed a better quality of life than the poor in England. It is possible therefore that he commissioned this painting not just to show the refinement and sensibility of his children but as pro-slavery propaganda, to show how his family cared for both the poor in England and his slaves in Barbados. Although not connected, Ford fell ill the year this painting was exhibited and died in 1801 after eight years of suffering from ‘a severe and painful illness’.

Finally, following a twenty-year campaign for abolition Parliament passed the Slave Trade Act in 1807. Many believed this would end slavery but it only slowed the trade and slavery continued for a further twenty-six years. The Royal Navy enforced the ban on the slave trade and between 1808 and 1860 they seized approximately 1,600 slave ships and freed 150,000 Africans who were aboard.

William Beechey

William Beechey was a sound artist known for his lack of extravagance which appealed to those like King George III, who regarded other artists, such as Thomas Lawrence, as too flamboyant and ‘eccentric’. When this painting was exhibited the well-known portrait painters Thomas Gainsborough and his rival Sir Joshua Reynolds had both died and the year was a turning point in Beechey’s life and career. He painted a full-length portrait of Queen Charlotte who appointed him her official portrait painter and he was elected an associate of the Royal Academy. He went on to paint not only the royal family but most of the famous and fashionable of the period. The up-and-coming portrait painter was Thomas Lawrence who also painted Queen Charlotte but she did not like the picture. Lawrence’s flamboyant approach appealed to many including the Prince Regent but Beechey remained the favourite of George III and Queen Charlotte while Lawrence went on to become the leading portrait artist of the period.

References

http://www.tate.org.uk/art/artworks/beechey-portrait-of-sir-francis-fords-children-giving-a-coin-to-a-beggar-boy-t06734

Thomas Gainsborough, ‘Giovanna Baccelli’, 1782

Thomas Gainsborough (1727-1788), 'Giovanna Baccelli', exhibited 1782, photo: © Tate, London, 2017

Thomas Gainsborough (1727-1788), ‘Giovanna Baccelli’, exhibited 1782, photo: © Tate, London, 2017

If you have visited Knole House near Sevenoaks, Kent, then you might have noticed a nude woman reclining on a chaise longue at the bottom of the stairs. This is a sculpture of Giovanna Baccelli by Giovanni Battista Locatelli. Baccelli was principal ballerina at the King’s Theatre Haymarket and mistress of John Frederick Sackville, 3rd Duke of Dorset, the owner of Knole. He was a handsome, extravagant man with a string of mistresses but he thought highly of Baccelli and set her up in a suite of rooms at Knole. A few years later he commissioned Thomas Gainsborough to paint her full-length in a dynamic pose with the costume and heavy make-up she wore for one of her roles. When the painting was exhibited at the Royal Academy in 1782 an embarrassing incident was narrowly avoided. Gainsborough had also painted the Duke and the picture was going to be shown at the same exhibition. To exhibit a Duke alongside his mistress would have been a flagrant breach of decorum and the Duke’s portrait was withdrawn, we presume for that reason.

When the Duke was appointed ambassador to France, Baccelli accompanied him to Paris and she was a great success as a ballet dancer at the Paris Opéra. It is said that when he was awarded the Order of the Garter she performed with the blue ribbon of the Garter tied around her head. They became friends of Queen Marie-Antoinette but as the French Revolution unfolded they returned to Knole. They separated the same year but remained friends and the Duke married a respectable heiress with a dowry of £140,000. According to an inventory of Knole in 1799, the statue of Baccelli was rechristened ‘A Naked Venus’ and moved to a less prominent position and she was given an annual pension of £400 and left her son to be brought up by the Duke as a gentleman.

The painting by Gainsborough is now at Tate Britain and is a wonderful example of his mature style with its lightly flicked brushstrokes and immediate sense of delicate animation. Accounts at the time describe her as charming rather than beautiful and her character was thought to have been captured by Gainsborough. As one contemporary critic wrote, the portrait was, ‘as the Original, light airy and elegant’.

References
http://www.nationaltrustcollections.org.uk/object/130059
http://www.tate.org.uk/art/artworks/gainsborough-giovanna-baccelli-t02000

Artificial Intelligence and Go

Go is the oldest game in existence, it was invented in China 2,500 years ago. The rules are simple but the strategic thinking required to play well led many people to claim that a world-class human player would never be beaten by a computer. However, in 2016 one of the best Go players in the world was beaten by a computer program called AlphaGo. AlphaGo is an example of Artificial Intelligence (AI).

The Approach

The number of possible Go games is so vast that the number of atoms in the universe and the age of the universe look tiny in comparison. Therefore, for a computer to learn to play through brute force, that is by examining millions of games, seems pointless as countless trillions of games are less than a pin-prick in the vast universe of possible games. Yet, that is exactly what AlphaGo did, it examined thousands of games played by humans and then played millions of games against itself and from that blind analysis, without any strategic thinking, it achieved a level of play beyond any human player.

Deep Mind

AlphaGo was produced by a company in London called Deep Mind, which was acquired by Google in 2010 and is now part of the Alphabet group. The latest version of AlphaGo is called AlphaGo Zero and it takes the program to a new level of capability. AlphaGo Zero started with just the rules and no prior knowledge of the game. It did not analyse any human games and only played itself yet within three hours it played like a human beginner, within 19 hours like a human expert and within 70 hours it surpassed all human players, and it never stops learning.

The next step is to apply the same approach to solving other complex problems, such as finding new drugs, diagnosing diseases, reducing energy consumption or searching for revolutionary new materials. We can only speculate about its ability to solve other problems such as weather forecasting, economic forecasting, predicting human behaviour and reading human thoughts. The latest AI research work at Google has found a way to use the same mechanism to solve many types of problem and to combine different problem solvers in order to tackle a wide variety of problems.

The Problem with Neural Nets

AlphaGo Zero and most of the other recent artificial intelligence (AI) systems are based on neural nets and suffer from one major problem, they are unable to explain their actions. Neural nets are impenetrable, like the human brain. Why do I say, ‘like the human brain’ when our thoughts appear open to us. When we are asked ‘why’ we can give a considered response that explains our reasoning. However, is this what is happening? Back in 1976 in my PhD thesis I speculated that there are two mechanisms at work. There is an underlying brain machine that analyses our environment, controls our body and makes decisions and there is a separate but integrated language system that believes it is in control and uses language to construct reasons why the other part of our brain has done something. The combination can model the world and forecast the future better than the brain system on its own. When we are asked ‘why’ it is our language system that constructs a set of reasons to explain what we have done or said.

Free Will

As a footnote, you may be wondering about free will. Well, that depends on what you mean. If you mean the ability to make decisions and control one’s behaviour without constraint and in response to reason then that can be achieved through the combination of the brain machine and the language system. However, generally our actions and decisions are constrained by our unique genetic makeup and the circumstances of our upbringing but this does not free us from responsibility or remove the appropriateness of praise or blame. We take what we consider to be the best action in all the circumstances of our individual makeup. So we are predisposed free agents, as is AlphaGo Zero.

A Go Board

A Go Board

See The Economist, 21 October 2017, pp. 80-81 and New Scientist, 21 October 2017, p. 9.

Sonnet 116

William Shakespeare

Let me not to the marriage of true minds
Admit impediments. Love is not love
Which alters when it alteration finds,
Or bends with the remover to remove:
O no; it is an ever-fixed mark,
That looks on tempests, and is never shaken;
It is the star to every wandering bark,
Whose worth’s unknown, although his height be taken.
Love’s not Time’s fool, though rosy lips and cheeks
Within his bending sickle’s compass come;
Love alters not with his brief hours and weeks,
But bears it out even to the edge of doom.
If this be error and upon me proved,
I never writ, nor no man ever loved.

Sea Fever

John Masefield (1878-1967), Poet Laureate 1930-1967, from Salt-Water Ballads (1902)

I must go down to the seas again, to the lonely sea and the sky,
And all I ask is a tall ship and a star to steer her by;
And the wheel’s kick and the wind’s song and the white sail’s shaking,
And a grey mist on the sea’s face, and a grey dawn breaking,

I must go down to the seas again, for the call of the running tide
Is a wild call and a clear call that may not be denied;
And all I ask is a windy day with the white clouds flying,
And the flung spray and the blown spume, and the sea-gulls crying.

I must go down to the seas again, to the vagrant gypsy life,
To the gull’s way and the whale’s way where the wind’s like a whetted knife;
And all I ask is a merry yarn from a laughing fellow-rover,
And quiet sleep and a sweet dream when the long trick’s over.

Do not go gentle into that good night

Dylan Thomas (1914-1953), originally published 1951

Do not go gentle into that good night,
Old age should burn and rave at close of day;
Rage, rage against the dying of the light.

Though wise men at their end know dark is right,
Because their words had forked no lightning they
Do not go gentle into that good night.

Good men, the last wave by, crying how bright
Their frail deeds might have danced in a green bay,
Rage, rage against the dying of the light.

Wild men who caught and sang the sun in flight,
And learn, too late, they grieved it on its way,
Do not go gentle into that good night.

Grave men, near death, who see with blinding sight
Blind eyes could blaze like meteors and be gay,
Rage, rage against the dying of the light.

And you, my father, there on the sad height,
Curse, bless, me now with your fierce tears, I pray.
Do not go gentle into that good night.
Rage, rage against the dying of the light.


One Interpretation

Thomas wrote this poem for his dying father. It has the strict form of a villanelle consisting of five stanzas of three lines and a final stanza of four lines.

The narrator is suggesting the way we should respond to the inevitability of death. Thomas has divided people into four types—wise, good, wild and grave—and the implication is that these are all the possible types of humanity. That is, some people are essentially rational in their approach to the world, some are moral, some are action orientated and some are serious.

People who approach the world rationally have thought about death and know that it is inevitable, a requirement for life to have evolved. However, because nothing they ever said or wrote made any difference (‘forked no lightning’), when death comes they do not accept it as inevitable but fight against it.

Good people see with hindsight that their moral acts were as nothing (‘frail’) and that if they had been better those deeds might have been ‘bright’. However, even then they would only have ‘danced in a green bay’, suggesting a bright flash of sunlight on the breaking crest of a wave. So, even the best people’s deeds last a fraction of a second and therefore they do not go gently towards death.

Action orientated people (‘wild men’) may have performed the most amazing feats (catching the sun in flight) but even they see, looking back on their lives, that it just resulted in sadness and grieving so they also fight the coming of death.

Serious people understand, too late, that their lives could have been different and they could have seen the world as a bright and happy place. But it is too late and so they also rage against death approaching.

Thomas then discusses his father. Someone who has been sad, perhaps a serious person, a ‘grave’ man. He wants him now to react strongly towards his son, whether it is cursing or blessing him, and through that reaction fight against the inevitability of death.

The Fermi Paradox

The Fermi paradox is the puzzling conflict between the high likelihood of finding alien life or signs of alien life on earth and the fact that we have found none.

There are many explanations for this discrepancy (see Wikipedia Fermi paradox) but they ignore one important factor—self-replicating spacecraft. If we could build a spacecraft that made a copy of itself after it had landed then it can be shown that our galaxy, despite its vast size could be filled with such spacecraft in a few tens of millions of years.

The key question is whether it will ever be possible to build a spacecraft that could replicate itself. Such a spacecraft, although we could not build one today, does not appear to require any breakthrough that is beyond the bounds of current science. If the human race could build such a device ‘soon’, that is within say a thousand years, then thousands, possibly millions, of other civilizations must have been able to build such devices before now. The earth should therefore be full of self-replicating spacecraft.

I can hear you raising many objections but I think they can all be answered. This leaves the ‘Shafe paradox’ :), a subset of the Fermi paradox, ‘where are all the self-replicating spacecraft?’.

Objections Answered

  1. The galaxy is very big so why do you say it could be filled in only a few tens of millions of years?
    1. Voyager I would take about 80,000 years to reach our nearest star Proxima Centauri which is 4.2 light years away. Assume that it is possible to build a self-replicating spacecraft (SRS) that is eighty times faster, about 1,300 kilometers per second, then it would take about 1,000 years. Our galaxy is about 100,000 light years across so it would take 24 million years to cross the galaxy. How many stars have planets or asteroids containing the material to replicate a SRS? If we assume one in 1,000 stars then our SRS will replicate itself every 1 million years. If our SRS makes just 10 copies each time then after 12 replications the number of copies travelling across our galaxy will exceed the number of stars in our galaxy, roughly 300 billion. After 100 million years our entire galaxy will have turned into self-replicating spacecraft.
  2. Is it possible for a spacecraft to survive in space for a million years?
    1. Not easily. It is possible a small craft could travel faster and it is possible more stars than one in a thousand have suitable material orbiting them. Another solution is that the self-replicating spacecraft is also self-repairing as the two requirements are similar. With suitable spare material and with multiple copies of each part when damage is detected in one copy that copy could be repaired.
  3. Why would any intelligent lifeform build such a device?
    1. Because it is possible. If only a small percentage of lifeforms built such a device then we should see them on earth.
  4. After landing on a planet to replicate could the spacecraft reach escape velocity?
    1. It may be sensible to never land on a planet as large as earth but to use the material from asteroids to replicate. We should see evidence of this is we examine asteroids closely.

 

 

Blue and Black Dress or White and Gold, an Explanation

Gold and white or blue and black dressSome people see this dress as white and gold and others see it as blue and black. The dress is actually blue and black but this is irrelevant as the key question is why do some people see the dress in this photograph as one combination of colours and other people see it as another combination?

The best explanation I have found is that we have all evolved to remove the colour of the light around us. In the early morning and late evening the light is reddish and at midday it is blueish. So to enable us to see the ‘real’ colour our visual cortex automatically removes the colour cast of the light falling on an object before we become conscious of the colour.

In the early morning our visual cortex removes red and at midday it removes blue. What happens when we look at a photograph as there is no way of judging the colour of the light when it was taken? It seems some people are born to automatically remove red and some blue. One way of describing this is to say that some of us are midday people and some evening/early morning people. The visual cortex of midday people removes blue and so those people see the dress as white and gold but evening/early morning people remove red and so see the dress as blue and black.

28 February 2015