Can ChatGPT Respond Using Words of Low Predicted Likelihood?

I wondered as a PS to the previous post (Can ChatGPT Detect GPT3 Generated Texts?) “are we human to the extent that the words we say are not reliably predicted using a large language model?“…

As I understand a large language model, your responses are constructed based on a prediction of the likelihood of the next word in a sentence having a hight probability. Is that correct?

Me to ChatGPT

What would happen if your answer was generated using words that have a very low probability of being used next according to your model. Please give an example.

Me to ChatGPT

Write 200 words on Macbeth using that approach, with words of a low predicted likelihood following next.

Me to ChatGPT

Interestingly, if I specified a specific low probability threshold, wherever I set the limit I got a response similar to the previous one:

Write 200 words on Macbeth using an approach where the next word in the sentence has a less than 0.0000000005% probability of appearing according to your model.

Me to ChatGPT

So I’m not sure what’s going on there? Is the stuff firther up the thread affecting things?

In a new thread:

As I understand a large language model, your responses are constructed based on a prediction of the likelihood of the next word in a sentence having a hight probability. Write 200 words on Macbeth using a generative approach in which the next word in the sentence has, according to your model, a near-zero probability of being the next word in the sentence.

Me to ChatGPT
Macbeth is a tragic play written by William Shakespeare. The play tells the story of a brave Scottish general named Macbeth who receives a prophecy from three witches that he will become the King of Scotland. Consumed by ambition and spurred on by his wife, Macbeth murders King Duncan and takes the throne for himself. However, he soon becomes wracked with guilt and fear, as he is haunted by the ghosts of those he has murdered and tormented by his own conscience. In the end, Macbeth is defeated and killed in a final battle against the forces of good.

Despite his initial success, Macbeth's fate is ultimately one of downfall and despair. He becomes a tyrant, mistreating those around him and losing the respect and loyalty of his subjects. His descent into madness and violence ultimately leads to his own downfall, as he is unable to cope with the weight of his guilt and the consequences of his actions.

Throughout the play, Macbeth is plagued by visions and hallucinations, which serve as a constant reminder of the evil deeds he has committed. These hallucinations ultimately drive him to the brink of madness, as he is unable to escape the consequences of his actions. Despite his best efforts to maintain his sanity and hold onto the throne, Macbeth is ultimately defeated by the forces of good, who seek to restore order and justice to Scotland.

In the end, Macbeth serves as a cautionary tale, warning against the dangers of ambition and the destructive nature of unchecked power. It is a tragic story of a man who is ultimately undone by his own flaws and weaknesses, and serves as a reminder of the consequences of succumbing to temptation and greed.

In repeated trials in new threads, it repeatedly keeps coming up with texts very similar to that.

So I wonder — can we prompt to modify the probabilities used in the selection of the next word or not?

Author: Tony Hirst

I'm a Senior Lecturer at The Open University, with an interest in #opendata policy and practice, as well as general web tinkering...

2 thoughts on “Can ChatGPT Respond Using Words of Low Predicted Likelihood?”

  1. Interesting that I do not get anything close to your answer when using your prompt in a new thread.

    —–ME—–
    As I understand a large language model, your responses are constructed based on a prediction of the likelihood of the next word in a sentence having a high probability. Write 200 words on Macbeth using a generative approach in which the next word in the sentence has, according to your model, a near-zero probability of being the next word in the sentence.

    —CHATGPT—–
    Macbeth is a play written by William Shakespeare that tells the story of a brave Scottish general named Macbeth who receives a prophecy from a trio of witches that he will become the King of Scotland. Despite his loyalty to the current king, Duncan, Macbeth becomes tempted by the idea of power and allows his ambition to consume him. He murders Duncan in order to fulfill the prophecy and become king himself.

    As he becomes more and more paranoid and guilty about his actions, Macbeth begins to descend into madness. He is tormented by the ghosts of those he has killed and becomes increasingly isolated from those around him, including his wife, Lady Macbeth. The guilt and turmoil ultimately lead to his downfall as he is defeated in battle and killed by Macduff, a nobleman who was seeking revenge for the murder of his family.

    Throughout the play, Shakespeare explores themes of ambition, guilt, and the corrupting nature of power. The character of Macbeth serves as a cautionary tale about the dangers of allowing one’s desires to overcome their sense of morality and reason. Despite his initial good intentions, Macbeth’s actions ultimately lead to his own demise and the destruction of those around him.

    1. I think the responses are generated probabalistically whenver they are run, so so you might expect a novel answer to the same prompt run in multiple new threads; if you are deep in a particular thread, I have noticed on occasion that responses can start to approach a fixed point and become repetitive.

Comments are closed.