ChatGPT and the Web (not), Citations, &c.

Noting that Bing may soon integrate ChatGPT (Microsoft aims for AI-powered version of Bing – The Information [Reuters]), we can only hope they sort out how URLs are parsed…

It got the PM wrong, but perhaps that’s because its training biases it to Johnson?

My querying is really sloppy here, and doesn’t really check whether ChatGPT is getting content from the page or not… Which in part goes to show how beguiling all this stuff can be and how it’s so easy to to make so many assumptions, as the apparent fit of the responses to the prompts takes you along with it (as you’d expect: the model chucks out the next token based on what it’s likely to be given all the historical training sentences that have been used to build the model).

Okay, so maybe it isn’t reading the page, it’s just parsing the URL and using the words from the page slug to prompt the faux summary? [That said, as Phil Bradley pointed out in a comment, the name of the PM isn’t actually mentioned in the linked to post. Also, as @arosha pointed out, the maths thing has been trailled in at least one news report from August 2022, although that is past the model cut-off point.] Let’s try it with a made-up URL:

Okay, so it seems to claim that it doesn’t recognise that URL. David Kane tried something less contentious, and did get a response based around a different made up URL:

So maybe the “plausibility” of the URL is relevant?

With a bit of fiddling, we can get a response where ChatGPT implies it can’t access the web:

If we are referring to URLs in Bing prompts, and the search engine is coming up with responses based on page indexes, whereas the ChatGPT component is hallucinating indexes based on the prompt and the terms in the URL, then, erm…, WTF? (For a quick take on current search engine + GPT3 integrations, see Combining GPT3 and Web Search — and

Elsewhere in the blogoverse, I notice that D’Arcy has also been playing with ChatGPT — ChatGPT Designs a University Learning Space Design Evaluation Plan — and spotted that ChatGPT is happy to make up plausible sounding but non-existent citations to strengthen the form of its response.

I’ve noticed that when trying to get ChatGPT to make up references (eg Information Literacy and Generating Fake Citations and Abstracts With ChatGPT), it often uses actual (and relevant) journal titles, the names of actual authors (and author combinations) and plausible titles. So… I wonder.. If ChatGPT makes up a citation claiming me as the author in some sort of plausible context, and an author then includes it in a published, peer reviewed work from a commercial publisher, and I get taken through an academic disciplinary committee because some sort of citation harvesting engine has picked up the fake citation and that citation harvester output somehow finds it way back into a reputation manageet system my institution is using and I am “rumbled” for making up fake citations, who do I sue?

I’ve noticed that ChatGPT does have post-processor filters that can flag content warnings, so should it also be providing an optional “fake citation” filter to highlight fake citations? There could also be value in identifying real authors and the “sort of” paper title they might publish, or the sort of journal they are likely to publish in, even if the actual paper (or even the actual journal) doesn’t exist. Do citation managers such as Zotero provide existence check tools so users can check that a citation actual exists, rather than just ensuring stylistic correctness for a particular citation format?

If Bing is to incorporate ChatGPT, and generate novel texts as well as returning links to third party texts, how will it filter out generated responses that are essentially bullshit? Particularly if it is rating or ranking the generated response (which is generated from indexed content) against the content pages that contributed to the underlying model?

And finally, there has been a reasonable amount of traffic on the wires with folk asking about what the effect on education and assessment is likely to be. Whilst “everyone” has been talking about ChatGPT, I suspect most people haven’t, and even fewer have signed up to play with it. If ChatGPT gets incorporated into Bing (or Google incporporates its own LLM into Google search), then the content will be just another content option for students pasting questions into the search box to copy and paste from. More “deliberate” use might result from incorporation into MS Word, eg as a Grammarly service [hmm, I wonder what percentage of OUr students use Grammarly, and whether we can detect its use?].

PS Thinks: just like Amazon spots popular products from its search and sales logs and then releases undercutting or competitively priced and highly ranked own-brand alternatives, is it hard to imagine a search engine that uses something like Common Crawl for a base level of web search, but also mints URLs and auto-generates content pages on-the-fly in response to queries that it (legitimately) ranks highly and pops a few ads onto, to give the appearance that the result is on a “legitimate” web page?

PPS TIme to read Richard Gregory’s Mind In Science again, I think, and wonder what he would have thought about LLMs…

Author: Tony Hirst

I'm a Senior Lecturer at The Open University, with an interest in #opendata policy and practice, as well as general web tinkering...

4 thoughts on “ChatGPT and the Web (not), Citations, &c.”

  1. I took a look at the maths to 18 article and while it obviously talks about ‘the PM’ at no point does it actually name him. The tool (ChatGPT that is, not the PM) was trained on data that was current to 2021, so it’s going to make the assumption, based on its knowledge that the reference to ‘PM’ is going to be Johnson, not Sunak.

    What I found interesting was the when I asked it for a summary of a BBC article and giving it an exact URL it gave me a summary of an entirely different article. Twice. And when I told it that it had it wrong it apologised and gave me a new summary, which was also wrong. To be honest, this doesn’t surprise me and I’m not particularly worried about it. The data it’s using is to all intents and purposes ancient, and it’s only in a sandbox anyway. It’s still got a lot of weaknesses, so I wouldn’t be overly concerned at your hypothetical citation issue. Worry about it when a system goes live!

    1. Good point re: no mentio of PM name in the announcement. Even so, I don’t think it is looking at the content of the web page. I donlt think the model embeds literal indexes of old web pages, does it (not done a test of that to see if I can pull out a web page from an old, unchanging URL)? Did the BBC page summary reflect any of the words in th URL that it could have hallucinated around?

      1. No, I tried a few searches to see if I could identify where it pulled the content from but no luck. I recently asked it to write a tourist brochure for where I live, Billericay near Basildon. It was a very good brochure with the exception that it talked about a historic building in Lower Basildon Park, which is actually in Reading. I informed it, and it apologised and immediately re-wrote the offending paragraph! It did make me laugh out loud. It’s obviously still making random jumps and assumptions that one Basildon is the same as a completely different one. Currently I wouldn’t trust it with anything factual that was important, and I’ll still be fact checking it for a long time to come. Most people however will blindly accept what they are given I suspect. For me, that’s the far bigger question and concern.

        1. Re: the concern – agreed; situating ChatGPT within a search engine context, where you might expect discovery of hopefully relevant resource (with the caveat you should still check provenance, quaity etc) rather than hallucination of something that sounds plausible that may still be nonsense.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: