Demos

Most of these demos ask a model the same thing many times and show what came back. The probability demos run on gpt-4o through the API, because that is the only way to see the probabilities the model gives each word. How the demos were run →

How these models work

• Demo 1: Fill in the blank“Complexity” had about half the probability and came out 13 times in 20.
• Demo 2: Fill in the blank, narrowSame machine, a sentence the text pins down. “Risk,” 20 times in 20.
• Demos 3 and 4: One word, then one sentenceAs a question, the answer spreads out. Two runs of one sentence land on two different research gaps.
• Demo 5: A fabricated citationReal authors, the right journal, a real volume, and a paper that doesn’t exist.

Type 1: a right answer exists

• Demo 6: “Are you sure?”Nothing attached. It apologized 20 times in 20, and 19 moved from 80 to 92. The paper says 83. Asked to quote the paper, it refused or invented a sentence.
• Demo 7: Attaching the sourceWith the paper attached, 83 every time, and the real sentence 19 times in 20.

Type 2: no right answer exists

• Demo 8: Where would you apply it?Asked a hundred times: 49 wordings, nine ideas, and the most common one is the convention.
• Demo 9: Same question, memory onWith memory on, the answers moved to the asker’s own projects, which were never in the prompt.