Signal Detection Theory

My wife asked me if I’ve ever heard of “signal detection theory”. I said “nup”, which surprised her. She reckons that you guys would have (I think she’s looking at you @Belfry ). It would be easy enough to ask ChatGPT (go ahead and laugh at me @techman ) but I think it would be more fun to ask all of you.

I’ve never heard of “signal detection theory”, and you’re way too big to laugh at :slight_smile:

I’m just a simple retired electronics technician!

I hadn’t heard of SDT before but anyone who has done a visual field test for glaucoma will know the problem. (Was that a faint flash way out at 7 o’clock??? Bugger it, I’ll press the button anyway.)

For “left field” questions like this, I use prompting queries to get the context. As a prompt repeat back as an with curious inquiry tone: “signal detection theory?” or “go on..?” or “and..?” or “please share?”. or “used for..?” or “what are you trying to treat?”. The answers to types of questions can be extensive. And tell a lot about the topic and what is concerning for the person. Can take some time.

Ok, so I asked my AI, and I’m still none the wiser really.

~]$ fabric “what is ‘signal detection theory’” -p summarize

ONE SENTENCE SUMMARY:
Signal Detection Theory measures the ability to distinguish signals from noise while separating sensitivity from decision bias.

MAIN POINTS:

  1. It quantifies how well an observer distinguishes signal from noise.
  2. The theory separates actual sensitivity from the observer’s decision-making bias.
  3. Outcomes are categorized as hits, misses, false alarms, or correct rejections.
  4. Sensitivity ($d’$) measures the distance between noise and signal distributions.
  5. Criterion represents the threshold level an observer uses to decide.
  6. Background noise always interferes with the detection of a target signal.
  7. Receiver Operating Characteristic curves plot the trade-off between hits and false alarms.
  8. Applications include psychology, medical diagnostics, and radar engineering fields.
  9. Decision criteria can shift based on motivations or expectations.
  10. Mathematical frameworks analyze perceptual decision-making processes under uncertainty.

TAKEAWAYS:

  1. Detection depends on both stimulus strength and internal decision bias.
  2. High sensitivity does not guarantee accuracy without proper decision thresholds.
  3. False alarms and misses are inevitable trade-offs in noisy environments.
  4. ROC curves provide a visual method to compare different diagnostic systems.
  5. Understanding bias helps improve the reliability of eyewitness testimony and medical tests.

@techman, I thought you didn’t allow bullet points in your fabric answers. However your AI has got it right in the first line.

I think we all understand signal to noise ratio and hits, misses, false negatives and false positives. What decision theory is doing is noting the ROC (receive operating characteristic) varies based on the view / bias of the observer.

Professor David Heeger explains this in more detail in the scenario of a radiologist detecting cancer.

The radiologist can be a cowboy and lets through a lot of cancers but minimises the trauma of followup investigations in the false positives. The nazi radiologist misses few cancers but the patients go through a lot more tests and trauma.

This variation is independent of the primary signal to noise ratio and signal decision theory elaborates on the variation and what one can do about it (viz increase the signal, or decrease the noise).

What puzzles me is why Mrs @jdownie was asking about it.

Not quite. I use a prompt that tells the AI not to allow bullet points. I didn’t design Fabric, the AI or the prompt myself.

OUTPUT INSTRUCTIONS

  • Output no more than 40 words.
  • Create the output using the formatting above.
  • You only output human readable Markdown.
  • Do not output numbered lists or bullets.
  • Do not output newlines.
  • Do not output warnings or notes.

Ah well, at least it hasn’t deleted the contents of my hard drive yet :slight_smile:

Haha, for the record, I had heard of it. Been struggling with the electronics version of it with the GPS clock shenanigans most recently! :joy: