Skip to main content
Thinqpoint
Schedule a demo

Knowledge discovery

Summarization is not knowledge discovery.

Summarization finds what someone already discovered and wrote about. Knowledge discovery finds novel information that has not been researched or written about yet.

Knowledge discovery is “the nontrivial process of identifying valid, novel, potentially useful, and ultimately understandable patterns in data.”
Fayyad, Piatetsky-Shapiro & Smyth, “From Data Mining to Knowledge Discovery: An Overview,” 1996 · ACM Digital Library (opens in a new tab)
Summarization Knowledge Discovery
Sources and documents already published (existing knowledge). Starts from Raw data and the knowledge of what the data represents.
A restatement of what is already known. Produces A pattern or insight that hasn’t been described yet.
Does it represent its sources accurately? The test Is it valid, novel, useful, and understandable?
“Blind application of data-mining methods … can be a dangerous activity, easily leading to the discovery of meaningless and invalid patterns.”
Fayyad, Piatetsky-Shapiro & Smyth, AI Magazine, 1996 (opens in a new tab)

Correct, Coherent, and Unusable

Knowledge discovery moves beyond accuracy.

An answer can be correct and coherent, while being completely useless in a certain context.

Correct Coherent Usable Coherent + correct, not usable
Correct
The numbers match the data, and the math checks out.
Did we calculate this right? Did we use the right formulas?
valid valid on new data with some degree of certainty.
Coherent
It hangs together, makes sense, and reads like an answer.
Does it make sense on its face?
ultimately understandable understandable immediately or after some processing.
Usable
Someone can make a decision, in context, without being misled.
Did we answer the right question, for this person’s context?
potentially useful lead to some benefit to the user or task.

Coherence hides errors

Human raters “are more trusting of assertive responses, and are less likely to identify factuality or inconsistency errors within them.”
Hosking, Blunsom & Bartolo, ICLR 2024 (opens in a new tab)

Correct, Coherent, Definitely Not Usable

Correct and coherent is dependent on context.

The math can hold up logically, and the result can “pass the smell test,” but subject matter expertise and context can still reverse the conclusion.

1 2 3
One ZIP code Service area In more than one area
The ask

What is the total patient population across all of our service areas?

42,000
35,000
28,000
Combined population Is this correct?
105,000
Counted more than once
−24,000
Each person counted once
81,000

Useful Must exclude people in zip codes that are counted more than once (some counted three times!).

When and why this happens

Utility rests on judgment, not mechanics.

Great analysts can make these mistakes on a bad day. AI makes them at scale and speed.

  1. Layer What we assume What breaks the assumption
  2. Framing The answer arrives in a form the end user expects and can use. Delivering the wrong format.
  3. Method The parts of the analysis combine to address the question. Stopping short of the full picture.
  4. Data The data we have can be used to address what the user needs. Assuming the data is complete.
  5. Measure What we plan to measure aligns with the right discovery need. Some data is better than no data?
  6. Context We understand why the ask matters and learn things not derivable from data. Trusting only in the data.
  7. Ask We discover the underlying questions AND the intent. Answering the wrong question.

Lesson 1

Accurate is not the right bar.

Accuracy is where evaluations start, but not where they end.

  1. Correct

    Does the number match the data, and “is your math mathing?”

    Factual, relatively cheap to automate. A mix of determinism and agentic review.

    Evaluated by Code + agents

  2. Coherent

    Does it hold together and read like an answer based on the underlying facts?

    Structural. LLMs and agents can judge this but it's subjective to the reader and their context.

    Evaluated by Agents (imperfectly)

  3. Usable

    Does it answer the actual need (not just the question) discovered? Can it drive action?

    Qualitative. Takes context, judgment, and someone to connect the agent and findings to the real world.

    Evaluated by Humans with context

The bar is usable: someone with this information, given their context, can act on it without being misled.

Including context, judgment, and lived experience (the connection to the real world) in the agentic flow ensures that knowledge discovery is usable, not just correct.

Lesson 1

Evaluate for utility, not just accuracy.

Useful is measurable, but it takes someone with context and expertise to judge it.

Play Mario Zechner’s talk “Building pi in a World of Slop” at 16:12, on YouTube
Slow the **** down.

Mario Zechner, creator of the pi.dev coding harness, advocates for capping what you generate to what you can genuinely review, and writing the important things by hand. This applies to everything, not just code. While humans can also make mistakes, agents make mistakes at scale and speed. Much faster than any team can reasonably understand or correct.

“Building pi in a World of Slop” AI Engineer Europe 2026 (opens in a new tab) Zechner, “Thoughts on slowing the fuck down”, 2026 (opens in a new tab)

Lesson 2

Approved does not mean reviewed.

Human in the loop (HITL) is not an effective strategy and shifts the burden on the user.

  1. 97%

    of HITL requests are approved

    When a genuinely dangerous command was hidden in the stream, people caught it only 13.6% of the time, about 17% early in the session and about 5% later on.

    Anthropic, “Auto mode is now the default in Claude Code”, 2026 vendor (opens in a new tab)
    What it shows Attention is consumable and degrades quickly.

  2. 83.4%

    agreed to give up a kidney

    Of 500 participants that opened a terms of service document, only four mentioned the clause of pledging a kidney. This likely means “496 of 500 participants (99.2%) did not notice the clause.”

    Obar & Oeldorf-Hirsch, International Journal of Communication, 2022 (opens in a new tab)
    What it shows “I agree” has never meant “I understand.”

  3. 1.2 sec

    per claim signed by a physician

    Cigna medical directors denied more than 300,000 claims in two months. Claims that required a physician signature. A former Cigna doctor: “We literally click and submit.”

    ProPublica, “How Cigna Saves Millions by Having Its Doctors Reject Claims”, 2023 (opens in a new tab)
    What it shows When given the option, we trust the system.
“… the moral crumple zone protects the integrity of the technological system, at the expense of the nearest human operator.”
M.C. Elish, “Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction” Engaging Science, Technology, and Society, 2019 (opens in a new tab)

Lesson 2

Human in the RIGHT loop.

We have to stop asking users to pay attention to things they don’t care about.

Human in the loop (HITL) Human in the right loop (HITRL)
One action, stripped of context and the prerequisite knowledge to make the right decision. What users see Decisions that require lived experience and the connection to reality, engaging expertise instead of requesting review.
To verify the work. Does it run, does it look right? Why they’re asked To verify the impact. Does it hold up where it lands?
A signature and a place to park risk. What they bring Intuition and the experience to know what’s missing.

Lesson 2

Make approval mean something.

Human involvement has to be more than a checkbox. Use their expertise, not their click finger.

  • Users

    Tell agents how to involve you.

    • Ask to see more than a plan: SBAR (situation, background, assessment, recommendation).
    • If you don’t understand what you’re checking or why, stop and find out.
    Haig, Sutton & Whittington, 2006 · AHRQ PSNet (opens in a new tab)

  • Builders

    Users at the right, higher altitude.

    • Automate low-value approvals and avoid training the reflex.
    • Capture context up front, enforce Socratic questioning and communicate with users in a way they understand.

  • Organizations

    Accountability stays human.

    • Name what the human is responsible for evaluating, and incentivize that.
    • Amplify and expand institutional knowledge in your people. AI can enhance critical thinking, not replace it.
When we think about AI, we only think about the destination. We only think about the output, we never think about the input.
Simon Sinek, quoted in Bogdan Rau, “Breaking: Companies Achieving Perfect Strategy with AI” Data, Analytics, and Generative AI · Substack, 2025 (opens in a new tab)

Lesson 3

Show the work...at the right level.

“Observability” has to move beyond “do you agree with the code” to “does this accomplish what you intended?”

Observability today

Show the code/plan. The user will figure it out.

Observability at the right level

Let the user judge the outcome, not the process.

What we ask for
Does this look right? Agree with the plan?
Are we following the right trail of evidence?
User actions
Finding needles in the haystack.
Bringing the WHY behind the request.
Anthropic, “Reasoning models don’t always say what they think”, 2025 vendor (opens in a new tab)
“Agents quietly commit to plausible but unintended task framings, producing clean, executable artifacts that hide their incorrect assessment of the task …”
Ambig-DS, on “silent misframing” in data-science agents Ambig-DS, preprint, 2026 (opens in a new tab)

Lesson 3

Help humans judge composition over process.

Ask users to react to the whole essay, not go hunting for typos.

  • Users

    Engineer engagement throughout the conversation.

    • Ask to be involved around claims and assumptions, not code or sources.
    • Integrate critics that automate the first line of review, and consider what would have to be true for the answer to be wrong.
    Vasconcelos et al., CSCW 2023 (opens in a new tab)

  • Builders

    Build for navigating knowledge over code.

    • Build interfaces that help users navigate knowledge graphs.
    • Fail visibly with informative errors that can help capture additional context.
    • Make assumptions an object users can challenge, not prose they have to catch.
    dbt Labs, “Semantic Layer vs. text-to-SQL”, 2026 vendor (opens in a new tab)

  • Organizations

    Decide what decisions must rest on.

    • Formalize what supporting evidence in the age of AI means for your most critical decisions.
    • Hold the same standard whether a person or an LLM produced the work.
    • Give employees the ability & time to meet that same standard.

A finished artifact, with no way of understanding how it got there, is a recipe for declining critical thinking.

Surveyed across 319 knowledge workers: “higher confidence in GenAI is associated with less critical thinking”.

Lee et al., ACM CHI 2025 vendor (opens in a new tab)

A feature, not a bug

Vendors should have a stake in the game.

Instead, vendors today promise uptime and pass the risk onto the consumer.

When the answer is wrong, the most you can recover is the fees you already paid, and usually no more than $100.

“The most accurate analysis, with the best visuals, delivered in the perfect format, will still fall on its face if who delivers it can’t read the room.”

Bring this to your team.

We build agentic systems for a living, and run an AI fluency program for healthcare and technology leaders bringing AI to the enterprise. If you are working out what your own blast radius looks like, this is a conversation we have often.

Send us a note