Skip Navigation

InitialsDiceBearhttps://github.com/dicebear/dicebearhttps://creativecommons.org/publicdomain/zero/1.0/„Initials” (https://github.com/dicebear/dicebear) by „DiceBear”, licensed under „CC0 1.0” (https://creativecommons.org/publicdomain/zero/1.0/)S

SuspciousCarrot78

@ SuspciousCarrot78 @lemmy.world

Posts
4
Comments
231
Joined
6 mo. ago

  • I'm tired of LLM bullshitting. So I fixed it.

    Jump

  • Replying in specific

    “SUMM -> human reviews That would be fixed, but will work only for small KBs, as otherwise the summary would be exhaustive.”

    Correct: filesystem SUMM + human review is intentionally for small/curated KBs, not “review 3,000 entities.” The point of SUMM is curation, not bulk ingestion at scale. If the KB is so large that summaries become exhaustive, that dataset is in the wrong layer.

    “Case in point: assume a Person model with 3-7 facts per Person. Assume small 3000 size set of Persons. How would the SUMM of work?”

    Poorly. It shouldn’t work via filesystem SUMM. A “Person table” is structured data; SUMM is for documents. For 3,000 people × (3–7 facts), you’d put that in a structured store (SQLite/CSV/JSONL/whatever) and query it via a non-LLM tool (exact lookup/filter) or via Vault retrieval if you insist on LLM synthesis on top.

    Do you expect a human to verify that SUMM?”

    No - not for that use case. Human verification is realistic when you’re curating dozens/hundreds of docs, not thousands of structured records. For 3,000 persons, verification is done by data validation rules (schema, constraints, unit tests, diff checks), not reading summaries.

    “How are you going to converse with your system to get the data from that KB Person set?”

    Not by attaching a folder and “asking the model nicely.” You’d do one of these -

    • Exact tool lookup: person("Alice") -> facts, or search by ID/name, return rows deterministically.
    • Hybrid: tool lookup returns the relevant rows, then the LLM formats/summarizes them.
    • Vault retrieval: embed/chunk rows and retrieve top-k, but that’s still weaker than exact lookup for structured “Person facts.”

    So: conversation is fine as UX, but the retrieval step should be tool-based (exact) for that dataset.

    But actually, you give me a good idea here. It wouldn't be the work of ages to build a >>look or >>find function into this thing. Maybe I will.

    My mental model for this was always "1 person, 1 box, personal scale" but maybe I need to think bigger. Then again, scope creep is a cruel bitch.

    “Because to me that sounds like case C, only works for small KBs.”

    For filesystem SUMM + human review: yes. That’s the design. It’s a personal, “curate your sources” workflow, not an enterprise entity store.

    This was never designed to be a multi-tenant look up system. I don't know how to build that and still keep it 1) small 2) potato friendly 3) account for ALL the moving part nightmares that brings.

    What I built is STRICTLY for personal use, not enterprise use.

    Fair. Except that you are still left with the original problem of you don't know WHEN the information is incorrect if you missed it at SUMM time.”

    Sort of. Summation via LLM was always going to be a lossy proposition. What this system changes is the failure mode:

    • Without this: errors can get injected and later you can’t tell where they came from.
    • With this: if a SUMM is wrong, it is pinned to a specific source file hash + summary hash, and you can fix it by re-summarizing or replacing the source.

    In other words: it doesn’t guarantee correctness; it guarantees traceability and non-silent drift. You still need to "trust but verify".

    TL;DR:

    You don’t query big, structured datasets (like 3,000 “Person” records) via SUMM at all. You use exact tools/lookup first (DB/JSON/CSV), then let the LLM format or explain the result. That can probably be added reasonably quickly, because I tried to build something that future me wouldn't hate past me for. We'll see if he/I succeeded.

    SUMM is for curated documents, not tables. I can try adding a >>find >>grep or similar tool (the system is modular so I should be able to accommodate a few things like that, but I don't want to end up with 1500 "micro tools" and hating my life)

    And yeah, you can still miss errors at SUMM time - the system doesn’t guarantee correctness. That's on you. Sorry.

    What it guarantees is traceability: every answer is tied to a specific source + hash, so when something’s wrong, you can see where it came from and fix it instead of having silent drift. That's the "glass box, not black box" part of the build.

    Sorry - really. This is the best I could figure out for caging the stochastic parrot. I built this while I was physically incapacitated and confined to be rest, and shooting the shit with Gippity all day. Built it for myself and then though "hmm, this might help someone else too. I can't be the only one that's noticed this problem".

    If you or anyone else has a better idea, I'm willing to consider.

  • I'm tired of LLM bullshitting. So I fixed it.

    Jump

  • Thank you. I appreciate you saying so!

  • I'm tired of LLM bullshitting. So I fixed it.

    Jump

  • Please enjoy. Make sure you use >>FR mode at least once. You probably won't like the seed quotes but maybe just maybe you might and I'll be able to hear the "ha" from here.

  • I'm tired of LLM bullshitting. So I fixed it.

    Jump

  • Spite based inference?

    You dirty pirate hooker.

    I don’t believe you.

  • I'm tired of LLM bullshitting. So I fixed it.

    Jump

  • Thank you! I appreciate you.

    PS: Where's the guy this should be targeted at?

  • I'm tired of LLM bullshitting. So I fixed it.

    Jump

  • Well, to butcher Sinatra: if it can make it on Lemmy and HN, it can make it anywhere :)

  • I'm tired of LLM bullshitting. So I fixed it.

    Jump

  • I’m not claiming I “fixed” bullshitting. I said I was TIRED of bullshit.

    So, the claim I’m making is: I made bullshit visible and bounded.

    The problem I’m solving isn’t “LLMs sometimes get things wrong.” That’s unsolvable AFAIK. What I'm solving for is "LLMs get things wrong in ways that are opaque and untraceable".

    That's solvable. That’s what hashes get you. Attribution, clear fail states and auditability. YOU still have to check sources if you care about correctness.

    The difference is - YOU are no longer checking a moving target or a black box. You're checking a frozen, reproducible input.

    That’s… not how any of this works…

    Please don't teach me to suck lemons. I have very strict parameters for fail states. When I say three strikes and your out, I do mean three strikes and you're out. Quants ain't quants, and models ain't models. I am very particular in what I run, how I run it and what I tolerate.

  • I'm tired of LLM bullshitting. So I fixed it.

    Jump

  • Oh boy. So hallucination will occur here, and all further retrievals will be deterministically poisoned?

    Huh? That is the literal opposite of what I said. Like, diametrically opposite.

    Let me try this a different way.

    Hallucination in SUMM doesn’t "poison" the KB, because SUMMs are not authoritative facts, they’re derived artifacts with provenance. They’re explicitly marked as model output tied to a specific source hash. Two key mechanics that stop the cascade you’re describing:

    1. SUMM is not a "source of truth"

    The source of truth is still the original document, not the summary. The summary is just a compressed view of it. That’s why it carries a SHA of the original file. If a SUMM looks wrong, you can:

    a) trace it back to the exact document version b) regenerate it c) discard it d) read the original doc yourself and manually curate it.

    Nothing is "silently accepted" as ground truth.

    1. Promotion is manual, not automatic

    The dangerous step would be: model output -> auto-ingest into long-term knowledge.

    That’s explicitly not how this works.

    The Flow is: Attach KB -> SUMM -> human reviews -> Ok, move to Vault -> Mentats runs against that

    Don't like a SUMM? Don't push it into the vault. There's a gate between “model said a thing” and “system treats this as curated knowledge.” That's you - the human. Don't GI and it won't GO.

    Determinism works for you here. The hash doesn’t freeze the hallucination; it freezes the input snapshot. That makes bad summaries:

    • reproducible
    • inspectable
    • fixable

    Which is the opposite of silent drift.

    If SUMM is wrong and you miss it, the system will be consistently wrong in a traceable way, not creatively wrong in a new way every time.

    That’s a much easier class of bug to detect and correct. Again: the proposition is not "the model will never hallucinate.". It's "it can't silently propagate hallucinations without a human explicitly allowing it to, and when it does, you trace it back to source version".

    And that, is ultimately what keeps the pipeline from becoming "poisoned".

  • I'm tired of LLM bullshitting. So I fixed it.

    Jump

  • Parts of this are RAG, sure

    RAG parts:

    • Vault / Mentats is classic retrieval + generation.
    • Vector store = Qdrant
    • Embedding and reranker

    So yes, that layer is RAG with extra steps.

    What’s not RAG -

    KB mode (filesystem SUMM path)

    This isn’t vector search. It’s deterministic, file-backed grounding. You attach folders as needed. The system summarizes and hashes docs. The model can only answer from those summaries in that mode. There’s no semantic retrieval step. It can style and jazz around the answer a little, but the answer is the answer is the answer.

    If the fact isn’t in the attached KB, the router forces a refusal. Put up or shut up.

    Vodka (facts memory)

    That’s not retrieval at all, in the LLM sense. It's verbatim key-value recall.

    • JSON on disk
    • Exact store (!!)
    • Exact recall (??)

    Again, no embeddings, no similarity search, no model interpretation.

    "Facts that aren’t RAG"

    In my set up, they land in one of two buckets.

    1. Short-term / user facts → Vodka. That for things like numbers, appointments, lists, one-off notes etc. Deterministic recall, no synthesis.

    2. Curated knowledge → KB / Vault. Things you want grounded, auditable, and reusable.

    In response to the implicit "why not just RAG then"

    Classic RAG failure mode is: retrieval is fuzzy → model fills gaps → user can’t tell which part came from where.

    The extra "steps" are there to separate memory from knowledge, separate retrieval from synthesis and make refusal a legal output, not a model choice.

    So yeah; some of it is RAG. RAG is good. The point is this system is designed so not everything of value is forced through a semantic search + generate loop. I don't trust LLMs. I am actively hostile to them. This is me telling my LLM to STFU and prove it, or GTFO. I know that's a weird way to operate maybe (advesarial, assume worst, engineer around issue) but that's how ASD brains work.

  • I'm tired of LLM bullshitting. So I fixed it.

    Jump

  • I had to look it up. Apparently, it's someone who over-optimises the bells and whistles and never ships a finished product.

    https://gooncode.dev/

  • I'm tired of LLM bullshitting. So I fixed it.

    Jump

  • ^ Yes! That. Exactly that. Thank you!

    I don't like the bullshit...and I'm not paid to optimize for bullshit-leading-to-engagment-chatty-chat.

    "LLM - tell me the answer and then go away. If you can't, say so and go away. Optionally, roast me like you've watched too many episodes of Futurama while doing it"

  • I'm tired of LLM bullshitting. So I fixed it.

    Jump

  • This is the way. Good luck with OSS-120B. Those OSS models, they

    • really
    • like
    • bullet
    • points

  • I'm tired of LLM bullshitting. So I fixed it.

    Jump

  • Yeah.

    The SHA isn’t there to make the model smarter. It’s there to make the source immutable and auditable.

    Having been burnt by LLMs (far too many times), I now start from a position of "fuck you, prove it".

    The hash proves which bytes the answer was grounded in, should I ever want to check it. If the model misreads or misinterprets, you can point to the source and say “the mistake is here, not in my memory of what the source was.”.

    If it does that more than twice, straight in the bin. I have zero chill any more.

    Secondly, drift detection. If someone edits or swaps a file later, the hash changes. That means yesterday’s answer can’t silently pretend it came from today’s document. I doubt my kids are going to sneak in and change the historical prices of 8 bit computers (well, the big one might...she's dead keen on being a hacker) but I wanted to be sure no one and no-thing was fucking with me.

    Finally, you (or someone else) can re-run the same question against the same hashed inputs and see if the system behaves the same way.

    So: the hashes don't fix hallucinations (I don't even think that's possible, even with magic). The hashes make it possible to audit the answer and spot why hallucinations might have happened.

    PS: You’re right that interpretation errors still exist. That's why Mentats does the triple-pass and why the system clearly flags “missing / unsupported” instead of filling gaps. The SHA is there to make the pipeline inspectable, instead of “trust me, bro.”.

    Guess what? I don't trust you. Prove it or GTFO.

  • I'm tired of LLM bullshitting. So I fixed it.

    Jump

  • I hope it does what it I claim it does for you. Choose a good LLM model. Not one of the sex-chat ones. Or maybe, exactly one of those. For uh...research.

  • I'm tired of LLM bullshitting. So I fixed it.

    Jump

  • Thank you :) I've been eating a lot of shit on HN (and other places) about this thing. It's nice not to be called a goon-coder or fantasist, just once.

  • I'm tired of LLM bullshitting. So I fixed it.

    Jump

  • Show off :)

    You're self hosting that, right? I will not be held responsible for some dogey OpenRouter quant hosted by ToTaLlY NoT a ScAM LLC :)

  • I'm tired of LLM bullshitting. So I fixed it.

    Jump

  • D) None of the above.

    I didn’t "solve hallucination". I changed the failure mode. The model can still hallucinate internally. The difference is it’s not allowed to surface claims unless they’re grounded in attached sources.

    If retrieval returns nothing relevant, the router forces a refusal instead of letting the model free-associate. So the guarantee isn’t “the model is always right.”

    The guarantee is “the system won’t pretend it knows when the sources don’t support it.” That's it. That's the whole trick.

    KB size doesn’t matter much here. Small or large, the constraint is the same: no source, no claim. GTFO.

    That’s a control-layer property, not a model property. If it helps: think of it as moving from “LLM answers questions” to “LLM summarizes evidence I give it, or says ‘insufficient evidence.’”

    Again, that’s the whole trick.

    You don't need to believe me. In fact, please don't. Test it.

    I could be wrong...but if I'm right (and if you attach this to a non-retarded LLM), then maybe, just maybe, this doesn't suck balls as much as you think it might.

    Maybe it's even useful to you.

    I dunno. Try it?

  • I'm tired of LLM bullshitting. So I fixed it.

    Jump

  • re: the KB tool, why not just skip the llm and do two chained fuzzy finds? (what knowledge base & question keywords)

    Yep, good question. You can do that, it's not wrong. If your KB is small + your question is basically “find me the paragraph that contains X,” then yeah: two-pass fuzzy find will dunk on any LLM for speed and correctness.

    But the reason I put an LLM in the loop is: retrieval isn’t the hard part. Synthesis + constraint are. What a LLM is doing in KB mode (basically) is this -

    1. Turns question into extraction task. Instead of “search keywords,” it’s: “given these snippets, answer only what is directly supported, and list what’s missing.”

    2. Then, rather that giving 6 fragments across multiple files, the LLM assembles the whole thing into a single answer, while staying source locked (and refusing fragments that don't contain the needed fact).

    3. Finally: it has "structured refusal" baked in. IOW, the whole point is that the LLM is forced to say "here are the facts I saw, and this is what I can't answer from those facts".

    TL;DR: fuzzy search gets you where the info lives. This gets you what you can safely claim from it, plus an explicit "missing list".

    For pure retreval: yeah - search. In fact, maybe I should bake in a >>grep or >>find commands. That would be the right trick for "show me the passage" not "answer the question".

    I hope that makes sense?

  • I'm tired of LLM bullshitting. So I fixed it.

    Jump

  • Oh shit! Uh...thank you! Umm. Yes. That was unexpected :)

    Re: collab. I'm away for a bit with work, but let me think on it for a bit? There's got to be a way to make this useful to more peeps.

    Believe it or not, I am not a CS guy at ALL (I work in health-care) and I made this for fun, in a cave, with a box of scraps.

    I'm not good at CS. I just have a ... "very special" brain. As in, I designed this thing from first principles using invariants, which I understand now is not typical CS practice.