​

I've been wondering about this lately. Many organizations are building in-house LLMs for GRC to answer security questionnaires, map compliance controls, generate policies, and support audit preparation. Keeping everything on-premises helps with data privacy, but it doesn't solve the biggest problem: a confident hallucination can still end up in an audit report or customer response if no one catches it.

The more capable these systems become, the more people are likely to trust them. That creates an interesting trade-off. If an AI-generated answer helps close deals faster but occasionally invents evidence or misinterprets a control, the financial and compliance impact could outweigh the productivity gains. At that point, is the technology truly ready for critical GRC work, or are we becoming overconfident because the responses sound convincing?

For those using an in-house LLM in GRC today, where do you draw the line between AI assistance and human accountability—and do you think we'll ever reach a point where an LLM can be trusted to answer security questionnaires without manual verification?

What it is

A self-hosted AI assistant for GRC (Governance, Risk & Compliance) and cybersecurity. You ask a plain-language question ("What are the DPDPA breach-notification timelines?") and it answers from a curated corpus of 2,681 real controls across 48 frameworks (GDPR, HIPAA, ISO 27001, NIST, PCI DSS, SOC 2, RBI, DPDPA, DORA, and more) with inline citations back to the specific control and its official source URL, plus a verification-tier badge so you know how trustworthy each source is.

Design goals: runs entirely on your own hardware, near-zero infra cost, no data ever leaves the machine (it's compliance data that's the whole point), and a clean integrable API so it's not locked to the bundled UI.

The stack

Backend (Rust)

• Axum 0.8 + Tokio async API, SSE streaming
• LanceDB (embedded vector DB) for dense retrieval + Tantivy (BM25) for sparse — merged with Reciprocal Rank Fusion.
• No external vector-DB service.
• Ollama for local inference: bge-m3 embeddings (1024-dim) + qwen2.5:7b-instruct generation
• SQLite (rusqlite) for users, hashed API keys, audit log, and persistent chat history — everything embedded, one file
• JWT + API-key dual auth with RBAC (viewer/admin); RFC 7807 error envelopes; OpenAPI/Swagger auto-generated via utoipa
• SearXNG (self-hosted metasearch) as a live-web-search fallback when the corpus is thin or the question is time-sensitive

Frontend: React + TypeScript + Vite + Tailwind v4, streaming chat UI, collapsible "thinking" reasoning trace, citation panel, admin console (coverage/audit/API-key issuance)

Ops

• Multi-stage Docker with cargo-chef layer caching, single-binary that also serves the built frontend (one port, no separate frontend host)
• Eval harness (retrieval precision@k + a grounding/hallucination check) and a drift monitor that re-checks high-stakes facts (penalties, deadlines, circular numbers) against their source URLs on a schedule

Features that actually work right now

• Hybrid retrieval -> grounded, cited answers (token-streamed)
• Persistent chat -> generation runs as a detached backend task, so an in-flight answer survives a reload/tab-close and gets saved regardless
• Thinking mode toggle -> runs a deeper pipeline (query rewriting → rerank → self critique) and streams the reasoning steps
• Live-web-search toggle, per-framework filtering, thumbs feedback logging
• Full auth/RBAC/audit trail

Being honest about the rough edges

• It's CPU-bound on my dev box (no GPU) a 7B model answer takes ~1-2 min; streaming hides most of it but a GPU is the real fix
• Small local models (1.5B/0.5B) drop citations under long RAG context instruction following degrades; 7B is the floor for reliable citing. Interesting problem I'd love input on.
• Single-node, self-hosted MVP not horizontally scaled yet
• The corpus went through two automated audit/fix passes but hasn't had a formal expert review

Why open source

Compliance tooling is either eye-wateringly expensive SaaS or spreadsheets. A self hostable, private, auditable RAG assistant over open framework data feels like something the community should own. Planning to open it up once the core is battle-tested.

Happy to go deep on any part the RRF hybrid retrieval, the detached-generation persistence model, the cargo-chef Docker setup, the citation-grounding eval, whatever. What are you all building in this space?

We recently conducted a manual audit of our google workspace environment at a mid-sized company and discovered numerous sensitive files including customer contracts and internal financial reports shared publicly with 'anyone with the link'

Many of these files were created years ago by former employees. I’m currently going through thousands of files manually, which is taking forever.

How are other GRC and security teams handling discovery and remediation of historical data exposure like this at scale? Looking for best practices, tools, or efficient processes that have worked for you?

thanks!

Currently got our IT risk register in Jira, it’s fine but a bit clunky.

Has anyone used a tool for IT risk that’s actually been worth it.

For context, we’re at the start of our risk management journey, we’ve got a register but it’s not really being managed properly. My main goal is just to make it easy for our risk owners to review risks assigned to them without being clouded by everything else, assign out tasks and manage and track things accordingly.

The great majority of society does not pay attention to the current threat of intellectual property loss as a result of leakage of proprietary data. Although it will take time for a quantum computer to be fully operational, state agencies could be creating vital intelligence in the meantime

At the time when new compliance audits will commence, companies dedicated to creating advanced cryptographic systems, like QuSecure and SandboxAQ, will become much more prominent in the industry. Is there any member of the audience who has tried out either of them or used them at the organizational level? I am wondering if anyone can share their experience: difficulties of integration, level of assistance, whether the price compensates for the usage of the PQC technological solution.

Have you thought of quantum readiness when trying out these approaches?

Nine questions in the career megathread. I noticed the last reply was two weeks ago, and before that, four. I’m guessing people posted there to get some sort of help, so I figured I’d mention it.

@mods or anyone. Appreciate it!

I want to build an incident response plan for my organization can someone guide me the resources I should follow to build the workable program?

My organization already has a good security stack they lack the IR plan I wanna know how a effective IR program looks like what to add and what to ignore

Any resources books, blogs, talks much appreciated.

Thanks in advance.

starting to build a digital asset risk framework before the business side commits to anything, and the standard counterparty risk models don't translate cleanly. The market spans unchartered technology companies to federally regulated banks and all of them describe themselves in broadly similar terms.

The failure mode question is hardest to structure. Key management failure in digital assets isn't a clean analog to traditional custody failure, and recoverability depends on the custody architecture in a way that doesn't apply to most other counterparty risk categories.

Has anyone built a taxonomy for digital asset infrastructure that maps onto existing operational risk frameworks, particularly around the charter tier distinction?

to be fair, printed assurance reports can be heavy

I'm evaluating GRC platforms for my own use and keep finding mixed opinions online.

For those of you who work with GRC tools every day, if you were starting from scratch today, which platform would you choose and why?

I'd be especially interested in hearing about what you like, what frustrates you, and whether you'd choose the same platform again.

Sanity checking our approach on the compliance side before an assessment. We're going for FedRAMP and a chunk of our vulnerability findings have no vendor fix available, the patch simply doesn't exist yet. A lot of them come from third-party components we don't build because remediation isn't in our hands. We can't close them and we can't ignore them, they go on the POA&M.

We prioritise with exploitability signals, CISA KEV and EPSS such that we know which no-fix items are worth escalating versus just monitoring. The part I'm unsure about is the long-term optics. That POA&M line just grows, criticals sitting as monitored with no close date because the fix is outside our control.

What I'm really after is whether this holds up at assessment. If you've taken a growing no-fix POA&M through a 3PAO, do they let it ride as monitored or force close dates and deviation requests on you.

I want to know how everyone structures their enterprise risk register.

Do you track:

• Inherent & residual risk?
• KRIs?
• Controls?
• Treatment plans?
• Review dates?

What's one field you've added that turned out to be surprisingly valuable?

The Cyber Resilience Act covers products with digital elements - software and connected products sold in the EU. The big compliance deadlines are still a way off, but a lot of organizations are already having to bake security requirements into their dev process now.

Curious how others are actually tackling this:
Have you started tying security requirements, vulnerability handling, patching, and risk assessment into the product lifecycle?

And what's the hardest part to nail down - the tech itself, the process, who's accountable, the documentation, interpreting the legal text, or the supply chain?

I'm a cybersecurity student in Brazil with ISO 27001 Foundation (PECB), currently completing my first real GRC project for a small business (security policies, risk matrix, LGPD compliance documentation). I'm looking to gain hands-on experience by assisting an experienced GRC or ISO 27001 consultant on real projects. I can help with documentation, asset inventories, risk assessments, and operational support. Happy to work for free or for a very small fee. DMs open.

It always felt like another compliance task until we started finding forgotten VMs, old cloud resources, and systems nobody remembered owning.

Turns out "you can't secure what you don't know exists" isn't just a cliché.

What's one compliance requirement you've ended up appreciating more over time?

Hi everyone, I’m doing some research on how privacy professionals stay current with privacy, AI governance, and cybersecurity regulations.I’m not selling anything—I’m genuinely trying to understand how people work because everyone I’ve spoken to seems to have a different system.

A few questions I have

1). Where do you usually hear about new regulatory developments?
Official regulators?
Law firms?
LinkedIn?
Newsletter subscriptions or RSS feeds?

2).Once you learn about a new regulation or enforcement action, what happens next….Do you save it or share and Forward it to people on ur team
Personally , I feel like I would forget until somebody asks me about it😬😂.

3).What’s the most frustrating part of staying current? and are there tools or anything that help with that

I’d love to understand your workflow.

Thanks❤️

our security team presents to the board next quarter and i've been tasked with the vuln management section. first time doing this at board level and i'm trying to figure out what lands versus what gets us a bunch of questions we can't answer cleanly.

right now we lean on mean time to remediate and SLA compliance by severity, plus total open vulns. none of those tell a clean story on their own. total open vulns goes up every time we onboard a new scanner or expand coverage, so it looks like we're getting worse when we're just seeing more. MTTR looks fine, but that's partly because crits with no patch never close, so they drop out of the average entirely, and the number reflects the vulns we could fix quickly, not the ones stuck with no fix. SLA compliance makes leadership feel good but tells you nothing about what's at risk.

i've seen people talk about exposure window and exploitability weighting, but we’ve looked at weighting exposure and exploitability more heavily but i'm not sure how to explain that to a board. you say "we prioritize by CVSS and EPSS" and half the room nods like they understood that.

whatever we show, someone asks how we compare to industry benchmarks. i never have a clean answer because benchmarks for this swing wildly by sector and company size, and that's before you account for what you're even scanning.

the only thing that's ever landed cleanly with our execs is exposure on internet-facing assets, how many known-exploitable vulns are sitting on something reachable from outside and how long they've been there. that maps to "what could hurt us" in language a board follows.

everything else we put up turns into a debate about methodology and we lose the room.

has anyone figured out how to make vuln management metrics meaningful at the board level without dumbing it down to uselessness or drowning them in numbers they have no context for. what do you show, and what questions does it usually kick off?

How's your workload or work/life balance working in this field? If you can include if your Entry/Mid/Sr. position and your company's industry (Tech, Hospitality, Finance), even better.

Thanks

Our board... unsurprisingly...started asking about AI risk last quarter and I'm struggling with how to present it. The cyber risk conversation took years to get right, and even that still sometimes falls flat. AI risk feels like starting from zero. The challenge is that most of what I can show them right now is qualitative. They nod or whatever but I can tell it's not landing. Nobody challenges. There's no conversation.

With cyber we eventually got traction when we started presenting risk in financial terms. The board understood exposure in dollars and could make real decisions about control investments. I'm wondering if the same approach works for AI risk. Translating governance gaps and shadow AI exposure into financial figures rather than compliance status updates.

For those of you who've been in front of a board on AI risk and have found any type of success.... What's worked?

Edit: Spent some time researching this after posting. The cyber risk quantification parallel seems to be exactly the right model. Found Kovrr, which does AI risk quantification alongside shadow AI discovery and EU AI Act compliance. The idea is translating AI governance gaps into financial exposure figures, which is exactly the format that worked for our cyber risk conversations with the board. Going to look into it further but flagging for anyone in the same boat.

I work for a small company who on occasion does surface level framework readiness compliance assessments. We do not have any enterprise software. All our work is on spreadsheets and word docs . After the assessment we complete a gap Analysis and provide a remediation roadmap. Are there any enterprise tools that provide a decent free tier for home use? How can I increase my skillsets in this space? What companies may have an interest in me at my level?

How does your intake / governance process work, in regards to the various process owners, your team, and the overarching platform team?

I'm the director of GRC, and I don't want to become a useless bottleneck between the people who do the work, and the platform team (the people who run change management within the larger ServiceNow platform).

But I'm also accountable for the structure of the various items within IRM (policy, issues management, risk management).

What type of governance and intake have you seen or conduct?

Hi everyone! This is my first post here.

Over the past few months, while building Halbarad, I've had the chance to speak with a lot of people working across third-party risk, GRC, compliance, cybersecurity, privacy, audit, resilience, procurement, and AI governance.

One thing that really stood out is how much this profession is changing.

AI is changing how risk teams operate. Organizations are asking more from risk professionals than ever before. New specialties are emerging, and more people are moving between cybersecurity, compliance, privacy, audit, and third-party risk than they were just a few years ago.

Something else I kept hearing was how frustrating it is to actually find jobs in this space. Most job boards are dominated by software engineering roles and risk jobs are scattered across hundreds of company career pages.

So I decided to build something to help.

Today we launched what I believe is the largest dedicated job board for the modern risk profession, with nearly 2,000 open roles across third-party risk, GRC, compliance, cybersecurity, privacy, resilience, audit, AI governance, and more, all aggregated from public hiring sources and continuously updated.

[https://community.halbarad.com/jobs]()

Since this community is exactly who we're building it for, I'd genuinely love your feedback.

• Are there companies we're missing?
• What filters or search features would make this more useful?
• If you were looking for a new role, what would you want from a job board that LinkedIn doesn't provide?

If this isn't appropriate for the subreddit, mods, please feel free to remove it. Otherwise, I hope some of you find it useful, and I'd really appreciate any feedback.