Team BehelitArun Kumar (solo)CoCo CLI Hackathon · GCC Editionv1.2.0

Risk Copilot, explained the Feynman way

Each part is told twice. First in plain words a new teammate would get on day one. Then the real machinery: the file, the function, the SQL and the numbers. If the plain version and the deep version ever disagree, the deep version is the truth.

Step 1 of the Feynman method · say it simply

The 30-second story

A bank analyst types a question like “draft a suspicious activity report for fast wires to high-risk countries.” The copilot does three things a careful junior would do. It pulls the matching transaction rows from the database as proof. It finds the three rule clauses that fit best. It writes a short draft report that names every proof row and every rule it used. Then it stops. Nothing is saved until a compliance officer, a named person, presses Approve. Every approval, and every blocked try, goes into a log that cannot be argued with later.

The analogy: a research assistant who brings you the receipts and the rulebook pages, writes the memo, and then hands the pen to the manager to sign. The assistant can never sign.

3law clauses cited per question (TOP_K)
5evidence rows per draft (EVIDENCE_TX_LIMIT)
0AI calls without the harm filter
102automated tests, no Snowflake needed
The map

The whole flow on one page

Read it left to right, top row then bottom row. The orange box is the one step only a human can do.

Step 2 of the Feynman method · go deep, one piece at a time

1 · The front door: what may come in

In plain words

Before anything happens, the question is checked like a bag at airport security. Too long, strange symbols or database command words are rejected. People who ask too often are slowed down. If the speed check itself breaks, the answer is “no”, not “ok”.

Under the hood

guardrail_check() in backend/hybrid_agent.py runs on every input:

  • Empty or over 2000 characters (MAX_INPUT_LEN) is rejected.
  • A character allow-list regex ^[A-Za-z0-9 .,?!#$@:/()_-]+$ (only these symbols pass). A semicolon fails, so chained SQL cannot even be typed.
  • Blocked whole words: DROP, DELETE, TRUNCATE, ALTER, GRANT, INSERT, UPDATE, UNION, INFORMATION_SCHEMA, XP_, EXEC.

Rate limits (streamlit_app.py):

  • 10 per minute per browser session, kept in memory.
  • 50 per hour per user in the RATE_LIMIT table, using one MERGE (insert-or-add-one in one statement) keyed by user and the hour window.
  • An audit guard also counts only APPROVED and APPROVE_DENIED rows. Failed logins are left out, so a stranger cannot lock out a real officer by typing wrong passwords.
  • Any error in the check blocks the request (fail closed).

2 · Sorting the question

In plain words

The app decides what kind of question it got: “write a report”, “show me rules” or “review transactions”. It only reacts to whole words, so “necessary” is not mistaken for “SAR”.

Under the hood

classify_intent() tests three compiled regexes with \b word edges, in this order:

sar_draft      \b(?:sars?|suspicious|report(?:s|ed|ing)?)\b
policy_lookup  \b(?:polic(?:y|ies)|rules?|clauses?|regulations?)\b
tx_review      \b(?:transactions?|payments?|transfer(?:s|red|ring)?|tx)\b
otherwise      general

No AI is used here. It is deterministic (same input, same answer), free, and testable offline. The user's text never becomes SQL. Queries come from fixed templates, and user values are passed as bound parameters (sent separately from the SQL text).

3 · Evidence rows: the receipts

In plain words

The app pulls recent transactions from the bank's tables as proof. Analysts never see accounts marked RESTRICTED; officers do. Customer names are hidden from non-officers.

Under the hood

fetch_transactions() builds one fixed query over TRANSACTIONS T:

SELECT T.TX_ID, T.ACCOUNT_ID, T.AMOUNT, T.CURRENCY, T.TX_DATE, T.MERCHANT
FROM TRANSACTIONS T WHERE 1=1
  [AND T.ACCOUNT_ID LIKE %s ESCAPE '\\']          -- filter, bound + LIKE-escaped
  [AND EXISTS (... A.RISK_TIER != 'RESTRICTED')]  -- non-officers only
ORDER BY T.TX_DATE DESC LIMIT 5 OFFSET 0
  • LIMIT/OFFSET are Python ints that are clamped (limit 1–20, offset 0–10000), then written in as digits. The docs do not prove that server-side binding of LIMIT works, so they are never bound and never come from user text.
  • Two layers of hiding:
    • the app-side filter and mask_rows();
    • Snowflake row access policies (ACCOUNTS_ROLE_FILTER, TRANSACTIONS_ROLE_FILTER) plus the masking policy MASK_CUSTOMER_NAME in sql/01_setup.sql.

4 · Finding the law by meaning, not by keyword

In plain words

Every rule clause was turned into a list of 768 numbers that captures its meaning. The question is turned into the same kind of list. Clauses whose numbers “point the same way” as the question are the best match, so the top three are shown with a score.

Under the hood

vector_search() runs inside Snowflake:

SELECT POLICY_ID, TITLE, CLAUSE_TEXT,
  VECTOR_COSINE_SIMILARITY(CLAUSE_VECTOR,
    AI_EMBED('snowflake-arctic-embed-m-v1.5', %s)) AS SIM
FROM REGULATORY_POLICIES ORDER BY SIM DESC LIMIT 3
  • CLAUSE_VECTOR is VECTOR(FLOAT, 768). Cosine similarity = (a·b) / (|a|·|b|), from −1 to 1, and near 1 means close in meaning.
  • The model name must be a literal in the SQL, so it is checked against ALLOWED_EMBED_MODELS first (it raises ValueError if not listed).
  • Each clause is cut to 400 characters (MAX_CLAUSE_CHARS).
  • Optional: Cortex Search (USE_CORTEX_SEARCH=TRUE, sql/04), which adds keyword match and reranking. Any error or zero hits falls back to cosine.
  • It runs once per question, and the same list feeds the screen, the prompt and the audit row.

5 · Writing the draft

In plain words

The app hands the AI a short, tightly fenced brief: the question, the proof rows and the three clauses, with the rule “facts only, list the IDs.” A harm filter is always on. If the main AI is unavailable, a smaller one takes over, with the same filter.

Under the hood

_build_prompt():

  • Uses up to 10 evidence lines (≤4000 characters) and up to 3 clauses.
  • Cuts the whole prompt to 350 words.

_run_ai_complete() sends:

SELECT AI_COMPLETE(model => %s, prompt => %s,
  model_parameters => {'guardrails': TRUE, 'temperature': 0})
  • Primary model: claude-sonnet-5 (GA, which means generally available; needs cross-region inference). Fallback: llama3.1-8b. Both are in an allow-list, and both run with Cortex Guard (Snowflake's harm filter).
  • temperature 0 gives the most repeatable wording, which suits an audit trail.
  • Fingerprints:
    • prompt_hash = SHA-256(prompt | model | version)
    • result_hash = SHA-256(report)
    Change one character and the hash changes.

6 · The officer gate: the one step AI never does

In plain words

Only a person on the officer list can approve. The check happens on the server at the moment of the click, not from what the screen says. If someone tries without permission, the attempt is written down under their real name.

Under the hood

approve_and_log() runs these checks in order:

  • The draft must have status == "DRAFT", and the approver must be non-empty.
  • Inside Snowflake (SiS): it reads the viewer from st.user.user_name at click time. The viewer must equal the approver and be in APP_OFFICERS (looked up with a bound value; a lookup error counts as “no”).
  • Local: the Snowflake login must hold COMPLIANCE_OFFICER (checked with IS_ROLE_IN_SESSION). ALLOW_LOCAL_APPROVE is a demo-only switch, default FALSE.
  • Pass: it inserts an APPROVED row with a Python uuid4 (no MAX(ID) race between two people saving at once), plus the evidence IDs, policy IDs, model, hashes and timestamp. It then saves the Snowflake QUERY_ID (cursor.sfqid) as extra proof.
  • Fail: it writes an APPROVE_DENIED row naming the real viewer, e.g. JSMITH (claimed BOSS), then raises PermissionError.

Where it runs: three homes, one codebase

HomeHow it is detectedHow it talks to dataWho it is for
LocaldefaultPython connector with secrets or a key file. %s placeholders. 30 s / 60 s query caps via ALTER SESSION.One developer or demo user (app password + role dropdown)
Inside Snowflake (SiS)Warehouse runtime: running_in_sis(). Container: SNOWFLAKE_HOST + /snowflake/session/token.st.connection("snowflake"). A small adapter switches %s → ? (qmark). There is no ALTER SESSION, so the time cap lives on the warehouse.Many users, each a real Snowflake identity
Browser demosys.platform == "emscripten" (Pyodide) or DEMO_MODE=TRUE. Never on inside SiS.No database. backend/demo.py has 8 sample rows and 4 sample clauses. The draft is a fixed template, and the audit log lives in memory.Judges and visitors on GitHub Pages (stlite 1.9.2 → Streamlit 1.62 in the browser)

Safe output

In plain words

Anything that came from a user or the AI is cleaned before it is shown, so it cannot sneak code into the page. The download contains findings and IDs, never the raw database queries.

Under the hood
  • clean_markdown() strips every <…> tag before display. The only HTML on the page is static CSS and badge shells, and badge labels pass through the stripper too. This stops XSS (script injection into a web page).
  • build_safe_export() writes the findings, evidence IDs, policy IDs and both hashes as Markdown, with no SQL.
  • AI and search failures are logged by error type only, because the full error text could echo prompt data.
Step 3 of the Feynman method · test what you claim

Proof it works

Test fileTestsWhat it proves
test_guards.py9Input guard and router basics
test_agent_sql.py35SQL shape, guardrails always on, model allow-lists, router edge cases, Cortex Search fallback, one search per question
test_app_smoke.py8Real Streamlit screens (AppTest) in local and SiS mode with no database
test_sis_mode.py41SiS detection, the ? adapter on real connector classes, officer list, denied-approval audit, masking, rate limit, int-only LIMIT
test_demo_mode.py9Demo never touches the database, hides RESTRICTED rows, only the officer approves, full run end to end

Every push runs:

  • CI: compile, ruff (code style checker) and the 102 tests.
  • Security: CodeQL (code flaw scanner, for Python and workflow files), Gitleaks (password leak scan) and pip-audit (known-bad library check).
  • Release: an automatic version number plus an SBOM (full list of parts).
  • Pages: redeploys the live demo.
Step 4 of the Feynman method · find the gaps

Honest limits

  • Not yet run on a live Snowflake account. The Cortex calls, SQL scripts, SiS deploy and Cortex Search are checked against the docs and fake-database tests only.
  • Evidence is the newest rows the viewer may see, not rows matched to the question's meaning. Matching evidence to the question is the next upgrade.
  • The browser demo does not call an AI. Its draft is a template, labelled as such on screen.
  • Local mode is single-user by design. Multi-user approval needs the Snowflake deploy.
  • Later: SSO (company login), multi-team roles, and shared per-minute limits.
Step 5 of the Feynman method · say it back simply

Say it back in five sentences

  1. The question is checked at the door and slowed down if it comes too often.
  2. The app sorts the question with whole-word rules, then pulls proof rows with fixed, safe SQL.
  3. It finds the three closest rule clauses by meaning, using 768-number vectors.
  4. An AI writes a short draft with its harm filter always on, and the draft is fingerprinted.
  5. Only a listed officer, checked on the server at click time, can turn the draft into a permanent audit record.

Word list

SAR
Suspicious Activity Report: the form a bank files when money movement looks wrong.
Cortex AI
Snowflake's built-in AI functions, called from SQL (AI_EMBED, AI_COMPLETE).
Embedding
A list of numbers that stores the meaning of a text, so meanings can be compared.
Cosine similarity
How much two number-lists point the same way; 1 = same meaning.
Bound parameter
A value sent separately from the SQL text, so it can never change the query.
Cortex Guard
Snowflake's filter that blocks harmful AI answers.
HITL
Human in the loop: a person must approve before anything final happens.
SiS
Streamlit in Snowflake: the app runs inside the customer's Snowflake account.
Fail closed
If a safety check breaks, the answer is “no”.
stlite / Pyodide
Python and Streamlit running inside a web browser, with no server.
SHA-256
A fingerprint of text; any change gives a completely different fingerprint.
SBOM
Software bill of materials: the full parts list of a release.