§6 Knowledge Units — The Pattern Registry

Layer §5 of the Debt Collector Stack

Author: Shrikant Bhosale (@debtcollector21)

---

6.1 What It Is

The Knowledge Unit (KU) engine is a pattern registry that stores, matches, and generalizes vulnerability patterns. It serves as the bridge between the VMF scanner’s raw findings and the AttackGraph’s formal chain model.

A KU is not a CVE entry. It is a mathematical abstraction of a vulnerability class — a geometric description of what the vulnerability looks like in terms of preconditions, postconditions, and the information debt that enables it.

---

6.2 Architecture

┌──────────────────────────────────────────────────────────────┐
│                      KU Library                                │
│     Built-in (27 KUs)  │  Auto-registered (from scans)        │
│     ┌────────────────┐ │  ┌────────────────┐                   │
│     │ Security CPP   │ │  │ [Vendor] Scanner     │                   │
│     │ CS Foundations │ │  │ Binder Scanner │                   │
│     │ Scope Theory   │ │  │ [Vendor]       │                   │
│     └────────────────┘ │  └────────────────┘                   │
└──────────────────────────┬───────────────────────────────────-┘
                          ↓
┌──────────────────────────────────────────────┐
│              Match Engine                      │
│     Input: VMF Finding Dict                   │
│     Process: Fuzzy word overlap + geometry    │
│              + failure mode matching           │
│     Output: Matched KU list + similarity      │
└──────────────────┬───────────────────────────-┘
                   ↓
┌──────────────────────────────────────────────┐
│           NKU Capture (Negative KUs)           │
│     For findings that DON'T match any KU       │
│     → Novel pattern detected                   │
│     → Auto-registered for future matching      │
└──────────────────────────────────────────────┘

---

6.3 Knowledge Unit Definition

Every KU is a dataclass with the following fields:

@dataclass
class KnowledgeUnit:
    id: str                    # e.g., "KU-CPP-001"
    name: str                  # e.g., "Raw Pointer Across Boundary"
    domain: KUDomain           # security_cpp, cs_foundations, scope_theory, etc.
    patterns: list[str]        # VMF pattern IDs this KU covers
    cwe: list[int]             # CWE identifiers (e.g., [822, 823])
    preconditions: set[str]    # What must be true for this to trigger
    postconditions: set[str]   # What becomes possible after exploitation
    severity: str              # critical, high, medium, low, info
    description: str           # Human-readable explanation
    remediation: str           # How to fix
    references: list[str]      # CVE IDs, papers, blog posts
    curvature_baseline: float  # Expected κ if confirmed (0-1)
    examples: list[str]        # Source code examples

---

6.4 The Five Core KU Domains

6.4.1 Security C++ (13 KUs)

ID	Name	Preconditions	Postconditions	CWE
KU-CPP-001	Raw Pointer Across Boundary	controlled_input, serialized_data, reinterpret_cast	type_confusion, arbitrary_read	822, 823
KU-CPP-002	Null Dereference	controlled_null, unchecked_alloc	crash, dos	476
KU-CPP-003	OOB Array Access	controlled_index, no_bounds_check	heap_overflow, stack_overflow	122, 787
KU-CPP-004	Two-Phase API TOCTOU	shared_state, check_then_use, no_revalidation	privilege_escalation, state_corruption	367
KU-CPP-005	Untrusted memcpy	controlled_src, controlled_len, no_size_check	heap_overflow, stack_overflow	120
KU-CPP-006	Use After Free	free_keep_ref, ref_reuse	arbitrary_read, type_confusion	416
KU-CPP-007	Double Free	free_no_reset, multiple_paths	heap_corruption	415
KU-CPP-008	Format String	user_format, printf_no_fmt	arbitrary_read, info_leak	134
KU-CPP-009	Integer Overflow	unchecked_arithmetic, size_computation	heap_overflow, buffer_overflow	190
KU-CPP-010	DCHECK-only Bounds	debug_only_check, release_no_check	oob_access, heap_overflow	1295
KU-CPP-011	Nullptr Element Validator	object_array, type_check_no_null	null_deref, type_confusion	822
KU-CPP-012	Capacity/Size Confusion	set_size_no_capacity, write_beyond_capacity	heap_overflow, out_of_bounds	122
KU-CPP-013	Error Debt (Information)	unhandled_exception, unchecked_return	state_corruption, bypassed_validation	754

6.4.2 Go (8 KUs)

ID	Name	Preconditions	Postconditions
KU-GO-001	Unsafe Pointer	unsafe_package, pointer_arithmetic	type_confusion, arbitrary_read
KU-GO-002	Nil Dereference	nil_assignment, no_nil_check	crash, dos
KU-GO-003	Slice OOB	controlled_index, no_bounds_check	heap_overflow, panic
KU-GO-004	Goroutine Leak	unbounded_goroutine, no_wait_group	memory_exhaustion
KU-GO-005	Deferred Panic	panic_in_defer, no_recover	crash, dos
KU-GO-006	Channel Deadlock	unbuffered_channel, mismatched_send_recv	dos, hang
KU-GO-007	Race Condition	shared_mutex, concurrent_write	state_corruption
KU-GO-008	SQL Injection	string_concat_query, no_parameterize	data_leak, sqli

6.4.3 Rust (7 KUs)

ID	Name	Preconditions	Postconditions
KU-RS-001	Unsafe Block	unsafe_block, raw_pointer	type_confusion, arbitrary_read
KU-RS-002	Unchecked Unwrap	option_unwrap, result_unwrap, no_match	panic, dos
KU-RS-003	Ref Cycle	rc_clone, ref_cycle, no_weak	memory_leak
KU-RS-004	Transmute	transmute_call, type_mismatch	type_confusion
KU-RS-005	FFI Unsafe	extern_fn, unsafe_ffi, no_validate	arbitrary_code
KU-RS-006	Lifetime Confusion	lifetime_annotation, unsafe_lifetime	use_after_free
KU-RS-007	Send/Sync Violation	unsafe_send_sync, shared_mutation	race_condition

6.4.4 Java (7 KUs)

ID	Name	Preconditions	Postconditions
KU-JV-001	Deserialization RCE	object_input_stream, serializable, no_filter	arbitrary_code
KU-JV-002	SQL Injection	string_concat_query, no_prepared	data_leak
KU-JV-003	XSS	user_input_html, no_escape	xss_execution
KU-JV-004	Insecure Crypto	static_key, weak_algorithm, ecb_mode	data_leak
KU-JV-005	Android IPC Export	exported_component, no_permission	intent_injection
KU-JV-006	Root Detection Bypass	runtime_exec, file_check, no_obfuscation	bypass
KU-JV-007	Unchecked Close	close_in_finally, no_try_with_resources	resource_leak

6.4.5 Solidity (5 KUs)

ID	Name	Preconditions	Postconditions
KU-SOL-001	Reentrancy	external_call, state_after_call	drain_funds
KU-SOL-002	tx.origin Auth	tx_origin_check	phishing_auth_bypass
KU-SOL-003	Delegatecall Injection	delegatecall, controlled_target	arbitrary_logic
KU-SOL-004	Unchecked Call	call_return_ignored	silent_failure
KU-SOL-005	Timestamp Manipulation	block_timestamp_dependency	frontrunning

---

6.5 The Match Engine

Match engine implements three matching strategies:

6.5.1 Fuzzy Word Overlap

def fuzzy_match(finding: dict, ku: KnowledgeUnit) -> float:
    """
    Match by word overlap between finding description and KU patterns.

    Score = |intersection| / sqrt(|finding_words| × |ku_words|)
    Range: [0, 1]
    """
    finding_words = set(tokenize(finding.get("description", "")))
    ku_words = set()
    for pattern in ku.patterns:
        ku_words.update(tokenize(pattern))

    overlap = finding_words & ku_words
    return len(overlap) / (sqrt(len(finding_words)) * sqrt(len(ku_words)))

6.5.2 Geometry Match

def geometry_match(finding: dict, ku: KnowledgeUnit) -> float:
    """
    Match by geometric properties.

    Each KU has a geometry signature:
    - Does the finding involve serialization? (KU-CPP-001)
    - Does it involve an allocation? (KU-CPP-006)
    - Is there a type confusion element? (KU-CPP-011)

    Score = matched_properties / ku_properties
    """
    finding_props = extract_geometry_properties(finding)
    ku_props = set(ku.preconditions) | set(ku.postconditions)

    matched = sum(1 for prop in ku_props if prop in finding_props)
    return matched / max(1, len(ku_props))

6.5.3 Failure Mode Match

def failure_mode_match(finding: dict, ku: KnowledgeUnit) -> float:
    """
    Match by failure mode (ASAN crash type, error pattern).

    Each KU maps to specific failure modes:
    - KU-CPP-003 (OOB): heap-buffer-overflow, stack-buffer-overflow
    - KU-CPP-006 (UAF): use-after-free
    - KU-CPP-011 (nullptr validator): null_deref

    Score: 1.0 if crash type is in expected failures, else 0.0
    """
    crash_type = finding.get("metadata", {}).get("crash_type")
    if not crash_type:
        return 0.0

    expected = KU_FAILURE_MODE_MAP.get(ku.id, set())
    return 1.0 if crash_type in expected else 0.0

6.5.4 Combined Score

def match(finding: dict, ku: KnowledgeUnit) -> float:
    """
    Weighted combination of all three strategies.

    Weights reflect empirical reliability:
    - Failure mode match is most reliable (ASAN doesn't lie)
    - Geometry match is next (pre/post conditions are structural)
    - Fuzzy word overlap is least reliable (description may be noisy)
    """
    fm = failure_mode_match(finding, ku) * 0.5
    gm = geometry_match(finding, ku) * 0.3
    fw = fuzzy_match(finding, ku) * 0.2

    return fm + gm + fw

---

6.6 NKU Capture — Negative Knowledge Units

When a finding does not match any existing KU, an NKU is created:

@dataclass
class NegativeKnowledgeUnit:
    id: str
    pattern_name: str
    domain: KUDomain
    severity: str
    description: str
    geometry_signature: set[str]
    failure_modes: set[str]
    source: str  # file:line of first instance
    instances: int

NKUs are the learning mechanism of the KU engine. Every NKU represents:
1. A gap in the VMF scanner’s pattern coverage
2. A vulnerability class that was not previously catalogued
3. An opportunity to improve the scanner

When an NKU reaches instances >= 3 (same pattern in 3 different codebases), it is automatically promoted to a full KU and added to the library.

---

6.7 Pre/Post Condition Chaining

Each KU’s preconditions and postconditions enable chain building:

def can_chain(ku_a: KnowledgeUnit, ku_b: KnowledgeUnit) -> bool:
    """
    Can KU_A and KU_B form a chain?

    Yes if any postcondition of KU_A satisfies any precondition of KU_B.
    """
    prerequisite_match = ku_a.postconditions & ku_b.preconditions
    return len(prerequisite_match) > 0

Example chain:

KU-CPP-001 (Raw Pointer Across Boundary)
  postconditions: {type_confusion, arbitrary_read}
  → satisfies precondition of:
KU-CPP-011 (Nullptr Element Validator)
  preconditions: {object_array, type_check_no_null, type_confusion}
  → chain: read primitive → null deref → write primitive

This is the same chain that underpins the A01/A02 AOSP finding (CVSS 8.8).

---

6.8 Validation

Metric	Value
Total KUs	40 (27 built-in + 13 auto-registered)
Total NKUs	0 (all novel findings so far matched existing KUs)
Match accuracy (known)	100% (43/43 on [Vendor] Wasm)
Chain composition success	100% (35 KUs form valid pre/post chains)
Cross-language chaining	3 chains (C++ → Java via JNI, C++ → Go via cgo)

---

6.9 CLI Usage

# List all KUs
vmf ku library

# Show specific KU
vmf ku library --id KU-CPP-001

# Extract KUs from VMF scan results
vmf ku extract findings.json

# Verify a finding against KU library
vmf ku verify finding.json

# Match a new finding
vmf ku match finding.json

# Show NKU capture log
vmf ku nku

# Export KU library
vmf ku export --format json
vmf ku export --format markdown