Ditch Distance

The Australian Government Information Security Manual (AUISM), published by the Australian Signals Directorate, and the New Zealand Information Security Manual (NZISM), published by the Government Communications Security Bureau, both set out baseline security controls for government systems in their respective jurisdictions.

Australia and New Zealand are close neighbours "across the ditch", with tightly integrated economies and a shared cyber-security heritage, so the two ISMs cover much of the same ground. Where they part company is in character and cadence. NZISM tends toward broader, multi-component controls written in prescriptive, RFC-2119-like MUST / SHOULD compliance language, and its technology posture still leans pre-cloud (traditional on-premise ICT, data centre co-location). AUISM presents more discrete, specific controls as outcome statements, with a more cloud-native posture. AUISM ships a new release roughly every quarter; the NZISM cycle is closer to 15–18 months. The gap that produces is easiest to see at the frontier: AUISM has been iterating on AI system risks for a while now, and NZISM v3.9 doesn't yet cover most of that ground. See Character of each manual below for how these differences affect the scoring.

Ditch Distance measures how far apart the two ISMs are, control by control. For each control on one side it finds the closest match on the other side using any of five scoring methods; the mechanics are described in Statistical Methods.

Same data, one fixed lens. Every AUISM release is scored against the current NZISM using RRF fusion at threshold 0.75, and each tier is drawn as its own line so you can watch it move from version to version. The dashed line is the median rank-1 RRF score for that release (right axis). When the median climbs and the gap band shrinks, that AUISM release has drifted into ground the NZISM already covers. When the median drops and the gap band widens, the AUISM has moved into new territory the NZISM hasn't caught up on. The second chart flips the direction and asks the mirror question: how well does each AUISM release actually cover the NZISM?

This is not a substitute for the official ISM releases. It's an analytical aid for security and compliance practitioners doing mapping, alignment, and crosswalk work between the two frameworks. If you need the authoritative text, go to the issuing agencies directly:

• AUISM — cyber.gov.au/business-government/asds-cyber-security-frameworks/ism
• NZISM — nzism.gcsb.govt.nz/ism-document

Both catalogs are compared in full. Every control in the AUISM OSCAL catalog is matched against every control in the NZISM OSCAL catalog (and vice versa) — no chapter is excluded up front.

Earlier iterations of this tool dropped NZISM's four prefatory / governance chapters (About Information Security, Information Security Services within Government, Information security governance — roles and responsibilities, System Certification and Accreditation) on the grounds that the AUISM OSCAL catalog carries only control statements and has no direct analogue for that framing material. Now that NZISM OSCAL is available (unofficial derivative of the GCSB-published NZISM), the cleaner move is to compare everything and let the scoring surface the asymmetry: those NZ chapters will naturally show up as low-score rank-1 matches (i.e. gaps) rather than being hidden. Reading the gap bucket, keep in mind that a portion of it is structural — NZ governance chapters with no AU counterpart — rather than genuine coverage drift.

Beyond raw coverage, the two manuals differ in how they express controls — which affects how the scoring behaves and how the results should be read:

• Granularity. NZISM tends to have broader, multi-component / combined requirement controls; AUISM presents more discrete and specific requirements. A single NZ control often maps to several AU controls, and vice versa.
• Language. NZISM leans prescriptive, written in RFC-2119-like MUST / SHOULD / MUST NOT / SHOULD NOT compliance language. AUISM generally provides outcome statements without the same normative verbs.
• Technology posture. NZISM has a significant focus on pre-cloud (traditional on-premise ICT, data centre co-location); AUISM is more cloud-native in posture.

These differences mean that a low similarity score is not automatically a coverage gap — it may just be the two manuals saying the same thing at different levels of granularity, in different registers.

Flip the perspective (AUISM → NZISM or the other way), change the scoring method, or move the match threshold — the tables below update live. The dual score-range slider is useful when you want to focus on one tier at a time. Dragging both handles down into the borderline / gap zone, for instance, hides the strong matches and leaves only the candidates worth investigating for genuine coverage gaps.

Under the hood, each source-side control is matched to its closest counterpart on the other side. There are five ways to do that matching, each with different strengths:

• RRF fusion (default) — Reciprocal Rank Fusion combines the ranks (not the raw scores) of the four methods below using sum(1/(60 + rank)). Consensus picks float to the top; outliers from any one method are damped.
• TF-IDF cosine — fast lexical overlap over 1–2 word n-grams. Rewards shared vocabulary; weak on paraphrase.
• Sentence-transformer embedding — pooled semantic vector (all-MiniLM-L6-v2), cosine similarity. Handles paraphrase but can over-score topically-adjacent controls that don't actually correspond.
• SecureBERT bi-encoder — same shape as the sentence-transformer method but using a cyber-security-domain BERT (ehsanaghaei/SecureBERT) with mean-pooled last-hidden-state, so domain-specific jargon (ACL, MFA, SIEM, SBOM, ...) is weighted more like a security practitioner would weight it.
• Cross-encoder rerank — the top candidates from TF-IDF and the sentence-transformer are re-scored jointly by a cross-encoder (bge-reranker-base) that reads both control statements together. Slower, but visibly reduces the false-positive "strong-solid" matches that pooled cosine sometimes produces.

Why RRF is the default. Every one of the four base methods has a weakness we already know about. TF-IDF can't see paraphrase. The MiniLM sentence-transformer over-scores anything topically nearby. SecureBERT out of the box crushes almost every pair into a narrow 0.95–0.99 window — without a proper fine-tuned sentence head it just isn't a good discriminator. And the cross-encoder is limited to whatever shortlist the cheaper methods put on its plate.

RRF sidesteps most of that by fusing on rank position rather than raw score. A target that shows up near the top for most methods rises regardless of what the absolute numbers say, and a target that only one method likes gets damped. In practice this knocks out a lot of the false-positive "strong-solid" matches you see when the bi-encoder methods run on their own. The other four are still there in the dropdown — often the more interesting signal is where they disagree with RRF, or with each other.

All five methods are precomputed in both directions and can be toggled above without regenerating the report.

Why not use an LLM for this? You can hand an LLM a pair of controls and ask it for a similarity score, and it will cheerfully give you a number that looks about right. The problem starts as soon as anyone asks why. Security and compliance work has to be defensible: an analyst needs to be able to point at a claim ("control X is equivalent to control Y", "control X is a gap") and walk through the reasoning. LLMs don't give you that. They can't faithfully report how they arrived at a score, they invent supporting detail when pressed, and they lean toward calling things a match even when the evidence is thin.

The methods used here are older, but every step is inspectable. Each score is a deterministic function of the input text and the published model weights. You can pull out the intermediate rankings. You can compare one method against another, and the places where they disagree are often the most interesting rows in the table. LLMs still have a role in this kind of work — most naturally as a downstream summariser, explaining a candidate match to a human reviewer once the shortlist is drawn — but not as the primary scoring engine.