Library Catalog Data Sync Pipelines

Automating FERPA Compliance in Student Patron Records

This page answers one narrow operational question: why do FERPA-protected education-record fields — enrollment status, advisor, GPA, disciplinary flags — keep surfacing in patron exports after a Student Information System (SIS) term sync, and how do you enforce automated, deterministic redaction so they can never reach a downstream feed? It sits under PII Masking in Patron Data Exports, which covers the export-masking contract end to end, and within the broader Patron Validation & Privacy Data Routing architecture. If you run an academic or K-12 library whose borrower records are provisioned from a campus SIS, the leak below is one you will eventually trigger on a term-rollover night.

Fail-closed FERPA allowlist gate for a SIS-to-ILS patron sync A wide SIS term export carrying a student's full education record — student_id, last_name, email, gpa, advisor_name, disciplinary_flag, branch_code and patron_type — enters a FERPA allowlist gate that classifies every field before storage. The gate fans out to four dispositions. TOKENIZE turns student_id into an HMAC-SHA-256 surrogate. KEEP retains library-operational fields such as branch_code and patron_type. DIRECTORY retains directory information such as last_name and email only when the student has not filed a directory-information opt-out. Any field with no declared disposition — including education records like gpa, advisor_name and disciplinary_flag, and any new field the SIS adds later — is treated as unclassified and dropped. The TOKENIZE, KEEP and DIRECTORY paths converge into a masked patron payload that is upserted into the ILS, while the unclassified path fails closed into a schema-validation quarantine queue for human review. A dashed boundary separates the SIS side, which handles the full record, from the ILS side, which sees masked fields only. An append-only audit log at the bottom records every keep, directory, drop and quarantine decision. SIS SIDE · full education record ILS SIDE · masked fields only classification boundary SIS term export default-open feed student_id last_name email gpa advisor_name disciplinary_flag branch_code patron_type FERPA allowlist gate classify every field declared → disposition unknown → fail closed TOKENIZE student_id → HMAC-SHA-256 KEEP branch_code · patron_type DIRECTORY last_name · email — opt-out gated UNCLASSIFIED → DROP gpa · advisor · disciplinary · new field masked patron payload allowlisted fields only ILS patron upsert schema-validation quarantine · review append-only audit log every KEEP · DIRECTORY · DROP · QUARANTINE decision recorded — provable at audit time

Problem Framing

The symptom is not a crash. The nightly SIS-to-ILS patron sync reports success, the borrower count is correct, and no exception fires. Weeks later a reporting extract — a consortium usage feed, a collection-development spreadsheet, a discovery-vendor log — is found to carry a column that should never have left the SIS: a student’s enrollment_status, an advisor_name, or worse, a disciplinary_flag. Under FERPA these are education records, and the library was never entitled to redistribute them.

The leak is visible if you diff what the SIS sent against what the patron index stored. A quick audit over a single term batch makes it obvious:

text
$ psql -c "SELECT jsonb_object_keys(raw_sis_payload) FROM patron_sync_audit \
           WHERE batch_id = '2026-fall-rollover' LIMIT 1 \\gx"
student_id
last_name
email
enrollment_status      <-- education record, should not be here
advisor_name           <-- education record, should not be here
gpa                    <-- education record, should not be here
disciplinary_flag      <-- education record, should not be here
branch_code
patron_type

Every one of those non-library fields was carried verbatim into the patron row because the sync mapper treated the SIS payload as an open dictionary and copied whatever keys arrived. The moment any of those keys reaches the masking stage of an export, it is already too late — the field is inside the trusted zone and one misconfigured extract away from disclosure.

Root Cause

The failure is a default-open field mapping. SIS exports are wide by design: a single Ellucian Banner, PowerSchool, or Workday Student feed emits the student’s full record because it does not know which consumer is reading it. The library patron sync typically consumes that feed with a permissive mapper — patron.update(**sis_record) or a SELECT *-style projection — that maps every inbound key to a patron attribute unless a field is explicitly excluded.

Deny-listing is the root cause. A denylist enumerates the fields you know are dangerous (gpa, disciplinary_flag) and drops them, but it silently admits any field the SIS adds later. When the registrar’s office enables a new immigration_status or financial_hold attribute mid-year, the denylist has no rule for it, so it flows straight through. Compounding this, FERPA draws a line the mapper does not encode: some fields are directory information (name, enrollment dates) that may be disclosed unless the student has filed a directory-information opt-out, while others are education records that may never be disclosed without consent. A mapper that keeps or drops by field name alone cannot represent “keep this field only when the opt-out flag is false.”

The fix is to invert the default. Every field must be classified before it is stored, exactly the disposition-per-field discipline used for masked exports in the parent PII Masking in Patron Data Exports contract — and any field with no classification must be rejected, not admitted.

Solution

Replace the permissive mapper with a strict allowlist gate. Fields are declared with an explicit disposition — KEEP for library-operational attributes, DIRECTORY for FERPA directory information subject to the opt-out flag, and everything else dropped. Any inbound key that is not declared is routed to a schema validation quarantine queue rather than stored, so a new SIS field fails closed and surfaces for human classification instead of leaking.

python
from __future__ import annotations

import enum
import hashlib
import hmac
import logging
from dataclasses import dataclass

logger = logging.getLogger("ferpa.gate")


class Disposition(enum.Enum):
    KEEP = "keep"            # library-operational, always retained
    DIRECTORY = "directory"  # FERPA directory info; retained unless opted out
    TOKENIZE = "tokenize"    # identity join key; stored only as HMAC surrogate


# Allowlist: any SIS field NOT listed here is dropped and quarantined.
FIELD_POLICY: dict[str, Disposition] = {
    "student_id": Disposition.TOKENIZE,
    "last_name": Disposition.DIRECTORY,
    "first_name": Disposition.DIRECTORY,
    "email": Disposition.DIRECTORY,
    "enrollment_dates": Disposition.DIRECTORY,
    "branch_code": Disposition.KEEP,
    "patron_type": Disposition.KEEP,
    "expiry_date": Disposition.KEEP,
}


@dataclass(frozen=True)
class GateResult:
    patron: dict[str, str]
    quarantined: dict[str, str]


def tokenize(value: str, *, secret: bytes) -> str:
    """Domain-separated HMAC-SHA-256 surrogate for a join key."""
    return hmac.new(secret, f"patron:{value}".encode(), hashlib.sha256).hexdigest()


def apply_ferpa_gate(
    sis_record: dict[str, str],
    *,
    secret: bytes,
    directory_opt_out: bool,
    batch_id: str,
) -> GateResult:
    """Classify every inbound SIS field before it can reach the patron index.

    Unknown fields are quarantined (fail-closed), education records are dropped,
    and directory information is suppressed when the student has opted out.
    """
    patron: dict[str, str] = {}
    quarantined: dict[str, str] = {}

    for key, value in sis_record.items():
        policy = FIELD_POLICY.get(key)
        if policy is None:
            quarantined[key] = value
            logger.warning(
                "unclassified_sis_field",
                extra={"field": key, "batch_id": batch_id, "action": "quarantine"},
            )
            continue
        if policy is Disposition.DIRECTORY and directory_opt_out:
            logger.info(
                "directory_suppressed",
                extra={"field": key, "batch_id": batch_id, "action": "drop"},
            )
            continue
        if policy is Disposition.TOKENIZE:
            patron[key] = tokenize(value, secret=secret)
        else:
            patron[key] = value

    return GateResult(patron=patron, quarantined=quarantined)

The behavioural change is the inversion of the default. Before, advisor_name and disciplinary_flag reached the patron row because nothing forbade them. After, they never appear in FIELD_POLICY, so they are dropped and their presence is logged as an unclassified field for review. When the registrar adds immigration_status next term, it too fails closed. The join key survives only as an HMAC-SHA-256 surrogate, matching the tokenization scheme the export-masking contract uses so that patron rows still join to circulation events without carrying a raw student_id.

Compliance or Privacy Impact

Automating the gate changes the FERPA posture in three concrete ways, and each has a downstream effect worth tracking.

The optimization does not widen the PII surface — it narrows it — but it shifts risk onto the correctness of FIELD_POLICY. A field mistakenly declared KEEP is a leak with the gate’s full authority behind it, so changes to the policy map deserve code review and the verification below.

Verification

Confirm the gate is fail-closed with a test that asserts every known education-record field is absent from the output and that an unseen field lands in quarantine rather than in the patron row.

python
import pytest

SECRET = b"unit-test-secret-not-for-prod"


def test_education_records_never_reach_patron() -> None:
    sis = {
        "student_id": "S1234567",
        "last_name": "Okafor",
        "gpa": "3.8",
        "advisor_name": "Dr. Reyes",
        "disciplinary_flag": "true",
        "branch_code": "MAIN",
        "immigration_status": "F-1",  # new field the SIS added this term
    }
    result = apply_ferpa_gate(
        sis, secret=SECRET, directory_opt_out=False, batch_id="test"
    )

    # No education record survives the gate.
    for forbidden in ("gpa", "advisor_name", "disciplinary_flag", "immigration_status"):
        assert forbidden not in result.patron

    # Unknown fields fail closed into quarantine, not the patron index.
    assert "immigration_status" in result.quarantined

    # The join key is present but only as an opaque surrogate.
    assert result.patron["student_id"] != "S1234567"
    assert len(result.patron["student_id"]) == 64  # sha256 hexdigest


def test_directory_opt_out_suppresses_name() -> None:
    sis = {"student_id": "S1", "last_name": "Okafor", "branch_code": "MAIN"}
    result = apply_ferpa_gate(
        sis, secret=SECRET, directory_opt_out=True, batch_id="test"
    )
    assert "last_name" not in result.patron  # suppressed on opt-out
    assert "branch_code" in result.patron    # operational field unaffected

For a running pipeline, add a continuous invariant on the patron index itself: periodically snapshot the column set of the stored patron table and assert it is a subset of the allowlisted keys. If a snapshot ever shows an unlisted column, halt the sync and quarantine the batch — the same fail-closed reflex the gate applies per field, applied to the table as a whole. Track unclassified_sis_field counts over time; a nonzero rate after a term rollover is your early signal that the SIS schema drifted and FIELD_POLICY needs a reviewed update.