Library Catalog Data Sync Pipelines

Mapping NCIP LookupUser Responses to ILS Patron Records

This page answers one narrow operational question: when a partner agency returns an NCIP LookupUserResponse for a consortial loan, why does the UserId it carries so often resolve to the wrong local patron — or to no patron at all — and how do you build a resolver that maps a borrowing agency’s identifier to the correct ILS record every time? It sits under NCIP Resource-Sharing Workflows, which covers the full request/response choreography, and within the broader Circulation Protocols & Interoperability architecture. If you run a resource-sharing hub where several ILS instances trade patrons and items, this identifier collision is one you will hit the first time two agencies mint the same bare UserId.

Resolving an NCIP LookupUser response to a local ILS patron via a composite key A partner agency LookupUserResponse carrying AgencyId NER-LIB and UserId 44821 enters a normalization adapter selected by AgencyId. The adapter rewrites the raw UserId into a canonical composite key, agency plus local identifier, which is used to look up the correct local ILS patron record. A dashed branch shows that keying on the bare UserId 44821 alone collides with a different local patron who happens to share that number, so that path is rejected. LookupUserResponse AgencyId: NER-LIB UserId: 44821 normalize adapter select by AgencyId emit composite key local ILS patron key: NER-LIB:44821 exactly one match bare UserId 44821 collides — wrong patron rejected path

Problem Framing

The symptom is a mis-routed loan, not an error. A patron at a partner library places a request, the resource-sharing hub sends a LookupUser to their home agency, and the response comes back clean — a UserId, a name, a set of privileges. Your local circulation code takes that UserId, looks up a patron, finds one, and charges the item out. Everything reports success. Days later a borrower complains that a book they never requested is on their account, or a partner reports that a legitimate request silently failed to resolve. What you have is an identifier collision: the UserId from the partner matched a local patron who happens to carry the same number.

Here is a representative LookupUserResponse from a partner agency, trimmed to the fields that matter:

text
<ns1:LookupUserResponse xmlns:ns1="http://www.niso.org/2008/ncip">
  <ns1:UserId>
    <ns1:AgencyId>NER-LIB</ns1:AgencyId>
    <ns1:UserIdentifierValue>44821</ns1:UserIdentifierValue>
  </ns1:UserId>
  <ns1:UserOptionalFields>
    <ns1:NameInformation>
      <ns1:PersonalNameInformation>
        <ns1:StructuredPersonalUserName>
          <ns1:GivenName>Amara</ns1:GivenName>
          <ns1:Surname>Okafor</ns1:Surname>
        </ns1:StructuredPersonalUserName>
      </ns1:PersonalNameInformation>
    </ns1:NameInformation>
    <ns1:UserPrivilege>
      <ns1:AgencyId>NER-LIB</ns1:AgencyId>
      <ns1:UserPrivilegeType>BORROWING</ns1:UserPrivilegeType>
      <ns1:UserPrivilegeStatus>
        <ns1:UserPrivilegeStatusType>OK</ns1:UserPrivilegeStatusType>
      </ns1:UserPrivilegeStatus>
    </ns1:UserPrivilege>
  </ns1:UserOptionalFields>
</ns1:UserId>

The UserIdentifierValue is 44821. Your local ILS also has a patron 44821 — a completely different person, minted years ago under your own numbering scheme. A lookup keyed on that value alone silently returns your patron, and the loan is charged to the wrong account.

Root Cause

The failure is a namespace assumption: the code treats a bare UserId as if it were globally unique when it is only locally unique. Every agency in a resource-sharing arrangement mints its own UserIdentifierValue from its own sequence. NER-LIB’s 44821 and your library’s 44821 are unrelated integers that collide by coincidence. NCIP anticipates exactly this — that is why the UserId element wraps the identifier value inside an AgencyId. The value is meaningful only in the scope of its agency.

Two further wrinkles compound the collision. First, agencies use different identifier schemes: one sends a bare integer, another sends a barcode with a check digit, a third sends an institution-prefixed string like NER-44821, and a fourth pads to a fixed width (0044821). Comparing these as raw strings fails even when they refer to the same person. Second, AgencyId values themselves can collide or drift — a partner may send NER-LIB in one message and NER_LIB or ner-lib in another, or the consortium may reassign a code. Keying on the identifier alone, or on a naively concatenated string, produces either a false match or a missed match.

The fix is to stop treating UserIdentifierValue as a primary key. Resolve on the composite of AgencyId and the normalized UserId, with a per-agency adapter that knows how that partner encodes its identifiers — the same disposition-per-source discipline the parent NCIP Resource-Sharing Workflows guide applies to the whole message exchange.

Solution

Parse the response with a hardened XML parser, extract the (AgencyId, UserIdentifierValue) pair, run it through a normalization adapter chosen by agency, and look the patron up on the composite key. Untrusted XML from a partner agency must be parsed with defusedxml to close entity-expansion and external-entity attacks; the standard-library parser is not safe for foreign input.

python
from __future__ import annotations

import logging
from dataclasses import dataclass

from defusedxml.ElementTree import fromstring
from xml.etree.ElementTree import Element

logger = logging.getLogger("ncip.lookupuser")

NCIP_NS = "http://www.niso.org/2008/ncip"


class LookupUserError(Exception):
    """Base class for unresolvable LookupUser responses."""


class MalformedResponseError(LookupUserError):
    """The response did not carry a well-formed UserId."""


class UnknownAgencyError(LookupUserError):
    """No normalization adapter is registered for the sending agency."""


class PatronNotFoundError(LookupUserError):
    """The composite key resolved to no local patron."""


@dataclass(frozen=True)
class PartnerUser:
    agency_id: str
    raw_user_id: str
    given_name: str | None
    surname: str | None
    borrowing_ok: bool


@dataclass(frozen=True)
class AgencyAdapter:
    """Per-agency identifier normalization rules."""

    canonical_agency: str          # collapse casing/underscore variants to one code
    strip_prefix: str | None = None  # e.g. "NER-" for institution-prefixed ids
    zero_pad_width: int | None = None

    def normalize(self, raw_user_id: str) -> str:
        value = raw_user_id.strip()
        if self.strip_prefix and value.startswith(self.strip_prefix):
            value = value[len(self.strip_prefix):]
        value = value.lstrip("0") or "0"
        if self.zero_pad_width:
            value = value.zfill(self.zero_pad_width)
        return value


# Registry keyed on the AgencyId as it appears on the wire, folded to lower case.
ADAPTERS: dict[str, AgencyAdapter] = {
    "ner-lib": AgencyAdapter(canonical_agency="NER-LIB"),
    "ner_lib": AgencyAdapter(canonical_agency="NER-LIB"),
    "wcpl": AgencyAdapter(canonical_agency="WCPL", strip_prefix="WCPL-"),
    "statelib": AgencyAdapter(canonical_agency="STATELIB", zero_pad_width=8),
}


def _text(node: Element | None) -> str | None:
    return node.text.strip() if node is not None and node.text else None


def parse_lookup_user(xml_bytes: bytes) -> PartnerUser:
    """Safely parse a LookupUserResponse into a typed PartnerUser."""
    root = fromstring(xml_bytes)  # defusedxml: no entity expansion, no external refs
    ns = {"ncip": NCIP_NS}

    user_id_el = root.find(".//ncip:UserId", ns)
    if user_id_el is None:
        raise MalformedResponseError("response carried no UserId element")

    agency_id = _text(user_id_el.find("ncip:AgencyId", ns))
    raw_user_id = _text(user_id_el.find("ncip:UserIdentifierValue", ns))
    if not agency_id or not raw_user_id:
        raise MalformedResponseError("UserId missing AgencyId or identifier value")

    priv = root.find(".//ncip:UserPrivilege/ncip:UserPrivilegeStatus"
                     "/ncip:UserPrivilegeStatusType", ns)
    return PartnerUser(
        agency_id=agency_id,
        raw_user_id=raw_user_id,
        given_name=_text(root.find(".//ncip:GivenName", ns)),
        surname=_text(root.find(".//ncip:Surname", ns)),
        borrowing_ok=_text(priv) == "OK",
    )


def composite_key(user: PartnerUser) -> tuple[str, str]:
    """Resolve (AgencyId, UserId) to a canonical, collision-free composite key."""
    adapter = ADAPTERS.get(user.agency_id.strip().lower())
    if adapter is None:
        logger.error(
            "unknown_partner_agency",
            extra={"agency_id": user.agency_id, "action": "reject"},
        )
        raise UnknownAgencyError(f"no adapter for agency {user.agency_id!r}")
    normalized = adapter.normalize(user.raw_user_id)
    return adapter.canonical_agency, normalized

The database lookup then binds both halves of the key, so a local 44821 can never satisfy a query scoped to agency NER-LIB:

python
def resolve_patron(user: PartnerUser, *, conn) -> str:
    """Return the local patron primary key for a partner user, or raise."""
    agency, local_id = composite_key(user)
    with conn.cursor() as cur:
        cur.execute(
            "SELECT patron_pk FROM partner_patron_link "
            "WHERE source_agency = %s AND source_user_id = %s",
            (agency, local_id),
        )
        row = cur.fetchone()
    if row is None:
        logger.warning(
            "partner_patron_unresolved",
            extra={"agency_id": agency, "source_user_id": local_id},
        )
        raise PatronNotFoundError(f"{agency}:{local_id} maps to no local patron")
    logger.info(
        "partner_patron_resolved",
        extra={"agency_id": agency, "patron_pk": row[0]},
    )
    return row[0]

The behavioural change is that the join key is now (source_agency, source_user_id), never the bare identifier. NER-LIB’s 44821 and your local 44821 occupy different rows in partner_patron_link and cannot be confused. An identifier from an agency with no registered adapter fails closed with UnknownAgencyError instead of guessing, and a normalized key that matches nothing raises PatronNotFoundError so the loan is refused rather than mis-charged.

Compliance or Privacy Impact

A LookupUserResponse carries another agency’s patron PII — a name, sometimes an address or email in UserOptionalFields. The resolver above deliberately keeps only what the loan needs: the composite key and the borrowing-privilege flag. Do not persist the partner’s NameInformation into your own patron table. Store the link row (source_agency, source_user_id, patron_pk) and let the partner remain the system of record for its own patrons’ details, exactly the minimisation stance described in PII Masking in Patron Data Exports.

Two further points follow from that. First, the source_user_id you retain is still a foreign identifier for a real person; treat it as PII subject to your retention schedule, and purge the link row when the consortial loan closes and the retention window lapses rather than keeping it indefinitely. Second, log the resolution decision, not the payload — the extra={...} fields above record agency and patron key but never the name or privilege detail, so your operational logs do not become a second, unmanaged copy of partner PII. If a partner agency requests erasure for one of its patrons, a link keyed on (source_agency, source_user_id) is trivial to locate and delete, whereas name-based matching would leave orphaned copies behind.

Verification

Confirm the resolver is collision-safe with a test that feeds two agencies the same bare identifier and asserts they resolve to different local patrons, plus a test that an unknown agency fails closed.

python
import pytest

RESPONSE = b"""<ns1:LookupUserResponse xmlns:ns1="http://www.niso.org/2008/ncip">
  <ns1:UserId>
    <ns1:AgencyId>NER_LIB</ns1:AgencyId>
    <ns1:UserIdentifierValue>0044821</ns1:UserIdentifierValue>
  </ns1:UserId>
  <ns1:UserOptionalFields>
    <ns1:UserPrivilege><ns1:UserPrivilegeStatus>
      <ns1:UserPrivilegeStatusType>OK</ns1:UserPrivilegeStatusType>
    </ns1:UserPrivilegeStatus></ns1:UserPrivilege>
  </ns1:UserOptionalFields>
</ns1:UserId>"""


def test_agency_variant_and_padding_normalize_to_one_key() -> None:
    user = parse_lookup_user(RESPONSE)
    # NER_LIB casing/underscore variant folds to canonical NER-LIB,
    # and the zero-padded 0044821 strips to 44821.
    assert composite_key(user) == ("NER-LIB", "44821")
    assert user.borrowing_ok is True


def test_bare_id_does_not_collide_across_agencies() -> None:
    ner = PartnerUser("NER-LIB", "44821", None, None, True)
    wcpl = PartnerUser("WCPL", "WCPL-44821", None, None, True)
    assert composite_key(ner) != composite_key(wcpl)


def test_unknown_agency_fails_closed() -> None:
    stranger = PartnerUser("XX-UNKNOWN", "44821", None, None, True)
    with pytest.raises(UnknownAgencyError):
        composite_key(stranger)

For a running hub, add an operational invariant: alert on any nonzero rate of partner_patron_unresolved or unknown_partner_agency after a new partner joins the consortium — it is the early signal that a partner’s identifier scheme needs a new AgencyAdapter before its loans can resolve. Feed the same events into the timeout-recovery path so an unresolved lookup does not later masquerade as a stalled CheckOutItem request.