SIP2 vs NCIP: Choosing the Right Circulation Protocol for Your ILS

Operating within the broader Circulation Protocols & Interoperability architecture, this guide answers the question every ILS admin eventually faces when wiring a new device or partner into circulation: is this a job for SIP2 or for NCIP? The two protocols are not competitors so much as tools shaped for different jobs — SIP2 protocol integration is a terse, socket-based dialogue built for self-check machines and payment kiosks standing next to the ILS, while NCIP resource-sharing workflows speak a verbose XML over HTTP designed to let two different library systems lend to each other across a consortium. Choosing wrongly is expensive: pick SIP2 for a cross-institution interlibrary-loan flow and you will hand-roll patron-mapping logic the protocol was never meant to carry; pick NCIP for a lobby self-check unit and you will pay XML-parsing and TLS-handshake latency on an operation that should complete in milliseconds. This page lays out the contract side by side, gives you a decision matrix, builds a Python selector that routes a circulation operation to the right backend, and shows how to run both protocols at once — because most real deployments do.

Specification & Contract

SIP2 and NCIP answer the same underlying question — may this patron do this thing with this item, and record it — but they were designed a decade and a philosophy apart. SIP2 (the 3M Standard Interchange Protocol, version 2) is a device protocol: a self-check unit opens a raw TCP socket to the ILS, logs in once, and then trades short fixed-position messages for the duration of a patron’s session. NCIP (the NISO Circulation Interchange Protocol, Z39.83) is an application protocol: each operation is a self-contained XML document posted over HTTP, carrying its own full context because the two systems on either end may belong to entirely different institutions that share no session state.

The decision almost always comes down to who is on the other end. If the far side is a fixed device that belongs to your library and talks only to your ILS, SIP2’s terseness is a feature. If the far side is another library’s ILS — a resource-sharing partner, an interlibrary-loan broker, a discovery layer requesting a hold on your behalf — NCIP’s self-describing, schema-validated messages are what make the exchange auditable and vendor-neutral.

Read the table as two commitments rather than a scorecard. SIP2’s commitments — a held-open socket, positional ASCII, an in-band checksum — all serve low latency at a single trusted endpoint. NCIP’s commitments — stateless HTTP, verbose XML, schema validation — all serve safe interoperation between systems that do not trust each other’s internals. Neither set is strictly better; each is the wrong set for the other’s job. The privacy row deserves special attention: SIP2 puts a patron barcode and PIN onto the wire in a form a packet capture can read unless you tunnel it, which is why the field-masking discipline in PII masking in patron data exports has to extend all the way down to the transport for self-check traffic.

Prerequisites & Environment Setup

The examples target Python 3.11+ and treat both protocols as pluggable backends behind one interface. You do not need both client libraries installed to follow the selector pattern, but a real deployment that runs both will pin both. SIP2 has no single canonical Python client, so socket handling is usually hand-rolled or wrapped around a thin library; NCIP is XML over HTTP, so httpx plus lxml is the common pairing.

• Python >= 3.11 in the circulation gateway image (python --version)
• httpx >= 0.27 pinned for the NCIP HTTP client (timeouts and TLS verification configured explicitly)
• lxml >= 5.0 for schema-validated NCIP document building and parsing
• SIP2 endpoint host/port and login credentials resolved from a secrets manager, never literals
• NCIP responder URL, FromAgencyId, and ToAgencyId configured per partner
• TLS material available for both lanes — an stunnel/tunnel config for SIP2, a CA bundle or client cert for NCIP
• Structured (JSON) log handler configured for the circulation_router logger, shipped to append-only storage
• A capability map reviewed against your ILS: not every ILS implements every NCIP service, and self-check vendors vary in which SIP2 fields they populate — the same normalization discipline described in ILS schema translation patterns applies before either protocol is dispatched

Core Implementation

The goal is to make the SIP2-versus-NCIP choice a configuration decision rather than a decision scattered through your call sites. A circulation operation — checkout, checkin, patron lookup — should be expressed once, in protocol-neutral terms, and a selector should route it to whichever backend serves that operation for that endpoint. That gives you one place to reason about the trade-offs in the table above, and one place to change when a partner migrates from one protocol to the other.

Step 1 — Define the protocol-neutral operation and result

Nothing in the calling code should mention sockets or XML. A small set of typed request/result objects carries only what both protocols need: the operation, the patron, the item, and the endpoint that decides the lane.

from __future__ import annotations

import logging
from dataclasses import dataclass
from enum import Enum
from typing import Optional, Protocol, runtime_checkable

router_logger = logging.getLogger("circulation_router")


class Operation(str, Enum):
    CHECKOUT = "checkout"
    CHECKIN = "checkin"
    LOOKUP_USER = "lookup_user"


@dataclass(frozen=True)
class CircRequest:
    """A protocol-neutral circulation request. Knows nothing about SIP2 or NCIP."""

    operation: Operation
    patron_barcode: str
    endpoint: str
    item_barcode: Optional[str] = None


@dataclass(frozen=True)
class CircResult:
    """A normalized result both backends map their native reply into."""

    ok: bool
    operation: Operation
    detail: str
    protocol: str

Step 2 — Declare the backend interface as a typed Protocol

A typing.Protocol states the contract each backend must satisfy without forcing a shared base class — the SIP2 and NCIP adapters can be written independently and still be interchangeable at the call site. @runtime_checkable lets the selector assert the contract at wiring time.

@runtime_checkable
class CirculationBackend(Protocol):
    """Structural contract every circulation backend implements."""

    name: str

    def supports(self, operation: Operation) -> bool:
        """True if this backend handles the given operation."""
        ...

    def execute(self, request: CircRequest) -> CircResult:
        """Perform the operation against the underlying protocol."""
        ...

Step 3 — Implement the two adapters

Each adapter hides its protocol’s mechanics behind execute. The SIP2 adapter manages a stateful socket session through a context manager; the NCIP adapter builds a self-contained XML document and posts it. The calling code sees neither detail.

class Sip2Backend:
    """Fixed-field ASCII over a persistent TCP socket. Built for self-check."""

    name = "sip2"
    _SUPPORTED = {Operation.CHECKOUT, Operation.CHECKIN}

    def __init__(self, session_factory) -> None:
        # session_factory yields a connected, logged-in SIP2 session
        # (a context manager wrapping the raw socket).
        self._session_factory = session_factory

    def supports(self, operation: Operation) -> bool:
        return operation in self._SUPPORTED

    def execute(self, request: CircRequest) -> CircResult:
        with self._session_factory() as session:
            # e.g. "11" = Checkout request; fields are positional + delimited.
            reply = session.transact(request)
        router_logger.info(
            "sip2 transaction complete",
            extra={"operation": request.operation.value, "endpoint": request.endpoint},
        )
        return CircResult(
            ok=reply.accepted,
            operation=request.operation,
            detail=reply.screen_message,
            protocol=self.name,
        )


class NcipBackend:
    """XML over HTTP. Built for cross-institution resource sharing."""

    name = "ncip"
    _SUPPORTED = {
        Operation.CHECKOUT, Operation.CHECKIN, Operation.LOOKUP_USER,
    }

    def __init__(self, client, responder_url: str) -> None:
        self._client = client            # an httpx.Client with TLS configured
        self._responder_url = responder_url

    def supports(self, operation: Operation) -> bool:
        return operation in self._SUPPORTED

    def execute(self, request: CircRequest) -> CircResult:
        document = build_ncip_document(request)   # returns validated XML bytes
        response = self._client.post(
            self._responder_url,
            content=document,
            headers={"Content-Type": "application/xml"},
            timeout=10.0,
        )
        response.raise_for_status()
        parsed = parse_ncip_response(response.content)
        router_logger.info(
            "ncip transaction complete",
            extra={"operation": request.operation.value, "endpoint": request.endpoint},
        )
        return CircResult(
            ok=parsed.accepted,
            operation=request.operation,
            detail=parsed.problem_detail or "ok",
            protocol=self.name,
        )

Step 4 — Route with the selector

The selector holds the routing policy: an endpoint-to-protocol map, plus a capability check so an operation is never sent to a backend that cannot perform it. This is the one place the SIP2-versus-NCIP decision lives.

class ProtocolSelector:
    """Routes a CircRequest to the backend configured for its endpoint."""

    def __init__(
        self,
        backends: dict[str, CirculationBackend],
        endpoint_protocol: dict[str, str],
        default_protocol: str,
    ) -> None:
        for backend in backends.values():
            if not isinstance(backend, CirculationBackend):
                raise TypeError(f"{backend!r} does not satisfy CirculationBackend")
        self._backends = backends
        self._endpoint_protocol = endpoint_protocol
        self._default_protocol = default_protocol

    def route(self, request: CircRequest) -> CircResult:
        protocol = self._endpoint_protocol.get(
            request.endpoint, self._default_protocol
        )
        backend = self._backends[protocol]
        if not backend.supports(request.operation):
            raise UnsupportedOperationError(
                f"{protocol} cannot perform {request.operation.value}"
            )
        router_logger.info(
            "routing circulation request",
            extra={
                "operation": request.operation.value,
                "endpoint": request.endpoint,
                "protocol": protocol,
            },
        )
        return backend.execute(request)

Pitfall: do not route on the operation alone. A checkout from a lobby kiosk and a checkout requested by an ILL partner are the same operation but belong to different lanes — the endpoint, not the verb, decides the protocol. Keying the map on the endpoint keeps that distinction explicit.

Running both protocols at once

Most production libraries do not choose SIP2 or NCIP — they run both, because the two jobs coexist. The self-check fleet in the lobby speaks SIP2 to the ILS for fast checkout and fine payment, while the same ILS answers NCIP over HTTPS for the consortium’s resource-sharing traffic. The selector above already supports this: register both backends, map the kiosk endpoints to sip2 and the partner endpoints to ncip, and let the routing policy keep them apart.

selector = ProtocolSelector(
    backends={"sip2": sip2_backend, "ncip": ncip_backend},
    endpoint_protocol={
        "lobby-selfcheck-01": "sip2",
        "lobby-selfcheck-02": "sip2",
        "consortium-ill-responder": "ncip",
        "statewide-sharing-hub": "ncip",
    },
    default_protocol="ncip",
)

Keep the two lanes operationally independent: separate connection pools, separate timeouts, separate health checks, and separate on-call alerts. A SIP2 socket storm from a wedged kiosk must not exhaust the HTTP client pool serving ILL partners, and an NCIP responder outage at a partner must not stall the lobby. When a library eventually consolidates a self-check fleet onto NCIP, that migration is a routing-map change plus hardware capability work rather than a rewrite — the mechanics are covered in migrating a self-check fleet from SIP2 to NCIP.

PII & Compliance Checkpoints

The protocol choice is also a privacy-surface choice, and it fails open in ways the routing map should account for. Two checkpoints keep it defensible.

Transport confidentiality. SIP2’s default plain-TCP framing carries the patron barcode and PIN in a form a network capture can read. Wherever a SIP2 lane is registered, the endpoint must be reachable only over a TLS tunnel or an isolated VLAN, never the open network. NCIP inherits HTTPS and standard web auth, so its transport story is easier — but a misconfigured client that disables certificate verification throws that advantage away. Both lanes must terminate on verified TLS before any patron identifier is emitted, and the masking rules in PII masking in patron data exports apply to anything the router logs or forwards downstream.

Minimal identifier propagation. A LookupUser response — richer under NCIP than SIP2 — can carry name, address, and block status. The router should normalize both backends’ replies down to the fields the caller is actually entitled to before returning, following the data-minimization boundary in data privacy boundaries in library systems. Never let a verbose NCIP UserOptionalFields block ride through to a log line or a downstream consumer unfiltered.

Error Handling & Quarantine Patterns

The two protocols report failure in incompatible ways, and the adapter layer is where those differences must be reconciled into one exception vocabulary so the selector and its callers never branch on protocol-specific error shapes. SIP2 signals trouble with a mismatched sequence number or a bad AZ checksum on an otherwise well-formed line; NCIP returns a Problem element inside an HTTP 200 body, or a transport-level HTTP error. A record that cannot be safely completed on either lane is diverted rather than retried blindly.

class UnsupportedOperationError(Exception):
    """The routed backend cannot perform the requested operation."""


class ProtocolTransactionError(Exception):
    """A backend reported a recoverable, protocol-specific failure."""


def route_with_quarantine(
    selector: ProtocolSelector, request: CircRequest, quarantine_sink
) -> Optional[CircResult]:
    try:
        return selector.route(request)
    except UnsupportedOperationError:
        # A routing/config error, not a data error: fail loudly, do not quarantine.
        raise
    except ProtocolTransactionError as exc:
        # A single transaction failed on its lane; divert for inspection and
        # keep the gateway serving other endpoints.
        quarantine_sink.put({
            "endpoint": request.endpoint,
            "operation": request.operation.value,
            "reason": str(exc),
            "patron_ref": request.patron_barcode[-4:],
        })
        router_logger.warning(
            "circulation transaction quarantined",
            extra={"endpoint": request.endpoint, "reason": str(exc)},
        )
        return None

An UnsupportedOperationError is deliberately not quarantined: it means the routing map sent an operation to a backend that cannot serve it — a configuration defect that should stop and be fixed, not a single bad transaction to divert. By contrast a ProtocolTransactionError — a SIP2 checksum mismatch, an NCIP Problem for a blocked patron — is a per-transaction failure that is routed to an isolated queue while the gateway keeps flowing, the same isolation discipline libraries use for a schema validation quarantine queue. The quarantine record stores only a truncated patron reference, never the full barcode, so the queue does not become an unmasked shadow copy of patron identifiers. SIP2’s own checksum-and-sequence failures have their own recovery playbook in handling SIP2 checksum and sequence number errors.

Performance Considerations

The lanes have opposite performance profiles, and conflating them is how a gateway ends up slow on the operation that should be fast. SIP2’s persistent socket amortizes connection setup across a whole patron session, so its per-message cost is essentially the round-trip plus a few microseconds of ASCII parsing — which is exactly why it is right for a kiosk where a patron is waiting at the machine. The failure mode is socket exhaustion: a wedged device that never closes its session ties up an ILS connection slot, so SIP2 lanes need aggressive idle timeouts and a bounded connection pool per device class.

NCIP pays a per-call HTTP handshake and XML serialize/parse cost that is negligible against network latency for a cross-institution call but ruinous if you put it on a lobby self-check hot path. Reuse a pooled httpx.Client so TLS sessions are kept alive across calls, cap the per-request timeout so a slow partner cannot stall the pool, and treat the XML build/parse as CPU work worth caching schema objects for. For high-volume resource-sharing bursts, offload NCIP calls to the broker rather than a single synchronous worker — the durable, at-least-once consumer pattern developed for async batch processing for catalog updates applies unchanged, with the NCIP transaction as the consumer’s side effect. And because both lanes ultimately hit an ILS that rate-limits, respect the throttle discipline in ILS REST API polling & rate limiting.

Verification & Testing

Two properties must be proven by test, not assumed: that routing sends each operation to the correct lane, and that both adapters normalize their native replies into the same CircResult shape so callers never see a protocol-specific field leak through.

import pytest


class _FakeBackend:
    def __init__(self, name: str, supported: set[Operation]) -> None:
        self.name = name
        self._supported = supported

    def supports(self, operation: Operation) -> bool:
        return operation in self._supported

    def execute(self, request: CircRequest) -> CircResult:
        return CircResult(
            ok=True, operation=request.operation, detail="ok", protocol=self.name
        )


@pytest.fixture
def selector() -> ProtocolSelector:
    return ProtocolSelector(
        backends={
            "sip2": _FakeBackend("sip2", {Operation.CHECKOUT}),
            "ncip": _FakeBackend("ncip", {Operation.CHECKOUT, Operation.LOOKUP_USER}),
        },
        endpoint_protocol={"kiosk-1": "sip2", "ill-1": "ncip"},
        default_protocol="ncip",
    )


def test_kiosk_endpoint_routes_to_sip2(selector):
    result = selector.route(
        CircRequest(Operation.CHECKOUT, "P1", "kiosk-1", item_barcode="I1")
    )
    assert result.protocol == "sip2"


def test_partner_endpoint_routes_to_ncip(selector):
    result = selector.route(
        CircRequest(Operation.LOOKUP_USER, "P1", "ill-1")
    )
    assert result.protocol == "ncip"


def test_unsupported_operation_raises(selector):
    with pytest.raises(UnsupportedOperationError):
        selector.route(CircRequest(Operation.LOOKUP_USER, "P1", "kiosk-1"))


def test_unknown_endpoint_uses_default(selector):
    result = selector.route(CircRequest(Operation.CHECKOUT, "P1", "new-device"))
    assert result.protocol == "ncip"

Run the routing tests against your actual endpoint map, not just fixtures — a mistyped endpoint key silently falls through to the default protocol, and the only way to catch that before a patron does is to assert the concrete map routes every known device to the lane you intend.

Troubleshooting

A self-check checkout works, but the same barcode fails when an ILL partner requests it. Why?

The two requests are taking different lanes, and the failure is on the NCIP side. A SIP2 checkout at a kiosk and an NCIP CheckOutItem from a partner exercise entirely different code, auth, and agency mappings even though they are the same logical operation. Check the NCIP Problem element in the response body — a blocked patron, an unknown ToAgencyId, or an item not owned by the responding agency all surface there, not in the transport layer.

Our packet capture shows patron PINs in cleartext. Is that a bug?

It is expected for a plain SIP2 lane and it is a serious exposure. SIP2 frames barcode and PIN as positional ASCII with no built-in encryption. Wrap the socket in a TLS tunnel (stunnel or the vendor’s TLS extension) or isolate the lane on a dedicated VLAN, and confirm the endpoint’s entry in the routing map points only at the secured listener. NCIP over HTTPS does not have this problem unless certificate verification was disabled in the client.

NCIP calls intermittently time out under load while SIP2 stays healthy. What is happening?

The HTTP client pool is exhausted. NCIP pays a per-call handshake and XML cost, and if you share one small httpx.Client pool across a burst of resource-sharing traffic, later calls queue behind stalled ones. Give the NCIP lane its own pool with a bounded size and a hard per-request timeout, and move burst traffic onto the broker so the synchronous path stays short. The SIP2 lane is unaffected because its persistent sockets do not compete for that pool.

The selector sent a LookupUser to SIP2 and it crashed. How do I prevent this?

LookupUser has no first-class equivalent in the SIP2 flow used for self-check, so the SIP2 backend’s supports() returns False and the selector raises UnsupportedOperationError. That is the guard working. The fix is in the routing map, not the adapter: route patron-lookup operations to an NCIP endpoint, or to the SIP2 Patron Information message only if your kiosk flow actually needs it. Never widen _SUPPORTED to silence the error.

We are consolidating our kiosks onto NCIP. Do we flip the whole fleet at once?

No. Migrate endpoint by endpoint: change one kiosk’s entry in the routing map from sip2 to ncip, verify it against the NCIP responder, then proceed. Because the selector routes per endpoint, the fleet can run mixed indefinitely while you convert, and a bad conversion rolls back by editing one map entry. The hardware and capability details are in migrating a self-check fleet from SIP2 to NCIP.

Related

• SIP2 Protocol Integration for Self-Service Circulation — the socket-based self-check lane in depth, including session and field handling
• NCIP Resource-Sharing Workflows — the XML-over-HTTP lane for cross-institution lending and holds
• Migrating a Self-Check Fleet from SIP2 to NCIP — the endpoint-by-endpoint path when a library consolidates onto one protocol
• PII Masking in Patron Data Exports — the field-masking discipline both lanes must extend down to the transport
• ILS Schema Translation Patterns — normalizing patron and item fields before either protocol dispatches