Migrating a Self-Check Fleet from SIP2 to NCIP
This page answers one narrow operational question: how do you move a fleet of self-check machines and their integrations from SIP2 to NCIP without downtime, and without double-applying circulation events during the window when both protocols are live against the same ILS? It sits under SIP2 vs NCIP: Choosing the Right Circulation Protocol for Your ILS, which weighs the two protocols against each other, and within the broader Circulation Protocols & Interoperability architecture. If you run twenty or two hundred self-check units against a single circulation backend, a naive cutover is the fastest way to hand a patron two loan records for one book — or take the whole fleet dark at open.
Problem Framing
The symptom is a phantom double loan. A patron scans one item at a self-check kiosk during the migration window and the ILS records two open loans for the same barcode — or a return posts twice and drives the patron’s account to a negative fine balance. Nothing crashes: both the SIP2 and NCIP paths report success to their respective clients, the kiosk prints a receipt, and the item leaves the building once. Only the circulation ledger disagrees with reality.
The doubling is visible if you group the loan table by item and patron over the cutover window and look for events that arrived on two different protocols within seconds of each other:
$ psql -c "SELECT item_barcode, patron_id, source_protocol, created_at
FROM loan_event
WHERE created_at > '2026-07-15 08:00'
ORDER BY item_barcode, created_at" | head
item_barcode | patron_id | source_protocol | created_at
--------------+-----------+-----------------+---------------------------
31234000551 | P0099821 | sip2 | 2026-07-15 09:02:11.310
31234000551 | P0099821 | ncip | 2026-07-15 09:02:11.844 <-- same loan, second protocol
31234000772 | P0100544 | ncip | 2026-07-15 09:14:03.007
31234000772 | P0100544 | sip2 | 2026-07-15 09:14:03.512 <-- duplicate 0.5s later
Two rows, half a second apart, same item and patron, different source_protocol. Either a single kiosk was talking both protocols during a config flip, or a middleware retry replayed one patron action down both adapters. The moment the second row commits, the ledger is wrong, and every downstream feed — holds, fines, consortium reporting — inherits the error.
Root Cause
SIP2 and NCIP model circulation differently but mutate the same ILS state. SIP2 is a terse field-delimited request/response protocol: a 11 Checkout message carries an item barcode and a patron identifier, and the ILS answers 12 with an ok flag. NCIP is an XML service with a richer object model — CheckOutItem wraps typed ItemId and UserId elements and returns a structured DateDue. Both, at the end of the call, create one loan row. Neither protocol knows the other exists.
Running both against one ILS therefore has no shared notion of “this circulation event already happened.” When a device, a load balancer, or an operator flips a kiosk mid-transaction — or a middleware layer fans one patron tap out to both adapters during a parallel-run test — the same physical action becomes two independent write paths, and the ILS obediently applies both. A big-bang cutover avoids the double-write but trades it for the opposite failure: flip every kiosk at once, hit one unhandled NCIP edge case, and the entire fleet is down at opening with no graceful path back to the protocol that was working an hour ago.
The fix is to make protocol selection an explicit, per-device decision and to give both protocol paths a shared idempotency key, so that whichever adapter handles a given circulation event, a replay of that same event down either path is recognized and deduplicated before it reaches the ILS. This is the same discipline the parent guide recommends for SIP2 protocol integration on its own, extended so the dedupe boundary spans both protocols at once.
Solution
Put an adapter in front of the ILS that routes each device by a per-device protocol flag read from a device registry, and thread a deterministic idempotency key through both the SIP2 and NCIP paths. Canary a handful of machines onto NCIP first; because the key is derived from the circulation event itself and not from the protocol, an event that somehow arrives on both paths collides in the idempotency store and commits exactly once. Rollback is a flag flip, not a redeploy.
from __future__ import annotations
import enum
import hashlib
import logging
from dataclasses import dataclass
from typing import Protocol
logger = logging.getLogger("circ.migration")
class ProtocolChoice(enum.Enum):
SIP2 = "sip2"
NCIP = "ncip"
@dataclass(frozen=True)
class CircEvent:
"""A protocol-neutral circulation event normalized from either adapter."""
action: str # "checkout" | "checkin"
item_barcode: str
patron_id: str
device_id: str
occurred_at: str # device-supplied ISO-8601, stable across retries
def idempotency_key(event: CircEvent) -> str:
"""Deterministic key shared by BOTH protocol paths.
Derived only from the circulation event, never from the protocol or
transport, so the same patron action hashes identically whether it
arrives via SIP2 or NCIP. A replay across protocols therefore collides.
"""
material = "|".join(
(event.action, event.item_barcode, event.patron_id, event.occurred_at)
)
return hashlib.sha256(material.encode()).hexdigest()
class Adapter(Protocol):
def commit(self, event: CircEvent) -> None: ...
class IdempotencyStore(Protocol):
def add_if_absent(self, key: str) -> bool:
"""Return True if the key was newly inserted, False if already seen."""
...
def discard(self, key: str) -> None:
"""Release a key so a genuine retry can proceed after a failed commit."""
...
class DeviceRegistry(Protocol):
def protocol_for(self, device_id: str) -> ProtocolChoice: ...
class CircRouter:
"""Route each device to SIP2 or NCIP behind a per-device flag, with a
shared idempotency gate so parallel operation cannot double-apply a loan."""
def __init__(
self,
registry: DeviceRegistry,
adapters: dict[ProtocolChoice, Adapter],
store: IdempotencyStore,
) -> None:
self._registry = registry
self._adapters = adapters
self._store = store
def handle(self, event: CircEvent) -> None:
key = idempotency_key(event)
if not self._store.add_if_absent(key):
logger.info(
"circ_event_deduped",
extra={
"idem_key": key,
"device_id": event.device_id,
"action": "dedupe",
},
)
return # already applied on this or the other protocol path
choice = self._registry.protocol_for(event.device_id)
adapter = self._adapters[choice]
try:
adapter.commit(event)
except Exception:
# Commit failed: release the key so a genuine retry can proceed.
self._store.discard(key)
logger.exception(
"circ_commit_failed",
extra={
"idem_key": key,
"device_id": event.device_id,
"protocol": choice.value,
"action": "release",
},
)
raise
logger.info(
"circ_event_committed",
extra={
"idem_key": key,
"device_id": event.device_id,
"protocol": choice.value,
"action": "commit",
},
)
The behavioural change is that protocol is now a property of the device, resolved at routing time, and dedupe happens before protocol selection. Before, a kiosk mid-flip or a fanned-out retry produced two independent writes because nothing tied them together. After, both derive the same idempotency_key from the item, patron, action and device-supplied timestamp, so the second arrival — on either protocol — loses the add_if_absent race and is logged as circ_event_deduped instead of committed. The discard on failure keeps the gate from swallowing a legitimate retry after a transient ILS error. To migrate a machine you flip one row in the device registry from sip2 to ncip; to roll it back you flip it the other way, with no code change and no fleet-wide restart.
Progressive rollout then becomes: canary two or three low-traffic kiosks onto NCIP, watch the circ_event_deduped and circ_commit_failed counters for a full open-to-close day, widen to a branch, then the fleet — reverting any device to SIP2 the instant its NCIP error rate rises.
Compliance or Privacy Impact
A protocol migration is a circulation-data change, so the audit obligations do not lift during the window — they get harder, because events now flow through two code paths. Audit both paths uniformly or the migration itself becomes an accountability gap.
- Log the same fields on both protocols. Every event, whether it committed via SIP2 or NCIP, must produce a structured record with the shared
idem_key,device_id,protocoland disposition (commit,dedupe,release). That uniform shape is what lets you prove a deduped event was not double-counted, and it is the only way to reconstruct what happened on a contested loan after the fleet is fully cut over. Route both paths’ events into the same immutable audit sink the routing boundary defines rather than two protocol-specific logs. - Patron identifiers cross both adapters. SIP2 patron barcodes and NCIP
UserIdvalues are the same personal identifiers, and they now transit two integrations. Any reporting or export drawn from the loan ledger during migration must run through the masking contract in PII Masking in Patron Data Exports so that a mid-migration extract cannot leak a raw patron id from either path. - The idempotency store is not a retention store. It holds event keys only long enough to dedupe within the replay window; it is not the system of record and should age out on a short TTL. Keep the durable circulation history under its normal retention schedule, and do not let the dedupe cache quietly become a second, unmanaged copy of loan activity.
The migration narrows nothing about the PII surface, but it does widen the number of paths that touch it, which is precisely why uniform auditing across SIP2 and NCIP is the compliance-critical part of the cutover rather than an afterthought.
Verification
Confirm the router is idempotent across protocols with a test that replays one event down both the SIP2 and NCIP adapters and asserts the ILS is committed exactly once.
class _FakeStore:
def __init__(self) -> None:
self._seen: set[str] = set()
def add_if_absent(self, key: str) -> bool:
if key in self._seen:
return False
self._seen.add(key)
return True
def discard(self, key: str) -> None:
self._seen.discard(key)
class _CountingAdapter:
def __init__(self) -> None:
self.commits = 0
def commit(self, event: CircEvent) -> None:
self.commits += 1
def test_cross_protocol_replay_commits_once() -> None:
sip2, ncip = _CountingAdapter(), _CountingAdapter()
store = _FakeStore()
# Same physical patron action, delivered on both paths during cutover.
event = CircEvent(
action="checkout",
item_barcode="31234000551",
patron_id="P0099821",
device_id="kiosk-07",
occurred_at="2026-07-15T09:02:11",
)
class _Reg:
def __init__(self, choice: ProtocolChoice) -> None:
self._choice = choice
def protocol_for(self, device_id: str) -> ProtocolChoice:
return self._choice
sip2_router = CircRouter(_Reg(ProtocolChoice.SIP2),
{ProtocolChoice.SIP2: sip2}, store)
ncip_router = CircRouter(_Reg(ProtocolChoice.NCIP),
{ProtocolChoice.NCIP: ncip}, store)
sip2_router.handle(event) # first arrival commits
ncip_router.handle(event) # replay across protocol is deduped
assert sip2.commits == 1
assert ncip.commits == 0 # exactly one loan reached the ILS
For a running cutover, add a continuous invariant on the loan ledger itself: over any rolling window, no (item_barcode, patron_id, action) triple should appear on two source_protocol values within the replay TTL. Alert if one does — that is the same phantom double loan from the Problem Framing, caught in flight instead of at month-end reconciliation. Track the circ_event_deduped rate per device; a canary whose dedupe count spikes is a machine still emitting on both protocols and should be rolled back to SIP2 until its client config is corrected.
Related
- SIP2 vs NCIP: Choosing the Right Circulation Protocol for Your ILS — the parent comparison that frames when a migration like this is worth undertaking.
- Circulation Protocols & Interoperability — the wider protocol-integration architecture this cutover sits inside.
- SIP2 Protocol Integration for Self-Service Circulation — the source protocol you are migrating off, and its idempotency patterns.
- NCIP Resource-Sharing Workflows — the target protocol’s message model and the workflows you are migrating onto.