SIP2 Protocol Integration for Self-Service Circulation

Operating within the broader Circulation Protocols & Interoperability architecture, this guide covers the wire protocol that every self-check machine, fine-payment station, and print-release kiosk in a library speaks to the ILS: 3M’s Standard Interchange Protocol, version 2, better known as SIP2. Library-tech staff hit this the moment a self-service device needs to do something transactional — check an item out, take it back, tell a patron why their card is blocked, debit a print account — because none of those actions are catalog reads; they mutate circulation state inside the ILS and must be spoken in SIP2’s fixed-and-variable-field message format over a raw TCP socket. This page walks through the SIP2 message contract, the field-code table that drives every parse, a production-grade Python client that frames and parses a Checkout exchange, the checkpoint that keeps the patron barcode out of your logs, and the checksum, quarantine, performance, and verification patterns that make the integration reliable on a busy circulation desk.

Specification & Contract

SIP2 is a synchronous, line-oriented, ASCII protocol. The terminal opens a TCP connection to the ILS’s SIP2 port (commonly 6001), and every exchange is a request message followed by exactly one response message, each terminated by a single carriage return (\r, 0x0D). There is no framing header and no length prefix — the carriage return is the frame delimiter — so a correct reader accumulates bytes until it sees a \r and treats everything before it as one message.

Each message begins with a two-digit message identifier that names the command, followed immediately by a run of fixed-length fields whose positions are defined by that identifier, then zero or more variable-length fields. A variable field is a two-letter code followed by its value and terminated by the field delimiter, which the login handshake declares (conventionally the pipe, |). The message ends with two optional-but-standard trailer fields: AY, a single-digit sequence number, and AZ, a four-hex-digit checksum computed over every byte up to and including AZ’s own code.

The identifiers you will implement first are the transactional pairs. The request/response numbering is always request n, response n+1:

Code	Message	Direction	Purpose
`93`	Login	terminal → ILS	Authenticate the terminal and declare the delimiter
`94`	Login Response	ILS → terminal	`1` = accepted, `0` = rejected
`11`	Checkout	terminal → ILS	Loan an item to a patron
`12`	Checkout Response	ILS → terminal	Loan granted/denied + due date
`09`	Checkin	terminal → ILS	Return an item
`10`	Checkin Response	ILS → terminal	Return accepted + routing instructions
`63`	Patron Information	terminal → ILS	Request status, fines, holds for a card
`64`	Patron Information Response	ILS → terminal	Block flags, balances, summary counts

The Checkout (11) request is the canonical example of the fixed-then-variable layout. Its fixed portion is positional; miscount a single character and every field after it shifts:

Offset / code	Field	Format	Notes
pos 2	SC renewal policy	1 char	`Y`/`N` — is a renewal allowed
pos 3	No-block	1 char	`Y` if the item was checked out offline
pos 4–21	Transaction date	18 chars	`YYYYMMDDZZZHHMMSS`
pos 22–39	NB due date	18 chars	blank unless no-block
`AO`	Institution ID	variable	scoping the request to a library
`AA`	Patron identifier	variable	the patron barcode — PII
`AB`	Item identifier	variable	the item barcode being loaned
`AC`	Terminal password	variable	shared secret for the SC unit
`AD`	Patron password	variable	optional PIN
`AY`	Sequence number	1 digit	echoed by the response for pairing
`AZ`	Checksum	4 hex	error-detection trailer

Two rules govern this table. First, the fixed fields are order- and width-sensitive — they have no delimiter between them, so the parser must slice by known offsets, not split on a separator. Second, AY and AZ are the protocol’s only integrity guarantees. SIP2 runs over plain TCP with no message signing, so the one-digit sequence number (which must match between request and response) and the checksum (which must recompute to the transmitted value) are what let you detect a dropped, duplicated, or corrupted frame. The mechanics of recovering when they don’t match are detailed in handling SIP2 checksum and sequence-number errors, and the closely related problem of a terminal replaying the same AY after a timeout is covered in deduplicating replayed SIP2 checkout messages.

Prerequisites & Environment Setup

The examples target Python 3.11+ and use only the standard library — socket, hashlib is not needed (the SIP2 checksum is a simple sum, not a cryptographic digest), and logging for structured audit output. No third-party SIP2 library is assumed; the wire format is small enough that owning the parser is safer than depending on an unmaintained package whose framing bugs you cannot see. The terminal password (AC) and the SIP2 host/port are resolved at runtime from configuration or a secrets manager, never committed, because the terminal password is a shared circulation credential.

Python >= 3.11 in the gateway worker image (python --version)
Network reachability to the ILS SIP2 port (default 6001); confirm with nc -vz ils-host 6001
SIP2_TERMINAL_PASSWORD (AC) resolved from a secrets manager at runtime, never committed
SIP2_INSTITUTION_ID (AO) for the branch the terminal serves
The field delimiter agreed with the ILS (default |) and the line terminator confirmed as \r
A socket read/write timeout set (self-check units abandon a request after a few seconds)
Structured (JSON) log handler configured for the sip2_audit logger, shipped to append-only storage
The patron-barcode masking rule reviewed against the field-classification discipline in PII Masking in Patron Data Exports before any message is logged

Core Implementation

The client runs each exchange through four labeled stages: model the message as a typed dataclass, serialize it with a computed checksum, send-and-receive over a carriage-return-delimited socket reader, then parse and validate the response. Keeping serialization and parsing symmetric is what makes the checksum trustworthy — the same byte range that the sender sums is the range the receiver re-sums.

Step 1 — Model the message as a typed dataclass

A SIP2 message is a message identifier, an ordered list of fixed fields, and a map of variable fields. Modelling it as a dataclass keeps the wire concerns (offsets, delimiters, checksums) out of the call sites that just want to check an item out.

from __future__ import annotations

import logging
import socket
from dataclasses import dataclass, field

logger = logging.getLogger("sip2_audit")

FIELD_DELIMITER = "|"
LINE_TERMINATOR = "\r"


@dataclass(slots=True)
class SIP2Message:
    """A parsed SIP2 message: identifier + fixed prefix + variable field map."""

    identifier: str            # e.g. "11" for Checkout
    fixed: str                 # positional fixed-length prefix, already formatted
    variables: dict[str, str] = field(default_factory=dict)
    sequence: int | None = None  # the AY value, 0-9

    def body(self) -> str:
        """Everything up to but excluding the AZ checksum field."""
        parts = [self.identifier, self.fixed]
        for code, value in self.variables.items():
            parts.append(f"{code}{value}{FIELD_DELIMITER}")
        if self.sequence is not None:
            parts.append(f"AY{self.sequence}")
        parts.append("AZ")
        return "".join(parts)

Pitfall: preserve insertion order in variables. Some ILS SIP2 servers are strict about AO preceding AA preceding AB; a plain dict on Python 3.7+ keeps insertion order, but do not sort the keys as a “cleanup” — sorting reorders the wire message and some servers will reject it.

Step 2 — Compute the checksum and serialize

The AZ checksum is the twos-complement of the unsigned 16-bit sum of every byte in the message up to and including the literal AZ code, rendered as four uppercase hex digits. Compute it over the exact bytes you will transmit, then append it and the terminator.

def checksum(body: str) -> str:
    """SIP2 AZ: negated 16-bit sum of body bytes, 4 uppercase hex digits."""
    total = sum(body.encode("ascii"))
    return f"{(-total) & 0xFFFF:04X}"


def serialize(message: SIP2Message) -> bytes:
    """Render a message to the exact bytes on the wire, checksum included."""
    body = message.body()               # ends with the literal "AZ"
    wire = f"{body}{checksum(body)}{LINE_TERMINATOR}"
    return wire.encode("ascii")

Pitfall: the sum runs over the literal characters A and Z, not over the checksum value. A parser that sums up to the delimiter before AZ, or that includes the four hex digits, will compute a value the ILS never agrees with and reject every message as corrupt.

Step 3 — Send and receive over a carriage-return-delimited socket

SIP2 has no length prefix, so the reader must accumulate until it sees \r. Wrapping the socket in a context manager guarantees the connection closes even when a read times out mid-message, and a bounded buffer stops a misbehaving server from streaming forever.

class SIP2Timeout(Exception):
    """The ILS did not return a complete, CR-terminated response in time."""


class SIP2Client:
    """A synchronous SIP2 client over a single TCP connection."""

    def __init__(self, host: str, port: int, timeout: float = 5.0) -> None:
        self._host = host
        self._port = port
        self._timeout = timeout

    def exchange(self, request: SIP2Message) -> str:
        """Send one request, block for its CR-terminated response line."""
        payload = serialize(request)
        with socket.create_connection(
            (self._host, self._port), timeout=self._timeout
        ) as sock:
            sock.sendall(payload)
            buffer = bytearray()
            while LINE_TERMINATOR.encode("ascii")[0] not in buffer:
                chunk = sock.recv(4096)
                if not chunk:
                    raise SIP2Timeout("connection closed before terminator")
                buffer.extend(chunk)
                if len(buffer) > 64_000:
                    raise SIP2Timeout("response exceeded frame ceiling")
            line, _, _ = buffer.partition(b"\r")
        logger.info(
            "sip2_exchange",
            extra={"identifier": request.identifier, "seq": request.sequence},
        )
        return line.decode("ascii")

Pitfall: a single recv(4096) is not one message. TCP is a stream, not a datagram service, so a response can arrive across several recv calls or with the next response’s first bytes already buffered. The accumulate-until-\r loop is mandatory; treating one recv as one frame produces intermittent, load-dependent parse failures that are miserable to reproduce.

Step 4 — Parse and validate the Checkout Response (12)

The response parser reverses serialization: verify the checksum, slice the fixed prefix by known offsets, then split the remainder on the delimiter into variable fields. Only after integrity passes do the field values mean anything.

@dataclass(slots=True)
class CheckoutResponse:
    ok: bool                    # fixed pos 2: "1" granted, "0" denied
    renewal_ok: bool
    due_date: str | None
    item_id: str | None
    sequence: int | None


def parse_checkout_response(line: str) -> CheckoutResponse:
    """Parse a 12 Checkout Response after verifying its AZ checksum."""
    az_index = line.rfind("AZ")
    if az_index == -1:
        raise SIP2ParseError("no AZ checksum field")
    body, transmitted = line[: az_index + 2], line[az_index + 2 :]
    if checksum(body) != transmitted:
        raise SIP2ChecksumError(expected=checksum(body), got=transmitted)

    if line[:2] != "12":
        raise SIP2ParseError(f"expected 12, got {line[:2]}")

    ok = line[2] == "1"
    renewal_ok = line[3] == "1"
    variables = _split_variables(line[26:az_index])  # after fixed prefix
    seq = variables.get("AY")
    return CheckoutResponse(
        ok=ok,
        renewal_ok=renewal_ok,
        due_date=variables.get("AH"),
        item_id=variables.get("AB"),
        sequence=int(seq) if seq and seq.isdigit() else None,
    )

Pitfall: verify the checksum before trusting any parsed field, and slice the fixed prefix by its documented width for message 12 (a status flags block plus an 18-character transaction date). Splitting the whole line on | and hoping the fixed prefix “looks fine” is how a one-character offset silently mislabels a granted loan as denied.

PII & Compliance Checkpoints

The patron identifier in the AA field is a direct identifier — it is the physical barcode on a library card, and in many systems it encodes or maps directly to a named person. A SIP2 gateway sees an AA value on every checkout, checkin, and patron-information message, which makes its log stream the single highest-volume source of patron PII in the circulation stack. Two checkpoints keep it defensible.

Mask the barcode at the log boundary. No log line, structured field, or trace span may carry the raw AA value. Before any message is logged, the barcode is rewritten to a masked surrogate — a truncated, non-reversible token that still lets an operator correlate the two sides of one exchange — using the same deterministic tokenization discipline described in PII Masking in Patron Data Exports. The extra={...} payloads in the code above deliberately carry the message identifier and sequence number, never the AA field.

def mask_barcode(barcode: str) -> str:
    """Non-reversible display surrogate for an AA patron barcode in logs."""
    if len(barcode) <= 4:
        return "****"
    return f"{barcode[:2]}***{barcode[-2:]}"

Minimize what the gateway retains. A SIP2 gateway is a relay, not a store. It should hold a message only long enough to forward it and pair its response; it must not persist the raw frame, because a persisted frame is an unmasked copy of the barcode. Where circulation events do flow onward into analytics, the one-way anonymization boundary in Circulation History Routing & Anonymization should run downstream of the gateway so the barcode is already a surrogate before it reaches any long-lived store.

Error Handling & Quarantine Patterns

The client distinguishes three failure classes and never lets a corrupt or mismatched frame be treated as a successful loan. A checksum failure means the bytes were mangled in transit; a sequence mismatch means the response does not belong to this request; a timeout means the ILS never answered.

class SIP2ParseError(Exception):
    """The response was structurally unparseable."""


class SIP2ChecksumError(Exception):
    """The recomputed AZ did not match the transmitted checksum."""

    def __init__(self, expected: str, got: str) -> None:
        super().__init__(f"checksum mismatch expected={expected} got={got}")
        self.expected, self.got = expected, got


class SIP2SequenceError(Exception):
    """The response AY did not echo the request AY."""


def exchange_checked(client: SIP2Client, request: SIP2Message,
                     quarantine_sink) -> CheckoutResponse | None:
    """Run one Checkout exchange with integrity checks and quarantine routing."""
    try:
        line = client.exchange(request)
        response = parse_checkout_response(line)
    except (SIP2ChecksumError, SIP2ParseError) as exc:
        quarantine_sink.put({
            "reason": type(exc).__name__,
            "identifier": request.identifier,
            "seq": request.sequence,
        })
        logger.warning("sip2_quarantine", extra={"reason": type(exc).__name__})
        return None

    if response.sequence != request.sequence:
        raise SIP2SequenceError(
            f"AY mismatch request={request.sequence} response={response.sequence}"
        )
    return response

A SIP2ChecksumError or SIP2ParseError diverts the single frame to an isolated quarantine queue for inspection and lets the terminal retry, exactly as an ingest pipeline routes a malformed record to a schema validation quarantine queue rather than failing the whole batch. A SIP2SequenceError is handled differently: it is not quarantined and retried blindly, because a mismatched AY may mean the ILS actually committed the loan and the terminal simply lost the acknowledgement — resending would double-charge the item. The safe recovery is to reconcile against patron status before retrying, the deduplication problem worked through in deduplicating replayed SIP2 checkout messages. Note that the quarantine record carries only the identifier and sequence number, never the raw frame, so the quarantine queue does not become an unmasked shadow copy of patron barcodes.

Performance Considerations

A single SIP2 exchange is cheap — a few hundred bytes each way and a trivial checksum — so throughput is never bound by parsing. It is bound by connection setup. Opening a fresh TCP connection per message, as the exchange method above does for clarity, adds a full handshake round-trip to every checkout; on a busy self-check fleet during a school rush that latency is what patrons feel as a sluggish terminal.

Pool the connection instead. Keep one long-lived socket per terminal session after the 93 Login handshake, reuse it for the sequence of 11/09/63 messages the patron generates, and only tear it down on idle timeout or error. Because SIP2 is strictly one-response-per-request, a pooled connection needs no multiplexing — just a lock so two threads never interleave frames on the same socket. For a large gateway fronting many ILS backends, the durable, at-least-once relay pattern developed for Async Batch Processing for Catalog Updates applies to the forwarding hop, and the throttle discipline in ILS REST API Polling & Rate Limiting keeps the gateway from overrunning an ILS whose SIP2 port has a hard concurrency cap.

Verification & Testing

Two properties must be proven by test, not assumed: round-trip integrity (a message you serialize parses back with a checksum the receiver accepts) and rejection (a corrupted frame is caught, not silently mis-parsed).

import pytest


def test_checksum_round_trips():
    msg = SIP2Message(identifier="11", fixed="NN20260716    120000",
                      variables={"AO": "MAIN", "AA": "21234", "AB": "31111"},
                      sequence=3)
    wire = serialize(msg).decode("ascii")
    body, transmitted = wire[: wire.rfind("AZ") + 2], wire[wire.rfind("AZ") + 2 : -1]
    assert checksum(body) == transmitted


def test_corrupted_frame_is_rejected():
    msg = SIP2Message(identifier="12", fixed="1N00120260716    120000",
                      variables={"AB": "31111", "AH": "20260730"}, sequence=3)
    wire = serialize(msg).decode("ascii")
    flipped = wire[:5] + ("X" if wire[5] != "X" else "Y") + wire[6:]
    with pytest.raises(SIP2ChecksumError):
        parse_checkout_response(flipped.rstrip("\r"))


def test_no_barcode_leaks_into_logs(caplog):
    with caplog.at_level(logging.INFO, logger="sip2_audit"):
        SIP2Client("localhost", 6001).__class__  # construction only
    assert "21234" not in caplog.text


def test_barcode_masking_is_non_reversible():
    assert mask_barcode("21234567890123") == "21***23"
    assert mask_barcode("abc") == "****"

Run the barcode-leak assertion as a post-deploy gate over a sample of the actual gateway log stream, not just unit fixtures — a new message type added later may log a field you never classified, and the live-log scan catches a raw AA value the moment it appears. Pair this with a conformance run against your ILS vendor’s SIP2 test harness, since real servers vary in which optional fields they require.

Troubleshooting

The ILS rejects every message with a checksum error, but my code looks right. Why?

The checksum is almost always being summed over the wrong byte range. It must cover every byte from the message identifier through the literal AZ code, and it must exclude the four hex digits themselves. The two most common bugs are summing up to the delimiter before AZ (missing the A and Z characters) and encoding the string as UTF-8 when a non-ASCII character is present. Confirm by hand: sum the ASCII values of the exact body string ending in AZ, negate, mask to 16 bits, format as four uppercase hex digits.

Responses occasionally parse as garbage under load but are fine when I test one at a time. What is happening?

You are treating a single recv as a whole message. TCP delivers a byte stream, so under load a response can span multiple recv calls, or a fast ILS can pack the tail of one response and the head of the next into one buffer. Use the accumulate-until-\r loop and keep any leftover bytes after the terminator for the next read. This class of bug is invisible at low volume and reliable at high volume.

A self-check unit charged a patron for an item twice. How did that happen?

The terminal sent a 11 Checkout, the ILS committed the loan, but the acknowledgement was lost to a timeout, so the terminal resent the same request with the same AY. Without deduplication the ILS booked a second loan. A mismatched or repeated sequence number must trigger reconciliation against patron status before any resend, never a blind retry — the full pattern is in deduplicating replayed SIP2 checkout messages.

The patron barcode is showing up in our centralized logs. How do I stop it?

A message is being logged before the AA field is masked, or a raw frame is being persisted for debugging. Route every log call through the masking surrogate shown under PII & Compliance Checkpoints, strip raw-frame logging from production, and add the live-log scan from Verification as a gate. The masking rule follows the same field-classification discipline as PII Masking in Patron Data Exports.

SIP2 is the right tool for self-service circulation at a single ILS — checkout, checkin, fine payment, print release. For cross-institution resource sharing and richer patron/agency semantics, NCIP is usually the better fit; the trade-offs are laid out in SIP2 vs NCIP: choosing a circulation protocol and the resource-sharing message flows in NCIP resource-sharing workflows.

Deduplicating Replayed SIP2 Checkout Messages — how to keep a lost acknowledgement and a resent AY from double-booking a loan
Handling SIP2 Checksum and Sequence-Number Errors — the AY/AZ recovery paths in depth, including quarantine and resend
SIP2 vs NCIP: Choosing a Circulation Protocol — when the fixed-field simplicity of SIP2 wins and when NCIP’s richer semantics are worth the cost
NCIP Resource-Sharing Workflows — the cross-institution message flows SIP2 was never designed to carry
PII Masking in Patron Data Exports — the deterministic masking discipline applied to the AA patron barcode at the log boundary