JSON-RPC Transport Focus Deep-Dive

🎯 The Problem

You have a single CLI binary that orchestrates an LLM agent. You want to embed it into a Node.js app, a Python script, a Go service, or a .NET API — without forcing any of them to fork the CLI source. The transport must: (1) handle request/response correlation with IDs, (2) allow either side to initiate calls (tool invocations flow CLI→SDK), (3) stream notifications (events with no reply), (4) work identically on Linux, macOS, Windows, (5) require zero runtime configuration from the user. It must also be simple enough to implement in Go with 400 lines and in Python with 300.

🔧 The Approach

Use JSON-RPC 2.0 as the payload format (well-known, self-describing, language-neutral), Content-Length framing borrowed from LSP (unambiguous message boundaries, works on any byte stream), and stdio as the transport (subprocess spawn is a solved problem; no port binding; no firewall). Each side runs two halves: a reader that pulls one framed message at a time and dispatches by kind, and a writer that serialises outgoing messages.

Message kinds (JSON-RPC 2.0):

• Request — {id, method, params}; requires a response.
• Response — {id, result} or {id, error}; correlated by id.
• Notification — {method, params} (no id); fire-and-forget.

🔄 Why Not Alternatives?

Overview Diagram

flowchart LR
  subgraph sdk["SDK (any language)"]
    WR[writer goroutine/task]
    RD[reader goroutine/task]
    PEND[pending requests map
id -> promise]
    DISP[dispatcher
method -> handler]
  end
  subgraph cli["CLI subprocess"]
    CW[writer]
    CR[reader]
    CP[pending map]
    CD[dispatcher]
  end
  STDIN[stdin pipe]
  STDOUT[stdout pipe]
  WR -->|Content-Length frame| STDIN
  STDIN --> CR
  CR --> CD
  CR --> CP
  CW -->|Content-Length frame| STDOUT
  STDOUT --> RD
  RD --> DISP
  RD --> PEND

Step-by-Step Frame Handling

// go/internal/jsonrpc2/frame.go
func ReadMessage(r *bufio.Reader) ([]byte, error) {
    var n int64
    for {
        line, err := r.ReadString('\n')
        if err != nil { return nil, err }
        if line == "\r\n" { break }              // end of headers
        if strings.HasPrefix(line, "Content-Length:") {
            fmt.Sscanf(line, "Content-Length: %d", &n)
        }
    }
    body := make([]byte, n)
    _, err := io.ReadFull(r, body)
    return body, err
}

var raw struct {
    ID     json.RawMessage `json:"id"`
    Method string          `json:"method"`
    Result json.RawMessage `json:"result"`
    Error  *Error          `json:"error"`
}
json.Unmarshal(body, &raw)

switch {
case raw.Method != "" && len(raw.ID) > 0:  // request
    go handleRequest(raw)
case raw.Method != "":                     // notification
    go handleNotification(raw)
default:                                   // response
    completePending(raw.ID, raw.Result, raw.Error)
}

pending := map[string]chan Response{}
// When sending:
ch := make(chan Response, 1)
pending[id] = ch
writeFrame(req)
// When the reader sees a response with matching id:
pending[id] <- resp
delete(pending, id)

func WriteMessage(w io.Writer, body []byte) error {
    hdr := fmt.Sprintf("Content-Length: %d\r\n\r\n", len(body))
    if _, err := w.Write([]byte(hdr)); err != nil { return err }
    _, err := w.Write(body)
    return err
}

Bidirectional Flow

sequenceDiagram
    participant USR as User code
    participant SDK as SDK Client
    participant CLI as CLI Server
    USR->>SDK: session.send(prompt)
    SDK->>CLI: session.send (req, id=7)
    CLI-->>SDK: assistantMessage (notification)
    CLI->>SDK: tool.invoke shell (req, id=42)
    Note over SDK: dispatches to user tool handler
    SDK-->>CLI: tool result (resp, id=42)
    CLI-->>SDK: toolCallCompleted (notification)
    CLI-->>SDK: sessionIdle (notification)
    CLI-->>SDK: response to session.send (id=7)
    SDK-->>USR: messageId returned

The sequence above shows that a single user call may trigger many notifications and at least one server→client request before its own response arrives.

Error Handling

⚖️ Complexity & Tradeoffs

Time. Frame read: O(bytes). Dispatch: O(1) map lookup. Response correlation: O(1). Space. Pending-requests map bounded by in-flight count (typically <10). Throughput. Limited by JSON serialisation cost and pipe buffer size (usually 64KB). Latency. 1-2ms round-trip locally; LSP-style stdio is faster than localhost TCP because no TCP handshake or Nagle.

Tradeoff: JSON is larger than binary. For a chat SDK where the dominant payload is LLM-generated text, JSON overhead is a rounding error. For a high-throughput RPC system, MessagePack would be worth it.

stdio vs TCP

All four SDKs support both. TCP is used when CLIUrl is set — typically for debugging (attach the SDK to a separately launched CLI) or for containerised setups where the CLI runs in a sidecar. The framing is identical; only the byte transport changes. stdio is default because:

• No port allocation or clash with user's services.
• Automatic lifetime binding: when the SDK process exits, the CLI's stdin EOFs and the CLI exits too. No orphan processes.
• No auth model needed — the subprocess inherits the parent's credentials; third parties can't connect.

Directory Map

nodejs/
├── src/client.ts            ← uses vscode-jsonrpc (external library)
└── node_modules/vscode-jsonrpc/ ← Content-Length framing implementation

python/copilot/
└── _jsonrpc.py              ← hand-rolled framing + threaded reader

go/internal/jsonrpc2/
├── jsonrpc2.go              ← Client, handlers, pending map
└── frame.go                 ← ReadMessage / WriteMessage

dotnet/src/                  ← STJ-based framing inside Client.cs

Related Topics

• Protocol Negotiation — how the handshake is itself a transport-layer exchange.
• Codegen Pipeline — how RPC method wrappers are generated on top of this transport.