Channels/Messaging — Media Handling

Focus Deep-Dive

Cross-Platform Media Attachment Processing & Format Conversion

Media handling manages the ingestion, validation, and conversion of media attachments (images, audio, video, documents) across all supported channels. Each platform has different size limits, supported formats, and delivery mechanisms that this module normalizes into a uniform internal representation.

Multi-Format Size Validation Format Conversion Platform-Aware

📚 What is this & why does it matter?

Users send images, voice messages, documents, and videos through different platforms. Each platform has unique constraints — WhatsApp limits file sizes differently than Telegram, Discord supports different image formats than Slack. The media handling module ensures consistent processing regardless of source.

💡 Real-world analogy

Think of an international courier service. Packages (media) arrive in different formats, sizes, and packaging standards from different countries (platforms). The receiving hub (media handler) inspects each package, checks it against the destination’s regulations (size/type limits), repackages if needed (format conversion), and standardizes the shipping label (metadata normalization) before routing.

Channel Media Capabilities

Each channel declares its media support via the ChannelCapabilities type:

export type ChannelCapabilities = {
  chatTypes: Array<ChatType | "thread">;
  media?: boolean;            // Supports media attachments
  reactions?: boolean;        // Supports emoji reactions
  edit?: boolean;             // Supports message editing
  unsend?: boolean;           // Supports message deletion
  reply?: boolean;            // Supports reply threading
  polls?: boolean;            // Supports polls
  effects?: boolean;          // Supports message effects
  threads?: boolean;          // Supports thread conversations
  blockStreaming?: boolean;   // Cannot handle streaming responses
};

Media Processing Flow

flowchart TD
    INBOUND["Inbound media attachment"]
    INBOUND --> DETECT["Detect media type"]
    DETECT --> VALIDATE["Validate against
channel limits"]
    VALIDATE --> PASS{"Within limits?"}
    PASS -->|Yes| NORMALIZE["Normalize metadata"]
    PASS -->|No| REJECT["Skip / warn"]
    NORMALIZE --> EXTRACT["Extract content
(download URL or inline)"]
    EXTRACT --> CONTEXT["Attach to MsgContext"]
    CONTEXT --> AGENT["Forward to agent"]
    AGENT --> RESPONSE["Agent processes
media content"]

Platform-Specific Limits

Platform	Max Image Size	Max Document Size	Supported Formats
WhatsApp	16 MB	100 MB	JPEG, PNG, WebP, GIF, PDF, Office docs
Telegram	10 MB (photo)	2 GB (file)	Most common formats
Discord	25 MB (Nitro: 100 MB)	25 MB	Most common formats
Slack	Workspace-dependent	1 GB	Most common formats
Signal	100 MB	100 MB	Most common formats
iMessage	100 MB	100 MB	Apple-native + common formats

Media Normalization

Regardless of source platform, media is normalized to a common structure:

// Normalized media attachment (conceptual)
type NormalizedMedia = {
  type: "image" | "audio" | "video" | "document" | "sticker";
  mimeType: string;
  size?: number;          // bytes
  url?: string;           // download URL
  data?: Buffer;          // inline data
  filename?: string;
  caption?: string;
  thumbnail?: {
    url?: string;
    data?: Buffer;
  };
};

Interaction with Debounce Policy

Media attachments bypass the inbound debounce policy. While text-only messages are batched within the debounce window, messages containing media are processed immediately:

flowchart TD
    MSG["Inbound message"]
    MSG --> CHECK{"Has media?"}
    CHECK -->|Yes| IMMEDIATE["Process immediately
(skip debounce)"]
    CHECK -->|No| DEBOUNCE["Enter debounce window"]
    DEBOUNCE --> WAIT["Wait debounceMs"]
    WAIT --> PROCESS["Process batched text"]
    IMMEDIATE --> PROCESS_MEDIA["Process with media"]

Outbound Media Formatting

When the agent sends media back, the outbound adapter converts to the target channel’s expected format. Channels that don’t support a media type gracefully degrade — e.g., an image becomes a text link on IRC.

Feature	Markdown-Capable Channel	Plain-Text Channel
Image response	Inline `![alt](url)`	URL link
Code block	Fenced ```code```	Indented plain text
Bold/italic	`bold` / `italic`	Stripped to plain text