Connect your game
- Start a WebSocket listener bound only to
127.0.0.1 at ws://127.0.0.1:PORT/rp-engine/socket. Do not expose it on your LAN or the internet.
- Use the same port in the game integration and RPEngine settings. The default is
38471.
- Accept the production browser origin
https://rpengine.gemtavern.com. Local development may additionally allow http://127.0.0.1:5173.
- Limit incoming messages to 8 MiB and accept only the expected WebSocket path and browser origins.
- Your game may open RPEngine with its port in the URL so the player does not need to enter it manually.
https://rpengine.gemtavern.com/#port=38471
Connection handshake
RPEngine sends hello immediately after opening the WebSocket. Reply within five seconds with welcome containing a new sessionId and your serverVersion. RPEngine repeats this handshake whenever it reconnects.
Every envelope includes protocol: "gemtavern.rp_engine", protocolVersion: 3, a unique messageId, an ISO 8601 timestamp, and type. The initial hello has no session yet; welcome establishes one, and later messages use that sessionId.
RPEngine returns ack for valid game or mod messages. A repeated messageId is acknowledged as a duplicate and is not processed twice. Do not acknowledge an ack.
{
"protocol": "gemtavern.rp_engine",
"protocolVersion": 3,
"type": "welcome",
"messageId": "unique-message-id",
"sessionId": "new-session-id",
"timestamp": "2026-07-16T00:00:00.000Z",
"serverVersion": "my-game-1.0"
}
Synchronize a character
Send character.sync to preload or update a character without starting inference. Include integrationId, characterId, and a Character Card V2 transfer. RPEngine replies with character.synced and the accepted targetHash.
Character caches belong to one connection session. Send a snapshot after every new handshake before using patch or reference transfers.
Reply request
The following example assumes this character hash was already acknowledged by character.synced or an earlier snapshot request in the current session.
{
"protocol": "gemtavern.rp_engine",
"protocolVersion": 3,
"type": "reply.request",
"messageId": "message-id",
"sessionId": "session-id",
"timestamp": "2026-07-14T00:00:00.000Z",
"requestId": "request-id",
"eventId": "event-id",
"integrationId": "my-game",
"characterId": "character-1",
"event": {
"text": "Optional typed context alongside the recording.",
"audio": {
"format": "pcm_s16le",
"sampleRate": 16000,
"channels": 1,
"language": "en",
"data": "BASE64_PCM_DATA"
}
},
"output": {
"modalities": ["text", "audio"],
"language": "en",
"audio": {
"model": "gemtavern-supertonic-3",
"voice": "F4",
"format": "pcm_s16le"
}
},
"player": { "displayName": "Player" },
"card": {
"format": "chara_card_v2",
"mode": "reference",
"targetHash": "sha256-canonical-card-hash"
}
}
Provide event.text, event.audio, or both. Audio must be mono, 16 kHz, base64-encoded pcm_s16le or pcm_f32le, and no longer than 30 seconds. Supported audio languages are en, ar, es, ja, ko, vi, uk, and zh.
Games may include interactionMode, promptScene, and promptDirective together. RPEngine uses autoEventGuide only for auto_event, and directUserGuide only for direct_user; sceneContext supplies the shared scene once.
Use ["text"] for a text response. Use ["text", "audio"] with model gemtavern-supertonic-3, format pcm_s16le, and voice F1–F5 or M1–M5 when your game also wants speech.
Supported speech-output languages are en, ko, es, pt, fr, de, it, pl, ru, nl, cs, ar, zh, ja, hu, tr, fi, sk, da, hr, el, sv, nb, he, uk, id, ms, vi, th, ro, and bg.
Browser microphone request
Use voice.capture.start when the player should speak through the microphone attached to the RPEngine browser. It uses the same requestId, eventId, integrationId, characterId, output, player, card transfer, and optional prompt fields as reply.request, but it does not include event.
The player must enable microphone access in RPEngine first. RPEngine reports voice.capture.level and voice.capture.state. FireRedVAD normally ends speech automatically; the game may send voice.capture.stop to finish manually or voice.capture.cancel to discard the capture. Successful speech then follows the normal reply stream.
Character card transfer
Character Card V2 is the canonical character format. Send a complete chara_card_v2 document with spec_version: "2.0" and all required fields. Use empty values where your game has no content; unknown fields and extension data are preserved.
snapshot: full Character Card V2 plus targetHash.patch: RFC 6902 operations plus baseHash and targetHash.reference: an unchanged session card identified by targetHash.
Hashes are lowercase SHA-256 over RFC 8785 canonical JSON. Begin a connection with a snapshot. If RPEngine returns card_resync_required, resend the request with a snapshot.
Response stream
After reply.accepted, text and audio stream concurrently. RPEngine sends reply.text.delta messages while generation continues and begins sending sentence-level reply.audio.chunk messages as soon as complete sentences are available. Audio may therefore begin before reply.text.completed. The request ends with reply.completed after all requested output is complete.
Audio chunks contain base64-encoded 44.1 kHz mono PCM16. Group chunks by segmentSequence, then play each completed segment in order while later text and audio continue generating. Your game controls volume, subtitles, interruption, and spatial placement.
reply.accepted: the request and card hash were accepted.reply.text.delta: clean incremental display text with a zero-based sequence.reply.text.completed: complete display text plus generation statistics.reply.audio.start: audio format, sample rate, and channel count.reply.audio.chunk: global and sentence-segment sequence fields plus base64 PCM bytes.reply.audio.completed: final segment, chunk, byte, duration, and synthesis totals.reply.completed: terminal success after all requested modalities finish.
Capacity, cancellation, and errors
RPEngine sends capacity.update after connection and whenever capacity changes. It contains acceptingRequests, queueDepth, and queueLimit. Submit new work only while acceptingRequests is true.
Send request.cancel with requestId to cancel a queued or active reply. RPEngine answers with reply.cancelled. For microphone capture, use voice.capture.cancel.
request.error contains requestId when available, sourceMessageId, code, message, and retryable. Common codes include card_resync_required, invalid_character_card, service_not_started, capacity_exceeded, duplicate_request, microphone_not_enabled, unsupported_language, unsupported_voice, and request_failed.
