Designing Message Formats

1. Designing Message Structure

FieldPurpose
vSchema version
typeDispatch key
idCorrelation ID
tsTimestamp (ms epoch)
seqPer-channel sequence
payloadType-specific body
metaTrace IDs, source, etc.

2. Using JSON Format

AspectDetail
ProsHuman-readable, debuggable, native parsers
ConsVerbose, no binary, slow parse for large msgs
ToolsJSON Schema, zod, ajv

Example: JSON envelope

{ "v": 1, "type": "trade", "id": "abc", "ts": 1715900000000, "payload": { "sym": "BTC", "px": 65000.5 } }

3. Using MessagePack

AspectDetail
EncodingSchema-less binary; JSON-like model
Size~30-50% smaller than JSON
JS library@msgpack/msgpack
Typesint, float, bool, str, bin, array, map, ext

Example: MessagePack

import { encode, decode } from "@msgpack/msgpack";
ws.binaryType = "arraybuffer";
ws.send(encode({ "type": "tick", "px": 65000.5 }));
ws.addEventListener("message", (e) => console.log(decode(new Uint8Array(e.data))));

4. Using Protocol Buffers

AspectDetail
SchemaRequired (.proto file)
Libraryprotobufjs, google-protobuf
SizeSmallest of common formats
EvolutionBackward/forward compatible if rules followed

Example: protobuf.js

const root = await protobuf.load("chat.proto");
const Msg = root.lookupType("chat.Message");
const buf = Msg.encode(Msg.create({ "text": "hi" })).finish();
ws.send(buf);

5. Using CBOR

AspectDetail
SpecRFC 8949
Librarycbor-x (fast)
StrengthsSelf-describing, IoT-friendly, supports tags
vs MessagePackSimilar; CBOR is IETF-standard

6. Implementing Custom Binary Format

Example: Fixed-layout header

// [u8 type][u32 length][bytes payload]
function encode(type, payload) {
  const buf = new ArrayBuffer(5 + payload.byteLength);
  const v = new DataView(buf);
  v.setUint8(0, type);
  v.setUint32(1, payload.byteLength, false);
  new Uint8Array(buf, 5).set(new Uint8Array(payload));
  return buf;
}
FieldUse
MagicValidate protocol
VersionSchema rev
TypeRouting
LengthPayload bound
CRCIntegrity (optional)

7. Adding Message Versioning

StrategyDetail
Envelope field vPer-message version
SubprotocolNegotiate at handshake (chat.v2)
URL path/v2/ws
Feature flagsServer announces capabilities

8. Including Message Metadata

MetadataUse
traceIdDistributed tracing
correlationIdMatch req/resp
userId / sessionIdAuth context
sourceOrigin service
ttlDrop after expiry

9. Implementing Message Acknowledgment

Example: Ack flow

// sender
ws.send(JSON.stringify({ "type": "evt", "id": id, "payload": data }));
// receiver
ws.send(JSON.stringify({ "type": "ack", "id": id }));
Ack TypeUse
Application ackConfirm processing
Negative ack (nack)Request resend
Cumulative ack"All up to seq N"

10. Validating Message Schema

ToolApproach
JSON Schema + ajvDeclarative, language-agnostic
zod / valibotTS-native, runtime + types
ProtobufSchema is compiled in
AvroSchema evolution heavy