GRPC Roadmap
47 sections • 649 topics
- 1. Understanding gRPC Architecture
- 2. Understanding HTTP/2 Benefits
- 3. Comparing gRPC vs REST
- 4. Understanding Protocol Buffers
- 5. Understanding RPC Communication Patterns
- 6. Understanding gRPC Service Definition
- Example: Minimal service definition
- 7. Understanding Wire Format
- 8. Understanding Channel Abstraction
- 9. Understanding Stub Generation
- 10. Understanding gRPC Ecosystem
- 1. Installing gRPC Runtime
- 2. Installing Protocol Buffer Compiler
- 3. Installing Language-Specific Plugins
- 4. Setting Up Project Structure
- 5. Configuring Go Modules
- Example: go.mod for a gRPC service
- 6. Verifying Installation
- 7. Setting Up IDE Support
- 8. Configuring Environment Variables
- 9. Installing Development Tools
- 10. Setting Up Buf CLI
- Example: buf.yaml + buf.gen.yaml
- 1. Understanding Proto3 Syntax
- 2. Understanding Message Structure
- Example: Anatomy of a message
- 3. Understanding Field Numbering
- 4. Understanding Field Types
- 5. Understanding Field Rules
- 6. Understanding Default Values
- 7. Understanding Package Namespaces
- 8. Understanding Import Mechanism
- 9. Understanding Proto Compilation Process
- Compilation Pipeline
- 10. Understanding Comments and Documentation
- 1. Creating Message Types
- Example: Basic message
- 2. Defining Scalar Field Types
- 3. Assigning Field Numbers
- 4. Using Optional Fields
- Example: Presence tracking
- 5. Using Repeated Fields
- Example: Lists
- 6. Defining Nested Messages
- Example: Nested types
- 7. Creating Enumerations
- Example: Enum with safe default
- 8. Using Reserved Fields
- Example: Reserve removed fields
- 9. Defining Map Fields
- Example: Map field
- 10. Using Oneof Fields
- Example: Mutually exclusive payload
- 11. Importing Proto Files
- Example: Importing well-known + local files
- 12. Setting Package Names
- 1. Using Timestamp Type
- Example: Timestamp in a message
- 2. Using Duration Type
- 3. Using Any Type
- Example: Packing/unpacking Any
- 4. Using Struct Type
- 5. Using Empty Type
- Example: RPC with no input or output
- 6. Using FieldMask Type
- Example: Partial update with FieldMask
- 7. Using Wrapper Types
- 8. Importing Well-Known Types
- 9. Converting Between Native and Proto Types
- 10. Using Value Type
- 1. Implementing Custom Options
- Example: Defining and using a custom field option
- 2. Using Message Extensions
- 3. Working with Unknown Fields
- 4. Using Proto Reflection API
- Example: Iterate fields via reflection (Go)
- 5. Implementing Dynamic Messages
- 6. Using JSON Mapping
- 7. Configuring Field Behavior
- 8. Using API Annotations
- Example: HTTP mapping annotation
- 9. Implementing Proto Validation
- 10. Using Service Options
- Example: Default host + OAuth scope
- 1. Creating Service Definitions
- 2. Defining Unary RPCs
- Example: Unary RPC
- 3. Defining Server Streaming RPCs
- Example: Server stream
- 4. Defining Client Streaming RPCs
- Example: Client stream
- 5. Defining Bidirectional Streaming RPCs
- Example: Bidi stream
- 6. Using Request Messages
- 7. Using Response Messages
- 8. Defining Multiple Services
- Example: Multiple services in one file
- 9. Using Custom Method Options
- 10. Defining Empty Request or Response
- 1. Generating Code with protoc
- Example: Generate Go + gRPC code
- 2. Specifying Output Directory
- 3. Setting Import Paths
- 4. Generating Multiple Language Outputs
- Example: Multi-language generation
- 5. Using Generation Plugins
- 6. Generating gRPC Service Code
- 7. Generating Message Code Only
- 8. Setting Module Paths
- 9. Using Buf for Compilation
- Example: Generate with Buf
- 10. Automating Generation with Makefiles
- Example: Makefile target
- 1. Creating Server Instance
- Example: Basic server (Go)
- 2. Registering Service Handlers
- Example: Register implementation
- 3. Creating Network Listener
- Example: TCP listener
- 4. Starting Server
- Example: Serve
- 5. Implementing Graceful Shutdown
- Example: Signal-driven shutdown
- 6. Handling Server Errors
- 7. Setting Server Options
- 8. Configuring Max Message Size
- 9. Setting Connection Timeout
- 10. Enabling Server Reflection
- Example: Enable reflection (Go)
- 11. Using Multiple Listeners
- 1. Creating Client Connection
- Example: Modern client connection (Go)
- 2. Setting Dial Options
- 3. Creating Service Stub
- 4. Configuring Transport Security
- 5. Setting Connection Timeout
- Example: Bounded dial via per-RPC context
- 6. Using Keep-Alive Settings
- Example: Client keep-alive
- 7. Closing Connections
- 8. Implementing Connection Pooling
- 9. Handling Connection Errors
- 10. Using WithBlock Option
- 1. Creating Server Handler
- Example: Unary handler
- 2. Receiving Request Context
- 3. Accessing Request Fields
- 4. Returning Response
- 5. Returning Errors
- Example: Rich error details
- 6. Validating Request Data
- 7. Creating Client Call
- Example: Client unary call
- 8. Setting Call Options
- 9. Handling Client Response
- 10. Using Context Cancellation
- Example: Cancel mid-flight
- 1. Creating Streaming Handler
- Example: Server stream handler
- 2. Sending Multiple Responses
- 3. Detecting Client Disconnect
- Example: Check ctx in loop
- 4. Handling Stream Errors
- 5. Creating Client Receiver
- Example: Client receives stream
- 6. Receiving Stream Messages
- 7. Detecting End of Stream
- 8. Handling Receive Errors
- Example: Status-aware receive
- 9. Processing Stream Data
- 10. Closing Server Stream
- 1. Receiving Client Stream
- Example: Server reads client stream
- 2. Processing Incoming Messages
- 3. Sending Final Response
- 4. Aggregating Stream Data
- Example: Hash + size summary
- 5. Creating Client Stream
- Example: Open client stream
- 6. Sending Multiple Requests
- 7. Closing Client Stream
- 8. Handling Send Errors
- 9. Receiving Final Response
- Example: Close + receive
- 10. Using Stream Context
- 1. Receiving and Sending Concurrently
- Example: Bidi handler with goroutines
- 2. Creating Receive Loop
- 3. Creating Send Loop
- 4. Coordinating Goroutines
- 5. Handling Stream Completion
- 6. Detecting Stream Errors
- 7. Creating Client Bidirectional Stream
- Example: Client bidi loop
- 8. Implementing Request-Response Pattern
- 9. Implementing Full-Duplex Pattern
- 10. Closing Bidirectional Stream
- 1. Implementing Batch Streaming
- Example: Batched server stream
- 2. Using Chunked File Upload
- 3. Using Chunked File Download
- 4. Implementing Stream Multiplexing
- 5. Using Flow Control
- 6. Implementing Stream Cancellation
- Example: Cancel from client
- 7. Using Stream Heartbeats
- 8. Implementing Stream Reconnection
- Example: Reconnect with backoff
- 9. Handling Partial Failures
- 10. Using Stream Buffering
- 1. Creating Context with Timeout
- Example: Timeout context
- 2. Creating Context with Deadline
- Example: Deadline
- 3. Creating Context with Cancellation
- 4. Propagating Context
- 5. Checking Context Cancellation
- Example: Cooperative check
- 6. Getting Context Error
- 7. Adding Context Values
- Example: Typed key
- 8. Using Background Context
- 9. Setting Default Timeout
- Example: Interceptor sets default deadline
- 10. Handling Context Timeout Errors
- 1. Creating Metadata
- Example: Build MD
- 2. Sending Metadata from Client
- Example: Outgoing context
- 3. Receiving Metadata on Server
- Example: Read incoming MD
- 4. Sending Headers from Server
- 5. Sending Trailers from Server
- 6. Receiving Headers on Client
- Example: Capture headers
- 7. Receiving Trailers on Client
- Example: Capture trailers
- 8. Appending Metadata
- 9. Using Binary Metadata
- Example: Binary key
- 10. Forwarding Metadata
- Example: Proxy metadata downstream
- 1. Creating Status Errors
- 2. Using Standard Status Codes
- 3. Adding Error Details
- 4. Extracting Status from Error
- Example: Status extraction
- 5. Creating Custom Error Details
- Example: Custom detail message
- 6. Checking Specific Status Codes
- Example: Code-based handling
- 7. Handling Canceled Errors
- 8. Handling Timeout Errors
- 9. Propagating Errors
- 10. Converting Errors
- Example: Domain → gRPC mapping
- 1. Using Circuit Breaker Pattern
- Example: sony/gobreaker
- 2. Implementing Retry Logic
- 3. Implementing Fallback Strategies
- 4. Using Bulkhead Pattern
- 5. Implementing Timeout Patterns
- 6. Handling Partial Failures
- 7. Using Dead Letter Queues
- 8. Implementing Compensating Transactions
- 9. Using Health Checks for Recovery
- 10. Implementing Graceful Degradation
- 1. Using protoc-gen-validate Plugin
- 2. Defining Validation Rules
- Example: protovalidate rules
- 3. Generating Validation Code
- 4. Calling Validate Method
- Example: Runtime validation (Go)
- 5. Handling Validation Errors
- 6. Implementing Custom Validators
- Example: CEL expression
- 7. Validating Nested Messages
- 8. Validating Repeated Fields
- 9. Using Validation Interceptor
- Example: Auto-validate interceptor
- 10. Setting Custom Error Messages
- 1. Creating Unary Server Interceptor
- Example: Logging unary server interceptor
- 2. Creating Stream Server Interceptor
- Example: Stream interceptor
- 3. Creating Unary Client Interceptor
- Example: Auth-injecting client interceptor
- 4. Creating Stream Client Interceptor
- 5. Chaining Interceptors
- Example: Chain on server & client
- 6. Implementing Logging Interceptor
- 7. Implementing Auth Interceptor
- Example: Token validation interceptor
- 8. Implementing Recovery Interceptor
- Example: Panic recovery
- 9. Modifying Request and Response
- 10. Registering Interceptors
- 1. Setting Client Deadline
- 2. Setting Client Timeout
- Example: Per-call timeout
- 3. Checking Remaining Time
- Example: Compute leftover budget
- 4. Propagating Deadlines
- 5. Setting Per-Call Timeout
- 6. Using Default Timeout
- Example: Service config default
- 7. Handling Deadline Exceeded
- 8. Implementing Timeout Retry
- 9. Setting Server-Side Timeout
- 10. Using WaitForReady Option
- Example: WaitForReady
- 1. Generating TLS Certificates
- Example: Self-signed dev certs
- 2. Loading Server Certificates
- Example: Load cert pair
- 3. Creating Server TLS Config
- Example: Server with TLS
- 4. Creating Client TLS Config
- Example: Client TLS
- 5. Using System Certificate Pool
- 6. Adding Custom CA Certificates
- Example: Append custom CA
- 7. Setting Server Name
- 8. Enabling Client Certificate Verification
- 9. Implementing Mutual TLS
- Example: Strict mTLS server
- 10. Using Let's Encrypt Certificates
- 1. Using TLS Credentials
- 2. Implementing Token-Based Auth
- Example: PerRPCCredentials with token
- 3. Creating Custom Credentials
- 4. Validating Tokens in Interceptor
- Example: JWT verification
- 5. Implementing JWT Authentication
- 6. Setting Insecure Connection
- Example: Insecure for local dev
- 7. Using Transport Credentials
- 8. Implementing OAuth2
- Example: oauth2 token source
- 9. Extracting Peer Information
- Example: Peer info on server
- 10. Implementing API Key Auth
- 1. Implementing Role-Based Access Control
- Example: Role check in interceptor
- 2. Implementing Attribute-Based Access Control
- 3. Creating Authorization Interceptor
- 4. Defining Permission Policies
- Example: OPA Rego policy
- 5. Using Open Policy Agent
- 6. Implementing Method-Level Authorization
- 7. Handling Authorization Errors
- 8. Using Claims from JWT
- Example: Read claims from ctx
- 9. Implementing Resource-Based Access
- 10. Auditing Authorization Decisions
- 1. Understanding Connection Lifecycle
- 2. Configuring Connection Timeout
- 3. Implementing Connection Pooling
- Example: Round-robin pool
- 4. Reusing Connections
- 5. Handling Connection Failures
- 6. Detecting Connection State
- Example: Read state
- 7. Waiting for Connection Ready
- Example: Wait for READY
- 8. Closing Connections Gracefully
- 9. Handling Connection Leaks
- 10. Using Connection State Callbacks
- 1. Configuring Client Keep-Alive
- 2. Setting Keep-Alive Time
- 3. Setting Keep-Alive Timeout
- 4. Allowing Pings Without Stream
- Example: Idle pings
- 5. Configuring Server Keep-Alive
- Example: Server keep-alive
- 6. Setting Enforcement Policy
- Example: Enforcement
- 7. Setting Min Time Between Pings
- 8. Permitting Without Stream on Server
- 9. Detecting Connection Loss
- 10. Tuning Keep-Alive for Production
- 1. Enabling gzip Compression
- Example: Per-call gzip
- 2. Setting Compression Level
- 3. Registering Custom Compressor
- Example: Custom compressor registration
- 4. Configuring Client Compression
- 5. Configuring Server Compression
- 6. Detecting Compression Support
- 7. Disabling Compression
- 8. Implementing Custom Compression
- 9. Setting Compression Threshold
- 10. Measuring Compression Efficiency
- 1. Using Round Robin Policy
- Example: Enable round_robin
- 2. Configuring Client-Side Load Balancing
- 3. Using DNS Resolver
- 4. Implementing Custom Resolver
- Example: Custom resolver skeleton
- 5. Registering Custom Balancer
- 6. Using Pick First Strategy
- 7. Configuring Service Config JSON
- Example: Service config
- 8. Implementing Health-Based Balancing
- 9. Using External Load Balancer
- 10. Understanding Subchannel Management
- 1. Using DNS-Based Discovery
- 2. Implementing Custom Resolver
- 3. Registering Custom Resolver
- Example: Register at init
- 4. Using Consul for Discovery
- 5. Using etcd for Discovery
- 6. Using Kubernetes Service Discovery
- 7. Implementing Static Resolution
- Example: Manual resolver with static list
- 8. Updating Resolver State
- Example: Push new addresses
- 9. Handling Service Updates
- 10. Testing Service Discovery
- 1. Configuring Retry Policy
- Example: Built-in retry via service config
- 2. Setting Max Attempts
- 3. Configuring Initial Backoff
- 4. Configuring Max Backoff
- 5. Setting Retryable Status Codes
- 6. Using Hedging Policy
- Example: Hedging
- 7. Implementing Custom Retry Logic
- Example: Custom retry interceptor
- 8. Setting Per-Attempt Timeout
- 9. Using Exponential Backoff
- 10. Handling Non-Idempotent Methods
- 1. Using Token Bucket Algorithm
- 2. Implementing Rate Limit Interceptor
- Example: Global rate limit
- 3. Using golang.org/x/time/rate Package
- 4. Setting Rate Limit per Client
- Example: Per-API-key limiter
- 5. Returning ResourceExhausted Status
- 6. Implementing Sliding Window
- 7. Using Redis for Distributed Limiting
- Example: Redis token bucket (Lua)
- 8. Adding Rate Limit Headers
- 9. Implementing Per-Method Limits
- 10. Configuring Burst Size
- 1. Using grpclog Package
- Example: Set custom grpc logger
- 2. Setting Log Level
- 3. Implementing Custom Logger
- 4. Logging Requests and Responses
- Example: Logging interceptor
- 5. Using Structured Logging
- 6. Logging Metadata
- 7. Logging Request Duration
- 8. Logging Error Details
- 9. Filtering Sensitive Data
- 10. Integrating with Logging Framework
- 1. Using OpenTelemetry SDK
- 2. Creating Trace Provider
- Example: OTLP exporter + provider
- 3. Instrumenting Client with Interceptor
- Example: Client otelgrpc
- 4. Instrumenting Server with Interceptor
- Example: Server otelgrpc
- 5. Creating Spans
- Example: Manual span
- 6. Adding Span Attributes
- Example: Set attrs
- 7. Recording Span Events
- Example: Event & error
- 8. Propagating Trace Context
- 9. Exporting Traces
- 10. Correlating Logs and Traces
- Example: Add trace id to log
- 1. Using Prometheus Client
- Example: Register collector + endpoint
- 2. Creating Counter Metrics
- Example: Counter
- 3. Creating Histogram Metrics
- Example: Histogram for latency
- 4. Creating Gauge Metrics
- 5. Registering Metrics
- 6. Exposing Metrics Endpoint
- 7. Instrumenting with Interceptor
- Example: Metric interceptor
- 8. Tracking Request Count
- 9. Tracking Request Duration
- 10. Adding Custom Labels
- 11. Monitoring Error Rates
- Example: PromQL error rate
- 12. Tracking Active Streams
- 1. Implementing Health Service
- 2. Registering Health Server
- Example: Register health
- 3. Setting Service Status
- 4. Creating Health Check Client
- Example: Client
- 5. Calling Check Method
- 6. Using Watch Method
- Example: Watch stream
- 7. Handling Status Changes
- 8. Implementing Custom Health Logic
- 9. Integrating with Load Balancer
- 10. Setting Overall Server Status
- 1. Registering Reflection Service
- Example: Enable reflection (Go)
- 2. Using grpc_cli Tool
- 3. Listing Services
- Example: grpcurl list
- 4. Describing Service Types
- 5. Calling Methods via CLI
- Example: grpcurl invoke
- 6. Using grpcurl Tool
- 7. Discovering Service Schema
- 8. Disabling Reflection in Production
- 9. Using Postman with Reflection
- 10. Implementing Custom Reflection
- 1. Understanding gRPC Gateway
- 2. Generating Gateway Code
- Example: buf.gen.yaml entry
- 3. Creating Gateway Server
- Example: Mux + register
- 4. Registering Gateway Handlers
- 5. Adding HTTP Annotations
- Example: HTTP rules
- 6. Configuring REST Paths
- 7. Mapping HTTP Methods
- 8. Handling Query Parameters
- 9. Handling Path Parameters
- 10. Configuring Custom Marshaling
- Example: Custom JSON marshaler
- 1. Understanding gRPC-Web Protocol
- 2. Installing gRPC-Web Package
- Example: npm install
- 3. Configuring grpcwebtext Mode
- 4. Configuring grpcweb Mode
- 5. Setting Up Envoy Proxy
- Example: Envoy filter snippet
- 6. Generating JavaScript Client
- Example: protoc gen js + grpc-web
- 7. Creating Browser Client
- Example: TypeScript client call
- 8. Handling CORS
- 9. Using Unary Calls in Browser
- 10. Using Server Streaming in Browser
- Example: Server streaming
- 1. Using Envoy as gRPC Proxy
- 2. Configuring Envoy Routes
- Example: gRPC route
- 3. Using NGINX for gRPC
- Example: nginx grpc_pass
- 4. Implementing Auth Middleware
- 5. Implementing Logging Middleware
- 6. Implementing Caching Middleware
- 7. Implementing Compression Middleware
- 8. Chaining Middleware
- Example: Chain with go-grpc-middleware
- 9. Setting Middleware Order
- 10. Implementing Custom Middleware
- 1. Writing Unit Tests for Services
- Example: Test service method directly
- 2. Using bufconn for In-Memory Server
- Example: bufconn listener
- 3. Mocking gRPC Server
- 4. Mocking gRPC Client
- Example: mockgen client
- 5. Testing Unary RPCs
- 6. Testing Streaming RPCs
- Example: Drain server stream
- 7. Testing Error Cases
- 8. Testing Interceptors
- 9. Using Test Doubles
- 10. Implementing Integration Tests
- 11. Implementing Load Tests
- 1. Enabling Verbose Logging
- 2. Using grpc.EnableTracing
- Example: Enable internal tracing
- 3. Inspecting Wire Traffic with tcpdump
- Example: Capture gRPC traffic
- 4. Using Wireshark for gRPC
- 5. Debugging Connection Issues
- 6. Debugging TLS Issues
- 7. Debugging Performance Issues
- 8. Using pprof for Profiling
- Example: Enable pprof
- 9. Debugging Memory Leaks
- 10. Using gRPC Channelz
- Example: Enable channelz
- 1. Tuning Message Size
- 2. Tuning Window Size
- Example: HTTP/2 windows
- 3. Using Connection Pooling
- 4. Enabling Multiplexing
- 5. Optimizing Message Encoding
- 6. Reducing Allocations
- 7. Implementing Caching
- 8. Batching Requests
- 9. Using Streaming for Bulk Data
- 10. Tuning Thread Pool
- 11. Profiling gRPC Performance
- 12. Optimizing Network Usage
- 1. Adding New Fields Safely
- 2. Deprecating Fields
- Example: Deprecated option
- 3. Reserving Field Numbers
- Example: Reserve after delete
- 4. Versioning Services
- 5. Handling Schema Evolution
- 6. Using Buf Breaking Detection
- Example: buf breaking
- 7. Maintaining API Compatibility
- 8. Migrating Clients
- 9. Supporting Multiple Versions
- 10. Sunsetting Old Versions
- 1. Following Naming Conventions
- 2. Versioning APIs Properly
- 3. Designing Idempotent Methods
- 4. Using Appropriate RPC Type
- 5. Setting Proper Deadlines
- 6. Handling Errors Properly
- 7. Documenting Services
- 8. Monitoring Services
- 9. Testing Thoroughly
- 10. Securing Communications
- 11. Implementing Graceful Shutdown
- Example: Signal-driven shutdown
- 12. Reviewing Code Regularly