GRPC_INTERNAL_PANIC_INTERCEPTOR grpc runtime_error ai_generated true

INTERNAL: panic caught in unary client interceptor: runtime error: invalid memory address or nil pointer dereference

ID: grpc/client-interceptor-panic-recovery

Also available as: JSON · Markdown · 中文
85%Fix Rate
88%Confidence
1Evidence
2024-03-15First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
gRPC-go v1.63.0 active — — —
gRPC-java v1.62.0 active — — —
envoy v1.29.0 active — — —

Root Cause

A client-side interceptor panicked due to a nil pointer dereference, which was recovered by gRPC's internal recovery interceptor but not properly handled upstream.

generic

中文

客户端拦截器因空指针解引用而恐慌，被 gRPC 内部恢复拦截器捕获，但未在上游妥善处理。

Official Documentation

https://github.com/grpc/grpc-go/blob/master/interceptor.go

Workarounds

  1. 90% success Add explicit nil checks before dereferencing context values in the interceptor. For example, in Go: if md, ok := metadata.FromIncomingContext(ctx); ok { ... } else { return nil, status.Error(codes.Internal, "missing metadata") } 
    Add explicit nil checks before dereferencing context values in the interceptor. For example, in Go: if md, ok := metadata.FromIncomingContext(ctx); ok { ... } else { return nil, status.Error(codes.Internal, "missing metadata") }
  2. 85% success Wrap the interceptor with a custom recovery middleware that converts panics to gRPC errors with stack traces: use grpc_recovery.UnaryServerInterceptor() from go-grpc-middleware. 
    Wrap the interceptor with a custom recovery middleware that converts panics to gRPC errors with stack traces: use grpc_recovery.UnaryServerInterceptor() from go-grpc-middleware.
  3. 75% success Upgrade to gRPC v1.64.0+ which includes improved panic recovery in interceptors, and enable verbose logging with GRPC_GO_LOG_SEVERITY_LEVEL=info to capture stack traces. 
    Upgrade to gRPC v1.64.0+ which includes improved panic recovery in interceptors, and enable verbose logging with GRPC_GO_LOG_SEVERITY_LEVEL=info to capture stack traces.

中文步骤

  1. Add explicit nil checks before dereferencing context values in the interceptor. For example, in Go: if md, ok := metadata.FromIncomingContext(ctx); ok { ... } else { return nil, status.Error(codes.Internal, "missing metadata") }
  2. Wrap the interceptor with a custom recovery middleware that converts panics to gRPC errors with stack traces: use grpc_recovery.UnaryServerInterceptor() from go-grpc-middleware.
  3. Upgrade to gRPC v1.64.0+ which includes improved panic recovery in interceptors, and enable verbose logging with GRPC_GO_LOG_SEVERITY_LEVEL=info to capture stack traces.

Dead Ends

Common approaches that don't work:

  1. 95% fail

    Restarting the client process without fixing the interceptor code will cause the same panic on the next request.

  2. 85% fail

    Adding a generic recovery handler at the application level that logs but returns a generic error masks the root cause, leading to silent data corruption.

  3. 70% fail

    Disabling all interceptors removes tracing and auth, but the underlying nil pointer might still exist in the call path.