go encoding_error ai_generated partial

invalid UTF-8 rune string: "\xff"

ID: go/invalid-utf-8-rune-string

Also available as: JSON · Markdown · 中文
78%Fix Rate
85%Confidence
1Evidence
2024-03-15First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
Go 1.21 active — — —
Go 1.22 active — — —
Go 1.23 active — — —

Root Cause

Attempting to decode a byte sequence that is not valid UTF-8, often from external data sources like network streams or file reads without proper encoding handling.

generic

中文

尝试解码非有效UTF-8的字节序列，通常来自未正确处理编码的外部数据源，如网络流或文件读取。

Official Documentation

https://pkg.go.dev/unicode/utf8#DecodeRune

Workarounds

  1. 85% success Use strings.ToValidUTF8() to replace invalid bytes with the Unicode replacement character U+FFFD 
    Use strings.ToValidUTF8() to replace invalid bytes with the Unicode replacement character U+FFFD
  2. 90% success Read data as []byte and decode with utf8.Valid() validation before string conversion 
    Read data as []byte and decode with utf8.Valid() validation before string conversion

中文步骤

  1. Use strings.ToValidUTF8() to replace invalid bytes with the Unicode replacement character U+FFFD
  2. Read data as []byte and decode with utf8.Valid() validation before string conversion

Dead Ends

Common approaches that don't work:

  1. 95% fail

    Binary data cannot be safely represented as UTF-8 strings; this leads to data corruption and runtime panics.

  2. 80% fail

    These operations still expect valid UTF-8 and will produce the same error or silently corrupt data.