A redactable string (or byte array) is a string where unsafe information is enclosed between special delimiters. For example,
ReadactableString(var s RedactableString = “hello ‹secret›“)
contains the safe word “hello” and unsafe word “secret”.
String redaction is a function that deletes the data between delimiters to produce a redacted string.
The remaining APIs can:
introduce safe information the guarantee of safety from scratch,
promise that unsafe information is, in fact, safe; or and that redactable strings are, in fact, redactable.
transform information and compose redactable strings in a way that is proven to preserve redactability without leaking sensitive information.
Where can users observe redactable information?
Redactable information Redactability (i.e. information where sensitive and safe bits can be separated from each other) can be found:
In log files or network log entries produced by CockroachDB with the
redactable: trueconfiguration flag set. Here is an example redactable log entry:
I210413 09:51:03.906798 14 heapprofiler.go:49 ⋮ [n2] 34 writing go heap profiles to ‹/home/kena/cockroach/cockroach-data/logs/heap_profiler› at least every 1h0m0s ^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this symbol means that the rest of the entry is redactable unsafe information
In crash reports, prior to sending by CockroachDB to Sentry.io (via the crash reporting facility). to Cockroach Labs for monitoring and error tracking. The unsafe information is redacted on the way out.
(The filename was marked as redactable-unsafe in the error message, and was redacted into “×” on the way outbefore sending. The web display erased the redaction markers.)
errorobjects produced / managed by the CockroachDB errors library.
Why? Because errors eventually get translated into log entries crash reports, see above. If the errors were not redactable to start with, we couldn’t make log entries / crash reports redactable.
(The redaction markers inside errors are hidden when looking via the
.Error()method, but appear when using
At some point in the future (relative to the date of this writing, May 2021), in distributed traces produced by CockroachDB, which can be inspected during troubleshooting.
(At the time of this writing, traces are not redactable and thus should be considered thoroughly unsafe as per the definition above.)
Certain data structures and strings held in RAM inside CockroachDB, when they are likely to be included in log messages or error payloads.
More redactability = more observability / + more troubleshootability.
The various APIs try to minimize the work needed by CockroachDB programmers, but sometimes extra care must be taken.
The constant literal string used as first argument to
Wrapetc, as well as
Infofetc is axiomatically considered safe.
The redactable contents of
errorobjects are automatically recognized and propagated when constructing further error objects or log entries from them.
(Certain common error types from the Go runtime are also properly recognized to separate their safe vs unsafe payloads.)
Certain data types outside of CockroachDB’s own source code have been marked as always-safe using the “safe type registry” (
redact.RegisterSafeType), because we consider that they can never been traced back to individual customers or PII. This includes, for example:
Go’s native booleans, integers & float types
Remaining data types are considered unsafe by default.
This is especially the case with struct types and other Go types that alias basic types.
To mark a data type as safe or redactable , when it would be considered unsafe otherwise, use:
For simple types that alias a Go numeric type, you can axiomatically mark the type as always-safe by marking it with the
For more complex types or types that alias the Go
stringtype, implement a
SafeFormatmethod (i.e. the
The primitives available in the body of
SafeFormatprovably generate redactable strings. See below for examples.
RedactableStringvalues upfront, then store them until later, instead of composing a
stringvalue, storing it into a struct, and then later trying to include it into an error or log message,
RedactableStringvalue upfront, then store that until later.
To compose a redactable string from a mix of safe and unsafe information, use:
redact.Sprintf()to create a
RedactableStringfrom various bits using fmt.Print / Printf-like formatting.
redact.JoinTo() to adjoin a list of various bits using a delimiter and form a
RedactableStringa bit like strings.Join
redact.StringBuilderto compose a