feat(materials): add binary-safe inline material support#3030
Merged
migmartri merged 5 commits intochainloop-dev:mainfrom Apr 14, 2026
Merged
feat(materials): add binary-safe inline material support#3030migmartri merged 5 commits intochainloop-dev:mainfrom
migmartri merged 5 commits intochainloop-dev:mainfrom
Conversation
Add a new `bytes raw_value` field to the AttestationItem.Material proto message to support binary content in inline materials. Previously, binary data stored inline caused gRPC marshaling errors because `string value` requires valid UTF-8. The change flows through the internal NormalizedMaterial.Value type (now []byte) and dual-populates at the API boundary: raw_value always set, deprecated value set only when content is valid UTF-8. Closes chainloop-dev#2065 Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
The FIELD_SAME_TYPE breaking change rule is expected for the NormalizedMaterial.value field change from string to bytes. This is wire-compatible and the fanout proto is an internal plugin SDK, not a public API. Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>
jiparis
reviewed
Apr 14, 2026
| // older control plane versions that don't populate raw_value. | ||
| var value string | ||
| if len(in.GetRawValue()) > 0 { | ||
| value = string(in.GetRawValue()) |
Member
There was a problem hiding this comment.
What if RawValue doesn't contain UTF8 content? This line will panic. I think you'll need to validate here as well with utf8.Valid
Check UTF-8 validity before allocating string to avoid wasted allocation for binary content. Add round-trip test verifying binary data survives the structpb.Struct serialization path. Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>
Revert fanout proto value field to string since plugins expect text. Guard the assignment with UTF-8 validation so binary inline content is only available via the Content bytes field. Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>
jiparis
approved these changes
Apr 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When binary data is stored inline (no CAS backend configured),
wf run describecrashes with:The root cause: binary file content flows through
NormalizedMaterial.Value(Gostring) and intoAttestationItem.Material.value(protostring). Protobuf 3 requiresstringfields to be valid UTF-8, so gRPC rejects the response. The write path is correct —Artifact.contentin the crafting state proto isbytes, and the in-totoResourceDescriptor.contentis alsobytes. The corruption only happens on the read/response path whennormalizeMaterial()casts[]bytetostring.Approach
Add a new
bytes raw_valuefield to the API response proto alongside the existing (now deprecated)string value. Internally, changeNormalizedMaterial.Valuefromstringto[]byteso binary content flows cleanly through the pipeline. Dual-populate at the API boundary for backward compatibility.Changes
bytes raw_valuefield toAttestationItem.Materialproto, markstring valueas deprecatedNormalizedMaterial.Valuefromstringto[]byteraw_valuealways set, deprecatedvalueset only for valid UTF-8raw_valuewith fallback tovaluefor backward compat with older serversstring valuewith UTF-8 guard — binary content available viaContentbytes fieldTesting
TestBinaryContentStructRoundTrip) verifies data survives thestructpb.Structserialization path (json.Marshal -> protojson.Unmarshal -> protojson.Marshal -> json.Unmarshal) with null bytes, ELF headers, all 256 byte values, and 64KB payloadsextractMaterialsconfirms binary content populatesraw_valuewhilevaluestays empty.tgz(gzip binary) round-trips intact viaraw_value, while a 348KB CycloneDX JSON SBOM populates both fields correctlyCompatibility
value(deprecated) — works for UTF-8 content, empty for binary (no crash)raw_value— works for all contentvalue— same as todayCloses #2065