* `protoc`: Included in zips alongside binary
* Java: Generated classes have been added to the Maven `util` artifact
* Ruby: Generated classes have been added to gems
* CMake: `.proto` files are included in installation
PiperOrigin-RevId: 957382561
(cherry picked from commit 713b6a4a24)
Avoid generating double-checked locking inside every generated message class's
dynamicMethod(GET_PARSER) by centralizing the parser instantiation and caching
in a thread-safe map inside GeneratedMessageLite.
This saves substantial DEX and ODEX size for apps with many protobuf classes.
Note: This change requires the generated code to run against a runtime of at
least the same version. Mixing new generated code with an older runtime version
is not supported (see https://protobuf.dev/support/cross-version-runtime-guarantee/).
PiperOrigin-RevId: 944139195
## Summary
`mergeDelimitedFrom` passes the return value of `readRawVarint32` directly to `LimitedInputStream` without checking for negative values. For 5-byte varints encoding values >= 2^31, `readRawVarint32` returns a negative int (e.g., `\x80\x80\x80\x80\x08` decodes to -2147483648).
`LimitedInputStream` with a negative limit immediately returns EOF (`if (limit <= 0) return -1`), causing the parser to produce a default/empty message and return `true` (success). The actual message body bytes remain unconsumed in the stream, permanently desynchronizing subsequent `parseDelimitedFrom` calls.
## Fix
Add `if (size < 0) throw InvalidProtocolBufferException.negativeSize()` before constructing `LimitedInputStream`.
## Test plan
- [x] Verified with PoC: 5-byte varint prefix encoding 2^31 previously produced empty message + stream desync; after fix, `negativeSize` exception thrown
- [ ] Existing unit tests pass
Closes#28097
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/28097 from TristanInSec:fix/merge-delimited-negative-size 7ccb18676e
PiperOrigin-RevId: 937547094
- Apply a fast path to Uint64 parse where we try to parse it directly before falling back to the BigDecimal path. This optimization is preexisting on int64 and int32 and uint32 but was missed on Uint64
- Avoid doing out.append(value.subSequence(start, end)) which forces an intermediate copy the string, instead use the out.append(value, start, end)
PiperOrigin-RevId: 937290790
Because JsonFormat uses GSON to parse JSON it has a variety of
leniencies, e.g. allowing keys and values to skip quotes:
```
{hello: world}
```
These behaviours are not part of the ProtoJSON format but they are
behaviours that JsonFormat's parser has. It's possible someone in the
wild depends upon these unknowingly. This adds explicit tests for these
behaviours to make sure that they are not accidentally lost in future.
Closes#27622
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/27622 from petedmarsh:json-format-lienent-tests edebd543cd
PiperOrigin-RevId: 933849754
Note that as this case is only reachable for known groups, its not a usable primitive for fully unconstrained recursion, since even if a recursive schema did use a group it would call back to a message between, meaning that even in such an obscure case this would only unlock reaching depth 200 not unconstrained.
PiperOrigin-RevId: 930082333
These interfaces were part of the experimental runtime, and before this change have exactly one concrete implementation each. This CL simply deletes the interfaces and inlines the corresponding concrete type wherever they were named.
PiperOrigin-RevId: 930019405
The previous change looked to get down to exactly 2 CodedOutputStream concrete classes: unfortunately some users are relying on the incidental-but-reliable behavior that CodedOutputStreams with ByteBuffers do immediately push the content without a flush() but that the position update does not apply until you call flush(). This oddity has been incidentally/accidentally true for both Heap and Direct ByteBuffer cases even though they didn't use a common codepath.
To move incrementally forward, we can merge in HeapNio into ArrayEncoder without touching DirectNio. For now this still leaves 3 totalconcrete classes, but it will achieve the 'exactly 2 concrete classes loaded' for any server if they never reach DirectNioEncoder or never reach the OutputStreamEncoder.
PiperOrigin-RevId: 929337270
This requires sprinkling some casts around locally, mainly surrounding the mismatch of apis that return `MessageLite` which on LiteRuntime is guaranteed to be `GeneratedMessageLite`, but statically thats not known because Full messages also implement it.
PiperOrigin-RevId: 928716180
This still retains the reflection path for these types so that it will continue to handle DynamicMessage printing correctly.
PiperOrigin-RevId: 928689274
- Use a lookup table for the replacement characters in the ascii range (which is all of the characters that gson escapes except for exactly 2).
- Stream the escapes out ourselves instead of going through gson.toJson(val.toString()) which goes through a lot of heavy gson machinery and makes several copies of the string.
We exactly match the choices that gson made for escapes here to make this an implementation detail change with no observable difference.
PiperOrigin-RevId: 928654085
The purpose of this change is guarantee the JIT can perform always bimorphization is possible in all paths which are handling a CodedOutputStream, which can in turn unlock significant other optimization wins.
PiperOrigin-RevId: 928539681
It used to be that the UnsafeProcessor was named that because it used sun.misc.Unsafe, and SafeProcessor did not.
However, over time as optimizations shifted the dominant difference was actually that UnsafeProcessor was targeting serverside and SafeProcess targeted Android (even though we otherwise heavily use sun.misc.Unsafe on Android).
Since we no longer sun.misc.Unsafe in either path now, its time to change the names.
PiperOrigin-RevId: 927336258
Lazy field should only be applicable to full message, whose `mergeFrom(Message)` supports merging from a different message but with the same descriptor. Therefore, this change abandons the default instance check in the lazy field impl.
Also note that InternalLazyField currently belongs to the lite runtime, so direct access to descriptors is infeasible, thus depending on the mergeFrom overrides to invoke the full merging method.
PiperOrigin-RevId: 926112976
We have to break up the descriptors into multiple strings because of the 64k cap on string literals and then join them back up at runtime. This change we avoid building the intermediate joined string before extracting it back as byte[], and instead pre-allocate the total byte[] array first and turns each 64kb segment to byte[] and uses System.arraycopy to copy them directly.
This avoids some pointless work, but also removes garbage copies. Note that its a garbage copy that should be escape-analysis by the JVM and so should be very cheap, but still cheaper not to.
PiperOrigin-RevId: 925969602
