Avoid generating double-checked locking inside every generated message class's
dynamicMethod(GET_PARSER) by centralizing the parser instantiation and caching
in a thread-safe map inside GeneratedMessageLite.
This saves substantial DEX and ODEX size for apps with many protobuf classes.
Note: This change requires the generated code to run against a runtime of at
least the same version. Mixing new generated code with an older runtime version
is not supported (see https://protobuf.dev/support/cross-version-runtime-guarantee/).
PiperOrigin-RevId: 944139195
## Summary
`mergeDelimitedFrom` passes the return value of `readRawVarint32` directly to `LimitedInputStream` without checking for negative values. For 5-byte varints encoding values >= 2^31, `readRawVarint32` returns a negative int (e.g., `\x80\x80\x80\x80\x08` decodes to -2147483648).
`LimitedInputStream` with a negative limit immediately returns EOF (`if (limit <= 0) return -1`), causing the parser to produce a default/empty message and return `true` (success). The actual message body bytes remain unconsumed in the stream, permanently desynchronizing subsequent `parseDelimitedFrom` calls.
## Fix
Add `if (size < 0) throw InvalidProtocolBufferException.negativeSize()` before constructing `LimitedInputStream`.
## Test plan
- [x] Verified with PoC: 5-byte varint prefix encoding 2^31 previously produced empty message + stream desync; after fix, `negativeSize` exception thrown
- [ ] Existing unit tests pass
Closes#28097
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/28097 from TristanInSec:fix/merge-delimited-negative-size 7ccb18676e
PiperOrigin-RevId: 937547094
- Apply a fast path to Uint64 parse where we try to parse it directly before falling back to the BigDecimal path. This optimization is preexisting on int64 and int32 and uint32 but was missed on Uint64
- Avoid doing out.append(value.subSequence(start, end)) which forces an intermediate copy the string, instead use the out.append(value, start, end)
PiperOrigin-RevId: 937290790
Because JsonFormat uses GSON to parse JSON it has a variety of
leniencies, e.g. allowing keys and values to skip quotes:
```
{hello: world}
```
These behaviours are not part of the ProtoJSON format but they are
behaviours that JsonFormat's parser has. It's possible someone in the
wild depends upon these unknowingly. This adds explicit tests for these
behaviours to make sure that they are not accidentally lost in future.
Closes#27622
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/27622 from petedmarsh:json-format-lienent-tests edebd543cd
PiperOrigin-RevId: 933849754
Note that as this case is only reachable for known groups, its not a usable primitive for fully unconstrained recursion, since even if a recursive schema did use a group it would call back to a message between, meaning that even in such an obscure case this would only unlock reaching depth 200 not unconstrained.
PiperOrigin-RevId: 930082333
These interfaces were part of the experimental runtime, and before this change have exactly one concrete implementation each. This CL simply deletes the interfaces and inlines the corresponding concrete type wherever they were named.
PiperOrigin-RevId: 930019405
The previous change looked to get down to exactly 2 CodedOutputStream concrete classes: unfortunately some users are relying on the incidental-but-reliable behavior that CodedOutputStreams with ByteBuffers do immediately push the content without a flush() but that the position update does not apply until you call flush(). This oddity has been incidentally/accidentally true for both Heap and Direct ByteBuffer cases even though they didn't use a common codepath.
To move incrementally forward, we can merge in HeapNio into ArrayEncoder without touching DirectNio. For now this still leaves 3 totalconcrete classes, but it will achieve the 'exactly 2 concrete classes loaded' for any server if they never reach DirectNioEncoder or never reach the OutputStreamEncoder.
PiperOrigin-RevId: 929337270
This requires sprinkling some casts around locally, mainly surrounding the mismatch of apis that return `MessageLite` which on LiteRuntime is guaranteed to be `GeneratedMessageLite`, but statically thats not known because Full messages also implement it.
PiperOrigin-RevId: 928716180
This still retains the reflection path for these types so that it will continue to handle DynamicMessage printing correctly.
PiperOrigin-RevId: 928689274
- Use a lookup table for the replacement characters in the ascii range (which is all of the characters that gson escapes except for exactly 2).
- Stream the escapes out ourselves instead of going through gson.toJson(val.toString()) which goes through a lot of heavy gson machinery and makes several copies of the string.
We exactly match the choices that gson made for escapes here to make this an implementation detail change with no observable difference.
PiperOrigin-RevId: 928654085
The purpose of this change is guarantee the JIT can perform always bimorphization is possible in all paths which are handling a CodedOutputStream, which can in turn unlock significant other optimization wins.
PiperOrigin-RevId: 928539681
It used to be that the UnsafeProcessor was named that because it used sun.misc.Unsafe, and SafeProcessor did not.
However, over time as optimizations shifted the dominant difference was actually that UnsafeProcessor was targeting serverside and SafeProcess targeted Android (even though we otherwise heavily use sun.misc.Unsafe on Android).
Since we no longer sun.misc.Unsafe in either path now, its time to change the names.
PiperOrigin-RevId: 927336258
Lazy field should only be applicable to full message, whose `mergeFrom(Message)` supports merging from a different message but with the same descriptor. Therefore, this change abandons the default instance check in the lazy field impl.
Also note that InternalLazyField currently belongs to the lite runtime, so direct access to descriptors is infeasible, thus depending on the mergeFrom overrides to invoke the full merging method.
PiperOrigin-RevId: 926112976
We have to break up the descriptors into multiple strings because of the 64k cap on string literals and then join them back up at runtime. This change we avoid building the intermediate joined string before extracting it back as byte[], and instead pre-allocate the total byte[] array first and turns each 64kb segment to byte[] and uses System.arraycopy to copy them directly.
This avoids some pointless work, but also removes garbage copies. Note that its a garbage copy that should be escape-analysis by the JVM and so should be very cheap, but still cheaper not to.
PiperOrigin-RevId: 925969602
By outlining the exception handling path, this notably shrinks this method to below 35 bytes in .class bytecode representation, which is one of the magic arbitrary limits that influences JIT inlining behaviors.
PiperOrigin-RevId: 925530732
In `AbstractMessageLite.Builder.mergeFrom(MessageLite)`, it either goes into (full runtime) `AbstractMessage` which supports merge from a different instance but the same *descriptor* type, or into the lite runtime path which should not support it in the absence of descriptors .e.g in case of `DynamicMessage`.
PiperOrigin-RevId: 924941126
Today, equals() causes all lazy extensions to be parsed before the equality check starts. After this change, equals() will try to avoid hitting lazy parsing if possible
Notably:
- When the count of extensions don't match, or if "which extensions are set" don't match, we can return false before looking at any extension values.
- When actually comparing extensions, it goes iteratively which means it'll only parse the extensions as it goes; if it finds any one ext which is not-equal it will will bail out and not parse any of the rest.
Note that when the case where two messages are equal, this unavoidably still cause all extensions to parse since it has to examine all fields.
To achieve this does require adding a protected `extensionsEquals()` method for the gencode to call, since the current base class API isn't otherwise powerful enough to do so.
PiperOrigin-RevId: 924903438
This only changes the sort order in an obscure edge case of map-keys which sort differently in Utf8 and Utf16, meaning a mix of certain characters that are "very high code point but still single char" and "utf16 surrogate pairs".
PiperOrigin-RevId: 922768203
JsonFormat used 'reader.setLenient(false)' to try to get spec JSON behavior out of GSON. However, this mode was still not actually spec JSON. In ~2024 GSON deprecated setLenient(), made that behavior LEGACY_STRICT and added a new level STRICT which is intended to be spec.
Switching unilaterally would be a breaking change for Protobuf: this change just adds the option for users to ask us to use GSON in strict mode. At a later date we will consider flipping the default of this (letting people opt into LEGACY_STRICT if they want to).
PiperOrigin-RevId: 921432331
