Lazy field should only be applicable to full message, whose `mergeFrom(Message)` supports merging from a different message but with the same descriptor. Therefore, this change abandons the default instance check in the lazy field impl.
Also note that InternalLazyField currently belongs to the lite runtime, so direct access to descriptors is infeasible, thus depending on the mergeFrom overrides to invoke the full merging method.
PiperOrigin-RevId: 926112976
We have to break up the descriptors into multiple strings because of the 64k cap on string literals and then join them back up at runtime. This change we avoid building the intermediate joined string before extracting it back as byte[], and instead pre-allocate the total byte[] array first and turns each 64kb segment to byte[] and uses System.arraycopy to copy them directly.
This avoids some pointless work, but also removes garbage copies. Note that its a garbage copy that should be escape-analysis by the JVM and so should be very cheap, but still cheaper not to.
PiperOrigin-RevId: 925969602
By outlining the exception handling path, this notably shrinks this method to below 35 bytes in .class bytecode representation, which is one of the magic arbitrary limits that influences JIT inlining behaviors.
PiperOrigin-RevId: 925530732
In `AbstractMessageLite.Builder.mergeFrom(MessageLite)`, it either goes into (full runtime) `AbstractMessage` which supports merge from a different instance but the same *descriptor* type, or into the lite runtime path which should not support it in the absence of descriptors .e.g in case of `DynamicMessage`.
PiperOrigin-RevId: 924941126
Today, equals() causes all lazy extensions to be parsed before the equality check starts. After this change, equals() will try to avoid hitting lazy parsing if possible
Notably:
- When the count of extensions don't match, or if "which extensions are set" don't match, we can return false before looking at any extension values.
- When actually comparing extensions, it goes iteratively which means it'll only parse the extensions as it goes; if it finds any one ext which is not-equal it will will bail out and not parse any of the rest.
Note that when the case where two messages are equal, this unavoidably still cause all extensions to parse since it has to examine all fields.
To achieve this does require adding a protected `extensionsEquals()` method for the gencode to call, since the current base class API isn't otherwise powerful enough to do so.
PiperOrigin-RevId: 924903438
This only changes the sort order in an obscure edge case of map-keys which sort differently in Utf8 and Utf16, meaning a mix of certain characters that are "very high code point but still single char" and "utf16 surrogate pairs".
PiperOrigin-RevId: 922768203
JsonFormat used 'reader.setLenient(false)' to try to get spec JSON behavior out of GSON. However, this mode was still not actually spec JSON. In ~2024 GSON deprecated setLenient(), made that behavior LEGACY_STRICT and added a new level STRICT which is intended to be spec.
Switching unilaterally would be a breaking change for Protobuf: this change just adds the option for users to ask us to use GSON in strict mode. At a later date we will consider flipping the default of this (letting people opt into LEGACY_STRICT if they want to).
PiperOrigin-RevId: 921432331
This exception is put at the spot where we will typically first notice that we cannot continue due to being on Lite gencode flows and that sun.misc.Unsafe is unavailable.
Add corresponding note on lite.md that it requires sun.misc.Unsafe
PiperOrigin-RevId: 919813377
These paths are 100% unreachable today via our public APIs (they're only reachable by our tests which pierce-the-veil of the package-protected api). The only entry point is via GeneratedMessageLite, and this code already checks for GeneratedMessageLite types and always uses the Lite types.
This behavior means that even if someone in OSS uses Lite gencode on Full runtime this path is never reachable today.
PiperOrigin-RevId: 917409414
The only field order used in practice is ASCENDING. DESCENDING was added for the experimental JavaProto runtime which is now defunct.
PiperOrigin-RevId: 917271023
When toBuilder() is called on an immutable ExtendableMessage whose extensions field is the singleton FieldSet.emptySet(), generated code calls ExtendableBuilder.mergeExtensionFields(other).
Because other.extensions is FieldSet.emptySet() (which is not null), mergeExtensionFields previously invoked ensureExtensionsIsMutable(), which unnecessarily allocated a brand new temporary FieldSet.Builder and SmallSortedMap. Later, when build() executed, FieldSet.Builder.buildImpl recognized that the builder's map was empty and correctly returned the singleton FieldSet.emptySet().
By checking !other.extensions.isEmpty() before mutating the builder, we completely eliminate these wasted temporary allocations.
PiperOrigin-RevId: 916174058
The refactor shrinks the method's bytecode to be below 35 bytes instead of above, and 35 bytes is one of the relevant default threshold conditions for inlining.
The theory behind this change is to improve a very hot case of:
- gencode always calls readMessage with the concrete builder types of fields
- That means readMessage itself is called with all reachable Builder types
- If readMessage is not inlined, it will be highly megamorphic (= too many different types for the JIT to try to monomorphize it), which makes it true dynamic dispatch at runtime. But if it is inlined instead, then it will trivially dispatch to one specific Builder type instead.
It isn't so easy to benchmark how much this will impact the fleet in reality, but since this change is otherwise very benign we can just make the change and then post-hoc see if it looks like it improved things.
PiperOrigin-RevId: 915625779
The last usages of UnsafeUtil from the UnsafeProcessor were removed in a previous change.
The name 'UnsafeProcessor' was already a bit stale, and is now very stale. But not renaming it now to reduce churn if this has to be rolled back. After this sticks, we should come back and do a pass to rename them to 'StandardProcessor' versus 'AndroidOnDeviceProcessor'
PiperOrigin-RevId: 914424161
This use was somewhat buggy: it would have broken if used on a non-Direct Buffer, or if it was executed in a context where sun.misc.Unsafe was not available.
This path is empirically extremely rarely reached, and so we will just use the 'natural' API as part of getting off of the terminally deprecated sun.misc.Unsafe
PiperOrigin-RevId: 914423267
This change effectively has the runtime behavior of hardcoding HAS_UNSAFE_ARRAY_OPERATIONS to false, which was measured as a net performance improvement after the recent loop unrolling changes.
This is one of the last steps towards no longer using the terminally deprecated sun.misc.Unsafe from JavaFull.
PiperOrigin-RevId: 914398839
This change is in preparation for a subsequent change which will disable sun.misc.Unsafe usage. Simply turning off s.m.Unsafe would result in some small but measurably regressions; unrolling these loops by contrast provides a significant speedup, which should more than outweighs any regression we will get from the disabling Unsafe. This will make it easier for us to roll out the change to stop using Unsafe without problems from regression.
PiperOrigin-RevId: 913906295
Right now we delegate negative int32s to the int64 encoder, which has to still figure out how many bytes its going to take, which is wasted work since we already checked the top bit and know it will be exactly 10 byte varint.
PiperOrigin-RevId: 913824216
This was changed in 2020 to an unrolled loop with Unsafe but was rolled back after external reports that it was a regression (https://github.com/protocolbuffers/protobuf/issues/6977). However, not only has JIT behavior evolved in that amount of time, the main external claim appears to have been that the `int position = this.position;` local variable optimization was lost at that time, with one the first comments on that issue noting that restoring that also fixed the regression.
Today, all available benchmarks suggest that this should be good change to make (with the local variable position optimization included), ~twice as fast for direct Uint32 writes usages.
PiperOrigin-RevId: 913780730
This per-processor path static method implementation in a previous change.
This is incremental progress towards migrating off of sun.misc.Unsafe which is terminally deprecated.
PiperOrigin-RevId: 912643167
This path uses sun.misc.Unsafe which is terminally deprecated. Although it is theoretically 'good' that we can optimize a Direct buffer faster than the basic APIs, this is empirically never reached in all of Google's internal use. With sun.misc.Unsafe going away and no easy replacement, this is turning down the corner case optimization.
PiperOrigin-RevId: 912177627
## Summary
`BigDecimal(String)` has O(N²) time complexity for N-digit strings on JDK versions before 18 ([JDK-8291514](https://bugs.openjdk.org/browse/JDK-8291514)). Five JSON parser methods — `parseInt32`, `parseInt64`, `parseUint32`, `parseUint64`, and `parseDouble` — pass user-controlled strings directly to `new BigDecimal()` without length validation.
A single JSON numeric value with 1,000,000 digits takes ~13 seconds to parse on JDK 17. This can be used to DoS any service that parses protobuf JSON messages with numeric fields from untrusted input.
### Benchmark (JDK 17, x86-64 Linux)
| Digits | BigDecimal construction time |
|--------|-----|
| 1,000 | 1.8 ms |
| 10,000 | 6.3 ms |
| 100,000 | 133 ms |
| 1,000,000 | **13.1 seconds** |
### Fix
Added a `parseBigDecimal()` helper that rejects strings longer than 1000 characters before constructing `BigDecimal`. This is generous — valid protobuf numeric values never exceed ~350 characters (Double.MAX_VALUE in non-scientific notation is ~309 digits).
### Affected JDK versions
- JDK 8, 11, 17 (all current LTS releases): **Vulnerable** — no built-in string length limit in BigDecimal
- JDK 18+: JDK itself limits BigDecimal string input to 1100 characters by default (JDK-8291514), but the protobuf-level check is still worthwhile as defense-in-depth
### Test
Added `testParserRejectOverlyLongNumericStrings` covering all 5 affected field types.
Closes#26908
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/26908 from MindflareX:fix/java-bigdecimal-length-check a461d0edc6
PiperOrigin-RevId: 904988136
In the cases where we historically didn't enforce a depth limit (C++ and Python) document that the default behavior having no depth limit is an intentional decision.
PiperOrigin-RevId: 903216142
## Fix unguarded recursion DoS in Protobuf Lite skipField
This PR addresses a missing recursion depth limit check in `ArrayDecoders.skipField()` that causes a `StackOverflowError` and subsequent Denial of Service.
### Vulnerability Context
The initial patch for CVE-2024-7254 correctly added recursion limits to the standard `CodedInputStream` and `decodeUnknownField` in the Lite fast-path array decoders. However, `ArrayDecoders.skipField()` handles the `WIRETYPE_START_GROUP` recursively without checking or incrementing `Registers.recursionDepth`.
This omission exposes any Protobuf Java Lite application that accepts unauthenticated binary input to a remote DoS via simple stack overflow payloads.
**The bug is reachable via two independent paths from Public APIs:**
1. **Legacy Vectors:** `message_set_wire_format`. A payload containing thousands of nested unknown groups inside a MessageSet forces `MessageSetSchema.mergeFrom` to invoke `skipField()`.
2. **Universal Vectors:** Standard Maps! `MessageSchema.decodeMapEntry` handles unrecognized tags inside a map entry by calling `ArrayDecoders.skipField()`. Nesting thousands of groups inside a standard map entry instantly crashes the JVM process.
### Fix
Added `registers.recursionDepth++` and `checkRecursionLimit()` to the `WIRETYPE_START_GROUP` branch in `skipField()`, aligning it securely with `decodeUnknownField()`.
### Issue Link
Additional discussion and reproduction code is documented internally at:
https://issuetracker.google.com/issues/498542124Closes#26670
COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/26670 from VenkatKwest:fix-unguarded-recursion-lite 3660dd6bce
PiperOrigin-RevId: 899005419
