The pyi generator now includes Kythe annotations for:
* Extension field constants (e.g., `EXTENSION_FIELD`).
* Field number constants (e.g., `STRING_FIELD_FIELD_NUMBER`).
* `Create` methods in generated Stubby client classes.
These annotations allow the Python indexer to link these generated symbols back to their definitions in the `.proto` files.
PiperOrigin-RevId: 852971617
This is only adding the annotation on cases that match standard idioms that return values can be ignored: like collections add/addAll/remove and Builders returning `this`
PiperOrigin-RevId: 852435246
On map fields the Serialize and SerializedLength create temporary instances of synthetic map entries and .build() on them: this change switches it to instead use buildPartial() in these cases.
This is a semantics observable change in a corner case, but in a bugfix direction: JavaProto's normal semantics is to only check required fields at .build() time, which allows for buildPartial() and then serialize if you want to intentionally serialize with required fields missing or as a microoptimization skip checking for required fields being set.
Only in the case of message-typed-values of map fields serialize may fail with missing required fields: it will happen if the user does buildPartial(), sets it on a map, and then serializes, since the build() on the synthetic map entry will recheck for required fields. After this change the behavior on that shape of flow is consistent with our standard serialize.
PiperOrigin-RevId: 849485221
After the prior cleanup, this is the last use of the base class left and we are unlikely to reuse the base class for another different type, so we can merge it in.
PiperOrigin-RevId: 848198069
The primary reason to do this is to reduce allocations, but it may give epsilon better runtime characteristics by hitting == paths more often compared to .equals() paths.
Some of the hottest cases of these are actually already handled in layers above, but there's many paths where this is reachable with 0-length strings where we currently will perform an allocation.
PiperOrigin-RevId: 845840649
Today when we let it ride it ends up allocating a new String() each timu which is equals but != the "" interned value.
By adding the trivial check up front we can avoid an arbitrary number of separate java.lang.String objects being created.
PiperOrigin-RevId: 845499246
The behavior before this change was that when a corrupted lazy extension was seen, the first time parse is reached it actually discarded and 0-length byte is stored.
This can led to exceptions in a case of
- Parse some parent message with a messageset with an extension that has malformed bytes
- Call getSerializedLength() on parent, which will see the lazy fields byte length and cache it
- Call .get() on the bad extension, which sees the corruption, this resets only the inner cached size to -1 not any parents
- Serialize the message, serialized length doesn't match the cached serialized length which will throw that the wrong number of bytes were written.
After this change, we instead round-trip the original bytes in the face of corrupted data. This is consistent with cases like wrong-tag cases where we stuff the data into unknown fields instead and don't discard it.
PiperOrigin-RevId: 844814654
This was briefly used by generated code internally as part of the experimental parser/runtime work. This method was never used by protoc generated code otherwise.
We don't consider removal of protected methods for the gencode base classes to be a breaking change if they were never called by any released generated code, as it would only break someone who subclassed GeneratedMessage by hand and wrote code who called this which is not supported. In this case there's essentially no reason for someone to have been interested in calling this anyway.
PiperOrigin-RevId: 843318803
JavaProto is already robust in the absence of sun.misc.Unsafe, and checks for its existence via reflection to use as a performance optimization.
As part of the planned removal of sun.misc.Unsafe, a future JDK will have Unsafe available but most of the methods will throw by default, which would break JavaProto. This creates an odd situation that if it was removed we wouldn't break, but the intermediate turndown state does break us. By adding a usage of one method to see if it throws an UnsupportedOperationException, we avoid breaking users if they use Protobuf in this intermediate turndown state of the API (which will become the default state in a future JDK release).
It is still TBD whether a future release of Protobuf may stop using sun.misc.Unsafe always, or if we may keep this code alive for users running old JDKs; that detail is still TBD depending on performance implications.
https://github.com/protocolbuffers/protobuf/issues/20760
PiperOrigin-RevId: 836715451
In the past, as part of supporting Ropes, our Utf8 path had to support verifying if some byte[] were individually invalid Utf8 but actually were valid Utf8 once concatenated, which required the validation library to have complex handling for propagating and checking with partial state.
In a prior change Ropes were simplified to no longer need that behavior and the 'partial' behavior became vestigial internal detail of the Utf8 library. This change is an internal no-op cleanup.
There are potentially more followup cleanup opportunities, this change is oriented towards doing nearly the minimum to replace the int return values with bools, and dropping `int state` input (which is at head always passed as state=COMPLETE). This involved inlining some methods.
PiperOrigin-RevId: 836703657
Rename SafeDirectNioEncoder to DirectNioEncoder now that there is no unsafe one.
This encoding path is rarely used (only used on direct Nio buffers, not heap ones). sun.misc.Unsafe is slated for removal, and a quick check of benchmarks shows that the safe version is actually only very slightly slower today, so we will just remove the unsafe path here.
https://github.com/protocolbuffers/protobuf/issues/20760
PiperOrigin-RevId: 833892566
This type was only used with non-array backed ByteBuffers, which in practice does not happen. Any fringe usages which may be invisible to our tracing (including any usages in open source) will not break with this change, which is done at the expense of a copy of the bytes on the way in.
PiperOrigin-RevId: 829088261
In practice the way we used this was:
- The UTF8 util refused to handle strings with dangling UTF16 surrogates
- The CodedOutputStream above always would have to catch the exception and fall back to the 'naive' implementation.
Broadly we're moving serverside things onto the 'naive implementation' which does replacement characters inside the Utf8 utility anyway, which makes those paths never throw this exception. So instead reshape this so the Utf8 utility does replacement character behavior, and the layer above gets that behavior already
This is nearly entirely a behavior no-op: the main thing is losing a log-warning in one path that warns that the unpaired surrogate situation was happening.
Note that replacement characters on encode of malformed Utf16 is just a pretty safe behavior: it can never occur when parsing wire format, instead it can only happen from someone doing something odd with their Java application code and there's realistically no real signal in the data being lost. That contrasts with malformed Utf8 from the wire at parse, where someone may realistically have encoding type confusion and a parse>serialize round trip silently leading to replacement characters is potentially dangerous for data loss.
PiperOrigin-RevId: 828473999
