Commit graph

1576 commits

Author SHA1 Message Date
Protobuf Team Bot
886d0b8f79 Use the ROOT locale when doing toLowerCase on names.
PiperOrigin-RevId: 815831657
2025-10-06 12:14:00 -07:00
Protobuf Team Bot
3354a3c7ed Fix bugs in Java Large Enums that have aliased values.
The old implementation tried to emit only the canonical values in the 'chunked' ranges and then emit aliases separately at the top level, which led to a number of problems including:

- On reflective lookup, it indexed into the values() array based on the all-value-index but that array only contained canonical-values. This is fixed to match the non-Large implementation where the public values() still returns only canonicals, but there's a second private array which contains all values for index-based indexing.

- The static init dependency graph potentially had cycles which could leave values as null if they were touched in the 'wrong' order.

This changes switches it to instead chunk up all enum values based on the original index range, unrelated to if they the value is an aliases or not. The non-aliases remain constructed as normal, the aliases are declared in their index-order but name the corresponding canonical value directly by its chunk (eg the SomeEnum2 chunk could have something like `public static final SomeEnum X_ALIAS = SomeEnum0.X`).

Since the definition of alias is "the lowest index of a given value is the canonical one and any higher index with same value are aliases", this should avoid init cycles since the aliases will only point to some chunk which is <= its own chunk.

PiperOrigin-RevId: 815717989
2025-10-06 07:34:52 -07:00
Mike Kruskal
dc08077d6b Fix a bug calculating the file name in the absense of directories.
PiperOrigin-RevId: 814799893
2025-10-03 13:04:38 -07:00
Mike Kruskal
cdb39db306 Clarify the public APIs of GeneratorNames helpers.
* FileJavaPackage - matches C++, no ambiguity
* FileClassName - matches C++ and our new style
* BytecodeClassName - returns the fully qualified java names using '$' separators for inner classes.  C++ will be similarly renamed from JavaClassName
* QualifiedClassName - returns the fully qualified java names using '.' sepators for inner classes.  C++ will be similarly renamed from ClassName

PiperOrigin-RevId: 814414729
2025-10-02 15:47:41 -07:00
Mike Kruskal
6fe1497e79 Internal changes
PiperOrigin-RevId: 814384166
2025-10-02 14:23:13 -07:00
Mike Kruskal
7a7f276dce Expose helpers to predict generated class names in java.
PiperOrigin-RevId: 814370138
2025-10-02 13:50:03 -07:00
honglooker
6d00d87830 Updating version.json and repo version numbers to: 34.0-dev (#23738)
NOTE: This should be reviewed and imported via Copybara per the normal PR review process.

Closes #23738

PiperOrigin-RevId: 814297739
2025-10-02 10:51:16 -07:00
Mark Hansen
dead64c507 Remove unused method com.google.protobuf.Protobuf.getTotalSchemaSize()
There appear to be no callers of this.

I'm removing this in preparation to try to offer a more performant
java.lang.ClassValue replacement for the ConcurrentHashMap.

Removing this method means I don't have to reimplement it for ClassValue.

PiperOrigin-RevId: 814272309
2025-10-02 09:49:23 -07:00
Protobuf Team Bot
c497523cb4 Add annotation to Hide Kotlin DSL factory functions from Objective-C
Introduces a new annotation that can be used specifically for hiding Kotlin generated functions that cause name collisions in Objective-C.

PiperOrigin-RevId: 814251585
2025-10-02 08:57:08 -07:00
Protobuf Team Bot
6245b7291c Use the simplest possible implementation of encodeUtf8 instead of a fancy one that uses sun.misc.Unsafe.
Empirically in 2025 this appears to be faster than our implementation that tries to hand roll this and avoid a string copy and use sun.misc.Unsafe (and much faster in the case of ascii strings).

The JDK implementation is able to leverage the internal representation of the strings when encoding, including that it can have a very fast path for "internal representation is Latin1 and the string happens to be ascii, then it is already utf8". The exposed Java API unfortunately demands us have a temporary copied array here, but it empirically doesn't show up as a regression to have this shortlived object and copy.

This delta is only changing UnsafeProcessor path which is currently held back from Android, so its OK that this was only benchmarked on server and we don't need to worry about a regression on Android at this time.

PiperOrigin-RevId: 813904984
2025-10-01 14:09:36 -07:00
Protobuf Team Bot
3ce5838cff Change back to using String instead of Class as the argument on the Poison Pill.
The reason to change this back is that one purpose of this method to provide a good message when e.g. someone tries to use 4.33.0 gencode against 4.30 runtime. The 4.30 runtime only has the `String` argument method, so by having new gencode try to call the `Class` one it will result in ABI behavior of a NoSuchMethodError instead of getting the clean message when that 'bad direction skew' happens.

This partial-rollback tries to retain the benefit of avoiding the small cost of the .class.getName() call by having new gencode simply emit the name as a string literal like `"MyMessageProto"` instead of  `MyMessageProto.class.getName()` when calling the poison pill.

Because we didn't do any open source releases with this change quite yet, we can remove the `Class` overload here instead of it having to linger forever.

PiperOrigin-RevId: 813433985
2025-09-30 14:42:35 -07:00
Protobuf Team Bot
7b0bee3fb5 Slightly relax Java Poison Pill on prerelease versions (-rc1, -dev, etc).
Before this, the poison pill intended to require exact match including suffix if there were suffixes on either gencode or runtime.

After this change:
- If gencode is suffixed, it must be exact match to runtime (including suffix).

- If runtime is suffixed and gencode is not suffixed, the gencode must be a strictly lower major.minor.point (4.32.1-rc1 runtime must be on <= 4.32.0 gencode). We log a warning once if this allowed inequality state happens.

This change also fixes a bug where a lower minor version was allowed if it did have the same suffix (4.33.0-rc1 runtime accidentally accepted 4.32.0-rc1 gencode while still rejecting 4.32.0).

PiperOrigin-RevId: 813286207
2025-09-30 08:56:06 -07:00
Protobuf Team Bot
be93dbcf73 Delete an unused internal field.
PiperOrigin-RevId: 811032599
2025-09-24 14:25:28 -07:00
Protobuf Team Bot
65896c1f4b Change ArrayIndexOutOfBounds message to exclude information about char and index.
The information doesn't seem very useful for debugging, and it isn't possible to have the char+index be accurate while allowing for fast-failure paths that can know an input string won't fit but can't possibly know the exact character that is at the boundary without doing a lot of expensive work instead.

The unit test already reflected the oddity that what it printed wasn't consistent/predictable depending on if it was going to hit a fast-path or not, and that test is fixed to shown the consistency after this change.

PiperOrigin-RevId: 810954092
2025-09-24 11:07:38 -07:00
Protobuf Team Bot
810272f3c4 Avoid boxing/unboxing varint, fixed32, and fixed64 fields in UnknownFieldSet.Field
PiperOrigin-RevId: 810678393
2025-09-23 20:06:59 -07:00
Protobuf Team Bot
9e05ca5325 Replace NumberGetter with ToIntFunction
PiperOrigin-RevId: 809025295
2025-09-19 07:16:57 -07:00
Protobuf Team Bot
eb4613c9f0 Avoid more array wrapping as list
PiperOrigin-RevId: 808792922
2025-09-18 16:55:59 -07:00
Protobuf Team Bot
badaf41ecf Readd new*List() methods on GeneratedMessageV3.
This will further restore compatibility on 4.x runtime with gencode to <= 3.21

This is a partial rollback from 787447430f (diff-94b93e396de2304f28fafc97968031b4e2a3a598c7efdab8b252a156ac7c57dbL451-L452) with the methods readded onto GeneratedMessageV3.

PiperOrigin-RevId: 807319417
2025-09-15 11:41:07 -07:00
Protobuf Team Bot
c518f2564e Add Values.of(Map<String, Value> values).
Also update Javadoc to include {@link} syntax and fix typo.

PiperOrigin-RevId: 806318745
2025-09-12 09:43:30 -07:00
Mike Kruskal
8d51e346ac Fix handling of optional dependencies in java generator.
Java reflection is used to fail smoothly when the extensions aren't
found in the classpath.

PiperOrigin-RevId: 805477788
2025-09-10 12:58:41 -07:00
Protobuf Team Bot
4edff4a971 internal change
PiperOrigin-RevId: 805423369
2025-09-10 10:42:54 -07:00
Protobuf Team Bot
ea33ae8615 Restore ABI compatibility for extension methods which was previously (knowingly) broken with 4.x: 94a2a44851
Having overloads of the subtypes is unnecessary for source compatibility (where javac will resolve the type to the base type method), but removal of them an ABI break if .class files compiled against PBJ 3.x are used with PBJ 4.x.

PiperOrigin-RevId: 805035553
2025-09-09 13:16:06 -07:00
Mike Kruskal
df7f12a35b Introduce edition 2024 transform
This just sets up the transform and implements one piece of it.  Specifically, it updates the edition to 2024 and migrates feature dependencies to `import option`.  Further transforms will be added in later changes.

PiperOrigin-RevId: 803656952
2025-09-05 17:16:00 -07:00
Protobuf Team Bot
cb873c8987 Fold hasInferredLegacyProtoFeatures() into inferLegacyProtoFeatures()
The logic is duplicated across both. Rather than have both, we just optimize `inferLegacyProtoFeatures()` to only return a new instance when the features would be set. This ensures there is only one source of truth and we do not need to worry about the two functions coming out of sync.

PiperOrigin-RevId: 803146647
2025-09-04 13:19:24 -07:00
Protobuf Team Bot
1932ccbd74 Cut guava dep from the Java Protobuf-Util package.
In a subsequent change we should be able to cut the Guava dep from our Maven package.

https://github.com/protocolbuffers/protobuf/issues/21173

PiperOrigin-RevId: 803086278
2025-09-04 10:40:18 -07:00
Protobuf Team Bot
e05db5cfe9 Optimize redaction state calculation
Memoizes the state, shifts to using a single `volatile` field, and updates `TextFormat` to look at the state directly.

PiperOrigin-RevId: 803053745
2025-09-04 09:17:57 -07:00
Protobuf Team Bot
c47f34ca1c Suppress some SelfAssertion warnings on a couple tests.
PiperOrigin-RevId: 803033489
2025-09-04 08:17:31 -07:00
Protobuf Team Bot
f978ec2040 Add isPlaceholder() accessors to file, message, and enum descriptors
This aligns with C++ and Go.

PiperOrigin-RevId: 803010915
2025-09-04 07:16:32 -07:00
Mike Kruskal
2cebd6c187 Internal changes
PiperOrigin-RevId: 802642771
2025-09-03 11:48:35 -07:00
Protobuf Team Bot
ed1496fcde Refactor field descriptors by using a single field for the resolved type_name reference
This is just a cleanup which more closely enforces the fact that a field descriptor can either reference a message type or enum type, never both.

PiperOrigin-RevId: 802550883
2025-09-03 07:50:35 -07:00
Protobuf Team Bot
b9066bd74c Refactor descriptors to avoid duplicate fields
This is just a cleanup to deduplicate the amount of redundant information, making it easier to refactor going forward.

PiperOrigin-RevId: 802540672
2025-09-03 07:16:25 -07:00
Protobuf Team Bot
41a8d46cdb Automated rollback of commit bd440bf93c.
PiperOrigin-RevId: 800651285
2025-08-28 16:09:52 -07:00
Protobuf Team Bot
bd440bf93c Restore ABI compatibility for extension methods which was previously (knowingly) broken with 4.x:
94a2a44851

Having overloads of the subtypes is unnecessary for source compatibility (where javac will resolve the type to the base type method), but removal of them an ABI break if .class files compiled against PBJ 3.x are used with PBJ 4.x.

PiperOrigin-RevId: 800614750
2025-08-28 14:33:34 -07:00
Protobuf Team Bot
894039e751 Automated rollback of commit 493f2fa8a0.
PiperOrigin-RevId: 799769468
2025-08-26 16:44:05 -07:00
Protobuf Team Bot
493f2fa8a0 Automated rollback of commit b68b673177.
PiperOrigin-RevId: 799763570
2025-08-26 16:28:51 -07:00
Protobuf Team Bot
b68b673177 Improve Java gencode static initialization to avoid unnecessary temporaries
PiperOrigin-RevId: 799633296
2025-08-26 11:04:08 -07:00
Protobuf Team Bot
ad4a58499d Optimization: avoid calling getDescriptor() twice in a row.
PiperOrigin-RevId: 797042579
2025-08-19 15:27:28 -07:00
Protobuf Team Bot
5768acddf2 Remove protobuf-util usages of guava except annotations.
The annotations will be handled in a subsequent change.

The change to use java.lang.String.split() instead of Guava's Splitter does introduce behavior changes in the case of handling of degenerate/malformed FieldMask paths (paths like `a///`). The handling of malformed paths shape are already arbitrary and inconsistent between the different methods in this file, and callers should not construct such paths. In a future change we may consider validating and throwing an IllegalArgumentException on such malformed paths more explicitly.

https://github.com/protocolbuffers/protobuf/issues/21173

PiperOrigin-RevId: 795126095
2025-08-14 11:34:28 -07:00
Protobuf Team Bot
d2d2115e70 Fix DynamicMessage.Builder.setField to clear repeated fields when appropriate
When setting a repeated field to an empty list, clear the corresponding entry in the `FieldSet`. Otherwise the resulting message wouldn't be treated as equal to an empty message, even though it is.

Generated messages already have the same behavior.

PiperOrigin-RevId: 795044555
2025-08-14 08:13:08 -07:00
Sandy Zhang
781d78cb4a Strip extensions from option imports that are known in CollectExtensions due to polluted pool from protoc parse when used with protoc full + java built in generator.
Fixes https://github.com/protocolbuffers/protobuf/issues/22918

PiperOrigin-RevId: 794764601
2025-08-13 15:40:43 -07:00
Joshua Haberman
8aab380912 Moved assumeLiteRuntime to Android.java, which avoids pulling in MessageSchema.
PiperOrigin-RevId: 793795963
2025-08-11 14:23:05 -07:00
Joshua Haberman
e729bc68e0 Removed dependency CodedOutputStream -> LazyFieldLite.
This is in line with the larger goal of making `CodedOutputStream` not depend on specific message representations, which `LazyFieldLite` is.

PiperOrigin-RevId: 793672999
2025-08-11 09:13:48 -07:00
Protobuf Team Bot
418399e528 Use number instead of recomputing it.
PiperOrigin-RevId: 792327771
2025-08-07 15:19:33 -07:00
Joshua Haberman
b82fe9843d Internal change.
PiperOrigin-RevId: 792302467
2025-08-07 14:13:13 -07:00
Kurt Alfred Kluever
1ae6b6bb9c Validate Duration instances produced by the static factories.
PiperOrigin-RevId: 792163897
2025-08-07 08:11:28 -07:00
Sandy Zhang
fee15f80c7 Migrate unittest_features.proto to edition 2024.
Updates tests to avoid setting java features on this proto, since its used frequently in custom pools in unit tests (descriptor_unittest.cc, command_line_interface_unittest.cc) which would require adding the addl. dep in a bunch of places.

PiperOrigin-RevId: 791866799
2025-08-06 15:31:45 -07:00
Sandy Zhang
370758725d Manually migrate top-level protobuf unittest protos from edition 2023 to edition 2024.
PiperOrigin-RevId: 791221060
2025-08-05 08:16:42 -07:00
Protobuf Team Bot
7c51e5b582 Restore compatibility of runtime with pre-3.22.x gencode impacted by CVE-2022-3171
Generated code from this range is covered by CVE-2022-3171 and potentially vulnerable to a Denial of Service issue.

JavaProto 4.x previously dropped compatibility with the potentially vulnerable generated code, having the behavior of:

* The vulnerable generated code was source-incompatible with new runtime (would not compile when built from source)

* The vulnerable generated code was ABI-incompatible with new runtime (when using a .class file compiled against old runtime, a NoSuchMethodException would be thrown at parse time).

After this change, instead:

* The vulnerable generated code is now source-compatible (will compile).
The first time each potentially vulnerable type is parsed,  an error message will be logged noting that potentially vulnerable generated code is in use and the name of the corresponding type.

* Environment variables may be set to either throw an exception instead (-Dcom.google.protobuf.error_on_unsafe_pre22_gencode) or to entirely silence the logged messages (-Dcom.google.protobuf.use_unsafe_pre22_gencode)
This change was made based on community feedback regarding the difficulty in identifying and quickly remediating stale gencode in their transitive dependencies weighed against a careful evaluation of the realistic risk exposure of DoS (with no risk of other concerns including information leak or RCE).

We strongly recommend that any users who observe the log messages to regenerate the corresponding code with a newer protoc. We recommend that any security-conscious services opt into error_on_unsafe_pre22_gencode to preclude any risk of a Denial of Service surface area being exposed.

A future release may flip the default behavior to error by default as a measure to further help the ecosystem avoid the Denial of Service risks.

PiperOrigin-RevId: 790798112
2025-08-04 10:20:16 -07:00
Joshua Haberman
273dbab6b0 Internal change.
PiperOrigin-RevId: 789367726
2025-07-31 09:43:43 -07:00
Joshua Haberman
ed94600681 Removed dependency from coded streams to the experimental writer/reader interfaces.
PiperOrigin-RevId: 789340420
2025-07-31 08:12:30 -07:00