dep-protobuf

mirror of https://github.com/protocolbuffers/protobuf synced 2026-08-26 02:23:14 -04:00

History

Tony Liao 333846acf4 Optimize pure Python parse path for custom JSON enum names. This implementation is similar to our Java idea in cl/949670944, basically the idea is that we will have a local map (dictionary) that will be populated the very first time we try to parse into an enum field. The lifetime of this dictionary will be tied to the Parser instance, similar to the Java implementation. BENCHMARKS (baseline stats are recorded at cl/952170250): Python: ``` [BENCHMARK] ParseJsonDefault: med 10.06 us/op \| p99 11.22 us/op \| mean 10.10 ± 0.20 us/op [BENCHMARK] ParseJsonCustom: med 20.33 us/op \| p99 22.23 us/op \| mean 20.40 ± 0.34 us/op [BENCHMARK] ParseJsonUnknownIgnored: med 24.21 us/op \| p99 26.25 us/op \| mean 24.31 ± 0.44 us/op [BENCHMARK] ParseRepeatedJsonDefault: med 1.94 us/item \| p99 2.05 us/item \| mean 1.95 ± 0.03 us/item [BENCHMARK] ParseRepeatedJsonCustom: med 2.26 us/item \| p99 2.45 us/item \| mean 2.27 ± 0.03 us/item [BENCHMARK] ParseRepeatedJsonUnknownIgnored: med 5.31 us/item \| p99 5.52 us/item \| mean 5.32 ± 0.05 us/item ``` Cpp: ``` [BENCHMARK] ParseJsonDefault: med 8.66 us/op \| p99 9.21 us/op \| mean 8.70 ± 0.13 us/op [BENCHMARK] ParseJsonCustom: med 26.33 us/op \| p99 29.71 us/op \| mean 26.44 ± 0.69 us/op [BENCHMARK] ParseJsonUnknownIgnored: med 28.46 us/op \| p99 31.80 us/op \| mean 28.59 ± 0.67 us/op [BENCHMARK] ParseRepeatedJsonDefault: med 1.72 us/item \| p99 2.19 us/item \| mean 1.74 ± 0.06 us/item [BENCHMARK] ParseRepeatedJsonCustom: med 2.12 us/item \| p99 2.38 us/item \| mean 2.13 ± 0.04 us/item [BENCHMARK] ParseRepeatedJsonUnknownIgnored: med 4.61 us/item \| p99 4.92 us/item \| mean 4.62 ± 0.07 us/item ``` UPB: ``` [BENCHMARK] ParseJsonDefault: med 11.22 us/op \| p99 12.29 us/op \| mean 11.27 ± 0.21 us/op [BENCHMARK] ParseJsonCustom: med 37.54 us/op \| p99 39.25 us/op \| mean 37.65 ± 0.46 us/op [BENCHMARK] ParseJsonUnknownIgnored: med 41.18 us/op \| p99 48.45 us/op \| mean 41.30 ± 0.89 us/op [BENCHMARK] ParseRepeatedJsonDefault: med 2.10 us/item \| p99 2.19 us/item \| mean 2.10 ± 0.02 us/item [BENCHMARK] ParseRepeatedJsonCustom: med 2.58 us/item \| p99 2.73 us/item \| mean 2.59 ± 0.03 us/item [BENCHMARK] ParseRepeatedJsonUnknownIgnored: med 5.70 us/item \| p99 5.90 us/item \| mean 5.71 ± 0.04 us/item ``` Seeing these benchmarks, we can see a very noticeable improvement in `ParseRepeatedJsonCustom` and `ParseRepeatedJsonUnknownIgnored` cases -- which correspond to the pathological cases that we are worried about. Singular fields are taking longer to parse in our microbenchmark because the cache that we've added is tied to each Parser instance and must be re-instantiated across every `Parse` call. PiperOrigin-RevId: 970528683	2026-08-25 13:16:01 -04:00
..
protobuf	Optimize pure Python parse path for custom JSON enum names.	2026-08-25 13:16:01 -04:00
__init__.py	drop-deprecated-pkg-resources-declare (#22442 )	2025-07-11 11:09:55 -07:00

Tony Liao 333846acf4 Optimize pure Python parse path for custom JSON enum names.

This implementation is similar to our Java idea in cl/949670944,
basically the idea is that we will have a local map (dictionary) that
will be populated the very first time we try to parse into an enum
field. The lifetime of this dictionary will be tied to the Parser
instance, similar to the Java implementation.

BENCHMARKS (baseline stats are recorded at cl/952170250):

Python:
```
[BENCHMARK] ParseJsonDefault:                med 10.06 us/op | p99 11.22 us/op | mean 10.10 ± 0.20 us/op
[BENCHMARK] ParseJsonCustom:                 med 20.33 us/op | p99 22.23 us/op | mean 20.40 ± 0.34 us/op
[BENCHMARK] ParseJsonUnknownIgnored:         med 24.21 us/op | p99 26.25 us/op | mean 24.31 ± 0.44 us/op
[BENCHMARK] ParseRepeatedJsonDefault:        med 1.94 us/item | p99 2.05 us/item | mean 1.95 ± 0.03 us/item
[BENCHMARK] ParseRepeatedJsonCustom:         med 2.26 us/item | p99 2.45 us/item | mean 2.27 ± 0.03 us/item
[BENCHMARK] ParseRepeatedJsonUnknownIgnored: med 5.31 us/item | p99 5.52 us/item | mean 5.32 ± 0.05 us/item
```

Cpp:
```
[BENCHMARK] ParseJsonDefault:                med 8.66 us/op | p99 9.21 us/op | mean 8.70 ± 0.13 us/op
[BENCHMARK] ParseJsonCustom:                 med 26.33 us/op | p99 29.71 us/op | mean 26.44 ± 0.69 us/op
[BENCHMARK] ParseJsonUnknownIgnored:         med 28.46 us/op | p99 31.80 us/op | mean 28.59 ± 0.67 us/op
[BENCHMARK] ParseRepeatedJsonDefault:        med 1.72 us/item | p99 2.19 us/item | mean 1.74 ± 0.06 us/item
[BENCHMARK] ParseRepeatedJsonCustom:         med 2.12 us/item | p99 2.38 us/item | mean 2.13 ± 0.04 us/item
[BENCHMARK] ParseRepeatedJsonUnknownIgnored: med 4.61 us/item | p99 4.92 us/item | mean 4.62 ± 0.07 us/item
```

UPB:
```
[BENCHMARK] ParseJsonDefault:                med 11.22 us/op | p99 12.29 us/op | mean 11.27 ± 0.21 us/op
[BENCHMARK] ParseJsonCustom:                 med 37.54 us/op | p99 39.25 us/op | mean 37.65 ± 0.46 us/op
[BENCHMARK] ParseJsonUnknownIgnored:         med 41.18 us/op | p99 48.45 us/op | mean 41.30 ± 0.89 us/op
[BENCHMARK] ParseRepeatedJsonDefault:        med 2.10 us/item | p99 2.19 us/item | mean 2.10 ± 0.02 us/item
[BENCHMARK] ParseRepeatedJsonCustom:         med 2.58 us/item | p99 2.73 us/item | mean 2.59 ± 0.03 us/item
[BENCHMARK] ParseRepeatedJsonUnknownIgnored: med 5.70 us/item | p99 5.90 us/item | mean 5.71 ± 0.04 us/item
```

Seeing these benchmarks, we can see a very noticeable improvement in
`ParseRepeatedJsonCustom` and `ParseRepeatedJsonUnknownIgnored` cases --
which correspond to the pathological cases that we are worried about.
Singular fields are taking longer to parse in our microbenchmark because
the cache that we've added is tied to each Parser instance and must be
re-instantiated across every `Parse` call.

PiperOrigin-RevId: 970528683

2026-08-25 13:16:01 -04:00

protobuf

Optimize pure Python parse path for custom JSON enum names.

2026-08-25 13:16:01 -04:00

__init__.py

drop-deprecated-pkg-resources-declare (#22442 )

2025-07-11 11:09:55 -07:00