Update the GitHub Actions CI tests to use the latest versions of the images, which include versions 8.7.0 and 9.2.0 of bazel and a few language specific updates.
#test-continuous
Note: The Continuous Linux TSAN test failure is pre-existing and currently broken at HEAD
PiperOrigin-RevId: 959198290
Bazel 8 no longer supports legacy toolchains, so all of our release toolchains have been updated to use platforms.
#test-continuous
PiperOrigin-RevId: 870017275
The pyi generator now includes Kythe annotations for:
* Extension field constants (e.g., `EXTENSION_FIELD`).
* Field number constants (e.g., `STRING_FIELD_FIELD_NUMBER`).
* `Create` methods in generated Stubby client classes.
These annotations allow the Python indexer to link these generated symbols back to their definitions in the `.proto` files.
PiperOrigin-RevId: 852971617
#test-continuous
This creates a module extension for system python so that it can be used in bzlmod builds. The pip_parse setup (dynamically building a new repo) doesn't work in the new bzlmod model, so we use the regular pip extension pointed at system_python's interpreter.
Remaining work to get off WORKSPACE:
* The pip extension quickly fails in the case where there's no python available, unlike our system_python setup
* Our windows upb tests were never actually running python, and system_python doesn't work (and breaks under bzlmod due to the above issue)
* Some of the upb wheel tests still don't work on bzlmod yet because they depend on some very custom http_archive dependencies.
Closes#18750Closes#23313Closes#23307
PiperOrigin-RevId: 810995270
Enables bzlmod for most CI tests, except python (and thus upb) since infra for system python headers for python dist are not bzlmod compatible and require further work.
#test-continuous
PiperOrigin-RevId: 721946253
The test wrappers were another way to document nonconformant behaviour between
different python backends. We can achieve the same by removing the wrapper
script and adding an if-condition in the test itself based on
api_implementation.Type(). Since we already do that for nonconformance between
pure Python vs. C++ backends, this change makes it easier to look for UPB
nonconformance instead of going through another layer of indirection.
Since this is the last remaining test file in the pb_unit_tests directory, I
have removed the entire directory. Future UPB tests will be in the same
directory as C++ and pure python implementations, namely:
//python/google/protobuf/internal.
Temporarily, we will need to hardcode the migrated test name in test_upb.yml
because not all tests under google.protobuf.internal support UPB yet.
(UPB testing for selected tests are added in 21e9aa6cac).
The reflection_test is a slightly more complicated test. In this commit I've
removed exemption of testParsingNestedClass in reflection_test. A follow-up
commit will ensure further conformance within the reflection_test. There are
quite a few if-conditions that were added to skip certain checks due to
nonconformant behavior -- they seem to no longer apply.
PiperOrigin-RevId: 712943652
Migrate all tests to run on bazel 7 and fix errors that came up in the process. 30.x will no longer guarantee support for bazel 6.
#test-continuous
PiperOrigin-RevId: 703590770
Enable Bazel 7 macOS test coverage which otherwise fails with
```
Undefined symbols for architecture arm64:
"_PyModule_AddIntConstant", referenced from:
_PyInit__api_implementation in api_implementation.o
"_PyModule_Create2", referenced from:
_PyInit__api_implementation in api_implementation.o
"__Py_Dealloc", referenced from:
_PyInit__api_implementation in api_implementation.o
ld: symbol(s) not found for architecture arm64
```
Fixes https://github.com/protocolbuffers/protobuf/issues/19454
#test-continuous
PiperOrigin-RevId: 702375059
Before this PR, we stored a list internally of tests that must pass on presubmit and tried to keep it up to date.
This PR moves that information keeping into GitHub by adding a 'continuous-only' variable to most testing matrices to allow authors to specify which of their tests should be skipped on presubmit. During presubmit, tests that were specified to not run on presubmit will not be run and their names will be prefixed with "[SKIPPED]". All continuous only tests will be suffixed with "(Continuous)".
At the end of running all the tests, we have a single "All Blocking Tests" signal that will tell us whether all of the necessary tests have passed (either for presubmit or continuous based on how the test was triggered).
I've tested this from a different branch [here](https://github.com/protocolbuffers/protobuf/actions/runs/9602443750?pr=17151) and from a different fork [here](https://github.com/protocolbuffers/protobuf/actions/runs/9602554500?pr=17192). These should be the same and are as far as I can tell.
I also have a continuous test run [here](https://github.com/protocolbuffers/protobuf/actions/runs/9603824200) which runs the entire test suite.
Closes#17198
PiperOrigin-RevId: 662940724
This will allow them to reuse our bazelrc and remote caching setup. This also silences the non-bzlmod windows test that's hitting the windows path length.
PiperOrigin-RevId: 626390416
There's a test run in test_python.yml that is non-trivial to get working with
Python 3.12 due to some refactoring of our Docker images that would be needed.
But this change updates everything else to add coverage for Python 3.12.
The main changes necessary to get the builds working were to upgrade some Pip
packages via requirements.txt, including in a patch to `rules_fuzzing` that I
plan to upstream soon. I also had to take an explicit dependency on
`setuptools`.
I removed tox.ini, since it was outdated and we have not been actively
maintaining it.
PiperOrigin-RevId: 580548224
This change moves almost everything in the `upb/` directory up one level, so
that for example `upb/upb/generated_code_support.h` becomes just
`upb/generated_code_support.h`. The only exceptions I made to this were that I
left `upb/cmake` and `upb/BUILD` where they are, mostly because that avoids
conflict with other files and the current locations seem reasonable for now.
The `python/` directory is a little bit of a challenge because we had to merge
the existing directory there with `upb/python/`. I made `upb/python/BUILD` into
the BUILD file for the merged directory, and it effectively loads the contents
of the other BUILD file via `python/build_targets.bzl`, but I plan to clean
this up soon.
PiperOrigin-RevId: 568651768
This will retry up to 3 times if we hit networks flakes updating our submodules. It will also allow us to easily inject other stability fixes to this step in the future.
PiperOrigin-RevId: 568306356
I am getting ready to move almost everything under the upb/ directory up one
level to integrate upb better into its new location in the protobuf repo. This
change makes a few tweaks to prepare for that:
- Delete upb's LICENSE and CONTRIBUTING.md files since we already have similar
files at the top level.
- Rename `//python:python_version` so that it won't conflict later with
`//upb/python:python_version`.
- Move the contents of python/BUILD.bazel out to a Bazel macro to facilitate
merging that BUILD.bazel file with upb/python/BUILD.
PiperOrigin-RevId: 567119840
A couple weeks ago we moved upb into the protobuf Git repo, and this change
continues the merger of the two repos by making them into a single Bazel repo.
This was mostly a matter of deleting upb's WORKSPACE file and fixing up a bunch
of references to reflect the new structure.
Most of the changes are pretty mechanical, but one thing that needed more
invasive changes was the Python script for generating CMakeLists.txt,
make_cmakelists.py. The WORKSPACE file it relied on no longer exists with this
change, so I updated it to hardcode the information it needed from that file.
PiperOrigin-RevId: 564810016
For now, this only covers linux on the two architectures we have testing support for. However, it serves as a good sanity check and can be expanded in the future.
PiperOrigin-RevId: 514449399
This will make PRs from forked repositories significantly less painful, since they'll agree on which version of each action to use. OTOH, we'll have a separate repo that needs to be maintained, and changes to it will need to be coordinated and versioned carefully. This will likely need to be done less often though now that our infrastructure is stable.
PiperOrigin-RevId: 512117705
