Fix _GetFvAttributes() to return True when it has successfully parsed
at least one attribute before encountering a non-attribute keyword.
Previously it always returned False on encountering an unrecognized
word, even after consuming prior attributes. This caused the outer
parsing loop to break prematurely when FvForceRebase, FvBaseAddress,
or FvAlignment appeared between FV attribute flags (e.g. between
ERASE_POLARITY and MEMORY_MAPPED), resulting in a Python stack trace.
Move IsWordToken assignment to after successful attribute parsing and
change the early return from 'return False' to 'return IsWordToken'.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Replace sequential if-statements for Align, Xip, and
RELOCS_STRIPPED/RELOCS_RETAINED parsing in _GetEfiSection() with a
while-loop that accepts these keywords in any permutation. Previously,
specifying Xip before Align in a [Rule] PE32 section caused a Python
stack trace.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Update ParseFvInf() to detect and strip ",XIP" suffix from
EFI_FILE_NAME values in the FV INF file. Store the per-file XIP
flag in the new XipFile[] array in the FV_INFO structure.
Add FileIndex parameter to FfsRebase() so it can look up the XIP
flag for the current file. When ForceRebase is TRUE, only rebase
files that have their XipFile[] entry set to TRUE.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Add Xip attribute to FDF Rule class and parse the Xip keyword in
EFI section rules of FDF files. Add XipEnabled attribute to
FfsInfStatement that is determined from the applicable FDF Rule's
section Xip setting. When generating the FV INF file, append ",XIP"
to EFI_FILE_NAME entries for modules whose Rule specifies Xip=TRUE.
This enables per-file XIP rebase control in GenFv by communicating
which files require XIP rebase directly in the FV INF file format.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
GenFw converts linked ELF images to PE/COFF images using a new section
layout. The generated PE/COFF section RVAs are not required to match the
linked ELF section addresses, so relocation fixups that rewrite section
contents must use the generated PE/COFF RVA space.
A linker script layout change can expose this on LoongArch64 when .text
and .data are linked with 0x1000 alignment while .hii still keeps a
0x4000 alignment. GenFw then keeps a 0x4000 PE/COFF section alignment
because of .hii, producing different ELF and PE/COFF layouts.
Fix two LoongArch relocation paths exposed by this layout mismatch.
R_LARCH_64 entries stored in section contents, such as switch jump
tables, must be translated from the linked ELF section address space to
the generated PE/COFF RVA space. R_LARCH_PCALA_* and R_LARCH_GOT_PC_*
must also convert the referenced symbol to its generated PE/COFF RVA
before calculating the PC-relative offset.
The LoongArch ELF ABI defines PCALA/GOT_PC relocations as page-based,
but this GenFw path rewrites the HI/LO pair to PCADDU12I plus ADDI.D.
After that rewrite, the offset split must be based on the generated
PE/COFF instruction-relative offset.
For example, in the failing LogoDxe image, _gUefiDriverRevision is at
ELF address 0x5400 in .text, and the relocation referencing it is at ELF
address 0x104c. With ELF .text at 0x1000 and PE/COFF .text at 0x4000,
the generated PE/COFF RVAs are 0x8400 for the symbol and 0x404c for the
relocation site. Mixing the ELF symbol address with the PE/COFF
relocation-site RVA makes the entry wrapper read PE/COFF RVA 0x5400
instead of 0x8400, causing EFI_INCOMPATIBLE_VERSION.
Use PE/COFF RVAs consistently when rewriting LoongArch absolute and
PC-relative references during WriteSections64(). The existing
WriteRelocations64() base relocation emission is kept for load-time
image rebasing.
Reported-by: Chao Li <lichao@loongson.cn>
Signed-off-by: Dongyan Qian <qiandongyan@loongson.cn>
Signed-off-by: Chao Li <lichao@loongson.cn>
Tested-by: Dongyan Qian <qiandongyan@loongson.cn>
The relocation methods of R_LARCH_PCALA_HI20, R_LARCH_PCALA_LO12,
R_LARCH_GOT_PC_HI20 and R_LARCH_GOT_PC_LO12 have been optimized to
support cases where HI is not adjacent to LO and where one HI
corresponds to multiple LOs.
Signed-off-by: Chao Li <lichao@loongson.cn>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Guillermo Antonio Palomino Sosa <guillermo.a.palomino.sosa@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Poncho Figueroa <poncho.figueroa.esqueda@intel.com>
Cc: Mike Beaton <mjsbeaton@gmail.com>
- Within the capsule generate script, it is using the STDOUT result
as signature while signing the hash digest via OpenSSL tool.
- There would have incorrect result when the user terminal have
the output when executing the startup script.
- Incorrect the content of signature would make the verification failed.
- Use the "-output" flag to export the signature then read it back
as the resolution.
Signed-off-by: Jason1 Lin <jason1.lin@intel.com>
When dumping LOONGARCH64 or RISCV64 Option Rom by `EfiRom -d test.rom`
command, the machine type is showed as `unknown`. This patch adds type
lookup strings for the two architectures.
Signed-off-by: Qihang Gao <gaoqihang@loongson.cn>
There is a bug in BaseTools currently when an inline
comment is used in tools_def. The comment is not
stripped out and wreaks havoc down the line,
causing BaseTools to get confused elsewhere and
drop build options it should be applying.
This fixes that behavior by following the build spec
which states:
Comments are only allows on separate lines and may not
be appended appear on actual entry lines.
Inline comments are now not allowed and the build will
fail and specify why and where.
Signed-off-by: Oliver Smith-Denny <osde@microsoft.com>
ANTLR 4.9 is broken in python 3.13 because
it uses a library in the autogenerated files
that is removed.
This updates to 4.13.2 and also updates the autogen
files, which contain support for python 3.13 as well
as backwards compat.
Signed-off-by: Oliver Smith-Denny <osde@microsoft.com>
The ANTLR autogen files are currently created
without an SPDX identifer. Add the BSD-2-Clause-Patent
ID.
While here, correct the command to do the autogeneration
by using the right filename.
Signed-off-by: Oliver Smith-Denny <osde@microsoft.com>
Certain environments require Control Flow Guard (CFG) to be enabled at
build time as part of their security hardening requirements.
This change adds the necessary compiler and linker flags to enable CFG
support.
Signed-off-by: Kun Qin <kun.qin@microsoft.com>
Add conditional suppression of MSVC warning C4028 (formal parameter
different from declaration) when building with Visual Studio 2017 or
earlier toolchain.
This warning is triggered by brotli submodule and not emitted by newer
compilers, hence a conditional supression is used.
Signed-off-by: Kun Qin <kun.qin@microsoft.com>
This change fixes a warning newly emitted by the latest MSVC, which now
treats this warning as an error and causes the build to fail.
This change aligns operand types in bitwise expressions by casting sizeof
results to UINT32, and promoting a UINT32 to UINTN where required to
correctly apply the bitmask.
Signed-off-by: Kun Qin <kun.qin@microsoft.com>
Add --source-code-format option that can be NASM or not
specified. This can be used for file format specific actions
when --source-code is used.
A NASM specific action is added to convert #line to %line to
preserve reference the originating NASM source file for source
level debug in NASM format.
Without this change, the source level debug of NASM files
loads the generated intermediate file in the build output
directory.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
The CParser4 Python parser files (CLexer.py, CParser.py, CListener.py)
were generated 7 years ago with ANTLR 4.7.1.
Meanwhile, pip-requirements.txt pins antlr4-python3-runtime to version
4.9 in commit 4a7dd50, but the files were patched, not fully
regenerated. This version mismatch could result in failures when
running against non-trivial C code.
This change regenerates the CParser4 files with ANTLR 4.9 to resolve
the version mismatch. It also updates import statements to correctly
reference Eot instead of Ecc.
Steps used to regenerate the files:
1. Download the ANTLR 4.9 complete tool JAR:
- `https://www.antlr.org/download/antlr-4.9-complete.jar`
2. Generate Python3 parser files
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
The CParser4 Python parser files (CLexer.py, CParser.py, CListener.py)
were generated 7 years ago with ANTLR 4.7.1.
Meanwhile, pip-requirements.txt pins antlr4-python3-runtime to version
4.9 in commit 4a7dd50, but the files were patched, not fully
regenerated. This version mismatch produced two failures when running
EccMain.py against non-trivial C code:
1. A runtime warning on every file parsed:
"ANTLR runtime and generated code versions disagree: 4.9!=4.7.1"
2. A crash when parsing complex C constructs that exercise the
struct/union definition rule in CParser.py:
TypeError: '<' not supported between instances of 'tuple' and 'int'
This occurs in antlr4/BufferedTokenStream.py getText() because the
4.9 runtime changed the expected argument types for that method,
and the 4.7.1-generated parser was passing a tuple where an int is
now required.
This change regenerates the CParser4 files with ANTLR 4.9 to resolve
the version mismatch.
Steps used to regenerate the files:
1. Download the ANTLR 4.9 complete tool JAR:
- `https://www.antlr.org/download/antlr-4.9-complete.jar`
2. Generate Python3 parser files from the grammar:
```
java -jar antlr-4.9-complete.jar `
-Dlanguage=Python3 -visitor `
-o BaseTools/Source/Python/Ecc/CParser4_new `
BaseTools/Source/Python/Ecc/CParser4/C.g4
```
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Add include guards back to include files that use the same
include guard macro in BaseTools/Source/C/Include/Common
and MdePkg or MdeModulePkg.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Update makefile rules to run antlr and dlg to completion
before compiling any of the generated cpp files.
Without this change, parallel make may start compiling some
of the cpp files before both antlr and dlg have finished
which produces syntax errors from compilation with partially
generated files.
Also use &: so the targets are treated as a group and the
rule is only executed once for the entire group. Without
this change, parallel make may run the rule actions more
than once and modify the output while it is being used by
another rule.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Replace traditional `#ifndef`/`#define`/`#endif` include guards with
`#pragma` once.
`#pragma once` is a widely supported preprocessor directive that
prevents header files from being included multiple times. It is
supported by all toolchains used to build edk2: GCC, Clang/LLVM, and
MSVC.
Note: Some files in BaseTools are excluded from the change if they
are autogenerated or direcly related to a header from a subproject,
etc. In particular, headers in these directories were ignored:
- BaseTools/Source/C/LzmaCompress/Sdk/
- BaseTools/Source/C/VfrCompile/Pccts/
Compared to macro-based include guards, `#pragma once`:
- Eliminates the risk of macro name collisions or copy/paste errors
where two headers inadvertently use the same guard macro.
- Eliminate inconsistency in the way include guard macros are named
(e.g., some files use `__FILE_H__`, others use `FILE_H_`, etc.).
- Reduces boilerplate (three lines replaced by one).
- Avoids polluting the macro namespace with guard symbols.
- Can improve build times as the preprocessor can skip re-opening the
file entirely, rather than re-reading it to find the matching
`#endif` ("multiple-include optimization").
- Note that some compilers may already optimize traditional include
guards, by recognzining the idiomatic pattern.
This change is made acknowledging that overall portability of the
code will technically be reduced, as `#pragma once` is not part of the
C/C++ standards.
However, this is considered acceptable given:
1. edk2 already defines a subset of supported compilers in
BaseTools/Conf/tools_def.template, all of which have supported
`#pragma once` for over two decades.
2. There have been concerns raised to the project about inconsistent
include guard naming and potential macro collisions.
Approximate compiler support dates:
- MSVC: Supported since Visual C++ 4.2 (1996)
- GCC: Supported since 3.4 (2004)
(http://gnu.ist.utl.pt/software/gcc/gcc-3.4/changes.html)
- Clang (LLVM based): Since initial release in 2007
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
The codebase has moved from traditional `#ifndef` include guards to
`#pragma once`. Remove the ECC checks that validated include guard
presence and naming conventions since they are no longer applicable.
The following checks are removed:
- IncludeFileCheckIfndefStatement: Verified all header file contents
were guarded by a `#ifndef` statement, that the `#ifndef` was the
first line of code after the file header comment, and that the
`#endif` appeared on the last line.
- NamingConventionCheckIfndefStatement: Verified that the `#ifndef`
guard name at the start of an include file used a postfix underscore
and no prefix underscore character.
Also removed related error codes and configuration settings that were
specific to these checks.
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
The C preprocessor turns each .vfr file into a pre-processed .i
file. At this step, the C preprocessor processes `#pragma once`.
Then, VfrCompile is called (with `-n` to prevent preprocessing)
to parse the pre-processed .i files.
The .i files may still contain `#pragma once` lines. Currently,
VfrCompile treats `once` as an unknown token, causing parse failures.
Originally, this change was going to add a `PragmaOnce` token rule
to the VFR lexer grammar (in VfrSyntax.g) that matched `#pragma once`
lines and silently skipped them using `skip()` and `newline()`. The
`newline()` call would keep line numbers stable for error reporting.
This was consistent with how other preprocessor artifacts were already
handled like `#line` directives (`LineDefinition` and
`GccLineDefinition` tokens) and `extern` declarations (skipped with
`mode(CPP_COMMENT)`).
Writing a regular expression to match `#pragma once` was simple
enough, but it makes overall pragma token recognition more fragile
at the lexer level. When the lexer is walking the DFA state table,
it could begin to match a `#pragma ` line but then not be able to
match remaining characters to recognize tokens other than `once`.
Instead, this change handles `#pragma once` lines in the VFR parser
grammar in `vfrPragmaDefinition` alongside where `pack` is already
handled.
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
This reverts commit 3fe1d56cc9.
PR https://github.com/tianocore/edk2/pull/11757 introduced a
"Breaking Change" feature for out of tree builds of tools.
This breaking change is blocking testing of edk2-stable202602
due to side effects on building FitGen tool in edk2-platforms.
Revert this feature for the edk2-stable202602 release and
work on this feature after the release.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
This reverts commit f0542ae07d.
PR https://github.com/tianocore/edk2/pull/11757 introduced a
"Breaking Change" feature for out of tree builds of tools.
This breaking change is blocking testing of edk2-stable202602
due to side effects on building FitGen tool in edk2-platforms.
Revert this feature for the edk2-stable202602 release and
work on this feature after the release.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Use $(SEP) with addprefix of $(OBJDIR) to support Windows MINGW
CLANG builds that use Windows path separators with GNU makefiles.
This fixes Windows MINGW CLANG builds of the PcdValueInit
application that is required for structured PCDs.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Main EDK2 build supports out-of-tree builds but BaseTools make process
still creates tools and object files in-tree. In order to make
out-of-tree build support complete move the generated tools and
interim obj files to $WORKSPACE location as well.
This patch also changes the location of BaseTools for in-tree builds
(default behavior when WORKSPACE is not provided before calling
edksetup) to $WORKSPACE/BaseTools/Build/... It may potentially break
external workflows that invoke tools from the default location outside
of the build tool.
Signed-off-by: Oleksandr Tymoshenko <ovt@google.com>
VfrLexer.h is built as a part of VfrCompile build and
shouldn't be present at the BaseTools/Source/C level.
Signed-off-by: Oleksandr Tymoshenko <ovt@google.com>
New warning after updating gcc:
EfiRom.c: In function ‘main’:
EfiRom.c:78:17: error: assignment discards ‘const’ qualifier from pointer target type [-Werror=discarded-qualifiers]
The assigned value is not used, so fix the warning by just removing it.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
StringFuncs.c: In function ‘SplitStringByWhitespace’:
StringFuncs.c:113:15: error: variable ‘Item’ set but not used [-Werror=unused-but-set-variable=]
113 | UINTN Item;
| ^~~~
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
The PCD value defined in module subsections can be added to global PCD
database. Therefore the unsolved expressions, even belongs to the global
scope, can incorrectly refer to the value from module subsection.
This only happens when the referred PCD has no value assignment in the
platform dsc file. Which also should raise an error.
Signed-off-by: Paddy Deng <paddydeng@ami.com>
Update _get_win32_parent_processes() to detect a
cycle in parent process ids that can cause
_get_win32_parent_processes() to never return.
If a pid cycle is detected, then return the list
of parent process ids detected up to the point
the cycle is detected.
GitHub Actions builds using windows-2025 can
reproduce this issue once in a while. It shows
up as job that runs until a timeout. If the job
is canceled, the logs show a python stack in the
loop in _get_win32_parent_processes().
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
The stuart tools automatically add -D WIN_HOST_BUILD to
edk2 build command line if a Windows build environment
is detected. This behavior is added to build.py so that
builds of the EmulatorPkg using build.py are not required
to add the option -D WIN_HOST_BUILD when building in a
Windows environment. This aligns Linux and Windows builds
of the EmulatorPkg removing the need to specify extra
defines.
In order to build the EmulatorPkg for Windows Mingw
environments, EmulatorPkg DSC/FDF files require a way
to detect if Windows Mingw environment is present.
The Windows Mingw environment can be detected if
CLANG_BIN is set and mingw32-make.exe is detected
in the CLANG_BIN directory.
If a Windows Mingw environment is detected, add
-D WIN_MINGW32_BUILD to the edk2 build command line.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
The ELF psABIs for all architectures stipulate that R_*_NONE relocations
require no action at relocation time, and merely exist to declare a
dependency on a symbol in a way that cannot be conveyed by the code
itself (i.e., using an actual symbol reference). Given that EFI PE/COFF
images are always fully linked binaries, such a dependency cannot be
translated, and there are no known reasons why this would be necessary.
So instead of ignoring such relocations specifically on x86_64 only,
ignore them on all architectures when converting ELF binaries to
PE/COFF.
Fixes: #11878
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
For nebulous reasons, the original ELF psABI deviated from common sense,
and decided to #define R_AARCH64_NONE as '256', in spite of the fact
that no other architecture uses anything other than 0x0.
This has now been fixed in the psABI, so fix it in our code as well.
Continuous-integration-options: PatchCheck.ignore-multi-package
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Use $(OS) in all GNUMakefiles to detect if the GNUMakefile
is being used in a Windows OS. If a Windows OS is detected,
then override SHELL to use cmd.exe. This prevents make
utility from using sh.exe if sh.exe happens to be in PATH.
If sh.exe is used, then backslash (\) characters in file
paths are removed and builds break for files not found.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
When the number of build threads multiplied by per-thread file
descriptor usage exceeds the system's open file descriptor limit,
some threads may fail to acquire necessary resources (e.g., pipes
or semaphores), leading to deadlocks or hangs during parallel builds.
To prevent this situation, calculate the safety upper limit of
concurrency by dividing the system's maximum file descriptor limit by
3 (An empirical value derived from balancing performance overhead
against the theoretical number of file descriptors consumed per thread).
The actual thread count is then clamped to this safe value.
Other usages of ThreadNum()—such as during actual compilation or log
queue creation—do not significantly contribute to file descriptor
consumption. Therefore, adjusting ThreadNum() globally would be
unwarranted, as it could unnecessarily restrict parallelism in stages
that are not FD-bound.
This ensures stable parallel builds even under constrained resource
limits.
Signed-off-by: Ayden Meng <mengxiangdong@loongson.cn>
Previously, when file descriptors were exhausted in high-concurrency
builds (e.g., 512 threads with 1024 FD limit), the build would hang or
fail silently without clear indication of the root cause.
This change catches relevant OSError instances and terminates the build,
ensuring failures due to resource limits are explicit.
Signed-off-by: Ayden Meng <mengxiangdong@loongson.cn>
There are no functional changes in this commit. It adds non-functional
minor code cleanups made while tracking down the bugs fixed in the
preceding commits.
- Fix spelling 'arrary' -> 'array' where it occurs in the tool code and comments.
- When renaming ExtractFieldNameAndArrary to ExtractFieldNameAndArrayIdx,
we fix the typo and also clarify what the method does: it returns the field name
and the array index when present. Since it is always used to populate a local
variable named ArrayIdx, we stick to that spelling in the renamed method.
- Add comment clarifying that mArrayNum holds array length (but again, don't
go for a bigger global rename).
- Rename CVfrVarDataTypeDB::GetFieldWidth to GetFieldType. It is only used
once, and the new name correctly matches what it does.
Signed-off-by: Mike Beaton <mjsbeaton@gmail.com>
Internal types such as pFormIdField, pFormSetGuidField have
no associated field type, however the Dump command assumes
that all types do. This causes null pointer dereferencing which
can be caught by sanitizers.
De facto, the field type for these types was already
dumped as <null>, this achieves the same safely.
Signed-off-by: Mike Beaton <mjsbeaton@gmail.com>
The previous logic for accessing bit fields is wrong when the bit field
starts beyond the first byte, in a storage unit larger than a byte.
This caused unaligned memory accesses which can be caught
by sanitizers.
We also bump the tool minor version number to reflect a functional bugfix.
Signed-off-by: Mike Beaton <mjsbeaton@gmail.com>
When traversing through the field definition of a nested type, such
as "MY_EFI_BITS_VARSTORE_DATA.BitsData.NestBitCheckbox", in method
CVfrVarDataTypeDB::GetDataFieldInfo, the mBitWidth member of the
parent (e.g. BitsData) of a bitfield is accessed, even though it
is not itself a bit field and thus its mBitWidth has never been
initialised.
This is not necessary, the Size and Type of each parent object
were being calculated then overwritten by the next lower object,
so we can just avoid calculating these until the final one which
we actually need.
This behaviour was caught by clang MSan.
Signed-off-by: Mike Beaton <mjsbeaton@gmail.com>
The memmove in SetClassGuid at line 684 generates a
`store to misaligned address` runtime warning when
VfrCompile is compiled with clang UBSan.
We can avoid generating code which stores to a misaligned
address in a least a couple of other ways than done here,
including casting the destination pointer of the memmove
to (VOID *), or compiling with `-fno-builtin-memmove`.
However, maintaining an unaligned pointer to an aligned
type is, strictly speaking, undefined behaviour in the
c/c++ standard (regardless of how it is or is not then
accessed).
So here we resolve the actual incorrect semantics of this
part of the code (and thereby fix the unaligned access) by
introducing a PACKED_EFI_GUID type in VfrUtilityLib.h, and
using it for this unaligned access.
Signed-off-by: Mike Beaton <mjsbeaton@gmail.com>
Using GCC 13.3.0 discovers an out of bounds memory access in VfrCompile
when building DriverSampleDxe. This is also discoverable with ASan.
The issue here is that EFI_IFR_TYPE_VALUE is a flexible type and
when passed by value for string types only the header part is accessible.
Assuming the remainder is zero seems to be ok as gZeroEfiIfrTypeValue
is used as a variable source.
This change also fixes a warning for new[]/delete[] mismatch
discovered by ASan.
Co-authored-by: Vitaly Cheptsov <vit9696@protonmail.com>
Signed-off-by: Mike Beaton <mjsbeaton@gmail.com>
Without this change, the macro causes unwanted 'member access within
null pointer' warnings with UBSan.
This is the same macro definition as used in MdePkg/Include/Base.h,
except for removing the legacy `__GNUC__ >= 4` test. Ideally the
orginal macro should have this removed too, as should the several
other instances with similar tests around the codebase, but we leave
that for a separate commit.
Signed-off-by: Mike Beaton <mjsbeaton@gmail.com>
