When Trim processes an ASL file (`--asl-file`), it textually inlines
the body of every `Include()`'d file directly into the constructed
preprocessor input, once per include site.
Its has duplicate protection in the form of a circular-include stack
(`gIncludedAslFile`), that prevents A->B->A cycles. But, as far as
the script is concerned, each `Include()` is a unique include site.
Various combinations of includes and file types are possible and
handled slightly differently.
Starting with file types as defined in
BaseTools\Conf\build_rule.template:
- `.aslc`, `.act` files fall under `Acpi-Table-Code-File` and are
compiled, linked, and processed by genfw.
- `.asl`, `.Asl`, and `.ASL` files fall in `Acpi-Source-Language-File`
and are processed by Trim:
1. `Trim --asl-file` to produce a single combined .i file with
includes inlined.
2. `ASLPP` (ASL preprocessor, a C preprocessor) on the output of
Trim to produce a .iii file with all macros expanded and
conditional branches resolved. AutoGen.h is also included and
processed here to resolve fixed PCD values if needed.
3. `Trim --source-code` which takes the pre-processed .iii file and
produces a .iiii file with content like linemarkers cleaned up.
4. The ACPI compiler compiles the .iiii file to produce AML bytecode
in a .aml file.
Because the `.aslc`/`.act` files are directly passed to normal C
processing tools, they are not part of the Trim change made in this
commit and the remainder of this message focuses on the ACPI Source
Language File case.
ASL files can use either an ASL `Include()` directive or a C-style
`#include` directive. In addition, different file types may be
included such as a `.asl` file or a `.h` file.
`Trim` handles these cases differently:
- For ASL `Include()` directives, `Trim` inlines the content of the
included file directly into the output at the include site. This is
done for all included ASL files regardless of their extension. The
inlining is purely textual and does not attempt to resolve or
preserve any preprocessor directives such as `#pragma once` or
include guards.
- For C-style `#include` directives, `Trim` checks the file extension
of the included file. If the file is an ASL file (`.asl` or `.asi`),
`Trim` treats the file the same as the `Include()` case. Otherwise,
`Trim` passes the directive through verbatim to the output, allowing
the downstream C preprocessor (`ASLPP`) to handle it according to
normal C preprocessor rules.
This creates a situtation in which the resulting `.i` might include:
- Inlined file content (from a `.asl` or `.h` file) depending on the
include type and file extension.
- Verbatim `#include` directives for non-ASL files which will be
processed by the C preprocessor.
Focusing on the "inlined" case, historically `.h` files would have
traditional C include guards (`#ifndef`/`#define`, `#endif`). However,
files might also include `#pragma once` as a guard.
In that case, the inlined content of the `.i` file could contain
multiple `#pragma once` directives, one per include site. When the
C preprocessor (`ASLPP`) processes the `.i` file, it sees multiple
`#pragma once` directives in what it considers the main file, and
could emit a warning like the following from gcc:
warning: '#pragma once' in main file [-Wpragma-once-outside-header]
The remainder of this commit message describes the change made to
address this warning.
This change strips "#pragma once" lines on the ASL content path in
`DoInclude()` in `Trim.py` so the directive is removed before it
reaches the C preprocessor.
- "#include" directives for non-ASL files are still passed through
verbatim for the C preprocessor to resolve where the contents of
those .h files might contain "#pragma once" or traditional guards.
- Traditional include guards are untouched and continue to behave as
before where multiple include sites might inline the same content
in the .i file before reaching the C preprocessor.
The change:
In the case that a file is inlined with a `#pragma once` directive,
the directive is stripped from the inlined content which prevents the
warning.
This is considered acceptable because it only removes the
`#pragma once` directive from the inlined content for these specific
cases. So, the `.i` file might contain multiple inlined copies of the
same header content (like always in this inline case) but without the
`#pragma once` directives. Because actual C content was already not
processed or trimmed out (e.g. `typedef struct`) duplicate content is
not considered to be a problem (`#define` multiple times is not a
problem for the C preprocessor).
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Add --source-code-format option that can be NASM or not
specified. This can be used for file format specific actions
when --source-code is used.
A NASM specific action is added to convert #line to %line to
preserve reference the originating NASM source file for source
level debug in NASM format.
Without this change, the source level debug of NASM files
loads the generated intermediate file in the build output
directory.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
When including one ASL file in another, add a header / footer to the
included file to easily tell where the included file starts and ends.
Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
Switches regex patterns to raw text to resolve python 3.12 syntax
warnings in regards to invalid escape sequences, as is suggested by the
re (regex) module in python.
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
As the error is changed to warning, Trim.py will skip the build
error when the source code have exactly issue.
This patch change warning to error to opens the checking.
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2972
When using incremental build to delete an included xxx.asi file from
the ASL file, the xxx.asl.trim.deps file generated by previous build
process will not be deleted from the OUTPUT directory, which caused
the dependency file still include the xxx.asl.trim.deps file.
If the include file is deleted and DEPs is empty.
Signed-off-by: Mingyue Liang <mingyuex.liang@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2770
The Trim.py would break the build process when the file not found
issue occures, however sometimes we do not care about this issue.
This patch changes the error with warning in order to solve this
kind of break.
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Yuwei Chen <yuwei.chen@intel.com>
Reviewed-by: Bob Feng<bob.c.feng@intel.com>
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2311
Implement the function in Trim tool to get the included
file list for ASL and ASM file.
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
This is a regression issue introduced
by commit 307e1650be
This patch is to fix this issue.
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2014
Add extra debugging to improve error identification.
Error while processing file if the file is read incorrectly
This patch is going to fix that issue.
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1815
The edk2 source tree contains four instances of the outdated mailing
list address "edk2-devel@lists.01.org".
I created a new variable, MSG_EDKII_MAIL_ADDR, to receive the
new email address and replaced the old one with this variable
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=1641
The command trim --asl-file -o test.i UTF8.asl will trig the exception.
Trim tool should report error message for unsupported UTF8 file instead
of the exception.
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
https://bugzilla.tianocore.org/show_bug.cgi?id=1379
Line 208 of BaseTools/Source/Python/Trim/Trim.py uses
'NewLines.append(os.linesep)' to insert a new line into
the list that will be written to the output file.
This causes the '\r\n' inserted with os.linesep to be
written as '\r\r\n', causing some assemblers to error.
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
This reverts commit 6693f359b3..
678f851312.
Python3 migration is the fundamental change. It requires every developer
to install Python3. Before this migration, the well communication and wide
verification must be done. But now, most people is not aware of this change,
and not try it. So, Python3 migration is reverted and be moved to edk2-staging
Python3 branch for the edk2 user evaluation.
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
1. Do not use tab characters
2. No trailing white space in one line
3. All files must end with CRLF
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
When using Linaro GCC5+ arm-eabi toolchain on Windows, the generated
DSDT.iii contains a canonicalized ("\.\" removed and lower case)
filepath for the preprocessed DSDT.i file in the first line.
Trim.exe is called on DSDT.iii to generate DSDT.iiii, which does a
line for line comparison of filepaths encountered to the preprocessed
DSDT.i filepath found in the first line to determine what lines to
place in DSDT.iiii. Since the DSDT.i filepath is canonicalized and
all later filepaths in DSDT.iii are not canonicalized, all comparisons
fail and the result is in an empty DSDT.iiii.
Issue was first reported to Linaro here:
https://bugs.linaro.org/show_bug.cgi?id=2909
where the recommendation was to address the issue in Trim.exe.
This patch normalizes the case and pathname of all filepaths
encountered during Trim.exe execution on preprocessed files. This
fixes comparisons of filepaths that contain mismatching case on
case-insensitive filesystems, redundant separators, and uplevel
references.
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Christopher Co <christopher.co@microsoft.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Replace StringIO.StringIO with io.BytesIO to be compatible with python3.
This commit also removes "import StringIO" from those python scripts
that don't really use it.
Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
Convert "except ... ," to "except ... as" to be compatible with python3.
Based on "futurize -f lib2to3.fixes.fix_except"
Contributed-under: TianoCore Contribution Agreement 1.1
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Gary Lin <glin@suse.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
BINARY_FILE_TYPE_FW was 'FW'
BINARY_FILE_TYPE_GUID was 'GUID'
BINARY_FILE_TYPE_PREEFORM was 'PREEFORM'
BINARY_FILE_TYPE_UEFI_APP was 'UEFI_APP'
BINARY_FILE_TYPE_UNI_UI was 'UNI_UI'
BINARY_FILE_TYPE_UNI_VER was 'UNI_VER'
BINARY_FILE_TYPE_LIB was 'LIB'
BINARY_FILE_TYPE_PE32 was 'PE32'
BINARY_FILE_TYPE_PIC was 'PIC'
BINARY_FILE_TYPE_PEI_DEPEX was 'PEI_DEPEX'
BINARY_FILE_TYPE_DXE_DEPEX was 'DXE_DEPEX'
BINARY_FILE_TYPE_SMM_DEPEX was 'SMM_DEPEX'
BINARY_FILE_TYPE_TE was 'TE'
BINARY_FILE_TYPE_VER was 'VER'
BINARY_FILE_TYPE_UI was 'UI'
BINARY_FILE_TYPE_BIN was 'BIN'
BINARY_FILE_TYPE_FV was 'FV'
v2 - split apart FV and GUID types with different meanings.
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
1. when inf file is binary module, not generate makefile,
so need generate ffs with previous method.
2. generate Ui section maybe override and the string is not
$(MODULE_NAME)
like as:
INF RuleOverride = UI MdeModulePkg/Application/UiApp/UiApp.inf
3. Trim generate incorrect Offset.raw when some vfr not generate .lst
file in Debug directory, Trim get the VFR name with the .c files
replacement.
4. fix some depex file not generate issue
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com>
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
Update the mail address from edk2-devel@lists.sourceforge.net to
edk2-devel@lists.01.org.
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
Reviewed-by: Andrew Fish <afish@apple.com>
when no line directive in file, the tool still need to strip the typedef
statement (eg: typedef struct, typedef union ..).
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
The long value substitution must move to the front of
HEX substitution, and updated build_rule to add --trim-long
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Yingke Liu <yingke.d.liu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@18170 6f19259b-4bc3-4df7-8a09-765794883524
1. Support use expression as DSC file PCD value.
2. Update FDF parser to fix bug to get complete macro value.
3. Fix bug to replace SET statement macro and evaluate SET statement PCD value in FDF file.
4. Fix a bug for MACRO defined in conditional block cannot be processed correctly
Signed-off-by: lgao4
Reviewed-by: gikidy
git-svn-id: https://edk2.svn.sourceforge.net/svnroot/edk2/trunk/edk2@12827 6f19259b-4bc3-4df7-8a09-765794883524
1. Fix !include issues
2. Fix Trim to skip the postfix 'U' for hexadecimal and decimal numbers
3. Fix building error C2733 when building C++ code.
4. Add GCC46 tool chain definition
5. Add new RVCT and RVCTLINUX tool chains
Signed-off-by: lgao4
git-svn-id: https://edk2.svn.sourceforge.net/svnroot/edk2/trunk/edk2@12782 6f19259b-4bc3-4df7-8a09-765794883524
