2010-02-28 23:39:39 +00:00
## @file
# Generate AutoGen.h, AutoGen.c and *.depex files
#
2018-01-22 11:17:09 +08:00
# Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
2018-05-09 17:03:23 +08:00
# Copyright (c) 2018, Hewlett Packard Enterprise Development, L.P.<BR>
#
2010-05-18 05:04:32 +00:00
# This program and the accompanying materials
2010-02-28 23:39:39 +00:00
# are licensed and made available under the terms and conditions of the BSD License
# which accompanies this distribution. The full text of the license may be found at
# http://opensource.org/licenses/bsd-license.php
#
# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
# WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
#
## Import Modules
#
2018-06-25 18:31:26 +08:00
from __future__ import print_function
2018-07-13 18:18:37 +08:00
from __future__ import absolute_import
2014-08-15 03:06:48 +00:00
import Common . LongFilePathOs as os
2010-02-28 23:39:39 +00:00
import re
import os . path as path
import copy
2015-06-10 07:50:59 +00:00
import uuid
2010-02-28 23:39:39 +00:00
2018-07-13 18:18:37 +08:00
from . import GenC
from . import GenMake
from . import GenDepex
2018-06-25 18:31:36 +08:00
from io import BytesIO
2010-02-28 23:39:39 +00:00
2018-07-13 18:18:37 +08:00
from . StrGather import *
from . BuildEngine import BuildRule
2010-02-28 23:39:39 +00:00
2014-08-15 03:06:48 +00:00
from Common . LongFilePathSupport import CopyLongFilePath
2010-02-28 23:39:39 +00:00
from Common . BuildToolError import *
from Common . DataType import *
from Common . Misc import *
2018-05-19 18:50:25 +08:00
from Common . StringUtils import *
2010-02-28 23:39:39 +00:00
import Common . GlobalData as GlobalData
from GenFds . FdfParser import *
from CommonDataClass . CommonClass import SkuInfoClass
from Workspace . BuildClassObject import *
2013-11-18 07:41:21 +00:00
from GenPatchPcdTable . GenPatchPcdTable import parsePcdInfoFromMapFile
2010-09-06 01:58:00 +00:00
import Common . VpdInfoFile as VpdInfoFile
2018-07-13 18:18:37 +08:00
from . GenPcdDb import CreatePcdDatabaseCode
2013-11-18 07:41:21 +00:00
from Workspace . MetaFileCommentParser import UsageList
2018-05-18 08:06:51 +08:00
from Workspace . WorkspaceCommon import GetModuleLibInstances
2015-10-08 09:27:14 +00:00
from Common . MultipleWorkspace import MultipleWorkspace as mws
2018-07-13 18:18:37 +08:00
from . import InfSectionParser
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
import datetime
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
import hashlib
2018-07-13 18:18:37 +08:00
from . GenVar import VariableMgr , var_info
2018-04-04 05:03:05 +08:00
from collections import OrderedDict
2018-04-06 07:13:51 +08:00
from collections import defaultdict
2018-05-18 08:06:52 +08:00
from Workspace . WorkspaceCommon import OrderedListDict
2014-08-28 13:53:34 +00:00
2018-08-03 23:11:07 +08:00
from Common . caching import cached_property , cached_class_function
2018-08-03 23:11:06 +08:00
2013-11-18 07:41:21 +00:00
## Regular expression for splitting Dependency Expression string into tokens
2010-02-28 23:39:39 +00:00
gDepexTokenPattern = re . compile ( " ( \ (| \ )| \ w+| \ S+ \ .inf) " )
2018-04-04 06:34:02 +08:00
## Regular expression for match: PCD(xxxx.yyy)
gPCDAsGuidPattern = re . compile ( r " ^PCD \ (.+ \ ..+ \ )$ " )
#
# Regular expression for finding Include Directories, the difference between MSFT and INTEL/GCC/RVCT
# is the former use /I , the Latter used -I to specify include directories
#
gBuildOptIncludePatternMsft = re . compile ( r " (?:.*?)/I[ \ t]*([^ ]*) " , re . MULTILINE | re . DOTALL )
gBuildOptIncludePatternOther = re . compile ( r " (?:.*?)-I[ \ t]*([^ ]*) " , re . MULTILINE | re . DOTALL )
2014-08-28 13:53:34 +00:00
#
# Match name = variable
#
gEfiVarStoreNamePattern = re . compile ( " \ s*name \ s*= \ s*( \ w+) " )
#
# The format of guid in efivarstore statement likes following and must be correct:
# guid = {0xA04A27f4, 0xDF00, 0x4D42, {0xB5, 0x52, 0x39, 0x51, 0x13, 0x02, 0x11, 0x3D}}
#
gEfiVarStoreGuidPattern = re . compile ( " \ s*guid \ s*= \ s*( { .*? { .*?} \ s*}) " )
2010-02-28 23:39:39 +00:00
## Mapping Makefile type
gMakeTypeMap = { " MSFT " : " nmake " , " GCC " : " gmake " }
## Build rule configuration file
2017-04-17 17:12:06 +08:00
gDefaultBuildRuleFile = ' build_rule.txt '
2010-02-28 23:39:39 +00:00
2017-04-11 13:20:08 +08:00
## Tools definition configuration file
2017-04-17 17:12:06 +08:00
gDefaultToolsDefFile = ' tools_def.txt '
2017-04-11 13:20:08 +08:00
2012-04-10 07:18:20 +00:00
## Build rule default version
AutoGenReqBuildRuleVerNum = " 0.1 "
2010-02-28 23:39:39 +00:00
## default file name for AutoGen
gAutoGenCodeFileName = " AutoGen.c "
gAutoGenHeaderFileName = " AutoGen.h "
gAutoGenStringFileName = " %(module_name)s StrDefs.h "
gAutoGenStringFormFileName = " %(module_name)s StrDefs.hpk "
gAutoGenDepexFileName = " %(module_name)s .depex "
2016-09-21 10:39:11 +08:00
gAutoGenImageDefFileName = " %(module_name)s ImgDefs.h "
gAutoGenIdfFileName = " %(module_name)s Idf.hpk "
2014-08-28 13:53:34 +00:00
gInfSpecVersion = " 0x00010017 "
2011-05-11 10:26:49 +00:00
#
# Template string to generic AsBuilt INF
#
2013-11-18 07:41:21 +00:00
gAsBuiltInfHeaderString = TemplateString ( """ $ {header_comments}
2011-05-11 10:26:49 +00:00
2014-08-28 13:53:34 +00:00
# DO NOT EDIT
# FILE auto-generated
2011-05-11 10:26:49 +00:00
[ Defines ]
2014-08-28 13:53:34 +00:00
INF_VERSION = $ { module_inf_version }
2011-05-11 10:26:49 +00:00
BASE_NAME = $ { module_name }
FILE_GUID = $ { module_guid }
2014-08-28 13:53:34 +00:00
MODULE_TYPE = $ { module_module_type } $ { BEGIN }
VERSION_STRING = $ { module_version_string } $ { END } $ { BEGIN }
2013-11-18 07:41:21 +00:00
PCD_IS_DRIVER = $ { pcd_is_driver_string } $ { END } $ { BEGIN }
2011-05-11 10:26:49 +00:00
UEFI_SPECIFICATION_VERSION = $ { module_uefi_specification_version } $ { END } $ { BEGIN }
2014-08-28 13:53:34 +00:00
PI_SPECIFICATION_VERSION = $ { module_pi_specification_version } $ { END } $ { BEGIN }
ENTRY_POINT = $ { module_entry_point } $ { END } $ { BEGIN }
UNLOAD_IMAGE = $ { module_unload_image } $ { END } $ { BEGIN }
CONSTRUCTOR = $ { module_constructor } $ { END } $ { BEGIN }
DESTRUCTOR = $ { module_destructor } $ { END } $ { BEGIN }
SHADOW = $ { module_shadow } $ { END } $ { BEGIN }
PCI_VENDOR_ID = $ { module_pci_vendor_id } $ { END } $ { BEGIN }
PCI_DEVICE_ID = $ { module_pci_device_id } $ { END } $ { BEGIN }
PCI_CLASS_CODE = $ { module_pci_class_code } $ { END } $ { BEGIN }
PCI_REVISION = $ { module_pci_revision } $ { END } $ { BEGIN }
BUILD_NUMBER = $ { module_build_number } $ { END } $ { BEGIN }
SPEC = $ { module_spec } $ { END } $ { BEGIN }
UEFI_HII_RESOURCE_SECTION = $ { module_uefi_hii_resource_section } $ { END } $ { BEGIN }
MODULE_UNI_FILE = $ { module_uni_file } $ { END }
[ Packages . $ { module_arch } ] $ { BEGIN }
2011-05-11 10:26:49 +00:00
$ { package_item } $ { END }
[ Binaries . $ { module_arch } ] $ { BEGIN }
$ { binary_item } $ { END }
2013-11-18 07:41:21 +00:00
[ PatchPcd . $ { module_arch } ] $ { BEGIN }
$ { patchablepcd_item }
$ { END }
2014-08-28 13:53:34 +00:00
2013-11-18 07:41:21 +00:00
[ Protocols . $ { module_arch } ] $ { BEGIN }
$ { protocol_item }
$ { END }
2014-08-28 13:53:34 +00:00
2013-11-18 07:41:21 +00:00
[ Ppis . $ { module_arch } ] $ { BEGIN }
$ { ppi_item }
$ { END }
2014-08-28 13:53:34 +00:00
2013-11-18 07:41:21 +00:00
[ Guids . $ { module_arch } ] $ { BEGIN }
$ { guid_item }
$ { END }
2014-08-28 13:53:34 +00:00
2013-11-18 07:41:21 +00:00
[ PcdEx . $ { module_arch } ] $ { BEGIN }
$ { pcd_item }
$ { END }
2011-05-11 10:26:49 +00:00
2014-08-28 13:53:34 +00:00
[ LibraryClasses . $ { module_arch } ]
## @LIB_INSTANCES${BEGIN}
# ${libraryclasses_item}${END}
$ { depexsection_item }
2017-04-13 14:33:05 +08:00
$ { userextension_tianocore_item }
2014-08-28 13:53:34 +00:00
$ { tail_comments }
[ BuildOptions . $ { module_arch } ]
2011-05-11 10:26:49 +00:00
## @AsBuilt${BEGIN}
## ${flags_item}${END}
""" )
2010-02-28 23:39:39 +00:00
## Base class for AutoGen
#
# This class just implements the cache mechanism of AutoGen objects.
#
class AutoGen ( object ) :
2018-03-15 08:20:27 +08:00
# database to maintain the objects in each child class
__ObjectCache = { } # (BuildTarget, ToolChain, ARCH, platform file): AutoGen object
2010-02-28 23:39:39 +00:00
## Factory method
#
# @param Class class object of real AutoGen class
# (WorkspaceAutoGen, ModuleAutoGen or PlatformAutoGen)
# @param Workspace Workspace directory or WorkspaceAutoGen object
# @param MetaFile The path of meta file
# @param Target Build target
# @param Toolchain Tool chain name
# @param Arch Target arch
# @param *args The specific class related parameters
# @param **kwargs The specific class related dict parameters
#
2018-03-15 08:20:27 +08:00
def __new__ ( cls , Workspace , MetaFile , Target , Toolchain , Arch , * args , * * kwargs ) :
2010-02-28 23:39:39 +00:00
# check if the object has been created
2018-03-15 08:20:27 +08:00
Key = ( Target , Toolchain , Arch , MetaFile )
2018-08-03 23:11:06 +08:00
if Key in cls . __ObjectCache :
2018-03-15 08:20:27 +08:00
# if it exists, just return it directly
return cls . __ObjectCache [ Key ]
# it didnt exist. create it, cache it, then return it
2018-08-03 23:11:06 +08:00
RetVal = cls . __ObjectCache [ Key ] = super ( AutoGen , cls ) . __new__ ( cls )
return RetVal
2018-03-15 08:20:27 +08:00
def __init__ ( self , Workspace , MetaFile , Target , Toolchain , Arch , * args , * * kwargs ) :
super ( AutoGen , self ) . __init__ ( self , Workspace , MetaFile , Target , Toolchain , Arch , * args , * * kwargs )
2010-02-28 23:39:39 +00:00
## hash() operator
#
# The file path of platform file will be used to represent hash value of this object
#
# @retval int Hash value of the file path of platform file
#
def __hash__ ( self ) :
return hash ( self . MetaFile )
## str() operator
#
# The file path of platform file will be used to represent this object
#
# @retval string String of platform file path
#
def __str__ ( self ) :
return str ( self . MetaFile )
## "==" operator
def __eq__ ( self , Other ) :
return Other and self . MetaFile == Other
## Workspace AutoGen class
#
# This class is used mainly to control the whole platform build for different
# architecture. This class will generate top level makefile.
#
class WorkspaceAutoGen ( AutoGen ) :
2018-03-15 08:20:27 +08:00
# call super().__init__ then call the worker function with different parameter count
def __init__ ( self , Workspace , MetaFile , Target , Toolchain , Arch , * args , * * kwargs ) :
2018-08-03 23:11:06 +08:00
if not hasattr ( self , " _Init " ) :
2018-03-15 08:20:27 +08:00
super ( WorkspaceAutoGen , self ) . __init__ ( Workspace , MetaFile , Target , Toolchain , Arch , * args , * * kwargs )
self . _InitWorker ( Workspace , MetaFile , Target , Toolchain , Arch , * args , * * kwargs )
self . _Init = True
2018-07-05 17:40:04 +08:00
2018-03-15 08:20:27 +08:00
## Initialize WorkspaceAutoGen
2010-02-28 23:39:39 +00:00
#
# @param WorkspaceDir Root directory of workspace
# @param ActivePlatform Meta-file of active platform
# @param Target Build target
# @param Toolchain Tool chain name
# @param ArchList List of architecture of current build
# @param MetaFileDb Database containing meta-files
# @param BuildConfig Configuration of build
# @param ToolDefinition Tool chain definitions
# @param FlashDefinitionFile File of flash definition
# @param Fds FD list to be generated
# @param Fvs FV list to be generated
2011-08-26 07:46:26 +00:00
# @param Caps Capsule list to be generated
2010-02-28 23:39:39 +00:00
# @param SkuId SKU id from command line
#
2018-03-15 08:20:27 +08:00
def _InitWorker ( self , WorkspaceDir , ActivePlatform , Target , Toolchain , ArchList , MetaFileDb ,
2015-12-01 04:22:16 +00:00
BuildConfig , ToolDefinition , FlashDefinitionFile = ' ' , Fds = None , Fvs = None , Caps = None , SkuId = ' ' , UniFlag = None ,
2011-12-16 08:52:13 +00:00
Progress = None , BuildModule = None ) :
2011-10-29 06:59:30 +00:00
self . BuildDatabase = MetaFileDb
self . MetaFile = ActivePlatform
2010-02-28 23:39:39 +00:00
self . WorkspaceDir = WorkspaceDir
2018-04-20 23:51:25 +08:00
self . Platform = self . BuildDatabase [ self . MetaFile , TAB_ARCH_COMMON , Target , Toolchain ]
2011-12-07 06:19:28 +00:00
GlobalData . gActivePlatform = self . Platform
2010-02-28 23:39:39 +00:00
self . BuildTarget = Target
self . ToolChain = Toolchain
self . ArchList = ArchList
self . SkuId = SkuId
2010-03-19 06:55:07 +00:00
self . UniFlag = UniFlag
2010-02-28 23:39:39 +00:00
self . TargetTxt = BuildConfig
self . ToolDef = ToolDefinition
self . FdfFile = FlashDefinitionFile
2018-04-11 07:17:22 +08:00
self . FdTargetList = Fds if Fds else [ ]
self . FvTargetList = Fvs if Fvs else [ ]
self . CapTargetList = Caps if Caps else [ ]
2010-02-28 23:39:39 +00:00
self . AutoGenObjectList = [ ]
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
self . _BuildDir = None
self . _FvDir = None
self . _MakeFileDir = None
self . _BuildCommand = None
2017-12-26 16:17:13 +08:00
self . _GuidDict = { }
2010-02-28 23:39:39 +00:00
# there's many relative directory operations, so ...
os . chdir ( self . WorkspaceDir )
2011-10-29 06:59:30 +00:00
#
# Merge Arch
#
if not self . ArchList :
ArchList = set ( self . Platform . SupArchList )
else :
ArchList = set ( self . ArchList ) & set ( self . Platform . SupArchList )
if not ArchList :
EdkLogger . error ( " build " , PARAMETER_INVALID ,
ExtraData = " Invalid ARCH specified. [Valid ARCH: %s ] " % ( " " . join ( self . Platform . SupArchList ) ) )
elif self . ArchList and len ( ArchList ) != len ( self . ArchList ) :
SkippedArchList = set ( self . ArchList ) . symmetric_difference ( set ( self . Platform . SupArchList ) )
EdkLogger . verbose ( " \n Arch [ %s ] is ignored because the platform supports [ %s ] only! "
% ( " " . join ( SkippedArchList ) , " " . join ( self . Platform . SupArchList ) ) )
self . ArchList = tuple ( ArchList )
# Validate build target
if self . BuildTarget not in self . Platform . BuildTargets :
2015-12-01 04:22:16 +00:00
EdkLogger . error ( " build " , PARAMETER_INVALID ,
2011-10-29 06:59:30 +00:00
ExtraData = " Build target [ %s ] is not supported by the platform. [Valid target: %s ] "
% ( self . BuildTarget , " " . join ( self . Platform . BuildTargets ) ) )
2018-07-05 17:40:04 +08:00
2010-02-28 23:39:39 +00:00
# parse FDF file to get PCDs in it, if any
2011-10-29 06:59:30 +00:00
if not self . FdfFile :
self . FdfFile = self . Platform . FlashDefinition
2015-12-01 04:22:16 +00:00
2011-12-16 08:52:13 +00:00
EdkLogger . info ( " " )
if self . ArchList :
EdkLogger . info ( ' %-16s = %s ' % ( " Architecture(s) " , ' ' . join ( self . ArchList ) ) )
EdkLogger . info ( ' %-16s = %s ' % ( " Build target " , self . BuildTarget ) )
2015-12-01 04:22:16 +00:00
EdkLogger . info ( ' %-16s = %s ' % ( " Toolchain " , self . ToolChain ) )
2011-12-16 08:52:13 +00:00
EdkLogger . info ( ' \n %-24s = %s ' % ( " Active Platform " , self . Platform ) )
if BuildModule :
EdkLogger . info ( ' %-24s = %s ' % ( " Active Module " , BuildModule ) )
2015-12-01 04:22:16 +00:00
2011-12-16 08:52:13 +00:00
if self . FdfFile :
EdkLogger . info ( ' %-24s = %s ' % ( " Flash Image Definition " , self . FdfFile ) )
2011-10-29 06:59:30 +00:00
2011-12-16 08:52:13 +00:00
EdkLogger . verbose ( " \n FLASH_DEFINITION = %s " % self . FdfFile )
2015-12-01 04:22:16 +00:00
2018-01-22 11:17:09 +08:00
if Progress :
Progress . Start ( " \n Processing meta-data " )
2015-12-01 04:22:16 +00:00
2011-10-29 06:59:30 +00:00
if self . FdfFile :
2011-09-19 09:03:59 +00:00
#
# Mark now build in AutoGen Phase
#
2015-12-01 04:22:16 +00:00
GlobalData . gAutoGenPhase = True
2010-02-28 23:39:39 +00:00
Fdf = FdfParser ( self . FdfFile . Path )
Fdf . ParseFile ( )
2014-08-26 02:52:24 +00:00
GlobalData . gFdfParser = Fdf
2011-10-29 06:59:30 +00:00
GlobalData . gAutoGenPhase = False
2010-02-28 23:39:39 +00:00
PcdSet = Fdf . Profile . PcdDict
2016-01-26 08:45:34 +00:00
if Fdf . CurrentFdName and Fdf . CurrentFdName in Fdf . Profile . FdDict :
FdDict = Fdf . Profile . FdDict [ Fdf . CurrentFdName ]
for FdRegion in FdDict . RegionList :
if str ( FdRegion . RegionType ) is ' FILE ' and self . Platform . VpdToolGuid in str ( FdRegion . RegionDataList ) :
if int ( FdRegion . Offset ) % 8 != 0 :
EdkLogger . error ( " build " , FORMAT_INVALID , ' The VPD Base Address %s must be 8-byte aligned. ' % ( FdRegion . Offset ) )
2010-02-28 23:39:39 +00:00
ModuleList = Fdf . Profile . InfList
self . FdfProfile = Fdf . Profile
2011-10-29 06:59:30 +00:00
for fvname in self . FvTargetList :
if fvname . upper ( ) not in self . FdfProfile . FvDict :
EdkLogger . error ( " build " , OPTION_VALUE_INVALID ,
" No such an FV in FDF file: %s " % fvname )
2016-07-20 13:58:03 +08:00
2016-08-03 17:09:18 +08:00
# In DSC file may use FILE_GUID to override the module, then in the Platform.Modules use FILE_GUIDmodule.inf as key,
# but the path (self.MetaFile.Path) is the real path
2016-07-20 13:58:03 +08:00
for key in self . FdfProfile . InfDict :
if key == ' ArchTBD ' :
2018-04-20 23:51:43 +08:00
MetaFile_cache = defaultdict ( set )
2016-07-20 13:58:03 +08:00
for Arch in self . ArchList :
2018-04-11 07:17:21 +08:00
Current_Platform_cache = self . BuildDatabase [ self . MetaFile , Arch , Target , Toolchain ]
for Pkey in Current_Platform_cache . Modules :
MetaFile_cache [ Arch ] . add ( Current_Platform_cache . Modules [ Pkey ] . MetaFile )
2016-07-20 13:58:03 +08:00
for Inf in self . FdfProfile . InfDict [ key ] :
ModuleFile = PathClass ( NormPath ( Inf ) , GlobalData . gWorkspace , Arch )
for Arch in self . ArchList :
2016-08-03 17:09:18 +08:00
if ModuleFile in MetaFile_cache [ Arch ] :
2016-07-20 13:58:03 +08:00
break
else :
ModuleData = self . BuildDatabase [ ModuleFile , Arch , Target , Toolchain ]
if not ModuleData . IsBinaryModule :
EdkLogger . error ( ' build ' , PARSER_ERROR , " Module %s NOT found in DSC file; Is it really a binary module? " % ModuleFile )
else :
for Arch in self . ArchList :
if Arch == key :
Platform = self . BuildDatabase [ self . MetaFile , Arch , Target , Toolchain ]
2018-04-06 07:13:52 +08:00
MetaFileList = set ( )
for Pkey in Platform . Modules :
MetaFileList . add ( Platform . Modules [ Pkey ] . MetaFile )
2016-07-20 13:58:03 +08:00
for Inf in self . FdfProfile . InfDict [ key ] :
ModuleFile = PathClass ( NormPath ( Inf ) , GlobalData . gWorkspace , Arch )
2016-08-03 17:09:18 +08:00
if ModuleFile in MetaFileList :
2016-07-20 13:58:03 +08:00
continue
ModuleData = self . BuildDatabase [ ModuleFile , Arch , Target , Toolchain ]
if not ModuleData . IsBinaryModule :
EdkLogger . error ( ' build ' , PARSER_ERROR , " Module %s NOT found in DSC file; Is it really a binary module? " % ModuleFile )
2010-02-28 23:39:39 +00:00
else :
PcdSet = { }
ModuleList = [ ]
self . FdfProfile = None
2011-10-29 06:59:30 +00:00
if self . FdTargetList :
EdkLogger . info ( " No flash definition file found. FD [ %s ] will be ignored. " % " " . join ( self . FdTargetList ) )
self . FdTargetList = [ ]
if self . FvTargetList :
EdkLogger . info ( " No flash definition file found. FV [ %s ] will be ignored. " % " " . join ( self . FvTargetList ) )
self . FvTargetList = [ ]
if self . CapTargetList :
EdkLogger . info ( " No flash definition file found. Capsule [ %s ] will be ignored. " % " " . join ( self . CapTargetList ) )
self . CapTargetList = [ ]
2015-12-01 04:22:16 +00:00
2010-02-28 23:39:39 +00:00
# apply SKU and inject PCDs from Flash Definition file
for Arch in self . ArchList :
2011-10-29 06:59:30 +00:00
Platform = self . BuildDatabase [ self . MetaFile , Arch , Target , Toolchain ]
2018-03-23 11:39:39 +08:00
PlatformPcds = Platform . Pcds
self . _GuidDict = Platform . _GuidDict
2018-04-20 23:51:25 +08:00
SourcePcdDict = { TAB_PCDS_DYNAMIC_EX : set ( ) , TAB_PCDS_PATCHABLE_IN_MODULE : set ( ) , TAB_PCDS_DYNAMIC : set ( ) , TAB_PCDS_FIXED_AT_BUILD : set ( ) }
BinaryPcdDict = { TAB_PCDS_DYNAMIC_EX : set ( ) , TAB_PCDS_PATCHABLE_IN_MODULE : set ( ) }
2016-04-12 10:31:55 +08:00
SourcePcdDict_Keys = SourcePcdDict . keys ( )
BinaryPcdDict_Keys = BinaryPcdDict . keys ( )
# generate the SourcePcdDict and BinaryPcdDict
2018-01-31 16:49:14 +08:00
PGen = PlatformAutoGen ( self , self . MetaFile , Target , Toolchain , Arch )
2016-04-12 10:31:55 +08:00
for BuildData in PGen . BuildDatabase . _CACHE_ . values ( ) :
if BuildData . Arch != Arch :
continue
if BuildData . MetaFile . Ext == ' .inf ' :
for key in BuildData . Pcds :
if BuildData . Pcds [ key ] . Pending :
if key in Platform . Pcds :
PcdInPlatform = Platform . Pcds [ key ]
2018-04-20 23:51:42 +08:00
if PcdInPlatform . Type :
2016-04-12 10:31:55 +08:00
BuildData . Pcds [ key ] . Type = PcdInPlatform . Type
2018-05-10 16:47:44 +08:00
BuildData . Pcds [ key ] . Pending = False
2016-04-12 10:31:55 +08:00
if BuildData . MetaFile in Platform . Modules :
PlatformModule = Platform . Modules [ str ( BuildData . MetaFile ) ]
if key in PlatformModule . Pcds :
PcdInPlatform = PlatformModule . Pcds [ key ]
2018-04-20 23:51:42 +08:00
if PcdInPlatform . Type :
2016-04-12 10:31:55 +08:00
BuildData . Pcds [ key ] . Type = PcdInPlatform . Type
2018-05-10 16:47:44 +08:00
BuildData . Pcds [ key ] . Pending = False
2018-05-14 09:54:46 +08:00
else :
#Pcd used in Library, Pcd Type from reference module if Pcd Type is Pending
if BuildData . Pcds [ key ] . Pending :
MGen = ModuleAutoGen ( self , BuildData . MetaFile , Target , Toolchain , Arch , self . MetaFile )
if MGen and MGen . IsLibrary :
if MGen in PGen . LibraryAutoGenList :
2018-08-03 23:11:06 +08:00
ReferenceModules = MGen . ReferenceModules
2018-05-14 09:54:46 +08:00
for ReferenceModule in ReferenceModules :
if ReferenceModule . MetaFile in Platform . Modules :
RefPlatformModule = Platform . Modules [ str ( ReferenceModule . MetaFile ) ]
if key in RefPlatformModule . Pcds :
PcdInReferenceModule = RefPlatformModule . Pcds [ key ]
if PcdInReferenceModule . Type :
BuildData . Pcds [ key ] . Type = PcdInReferenceModule . Type
BuildData . Pcds [ key ] . Pending = False
break
2016-04-12 10:31:55 +08:00
2018-04-20 23:51:25 +08:00
if TAB_PCDS_DYNAMIC_EX in BuildData . Pcds [ key ] . Type :
2016-04-12 10:31:55 +08:00
if BuildData . IsBinaryModule :
2018-04-20 23:51:25 +08:00
BinaryPcdDict [ TAB_PCDS_DYNAMIC_EX ] . add ( ( BuildData . Pcds [ key ] . TokenCName , BuildData . Pcds [ key ] . TokenSpaceGuidCName ) )
2016-04-12 10:31:55 +08:00
else :
2018-04-20 23:51:25 +08:00
SourcePcdDict [ TAB_PCDS_DYNAMIC_EX ] . add ( ( BuildData . Pcds [ key ] . TokenCName , BuildData . Pcds [ key ] . TokenSpaceGuidCName ) )
2016-04-12 10:31:55 +08:00
2018-04-20 23:51:25 +08:00
elif TAB_PCDS_PATCHABLE_IN_MODULE in BuildData . Pcds [ key ] . Type :
2016-04-12 10:31:55 +08:00
if BuildData . MetaFile . Ext == ' .inf ' :
if BuildData . IsBinaryModule :
2018-04-20 23:51:25 +08:00
BinaryPcdDict [ TAB_PCDS_PATCHABLE_IN_MODULE ] . add ( ( BuildData . Pcds [ key ] . TokenCName , BuildData . Pcds [ key ] . TokenSpaceGuidCName ) )
2016-04-12 10:31:55 +08:00
else :
2018-04-20 23:51:25 +08:00
SourcePcdDict [ TAB_PCDS_PATCHABLE_IN_MODULE ] . add ( ( BuildData . Pcds [ key ] . TokenCName , BuildData . Pcds [ key ] . TokenSpaceGuidCName ) )
2016-04-12 10:31:55 +08:00
2018-04-20 23:51:25 +08:00
elif TAB_PCDS_DYNAMIC in BuildData . Pcds [ key ] . Type :
SourcePcdDict [ TAB_PCDS_DYNAMIC ] . add ( ( BuildData . Pcds [ key ] . TokenCName , BuildData . Pcds [ key ] . TokenSpaceGuidCName ) )
elif TAB_PCDS_FIXED_AT_BUILD in BuildData . Pcds [ key ] . Type :
SourcePcdDict [ TAB_PCDS_FIXED_AT_BUILD ] . add ( ( BuildData . Pcds [ key ] . TokenCName , BuildData . Pcds [ key ] . TokenSpaceGuidCName ) )
2016-04-12 10:31:55 +08:00
else :
pass
2017-05-03 15:19:21 +08:00
#
# A PCD can only use one type for all source modules
#
for i in SourcePcdDict_Keys :
for j in SourcePcdDict_Keys :
if i != j :
2018-04-05 04:56:56 +08:00
Intersections = SourcePcdDict [ i ] . intersection ( SourcePcdDict [ j ] )
if len ( Intersections ) > 0 :
2017-05-03 15:19:21 +08:00
EdkLogger . error (
' build ' ,
FORMAT_INVALID ,
" Building modules from source INFs, following PCD use %s and %s access method. It must be corrected to use only one access method. " % ( i , j ) ,
2018-05-18 08:06:52 +08:00
ExtraData = ' \n \t ' . join ( str ( P [ 1 ] + ' . ' + P [ 0 ] ) for P in Intersections )
2017-05-03 15:19:21 +08:00
)
2016-04-12 10:31:55 +08:00
#
# intersection the BinaryPCD for Mixed PCD
#
for i in BinaryPcdDict_Keys :
for j in BinaryPcdDict_Keys :
if i != j :
2018-04-05 04:56:56 +08:00
Intersections = BinaryPcdDict [ i ] . intersection ( BinaryPcdDict [ j ] )
for item in Intersections :
2016-04-12 10:31:55 +08:00
NewPcd1 = ( item [ 0 ] + ' _ ' + i , item [ 1 ] )
NewPcd2 = ( item [ 0 ] + ' _ ' + j , item [ 1 ] )
if item not in GlobalData . MixedPcd :
GlobalData . MixedPcd [ item ] = [ NewPcd1 , NewPcd2 ]
else :
if NewPcd1 not in GlobalData . MixedPcd [ item ] :
GlobalData . MixedPcd [ item ] . append ( NewPcd1 )
if NewPcd2 not in GlobalData . MixedPcd [ item ] :
GlobalData . MixedPcd [ item ] . append ( NewPcd2 )
#
# intersection the SourcePCD and BinaryPCD for Mixed PCD
#
for i in SourcePcdDict_Keys :
for j in BinaryPcdDict_Keys :
if i != j :
2018-04-05 04:56:56 +08:00
Intersections = SourcePcdDict [ i ] . intersection ( BinaryPcdDict [ j ] )
for item in Intersections :
2016-04-12 10:31:55 +08:00
NewPcd1 = ( item [ 0 ] + ' _ ' + i , item [ 1 ] )
NewPcd2 = ( item [ 0 ] + ' _ ' + j , item [ 1 ] )
if item not in GlobalData . MixedPcd :
GlobalData . MixedPcd [ item ] = [ NewPcd1 , NewPcd2 ]
else :
if NewPcd1 not in GlobalData . MixedPcd [ item ] :
GlobalData . MixedPcd [ item ] . append ( NewPcd1 )
if NewPcd2 not in GlobalData . MixedPcd [ item ] :
GlobalData . MixedPcd [ item ] . append ( NewPcd2 )
for BuildData in PGen . BuildDatabase . _CACHE_ . values ( ) :
if BuildData . Arch != Arch :
continue
for key in BuildData . Pcds :
for SinglePcd in GlobalData . MixedPcd :
if ( BuildData . Pcds [ key ] . TokenCName , BuildData . Pcds [ key ] . TokenSpaceGuidCName ) == SinglePcd :
for item in GlobalData . MixedPcd [ SinglePcd ] :
Pcd_Type = item [ 0 ] . split ( ' _ ' ) [ - 1 ]
2018-04-20 23:51:37 +08:00
if ( Pcd_Type == BuildData . Pcds [ key ] . Type ) or ( Pcd_Type == TAB_PCDS_DYNAMIC_EX and BuildData . Pcds [ key ] . Type in PCD_DYNAMIC_EX_TYPE_SET ) or \
( Pcd_Type == TAB_PCDS_DYNAMIC and BuildData . Pcds [ key ] . Type in PCD_DYNAMIC_TYPE_SET ) :
2016-04-12 10:31:55 +08:00
Value = BuildData . Pcds [ key ]
Value . TokenCName = BuildData . Pcds [ key ] . TokenCName + ' _ ' + Pcd_Type
if len ( key ) == 2 :
newkey = ( Value . TokenCName , key [ 1 ] )
elif len ( key ) == 3 :
newkey = ( Value . TokenCName , key [ 1 ] , key [ 2 ] )
del BuildData . Pcds [ key ]
BuildData . Pcds [ newkey ] = Value
break
break
# handle the mixed pcd in FDF file
for key in PcdSet :
if key in GlobalData . MixedPcd :
Value = PcdSet [ key ]
del PcdSet [ key ]
for item in GlobalData . MixedPcd [ key ] :
PcdSet [ item ] = Value
2014-08-28 13:53:34 +00:00
#Collect package set information from INF of FDF
2014-08-26 02:52:24 +00:00
PkgSet = set ( )
for Inf in ModuleList :
ModuleFile = PathClass ( NormPath ( Inf ) , GlobalData . gWorkspace , Arch )
if ModuleFile in Platform . Modules :
continue
ModuleData = self . BuildDatabase [ ModuleFile , Arch , Target , Toolchain ]
PkgSet . update ( ModuleData . Packages )
Pkgs = list ( PkgSet ) + list ( PGen . PackageList )
2018-04-06 07:13:53 +08:00
DecPcds = set ( )
2018-01-31 16:49:14 +08:00
DecPcdsKey = set ( )
2012-04-10 07:18:20 +00:00
for Pkg in Pkgs :
2012-05-23 08:27:14 +00:00
for Pcd in Pkg . Pcds :
2018-04-06 07:13:53 +08:00
DecPcds . add ( ( Pcd [ 0 ] , Pcd [ 1 ] ) )
2013-08-23 02:18:16 +00:00
DecPcdsKey . add ( ( Pcd [ 0 ] , Pcd [ 1 ] , Pcd [ 2 ] ) )
2012-04-10 07:18:20 +00:00
2010-02-28 23:39:39 +00:00
Platform . SkuName = self . SkuId
2018-06-22 17:14:13 +08:00
for Name , Guid , Fileds in PcdSet :
2012-04-10 07:18:20 +00:00
if ( Name , Guid ) not in DecPcds :
EdkLogger . error (
' build ' ,
PARSER_ERROR ,
" PCD ( %s . %s ) used in FDF is not declared in DEC files. " % ( Guid , Name ) ,
File = self . FdfProfile . PcdFileLineDict [ Name , Guid ] [ 0 ] ,
Line = self . FdfProfile . PcdFileLineDict [ Name , Guid ] [ 1 ]
)
2013-08-23 02:18:16 +00:00
else :
# Check whether Dynamic or DynamicEx PCD used in FDF file. If used, build break and give a error message.
if ( Name , Guid , TAB_PCDS_FIXED_AT_BUILD ) in DecPcdsKey \
or ( Name , Guid , TAB_PCDS_PATCHABLE_IN_MODULE ) in DecPcdsKey \
or ( Name , Guid , TAB_PCDS_FEATURE_FLAG ) in DecPcdsKey :
continue
elif ( Name , Guid , TAB_PCDS_DYNAMIC ) in DecPcdsKey or ( Name , Guid , TAB_PCDS_DYNAMIC_EX ) in DecPcdsKey :
EdkLogger . error (
' build ' ,
PARSER_ERROR ,
" Using Dynamic or DynamicEx type of PCD [ %s . %s ] in FDF file is not allowed. " % ( Guid , Name ) ,
File = self . FdfProfile . PcdFileLineDict [ Name , Guid ] [ 0 ] ,
Line = self . FdfProfile . PcdFileLineDict [ Name , Guid ] [ 1 ]
)
2010-02-28 23:39:39 +00:00
Pa = PlatformAutoGen ( self , self . MetaFile , Target , Toolchain , Arch )
#
# Explicitly collect platform's dynamic PCDs
#
Pa . CollectPlatformDynamicPcds ( )
2014-01-10 05:25:50 +00:00
Pa . CollectFixedAtBuildPcds ( )
2010-02-28 23:39:39 +00:00
self . AutoGenObjectList . append ( Pa )
2015-12-01 04:22:16 +00:00
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
#
# Generate Package level hash value
#
GlobalData . gPackageHash [ Arch ] = { }
if GlobalData . gUseHashCache :
for Pkg in Pkgs :
self . _GenPkgLevelHash ( Pkg )
2010-11-15 02:51:34 +00:00
#
# Check PCDs token value conflict in each DEC file.
#
self . _CheckAllPcdsTokenValueConflict ( )
2015-12-01 04:22:16 +00:00
2011-08-26 07:46:26 +00:00
#
# Check PCD type and definition between DSC and DEC
#
self . _CheckPcdDefineAndType ( )
2014-08-28 13:53:34 +00:00
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
#
2017-05-23 17:02:05 +08:00
# Create BuildOptions Macro & PCD metafile, also add the Active Platform and FDF file.
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
#
content = ' gCommandLineDefines: '
content + = str ( GlobalData . gCommandLineDefines )
content + = os . linesep
content + = ' BuildOptionPcd: '
content + = str ( GlobalData . BuildOptionPcd )
2017-05-23 17:02:05 +08:00
content + = os . linesep
content + = ' Active Platform: '
content + = str ( self . Platform )
content + = os . linesep
if self . FdfFile :
content + = ' Flash Image Definition: '
content + = str ( self . FdfFile )
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
content + = os . linesep
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
SaveFileOnChange ( os . path . join ( self . BuildDir , ' BuildOptions ' ) , content , False )
2017-05-22 22:01:26 +08:00
#
# Create PcdToken Number file for Dynamic/DynamicEx Pcd.
#
PcdTokenNumber = ' PcdTokenNumber: '
if Pa . PcdTokenNumber :
if Pa . DynamicPcdList :
for Pcd in Pa . DynamicPcdList :
PcdTokenNumber + = os . linesep
PcdTokenNumber + = str ( ( Pcd . TokenCName , Pcd . TokenSpaceGuidCName ) )
PcdTokenNumber + = ' : '
PcdTokenNumber + = str ( Pa . PcdTokenNumber [ Pcd . TokenCName , Pcd . TokenSpaceGuidCName ] )
SaveFileOnChange ( os . path . join ( self . BuildDir , ' PcdTokenNumber ' ) , PcdTokenNumber , False )
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
#
# Get set of workspace metafiles
#
AllWorkSpaceMetaFiles = self . _GetMetaFiles ( Target , Toolchain , Arch )
#
# Retrieve latest modified time of all metafiles
#
SrcTimeStamp = 0
for f in AllWorkSpaceMetaFiles :
if os . stat ( f ) [ 8 ] > SrcTimeStamp :
SrcTimeStamp = os . stat ( f ) [ 8 ]
self . _SrcTimeStamp = SrcTimeStamp
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
if GlobalData . gUseHashCache :
m = hashlib . md5 ( )
for files in AllWorkSpaceMetaFiles :
if files . endswith ( ' .dec ' ) :
continue
f = open ( files , ' r ' )
Content = f . read ( )
f . close ( )
m . update ( Content )
SaveFileOnChange ( os . path . join ( self . BuildDir , ' AutoGen.hash ' ) , m . hexdigest ( ) , True )
GlobalData . gPlatformHash = m . hexdigest ( )
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
#
# Write metafile list to build directory
#
AutoGenFilePath = os . path . join ( self . BuildDir , ' AutoGen ' )
if os . path . exists ( AutoGenFilePath ) :
os . remove ( AutoGenFilePath )
if not os . path . exists ( self . BuildDir ) :
os . makedirs ( self . BuildDir )
with open ( os . path . join ( self . BuildDir , ' AutoGen ' ) , ' w+ ' ) as file :
for f in AllWorkSpaceMetaFiles :
2018-06-25 18:31:26 +08:00
print ( f , file = file )
2010-02-28 23:39:39 +00:00
return True
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
def _GenPkgLevelHash ( self , Pkg ) :
2018-05-09 17:03:23 +08:00
if Pkg . PackageName in GlobalData . gPackageHash [ Pkg . Arch ] :
return
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
PkgDir = os . path . join ( self . BuildDir , Pkg . Arch , Pkg . PackageName )
CreateDirectory ( PkgDir )
HashFile = os . path . join ( PkgDir , Pkg . PackageName + ' .hash ' )
m = hashlib . md5 ( )
# Get .dec file's hash value
f = open ( Pkg . MetaFile . Path , ' r ' )
Content = f . read ( )
f . close ( )
m . update ( Content )
# Get include files hash value
if Pkg . Includes :
2018-05-09 17:03:23 +08:00
for inc in sorted ( Pkg . Includes , key = lambda x : str ( x ) ) :
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
for Root , Dirs , Files in os . walk ( str ( inc ) ) :
2018-05-09 17:03:23 +08:00
for File in sorted ( Files ) :
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
File_Path = os . path . join ( Root , File )
f = open ( File_Path , ' r ' )
Content = f . read ( )
f . close ( )
m . update ( Content )
SaveFileOnChange ( HashFile , m . hexdigest ( ) , True )
2018-05-09 17:03:23 +08:00
GlobalData . gPackageHash [ Pkg . Arch ] [ Pkg . PackageName ] = m . hexdigest ( )
2016-02-25 16:13:31 +08:00
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
def _GetMetaFiles ( self , Target , Toolchain , Arch ) :
AllWorkSpaceMetaFiles = set ( )
#
# add fdf
#
if self . FdfFile :
AllWorkSpaceMetaFiles . add ( self . FdfFile . Path )
2018-05-18 08:06:52 +08:00
for f in GlobalData . gFdfParser . GetAllIncludedFile ( ) :
AllWorkSpaceMetaFiles . add ( f . FileName )
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
#
# add dsc
#
AllWorkSpaceMetaFiles . add ( self . MetaFile . Path )
#
2017-04-11 13:20:08 +08:00
# add build_rule.txt & tools_def.txt
#
2017-04-17 17:12:06 +08:00
AllWorkSpaceMetaFiles . add ( os . path . join ( GlobalData . gConfDirectory , gDefaultBuildRuleFile ) )
AllWorkSpaceMetaFiles . add ( os . path . join ( GlobalData . gConfDirectory , gDefaultToolsDefFile ) )
2017-04-11 13:20:08 +08:00
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
# add BuildOption metafile
#
AllWorkSpaceMetaFiles . add ( os . path . join ( self . BuildDir , ' BuildOptions ' ) )
2017-05-22 22:01:26 +08:00
# add PcdToken Number file for Dynamic/DynamicEx Pcd
#
AllWorkSpaceMetaFiles . add ( os . path . join ( self . BuildDir , ' PcdTokenNumber ' ) )
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
for Arch in self . ArchList :
#
# add dec
#
2018-05-18 08:06:52 +08:00
for Package in PlatformAutoGen ( self , self . MetaFile , Target , Toolchain , Arch ) . PackageList :
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
AllWorkSpaceMetaFiles . add ( Package . MetaFile . Path )
#
# add included dsc
#
2018-05-18 08:06:52 +08:00
for filePath in self . BuildDatabase [ self . MetaFile , Arch , Target , Toolchain ] . _RawData . IncludedFiles :
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
AllWorkSpaceMetaFiles . add ( filePath . Path )
return AllWorkSpaceMetaFiles
2011-08-26 07:46:26 +00:00
def _CheckPcdDefineAndType ( self ) :
2018-05-18 08:06:52 +08:00
PcdTypeSet = { TAB_PCDS_FIXED_AT_BUILD ,
TAB_PCDS_PATCHABLE_IN_MODULE ,
TAB_PCDS_FEATURE_FLAG ,
TAB_PCDS_DYNAMIC ,
TAB_PCDS_DYNAMIC_EX }
2011-08-26 07:46:26 +00:00
# This dict store PCDs which are not used by any modules with specified arches
2018-04-04 05:03:05 +08:00
UnusedPcd = OrderedDict ( )
2011-08-26 07:46:26 +00:00
for Pa in self . AutoGenObjectList :
# Key of DSC's Pcds dictionary is PcdCName, TokenSpaceGuid
for Pcd in Pa . Platform . Pcds :
PcdType = Pa . Platform . Pcds [ Pcd ] . Type
2015-12-01 04:22:16 +00:00
2018-07-05 17:40:04 +08:00
# If no PCD type, this PCD comes from FDF
2011-08-26 07:46:26 +00:00
if not PcdType :
continue
2015-12-01 04:22:16 +00:00
2011-08-26 07:46:26 +00:00
# Try to remove Hii and Vpd suffix
2018-04-20 23:51:25 +08:00
if PcdType . startswith ( TAB_PCDS_DYNAMIC_EX ) :
PcdType = TAB_PCDS_DYNAMIC_EX
elif PcdType . startswith ( TAB_PCDS_DYNAMIC ) :
PcdType = TAB_PCDS_DYNAMIC
2015-12-01 04:22:16 +00:00
2011-08-26 07:46:26 +00:00
for Package in Pa . PackageList :
# Key of DEC's Pcds dictionary is PcdCName, TokenSpaceGuid, PcdType
if ( Pcd [ 0 ] , Pcd [ 1 ] , PcdType ) in Package . Pcds :
break
2018-05-18 08:06:52 +08:00
for Type in PcdTypeSet :
2011-08-26 07:46:26 +00:00
if ( Pcd [ 0 ] , Pcd [ 1 ] , Type ) in Package . Pcds :
EdkLogger . error (
' build ' ,
FORMAT_INVALID ,
" Type [ %s ] of PCD [ %s . %s ] in DSC file doesn ' t match the type [ %s ] defined in DEC file. " \
% ( Pa . Platform . Pcds [ Pcd ] . Type , Pcd [ 1 ] , Pcd [ 0 ] , Type ) ,
ExtraData = None
)
return
else :
UnusedPcd . setdefault ( Pcd , [ ] ) . append ( Pa . Arch )
for Pcd in UnusedPcd :
EdkLogger . warn (
' build ' ,
" The PCD was not specified by any INF module in the platform for the given architecture. \n "
" \t PCD: [ %s . %s ] \n \t Platform: [ %s ] \n \t Arch: %s "
% ( Pcd [ 1 ] , Pcd [ 0 ] , os . path . basename ( str ( self . MetaFile ) ) , str ( UnusedPcd [ Pcd ] ) ) ,
ExtraData = None
)
2010-02-28 23:39:39 +00:00
def __repr__ ( self ) :
return " %s [ %s ] " % ( self . MetaFile , " , " . join ( self . ArchList ) )
## Return the directory to store FV files
def _GetFvDir ( self ) :
2018-03-27 04:25:43 +08:00
if self . _FvDir is None :
2018-04-27 00:57:56 +08:00
self . _FvDir = path . join ( self . BuildDir , TAB_FV_DIRECTORY )
2010-02-28 23:39:39 +00:00
return self . _FvDir
## Return the directory to store all intermediate and final files built
def _GetBuildDir ( self ) :
2018-03-27 04:25:43 +08:00
if self . _BuildDir is None :
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
return self . AutoGenObjectList [ 0 ] . BuildDir
2010-02-28 23:39:39 +00:00
## Return the build output directory platform specifies
def _GetOutputDir ( self ) :
return self . Platform . OutputDirectory
## Return platform name
def _GetName ( self ) :
return self . Platform . PlatformName
## Return meta-file GUID
def _GetGuid ( self ) :
return self . Platform . Guid
## Return platform version
def _GetVersion ( self ) :
return self . Platform . Version
## Return paths of tools
def _GetToolDefinition ( self ) :
return self . AutoGenObjectList [ 0 ] . ToolDefinition
## Return directory of platform makefile
#
# @retval string Makefile directory
#
def _GetMakeFileDir ( self ) :
2018-03-27 04:25:43 +08:00
if self . _MakeFileDir is None :
2010-02-28 23:39:39 +00:00
self . _MakeFileDir = self . BuildDir
return self . _MakeFileDir
## Return build command string
#
# @retval string Build command string
#
def _GetBuildCommand ( self ) :
2018-03-27 04:25:43 +08:00
if self . _BuildCommand is None :
2010-02-28 23:39:39 +00:00
# BuildCommand should be all the same. So just get one from platform AutoGen
self . _BuildCommand = self . AutoGenObjectList [ 0 ] . BuildCommand
return self . _BuildCommand
2015-12-01 04:22:16 +00:00
2010-11-15 02:51:34 +00:00
## Check the PCDs token value conflict in each DEC file.
#
# Will cause build break and raise error message while two PCDs conflict.
2018-07-05 17:40:04 +08:00
#
2010-11-15 02:51:34 +00:00
# @return None
#
def _CheckAllPcdsTokenValueConflict ( self ) :
2011-09-18 12:17:25 +00:00
for Pa in self . AutoGenObjectList :
for Package in Pa . PackageList :
2010-11-15 02:51:34 +00:00
PcdList = Package . Pcds . values ( )
2018-07-05 17:40:04 +08:00
PcdList . sort ( lambda x , y : cmp ( int ( x . TokenValue , 0 ) , int ( y . TokenValue , 0 ) ) )
2010-11-15 02:51:34 +00:00
Count = 0
while ( Count < len ( PcdList ) - 1 ) :
Item = PcdList [ Count ]
ItemNext = PcdList [ Count + 1 ]
#
# Make sure in the same token space the TokenValue should be unique
#
2015-06-23 07:02:03 +00:00
if ( int ( Item . TokenValue , 0 ) == int ( ItemNext . TokenValue , 0 ) ) :
2010-11-15 02:51:34 +00:00
SameTokenValuePcdList = [ ]
SameTokenValuePcdList . append ( Item )
SameTokenValuePcdList . append ( ItemNext )
RemainPcdListLength = len ( PcdList ) - Count - 2
for ValueSameCount in range ( RemainPcdListLength ) :
2015-06-23 07:02:03 +00:00
if int ( PcdList [ len ( PcdList ) - RemainPcdListLength + ValueSameCount ] . TokenValue , 0 ) == int ( Item . TokenValue , 0 ) :
2010-11-15 02:51:34 +00:00
SameTokenValuePcdList . append ( PcdList [ len ( PcdList ) - RemainPcdListLength + ValueSameCount ] )
else :
break ;
#
# Sort same token value PCD list with TokenGuid and TokenCName
#
2015-12-01 04:22:16 +00:00
SameTokenValuePcdList . sort ( lambda x , y : cmp ( " %s . %s " % ( x . TokenSpaceGuidCName , x . TokenCName ) , " %s . %s " % ( y . TokenSpaceGuidCName , y . TokenCName ) ) )
SameTokenValuePcdListCount = 0
2010-11-15 02:51:34 +00:00
while ( SameTokenValuePcdListCount < len ( SameTokenValuePcdList ) - 1 ) :
2016-04-12 10:31:55 +08:00
Flag = False
2015-12-01 04:22:16 +00:00
TemListItem = SameTokenValuePcdList [ SameTokenValuePcdListCount ]
TemListItemNext = SameTokenValuePcdList [ SameTokenValuePcdListCount + 1 ]
2010-11-15 02:51:34 +00:00
if ( TemListItem . TokenSpaceGuidCName == TemListItemNext . TokenSpaceGuidCName ) and ( TemListItem . TokenCName != TemListItemNext . TokenCName ) :
2016-04-12 10:31:55 +08:00
for PcdItem in GlobalData . MixedPcd :
if ( TemListItem . TokenCName , TemListItem . TokenSpaceGuidCName ) in GlobalData . MixedPcd [ PcdItem ] or \
( TemListItemNext . TokenCName , TemListItemNext . TokenSpaceGuidCName ) in GlobalData . MixedPcd [ PcdItem ] :
Flag = True
if not Flag :
EdkLogger . error (
' build ' ,
FORMAT_INVALID ,
" The TokenValue [ %s ] of PCD [ %s . %s ] is conflict with: [ %s . %s ] in %s " \
% ( TemListItem . TokenValue , TemListItem . TokenSpaceGuidCName , TemListItem . TokenCName , TemListItemNext . TokenSpaceGuidCName , TemListItemNext . TokenCName , Package ) ,
ExtraData = None
)
2010-11-15 02:51:34 +00:00
SameTokenValuePcdListCount + = 1
Count + = SameTokenValuePcdListCount
Count + = 1
2015-12-01 04:22:16 +00:00
2010-11-15 02:51:34 +00:00
PcdList = Package . Pcds . values ( )
2015-12-01 04:22:16 +00:00
PcdList . sort ( lambda x , y : cmp ( " %s . %s " % ( x . TokenSpaceGuidCName , x . TokenCName ) , " %s . %s " % ( y . TokenSpaceGuidCName , y . TokenCName ) ) )
2010-11-15 02:51:34 +00:00
Count = 0
while ( Count < len ( PcdList ) - 1 ) :
Item = PcdList [ Count ]
2015-12-01 04:22:16 +00:00
ItemNext = PcdList [ Count + 1 ]
2010-11-15 02:51:34 +00:00
#
# Check PCDs with same TokenSpaceGuidCName.TokenCName have same token value as well.
#
2015-06-23 07:02:03 +00:00
if ( Item . TokenSpaceGuidCName == ItemNext . TokenSpaceGuidCName ) and ( Item . TokenCName == ItemNext . TokenCName ) and ( int ( Item . TokenValue , 0 ) != int ( ItemNext . TokenValue , 0 ) ) :
2010-11-15 02:51:34 +00:00
EdkLogger . error (
' build ' ,
FORMAT_INVALID ,
" The TokenValue [ %s ] of PCD [ %s . %s ] in %s defined in two places should be same as well. " \
% ( Item . TokenValue , Item . TokenSpaceGuidCName , Item . TokenCName , Package ) ,
ExtraData = None
)
Count + = 1
2014-08-28 13:53:34 +00:00
## Generate fds command
2014-08-18 04:59:01 +00:00
def _GenFdsCommand ( self ) :
return ( GenMake . TopLevelMakefile ( self ) . _TEMPLATE_ . Replace ( GenMake . TopLevelMakefile ( self ) . _TemplateDict ) ) . strip ( )
2010-02-28 23:39:39 +00:00
2010-09-06 01:58:00 +00:00
## Create makefile for the platform and modules in it
2010-02-28 23:39:39 +00:00
#
# @param CreateDepsMakeFile Flag indicating if the makefile for
# modules will be created as well
#
def CreateMakeFile ( self , CreateDepsMakeFile = False ) :
2018-05-18 08:06:52 +08:00
if not CreateDepsMakeFile :
return
for Pa in self . AutoGenObjectList :
Pa . CreateMakeFile ( True )
2010-02-28 23:39:39 +00:00
## Create autogen code for platform and modules
#
# Since there's no autogen code for platform, this method will do nothing
# if CreateModuleCodeFile is set to False.
#
# @param CreateDepsCodeFile Flag indicating if creating module's
# autogen code file or not
#
def CreateCodeFile ( self , CreateDepsCodeFile = False ) :
if not CreateDepsCodeFile :
return
for Pa in self . AutoGenObjectList :
2018-05-18 08:06:52 +08:00
Pa . CreateCodeFile ( True )
2010-02-28 23:39:39 +00:00
2011-05-26 07:35:18 +00:00
## Create AsBuilt INF file the platform
#
def CreateAsBuiltInf ( self ) :
return
2010-02-28 23:39:39 +00:00
Name = property ( _GetName )
Guid = property ( _GetGuid )
Version = property ( _GetVersion )
OutputDir = property ( _GetOutputDir )
ToolDefinition = property ( _GetToolDefinition ) # toolcode : tool path
BuildDir = property ( _GetBuildDir )
FvDir = property ( _GetFvDir )
MakeFileDir = property ( _GetMakeFileDir )
BuildCommand = property ( _GetBuildCommand )
2014-08-18 04:59:01 +00:00
GenFdsCommand = property ( _GenFdsCommand )
2010-02-28 23:39:39 +00:00
## AutoGen class for platform
#
# PlatformAutoGen class will process the original information in platform
# file in order to generate makefile for platform.
#
class PlatformAutoGen ( AutoGen ) :
2018-03-15 08:20:27 +08:00
# call super().__init__ then call the worker function with different parameter count
def __init__ ( self , Workspace , MetaFile , Target , Toolchain , Arch , * args , * * kwargs ) :
2018-08-03 23:11:06 +08:00
if not hasattr ( self , " _Init " ) :
2018-03-15 08:20:27 +08:00
super ( PlatformAutoGen , self ) . __init__ ( self , Workspace , MetaFile , Target , Toolchain , Arch , * args , * * kwargs )
self . _InitWorker ( Workspace , MetaFile , Target , Toolchain , Arch )
self . _Init = True
2010-02-28 23:39:39 +00:00
#
2018-07-05 17:40:04 +08:00
# Used to store all PCDs for both PEI and DXE phase, in order to generate
2010-02-28 23:39:39 +00:00
# correct PCD database
2018-07-05 17:40:04 +08:00
#
2010-02-28 23:39:39 +00:00
_DynaPcdList_ = [ ]
_NonDynaPcdList_ = [ ]
2016-04-19 10:43:57 +08:00
_PlatformPcds = { }
2018-07-05 17:40:04 +08:00
2010-11-15 02:51:34 +00:00
#
2018-07-05 17:40:04 +08:00
# The priority list while override build option
2010-11-15 02:51:34 +00:00
#
PrioList = { " 0x11111 " : 16 , # TARGET_TOOLCHAIN_ARCH_COMMANDTYPE_ATTRIBUTE (Highest)
" 0x01111 " : 15 , # ******_TOOLCHAIN_ARCH_COMMANDTYPE_ATTRIBUTE
" 0x10111 " : 14 , # TARGET_*********_ARCH_COMMANDTYPE_ATTRIBUTE
2018-07-05 17:40:04 +08:00
" 0x00111 " : 13 , # ******_*********_ARCH_COMMANDTYPE_ATTRIBUTE
2010-11-15 02:51:34 +00:00
" 0x11011 " : 12 , # TARGET_TOOLCHAIN_****_COMMANDTYPE_ATTRIBUTE
" 0x01011 " : 11 , # ******_TOOLCHAIN_****_COMMANDTYPE_ATTRIBUTE
" 0x10011 " : 10 , # TARGET_*********_****_COMMANDTYPE_ATTRIBUTE
" 0x00011 " : 9 , # ******_*********_****_COMMANDTYPE_ATTRIBUTE
" 0x11101 " : 8 , # TARGET_TOOLCHAIN_ARCH_***********_ATTRIBUTE
" 0x01101 " : 7 , # ******_TOOLCHAIN_ARCH_***********_ATTRIBUTE
" 0x10101 " : 6 , # TARGET_*********_ARCH_***********_ATTRIBUTE
" 0x00101 " : 5 , # ******_*********_ARCH_***********_ATTRIBUTE
" 0x11001 " : 4 , # TARGET_TOOLCHAIN_****_***********_ATTRIBUTE
" 0x01001 " : 3 , # ******_TOOLCHAIN_****_***********_ATTRIBUTE
" 0x10001 " : 2 , # TARGET_*********_****_***********_ATTRIBUTE
" 0x00001 " : 1 } # ******_*********_****_***********_ATTRIBUTE (Lowest)
2015-12-01 04:22:16 +00:00
2018-03-15 08:20:27 +08:00
## Initialize PlatformAutoGen
2010-02-28 23:39:39 +00:00
#
#
# @param Workspace WorkspaceAutoGen object
# @param PlatformFile Platform file (DSC file)
# @param Target Build target (DEBUG, RELEASE)
# @param Toolchain Name of tool chain
# @param Arch arch of the platform supports
#
2018-03-15 08:20:27 +08:00
def _InitWorker ( self , Workspace , PlatformFile , Target , Toolchain , Arch ) :
2010-02-28 23:39:39 +00:00
EdkLogger . debug ( EdkLogger . DEBUG_9 , " AutoGen platform [ %s ] [ %s ] " % ( PlatformFile , Arch ) )
GlobalData . gProcessingFile = " %s [ %s , %s , %s ] " % ( PlatformFile , Arch , Toolchain , Target )
self . MetaFile = PlatformFile
self . Workspace = Workspace
self . WorkspaceDir = Workspace . WorkspaceDir
self . ToolChain = Toolchain
self . BuildTarget = Target
self . Arch = Arch
self . SourceDir = PlatformFile . SubDir
self . SourceOverrideDir = None
self . FdTargetList = self . Workspace . FdTargetList
self . FvTargetList = self . Workspace . FvTargetList
self . AllPcdList = [ ]
2014-08-26 02:52:24 +00:00
# get the original module/package/platform objects
self . BuildDatabase = Workspace . BuildDatabase
2017-12-22 20:46:15 +08:00
self . DscBuildDataObj = Workspace . Platform
2018-01-27 00:28:05 +08:00
self . _GuidDict = Workspace . _GuidDict
2010-02-28 23:39:39 +00:00
# flag indicating if the makefile/C-code file has been created or not
self . IsMakeFileCreated = False
self . IsCodeFileCreated = False
self . _Platform = None
self . _Name = None
self . _Guid = None
self . _Version = None
self . _BuildRule = None
self . _SourceDir = None
self . _BuildDir = None
self . _OutputDir = None
self . _FvDir = None
self . _MakeFileDir = None
self . _FdfFile = None
self . _PcdTokenNumber = None # (TokenCName, TokenSpaceGuidCName) : GeneratedTokenNumber
self . _DynamicPcdList = None # [(TokenCName1, TokenSpaceGuidCName1), (TokenCName2, TokenSpaceGuidCName2), ...]
self . _NonDynamicPcdList = None # [(TokenCName1, TokenSpaceGuidCName1), (TokenCName2, TokenSpaceGuidCName2), ...]
2014-01-10 05:25:50 +00:00
self . _NonDynamicPcdDict = { }
2010-02-28 23:39:39 +00:00
self . _ToolDefinitions = None
self . _ToolDefFile = None # toolcode : tool path
self . _ToolChainFamily = None
self . _BuildRuleFamily = None
self . _BuildOption = None # toolcode : option
self . _EdkBuildOption = None # edktoolcode : option
self . _EdkIIBuildOption = None # edkiitoolcode : option
self . _PackageList = None
self . _ModuleAutoGenList = None
self . _LibraryAutoGenList = None
self . _BuildCommand = None
2014-08-26 02:52:24 +00:00
self . _AsBuildInfList = [ ]
self . _AsBuildModuleList = [ ]
2017-12-22 20:14:29 +08:00
self . VariableInfo = None
2018-03-27 04:25:43 +08:00
if GlobalData . gFdfParser is not None :
2014-08-26 02:52:24 +00:00
self . _AsBuildInfList = GlobalData . gFdfParser . Profile . InfList
for Inf in self . _AsBuildInfList :
InfClass = PathClass ( NormPath ( Inf ) , GlobalData . gWorkspace , self . Arch )
M = self . BuildDatabase [ InfClass , self . Arch , self . BuildTarget , self . ToolChain ]
if not M . IsSupportedArch :
continue
self . _AsBuildModuleList . append ( InfClass )
2014-08-18 04:59:01 +00:00
# get library/modules for build
self . LibraryBuildDirectoryList = [ ]
self . ModuleBuildDirectoryList = [ ]
2017-12-22 20:14:29 +08:00
2010-02-28 23:39:39 +00:00
return True
def __repr__ ( self ) :
return " %s [ %s ] " % ( self . MetaFile , self . Arch )
## Create autogen code for platform and modules
#
# Since there's no autogen code for platform, this method will do nothing
# if CreateModuleCodeFile is set to False.
#
# @param CreateModuleCodeFile Flag indicating if creating module's
# autogen code file or not
#
def CreateCodeFile ( self , CreateModuleCodeFile = False ) :
# only module has code to be greated, so do nothing if CreateModuleCodeFile is False
if self . IsCodeFileCreated or not CreateModuleCodeFile :
return
for Ma in self . ModuleAutoGenList :
Ma . CreateCodeFile ( True )
# don't do this twice
self . IsCodeFileCreated = True
2014-08-18 04:59:01 +00:00
## Generate Fds Command
def _GenFdsCommand ( self ) :
return self . Workspace . GenFdsCommand
2016-04-22 15:08:47 +08:00
2010-02-28 23:39:39 +00:00
## Create makefile for the platform and mdoules in it
#
# @param CreateModuleMakeFile Flag indicating if the makefile for
# modules will be created as well
#
2017-11-22 15:42:25 +08:00
def CreateMakeFile ( self , CreateModuleMakeFile = False , FfsCommand = { } ) :
2010-02-28 23:39:39 +00:00
if CreateModuleMakeFile :
for ModuleFile in self . Platform . Modules :
Ma = ModuleAutoGen ( self . Workspace , ModuleFile , self . BuildTarget ,
self . ToolChain , self . Arch , self . MetaFile )
2017-11-22 15:42:25 +08:00
if ( ModuleFile . File , self . Arch ) in FfsCommand :
Ma . CreateMakeFile ( True , FfsCommand [ ModuleFile . File , self . Arch ] )
else :
Ma . CreateMakeFile ( True )
2014-08-28 13:53:34 +00:00
#Ma.CreateAsBuiltInf()
2010-02-28 23:39:39 +00:00
# no need to create makefile for the platform more than once
if self . IsMakeFileCreated :
return
2014-08-18 04:59:01 +00:00
# create library/module build dirs for platform
2010-02-28 23:39:39 +00:00
Makefile = GenMake . PlatformMakefile ( self )
2014-08-18 04:59:01 +00:00
self . LibraryBuildDirectoryList = Makefile . GetLibraryBuildDirectoryList ( )
self . ModuleBuildDirectoryList = Makefile . GetModuleBuildDirectoryList ( )
2010-02-28 23:39:39 +00:00
self . IsMakeFileCreated = True
2014-01-10 05:25:50 +00:00
## Deal with Shared FixedAtBuild Pcds
#
def CollectFixedAtBuildPcds ( self ) :
for LibAuto in self . LibraryAutoGenList :
2018-07-05 17:40:04 +08:00
FixedAtBuildPcds = { }
ShareFixedAtBuildPcdsSameValue = { }
2018-08-03 23:11:06 +08:00
for Module in LibAuto . ReferenceModules :
2018-07-13 16:07:06 +08:00
for Pcd in set ( Module . FixedAtBuildPcds + LibAuto . FixedAtBuildPcds ) :
2018-07-09 20:14:38 +08:00
DefaultValue = Pcd . DefaultValue
# Cover the case: DSC component override the Pcd value and the Pcd only used in one Lib
if Pcd in Module . LibraryPcdList :
Index = Module . LibraryPcdList . index ( Pcd )
DefaultValue = Module . LibraryPcdList [ Index ] . DefaultValue
2018-06-25 18:31:33 +08:00
key = " . " . join ( ( Pcd . TokenSpaceGuidCName , Pcd . TokenCName ) )
2014-01-10 05:25:50 +00:00
if key not in FixedAtBuildPcds :
ShareFixedAtBuildPcdsSameValue [ key ] = True
2018-07-09 20:14:38 +08:00
FixedAtBuildPcds [ key ] = DefaultValue
2014-01-10 05:25:50 +00:00
else :
2018-07-09 20:14:38 +08:00
if FixedAtBuildPcds [ key ] != DefaultValue :
2018-07-05 17:40:04 +08:00
ShareFixedAtBuildPcdsSameValue [ key ] = False
2014-01-10 05:25:50 +00:00
for Pcd in LibAuto . FixedAtBuildPcds :
2018-06-25 18:31:33 +08:00
key = " . " . join ( ( Pcd . TokenSpaceGuidCName , Pcd . TokenCName ) )
if ( Pcd . TokenCName , Pcd . TokenSpaceGuidCName ) not in self . NonDynamicPcdDict :
2014-01-10 05:25:50 +00:00
continue
else :
2018-06-25 18:31:33 +08:00
DscPcd = self . NonDynamicPcdDict [ ( Pcd . TokenCName , Pcd . TokenSpaceGuidCName ) ]
2018-04-20 23:51:25 +08:00
if DscPcd . Type != TAB_PCDS_FIXED_AT_BUILD :
2014-01-10 05:25:50 +00:00
continue
2018-07-05 17:40:04 +08:00
if key in ShareFixedAtBuildPcdsSameValue and ShareFixedAtBuildPcdsSameValue [ key ] :
2018-03-14 16:51:04 +08:00
LibAuto . ConstPcd [ key ] = FixedAtBuildPcds [ key ]
2014-01-10 05:25:50 +00:00
2017-11-24 14:46:58 +08:00
def CollectVariables ( self , DynamicPcdSet ) :
2017-12-22 20:14:29 +08:00
VpdRegionSize = 0
VpdRegionBase = 0
if self . Workspace . FdfFile :
FdDict = self . Workspace . FdfProfile . FdDict [ GlobalData . gFdfParser . CurrentFdName ]
for FdRegion in FdDict . RegionList :
for item in FdRegion . RegionDataList :
if self . Platform . VpdToolGuid . strip ( ) and self . Platform . VpdToolGuid in item :
VpdRegionSize = FdRegion . Size
VpdRegionBase = FdRegion . Offset
break
2018-06-25 18:31:33 +08:00
VariableInfo = VariableMgr ( self . DscBuildDataObj . _GetDefaultStores ( ) , self . DscBuildDataObj . _GetSkuIds ( ) )
2017-12-22 20:14:29 +08:00
VariableInfo . SetVpdRegionMaxSize ( VpdRegionSize )
VariableInfo . SetVpdRegionOffset ( VpdRegionBase )
2017-11-24 14:46:58 +08:00
Index = 0
for Pcd in DynamicPcdSet :
2018-06-25 18:31:33 +08:00
pcdname = " . " . join ( ( Pcd . TokenSpaceGuidCName , Pcd . TokenCName ) )
2017-12-22 20:46:15 +08:00
for SkuName in Pcd . SkuInfoList :
Sku = Pcd . SkuInfoList [ SkuName ]
SkuId = Sku . SkuId
2018-03-27 04:25:43 +08:00
if SkuId is None or SkuId == ' ' :
2017-12-22 20:46:15 +08:00
continue
if len ( Sku . VariableName ) > 0 :
VariableGuidStructure = Sku . VariableGuidValue
VariableGuid = GuidStructureStringToGuidString ( VariableGuidStructure )
2018-01-02 08:58:33 +08:00
for StorageName in Sku . DefaultStoreDict :
2018-06-25 18:31:33 +08:00
VariableInfo . append_variable ( var_info ( Index , pcdname , StorageName , SkuName , StringToArray ( Sku . VariableName ) , VariableGuid , Sku . VariableOffset , Sku . VariableAttribute , Sku . HiiDefaultValue , Sku . DefaultStoreDict [ StorageName ] , Pcd . DatumType ) )
2017-11-24 14:46:58 +08:00
Index + = 1
return VariableInfo
2017-12-22 20:14:29 +08:00
2018-06-25 18:31:33 +08:00
def UpdateNVStoreMaxSize ( self , OrgVpdFile ) :
2017-12-22 20:23:21 +08:00
if self . VariableInfo :
2018-04-27 00:57:56 +08:00
VpdMapFilePath = os . path . join ( self . BuildDir , TAB_FV_DIRECTORY , " %s .map " % self . Platform . VpdToolGuid )
2017-12-22 20:23:21 +08:00
PcdNvStoreDfBuffer = [ item for item in self . _DynamicPcdList if item . TokenCName == " PcdNvStoreDefaultValueBuffer " and item . TokenSpaceGuidCName == " gEfiMdeModulePkgTokenSpaceGuid " ]
if PcdNvStoreDfBuffer :
if os . path . exists ( VpdMapFilePath ) :
OrgVpdFile . Read ( VpdMapFilePath )
PcdItems = OrgVpdFile . GetOffset ( PcdNvStoreDfBuffer [ 0 ] )
2017-12-19 16:26:16 +08:00
NvStoreOffset = PcdItems . values ( ) [ 0 ] . strip ( ) if PcdItems else ' 0 '
2017-12-22 20:23:21 +08:00
else :
EdkLogger . error ( " build " , FILE_READ_FAILURE , " Can not find VPD map file %s to fix up VPD offset. " % VpdMapFilePath )
2018-06-25 18:31:33 +08:00
NvStoreOffset = int ( NvStoreOffset , 16 ) if NvStoreOffset . upper ( ) . startswith ( " 0X " ) else int ( NvStoreOffset )
2018-04-16 21:52:13 +08:00
default_skuobj = PcdNvStoreDfBuffer [ 0 ] . SkuInfoList . get ( TAB_DEFAULT )
2017-12-22 20:23:21 +08:00
maxsize = self . VariableInfo . VpdRegionSize - NvStoreOffset if self . VariableInfo . VpdRegionSize else len ( default_skuobj . DefaultValue . split ( " , " ) )
var_data = self . VariableInfo . PatchNVStoreDefaultMaxSize ( maxsize )
if var_data and default_skuobj :
default_skuobj . DefaultValue = var_data
PcdNvStoreDfBuffer [ 0 ] . DefaultValue = var_data
PcdNvStoreDfBuffer [ 0 ] . SkuInfoList . clear ( )
2018-04-16 21:52:13 +08:00
PcdNvStoreDfBuffer [ 0 ] . SkuInfoList [ TAB_DEFAULT ] = default_skuobj
2017-12-22 20:23:21 +08:00
PcdNvStoreDfBuffer [ 0 ] . MaxDatumSize = str ( len ( default_skuobj . DefaultValue . split ( " , " ) ) )
2017-12-22 20:14:29 +08:00
return OrgVpdFile
2010-02-28 23:39:39 +00:00
## Collect dynamic PCDs
#
# Gather dynamic PCDs list from each module and their settings from platform
# This interface should be invoked explicitly when platform action is created.
#
def CollectPlatformDynamicPcds ( self ) :
2016-02-25 16:13:31 +08:00
2016-04-12 10:31:55 +08:00
for key in self . Platform . Pcds :
for SinglePcd in GlobalData . MixedPcd :
if ( self . Platform . Pcds [ key ] . TokenCName , self . Platform . Pcds [ key ] . TokenSpaceGuidCName ) == SinglePcd :
for item in GlobalData . MixedPcd [ SinglePcd ] :
Pcd_Type = item [ 0 ] . split ( ' _ ' ) [ - 1 ]
2018-04-20 23:51:37 +08:00
if ( Pcd_Type == self . Platform . Pcds [ key ] . Type ) or ( Pcd_Type == TAB_PCDS_DYNAMIC_EX and self . Platform . Pcds [ key ] . Type in PCD_DYNAMIC_EX_TYPE_SET ) or \
( Pcd_Type == TAB_PCDS_DYNAMIC and self . Platform . Pcds [ key ] . Type in PCD_DYNAMIC_TYPE_SET ) :
2016-04-12 10:31:55 +08:00
Value = self . Platform . Pcds [ key ]
Value . TokenCName = self . Platform . Pcds [ key ] . TokenCName + ' _ ' + Pcd_Type
if len ( key ) == 2 :
newkey = ( Value . TokenCName , key [ 1 ] )
elif len ( key ) == 3 :
newkey = ( Value . TokenCName , key [ 1 ] , key [ 2 ] )
del self . Platform . Pcds [ key ]
self . Platform . Pcds [ newkey ] = Value
break
break
2010-02-28 23:39:39 +00:00
# for gathering error information
NoDatumTypePcdList = set ( )
2014-08-26 02:52:24 +00:00
FdfModuleList = [ ]
for InfName in self . _AsBuildInfList :
2015-10-08 09:27:14 +00:00
InfName = mws . join ( self . WorkspaceDir , InfName )
2014-08-26 02:52:24 +00:00
FdfModuleList . append ( os . path . normpath ( InfName ) )
2010-02-28 23:39:39 +00:00
for F in self . Platform . Modules . keys ( ) :
M = ModuleAutoGen ( self . Workspace , F , self . BuildTarget , self . ToolChain , self . Arch , self . MetaFile )
#GuidValue.update(M.Guids)
2018-07-05 17:40:04 +08:00
2010-02-28 23:39:39 +00:00
self . Platform . Modules [ F ] . M = M
2015-12-01 04:22:16 +00:00
for PcdFromModule in M . ModulePcdList + M . LibraryPcdList :
2010-02-28 23:39:39 +00:00
# make sure that the "VOID*" kind of datum has MaxDatumSize set
2018-04-20 23:51:42 +08:00
if PcdFromModule . DatumType == TAB_VOID and not PcdFromModule . MaxDatumSize :
2010-02-28 23:39:39 +00:00
NoDatumTypePcdList . add ( " %s . %s [ %s ] " % ( PcdFromModule . TokenSpaceGuidCName , PcdFromModule . TokenCName , F ) )
2014-08-28 13:53:34 +00:00
# Check the PCD from Binary INF or Source INF
2014-08-26 02:52:24 +00:00
if M . IsBinaryModule == True :
PcdFromModule . IsFromBinaryInf = True
2014-08-28 13:53:34 +00:00
2018-07-05 17:40:04 +08:00
# Check the PCD from DSC or not
2018-04-17 22:40:15 +08:00
PcdFromModule . IsFromDsc = ( PcdFromModule . TokenCName , PcdFromModule . TokenSpaceGuidCName ) in self . Platform . Pcds
2018-04-20 23:51:37 +08:00
if PcdFromModule . Type in PCD_DYNAMIC_TYPE_SET or PcdFromModule . Type in PCD_DYNAMIC_EX_TYPE_SET :
2014-08-26 02:52:24 +00:00
if F . Path not in FdfModuleList :
2018-07-05 17:40:04 +08:00
# If one of the Source built modules listed in the DSC is not listed
# in FDF modules, and the INF lists a PCD can only use the PcdsDynamic
# access method (it is only listed in the DEC file that declares the
2014-08-26 02:52:24 +00:00
# PCD as PcdsDynamic), then build tool will report warning message
2018-07-05 17:40:04 +08:00
# notify the PI that they are attempting to build a module that must
# be included in a flash image in order to be functional. These Dynamic
# PCD will not be added into the Database unless it is used by other
2014-08-26 02:52:24 +00:00
# modules that are included in the FDF file.
2018-04-20 23:51:37 +08:00
if PcdFromModule . Type in PCD_DYNAMIC_TYPE_SET and \
2014-08-26 02:52:24 +00:00
PcdFromModule . IsFromBinaryInf == False :
# Print warning message to let the developer make a determine.
continue
2018-07-05 17:40:04 +08:00
# If one of the Source built modules listed in the DSC is not listed in
# FDF modules, and the INF lists a PCD can only use the PcdsDynamicEx
# access method (it is only listed in the DEC file that declares the
# PCD as PcdsDynamicEx), then DO NOT break the build; DO NOT add the
2014-08-26 02:52:24 +00:00
# PCD to the Platform's PCD Database.
2018-04-20 23:51:37 +08:00
if PcdFromModule . Type in PCD_DYNAMIC_EX_TYPE_SET :
2014-08-26 02:52:24 +00:00
continue
2010-02-28 23:39:39 +00:00
#
# If a dynamic PCD used by a PEM module/PEI module & DXE module,
# it should be stored in Pcd PEI database, If a dynamic only
# used by DXE module, it should be stored in DXE PCD database.
# The default Phase is DXE
#
2018-04-20 23:51:34 +08:00
if M . ModuleType in SUP_MODULE_SET_PEI :
2010-02-28 23:39:39 +00:00
PcdFromModule . Phase = " PEI "
if PcdFromModule not in self . _DynaPcdList_ :
self . _DynaPcdList_ . append ( PcdFromModule )
elif PcdFromModule . Phase == ' PEI ' :
# overwrite any the same PCD existing, if Phase is PEI
Index = self . _DynaPcdList_ . index ( PcdFromModule )
self . _DynaPcdList_ [ Index ] = PcdFromModule
elif PcdFromModule not in self . _NonDynaPcdList_ :
self . _NonDynaPcdList_ . append ( PcdFromModule )
2014-08-26 02:52:24 +00:00
elif PcdFromModule in self . _NonDynaPcdList_ and PcdFromModule . IsFromBinaryInf == True :
Index = self . _NonDynaPcdList_ . index ( PcdFromModule )
if self . _NonDynaPcdList_ [ Index ] . IsFromBinaryInf == False :
#The PCD from Binary INF will override the same one from source INF
self . _NonDynaPcdList_ . remove ( self . _NonDynaPcdList_ [ Index ] )
PcdFromModule . Pending = False
self . _NonDynaPcdList_ . append ( PcdFromModule )
2018-04-11 07:17:23 +08:00
DscModuleSet = { os . path . normpath ( ModuleInf . Path ) for ModuleInf in self . Platform . Modules }
2018-07-05 17:40:04 +08:00
# add the PCD from modules that listed in FDF but not in DSC to Database
2014-08-26 02:52:24 +00:00
for InfName in FdfModuleList :
2018-04-11 07:17:23 +08:00
if InfName not in DscModuleSet :
2014-08-26 02:52:24 +00:00
InfClass = PathClass ( InfName )
M = self . BuildDatabase [ InfClass , self . Arch , self . BuildTarget , self . ToolChain ]
2018-07-05 17:40:04 +08:00
# If a module INF in FDF but not in current arch's DSC module list, it must be module (either binary or source)
# for different Arch. PCDs in source module for different Arch is already added before, so skip the source module here.
# For binary module, if in current arch, we need to list the PCDs into database.
2014-08-26 02:52:24 +00:00
if not M . IsSupportedArch :
continue
# Override the module PCD setting by platform setting
ModulePcdList = self . ApplyPcdSetting ( M , M . Pcds )
for PcdFromModule in ModulePcdList :
PcdFromModule . IsFromBinaryInf = True
PcdFromModule . IsFromDsc = False
# Only allow the DynamicEx and Patchable PCD in AsBuild INF
2018-04-20 23:51:37 +08:00
if PcdFromModule . Type not in PCD_DYNAMIC_EX_TYPE_SET and PcdFromModule . Type not in TAB_PCDS_PATCHABLE_IN_MODULE :
2014-08-26 02:52:24 +00:00
EdkLogger . error ( " build " , AUTOGEN_ERROR , " PCD setting error " ,
File = self . MetaFile ,
ExtraData = " \n \t Existed %s PCD %s in: \n \t \t %s \n "
% ( PcdFromModule . Type , PcdFromModule . TokenCName , InfName ) )
# make sure that the "VOID*" kind of datum has MaxDatumSize set
2018-04-20 23:51:42 +08:00
if PcdFromModule . DatumType == TAB_VOID and not PcdFromModule . MaxDatumSize :
2014-08-26 02:52:24 +00:00
NoDatumTypePcdList . add ( " %s . %s [ %s ] " % ( PcdFromModule . TokenSpaceGuidCName , PcdFromModule . TokenCName , InfName ) )
2018-04-20 23:51:34 +08:00
if M . ModuleType in SUP_MODULE_SET_PEI :
2014-08-26 02:52:24 +00:00
PcdFromModule . Phase = " PEI "
2018-04-20 23:51:37 +08:00
if PcdFromModule not in self . _DynaPcdList_ and PcdFromModule . Type in PCD_DYNAMIC_EX_TYPE_SET :
2014-08-26 02:52:24 +00:00
self . _DynaPcdList_ . append ( PcdFromModule )
elif PcdFromModule not in self . _NonDynaPcdList_ and PcdFromModule . Type in TAB_PCDS_PATCHABLE_IN_MODULE :
self . _NonDynaPcdList_ . append ( PcdFromModule )
2018-04-20 23:51:37 +08:00
if PcdFromModule in self . _DynaPcdList_ and PcdFromModule . Phase == ' PEI ' and PcdFromModule . Type in PCD_DYNAMIC_EX_TYPE_SET :
2014-08-28 13:53:34 +00:00
# Overwrite the phase of any the same PCD existing, if Phase is PEI.
2018-07-05 17:40:04 +08:00
# It is to solve the case that a dynamic PCD used by a PEM module/PEI
2014-08-28 13:53:34 +00:00
# module & DXE module at a same time.
# Overwrite the type of the PCDs in source INF by the type of AsBuild
2018-07-05 17:40:04 +08:00
# INF file as DynamicEx.
2014-08-26 02:52:24 +00:00
Index = self . _DynaPcdList_ . index ( PcdFromModule )
self . _DynaPcdList_ [ Index ] . Phase = PcdFromModule . Phase
self . _DynaPcdList_ [ Index ] . Type = PcdFromModule . Type
for PcdFromModule in self . _NonDynaPcdList_ :
2018-07-05 17:40:04 +08:00
# If a PCD is not listed in the DSC file, but binary INF files used by
# this platform all (that use this PCD) list the PCD in a [PatchPcds]
# section, AND all source INF files used by this platform the build
# that use the PCD list the PCD in either a [Pcds] or [PatchPcds]
2014-08-26 02:52:24 +00:00
# section, then the tools must NOT add the PCD to the Platform's PCD
2018-07-05 17:40:04 +08:00
# Database; the build must assign the access method for this PCD as
2014-08-26 02:52:24 +00:00
# PcdsPatchableInModule.
if PcdFromModule not in self . _DynaPcdList_ :
continue
Index = self . _DynaPcdList_ . index ( PcdFromModule )
if PcdFromModule . IsFromDsc == False and \
PcdFromModule . Type in TAB_PCDS_PATCHABLE_IN_MODULE and \
PcdFromModule . IsFromBinaryInf == True and \
self . _DynaPcdList_ [ Index ] . IsFromBinaryInf == False :
Index = self . _DynaPcdList_ . index ( PcdFromModule )
self . _DynaPcdList_ . remove ( self . _DynaPcdList_ [ Index ] )
2010-02-28 23:39:39 +00:00
# print out error information and break the build, if error found
if len ( NoDatumTypePcdList ) > 0 :
NoDatumTypePcdListString = " \n \t \t " . join ( NoDatumTypePcdList )
EdkLogger . error ( " build " , AUTOGEN_ERROR , " PCD setting error " ,
File = self . MetaFile ,
ExtraData = " \n \t PCD(s) without MaxDatumSize: \n \t \t %s \n "
% NoDatumTypePcdListString )
self . _NonDynamicPcdList = self . _NonDynaPcdList_
self . _DynamicPcdList = self . _DynaPcdList_
#
# Sort dynamic PCD list to:
2018-07-05 17:40:04 +08:00
# 1) If PCD's datum type is VOID* and value is unicode string which starts with L, the PCD item should
2010-02-28 23:39:39 +00:00
# try to be put header of dynamicd List
# 2) If PCD is HII type, the PCD item should be put after unicode type PCD
#
# The reason of sorting is make sure the unicode string is in double-byte alignment in string table.
#
2017-12-22 20:23:21 +08:00
UnicodePcdArray = set ( )
HiiPcdArray = set ( )
OtherPcdArray = set ( )
2010-11-15 02:51:34 +00:00
VpdPcdDict = { }
2010-09-06 01:58:00 +00:00
VpdFile = VpdInfoFile . VpdInfoFile ( )
2016-04-19 10:43:57 +08:00
NeedProcessVpdMapFile = False
2018-04-17 22:40:15 +08:00
for pcd in self . Platform . Pcds :
if pcd not in self . _PlatformPcds :
2016-04-19 10:43:57 +08:00
self . _PlatformPcds [ pcd ] = self . Platform . Pcds [ pcd ]
2017-11-24 14:30:11 +08:00
for item in self . _PlatformPcds :
if self . _PlatformPcds [ item ] . DatumType and self . _PlatformPcds [ item ] . DatumType not in [ TAB_UINT8 , TAB_UINT16 , TAB_UINT32 , TAB_UINT64 , TAB_VOID , " BOOLEAN " ] :
2018-04-11 09:14:05 -07:00
self . _PlatformPcds [ item ] . DatumType = TAB_VOID
2017-11-24 14:30:11 +08:00
2018-07-05 17:40:04 +08:00
if ( self . Workspace . ArchList [ - 1 ] == self . Arch ) :
2010-09-06 01:58:00 +00:00
for Pcd in self . _DynamicPcdList :
# just pick the a value to determine whether is unicode string type
2018-04-17 22:40:15 +08:00
Sku = Pcd . SkuInfoList . values ( ) [ 0 ]
2010-09-06 01:58:00 +00:00
Sku . VpdOffset = Sku . VpdOffset . strip ( )
2015-12-01 04:22:16 +00:00
2017-11-24 14:30:11 +08:00
if Pcd . DatumType not in [ TAB_UINT8 , TAB_UINT16 , TAB_UINT32 , TAB_UINT64 , TAB_VOID , " BOOLEAN " ] :
2018-04-11 09:14:05 -07:00
Pcd . DatumType = TAB_VOID
2017-11-24 14:30:11 +08:00
2010-09-06 01:58:00 +00:00
# if found PCD which datum value is unicode string the insert to left size of UnicodeIndex
# if found HII type PCD then insert to right of UnicodeIndex
if Pcd . Type in [ TAB_PCDS_DYNAMIC_VPD , TAB_PCDS_DYNAMIC_EX_VPD ] :
2015-12-01 04:22:16 +00:00
VpdPcdDict [ ( Pcd . TokenCName , Pcd . TokenSpaceGuidCName ) ] = Pcd
2017-11-24 14:46:58 +08:00
#Collect DynamicHii PCD values and assign it to DynamicExVpd PCD gEfiMdeModulePkgTokenSpaceGuid.PcdNvStoreDefaultValueBuffer
2018-06-25 18:31:33 +08:00
PcdNvStoreDfBuffer = VpdPcdDict . get ( ( " PcdNvStoreDefaultValueBuffer " , " gEfiMdeModulePkgTokenSpaceGuid " ) )
2017-11-24 14:46:58 +08:00
if PcdNvStoreDfBuffer :
2017-12-22 20:14:29 +08:00
self . VariableInfo = self . CollectVariables ( self . _DynamicPcdList )
vardump = self . VariableInfo . dump ( )
2017-12-22 20:19:20 +08:00
if vardump :
2017-12-22 20:04:04 +08:00
PcdNvStoreDfBuffer . DefaultValue = vardump
2017-12-22 20:19:20 +08:00
for skuname in PcdNvStoreDfBuffer . SkuInfoList :
PcdNvStoreDfBuffer . SkuInfoList [ skuname ] . DefaultValue = vardump
PcdNvStoreDfBuffer . MaxDatumSize = str ( len ( vardump . split ( " , " ) ) )
2017-11-24 14:46:58 +08:00
2018-05-18 08:06:52 +08:00
PlatformPcds = sorted ( self . _PlatformPcds . keys ( ) )
2010-11-15 02:51:34 +00:00
#
# Add VPD type PCD into VpdFile and determine whether the VPD PCD need to be fixed up.
#
2017-12-22 20:19:20 +08:00
VpdSkuMap = { }
2010-11-15 02:51:34 +00:00
for PcdKey in PlatformPcds :
2016-04-19 10:43:57 +08:00
Pcd = self . _PlatformPcds [ PcdKey ]
2013-11-18 07:41:21 +00:00
if Pcd . Type in [ TAB_PCDS_DYNAMIC_VPD , TAB_PCDS_DYNAMIC_EX_VPD ] and \
PcdKey in VpdPcdDict :
Pcd = VpdPcdDict [ PcdKey ]
2017-12-22 20:19:20 +08:00
SkuValueMap = { }
2018-04-16 21:52:13 +08:00
DefaultSku = Pcd . SkuInfoList . get ( TAB_DEFAULT )
2017-12-19 16:26:16 +08:00
if DefaultSku :
PcdValue = DefaultSku . DefaultValue
if PcdValue not in SkuValueMap :
SkuValueMap [ PcdValue ] = [ ]
2018-06-25 18:31:33 +08:00
VpdFile . Add ( Pcd , TAB_DEFAULT , DefaultSku . VpdOffset )
2017-12-19 16:26:16 +08:00
SkuValueMap [ PcdValue ] . append ( DefaultSku )
2018-06-25 18:31:33 +08:00
for ( SkuName , Sku ) in Pcd . SkuInfoList . items ( ) :
2013-11-18 07:41:21 +00:00
Sku . VpdOffset = Sku . VpdOffset . strip ( )
2016-01-18 01:46:25 +00:00
PcdValue = Sku . DefaultValue
if PcdValue == " " :
PcdValue = Pcd . DefaultValue
if Sku . VpdOffset != ' * ' :
if PcdValue . startswith ( " { " ) :
Alignment = 8
elif PcdValue . startswith ( " L " ) :
Alignment = 2
else :
Alignment = 1
2016-01-29 04:44:54 +00:00
try :
VpdOffset = int ( Sku . VpdOffset )
except :
try :
VpdOffset = int ( Sku . VpdOffset , 16 )
except :
EdkLogger . error ( " build " , FORMAT_INVALID , " Invalid offset value %s for PCD %s . %s . " % ( Sku . VpdOffset , Pcd . TokenSpaceGuidCName , Pcd . TokenCName ) )
if VpdOffset % Alignment != 0 :
2016-03-09 14:03:39 +08:00
if PcdValue . startswith ( " { " ) :
EdkLogger . warn ( " build " , " The offset value of PCD %s . %s is not 8-byte aligned! " % ( Pcd . TokenSpaceGuidCName , Pcd . TokenCName ) , File = self . MetaFile )
else :
EdkLogger . error ( " build " , FORMAT_INVALID , ' The offset value of PCD %s . %s should be %s -byte aligned. ' % ( Pcd . TokenSpaceGuidCName , Pcd . TokenCName , Alignment ) )
2017-12-22 20:19:20 +08:00
if PcdValue not in SkuValueMap :
SkuValueMap [ PcdValue ] = [ ]
2018-06-25 18:31:33 +08:00
VpdFile . Add ( Pcd , SkuName , Sku . VpdOffset )
2017-12-22 20:19:20 +08:00
SkuValueMap [ PcdValue ] . append ( Sku )
2013-11-18 07:41:21 +00:00
# if the offset of a VPD is *, then it need to be fixed up by third party tool.
if not NeedProcessVpdMapFile and Sku . VpdOffset == " * " :
NeedProcessVpdMapFile = True
2018-03-27 04:25:43 +08:00
if self . Platform . VpdToolGuid is None or self . Platform . VpdToolGuid == ' ' :
2013-11-18 07:41:21 +00:00
EdkLogger . error ( " Build " , FILE_NOT_FOUND , \
" Fail to find third-party BPDG tool to process VPD PCDs. BPDG Guid tool need to be defined in tools_def.txt and VPD_TOOL_GUID need to be provided in DSC file. " )
2015-12-01 04:22:16 +00:00
2017-12-22 20:19:20 +08:00
VpdSkuMap [ PcdKey ] = SkuValueMap
2010-09-06 01:58:00 +00:00
#
# Fix the PCDs define in VPD PCD section that never referenced by module.
# An example is PCD for signature usage.
2018-07-05 17:40:04 +08:00
#
2010-11-15 02:51:34 +00:00
for DscPcd in PlatformPcds :
2016-04-19 10:43:57 +08:00
DscPcdEntry = self . _PlatformPcds [ DscPcd ]
2010-09-06 01:58:00 +00:00
if DscPcdEntry . Type in [ TAB_PCDS_DYNAMIC_VPD , TAB_PCDS_DYNAMIC_EX_VPD ] :
2018-03-27 04:25:43 +08:00
if not ( self . Platform . VpdToolGuid is None or self . Platform . VpdToolGuid == ' ' ) :
2010-09-06 01:58:00 +00:00
FoundFlag = False
2018-04-17 22:40:15 +08:00
for VpdPcd in VpdFile . _VpdArray :
2010-09-06 01:58:00 +00:00
# This PCD has been referenced by module
if ( VpdPcd . TokenSpaceGuidCName == DscPcdEntry . TokenSpaceGuidCName ) and \
( VpdPcd . TokenCName == DscPcdEntry . TokenCName ) :
FoundFlag = True
2015-12-01 04:22:16 +00:00
2010-09-06 01:58:00 +00:00
# Not found, it should be signature
if not FoundFlag :
# just pick the a value to determine whether is unicode string type
2017-12-22 20:19:20 +08:00
SkuValueMap = { }
2018-01-23 16:00:59 +08:00
SkuObjList = DscPcdEntry . SkuInfoList . items ( )
2018-04-16 21:52:13 +08:00
DefaultSku = DscPcdEntry . SkuInfoList . get ( TAB_DEFAULT )
2017-12-19 16:26:16 +08:00
if DefaultSku :
2018-06-25 18:31:33 +08:00
defaultindex = SkuObjList . index ( ( TAB_DEFAULT , DefaultSku ) )
SkuObjList [ 0 ] , SkuObjList [ defaultindex ] = SkuObjList [ defaultindex ] , SkuObjList [ 0 ]
for ( SkuName , Sku ) in SkuObjList :
2018-07-05 17:40:04 +08:00
Sku . VpdOffset = Sku . VpdOffset . strip ( )
2013-11-18 07:41:21 +00:00
# Need to iterate DEC pcd information to get the value & datumtype
for eachDec in self . PackageList :
for DecPcd in eachDec . Pcds :
DecPcdEntry = eachDec . Pcds [ DecPcd ]
if ( DecPcdEntry . TokenSpaceGuidCName == DscPcdEntry . TokenSpaceGuidCName ) and \
( DecPcdEntry . TokenCName == DscPcdEntry . TokenCName ) :
# Print warning message to let the developer make a determine.
EdkLogger . warn ( " build " , " Unreferenced vpd pcd used! " ,
File = self . MetaFile , \
ExtraData = " PCD: %s . %s used in the DSC file %s is unreferenced. " \
2018-07-05 17:40:04 +08:00
% ( DscPcdEntry . TokenSpaceGuidCName , DscPcdEntry . TokenCName , self . Platform . MetaFile . Path ) )
2013-11-18 07:41:21 +00:00
DscPcdEntry . DatumType = DecPcdEntry . DatumType
DscPcdEntry . DefaultValue = DecPcdEntry . DefaultValue
DscPcdEntry . TokenValue = DecPcdEntry . TokenValue
DscPcdEntry . TokenSpaceGuidValue = eachDec . Guids [ DecPcdEntry . TokenSpaceGuidCName ]
# Only fix the value while no value provided in DSC file.
2018-04-13 08:02:10 +08:00
if not Sku . DefaultValue :
2013-11-18 07:41:21 +00:00
DscPcdEntry . SkuInfoList [ DscPcdEntry . SkuInfoList . keys ( ) [ 0 ] ] . DefaultValue = DecPcdEntry . DefaultValue
2018-07-05 17:40:04 +08:00
2013-11-18 07:41:21 +00:00
if DscPcdEntry not in self . _DynamicPcdList :
self . _DynamicPcdList . append ( DscPcdEntry )
Sku . VpdOffset = Sku . VpdOffset . strip ( )
PcdValue = Sku . DefaultValue
2016-01-18 01:46:25 +00:00
if PcdValue == " " :
PcdValue = DscPcdEntry . DefaultValue
if Sku . VpdOffset != ' * ' :
if PcdValue . startswith ( " { " ) :
Alignment = 8
elif PcdValue . startswith ( " L " ) :
Alignment = 2
else :
Alignment = 1
2016-01-29 04:44:54 +00:00
try :
VpdOffset = int ( Sku . VpdOffset )
except :
try :
VpdOffset = int ( Sku . VpdOffset , 16 )
except :
EdkLogger . error ( " build " , FORMAT_INVALID , " Invalid offset value %s for PCD %s . %s . " % ( Sku . VpdOffset , DscPcdEntry . TokenSpaceGuidCName , DscPcdEntry . TokenCName ) )
if VpdOffset % Alignment != 0 :
2016-03-09 14:03:39 +08:00
if PcdValue . startswith ( " { " ) :
EdkLogger . warn ( " build " , " The offset value of PCD %s . %s is not 8-byte aligned! " % ( DscPcdEntry . TokenSpaceGuidCName , DscPcdEntry . TokenCName ) , File = self . MetaFile )
else :
EdkLogger . error ( " build " , FORMAT_INVALID , ' The offset value of PCD %s . %s should be %s -byte aligned. ' % ( DscPcdEntry . TokenSpaceGuidCName , DscPcdEntry . TokenCName , Alignment ) )
2017-12-22 20:19:20 +08:00
if PcdValue not in SkuValueMap :
SkuValueMap [ PcdValue ] = [ ]
2018-06-25 18:31:33 +08:00
VpdFile . Add ( DscPcdEntry , SkuName , Sku . VpdOffset )
2017-12-22 20:19:20 +08:00
SkuValueMap [ PcdValue ] . append ( Sku )
2013-11-18 07:41:21 +00:00
if not NeedProcessVpdMapFile and Sku . VpdOffset == " * " :
2018-07-05 17:40:04 +08:00
NeedProcessVpdMapFile = True
2018-04-11 09:14:05 -07:00
if DscPcdEntry . DatumType == TAB_VOID and PcdValue . startswith ( " L " ) :
2017-12-22 20:23:21 +08:00
UnicodePcdArray . add ( DscPcdEntry )
2013-11-18 07:41:21 +00:00
elif len ( Sku . VariableName ) > 0 :
2017-12-22 20:23:21 +08:00
HiiPcdArray . add ( DscPcdEntry )
2013-11-18 07:41:21 +00:00
else :
2017-12-22 20:23:21 +08:00
OtherPcdArray . add ( DscPcdEntry )
2013-11-18 07:41:21 +00:00
# if the offset of a VPD is *, then it need to be fixed up by third party tool.
2017-12-22 20:19:20 +08:00
VpdSkuMap [ DscPcd ] = SkuValueMap
2018-03-27 04:25:43 +08:00
if ( self . Platform . FlashDefinition is None or self . Platform . FlashDefinition == ' ' ) and \
2010-09-06 01:58:00 +00:00
VpdFile . GetCount ( ) != 0 :
2018-07-05 17:40:04 +08:00
EdkLogger . error ( " build " , ATTRIBUTE_NOT_AVAILABLE ,
2010-09-06 01:58:00 +00:00
" Fail to get FLASH_DEFINITION definition in DSC file %s which is required when DSC contains VPD PCD. " % str ( self . Platform . MetaFile ) )
2015-12-01 04:22:16 +00:00
2010-09-06 01:58:00 +00:00
if VpdFile . GetCount ( ) != 0 :
2017-12-22 20:14:29 +08:00
self . FixVpdOffset ( VpdFile )
self . FixVpdOffset ( self . UpdateNVStoreMaxSize ( VpdFile ) )
2015-12-01 04:22:16 +00:00
2010-09-06 01:58:00 +00:00
# Process VPD map file generated by third party BPDG tool
if NeedProcessVpdMapFile :
2018-04-27 00:57:56 +08:00
VpdMapFilePath = os . path . join ( self . BuildDir , TAB_FV_DIRECTORY , " %s .map " % self . Platform . VpdToolGuid )
2010-09-06 01:58:00 +00:00
if os . path . exists ( VpdMapFilePath ) :
VpdFile . Read ( VpdMapFilePath )
2015-12-01 04:22:16 +00:00
2010-09-06 01:58:00 +00:00
# Fixup "*" offset
2017-12-22 20:19:20 +08:00
for pcd in VpdSkuMap :
vpdinfo = VpdFile . GetVpdInfo ( pcd )
if vpdinfo is None :
2010-09-06 01:58:00 +00:00
# just pick the a value to determine whether is unicode string type
2017-12-22 20:19:20 +08:00
continue
for pcdvalue in VpdSkuMap [ pcd ] :
for sku in VpdSkuMap [ pcd ] [ pcdvalue ] :
for item in vpdinfo :
if item [ 2 ] == pcdvalue :
sku . VpdOffset = item [ 1 ]
2010-09-06 01:58:00 +00:00
else :
EdkLogger . error ( " build " , FILE_READ_FAILURE , " Can not find VPD map file %s to fix up VPD offset. " % VpdMapFilePath )
2015-12-01 04:22:16 +00:00
2017-12-22 20:19:20 +08:00
# Delete the DynamicPcdList At the last time enter into this function
for Pcd in self . _DynamicPcdList :
# just pick the a value to determine whether is unicode string type
2018-04-17 22:40:15 +08:00
Sku = Pcd . SkuInfoList . values ( ) [ 0 ]
2017-12-22 20:19:20 +08:00
Sku . VpdOffset = Sku . VpdOffset . strip ( )
if Pcd . DatumType not in [ TAB_UINT8 , TAB_UINT16 , TAB_UINT32 , TAB_UINT64 , TAB_VOID , " BOOLEAN " ] :
2018-04-11 09:14:05 -07:00
Pcd . DatumType = TAB_VOID
2017-12-22 20:19:20 +08:00
PcdValue = Sku . DefaultValue
2018-04-11 09:14:05 -07:00
if Pcd . DatumType == TAB_VOID and PcdValue . startswith ( " L " ) :
2017-12-22 20:19:20 +08:00
# if found PCD which datum value is unicode string the insert to left size of UnicodeIndex
2017-12-22 20:23:21 +08:00
UnicodePcdArray . add ( Pcd )
2017-12-22 20:19:20 +08:00
elif len ( Sku . VariableName ) > 0 :
# if found HII type PCD then insert to right of UnicodeIndex
2017-12-22 20:23:21 +08:00
HiiPcdArray . add ( Pcd )
2017-12-22 20:19:20 +08:00
else :
2017-12-22 20:23:21 +08:00
OtherPcdArray . add ( Pcd )
2015-12-01 04:22:16 +00:00
del self . _DynamicPcdList [ : ]
2017-12-22 20:23:21 +08:00
self . _DynamicPcdList . extend ( list ( UnicodePcdArray ) )
self . _DynamicPcdList . extend ( list ( HiiPcdArray ) )
self . _DynamicPcdList . extend ( list ( OtherPcdArray ) )
2018-06-25 18:31:33 +08:00
allskuset = [ ( SkuName , Sku . SkuId ) for pcd in self . _DynamicPcdList for ( SkuName , Sku ) in pcd . SkuInfoList . items ( ) ]
2017-12-22 20:04:04 +08:00
for pcd in self . _DynamicPcdList :
if len ( pcd . SkuInfoList ) == 1 :
2018-06-25 18:31:33 +08:00
for ( SkuName , SkuId ) in allskuset :
if type ( SkuId ) in ( str , unicode ) and eval ( SkuId ) == 0 or SkuId == 0 :
2017-12-22 20:04:04 +08:00
continue
2018-04-16 21:52:13 +08:00
pcd . SkuInfoList [ SkuName ] = copy . deepcopy ( pcd . SkuInfoList [ TAB_DEFAULT ] )
2017-12-01 22:00:07 +08:00
pcd . SkuInfoList [ SkuName ] . SkuId = SkuId
2014-08-26 02:52:24 +00:00
self . AllPcdList = self . _NonDynamicPcdList + self . _DynamicPcdList
2017-12-22 20:14:29 +08:00
2018-06-25 18:31:33 +08:00
def FixVpdOffset ( self , VpdFile ) :
2018-04-27 00:57:56 +08:00
FvPath = os . path . join ( self . BuildDir , TAB_FV_DIRECTORY )
2017-12-22 20:14:29 +08:00
if not os . path . exists ( FvPath ) :
try :
os . makedirs ( FvPath )
except :
EdkLogger . error ( " build " , FILE_WRITE_FAILURE , " Fail to create FV folder under %s " % self . BuildDir )
VpdFilePath = os . path . join ( FvPath , " %s .txt " % self . Platform . VpdToolGuid )
if VpdFile . Write ( VpdFilePath ) :
# retrieve BPDG tool's path from tool_def.txt according to VPD_TOOL_GUID defined in DSC file.
BPDGToolName = None
for ToolDef in self . ToolDefinition . values ( ) :
2018-06-25 18:31:29 +08:00
if TAB_GUID in ToolDef and ToolDef [ TAB_GUID ] == self . Platform . VpdToolGuid :
if " PATH " not in ToolDef :
2017-12-22 20:14:29 +08:00
EdkLogger . error ( " build " , ATTRIBUTE_NOT_AVAILABLE , " PATH attribute was not provided for BPDG guid tool %s in tools_def.txt " % self . Platform . VpdToolGuid )
BPDGToolName = ToolDef [ " PATH " ]
break
# Call third party GUID BPDG tool.
2018-03-27 04:25:43 +08:00
if BPDGToolName is not None :
2017-12-22 20:14:29 +08:00
VpdInfoFile . CallExtenalBPDGTool ( BPDGToolName , VpdFilePath )
else :
EdkLogger . error ( " Build " , FILE_NOT_FOUND , " Fail to find third-party BPDG tool to process VPD PCDs. BPDG Guid tool need to be defined in tools_def.txt and VPD_TOOL_GUID need to be provided in DSC file. " )
2010-02-28 23:39:39 +00:00
## Return the platform build data object
def _GetPlatform ( self ) :
2018-03-27 04:25:43 +08:00
if self . _Platform is None :
2011-10-29 06:59:30 +00:00
self . _Platform = self . BuildDatabase [ self . MetaFile , self . Arch , self . BuildTarget , self . ToolChain ]
2010-02-28 23:39:39 +00:00
return self . _Platform
## Return platform name
def _GetName ( self ) :
return self . Platform . PlatformName
## Return the meta file GUID
def _GetGuid ( self ) :
return self . Platform . Guid
## Return the platform version
def _GetVersion ( self ) :
return self . Platform . Version
## Return the FDF file name
def _GetFdfFile ( self ) :
2018-03-27 04:25:43 +08:00
if self . _FdfFile is None :
2010-02-28 23:39:39 +00:00
if self . Workspace . FdfFile != " " :
2015-10-08 09:27:14 +00:00
self . _FdfFile = mws . join ( self . WorkspaceDir , self . Workspace . FdfFile )
2010-02-28 23:39:39 +00:00
else :
self . _FdfFile = ' '
return self . _FdfFile
## Return the build output directory platform specifies
def _GetOutputDir ( self ) :
return self . Platform . OutputDirectory
## Return the directory to store all intermediate and final files built
def _GetBuildDir ( self ) :
2018-03-27 04:25:43 +08:00
if self . _BuildDir is None :
2010-02-28 23:39:39 +00:00
if os . path . isabs ( self . OutputDir ) :
self . _BuildDir = path . join (
path . abspath ( self . OutputDir ) ,
self . BuildTarget + " _ " + self . ToolChain ,
)
else :
self . _BuildDir = path . join (
self . WorkspaceDir ,
self . OutputDir ,
self . BuildTarget + " _ " + self . ToolChain ,
)
2017-10-13 10:47:24 +08:00
GlobalData . gBuildDirectory = self . _BuildDir
2010-02-28 23:39:39 +00:00
return self . _BuildDir
## Return directory of platform makefile
#
# @retval string Makefile directory
#
def _GetMakeFileDir ( self ) :
2018-03-27 04:25:43 +08:00
if self . _MakeFileDir is None :
2010-02-28 23:39:39 +00:00
self . _MakeFileDir = path . join ( self . BuildDir , self . Arch )
return self . _MakeFileDir
## Return build command string
#
# @retval string Build command string
#
def _GetBuildCommand ( self ) :
2018-03-27 04:25:43 +08:00
if self . _BuildCommand is None :
2010-02-28 23:39:39 +00:00
self . _BuildCommand = [ ]
if " MAKE " in self . ToolDefinition and " PATH " in self . ToolDefinition [ " MAKE " ] :
self . _BuildCommand + = SplitOption ( self . ToolDefinition [ " MAKE " ] [ " PATH " ] )
if " FLAGS " in self . ToolDefinition [ " MAKE " ] :
NewOption = self . ToolDefinition [ " MAKE " ] [ " FLAGS " ] . strip ( )
if NewOption != ' ' :
2010-11-15 02:51:34 +00:00
self . _BuildCommand + = SplitOption ( NewOption )
2018-02-24 21:41:13 +08:00
if " MAKE " in self . EdkIIBuildOption :
if " FLAGS " in self . EdkIIBuildOption [ " MAKE " ] :
Flags = self . EdkIIBuildOption [ " MAKE " ] [ " FLAGS " ]
if Flags . startswith ( ' = ' ) :
self . _BuildCommand = [ self . _BuildCommand [ 0 ] ] + [ Flags [ 1 : ] ]
else :
2018-05-18 08:06:52 +08:00
self . _BuildCommand . append ( Flags )
2010-02-28 23:39:39 +00:00
return self . _BuildCommand
## Get tool chain definition
#
# Get each tool defition for given tool chain from tools_def.txt and platform
#
def _GetToolDefinition ( self ) :
2018-03-27 04:25:43 +08:00
if self . _ToolDefinitions is None :
2010-02-28 23:39:39 +00:00
ToolDefinition = self . Workspace . ToolDef . ToolsDefTxtDictionary
if TAB_TOD_DEFINES_COMMAND_TYPE not in self . Workspace . ToolDef . ToolsDefTxtDatabase :
EdkLogger . error ( ' build ' , RESOURCE_NOT_AVAILABLE , " No tools found in configuration " ,
ExtraData = " [ %s ] " % self . MetaFile )
self . _ToolDefinitions = { }
DllPathList = set ( )
for Def in ToolDefinition :
Target , Tag , Arch , Tool , Attr = Def . split ( " _ " )
if Target != self . BuildTarget or Tag != self . ToolChain or Arch != self . Arch :
continue
Value = ToolDefinition [ Def ]
# don't record the DLL
if Attr == " DLL " :
DllPathList . add ( Value )
continue
if Tool not in self . _ToolDefinitions :
self . _ToolDefinitions [ Tool ] = { }
self . _ToolDefinitions [ Tool ] [ Attr ] = Value
ToolsDef = ' '
if GlobalData . gOptions . SilentMode and " MAKE " in self . _ToolDefinitions :
if " FLAGS " not in self . _ToolDefinitions [ " MAKE " ] :
self . _ToolDefinitions [ " MAKE " ] [ " FLAGS " ] = " "
self . _ToolDefinitions [ " MAKE " ] [ " FLAGS " ] + = " -s "
MakeFlags = ' '
for Tool in self . _ToolDefinitions :
for Attr in self . _ToolDefinitions [ Tool ] :
Value = self . _ToolDefinitions [ Tool ] [ Attr ]
if Tool in self . BuildOption and Attr in self . BuildOption [ Tool ] :
# check if override is indicated
if self . BuildOption [ Tool ] [ Attr ] . startswith ( ' = ' ) :
Value = self . BuildOption [ Tool ] [ Attr ] [ 1 : ]
else :
2017-02-15 14:47:47 +08:00
if Attr != ' PATH ' :
Value + = " " + self . BuildOption [ Tool ] [ Attr ]
else :
Value = self . BuildOption [ Tool ] [ Attr ]
2010-02-28 23:39:39 +00:00
if Attr == " PATH " :
# Don't put MAKE definition in the file
2018-04-06 07:13:54 +08:00
if Tool != " MAKE " :
2010-02-28 23:39:39 +00:00
ToolsDef + = " %s = %s \n " % ( Tool , Value )
elif Attr != " DLL " :
# Don't put MAKE definition in the file
if Tool == " MAKE " :
if Attr == " FLAGS " :
MakeFlags = Value
else :
ToolsDef + = " %s _ %s = %s \n " % ( Tool , Attr , Value )
ToolsDef + = " \n "
SaveFileOnChange ( self . ToolDefinitionFile , ToolsDef )
for DllPath in DllPathList :
os . environ [ " PATH " ] = DllPath + os . pathsep + os . environ [ " PATH " ]
os . environ [ " MAKE_FLAGS " ] = MakeFlags
return self . _ToolDefinitions
## Return the paths of tools
def _GetToolDefFile ( self ) :
2018-03-27 04:25:43 +08:00
if self . _ToolDefFile is None :
2010-02-28 23:39:39 +00:00
self . _ToolDefFile = os . path . join ( self . MakeFileDir , " TOOLS_DEF. " + self . Arch )
return self . _ToolDefFile
## Retrieve the toolchain family of given toolchain tag. Default to 'MSFT'.
def _GetToolChainFamily ( self ) :
2018-03-27 04:25:43 +08:00
if self . _ToolChainFamily is None :
2010-02-28 23:39:39 +00:00
ToolDefinition = self . Workspace . ToolDef . ToolsDefTxtDatabase
if TAB_TOD_DEFINES_FAMILY not in ToolDefinition \
or self . ToolChain not in ToolDefinition [ TAB_TOD_DEFINES_FAMILY ] \
or not ToolDefinition [ TAB_TOD_DEFINES_FAMILY ] [ self . ToolChain ] :
EdkLogger . verbose ( " No tool chain family found in configuration for %s . Default to MSFT. " \
% self . ToolChain )
self . _ToolChainFamily = " MSFT "
else :
self . _ToolChainFamily = ToolDefinition [ TAB_TOD_DEFINES_FAMILY ] [ self . ToolChain ]
return self . _ToolChainFamily
def _GetBuildRuleFamily ( self ) :
2018-03-27 04:25:43 +08:00
if self . _BuildRuleFamily is None :
2010-02-28 23:39:39 +00:00
ToolDefinition = self . Workspace . ToolDef . ToolsDefTxtDatabase
if TAB_TOD_DEFINES_BUILDRULEFAMILY not in ToolDefinition \
or self . ToolChain not in ToolDefinition [ TAB_TOD_DEFINES_BUILDRULEFAMILY ] \
or not ToolDefinition [ TAB_TOD_DEFINES_BUILDRULEFAMILY ] [ self . ToolChain ] :
EdkLogger . verbose ( " No tool chain family found in configuration for %s . Default to MSFT. " \
% self . ToolChain )
self . _BuildRuleFamily = " MSFT "
else :
self . _BuildRuleFamily = ToolDefinition [ TAB_TOD_DEFINES_BUILDRULEFAMILY ] [ self . ToolChain ]
return self . _BuildRuleFamily
## Return the build options specific for all modules in this platform
def _GetBuildOptions ( self ) :
2018-03-27 04:25:43 +08:00
if self . _BuildOption is None :
2010-02-28 23:39:39 +00:00
self . _BuildOption = self . _ExpandBuildOption ( self . Platform . BuildOptions )
return self . _BuildOption
## Return the build options specific for EDK modules in this platform
def _GetEdkBuildOptions ( self ) :
2018-03-27 04:25:43 +08:00
if self . _EdkBuildOption is None :
2010-02-28 23:39:39 +00:00
self . _EdkBuildOption = self . _ExpandBuildOption ( self . Platform . BuildOptions , EDK_NAME )
return self . _EdkBuildOption
## Return the build options specific for EDKII modules in this platform
def _GetEdkIIBuildOptions ( self ) :
2018-03-27 04:25:43 +08:00
if self . _EdkIIBuildOption is None :
2010-02-28 23:39:39 +00:00
self . _EdkIIBuildOption = self . _ExpandBuildOption ( self . Platform . BuildOptions , EDKII_NAME )
return self . _EdkIIBuildOption
2014-08-28 13:53:34 +00:00
## Parse build_rule.txt in Conf Directory.
2010-02-28 23:39:39 +00:00
#
# @retval BuildRule object
#
def _GetBuildRule ( self ) :
2018-03-27 04:25:43 +08:00
if self . _BuildRule is None :
2010-02-28 23:39:39 +00:00
BuildRuleFile = None
if TAB_TAT_DEFINES_BUILD_RULE_CONF in self . Workspace . TargetTxt . TargetTxtDictionary :
BuildRuleFile = self . Workspace . TargetTxt . TargetTxtDictionary [ TAB_TAT_DEFINES_BUILD_RULE_CONF ]
2018-04-20 23:51:42 +08:00
if not BuildRuleFile :
2014-08-28 13:53:34 +00:00
BuildRuleFile = gDefaultBuildRuleFile
2010-02-28 23:39:39 +00:00
self . _BuildRule = BuildRule ( BuildRuleFile )
2012-04-10 07:18:20 +00:00
if self . _BuildRule . _FileVersion == " " :
self . _BuildRule . _FileVersion = AutoGenReqBuildRuleVerNum
else :
if self . _BuildRule . _FileVersion < AutoGenReqBuildRuleVerNum :
# If Build Rule's version is less than the version number required by the tools, halting the build.
2015-12-01 04:22:16 +00:00
EdkLogger . error ( " build " , AUTOGEN_ERROR ,
2012-04-10 07:18:20 +00:00
ExtraData = " The version number [ %s ] of build_rule.txt is less than the version number required by the AutoGen.(the minimum required version number is [ %s ]) " \
% ( self . _BuildRule . _FileVersion , AutoGenReqBuildRuleVerNum ) )
2015-12-01 04:22:16 +00:00
2010-02-28 23:39:39 +00:00
return self . _BuildRule
## Summarize the packages used by modules in this platform
def _GetPackageList ( self ) :
2018-03-27 04:25:43 +08:00
if self . _PackageList is None :
2010-02-28 23:39:39 +00:00
self . _PackageList = set ( )
for La in self . LibraryAutoGenList :
self . _PackageList . update ( La . DependentPackageList )
for Ma in self . ModuleAutoGenList :
self . _PackageList . update ( Ma . DependentPackageList )
2014-08-26 02:52:24 +00:00
#Collect package set information from INF of FDF
PkgSet = set ( )
for ModuleFile in self . _AsBuildModuleList :
if ModuleFile in self . Platform . Modules :
continue
ModuleData = self . BuildDatabase [ ModuleFile , self . Arch , self . BuildTarget , self . ToolChain ]
PkgSet . update ( ModuleData . Packages )
self . _PackageList = list ( self . _PackageList ) + list ( PkgSet )
2010-02-28 23:39:39 +00:00
return self . _PackageList
2014-01-10 05:25:50 +00:00
def _GetNonDynamicPcdDict ( self ) :
if self . _NonDynamicPcdDict :
return self . _NonDynamicPcdDict
for Pcd in self . NonDynamicPcdList :
2018-06-25 18:31:33 +08:00
self . _NonDynamicPcdDict [ ( Pcd . TokenCName , Pcd . TokenSpaceGuidCName ) ] = Pcd
2014-01-10 05:25:50 +00:00
return self . _NonDynamicPcdDict
2010-02-28 23:39:39 +00:00
## Get list of non-dynamic PCDs
def _GetNonDynamicPcdList ( self ) :
2018-03-27 04:25:43 +08:00
if self . _NonDynamicPcdList is None :
2010-09-06 01:58:00 +00:00
self . CollectPlatformDynamicPcds ( )
2010-02-28 23:39:39 +00:00
return self . _NonDynamicPcdList
## Get list of dynamic PCDs
def _GetDynamicPcdList ( self ) :
2018-03-27 04:25:43 +08:00
if self . _DynamicPcdList is None :
2010-09-06 01:58:00 +00:00
self . CollectPlatformDynamicPcds ( )
2010-02-28 23:39:39 +00:00
return self . _DynamicPcdList
## Generate Token Number for all PCD
def _GetPcdTokenNumbers ( self ) :
2018-03-27 04:25:43 +08:00
if self . _PcdTokenNumber is None :
2018-04-04 05:03:05 +08:00
self . _PcdTokenNumber = OrderedDict ( )
2010-02-28 23:39:39 +00:00
TokenNumber = 1
2011-12-07 06:19:28 +00:00
#
2018-07-05 17:40:04 +08:00
# Make the Dynamic and DynamicEx PCD use within different TokenNumber area.
2011-12-07 06:19:28 +00:00
# Such as:
2018-07-05 17:40:04 +08:00
#
2011-12-07 06:19:28 +00:00
# Dynamic PCD:
# TokenNumber 0 ~ 10
# DynamicEx PCD:
# TokeNumber 11 ~ 20
#
2010-02-28 23:39:39 +00:00
for Pcd in self . DynamicPcdList :
2018-05-18 08:06:52 +08:00
if Pcd . Phase == " PEI " and Pcd . Type in PCD_DYNAMIC_TYPE_SET :
EdkLogger . debug ( EdkLogger . DEBUG_5 , " %s %s ( %s ) -> %d " % ( Pcd . TokenCName , Pcd . TokenSpaceGuidCName , Pcd . Phase , TokenNumber ) )
self . _PcdTokenNumber [ Pcd . TokenCName , Pcd . TokenSpaceGuidCName ] = TokenNumber
TokenNumber + = 1
2015-12-01 04:22:16 +00:00
2011-12-07 06:19:28 +00:00
for Pcd in self . DynamicPcdList :
2018-05-18 08:06:52 +08:00
if Pcd . Phase == " PEI " and Pcd . Type in PCD_DYNAMIC_EX_TYPE_SET :
EdkLogger . debug ( EdkLogger . DEBUG_5 , " %s %s ( %s ) -> %d " % ( Pcd . TokenCName , Pcd . TokenSpaceGuidCName , Pcd . Phase , TokenNumber ) )
self . _PcdTokenNumber [ Pcd . TokenCName , Pcd . TokenSpaceGuidCName ] = TokenNumber
TokenNumber + = 1
2015-12-01 04:22:16 +00:00
2010-02-28 23:39:39 +00:00
for Pcd in self . DynamicPcdList :
2018-05-18 08:06:52 +08:00
if Pcd . Phase == " DXE " and Pcd . Type in PCD_DYNAMIC_TYPE_SET :
EdkLogger . debug ( EdkLogger . DEBUG_5 , " %s %s ( %s ) -> %d " % ( Pcd . TokenCName , Pcd . TokenSpaceGuidCName , Pcd . Phase , TokenNumber ) )
self . _PcdTokenNumber [ Pcd . TokenCName , Pcd . TokenSpaceGuidCName ] = TokenNumber
TokenNumber + = 1
2015-12-01 04:22:16 +00:00
2011-12-07 06:19:28 +00:00
for Pcd in self . DynamicPcdList :
2018-05-18 08:06:52 +08:00
if Pcd . Phase == " DXE " and Pcd . Type in PCD_DYNAMIC_EX_TYPE_SET :
EdkLogger . debug ( EdkLogger . DEBUG_5 , " %s %s ( %s ) -> %d " % ( Pcd . TokenCName , Pcd . TokenSpaceGuidCName , Pcd . Phase , TokenNumber ) )
self . _PcdTokenNumber [ Pcd . TokenCName , Pcd . TokenSpaceGuidCName ] = TokenNumber
TokenNumber + = 1
2015-12-01 04:22:16 +00:00
2010-02-28 23:39:39 +00:00
for Pcd in self . NonDynamicPcdList :
self . _PcdTokenNumber [ Pcd . TokenCName , Pcd . TokenSpaceGuidCName ] = TokenNumber
TokenNumber + = 1
return self . _PcdTokenNumber
## Summarize ModuleAutoGen objects of all modules/libraries to be built for this platform
def _GetAutoGenObjectList ( self ) :
self . _ModuleAutoGenList = [ ]
self . _LibraryAutoGenList = [ ]
for ModuleFile in self . Platform . Modules :
Ma = ModuleAutoGen (
self . Workspace ,
ModuleFile ,
self . BuildTarget ,
self . ToolChain ,
self . Arch ,
self . MetaFile
)
if Ma not in self . _ModuleAutoGenList :
self . _ModuleAutoGenList . append ( Ma )
for La in Ma . LibraryAutoGenList :
if La not in self . _LibraryAutoGenList :
self . _LibraryAutoGenList . append ( La )
2018-08-03 23:11:06 +08:00
if Ma not in La . ReferenceModules :
La . ReferenceModules . append ( Ma )
2010-02-28 23:39:39 +00:00
## Summarize ModuleAutoGen objects of all modules to be built for this platform
def _GetModuleAutoGenList ( self ) :
2018-03-27 04:25:43 +08:00
if self . _ModuleAutoGenList is None :
2010-02-28 23:39:39 +00:00
self . _GetAutoGenObjectList ( )
return self . _ModuleAutoGenList
## Summarize ModuleAutoGen objects of all libraries to be built for this platform
def _GetLibraryAutoGenList ( self ) :
2018-03-27 04:25:43 +08:00
if self . _LibraryAutoGenList is None :
2010-02-28 23:39:39 +00:00
self . _GetAutoGenObjectList ( )
return self . _LibraryAutoGenList
## Test if a module is supported by the platform
#
# An error will be raised directly if the module or its arch is not supported
# by the platform or current configuration
#
def ValidModule ( self , Module ) :
2014-08-26 02:52:24 +00:00
return Module in self . Platform . Modules or Module in self . Platform . LibraryInstances \
or Module in self . _AsBuildModuleList
2010-02-28 23:39:39 +00:00
## Resolve the library classes in a module to library instances
#
# This method will not only resolve library classes but also sort the library
# instances according to the dependency-ship.
#
# @param Module The module from which the library classes will be resolved
#
# @retval library_list List of library instances sorted
#
def ApplyLibraryInstance ( self , Module ) :
2016-07-27 16:29:38 +08:00
# Cover the case that the binary INF file is list in the FDF file but not DSC file, return empty list directly
if str ( Module ) not in self . Platform . Modules :
return [ ]
2018-05-18 08:06:51 +08:00
return GetModuleLibInstances ( Module ,
self . Platform ,
self . BuildDatabase ,
self . Arch ,
self . BuildTarget ,
self . ToolChain ,
self . MetaFile ,
EdkLogger )
2010-02-28 23:39:39 +00:00
## Override PCD setting (type, value, ...)
#
# @param ToPcd The PCD to be overrided
# @param FromPcd The PCD overrideing from
#
2018-05-16 13:59:59 +08:00
def _OverridePcd ( self , ToPcd , FromPcd , Module = " " , Msg = " " , Library = " " ) :
2010-02-28 23:39:39 +00:00
#
# in case there's PCDs coming from FDF file, which have no type given.
# at this point, ToPcd.Type has the type found from dependent
# package
#
2016-04-12 10:31:55 +08:00
TokenCName = ToPcd . TokenCName
for PcdItem in GlobalData . MixedPcd :
if ( ToPcd . TokenCName , ToPcd . TokenSpaceGuidCName ) in GlobalData . MixedPcd [ PcdItem ] :
TokenCName = PcdItem [ 0 ]
break
2018-03-27 04:25:43 +08:00
if FromPcd is not None :
2018-04-20 23:51:42 +08:00
if ToPcd . Pending and FromPcd . Type :
2010-02-28 23:39:39 +00:00
ToPcd . Type = FromPcd . Type
2018-05-18 08:06:52 +08:00
elif ToPcd . Type and FromPcd . Type \
and ToPcd . Type != FromPcd . Type and ToPcd . Type in FromPcd . Type :
2018-04-20 23:51:25 +08:00
if ToPcd . Type . strip ( ) == TAB_PCDS_DYNAMIC_EX :
2015-12-01 04:22:16 +00:00
ToPcd . Type = FromPcd . Type
2018-04-20 23:51:42 +08:00
elif ToPcd . Type and FromPcd . Type \
2010-02-28 23:39:39 +00:00
and ToPcd . Type != FromPcd . Type :
2018-05-16 13:59:59 +08:00
if Library :
Module = str ( Module ) + " ' s library file ( " + str ( Library ) + " ) "
2010-02-28 23:39:39 +00:00
EdkLogger . error ( " build " , OPTION_CONFLICT , " Mismatched PCD type " ,
2018-05-16 13:59:59 +08:00
ExtraData = " %s . %s is used as [ %s ] in module %s , but as [ %s ] in %s . " \
2016-04-12 10:31:55 +08:00
% ( ToPcd . TokenSpaceGuidCName , TokenCName ,
2018-05-16 13:59:59 +08:00
ToPcd . Type , Module , FromPcd . Type , Msg ) ,
2010-02-28 23:39:39 +00:00
File = self . MetaFile )
2018-04-06 07:13:56 +08:00
if FromPcd . MaxDatumSize :
2010-02-28 23:39:39 +00:00
ToPcd . MaxDatumSize = FromPcd . MaxDatumSize
2018-03-17 15:25:32 +08:00
ToPcd . MaxSizeUserSet = FromPcd . MaxDatumSize
2018-04-06 07:13:56 +08:00
if FromPcd . DefaultValue :
2010-02-28 23:39:39 +00:00
ToPcd . DefaultValue = FromPcd . DefaultValue
2018-04-06 07:13:56 +08:00
if FromPcd . TokenValue :
2010-02-28 23:39:39 +00:00
ToPcd . TokenValue = FromPcd . TokenValue
2018-04-06 07:13:56 +08:00
if FromPcd . DatumType :
2010-02-28 23:39:39 +00:00
ToPcd . DatumType = FromPcd . DatumType
2018-04-06 07:13:56 +08:00
if FromPcd . SkuInfoList :
2010-02-28 23:39:39 +00:00
ToPcd . SkuInfoList = FromPcd . SkuInfoList
2017-12-26 16:17:13 +08:00
# Add Flexible PCD format parse
if ToPcd . DefaultValue :
try :
2018-01-27 00:28:05 +08:00
ToPcd . DefaultValue = ValueExpressionEx ( ToPcd . DefaultValue , ToPcd . DatumType , self . _GuidDict ) ( True )
2018-06-25 18:31:25 +08:00
except BadExpression as Value :
2017-12-26 16:17:13 +08:00
EdkLogger . error ( ' Parser ' , FORMAT_INVALID , ' PCD [ %s . %s ] Value " %s " , %s ' % ( ToPcd . TokenSpaceGuidCName , ToPcd . TokenCName , ToPcd . DefaultValue , Value ) ,
File = self . MetaFile )
2010-02-28 23:39:39 +00:00
# check the validation of datum
IsValid , Cause = CheckPcdDatum ( ToPcd . DatumType , ToPcd . DefaultValue )
if not IsValid :
EdkLogger . error ( ' build ' , FORMAT_INVALID , Cause , File = self . MetaFile ,
2016-04-12 10:31:55 +08:00
ExtraData = " %s . %s " % ( ToPcd . TokenSpaceGuidCName , TokenCName ) )
2015-04-10 06:59:47 +00:00
ToPcd . validateranges = FromPcd . validateranges
ToPcd . validlists = FromPcd . validlists
ToPcd . expressions = FromPcd . expressions
2010-02-28 23:39:39 +00:00
2018-04-20 23:51:42 +08:00
if FromPcd is not None and ToPcd . DatumType == TAB_VOID and not ToPcd . MaxDatumSize :
2010-02-28 23:39:39 +00:00
EdkLogger . debug ( EdkLogger . DEBUG_9 , " No MaxDatumSize specified for PCD %s . %s " \
2016-04-12 10:31:55 +08:00
% ( ToPcd . TokenSpaceGuidCName , TokenCName ) )
2010-02-28 23:39:39 +00:00
Value = ToPcd . DefaultValue
2018-04-20 23:51:42 +08:00
if not Value :
2013-11-18 07:41:21 +00:00
ToPcd . MaxDatumSize = ' 1 '
2010-02-28 23:39:39 +00:00
elif Value [ 0 ] == ' L ' :
2013-11-18 07:41:21 +00:00
ToPcd . MaxDatumSize = str ( ( len ( Value ) - 2 ) * 2 )
2010-02-28 23:39:39 +00:00
elif Value [ 0 ] == ' { ' :
ToPcd . MaxDatumSize = str ( len ( Value . split ( ' , ' ) ) )
else :
2013-11-18 07:41:21 +00:00
ToPcd . MaxDatumSize = str ( len ( Value ) - 1 )
2010-02-28 23:39:39 +00:00
# apply default SKU for dynamic PCDS if specified one is not available
2018-04-20 23:51:37 +08:00
if ( ToPcd . Type in PCD_DYNAMIC_TYPE_SET or ToPcd . Type in PCD_DYNAMIC_EX_TYPE_SET ) \
2018-04-20 23:51:42 +08:00
and not ToPcd . SkuInfoList :
2010-02-28 23:39:39 +00:00
if self . Platform . SkuName in self . Platform . SkuIds :
SkuName = self . Platform . SkuName
else :
2018-04-16 21:52:13 +08:00
SkuName = TAB_DEFAULT
2010-02-28 23:39:39 +00:00
ToPcd . SkuInfoList = {
2017-12-22 20:46:15 +08:00
SkuName : SkuInfoClass ( SkuName , self . Platform . SkuIds [ SkuName ] [ 0 ] , ' ' , ' ' , ' ' , ' ' , ' ' , ToPcd . DefaultValue )
2010-02-28 23:39:39 +00:00
}
## Apply PCD setting defined platform to a module
#
# @param Module The module from which the PCD setting will be overrided
#
# @retval PCD_list The list PCDs with settings from platform
#
2018-05-16 13:59:59 +08:00
def ApplyPcdSetting ( self , Module , Pcds , Library = " " ) :
2010-02-28 23:39:39 +00:00
# for each PCD in module
2015-12-01 04:22:16 +00:00
for Name , Guid in Pcds :
PcdInModule = Pcds [ Name , Guid ]
2010-02-28 23:39:39 +00:00
# find out the PCD setting in platform
2015-12-01 04:22:16 +00:00
if ( Name , Guid ) in self . Platform . Pcds :
PcdInPlatform = self . Platform . Pcds [ Name , Guid ]
2010-02-28 23:39:39 +00:00
else :
PcdInPlatform = None
# then override the settings if any
2018-05-16 13:59:59 +08:00
self . _OverridePcd ( PcdInModule , PcdInPlatform , Module , Msg = " DSC PCD sections " , Library = Library )
2010-02-28 23:39:39 +00:00
# resolve the VariableGuid value
for SkuId in PcdInModule . SkuInfoList :
Sku = PcdInModule . SkuInfoList [ SkuId ]
if Sku . VariableGuid == ' ' : continue
2016-05-10 17:58:26 +08:00
Sku . VariableGuidValue = GuidValue ( Sku . VariableGuid , self . PackageList , self . MetaFile . Path )
2018-03-27 04:25:43 +08:00
if Sku . VariableGuidValue is None :
2018-04-28 06:32:54 +08:00
PackageList = " \n \t " . join ( str ( P ) for P in self . PackageList )
2010-02-28 23:39:39 +00:00
EdkLogger . error (
' build ' ,
RESOURCE_NOT_AVAILABLE ,
" Value of GUID [ %s ] is not found in " % Sku . VariableGuid ,
ExtraData = PackageList + " \n \t (used with %s . %s from module %s ) " \
% ( Guid , Name , str ( Module ) ) ,
File = self . MetaFile
)
# override PCD settings with module specific setting
if Module in self . Platform . Modules :
PlatformModule = self . Platform . Modules [ str ( Module ) ]
for Key in PlatformModule . Pcds :
2017-09-10 16:23:09 +08:00
Flag = False
2010-02-28 23:39:39 +00:00
if Key in Pcds :
2017-09-10 16:23:09 +08:00
ToPcd = Pcds [ Key ]
Flag = True
elif Key in GlobalData . MixedPcd :
for PcdItem in GlobalData . MixedPcd [ Key ] :
if PcdItem in Pcds :
ToPcd = Pcds [ PcdItem ]
Flag = True
break
if Flag :
2018-05-16 13:59:59 +08:00
self . _OverridePcd ( ToPcd , PlatformModule . Pcds [ Key ] , Module , Msg = " DSC Components Module scoped PCD section " , Library = Library )
2018-02-26 15:36:47 +08:00
# use PCD value to calculate the MaxDatumSize when it is not specified
for Name , Guid in Pcds :
Pcd = Pcds [ Name , Guid ]
2018-04-20 23:51:42 +08:00
if Pcd . DatumType == TAB_VOID and not Pcd . MaxDatumSize :
2018-03-17 15:25:32 +08:00
Pcd . MaxSizeUserSet = None
2018-02-26 15:36:47 +08:00
Value = Pcd . DefaultValue
2018-04-20 23:51:42 +08:00
if not Value :
2018-02-26 15:36:47 +08:00
Pcd . MaxDatumSize = ' 1 '
elif Value [ 0 ] == ' L ' :
Pcd . MaxDatumSize = str ( ( len ( Value ) - 2 ) * 2 )
elif Value [ 0 ] == ' { ' :
Pcd . MaxDatumSize = str ( len ( Value . split ( ' , ' ) ) )
else :
Pcd . MaxDatumSize = str ( len ( Value ) - 1 )
2010-02-28 23:39:39 +00:00
return Pcds . values ( )
## Resolve library names to library modules
#
2011-09-18 12:17:25 +00:00
# (for Edk.x modules)
2010-02-28 23:39:39 +00:00
#
# @param Module The module from which the library names will be resolved
#
# @retval library_list The list of library modules
#
def ResolveLibraryReference ( self , Module ) :
EdkLogger . verbose ( " " )
EdkLogger . verbose ( " Library instances of module [ %s ] [ %s ]: " % ( str ( Module ) , self . Arch ) )
LibraryConsumerList = [ Module ]
2011-09-18 12:17:25 +00:00
# "CompilerStub" is a must for Edk modules
2010-02-28 23:39:39 +00:00
if Module . Libraries :
Module . Libraries . append ( " CompilerStub " )
LibraryList = [ ]
while len ( LibraryConsumerList ) > 0 :
M = LibraryConsumerList . pop ( )
for LibraryName in M . Libraries :
Library = self . Platform . LibraryClasses [ LibraryName , ' :dummy: ' ]
2018-03-27 04:25:43 +08:00
if Library is None :
2018-04-17 22:40:15 +08:00
for Key in self . Platform . LibraryClasses . data :
2010-02-28 23:39:39 +00:00
if LibraryName . upper ( ) == Key . upper ( ) :
Library = self . Platform . LibraryClasses [ Key , ' :dummy: ' ]
break
2018-03-27 04:25:43 +08:00
if Library is None :
2010-02-28 23:39:39 +00:00
EdkLogger . warn ( " build " , " Library [ %s ] is not found " % LibraryName , File = str ( M ) ,
ExtraData = " \t %s [ %s ] " % ( str ( Module ) , self . Arch ) )
continue
if Library not in LibraryList :
LibraryList . append ( Library )
LibraryConsumerList . append ( Library )
EdkLogger . verbose ( " \t " + LibraryName + " : " + str ( Library ) + ' ' + str ( type ( Library ) ) )
return LibraryList
2010-11-15 02:51:34 +00:00
## Calculate the priority value of the build option
#
# @param Key Build option definition contain: TARGET_TOOLCHAIN_ARCH_COMMANDTYPE_ATTRIBUTE
#
# @retval Value Priority value based on the priority list.
#
def CalculatePriorityValue ( self , Key ) :
2015-12-01 04:22:16 +00:00
Target , ToolChain , Arch , CommandType , Attr = Key . split ( ' _ ' )
PriorityValue = 0x11111
2010-11-15 02:51:34 +00:00
if Target == " * " :
PriorityValue & = 0x01111
if ToolChain == " * " :
PriorityValue & = 0x10111
if Arch == " * " :
PriorityValue & = 0x11011
if CommandType == " * " :
PriorityValue & = 0x11101
if Attr == " * " :
PriorityValue & = 0x11110
2015-12-01 04:22:16 +00:00
return self . PrioList [ " 0x %0.5x " % PriorityValue ]
2010-11-15 02:51:34 +00:00
2010-02-28 23:39:39 +00:00
## Expand * in build option key
#
# @param Options Options to be expanded
#
# @retval options Options expanded
2018-07-05 17:40:04 +08:00
#
2010-02-28 23:39:39 +00:00
def _ExpandBuildOption ( self , Options , ModuleStyle = None ) :
BuildOptions = { }
FamilyMatch = False
FamilyIsNull = True
2015-12-01 04:22:16 +00:00
2010-11-15 02:51:34 +00:00
OverrideList = { }
#
# Construct a list contain the build options which need override.
#
for Key in Options :
#
# Key[0] -- tool family
# Key[1] -- TARGET_TOOLCHAIN_ARCH_COMMANDTYPE_ATTRIBUTE
#
2015-06-19 01:43:45 +00:00
if ( Key [ 0 ] == self . BuildRuleFamily and
2018-03-27 04:25:43 +08:00
( ModuleStyle is None or len ( Key ) < 3 or ( len ( Key ) > 2 and Key [ 2 ] == ModuleStyle ) ) ) :
2010-11-15 02:51:34 +00:00
Target , ToolChain , Arch , CommandType , Attr = Key [ 1 ] . split ( ' _ ' )
2018-05-18 08:06:52 +08:00
if ( Target == self . BuildTarget or Target == " * " ) and \
( ToolChain == self . ToolChain or ToolChain == " * " ) and \
( Arch == self . Arch or Arch == " * " ) and \
Options [ Key ] . startswith ( " = " ) :
if OverrideList . get ( Key [ 1 ] ) is not None :
OverrideList . pop ( Key [ 1 ] )
OverrideList [ Key [ 1 ] ] = Options [ Key ]
2018-07-05 17:40:04 +08:00
2010-11-15 02:51:34 +00:00
#
2018-07-05 17:40:04 +08:00
# Use the highest priority value.
2010-11-15 02:51:34 +00:00
#
if ( len ( OverrideList ) > = 2 ) :
2015-12-01 04:22:16 +00:00
KeyList = OverrideList . keys ( )
2010-11-15 02:51:34 +00:00
for Index in range ( len ( KeyList ) ) :
2015-12-01 04:22:16 +00:00
NowKey = KeyList [ Index ]
2010-11-15 02:51:34 +00:00
Target1 , ToolChain1 , Arch1 , CommandType1 , Attr1 = NowKey . split ( " _ " )
for Index1 in range ( len ( KeyList ) - Index - 1 ) :
NextKey = KeyList [ Index1 + Index + 1 ]
#
# Compare two Key, if one is included by another, choose the higher priority one
2018-07-05 17:40:04 +08:00
#
2010-11-15 02:51:34 +00:00
Target2 , ToolChain2 , Arch2 , CommandType2 , Attr2 = NextKey . split ( " _ " )
2018-05-18 08:06:52 +08:00
if ( Target1 == Target2 or Target1 == " * " or Target2 == " * " ) and \
( ToolChain1 == ToolChain2 or ToolChain1 == " * " or ToolChain2 == " * " ) and \
( Arch1 == Arch2 or Arch1 == " * " or Arch2 == " * " ) and \
( CommandType1 == CommandType2 or CommandType1 == " * " or CommandType2 == " * " ) and \
( Attr1 == Attr2 or Attr1 == " * " or Attr2 == " * " ) :
if self . CalculatePriorityValue ( NowKey ) > self . CalculatePriorityValue ( NextKey ) :
if Options . get ( ( self . BuildRuleFamily , NextKey ) ) is not None :
Options . pop ( ( self . BuildRuleFamily , NextKey ) )
else :
if Options . get ( ( self . BuildRuleFamily , NowKey ) ) is not None :
Options . pop ( ( self . BuildRuleFamily , NowKey ) )
2018-07-05 17:40:04 +08:00
2010-02-28 23:39:39 +00:00
for Key in Options :
2018-03-27 04:25:43 +08:00
if ModuleStyle is not None and len ( Key ) > 2 :
2010-02-28 23:39:39 +00:00
# Check Module style is EDK or EDKII.
# Only append build option for the matched style module.
if ModuleStyle == EDK_NAME and Key [ 2 ] != EDK_NAME :
continue
elif ModuleStyle == EDKII_NAME and Key [ 2 ] != EDKII_NAME :
continue
Family = Key [ 0 ]
Target , Tag , Arch , Tool , Attr = Key [ 1 ] . split ( " _ " )
# if tool chain family doesn't match, skip it
if Tool in self . ToolDefinition and Family != " " :
FamilyIsNull = False
if self . ToolDefinition [ Tool ] . get ( TAB_TOD_DEFINES_BUILDRULEFAMILY , " " ) != " " :
if Family != self . ToolDefinition [ Tool ] [ TAB_TOD_DEFINES_BUILDRULEFAMILY ] :
continue
elif Family != self . ToolDefinition [ Tool ] [ TAB_TOD_DEFINES_FAMILY ] :
continue
FamilyMatch = True
# expand any wildcard
if Target == " * " or Target == self . BuildTarget :
if Tag == " * " or Tag == self . ToolChain :
if Arch == " * " or Arch == self . Arch :
if Tool not in BuildOptions :
BuildOptions [ Tool ] = { }
2015-07-02 03:42:34 +00:00
if Attr != " FLAGS " or Attr not in BuildOptions [ Tool ] or Options [ Key ] . startswith ( ' = ' ) :
2010-02-28 23:39:39 +00:00
BuildOptions [ Tool ] [ Attr ] = Options [ Key ]
else :
2017-02-15 14:47:47 +08:00
# append options for the same tool except PATH
if Attr != ' PATH ' :
BuildOptions [ Tool ] [ Attr ] + = " " + Options [ Key ]
else :
BuildOptions [ Tool ] [ Attr ] = Options [ Key ]
2010-02-28 23:39:39 +00:00
# Build Option Family has been checked, which need't to be checked again for family.
if FamilyMatch or FamilyIsNull :
return BuildOptions
2015-06-19 01:43:45 +00:00
2010-02-28 23:39:39 +00:00
for Key in Options :
2018-03-27 04:25:43 +08:00
if ModuleStyle is not None and len ( Key ) > 2 :
2010-02-28 23:39:39 +00:00
# Check Module style is EDK or EDKII.
# Only append build option for the matched style module.
if ModuleStyle == EDK_NAME and Key [ 2 ] != EDK_NAME :
continue
elif ModuleStyle == EDKII_NAME and Key [ 2 ] != EDKII_NAME :
continue
Family = Key [ 0 ]
Target , Tag , Arch , Tool , Attr = Key [ 1 ] . split ( " _ " )
# if tool chain family doesn't match, skip it
2015-12-01 04:22:16 +00:00
if Tool not in self . ToolDefinition or Family == " " :
2010-02-28 23:39:39 +00:00
continue
# option has been added before
if Family != self . ToolDefinition [ Tool ] [ TAB_TOD_DEFINES_FAMILY ] :
continue
# expand any wildcard
if Target == " * " or Target == self . BuildTarget :
if Tag == " * " or Tag == self . ToolChain :
if Arch == " * " or Arch == self . Arch :
if Tool not in BuildOptions :
BuildOptions [ Tool ] = { }
2015-07-02 03:42:34 +00:00
if Attr != " FLAGS " or Attr not in BuildOptions [ Tool ] or Options [ Key ] . startswith ( ' = ' ) :
2010-02-28 23:39:39 +00:00
BuildOptions [ Tool ] [ Attr ] = Options [ Key ]
else :
2017-02-15 14:47:47 +08:00
# append options for the same tool except PATH
if Attr != ' PATH ' :
BuildOptions [ Tool ] [ Attr ] + = " " + Options [ Key ]
else :
BuildOptions [ Tool ] [ Attr ] = Options [ Key ]
2010-02-28 23:39:39 +00:00
return BuildOptions
## Append build options in platform to a module
#
# @param Module The module to which the build options will be appened
#
# @retval options The options appended with build options in platform
#
def ApplyBuildOption ( self , Module ) :
# Get the different options for the different style module
if Module . AutoGenVersion < 0x00010005 :
PlatformOptions = self . EdkBuildOption
2015-06-11 05:16:40 +00:00
ModuleTypeOptions = self . Platform . GetBuildOptionsByModuleType ( EDK_NAME , Module . ModuleType )
2010-02-28 23:39:39 +00:00
else :
PlatformOptions = self . EdkIIBuildOption
2015-06-11 05:16:40 +00:00
ModuleTypeOptions = self . Platform . GetBuildOptionsByModuleType ( EDKII_NAME , Module . ModuleType )
ModuleTypeOptions = self . _ExpandBuildOption ( ModuleTypeOptions )
2010-02-28 23:39:39 +00:00
ModuleOptions = self . _ExpandBuildOption ( Module . BuildOptions )
if Module in self . Platform . Modules :
PlatformModule = self . Platform . Modules [ str ( Module ) ]
PlatformModuleOptions = self . _ExpandBuildOption ( PlatformModule . BuildOptions )
else :
PlatformModuleOptions = { }
2015-05-26 10:32:07 +00:00
BuildRuleOrder = None
2015-06-11 05:16:40 +00:00
for Options in [ self . ToolDefinition , ModuleOptions , PlatformOptions , ModuleTypeOptions , PlatformModuleOptions ] :
2015-05-26 10:32:07 +00:00
for Tool in Options :
for Attr in Options [ Tool ] :
if Attr == TAB_TOD_DEFINES_BUILDRULEORDER :
BuildRuleOrder = Options [ Tool ] [ Attr ]
2015-06-11 05:16:40 +00:00
AllTools = set ( ModuleOptions . keys ( ) + PlatformOptions . keys ( ) +
PlatformModuleOptions . keys ( ) + ModuleTypeOptions . keys ( ) +
self . ToolDefinition . keys ( ) )
2018-04-06 07:13:57 +08:00
BuildOptions = defaultdict ( lambda : defaultdict ( str ) )
2010-02-28 23:39:39 +00:00
for Tool in AllTools :
2015-06-11 05:16:40 +00:00
for Options in [ self . ToolDefinition , ModuleOptions , PlatformOptions , ModuleTypeOptions , PlatformModuleOptions ] :
2010-02-28 23:39:39 +00:00
if Tool not in Options :
continue
for Attr in Options [ Tool ] :
2015-05-26 10:32:07 +00:00
#
# Do not generate it in Makefile
#
if Attr == TAB_TOD_DEFINES_BUILDRULEORDER :
continue
2018-04-06 07:13:57 +08:00
Value = Options [ Tool ] [ Attr ]
2010-02-28 23:39:39 +00:00
# check if override is indicated
if Value . startswith ( ' = ' ) :
2018-04-06 07:13:57 +08:00
BuildOptions [ Tool ] [ Attr ] = mws . handleWsMacro ( Value [ 1 : ] )
2015-07-02 03:42:34 +00:00
else :
2017-02-15 14:47:47 +08:00
if Attr != ' PATH ' :
2018-04-06 07:13:57 +08:00
BuildOptions [ Tool ] [ Attr ] + = " " + mws . handleWsMacro ( Value )
2017-02-15 14:47:47 +08:00
else :
2018-04-06 07:13:57 +08:00
BuildOptions [ Tool ] [ Attr ] = mws . handleWsMacro ( Value )
2018-03-27 04:25:43 +08:00
if Module . AutoGenVersion < 0x00010005 and self . Workspace . UniFlag is not None :
2010-03-19 06:55:07 +00:00
#
# Override UNI flag only for EDK module.
#
BuildOptions [ ' BUILD ' ] [ ' FLAGS ' ] = self . Workspace . UniFlag
2015-05-26 10:32:07 +00:00
return BuildOptions , BuildRuleOrder
2010-02-28 23:39:39 +00:00
Platform = property ( _GetPlatform )
Name = property ( _GetName )
Guid = property ( _GetGuid )
Version = property ( _GetVersion )
OutputDir = property ( _GetOutputDir )
BuildDir = property ( _GetBuildDir )
MakeFileDir = property ( _GetMakeFileDir )
FdfFile = property ( _GetFdfFile )
PcdTokenNumber = property ( _GetPcdTokenNumbers ) # (TokenCName, TokenSpaceGuidCName) : GeneratedTokenNumber
DynamicPcdList = property ( _GetDynamicPcdList ) # [(TokenCName1, TokenSpaceGuidCName1), (TokenCName2, TokenSpaceGuidCName2), ...]
NonDynamicPcdList = property ( _GetNonDynamicPcdList ) # [(TokenCName1, TokenSpaceGuidCName1), (TokenCName2, TokenSpaceGuidCName2), ...]
2014-01-10 05:25:50 +00:00
NonDynamicPcdDict = property ( _GetNonDynamicPcdDict )
2010-02-28 23:39:39 +00:00
PackageList = property ( _GetPackageList )
ToolDefinition = property ( _GetToolDefinition ) # toolcode : tool path
ToolDefinitionFile = property ( _GetToolDefFile ) # toolcode : lib path
ToolChainFamily = property ( _GetToolChainFamily )
BuildRuleFamily = property ( _GetBuildRuleFamily )
BuildOption = property ( _GetBuildOptions ) # toolcode : option
EdkBuildOption = property ( _GetEdkBuildOptions ) # edktoolcode : option
EdkIIBuildOption = property ( _GetEdkIIBuildOptions ) # edkiitoolcode : option
BuildCommand = property ( _GetBuildCommand )
BuildRule = property ( _GetBuildRule )
ModuleAutoGenList = property ( _GetModuleAutoGenList )
LibraryAutoGenList = property ( _GetLibraryAutoGenList )
2014-08-21 07:51:21 +00:00
GenFdsCommand = property ( _GenFdsCommand )
2010-02-28 23:39:39 +00:00
2018-05-18 08:06:52 +08:00
#
# extend lists contained in a dictionary with lists stored in another dictionary
# if CopyToDict is not derived from DefaultDict(list) then this may raise exception
#
def ExtendCopyDictionaryLists ( CopyToDict , CopyFromDict ) :
for Key in CopyFromDict :
CopyToDict [ Key ] . extend ( CopyFromDict [ Key ] )
2018-08-03 23:11:06 +08:00
# Create a directory specified by a set of path elements and return the full path
def _MakeDir ( PathList ) :
RetVal = path . join ( * PathList )
CreateDirectory ( RetVal )
return RetVal
2018-05-18 08:06:52 +08:00
2010-02-28 23:39:39 +00:00
## ModuleAutoGen class
#
# This class encapsules the AutoGen behaviors for the build tools. In addition to
# the generation of AutoGen.h and AutoGen.c, it will generate *.depex file according
# to the [depex] section in module's inf file.
#
class ModuleAutoGen ( AutoGen ) :
2018-03-15 08:20:27 +08:00
# call super().__init__ then call the worker function with different parameter count
def __init__ ( self , Workspace , MetaFile , Target , Toolchain , Arch , * args , * * kwargs ) :
2018-08-03 23:11:06 +08:00
if not hasattr ( self , " _Init " ) :
2018-03-15 08:20:27 +08:00
super ( ModuleAutoGen , self ) . __init__ ( Workspace , MetaFile , Target , Toolchain , Arch , * args , * * kwargs )
self . _InitWorker ( Workspace , MetaFile , Target , Toolchain , Arch , * args )
self . _Init = True
2018-08-03 23:11:06 +08:00
## Cache the timestamps of metafiles of every module in a class attribute
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
#
TimeDict = { }
2018-03-15 08:20:27 +08:00
def __new__ ( cls , Workspace , MetaFile , Target , Toolchain , Arch , * args , * * kwargs ) :
# check if this module is employed by active platform
if not PlatformAutoGen ( Workspace , args [ 0 ] , Target , Toolchain , Arch ) . ValidModule ( MetaFile ) :
EdkLogger . verbose ( " Module [ %s ] for [ %s ] is not employed by active platform \n " \
% ( MetaFile , Arch ) )
return None
2018-08-03 23:11:06 +08:00
return super ( ModuleAutoGen , cls ) . __new__ ( cls , Workspace , MetaFile , Target , Toolchain , Arch , * args , * * kwargs )
2018-07-05 17:40:04 +08:00
2018-03-15 08:20:27 +08:00
## Initialize ModuleAutoGen
2010-02-28 23:39:39 +00:00
#
# @param Workspace EdkIIWorkspaceBuild object
# @param ModuleFile The path of module file
# @param Target Build target (DEBUG, RELEASE)
# @param Toolchain Name of tool chain
# @param Arch The arch the module supports
# @param PlatformFile Platform meta-file
#
2018-03-15 08:20:27 +08:00
def _InitWorker ( self , Workspace , ModuleFile , Target , Toolchain , Arch , PlatformFile ) :
2010-02-28 23:39:39 +00:00
EdkLogger . debug ( EdkLogger . DEBUG_9 , " AutoGen module [ %s ] [ %s ] " % ( ModuleFile , Arch ) )
GlobalData . gProcessingFile = " %s [ %s , %s , %s ] " % ( ModuleFile , Arch , Toolchain , Target )
self . Workspace = Workspace
self . WorkspaceDir = Workspace . WorkspaceDir
2018-03-23 11:39:39 +08:00
self . _GuidDict = Workspace . _GuidDict
2010-02-28 23:39:39 +00:00
self . MetaFile = ModuleFile
self . PlatformInfo = PlatformAutoGen ( Workspace , PlatformFile , Target , Toolchain , Arch )
self . SourceDir = self . MetaFile . SubDir
2015-10-08 09:27:14 +00:00
self . SourceDir = mws . relpath ( self . SourceDir , self . WorkspaceDir )
2014-08-28 13:53:34 +00:00
2010-02-28 23:39:39 +00:00
self . SourceOverrideDir = None
# use overrided path defined in DSC file
if self . MetaFile . Key in GlobalData . gOverrideDir :
self . SourceOverrideDir = GlobalData . gOverrideDir [ self . MetaFile . Key ]
self . ToolChain = Toolchain
self . BuildTarget = Target
self . Arch = Arch
self . ToolChainFamily = self . PlatformInfo . ToolChainFamily
self . BuildRuleFamily = self . PlatformInfo . BuildRuleFamily
self . IsCodeFileCreated = False
2011-05-11 10:26:49 +00:00
self . IsAsBuiltInfCreated = False
self . DepexGenerated = False
2010-02-28 23:39:39 +00:00
self . BuildDatabase = self . Workspace . BuildDatabase
2015-05-26 10:32:07 +00:00
self . BuildRuleOrder = None
2017-09-11 16:50:07 +08:00
self . BuildTime = 0
2010-02-28 23:39:39 +00:00
2018-05-18 08:06:52 +08:00
self . _PcdComments = OrderedListDict ( )
self . _GuidComments = OrderedListDict ( )
self . _ProtocolComments = OrderedListDict ( )
self . _PpiComments = OrderedListDict ( )
2010-02-28 23:39:39 +00:00
self . _BuildTargets = None
self . _IntroBuildTargetList = None
self . _FinalBuildTargetList = None
self . _FileTypes = None
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
self . AutoGenDepSet = set ( )
2018-08-03 23:11:06 +08:00
self . ReferenceModules = [ ]
2014-01-10 05:25:50 +00:00
self . ConstPcd = { }
2010-02-28 23:39:39 +00:00
2018-07-13 17:05:20 +08:00
2010-02-28 23:39:39 +00:00
def __repr__ ( self ) :
return " %s [ %s ] " % ( self . MetaFile , self . Arch )
2014-01-10 05:25:50 +00:00
# Get FixedAtBuild Pcds of this Module
2018-08-03 23:11:06 +08:00
@cached_property
def FixedAtBuildPcds ( self ) :
RetVal = [ ]
2014-01-10 05:25:50 +00:00
for Pcd in self . ModulePcdList :
2018-04-20 23:51:25 +08:00
if Pcd . Type != TAB_PCDS_FIXED_AT_BUILD :
2014-01-10 05:25:50 +00:00
continue
2018-08-03 23:11:06 +08:00
if Pcd not in RetVal :
RetVal . append ( Pcd )
return RetVal
@cached_property
def FixedVoidTypePcds ( self ) :
RetVal = { }
for Pcd in self . FixedAtBuildPcds :
if Pcd . DatumType == TAB_VOID :
if ' {} . {} ' . format ( Pcd . TokenSpaceGuidCName , Pcd . TokenCName ) not in RetVal :
RetVal [ ' {} . {} ' . format ( Pcd . TokenSpaceGuidCName , Pcd . TokenCName ) ] = Pcd . DefaultValue
return RetVal
@property
def UniqueBaseName ( self ) :
2015-06-10 07:50:59 +00:00
BaseName = self . Name
for Module in self . PlatformInfo . ModuleAutoGenList :
if Module . MetaFile == self . MetaFile :
continue
if Module . Name == self . Name :
if uuid . UUID ( Module . Guid ) == uuid . UUID ( self . Guid ) :
EdkLogger . error ( " build " , FILE_DUPLICATED , ' Modules have same BaseName and FILE_GUID: \n '
' %s \n %s ' % ( Module . MetaFile , self . MetaFile ) )
BaseName = ' %s _ %s ' % ( self . Name , self . Guid )
return BaseName
2010-02-28 23:39:39 +00:00
# Macros could be used in build_rule.txt (also Makefile)
2018-08-03 23:11:06 +08:00
@cached_property
def Macros ( self ) :
return OrderedDict ( (
( " WORKSPACE " , self . WorkspaceDir ) ,
( " MODULE_NAME " , self . Name ) ,
( " MODULE_NAME_GUID " , self . UniqueBaseName ) ,
( " MODULE_GUID " , self . Guid ) ,
( " MODULE_VERSION " , self . Version ) ,
( " MODULE_TYPE " , self . ModuleType ) ,
( " MODULE_FILE " , str ( self . MetaFile ) ) ,
( " MODULE_FILE_BASE_NAME " , self . MetaFile . BaseName ) ,
( " MODULE_RELATIVE_DIR " , self . SourceDir ) ,
( " MODULE_DIR " , self . SourceDir ) ,
( " BASE_NAME " , self . Name ) ,
( " ARCH " , self . Arch ) ,
( " TOOLCHAIN " , self . ToolChain ) ,
( " TOOLCHAIN_TAG " , self . ToolChain ) ,
( " TOOL_CHAIN_TAG " , self . ToolChain ) ,
( " TARGET " , self . BuildTarget ) ,
( " BUILD_DIR " , self . PlatformInfo . BuildDir ) ,
( " BIN_DIR " , os . path . join ( self . PlatformInfo . BuildDir , self . Arch ) ) ,
( " LIB_DIR " , os . path . join ( self . PlatformInfo . BuildDir , self . Arch ) ) ,
( " MODULE_BUILD_DIR " , self . BuildDir ) ,
( " OUTPUT_DIR " , self . OutputDir ) ,
( " DEBUG_DIR " , self . DebugDir ) ,
( " DEST_DIR_OUTPUT " , self . OutputDir ) ,
( " DEST_DIR_DEBUG " , self . DebugDir ) ,
( " PLATFORM_NAME " , self . PlatformInfo . Name ) ,
( " PLATFORM_GUID " , self . PlatformInfo . Guid ) ,
( " PLATFORM_VERSION " , self . PlatformInfo . Version ) ,
( " PLATFORM_RELATIVE_DIR " , self . PlatformInfo . SourceDir ) ,
( " PLATFORM_DIR " , mws . join ( self . WorkspaceDir , self . PlatformInfo . SourceDir ) ) ,
( " PLATFORM_OUTPUT_DIR " , self . PlatformInfo . OutputDir ) ,
( " FFS_OUTPUT_DIR " , self . FfsOutputDir )
) )
2010-02-28 23:39:39 +00:00
## Return the module build data object
2018-08-03 23:11:06 +08:00
@cached_property
def Module ( self ) :
return self . Workspace . BuildDatabase [ self . MetaFile , self . Arch , self . BuildTarget , self . ToolChain ]
2010-02-28 23:39:39 +00:00
## Return the module name
2018-08-03 23:11:06 +08:00
@cached_property
def Name ( self ) :
2010-02-28 23:39:39 +00:00
return self . Module . BaseName
2011-09-18 12:17:25 +00:00
## Return the module DxsFile if exist
2018-08-03 23:11:06 +08:00
@cached_property
def DxsFile ( self ) :
2011-09-18 12:17:25 +00:00
return self . Module . DxsFile
2010-02-28 23:39:39 +00:00
## Return the module meta-file GUID
2018-08-03 23:11:06 +08:00
@cached_property
def Guid ( self ) :
2014-08-28 13:53:34 +00:00
#
# To build same module more than once, the module path with FILE_GUID overridden has
# the file name FILE_GUIDmodule.inf, but the relative path (self.MetaFile.File) is the realy path
# in DSC. The overridden GUID can be retrieved from file name
#
if os . path . basename ( self . MetaFile . File ) != os . path . basename ( self . MetaFile . Path ) :
#
# Length of GUID is 36
#
return os . path . basename ( self . MetaFile . Path ) [ : 36 ]
2010-02-28 23:39:39 +00:00
return self . Module . Guid
## Return the module version
2018-08-03 23:11:06 +08:00
@cached_property
def Version ( self ) :
2010-02-28 23:39:39 +00:00
return self . Module . Version
## Return the module type
2018-08-03 23:11:06 +08:00
@cached_property
def ModuleType ( self ) :
2010-02-28 23:39:39 +00:00
return self . Module . ModuleType
2011-09-18 12:17:25 +00:00
## Return the component type (for Edk.x style of module)
2018-08-03 23:11:06 +08:00
@cached_property
def ComponentType ( self ) :
2010-02-28 23:39:39 +00:00
return self . Module . ComponentType
## Return the build type
2018-08-03 23:11:06 +08:00
@cached_property
def BuildType ( self ) :
2010-02-28 23:39:39 +00:00
return self . Module . BuildType
## Return the PCD_IS_DRIVER setting
2018-08-03 23:11:06 +08:00
@cached_property
def PcdIsDriver ( self ) :
2010-02-28 23:39:39 +00:00
return self . Module . PcdIsDriver
## Return the autogen version, i.e. module meta-file version
2018-08-03 23:11:06 +08:00
@cached_property
def AutoGenVersion ( self ) :
2010-02-28 23:39:39 +00:00
return self . Module . AutoGenVersion
## Check if the module is library or not
2018-08-03 23:11:06 +08:00
@cached_property
def IsLibrary ( self ) :
return bool ( self . Module . LibraryClass )
2010-02-28 23:39:39 +00:00
2013-11-18 07:41:21 +00:00
## Check if the module is binary module or not
2018-08-03 23:11:06 +08:00
@cached_property
def IsBinaryModule ( self ) :
2013-11-18 07:41:21 +00:00
return self . Module . IsBinaryModule
2010-02-28 23:39:39 +00:00
## Return the directory to store intermediate files of the module
2018-08-03 23:11:06 +08:00
@cached_property
def BuildDir ( self ) :
return _MakeDir ( (
2010-02-28 23:39:39 +00:00
self . PlatformInfo . BuildDir ,
self . Arch ,
self . SourceDir ,
self . MetaFile . BaseName
2018-08-03 23:11:06 +08:00
) )
2010-02-28 23:39:39 +00:00
## Return the directory to store the intermediate object files of the mdoule
2018-08-03 23:11:06 +08:00
@cached_property
def OutputDir ( self ) :
return _MakeDir ( ( self . BuildDir , " OUTPUT " ) )
## Return the directory path to store ffs file
@cached_property
def FfsOutputDir ( self ) :
if GlobalData . gFdfParser :
return path . join ( self . PlatformInfo . BuildDir , TAB_FV_DIRECTORY , " Ffs " , self . Guid + self . Name )
return ' '
2017-11-22 15:42:25 +08:00
2010-02-28 23:39:39 +00:00
## Return the directory to store auto-gened source files of the mdoule
2018-08-03 23:11:06 +08:00
@cached_property
def DebugDir ( self ) :
return _MakeDir ( ( self . BuildDir , " DEBUG " ) )
2010-02-28 23:39:39 +00:00
## Return the path of custom file
2018-08-03 23:11:06 +08:00
@cached_property
def CustomMakefile ( self ) :
RetVal = { }
for Type in self . Module . CustomMakefile :
MakeType = gMakeTypeMap [ Type ] if Type in gMakeTypeMap else ' nmake '
if self . SourceOverrideDir is not None :
File = os . path . join ( self . SourceOverrideDir , self . Module . CustomMakefile [ Type ] )
if not os . path . exists ( File ) :
2010-02-28 23:39:39 +00:00
File = os . path . join ( self . SourceDir , self . Module . CustomMakefile [ Type ] )
2018-08-03 23:11:06 +08:00
else :
File = os . path . join ( self . SourceDir , self . Module . CustomMakefile [ Type ] )
RetVal [ MakeType ] = File
return RetVal
2010-02-28 23:39:39 +00:00
## Return the directory of the makefile
#
# @retval string The directory string of module's makefile
#
2018-08-03 23:11:06 +08:00
@cached_property
def MakeFileDir ( self ) :
2010-02-28 23:39:39 +00:00
return self . BuildDir
## Return build command string
#
# @retval string Build command string
#
2018-08-03 23:11:06 +08:00
@cached_property
def BuildCommand ( self ) :
2010-02-28 23:39:39 +00:00
return self . PlatformInfo . BuildCommand
## Get object list of all packages the module and its dependent libraries belong to
#
# @retval list The list of package object
#
2018-08-03 23:11:06 +08:00
@cached_property
def DerivedPackageList ( self ) :
2010-02-28 23:39:39 +00:00
PackageList = [ ]
for M in [ self . Module ] + self . DependentLibraryList :
for Package in M . Packages :
if Package in PackageList :
continue
PackageList . append ( Package )
return PackageList
2018-07-05 17:40:04 +08:00
2014-08-28 13:53:34 +00:00
## Get the depex string
#
# @return : a string contain all depex expresion.
def _GetDepexExpresionString ( self ) :
DepexStr = ' '
DepexList = [ ]
## DPX_SOURCE IN Define section.
if self . Module . DxsFile :
return DepexStr
for M in [ self . Module ] + self . DependentLibraryList :
Filename = M . MetaFile . Path
InfObj = InfSectionParser . InfSectionParser ( Filename )
DepexExpresionList = InfObj . GetDepexExpresionList ( )
for DepexExpresion in DepexExpresionList :
2018-04-17 22:40:15 +08:00
for key in DepexExpresion :
2014-08-28 13:53:34 +00:00
Arch , ModuleType = key
2016-05-11 13:46:44 +08:00
DepexExpr = [ x for x in DepexExpresion [ key ] if not str ( x ) . startswith ( ' # ' ) ]
2014-08-28 13:53:34 +00:00
# the type of build module is USER_DEFINED.
# All different DEPEX section tags would be copied into the As Built INF file
# and there would be separate DEPEX section tags
if self . ModuleType . upper ( ) == SUP_MODULE_USER_DEFINED :
if ( Arch . upper ( ) == self . Arch . upper ( ) ) and ( ModuleType . upper ( ) != TAB_ARCH_COMMON ) :
2016-05-11 13:46:44 +08:00
DepexList . append ( { ( Arch , ModuleType ) : DepexExpr } )
2014-08-28 13:53:34 +00:00
else :
if Arch . upper ( ) == TAB_ARCH_COMMON or \
( Arch . upper ( ) == self . Arch . upper ( ) and \
ModuleType . upper ( ) in [ TAB_ARCH_COMMON , self . ModuleType . upper ( ) ] ) :
2016-05-11 13:46:44 +08:00
DepexList . append ( { ( Arch , ModuleType ) : DepexExpr } )
2018-07-05 17:40:04 +08:00
2014-08-28 13:53:34 +00:00
#the type of build module is USER_DEFINED.
if self . ModuleType . upper ( ) == SUP_MODULE_USER_DEFINED :
for Depex in DepexList :
2018-04-17 22:40:15 +08:00
for key in Depex :
2014-08-28 13:53:34 +00:00
DepexStr + = ' [Depex. %s . %s ] \n ' % key
2018-04-28 06:32:54 +08:00
DepexStr + = ' \n ' . join ( ' # ' + val for val in Depex [ key ] )
2014-08-28 13:53:34 +00:00
DepexStr + = ' \n \n '
if not DepexStr :
return ' [Depex. %s ] \n ' % self . Arch
return DepexStr
2018-07-05 17:40:04 +08:00
2014-08-28 13:53:34 +00:00
#the type of build module not is USER_DEFINED.
Count = 0
for Depex in DepexList :
Count + = 1
if DepexStr != ' ' :
DepexStr + = ' AND '
DepexStr + = ' ( '
for D in Depex . values ( ) :
2018-04-28 06:32:54 +08:00
DepexStr + = ' ' . join ( val for val in D )
2014-08-28 13:53:34 +00:00
Index = DepexStr . find ( ' END ' )
if Index > - 1 and Index == len ( DepexStr ) - 3 :
DepexStr = DepexStr [ : - 3 ]
DepexStr = DepexStr . strip ( )
DepexStr + = ' ) '
if Count == 1 :
DepexStr = DepexStr . lstrip ( ' ( ' ) . rstrip ( ' ) ' ) . strip ( )
if not DepexStr :
return ' [Depex. %s ] \n ' % self . Arch
return ' [Depex. %s ] \n # ' % self . Arch + DepexStr
2018-07-05 17:40:04 +08:00
2010-02-28 23:39:39 +00:00
## Merge dependency expression
#
# @retval list The token list of the dependency expression after parsed
#
2018-08-03 23:11:06 +08:00
@cached_property
def DepexList ( self ) :
if self . DxsFile or self . IsLibrary or TAB_DEPENDENCY_EXPRESSION_FILE in self . FileTypes :
return { }
2018-08-03 23:11:08 +08:00
DepexList = [ ]
#
# Append depex from dependent libraries, if not "BEFORE", "AFTER" expresion
#
for M in [ self . Module ] + self . DependentLibraryList :
Inherited = False
for D in M . Depex [ self . Arch , self . ModuleType ] :
if DepexList != [ ] :
DepexList . append ( ' AND ' )
DepexList . append ( ' ( ' )
#replace D with value if D is FixedAtBuild PCD
NewList = [ ]
for item in D :
if ' . ' not in item :
NewList . append ( item )
else :
if item not in self . _FixedPcdVoidTypeDict :
EdkLogger . error ( " build " , FORMAT_INVALID , " {} used in [Depex] section should be used as FixedAtBuild type and VOID* datum type in the module. " . format ( item ) )
2018-08-03 23:11:06 +08:00
else :
2018-08-03 23:11:08 +08:00
Value = self . _FixedPcdVoidTypeDict [ item ]
if len ( Value . split ( ' , ' ) ) != 16 :
EdkLogger . error ( " build " , FORMAT_INVALID ,
" {} used in [Depex] section should be used as FixedAtBuild type and VOID* datum type and 16 bytes in the module. " . format ( item ) )
NewList . append ( Value )
DepexList . extend ( NewList )
if DepexList [ - 1 ] == ' END ' : # no need of a END at this time
DepexList . pop ( )
DepexList . append ( ' ) ' )
Inherited = True
if Inherited :
EdkLogger . verbose ( " DEPEX[ %s ] (+ %s ) = %s " % ( self . Name , M . BaseName , DepexList ) )
if ' BEFORE ' in DepexList or ' AFTER ' in DepexList :
break
if len ( DepexList ) > 0 :
EdkLogger . verbose ( ' ' )
return { self . ModuleType : DepexList }
2010-02-28 23:39:39 +00:00
## Merge dependency expression
#
# @retval list The token list of the dependency expression after parsed
#
2018-08-03 23:11:06 +08:00
@cached_property
def DepexExpressionTokenList ( self ) :
if self . DxsFile or self . IsLibrary or TAB_DEPENDENCY_EXPRESSION_FILE in self . FileTypes :
return { }
2010-02-28 23:39:39 +00:00
2018-08-03 23:11:08 +08:00
DepexExpressionString = ' '
#
# Append depex from dependent libraries, if not "BEFORE", "AFTER" expresion
#
for M in [ self . Module ] + self . DependentLibraryList :
Inherited = False
for D in M . DepexExpression [ self . Arch , self . ModuleType ] :
if DepexExpressionString != ' ' :
DepexExpressionString + = ' AND '
DepexExpressionString + = ' ( '
DepexExpressionString + = D
DepexExpressionString = DepexExpressionString . rstrip ( ' END ' ) . strip ( )
DepexExpressionString + = ' ) '
Inherited = True
if Inherited :
EdkLogger . verbose ( " DEPEX[ %s ] (+ %s ) = %s " % ( self . Name , M . BaseName , DepexExpressionString ) )
if ' BEFORE ' in DepexExpressionString or ' AFTER ' in DepexExpressionString :
break
if len ( DepexExpressionString ) > 0 :
EdkLogger . verbose ( ' ' )
2010-02-28 23:39:39 +00:00
2018-08-03 23:11:08 +08:00
return { self . ModuleType : DepexExpressionString }
2010-02-28 23:39:39 +00:00
2017-04-13 14:33:05 +08:00
# Get the tiano core user extension, it is contain dependent library.
# @retval: a list contain tiano core userextension.
#
def _GetTianoCoreUserExtensionList ( self ) :
TianoCoreUserExtentionList = [ ]
for M in [ self . Module ] + self . DependentLibraryList :
Filename = M . MetaFile . Path
InfObj = InfSectionParser . InfSectionParser ( Filename )
TianoCoreUserExtenList = InfObj . GetUserExtensionTianoCore ( )
for TianoCoreUserExtent in TianoCoreUserExtenList :
2018-04-17 22:40:15 +08:00
for Section in TianoCoreUserExtent :
2017-04-13 14:33:05 +08:00
ItemList = Section . split ( TAB_SPLIT )
Arch = self . Arch
if len ( ItemList ) == 4 :
Arch = ItemList [ 3 ]
if Arch . upper ( ) == TAB_ARCH_COMMON or Arch . upper ( ) == self . Arch . upper ( ) :
TianoCoreList = [ ]
TianoCoreList . extend ( [ TAB_SECTION_START + Section + TAB_SECTION_END ] )
TianoCoreList . extend ( TianoCoreUserExtent [ Section ] [ : ] )
TianoCoreList . append ( ' \n ' )
TianoCoreUserExtentionList . append ( TianoCoreList )
return TianoCoreUserExtentionList
2010-02-28 23:39:39 +00:00
## Return the list of specification version required for the module
#
# @retval list The list of specification defined in module file
#
2018-08-03 23:11:06 +08:00
@cached_property
def Specification ( self ) :
2010-02-28 23:39:39 +00:00
return self . Module . Specification
## Tool option for the module build
#
# @param PlatformInfo The object of PlatformBuildInfo
# @retval dict The dict containing valid options
#
2018-08-03 23:11:06 +08:00
@cached_property
def BuildOption ( self ) :
RetVal , self . BuildRuleOrder = self . PlatformInfo . ApplyBuildOption ( self . Module )
if self . BuildRuleOrder :
self . BuildRuleOrder = [ ' . %s ' % Ext for Ext in self . BuildRuleOrder . split ( ) ]
return RetVal
2010-02-28 23:39:39 +00:00
2011-10-11 02:49:48 +00:00
## Get include path list from tool option for the module build
#
# @retval list The include path list
#
2018-08-03 23:11:06 +08:00
@cached_property
def BuildOptionIncPathList ( self ) :
#
# Regular expression for finding Include Directories, the difference between MSFT and INTEL/GCC/RVCT
# is the former use /I , the Latter used -I to specify include directories
#
if self . PlatformInfo . ToolChainFamily in ( ' MSFT ' ) :
BuildOptIncludeRegEx = gBuildOptIncludePatternMsft
elif self . PlatformInfo . ToolChainFamily in ( ' INTEL ' , ' GCC ' , ' RVCT ' ) :
BuildOptIncludeRegEx = gBuildOptIncludePatternOther
else :
2011-10-11 02:49:48 +00:00
#
2018-08-03 23:11:06 +08:00
# New ToolChainFamily, don't known whether there is option to specify include directories
2011-10-11 02:49:48 +00:00
#
2018-08-03 23:11:06 +08:00
return [ ]
2018-07-05 17:40:04 +08:00
2018-08-03 23:11:06 +08:00
RetVal = [ ]
for Tool in ( ' CC ' , ' PP ' , ' VFRPP ' , ' ASLPP ' , ' ASLCC ' , ' APP ' , ' ASM ' ) :
try :
FlagOption = self . BuildOption [ Tool ] [ ' FLAGS ' ]
except KeyError :
FlagOption = ' '
2011-11-09 04:32:08 +00:00
2018-08-03 23:11:06 +08:00
if self . PlatformInfo . ToolChainFamily != ' RVCT ' :
IncPathList = [ NormPath ( Path , self . Macros ) for Path in BuildOptIncludeRegEx . findall ( FlagOption ) ]
else :
2011-10-11 02:49:48 +00:00
#
2018-08-03 23:11:06 +08:00
# RVCT may specify a list of directory seperated by commas
2011-10-11 02:49:48 +00:00
#
2018-08-03 23:11:06 +08:00
IncPathList = [ ]
for Path in BuildOptIncludeRegEx . findall ( FlagOption ) :
PathList = GetSplitList ( Path , TAB_COMMA_SPLIT )
IncPathList . extend ( NormPath ( PathEntry , self . Macros ) for PathEntry in PathList )
2018-07-05 17:40:04 +08:00
2018-08-03 23:11:06 +08:00
#
# EDK II modules must not reference header files outside of the packages they depend on or
# within the module's directory tree. Report error if violation.
#
if self . AutoGenVersion > = 0x00010005 :
for Path in IncPathList :
if ( Path not in self . IncludePathList ) and ( CommonPath ( [ Path , self . MetaFile . Dir ] ) != self . MetaFile . Dir ) :
ErrMsg = " The include directory for the EDK II module in this line is invalid %s specified in %s FLAGS ' %s ' " % ( Path , Tool , FlagOption )
EdkLogger . error ( " build " ,
PARAMETER_INVALID ,
ExtraData = ErrMsg ,
File = str ( self . MetaFile ) )
RetVal + = IncPathList
return RetVal
2018-07-05 17:40:04 +08:00
2010-02-28 23:39:39 +00:00
## Return a list of files which can be built from source
#
# What kind of files can be built is determined by build rules in
2014-08-28 13:53:34 +00:00
# $(CONF_DIRECTORY)/build_rule.txt and toolchain family.
2010-02-28 23:39:39 +00:00
#
2018-08-03 23:11:06 +08:00
@cached_property
def SourceFileList ( self ) :
RetVal = [ ]
ToolChainTagSet = { " " , " * " , self . ToolChain }
ToolChainFamilySet = { " " , " * " , self . ToolChainFamily , self . BuildRuleFamily }
for F in self . Module . Sources :
# match tool chain
if F . TagName not in ToolChainTagSet :
EdkLogger . debug ( EdkLogger . DEBUG_9 , " The toolchain [ %s ] for processing file [ %s ] is found, "
" but [ %s ] is currently used " % ( F . TagName , str ( F ) , self . ToolChain ) )
continue
# match tool chain family or build rule family
if F . ToolChainFamily not in ToolChainFamilySet :
EdkLogger . debug (
EdkLogger . DEBUG_0 ,
" The file [ %s ] must be built by tools of [ %s ], " \
" but current toolchain family is [ %s ], buildrule family is [ %s ] " \
% ( str ( F ) , F . ToolChainFamily , self . ToolChainFamily , self . BuildRuleFamily ) )
continue
2010-02-28 23:39:39 +00:00
2018-08-03 23:11:06 +08:00
# add the file path into search path list for file including
if F . Dir not in self . IncludePathList and self . AutoGenVersion > = 0x00010005 :
self . IncludePathList . insert ( 0 , F . Dir )
RetVal . append ( F )
2016-01-19 12:58:52 +00:00
2018-08-03 23:11:06 +08:00
self . _MatchBuildRuleOrder ( RetVal )
2016-01-19 12:58:52 +00:00
2018-08-03 23:11:06 +08:00
for F in RetVal :
self . _ApplyBuildRule ( F , TAB_UNKNOWN_FILE )
return RetVal
2010-02-28 23:39:39 +00:00
2016-01-19 12:58:52 +00:00
def _MatchBuildRuleOrder ( self , FileList ) :
Order_Dict = { }
2018-08-03 23:11:06 +08:00
self . BuildOption
2016-01-19 12:58:52 +00:00
for SingleFile in FileList :
if self . BuildRuleOrder and SingleFile . Ext in self . BuildRuleOrder and SingleFile . Ext in self . BuildRules :
key = SingleFile . Path . split ( SingleFile . Ext ) [ 0 ]
if key in Order_Dict :
Order_Dict [ key ] . append ( SingleFile . Ext )
else :
Order_Dict [ key ] = [ SingleFile . Ext ]
RemoveList = [ ]
for F in Order_Dict :
if len ( Order_Dict [ F ] ) > 1 :
Order_Dict [ F ] . sort ( key = lambda i : self . BuildRuleOrder . index ( i ) )
for Ext in Order_Dict [ F ] [ 1 : ] :
RemoveList . append ( F + Ext )
2018-07-05 17:40:04 +08:00
2016-01-19 12:58:52 +00:00
for item in RemoveList :
FileList . remove ( item )
return FileList
2010-02-28 23:39:39 +00:00
## Return the list of unicode files
2018-08-03 23:11:06 +08:00
@cached_property
def UnicodeFileList ( self ) :
return self . FileTypes . get ( TAB_UNICODE_FILE , [ ] )
2010-02-28 23:39:39 +00:00
2017-02-13 13:10:43 +08:00
## Return the list of vfr files
2018-08-03 23:11:06 +08:00
@cached_property
def VfrFileList ( self ) :
return self . FileTypes . get ( TAB_VFR_FILE , [ ] )
2017-02-13 13:10:43 +08:00
2016-09-21 10:39:11 +08:00
## Return the list of Image Definition files
2018-08-03 23:11:06 +08:00
@cached_property
def IdfFileList ( self ) :
return self . FileTypes . get ( TAB_IMAGE_FILE , [ ] )
2016-09-21 10:39:11 +08:00
2010-02-28 23:39:39 +00:00
## Return a list of files which can be built from binary
#
# "Build" binary files are just to copy them to build directory.
#
# @retval list The list of files which can be built later
#
2018-08-03 23:11:06 +08:00
@cached_property
def BinaryFileList ( self ) :
RetVal = [ ]
for F in self . Module . Binaries :
if F . Target not in [ TAB_ARCH_COMMON , ' * ' ] and F . Target != self . BuildTarget :
continue
RetVal . append ( F )
self . _ApplyBuildRule ( F , F . Type , BinaryFileList = RetVal )
return RetVal
@cached_property
def BuildRules ( self ) :
RetVal = { }
BuildRuleDatabase = self . PlatformInfo . BuildRule
for Type in BuildRuleDatabase . FileTypeList :
#first try getting build rule by BuildRuleFamily
RuleObject = BuildRuleDatabase [ Type , self . BuildType , self . Arch , self . BuildRuleFamily ]
if not RuleObject :
# build type is always module type, but ...
if self . ModuleType != self . BuildType :
RuleObject = BuildRuleDatabase [ Type , self . ModuleType , self . Arch , self . BuildRuleFamily ]
#second try getting build rule by ToolChainFamily
if not RuleObject :
RuleObject = BuildRuleDatabase [ Type , self . BuildType , self . Arch , self . ToolChainFamily ]
2010-02-28 23:39:39 +00:00
if not RuleObject :
# build type is always module type, but ...
if self . ModuleType != self . BuildType :
2018-08-03 23:11:06 +08:00
RuleObject = BuildRuleDatabase [ Type , self . ModuleType , self . Arch , self . ToolChainFamily ]
if not RuleObject :
continue
RuleObject = RuleObject . Instantiate ( self . Macros )
RetVal [ Type ] = RuleObject
for Ext in RuleObject . SourceFileExtList :
RetVal [ Ext ] = RuleObject
return RetVal
def _ApplyBuildRule ( self , File , FileType , BinaryFileList = None ) :
2018-03-27 04:25:43 +08:00
if self . _BuildTargets is None :
2010-02-28 23:39:39 +00:00
self . _IntroBuildTargetList = set ( )
self . _FinalBuildTargetList = set ( )
2018-04-06 07:13:51 +08:00
self . _BuildTargets = defaultdict ( set )
self . _FileTypes = defaultdict ( set )
2010-02-28 23:39:39 +00:00
2018-08-03 23:11:06 +08:00
if not BinaryFileList :
BinaryFileList = self . BinaryFileList
2014-08-21 01:39:18 +00:00
SubDirectory = os . path . join ( self . OutputDir , File . SubDir )
if not os . path . exists ( SubDirectory ) :
CreateDirectory ( SubDirectory )
2010-02-28 23:39:39 +00:00
LastTarget = None
2018-05-18 08:06:52 +08:00
RuleChain = set ( )
2010-02-28 23:39:39 +00:00
SourceList = [ File ]
Index = 0
2015-05-26 10:32:07 +00:00
#
# Make sure to get build rule order value
#
2018-08-03 23:11:06 +08:00
self . BuildOption
2015-05-26 10:32:07 +00:00
2010-02-28 23:39:39 +00:00
while Index < len ( SourceList ) :
Source = SourceList [ Index ]
Index = Index + 1
if Source != File :
CreateDirectory ( Source . Dir )
2018-08-03 23:11:06 +08:00
if File . IsBinary and File == Source and File in BinaryFileList :
2011-05-11 10:26:49 +00:00
# Skip all files that are not binary libraries
if not self . IsLibrary :
2015-12-01 04:22:16 +00:00
continue
2010-02-28 23:39:39 +00:00
RuleObject = self . BuildRules [ TAB_DEFAULT_BINARY_FILE ]
elif FileType in self . BuildRules :
RuleObject = self . BuildRules [ FileType ]
elif Source . Ext in self . BuildRules :
RuleObject = self . BuildRules [ Source . Ext ]
else :
# stop at no more rules
if LastTarget :
self . _FinalBuildTargetList . add ( LastTarget )
break
FileType = RuleObject . SourceFileType
self . _FileTypes [ FileType ] . add ( Source )
# stop at STATIC_LIBRARY for library
if self . IsLibrary and FileType == TAB_STATIC_LIBRARY :
if LastTarget :
self . _FinalBuildTargetList . add ( LastTarget )
break
2015-05-26 10:32:07 +00:00
Target = RuleObject . Apply ( Source , self . BuildRuleOrder )
2010-02-28 23:39:39 +00:00
if not Target :
if LastTarget :
self . _FinalBuildTargetList . add ( LastTarget )
break
elif not Target . Outputs :
# Only do build for target with outputs
self . _FinalBuildTargetList . add ( Target )
self . _BuildTargets [ FileType ] . add ( Target )
if not Source . IsBinary and Source == File :
self . _IntroBuildTargetList . add ( Target )
# to avoid cyclic rule
if FileType in RuleChain :
break
2018-05-18 08:06:52 +08:00
RuleChain . add ( FileType )
2010-02-28 23:39:39 +00:00
SourceList . extend ( Target . Outputs )
LastTarget = Target
FileType = TAB_UNKNOWN_FILE
2018-08-03 23:11:06 +08:00
@cached_property
def Targets ( self ) :
2018-03-27 04:25:43 +08:00
if self . _BuildTargets is None :
2010-02-28 23:39:39 +00:00
self . _IntroBuildTargetList = set ( )
self . _FinalBuildTargetList = set ( )
2018-04-06 07:13:51 +08:00
self . _BuildTargets = defaultdict ( set )
self . _FileTypes = defaultdict ( set )
2010-02-28 23:39:39 +00:00
2018-08-03 23:11:06 +08:00
#TRICK: call SourceFileList property to apply build rule for source files
self . SourceFileList
2010-02-28 23:39:39 +00:00
#TRICK: call _GetBinaryFileList to apply build rule for binary files
2018-08-03 23:11:06 +08:00
self . BinaryFileList
2010-02-28 23:39:39 +00:00
return self . _BuildTargets
2018-08-03 23:11:06 +08:00
@cached_property
def IntroTargetList ( self ) :
self . Targets
2010-02-28 23:39:39 +00:00
return self . _IntroBuildTargetList
2018-08-03 23:11:06 +08:00
@cached_property
def CodaTargetList ( self ) :
self . Targets
2010-02-28 23:39:39 +00:00
return self . _FinalBuildTargetList
2018-08-03 23:11:06 +08:00
@cached_property
def FileTypes ( self ) :
self . Targets
2010-02-28 23:39:39 +00:00
return self . _FileTypes
## Get the list of package object the module depends on
#
# @retval list The package object list
#
2018-08-03 23:11:06 +08:00
@cached_property
def DependentPackageList ( self ) :
2010-02-28 23:39:39 +00:00
return self . Module . Packages
## Return the list of auto-generated code file
#
# @retval list The list of auto-generated file
#
2018-08-03 23:11:06 +08:00
@cached_property
def AutoGenFileList ( self ) :
AutoGenUniIdf = self . BuildType != ' UEFI_HII '
2018-06-25 18:31:36 +08:00
UniStringBinBuffer = BytesIO ( )
IdfGenBinBuffer = BytesIO ( )
2018-08-03 23:11:06 +08:00
RetVal = { }
AutoGenC = TemplateString ( )
AutoGenH = TemplateString ( )
StringH = TemplateString ( )
StringIdf = TemplateString ( )
GenC . CreateCode ( self , AutoGenC , AutoGenH , StringH , AutoGenUniIdf , UniStringBinBuffer , StringIdf , AutoGenUniIdf , IdfGenBinBuffer )
#
# AutoGen.c is generated if there are library classes in inf, or there are object files
#
if str ( AutoGenC ) != " " and ( len ( self . Module . LibraryClasses ) > 0
or TAB_OBJECT_FILE in self . FileTypes ) :
AutoFile = PathClass ( gAutoGenCodeFileName , self . DebugDir )
RetVal [ AutoFile ] = str ( AutoGenC )
self . _ApplyBuildRule ( AutoFile , TAB_UNKNOWN_FILE )
if str ( AutoGenH ) != " " :
AutoFile = PathClass ( gAutoGenHeaderFileName , self . DebugDir )
RetVal [ AutoFile ] = str ( AutoGenH )
self . _ApplyBuildRule ( AutoFile , TAB_UNKNOWN_FILE )
if str ( StringH ) != " " :
AutoFile = PathClass ( gAutoGenStringFileName % { " module_name " : self . Name } , self . DebugDir )
RetVal [ AutoFile ] = str ( StringH )
self . _ApplyBuildRule ( AutoFile , TAB_UNKNOWN_FILE )
if UniStringBinBuffer is not None and UniStringBinBuffer . getvalue ( ) != " " :
AutoFile = PathClass ( gAutoGenStringFormFileName % { " module_name " : self . Name } , self . OutputDir )
RetVal [ AutoFile ] = UniStringBinBuffer . getvalue ( )
AutoFile . IsBinary = True
self . _ApplyBuildRule ( AutoFile , TAB_UNKNOWN_FILE )
if UniStringBinBuffer is not None :
UniStringBinBuffer . close ( )
if str ( StringIdf ) != " " :
AutoFile = PathClass ( gAutoGenImageDefFileName % { " module_name " : self . Name } , self . DebugDir )
RetVal [ AutoFile ] = str ( StringIdf )
self . _ApplyBuildRule ( AutoFile , TAB_UNKNOWN_FILE )
if IdfGenBinBuffer is not None and IdfGenBinBuffer . getvalue ( ) != " " :
AutoFile = PathClass ( gAutoGenIdfFileName % { " module_name " : self . Name } , self . OutputDir )
RetVal [ AutoFile ] = IdfGenBinBuffer . getvalue ( )
AutoFile . IsBinary = True
self . _ApplyBuildRule ( AutoFile , TAB_UNKNOWN_FILE )
if IdfGenBinBuffer is not None :
IdfGenBinBuffer . close ( )
return RetVal
2010-02-28 23:39:39 +00:00
## Return the list of library modules explicitly or implicityly used by this module
2018-08-03 23:11:06 +08:00
@cached_property
def DependentLibraryList ( self ) :
# only merge library classes and PCD for non-library module
if self . IsLibrary :
return [ ]
if self . AutoGenVersion < 0x00010005 :
return self . PlatformInfo . ResolveLibraryReference ( self . Module )
return self . PlatformInfo . ApplyLibraryInstance ( self . Module )
2010-02-28 23:39:39 +00:00
## Get the list of PCDs from current module
#
# @retval list The list of PCD
#
2018-08-03 23:11:06 +08:00
@cached_property
def ModulePcdList ( self ) :
# apply PCD settings from platform
RetVal = self . PlatformInfo . ApplyPcdSetting ( self . Module , self . Module . Pcds )
ExtendCopyDictionaryLists ( self . _PcdComments , self . Module . PcdComments )
return RetVal
2010-02-28 23:39:39 +00:00
## Get the list of PCDs from dependent libraries
#
# @retval list The list of PCD
#
2018-08-03 23:11:06 +08:00
@cached_property
def LibraryPcdList ( self ) :
if self . IsLibrary :
return [ ]
RetVal = [ ]
Pcds = set ( )
# get PCDs from dependent libraries
for Library in self . DependentLibraryList :
PcdsInLibrary = OrderedDict ( )
ExtendCopyDictionaryLists ( self . _PcdComments , Library . PcdComments )
for Key in Library . Pcds :
# skip duplicated PCDs
if Key in self . Module . Pcds or Key in Pcds :
continue
Pcds . add ( Key )
PcdsInLibrary [ Key ] = copy . copy ( Library . Pcds [ Key ] )
RetVal . extend ( self . PlatformInfo . ApplyPcdSetting ( self . Module , PcdsInLibrary , Library = Library ) )
return RetVal
2010-02-28 23:39:39 +00:00
## Get the GUID value mapping
#
# @retval dict The mapping between GUID cname and its value
#
2018-08-03 23:11:06 +08:00
@cached_property
def GuidList ( self ) :
RetVal = OrderedDict ( self . Module . Guids )
for Library in self . DependentLibraryList :
RetVal . update ( Library . Guids )
ExtendCopyDictionaryLists ( self . _GuidComments , Library . GuidComments )
ExtendCopyDictionaryLists ( self . _GuidComments , self . Module . GuidComments )
return RetVal
@cached_property
2013-11-18 07:41:21 +00:00
def GetGuidsUsedByPcd ( self ) :
2018-08-03 23:11:06 +08:00
RetVal = OrderedDict ( self . Module . GetGuidsUsedByPcd ( ) )
for Library in self . DependentLibraryList :
RetVal . update ( Library . GetGuidsUsedByPcd ( ) )
return RetVal
2010-02-28 23:39:39 +00:00
## Get the protocol value mapping
#
# @retval dict The mapping between protocol cname and its value
#
2018-08-03 23:11:06 +08:00
@cached_property
def ProtocolList ( self ) :
RetVal = OrderedDict ( self . Module . Protocols )
for Library in self . DependentLibraryList :
RetVal . update ( Library . Protocols )
ExtendCopyDictionaryLists ( self . _ProtocolComments , Library . ProtocolComments )
ExtendCopyDictionaryLists ( self . _ProtocolComments , self . Module . ProtocolComments )
return RetVal
2010-02-28 23:39:39 +00:00
## Get the PPI value mapping
#
# @retval dict The mapping between PPI cname and its value
#
2018-08-03 23:11:06 +08:00
@cached_property
def PpiList ( self ) :
RetVal = OrderedDict ( self . Module . Ppis )
for Library in self . DependentLibraryList :
RetVal . update ( Library . Ppis )
ExtendCopyDictionaryLists ( self . _PpiComments , Library . PpiComments )
ExtendCopyDictionaryLists ( self . _PpiComments , self . Module . PpiComments )
return RetVal
2010-02-28 23:39:39 +00:00
## Get the list of include search path
#
# @retval list The list path
#
2018-08-03 23:11:06 +08:00
@cached_property
def IncludePathList ( self ) :
RetVal = [ ]
if self . AutoGenVersion < 0x00010005 :
for Inc in self . Module . Includes :
if Inc not in RetVal :
RetVal . append ( Inc )
# for Edk modules
Inc = path . join ( Inc , self . Arch . capitalize ( ) )
if os . path . exists ( Inc ) and Inc not in RetVal :
RetVal . append ( Inc )
# Edk module needs to put DEBUG_DIR at the end of search path and not to use SOURCE_DIR all the time
RetVal . append ( self . DebugDir )
else :
RetVal . append ( self . MetaFile . Dir )
RetVal . append ( self . DebugDir )
for Package in self . Module . Packages :
PackageDir = mws . join ( self . WorkspaceDir , Package . MetaFile . Dir )
if PackageDir not in RetVal :
RetVal . append ( PackageDir )
IncludesList = Package . Includes
if Package . _PrivateIncludes :
if not self . MetaFile . Path . startswith ( PackageDir ) :
IncludesList = list ( set ( Package . Includes ) . difference ( set ( Package . _PrivateIncludes ) ) )
for Inc in IncludesList :
if Inc not in RetVal :
RetVal . append ( str ( Inc ) )
return RetVal
@cached_property
def IncludePathLength ( self ) :
return sum ( len ( inc ) + 1 for inc in self . IncludePathList )
2016-03-16 11:06:44 +08:00
2014-08-28 13:53:34 +00:00
## Get HII EX PCDs which maybe used by VFR
#
# efivarstore used by VFR may relate with HII EX PCDs
# Get the variable name and GUID from efivarstore and HII EX PCD
# List the HII EX PCDs in As Built INF if both name and GUID match.
#
# @retval list HII EX PCDs
#
def _GetPcdsMaybeUsedByVfr ( self ) :
if not self . SourceFileList :
return [ ]
2018-04-06 07:13:58 +08:00
NameGuids = set ( )
2014-08-28 13:53:34 +00:00
for SrcFile in self . SourceFileList :
if SrcFile . Ext . lower ( ) != ' .vfr ' :
continue
Vfri = os . path . join ( self . OutputDir , SrcFile . BaseName + ' .i ' )
if not os . path . exists ( Vfri ) :
continue
VfriFile = open ( Vfri , ' r ' )
Content = VfriFile . read ( )
VfriFile . close ( )
Pos = Content . find ( ' efivarstore ' )
while Pos != - 1 :
#
# Make sure 'efivarstore' is the start of efivarstore statement
# In case of the value of 'name' (name = efivarstore) is equal to 'efivarstore'
#
Index = Pos - 1
while Index > = 0 and Content [ Index ] in ' \t \r \n ' :
Index - = 1
if Index > = 0 and Content [ Index ] != ' ; ' :
Pos = Content . find ( ' efivarstore ' , Pos + len ( ' efivarstore ' ) )
continue
#
# 'efivarstore' must be followed by name and guid
#
Name = gEfiVarStoreNamePattern . search ( Content , Pos )
if not Name :
break
Guid = gEfiVarStoreGuidPattern . search ( Content , Pos )
if not Guid :
break
NameArray = ConvertStringToByteArray ( ' L " ' + Name . group ( 1 ) + ' " ' )
2018-04-06 07:13:58 +08:00
NameGuids . add ( ( NameArray , GuidStructureStringToGuidString ( Guid . group ( 1 ) ) ) )
2014-08-28 13:53:34 +00:00
Pos = Content . find ( ' efivarstore ' , Name . end ( ) )
if not NameGuids :
return [ ]
HiiExPcds = [ ]
for Pcd in self . PlatformInfo . Platform . Pcds . values ( ) :
if Pcd . Type != TAB_PCDS_DYNAMIC_EX_HII :
continue
2018-05-18 08:06:52 +08:00
for SkuInfo in Pcd . SkuInfoList . values ( ) :
2016-05-10 17:58:26 +08:00
Value = GuidValue ( SkuInfo . VariableGuid , self . PlatformInfo . PackageList , self . MetaFile . Path )
2014-08-28 13:53:34 +00:00
if not Value :
continue
2018-05-18 08:06:52 +08:00
Name = ConvertStringToByteArray ( SkuInfo . VariableName )
2014-08-28 13:53:34 +00:00
Guid = GuidStructureStringToGuidString ( Value )
if ( Name , Guid ) in NameGuids and Pcd not in HiiExPcds :
HiiExPcds . append ( Pcd )
break
return HiiExPcds
2015-06-11 05:20:00 +00:00
def _GenOffsetBin ( self ) :
VfrUniBaseName = { }
for SourceFile in self . Module . Sources :
if SourceFile . Type . upper ( ) == " .VFR " :
#
2018-07-05 17:40:04 +08:00
# search the .map file to find the offset of vfr binary in the PE32+/TE file.
2015-06-11 05:20:00 +00:00
#
VfrUniBaseName [ SourceFile . BaseName ] = ( SourceFile . BaseName + " Bin " )
2018-05-18 08:06:52 +08:00
elif SourceFile . Type . upper ( ) == " .UNI " :
2015-06-11 05:20:00 +00:00
#
2018-07-05 17:40:04 +08:00
# search the .map file to find the offset of Uni strings binary in the PE32+/TE file.
2015-06-11 05:20:00 +00:00
#
VfrUniBaseName [ " UniOffsetName " ] = ( self . Name + " Strings " )
2018-05-18 08:06:52 +08:00
if not VfrUniBaseName :
2015-06-11 05:20:00 +00:00
return None
MapFileName = os . path . join ( self . OutputDir , self . Name + " .map " )
EfiFileName = os . path . join ( self . OutputDir , self . Name + " .efi " )
VfrUniOffsetList = GetVariableOffset ( MapFileName , EfiFileName , VfrUniBaseName . values ( ) )
if not VfrUniOffsetList :
return None
OutputName = ' %s Offset.bin ' % self . Name
UniVfrOffsetFileName = os . path . join ( self . OutputDir , OutputName )
try :
fInputfile = open ( UniVfrOffsetFileName , " wb+ " , 0 )
except :
2018-06-25 18:31:33 +08:00
EdkLogger . error ( " build " , FILE_OPEN_FAILURE , " File open failed for %s " % UniVfrOffsetFileName , None )
2015-06-11 05:20:00 +00:00
2018-06-25 18:31:36 +08:00
# Use a instance of BytesIO to cache data
fStringIO = BytesIO ( ' ' )
2015-06-11 05:20:00 +00:00
for Item in VfrUniOffsetList :
if ( Item [ 0 ] . find ( " Strings " ) != - 1 ) :
#
# UNI offset in image.
# GUID + Offset
# { 0x8913c5e0, 0x33f6, 0x4d86, { 0x9b, 0xf1, 0x43, 0xef, 0x89, 0xfc, 0x6, 0x66 } }
#
UniGuid = [ 0xe0 , 0xc5 , 0x13 , 0x89 , 0xf6 , 0x33 , 0x86 , 0x4d , 0x9b , 0xf1 , 0x43 , 0xef , 0x89 , 0xfc , 0x6 , 0x66 ]
UniGuid = [ chr ( ItemGuid ) for ItemGuid in UniGuid ]
2018-07-05 17:40:04 +08:00
fStringIO . write ( ' ' . join ( UniGuid ) )
2015-06-11 05:20:00 +00:00
UniValue = pack ( ' Q ' , int ( Item [ 1 ] , 16 ) )
fStringIO . write ( UniValue )
else :
#
# VFR binary offset in image.
# GUID + Offset
# { 0xd0bc7cb4, 0x6a47, 0x495f, { 0xaa, 0x11, 0x71, 0x7, 0x46, 0xda, 0x6, 0xa2 } };
#
VfrGuid = [ 0xb4 , 0x7c , 0xbc , 0xd0 , 0x47 , 0x6a , 0x5f , 0x49 , 0xaa , 0x11 , 0x71 , 0x7 , 0x46 , 0xda , 0x6 , 0xa2 ]
VfrGuid = [ chr ( ItemGuid ) for ItemGuid in VfrGuid ]
2018-07-05 17:40:04 +08:00
fStringIO . write ( ' ' . join ( VfrGuid ) )
2015-06-11 05:20:00 +00:00
VfrValue = pack ( ' Q ' , int ( Item [ 1 ] , 16 ) )
fStringIO . write ( VfrValue )
#
# write data into file.
#
2018-07-05 17:40:04 +08:00
try :
2015-06-11 05:20:00 +00:00
fInputfile . write ( fStringIO . getvalue ( ) )
except :
EdkLogger . error ( " build " , FILE_WRITE_FAILURE , " Write data to file %s failed, please check whether the "
2018-06-25 18:31:33 +08:00
" file been locked or using by other applications. " % UniVfrOffsetFileName , None )
2015-06-11 05:20:00 +00:00
fStringIO . close ( )
fInputfile . close ( )
return OutputName
2011-05-11 10:26:49 +00:00
## Create AsBuilt INF file the module
#
2017-12-12 15:18:22 +08:00
def CreateAsBuiltInf ( self , IsOnlyCopy = False ) :
2018-04-06 07:13:55 +08:00
self . OutputFile = set ( )
2018-05-18 08:06:52 +08:00
if IsOnlyCopy and GlobalData . gBinCacheDest :
self . CopyModuleToCache ( )
return
2017-12-12 15:18:22 +08:00
2011-05-11 10:26:49 +00:00
if self . IsAsBuiltInfCreated :
return
2018-07-05 17:40:04 +08:00
2011-05-11 10:26:49 +00:00
# Skip the following code for EDK I inf
if self . AutoGenVersion < 0x00010005 :
return
2018-07-05 17:40:04 +08:00
2011-05-11 10:26:49 +00:00
# Skip the following code for libraries
if self . IsLibrary :
return
2018-07-05 17:40:04 +08:00
2011-05-11 10:26:49 +00:00
# Skip the following code for modules with no source files
2018-04-06 07:13:56 +08:00
if not self . SourceFileList :
2011-05-11 10:26:49 +00:00
return
# Skip the following code for modules without any binary files
2018-04-25 08:46:36 +08:00
if self . BinaryFileList :
2011-05-11 10:26:49 +00:00
return
2018-07-05 17:40:04 +08:00
2011-05-11 10:26:49 +00:00
### TODO: How to handles mixed source and binary modules
2013-11-18 07:41:21 +00:00
# Find all DynamicEx and PatchableInModule PCDs used by this module and dependent libraries
2011-05-11 10:26:49 +00:00
# Also find all packages that the DynamicEx PCDs depend on
Pcds = [ ]
2016-07-26 15:17:15 +08:00
PatchablePcds = [ ]
2015-12-01 04:22:16 +00:00
Packages = [ ]
2013-11-18 07:41:21 +00:00
PcdCheckList = [ ]
PcdTokenSpaceList = [ ]
2011-05-11 10:26:49 +00:00
for Pcd in self . ModulePcdList + self . LibraryPcdList :
2013-11-18 07:41:21 +00:00
if Pcd . Type == TAB_PCDS_PATCHABLE_IN_MODULE :
2018-05-18 08:06:52 +08:00
PatchablePcds . append ( Pcd )
2018-04-20 23:51:25 +08:00
PcdCheckList . append ( ( Pcd . TokenCName , Pcd . TokenSpaceGuidCName , TAB_PCDS_PATCHABLE_IN_MODULE ) )
2018-04-20 23:51:37 +08:00
elif Pcd . Type in PCD_DYNAMIC_EX_TYPE_SET :
2013-11-18 07:41:21 +00:00
if Pcd not in Pcds :
2018-05-18 08:06:52 +08:00
Pcds . append ( Pcd )
2018-04-20 23:51:25 +08:00
PcdCheckList . append ( ( Pcd . TokenCName , Pcd . TokenSpaceGuidCName , TAB_PCDS_DYNAMIC_EX ) )
PcdCheckList . append ( ( Pcd . TokenCName , Pcd . TokenSpaceGuidCName , TAB_PCDS_DYNAMIC ) )
2013-11-18 07:41:21 +00:00
PcdTokenSpaceList . append ( Pcd . TokenSpaceGuidCName )
2018-08-03 23:11:06 +08:00
GuidList = OrderedDict ( self . GuidList )
for TokenSpace in self . GetGuidsUsedByPcd :
2013-11-18 07:41:21 +00:00
# If token space is not referred by patch PCD or Ex PCD, remove the GUID from GUID list
# The GUIDs in GUIDs section should really be the GUIDs in source INF or referred by Ex an patch PCDs
if TokenSpace not in PcdTokenSpaceList and TokenSpace in GuidList :
GuidList . pop ( TokenSpace )
CheckList = ( GuidList , self . PpiList , self . ProtocolList , PcdCheckList )
for Package in self . DerivedPackageList :
if Package in Packages :
continue
BeChecked = ( Package . Guids , Package . Ppis , Package . Protocols , Package . Pcds )
Found = False
for Index in range ( len ( BeChecked ) ) :
for Item in CheckList [ Index ] :
if Item in BeChecked [ Index ] :
2018-05-18 08:06:52 +08:00
Packages . append ( Package )
2013-11-18 07:41:21 +00:00
Found = True
break
2018-05-18 08:06:52 +08:00
if Found :
break
2011-05-11 10:26:49 +00:00
2014-08-28 13:53:34 +00:00
VfrPcds = self . _GetPcdsMaybeUsedByVfr ( )
for Pkg in self . PlatformInfo . PackageList :
if Pkg in Packages :
continue
for VfrPcd in VfrPcds :
2018-04-20 23:51:25 +08:00
if ( ( VfrPcd . TokenCName , VfrPcd . TokenSpaceGuidCName , TAB_PCDS_DYNAMIC_EX ) in Pkg . Pcds or
( VfrPcd . TokenCName , VfrPcd . TokenSpaceGuidCName , TAB_PCDS_DYNAMIC ) in Pkg . Pcds ) :
2018-05-18 08:06:52 +08:00
Packages . append ( Pkg )
2014-08-28 13:53:34 +00:00
break
2018-05-18 08:06:52 +08:00
ModuleType = SUP_MODULE_DXE_DRIVER if self . ModuleType == SUP_MODULE_UEFI_DRIVER and self . DepexGenerated else self . ModuleType
DriverType = self . PcdIsDriver if self . PcdIsDriver else ' '
2014-08-28 13:53:34 +00:00
Guid = self . Guid
MDefs = self . Module . Defines
2011-05-11 10:26:49 +00:00
AsBuiltInfDict = {
' module_name ' : self . Name ,
2014-08-28 13:53:34 +00:00
' module_guid ' : Guid ,
2011-05-11 10:26:49 +00:00
' module_module_type ' : ModuleType ,
2014-08-28 13:53:34 +00:00
' module_version_string ' : [ MDefs [ ' VERSION_STRING ' ] ] if ' VERSION_STRING ' in MDefs else [ ] ,
2013-11-18 07:41:21 +00:00
' pcd_is_driver_string ' : [ ] ,
2011-05-11 10:26:49 +00:00
' module_uefi_specification_version ' : [ ] ,
' module_pi_specification_version ' : [ ] ,
2014-08-28 13:53:34 +00:00
' module_entry_point ' : self . Module . ModuleEntryPointList ,
' module_unload_image ' : self . Module . ModuleUnloadImageList ,
' module_constructor ' : self . Module . ConstructorList ,
' module_destructor ' : self . Module . DestructorList ,
' module_shadow ' : [ MDefs [ ' SHADOW ' ] ] if ' SHADOW ' in MDefs else [ ] ,
' module_pci_vendor_id ' : [ MDefs [ ' PCI_VENDOR_ID ' ] ] if ' PCI_VENDOR_ID ' in MDefs else [ ] ,
' module_pci_device_id ' : [ MDefs [ ' PCI_DEVICE_ID ' ] ] if ' PCI_DEVICE_ID ' in MDefs else [ ] ,
' module_pci_class_code ' : [ MDefs [ ' PCI_CLASS_CODE ' ] ] if ' PCI_CLASS_CODE ' in MDefs else [ ] ,
' module_pci_revision ' : [ MDefs [ ' PCI_REVISION ' ] ] if ' PCI_REVISION ' in MDefs else [ ] ,
' module_build_number ' : [ MDefs [ ' BUILD_NUMBER ' ] ] if ' BUILD_NUMBER ' in MDefs else [ ] ,
' module_spec ' : [ MDefs [ ' SPEC ' ] ] if ' SPEC ' in MDefs else [ ] ,
' module_uefi_hii_resource_section ' : [ MDefs [ ' UEFI_HII_RESOURCE_SECTION ' ] ] if ' UEFI_HII_RESOURCE_SECTION ' in MDefs else [ ] ,
' module_uni_file ' : [ MDefs [ ' MODULE_UNI_FILE ' ] ] if ' MODULE_UNI_FILE ' in MDefs else [ ] ,
2011-05-11 10:26:49 +00:00
' module_arch ' : self . Arch ,
2018-05-18 08:06:52 +08:00
' package_item ' : [ Package . MetaFile . File . replace ( ' \\ ' , ' / ' ) for Package in Packages ] ,
2011-05-11 10:26:49 +00:00
' binary_item ' : [ ] ,
2013-11-18 07:41:21 +00:00
' patchablepcd_item ' : [ ] ,
2011-05-11 10:26:49 +00:00
' pcd_item ' : [ ] ,
2013-11-18 07:41:21 +00:00
' protocol_item ' : [ ] ,
' ppi_item ' : [ ] ,
' guid_item ' : [ ] ,
' flags_item ' : [ ] ,
' libraryclasses_item ' : [ ]
2011-05-11 10:26:49 +00:00
}
2014-08-28 13:53:34 +00:00
2017-04-13 14:37:40 +08:00
if ' MODULE_UNI_FILE ' in MDefs :
UNIFile = os . path . join ( self . MetaFile . Dir , MDefs [ ' MODULE_UNI_FILE ' ] )
if os . path . isfile ( UNIFile ) :
shutil . copy2 ( UNIFile , self . OutputDir )
2014-08-28 13:53:34 +00:00
if self . AutoGenVersion > int ( gInfSpecVersion , 0 ) :
AsBuiltInfDict [ ' module_inf_version ' ] = ' 0x %08x ' % self . AutoGenVersion
else :
AsBuiltInfDict [ ' module_inf_version ' ] = gInfSpecVersion
2013-11-18 07:41:21 +00:00
if DriverType :
2018-05-18 08:06:52 +08:00
AsBuiltInfDict [ ' pcd_is_driver_string ' ] . append ( DriverType )
2011-05-11 10:26:49 +00:00
if ' UEFI_SPECIFICATION_VERSION ' in self . Specification :
2018-05-18 08:06:52 +08:00
AsBuiltInfDict [ ' module_uefi_specification_version ' ] . append ( self . Specification [ ' UEFI_SPECIFICATION_VERSION ' ] )
2011-05-11 10:26:49 +00:00
if ' PI_SPECIFICATION_VERSION ' in self . Specification :
2018-05-18 08:06:52 +08:00
AsBuiltInfDict [ ' module_pi_specification_version ' ] . append ( self . Specification [ ' PI_SPECIFICATION_VERSION ' ] )
2011-05-11 10:26:49 +00:00
2015-12-01 04:22:16 +00:00
OutputDir = self . OutputDir . replace ( ' \\ ' , ' / ' ) . strip ( ' / ' )
2018-01-03 12:36:51 +08:00
DebugDir = self . DebugDir . replace ( ' \\ ' , ' / ' ) . strip ( ' / ' )
2016-04-22 15:08:47 +08:00
for Item in self . CodaTargetList :
2018-03-16 15:03:16 +08:00
File = Item . Target . Path . replace ( ' \\ ' , ' / ' ) . strip ( ' / ' ) . replace ( DebugDir , ' ' ) . replace ( OutputDir , ' ' ) . strip ( ' / ' )
2018-04-06 07:13:55 +08:00
self . OutputFile . add ( File )
2018-01-22 17:55:17 +08:00
if os . path . isabs ( File ) :
File = File . replace ( ' \\ ' , ' / ' ) . strip ( ' / ' ) . replace ( OutputDir , ' ' ) . strip ( ' / ' )
2015-12-01 04:22:16 +00:00
if Item . Target . Ext . lower ( ) == ' .aml ' :
2018-05-18 08:06:52 +08:00
AsBuiltInfDict [ ' binary_item ' ] . append ( ' ASL| ' + File )
2015-12-01 04:22:16 +00:00
elif Item . Target . Ext . lower ( ) == ' .acpi ' :
2018-05-18 08:06:52 +08:00
AsBuiltInfDict [ ' binary_item ' ] . append ( ' ACPI| ' + File )
2016-04-22 15:08:47 +08:00
elif Item . Target . Ext . lower ( ) == ' .efi ' :
2018-05-18 08:06:52 +08:00
AsBuiltInfDict [ ' binary_item ' ] . append ( ' PE32| ' + self . Name + ' .efi ' )
2011-05-11 10:26:49 +00:00
else :
2018-05-18 08:06:52 +08:00
AsBuiltInfDict [ ' binary_item ' ] . append ( ' BIN| ' + File )
2016-04-22 15:08:47 +08:00
if self . DepexGenerated :
2018-04-06 07:13:55 +08:00
self . OutputFile . add ( self . Name + ' .depex ' )
2018-04-27 00:57:53 +08:00
if self . ModuleType in [ SUP_MODULE_PEIM ] :
2018-05-18 08:06:52 +08:00
AsBuiltInfDict [ ' binary_item ' ] . append ( ' PEI_DEPEX| ' + self . Name + ' .depex ' )
elif self . ModuleType in [ SUP_MODULE_DXE_DRIVER , SUP_MODULE_DXE_RUNTIME_DRIVER , SUP_MODULE_DXE_SAL_DRIVER , SUP_MODULE_UEFI_DRIVER ] :
AsBuiltInfDict [ ' binary_item ' ] . append ( ' DXE_DEPEX| ' + self . Name + ' .depex ' )
elif self . ModuleType in [ SUP_MODULE_DXE_SMM_DRIVER ] :
AsBuiltInfDict [ ' binary_item ' ] . append ( ' SMM_DEPEX| ' + self . Name + ' .depex ' )
2011-05-11 10:26:49 +00:00
2015-06-11 05:20:00 +00:00
Bin = self . _GenOffsetBin ( )
if Bin :
2018-05-18 08:06:52 +08:00
AsBuiltInfDict [ ' binary_item ' ] . append ( ' BIN| %s ' % Bin )
2018-04-06 07:13:55 +08:00
self . OutputFile . add ( Bin )
2015-06-11 05:20:00 +00:00
2013-11-18 07:41:21 +00:00
for Root , Dirs , Files in os . walk ( OutputDir ) :
for File in Files :
if File . lower ( ) . endswith ( ' .pdb ' ) :
2018-05-18 08:06:52 +08:00
AsBuiltInfDict [ ' binary_item ' ] . append ( ' DISPOSABLE| ' + File )
2018-04-06 07:13:55 +08:00
self . OutputFile . add ( File )
2013-11-18 07:41:21 +00:00
HeaderComments = self . Module . HeaderComments
StartPos = 0
for Index in range ( len ( HeaderComments ) ) :
if HeaderComments [ Index ] . find ( ' @BinaryHeader ' ) != - 1 :
HeaderComments [ Index ] = HeaderComments [ Index ] . replace ( ' @BinaryHeader ' , ' @file ' )
StartPos = Index
break
AsBuiltInfDict [ ' header_comments ' ] = ' \n ' . join ( HeaderComments [ StartPos : ] ) . replace ( ' :# ' , ' :// ' )
2014-08-28 13:53:34 +00:00
AsBuiltInfDict [ ' tail_comments ' ] = ' \n ' . join ( self . Module . TailComments )
2013-11-18 07:41:21 +00:00
GenList = [
( self . ProtocolList , self . _ProtocolComments , ' protocol_item ' ) ,
( self . PpiList , self . _PpiComments , ' ppi_item ' ) ,
( GuidList , self . _GuidComments , ' guid_item ' )
]
for Item in GenList :
for CName in Item [ 0 ] :
2018-05-18 08:06:52 +08:00
Comments = ' \n ' . join ( Item [ 1 ] [ CName ] ) if CName in Item [ 1 ] else ' '
Entry = Comments + ' \n ' + CName if Comments else CName
2013-11-18 07:41:21 +00:00
AsBuiltInfDict [ Item [ 2 ] ] . append ( Entry )
PatchList = parsePcdInfoFromMapFile (
os . path . join ( self . OutputDir , self . Name + ' .map ' ) ,
os . path . join ( self . OutputDir , self . Name + ' .efi ' )
)
if PatchList :
2016-07-26 15:17:15 +08:00
for Pcd in PatchablePcds :
TokenCName = Pcd . TokenCName
for PcdItem in GlobalData . MixedPcd :
if ( Pcd . TokenCName , Pcd . TokenSpaceGuidCName ) in GlobalData . MixedPcd [ PcdItem ] :
TokenCName = PcdItem [ 0 ]
break
for PatchPcd in PatchList :
if TokenCName == PatchPcd [ 0 ] :
break
2016-04-12 10:31:55 +08:00
else :
2013-11-18 07:41:21 +00:00
continue
PcdValue = ' '
2017-04-26 18:19:20 +08:00
if Pcd . DatumType == ' BOOLEAN ' :
BoolValue = Pcd . DefaultValue . upper ( )
if BoolValue == ' TRUE ' :
Pcd . DefaultValue = ' 1 '
elif BoolValue == ' FALSE ' :
Pcd . DefaultValue = ' 0 '
2018-04-11 09:14:05 -07:00
if Pcd . DatumType in TAB_PCD_NUMERIC_TYPES :
2013-11-18 07:41:21 +00:00
HexFormat = ' 0x %02x '
2018-04-11 09:14:05 -07:00
if Pcd . DatumType == TAB_UINT16 :
2013-11-18 07:41:21 +00:00
HexFormat = ' 0x %04x '
2018-04-11 09:14:05 -07:00
elif Pcd . DatumType == TAB_UINT32 :
2013-11-18 07:41:21 +00:00
HexFormat = ' 0x %08x '
2018-04-11 09:14:05 -07:00
elif Pcd . DatumType == TAB_UINT64 :
2013-11-18 07:41:21 +00:00
HexFormat = ' 0x %016x '
PcdValue = HexFormat % int ( Pcd . DefaultValue , 0 )
else :
2018-03-27 04:25:43 +08:00
if Pcd . MaxDatumSize is None or Pcd . MaxDatumSize == ' ' :
2013-11-18 07:41:21 +00:00
EdkLogger . error ( " build " , AUTOGEN_ERROR ,
2016-04-12 10:31:55 +08:00
" Unknown [MaxDatumSize] of PCD [ %s . %s ] " % ( Pcd . TokenSpaceGuidCName , TokenCName )
2013-11-18 07:41:21 +00:00
)
ArraySize = int ( Pcd . MaxDatumSize , 0 )
PcdValue = Pcd . DefaultValue
if PcdValue [ 0 ] != ' { ' :
Unicode = False
if PcdValue [ 0 ] == ' L ' :
Unicode = True
PcdValue = PcdValue . lstrip ( ' L ' )
PcdValue = eval ( PcdValue )
NewValue = ' { '
for Index in range ( 0 , len ( PcdValue ) ) :
if Unicode :
CharVal = ord ( PcdValue [ Index ] )
NewValue = NewValue + ' 0x %02x ' % ( CharVal & 0x00FF ) + ' , ' \
+ ' 0x %02x ' % ( CharVal >> 8 ) + ' , '
else :
NewValue = NewValue + ' 0x %02x ' % ( ord ( PcdValue [ Index ] ) % 0x100 ) + ' , '
Padding = ' 0x00, '
if Unicode :
Padding = Padding * 2
ArraySize = ArraySize / 2
if ArraySize < ( len ( PcdValue ) + 1 ) :
2018-03-17 15:25:32 +08:00
if Pcd . MaxSizeUserSet :
EdkLogger . error ( " build " , AUTOGEN_ERROR ,
2016-04-12 10:31:55 +08:00
" The maximum size of VOID* type PCD ' %s . %s ' is less than its actual size occupied. " % ( Pcd . TokenSpaceGuidCName , TokenCName )
2013-11-18 07:41:21 +00:00
)
2018-03-17 15:25:32 +08:00
else :
ArraySize = len ( PcdValue ) + 1
2013-11-18 07:41:21 +00:00
if ArraySize > len ( PcdValue ) + 1 :
NewValue = NewValue + Padding * ( ArraySize - len ( PcdValue ) - 1 )
PcdValue = NewValue + Padding . strip ( ) . rstrip ( ' , ' ) + ' } '
elif len ( PcdValue . split ( ' , ' ) ) < = ArraySize :
PcdValue = PcdValue . rstrip ( ' } ' ) + ' , 0x00 ' * ( ArraySize - len ( PcdValue . split ( ' , ' ) ) )
PcdValue + = ' } '
else :
2018-03-17 15:25:32 +08:00
if Pcd . MaxSizeUserSet :
EdkLogger . error ( " build " , AUTOGEN_ERROR ,
2016-04-12 10:31:55 +08:00
" The maximum size of VOID* type PCD ' %s . %s ' is less than its actual size occupied. " % ( Pcd . TokenSpaceGuidCName , TokenCName )
2013-11-18 07:41:21 +00:00
)
2018-03-17 15:25:32 +08:00
else :
ArraySize = len ( PcdValue ) + 1
2013-11-18 07:41:21 +00:00
PcdItem = ' %s . %s | %s |0x %X ' % \
2016-04-12 10:31:55 +08:00
( Pcd . TokenSpaceGuidCName , TokenCName , PcdValue , PatchPcd [ 1 ] )
2013-11-18 07:41:21 +00:00
PcdComments = ' '
if ( Pcd . TokenSpaceGuidCName , Pcd . TokenCName ) in self . _PcdComments :
PcdComments = ' \n ' . join ( self . _PcdComments [ Pcd . TokenSpaceGuidCName , Pcd . TokenCName ] )
if PcdComments :
PcdItem = PcdComments + ' \n ' + PcdItem
AsBuiltInfDict [ ' patchablepcd_item ' ] . append ( PcdItem )
2014-08-28 13:53:34 +00:00
for Pcd in Pcds + VfrPcds :
2013-11-18 07:41:21 +00:00
PcdCommentList = [ ]
HiiInfo = ' '
2016-04-12 10:31:55 +08:00
TokenCName = Pcd . TokenCName
for PcdItem in GlobalData . MixedPcd :
if ( Pcd . TokenCName , Pcd . TokenSpaceGuidCName ) in GlobalData . MixedPcd [ PcdItem ] :
TokenCName = PcdItem [ 0 ]
break
2013-11-18 07:41:21 +00:00
if Pcd . Type == TAB_PCDS_DYNAMIC_EX_HII :
for SkuName in Pcd . SkuInfoList :
SkuInfo = Pcd . SkuInfoList [ SkuName ]
HiiInfo = ' ## %s | %s | %s ' % ( SkuInfo . VariableName , SkuInfo . VariableGuid , SkuInfo . VariableOffset )
break
if ( Pcd . TokenSpaceGuidCName , Pcd . TokenCName ) in self . _PcdComments :
PcdCommentList = self . _PcdComments [ Pcd . TokenSpaceGuidCName , Pcd . TokenCName ] [ : ]
if HiiInfo :
UsageIndex = - 1
2014-08-28 13:53:34 +00:00
UsageStr = ' '
2013-11-18 07:41:21 +00:00
for Index , Comment in enumerate ( PcdCommentList ) :
for Usage in UsageList :
if Comment . find ( Usage ) != - 1 :
2014-08-28 13:53:34 +00:00
UsageStr = Usage
2013-11-18 07:41:21 +00:00
UsageIndex = Index
break
if UsageIndex != - 1 :
2018-07-05 17:40:04 +08:00
PcdCommentList [ UsageIndex ] = ' ## %s %s %s ' % ( UsageStr , HiiInfo , PcdCommentList [ UsageIndex ] . replace ( UsageStr , ' ' ) )
2013-11-18 07:41:21 +00:00
else :
2014-08-28 13:53:34 +00:00
PcdCommentList . append ( ' ## UNDEFINED ' + HiiInfo )
2013-11-18 07:41:21 +00:00
PcdComments = ' \n ' . join ( PcdCommentList )
2016-04-12 10:31:55 +08:00
PcdEntry = Pcd . TokenSpaceGuidCName + ' . ' + TokenCName
2013-11-18 07:41:21 +00:00
if PcdComments :
PcdEntry = PcdComments + ' \n ' + PcdEntry
2018-05-18 08:06:52 +08:00
AsBuiltInfDict [ ' pcd_item ' ] . append ( PcdEntry )
2011-05-11 10:26:49 +00:00
for Item in self . BuildOption :
2016-04-22 15:08:47 +08:00
if ' FLAGS ' in self . BuildOption [ Item ] :
2018-05-18 08:06:52 +08:00
AsBuiltInfDict [ ' flags_item ' ] . append ( ' %s : %s _ %s _ %s _ %s _FLAGS = %s ' % ( self . ToolChainFamily , self . BuildTarget , self . ToolChain , self . Arch , Item , self . BuildOption [ Item ] [ ' FLAGS ' ] . strip ( ) ) )
2014-08-28 13:53:34 +00:00
# Generated LibraryClasses section in comments.
for Library in self . LibraryAutoGenList :
2018-05-18 08:06:52 +08:00
AsBuiltInfDict [ ' libraryclasses_item ' ] . append ( Library . MetaFile . File . replace ( ' \\ ' , ' / ' ) )
2018-07-05 17:40:04 +08:00
2017-04-13 14:33:05 +08:00
# Generated UserExtensions TianoCore section.
# All tianocore user extensions are copied.
UserExtStr = ' '
for TianoCore in self . _GetTianoCoreUserExtensionList ( ) :
UserExtStr + = ' \n ' . join ( TianoCore )
ExtensionFile = os . path . join ( self . MetaFile . Dir , TianoCore [ 1 ] )
if os . path . isfile ( ExtensionFile ) :
shutil . copy2 ( ExtensionFile , self . OutputDir )
AsBuiltInfDict [ ' userextension_tianocore_item ' ] = UserExtStr
2014-08-28 13:53:34 +00:00
# Generated depex expression section in comments.
DepexExpresion = self . _GetDepexExpresionString ( )
2018-05-18 08:06:52 +08:00
AsBuiltInfDict [ ' depexsection_item ' ] = DepexExpresion if DepexExpresion else ' '
2018-07-05 17:40:04 +08:00
2011-05-11 10:26:49 +00:00
AsBuiltInf = TemplateString ( )
AsBuiltInf . Append ( gAsBuiltInfHeaderString . Replace ( AsBuiltInfDict ) )
2018-07-05 17:40:04 +08:00
2011-05-11 10:26:49 +00:00
SaveFileOnChange ( os . path . join ( self . OutputDir , self . Name + ' .inf ' ) , str ( AsBuiltInf ) , False )
2018-07-05 17:40:04 +08:00
2011-05-11 10:26:49 +00:00
self . IsAsBuiltInfCreated = True
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
if GlobalData . gBinCacheDest :
self . CopyModuleToCache ( )
def CopyModuleToCache ( self ) :
FileDir = path . join ( GlobalData . gBinCacheDest , self . Arch , self . SourceDir , self . MetaFile . BaseName )
CreateDirectory ( FileDir )
HashFile = path . join ( self . BuildDir , self . Name + ' .hash ' )
ModuleFile = path . join ( self . OutputDir , self . Name + ' .inf ' )
if os . path . exists ( HashFile ) :
shutil . copy2 ( HashFile , FileDir )
if os . path . exists ( ModuleFile ) :
shutil . copy2 ( ModuleFile , FileDir )
2017-12-12 15:18:22 +08:00
if not self . OutputFile :
Ma = self . Workspace . BuildDatabase [ PathClass ( ModuleFile ) , self . Arch , self . BuildTarget , self . ToolChain ]
self . OutputFile = Ma . Binaries
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
if self . OutputFile :
for File in self . OutputFile :
2017-12-12 15:18:22 +08:00
File = str ( File )
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
if not os . path . isabs ( File ) :
File = os . path . join ( self . OutputDir , File )
if os . path . exists ( File ) :
shutil . copy2 ( File , FileDir )
def AttemptModuleCacheCopy ( self ) :
if self . IsBinaryModule :
return False
FileDir = path . join ( GlobalData . gBinCacheSource , self . Arch , self . SourceDir , self . MetaFile . BaseName )
HashFile = path . join ( FileDir , self . Name + ' .hash ' )
if os . path . exists ( HashFile ) :
f = open ( HashFile , ' r ' )
CacheHash = f . read ( )
f . close ( )
if GlobalData . gModuleHash [ self . Arch ] [ self . Name ] :
if CacheHash == GlobalData . gModuleHash [ self . Arch ] [ self . Name ] :
for root , dir , files in os . walk ( FileDir ) :
for f in files :
if self . Name + ' .hash ' in f :
shutil . copy2 ( HashFile , self . BuildDir )
else :
File = path . join ( root , f )
shutil . copy2 ( File , self . OutputDir )
if self . Name == " PcdPeim " or self . Name == " PcdDxe " :
CreatePcdDatabaseCode ( self , TemplateString ( ) , TemplateString ( ) )
return True
return False
2010-02-28 23:39:39 +00:00
## Create makefile for the module and its dependent libraries
#
# @param CreateLibraryMakeFile Flag indicating if or not the makefiles of
# dependent libraries will be created
#
2018-08-03 23:11:07 +08:00
@cached_class_function
2017-11-22 15:42:25 +08:00
def CreateMakeFile ( self , CreateLibraryMakeFile = True , GenFfsList = [ ] ) :
2018-08-03 23:11:07 +08:00
# nest this function inside it's only caller.
def CreateTimeStamp ( ) :
FileSet = { self . MetaFile . Path }
for SourceFile in self . Module . Sources :
FileSet . add ( SourceFile . Path )
for Lib in self . DependentLibraryList :
FileSet . add ( Lib . MetaFile . Path )
for f in self . AutoGenDepSet :
FileSet . add ( f . Path )
if os . path . exists ( self . TimeStampPath ) :
os . remove ( self . TimeStampPath )
with open ( self . TimeStampPath , ' w+ ' ) as file :
for f in FileSet :
print ( f , file = file )
2014-08-26 07:14:38 +00:00
# Ignore generating makefile when it is a binary module
2014-08-26 02:52:24 +00:00
if self . IsBinaryModule :
return
2014-08-26 07:14:38 +00:00
2017-11-22 15:42:25 +08:00
self . GenFfsList = GenFfsList
2010-02-28 23:39:39 +00:00
if not self . IsLibrary and CreateLibraryMakeFile :
for LibraryAutoGen in self . LibraryAutoGenList :
LibraryAutoGen . CreateMakeFile ( )
2017-11-13 10:50:49 +08:00
if self . CanSkip ( ) :
return
2010-02-28 23:39:39 +00:00
if len ( self . CustomMakefile ) == 0 :
Makefile = GenMake . ModuleMakefile ( self )
else :
Makefile = GenMake . CustomMakefile ( self )
if Makefile . Generate ( ) :
EdkLogger . debug ( EdkLogger . DEBUG_9 , " Generated makefile for module %s [ %s ] " %
( self . Name , self . Arch ) )
else :
EdkLogger . debug ( EdkLogger . DEBUG_9 , " Skipped the generation of makefile for module %s [ %s ] " %
( self . Name , self . Arch ) )
2018-08-03 23:11:07 +08:00
CreateTimeStamp ( )
2010-02-28 23:39:39 +00:00
2014-08-26 02:52:24 +00:00
def CopyBinaryFiles ( self ) :
for File in self . Module . Binaries :
SrcPath = File . Path
2018-06-25 18:31:33 +08:00
DstPath = os . path . join ( self . OutputDir , os . path . basename ( SrcPath ) )
2014-08-26 02:52:24 +00:00
CopyLongFilePath ( SrcPath , DstPath )
2010-02-28 23:39:39 +00:00
## Create autogen code for the module and its dependent libraries
#
# @param CreateLibraryCodeFile Flag indicating if or not the code of
# dependent libraries will be created
#
def CreateCodeFile ( self , CreateLibraryCodeFile = True ) :
if self . IsCodeFileCreated :
return
2013-11-18 07:41:21 +00:00
# Need to generate PcdDatabase even PcdDriver is binarymodule
if self . IsBinaryModule and self . PcdIsDriver != ' ' :
CreatePcdDatabaseCode ( self , TemplateString ( ) , TemplateString ( ) )
return
2014-08-26 02:52:24 +00:00
if self . IsBinaryModule :
2014-09-04 08:32:44 +00:00
if self . IsLibrary :
self . CopyBinaryFiles ( )
2014-08-26 02:52:24 +00:00
return
2013-11-18 07:41:21 +00:00
2010-02-28 23:39:39 +00:00
if not self . IsLibrary and CreateLibraryCodeFile :
for LibraryAutoGen in self . LibraryAutoGenList :
LibraryAutoGen . CreateCodeFile ( )
2017-11-13 10:50:49 +08:00
if self . CanSkip ( ) :
return
2010-02-28 23:39:39 +00:00
AutoGenList = [ ]
IgoredAutoGenList = [ ]
for File in self . AutoGenFileList :
if GenC . Generate ( File . Path , self . AutoGenFileList [ File ] , File . IsBinary ) :
2011-09-18 12:17:25 +00:00
#Ignore Edk AutoGen.c
2010-02-28 23:39:39 +00:00
if self . AutoGenVersion < 0x00010005 and File . Name == ' AutoGen.c ' :
continue
AutoGenList . append ( str ( File ) )
else :
IgoredAutoGenList . append ( str ( File ) )
# Skip the following code for EDK I inf
if self . AutoGenVersion < 0x00010005 :
return
for ModuleType in self . DepexList :
2018-04-27 00:57:53 +08:00
# Ignore empty [depex] section or [depex] section for SUP_MODULE_USER_DEFINED module
if len ( self . DepexList [ ModuleType ] ) == 0 or ModuleType == SUP_MODULE_USER_DEFINED :
2010-02-28 23:39:39 +00:00
continue
2010-05-18 05:04:32 +00:00
2010-02-28 23:39:39 +00:00
Dpx = GenDepex . DependencyExpression ( self . DepexList [ ModuleType ] , ModuleType , True )
DpxFile = gAutoGenDepexFileName % { " module_name " : self . Name }
2018-06-25 18:31:27 +08:00
if len ( Dpx . PostfixNotation ) != 0 :
2011-10-29 06:59:30 +00:00
self . DepexGenerated = True
2011-05-11 10:26:49 +00:00
2010-02-28 23:39:39 +00:00
if Dpx . Generate ( path . join ( self . OutputDir , DpxFile ) ) :
AutoGenList . append ( str ( DpxFile ) )
else :
IgoredAutoGenList . append ( str ( DpxFile ) )
if IgoredAutoGenList == [ ] :
EdkLogger . debug ( EdkLogger . DEBUG_9 , " Generated [ %s ] files for module %s [ %s ] " %
( " " . join ( AutoGenList ) , self . Name , self . Arch ) )
elif AutoGenList == [ ] :
EdkLogger . debug ( EdkLogger . DEBUG_9 , " Skipped the generation of [ %s ] files for module %s [ %s ] " %
( " " . join ( IgoredAutoGenList ) , self . Name , self . Arch ) )
else :
EdkLogger . debug ( EdkLogger . DEBUG_9 , " Generated [ %s ] (skipped %s ) files for module %s [ %s ] " %
( " " . join ( AutoGenList ) , " " . join ( IgoredAutoGenList ) , self . Name , self . Arch ) )
self . IsCodeFileCreated = True
return AutoGenList
## Summarize the ModuleAutoGen objects of all libraries used by this module
2018-08-03 23:11:06 +08:00
@cached_property
def LibraryAutoGenList ( self ) :
RetVal = [ ]
for Library in self . DependentLibraryList :
La = ModuleAutoGen (
2010-02-28 23:39:39 +00:00
self . Workspace ,
Library . MetaFile ,
self . BuildTarget ,
self . ToolChain ,
self . Arch ,
self . PlatformInfo . MetaFile
)
2018-08-03 23:11:06 +08:00
if La not in RetVal :
RetVal . append ( La )
for Lib in La . CodaTargetList :
self . _ApplyBuildRule ( Lib . Target , TAB_UNKNOWN_FILE )
return RetVal
2010-02-28 23:39:39 +00:00
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
def GenModuleHash ( self ) :
if self . Arch not in GlobalData . gModuleHash :
GlobalData . gModuleHash [ self . Arch ] = { }
m = hashlib . md5 ( )
# Add Platform level hash
m . update ( GlobalData . gPlatformHash )
# Add Package level hash
if self . DependentPackageList :
2018-05-09 17:03:23 +08:00
for Pkg in sorted ( self . DependentPackageList , key = lambda x : x . PackageName ) :
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
if Pkg . PackageName in GlobalData . gPackageHash [ self . Arch ] :
m . update ( GlobalData . gPackageHash [ self . Arch ] [ Pkg . PackageName ] )
# Add Library hash
if self . LibraryAutoGenList :
2018-05-09 17:03:23 +08:00
for Lib in sorted ( self . LibraryAutoGenList , key = lambda x : x . Name ) :
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
if Lib . Name not in GlobalData . gModuleHash [ self . Arch ] :
Lib . GenModuleHash ( )
m . update ( GlobalData . gModuleHash [ self . Arch ] [ Lib . Name ] )
# Add Module self
f = open ( str ( self . MetaFile ) , ' r ' )
Content = f . read ( )
f . close ( )
m . update ( Content )
# Add Module's source files
if self . SourceFileList :
2018-05-09 17:03:23 +08:00
for File in sorted ( self . SourceFileList , key = lambda x : str ( x ) ) :
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
f = open ( str ( File ) , ' r ' )
Content = f . read ( )
f . close ( )
m . update ( Content )
ModuleHashFile = path . join ( self . BuildDir , self . Name + " .hash " )
if self . Name not in GlobalData . gModuleHash [ self . Arch ] :
GlobalData . gModuleHash [ self . Arch ] [ self . Name ] = m . hexdigest ( )
if GlobalData . gBinCacheSource :
2018-05-18 08:06:52 +08:00
if self . AttemptModuleCacheCopy ( ) :
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
return False
return SaveFileOnChange ( ModuleHashFile , m . hexdigest ( ) , True )
## Decide whether we can skip the ModuleAutoGen process
def CanSkipbyHash ( self ) :
if GlobalData . gUseHashCache :
return not self . GenModuleHash ( )
2018-07-02 09:15:47 +08:00
return False
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
## Decide whether we can skip the ModuleAutoGen process
BaseTools: add support for BIOS build with binary cache
Add three new options:
--hash enables hash-based caching during build process. when --hash is
enabled, build tool will base on the module hash value to do the
incremental build, without --hash, build tool will base on the
timestamp to do the incremental build. --hash option use md5 method to
get every hash value, DSC/FDF, tools_def.txt, build_rule.txt and build
command are calculated as global hash value, Package DEC and its
include header files are calculated as package hash value, Module
source files and its INF file are calculated as module hash value.
Library hash value will combine the global hash value and its dependent
package hash value. Driver hash value will combine the global hash
value, its dependent package hash value and its linked library hash
value.
When --hash and --binary-destination are specified, build tool will
copy generated binary files for each module into the directory specified
by binary-destination at the build phase. Binary-destination directory
caches all generated binary files.
When --hash and --binary-source are specified, build tool will try to
get the binary files from the binary source directory at the build
phase.If the cached binary has the same hash value, it will be directly
used. Otherwise, build tool will compile the source files and generate
the binary files.
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
2017-09-19 14:31:09 +08:00
# If any source file is newer than the module than we cannot skip
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
#
def CanSkip ( self ) :
2018-07-11 00:19:24 +08:00
if self . MakeFileDir in GlobalData . gSikpAutoGenCache :
2018-07-02 09:15:47 +08:00
return True
2018-08-03 23:11:06 +08:00
if not os . path . exists ( self . TimeStampPath ) :
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
return False
#last creation time of the module
2018-08-03 23:11:06 +08:00
DstTimeStamp = os . stat ( self . TimeStampPath ) [ 8 ]
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
SrcTimeStamp = self . Workspace . _SrcTimeStamp
if SrcTimeStamp > DstTimeStamp :
return False
2018-08-03 23:11:06 +08:00
with open ( self . TimeStampPath , ' r ' ) as f :
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
for source in f :
source = source . rstrip ( ' \n ' )
2017-06-06 16:35:44 +08:00
if not os . path . exists ( source ) :
return False
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
if source not in ModuleAutoGen . TimeDict :
ModuleAutoGen . TimeDict [ source ] = os . stat ( source ) [ 8 ]
if ModuleAutoGen . TimeDict [ source ] > DstTimeStamp :
return False
2018-07-11 00:19:24 +08:00
GlobalData . gSikpAutoGenCache . add ( self . MakeFileDir )
BaseTools: Skip module AutoGen by comparing timestamp.
[Introduction]
The BaseTool Build.py AutoGen parse INF meta-file and generate
AutoGen.c/AutoGen.h/makefile. When we only change .c .h code, the
AutoGen might be not necessary, but Build.py spend a lot of time on it.
There's a -u flag to skip all module's AutoGen. In my environment, it save
35%~50% of time in rebuild a ROM.
However, if user change one .INF meta-file, then -u flag is not available.
[Idea]
AutoGen can compare meta-file's timestamp and decide if the module's
AutoGen can be skipped. With this, when a module's INF is changed, we
only run this module's AutoGen, we don't need to run other module's.
[Implementation]
In the end of a module's AutoGen, we create a AutoGenTimeStamp.
The file save a file list that related to this module's AutoGen.
In other word, the file list in AutoGenTimeStamp is INPUT files of
module AutoGen, AutoGenTimeStamp file is OUTPUT.
During rebuild, we compare time stamp between INPUT and OUTPUT, and
decide if we can skip it.
Below is the Input/Output of a module's AutoGen.
[Input]
1. All the DSC/DEC/FDF used by the platform.
2. Macro and PCD defined by Build Options such as "build -D AAA=TRUE
--pcd BbbPcd=0".
3. INF file of a module.
4. Source files of a module, list in [Sources] section of INF.
5. All the library link by the module.
6. All the .h files included by the module's sources.
[Output]
AutoGen.c/AutoGen.h/makefile/AutoGenTimeStamp
[Testing]
This patch save my build time. When I make a change without touching
DSC/DEC/FDF, it is absolutely much faster than original rebuild,
35%~50% time saving in my environment
(compare to original tool rebuild time).
If I change any DSC/DEC/FDF, there's no performance improve, because it
can't skip any module's AutoGen.
Please note that if your environment will generate DSC/FDF during prebuild,
it will not skip any AutoGen because of DSC timestamp is changed. This will
require prebuild script not to update metafile when content is not changed.
2017-02-24 15:26:19 +08:00
return True
2018-08-03 23:11:06 +08:00
@cached_property
def TimeStampPath ( self ) :
return os . path . join ( self . MakeFileDir , ' AutoGenTimeStamp ' )