[edk2] Failures building due to tool problems.

Discussion:

Roberts, Bruce (CoSpgs)

2015-06-05 15:48:08 UTC

Good morning.

We have been seeing this sort of problem for some time and a build retry seemed to work but now this is getting worse and it can take 10 retries or more to get a build to work. I am not sure how to begin debugging the toolset so I wanted to ask for some suggestions or help. I have found some information about rebuilding the Python tool set with the freeze tool but so far that tool has proven elusive. Notes I have refer to running exe files to accomplish that but the version I installed has nothing by Python scripts and did not install where it indicated it would so I am still trying to figure that out.

But the root problems I am trying to solve is the operational error shown here. As requested the Python error stack is included.

*********************
* 64-bit UEFI build *
*********************
G:\uefi_Hurricane
Get UEFI Build Data, Version 1.7, Built Dec 16 2014
- Recognized build type argument: Release (2)
- Generating Rev & Build data in: G:\uefi_Hurricane\Build\RomleyPkg\DEBUG_VS2008x86\FV\HPUefiVersion.bin
- Parsing the UEFI code from: G:\uefi_Hurricane\...
Build environment: Windows-Vista-6.1.7601
Build start time: 07:43:23, Jun.05 2015

WORKSPACE = g:\uefi_hurricane
ECP_SOURCE = g:\uefi_hurricane\edkcompatibilitypkg
EDK_SOURCE = g:\uefi_hurricane\edkcompatibilitypkg
EFI_SOURCE = g:\uefi_hurricane\r8romleypkg
EDK_TOOLS_PATH = g:\uefi_hurricane\basetools

TARGET_ARCH = IA32 X64
TARGET = DEBUG
TOOL_CHAIN_TAG = VS2008x86

Active Platform = g:\uefi_hurricane\RomleyPkg\RomleyPkg.dsc
Flash Image Definition = g:\uefi_hurricane\RomleyPkg\RomleyPkg.fdf

Processing meta-data ......

build...
: error C0DE: Unknown fatal error when processing [g:\uefi_hurricane\RomleyPkg\RomleyPkg.dsc]

(Please send email to edk2-buildtools-***@lists.sourceforge.net<mailto:edk2-buildtools-***@lists.sourceforge.net> for help, attaching following call stack trace!)

(Python 2.5.2 on win32) Traceback (most recent call last):
File "build.py", line 1863, in Main
File "build.py", line 1628, in Launch
File "build.py", line 1463, in _MultiThreadBuildPlatform
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\AutoGen\AutoGen.py", line 83, in __new__
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\AutoGen\AutoGen.py", line 190, in _Init
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\AutoGen\AutoGen.py", line 533, in CollectPlatformDynamicPcds
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\AutoGen\AutoGen.py", line 2142, in _GetLibraryPcdList
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\AutoGen\AutoGen.py", line 2120, in _GetLibraryList
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\AutoGen\AutoGen.py", line 1073, in lyLibraryInstance
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\Workspace\WorkspaceDatabase.py", line 2221, in __getitem__
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\Workspace\WorkspaceDatabase.py", line 2470, in __getitem__
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\Workspace\MetaDataTable.py", line 65, in Create
OperationalError: unable to open database file

- Failed -
Build end time: 07:43:31, Jun.05 2015
Build total time: 00:08
R. Bruce Roberts
Systems/Software Engineer
Utility Development Unit / HP Storage
Hewlett-Packard Company
719-548-2736 or 719-551-3303
***@hp.com<mailto:***@hp.com>

Andrew Fish

2015-06-05 15:58:28 UTC

Permalink

Try deleting Conf/.cache
You can add --no-cache to the build command.

I don't know what the root cause is, probably some threading issue. It would be nice if the tools would just fix this error (delete cache and rebuild it)

Sent from my iPhone

Post by Roberts, Bruce (CoSpgs)
Good morning.
We have been seeing this sort of problem for some time and a build retry seemed to work but now this is getting worse and it can take 10 retries or more to get a build to work. I am not sure how to begin debugging the toolset so I wanted to ask for some suggestions or help. I have found some information about rebuilding the Python tool set with the freeze tool but so far that tool has proven elusive. Notes I have refer to running exe files to accomplish that but the version I installed has nothing by Python scripts and did not install where it indicated it would so I am still trying to figure that out.
But the root problems I am trying to solve is the operational error shown here. As requested the Python error stack is included.
*********************
* 64-bit UEFI build *
*********************
G:\uefi_Hurricane
Get UEFI Build Data, Version 1.7, Built Dec 16 2014
- Recognized build type argument: Release (2)
- Generating Rev & Build data in: G:\uefi_Hurricane\Build\RomleyPkg\DEBUG_VS2008x86\FV\HPUefiVersion.bin
- Parsing the UEFI code from: G:\uefi_Hurricane\...
Build environment: Windows-Vista-6.1.7601
Build start time: 07:43:23, Jun.05 2015
WORKSPACE = g:\uefi_hurricane
ECP_SOURCE = g:\uefi_hurricane\edkcompatibilitypkg
EDK_SOURCE = g:\uefi_hurricane\edkcompatibilitypkg
EFI_SOURCE = g:\uefi_hurricane\r8romleypkg
EDK_TOOLS_PATH = g:\uefi_hurricane\basetools
TARGET_ARCH = IA32 X64
TARGET = DEBUG
TOOL_CHAIN_TAG = VS2008x86
Active Platform = g:\uefi_hurricane\RomleyPkg\RomleyPkg.dsc
Flash Image Definition = g:\uefi_hurricane\RomleyPkg\RomleyPkg.fdf
Processing meta-data ......
build...
: error C0DE: Unknown fatal error when processing [g:\uefi_hurricane\RomleyPkg\RomleyPkg.dsc]
File "build.py", line 1863, in Main
File "build.py", line 1628, in Launch
File "build.py", line 1463, in _MultiThreadBuildPlatform
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\AutoGen\AutoGen.py", line 83, in __new__
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\AutoGen\AutoGen.py", line 190, in _Init
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\AutoGen\AutoGen.py", line 533, in CollectPlatformDynamicPcds
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\AutoGen\AutoGen.py", line 2142, in _GetLibraryPcdList
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\AutoGen\AutoGen.py", line 2120, in _GetLibraryList
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\AutoGen\AutoGen.py", line 1073, in lyLibraryInstance
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\Workspace\WorkspaceDatabase.py", line 2221, in __getitem__
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\Workspace\WorkspaceDatabase.py", line 2470, in __getitem__
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\Workspace\MetaDataTable.py", line 65, in Create
OperationalError: unable to open database file
- Failed -
Build end time: 07:43:31, Jun.05 2015
Build total time: 00:08
R. Bruce Roberts
Systems/Software Engineer
Utility Development Unit / HP Storage
Hewlett-Packard Company
719-548-2736 or 719-551-3303
------------------------------------------------------------------------------
_______________________________________________
edk2-devel mailing list
https://lists.sourceforge.net/lists/listinfo/edk2-devel

Roberts, Bruce (CoSpgs)

2015-06-05 20:48:21 UTC

Permalink

Andrew,

I neglected to add some additional information that makes your suggestion moot. This is occurring on a new clone from GIT on an VM system used for automated builds under the control of a script and the Conf\.cache directory (windows box) does not exist until the make is executed the first time so there is nothing to delete on the first attempt to make the build work.

However, in the mean time we have looked at other things and discovered that the IT department here pushed out some antivirus changes June 1 and it appears that the AV code is locking up that file as soon as it is created and when the next step in the build process tries to open it again for write the AV code has it open for a scan and that is causing the failure. This started occurring 100% of the time after that update by IT where it was intermittent before that.

To prove this was the root of the problem I disabled the AV and have been able to run several successful builds without seeing another failure. So now I need a way to selectively not scan those files or critical directories, disable the AV scan from the script for a short period during a build or some other equivalent fix to make these software packages play together. And then get IT to agree to those changes.

Thanks for the quick response. I think I have all the information I need for this issue.
R. Bruce Roberts
Systems/Software Engineer
Utility Development Unit / HP Storage
Hewlett-Packard Company
719-548-2736 or 719-551-3303
***@hp.com<mailto:***@hp.com>

From: Andrew Fish [mailto:***@apple.com]
Sent: Friday, June 05, 2015 9:58 AM
To: edk2-***@lists.sourceforge.net
Cc: Brooks, Ted; Roberts, Bruce (CoSpgs)
Subject: Re: [edk2] Failures building due to tool problems.

Try deleting Conf/.cache
You can add --no-cache to the build command.

I don't know what the root cause is, probably some threading issue. It would be nice if the tools would just fix this error (delete cache and rebuild it)

Sent from my iPhone

On Jun 5, 2015, at 8:48 AM, Roberts, Bruce (CoSpgs) <***@hp.com<mailto:***@hp.com>> wrote:
Good morning.

We have been seeing this sort of problem for some time and a build retry seemed to work but now this is getting worse and it can take 10 retries or more to get a build to work. I am not sure how to begin debugging the toolset so I wanted to ask for some suggestions or help. I have found some information about rebuilding the Python tool set with the freeze tool but so far that tool has proven elusive. Notes I have refer to running exe files to accomplish that but the version I installed has nothing by Python scripts and did not install where it indicated it would so I am still trying to figure that out.

But the root problems I am trying to solve is the operational error shown here. As requested the Python error stack is included.

*********************
* 64-bit UEFI build *
*********************
G:\uefi_Hurricane
Get UEFI Build Data, Version 1.7, Built Dec 16 2014
- Recognized build type argument: Release (2)
- Generating Rev & Build data in: G:\uefi_Hurricane\Build\RomleyPkg\DEBUG_VS2008x86\FV\HPUefiVersion.bin
- Parsing the UEFI code from: G:\uefi_Hurricane\...
Build environment: Windows-Vista-6.1.7601
Build start time: 07:43:23, Jun.05 2015

WORKSPACE = g:\uefi_hurricane
ECP_SOURCE = g:\uefi_hurricane\edkcompatibilitypkg
EDK_SOURCE = g:\uefi_hurricane\edkcompatibilitypkg
EFI_SOURCE = g:\uefi_hurricane\r8romleypkg
EDK_TOOLS_PATH = g:\uefi_hurricane\basetools

TARGET_ARCH = IA32 X64
TARGET = DEBUG
TOOL_CHAIN_TAG = VS2008x86

Active Platform = g:\uefi_hurricane\RomleyPkg\RomleyPkg.dsc
Flash Image Definition = g:\uefi_hurricane\RomleyPkg\RomleyPkg.fdf

Processing meta-data ......

build...
: error C0DE: Unknown fatal error when processing [g:\uefi_hurricane\RomleyPkg\RomleyPkg.dsc]

(Please send email to edk2-buildtools-***@lists.sourceforge.net<mailto:edk2-buildtools-***@lists.sourceforge.net> for help, attaching following call stack trace!)

(Python 2.5.2 on win32) Traceback (most recent call last):
File "build.py", line 1863, in Main
File "build.py", line 1628, in Launch
File "build.py", line 1463, in _MultiThreadBuildPlatform
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\AutoGen\AutoGen.py", line 83, in __new__
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\AutoGen\AutoGen.py", line 190, in _Init
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\AutoGen\AutoGen.py", line 533, in CollectPlatformDynamicPcds
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\AutoGen\AutoGen.py", line 2142, in _GetLibraryPcdList
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\AutoGen\AutoGen.py", line 2120, in _GetLibraryList
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\AutoGen\AutoGen.py", line 1073, in lyLibraryInstance
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\Workspace\WorkspaceDatabase.py", line 2221, in __getitem__
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\Workspace\WorkspaceDatabase.py", line 2470, in __getitem__
File "C:\NightlyTest\FreezeTool\BaseTools\Source\Python\Workspace\MetaDataTable.py", line 65, in Create
OperationalError: unable to open database file

- Failed -
Build end time: 07:43:31, Jun.05 2015
Build total time: 00:08
R. Bruce Roberts
Systems/Software Engineer
Utility Development Unit / HP Storage
Hewlett-Packard Company
719-548-2736 or 719-551-3303
***@hp.com<mailto:***@hp.com>

------------------------------------------------------------------------------
_______________________________________________
edk2-devel mailing list
edk2-***@lists.sourceforge.net<mailto:edk2-***@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/edk2-devel