[edk2] [PATCH] BaseTools: Fix back-to-back !include in FDF files

[Samer El-Haj-Mahmoud]

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Samer El-Haj-Mahmoud <samer.el-haj-***@hp.com>
---
BaseTools/Source/Python/GenFds/FdfParser.py | 9 +++++++++
1 file changed, 9 insertions(+)

diff --git a/BaseTools/Source/Python/GenFds/FdfParser.py b/BaseTools/Source/Python/GenFds/FdfParser.py
index 8091a51..15ec886 100644
--- a/BaseTools/Source/Python/GenFds/FdfParser.py
+++ b/BaseTools/Source/Python/GenFds/FdfParser.py
@@ -1,6 +1,7 @@
## @file
# parse FDF file
#
+# (C) Copyright 2015 Hewlett-Packard Development Company, L.P.<BR>
# Copyright (c) 2007 - 2014, Intel Corporation. All rights reserved.<BR>
#
# This program and the accompanying materials
@@ -639,6 +640,14 @@ class FdfParser:
TempList.insert(IncludeOffset, '#')
self.Profile.FileLinesList[IncludeLine - 1] = ''.join(TempList)

+ #
+ # Fix back-to-back !include in fdf parsing
+ # For example:
+ # !include Nt32Pkg/a.fdf
+ # !include Nt32Pkg/b.fdf
+ # Without this change the preprocessor will not expand b.fdf.
+ #
+ self.CurrentOffsetWithinLine = 0
self.Rewind()

def __GetIfListCurrentItemStat(self, IfList):
--
1.9.5.msysgit.1

[Liu, Yingke D]

Hi Samer,

Thanks for your patch, after take a look at the issue, I found another bug in FDF parser. Following patch fixed both issues:

If there are two continuous !include lines, the second !include is ignored.
Also if the last line in !include file is not ends with '\r' or '\n', a '\n' should be appended, otherwise index out of range exceptions is raised.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Yingke Liu <***@intel.com>
---
BaseTools/Source/Python/GenFds/FdfParser.py | 4 ++++
1 file changed, 4 insertions(+)

diff --git a/BaseTools/Source/Python/GenFds/FdfParser.py b/BaseTools/Source/Python/GenFds/FdfParser.py
index ffc54ab..2313174 100644
--- a/BaseTools/Source/Python/GenFds/FdfParser.py
+++ b/BaseTools/Source/Python/GenFds/FdfParser.py
@@ -149,6 +149,8 @@ class IncludeFileProfile :
fsock = open(FileName, "rb", 0)
try:
self.FileLinesList = fsock.readlines()
+ if self.FileLinesList and self.FileLinesList[-1][-1] not in ('\r', '\n'):
+ self.FileLinesList[-1] = self.FileLinesList[-1] + '\n'
finally:
fsock.close()

@@ -627,6 +629,8 @@ class FdfParser:
IncFileProfile.InsertAdjust += 1
self.CurrentLineNumber += 1
self.CurrentOffsetWithinLine = 0
+ else:
+ self.__UndoToken()

for Line in IncFileProfile.FileLinesList:
self.Profile.FileLinesList.insert(InsertAtLine, Line)
--
1.9.5.msysgit.0

Dennis

-----Original Message-----
From: Samer El-Haj-Mahmoud [mailto:samer.el-haj-***@hp.com]
Sent: Thursday, July 02, 2015 9:12
To: edk2-***@lists.sourceforge.net
Cc: Liu, Yingke D; Samer El-Haj-Mahmoud
Subject: [PATCH] BaseTools: Fix back-to-back !include in FDF files

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Samer El-Haj-Mahmoud <samer.el-haj-***@hp.com>
---
BaseTools/Source/Python/GenFds/FdfParser.py | 9 +++++++++
1 file changed, 9 insertions(+)

diff --git a/BaseTools/Source/Python/GenFds/FdfParser.py b/BaseTools/Source/Python/GenFds/FdfParser.py
index 8091a51..15ec886 100644
--- a/BaseTools/Source/Python/GenFds/FdfParser.py
+++ b/BaseTools/Source/Python/GenFds/FdfParser.py
@@ -1,6 +1,7 @@
## @file
# parse FDF file
#
+# (C) Copyright 2015 Hewlett-Packard Development Company, L.P.<BR>
# Copyright (c) 2007 - 2014, Intel Corporation. All rights reserved.<BR> # # This program and the accompanying materials @@ -639,6 +640,14 @@ class FdfParser:
TempList.insert(IncludeOffset, '#')
self.Profile.FileLinesList[IncludeLine - 1] = ''.join(TempList)

+ #
+ # Fix back-to-back !include in fdf parsing
+ # For example:
+ # !include Nt32Pkg/a.fdf
+ # !include Nt32Pkg/b.fdf
+ # Without this change the preprocessor will not expand b.fdf.
+ #
+ self.CurrentOffsetWithinLine = 0
self.Rewind()

def __GetIfListCurrentItemStat(self, IfList):
--
1.9.5.msysgit.1

[El-Haj-Mahmoud, Samer]

Thanks Yingke,

If your patch fixes both issues then that is great! Let's go with your patch.

Reviewed-by: Samer El-Haj-Mahmoud <***@hp.com>


Thanks,
Samer





-----Original Message-----
From: Liu, Yingke D [***@intel.com]
Received: Wednesday, 01 Jul 2015, 11:21PM
To: El-Haj-Mahmoud, Samer [samer.el-haj-***@hp.com]; edk2-***@lists.sourceforge.net [edk2-***@lists.sourceforge.net]
Subject: RE: [PATCH] BaseTools: Fix back-to-back !include in FDF files

Hi Samer,

Thanks for your patch, after take a look at the issue, I found another bug in FDF parser. Following patch fixed both issues:

If there are two continuous !include lines, the second !include is ignored.
Also if the last line in !include file is not ends with '\r' or '\n', a '\n' should be appended, otherwise index out of range exceptions is raised.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Yingke Liu <***@intel.com>
---
BaseTools/Source/Python/GenFds/FdfParser.py | 4 ++++
1 file changed, 4 insertions(+)

diff --git a/BaseTools/Source/Python/GenFds/FdfParser.py b/BaseTools/Source/Python/GenFds/FdfParser.py
index ffc54ab..2313174 100644
--- a/BaseTools/Source/Python/GenFds/FdfParser.py
+++ b/BaseTools/Source/Python/GenFds/FdfParser.py
@@ -149,6 +149,8 @@ class IncludeFileProfile :
fsock = open(FileName, "rb", 0)
try:
self.FileLinesList = fsock.readlines()
+ if self.FileLinesList and self.FileLinesList[-1][-1] not in ('\r', '\n'):
+ self.FileLinesList[-1] = self.FileLinesList[-1] + '\n'
finally:
fsock.close()

@@ -627,6 +629,8 @@ class FdfParser:
IncFileProfile.InsertAdjust += 1
self.CurrentLineNumber += 1
self.CurrentOffsetWithinLine = 0
+ else:
+ self.__UndoToken()

for Line in IncFileProfile.FileLinesList:
self.Profile.FileLinesList.insert(InsertAtLine, Line)
--
1.9.5.msysgit.0

Dennis

-----Original Message-----
From: Samer El-Haj-Mahmoud [mailto:samer.el-haj-***@hp.com]
Sent: Thursday, July 02, 2015 9:12
To: edk2-***@lists.sourceforge.net
Cc: Liu, Yingke D; Samer El-Haj-Mahmoud
Subject: [PATCH] BaseTools: Fix back-to-back !include in FDF files

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Samer El-Haj-Mahmoud <samer.el-haj-***@hp.com>
---
BaseTools/Source/Python/GenFds/FdfParser.py | 9 +++++++++
1 file changed, 9 insertions(+)

diff --git a/BaseTools/Source/Python/GenFds/FdfParser.py b/BaseTools/Source/Python/GenFds/FdfParser.py
index 8091a51..15ec886 100644
--- a/BaseTools/Source/Python/GenFds/FdfParser.py
+++ b/BaseTools/Source/Python/GenFds/FdfParser.py
@@ -1,6 +1,7 @@
## @file
# parse FDF file
#
+# (C) Copyright 2015 Hewlett-Packard Development Company, L.P.<BR>
# Copyright (c) 2007 - 2014, Intel Corporation. All rights reserved.<BR> # # This program and the accompanying materials @@ -639,6 +640,14 @@ class FdfParser:
TempList.insert(IncludeOffset, '#')
self.Profile.FileLinesList[IncludeLine - 1] = ''.join(TempList)

+ #
+ # Fix back-to-back !include in fdf parsing
+ # For example:
+ # !include Nt32Pkg/a.fdf
+ # !include Nt32Pkg/b.fdf
+ # Without this change the preprocessor will not expand b.fdf.
+ #
+ self.CurrentOffsetWithinLine = 0
self.Rewind()

def __GetIfListCurrentItemStat(self, IfList):
--
1.9.5.msysgit.1