Discussion:
[edk2] Question about the UEFI. Can you help me?
Andrei Warkentin
2011-03-27 10:04:35 UTC
Permalink
Hi,

As you work on your proposal, keep a few things in mind. Google Summer
of Code is about 12 weeks, which isn't as much time as it might seem.
I haven't seen your draft proposal, but I would suggest to think about
what you feel is an important goal (or a few goals) that you want to
achieve for sure, then think what you can work on after that, given
the time constraints. Think about the time management, potential
issues you might run into, and what you think you should have by
mid-term and beyond. Cut down the idea of "OVMF on Xen" into
manageable, implementable in 12-weeks chunks. As a personal exercise,
you could plan out what you feel you should have week-by-week. It's
not likely you will stick to that schedule, but it should give you a
realistic idea of what you can expect to finish up by end of GSoC.
Think about what items are within the scope of "OVMF on Xen"
specifically, and not just within the scope of "OVMF" or "UEFI" in
general. For example, legacy boot support (i.e. "CSM - Compatibility
Support Module") in OVMF is a great idea, but it's a fairly
complicated project in itself (I'd rate it "Hard", see
ftp://download.intel.com/technology/framework/docs/csm_0_96.pdf for an
outdated picture of the problem space, although you could do some
intelligent chain-load thing, I suppose from OVMF into SeaBios), and
definitely outside the scope for the "OVMF on Xen" project.

I see the project overall as being fairly open-ended. The basic thing
that you should aim for, is to have OVMF instead of the regular BIOS
boot up inside a Xen HVM domain, thinking it's running on real
hardware - Ide controller, VGA video, ps/2 keyboard, etc. Ideally, it
should work in same fashion as on QEMU, since Xen uses QEMU-derived
emulators... You should be able to interact with the UEFI Shell and
access block devices. After that, it's up to you to figure out where
you want to take this. Here are some ideas, grouped by categories, and
listed in the order of complexity. This is by no means an exhaustive
list, you might have great ideas I haven't listed below...

Category A - Ensure OVMF-on-Xen has feature parity with OVMF-on-Qemu.

1) Verify you can boot into OSes UEFI-way from your OVMF-on-Xen. You
should be able to boot whatever is bootable in OVMF on QEMU.
2) ???

Category B - optimize OVMF for Xen HVM domains.

0) Develop a driver that handles Xen hypervisor support (grant tables,
event channels, hypercalls).
1) XenStore/XenBus is an information storage space, akin to a
registry. It is necessary to get any Xen ParaVirtualized devices
working in UEFI. Depends on (0).
2) Implement a SIMPLE_FILE_SYSTEM_PROTOCOL that exposes the XenStore
namespace as a file system. This might make the next steps easier.
3) Implement virtual frame buffer support instead of using vga text.
Depends on (1).
4) Implement ParaVirtualized block storage. This means that instead of
using the IDE hardware emulator in qemu via the UEFI IDE driver, you
use the the virtual xen device. So instead of touching I/O space,
causing an intercept every access that causing a domain switch so the
access is handled inside the I/O emulators, block i/o requests are
handled using a simple ring-buffer and an event channel. This should
provide a boost on storage access. Depends on (1).
5) Implement virtual network support. Depends on (1).
6) UEFI has a concept of "variable services". The most interesting use
for variables is boot device selection. In OVMF, the variables are
synced with an NvVars file on a filesystem. Investigate what it takes
to create a simple Xen device that can be used to store variables.
Depends on (1).

Category C - Depends on B for PV device support.

1) Make OVMF run inside a PV domain. I.e. - you have no emulated I/O
devices, only the Xen abstractions.

Thanks and good luck with your proposal. I've added some inlined
comments below as well.
I have a question about UEFI variables. So, what are the UEFI variables and
what are they used for?
Bei Guan
2011-03-29 16:28:09 UTC
Permalink
Post by Andrei Warkentin
Hi,
As you work on your proposal, keep a few things in mind. Google Summer
of Code is about 12 weeks, which isn't as much time as it might seem.
I haven't seen your draft proposal, but I would suggest to think about
what you feel is an important goal (or a few goals) that you want to
achieve for sure, then think what you can work on after that, given
the time constraints. Think about the time management, potential
issues you might run into, and what you think you should have by
mid-term and beyond. Cut down the idea of "OVMF on Xen" into
manageable, implementable in 12-weeks chunks. As a personal exercise,
you could plan out what you feel you should have week-by-week. It's
not likely you will stick to that schedule, but it should give you a
realistic idea of what you can expect to finish up by end of GSoC.
Think about what items are within the scope of "OVMF on Xen"
specifically, and not just within the scope of "OVMF" or "UEFI" in
general. For example, legacy boot support (i.e. "CSM - Compatibility
Support Module") in OVMF is a great idea, but it's a fairly
complicated project in itself (I'd rate it "Hard", see
ftp://download.intel.com/technology/framework/docs/csm_0_96.pdf for an
outdated picture of the problem space, although you could do some
intelligent chain-load thing, I suppose from OVMF into SeaBios), and
definitely outside the scope for the "OVMF on Xen" project.
I see the project overall as being fairly open-ended. The basic thing
that you should aim for, is to have OVMF instead of the regular BIOS
boot up inside a Xen HVM domain, thinking it's running on real
hardware - Ide controller, VGA video, ps/2 keyboard, etc. Ideally, it
should work in same fashion as on QEMU, since Xen uses QEMU-derived
emulators... You should be able to interact with the UEFI Shell and
access block devices. After that, it's up to you to figure out where
you want to take this. Here are some ideas, grouped by categories, and
listed in the order of complexity. This is by no means an exhaustive
list, you might have great ideas I haven't listed below...
Category A - Ensure OVMF-on-Xen has feature parity with OVMF-on-Qemu.
1) Verify you can boot into OSes UEFI-way from your OVMF-on-Xen. You
should be able to boot whatever is bootable in OVMF on QEMU.
2) ???
Category B - optimize OVMF for Xen HVM domains.
0) Develop a driver that handles Xen hypervisor support (grant tables,
event channels, hypercalls).
1) XenStore/XenBus is an information storage space, akin to a
registry. It is necessary to get any Xen ParaVirtualized devices
working in UEFI. Depends on (0).
2) Implement a SIMPLE_FILE_SYSTEM_PROTOCOL that exposes the XenStore
namespace as a file system. This might make the next steps easier.
3) Implement virtual frame buffer support instead of using vga text.
Post by Andrei Warkentin
Depends on (1).
Hi Andrei,

I don't understand why we need a virtual frame buffer here. OVMF-on-QEMU has
a vga text console, where we can run some UEFI app, such as a shell. Do you
mean maybe we can use a graphic interface to replace the vga text console
here?

Thank you very much.

Bei Guan
Post by Andrei Warkentin
4) Implement ParaVirtualized block storage. This means that instead of
using the IDE hardware emulator in qemu via the UEFI IDE driver, you
use the the virtual xen device. So instead of touching I/O space,
causing an intercept every access that causing a domain switch so the
access is handled inside the I/O emulators, block i/o requests are
handled using a simple ring-buffer and an event channel. This should
provide a boost on storage access. Depends on (1).
5) Implement virtual network support. Depends on (1).
6) UEFI has a concept of "variable services". The most interesting use
for variables is boot device selection. In OVMF, the variables are
synced with an NvVars file on a filesystem. Investigate what it takes
to create a simple Xen device that can be used to store variables.
Depends on (1).
Category C - Depends on B for PV device support.
1) Make OVMF run inside a PV domain. I.e. - you have no emulated I/O
devices, only the Xen abstractions.
Thanks and good luck with your proposal. I've added some inlined
comments below as well.
I have a question about UEFI variables. So, what are the UEFI variables
and
Jordan Justen
2011-03-29 18:02:10 UTC
Permalink
Post by Bei Guan
Post by Andrei Warkentin
Category B - optimize OVMF for Xen HVM domains.
0) Develop a driver that handles Xen hypervisor support (grant tables,
event channels, hypercalls).
1) XenStore/XenBus is an information storage space, akin to a
registry. It is necessary to get any Xen ParaVirtualized devices
working in UEFI. Depends on (0).
2) Implement a SIMPLE_FILE_SYSTEM_PROTOCOL that exposes the XenStore
namespace as a file system. This might make the next steps easier.
3) Implement virtual frame buffer support instead of using vga text.
Depends on (1).
Hi Andrei,
I don't understand why we need a virtual frame buffer here. OVMF-on-QEMU has
a vga text console, where we can run some UEFI app, such as a shell. Do you
mean maybe we can use a graphic interface to replace the vga text console
here?
The VGA emulation can in some cases be a burden within the VM. For
example, the VGA hardware accelerated drawing operations can cause
trapping for the VM, and therefore produce slower results than using a
framebuffer directly.

Actually, I plan to check in a new video driver for OVMF soon that
uses the VGA hardware for mode setting, but only uses framebuffer
operations for updating the screen. The performance is much better...

Regarding the Xen OVMF project, I think the main goals should be
booting an IA32 and X64 UEFI OS on Xen (with QEMU emulated hardware).

Andrei has provided a lot more interesting ideas beyond this. You can
consider making these additional primary goals or extra goals in your
proposal.

-Jordan
Andrei Warkentin
2011-03-29 19:59:11 UTC
Permalink
Post by Jordan Justen
Regarding the Xen OVMF project, I think the main goals should be
booting an IA32 and X64 UEFI OS on Xen (with QEMU emulated hardware).
Exactly. First make sure OVMF works just as well on Xen HVM as it does
on QEMU. Then you can make the incremental improvements to use less
QEMU emulators, and more Xen virtual devices.

A
Andrei Warkentin
2011-03-29 19:55:40 UTC
Permalink
Hi Bei,

I'll respond inline.
Hi,
Thank you for reminding me. I greatly agree with you on the idea "Category
C". That is to put OVMF in a PV domain. IOEMU stubdom, which is in Xen's
source code, adopts the same idea to put ioemu (alias for xen-QEMU) in a
lightweight domain, and it can boost the HVM performance. So we can take
this architecture into our "Enable Xen Support in OVMF" project.
You might be confusing several things here. Putting OVMF into a PV
domain is an interesting goal, but there was a good reason I listed it
last (beyond needing PV drivers for everything from timer to block
storage). If you are running OVMF in a purely virtual container, you
can only boot an OS that can boot in such an environment - i.e. a
purely virtual OS. This is not so interesting for Linux, which has a
domain builder already, but it could be interesting for ReactOS or
other OSes where loading the kernel is not as simple as plopping it
into memory.

The point I want to make, and I'll make it again below, is that you
should first optimize OVMF for HVM domains. That's where 95% of
performance improvement will be seen, that can apply to any OS you
wish to boot through OVMF UEFI-way. You make the timer and block
storage independent of the I/O emulators and your boot times will
reduce drastically. Everything else you should treat as icing on the
cake as far as GSoC goes.

OVMF fully running in a PV domain is an interesting academic
experiment. One that you could get to if you rewrite all I/O device
support in OVMF to use Xen equivalents, but one that overall just
isn't very useful.

You definitely do not want to run OVMF in a stub domain. Stub domains
are meant to contain Xen service add ons. The point of putting the
emulators into a stub domain is to "contain the damage" in case of a
bug in the emulators, so that other domains are not affected, to
reduce scheduling latencies, improve I/O throughput and reduce dom0
burden.

The point of making OVMF avoid "real I/O" handled by those emulators
is similar - to avoid needless scheduling-out of the VM, reduce
latencies, improve I/O throughput and reduce dom0 burden. Ultimately
it will allow better server scalability.
Firstly, we port the OVMF to IOEMU stubdom.
IOEMU stubdom is based on mini-os, which is a lightweight PV kernel, and
just provides an environment for QEMU to run. When this subtask is finished,
we can boot into OSes by UEFI-way. And then, we need to verify whether OS
booted in OVMF on QEMU can boot in OVMF on Xen.
Why would you want to run OVMF in the stub domain? That implies you're
booting your OS in the stub domain as well. Additionally, when you
finish this gargantuan task (of porting OVMF to run in a PV, not stub,
domain), you will end up with firmware that cannot boot any existing
OS, as none expect to be booted UEFI way without any "real I/O" or CPU
privileged operations present. So I disagree.
Secondly, we optimize the OVMF for Xen HVM domains.
This is your most important goal. You are going to replace
architectural protocols and drivers that use "real I/O" to instead use
hypervisor interfaces and PV back-end drivers.
This task is enlightened from your idea "Catefory B". Thanks to the mini-os,
we would get a perfect Paravirtualization environment if the first task has
been done. Up to now, mini-os has been added many front-end drivers, such as
network, framebuffer, pci, block device and keyboard. So, we can just use or
extend these front-end drivers to provide device information needed by OVMF.
You might be inspired by MiniOS, but you're going to write the
frontends yourself to run in OVMF without depending on anything else.
I have a question here. The idea in your "Category B 4)" is that "Implement
ParaVirtualized block storage". Do you mean that we just directly use the
front-drivers to provide virtual storage devices for OVMF instead of the
storage devices emulated by QEMU for OVMF?
Right. You will implement the frontend drivers that will talk to
backend drivers through XenBus.
I also take other implementation into consideration, such as the virtual
framebuffer, virtual network support, and so on.
These should be the icing on the cake after the burden of the work, which is -
1) Hypervisor support (grant tables, event channels - keep in mind
UEFI has to have only one interrupt)
2) Time enlightment to use Xen facilities instead of emulated PIT/APIC timer.
3) XenBus/XenStore
4) Block storage frontend

Then you can -
5) Framebuffer frontend
6) Network frontend

If you end up finishing with a good 4 weeks left, maybe -
7) Explore the idea of a fully PV firmware, with possible requisite
changes to $FAVORITE_OS to boot somewhat.

Thanks,
A
Bei Guan
2011-03-30 11:37:19 UTC
Permalink
Post by Andrei Warkentin
Hi Bei,
I'll respond inline.
Hi,
Thank you for reminding me. I greatly agree with you on the idea
"Category
C". That is to put OVMF in a PV domain. IOEMU stubdom, which is in Xen's
source code, adopts the same idea to put ioemu (alias for xen-QEMU) in a
lightweight domain, and it can boost the HVM performance. So we can take
this architecture into our "Enable Xen Support in OVMF" project.
You might be confusing several things here. Putting OVMF into a PV
domain is an interesting goal, but there was a good reason I listed it
last (beyond needing PV drivers for everything from timer to block
storage). If you are running OVMF in a purely virtual container, you
can only boot an OS that can boot in such an environment - i.e. a
purely virtual OS. This is not so interesting for Linux, which has a
domain builder already, but it could be interesting for ReactOS or
other OSes where loading the kernel is not as simple as plopping it
into memory.
The point I want to make, and I'll make it again below, is that you
should first optimize OVMF for HVM domains. That's where 95% of
performance improvement will be seen, that can apply to any OS you
wish to boot through OVMF UEFI-way. You make the timer and block
storage independent of the I/O emulators and your boot times will
reduce drastically. Everything else you should treat as icing on the
cake as far as GSoC goes.
Hi Andrei,
Up to now, I think I have understand what you mean. I will talk about it
later.

Firstly, also some question here.

Does the word "any OS" in "that can apply to any OS you wish ..." here just
mean UEFI-compliant OS?
Could you tell me any more about the "timer" here? What's the difference
between the timer got from Xen device and that got from QEMU?
Post by Andrei Warkentin
OVMF fully running in a PV domain is an interesting academic
experiment. One that you could get to if you rewrite all I/O device
support in OVMF to use Xen equivalents, but one that overall just
isn't very useful.
You definitely do not want to run OVMF in a stub domain. Stub domains
are meant to contain Xen service add ons. The point of putting the
emulators into a stub domain is to "contain the damage" in case of a
bug in the emulators, so that other domains are not affected, to
reduce scheduling latencies, improve I/O throughput and reduce dom0
burden.
The point of making OVMF avoid "real I/O" handled by those emulators
is similar - to avoid needless scheduling-out of the VM, reduce
latencies, improve I/O throughput and reduce dom0 burden. Ultimately
it will allow better server scalability.
Firstly, we port the OVMF to IOEMU stubdom.
IOEMU stubdom is based on mini-os, which is a lightweight PV kernel, and
just provides an environment for QEMU to run. When this subtask is
finished,
we can boot into OSes by UEFI-way. And then, we need to verify whether OS
booted in OVMF on QEMU can boot in OVMF on Xen.
Why would you want to run OVMF in the stub domain? That implies you're
booting your OS in the stub domain as well. Additionally, when you
finish this gargantuan task (of porting OVMF to run in a PV, not stub,
domain), you will end up with firmware that cannot boot any existing
OS, as none expect to be booted UEFI way without any "real I/O" or CPU
privileged operations present. So I disagree.
I chose the stub domain to hold the OVMF, because I misunderstand what you
mean. I though the front-end driver of Mini-OS and QEMU, which are all in
stubdom, can meet what you said.

Well, I think I have understand what you mean. You mean we need to write the
our front-end driver in OVMF to communicate with back-end driver in Xen
Dom0. And then, OVMF can use the devices provided by our front-end driver
(actually, by the communication with back-end driver). Because of the higher
efficiency of split driver in Xen than QEMU emulated devices, OVMF can get
better performance. Is it right?

So, I think the main tasks of this project are:
1) Modify the code of Xen to make the Xen-QEMU can call OVMF.(Also known
porting OVMF to Xen.)
2) Write some front-drivers in OVMF to use the Xen devices.(hypercall, event
channel, and others may be need here. )

In the future, we can exploit the fully PV for OVMF. That it to say, OVMF
don't need any emulated devices provided by QEMU any more, but instead of
Xen devices. With this PV OVMF architecture, all the OS supported by OVMF
can boot on Xen. Is it right?

Thanks

Bei Guan
Post by Andrei Warkentin
Secondly, we optimize the OVMF for Xen HVM domains.
This is your most important goal. You are going to replace
architectural protocols and drivers that use "real I/O" to instead use
hypervisor interfaces and PV back-end drivers.
This task is enlightened from your idea "Catefory B". Thanks to the
mini-os,
we would get a perfect Paravirtualization environment if the first task
has
been done. Up to now, mini-os has been added many front-end drivers, such
as
network, framebuffer, pci, block device and keyboard. So, we can just use
or
extend these front-end drivers to provide device information needed by
OVMF.
You might be inspired by MiniOS, but you're going to write the
frontends yourself to run in OVMF without depending on anything else.
I have a question here. The idea in your "Category B 4)" is that
"Implement
ParaVirtualized block storage". Do you mean that we just directly use the
front-drivers to provide virtual storage devices for OVMF instead of the
storage devices emulated by QEMU for OVMF?
Right. You will implement the frontend drivers that will talk to
backend drivers through XenBus.
I also take other implementation into consideration, such as the virtual
framebuffer, virtual network support, and so on.
These should be the icing on the cake after the burden of the work, which is -
1) Hypervisor support (grant tables, event channels - keep in mind
UEFI has to have only one interrupt)
2) Time enlightment to use Xen facilities instead of emulated PIT/APIC timer.
3) XenBus/XenStore
4) Block storage frontend
Then you can -
5) Framebuffer frontend
6) Network frontend
If you end up finishing with a good 4 weeks left, maybe -
7) Explore the idea of a fully PV firmware, with possible requisite
changes to $FAVORITE_OS to boot somewhat.
Thanks,
A
Andrei Warkentin
2011-03-30 19:48:25 UTC
Permalink
Post by Bei Guan
Hi Andrei,
Up to now, I think I have understand what you mean. I will talk about it
later.
Firstly, also some question here.
Does the word "any OS" in "that can apply to any OS you wish ..." here just
mean UEFI-compliant OS?
Could you tell me any more about the "timer" here? What's the difference
between the timer got from Xen device and that got from QEMU?
In the context of booting an OS from UEFI, I mean booting an
UEFI-compliant OS - one whose primary bootloader is an EFI
application. OVMF (AFAIK) doesn't do legacy booting, I don't think you
need to concern yourself with that.
Post by Bei Guan
Well, I think I have understand what you mean. You mean we need to write the
our front-end driver in OVMF to communicate with back-end driver in Xen
Dom0. And then, OVMF can use the devices provided by our front-end driver
(actually, by the communication with back-end driver). Because of the higher
efficiency of split driver in Xen than QEMU emulated devices, OVMF can get
better performance. Is it right?
Yes.
Post by Bei Guan
1) Modify the code of Xen to make the Xen-QEMU can call OVMF.(Also known
porting OVMF to Xen.)
2) Write some front-drivers in OVMF to use the Xen devices.(hypercall, event
channel, and others may be need here. )
I'd be more specific, but yes.

Step 1. OVMF runs in HVM domain. OVMF is not aware it is running
inside a VM, hence all all I/O accesses are redirected by Xen to the
ioemu stubdomain and handled there.
Step 2. OVMF runs in HVM domain. OVMF is now aware it is running
inside a Xen domain. It is able to perform hypercalls. It is able to
use grant tables. It is able to use event channels.
Step 3. OVMF runs in HVM domain. OVMF can now use Xen timer facilities
(set_timer_op) instead of using emulated timer provided by QEMU.
Step 4. OVMF runs in HVM domain. OVMF can now perform transactions
over XenBus/XenStore.
Step 5. OVMF runs in HVM domain. OVMF now has a block front-end
driver, that talks to Dom0 block back-end driver via XenBus+Event
Channel. Block I/O is now done solely through Xen facilities and not
through the IDE Emulator provided by QEMU in IOEMU stub domain,

Step 1 is really important and you should treat it as such. It is
important to have OVMF work under Xen without any PV drivers.
Post by Bei Guan
In the future, we can exploit the fully PV for OVMF. That it to say, OVMF
don't need any emulated devices provided by QEMU any more, but instead of
Xen devices. With this PV OVMF architecture, all the OS supported by OVMF
can boot on Xen. Is it right?
That is right (as long as you intend on booting within HVM domain)

A

Continue reading on narkive:
Loading...