[syslinux] EFI & PXE-booting: very slow TFTP performance on a VMWare test setup

[Oscar Roozen]

>> Are there any known problems with the performance of TFTP in (U)EFI
>> environments in general or maybe just on VMWare?
> I don't believe so.  I use Workstation 10.0.2 on Ubuntu 12.04 for the
> moment.

<stupid question> and have EFI enabled? </stupid>

> I hope this is a hex-core host with 6+GiB of RAM.

Fedora runs on an AMD A8-6600K at 3Ghz and has 8G of RAM. Ubuntu runs
on an Intel i7-2600K at 3.4Ghz with 16G RAM. I think they perform at
least "good enough" to quikly boot and install in BIOS mode. The
trouble only starts when I toggle the EFI switch.


> Have you considered single vCPU client VMs?

Yes. That's what I started with. But then I saw this piece of code in
efi/udp.c and thought it would be wise to enable an extra CPU to
receive the UDP packet. A single CPU has a 100% load here:

    start = jiffies();
    while (cb_status == -1) {
        /* 15ms receive timeout... */
        if (jiffies() - start >= 30) {
            if (jiffies() - start >= 60)
                printf("Failed to cancel UDP\n");

            uefi_call_wrapper(udp->Cancel, 2, udp, &token);
            printf("core_udp_recv: timed out\n");
        }
        uefi_call_wrapper(udp->Poll, 1, udp);
    }

Just to rule out some stupid missed interrupt or so, I gave it an extra CPU.


> I presume you disabled VMware's isc-dhcp on vmnet1?

Yes. I should have mentioned that. Found out the hard way.. . ;-)


> Good idea to serve from a subtree.

Thanks. We need to support both modes, so we now have /tftpboot/bios
and ../efi64.


>> # make spotless
>> # make bios
>> # make efi64
>
> 'make bios' should be unnecessary.

I have to test both modes, but while debugging I mostly did just "make
efi64 && install-stuff-to-/tftpboot/efi64".

>> The server does just fine. It typically reacts within 0.000100 seconds.
>> The client also starts out fine, reacting within about 0.000250 seconds,
>> but this delay slowly increases.
>
> This makes me wonder the source and there's at least 3 factors: VMware UEFI
> on multiple vCPU, memory or other leak, a strange bug in Syslinux.

This also happens on a single vCPU.

I did check efi/udp.c over and over again (especially the event
handling) but I don't see a place that could cause this behaviour.

One thing I did just now is a bit - ehm - hackish. I symlinked
/tftpboot/efi64/syslinux.efi to /dev/zero and tried booting from that.
I wanted to see if the "built-in" tftp-client of VMWare's UEFI
firmware also showed this behaviour, but it didn't. Well, it got a bit
slower, but not as much as the client from syslinux.


> I recall finding the UEFI 2.4 spec really for free.

I recall running into some NDA that had to be manually approved before
I could download anything. I also remember preferring to go on hacking
in more debug statements to find out where the delay comes from.
Unfortunately I still have no clue and no specs as a result. Even with
the specs, my guess is it will take some time understanding them. I'm
just becoming a bit more comfortable reading syslinux code.

> By the way, what version/commit?

A fresh syslinux-6.03.tar.gz from kernel.org.
I also tried 81ad566f155fac31089fde69c87059b217e7e9b6 with the same results.

-- 

Oscar Roozen
Linux Developer
Bright Computing BV