[syslinux] Having problem with menu.c32 on serial console

[John Rouillard]

On Wed, Dec 17, 2008 at 03:56:17PM -0800, H. Peter Anvin wrote:
> John Rouillard wrote:
> > When I try to use it with the serial settings all I see is the one
> > character that draws the top left corner of the menu against a blue
> > background and that's it. Nothing I type seems to have any effect. ESC
> > doesn't do anything either.
> 
> That is extremely odd.  I'm wondering if rxvt doesn't implement some of 
> the mode setting escape sequences that the menu system uses, or does so 
> incorrectly.
> 
> The fact that you're getting the proper background color indicates that 
> *something* is working right.
> 
> I would try to seriously lower the baud rate (to, say, 2400) just to 
> rule out signal integrity problems or buffering issues.

I used:

 serial 0 2400

and

 serial 0 9600

with no change. It still hangs displaying a white background and
little blue box with the top left hand box character in it.

The log file from teraterm pro reads:
====

PXELINUX 3.72 2008-09-25  Copyright (C) 1994-2008 H. Peter Anvin

Boot kernels:
Centos 4 i386:
res38400, res57600 - serial rescue mode at 38400(57600) baud
(default-38400)
resVGA   - vga rescue mode
vnc38400, vnc57600 - serial/VNC install mode at 38400(57600) baud
VGA      - VGA (local console) install mode/generic install mode.
           Add your own arguments. VGA console=48200n8 acpi=off

Centos 5 i386:
c5-res38400, c5-vnc38400, c5-VGA

Centos 4 x86-64:
xres38400, xresVGA, xvnc38400, xVGA and
xks38400 - kickstart server64.cfg config

Centos 5 x86-64:
x5res38400, x5vnc38400 and
x5ks38400 - kickstart, serial 38400 baud, server64_cent5.cfg
x5ks115200 - kickstart, serial 115200 baud, server64_cent5.cfg
x5g38400 - generic centos5/serial 38400 boot - add your own args
x5vmware - Vmware virtual host, console install

Other:
memvga,mem38400  - run memtest86 vga or serial console baud 38400 
                   (can also be used for reboot)
reboot  - reboot the system
flashWD - boot centos 4.4 to flash Western Digital firmware
dos     - boot to freedos
disk    - boot from hard disk
. Type any key for manual boot. <F1> or <Ctrl-F> 1 for help 
boot: help
Could not find kernel image: help
boot: menu32
Could not find kernel image: menu32
boot: menu
l
====

I am a little surpised 'menu' works to pull up the menu as I don't
have a 'label menu' entry, but it works fine on vga and does what I
expect (but not desire) on a serial port.

If I use "default menu.c32" and "prompt 0" with no display or say, the
log shows:

===========
PXELINUX 3.72 2008-09-25  Copyright (C) 1994-2008 H. Peter Anvin
l
===========

So other ideas? I am wondering if logging in teraterm is incapable of
logging binary traffic. I would expect to see at least a couple of
escape sequences in the logging. I am looking at the log file in emacs
which would show the binary characters if they are there.

I have a trace using sysinternal portmon if that is useful to anybody
(and it does show some escape sequences). It's pretty long, so I am
not going to spam the mailing list, I can send via personal email but
the serial port settings section are:

0.07563336	IRP_MJ_CREATE	Serial2	SUCCESS	Options: Open 	
0.00001062	IOCTL_SERIAL_GET_COMMSTATUS	Serial2	SUCCESS		
0.00000391	IOCTL_SERIAL_SET_QUEUE_SIZE	Serial2	SUCCESS	InSize: 8192 OutSize: 2048	
0.00000223	IOCTL_SERIAL_PURGE	Serial2	SUCCESS	Purge: TXABORT RXABORT TXCLEAR RXCLEAR	
0.00000140	IOCTL_SERIAL_SET_TIMEOUTS	Serial2	SUCCESS	RI:-1 RM:0 RC:0 WM:0 WC:500	
0.00000112	IOCTL_SERIAL_GET_BAUD_RATE	Serial2	SUCCESS		
0.00000140	IOCTL_SERIAL_GET_LINE_CONTROL	Serial2	SUCCESS		
0.00000140	IOCTL_SERIAL_GET_CHARS	Serial2	SUCCESS		
0.00000112	IOCTL_SERIAL_GET_HANDFLOW	Serial2	SUCCESS		
0.00000112	IOCTL_SERIAL_SET_BAUD_RATE	Serial2	SUCCESS	Rate: 9600	
0.00286349	IOCTL_SERIAL_SET_RTS	Serial2	SUCCESS		
0.00298921	IOCTL_SERIAL_SET_DTR	Serial2	SUCCESS		
0.00000168	IOCTL_SERIAL_SET_LINE_CONTROL	Serial2	SUCCESS	StopBits: 1 Parity: NONE WordLength: 8	
0.00000168	IOCTL_SERIAL_SET_CHAR	Serial2	SUCCESS	EOF:0 ERR:0 BRK:0 EVT:0 XON:0 XOFF:0	
0.00000168	IOCTL_SERIAL_SET_HANDFLOW	Serial2	SUCCESS	Shake:1 Replace:40 XonLimit:0 XoffLimit:0	
0.00000223	IOCTL_SERIAL_SET_WAIT_MASK	Serial2	SUCCESS	Mask: 	
0.00000140	IOCTL_SERIAL_SET_WAIT_MASK	Serial2	SUCCESS	Mask: RXCHAR 	
23.2489405	IOCTL_SERIAL_WAIT_ON_MASK	Serial2	SUCCESS		
0.00000503	IOCTL_SERIAL_GET_COMMSTATUS	Serial2	SUCCESS		


-- 
				-- rouilj

John Rouillard
System Administrator
Renesys Corporation
603-244-9084 (cell)
603-643-9300 x 111