[syslinux] PATH directive

[Matt Fleming]

On Sat, 20 Dec, at 12:48:18AM, Ady wrote:
>  
> So, my first (still incomplete and still inaccurate) attempt to write 
> some rules about the PATH directive, and ask about remaining doubts!...
> 
> 
> The search (for c32 files) is supposed to respect the following rules:
> 
> 1_ The search for c32 files is performed according to the following 
> rules. The search for each required c32 file is stopped once the first 
> matching file name is found; remnant potential paths are not searched 
> (even if the execution of the firstly-found c32 file fails in some way; 
> e.g. unmatching version, unmatching firmware architecture).
 
Correct.

> 2_ Every required c32 file starts a new search (for itself); i.e. once 
> a c32 file is found, if additional c32 (dependency) files are also 
> required then each c32 file triggers its own new search, according to 
> all these rules.
 
Correct.

> 3_ When the c32 file is explicitly referenced by an absolute path, the 
> explicitly-referenced c32 file is search-for in such location first.
 
Correct.

This question forced me to go stare at the code ;-) Specifically
findpath() and open_file().

Every time we attempt to open a .c32 module we call fopen() and the call
stack ultimately looks like this,

  fopen()
    open()
      open_file()
        searchdir()

So this is where the "search for absolute path first" semantics come
from. This part is the same for every file that gets opened, not just
.c32 files.

I've noticed a bug in the PATH code because lookups of an absolute path
happen relative to a directory in PATH.

That sentence sounds pretty confusing, so an example might help explain
this bug...

If you try to open "/absolute/path/to/my/file.c32" and that file doesn't
exist, the PATH lookup code will append that path to every entry in the
PATH list.

Assuming your PATH contains "/boot:/foobar", the PATH lookup code will
try and open the following files,

  /boot/absolute/path/to/my/file.c32
  /foobar/absolute/path/to/my/file.c32

Which is totally bogus, and contrary to what "absolute path" means.

I've created the following bug report to track this issue,

  http://bugzilla.syslinux.org/show_bug.cgi?id=58

> 4_ When the c32 file is explicitly referenced by a relative path, the 
> explicitly-referenced c32 file is search-for in such location first, 
> where the base for the relative path is the Current Working Directory. 
> (Note: this is not the same as searching in the CWD itself.)
 
Yes, this is the same as for the regular file lookup (see answer to 3.)

> 5_ If the CONFIG directive was used so to change the Working Directory, 
> then the _prior_ Working Directory is no longer relevant; only the 
> *Current* Working Directory is considered as base for relative paths 
> where the c32 files are searched-for. The (new) CWD takes precedence 
> over any PATH directive.
 
Wow, good question. Yes, the new current working directory should take
precedence.

> 6_ After the relevant aforementioned paths, the paths stated in all the 
> parsed PATH directives are searched-for, one-by-one until the file is 
> found, in the same order as they are stated in the parsed configuration 
> files. Note that all paths referenced in PATH directives are parsed as 
> *absolute* paths, even when they are not explicitly written with a 
> starting slash symbol ("/").
 
I'm not sure the absolute part is true. It should make sense to require
that, but from reading the code, I think any directories not prefixed
with "/" are treated as being relative to the current working directory.

Do you have some data to suggest otherwise?

> 7_ If the CONFIG directive was used, the paths in prior PATH directives 
> are still valid, and they take precedence over new PATH directives. In 
> other words, newly-parsed PATH directives are appended after 
> previously-parsed PATH directives (the new ones do not replace prior 
> ones). Once a configuration file was parsed:
> 7.1_ there is no method to "reset" the PATH directive.
> 7.2_ there is no method to eliminate a path already stated in the PATH 
> directive.
> 7.3_ there is no method to alternate the order in which the paths 
> stated in PATH directives are searched-for.
> 7.4_ all aforementioned "PATH rules" (starting from rule #1) are 
> re-evaluated, according to the new CWD.
 
Correct.
 
> 8_ These "PATH rules" apply for almost every-and-any c32 file, except 
> for "ldlinux.c32". Other types of files (such as configuration files, 
> kernels, initrd, memdisk,...) are only searched-for according to the 
> paths being used in their respective lines in the Syslinux 
> configuration file, or according to the paths used in the boot command 
> prompt; i.e. if non-c32 files are not found in the firstly-expected 
> location, no additional searching is performed for such files.
 
I think this is also correct.
 
> And now, the current remaining doubts!!!:
> 
> D1_ Matt Fleming said:
> > Yes, the PATH directive is used as a fallback if the file denoted by 
> an absolute path doesn't exist.
> 
> D1Q: When using an *absolute* path and the c32 file is not found in 
> such location, is the CWD expected to be searched as first fallback for 
> the *explicitly-referenced* c32 file, before searching according to the 
> PATH directive(s)? Note that I am not asking about the c32 dependency 
> files, but about the explicitly-referenced one(s) (e.g. not about 
> lib*.c32 files but about "/some/absolute/path/menu.c32").
 
When using an absolute path, the current working directory is not
searched.
 
> D2_ For this doubt I will use an example:
>  UI some/relative/path/menu.c32
> 
> Whether typed in CLI or parsing a configuration file, note that this is 
> a relative path *based* on the CWD, but not the CWD itself.
> 
> The "PATH rules" as documented in the official Syslinux 6.03 
> distribution archives state that the CWD is "always" searched-for 
> first, before the PATH directive(s). This statement (and the example 
> included in the official Syslinux v.6.03 documentation) clarifies the 
> expected behavior for paths written in absolute notation. But...
> 
> 
> D2Q1: In the case I am presenting here (c32 module explicitly 
> referenced in relative notation but not referenced to be located in the 
> CWD itself), if menu.c32 is not found first in the referenced 
> directory:
>  <CWD>some/relative/path/ 
> then, is "menu.c32" supposed to be searched in the CWD *itself* too 
> (before the PATH directive paths)? Note that I am not asking here about 
> the non-referenced c32 files such as lib*.c32.
 
I think I understand this question, and the answer is "no". The file
lookup code will not strip "some/relative/path/" when performing the
lookup, which I think is the question you're asking.

> D2Q2: In the case I am presenting here (c32 module explicitly 
> referenced in relative notation but not referenced to be located in the 
> CWD itself), in which locations are the c32 *dependencies* of 
> "menu.c32" searched-for and in which order (before searching according 
> to the PATH directive)? Are its *dependencies* searched-for in 
> "<CWD>some/relative/path/"? Are its *dependencies* searched-for in the 
> CWD (too)?
 
Loading of dependencies follows the same rules as loading any .c32 file,
there's nothing special about the actual lookup process for
dependencies.

However, it's worth pointing out that dependencies are specified as
simple filenames without any path component in the ELF sections for .c32
files. You can see this for yourself using readelf(1) or some other ELF
header parsing tool.

Here's the output for chain.c32,

Dynamic section at offset 0x63b8 contains 16 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libutil.c32]
 0x0000000000000001 (NEEDED)             Shared library: [libcom32.c32]

See how there's no path component for libutil.c32 and libcom32.c32? This
is the reason that dependencies are first searched for in the CWD.

But the file lookup code never distinguishes between .c32 and their
dependencies.

In your example, dependencies will be searched for in CWD, but not
"<CWD>some/relative/path", because lookup is performed using the CWD of
the process doing the loading.

> D2Q3: Could the search for each of the c32 dependencies be different in 
> each case, according to, or depending on, where the *prior* c32 file 
> was found (e.g. hdt.c32, or libutil.c32, before searching for 
> libcom32.c32, or libmenu.c32, or..., all so eventually hdt.c32 will 
> load and work)? Or, instead, is each search fully independent of the 
> result of the prior (library) c32 file that have been just 
> searched-for?
 
Each search should be independent.
 
> D3Q1: I know that in the CLI there is no equivalent to the PATH 
> directive, but the PATH *rules* are still valid in the command prompt, 
> right? Are still _all_ the rules valid? Are there any differences?
 
No, there shouldn't be any differences.

> D3Q2: Are there any *different* "PATH rules" for the boot prompt, 
> whether using (typing-in) relative paths or absolute paths notation?

Nope.
 
> D4_ When typing in the boot prompt, we know that the ".c32" file name 
> extension is used as fallback. For example, (IIRC) if I type in the 
> boot prompt:
> 
>  ls
> 
> then Syslinux will try to find such file name (or label), "as-is" 
> ("ls"). If such exact file name (or label) is not found in the expected 
> location, then Syslinux tries to find "ls.c32" in the same location.
> 
> 
> D4Q1: Are any "PATH rules" applied to such "fallback" file name, 
> "ls(.c32)"?
 
Yes, the usual PATH rules. There are no different PATH rules.

> D4Q2: Are any "PATH rules" applied to the dependencies of such 
> "fallback" file name, "ls(.c32)"?
 
Yes, the usual PATH rules.

> Matt, my goal is to improve documentation and to be able to test
> whether Syslinux actually behaves as expected regarding to the PATH
> rules. I would appreciate if these doubts could be clarified (and if I
> made/make a mistake or inaccuracy, please correct me), so the PATH
> rules' documentation can be improved.

Thanks, I appreciate that you want to provide better documentation.

-- 
Matt Fleming, Intel Open Source Technology Center