Writing a BIOS bootloader for 64-bit mode from scratch

And then you're free from dealing with the somewhat convoluted processor init stuff, but instead depend on the Windows PE format, FAT filesystem, and an overcomplicated API.

Seems like a bad tradeoff, and part of a slippery slope towards a completely locked down system, where writing your own code and getting it to run on the 'bare metal' is flat out impossible.

This is fine if you're only running on a single core, however if you're a multiprocessor OS you still need to deal with legacy when bringing up the other cores. Intel and AMD should consider a mode that disables that and brings up the other cores using a 64bit SIPI. While I applaud Intel on the X86S idea... I think there is room for bits of that without throwing out all the backwards compat. An X86SC which drops real mode and only supports 16bit 'real mode' in virtualization.

Yes, I see the argument that if you go to that point you might as well just use emulation. However running mixed 32bit/16bit code (Windows 98/95) becomes problematic just because of performance reasons. DosBox does well, but good luck supporting games for the Pentium 3 era that still used 16bit libraries because of course they did. (16bit Installshield was so common going into 64bit that MS just has code to replace it so 32bit applications can still be installed despite having a 16bit installer)

Backwards compatibility is good and necessary. But I don't think backwards compatibility going all the way back to the 8086 is. If someone has software written for the 8086 at this point, they would be far better served by running it in dosbox or something than on bare metal.

This killed FreeDOS (and presumably all the other *DOS as well) on modern hardware unfortunately. It was fun as long as it lasted. I do not know what the next-best single-user, single-process, non-bloated OS would be to run on modern hardware that still has some reasonably modern software and can be used for distraction-free (hobby) development the way FreeDOS could.

I've been thinking about how to structure a filesystem such that code complexity - in the boot loader and elsewhere - can be minimized. You'd want something similar to NTFS, except that it should be possible to refer to a file (MFT entry) directly by starting sector number. So they would either need to always remain at a fixed position, or have a link pointing back to any reference so it can be updated.

Somewhere in the first sector (aligned to a 64 bit boundary), there would be a small structure that just contains a unique signature (not ASCII but some random value; 64 bits seems like more than enough as opposed to a GUID), and a pointer to the "FS info block". Leaving all remaining space in the sector available for boot code or possibly another "overlayed" filesystem.

That info block in turn would point to the MFT entry for the stage 2 boot code. An MFT entry would contain at a minimum an easy to locate list of (start sector,count) extents. Maybe there could be a flag that tells the OS that a certain file should never be fragmented?

File names would be entirely optional and for human consumption; for direct links within a filesystem, sector numbers would be used, and some kind of unique numeric identifier for anything "higher-level".

I'm genuinely wondering if some expert here sees any problem with this scheme, other than it not conforming to how mainstream OSes do things?

Because just dealing with file fragmentation should bump you way over 512 bytes I would imagine.

Historically, many filesystems have had a special file type or attribute for contiguous files - a file guaranteed to not be fragmented on disk. Most commonly used for boot loaders, OS kernels, or other essential system files - although historically some people used them for database files for a performance boost (which is likely rather marginal with contemporary systems, but decades ago could be much more significant).

Some systems required certain system files to be contiguous without having any special file metadata to mark them as such - for example, MS-DOS required IO.SYS and MSDOS.SYS to be contiguous on disk, but didn’t have any file attribute to mark them as contiguous. Unlike an operating system with proper support for contiguous files, DOS won’t do anything to stop you fragmenting IO.SYS or MSDOS.SYS, it is just the system will fail to start if you do. (Some might interpret the System attribute as implying Contiguous, but officially speaking it doesn’t.)

you can just stick the kernel at sector 2 and read it from there using extended bios disk read. specify your DAP to read the kernels amount of sectors starting from sector 2, load it to something like 0x7e00 or some reachable place. It will have still limits on how much it can read, per read and in total.

If you do this all within 1 sector, equally you do not do any error checking. just ram stuff into memory and yolojump into it.

the basic would be: load kernel from disk using bios interrupt get memory map using bios interrupt parse kernel ELF header / program headers and relocate it - elf header to find ph_off and ph_num and entry_point - program headers to find all pt_load and rep movb them into tgt phys addr.

Also with 510 bytes generally u will not make nice 'page tables' though this is actually possible with only few bytes. - i did not manage it yet in 510 :D but i am sure there's someone who can do it... it can be done really efficiently.

the disk would be formed by assemblding the mbr. then doing something like

cat mbr.bin kernel.bin > disk.bin (maybe here use truncate to padd the disk.bin to a certain size like 10+MB or so - helps some controllers recognize it better)

all that said it's not useful to do this. you will find it an interesting excersize at best. like trying to make 'tiny ELF' file or so. fun to learn, useless in practice.

yes, the kernel was in a known location on disk (directly after the bootsector).

the whole boot disk image was generated during build, which is common for small systems.

ACHI (which is how SATA is exposed) doesn't change any of this. The only thing that is affected is how the loaded OS has to talk to the disk controller. The real thing that loses space is the BPB (a 60 bytes or so, iirc) because some BIOSes are broken and require it and the MBR (but that's only a couple bytes). At least [bootelf] manages to fit in (without a BPB or an MBR) with 128 bytes to spare, enough for a dummy BPB that makes all BIOSes happy.

Additionally, UEFI's reliability is.. sketchy, as far as I know (using the classic logic of "if Windows doesn't use it does it really matter?"). And GNU-EFI suffers from build portability troubles, AFAIK.

[bootelf]: https://github.com/n00byEdge/bootelf

To be fair, writing a bootloader is an interesting and educational project in its own right. But yes, for most people interested in osdev they should just use an existing bootloader. It gets you to the part that interests most people (writing the kernel) faster, and without having to worry if you are going to run into gnarly bugs because the bootloader is buggy and you never realized.

I don't see how riscv can be anything but worse than Arm in this regard. With Arm at least Arm Holdings has some nominal power to steer towards sanity (devicetrees, systemready etc), with riscv it's again full freedom for vendors to make their own bespoke crap.

The flag register layout is another case of extreme backwards compatibility - its lower bits have the same definitions they had on the 8-bit 8080, even the same fixed values:

Sign : Zero : always '0' : AuxCarry : always '0' : Parity : always '1' : Carry

(the parity flag came all the way from the 8008 / Datapoint 2200[1], and is the inverted XOR of the result's lower 8 bits; aux carry is the carry out of bit 3, used for BCD arithmetic)

Flag bit 15 has also stayed reserved, except at one time it was used by the NEC Vxx chips for their 8080 compatibility mode. That feature had to be first unlocked by executing a special instruction, because there is code out there that loads the entire (16 bit) flag register with 0000 or FFFF. With the mode bit unlocked, that would inadvertently switch the CPU to running a completely different set of opcodes!

[1] https://www.righto.com/2023/08/datapoint-to-8086.html

Particularly AMD made the 64 bit extension without any real input from Intel and didn't want to use any bits that would later conflict with a bit Intel might use in CR0. So a brand new register was in order.

I thought many UEFI implementations support legacy bios mode as well. Or well they used to.

From how I view linker scripts:

U can use a linker script to create file layout. If you want a flat binary file... u dont want a file layout. So linkerscript is really useless for a flat binary blob. Even if you make it with multiple files. It'd be just the same as saying: cat blob1 blob2 blob2 > finalyblob.

If you'd say have multiple blobs, and use linker directives to align them, the position dependent code within the assembled files will be wrong, unless you specifically define that using for example ORG directive in NASM. If you use the ORG directive in NASM, u will need to keep that synchronized with the linker script in order for all the labels etc. to keep the right offsets calculated.

So essentially.. this linker script might even add complexity and issues when working with multiple binary blobs. u can't align them or use nice linker features....

If you use more structured files, which allow for example for relocation... then ur already using ELF or PE and can simple produce those files. They can be more masterfuly linked with nice features, and linker scripts are then essential.

You can add your binary blob in the right location in the output file of a linking run, using a linker script. This is useful to add data into your files but for an MBR it's not particularly useful. People do it, but it adds no benefit over just sticking it on the front if your disk using 'dd' for example. ------

That being my views, I am wondering what you see the benefit here? Are there some linker features i am unaware of that are particularly useful here? (I really know only alignment stuff in there... and include blobs or put stuff into /discard/ and some basic define sections/segments etc.). I am not familiar with perhaps more advanced linking features.