[ Curiosity,Experimentation ]

Random stuff from the parallel universe of Ones and Zeroes

Writing a 16-bit Real mode OS [NASM]

Posted by appusajeev on January 27, 2011


This article is about writing a minimal 16 bit, real modes DOS like operating system that boots off a pen drive and provides a shell to run pure binary executables(aka COM files in the DOS era)  with a custom file system implemented. This means that the OS could run COM files directly if you have one. COM files are pure binary files in the sense that they don’t have a header, contains machine instructions only. For demonstration, I have developed some sample executables which could be run using our OS (apps like clone of unix echo, register dump etc). See the end of the post to see some pictures of the OS in action.

Download Source

The post explains how to

  • Write a bootloader
  • Write a shell, a kernel placeholder
  • Implement a basic filesystem
  • Write the OS to disk
  • Boot from the OS

The OS is written in the open source NASM assembler in Linux. To understand the working, you need to have some understanding about x86 booting process, bootloaders and real mode of processor operation.

Booting Process Basics

When the system is powered on, BIOS pops into action and performs what is known as Power On Self Test(POST) to verify the working of devices, initializing them etc. Immediately after that, POST loads BIOS executable code, present in the BIOS ROM into memory at address which is usually 0xF00000.  POST then places a jump instruction in the first byte of memory (CS:IP = 0). The jump instruction is nothing but a jump to the address 0xF00000 where the BIOS code is loaded.  Now the BIOS code takes control and performs certain operations like setting up the Interrupt Vector Table(IVT), finding a boot device, setting up certain information in RAM(BIOS Data Area), loading the bootloader  etc.

BIOS provides certain basic interrupts for the programmer for basic functions like loading and storing disk sectors, reading keyboard, printing to screen etc. These interrupts are similar to DOS interrupts but are not DOS interrupts.

Back to topic,  once these processes are over, the BIOS iterates through the list of boot devices and according the boot order preferences, the bios searches for a bootloader in each boot device. If a bootloader is found, its loaded into memory and is given control.

A point to note is that whenever system is powered on, whatever be the processor, Core 2 Duo or Core i7 or whatever operates in 16 bit real mode by default until it is explicitly asked to switch to 32  bit protected mode (by setting PE bit in CR0 register and doing a far jump to fix CS to point to a segment descriptor after setting up GDT).

Bootloader Basics

Bootloader is basically a 512 byte piece of machine code that is present in the first sector a boot device.  Bootloader is the first user defined program that’s loaded into memory and given control of. It is the duty of the bootloader to load the OS into memory and pass control to it.

A bootloader must be exactly 512 bytes in size. BIOS identifies  a valid bootloader by means of a signature. The 511th byte of the bootloader should contain the value 0x55 and 512th byte should have the value 0xAA.

A Bootloader will always be loaded at address 0x7C00 in RAM. Usually, this corresponds to CS:IP pair of 00:0x7C00 but some BIOSes set CS:IP as 0x7C0:0 which is essentially points to the same address  but leads to issues in writing bootloader when we have to specify the offset where our code will be loaded.
This can be easily dealt by defining an offset 0 and then jumping to 0x7C0:start , where start is the label of the next instruction following  the far jump. This jump sets CS = 0x7C0

(Note: Physical address = CS x 16 + IP)

Our Bootloader

Our bootloader, present in the first sector of the pendrive  (the mechanism for writing the bootloader and the OS into the pendrive will be discussed later) will be loaded into memory and execution will immediately start.

Our bootloader serves 3 purposes, it displays a welcome message and then loads the OS and file table from the disk and jumps to OS entry point. The os is loaded at address 0x1000 and file table at address 0x2000

Heres our bootloader source

Download full Source

Our Bootloader

Our Bootloader

Bios provides interrupts for displaying characters as well as strings on screen. Here I have used character display interrupt to write a function to print a zero-terminated string like in C.

The interrupt for displaying character on screen is

INT 0x10, BX = character color, AL = character to display, AH = 0x0E

Our bootloader, OS and executables are stored on  disk . For execution, they need to be loaded to RAM. The bootloader will be loaded by BIOS and the rest we have to load when required through the sector loading interrupt.A block of 512 bytes (in this case, need not always be) is called a sector.
For loading sectors off the disk to RAM, bios provides an interrupt, INT 0x13 and the parameters to set are

AH = 2

DL = drive,  DL = 0x80 for hard disk and this applies to our pen drive

DH = head number

CH = track number

CL= sector number of the sector to be loaded

AL = number of sectors to load

ES = segment to load the sector

BX = offset from ES where the sector is to be loaded

Our  bootloader occupies the first sector, file table in the third sector and OS in the third sector.

The bootloader loads the os into address 0x1000:0 and filetable into address 0x2000

Implementing Filesystem

The shell provided by our os enables the user  to type an executable name and run it. For that , the os must know where exactly each executable is located on disk and this is where the concept of file system surfaces. A file system, in simple words is basically a specification that tells how to locate of a file on disk given its name.

For our purpose, I made a simple filesystem. Each file is mapped to a sector in disk to form a string with following structure

{file1-sector,file2-sector,…,filen-sector,}

This filetable is loaded to 0x2000:0 by the bootloader this address space is scanned to find the sector where the requested file is stored on disk and that sector is brought to RAM.

Our OS/Shell

So, the bootloader has loaded our OS at 0x1000:0 and filetable at 0x2000:0 and it makes a far jump to 0x1000:0, the OS entry point.  The working of the shell is simple, using BIOS interrupt to reach character, we read the executable name from the user. The filetable loaded at 0x2000:0 is scanned for a match, the associated sector number is read and that sector is loaded into memory at 0x9000:0 and the shell makes a far jump to this address. If the name entered by the user cannot be matched in the file table, an error message is shown.  After the executable completes execution, for it to return back to the OS, it must make a far jump to 0x1000:0, the OS entry point. This step is functionally similar to executing (AH= 4ch, INT 0x21 in DOS)

The interrupt for reading a character from a keyboard is INT 0x16 with AH=0, the read character will be available in AL.

Heres our OS source

Download full Source

Our Shell

Our Shell

Writing the OS to disk

Okay so everything is done, the final thing to do to boot from the pen drive, is to write our os into it. It can’t just be done by copying the files to the pen drive. It doesn’t work in our case for two reasons:

1.   We cannot write the bootloader to the first sector using this method. When we ask the OS to copy a file, it copies the file to some free sectors and add this information to the file table.

2. If were to copy the files to the pendrive, our filesystem has to be known to the host OS, like FAT  which means we have to write code to parse that file system during boot time which, well is an overkill for this os

So, to write the bootloader and custom file system, we need to have low level disk access, for which the obvious choice is Linux.  Linux treats devices like files which can be read and written to. This is a really powerful and useful concept. Commands like ‘dd’ use this concept. The file representing our harddisk would be something like sda, something  like sdc for pen drive, it varies. To know the file allocated, after the pen drive is attached, run dmesg|tail in the terminal.

Now whatever we write to the device file will be written as such into the device which is exactly what we want. To write the bootloader, write the compiled boodloader into the first 512 bytes of the file and this would be written to first 512 bytes of the disk and this can be done using C file operations. Pretty easy, isn’t it. Now to write the file table and OS  to the 2nd and 3rd sectors of the disk, just write these files to the next two 512 bytes of the device file . The same procedure applies to writing the executables to disk

The following C code does the job of writing everything to disk. It takes the path of device file, bootloader, os and list of executable as command line arguments.

Copy Program

Copy Program

Booting the OS

Plug in the  pen drive, find its corresponding device file (use dmesg|tail), open the makefile, substitute its path in dev variable. Then run ‘make’ , reboot the system, choose to boot from mass storage and you could see the sweet sight of our OS booting . At the prompt,  type ‘help’ and hit enter, you could see list of available executables, like reg to dump registers, echo that echoes back a string read from the user

Here are some pics of the OS in action(Click to enlarge).

OS in action

OS in action

Our OS in action

Our OS in action

Advertisements

42 Responses to “Writing a 16-bit Real mode OS [NASM]”

  1. Arun said

    Awesome stuff, dude!
    I’m tempted to try it out.

    • appusajeev said

      You sure can, all thats needed is a pen drive and a linux system 🙂

      • Sana said

        Can you help me to build os in Tasm ? in windows

      • appusajeev said

        Sorry I have not used TASM. As for the code, you dont need to change the semantics. Only syntax changes, if any, might be needed. Also, I am not sure how to write the compiled OS to disk from Windows. I have used Linux and I think its the easier way.

  2. jishnu said

    great work….this will be of great help.

  3. vidya said

    :* great job.. 😀

  4. aju krishnan said

    awesome job man 🙂

  5. Pramode said

    Good work!

    You can try to do something like this:

    http://linuxgazette.net/82/raghu.html

    Also, it will be good to implement some form of pre-emptive multithreading!

  6. Indrajeet said

    Nice post ….

  7. M Farid Abdullah said

    I downloaded the source file and planned to write (but not using the c supplied file) the required files to a pendrive. I know nothing about Linux. I will be using windows to execute all your instruction. I have a number of question as below.

    1) What is the name of the file related to filetable to be put in to sector 1. I do not think the source asm for this is included in the download.
    2) Is the compiled copy of OS.asm to be put to sector 2?
    3) What extention to use as output for all the files during the nasm compilation.

    • appusajeev said

      Please refer my reply to your most recent question.

      file table is generated by the program cpy.c. Hope that is clear now if you have read my reply..

      the compiled os.asm file is to be placed in the 3rd sector of the disk. This is done by the program ‘cpy.c’. The extension of the compiled file is immaterial. Its perfectly fine not to have any extension. As you know, file extension is mere mechanism to know the type of file at a quick glance.

      the synatax of NASM is nasm -f bin filename.asm

  8. Henry said

    how to run it directly in a virtual machine like virtual box?

    • appusajeev said

      Easy ! Install ubuntu in virtual box, install nasm in it using ‘apt-get’…. For emulating our OS for testing, use qemu-kvm , ‘apt-get install qemu-kvm’

  9. Farid said

    A very good tutorial. A very good source of reference. For me the idea of having a simple custom file system will greatly help.

    I have a few queries:

    1) Is this sentence in your article a mistake, (because both file table and OS occupy the 3rd sector): “Our bootloader occupies the first sector, file table in the third sector and OS in the third sector.”

    2) But I am more interested in the sentence following the above: “The bootloader loads the os into address 0×1000:0 and filetable into address 0×2000”. I am not able to identify the range of source code for the filetable (the custom file system).

    Do you supply a separate source code file of the Filetable/custom file system in the downloaded files you offered or do you incorporate it (filetable source) in the OS source code? Can you please identify that code to us if it is incorporated in the OS source code.

    3) Once you identify the filetable code, I will try all this in my Windows system since I know nothing about Linux. Any advice (or expected problems) in trying these in Windows?

    Thank you for sharing your ideas.

    • appusajeev said

      Hello Farid

      1. Sorry that was a mistake. Bootloader occupies the first sector, filetable occupies the second sector and OS occupies the third sector. Thanks for pointing it out.

      2. If you observe, the ‘cpy.c’ when compiled and run generates the file table (a file called filetable is generated which is eventually written to the disk) and writes the file table into the second sector of the disk. ‘cpy’ takes the list of files to be written to the disk as commandline arguments, and generates the filetable using this list as the writing is done to different sectors. Once all files have been allocated different sectors, filetable generation completes. This table(the file ‘filetable’) is then written onto the disk in the second sector. Hope you understood. See the source of ‘cpy.c’

      In the bootloader, i have defined 2 macros – ‘loc‘ pointing to the location where OS is to be loaded(0x1000:0) , and ‘ftable‘ pointing to segment where file table is to be loaded(0x2000:0) from the second sector in the disk.

      3. Hope you understood till now. The reason why i chose Linux is because it provides raw access to disks, you can access individual sectors by accessing the corresponding device files. I explained it in the post. Hope you noticed it.

      On windows, if you want raw disk access, to write individual sectors, use the tool ‘Partcopy‘.
      http://www.osdever.net/downloads/other/pcopy02.zip

      I have not tried the above tool but i think it should do the trick once you have the compiled files. Install NASM first

  10. Henry said

    i will write it and test it with kemu, good work man. by the way: The links are broken

    • appusajeev said

      I hope you meant qemu, and yes, its a very good idea to try out things. I will fix the broken links ASAP.

  11. Devendra Vyas said

    The link is broken to download os.zip

  12. Chris said

    I am on windows. I assembled bootloader.asm using nasm and then used MagicISO to put bootloader.bin onto a virtual CD to run it in virtualbox. When I load it into VBox, it starts up fine, but it then won’t do anything after printing press any key to continue. Does cpy.c do more than put the files onto a drive?

    • appusajeev said

      Actually bootloader.bin is supposed to go to the first 512 bytes of the disk. How you managed to do that using MagicISO?
      Anyway I suppose you have installed it in the bootsector . If you open bootloader.asm, I have defined a macro ‘drive‘ with value 0x80, this value is used(this value is stored in DL register) by bios interrupt to to load sectors from hard disk(the bootloader does that). That value is to be set as 0 if you are booting from a floppy. In your case, you are attempting to boot from a CD. For that to work, you may have to use some other value for ‘drive’. I am not sure which value to use 😦 . (Try setting the value to 0, it may work)
      If you install Linux, things will be a whole lot easier.

      Regarding cpy.c, it writes the bootloader and other files to the disk specified in the makefile. Other files are written to the succeeding sectors. Doing this corrupts the earlier file system in the drive. Be careful about that

  13. jeans outlet…

    […]Writing a 16-bit Real mode OS [NASM] « [ Curiosity,Expermentation ][…]…

  14. Jose Papel said

    I can’t write it to the pendrive, I have Ubuntu 11.04 and the device file is /dev/sdb1 and as I execute the cpy executable it just compiles in the folder but writes nothing in the pendrive. Do I need to have some libraries in order to compile the cpy.c file?

  15. Kuladeep said

    Good work boy.. Keep it up..

  16. I’m a 14 year old programmer, and am really interested in these kind of things. Tell me, could you build off of this? Like turn it into a 16-bit Mac? Also, where are some good resources for assembly programming? I can’t seem to find books or any good tutorials for it.

  17. Noah said

    I compiled and ran the cpy.exe with:
    cpy.exe J:\ boot.bin os.bin reg.bin vidmem.bin about.bin echo.bin help.bin exit.bin

    I get the error: Assertion failed: dev>0, file cpy.c, line 24

    I am using Windows Vista.

    Email : bloddyrunes@live.com

    • appusajeev said

      This code was written to be compiled on a Linux environment. It is not tested on Windows. Kindly use Linux. I guess the problem here is ‘J:’, you are not supposed to be providing that, you are supposed to be providing the path to the device file. If you can find a way to get the device file for a device, you might succeed.

      • Noah said

        Would it be possible to generate the filetable with a something else then just use partcopy? I’ve never worked with filetables so i’m not really sure how that works. I’d use linux, if I could run it on my windows me, maby via a virtual machine?

  18. Noah said

    I installed ubuntu: i used, dmesg|tail But what am i supposed to accomplish with this?

  19. Noah said

    I figured it out thanks a lot for the tutorial!

  20. Marcos said

    The link is broken to download os.zip

  21. jay said

    I am building a new OS in windows.I have the boot.bin file and converted it to boot.iso . While am loading the iso file in virtual box it is returning an error “No Bootable medium found”.Please anyone help me??? 🙂

    • appusajeev said

      how did u convert it to boot.iso? i think the bootloader has not been installed properly in the iso file

      • jay said

        I had used magic.iso software and also tried to write the boot.bin to usb drive using ur cpy.c file.Bt while i am loading d os after changing d boot order,am not able to view d prompt…..

  22. Cătă said

    You said the bootloader is loaded at 0x7c0. How is it that it can run the first jmp instruction ?
    What type of file do you actually get loaded onto the pendrive ? Obj or what 😀 ?

    • Cătă said

      The thing is I don’t understand why that first jmp instruction exists.

      • appusajeev said

        A Bootloader will always be loaded at address 0x7C00 in RAM. Usually, this corresponds to CS:IP pair of 00:0x7C00 but some BIOSes set CS:IP as 0x7C0:0 which is essentially points to the same address but leads to issues in writing bootloader when we have to specify the offset where our code will be loaded.
        This can be easily dealt by defining an offset 0 and then jumping to 0x7C0:start , where start is the label of the next instruction following the far jump. This jump sets CS = 0x7C0

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: