Sunday, May 07, 2006

Linux's boot process explained

Short history of the UNIX operating system

Linux is an implementation of the UNIX operating system concept. UNIX was derived from AT&T's "Sys V" (System 5). The initialization process is meant to control the starting and ending of services and/or daemons in a system, and permits different start-up configurations on different execution levels ("run levels").
Some Linux distribution, like SlackWare, use the BSD init system, developed at the University of California, Berkeley.
Sys V uses a much more complex set of command files and directives to determine which services are available at different levels of execution, than the BSD's do.

Booting the Linux operating system

The first thing a computer does on start-up is a primer test (POST - Power On Self Test). This way several devices are tested, including the processor, memory, graphics card and the keyboard. Here is tested the boot medium (hard disk, floppy unit, CD-ROMs). After POST, the loader from a ROM loads the boot sector, which in turn loads the operating system from the active partition.
The boot blocks is always at the same place: track 0, cylinder 0, head 0 of the device from which we're booting. This block contains a program called loader, which in Linux's case is LiLo (Linux Loader), or Grub (GNU Grub Unified Boot Loader), which actually boots the operating system. These loaders in Linux , in case of a multi-boot configuration (more operating systems on a computer), permit the selection of the operating system to be booted. Lilo and Grub are installed or at the MBR (Master Boot Record), or at the first sector of the active partition.
In the following we will refer to LiLO as boot loader. This is usually installed in the boot sector, also known as MBR. If the user decides to boot Linux, LiLo will try to load the kernel. Now I will present step-by-step LiLo's attempt to load the operating system.

1. In case of a multi-boot config, LiLo permits the user two choose an operating system from the menu. The LiLo settings are stored at /etc/lilo.conf. System administrators use this file for a very detailed finement of the loader. Here can be manually set what operating systems are installed, as well as the method for loading any of them. If on the computer there is only Linux, LiLo can be set to load directly the kernel, and skip the selection menu.

2. The Linux kernel is compressed, and contains a small bit, which will decompress it. Immediately after the first step begins the decompression and the loading of the kernel.

3. If the kernel detects that your graphics card supports more complex text modes, Linux allows the usage of them - this can be specified or during the recompilation of the kernel, or right inside Lilo, or other program, like rdev.

4. The kernel verifies hardware configuration (floppy drive, hard disk, network adapters, etc) and configures the drivers for the system. During this operation, several informative messages are shown to the user.

5. The kernel tries to mount the file system and the system files. The location of system files is configurable during recompilation, or with other programs - LiLo and rdev. The file system type is automatically detected. The most used file systems on Linux are ext2 and ext3. If the mount fails, a so-called kernel panic will occur, and the system will "freeze".
System files are usually mounted in read-only mode, to permit a verification of them during the mount. This verification isn't indicated if the files were mounted in read-write mode.

6. After these steps, the kernel will start init, which will become process number 1, and will start the rest of the system.

The init process

It's Linux's first process, and parent of all the other processes. This process is the first running process on any Linux/UNIX system, and is started directly by the kernel. It is what loads the rest of the system, and always has a PID of 1.

The initialization files in /etc/inittab

First time the initialization process (init) examines the file /etc/inittab to determine what processes have to be launched after. This file provides init information on runlevels, and on what process should be launched on each runlevel.
After that, init looks up the first line with a sysinit (system initialization) action and executes the specified command file, in this case /etc/rc.d/rc.sysinit. After the execution of the scripts in /etc/rc.d/rc.sysinit, init starts to launch the processes associated with the initial runlevel.
The next few lines in /etc/inittab are specific to the different execution (run-) levels. Every line runs as a single script (/etc/rc.d/rc), which has a number from 1 to 6 as argument to specify the runlevel.
The most used action in /etc/inittab is wait, which means init executes the command file for a specified runlevel, and then waits until that level is terminated.

The files in /etc/rc.d/rc.sysinit

The commands defined in /etc/inittab are executed only once, by the init process, every time when the operating system boots. Usually these scripts are running as a succession of commands, and usually realise the following:

1. Determine whether the system takes part of a network, depending on the content of /etc/sysconfig/network

2. Mount /proc, the file system used in Linux to determine the state of the diverse processes.

3. Set the system time in fuction to the BIOS settings, as well as realises other settings (setting of time zone, etc), stabilized and configured during the installation of the system.

4. Enables virtual memory, activating and mounting the swap partition, specified in /etc/fstab (File System Table)

5. Sets the host name for the network and system wide authentication, like NIS (Network Information Service), NIS+ (an improved version of NIS), and so on.

6. Verifies the root fily system, and if no problems, mounts it.

7. Verifies the other file systems specified in /etc/fstab.

8. Identifies, if case of, special routines used by the operating system to recognize installed hardware to configure Plug'n'Play devices, and to activate other prime devices, like the sound card, for example.

9. Verifies the state of special disk devices, like RAID (Redundant Array of Inexpensive Disks)

10. Mounts all the specified file systems in /etc/fstab.

11. Executes other system-specific tasks.

The /etc/rc.d/init.d directory

The directory /etc/rc.d/init.d contains all the commands which start or stop services which are associated with all the execution levels.
All the files in /etc/rc.d/init.d have a short name which describes the services to which they're associated. For example, /etc/rc.d/init.d/amd starts and stops the auto mount daemon, which mounts the NFS host and devices anytime when needed.

The login process

After the init process executes all the commands, files and scripts, the last few processes are the /sbin/mingetty ones, which shows the banner and log-in message of the distribution you have installed. The system is loaded and prepared so the user could log in.

Linux's execution levels

The execution levels represent the mode in which the computer operates. They are defined by a set of available services at any time they are started. The execution levels represent different ways Linux uses to be available to you, the user, or eventually the administrator.
As daily user you don't have to bother with the execution levels, although the multi-user level makes the services which you need while using Linux in a network (though in a transparent mode) available.
In the next few sentences I'll present the execution levels, one by one:

0: Halt (stops all running processes and executes shutdown)

1: Known under the name "Single-user mode". In this case the system runs with a reduced set of services and daemons. The root file system is mounted read-only. This runlevel is used when the others fail while booting.

2: On this level run the most of the services, with the exception of network services (httpd, named, nfs, etc). This execution level is ideal for the debug of network services, keeping the file system shared.

3: Complete multi-user mode, with network support enabled.

4: Unused, in most of the distributions. In Slackware this level is equivalent with 3, the only difference is that this has graphic login enabled.

5: Complete multi-user mode, with network and graphic subsystem support enabled.

6: Reboot. Stops all running processes and reboots the system to the initial execution level.

Modification of execution levels

The most used facility of init, and maybe the most confusing one, is the ability to move from an execution level to an other.
The system boots into a runlevel specified in /etc/inittab, or to a level specified at the LiLo prompt. To change the execution level, use the command init. For example, to change the execution level to 3, type
init 3

This stops most of the processes and takes the system into a multi-user mode with networking enabled. Attention, changing the init level might force several daemons used at the moment to stop!

The directories of execution levels

Every execution level has a directory with a symbolic links (symlinks) pointing to the corresponding scripts in /etc/rc.d/init.d. These directories are:

/etc/rc.d/rc0.d
/etc/rc.d/rc1.d
/etc/rc.d/rc2.d
/etc/rc.d/rc3.d
/etc/rc.d/rc4.d
/etc/rc.d/rc5.d
/etc/rc.d/rc6.d

The name of the symlinks are semnificative. It specifies which service has to be stopped, started and when. The links starting with an "S" are programmed to start in various execution levels. The links also have a number in their name (01-99). Now some examples of symlinks in the directory /etc/rc.d/rc2.d:

K20nfs -> ../init.d/nfs
K50inet -> ../init.d/inet
S60lpd -> ../init.d/lpd
S80sendmail -> ../init.d/sendmail

When operating systems change the execution level, init compares the list of the terminated processes (links which start with "K") from the directory of the current execution level with the list of processes which have to be started (starting with "S"), found in the destination directory.

Example:

When the system boots into runlevel 3, will execute all the corresponding links starting with "S", in an order accorind to their number:

/etc/rc.d/rc3.d/S60lpd start
/etc/rc.d/rc3.d/S80sendmail start
(and so on)

If the system now changes to runlevel 1, will execute:

/etc/rc.d/rc3.d/K20nfs stop
/etc/rc.d/rc3.d/K50inet stop
(presuming that nfs and inet are NOT in /etc/rc.d/rc1.d)

After that it will start all the processes mentioned in /etc/rc.d/rc1.d except which are already running. In this example there's a single one only:

/etc/rc.d/rc1.d/S00single

Changing the current execution level

To change the current execution level for example to level 3, edit /etc/inittab in a text editor, and edit the following line:

id:3:initdefault:

(do not change the initial runlevel to 0 or 6!)

Booting into an alternative execution level

At the LiLo prompt you have to write the number of the wanted execution level, before booting the operating system. This way to boot into the third level, type for example:

linux 3


Eliminating a service from an execution level

To disable a service from a runlevel, you might simply delete or modify the corresponding symlink.
For example, to disable pcmcia, and don't start in the future, type:

rm /etc/rc.d/rc3.d/S45pcmcia


Adding a service to an execution level

To add a service, it is needed to create a symlink pointing to the corresponding scripts in /etc/rc.d/init.d. After the symlink is created, be sure to assign it a number, so it would be started in the right time:

To add "lpd" to runlevel 3, type:

ln -s /etc/rc.d/init.d/lpd /etc/rc.d/rc3.d/S64lpd




P.S.: This article has been originally written by Ovidiu T. He gave me the permission to translate and publish it, and since I found the article interesting, I did that way as well, although I don't treat myself responsible about the content of it.

Saturday, May 06, 2006

Sebastian Trueg @ K3B interview


I saw the success of my MPlayer review, and I looked for another open-source project, which developers I should interview. The first name came to my mind was "K3B". I dropped the maintainer a mail, and today evening we were ready. So who's the guy on the picture on the left?

Sebastian Trueg - K3B project founder, maintainer, and main programmer


Me: Please introduce yourself!

Trueg: I live in Freiburg, Germany since I moved here from northern germany for my studies of computer science which I finished this year. :)


Me: I bet you were among the first students.

Trueg: Not really. I never attended that much lessons but I am quite happy with my final score. And it could not have been that bad becasue my prof offered me a job and the opportunity to make my PhD. so at the moment I am employed at the University of Freiburg.


Me: When did you start the K3B project? Why?

Trueg: K3b, well here comes the main reason for me not being among the best students... I started working on K3b in 1998, shortly after I arrived in Freiburg, and A LOT OF TIME was spent on K3b, at times way more than on my studies. The reason for K3b... Well, when I came to Freiburg I had never tried a linux system and didn't know anything about it. A friend gave me, I think it was SuSE 5.something. I gave it a try, but at the time I was not too interested. Some time later I tried again, became curious to see if I could perform all the tasks I use a computer for on a linux system. To be honest: it were not that many tasks. ;) CD-burning was one of them, especially MP3->Audio CD. And that's when it started. There was simply no program around which satified my needs. You either had to use cdrecord on the command line or do the exact same thing in xcdroast (which was a real front-end, nothing more). I wanted to burn mp3 files to an audio cd without converting them beforehand, so I thought: "Well, try it yourself!". I had never programmed something this big but I was eager to learn so I started and it went quite well (mainly becasue of the good QT/KDE documentation). At some point (I don't think the first version of K3b could actually burn mp3 files on-the-fly...) I heard of sourceforge and put it online. The response was incredible, I had never thought that this many people would use my program... the rest is history :)


Me: How much time do you invest in developing K3B?

Trueg: Hard to say. There were times when I worked on it every day for many hours, but there was also a year in which I did nothing at all (that's the reason for the 500+ open bugs on bugs.kde.org :) At the moment I do not have that much time for K3b since I work full time at the university, but I hope that I will find a way to tend to K3b more often in the future.


Me: What is your favourite feature in K3B?

Trueg: Well, my favorite feature. There is, of course, the audio decoding which is nicely integrated using the plugin structure. Since it was the main reason I started K3b it is one of my favorite features since it works so well (IMHO). But then there is one thing in K3b which you don't fin in any other burning app (at least I know of none): the "auto" modes which I try to implement for as many things as possible. There is the auto-writing mode selection, the auto-multisession-mode selection, the auto-data-mode selection (for data cds). I think that's a very cool feature becasue it allows K3b to provide the parameters for advanced users on the one hand while being simple for beginners on the other hand. Oh, yeah, CD-Copy for multisession CDs. I don't think there are that many applications that do that. :)


Me: Speaking of other applications, have you ever tried any other CD-burning software except the mentioned XCdRoast and cdrecord?

Trueg: I knew Nero from Windows. It was my reference which probably shows... I tried KCDsomething, Arson, CDBakeoven, and GnoeToaster, but all of them did not get very far. At least Arson and CDBakeoven seem to be dead.


Me: Do you fear NeroLinux? I mean, do you treat it as a powerful competitor, or trust in the open source community which tends to use mainly open source software (and Nero isn't open-source).

Trueg: I do not trust in all linux users to use open-source. They will use what works best. I do fear NeroLinux a little. For the time being they do not put very much effort in NeroLinux, but that might change and if that day comes I might have a problem... But for now NeroLinux is far from being as powerful as K3b, (which I only know from feature descriptions since it never finds any devices on my system..)


Me: You had a fundraiser.. What success have you had?

Trueg: Incredible! Awesome! I set out to raise 1000 euros and was not sure at all if I would get that far... But the community prooved me wrong: within two days! the goal of 1000 euros was reached and by the end of the fund-raiser I received nearly 5000 euros. It was such a great feeling to be appriciated in such a way. The reson I started the fund-raiser was a new computer system. With that kind of money I was able to get a really cool one: a silent PC which does not make a sound (except if you put your ear to it you may hear the harddisk spinning). Most of the money came from private users, but I received two big donations from Mandriva and Linspire. I have to say that I had never dared to dream about such a success.


Me: At the announcement of the fundraiser, you promised K3B 1.0 as a "reward". Are you planning to release it soon, or it needs some more work?

Trueg: I am planning on releasing a preview soon. K3b 1.0 is almost feature-complete but there are still some quirks to work out. I am planning on releasing a preview soon. K3b 1.0 is almost feature-complete but there are still some quirks to work out. Data Project verification has been an issue for many years and I did not get it right yet. And I'd like to close most of the bugs on bugs.kde.org before releasing k3b 1.0 final. But it's not that far away. :) K3b 1.0, that is. I hope to release the first preview by the end of May, but who knows...


Me: There were some gossips about porting KDElibs to Windows. Will you release a Windows version if that will need some extra coding to do (or will you anyways release a Windows version)?

Trueg: I never had any plans for a Windows version. If anyone is interested in porting K3b, be my guest. I don't know if it would be that hard, probably there is only one file which needs to be changed (as was done for the FreeBSD port) and that is k3bscsicommand.cpp. Well, of course one had to have the kdelibs installed on the Windows system...


Me: What would you like to transmit to the OSS community, in particular to my readers?

Trueg: Well, especially after the fund-raiser I think the OSS community is an incredible bunch. ;) I find it astonishing how this many different people come together and create something with (mostly) only one reward: compliments. The free distribution of knowledge and the impact that this distribution has on the world (even the commercial one) is just incredible. I really hope that I can be a part of it for a long time to come and there is one thing I missed up to now: I've never met any of "the big guys" in person. I've never attended any of the KDE meetings and such. I really have to change that. As for your readers: support Open-Source. Install latest versions and help out where ever you can. :)


Me: Thank you for the interview, for your time spent with me. I wish you success in life and with your project!

Trueg: Thanks a lot. It was a pleasure!

Thursday, May 04, 2006

Gentoo GNU/Linux - Speed, Stability, Reliability.


Introduction
I guess most of those, who are familiar with linux already heard of the fierce beast, which seeks for its prey in the Linux jungle, the Gentoo GNU/Linux distribution. Despite of what most of the people think, Gentoo is not about building everything from source, optimization flags, and text-mode installation.
What really defines Gentoo is the philosophy behind it, the idea of letting the user make the choice. Yepp, that's great isn't it? But what does it mean? Basically, it means that one can customize his/her system as she/he wishes -- Using Gentoo, noone will ever force you to share someone else's concept (and this includes the developer too) of how a Linux system should look like, that is you are free to decide which and how many programs you will use, what features should they contain, what architectures should they support, etc.

Installation
As far as I know, the newest(2006.0) release also contains a program, which provides a graphical interface for installing Gentoo. It is non-sense to discuss the gui installer, so i'll carry on with giving you a view-in to the nemesis of all ignorants, the text-mode installation.
It is far from being so complicated as it would look like for the first time. It's kinda like compiling a kernel - it sure _is_ hard for the first time, and you'll probably mess something up - but once you have done it, you'll find it easier than pushing the shiny little reboot button on your PC.
The installation can be divided to 10 easy steps:
1. Choosing the installation medium
2. Configuring the network devices (optional)
3. Preparing the disks
4. Extracting the stage archive and configuring it
5. Installing the base system
6. Configuring the kernel
7. Configuring the system
8. Installing system tools
9. Configuring the bootloader
10. Adding some users
Well... that would be it. After you've done it, you can't even imagine an installation without the command line. But why is the CLI installation better than the GUI one anyway? In the first place it gives you a clear overview of the installation process, it lets you make some important choices about how your system will work in the future.

Usage
One of the greatest features of Gentoo is Portage, the package managment system it provides, about which I, intentionally, haven't spoken yet.
Portage is a very powerfull and versatile system for adding, removing and updating software packages(at the moment of writing it features more than 10000 packages). It was originaly designed with ports, the BSD package managment system, in mind. Gentoo's Portage system works similarly to Debian's APT. Portage is written in the Python programming language.
The main idea behind Portage is to compile every package from source, with the ability of choosing between wanted and unwanted features(USE flags), and of predefining compiler flags for the package manager to use. Though the system itself is known as Portage, operations are actually done with a command line program called "emerge". A GUI front-end is also available, known as "Kuroo".
The init system is another important feature of Gentoo. It is similar to the System V init system that most distributions use, however the Gentoo one uses dependency based scripts and named runlevels.

Conclusion
If you are the type of person, who prefers somebody else making choices suitable for you, rather than making your own, or simply you don't have enough experience in the *NIX world to decide which programs, which security model, what layout fits you the most, you generally shouldn't bother with using Gentoo. In the rest, I strongly recommend it for anyone, who is looking for a stable, reliable, fast and highly configurable operating system.