Tuesday, January 17, 2006








YoLinux Tutorial: System Enhancements, Optimization and Compiling the Kernel













This tutorial covers optimization techniques to make your system run faster or
support your needs better.
This may include tweeking configuration specifications, turning off un-needed
processes, and compiling a new kernel tweeked for your system.
Note that graphics device drivers are NOT compiled into the kernel athough
support for interfacing with their particular chipset can be compiled in.
This is because X-Windows is an application running on Linux and not
integral to the kernel.













Optimize hard drive access performance:


Hdparm:



The following optimizations techniques can be applied to (E)IDE drives to
utilize 32 bit I/O and DMA more effectively.



  • Get info: hdparm /dev/hda

  • Get more info: hdparm -I /dev/hda
  • Test Speed: hdparm -tT /dev/hda
  • Increase Speed: hdparm -d1 -c3 -u1 /dev/hda

    • -d1: Turn on DMA (0: off)
    • -c3: IDE chipset support

      • 0: Disable
      • 1: Enable
      • 3: 32 bit transfer


    • -u1: Get/set interrupt umask. Driver ignores other interrupts during processing of disk interrupts. (0: off)

  • Test Speed Again: hdparm -tT /dev/hda

The Red Hat 7.1+/8.0 configuration uses /etc/rc.sysinit to call scripts
/etc/sysconfig/harddisks. (Earlier systems used: /etc/sysconfig/harddiskhda, /etc/sysconfig/harddiskhdb, etc)
to run hdparm upon boot.


File: /etc/sysconfig/harddisks:


USE_DMA=1
EIDE_32BIT=1

Turn off hdparm DMA access: hdparm -d0 /dev/hda


Links:





Ramdisk:



Go even faster by using ramdisk - Memory mapped as a disk.
See: How to use a Ramdisk for Linux


Also see: /usr/src/linux-2.4/Documentation/ramdisk.txt


Tools:







Kernel enhancements:



Kernel enhancements to optimize SCSI for scanner access:



  • See man page for sane: man sane-scsi
  • Excerp from man page:

    First, make sure your kernel has SCSI generic support
    enabled. In ``make xconfig'', this shows up under ``SCSI
    support->SCSI generic support''.

    To keep scanning times to a minimum, it is strongly recom�
    mended to use a large buffer size for the generic SCSI
    driver. By default, Linux uses a buffer of size 32KB.
    This works, but for many cheaper scanners this causes
    scanning to be slower by about a factor of four than when
    using a size of 127KB. Linux defines the size of this
    buffer by macro SG_BIG_BUFF in header file
    /usr/include/scsi/sg.h. Unless a system is seriously
    short on memory, it is recommended to increase this value
    to the maximum legal value of 128*1024-512=130560 bytes.
    After changing this value, it is necessary to recompile
    both the kernel (or the SCSI generic module) and the SCSI
    backends.

    AUTHOR
    David Mosberger 13 May 1998





Kernel enhancements to increase the per process limit of open files:




Change the definition of NR_OPEN in local files:










Compiling the Kernel:


The default Red Hat Kernel comes with as much stuff compiled into the Kernel as
they could fit.
This allows it to support a vast array of devices and system needs.
If you must add support for something new or if you wish to streamline your
kernel with only what is needed so that it uses less memory, operates faster,
and is optimized for your processor, then one must recompile the kernel.


To find out about the latest kernel, issue the command: finger @finger.kernel.org


Typically the source code for the Red Hat installation is located in:


  • /usr/src/redhat/SOURCES/ - patches and kernel will be placed here by kernel source RPM. (Fedora Core 3)

    (bzip2 -d linux-2.6.9.tar.bz2, tar xf linux-2.6.9.tar)

  • /usr/src/linux/ - (Soft linked to /usr/src/linux-2.2.xx) Kernel 2.2 and older. (Red Hat 6.x, 7.0)
  • /usr/src/linux-2.4/ - (Soft linked to /usr/src/linux-2.4.xx) Kernel 2.4. (Red Hat 7.1)

This separation to a new directory for kernel 2.4 is to accomodate the
differences between the C library and the kernel.


If you get new kernal source download to a new directory.
i.e. /usr/src/linux-2.2.16 then ln -s /usr/src/linux-2.2.16 /usr/src/linux



Typically the kernel RPM (kernel-source-XXX.i386.rpm) will install the source
and generate the appropriate links.


Perform the following steps: (as root)


  1. Setup:



    • The source may already be installed as part of the installation or
      from an RPM (See /usr/src/).

      OR

      You may also download the latest source from
      http://www.kernel.org/pub/linux/kernel/.

      Move the downloaded file linux-2.X.XX.tar.gz

      to /usr/src/.

      Uncompress file: tar -xzf linux-2.X.XX.tar.gz which will
      install everything in a subdirectory linux/. (To view contents use command tar -tzf linux-...tar.gz)

      If using the file with the bz2 extention: bunzip2 linux-2.X.XX.tar.bz2
    • cd /usr/src/linux/ (or /usr/src/linux-2.4/)

    • less README (Read the documentation. It tells you everything. Use space bar to view next page)


      (Local links to: /usr/src/linux/README or
      /usr/src/linux-2.4/README)

    • Update /usr/include/


      NOTE: PERFORM THIS STEP FOR KERNEL 2.2.x AND OLDER. DO NOT DO THIS FOR KERNEL 2.4.



      The header files for Kernel 2.4 and the C librariy header files may no longer match and thus must remain separated.

      • cd /usr/include
      • rm -fr asm linux scsi
      • ln -s /usr/src/linux/include/asm asm
      • ln -s /usr/src/linux/include/linux linux
      • ln -s /usr/src/linux/include/scsi scsi


      The soft links (ln -s) to the
      include directories are important to software developers who need to compile
      with the new include files.


      Note: The sym link is optional and some don't do it due to potential
      differences in the header files used by glibc and the kernel.
      (A response from Linus)


    • cd /usr/src/linux (or /usr/src/linux-2.4)
    • make distclean (Cleans up junk from old compiles) (Warning: removes .config)

    • make mrproper (build the source tree)


    At this time the sources are correctly installed. This also resets the
    /usr/src/linux/.config file to the system default.


  2. Configuration:



    Use one of the following tools to create the .config file.
    This gives you the chance to choose what goes into the kernel. You can
    choose support for many of the latest capabilities, device drivers,
    and can tune the kernel for particular uses. i.e. you can also
    optimize the compilation for the Pentium. (Default for RH 5.2 was a
    386 with no math co-processor!)
    The default config file used to compile the default distribution kernel can
    be found in the source directory. (i.e. Fedora Core 3: /usr/src/redhat/SOURCES/kernel-2.6.9-i686.config, /usr/src/redhat/SOURCES/kernel-2.6.9-x86_64-smp.config, ... and all the others for the various hardware platforms.)


    Pick one of the following:


    • make config (Bash shell script)
    • make menuconfig (uses text window curses)
    • make xconfig (uses tck and wish) - recommended due to online help feature and intuitive interface. Save configuration to file: .config
    • make oldconfig Build a configuration file based on defaults found in current .config file.



    menuconfig Notes: "< > options can be compiled as a module.


    • <*> = Compile into kernel (y)
    • <M> = Compile as a module (m)


    Recommendation for Pentium owners: General Setup choose :



    • Kernel math emulations: N
    • Processor type: Pentium
    • Development, experimental or kernel debugging features: N
    • If using netfilter capabilities, do not enable the kernel fast-switching
      option. That code will permits fast switching at a lower level in the IP
      stack than netfilter and thus will bypass the netfilter capabilities.
    • Note: When requiring gobs of memory (> 1Gb), there is a kernel compile
      option which sets the limit on the amount of user memory one can use:
      "Processor type and features / Maximum Virtual Memory".


  3. Compile:


    Next: (from /usr/src/linux)

    • make dep (make dependencies)

    • make clean (At this point you have all the source gathered)
    • Choose one of the following to create the new kernel and then move it to /boot/vmlinux:

      • make zImage (compressed image)
      • make bzImage (very compressed image good for recovery floppies - slower to boot because of time to uncompress)
      • make image
      • make bzdisk (Boot floppy with no LILO)




  4. Install:



    • mv /usr/src/linux/arch/i386/boot/zImage /boot/vmlinuz-release


      ln -s /boot/vmlinuz-release /boot/vmlinuz


      (If "make zImage" was chosen)

      or

      mv /usr/src/linux/arch/i386/boot/bzImage /boot/vmlinuz-release


      ln -s /boot/bzImage-release /boot/vmlinuz


      (If "make bzImage" was chosen)

      Also move/copy the file /usr/src/linux.../System.map to /boot/System.map-2.X.XX

      OR

      make install
      (Preferred. Potential Pitfall: On my SCSI system I had to execute this command after issuing the mkinitrd command below.)

    • Modules documentation: (not that helpful)

    • make modules
    • rm /lib/modules/Kernel_release (i.e. /lib/modules/2.2.16-3 )
    • ln -s /lib/modules/new_modules_directory /lib/modules/Kernel_release

    • make modules_install

      Generates the file: /lib/modules/2.X.XX/modules.dep


  5. Configure lilo:


    Lilo must point to the new kernel. Edit /etc/lilo.conf.
    Add a new image statement to point to the new kernel.
    Keep the old as backup in case you need to boot it.


    SCSI systems must perform an additional step to load drivers upon system boot:


    • Execute the following command which creates the file used to let the system boot:


      mkinitrd /boot/initrd-2.2.16-3.img 2.2.16-3

      The second argument is the name of the sub-directory of the modules under the directory /lib/modules/. (In this case /lib/modules/2.2.16-3/


      This step is required by systems using modular SCSI support.
      If the SCSI support is compiled into the kernel and is NOT a module, then this
      step may not be necessary.
      The file /boot/initrd-2.2.16-3.img is specified as the RAM disk image
      used before the file system is available. (initrd = initial ram disk)




    Run /sbin/lilo -v to configure the master boot record with data from lilo.conf.


    Sample lilo.conf:






    boot=/dev/sda
    map=/boot/map - Locations on hard drive where the kernel can be found for boot
    install=/boot/boot.b
    prompt
    timeout=50
    linear - Specific to SCSI configurations
    default=linux

    image=/boot/vmlinuz-2.2.16-3 - New kernel
    label=linux - New name to be displayed by Lilo boot manager

    initrd=/boot/initrd-2.2.16-3.img - This is specific to systems using SCSI hard drives. This line is omitted for IDE.
    read-only
    root=/dev/sdb6

    image=/boot/vmlinuz-2.2.14-5.0
    label=OldLinux
    initrd=/boot/initrd-2.2.14-5.0.img - SCSI specific.
    read-only
    root=/dev/sdb6




    When the system boots and gives you the "Lilo:" prompt, you can type "?"
    to see your boot choices as defined by the above labels in lilo.conf.
    Type the label name of the kernel or system you wish to boot. A cairrage return
    will choose the default.


    For a full list of boot parameters see: bootparam man page


    Also see the Boot Prompt Howto.



Kernel version verification: uname -rv will print the compile date of the kernel currently running.






Modules:


Modules are used to reduce the amount of memory used to hold the kernel.
There is a slight penalty for the time taken to load and unload the module.


If the code is required for general operation of the kernel or is needed often
or required by the boot process, it is best to compile it into the kernel
and it should NOT be compiled as a module.


Module commands:


The kernel will use the modprobe utility to determine if the module is
compatable with the kernel.


The program used is specified by a proc file:


cat /proc/sys/kernel/modprobe
/sbin/modprobe - Result of cat operation



Modules are loaded by init scripts which call insmod/rmmod to load/unload
modules.

























CommandDescription
lsmodList loaded modules






[ /root]# lsmod
Module Size Used by
3c59x 19716 0 (autoclean) (unused)
autofs 9120 1 (autoclean)
emu10k1 45688 1
soundcore 2596 4 [emu10k1]
aic7xxx 137400 3

insmodInserts a module into the active kernel
rmmodRemove a loaded module. Just specify the module name. No ".o" or path necessary.
modprobeHigh level handling of loadable modules.


Loads module and dependancies.
depmodCreates dependencies file for a module (used by modprobe)
modinfoDisplay information about a kernel module



List of modules held in /etc/modules.conf (or for older systems: /etc/conf.modules)



Sample:




alias scsi_hostadapter aic7xxx
alias eth0 eepro100
alias eth1 eepro100
alias parport_lowlevel parport_pc
alias sound-slot-0 emu10k1
alias char-major-81 bttv



Other usefull commands:


  • Display system status gathered from /proc: /usr/bin/procinfo -a
  • List Processor type: cat /proc/cpuinfo

  • List devices:

    • cat /proc/devices
    • /sbin/lsmod

  • List IO ports (devivce address used drivers): cat /proc/ioports
  • List DMA channels: cat /proc/dma
  • View interrupts used by the system:


    • cat /proc/interrupts
    • procinfo -i
























    IRQDescription
    0Timer channel 0
    1Keyboard
    2Cascade for controller 2 (which controls IRQ 8-15)
    3Serial Port 2 (COM2)
    4Serial Port 1 (COM1)
    5Parallel Port 2 (LPT2)
    6Floppy Diskette Controller
    7Parallel Port 1 (Printer LPT1)
    8Real-time Clock
    9Redirected to IRQ2
    10--
    11--
    12PS/2 Mouse
    13Math Coprocessor
    14Hard Disk Controller (IDE)
    15--




    Also see file:

    • /etc/isapnp.gone for a list of protected IRQ's
      which will not be allocated to ISA Plug and Play device.
    • Command: pnpdump


      This command scans ISA PnP cards and can be used to generate
      /etc/isapnp.conf

      pnpdump > /etc/isapnp.conf
    • Command: isapnp -C /etc/isapnp.conf

      Uses file /etc/isapnp.gone and /proc files
      to avoid IRQ's already allocated and to determine safe IRQ's.
    • Home page for "isapnptools"


      Plug and Play ISA Specification

  • Create devices. Run script: /dev/MAKEDEV
  • Display kernel version in current use: /proc/version
  • Display boot messages: dmesg
  • Display sound driver status: cat /dev/sndstat







Tips: Kernel Sources and Patches



  • Obtain source: ftp://ftp.us.kernel.org/pub/linux/kernel/v2.X/linux-2.X.X.tar.gz
  • Uncompress: tar xzf linux-2.X.X.tar.gz

    This creates the local directory: linux/...

  • Apply patch:

    • Download patch: ftp://ftp.us.kernel.org/pub/linux/kernel/v2.X/patch-2.X.X
    • Execute patch script: /usr/src/linux/scripts/patch-kernel (Script located in directory linked to kernel source)

      • Source directory default: /usr/src/linux
      • Patch directory assumed to be the current directory.


      This script executes the patch command against the source code to make the appropriate updates.
      (cat linux-2.X.X.patch patch -p0)

  • Set-up directory links:

    • mv linux linux-2.X.XX
    • ln -s linux-2.X.XX linux
    • /usr/include/asm should point to /usr/src/linux/include/asm-i386

    • /usr/include/linux should point to /usr/src/linux/include/linux
    • /usr/include/scsi should point to /usr/src/linux/include/scsi







Example: Applying the devfs patch to kernel 2.2.20



  • Download kernel: http://www.kernel.org/pub/linux/kernel/v2.2/linux-2.2.20.tar.gz
  • mv linux-2.2.20.tar.gz /usr/src
  • cd /usr/src/
  • Remove previous source: rm linux (May use rm -Rf linux/)

  • Unload to subdirectory linux/: tar xzf linux-2.2.20.tar.gz
  • Download patch: http://www.atnf.csiro.au/~rgooch/linux/kernel-patches.html
  • Download patch file devfs-patch-current to /usr/src/
  • cd /usr/src
  • Test: patch -p0 --dry-run <>

  • Apply patch: patch -p0 <>
  • Now go through regular kernel build as described above. (From the beginning) make distclean; make mrproper
  • The configuration has been altered by the patch thus make xconfig
    will have options not seen in the original 2.2.20 kernel.

    • "Code maturity level" option
    • Select "development and/or incomplete code/drivers"

      This will provide lots of options for kernel.
    • Go to Filesystems - check /dev file support "Experimental"

      This feature was introduced by the patch.
      If it is not selected the devfs feature introduced by the patch will not be included.



Kernel source must be in /usr/src/linux/ (From kernel.org) or /usr/src/linux-2.2.20/ for the patch command as given, to work.
The patch is preconfigured for this.


patch man page.






Tips:










More Information:












Return to http://YoLinux.com for more Linux links, information and tutorials


Return to YoLinux Tutorial Index


Feedback Form


Copyright © 2000, 2001, 2002, 2003, 2005 by Greg Ippolito










No comments: