Ubuntu Karmic has been released to the public a few days ago and it’s time to write up a How To Compile a Kernel article. Only a few small changes have been made to the way we have compile a kernel for Ubuntu Karmic compared to Ubuntu Jaunty but they are significant enough for it’s own how-to.
I wrote this how to because my laptop has 4GB of internal memory and with the default kernel that ships with Ubuntu Karmic the entire 4GB is not addressed.
Lets get started building our own custom kernel for Ubuntu Karmic from scratch.
Updates
November 13, 2009: Compilation: The NOEXTRAS is no longer used . CONCURRENCY_LEVEL is automatically determined in the rules files.
November 10, 2009 : Changed the installation command. Thanks to comment by HotShotDJ
November 8, 2009 : Typo in the installing the kernel section. Only the headers were installed, not the image.
I wrote the follow up article: How to update your custom Ubuntu Karmic kernel after a new kernel release
Introduction
git
I’ll be using git to get the latest kernel version. This is my favorite way to get the sources and it is in my opinion the fastest way to make changes later on when you want to update your own kernel to the latest version.
I suggest adding my Launchpad repository to your system. The repository holds the latest version of git and is usually updated within a day of a new release of git, follow the instructions on the page Git Packages for Ubuntu to add my repository.
Arch
I am compiling the i386 version, if you want to compile for amd64 you need replace i386 for amd64 throughout this article.
Flavor
I choose the name core2 as the flavor name as for my personal use I’ll build a kernel for a Core2 processor. Besides the change of processor type in the configuration I also select support for 64GB as my laptop has 4GB, which is the main reason I started compiling my own kernels. I have some other changes but that’s beyond this article.
Preparations
Let’s get started by preparing our machine for compiling a kernel for Ubuntu Karmic.
Open a terminal.
sudo su - apt-get install fakeroot build-essential apt-get install crash kexec-tools makedumpfile kernel-wedge apt-get build-dep linux apt-get install git-core libncurses5 libncurses5-dev
Create a directory where you would like to build your kernel, this directory will hold the kernel source in a sub directory and all the deb files will end up in this folder. I choose /d1/development/kernel/karmic
Getting the source
cd /d1/development/kernel/karmic git clone git://kernel.ubuntu.com/ubuntu/ubuntu-karmic.git source
The source code is installed in the directory source.
We will create a branch in which we will be doing our modifications. That way the master branch will stay in tact which will make future updates a whole lot easier.
To see the latest version:
cd source cat debian.master/changelog|more
Press q to get back to the prompt.
This results in something like this:
linux (2.6.31-15.50) UNRELEASED; urgency=low CHANGELOG: Do not edit directly. Autogenerated at release. CHANGELOG: Use the printchanges target to see the curent changes. CHANGELOG: Use the insertchanges target to create the final log. -- Stefan Bader <stefan.bader@canonical.com> Tue, 03 Nov 2009 09:58:14 +0100 linux (2.6.31-15.49) karmic-proposed; urgency=low
The latest version is the first version after the UNRELEASED section. In this case it is 2.6.31-15.49. Sometime there is no UNRELEASED section and in that case the latest version is the first version you see.
You could select any version you see in the changelog but for the sake of this article we’ll use the latest version.
git checkout Ubuntu-2.6.31-15.49 -b core2
This will create a branch called core2.
Creating a new config
I’ll be using the method of creating a new flavor, this adds a bit more work but you can always compile the original kernels.
We’ll use the generic flavor as the base for our own flavor,core2.
cp debian.master/config/i386/config.flavour.generic debian.master/config/i386/config.flavour.core2 fakeroot debian/rules clean debian.master/scripts/misc/kernelconfig oldconfig
To make changes to the configuration file we need to edit the configuration file. The kernel developers have created a script to edit kernel configurations, unfortunately you will have to go through all the flavors for this script to work properly.
debian.master/scripts/misc/kernelconfig editconfig
This script will show you which configuration is next to be edited. You should not make changes, you can save the configuration if you like, the only time you have to edit the configuration is when you see
* Run menuconfig on i386/config.flavour.core2... Press a key.
Make your changes, save the configuration and then keep going until the script ends.
When you’re done, make a backup of the config flavor file.
cp debian.master/config/i386/config.flavour.core2 ../.
Now we need to clean up the git tree
git reset --hard git clean -xdf
Getting ready for compilation
Because we are going to be creating a new flavor based on a existing flavor (generic in my case) we need to create some extra files. During compilation the process checks the previous release for some settings, as we’re creating a local flavor it doesn’t exist in the source, so we’re creating it. The previous release in this case is 2.6.31-14.48
ls debian.master/abi
cp debian.master/abi/2.6.31-14.48/i386/generic debian.master/abi/2.6.31-14.48/i386/core2 cp debian.master/abi/2.6.31-14.48/i386/generic.modules debian.master/abi/2.6.31-14.48/i386/core2.modules
Copy our flavored configuration file back.
cp ../config.flavour.core2 debian.master/config/i386/
We need to edit the following files
- debian.master/scripts/misc/getabis
- debian.master/rules.d/i386.mk
In the next few parts I’ll explain what you need to change
File: debian.master/scripts/misc/getabis
Search for the line:
getall i386 generic generic-pae 386
Change it in:
getall i386 generic generic-pae 386 core2
File: debian.master/rules.d/i386.mk
Search for the line:
flavours = generic generic-pae 386
Change it in:
flavours = generic generic-pae 386 core2
We need to make the compilation process aware of our own flavor we want to compile.
cp debian.master/control.d/vars.generic debian.master/control.d/vars.core2
You can edit the file and make it your own.
arch="i386 amd64" supported="Core2" target="Geared toward Core2 systems." desc="x86/x86_64" bootloader="grub | lilo (>= 19.1)" provides="kvm-api-4, redhat-cluster-modules, ivtv-modules, ndiswrapper-modules-1.9"
We need to commit our changes in the git repository.
git add . git commit -a -m "Core2 modifications"
The text after -m is the message you add to your commit.
Compilation
It’s finally time for compiling but before we can start the compilation process there is one more step to do. I didn’t put this in the Preparations section as you need to the following step whether you make changes to the configuration or not
fakeroot debian/rules cleanTo start the compilation.
skipabi=true skipmodule=true fakeroot debian/rules binary-core2
Next you need to run
skipabi=true skipmodule=true fakeroot debian/rules binary-indep
Installation
After the compilation is finished we’ll have several deb files in the parent directory.
linux-doc_2.6.31-15.49_all.deb linux-headers-2.6.31-15-core2_2.6.31-15.49_i386.deb linux-source-2.6.31_2.6.31-15.49_all.deb linux-headers-2.6.31-15_2.6.31-15.49_all.deb linux-image-2.6.31-15-core2_2.6.31-15.49_i386.deb
To install the files
cd .. dpkg -i linux-headers-2.6.31-15-core2_2.6.31-15.49_i386.deb linux-headers-2.6.31-15_2.6.31-15.49_all.deb linux-image-2.6.31-15-core2_2.6.31-15.49_i386.deb
Check your bootloader if the newly installed kernel is the default one, for grub check the file /boot/grub/menu.lst or if you run grub2 check /boot/grub/grub.cfg
Reboot and enjoy your newly installed kernel.
Entries (RSS)
Just a quick one, wouldn’t doing a
“sudo apt-get install linux-image-2.6.31-14-generic-pae”
be what you are looking for, sure you won’t get the lastest kernel but you will get past the 3gig limit
You are correct that will do the trick as well but I also changed some other kernel settings, like changing the Processor family to Core2/newer Xeon, disabling Generic X86 support. to name a few.
I don’t know if this might have been pulled from the release, but I installed the beta and upgraded from there and my version of karmic uses grub2 and has no menu.lst
The default grub in Karmic is the older grub but if you run grub2 you’ll have to check /boot/grub.cfgI have to correct myself. The default grub with a new installation of Karmic is grub v2(grub-pc). I did an upgrade from Jaunty which doesn’t install grub-pc.
I still do it by downloading the latest kernel source from kernel.org. Then I do the ‘ol make xconfig, make, make install, make modules_install and then update-grub.
Tried this, and everything worked fine up until I ran the
CONCURRENCY_LEVEL=2 NOEXTRAS=1 skipabi=true skipmodule=true fakeroot debian/rules binary-XXXX
command. (where XXXX is replaced by my special name). When I ran this, I got the following error:
make[1]: *** No rule to make target ‘binary-XXXX’. Stop.
I went back and checked my steps. I verified files were in place and edited as required. Still fails.
Any ideas?
Did you run the clean command?
fakeroot debian/rules cleanHi Peter,
Yes, I did run the Clean command at you typed it. Is there a config or makefile I can look in to verify that my branch name has been included? Note: when I run the second command that you recommend:
CONCURRENCY_LEVEL=2 NOEXTRAS=1 skipabi=true skipmodule=true fakeroot debian/rules binary-indep
that command actually completes without an error. But with my branch name, it fails. Is there any limit on the name or length of name chosen? My XXXX name is actually 8 characters long.
Thanks for your help.
After the clean command, you can check debian/control
cat debian/control | grep Package| grep linux-image
There should be a line in there like:
Package: linux-image-2.6.####-XXXX
#### is the kernel version
XXXX is your flavor name.
Hi Peter,
After I retyped my last message and sent it the second time, the first copy and your response to it then showed up. Arghh! Please forgive the duplication.
Anyway, thanks for the response. I executed the above command, and sure enough, the control file has the line with the kernel number and with my ‘flavor’ in it. A wider grep shows my flavor appearing in 3 Package lines within the control file: linux-image, linux-headers, and linux-image-debug.
I then followed that up with the following two commands:
fakeroot debian/rules clean
and
CONCURRENCY_LEVEL=2 NOEXTRAS=1 skipabi=true skipmodule=true fakeroot debian/rules binary-XXXX
And I get the same make “No Rule” Stop error as before.
Thanks,
David
Hi Peter,
Responded earlier, but it didn’t show up on this page, so I’m responding again. Hope its not a duplicate.
Yes, I did execute the clean function. I did so on both attempts I made to build the new kernel (scrolled back to verify), and the clean command ran successfully. But the following execution of ‘rules’ to build binary-XXXX still failed with the following result:
make[1]: *** No rule to make target ‘binary-XXXX’. Stop.
I did try to run the second version of ‘rules’ that you indicated would be ‘wise’ to run – the one that builds binary-indep. That one ran without error, although it didn’t appear to build any usable kernel.
Question: XXXX represents my custom kernel name, similar to your ‘core2′ example. However, the XXXX name I use is actually 8 characters long. Is there any limit on name length that might cause problems?
Not sure where to go from here. Thanks for the help.
Interesting you don’t see your previous reply, I replied:
After the clean command, you can check debian/control
cat debian/control | grep Package| grep linux-image
There should be a line in there like:
Package: linux-image-2.6.####-XXXX
#### is the kernel version
XXXX is your flavor name.
I don’t think there is a limit, especially not 8 characters.
Do you have special characters in the name? Like spaces?
What is the result of:
dpkg-architecture -qDEB_HOST_ARCHAhh.
It’s AMD64. That’s the wrong architecture, as this is a CORE2 processor. I must be setting something wrong in the menuconfig. I thought I had set things correctly, but apparently not. Any hints?
Thanks,
David
Because it says AMD64 you will need to make the changes to the amd64 files.
the command you run is ran in the compilation process to determine which files to use, and even though you have a Core2 proc, the system is said to be AMD64
Hi !
Very interesting ! Only this one little thing that annoys me : I have checked out 2.6.31-14.51 and when I launch compilation, i see “arbitrary signature 2.6.31-14.48″ … Not that I worry that much about signatures, but I’d like to get this right… any idea ?
Thanks,
François
Francois,
I see no 2.6.31-14.51 in the repository.
Hello!
Very detailed tutorial!Thanks for that!
This is the first time when I’ve compiled my one kernel.
The issue is that this new kernel doesn’t appeared after installed in /boot/grub/menu.lst.
Do you have any ideea how to correct this?
Thanks!
Is the kernel installed in /boot?
You should have the following files:
abi*
config*
initrd*
System.map*
vmcoreinfo*
vmlinuz*
If you have these files you can run:
sudo update-grub
Hello!
Thanks for the reply!
I’ve looked in /boot and I don’t see new kernel there, although I’ve installed and run all the above steps.
I think that’s because of a typo:
“cd ..
dpkg -i linux-headers-2.6.31-15_2.6.31-15.49_all.deb
dpkg -i linux-headers-2.6.31-15-core2_2.6.31-15.49_i386.deb linux-headers-2.6.31-15_2.6.31-15.49_all.deb”
This two commands will install only headers not actual image. I think that first line should be :
dpkg -i linux-image-2.6.31-15-core2_2.6.31-15.49_i386.deb
Hope that helps.
Thank you for this catch. I’ll update the article
Thanks Adam!
It worked!
Next step for me is to understand what every configuration option means. After wards I will compile a kernel specific to my system.
Seems like running:
$ head debian.master/changelog
would be a bit more succinct than:
$ cat debian.master/changelog|more
followed by ‘q’. Regardless, helpful how-to.
If one has modules that are managed by DKMS, the headers need to be installed before the kernel or the compilation of those modules will fail. The better command to install your newly compiled kernel would be:
dpkg -i linux-headers-2.6.31-15-core2_2.6.31-15.49_i386.deb linux-headers-2.6.31-15_2.6.31-15.49_all.deb linux-image-2.6.31-15_2.6.31-15.49_all.deb
Thanks for this insight. I changed the how to accordingly.
Could you please explain the purpose of NOEXTRAS, skipabi, and skipmodule?
I’ve been using the make-kpkg method all this time. I’m very happy to have come across this article.
I have been using INSTALL_MOD_STRIP=1 lately in hopes of making the kernel smaller. Is there any specific reason you do not use this?
Thank’s for the article, I look forward to reading your response.
Also one quick question, how are patches handled? Say I want to patch my kernel with BFS 310 and ureadahead, at what point would it be best to patch? I would guess it would have to be some point before commit? Maybe before cleaning up the git tree?
Thanks again!
You will have to patch before you start compiling of course. You can add your patch before the commit in this article, which means you would have to patch just after you copied the vars.core2 file.
So it would be:
copy the vars.core2 file
Do your patching
git add .
git commit -a -m “Core2 modifications”
Keep in mind if you do patch upstream files you might get git merge failures when you follow the article: How to update your custom Ubuntu Karmic kernel after a new kernel release
Actually the NOEXTRAS is no longer used in the makefiles, so you can skip it. You can skip the CONCURRENCY_LEVEL as well, it’s automatically determined in the rules files.
skipabi and skipmodule disables a comparison check of modules of the current abi and the previous abi.
As there were no previous core2 modules we can’t really to a comparison.
INSTALL_MOD_STRIP is set to 1 in the rules. There is no need to add it to the cmd-line. See debian.master/rules.d/2-binary-arch.mk lines 67-69
Congrats on a very detailed and well written guide. It was a pleasure to follow it and build a new kernel
The last bit of the last command is not quite correct
dpkg -i linux-headers-2.6.31-15-core2_2.6.31-15.49_i386.deb linux-headers-2.6.31-15_2.6.31-15.49_all.deb linux-image-2.6.31-15-core2_2.6.31-15.49_i386.deb
I would like to whole-heartedly agree with deChrLam. You guide is EXTREMELY well written. Just about anybody should be able to create a customized kernel using your guide. Thank you so much for your dedication to helping the community.
Hey,
still a question : is it possible to compile with your introduction listed above on a amd64 machine a 32bit kernel?
Reason:
My netbook needs at least 4 hours to compile and i have to fix something that needs many different compilations… please tell is this possible.
Thx in advance
knuutsen
You can compile a 32bit kernel on a 64bit OS but you need to set up a 32bit environment with the help of debootstrap.
All I would like to do is add the patch for app armor support. Yet I can’t seem to find just a specific patch for that either in ubuntu or a new enough one at Novell. Other than that, I’m the type of person who prefers vanilla over distro kernels. Any tips on that?
Try using this git repository:
git clone git://kernel.ubuntu.com/jj/apparmor-mainline.git source
Let us know if that is what you were looking for.
This repository can not be used to compile a kernel as described in this article. It is a vanilla kernel and should be treated as such.
Is there a howto for compiling from a vanilla kernel? I’ve tried to start from the standard config file in /boot (config-2.6.31-16-generic) applied to a vanilla kernel and successfully compiled but it won’t boot. My thoughts are that this might be due to some new process introduced in Karmic. Any thoughts or suggestions?
Question, if I git clone from git clone git://kernel.ubuntu.com/ubuntu/ubuntu-karmic.git. In future, when there is a ubuntu-lucid.git, do i need to git clone the whole thing again? Or is there someway I can just continue with some kind of “git pull” so that the download is less?
For example, I only need to git clone http://www.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git once, and from there only need to git pull.
I notice when I do a git clone –reference linux-2.6 git://kernel.ubuntu.com/ubuntu/ubuntu-karmic.git, the amount being downloaded is quite big.
Thanks.
As far as I know now you will have to do a new clone. The Ubuntu kernel developers create a new repository for each Ubuntu release.
By using the reference option you do cut down the amount of data and time quite a lot.
time git clone --reference /d1/development/kernel/vanilla/source/ git://kernel.ubuntu.com/ubuntu/ubuntu-lucid.git sourceInitialized empty Git repository in /d1/development/kernel/lucid/source/.git/
remote: Counting objects: 30893, done.
remote: Compressing objects: 100% (8089/8089), done.
remote: Total 28274 (delta 21618), reused 26489 (delta 19892)
Receiving objects: 100% (28274/28274), 30.99 MiB | 763 KiB/s, done.
Resolving deltas: 100% (21618/21618), completed with 1356 local objects.
Checking out files: 100% (30919/30919), done.
real 1m13.555s
user 0m14.417s
sys 0m3.728s
compared to
time git clone git://kernel.ubuntu.com/ubuntu/ubuntu-lucid.git source2Initialized empty Git repository in /d1/development/kernel/lucid/source2/.git/
remote: Counting objects: 1390941, done.
remote: Compressing objects: 100% (233559/233559), done.
remote: Total 1390941 (delta 1152243), reused 1386311 (delta 1147672)
Receiving objects: 100% (1390941/1390941), 394.43 MiB | 763 KiB/s, done.
Resolving deltas: 100% (1152243/1152243), done.
Checking out files: 100% (30919/30919), done.
real 11m19.735s
user 1m54.743s
sys 0m26.142s
Hi, Peter.
I didn’t get the _all.deb files at the end of the process, only the _i386.deb (in my case amd64). Maybe for that reason i’m not able to install the headers, just the binary. The installer complains about missing dependencies for the header files
Did you also ran:
skipabi=true skipmodule=true fakeroot debian/rules binary-indep
Not sure anymore. Maybe I inadvertently skipped this one. I’ll go through the whole process again and let you know if things change.
Thnx a lot!
Hi, Peter.
You were right! I did miss that step. Now it works like a charm. Thanks a lot for this extremely helpfull (and well written) how-to!
-fm
I am happy you were able to compile your own Ubuntu kernel.
Hello,
I am using your description which is pretty clear but I am running into a problem. I have checked out the 2.6.31-18.55 version and the result is that debian/rules has moved to debian.master (which is not a real problem) but also there is no debian.env file in the debian directory. I have now “solved” this by adding DEBIAN=debian.master to the kernelconfig script which obviously is not the correct solution. How should I deal with this?
You should run : fakeroot debian/rules clean
This will create the rules file in debian, you have to run it twice
Once during the configuration
The 2nd time before compilation.
Hello Peter,
Thanks for the quick reply. This indeed solves my problem. (As you can see I am new to KARMIC and Debian).
Best regards,
Wim
Hello,
For some reason I get the error :
“dh_installchangelogs : package linux-image……. is not in control info” when executing the fakeroot debian/rules binary-core2 command in your description.
I check the debian/control to see if there is any reference to “linux-image…..” in there but there isn’t. I verified the changes in getabis and i386.mk and the changes are there.
Do you have any suggestions of what could have gone wrong and where to look for the problem?
Thanks in advance.
Wim
Forget about this. It was a mistake that occurred somewhere in the configuration. The control file is created correctly now!
After reading this post, I started compiling my own kernel. Unfortunately, I did the compilation on a 900MB disk and found out that 900MB is not enough.
I have aCore2Duo T6570 cpu. Could those who finished the compilation and have a core2duo please let me and others know if you can see any significant performance improvement on your systems?
Thank you for this good article. I was able to compile my own kernel with your help.
How would be the workflow for compiling several versions of the kernel with different configurations? I want to change the configuration frequently (for finding out what’s good and what is not) without recompiling everything and without conflicts in the package manager or elsewhere. Would be nice if you could help me with that.
If you want to change the configuration I do believe you’ll have to compile the kernel all over again.
By recompiling you won’t have conflicts, what you could do to keep track of your configurations is make backups of every configuration under a different name.