Small Linux kernal patch = big improvement in desktop experience

[40hz]

Just in time for the Holidays - this news from the folks over at Phoronix.com (emphasis added):

The ~200 Line Linux Kernel Patch That Does Wonders

Published on November 16, 2010
Written by Michael Larabe

In recent weeks and months there has been quite a bit of work towards improving the responsiveness of the Linux desktop with some very significant milestones building up recently and new patches continuing to come. This work is greatly improving the experience of the Linux desktop when the computer is withstanding a great deal of CPU load and memory strain. Fortunately, the exciting improvements are far from over. There is a new patch that has not yet been merged but has undergone a few revisions over the past several weeks and it is quite small -- just over 200 lines of code -- but it does wonders for the Linux desktop.

The patch being talked about is designed to automatically create task groups per TTY in an effort to improve the desktop interactivity under system strain. Mike Galbraith wrote the patch, which is currently in its third version in recent weeks, after Linus Torvalds inspired this idea. In its third form (patch), this patch only adds 224 lines of code to the kernel's scheduler while stripping away nine lines of code, thus only 233 lines of code are in play.

Tests done by Mike show the maximum latency dropping by over ten times and the average latency of the desktop by about 60 times. Linus Torvalds has already heavily praised (in an email) this miracle patch

-more-

http://www.phoronix....2637_video&num=1

Phoronix has a couple of (badly produced) quick & dirty vids to give you an idea of what this performance boost will look like.

----------

If you'd like to give this patch a try, but don't feel like waiting until it's built into your distro (and aren't keen on hacking the kernal yourself) an alternative method can be found here.

Alternative To The "200 Lines Kernel Patch That Does Wonders" Which You Can Use Right Away

Phoronix recently published an article regarding a ~200 lines Linux Kernel patch that improves responsiveness under system strain. Well, Lennart Poettering, a RedHat developer replied to Linus Torvalds on a maling list with an alternative to this patch that does the same thing yet all you have to do is run 2 commands and paste 4 lines in your ~/.bashrc file. I know it sounds unbelievable, but apparently someone even ran some tests which prove that Lennart's solution works. Read on!

Basically, Lennart explains you have to add this to your ~/.bashrc file (important: this won't work on Ubuntu. See instructions for Ubuntu further down the post!)

This is something I'm definitely going to try as soon as I can get some free time! 

Has anybody else given it a shot, either by patching the kernal, or using the alternative "install"?

 

[Deozaan]

My wife just walked into the room while I was reading this post and saw this:

Phoronix has a couple of (badly produced) quick & dirty vids to give you an idea of what this performance boost will look like.

-40hz (November 21, 2010, 05:19 AM)

Her: "Quick and dirty videos?" in a tone of disgust.

Me: "It's not what you think! 'Quick and dirty' is a phrase that means hastily put together!"

Her: "It better be!"

Thanks for getting me in trouble, 40hz! Next time mark the thread NSFW! Not Safe For Wives! Pretty funny how that sentence could be interpreted as a euphemism when taken out of context.



Oh, and that kernel patch sure sounds interesting. I'll probably wait until it comes built in before I try it though.

[40hz]

@Deo: Sorry man! I forgot how lucky I am...  

===

Her: Quick & Dirty huh? Lemme see...

Me: Oh it's not like that. It just means "done fast."

Her: Oh really?

      (BEAT)

      What a shame...

---

At which point she flashes me one of her patented "over the shoulder" smiles as she gracefully drifts out of the room...

===

Some people are amazed how long we've both been together.

I'm not.

[Stoic Joker]

Some people are amazed how long we've both been together.

I'm not.

-40hz (November 22, 2010, 10:56 AM)

...and for that I applaud you!

(The wife and I will have been married 19 years in December)

[40hz]

Some people are amazed how long we've both been together.

I'm not.

-40hz (November 22, 2010, 10:56 AM)

...and for that I applaud you!

(The wife and I will have been married 19 years in December)

-Stoic Joker (November 22, 2010, 11:30 AM)

No applause necessary. For once in my life I got something I truly wanted.

Hats off to you and yours too! 19 is a good number. Looks like the two of you have got it down cold. Keep up the good work!

 

[housetier]

I'll wait for the kernel patch because I did not understand what this alternative does and I didn't have the attention span to find out.

Until then I'll live with mediocre desktop performance

[Edvard]

After suffering with hand-me-down systems for so long, a laggy Linux desktop didn't surprise me any more, but since I moved on to a decent 64-bit system (albeit a few years old) I don't notice it so much.
I saw this last week sometime and got pretty excited for some of the older boxes in my house, then I saw the numbers that Lennart Poettering put up for the alternative patch and knew I HAD to try it.

It seems to be doing something...
For the record, this is a 64-bit single-core Desktop with 4 Megs RAM, 250Gb HD, and a Radeon HD 2600XT with ATI Catalyst drivers running on Xubuntu 9.10.
I opened a Vmware Player session running Mint with 1Meg of memory, a bunch of miscellaneous apps just to take up the memory, Thunderbird, Firefox open with ~15 tabs, didn't notice any slowdowns and switching between tabs was normal.

Then I thought I'd really kick it in the pants and run Glxgears and the Unigine Sanctuary demo in benchmark mode simultaneously.
Glxgears showed between 600-1000 fps while Unigine maintained between 14-20 (ick).
Firefox still opened websites just fine and switching to other windows was just as snappy as ever.

Then I opened the Amnesia:Dark Descent demo.
Suddenly glxgears went down between 300-500 fps and A:DD was crawling.
Unigine Sanctuary never broke a sweat though, maintaining its measly 14-20 fps the whole time.
I couldn't go opening websites while A:DD was running, so couldn't test that.

Probably a horrible method for testing, but I think it's doing what it says on the tin.

[40hz]

@Edvard - The Man!

I somehow had the feeling you were gonna beat me to trying that bad puppy out first.

Pretty cool findings you're seeing. Question: did you use the recompiled kernel for 64-bit, compile your own, or use the "alternate method" to set it up?

[Edvard]

The alternate.
To this day, I have not been able to successfully compile a kernel (this too I shall conquer, but not today), and all the pre-compiled ones are for Ubuntu 10.10 (I'm running 9.04) so a few userspace scripts works better.
Besides, reading some of the post/riposte between Linus and Lennart got me real curious.
http://lkml.org/lkml/2010/11/16/392

Like I said, I'm right now running the fastest box I've ever owned so speed is a bit of a relative term.
Later I'm going to test it on a lower powered box (32-bit XP1600 with 1G RAM) and see how it flies.

[housetier]

I wish I knew what the alternate method does. Afaic, it doesn't matter which method I try: they both do things I do not fully understand, and both aren't explained anywhere.

I think this is beyond my level hack-fu

[40hz]

The alternate.
To this day, I have not been able to successfully compile a kernel (this too I shall conquer, but not today), and all the pre-compiled ones are for Ubuntu 10.10 (I'm running 9.04) so a few userspace scripts works better.

-Edvard (November 27, 2010, 07:25 PM)

That was my thought too. I'm not a big believer when it comes to customizing a kernal unless the incorporated change is absolutely essential and there's no alternative way to incorporate it. Neither is the case with this patch.

Besides, reading some of the post/riposte between Linus and Lennart got me real curious.
http://lkml.org/lkml/2010/11/16/392

Ah....Linus Torvalds!   Once again being simultaneously 'spot on' - and utterly graceless.

I think that sporadic and unpredictable crankiness has become his trademark by now. Just like almost every other übergeek in that regard.

Good 'coder' though, so I guess I'll continue to cut him some slack.

[f0dder]

To this day, I have not been able to successfully compile a kernel (this too I shall conquer, but not today)

-Edvard (November 27, 2010, 07:25 PM)

user@box /usr/src/linux # make menuconfig modules bzimage modules_install

[MilesAhead]

It's been several years since I did any kernel compiling.  But the nice thing about Linux is you can set up the boot loader to allow kernel selection on start-up.  The old one that presumably works is still there to select if the modified one is giving trouble.  I think it was grub that had the easier configuration.  Lilo was a pain in that every time you compiled a kernel, even if the output was in the same spot, if may have changed its absolute location on the drive, requiring running the setup every time to be sure.  They may have fixed it eventually though.

With a graphical configuration program that works, modifying the kernel is kind of fun.  When it sucked was the script based text mode deal where it asked you about 250 questions, do you want this? yes, no maybe? and if you messed up hitting Enter too quickly, back to square one and run the script all over again!!  Loads of laughs!!

The Gui one is easy. Don't know about Ubuntu, but the old Red Hat, Mandrake, Debian, and even Slackware distros, once you had the Gui config app, came set up pretty logically.  And with modules you could configure things to be optional on start up.  No longer using that scanner? Don't load the module in the start up scripts.

Also you set up different sections of the source tree for different kernel versions. Usually you would compile something newer than came as the default kernel.  Start up scripts would branch using kernel version detection. If you wanted to go back to the kernel that came with the distro it was usually a matter of copying vmlinuz file and running grub configuration at the most.

[Edvard]

@housetier: No hack-fu required, just follow the directions, reboot, and you should be good.
@f0dder: Thanks, I'll give that a shot next time I'm in the mood for self-torture

OK, I think I figured out what's going on with these contraptions but I may be (more or less, completely) wrong.

I don't understand why this makes a difference, but what the kernel patch does is group tasks per tty and kill the group when there are no more tasks (I think...).
The result is overall performance increase (as seen in the Phoronix video), but it's mostly useful for servers and kernel developers as it's function is tied to the tty's.

The userspace hack does kinda the same thing (they both make heavy use of cgroups to do their magic), but deploys it in userspace and so directly affects desktop performance.

It doesn't make anything faster per se, it just redistributes resources when the system is under heavy load from some resource-hogging program (updatedb, ferinstance  ) so you can still get work done while said program flails away in the background.

Some questions I still have:
1- What happens when the script hack and the kernel patch are used at the same time?
2- Does this affect the expected performance of a realtime kernel, where you want certain things to take the limelight (which is why you're running a realtime kernel in the first place).
3- Will this help with the problem of stuttering during heavy disk I/O?

More tests, Igor!!

[f0dder]

Edvard: I can't see why kernel vs. usermode "cgrouping" should affect server vs. desktop loads - that's probably some urban legend. There might be some slight differences in how stuff is done, though; I've seen mentioned that the usermode script thingy gives a bit more visibility to what's going on, while the kernel patch is more like "silent magic".

I dunno if each X11 app gets it's own PTY, and I haven't looked into what cgroups exactly are - but if stuff in a cgroup is treated as "something that should be considered as a whole" for scheduling purposes, it makes sense that eg. a heavy make job is less disruptive to the system if each instance isn't treated as it's own little cpu-greedy entity.

It would definitely be interesting with some more info - in easy-to-follow-without-reading-a-zillion-different-documents-and-mailing-lists-and-browsing-kernel-source on what exactly happens and what the consequences exactly are

[Edvard]

I was hoping you'd chime in, I figured you'd know a lot more about this stuff than I do. 

RE: server vs. desktop:
To clarify, 'Desktop' means to me a graphical environment with applications launched by clicking a mouse on a menu item or icon, etc.
'Server' would then mean a text environment with mostly services and utilities typically launched from script or typing commands in a TTY (virtual login terminal) of which Linux normally reserves 7.
More could be said about that, but I wish to be brief...

According to Lennart's ranting comments, since the kernel patch only groups things per a given TTY, typical desktop applications wouldn't benefit directly because their origin is a graphical environment in userspace and therefore wouldn't be bound to any TTY-based 'cgroup'.
His alternate method does it's mojo in userspace and not bound by TTY.
Tests by somebody (I can't remember who now) showed that the alternate method increased perceived desktop performance at least as well as the kernel patch, and evenput forth numbers numbers to back it up.

That said, I can see how the kernel patch would help perceived (hehe, I said it again...) desktop performance because background processes in a typical desktop environment are normally started pre-X11 in the startup process, (therefore bound to TTY's and herded into their cgroups) thus leaving computing power left over for the graphical interface.
Also, isn't it true that X11 is itself bound to TTY8, so it wouldn't exactly be left out of the party, right?

You are correct about the "visibility" versus "silent magic" discussion.
Apparently, Mr. Poettering's argument lies with his opinion that a userspace implementation will benefit typical desktop users better, and the numbers appear to bear that out.
Linus likes the 'silent magic' of the kernel option because it "just works", i.e. it benefits everybody; from server admin to kernel hacker to Joe Desktop, and all it takes is to have it available in the kernel as a routine function of the scheduler.

MY question would be could these two methods be employed concurrently, (in other words, would there be any conflicts which would make performance even worse)?
If so, would it work by separating kernelspace and userspace "cgrouping"?

Like you said, we need plain english (or your preferred language pack  ) descriptions of what exactly is happening.

[Edvard]

A fairly lively article at technewsworld reveals a few key arguments:
http://www.technewsw...com/story/71328.html

'Do It Once and Leave It Built-In'

"The kernel patch groups processes by owning TTY. The bash shell change groups them by session," wrote Slashdot blogger Anonymous Coward, citing an explanation (for subscribers) on LWN.

Alternatively: "The differences between the change to the kernel and the shell script are basically two: one, they apparently have slightly different algorithms for choosing how to group the processes," brion wrote.

"That's not due to it being in-kernel vs out-of-kernel, though -- that's just because they are slightly different," brion added. "Both can be implemented in both ways, and both work with the same actual implementation mechanism -- simply one works from userspace through the interfaces and one's built-in to the kernel."

Auto-tuning behavior "that's built in will probably be the most reliable, easiest, and best-performing way to do this, rather than requiring every Linux distribution to ensure that they're running the same extra scripts and keeping the userspace stuff in sync," brion concluded. "Do it once and leave it built-in to the kernel."

And again: "One requires a kernel patch. One uses functionality *already present in the kernel* to do the same thing," chimed in spun. "Testing reveals the one that doesn't require a kernel patch is more responsive. You tell me which is best."