Category Archives: Linux

Tuxedo Computers: How not to be a Linux OEM

Tuxedo Computers Logo

Tuxedo Computers Logo taken from their Facebook page. Logo belongs to TUXEDO Computers GmbH.

My Fathers Alienware laptop has been dying lately. He’s also been very unhappy with the latest Microsoft offerings, and since Mom has been using Ubuntu for years, he was willing to give it a shot. My father likes nice hardware and is willing to spend money on it, unlike his son who just buys the cheapest crap. So the decision was: Dad wants a laptop with Linux.

My eye has been on Tuxedo Computers for a while. I intended to buy there, once either my Acer Aspire S3-391 or Dell XPS L502x would give up the ghost. Not happening any time soon, but I did recommend my father to buy a laptop at Tuxedo Computers because, as far I could tell, they’d ship with Ubuntu pre-installed.

I am, alas, a busy man, and the Tuxedobook arrived, and I won’t be having time to look at it the next weeks. I managed to clear up one measly hour to assist my father in doing the first setup. I, naively, assumed that would be all I need. Let’s see how I imagined this:

  • Unbox the laptop
  • Plug it in
  • Boot up
  • Get presented with the Ubuntu OEM installer
  • Set the hostname, set up his account
  • Copy his Thunderbird and Firefox settings from his Alienware Windows machine
  • Set up the samba share to our family server
  • Have a coffee, and have a chat with him and be on my merry way

I know this is possible.  I have done it myself, I sent my Zotac Nano ID61 to my friend John in Germany.  I did the OEM install, tested it (and reset it back to OEM install).  Approved it for good and I know it worked for him.  He told me he was suprised I did it for him, as he obviously could do i installs from scratch himself, but the gesture was appreciated.  Really, an OEM Ubuntu install is as easy as setting up a Windows OEM machine.  If not easier.  This is how OEMs, and Tuxedo Computers is one, should sent the hardware to their customers.

So, naively, I went to my parents without any of my gear.  No cables, no USB DVD Rom drives, not even a long network cable (my father uses wireless and while there is cabled infrastructure, there are no RJ45 in the living room).

I get there and my father hasn’t even unpacked it.  He let me do that.  Shiny!  We figure out how to plug in the battery, connect the power supply and then the magical moment of booting up comes.  It boots up, and the magnificent logo of Tuxedo Computers comes up.  Of course, I expect the Ubuntu logo next, but… nope… PXE booting starts.  Odd?  Is there no system on the SSD?  Perhaps on the HDD?  Let’s try booting from the HDD instead.  Nope.  Okay, fine.  First, I think: I’ll do a PXE boot and we’ll have this running in no time.  Oh. No network cable, right…

Let me think, ah, they added a Ubuntu DVD!  Great, let’s boot from… oh, yes, right.  Modern laptops don’t have optical drives.  Obviously my USB DVD drive is at home.

Well, Dads got 100Mbps fibre.  I’ll download Ubuntu 16.04 on my Moms computer.  That takes less than 5 minutes from releases.ubuntu.com, then I’ll drop it on a USB stick and we’re  running.  Look, I even have a brand new 8GB USB stick in my car.  As expected, the download takes no time.  Startup disk creator fails on it.  I try a direct dd of the ISO on it.  Nope, somehow hangs.  Weird.  I guess that USB stick was dead.  Dad gives me an old 2GB USB stick.  Startup disk creator fails, dd from iso succeeds.  Sweet.  We’re 40 minutes in my allocated time.  I may be lucky and get it running within the remaining time.

I boot from the stick and during the installation process, the Ubuntu installer tells me: “Hey, buddy… This thing already has Ubuntu on it.  Wanna overwrite it?”  What?!? How?  I notice it’s an EFI installation (hey, the “EFI” partition gives it away, right?”).  I abort the installation and  boot into the firmware of the computer.  I notice that EFI is disabled.  Keep in mind: it was delivered this way!  I turn EFI on and reboot!

It does boot into Ubuntu. Sweet!  I’ll get this done in… Wait?  What?  It boots to a German version of Ubuntu (fair enough) and the login screen and presents us with a user named after my fathers first name.  I ask my dad whether he got a password from Tuxedo Computers.  He says that he hasn’t.  Sure, he might have overseen it and perhaps we didn’t look enough in all the enclosed papers, but that’s not how you pre-install Linux.  I tried a few passwords, like his first name again, our last name, 1234.  Nothing let us in.

Obviously, I said “Fuck this shit, I’ll do it myself”.

Booting from the USB stick I created earlier, I tried installing Ubuntu.  Which then gives me crap about EFI and I notice that the Ubuntu installer doesn’t allow me to create GPT partition tables.  It also tells about SSD alignemnt problems (SSD tells sectors are 2048 based, Linux tells it’s 512 based.  Partitioning stupidly believes the Linux kernel) I tried the classic MBR setup.  That’s easy and usually works.  Grub fails to install.
I tried to install grub manually, I don’t manage.  Surely my fault, because I’m really angry by now.
At that point, I am through my time and I have to give up.  I was really, really, enraged.  As in “physically violently” enraged.  I wanted to smash something.  This was supposed to be easy, this was supposed to be working out of the box!

Yes, I do realize that the majority of Tuxedo Computers customers are people like me that nuke and install anyway, but that’s no excuse!  This was the most crappy out of the box experience I’ve ever had and I now have a ton of work, waiting for me and a father who cannot use his new toy.  Earliest I can take time would be begin August.

If I’d be a customer who just wanted a “Linux computer” that worked out of the box, I’d definitely return it.  Sure, I’ll fix it, I’ll get it running, but you shouldn’t actually need a guy like me to get a Linux machine up and running.

This is exactly the reason why people think that Linux is hard.  Exactly this!

Steam, Linux and Intel Graphics

Jorg… are you using Steam on Linux?
No motivation to pay close to 1000 for a steam box, but an i3/i5 with Intel graphics would be ok price wise… question is, would it be worth it?

Short answer: Yno.
Long answer:

This is a question in multiple parts, namely:

  1. Do you use Steam on Linux?
  2. Does Steam on Linux work well?
  3. Are integrated graphics sufficient these days for games?
  4. What about graphics drivers?

Now, let start with the easy part: Yes, I use Steam on Linux.  I basically only use it on Linux and one of the main criteria for buying games on Steam, or from Humble Bundle, is Linux support.  Steam, the application, works very well on Linux and it comes down to installing a package and that’s it.  That said, Steam is just Steam, being a kind of game library with a useful “Big Screen” (aka, “use it on TV”) mode.  On no configuration, did Steam refuse to run, but nobody runs Steam to run Steam.  What you want are the games.

That’s where question #3 comes in, and sorry, no… Linux won’t compensate your underpowered hardware.  Just as in Windows, you might end up buying a game and figuring out that “hell, fuck no, this won’t work“.  If it’s not because the game doesn’t start up at all, it will be because it is so horribly slow, that it’s unplayable even when turning down all the settings.  Now, in all fairness, my experience with Steam on Intel, limits itself to:

  • Zotac Nano ID61 – Celeron 867 / 8GB RAM / “Intel HD Graphics” (I have given this machine away, I can’t do tests any more)
  • Dell XPS 15 – Core i7-2630QM / 16GB RAM / “Intel HD 3000″ (Paired with an NVidia GT 525M)
  • Acer Aspire S3 – Core i5-3317U / 4GB RAM / “Intel HD 4000″

All run Ubuntu 14.04LTS with Steam installed from the Valve repos.  These are all older chips based on Sandy Bridge (Celeron 867 and Core i7-2630QM) and Ivy Bridge (Core i5-3317U) architectures, so obviously it’s not very representative for modern Intel hardware.  Take everything I say with a huge grain of salt.  Haswell, Broadwell and Skylake based processors might be so much better, especially with the Iris Pro graphics.  Intel graphics are so very diverse and the labels stuck on them make it next to impossible to find accurate benchmarks.  So basically, what it means is that I must go on feeling which really, is very subjective.

However, it’s not the only thing you can go on.  You can simply see what the minimum requirements for some games say.  A very well known AAA type game working on Linux is Sid Meiers Civilisation V.  Go and visit it, and you’ll notice that in the PC world, you damn better do your homework about your hardware and what this hardware means in relation to recommended and minimum settings.  Also, keep in mind this is a strategy game, and really shouldn’t tax your graphics card all that much.  Well, go and look:

  • Windows: Core i3 or better integrated graphics
  • Mac OS X: “Intel GMA (950/X3100), HD 3000″ not supported
  • SteamOS/Linux: Intel Integrated video chipsets (GMA 9XX, HD 3XXX) will not run Civilization V for SteamOS and Linux, and are unsupported.

So, the Zotac is out as is the i7 (using Intel graphics).  The i5 might run it, I don’t think I tried.

Another one?  Let’s take Victor Vran.  I bought this game because the gameplay reminded me of the PlayStation 2 version of Baldurs Gate.  I didn’t check specifications and I tried on the i5-3317U, connected using HDMI to my TV.  The game loads, but it is unplayable at any detail level.  Specs, here you go:

  • Windows: GeForce 8800 or higher, AMD Radeon HD 4000 or higher, Intel HD 4000 or higher (min. 512 MB VRAM)
  •  Mac OS X: OpenGL 4.1 (GeForce 600 or higher, AMD Radeon 5000 or higher, Intel HD 4000 or higher)
  • SteamOS/Linux: OpenGL 4.1 (GeForce 600 or higher, AMD Radeon 5000 or higher)

Hey!  Intel isn’t even supported on SteamOS/Linux.  I should try it on Windows to see how playable the game is, but I doubt it will be a very agreeable experience.

… you’ll notice that neither of these games is even a first person shooter-puzzler.
Indie games will work, right?  Right?

Yeah, kinda… Again better check what the specs are.  Rochard works fine, for example, and is a fun game.  Less graphically intensive games like World Of Goo or Thomas was alone work perfectly.  Rochard and Thomas was alone and are a joy to be played with an XBox controller, which works out of the box on Linux.  Are these your kind of games?  You’ll be happy with Integrated graphics and Steam.

However, not all is perfect. Which brings us to the drivers and question #4.

How are the drivers?  Intel has a good reputation on Linux for drivers, but, and that’s really a but… that’s because AMDs drivers are quite horrible and NVidias drivers are closed.

Are Intels drivers good?  Generally, yes, and it might just be because I use whatever comes stock with Ubuntu 14.04LTS that I have problems in some games.  Most of the issues here are in my i5-3317U (which is what I use for most casual gaming).  First time I noticed that something was wrong was that in certain scenes of Book of the unwritten tales – Critter Chronicles, the characters were very dark.  That’s a point ‘n click adventure, nothing graphically too fancy.  My guess, is that they used a badly implemented shader.  The game was playable, but when it happened, it looked ugly.
Recently I’ve been playing Never Alone, which is very playable on the i5 from a performance perspective.  However, the game uses some shader that is broken.  Probably the same issue as mentioned before, but it’s so much more annoying because the shaders move and look so very much out of place.  Is it playable?  Yes.  Is it pretty?  No.

There is also something strange going on with the i5.  Due to a flash of genius, I had the idea to just connect the laptop to my TV and play on the TV.  I set the TV as primary display and disabled the internal panel.  The surprise came when I tried playing something like Stanley Parable or Talos Principle.  They were unplayable, at any resolution.  Why is this remarkable?  Because I played these games on the Zotac ID61, which is immensely lower spec than that laptop.  Something is going, on, but I have no idea what.  Given these games play fine on the internal panel, my guess is it has something to do with the drivers and how a dual screen configuration is handled.

Back on topic: if you’re using Intel Integrated for gaming (within the range of chips I talked about), you get what you pay for.  I could recommend looking into AMD offerings, as their integrated graphics solutions are better, but you’ll still end up with disappointment.  I was rather happy with my A8-3850 (Radeon HD 6550D), but as a matter of fact, you’ll end up running most games on low settings any way.  I have given up and just got myself an low-range NVidia gaming card.  (Which I haven’t tested properly to tell whether it was a good idea.)

What does this mean?
Linux gaming will work, if you have the hardware to match.  If you don’t the experience will be as crappy or crappier than on Windows.  That 1100€ SteamBox suddenly does look more attractive, doesn’t it?   No, I don’t have that kind of money for toys either, but my fans can of course send me one as a present ;-).  Whether such a SteamBox will actually play games well enough, that, I cannot say.  Maybe consider a Steam Link to your real game box?

Update on the Debian kernel bug.

It might not be caused by the kernel, but by the Xen hypervisor.  What I did up to know:

  • I installed the problematic kernel in a virtual machine (DomU) while the host (Dom0) was running Jessie and a thus different kernel.  Within that environment, no problem occurred.
  • I reinstalled Wheezy on the machine, but this time, I did not install Xen and did exactly the same dd command.  The problem did not arise.  (I also simplified the disk setup and upgraded my BIOS to the latest version, for good measure.  It shouldn’t make much of a difference, I was surprised there was a new BIOS in the first place)
  • Being confident, it might perhaps be caused by the disk setup (Originally I had 4 disk raid6 with one spare, and now I simply have a 2-disk raid1, with no spares), I installed Xen and rebooted.  When I tried the dd command, I got my Oops.

Conclusion:  the error only seems to occur on a Dom0 while using Xen.  It can be avoided by upgrading to Jessie.

While that is good news for the new setup, it still implies that under no circumstances, I can reboot hammerhead before mako is ready.  It might of course be linked to AMD specific code, but I’m really not willing to take that risk.

I wonder if I should file a bug with the Debian kernel team.

Recently, I decided to reinstall my old self-built rack server (AMD A6-3650, 16GB RAM, Asus F1A75-V PRO)  It wasn’t really being used and since I want to reconfigure my Dell R210-II, I decided the AMD should, at least temporary, take over the Dells tasks.  Yes, I know it’s not real server hardware, and yes, I think of buying another R2xx when I’ve got a bit money to waste, which is not now.

So, I installed Debian Wheezy and the Xen Hypervisor on it, as always using PXE, which means you end up with an installation that is fully up-to-date, unlike my other machines who tend to have older kernels because I rarely see a reason to reboot.

Then, one of the first things I tried was to clone a disk over network ( dd if=”/dev/vg0/vm-root” | ssh root@mako “dd if=/dev/vg0/vm-root”).  I have done these things before, and I know they work.  It’s not the quickest way, but I had my reasons to do as such.  Thing is: I got a kernel oops.  While only a “oops”, it does make the system unstable so a reboot is truly recommended.

I thought it would perhaps be a fluke, so I tried again… Same thing, so I removed the networking component and tried a simple dd if=/dev/zero of=/dev/vg0/big-lv bs=1073741824 and, yes again a kernel oops.  It looks something like this:

Feb 11 23:00:08 mako kernel: [ 8450.177200] BUG: unable to handle kernel paging request at ffff88013f800000
Feb 11 23:00:08 mako kernel: [ 8450.177222] IP: [<ffffffff811b3e27>] clear_page_c+0x7/0x10
Feb 11 23:00:08 mako kernel: [ 8450.177237] PGD 1606067 PUD bdd89067 PMD bdf86067 PTE 0
Feb 11 23:00:08 mako kernel: [ 8450.177256] Oops: 0002 [#1] SMP 
Feb 11 23:00:08 mako kernel: [ 8450.177268] CPU 2 
Feb 11 23:00:08 mako kernel: [ 8450.177272] Modules linked in: fuse btrfs crc32c libcrc32c zlib_deflate ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs reiserfs ext3 jbd ext2 efivars xen_gntdev xen_evtchn xenfs nfsd nfs nfs_acl auth_rpcgss fscache lockd sunrpc bridge stp loop radeon ttm drm_kms_helper eeepc_wmi snd_hda_codec_hdmi psmouse asus_wmi sparse_keymap snd_hda_intel snd_hda_codec rfkill drm snd_hwdep snd_pcm powernow_k8 mperf pl2303 snd_page_alloc power_supply serio_raw pcspkr i2c_piix4 evdev snd_timer k10temp wmi snd usbserial soundcore button processor thermal_sys ext4 crc16 jbd2 mbcache dm_mod raid456 async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq md_mod usbhid hid sg sd_mod crc_t10dif r8169 mii ohci_hcd ahci libahci xhci_hcd ehci_hcd libata igb i2c_algo_bit i2c_core dca scsi_mod usbcore usb_common [last unloaded: scsi_wait_scan]
Feb 11 23:00:08 mako kernel: [ 8450.177604] 
Feb 11 23:00:08 mako kernel: [ 8450.177610] Pid: 4221, comm: sshd Not tainted 3.2.0-4-amd64 #1 Debian 3.2.65-1+deb7u1 System manufacturer System Product Name/F1A75-V PRO
Feb 11 23:00:08 mako kernel: [ 8450.177630] RIP: e030:[<ffffffff811b3e27>]  [<ffffffff811b3e27>] clear_page_c+0x7/0x10
Feb 11 23:00:08 mako kernel: [ 8450.177647] RSP: e02b:ffff8801f1f17b30  EFLAGS: 00010246
Feb 11 23:00:08 mako kernel: [ 8450.177658] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000200
Feb 11 23:00:08 mako kernel: [ 8450.177669] RDX: ffffea00045e4000 RSI: 0000000000000000 RDI: ffff88013f800000
Feb 11 23:00:08 mako kernel: [ 8450.177681] RBP: ffffea00045e4000 R08: 0000000000000000 R09: 00000000000401d7
Feb 11 23:00:08 mako kernel: [ 8450.177693] R10: 0000000000000002 R11: 0000000000000fc4 R12: 0000000000000000
Feb 11 23:00:08 mako kernel: [ 8450.177704] R13: 0000000000000000 R14: 0000000000000000 R15: ffff8801f1f16000
Feb 11 23:00:08 mako kernel: [ 8450.177718] FS:  00007f7904fce7c0(0000) GS:ffff8803cb500000(0000) knlGS:0000000000000000
Feb 11 23:00:08 mako kernel: [ 8450.177734] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
Feb 11 23:00:08 mako kernel: [ 8450.177745] CR2: ffff88013f800000 CR3: 000000014239e000 CR4: 0000000000000660
Feb 11 23:00:08 mako kernel: [ 8450.177757] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Feb 11 23:00:08 mako kernel: [ 8450.177769] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Feb 11 23:00:08 mako kernel: [ 8450.177781] Process sshd (pid: 4221, threadinfo ffff8801f1f16000, task ffff8803b356e100)
Feb 11 23:00:08 mako kernel: [ 8450.177796] Stack:
Feb 11 23:00:08 mako kernel: [ 8450.177804]  ffffffff810bb8cd ffff8803cb515628 ffffea00045e4000 0000000000000000
Feb 11 23:00:08 mako kernel: [ 8450.177830]  00000001000280da ffffffff00000041 00000003caf73025 ffff8803cb72ac08
Feb 11 23:00:08 mako kernel: [ 8450.177856]  ffff8803cb72ac00 0000000081004f2f 0000000000000030 ffff8803cb72ac08
Feb 11 23:00:08 mako kernel: [ 8450.177882] Call Trace:
Feb 11 23:00:08 mako kernel: [ 8450.177894]  [<ffffffff810bb8cd>] ? get_page_from_freelist+0x57a/0x665
Feb 11 23:00:08 mako kernel: [ 8450.177907]  [<ffffffff810bbb3e>] ? __alloc_pages_nodemask+0x186/0x7ab
Feb 11 23:00:08 mako kernel: [ 8450.177921]  [<ffffffff810d1a97>] ? handle_pte_fault+0x298/0x79f
Feb 11 23:00:08 mako kernel: [ 8450.177933]  [<ffffffff81004e44>] ? pte_pfn_to_mfn+0x26/0x77
Feb 11 23:00:08 mako kernel: [ 8450.177945]  [<ffffffff8100569f>] ? __xen_set_pte+0x11/0x51
Feb 11 23:00:08 mako kernel: [ 8450.177957]  [<ffffffff810e6ee9>] ? alloc_pages_vma+0x12d/0x136
Feb 11 23:00:08 mako kernel: [ 8450.177969]  [<ffffffff810d1964>] ? handle_pte_fault+0x165/0x79f
Feb 11 23:00:08 mako kernel: [ 8450.177981]  [<ffffffff810cefaf>] ? pmd_val+0x7/0x8
Feb 11 23:00:08 mako kernel: [ 8450.177992]  [<ffffffff810cf02d>] ? pte_offset_kernel+0x16/0x35
Feb 11 23:00:08 mako kernel: [ 8450.178005]  [<ffffffff81353e74>] ? do_page_fault+0x320/0x345
Feb 11 23:00:08 mako kernel: [ 8450.178018]  [<ffffffff81095461>] ? arch_local_irq_save+0x11/0x15
Feb 11 23:00:08 mako kernel: [ 8450.178029]  [<ffffffff81095e17>] ? __call_rcu+0x21/0x12c
Feb 11 23:00:08 mako kernel: [ 8450.178041]  [<ffffffff8110b26f>] ? dput+0x27/0xee
Feb 11 23:00:08 mako kernel: [ 8450.178052]  [<ffffffff810fc21e>] ? fput+0x17a/0x1a1
Feb 11 23:00:08 mako kernel: [ 8450.178063]  [<ffffffff810eb3fb>] ? arch_local_irq_restore+0x7/0x8
Feb 11 23:00:08 mako kernel: [ 8450.178074]  [<ffffffff81351415>] ? page_fault+0x25/0x30
Feb 11 23:00:08 mako kernel: [ 8450.178084] Code: 20 4c 89 4c 24 48 c7 44 24 08 10 00 00 00 48 89 44 24 18 e8 8c f9 ff ff 48 83 c4 58 c3 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 f3 aa c3 66 0f 
Feb 11 23:00:08 mako kernel: [ 8450.178270] RIP  [<ffffffff811b3e27>] clear_page_c+0x7/0x10
Feb 11 23:00:08 mako kernel: [ 8450.178283]  RSP <ffff8801f1f17b30>
Feb 11 23:00:08 mako kernel: [ 8450.178291] CR2: ffff88013f800000
Feb 11 23:00:08 mako kernel: [ 8450.178436] ---[ end trace c0e1c75d9283be10 ]---
Feb 11 23:00:08 mako kernel: [ 8450.178466] note: sshd[4221] exited with preempt_count 1
Feb 11 23:00:08 mako kernel: [ 8450.178971] BUG: scheduling while atomic: sshd/4221/0x10000001
Feb 11 23:00:08 mako kernel: [ 8450.179002] Modules linked in: fuse btrfs crc32c libcrc32c zlib_deflate ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs reiserfs ext3 jbd ext2 efivars xen_gntdev xen_evtchn xenfs nfsd nfs nfs_acl auth_rpcgss fscache lockd sunrpc bridge stp loop radeon ttm drm_kms_helper eeepc_wmi snd_hda_codec_hdmi psmouse asus_wmi sparse_keymap snd_hda_intel snd_hda_codec rfkill drm snd_hwdep snd_pcm powernow_k8 mperf pl2303 snd_page_alloc power_supply serio_raw pcspkr i2c_piix4 evdev snd_timer k10temp wmi snd usbserial soundcore button processor thermal_sys ext4 crc16 jbd2 mbcache dm_mod raid456 async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq md_mod usbhid hid sg sd_mod crc_t10dif r8169 mii ohci_hcd ahci libahci xhci_hcd ehci_hcd libata igb i2c_algo_bit i2c_core dca scsi_mod usbcore usb_common [last unloaded: scsi_wait_scan]
Feb 11 23:00:08 mako kernel: [ 8450.181496] Pid: 4221, comm: sshd Tainted: G      D      3.2.0-4-amd64 #1 Debian 3.2.65-1+deb7u1
Feb 11 23:00:08 mako kernel: [ 8450.181530] Call Trace:
Feb 11 23:00:08 mako kernel: [ 8450.181560]  [<ffffffff8134a2be>] ? __schedule_bug+0x3e/0x52
Feb 11 23:00:08 mako kernel: [ 8450.181591]  [<ffffffff8134f4a5>] ? __schedule+0x85/0x610
Feb 11 23:00:08 mako kernel: [ 8450.181621]  [<ffffffff8110b26f>] ? dput+0x27/0xee
Feb 11 23:00:08 mako kernel: [ 8450.181652]  [<ffffffff81042090>] ? __cond_resched+0x1d/0x26
Feb 11 23:00:08 mako kernel: [ 8450.181682]  [<ffffffff8134fa7f>] ? _cond_resched+0x12/0x1c
Feb 11 23:00:08 mako kernel: [ 8450.181713]  [<ffffffff81049a2a>] ? put_files_struct+0x65/0xad
Feb 11 23:00:08 mako kernel: [ 8450.181743]  [<ffffffff8104a02c>] ? do_exit+0x292/0x713
Feb 11 23:00:08 mako kernel: [ 8450.181774]  [<ffffffff8107130f>] ? arch_local_irq_disable+0x7/0x8
Feb 11 23:00:08 mako kernel: [ 8450.184239]  [<ffffffff81071307>] ? arch_local_irq_restore+0x7/0x8
Feb 11 23:00:08 mako kernel: [ 8450.184271]  [<ffffffff81350e3f>] ? _raw_spin_unlock_irqrestore+0xe/0xf
Feb 11 23:00:08 mako kernel: [ 8450.184304]  [<ffffffff81048345>] ? kmsg_dump+0x52/0xdd
Feb 11 23:00:08 mako kernel: [ 8450.184336]  [<ffffffff81350e3f>] ? _raw_spin_unlock_irqrestore+0xe/0xf
Feb 11 23:00:08 mako kernel: [ 8450.184368]  [<ffffffff81351d14>] ? oops_end+0xb1/0xb6
Feb 11 23:00:08 mako kernel: [ 8450.184399]  [<ffffffff81349d8b>] ? no_context+0x1ff/0x20e
Feb 11 23:00:08 mako kernel: [ 8450.184430]  [<ffffffff81349619>] ? pmd_val+0x7/0x8
Feb 11 23:00:08 mako kernel: [ 8450.184460]  [<ffffffff81349638>] ? pte_offset_kernel+0x16/0x35
Feb 11 23:00:08 mako kernel: [ 8450.184491]  [<ffffffff81353d0a>] ? do_page_fault+0x1b6/0x345
Feb 11 23:00:08 mako kernel: [ 8450.184522]  [<ffffffff81004e44>] ? pte_pfn_to_mfn+0x26/0x77
Feb 11 23:00:08 mako kernel: [ 8450.184553]  [<ffffffff81004375>] ? __raw_callee_save_xen_make_pte+0x11/0x1e
Feb 11 23:00:08 mako kernel: [ 8450.184584]  [<ffffffff81351415>] ? page_fault+0x25/0x30
Feb 11 23:00:08 mako kernel: [ 8450.184615]  [<ffffffff811b3e27>] ? clear_page_c+0x7/0x10
Feb 11 23:00:08 mako kernel: [ 8450.184646]  [<ffffffff810bb8cd>] ? get_page_from_freelist+0x57a/0x665
Feb 11 23:00:08 mako kernel: [ 8450.184677]  [<ffffffff810bbb3e>] ? __alloc_pages_nodemask+0x186/0x7ab
Feb 11 23:00:08 mako kernel: [ 8450.184709]  [<ffffffff810d1a97>] ? handle_pte_fault+0x298/0x79f
Feb 11 23:00:08 mako kernel: [ 8450.184739]  [<ffffffff81004e44>] ? pte_pfn_to_mfn+0x26/0x77
Feb 11 23:00:08 mako kernel: [ 8450.184770]  [<ffffffff8100569f>] ? __xen_set_pte+0x11/0x51
Feb 11 23:00:08 mako kernel: [ 8450.184800]  [<ffffffff810e6ee9>] ? alloc_pages_vma+0x12d/0x136
Feb 11 23:00:08 mako kernel: [ 8450.184831]  [<ffffffff810d1964>] ? handle_pte_fault+0x165/0x79f
Feb 11 23:00:08 mako kernel: [ 8450.184862]  [<ffffffff810cefaf>] ? pmd_val+0x7/0x8
Feb 11 23:00:08 mako kernel: [ 8450.184892]  [<ffffffff810cf02d>] ? pte_offset_kernel+0x16/0x35
Feb 11 23:00:08 mako kernel: [ 8450.184922]  [<ffffffff81353e74>] ? do_page_fault+0x320/0x345
Feb 11 23:00:08 mako kernel: [ 8450.184954]  [<ffffffff81095461>] ? arch_local_irq_save+0x11/0x15
Feb 11 23:00:08 mako kernel: [ 8450.184984]  [<ffffffff81095e17>] ? __call_rcu+0x21/0x12c
Feb 11 23:00:08 mako kernel: [ 8450.185014]  [<ffffffff8110b26f>] ? dput+0x27/0xee
Feb 11 23:00:08 mako kernel: [ 8450.185044]  [<ffffffff810fc21e>] ? fput+0x17a/0x1a1
Feb 11 23:00:08 mako kernel: [ 8450.185074]  [<ffffffff810eb3fb>] ? arch_local_irq_restore+0x7/0x8
Feb 11 23:00:08 mako kernel: [ 8450.185105]  [<ffffffff81351415>] ? page_fault+0x25/0x30

Okay, a Debian stable kernel causing kernel oopses?  Nah, can’t be…  Damn, probably the memory is broken.  As such, an overnight memtestx86+ is scheduled and in the morning, it tells me everything is just fine.

At this point, I worry that it truly is a kernel bug.  I verify my other machines and none of them run 3.2.65-1+deb7u1, but all of them have it installed already.  Unlike Ubuntu, Debian doesn’t seem to amass old kernels in /boot.  I would have tried using an older kernel, but somehow I didn’t find the magic apt invocation to do so.

I still wanted to verify whether it’s the kernel causing this, so I upgraded the AMD machine to Jessie.  After I did so, I tried the same tests as on the original install and it works exactly as expected.  No more oops.

I also realize that my other machines are a reboot away from instability!  Scary thought.  Now, I’ll probably just trash the system, try wheezy again and see whether the problem comes back.  If so, it must be kernel bug.  The question is whether I should report it to the Debian kernel team.  I’m not sure I can really help them, also I could just ignore it and go Jessie (getting rid of systemd isn’t all that hard on a server as I found out today).