How to set up X and VAAPI Accelerated HD Video on the MSI Wind U110 (GMA500 - Poulsbo) and Ubuntu Karmic 9.10

The following guide is now slightly outdated as Ubuntu 10.04 is out and new drivers are necessary, also some of the repositories I originally used have disappeared. If you use Ubuntu 10.04, try following my much easier directions here instead. (The largest detriment to the Ubuntu 9.10 driver is a complete lack of suspend to ram for me. I will leave my guide for 9.10 online as reference, though. It seems to have been the first good description and I'm happy about the major online media coverage. :)

I've finally managed to get it running. Here is what I did. First you need to get Ubuntu 9.10 running with poulsbo. The first steps 1.-4. describe how to set up the driver for X. The second part (5.-10.) describes how to set up VAAPI and mplayer to get video acceleration. (There is already an early stage GMA500 driver for Ubuntu 10.04.)

The image shows the phoronix test suite benchmarking when I play Grey.ts in a loop (-loop 0) with mplayer and vaapi but without sound (-nosound) on my netbook. During the entire time, the CPU frequency is also downscaled to 800 Mhz. Compare to phoronix vdpau benchmark on a core 2 duo system (which obviously shows even less cpu usage, because the CPU is much more powerful) or the Atom and Nvidia ION benchmark on Phoronix.
CPU usage when playing a 10 minute fireworks recording in 1080p @ 50 fps without sound and then two FullHD movie trailers. The CPU is clocked at 800Mhz the entire time. The spike is caused by a bug in my test sequence: I didn't start a new mplayer process for new files but passed them all at once on the command line.

I can watch FullHD (1080p @ 50 fps + AC3) videos with about 30% or less CPU utilization (Atom@800 Mhz) on my netbook now. Looks really nice, too. Some videos produce artefacts, especially self-recorded AVCHD m2ts files. If you use the right demuxer, AVCHD videos play fine, but deinterlacing does not work. Sometimes the video seems not to run very smooth, etc. It's not perfect yet, don't expect too much! But most videos work perfectly, no matter what resolution.

The only real problem is with suspend to ram. You can't use suspend to ram or VAAPI at the same time: If you used VAAPI and try to suspend, the system doesn't suspend and/or crashes. If you suspend, then use VAAPI or even xv I think, the system crashes really hard. So suspend doesn't really work as expected yet. I think the sound also doesn't work properly after suspend. Suspend works fine for me even after resume if I do a double console switch (Ctrl-Alt-F1 wait until you see the console, then Ctrl-Alt-F7).

Here's what you need to do
1. Basically you add the GMA500 Repositories for Ubuntu 9.10 (karmic) and 10.04 (Lucid) to /etc/apt/sources.list.d/mobile.list. (If the entry below doesn't work, check this page.)
sudo add-apt-repository ppa:gma500/ppa && sudo apt-get update

2. Then you install the poulsbo-driver-3d including the firmware and everything.

2.a. Check if you can load the psb module: sudo modprobe psb. If you see this in dmesg instead of a switch to the correct resolution,
"kernel BUG: unable to handle kernel NULL pointer dereference at (null)" (2.6.31)
you need a manual hack to get the driver working, as I just reported.


3. Once that's done, for a more stable and speedy operation, you need to add an /etc/X11/xorg.conf:

Section "Device"
Identifier "GMA500"
Option "AccelMethod" "EXA"
# someone suggested UXA? but that's for the i965 drivers
Option "DRI" "on"
Option "MigrationHeuristic" "greedy"
Option "IgnoreACPI" "yes"
Driver "psb"
EndSection

Section "DRI"
Mode 0666
EndSection

4. Reboot and make sure that it works! If X doesn't work properly yet, VAAPI won't, either. After a short black screen, you should be greeted in the correct resolution. Full screen xv video should already work fine. Suspend to RAM also worked perfectly at this point for me.

If  the system goes to DPMS standby and from there switches to other modes (DPMS suspend, etc.), then my screen actually turns on again (showing all black with backlight active). So make sure you chose only one setting for DPMS. A constantly running screen is not good at all.

VAAPI Video Acceleration setup
Now comes the part where we install the hardware decoding acceleration. If you just want X you can stop now. ;) This is written for Ubuntu 9.10 with old repositories, so 5+6 might not be necessary anymore or work for Ubuntu 10.04!

5. Install the Libva library from http://www.splitted-desktop.com/~gbeauchesne/libva/pkgs/i386/libva1_0.31.0-1+sds8_i386.deb. This replaces your current libva1 with a newer version.  also works for me. Don't forget to install the -dev version (http://www.splitted-desktop.com/~gbeauchesne/libva/pkgs/i386/libva-dev_0.31.0-1+sds8_i386.deb) as well if you want to compile mplayer later.


6. Create a link of your video driver to the vaapi driver directory:
sudo ln -s /usr/X11R6/lib/modules/dri/psb_drv_video.so /usr/lib/va/drivers/

A. Precompiled setup. (You can alternatively skip to part B: compiling the source code setup)

7. Download and unpack the precompiled mplayer:

8. Install a normal mplayer and libmad with apt-get.
sudo apt-get install mplayer libmad0

9. Unpack mplayer and create necessary links to the libraries it expects.
"ldd mplayer" tells you what library (names) mplayer expects.
Just to go /usr/lib and create links to them:
e.g
ln -s libx264.so.67 libx264.so.65
This should work as long as your libraries are newer than the expected ones. It may actually work out of the box in Ubuntu Jaunty.

10. Start mplayer from the package you downloaded:
e.g. ~/mplayer-vaapi-20090914.i686/mplayer -fs -vo vaapi -va vaapi HDtest.avi

B. Compiling the Source Code

I've also manged to get the source code to work. This is less bothersome, because you need not link the libaries. Hence less man hours, but more disk space and computer hours for compiling. Didn't take as long as I feared, though. And it's using a slightly newer version of mplayer-vaapi, which includes some rudimentary OSD in vaapi mode now.

7. Install the build environment:

sudo apt-get build-dep mplayer
sudo apt-get install libdrm-dev
maybe: sudo apt-get -f install


8. Download the source at http://www.splitted-desktop.com/~gbeauchesne/mplayer-vaapi/mplayer-vaapi-20091106-FULL.tar.bz2. Unpack it and execute the checkout-patch-build.sh.

9. Check if it works, e.g.
mplayer -fs -vo vaapi -va vaapi ~/HDtest.avi

10.
sudo make install.

X. Done! Enjoy e.g. a nice HD trailer for a start! Post your results! I can recommend using it in smplayer for more comfort(e.g. automatically continue where you left of last). If it crashed, check out by blog entry about smplayer crashing in vaapi mode.

Update: It turns out deinterlacing not working is not due to VAAPI, but a limit in the psb driver. It works with the current iegd driver and other drivers from AMD and NVIDIA (press the D putton in mplayer I think). If I could just get one of those iegd drivers... Thanks Gwenole from splitted-desktop.com for answering my email and of course for writing the mplayer-vaapi patches in the first place! He seems to be the absolute VAAPI expert.

Also some experiments show that I can attach an external screen and watch videos with VAAPI on it. That means I should be able to actually use the netbook for decoding HD video and displaying it on an external screen. (But it does not work for me with except in mirror or single display mode.) Only thing missing now is a digital display port to connect a FullHD display... ;)

You might also be interested in Surround Sound and Battery Life
And I did get a USB surround sound card working wonderfully now. Also here's some advice I wrote on decreasing your power consumption.

X Video finally works - Poulsbo and Linux on the MSI Wind U110 - Fix for BUG: kernel NULL pointer dereference at 00000000

If you tried to get poulsbo to run on your system and you get a message like this, you should try the fix posted below.

"kernel BUG: unable to handle kernel NULL pointer dereference at 00000000" (2.6.28)
"kernel BUG: unable to handle kernel NULL pointer dereference at (null)" (2.6.31)

Thanks to a post on Ubuntu's bugzilla, the poulsbo video module now finally works on my MSI Wind U110 (full review & installation guide). Download the psb-kernel-source module You only have to comment out the two lines in intel_lvds.c starting with

if (edid)
....

e.g.
/* if (edid)
drm_add_edid_modes(output, edid); */

Then reconfigure the package and it should be recompiled. The next time you start it, it should work instead of creating kernel BUG

Update:
Xvideo actually works better than in Directx video in Windows XP, where I often see blocky video scaling. The CPU usage is not too high, either. Currently it's only working under Ubuntu 9.04 for me. Though I had managed to compile the module with kernel 2.6.30 before, I don't remember how I did that... And I haven't gotten actual video acceleration to work yet.

Of course I'll keep you updated and I'll post a little howto once it's all working. But feel free to ask, anyway. And thanks so much to the person from the Ubuntu bugzilla who helped me to finally get it all working!!!

Update2: Keep checking the page. I will post a link to an archive with a script that will manage the compilation in Ubuntu 9.10. If someone could help me figure out how to, I'd love to create, upload and post a .deb for dkms so it all works automatically... Or ultimately, we could modify the driver to check for the MSI's signature and then skip the edid check.

I've created a package for Karmic that you can use to manually install the driver with the fix already applied: http://www.megaupload.com/?d=BB23R7K2. Extract it and execute the script make_psb.sh, then reboot and try to modprobe psb. Let me know when the 10 downloads run out and it it works for you.

Update3: Here is a step by step howto. It explains setting up X and then VAAPI (hardware video decoding) in Kubuntu 9.10.

My script for making the psb module out of the package:

KERNEL=/lib/modules/`uname -r`/
make KERNELRELEASE="$(uname -r)" LINUXDIR="/lib/modules/$(uname -r)/build" DRM_MODULES=psb || exit 1;
sudo cp drm.ko $KERNEL/kernel/drivers/gpu/drm/drm.ko
sudo mkdir -p $KERNEL/updates/dkms
sudo cp psb.ko $KERNEL/updates/dkms/psb.ko
ls -l $KERNEL/kernel/drivers/gpu/drm/drm.ko $KERNEL/updates/dkms/psb.ko
sudo depmod -a

Linux System Hardening Done Professionally

I just took a short look at the measures Google took to harden their Chromium OS. But I must say, I was pretty impressed. It seems to be a very, very secure system.

Linux Boots Faster!

Check out this live video comparison at TuxRadar:

Kubuntu 9.10 is a Beauty

It works nicely on intel graphics (except GMA500!), installs fine and seems much snappier than the 8.10 I was still using. It's a definite upgrade recommendation, as well for your friends who are thinking about Windows 7. ;)

I'll let you know if I'll have any worries. But so far, everything has been wonderful. KMS works great, suspend-resume is fine, KDE 4.3 is lovely... :)

Good work guys!

More News on Intel and its GMA 500 under Linux

The LinuxJournal wrote two articles about the horrible Linux support for Intel's GMA 500 graphics chipset.
How to kick your friends in the face - GMA500
More on Poulsbo GMA 500 - Intel and the Community

Also see the Intel defending response.

Convert 1080i AVCHD directly to 720p avi with ffmpeg 0.5

With the realease off ffmpeg 0.5 there is much better AVCHD support in my experience, so you can use it for perfect conversions. You can download ffmpeg from ffmpeg.org and then use this script:

#!/bin/sh
# ffmpeg-avchd script by linux-tipps.blogspot.com
# to encode a directory use this command:
# for i in *.m2ts; do ffmpeg-avchd $i; done
IN="$1"; shift
OUT=$(echo $IN | sed 's/.m2ts//')-720p.avi
echo Encoding $IN to $OUT.
ff="ffmpeg -deinterlace -i "$IN" -acodec copy -vcodec libx264 -vpre normal -crf 25 -sws_flags lanczos -s hd720 -r 25"
echo $ff $OUT
nice $ff "$OUT"

The only thing I'm not really happy with yet is the deinterlacing and deshaking. I would like to use a sharper deinterlacer, but I guess I'd need mencoder for that. The Lanczos software scaler does make steady images pretty sharp already, though.

Why you're screwed with the Intel Poulsbo Chipset

Let me tell you why you're not only screwed when you want to run Poulsbo systems on Linux:

Intel does not only have bad support for Linux, it even excludes any email support, even for Windows for the Intel Poulsbo Chipset: "Intel does not support this product via email."
see http://supportmail.intel.com/scripts-emf/defaultlanding.aspx?productid=3180&srchidstr=39,1101,3180 and it seems does no longer supply driver downloads: http://downloadcenter.intel.com/NotFound.aspx?ProductID=3180&lang=eng. Ok, the last is wrong: http://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&ProductID=3001&DwnldID=17992&strOSs=45&OSFullName=Windows%20XP%20Home%20Edition*&lang=eng

Now what other support options are there from Intel? Contacting your manufacturer often does not prove helpful either. MSI's support even seem aware that the Poulsbo chipset is supposed to have active video decoding acceleration. They inform that the Poulsbo chipset is not a high end chipset and you shall just disable video decoding acceleration in case it crashes your Windows system.

Considering the overal bad support for the Poulsbo chipset I can not recommend using it: No good Linux driver, no good Windows driver, DXVA announced but not working(not even in Windows), no email support, no Windows driver downloads, etc.

So I think: don't buy Poulsbo systems.

Intel advertises features it can't deliver and doesn't support the chipset. And it's really sad to bring out a chipset that won't be supported well until maybe 2010.

Update: I've finally managed to get poulsbo working on my MSI Wind U110 netbook. Check out the dirty hack and the details.

MSI Wind U110 Linux Review

The MSI Wind U110 netbook has an excellent basic design: The new generation Intel hardware is made for long battery life and the MSI can really make it over 9 hours under Windows. The MSI's are one of the few netbooks of this generation which include a nice and bright non-glossy display.

The keyboard is excellent and can from my experience be used immediately without any issues. I actually like it better than my external keyboard. The only problem here was the tiny "'" key. But that's only important for English text and I'm sure in the version they sell in English speaking countries the keyboard doesn't have that flaw.

So they only thing that might be seen as negative is the lower display resolution compared to some other new netbooks that have an 11.6" display. But their display is usually glossy and thus disqualified for really using it anywhere outside and also of course costs battery time.

But for Linux it looks much worse. While the Poulsbo chipset was already roughly supported in Ubuntu 8.04, which was released over a year ago, the support for Poulsbo is still at least flaky. It needs binary blobs to work anywhere near well as far as I have read.

And unfortunately it's not yet working on the MSI Wind U110 or U115 netbooks. You can normally get the chipset to work pretty well - with the appropriate resolution and at least some 2D acceleration, but no hardware video acceleration (aka VDPAU or VA-API).

But on the MSI's the support isn't working yet. You get the message that the there was a null pointer dereference when trying to load the psb kernel modules that is "necessary for the driver to work" as the Xorg driver tells you. Though I've read it might actually work on a certain SuSe distribution.

That unfortunately means that the netbook can not currently be used for Linux. A small display combined with a bad resolution will definitely lead to headaches. Linux users will either have to use Windows until sometime - hopefully - Intel fixes it's drivers (and maybe even includes a working VA-API support!).

What else I've checked worked fine: Sound, USB, Brightness Adjustments. I haven't tested standby enough yet, but I think there were still problems. But that may very well also be related to the graphics chipset. And as far as I've read the standby to ram works fine at least with certain BIOS versions.

The MSI U110 comes with one RAM slot, which held 1 GB ram in this case and is extensible to 2 GB. There is of course an internal card reader included. A microphone and a headset jack and a USB port, a VGA and a 100 Mbit LAN port are on the right side, two more USB ports, the energy plug, a kensington lock connection and a fan on the left side.

Summary: For now only for Linux experts who are either willing to debug and fix the kernel module or figure out how to get SuSe running on it or people who are willing to use Windows until the problems are fixed (supposedly that might happen by the end of this year). But it's a great netbook with lots of battery time, a nice display and a great keyboard. Perfect for writing blog entries or a large word document.

Once the graphics work, I'll try installing and running Linux from an SDHC card to see it that saves even more power. But right now with the vesa driver the power savings aren't that good. I wonder if they are better with the real driver.

What works&What doesn't
  • Grafics (works pretty well now!)
  • Sounds works (needs a fix in Ubuntu 9.10)
  • SD-Card Adapter works (it's generally not very fast, though)
  • Standby (works in Ubuntu 9.10)
  • CPU low power mode (works for me, reportedly doesn't work in certain configurations)
  • Grafics low power mode (?)
  • Sound low power mode (creates problems and had to be deactivated in 9.10)
  • Webcam works with uvcvideo after pressing the short cut (Fn-F6).
  • Ethernet works in 9.10 after plugging in the cable (ASIX AX88772 USB 2.0 Ethernet).
  • Wifi works out of the box in 9.10. Needs additional drivers in 9.04.
If there are any questions feel free to ask.

UPDATE: X video works nicely now. Finally!!! :)

I even managed to get to work vaapi hardware video decoding up to FullHD 1080i.

Sound does not always work without problems. In Karmic 9.10 you have to comment out
#options snd-hda-intel power_save=10 power_save_controller=N
in /etc/modprobe.d/alsa-base.conf.
options snd-hda-intel position_fix=1 bdl_pos_adj=64
instead helped me as well.

Suspend to RAM works fine with newer kernels from Ubuntu 9.10 and the psb module active. But I think it crashed when I tried to use vaapi after that. It did. But that's fixed with the newest BIOS version (the most current one on 01. June 2010) for me now.

I've run powertop on a full battery(half a year old) to test how low I can get the power usage in X with it.

PowerTOP version 1.11 (C) 2007 Intel Corporation

Cn Avg residency P-states (frequencies)
C0 (cpu running) ( 1.8%) 1.60 Ghz 0.4%
C0 0.0ms ( 0.0%) 1333 Mhz 0.2%
C1 mwait 0.0ms ( 0.0%) 1067 Mhz 0.6%
C2 mwait 0.2ms ( 0.0%) 800 Mhz 98.7%
C4 mwait 0.3ms ( 0.0%)
C6 mwait 9.6ms (98.1%)

Wakeups-from-idle per second : 103.4 interval: 15.0s
Power usage (ACPI estimate): 5.6W (10.3 hours)

Top causes for wakeups:
53.7% (126.9) : acpi
26.4% ( 62.3) : psb@pci:0000:00:02.0
11.2% ( 26.4) : hrtimer_start_range_ns (tick_sched_timer)
2.3% ( 5.5) : Rescheduling interrupts
1.8% ( 4.3) : extra timer interrupt
1.3% ( 3.1) : hrtimer_start (tick_sched_timer)

I think the psb driver is not highly optimized for low power consumption yet, but the biggest problem is the pretty high number of "acpi" interrupts. We can see that with neither ethernet nor wifi active, the netbook can reach over 10 hours of run time. And this is with a battery that already lost 10% capacity. We should be able to get this down a lot more by finding the cause for the acpi interrupts and with some optimizations, mostly in psb.

A Netbook for Linux

While most netbooks use common intel chips, some use the GMA500, which is yet badly supported for linux. An open source driver "may" be there in Q4 2009. But as the video decoding acceleration benchmarks show, it might be worth waiting for it. It's not only good, but also energy-efficient at decoding even HD material. The bad thing is though - there's at the moment no yet any well working Linux driver available. And you can't even know for sure it will ever come. If you're still interested, check out the MSI Wind U110.

Adjust Linux Network Receive Window for Fast Networks

Simply execute the following command in your shell to increase the receive window buffers. That should improve the performance, especially from servers with a high latency.
sudo sysctl -w \
net.core.wmem_max=1075200 \
net.core.rmem_max=1075200 \
net.ipv4.tcp_rmem="4096 87380 8388608" \
net.ipv4.tcp_wmem="4096 87380 8388608" \
net.core.rmem_default=1075200 \
net.core.wmem_default=1075200

Booting Linux over the Network

Here's a quick how to.

Refreshing Linux Group Rights

Maybe I'm mistaken, but I think there is no other way to let your system know that you e.g. are now member of another group than logging out and back in. And I'm wondering - why? Can it be so complicated? We can reload, refresh everything, but for that you'd have to log out, which is especially annoying in X...

But that' wrong! You can work around the problem with newgrp. The program updates the environment variables so you can immediately use the group, at least in the shell started via newgrp.

The Essence of Open Source

Imagine your computer breaks down. It doesn't power on anymore. Now imagine you're not allowed to open it and the only person who may fix it would be the producer. It would probably take weeks for you to send it in, get the machine fixed. That's if the error was found and it got fixed. And then it might end up being pretty expensive. Because there's a monopoly on the repairs.

That's exactly how closed source software works. And that's exactly why you should prefer open source software. The more important the software, the more important that it's open source.

Pretending a Package is Installed by Creating an Empty Package with Checkinstall (for Debian-based Distributions)

After installing ffmpeg from svn with checkinstall, I had the problem that the also installed library libavcodec51 is not compatible with the one delivered with Ubuntu. But I could not simply uninstall it so that the manually installed version was used because that caused problems with libxine1-ffmpeg, which stopped kaffeine, amarok and other software from running properly.

Unfortunately, checkinstall's --provides option did not work as expected. So I had to find a way to have them use the manually installed version I compiled from svn and hiding that fact that my package manager. I had to let the package management system know the package was already installed.

So I ended up creating an artificial package that only has the same name and a similar package version, but no actual contents. Create an empty directory, place the following Makefile into it and execute the following checkinstall command. You may need to adjust the parameters for different packages, refer to the output of apt-cache policy somepackage for an appropriate version number)
Makefile:

install:
install -d /usr/local/bin

checkinstall commands:
checkinstall --nodoc --install=no --pkgname=libavcodec51 --pkgversion=3:0.svn$(date +%Y%m%d)-12ubuntu5
checkinstall --nodoc --install=no --pkgname=libavutil49 --pkgversion=3:0.svn$(date +%Y%m%d)-12ubuntu5
checkinstall --install=no --nodoc --pkgname=libavformat52 --pkgversion=3:0.svn$(date +%Y%m%d)-12ubuntu5
sudo dpkg --install ./*.deb
sudo ldconfig


This elegantly fixed the error "ffplay: symbol lookup error: /usr/local/lib/libavcodec.so.52: undefined symbol: av_gcd" and "ffmpeg: error while loading shared libraries: libavformat.so.52: cannot open shared object file: No such file or directory" for me.

Circumventing Censorship - Evading the "20th Century's Berlin Wall"

The New York Times presents the status quo in censorship evasion with examples of Iran and China. A quite interesting read. Don't expect a how to, though.

Mpeg Artifacts

I wonder why mpeg decoders always produce artifacts (bad blocks) when decoding a stream with errors. Shouldn't there be a much better way to handle stream errors? What about discarding the block, there should only be a little glitch then instead of a completely bad block.

The other, more difficult option would seem to be to try to find a block that would match the crc and replace the bad block with it.

At least the current situation seems to be less that optimal.

It seems a similar idea has already been patented: "(WO/2004/028160) DETECTION AND REPAIR OF MPEG-2 CHROMA UPCONVERSION ARTIFACTS"