XSAVE bug discovered for Skylake cpu

At first, serial console put us behind the schedule. As it was very unusual problem due to malfunctioned serial cable. And now some another bug has cost us some very precious time. The anomaly started while I was trying to boot Xen on my system, I found Xen stuck in an infinite loop of the line:

(XEN) traps.c:3290: GPF (0000): ffff82d0801af585 -> ffff82d080240e5a

(full log here)

After inspection it turned out to be dom0 kernel bug. We Located the hex-address by using

$ addr2line -e xen-syms ffff82d0801af585

And it returned ‘xen/arch/x86/xstate.c:387’ which is position for XSAVE in the file. Adding ‘xsave=0’ made xen boot just fine. This is further evidence that the error was caused by XSAVE. This bug is also reported by another Xen user using Skylake cpu. So, this bug can be due to Xen’s incompatibility with Skylake cpu (for now).

Using the workaround (add ‘xsave=0’ to xen line parameter while booting) I am able to run on Skylake cpu. Work is in progress to patch this bug. Let’s hope that it’ll be out soon.

Detailed conversation at Xen-devel mailing list

Setting Devlopment Environment for Xen on Ubuntu

Development environment setup for Xen Project sounded easy to me. But it proved to be a worthy task of a standalone article. So, now I will be guiding you through the process of installing Xen Project software from source code. This article was written targeting the Xen Project 4.7-unstable on Ubuntu 15.10 (4.2.0-19-generic), but majority of steps may remain same for future versions.

Getting Xen Project Source Code

Primary ways to get the Xen Project source code are:

  • Release Tarballs: Latest stable release can be downloaded from here as tarball.
  • Git: This is the preferred way if you want to get latest unstable release. Just run the following command:
$ git clone git://xenbits.xen.org/xen.git


Before actual compiling you need to fulfill some requirements:

  • Updated /sbin/installkernel: You need to update /sbin/installkernel so that it copies the kernel configuration upon a custom kernel install time. Edit the file by using following command:
# nano /sbin/installkernel

And then add these lines and save the file.

  • Dependencies: Xen Project uses several external libraries and tools which can be installed using:
# apt-get install build-essential
# apt-get install bcc bin86 gawk bridge-utils iproute libcurl3 libcurl4-openssl-dev bzip2 module-init-tools transfig tgif 
# apt-get install texinfo texlive-latex-base texlive-latex-recommended texlive-fonts-extra texlive-fonts-recommended pciutils-dev mercurial
# apt-get install make gcc libc6-dev zlib1g-dev python python-dev python-twisted libncurses5-dev patch libvncserver-dev libsdl-dev libjpeg62-dev
# apt-get install iasl libbz2-dev e2fslibs-dev git-core uuid-dev ocaml ocaml-findlib libx11-dev bison flex xz-utils libyajl-dev
# apt-get install gettext libpixman-1-dev libaio-dev markdown pandoc libc6-dev-i386 libssl-dev
  • Configure: It matches the libraries required by the program with those installed on the user’s computer to that program can compile successfully. Run:
$ ./configure

Build & Install

To build all components run:

$ make dist

After successfully compiling you can install it by running:

# make install

Linux grub2 update

To make sure grub2 pick up your new Xen hypervisor update grub by running:

# update-grub

And that’s about it. You can also refer to original guide, for more generalized instructions and information.

Kicking-off Outreachy

Finally, I got selected for Outreachy 2015. I will be working on the project “Introducing PowerClamp-like driver for Xen” with Xen Project with Dario Faggioli and George Dunlap . And this is my first blog to share my Outreachy experiences. Before jumping to my experience, I would like to mention about Outreachy.

Reflective of a generally low number of women participating in the FOSS development. Outreachy provides an equal opportunity for women in FOSS. It provides Free and Open Source Software Outreach Program internships. December 2015 – March 2016 is the 11th round of the program starting with one round in 2006 and then in 2010 with rounds organized twice a year.

Being a part of Outreachy is matter of immense pride for me. I also believe having a program targeted specifically towards women makes easier to reach talented and passionate participants, who are uncertain about how to start otherwise.

I applied for the program so that I can start contributing to the open source organizations and I expect to learn a lot from the cool developers here at Xen Project. As a contributor I think Xen Project has a great team of developers. I enjoyed working with Xen Project developers. They are fun, highly knowledgeable, understanding and easy to work with, even for newbies like me. I am grateful to them for their help on my first patch (linked below) and feel lucky to work besides them in future.

My first bite-sized contribution:



About my project “Introducing PowerClamp-like driver for Xen”

PowerClamp was introduced to Linux in late 2012 in order to allow users to set a system-wide maximum power usage limit. This is particularly useful for data centers, where there may be a need to reduce power consumption based on availability of electricity or cooling. A more complete writeup is available at LWN. These same arguments apply to Xen. The purpose of this project would be to implement a similar functionality in Xen, and to make it interface as well as possible with the Linux PowerClamp tools, so that the same tools could be used for both. The basic mechanism for PowerClamp in Linux is to monitor the percentage of time spent idling. When this time goes below a user-specified threshold, it activates a high-priority real-time process to force the CPU to idle for the specified amount of time. The idea is fairly straightforward, but working with the scheduler involves dealing with very tricky race conditions and deadlock.

Project includes two main tasks. First one is to apply the idea to the main Xen scheduler, the Credit1 scheduler. Here, we want something that is generic, i.e., that can be applied to all schedulers (Credit1, Credit2, etc.). So, when dealing with the interface and with some high level infrastructure for the feature, we will try to make this happen in generic code (like schedule.c, sysctl.c, etc.). Although some bits of the actual implementation can be scheduler specific. In this phase we will decide what we want to be generic, and what we want to put in per -scheduler code.Then, we have two main ways to implement this:

  1. Adding an extra priority level.
  2. Re-using the existing credit mechanism.

Depending on the scheduler we can figure this out and then implement the better one.

Second one is to design an appropriate interface. We can check for any PowerClamp user-space tools for Linux and we can try to integrate those tools with Xen. Or we can just have this be a separate Xen feature, accessible via Xen’s xl command-line interface. Or we can try to combine both of the ideas.