Building x264 on Windows with Visual Studio

For anyone who has ever done any video encoding (or just watching for that matter) will probably have heard of x264. And if you havnt then you should have at least have heard of h264 (and if you still havnt then this post is probably not for you) which is the standard name for the video codec that the x264 encoder implements. So in the world of h264 video encoding x264 is probably one of, if not the best. And it is completely free and open source. So anyone who wants to get their hands on it can do so easily.

One downside of the x264 project (much like most open source projects) is that the default build tool is a gnu make style build chain. These don't run natively on Windows and generally requires an emulated shell such as MSYS or Cygwin. And even from within these shells they only support gcc (MinGW on Windows) based compilers.

But for those wanting to compile x264 natively on Windows using the native Windows build chain (i.e. Visual Studio) then that can actually be done rather simply. For those with Visual Studio 2013 then compiling many similar open source projects becomes a lot easier due to the addition of partial C99 support. C99 is something Microsoft have been neglecting for many years but with the 2013 updates it is a lot closer. x264 however still requires some manipulation in order to get it compile under MSVC's mostly C89 world.
Luckily changing x264 for be MSVC compatiblilty is rather straightforward. Most of the issues are a result of C89 requiring all variable declarations to be altogether and the start of each logical block of code. Straight of the bat most of the errors that Visual Studio will spit out about the x264 code will be a result of this issue (although the error codes don't do you any favours in realising this). So most errors will be due to code such as the following:

int padv = PADV >> v_shift;
// buffer: 2 chroma, 3 luma (rounded to 4) because deblocking goes beyond the top of the mb
if( b_end && !b_start )
    height += 4 >> (v_shift + SLICE_MBAFF);
pixel *pix;
int starty = 16*mb_y - 4*!b_start;

This will generate an error on the variable 'pix' because it is declared mid way through a block of code. Luckily the specifications say 'block' of code, which does not mean function or something similar. Instead it essential means anything between a set of '{' or '}'s (There iss actually a bit more to it than that but for our purposes - as you'll see later - it is good enough). In the above example we can see that the declaration of 'pix' occurs after an if statement. So if all we need to separate blocks are some '{}'s then modifying the code to the following will actually work:

int padv = PADV >> v_shift;
// buffer: 2 chroma, 3 luma (rounded to 4) because deblocking goes beyond the top of the mb
if( b_end && !b_start )
{
    height += 4 >> (v_shift + SLICE_MBAFF);
}
pixel *pix;
int starty = 16*mb_y - 4*!b_start;

All we did here was add the '{' and the '}' to the if statement. This makes that statement a block and so the line following it becomes a new block which makes everything work. This is surprisingly simple fix and will work for all of the cases found in libx264. In fact a complete working libx264 can be achieved by just performing the above operation at 10 different location in code. Or if you couldn't be bothered doing it yourself you can just apply the following patch that I have already made up for you.

Download patch file:
https://github.com/ShiftMediaProject/x264/commit/d9004ba604283fb70a3b67d444f67576c00a0e2e.diff

Now for those who don't just want the lib for x264 but actually want to compile the command line x264.exe then you'll have to perform the same operation a few more times. However there are 2 additional things youll need to do.
First is related to an issue with the use of unions. Specifically the following piece of code:

return (union {double f; uint64_t i;}){value}.i;

The above is too much for MSVC to handle. But with a bit of massaging it can be made to work. Massaging such as this:

union { double f; uint64_t i; } ulld = { value };
return ulld.i

The second issue is due to initialization lists being used on an array of structs. MSVC defaults to thinking that each element in the initializer list is actually an input for each component for the actual struct. So in the following piece of code MSVC treats the initializer list as actually an initializer list for the first 'AVS_Value' in the array.

AVS_Value arg_arr[] = { res, avs_new_value_bool( info->interlaced ), avs_new_value_string( matrix ) };

This will cause some nonsensical errors such as how a type of AVS_Value can not be converted to type short (short here being the type of the first member of AVS_Value). There is pretty much no combination of additional '('s and '{' that will fix this problem. So we have to fallback to the slightly less convenient way of just specifying each array element individually.

AVS_Value arg_arr[3];
arg_arr[0] = res;
arg_arr[1] = avs_new_value_bool( info->interlaced );
arg_arr[2] = avs_new_value_string( matrix );

This is not as nice to look at but it works. Putting all these pieces together and x264cli will compile under Visual Studio without any further problems. Again for those who dont want to do all this themselves then the appropriate patch can be acquired from below.

Download patch file:
https://github.com/ShiftMediaProject/x264/commit/4c51a4fc51737d932eddb4060bcd03d861dfec7d.diff

Of course if you don't want to worry about any of the above then you can check out my git repository that has all the necessary changes already applied and even comes with a pre-built Visual Studio project file. My repo is up to date with the current upstream master and with x264 development slowing down due to the upcoming x265 then its unlikely my repo will fall behind the upstream master so can be treated as up to date.

git repository:
https://github.com/ShiftMediaProject/x264

So with all of those above fixes it should be reasonable trivial to get x264 building natively in Windows. As to whether this is worth it is a debate for another time. For those who aren't familiar with MSYS/Cygwin but are familiar with Visual Studio then this should be a big win. What will be interesting is to test the performance of the compiled binaries between the different compilers. Perhaps ill have more on that soon.....