Symfony benchmark using HHVM

on in Symfony, Tips

Yesterday Asm89 blogged about a custom version of symfony edited to run on Facebook’s HHVM. I’ve followed his great blog post (you can read it here) in how to setup HHVM to run the Symfony standard web application.

After that I’ve tested the performances compared to a nginx + php-fpm configuration and I wanted to share the results here.

N.B. This is a benchmark of the symfony standard application just to have a starting point on how much the difference can be YMMV

Test setup

I’ve run the tests on a vagrant VM on a i5 3750k with assigned 4 cores. The box was running Ubuntu 12.04 64-bit. I’ve installed both HHVM and nginx + php-fpm on the same box running on different ports. Nginx and php-fpm was tuned following this article and php-fpm with max_children = 10.

Tests

I’ve used apache bench to test the default symfony webapp demo page using the production environment, all tests used 2000 requests with 1, 10, 50, 100 concurrent clients.

This is what I’ve got, enjoy.

Server - concurrency Requests per second Time per request (ms) 50% req served within (ms) 90% req served within (ms) 99% req served within (ms) 100% req served within (ms)
Nginx - 1 user 25.48 39.245 39 40 41 52
Nginx (APC) - 1 user 93.74 10.668 10 11 14 17
Nginx 5.5 (OPcache) - 1 user 135.01 7.407 7 8 10 21
HHVM - 1 user 105.79 9.453 9 10 13 16
Nginx - 10 user 78.90 126.736 125 177 220 334
Nginx (APC) - 10 user 259.74 38.492 37 56 79 106
Nginx 5.5 (OPcache) - 10 user 399.23 25.048 24 32 42 66
HHVM - 10 user 362.79 27.564 24 47 81 144
Nginx - 50 user 73.74 678.035 665 869 1079 1249
Nginx (APC) - 50 user 239.62 208.661 190 275 371 586
Nginx 5.5 (OPcache) - 50 user 378.44 132.120 131 146 163 182
HHVM - 50 user 369.74 135.231 126 203 319 898
Nginx - 100 user 76.66 1304.518 1292 1481 1714 1980
Nginx (APC) - 50 user 278.30 359.326 357 384 408 455
Nginx 5.5 (OPcache) - 100 user 385.82 259.185 254 292 316 336
HHVM - 100 user 364.71 274.191 269 332 389 494

As you can see HHVM is about 4 times faster then php so as soon it gets stable and more usable you should really consider looking into it.

As I’ve already said, this is just Symfony standard application, you can get worse or better results using HHVM on our own project, but for such a big performance improvement, it should be worth the work :)

Update: I’ve added the tests with APC enabled, it increased a lot compared to the plain PHP version, but still HHVM is faster.

Update 2: I’ve added also tests with PHP 5.5 using the Zend OPcache enabled, it seems they perform even better than HHVM! I’ve used both PHP and HHVM out of the box, without any config changes, if you think there is another test that I haven’t done drop a comment and I’ll try to run it!

Complexity heads up!

This is just an hello world example, as pointed out on Twitter by @chregu when more work has to be done by PHP, HHVM performs better than PHP 5.5 + OPcache.

As a simple test I’ve tried to run this function:

<?php

function fib($n)
{
    if ($n <= 2)
        return 1;
    else
        return fib($n-1) + fib($n-2);
}

$n = 32;
printf("fib(%d) = %d\n", $n, fib($n, 2));

on both HHVM and PHP 5.5 + OPcache (using only 500 requests, too much to wait with just PHP), these are the results:

Server - concurrency Requests per second Time per request (ms) 50% req served within (ms) 90% req served within (ms) 99% req served within (ms) 100% req served within (ms)
Nginx 5.5 (OPcache) - 1 user 2.12 472.811 472 477 491 493
HHVM - 1 user 47.30 21.140 20 21 41 66
Nginx 5.5 (OPcache) - 10 user 7.39 1352.937 1277 1598 1755 1755
HHVM - 10 user 182.67 54.743 53 81 109 134

So HHVM performs way better even compared to PHP 5.5 + OPcache when you’ve to do some hard work (who said Doctrine hydration?). So basically as I said before, you should test your application with HHVM (if it’s compatbile) to know if it’s worth the switch.

I have to remind you that these are just simple, stupid, tests, I’ll try to keep the post updated when real-life examples are done.