okami101/blog
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 46 Wiki Activity
Files
7cad7ec1bfeebe69b0f2c549f0937692732647e6
blog/content/posts/22-web-api-benchmarks-2024/index.md

128 KiB
Raw Blame History

title, date, tags
title date tags
A 2024 benchmark of main Web API frameworks 2023-12-26
kubernetes
docker
load-testing
k6
webapi

{{< lead >}} We'll be comparing the read performance of 6 Web APIs frameworks, sharing the same OpenAPI contract from realworld app, a medium-like clone, implemented under multiple languages (PHP, Python, Javascript, Java and C#). {{< /lead >}}

This is not a basic synthetic benchmark, but a real world benchmark with DB data tests, and multiple scenarios. This post may be updated when new versions of frameworks will be released or any suggestions for performance related improvement in below commentary section.

A state of the art of real world benchmarks comparison of Web APIs is difficult to achieve and very time-consuming as it forces to master each framework. As performance can highly dependent of:

  • Code implementation, all made by my own
  • Fine-tuning for each runtime, so I mostly take the default configuration

Now that's said, let's fight !

The contenders

We'll be using the very last up-to-date stable versions of the frameworks, and the latest stable version of the runtime.

Framework & Source code Runtime ORM Tested Database
Laravel 11 (api) PHP 8.3 Eloquent MySQL & PostgreSQL
Symfony 7 (api) PHP 8.3 Doctrine MySQL & PostgreSQL
FastAPI (api) Python 3.12 SQLAlchemy 2.0 PostgreSQL
NestJS 10 (api) Node 20 Prisma 5 PostgreSQL
Spring Boot 3.2 (api) Java 21 Hibernate 6 PostgreSQL
ASP.NET Core 8 (api) .NET 8.0 EF Core 8 PostgreSQL

Each project are:

  • Using the same OpenAPI contract
  • Fully tested and fonctional against same Postman collection
  • Highly tooled with high code quality in mind (static analyzers, formatter, linters, good code coverage, etc.)
  • Share roughly the same amount of DB datasets, 50 users, 500 articles, 5000 comments, generated by faker-like library for each language
  • Avoiding N+1 queries with eager loading (normally)
  • Containerized with Docker, and deployed on a monitored Docker Swarm cluster

Side note on PHP configuration

Note as I tested against PostgreSQL for all frameworks as main Database, but I added MySQL for Laravel and Symfony too (OPcache enabled), just by curiosity, and because of simplicity of PHP for switching database without changing code base, as both DB drivers integrated into base PHP Docker image. It allows to have an interesting Eloquent VS Doctrine ORM comparison for each database.

{{< alert >}} Update April 2024: I added worker tests via FrankenPHP. {{< /alert >}}

The target hardware

We'll running all Web APIs project on a Docker swarm cluster, where each node are composed of 2 dedicated CPUs for stable performance and 8 GB of RAM.

Traefik will be used as a reverse proxy, load balancing the requests to the replicas of each node.

{{< mermaid >}} flowchart TD client((k6)) client -- Port 80 443 --> traefik-01 subgraph manager-01 traefik-01{Traefik SSL} end subgraph worker-01 app-01([Conduit replica 1]) traefik-01 --> app-01 end subgraph worker-02 app-02([Conduit replica 2]) traefik-01 --> app-02 end subgraph storage-01 DB[(MySQL or PostgreSQL)] app-01 --> DB app-02 --> DB end {{< /mermaid >}}

The Swarm cluster is fully monitored with Prometheus and Grafana, allowing to get relevant performance result.

The scenarios

We'll be using k6 to run the tests, with constant-arrival-rate executor for progressive load testing, following 2 different scenarios :

  • Scenario 1 : fetch all articles, following the pagination
  • Scenario 2 : fetch all articles, calling each single article with slug, fetch associated comments for each article, and fetch profile of each related author

Duration of each scenario is 1 minute, with a 30 seconds graceful for finishing last started iterations. Results with one single test failures, i.e. any response status different than 200 or any response json error parsing, are not accepted.

The Iteration creation rate (rate / timeUnit) will be choosen in order to obtain the highest possible request rate, without any test failures.

Scenario 1

The interest of this scenario is to be very database intensive, by fetching all articles, authors, and favorites, following the pagination, with a couple of SQL queries. Note as each code implementation normally use eager loading to avoid N+1 queries, which can have high influence in this test.

import http from "k6/http";
import { check } from "k6";

export const options = {
    scenarios: {
        articles: {
            env: { CONDUIT_URL: '<framework_url>' },
            duration: '1m',
            executor: 'constant-arrival-rate',
            rate: '<rate>',
            timeUnit: '1s',
            preAllocatedVUs: 50,
        },
    },
};

export default function () {
    const apiUrl = `https://${__ENV.CONDUIT_URL}/api`;

    const limit = 10;
    let offset = 0;

    let articles = []

    do {
        const articlesResponse = http.get(`${apiUrl}/articles?limit=${limit}&offset=${offset}`);
        check(articlesResponse, {
            "status is 200": (r) => r.status == 200,
        });

        articles = articlesResponse.json().articles;

        offset += limit;
    }
    while (articles && articles.length >= limit);
}

Here the expected JSON response format:

{
    "articles": [
        {
            "title": "Laboriosam aliquid dolore sed dolore",
            "slug": "laboriosam-aliquid-dolore-sed-dolore",
            "description": "Rerum beatae est enim cum similique.",
            "body": "Voluptas maxime incidunt...",
            "createdAt": "2023-12-23T16:02:03.000000Z",
            "updatedAt": "2023-12-23T16:02:03.000000Z",
            "author": {
                "username": "Devin Swift III",
                "bio": "Nihil impedit totam....",
                "image": "https:\/\/randomuser.me\/api\/portraits\/men\/47.jpg",
                "following": false
            },
            "tagList": [
                "aut",
                "cumque"
            ],
            "favorited": false,
            "favoritesCount": 5
        }
    ],
    //...
    "articlesCount": 500
}

The expected pseudocode SQL queries to build this response:

SELECT * FROM articles LIMIT 10 OFFSET 0;
SELECT count(*) FROM articles;
SELECT * FROM users WHERE id IN (<articles.author_id...>);
SELECT * FROM article_tag WHERE article_id IN (<articles.id...>);
SELECT * FROM favorites WHERE article_id IN (<articles.id...>);

{{< alert >}} It can highly differ according to each ORM, as few of them can prefer to reduce the queries by using subselect, but it's a good approximation. {{< /alert >}}

Scenario 2

The interest of this scenario is to be mainly runtime intensive, by calling each endpoint of the API.

import http from "k6/http";
import { check } from "k6";

export const options = {
    scenarios: {
        articles: {
            env: { CONDUIT_URL: '<framework_url>' },
            duration: '1m',
            executor: 'constant-arrival-rate',
            rate: '<rate>',
            timeUnit: '1s',
            preAllocatedVUs: 50,
        },
    },
};

export default function () {
    const apiUrl = `https://${__ENV.CONDUIT_URL}.sw.okami101.io/api`;

    const limit = 10;
    let offset = 0;

    const tagsResponse = http.get(`${apiUrl}/tags`);
    check(tagsResponse, {
        "status is 200": (r) => r.status == 200,
    });

    let articles = []

    do {
        const articlesResponse = http.get(`${apiUrl}/articles?limit=${limit}&offset=${offset}`);
        check(articlesResponse, {
            "status is 200": (r) => r.status == 200,
        });

        articles = articlesResponse.json().articles;

        for (let i = 0; i < articles.length; i++) {
            const article = articles[i];
            const articleResponse = http.get(`${apiUrl}/articles/${article.slug}`);
            check(articleResponse, {
                "status is 200": (r) => r.status == 200,
            });

            const commentsResponse = http.get(`${apiUrl}/articles/${article.slug}/comments`);
            check(commentsResponse, {
                "status is 200": (r) => r.status == 200,
            });

            const authorsResponse = http.get(`${apiUrl}/profiles/${article.author.username}`);
            check(authorsResponse, {
                "status is 200": (r) => r.status == 200,
            });
        }
        offset += limit;
    }
    while (articles && articles.length >= limit);
}

The results

Laravel

Laravel MySQL scenario 1

Iteration creation rate = 5/s

checks.........................: 100.00% ✓ 8160       ✗ 0
data_received..................: 87 MB   1.2 MB/s
data_sent......................: 768 kB  11 kB/s
dropped_iterations.............: 140     1.976351/s
http_req_blocked...............: avg=213.72µs min=267ns   med=1.09µs   max=48.53ms  p(90)=1.62µs   p(95)=1.84µs
http_req_connecting............: avg=8.15µs   min=0s      med=0s       max=6.3ms    p(90)=0s       p(95)=0s
http_req_duration..............: avg=378.29ms min=11.56ms med=376.43ms max=1.2s     p(90)=562.76ms p(95)=628.13ms
  { expected_response:true }...: avg=378.29ms min=11.56ms med=376.43ms max=1.2s     p(90)=562.76ms p(95)=628.13ms
http_req_failed................: 0.00%   ✓ 0          ✗ 8160
http_req_receiving.............: avg=1.53ms   min=32.73µs med=601.76µs max=214.52ms p(90)=2.25ms   p(95)=5.06ms
http_req_sending...............: avg=209.07µs min=31.58µs med=128.42µs max=80.08ms  p(90)=217.25µs p(95)=293.37µs
http_req_tls_handshaking.......: avg=199.75µs min=0s      med=0s       max=46.99ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=376.54ms min=10.71ms med=374.89ms max=1.2s     p(90)=561.49ms p(95)=626.28ms
http_reqs......................: 8160    115.193022/s
iteration_duration.............: avg=19.35s   min=5.25s   med=20.28s   max=23.11s   p(90)=22s      p(95)=22.36s
iterations.....................: 160     2.258687/s
vus............................: 12      min=5        max=50
vus_max........................: 50      min=50       max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 59, 82, 103, 97, 104, 99, 107, 112, 116, 107, 143, 99, 133, 82, 119, 128, 124, 121, 131, 115, 113, 109, 121, 151, 98, 119, 118, 107, 112, 125, 126, 119, 114, 122, 126, 141, 98, 121, 128, 124, 104, 116, 121, 103, 113, 119, 117, 121, 125, 126, 125, 109, 123, 138, 104, 130, 114, 127, 114, 141, 103, 113, 127, 114, 123, 111, 109, 109, 111, 107, 70 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 5, 10, 15, 20, 25, 28, 33, 37, 42, 46, 49, 50, 50, 50, 50, 50, 50, 50, 49, 50, 50, 50, 49, 49, 50, 50, 48, 49, 50, 49, 48, 49, 50, 49, 50, 50, 50, 49, 50, 50, 50, 50, 48, 50, 49, 49, 49, 49, 50, 48, 48, 49, 50, 49, 49, 50, 50, 49, 50, 49, 49, 48, 46, 41, 39, 36, 32, 29, 19, 12 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 47, 80, 125, 165, 215, 237, 306, 294, 355, 364, 374, 393, 446, 495, 452, 399, 402, 414, 400, 388, 478, 442, 390, 369, 455, 411, 420, 391, 440, 455, 395, 390, 388, 457, 379, 378, 481, 411, 416, 372, 471, 400, 443, 471, 406, 470, 395, 422, 408, 380, 414, 421, 419, 384, 410, 403, 421, 410, 392, 413, 437, 413, 394, 355, 370, 326, 320, 274, 236, 159, 85 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.04, 0.18, 0.42, 0.41, 0.4, 0.42, 0.43, 0.42, 0.42, 0.42, 0.41, 0.4, 0.42, 0.4, 0.4, 0.26, 0.04, 0.04 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.04, 0.08, 0.08, 0.09, 0.08, 0.08, 0.09, 0.08, 0.08, 0.08, 0.08, 0.09, 0.09, 0.08, 0.06, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.02, 0.5, 0.93, 0.93, 0.93, 0.92, 0.93, 0.93, 0.92, 0.92, 0.93, 0.93, 0.94, 0.93, 0.92, 0.33, 0.02, 0.02 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.04, 0.06, 0.06, 0.07, 0.07, 0.06, 0.07, 0.07, 0.07, 0.07, 0.07, 0.06, 0.07, 0.06, 0.03, 0.01, 0.01 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

As expected here, database is the bottleneck. We'll get slow response time at full load (> 500ms).

Laravel MySQL scenario 2

Iteration creation rate = 1/2/s

checks.........................: 100.00% ✓ 24309      ✗ 0
data_received..................: 57 MB   628 kB/s
data_sent......................: 2.0 MB  22 kB/s
http_req_blocked...............: avg=33.87µs  min=222ns   med=1.09µs   max=62.31ms  p(90)=1.54µs   p(95)=1.71µs
http_req_connecting............: avg=2.26µs   min=0s      med=0s       max=15.76ms  p(90)=0s       p(95)=0s
http_req_duration..............: avg=74.01ms  min=8.8ms   med=54.64ms  max=821.77ms p(90)=157.96ms p(95)=194.87ms
  { expected_response:true }...: avg=74.01ms  min=8.8ms   med=54.64ms  max=821.77ms p(90)=157.96ms p(95)=194.87ms
http_req_failed................: 0.00%   ✓ 0          ✗ 24309
http_req_receiving.............: avg=1.58ms   min=19.96µs med=277.8µs  max=219.84ms p(90)=1.85ms   p(95)=7.33ms
http_req_sending...............: avg=190.85µs min=27.93µs med=120.41µs max=51.12ms  p(90)=204.95µs p(95)=267.48µs
http_req_tls_handshaking.......: avg=28.34µs  min=0s      med=0s       max=51.19ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=72.24ms  min=0s      med=53.36ms  max=821.49ms p(90)=154.74ms p(95)=190.84ms
http_reqs......................: 24309   270.071678/s
iteration_duration.............: avg=1m19s    min=1m18s   med=1m19s    max=1m19s    p(90)=1m19s    p(95)=1m19s
iterations.....................: 2       0.02222/s
vus............................: 28      min=1        max=30
vus_max........................: 50      min=50       max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 20, 32, 68, 122, 135, 126, 157, 171, 214, 212, 242, 260, 247, 272, 274, 271, 281, 272, 294, 286, 272, 272, 280, 302, 294, 263, 292, 278, 306, 299, 284, 291, 278, 298, 299, 283, 282, 293, 288, 293, 276, 287, 289, 296, 302, 284, 299, 284, 303, 304, 272, 280, 288, 303, 293, 270, 285, 283, 297, 304, 285, 274, 282, 303, 295, 284, 285, 291, 291, 292, 298, 290, 286, 301, 297, 280, 291, 272, 294, 298, 290, 279, 290, 311, 299, 272, 300, 280, 311, 300, 86 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13, 14, 14, 15, 15, 16, 16, 17, 17, 18, 18, 19, 19, 20, 20, 21, 21, 22, 22, 23, 23, 24, 24, 25, 25, 26, 26, 27, 27, 28, 28, 29, 29, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 29, 29, 29, 28, 28, 28, 28, 28, 28, 28, 28, 28 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 31, 30, 24, 16, 19, 23, 22, 23, 22, 23, 23, 23, 26, 26, 27, 29, 31, 33, 33, 35, 38, 40, 41, 40, 43, 47, 48, 49, 48, 50, 53, 57, 59, 56, 59, 60, 69, 65, 67, 67, 67, 80, 74, 74, 76, 74, 85, 82, 83, 83, 83, 99, 95, 88, 91, 100, 107, 103, 98, 98, 99, 112, 109, 97, 100, 102, 111, 104, 98, 105, 96, 110, 104, 98, 101, 99, 110, 110, 103, 97, 97, 104, 99, 91, 93, 91, 103, 100, 91, 91, 97 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.04, 0.04, 0.19, 0.41, 0.61, 0.7, 0.73, 0.73, 0.74, 0.75, 0.76, 0.76, 0.75, 0.73, 0.75, 0.73, 0.78, 0.75, 0.78 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.02, 0.04, 0.1, 0.13, 0.15, 0.16, 0.17, 0.17, 0.16, 0.17, 0.19, 0.17, 0.18, 0.17, 0.18, 0.17, 0.17, 0.18 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.03, 0.08, 0.16, 0.13, 0.14, 0.13, 0.14, 0.15, 0.16, 0.13, 0.16, 0.14, 0.15, 0.15, 0.15, 0.16, 0.16, 0.15 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.01, 0.06, 0.11, 0.1, 0.11, 0.1, 0.11, 0.11, 0.1, 0.1, 0.1, 0.09, 0.12, 0.1, 0.11, 0.12, 0.1, 0.1 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

Now we have a very runtime intensive scenario, with workers as bottleneck, database not very loaded, API is keeping up with a low response time (~100ms).

Laravel FrankenPHP MySQL scenario 1

Iteration creation rate = 5/s

checks.........................: 100.00% ✓ 7548      ✗ 0
data_received..................: 81 MB   1.1 MB/s
data_sent......................: 718 kB  9.4 kB/s
dropped_iterations.............: 153     2.011418/s
http_req_blocked...............: avg=227.08µs min=271ns   med=1.14µs   max=56.05ms  p(90)=1.69µs   p(95)=1.9µs
http_req_connecting............: avg=7.95µs   min=0s      med=0s       max=3.72ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=435.71ms min=5.62ms  med=205.37ms max=1.16s    p(90)=927.75ms p(95)=974.12ms
  { expected_response:true }...: avg=435.71ms min=5.62ms  med=205.37ms max=1.16s    p(90)=927.75ms p(95)=974.12ms
http_req_failed................: 0.00%   ✓ 0         ✗ 7548
http_req_receiving.............: avg=1.23ms   min=40.25µs med=700.17µs max=208.48ms p(90)=1.7ms    p(95)=2.67ms
http_req_sending...............: avg=231.75µs min=32.25µs med=132.92µs max=60.91ms  p(90)=228.32µs p(95)=316.55µs
http_req_tls_handshaking.......: avg=212.68µs min=0s      med=0s       max=54.71ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=434.24ms min=5.41ms  med=203.73ms max=1.16s    p(90)=925.77ms p(95)=972.64ms
http_reqs......................: 7548    99.229974/s
iteration_duration.............: avg=22.28s   min=3.08s   med=23.16s   max=29.88s   p(90)=27.92s   p(95)=28.86s
iterations.....................: 148     1.945686/s
vus............................: 3       min=3       max=50
vus_max........................: 50      min=50      max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 17, 102, 87, 98, 98, 112, 101, 96, 106, 96, 99, 104, 94, 91, 81, 89, 98, 111, 102, 100, 93, 107, 107, 94, 95, 102, 91, 106, 97, 102, 97, 101, 97, 108, 106, 89, 116, 104, 96, 99, 101, 95, 110, 91, 94, 106, 109, 91, 103, 99, 98, 111, 109, 112, 97, 92, 102, 104, 98, 97, 96, 98, 91, 99, 102, 97, 95, 102, 104, 108, 104, 96, 104 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 5, 10, 15, 19, 24, 28, 32, 36, 41, 46, 50, 50, 50, 50, 49, 49, 50, 50, 50, 50, 50, 50, 50, 49, 50, 50, 50, 50, 48, 49, 49, 50, 48, 49, 49, 50, 50, 50, 50, 50, 48, 50, 49, 50, 50, 49, 50, 50, 50, 50, 49, 49, 50, 48, 50, 50, 50, 49, 50, 49, 47, 46, 44, 43, 43, 42, 42, 41, 38, 36, 30, 29 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 30, 47, 110, 148, 179, 196, 270, 330, 282, 423, 421, 451, 503, 547, 586, 573, 526, 493, 464, 452, 593, 506, 448, 492, 497, 507, 531, 493, 483, 484, 516, 496, 465, 495, 465, 494, 477, 454, 525, 486, 498, 538, 479, 488, 531, 516, 440, 523, 516, 467, 542, 430, 455, 458, 471, 484, 525, 480, 509, 507, 511, 516, 510, 472, 433, 432, 483, 423, 369, 376, 369, 349, 284 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.04, 0.35, 0.35, 0.35, 0.36, 0.35, 0.35, 0.34, 0.36, 0.35, 0.36, 0.35, 0.35, 0.35, 0.37, 0.36, 0.1, 0.04 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.02, 0.07, 0.06, 0.06, 0.06, 0.06, 0.06, 0.05, 0.07, 0.06, 0.06, 0.07, 0.07, 0.06, 0.06, 0.06, 0.03, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.02, 0.11, 0.82, 0.8, 0.82, 0.81, 0.8, 0.79, 0.81, 0.83, 0.82, 0.81, 0.82, 0.83, 0.8, 0.8, 0.77, 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.03, 0.04, 0.05, 0.03, 0.04, 0.04, 0.04, 0.04, 0.04, 0.05, 0.05, 0.04, 0.04, 0.05, 0.05, 0.05, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

No surprise here, as we are database limited.

Laravel FrankenPHP MySQL scenario 2

Iteration creation rate = 1/s

checks.........................: 100.00% ✓ 60530      ✗ 0
data_received..................: 140 MB  1.6 MB/s
data_sent......................: 5.0 MB  56 kB/s
dropped_iterations.............: 4       0.044438/s
http_req_blocked...............: avg=21.5µs   min=191ns   med=895ns    max=52.03ms  p(90)=1.38µs   p(95)=1.56µs
http_req_connecting............: avg=1.46µs   min=0s      med=0s       max=10.41ms  p(90)=0s       p(95)=0s
http_req_duration..............: avg=50.82ms  min=3.01ms  med=46.99ms  max=323.69ms p(90)=94.36ms  p(95)=109.42ms
  { expected_response:true }...: avg=50.82ms  min=3.01ms  med=46.99ms  max=323.69ms p(90)=94.36ms  p(95)=109.42ms
http_req_failed................: 0.00%   ✓ 0          ✗ 60530
http_req_receiving.............: avg=700.42µs min=16.66µs med=199.85µs max=208.14ms p(90)=1.03ms   p(95)=1.92ms
http_req_sending...............: avg=179.82µs min=16.56µs med=99.19µs  max=122.8ms  p(90)=186.74µs p(95)=258.75µs
http_req_tls_handshaking.......: avg=18.4µs   min=0s      med=0s       max=47.62ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=49.94ms  min=0s      med=46.2ms   max=323.42ms p(90)=93.11ms  p(95)=107.84ms
http_reqs......................: 60530   672.458785/s
iteration_duration.............: avg=53.18s   min=30.44s  med=54.75s   max=1m15s    p(90)=1m13s    p(95)=1m14s
iterations.....................: 13      0.144424/s
vus............................: 43      min=1        max=50
vus_max........................: 50      min=50       max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 65, 149, 216, 476, 507, 578, 561, 625, 668, 647, 623, 611, 626, 666, 677, 707, 671, 665, 652, 686, 699, 711, 722, 667, 694, 698, 727, 712, 729, 677, 730, 750, 694, 736, 676, 700, 727, 704, 701, 712, 705, 646, 713, 734, 699, 747, 668, 714, 728, 721, 695, 741, 710, 664, 692, 752, 712, 707, 722, 692, 739, 648, 745, 616, 666, 686, 736, 704, 721, 716, 732, 686, 712, 710, 682, 719, 720, 723, 696, 690, 734, 724, 696, 670, 705, 698, 739, 706 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 30, 31, 32, 33, 33, 34, 35, 36, 37, 37, 38, 39, 40, 41, 42, 43, 44, 44, 45, 46, 47, 48, 49, 49, 50, 50, 50, 50, 50, 49, 49, 49, 49, 49, 49, 48, 48, 48, 48, 48, 48, 47, 47, 46, 46, 46, 46, 46, 46, 46, 45, 45, 45, 45, 44, 43 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 14, 13, 13, 8, 9, 10, 12, 13, 13, 15, 17, 19, 20, 21, 22, 22, 25, 26, 30, 29, 30, 30, 30, 37, 36, 37, 37, 39, 39, 44, 43, 41, 43, 44, 48, 49, 46, 48, 52, 52, 53, 57, 54, 56, 57, 56, 62, 60, 61, 62, 64, 65, 65, 74, 73, 67, 70, 69, 70, 73, 65, 77, 63, 79, 76, 72, 63, 68, 68, 67, 65, 68, 63, 69, 68, 64, 63, 62, 69, 66, 62, 62, 61, 69, 64, 61, 61, 61 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.04, 0.08, 0.44, 0.58, 0.61, 0.64, 0.67, 0.69, 0.7, 0.66, 0.67, 0.7, 0.68, 0.68, 0.66, 0.69, 0.66, 0.69, 0.66 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.03, 0.09, 0.13, 0.14, 0.16, 0.15, 0.16, 0.15, 0.15, 0.15, 0.15, 0.16, 0.17, 0.15, 0.16, 0.14, 0.16, 0.15 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.07, 0.2, 0.27, 0.3, 0.3, 0.31, 0.29, 0.3, 0.27, 0.32, 0.3, 0.31, 0.3, 0.3, 0.3, 0.33, 0.3, 0.32 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.04, 0.12, 0.14, 0.14, 0.13, 0.14, 0.14, 0.14, 0.14, 0.14, 0.14, 0.14, 0.15, 0.14, 0.13, 0.15, 0.13, 0.13 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

This is where Laravel Octane really shines, as it can handle more than twice requests than Apache.

Laravel PgSQL scenario 1

Iteration creation rate = 5/s

checks.........................: 100.00% ✓ 10710      ✗ 0
data_received..................: 115 MB  1.7 MB/s
data_sent......................: 980 kB  14 kB/s
dropped_iterations.............: 91      1.338864/s
http_req_blocked...............: avg=147.78µs min=207ns   med=1.14µs   max=46.93ms  p(90)=1.6µs    p(95)=1.79µs
http_req_connecting............: avg=8.16µs   min=0s      med=0s       max=11.33ms  p(90)=0s       p(95)=0s
http_req_duration..............: avg=267.09ms min=17.97ms med=234.63ms max=1.95s    p(90)=489.48ms p(95)=558.7ms
  { expected_response:true }...: avg=267.09ms min=17.97ms med=234.63ms max=1.95s    p(90)=489.48ms p(95)=558.7ms
http_req_failed................: 0.00%   ✓ 0          ✗ 10710
http_req_receiving.............: avg=6.26ms   min=34.64µs med=527.47µs max=301.14ms p(90)=18.09ms  p(95)=43.35ms
http_req_sending...............: avg=204.36µs min=14.62µs med=132.1µs  max=28.75ms  p(90)=232.5µs  p(95)=332.46µs
http_req_tls_handshaking.......: avg=136.14µs min=0s      med=0s       max=44.92ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=260.62ms min=17.09ms med=228.96ms max=1.95s    p(90)=476.98ms p(95)=544.88ms
http_reqs......................: 10710   157.574012/s
iteration_duration.............: avg=13.68s   min=2.62s   med=14.91s   max=18.1s    p(90)=16.51s   p(95)=17.12s
iterations.....................: 210     3.089687/s
vus............................: 16      min=5        max=50
vus_max........................: 50      min=50       max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 27, 129, 147, 150, 154, 154, 161, 162, 158, 154, 152, 163, 161, 157, 154, 160, 166, 158, 146, 158, 158, 171, 152, 166, 156, 164, 157, 161, 159, 154, 166, 167, 161, 155, 155, 162, 165, 162, 155, 157, 158, 161, 165, 155, 157, 158, 173, 154, 158, 158, 160, 172, 156, 155, 155, 170, 160, 159, 156, 160, 170, 166, 160, 159, 164, 165, 166, 157, 29 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 5, 10, 13, 17, 19, 23, 26, 30, 34, 37, 40, 44, 47, 49, 49, 50, 49, 50, 49, 50, 50, 49, 48, 50, 48, 47, 48, 49, 49, 48, 50, 50, 50, 48, 50, 50, 50, 50, 48, 47, 49, 50, 47, 46, 47, 48, 50, 49, 49, 50, 50, 50, 50, 48, 49, 49, 50, 49, 48, 46, 39, 36, 35, 33, 29, 23, 16 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 45, 46, 69, 96, 110, 131, 145, 168, 180, 227, 242, 249, 265, 282, 313, 313, 313, 296, 341, 329, 306, 307, 288, 306, 321, 314, 295, 286, 314, 317, 296, 304, 296, 314, 318, 304, 308, 309, 299, 329, 296, 294, 292, 326, 291, 291, 298, 290, 330, 307, 311, 299, 300, 292, 338, 305, 296, 294, 324, 323, 285, 238, 219, 226, 205, 170, 135, 91, 40 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.04, 0.11, 0.75, 0.82, 0.83, 0.79, 0.82, 0.82, 0.82, 0.82, 0.81, 0.84, 0.81, 0.81, 0.81, 0.29, 0.03, 0.04, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.03, 0.14, 0.16, 0.15, 0.16, 0.15, 0.15, 0.15, 0.15, 0.14, 0.15, 0.14, 0.15, 0.15, 0.06, 0.02, 0.03, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.11, 0.28, 0.27, 0.28, 0.27, 0.29, 0.27, 0.3, 0.29, 0.29, 0.27, 0.27, 0.27, 0.26, 0.04, 0.02, 0.02, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.14, 0.36, 0.4, 0.4, 0.4, 0.38, 0.41, 0.39, 0.38, 0.38, 0.37, 0.36, 0.36, 0.37, 0.04, 0.02, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

Laravel performs slightly better than MySQL in this scenario, and we are not limited by database this time.

Laravel PgSQL scenario 2

Iteration creation rate = 1/2/s

checks.........................: 100.00% ✓ 15649      ✗ 0
data_received..................: 36 MB   399 kB/s
data_sent......................: 1.3 MB  15 kB/s
http_req_blocked...............: avg=64.09µs  min=197ns   med=1.14µs   max=73.06ms  p(90)=1.6µs    p(95)=1.79µs
http_req_connecting............: avg=7.83µs   min=0s      med=0s       max=47.56ms  p(90)=0s       p(95)=0s
http_req_duration..............: avg=118.19ms min=16.33ms med=108.14ms max=506.31ms p(90)=208.01ms p(95)=241.86ms
  { expected_response:true }...: avg=118.19ms min=16.33ms med=108.14ms max=506.31ms p(90)=208.01ms p(95)=241.86ms
http_req_failed................: 0.00%   ✓ 0          ✗ 15649
http_req_receiving.............: avg=1.31ms   min=20.12µs med=294.79µs max=206.57ms p(90)=1.39ms   p(95)=4.84ms
http_req_sending...............: avg=208.28µs min=27.88µs med=129.12µs max=63.13ms  p(90)=231.66µs p(95)=320.73µs
http_req_tls_handshaking.......: avg=51.84µs  min=0s      med=0s       max=47.04ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=116.67ms min=14.17ms med=106.7ms  max=480.58ms p(90)=205.2ms  p(95)=239.05ms
http_reqs......................: 15649   173.850269/s
vus............................: 31      min=1        max=31
vus_max........................: 50      min=50       max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 19, 21, 54, 83, 89, 90, 130, 127, 135, 130, 151, 157, 159, 162, 166, 156, 155, 167, 176, 179, 179, 181, 172, 182, 187, 178, 182, 175, 183, 191, 173, 190, 181, 185, 178, 156, 182, 183, 192, 190, 185, 187, 186, 191, 195, 192, 188, 181, 190, 193, 190, 192, 183, 188, 203, 190, 186, 190, 190, 197, 180, 190, 190, 197, 188, 194, 190, 192, 191, 201, 193, 188, 198, 188, 197, 186, 197, 182, 189, 193, 183, 200, 183, 192, 192, 194, 190, 184, 194, 190 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 1, 1, 2, 2, 3, 3, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13, 14, 14, 15, 15, 16, 16, 17, 17, 18, 18, 19, 19, 20, 20, 21, 21, 22, 22, 23, 23, 24, 24, 25, 25, 26, 26, 27, 27, 28, 28, 29, 29, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 49, 45, 36, 24, 31, 33, 30, 30, 37, 37, 39, 38, 44, 42, 47, 50, 58, 54, 55, 56, 60, 62, 67, 67, 70, 70, 78, 79, 81, 79, 88, 87, 90, 95, 99, 109, 107, 98, 109, 106, 106, 117, 115, 118, 116, 116, 131, 120, 141, 131, 129, 139, 142, 148, 140, 142, 158, 152, 156, 153, 160, 172, 164, 161, 158, 159, 163, 162, 160, 158, 156, 167, 159, 159, 161, 163, 159, 166, 162, 159, 165, 164, 162, 166, 156, 159, 170, 159, 168, 158 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.05, 0.2, 0.41, 0.59, 0.66, 0.73, 0.76, 0.75, 0.76, 0.8, 0.8, 0.81, 0.82, 0.79, 0.81, 0.82, 0.83, 0.79 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.03, 0.04, 0.08, 0.11, 0.12, 0.14, 0.14, 0.15, 0.13, 0.14, 0.15, 0.14, 0.15, 0.15, 0.15, 0.15, 0.14, 0.16 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.02, 0.03, 0.14, 0.19, 0.25, 0.25, 0.27, 0.27, 0.27, 0.27, 0.29, 0.28, 0.28, 0.29, 0.29, 0.29, 0.28, 0.3, 0.31 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.03, 0.2, 0.27, 0.33, 0.34, 0.35, 0.36, 0.38, 0.39, 0.39, 0.4, 0.4, 0.39, 0.4, 0.41, 0.42, 0.42, 0.42 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

Laravel performing slower than MySQL in this context. Workers and databases are both heavy loaded, and we didn't complete a single scenario iteration.

Laravel FrankenPHP PgSQL scenario 1

Iteration creation rate = 5/s

checks.........................: 100.00% ✓ 15351     ✗ 0
data_received..................: 164 MB  2.7 MB/s
data_sent......................: 1.4 MB  22 kB/s
http_req_blocked...............: avg=92.69µs min=244ns    med=1.05µs   max=58.08ms  p(90)=1.51µs   p(95)=1.69µs
http_req_connecting............: avg=4.66µs  min=0s       med=0s       max=5.16ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=18.51ms min=4.14ms   med=16.47ms  max=428.74ms p(90)=25.07ms  p(95)=30.62ms
  { expected_response:true }...: avg=18.51ms min=4.14ms   med=16.47ms  max=428.74ms p(90)=25.07ms  p(95)=30.62ms
http_req_failed................: 0.00%   ✓ 0         ✗ 15351
http_req_receiving.............: avg=907µs   min=26.82µs  med=486.27µs max=207.67ms p(90)=1.42ms   p(95)=2.26ms
http_req_sending...............: avg=205.9µs min=27.86µs  med=123.52µs max=70.97ms  p(90)=214.68µs p(95)=314.53µs
http_req_tls_handshaking.......: avg=84.14µs min=0s       med=0s       max=43.29ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=17.4ms  min=0s       med=15.58ms  max=428.19ms p(90)=23.57ms  p(95)=28.49ms
http_reqs......................: 15351   251.56363/s
iteration_duration.............: avg=1s      min=777.67ms med=944.47ms max=1.71s    p(90)=1.25s    p(95)=1.37s
iterations.....................: 301     4.93262/s
vus............................: 1       min=1       max=8
vus_max........................: 50      min=50      max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 68, 243, 275, 268, 243, 277, 229, 261, 255, 263, 270, 245, 261, 258, 227, 281, 242, 238, 279, 257, 249, 248, 235, 281, 255, 259, 227, 239, 296, 256, 265, 248, 240, 276, 244, 266, 244, 273, 243, 256, 239, 213, 222, 285, 255, 297, 242, 249, 282, 264, 269, 235, 273, 253, 245, 265, 210, 281, 276, 242, 203, 11 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 5, 5, 5, 5, 4, 4, 5, 5, 5, 4, 4, 4, 4, 4, 5, 4, 4, 4, 4, 4, 4, 4, 5, 4, 4, 4, 6, 6, 4, 4, 3, 4, 5, 4, 4, 4, 5, 4, 4, 4, 5, 7, 8, 7, 6, 5, 5, 5, 5, 4, 4, 4, 4, 4, 4, 4, 6, 5, 4, 4, 1 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 22, 21, 19, 16, 17, 17, 19, 18, 17, 17, 16, 16, 17, 16, 20, 18, 17, 20, 18, 16, 18, 18, 22, 19, 17, 18, 20, 25, 19, 17, 16, 17, 19, 17, 17, 17, 17, 16, 18, 16, 19, 23, 29, 27, 27, 21, 18, 24, 19, 17, 16, 18, 16, 16, 17, 16, 21, 20, 16, 16, 16, 25 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.05, 0.03, 0.29, 0.45, 0.44, 0.44, 0.47, 0.45, 0.45, 0.46, 0.43, 0.5, 0.46, 0.45, 0.22, 0.03, 0.03, 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.03, 0.02, 0.06, 0.09, 0.08, 0.1, 0.08, 0.08, 0.09, 0.08, 0.07, 0.09, 0.1, 0.09, 0.05, 0.02, 0.02, 0.03, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.02, 0.18, 0.17, 0.19, 0.19, 0.2, 0.21, 0.18, 0.2, 0.23, 0.18, 0.17, 0.19, 0.07, 0.03, 0.03, 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.02, 0.12, 0.11, 0.1, 0.09, 0.13, 0.12, 0.11, 0.14, 0.14, 0.1, 0.1, 0.13, 0.04, 0.02, 0.02, 0.01, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

Way better than with MySQL but no CPU limitations...

Laravel FrankenPHP PgSQL scenario 2

Iteration creation rate = 1/s

checks.........................: 100.00% ✓ 53448      ✗ 0
data_received..................: 122 MB  1.4 MB/s
data_sent......................: 4.4 MB  49 kB/s
dropped_iterations.............: 8       0.088883/s
http_req_blocked...............: avg=28.83µs  min=188ns   med=900ns    max=130.65ms p(90)=1.4µs    p(95)=1.57µs
http_req_connecting............: avg=3.26µs   min=0s      med=0s       max=40.47ms  p(90)=0s       p(95)=0s
http_req_duration..............: avg=59.09ms  min=3.07ms  med=49.91ms  max=426.79ms p(90)=121.42ms p(95)=137.82ms
  { expected_response:true }...: avg=59.09ms  min=3.07ms  med=49.91ms  max=426.79ms p(90)=121.42ms p(95)=137.82ms
http_req_failed................: 0.00%   ✓ 0          ✗ 53448
http_req_receiving.............: avg=1.3ms    min=17.89µs med=202.25µs max=277.84ms p(90)=1.17ms   p(95)=2.63ms
http_req_sending...............: avg=271.85µs min=22.02µs med=101.15µs max=269.13ms p(90)=197.28µs p(95)=281.99µs
http_req_tls_handshaking.......: avg=23.83µs  min=0s      med=0s       max=89.85ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=57.51ms  min=0s      med=48.78ms  max=423.57ms p(90)=119.42ms p(95)=135.29ms
http_reqs......................: 53448   593.825611/s
iteration_duration.............: avg=1m3s     min=45.9s   med=1m4s     max=1m19s    p(90)=1m17s    p(95)=1m18s
iterations.....................: 9       0.099993/s
vus............................: 44      min=1        max=50
vus_max........................: 50      min=50       max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 59, 137, 245, 368, 393, 519, 334, 446, 490, 576, 526, 397, 379, 482, 597, 623, 652, 626, 631, 612, 631, 637, 608, 630, 651, 625, 637, 632, 671, 622, 683, 631, 655, 641, 632, 673, 659, 603, 650, 652, 642, 623, 668, 678, 642, 640, 498, 628, 638, 665, 624, 665, 630, 650, 669, 692, 628, 588, 650, 628, 687, 687, 639, 629, 632, 649, 657, 608, 669, 648, 658, 601, 618, 654, 654, 654, 521, 508, 317, 634, 685, 608, 662, 593, 675 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 1, 2, 3, 4, 5, 6, 8, 8, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 45, 46, 47, 47, 48, 49, 50, 50, 50, 50, 49, 50, 50, 50, 50, 49, 49, 49, 49, 49, 49, 49, 49, 48, 48, 48, 48, 47, 47, 47, 47, 47, 47, 47, 47, 46, 46, 45, 45 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 16, 14, 12, 11, 12, 11, 20, 18, 18, 17, 21, 29, 31, 30, 25, 25, 26, 27, 30, 33, 33, 34, 37, 38, 39, 41, 42, 43, 44, 48, 46, 48, 50, 53, 56, 54, 54, 61, 61, 62, 63, 66, 64, 63, 72, 71, 92, 71, 75, 73, 77, 74, 76, 79, 75, 70, 78, 82, 80, 81, 70, 71, 72, 81, 78, 75, 75, 78, 75, 75, 73, 77, 76, 75, 72, 72, 89, 89, 147, 74, 67, 77, 67, 80, 65 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.08, 0.37, 0.46, 0.51, 0.59, 0.58, 0.6, 0.62, 0.61, 0.6, 0.59, 0.6, 0.59, 0.58, 0.61, 0.58, 0.51, 0.64 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.03, 0.08, 0.11, 0.12, 0.13, 0.13, 0.14, 0.13, 0.16, 0.14, 0.14, 0.14, 0.14, 0.15, 0.16, 0.15, 0.12, 0.14 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.02, 0.08, 0.22, 0.22, 0.25, 0.25, 0.27, 0.26, 0.27, 0.26, 0.27, 0.26, 0.28, 0.27, 0.26, 0.26, 0.26, 0.26, 0.27 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.05, 0.13, 0.14, 0.16, 0.16, 0.16, 0.17, 0.16, 0.16, 0.16, 0.17, 0.16, 0.16, 0.15, 0.17, 0.15, 0.16, 0.18 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

Very close to MySQL equivalent.

Symfony

Symfony MySQL scenario 1

Iteration creation rate = 5/s

checks.........................: 100.00% ✓ 10098      ✗ 0
data_received..................: 91 MB   1.4 MB/s
data_sent......................: 929 kB  14 kB/s
dropped_iterations.............: 103     1.525577/s
http_req_blocked...............: avg=176.31µs min=222ns   med=1.09µs   max=75.92ms  p(90)=1.55µs   p(95)=1.74µs
http_req_connecting............: avg=6.31µs   min=0s      med=0s       max=4.4ms    p(90)=0s       p(95)=0s
http_req_duration..............: avg=283.48ms min=16.12ms med=299.04ms max=648.58ms p(90)=411.63ms p(95)=444.18ms
  { expected_response:true }...: avg=283.48ms min=16.12ms med=299.04ms max=648.58ms p(90)=411.63ms p(95)=444.18ms
http_req_failed................: 0.00%   ✓ 0          ✗ 10098
http_req_receiving.............: avg=1.03ms   min=22.06µs med=363.49µs max=204.94ms p(90)=1.63ms   p(95)=3.31ms
http_req_sending...............: avg=260µs    min=25.52µs med=124.22µs max=76.41ms  p(90)=223.35µs p(95)=316.73µs
http_req_tls_handshaking.......: avg=162.58µs min=0s      med=0s       max=49.02ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=282.18ms min=15.78ms med=297.55ms max=648.25ms p(90)=410.27ms p(95)=442.18ms
http_reqs......................: 10098   149.565835/s
iteration_duration.............: avg=14.51s   min=2.45s   med=16.39s   max=18.09s   p(90)=17.42s   p(95)=17.59s
iterations.....................: 198     2.932663/s
vus............................: 13      min=5        max=50
vus_max........................: 50      min=50       max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 10, 123, 159, 155, 163, 154, 149, 170, 145, 154, 150, 155, 152, 150, 153, 140, 156, 154, 149, 160, 140, 148, 151, 137, 166, 125, 150, 150, 145, 148, 135, 155, 136, 148, 152, 148, 160, 150, 146, 157, 148, 145, 158, 164, 144, 141, 153, 136, 157, 159, 158, 137, 147, 145, 161, 139, 144, 146, 150, 157, 150, 162, 135, 167, 148, 154, 162, 169, 14 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 5, 10, 13, 16, 20, 23, 26, 29, 32, 35, 39, 42, 46, 50, 49, 50, 49, 50, 50, 50, 50, 50, 49, 50, 47, 50, 48, 50, 50, 49, 49, 49, 50, 49, 49, 50, 50, 50, 49, 50, 49, 50, 50, 49, 46, 49, 49, 49, 49, 49, 50, 50, 49, 50, 50, 49, 48, 49, 48, 49, 46, 41, 35, 28, 22, 20, 13 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 43, 37, 59, 80, 94, 126, 140, 158, 189, 193, 231, 255, 266, 296, 327, 322, 346, 319, 325, 319, 337, 345, 335, 334, 306, 365, 346, 340, 332, 336, 351, 332, 337, 333, 336, 326, 322, 337, 327, 310, 336, 328, 327, 326, 326, 324, 340, 338, 326, 306, 339, 345, 332, 330, 324, 356, 335, 341, 330, 319, 336, 313, 262, 240, 210, 152, 128, 84, 28 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.06, 0.33, 0.33, 0.32, 0.31, 0.32, 0.31, 0.32, 0.32, 0.3, 0.31, 0.31, 0.31, 0.33, 0.13, 0.04, 0.03, 0.04 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.03, 0.16, 0.15, 0.16, 0.14, 0.14, 0.15, 0.14, 0.13, 0.16, 0.14, 0.15, 0.15, 0.16, 0.07, 0.02, 0.02, 0.03 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.02, 0.29, 0.91, 0.93, 0.94, 0.95, 0.95, 0.94, 0.95, 0.95, 0.95, 0.94, 0.95, 0.93, 0.93, 0.08, 0.03, 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.03, 0.06, 0.07, 0.06, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.06, 0.05, 0.06, 0.06, 0.02, 0.02, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

It's very similar to Laravel.

Symfony MySQL scenario 2

Iteration creation rate = 1/2/s

checks.........................: 100.00% ✓ 29892      ✗ 0
data_received..................: 54 MB   604 kB/s
data_sent......................: 2.4 MB  26 kB/s
http_req_blocked...............: avg=28.17µs  min=186ns   med=1.1µs    max=51.44ms  p(90)=1.55µs   p(95)=1.72µs
http_req_connecting............: avg=1.41µs   min=0s      med=0s       max=5.2ms    p(90)=0s       p(95)=0s
http_req_duration..............: avg=56.52ms  min=7.06ms  med=31.69ms  max=1.02s    p(90)=137.72ms p(95)=166.41ms
  { expected_response:true }...: avg=56.52ms  min=7.06ms  med=31.69ms  max=1.02s    p(90)=137.72ms p(95)=166.41ms
http_req_failed................: 0.00%   ✓ 0          ✗ 29892
http_req_receiving.............: avg=1.25ms   min=20.02µs med=209.22µs max=209.68ms p(90)=1.23ms   p(95)=5.37ms
http_req_sending...............: avg=189.98µs min=26.6µs  med=120.12µs max=70.73ms  p(90)=199.97µs p(95)=255.88µs
http_req_tls_handshaking.......: avg=24.33µs  min=0s      med=0s       max=49.8ms   p(90)=0s       p(95)=0s
http_req_waiting...............: avg=55.07ms  min=0s      med=30.73ms  max=1.02s    p(90)=134.97ms p(95)=163.13ms
http_reqs......................: 29892   332.049675/s
iteration_duration.............: avg=54.72s   min=42.96s  med=55.42s   max=1m9s     p(90)=1m6s     p(95)=1m8s
iterations.....................: 5       0.055542/s
vus............................: 26      min=1        max=28
vus_max........................: 50      min=50       max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 37, 44, 133, 154, 225, 230, 269, 273, 306, 292, 290, 272, 324, 339, 345, 342, 345, 351, 373, 354, 362, 318, 337, 348, 303, 301, 346, 370, 344, 335, 336, 355, 385, 373, 359, 351, 357, 382, 366, 350, 360, 347, 364, 342, 336, 362, 379, 375, 342, 348, 352, 352, 385, 360, 341, 357, 364, 363, 347, 314, 338, 359, 357, 344, 312, 344, 334, 390, 358, 339, 352, 359, 364, 335, 350, 368, 350, 378, 347, 357, 355, 331, 389, 343, 331, 342, 359, 369, 351, 321 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13, 14, 14, 15, 15, 16, 16, 17, 17, 18, 18, 19, 19, 20, 20, 21, 21, 21, 21, 22, 21, 22, 22, 23, 23, 24, 24, 25, 25, 26, 26, 27, 27, 28, 27, 28, 28, 28, 28, 28, 28, 28, 27, 27, 27, 27, 27, 27, 27, 27, 27, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26, 26 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 25, 22, 14, 13, 13, 13, 15, 14, 15, 17, 20, 22, 20, 22, 22, 23, 26, 25, 26, 27, 30, 34, 35, 34, 41, 43, 40, 38, 43, 43, 47, 47, 43, 44, 49, 53, 53, 49, 54, 55, 60, 61, 58, 63, 60, 63, 59, 55, 68, 64, 70, 70, 63, 70, 70, 78, 75, 72, 81, 81, 87, 80, 76, 81, 82, 89, 80, 73, 76, 74, 80, 78, 70, 82, 73, 78, 77, 69, 74, 71, 76, 76, 68, 76, 75, 78, 75, 69, 75, 74 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.04, 0.13, 0.33, 0.43, 0.5, 0.51, 0.5, 0.5, 0.54, 0.52, 0.51, 0.52, 0.53, 0.49, 0.52, 0.51, 0.52, 0.52 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.02, 0.08, 0.19, 0.22, 0.28, 0.27, 0.3, 0.3, 0.3, 0.29, 0.31, 0.29, 0.31, 0.29, 0.31, 0.28, 0.32, 0.31 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.03, 0.08, 0.15, 0.16, 0.13, 0.18, 0.16, 0.13, 0.12, 0.14, 0.13, 0.14, 0.13, 0.14, 0.12, 0.14, 0.13, 0.14 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.02, 0.04, 0.09, 0.09, 0.07, 0.1, 0.08, 0.07, 0.07, 0.07, 0.07, 0.06, 0.07, 0.07, 0.07, 0.07, 0.07, 0.08 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

Similar to Laravel too, just slightly better in the same context. Let's see if it's able to keep up with the same performance with PostgreSQL.

Symfony FrankenPHP MySQL scenario 1

Iteration creation rate = 5/s

checks.........................: 100.00% ✓ 11679     ✗ 0
data_received..................: 106 MB  1.6 MB/s
data_sent......................: 1.1 MB  16 kB/s
dropped_iterations.............: 72      1.058701/s
http_req_blocked...............: avg=127.87µs min=180ns   med=1.12µs   max=48.28ms  p(90)=1.59µs   p(95)=1.78µs
http_req_connecting............: avg=5.13µs   min=0s      med=0s       max=5.29ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=241.86ms min=10.69ms med=165.34ms max=935.06ms p(90)=495.74ms p(95)=513.79ms
  { expected_response:true }...: avg=241.86ms min=10.69ms med=165.34ms max=935.06ms p(90)=495.74ms p(95)=513.79ms
http_req_failed................: 0.00%   ✓ 0         ✗ 11679
http_req_receiving.............: avg=745.15µs min=27.68µs med=385.42µs max=207.74ms p(90)=1.02ms   p(95)=1.58ms
http_req_sending...............: avg=171.74µs min=27.68µs med=124.35µs max=23.65ms  p(90)=202.8µs  p(95)=258.4µs
http_req_tls_handshaking.......: avg=118.83µs min=0s      med=0s       max=47.04ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=240.95ms min=10.4ms  med=164.75ms max=934.19ms p(90)=495.14ms p(95)=513.11ms
http_reqs......................: 11679   171.73007/s
iteration_duration.............: avg=12.38s   min=1.78s   med=13.5s    max=17.82s   p(90)=15.61s   p(95)=16.21s
iterations.....................: 229     3.367256/s
vus............................: 1       min=1       max=50
vus_max........................: 50      min=50      max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 46, 172, 170, 177, 165, 173, 176, 168, 170, 162, 174, 171, 168, 177, 169, 178, 176, 170, 116, 174, 179, 180, 172, 181, 172, 180, 178, 171, 179, 168, 172, 177, 172, 175, 170, 181, 176, 172, 180, 170, 178, 182, 171, 179, 171, 176, 177, 167, 181, 170, 177, 175, 173, 179, 171, 177, 179, 172, 178, 172, 170, 179, 172, 179, 167, 174, 177, 165, 34 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 5, 9, 11, 14, 17, 20, 23, 25, 28, 31, 34, 37, 40, 42, 45, 49, 49, 50, 48, 50, 50, 49, 48, 48, 50, 49, 49, 47, 49, 44, 49, 50, 49, 50, 50, 49, 50, 49, 50, 48, 48, 50, 47, 47, 48, 49, 49, 49, 50, 49, 49, 48, 50, 49, 49, 48, 50, 50, 47, 47, 47, 43, 42, 35, 30, 23, 16, 1 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 29, 35, 55, 66, 89, 101, 115, 136, 149, 179, 183, 201, 222, 225, 253, 254, 274, 246, 454, 293, 279, 268, 290, 274, 286, 277, 277, 288, 276, 290, 260, 275, 280, 286, 282, 276, 278, 277, 273, 293, 266, 276, 296, 268, 277, 275, 279, 293, 286, 293, 274, 288, 280, 275, 288, 274, 275, 286, 269, 289, 269, 268, 251, 232, 214, 170, 131, 80, 25 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.1, 0.04, 0.25, 0.37, 0.39, 0.33, 0.38, 0.36, 0.36, 0.37, 0.32, 0.37, 0.39, 0.37, 0.36, 0.3, 0.04, 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.03, 0.02, 0.04, 0.05, 0.06, 0.04, 0.07, 0.06, 0.07, 0.07, 0.05, 0.07, 0.06, 0.07, 0.06, 0.05, 0.02, 0.02, 0.01 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.03, 0.33, 0.9, 0.87, 0.9, 0.88, 0.93, 0.94, 0.94, 0.94, 0.93, 0.94, 0.94, 0.94, 0.93, 0.15, 0.03, 0.02 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.02, 0.03, 0.03, 0.04, 0.03, 0.04, 0.04, 0.04, 0.04, 0.04, 0.04, 0.03, 0.05, 0.04, 0.04, 0.02, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

No enhancements here, it's even worse than Apache equivalent. No CPU limitations whatsoever, so strange behavior.

Symfony FrankenPHP MySQL scenario 2

Iteration creation rate = 1/s

checks.........................: 100.00% ✓ 94672       ✗ 0
data_received..................: 171 MB  2.1 MB/s
data_sent......................: 7.5 MB  94 kB/s
http_req_blocked...............: avg=14.88µs  min=182ns   med=643ns    max=93.7ms   p(90)=1.17µs   p(95)=1.34µs
http_req_connecting............: avg=945ns    min=0s      med=0s       max=9.67ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=14.55ms  min=1.89ms  med=10.7ms   max=318.64ms p(90)=27.5ms   p(95)=35.48ms
  { expected_response:true }...: avg=14.55ms  min=1.89ms  med=10.7ms   max=318.64ms p(90)=27.5ms   p(95)=35.48ms
http_req_failed................: 0.00%   ✓ 0           ✗ 94672
http_req_receiving.............: avg=895.52µs min=16.57µs med=242.05µs max=212.14ms p(90)=1.51ms   p(95)=2.88ms
http_req_sending...............: avg=172.15µs min=15.82µs med=77.99µs  max=151.35ms p(90)=163.13µs p(95)=228.85µs
http_req_tls_handshaking.......: avg=12.6µs   min=0s      med=0s       max=92.96ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=13.48ms  min=0s      med=9.93ms   max=317.4ms  p(90)=25.91ms  p(95)=32.93ms
http_reqs......................: 94672   1191.229807/s
iteration_duration.............: avg=22.99s   min=10.92s  med=23.68s   max=31.86s   p(90)=30.33s   p(95)=31.11s
iterations.....................: 61      0.767545/s
vus............................: 2       min=1         max=30
vus_max........................: 50      min=50        max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 77, 197, 295, 628, 880, 1008, 939, 1206, 1212, 1197, 1128, 1171, 1326, 1229, 778, 1325, 1201, 1321, 1315, 1316, 1314, 1217, 1359, 1275, 1335, 1364, 1199, 1418, 1350, 1368, 1327, 1253, 983, 1417, 1405, 1338, 1248, 1403, 1455, 1384, 823, 853, 1121, 815, 961, 1082, 1243, 1474, 1332, 1311, 1256, 555, 1016, 1241, 1055, 1218, 1259, 1447, 1492, 1485, 1487, 1289, 1481, 1403, 1506, 1399, 1250, 1433, 1460, 1157, 1386, 1231, 1471, 1318, 1377, 1332, 1127, 1233, 903, 229 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 10, 11, 11, 12, 12, 13, 13, 14, 14, 14, 15, 15, 16, 16, 16, 16, 17, 17, 17, 17, 18, 18, 19, 19, 19, 20, 20, 20, 20, 20, 21, 22, 22, 23, 23, 23, 24, 24, 26, 25, 25, 25, 26, 27, 27, 28, 29, 29, 30, 28, 27, 27, 26, 25, 24, 23, 23, 22, 21, 18, 18, 15, 14, 13, 10, 6, 2 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 9, 9, 9, 6, 5, 6, 7, 6, 7, 8, 9, 10, 8, 9, 14, 9, 10, 9, 10, 10, 11, 12, 11, 12, 12, 12, 13, 11, 12, 12, 13, 14, 18, 12, 13, 15, 15, 14, 13, 15, 24, 24, 18, 27, 23, 21, 19, 15, 18, 18, 19, 45, 24, 21, 24, 22, 21, 19, 19, 19, 20, 22, 18, 19, 17, 18, 19, 16, 16, 20, 15, 16, 12, 12, 10, 10, 10, 7, 5, 5 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.04, 0.31, 0.52, 0.53, 0.59, 0.6, 0.58, 0.59, 0.61, 0.47, 0.57, 0.48, 0.64, 0.64, 0.62, 0.61, 0.39, 0.04 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.01, 0.07, 0.11, 0.1, 0.11, 0.12, 0.12, 0.12, 0.13, 0.1, 0.14, 0.1, 0.13, 0.14, 0.13, 0.13, 0.08, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.04, 0.16, 0.3, 0.34, 0.35, 0.38, 0.39, 0.36, 0.38, 0.34, 0.38, 0.32, 0.39, 0.41, 0.39, 0.39, 0.32, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.02, 0.05, 0.06, 0.09, 0.09, 0.08, 0.08, 0.09, 0.08, 0.08, 0.09, 0.08, 0.1, 0.1, 0.09, 0.09, 0.07, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

Contrary to Laravel, Symfony is unable to get FrankenPHP really working, and is worst than Apache equivalent. Is it a bug from the runtime ?

Symfony PgSQL scenario 1

Iteration creation rate = 5/s

checks.........................: 100.00% ✓ 10608      ✗ 0
data_received..................: 96 MB   1.4 MB/s
data_sent......................: 971 kB  14 kB/s
dropped_iterations.............: 93      1.362698/s
http_req_blocked...............: avg=154.31µs min=248ns   med=1.07µs   max=60.23ms  p(90)=1.61µs   p(95)=1.86µs
http_req_connecting............: avg=5.19µs   min=0s      med=0s       max=3.83ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=273.06ms min=19.44ms med=291.33ms max=566.8ms  p(90)=362.38ms p(95)=388.91ms
{ expected_response:true }...: avg=273.06ms min=19.44ms med=291.33ms max=566.8ms  p(90)=362.38ms p(95)=388.91ms
http_req_failed................: 0.00%   ✓ 0          ✗ 10608
http_req_receiving.............: avg=743.59µs min=31.48µs med=265.43µs max=72ms     p(90)=757.9µs  p(95)=2.34ms
http_req_sending...............: avg=140.55µs min=33.65µs med=118.47µs max=5.54ms   p(90)=188.99µs p(95)=232.62µs
http_req_tls_handshaking.......: avg=144.54µs min=0s      med=0s       max=50.24ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=272.18ms min=19.24ms med=290.43ms max=566.36ms p(90)=361.47ms p(95)=387.77ms
http_reqs......................: 10608   155.435505/s
iteration_duration.............: avg=13.96s   min=2.62s   med=15.67s   max=16.83s   p(90)=16.36s   p(95)=16.48s
iterations.....................: 208     3.047755/s
vus............................: 9       min=5        max=50
vus_max........................: 50      min=50       max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 50, 137, 158, 159, 158, 144, 155, 160, 159, 155, 149, 158, 157, 165, 150, 155, 158, 159, 161, 159, 143, 163, 161, 162, 162, 144, 158, 165, 160, 159, 154, 153, 156, 165, 159, 146, 156, 158, 159, 158, 148, 154, 161, 161, 163, 142, 155, 163, 163, 155, 150, 158, 159, 160, 159, 149, 157, 163, 158, 159, 154, 154, 163, 163, 157, 152, 160, 159, 62 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 5, 10, 13, 16, 20, 23, 27, 30, 32, 36, 40, 43, 46, 49, 50, 50, 49, 50, 50, 49, 50, 50, 50, 50, 49, 49, 48, 49, 48, 49, 50, 50, 48, 50, 50, 49, 50, 49, 49, 50, 49, 48, 49, 48, 47, 49, 48, 50, 49, 50, 49, 50, 49, 50, 49, 48, 50, 48, 47, 49, 45, 39, 35, 32, 30, 25, 19, 9 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 36, 48, 72, 90, 109, 137, 156, 170, 196, 213, 240, 260, 272, 286, 309, 340, 310, 313, 307, 306, 329, 323, 310, 307, 307, 333, 313, 303, 302, 306, 325, 322, 310, 309, 307, 339, 312, 317, 310, 319, 332, 322, 306, 306, 296, 332, 316, 308, 306, 309, 331, 321, 315, 309, 306, 337, 316, 307, 309, 301, 324, 286, 245, 217, 197, 196, 151, 104, 57 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.08, 0.02, 0.17, 0.53, 0.54, 0.55, 0.55, 0.54, 0.54, 0.55, 0.54, 0.54, 0.54, 0.56, 0.55, 0.54, 0.15, 0.03, 0.02 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.03, 0.02, 0.05, 0.17, 0.17, 0.17, 0.15, 0.17, 0.16, 0.16, 0.16, 0.17, 0.17, 0.16, 0.16, 0.17, 0.05, 0.01, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.14, 0.61, 0.63, 0.64, 0.63, 0.64, 0.64, 0.64, 0.62, 0.65, 0.63, 0.64, 0.64, 0.64, 0.24, 0.03, 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.08, 0.33, 0.36, 0.36, 0.36, 0.36, 0.36, 0.36, 0.38, 0.35, 0.37, 0.36, 0.36, 0.35, 0.12, 0.02, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

Symfony performs same with PostgreSQL and MySQL, but we are limited by database contrary to Laravel case, and performing a little less.

Symfony PgSQL scenario 2

Iteration creation rate = 1/2/s

checks.........................: 100.00% ✓ 20817      ✗ 0
data_received..................: 38 MB   426 kB/s
data_sent......................: 1.7 MB  19 kB/s
http_req_blocked...............: avg=48.76µs  min=232ns   med=1.07µs   max=52.14ms p(90)=1.56µs   p(95)=1.78µs
http_req_connecting............: avg=2.93µs   min=0s      med=0s       max=23.78ms p(90)=0s       p(95)=0s
http_req_duration..............: avg=88.89ms  min=13.36ms med=81.73ms  max=1.74s   p(90)=157.48ms p(95)=179.93ms
{ expected_response:true }...: avg=88.89ms  min=13.36ms med=81.73ms  max=1.74s   p(90)=157.48ms p(95)=179.93ms
http_req_failed................: 0.00%   ✓ 0          ✗ 20817
http_req_receiving.............: avg=753.58µs min=22.88µs med=147.77µs max=69.37ms p(90)=574.84µs p(95)=2.86ms
http_req_sending...............: avg=139.52µs min=30.26µs med=120.37µs max=13.1ms  p(90)=191.03µs p(95)=225.35µs
http_req_tls_handshaking.......: avg=43.19µs  min=0s      med=0s       max=44.53ms p(90)=0s       p(95)=0s
http_req_waiting...............: avg=88ms     min=13.11ms med=80.82ms  max=1.74s   p(90)=156.15ms p(95)=178.32ms
http_reqs......................: 20817   231.261996/s
vus............................: 31      min=1        max=31
vus_max........................: 50      min=50       max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 14, 37, 66, 114, 147, 159, 161, 176, 202, 203, 207, 213, 202, 222, 226, 229, 226, 217, 236, 243, 242, 225, 226, 242, 248, 253, 235, 230, 246, 254, 249, 243, 225, 251, 251, 252, 221, 229, 247, 255, 250, 241, 238, 252, 255, 259, 242, 234, 252, 253, 253, 240, 238, 255, 250, 253, 246, 240, 250, 255, 254, 245, 240, 252, 258, 254, 234, 234, 252, 251, 254, 250, 236, 248, 256, 260, 244, 234, 248, 254, 253, 244, 246, 253, 252, 254, 244, 239, 252, 258, 84 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13, 14, 14, 15, 15, 16, 16, 17, 17, 18, 18, 19, 19, 20, 20, 21, 21, 22, 22, 23, 23, 24, 24, 25, 25, 26, 26, 27, 27, 28, 28, 29, 29, 30, 30, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31, 31 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 41, 27, 24, 17, 18, 19, 22, 23, 22, 24, 27, 28, 32, 31, 33, 35, 38, 41, 40, 41, 43, 49, 51, 49, 51, 51, 57, 61, 59, 58, 62, 65, 71, 70, 69, 70, 83, 82, 80, 78, 82, 86, 90, 87, 88, 90, 95, 102, 98, 99, 101, 106, 107, 109, 110, 111, 116, 119, 118, 119, 118, 127, 127, 125, 121, 121, 127, 128, 132, 122, 121, 124, 132, 125, 120, 120, 126, 127, 130, 121, 122, 126, 127, 123, 124, 122, 125, 130, 121, 122, 122 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.02, 0.02, 0.16, 0.44, 0.58, 0.63, 0.69, 0.71, 0.72, 0.74, 0.74, 0.73, 0.75, 0.73, 0.75, 0.75, 0.76, 0.76, 0.76 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.02, 0.07, 0.13, 0.18, 0.21, 0.21, 0.21, 0.22, 0.21, 0.22, 0.24, 0.22, 0.23, 0.23, 0.23, 0.22, 0.23, 0.23 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.07, 0.19, 0.23, 0.26, 0.27, 0.27, 0.29, 0.28, 0.31, 0.32, 0.31, 0.3, 0.31, 0.32, 0.31, 0.31, 0.3, 0.32 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.07, 0.26, 0.34, 0.38, 0.4, 0.42, 0.41, 0.43, 0.42, 0.41, 0.42, 0.44, 0.43, 0.41, 0.43, 0.43, 0.44, 0.42 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

Now it performs clearly slower than with MySQL in same scenario. Slightly better than Laravel in same context. To summary the 2nd scenario give MySQL a good advantage against PostgreSQL with PHP.

Symfony FrankenPHP PgSQL scenario 1

Iteration creation rate = 10/s

checks.........................: 100.00% ✓ 22287      ✗ 0
data_received..................: 201 MB  3.1 MB/s
data_sent......................: 1.9 MB  30 kB/s
dropped_iterations.............: 164     2.548995/s
http_req_blocked...............: avg=61.02µs  min=234ns   med=1.01µs   max=131.72ms p(90)=1.48µs   p(95)=1.68µs
http_req_connecting............: avg=6.79µs   min=0s      med=0s       max=62.5ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=130.52ms min=6.1ms   med=129.59ms max=404.9ms  p(90)=199.67ms p(95)=213.58ms
  { expected_response:true }...: avg=130.52ms min=6.1ms   med=129.59ms max=404.9ms  p(90)=199.67ms p(95)=213.58ms
http_req_failed................: 0.00%   ✓ 0          ✗ 22287
http_req_receiving.............: avg=880.47µs min=20.89µs med=297.79µs max=216.18ms p(90)=904.22µs p(95)=1.78ms
http_req_sending...............: avg=214.44µs min=29.19µs med=114.19µs max=92.18ms  p(90)=204.97µs p(95)=297.4µs
http_req_tls_handshaking.......: avg=50.13µs  min=0s      med=0s       max=54.56ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=129.43ms min=0s      med=128.65ms max=396.55ms p(90)=198.43ms p(95)=212.35ms
http_reqs......................: 22287   346.399123/s
iteration_duration.............: avg=6.7s     min=1.75s   med=7.06s    max=8.21s    p(90)=7.47s    p(95)=7.63s
iterations.....................: 437     6.79214/s
vus............................: 14      min=11       max=50
vus_max........................: 50      min=50       max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 122, 341, 348, 346, 359, 338, 361, 359, 345, 360, 330, 352, 361, 344, 359, 334, 362, 323, 346, 358, 335, 357, 352, 348, 358, 337, 359, 357, 345, 357, 339, 358, 350, 333, 355, 332, 359, 359, 349, 358, 333, 356, 359, 341, 352, 333, 355, 360, 344, 356, 340, 360, 360, 348, 355, 337, 346, 354, 345, 359, 337, 354, 341, 338, 179 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 11, 18, 23, 28, 34, 38, 44, 49, 49, 50, 50, 48, 49, 47, 48, 49, 49, 50, 48, 45, 50, 48, 49, 50, 50, 49, 49, 47, 46, 46, 50, 50, 48, 49, 47, 48, 47, 47, 50, 50, 48, 50, 48, 48, 48, 50, 50, 48, 50, 50, 47, 49, 49, 50, 48, 50, 47, 46, 48, 49, 47, 39, 31, 14 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 25, 37, 54, 69, 82, 104, 111, 128, 142, 136, 149, 141, 134, 142, 130, 142, 137, 154, 141, 134, 137, 136, 138, 141, 137, 146, 138, 136, 138, 133, 140, 137, 142, 148, 140, 144, 135, 134, 141, 137, 147, 138, 137, 141, 134, 151, 139, 137, 144, 138, 145, 136, 136, 143, 137, 146, 141, 136, 137, 134, 144, 127, 109, 85, 40 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.03, 0.28, 0.33, 0.34, 0.33, 0.34, 0.35, 0.34, 0.34, 0.35, 0.34, 0.33, 0.34, 0.32, 0.03, 0.04, 0.04, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.02, 0.05, 0.05, 0.06, 0.06, 0.05, 0.05, 0.06, 0.05, 0.05, 0.06, 0.05, 0.07, 0.05, 0.01, 0.02, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.03, 0.54, 0.93, 0.93, 0.91, 0.92, 0.93, 0.91, 0.92, 0.93, 0.93, 0.93, 0.92, 0.93, 0.21, 0.03, 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.02, 0.05, 0.07, 0.07, 0.07, 0.08, 0.07, 0.09, 0.08, 0.07, 0.07, 0.07, 0.08, 0.07, 0.03, 0.02, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

Symfony FrankenPHP PgSQL scenario 2

Iteration creation rate = 1/s

checks.........................: 100.00% ✓ 94672       ✗ 0
data_received..................: 171 MB  2.2 MB/s
data_sent......................: 7.5 MB  95 kB/s
http_req_blocked...............: avg=17.77µs  min=172ns   med=626ns    max=132.86ms p(90)=1.19µs   p(95)=1.36µs
http_req_connecting............: avg=2.07µs   min=0s      med=0s       max=62.8ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=14.36ms  min=1.87ms  med=10.33ms  max=358.31ms p(90)=26.54ms  p(95)=36.05ms
  { expected_response:true }...: avg=14.36ms  min=1.87ms  med=10.33ms  max=358.31ms p(90)=26.54ms  p(95)=36.05ms
http_req_failed................: 0.00%   ✓ 0           ✗ 94672
http_req_receiving.............: avg=946.99µs min=15.83µs med=251.63µs max=213.84ms p(90)=1.58ms   p(95)=3ms
http_req_sending...............: avg=180.14µs min=14.98µs med=75.49µs  max=199.41ms p(90)=161.75µs p(95)=230.84µs
http_req_tls_handshaking.......: avg=14.39µs  min=0s      med=0s       max=132.05ms p(90)=0s       p(95)=0s
http_req_waiting...............: avg=13.23ms  min=0s      med=9.58ms   max=278.53ms p(90)=24.63ms  p(95)=33.11ms
http_reqs......................: 94672   1199.909891/s
iteration_duration.............: avg=22.69s   min=10.8s   med=23.88s   max=30.11s   p(90)=29.54s   p(95)=29.85s
iterations.....................: 61      0.773138/s
vus............................: 5       min=1         max=29
vus_max........................: 50      min=50        max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 91, 207, 385, 748, 820, 959, 1075, 964, 1238, 1212, 1320, 1289, 1154, 1361, 1365, 1307, 1346, 1115, 1328, 1461, 1238, 1417, 1237, 953, 1350, 1272, 1331, 1155, 1105, 1017, 1196, 1108, 815, 824, 1327, 1224, 1334, 1213, 1410, 1430, 1365, 1394, 1299, 1165, 1427, 1419, 1341, 1234, 1418, 957, 1035, 1028, 929, 880, 1414, 1491, 1388, 1226, 1510, 1408, 1366, 1571, 1048, 1439, 1486, 1360, 1401, 1216, 1392, 1408, 1486, 1440, 1260, 1312, 1341, 1271, 1250, 964, 628, 4 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 10, 11, 11, 11, 11, 12, 12, 13, 13, 13, 13, 14, 14, 15, 15, 16, 15, 16, 16, 16, 17, 18, 19, 19, 20, 20, 20, 20, 21, 21, 22, 21, 22, 22, 23, 23, 24, 24, 25, 25, 25, 26, 27, 27, 27, 28, 28, 28, 28, 29, 27, 26, 26, 26, 23, 23, 22, 21, 20, 19, 18, 18, 15, 13, 10, 8, 5 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 8, 8, 7, 5, 6, 6, 6, 8, 7, 8, 8, 8, 9, 8, 8, 9, 8, 11, 9, 9, 11, 10, 11, 15, 11, 12, 12, 14, 14, 15, 14, 15, 21, 22, 14, 16, 15, 17, 14, 14, 15, 15, 17, 19, 16, 16, 18, 19, 17, 26, 24, 26, 24, 33, 19, 17, 20, 24, 19, 20, 21, 18, 25, 18, 17, 18, 16, 19, 15, 15, 13, 13, 14, 12, 10, 10, 8, 7, 5, 9 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.07, 0.37, 0.54, 0.58, 0.58, 0.55, 0.5, 0.52, 0.6, 0.6, 0.56, 0.55, 0.6, 0.6, 0.61, 0.58, 0.21, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.02, 0.07, 0.11, 0.12, 0.12, 0.12, 0.11, 0.12, 0.12, 0.12, 0.12, 0.12, 0.13, 0.13, 0.13, 0.11, 0.05, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.03, 0.19, 0.27, 0.33, 0.33, 0.3, 0.3, 0.3, 0.33, 0.32, 0.33, 0.3, 0.35, 0.34, 0.35, 0.34, 0.19, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.02, 0.07, 0.09, 0.1, 0.1, 0.1, 0.09, 0.09, 0.11, 0.1, 0.09, 0.11, 0.11, 0.11, 0.1, 0.11, 0.06, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

FastAPI

As a side note here, uvicorn is limited to 1 CPU core, so I use 2 replicas on each worker to use all CPU cores.

FastAPI PgSQL scenario 1

Iteration creation rate = 15/s

checks.........................: 100.00% ✓ 33048     ✗ 0
data_received..................: 272 MB  4.3 MB/s
data_sent......................: 2.9 MB  46 kB/s
dropped_iterations.............: 253     4.042284/s
http_req_blocked...............: avg=44.03µs  min=197ns   med=873ns    max=51.66ms  p(90)=1.31µs   p(95)=1.48µs
http_req_connecting............: avg=1.59µs   min=0s      med=0s       max=3.33ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=87.55ms  min=5.74ms  med=79.16ms  max=449.45ms p(90)=160.7ms  p(95)=187.45ms
{ expected_response:true }...: avg=87.55ms  min=5.74ms  med=79.16ms  max=449.45ms p(90)=160.7ms  p(95)=187.45ms
http_req_failed................: 0.00%   ✓ 0         ✗ 33048
http_req_receiving.............: avg=809.01µs min=18.38µs med=273.27µs max=53.15ms  p(90)=1.95ms   p(95)=3.12ms
http_req_sending...............: avg=156.85µs min=23.21µs med=95.41µs  max=45.6ms   p(90)=181.16µs p(95)=248.5µs
http_req_tls_handshaking.......: avg=40.32µs  min=0s      med=0s       max=44.77ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=86.59ms  min=0s      med=78.14ms  max=448.54ms p(90)=159.53ms p(95)=186.05ms
http_reqs......................: 33048   528.02138/s
iteration_duration.............: avg=4.49s    min=1.14s   med=4.65s    max=6.25s    p(90)=5.17s    p(95)=5.3s
iterations.....................: 648     10.35336/s
vus............................: 22      min=15      max=50
vus_max........................: 50      min=50      max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 270, 514, 530, 535, 552, 541, 524, 493, 545, 555, 560, 539, 519, 545, 540, 531, 525, 514, 547, 540, 537, 533, 485, 511, 534, 525, 508, 500, 550, 527, 538, 516, 500, 542, 532, 530, 504, 508, 540, 538, 553, 537, 497, 560, 517, 578, 559, 487, 551, 546, 538, 531, 517, 518, 578, 559, 521, 516, 556, 567, 517, 517, 351 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 15, 23, 31, 41, 46, 50, 50, 50, 48, 49, 50, 48, 48, 45, 49, 50, 49, 48, 50, 49, 50, 49, 49, 50, 50, 50, 50, 48, 50, 47, 49, 48, 48, 49, 49, 50, 48, 48, 50, 49, 49, 50, 48, 48, 49, 48, 45, 50, 48, 49, 49, 48, 47, 50, 49, 50, 48, 47, 50, 48, 39, 22 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 20, 33, 45, 65, 74, 88, 93, 96, 90, 89, 90, 92, 94, 91, 89, 91, 91, 95, 90, 93, 92, 93, 101, 93, 90, 94, 96, 101, 87, 92, 89, 96, 98, 91, 91, 93, 94, 97, 92, 90, 88, 91, 95, 91, 93, 86, 88, 94, 89, 91, 90, 92, 93, 92, 87, 86, 94, 90, 91, 86, 86, 63, 36 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.02, 0.52, 0.71, 0.74, 0.73, 0.71, 0.69, 0.71, 0.73, 0.73, 0.75, 0.71, 0.74, 0.5, 0.04, 0.03, 0.03, 0.02 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.01, 0.12, 0.15, 0.15, 0.15, 0.15, 0.17, 0.16, 0.15, 0.15, 0.15, 0.15, 0.15, 0.11, 0.02, 0.01, 0.01, 0.01 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.3, 0.46, 0.47, 0.47, 0.48, 0.47, 0.48, 0.49, 0.49, 0.49, 0.46, 0.49, 0.35, 0.03, 0.03, 0.03, 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.11, 0.25, 0.25, 0.3, 0.27, 0.27, 0.25, 0.29, 0.28, 0.28, 0.26, 0.29, 0.19, 0.02, 0.02, 0.02, 0.01, 0.01 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

Now we are talking, FastAPI outperforms above PHP frameworks, and database isn't the bottleneck anymore.

FastAPI PgSQL scenario 2

Iteration creation rate = 2/s

checks.........................: 100.00% ✓ 72075     ✗ 0
data_received..................: 147 MB  1.6 MB/s
data_sent......................: 5.4 MB  60 kB/s
dropped_iterations.............: 68      0.755514/s
http_req_blocked...............: avg=20.35µs  min=213ns    med=903ns    max=52.94ms p(90)=1.31µs   p(95)=1.48µs
http_req_connecting............: avg=909ns    min=0s       med=0s       max=10.34ms p(90)=0s       p(95)=0s
http_req_duration..............: avg=51.06ms  min=3.06ms   med=32.91ms  max=1.07s   p(90)=117.46ms p(95)=138.49ms
{ expected_response:true }...: avg=51.06ms  min=3.06ms   med=32.91ms  max=1.07s   p(90)=117.46ms p(95)=138.49ms
http_req_failed................: 0.00%   ✓ 0         ✗ 72075
http_req_receiving.............: avg=301.26µs min=17.32µs  med=125.14µs max=26.38ms p(90)=678.97µs p(95)=1.14ms
http_req_sending...............: avg=118.25µs min=21.81µs  med=94.18µs  max=20.22ms p(90)=163.96µs p(95)=204.06µs
http_req_tls_handshaking.......: avg=17.89µs  min=0s       med=0s       max=37.55ms p(90)=0s       p(95)=0s
http_req_waiting...............: avg=50.64ms  min=911.97µs med=32.54ms  max=1.07s   p(90)=116.83ms p(95)=137.77ms
http_reqs......................: 72075   800.78886/s
iteration_duration.............: avg=1m10s    min=51.94s   med=1m14s    max=1m21s   p(90)=1m20s    p(95)=1m21s
iterations.....................: 20      0.22221/s
vus............................: 33      min=2       max=50
vus_max........................: 50      min=50      max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 19, 168, 465, 631, 720, 792, 752, 758, 757, 763, 839, 819, 777, 770, 821, 828, 760, 792, 741, 869, 862, 831, 846, 811, 820, 878, 792, 811, 815, 829, 804, 807, 842, 819, 791, 804, 744, 839, 810, 828, 841, 890, 841, 834, 804, 829, 821, 837, 852, 853, 853, 884, 871, 773, 774, 825, 794, 832, 825, 787, 807, 872, 837, 815, 826, 778, 811, 810, 823, 807, 786, 872, 886, 810, 808, 831, 824, 853, 770, 818, 793, 827, 795, 813, 795, 858, 869, 805, 846, 823, 563 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 49, 50, 50, 50, 50, 49, 50, 50, 50, 49, 49, 49, 48, 48, 48, 47, 46, 46, 46, 46, 46, 46, 44, 44, 44, 44, 43, 43, 43, 42, 42, 40, 39, 39, 39, 37, 37, 36, 33 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 13, 12, 8, 9, 11, 12, 16, 18, 21, 23, 24, 27, 30, 34, 34, 36, 42, 42, 48, 44, 46, 51, 51, 56, 59, 56, 63, 59, 64, 59, 63, 62, 58, 61, 62, 62, 68, 60, 61, 60, 58, 57, 59, 59, 60, 61, 61, 58, 60, 58, 59, 56, 56, 65, 64, 58, 66, 61, 60, 62, 61, 57, 58, 61, 55, 62, 60, 59, 57, 57, 58, 53, 51, 56, 56, 53, 52, 52, 56, 51, 56, 51, 51, 52, 46, 48, 44, 46, 44, 43, 44 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.02, 0.02, 0.03, 0.54, 0.65, 0.68, 0.69, 0.72, 0.71, 0.7, 0.71, 0.71, 0.72, 0.69, 0.68, 0.72, 0.69, 0.71, 0.7 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.01, 0.01, 0.11, 0.18, 0.18, 0.16, 0.17, 0.17, 0.16, 0.18, 0.18, 0.18, 0.18, 0.18, 0.15, 0.18, 0.17, 0.17 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.03, 0.22, 0.22, 0.25, 0.28, 0.31, 0.29, 0.31, 0.3, 0.3, 0.32, 0.33, 0.3, 0.28, 0.3, 0.31, 0.3, 0.28 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.01, 0.11, 0.14, 0.18, 0.2, 0.23, 0.23, 0.22, 0.24, 0.26, 0.24, 0.25, 0.21, 0.2, 0.23, 0.24, 0.22, 0.19 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

FastAPI performs around at least twice better than PHP main frameworks in every situation. I'm not sure that testing it on MySQL change anything.

NestJS

NestJS PgSQL scenario 1

Iteration creation rate = 15/s

checks.........................: 100.00% ✓ 37434      ✗ 0
data_received..................: 648 MB  11 MB/s
data_sent......................: 3.5 MB  57 kB/s
dropped_iterations.............: 166     2.680206/s
http_req_blocked...............: avg=35.23µs  min=216ns   med=702ns    max=49.57ms  p(90)=1.19µs   p(95)=1.33µs
http_req_connecting............: avg=1.44µs   min=0s      med=0s       max=5.62ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=75.64ms  min=3.2ms   med=70.44ms  max=346.41ms p(90)=134.43ms p(95)=146.32ms
{ expected_response:true }...: avg=75.64ms  min=3.2ms   med=70.44ms  max=346.41ms p(90)=134.43ms p(95)=146.32ms
http_req_failed................: 0.00%   ✓ 0          ✗ 37434
http_req_receiving.............: avg=408.39µs min=19.38µs med=219.61µs max=42.89ms  p(90)=653.52µs p(95)=1.28ms
http_req_sending...............: avg=134.99µs min=18.84µs med=83.94µs  max=26.64ms  p(90)=156.84µs p(95)=222.84µs
http_req_tls_handshaking.......: avg=31.9µs   min=0s      med=0s       max=40.37ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=75.09ms  min=2.92ms  med=69.92ms  max=345.47ms p(90)=133.87ms p(95)=145.68ms
http_reqs......................: 37434   604.402491/s
iteration_duration.............: avg=3.89s    min=1.2s    med=4s       max=5.16s    p(90)=4.41s    p(95)=4.54s
iterations.....................: 734     11.851029/s
vus............................: 31      min=15       max=50
vus_max........................: 50      min=50       max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 273, 496, 577, 588, 599, 624, 621, 610, 641, 638, 641, 614, 585, 580, 600, 604, 601, 571, 606, 663, 643, 572, 596, 585, 616, 650, 680, 615, 612, 611, 617, 572, 587, 593, 605, 633, 633, 573, 646, 645, 650, 570, 629, 653, 691, 650, 580, 555, 590, 646, 565, 585, 638, 594, 567, 555, 602, 570, 641, 648, 601, 630, 8 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 15, 23, 32, 39, 44, 50, 49, 50, 46, 47, 49, 49, 48, 49, 49, 49, 49, 49, 50, 50, 49, 50, 48, 50, 49, 50, 49, 50, 46, 48, 46, 49, 50, 46, 47, 47, 45, 49, 47, 45, 49, 49, 49, 48, 45, 48, 50, 50, 49, 46, 45, 48, 50, 50, 49, 45, 48, 50, 49, 50, 31 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 21, 38, 45, 58, 65, 74, 79, 79, 75, 74, 75, 78, 85, 80, 80, 82, 81, 86, 78, 74, 76, 85, 81, 83, 77, 76, 72, 75, 78, 77, 81, 82, 83, 83, 79, 76, 76, 84, 75, 74, 73, 81, 81, 74, 68, 72, 81, 88, 82, 77, 75, 81, 76, 83, 86, 85, 74, 84, 80, 73, 74, 33, 6 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.02, 0.03, 0.27, 0.49, 0.46, 0.42, 0.45, 0.47, 0.42, 0.42, 0.46, 0.45, 0.42, 0.43, 0.35, 0.02, 0.02, 0.02, 0.02 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.01, 0.29, 0.48, 0.51, 0.55, 0.52, 0.49, 0.54, 0.54, 0.49, 0.51, 0.55, 0.53, 0.4, 0.01, 0.01, 0.01, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.04, 0.12, 0.21, 0.24, 0.21, 0.22, 0.22, 0.21, 0.21, 0.22, 0.23, 0.21, 0.21, 0.18, 0.03, 0.04, 0.03, 0.03, 0.04 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.08, 0.17, 0.16, 0.16, 0.15, 0.18, 0.17, 0.17, 0.17, 0.17, 0.17, 0.16, 0.16, 0.02, 0.02, 0.02, 0.01, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

It's slightly better than FastAPI, and database is strangely sleeping far more than with FastAPI, let's keep up on scenario 2.

NestJS PgSQL scenario 2

Iteration creation rate = 3/s

checks.........................: 99.93% ✓ 111672      ✗ 72
data_received..................: 530 MB 6.1 MB/s
data_sent......................: 8.9 MB 103 kB/s
dropped_iterations.............: 109    1.255685/s
http_req_blocked...............: avg=13.34µs  min=209ns   med=701ns    max=48.9ms  p(90)=1.15µs   p(95)=1.3µs
http_req_connecting............: avg=721ns    min=0s      med=0s       max=18.51ms p(90)=0s       p(95)=0s
http_req_duration..............: avg=29.19ms  min=1.38ms  med=26.19ms  max=247.1ms p(90)=53.2ms   p(95)=61.63ms
{ expected_response:true }...: avg=29.2ms   min=2.01ms  med=26.21ms  max=247.1ms p(90)=53.21ms  p(95)=61.65ms
http_req_failed................: 0.06%  ✓ 72          ✗ 111672
http_req_receiving.............: avg=501.09µs min=16.93µs med=189.85µs max=46.9ms  p(90)=1.11ms   p(95)=1.88ms
http_req_sending...............: avg=107.93µs min=13.6µs  med=76.9µs   max=27.43ms p(90)=147.26µs p(95)=188.72µs
http_req_tls_handshaking.......: avg=11.39µs  min=0s      med=0s       max=39.98ms p(90)=0s       p(95)=0s
http_req_waiting...............: avg=28.58ms  min=0s      med=25.61ms  max=246.9ms p(90)=52.33ms  p(95)=60.73ms
http_reqs......................: 111744 1287.295935/s
iteration_duration.............: avg=45.63s   min=26.8s   med=50.21s   max=55.45s  p(90)=54.19s   p(95)=54.62s
iterations.....................: 72     0.829443/s
vus............................: 8      min=3         max=50
vus_max........................: 50     min=50        max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 121, 508, 755, 899, 1012, 1007, 1156, 1177, 1096, 1180, 1224, 1237, 1208, 1244, 1295, 1437, 1445, 1345, 1338, 1405, 1378, 1380, 1411, 1293, 1420, 1441, 1451, 1365, 1264, 1439, 1384, 1584, 1241, 1361, 1319, 1427, 1398, 1362, 1320, 1448, 1482, 1458, 1311, 1256, 1399, 1363, 1345, 1259, 1346, 1443, 1499, 1445, 1438, 1451, 1425, 1472, 1479, 1367, 1322, 1450, 1414, 1360, 1355, 1457, 1326, 1411, 1363, 1350, 1277, 1279, 1168, 1216, 1198, 1256, 1314, 1248, 1236, 1192, 1183, 1227, 1263, 1357, 1148, 1141, 1168, 1127, 900, 25 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 3, 6, 9, 12, 15, 18, 21, 24, 27, 30, 33, 36, 39, 42, 45, 48, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 49, 50, 49, 50, 50, 49, 49, 50, 50, 50, 50, 50, 50, 49, 50, 49, 50, 49, 49, 47, 45, 42, 40, 38, 32, 28, 25, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 20, 20, 18, 16, 15, 11, 8 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 8, 8, 9, 11, 13, 16, 16, 18, 23, 23, 25, 27, 30, 32, 33, 32, 34, 37, 37, 35, 36, 36, 35, 38, 35, 35, 34, 36, 39, 34, 36, 31, 40, 37, 38, 35, 36, 36, 36, 35, 33, 34, 38, 39, 36, 36, 37, 40, 37, 34, 33, 34, 35, 34, 35, 34, 33, 37, 37, 35, 35, 35, 34, 30, 32, 28, 27, 23, 21, 19, 18, 18, 18, 17, 17, 17, 18, 18, 19, 18, 16, 14, 15, 14, 12, 9, 7, 6 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.02, 0.03, 0.34, 0.45, 0.47, 0.48, 0.46, 0.49, 0.47, 0.48, 0.46, 0.48, 0.51, 0.49, 0.46, 0.46, 0.43, 0.42, 0.41 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.01, 0.5, 0.53, 0.51, 0.5, 0.53, 0.49, 0.51, 0.51, 0.52, 0.51, 0.47, 0.49, 0.52, 0.52, 0.54, 0.57, 0.56 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.16, 0.23, 0.26, 0.29, 0.29, 0.29, 0.29, 0.28, 0.29, 0.27, 0.3, 0.3, 0.3, 0.27, 0.24, 0.24, 0.25, 0.1 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.08, 0.12, 0.13, 0.16, 0.14, 0.15, 0.16, 0.16, 0.15, 0.15, 0.16, 0.15, 0.15, 0.15, 0.14, 0.14, 0.13, 0.06 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

Huge gap now, NestJS is the clear winner so far. The native even loop system seems to make miracles. It's time to test it against compiled language.

Spring Boot

Spring Boot PgSQL scenario 1

Iteration creation rate = 30/s

checks.........................: 100.00% ✓ 91851       ✗ 0
data_received..................: 1.6 GB  26 MB/s
data_sent......................: 7.8 MB  129 kB/s
http_req_blocked...............: avg=16.33µs  min=191ns    med=419ns    max=71.26ms  p(90)=723ns    p(95)=925ns
http_req_connecting............: avg=978ns    min=0s       med=0s       max=19.89ms  p(90)=0s       p(95)=0s
http_req_duration..............: avg=14.04ms  min=2.37ms   med=12.32ms  max=223.11ms p(90)=24.19ms  p(95)=28.67ms
{ expected_response:true }...: avg=14.04ms  min=2.37ms   med=12.32ms  max=223.11ms p(90)=24.19ms  p(95)=28.67ms
http_req_failed................: 0.00%   ✓ 0           ✗ 91851
http_req_receiving.............: avg=1.76ms   min=19.18µs  med=758.76µs max=63.89ms  p(90)=4.48ms   p(95)=6.73ms
http_req_sending...............: avg=147.52µs min=21.29µs  med=51.49µs  max=43.07ms  p(90)=130.77µs p(95)=286.91µs
http_req_tls_handshaking.......: avg=14.4µs   min=0s       med=0s       max=47.91ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=12.12ms  min=0s       med=10.43ms  max=220.97ms p(90)=21.23ms  p(95)=24.99ms
http_reqs......................: 91851   1518.447027/s
iteration_duration.............: avg=741.16ms min=485.58ms med=732.29ms max=1.18s    p(90)=865.14ms p(95)=909.61ms
iterations.....................: 1801    29.773471/s
vus............................: 25      min=17        max=29
vus_max........................: 50      min=50        max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 353, 1407, 1575, 1522, 1483, 1562, 1587, 1578, 1491, 1521, 1545, 1561, 1523, 1493, 1392, 1604, 1609, 1526, 1554, 1493, 1547, 1558, 1531, 1484, 1511, 1530, 1606, 1548, 1479, 1459, 1574, 1582, 1575, 1481, 1439, 1615, 1304, 1567, 1571, 1530, 1610, 1604, 1516, 1523, 1433, 1630, 1503, 1532, 1557, 1492, 1559, 1577, 1521, 1497, 1446, 1583, 1566, 1509, 1424, 1514, 1385 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 20, 23, 24, 25, 25, 24, 20, 20, 22, 21, 19, 18, 21, 22, 26, 20, 21, 22, 20, 21, 20, 18, 20, 21, 21, 21, 18, 21, 21, 23, 22, 19, 19, 21, 22, 19, 29, 26, 27, 25, 22, 18, 21, 24, 24, 21, 21, 23, 21, 23, 20, 17, 19, 23, 22, 20, 21, 22, 25, 25 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 11, 14, 15, 15, 16, 16, 14, 13, 14, 15, 13, 12, 12, 14, 16, 15, 14, 13, 14, 14, 13, 13, 12, 14, 15, 14, 13, 12, 14, 15, 14, 13, 12, 13, 15, 14, 16, 18, 17, 17, 15, 13, 12, 14, 15, 14, 13, 13, 14, 14, 13, 12, 12, 13, 15, 14, 13, 13, 16, 17, 14 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.02, 0.02, 0.06, 0.29, 0.29, 0.29, 0.29, 0.28, 0.3, 0.27, 0.28, 0.28, 0.29, 0.29, 0.26, 0.03, 0.02, 0.02, 0.02 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.01, 0.04, 0.22, 0.22, 0.21, 0.21, 0.21, 0.21, 0.22, 0.22, 0.21, 0.21, 0.21, 0.21, 0.01, 0.01, 0.02, 0.01 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.05, 0.56, 0.6, 0.58, 0.59, 0.58, 0.58, 0.56, 0.59, 0.59, 0.59, 0.57, 0.57, 0.04, 0.03, 0.03, 0.04, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.03, 0.28, 0.29, 0.28, 0.28, 0.28, 0.3, 0.3, 0.27, 0.28, 0.27, 0.29, 0.3, 0.02, 0.02, 0.02, 0.01, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

End of debate, Spring Boot destroys competition for 1st scenario. Moreover, database is the bottleneck, and java runtime is clearly sleeping here. But JPA Hibernate was difficult to tune for optimal performance, and finally the magic @BatchSize annotation was the key, allowing to merge n+1 queries into 1+1 queries. Without it, Spring Boot was performing 3 times slower !

Spring Boot PgSQL scenario 2

Iteration creation rate = 10/s

checks.........................: 100.00% ✓ 225040     ✗ 0
data_received..................: 921 MB  11 MB/s
data_sent......................: 19 MB   235 kB/s
dropped_iterations.............: 456     5.549905/s
http_req_blocked...............: avg=6.76µs  min=202ns   med=389ns    max=52.58ms  p(90)=662ns   p(95)=859ns
http_req_connecting............: avg=275ns   min=0s      med=0s       max=8.36ms   p(90)=0s      p(95)=0s
http_req_duration..............: avg=16.7ms  min=1.6ms   med=13.62ms  max=237.91ms p(90)=32.02ms p(95)=39.52ms
{ expected_response:true }...: avg=16.7ms  min=1.6ms   med=13.62ms  max=237.91ms p(90)=32.02ms p(95)=39.52ms
http_req_failed................: 0.00%   ✓ 0          ✗ 225040
http_req_receiving.............: avg=1.91ms  min=16.76µs med=886.69µs max=211.62ms p(90)=4.52ms  p(95)=7.22ms
http_req_sending...............: avg=85.38µs min=17.86µs med=45.75µs  max=77.73ms  p(90)=86.49µs p(95)=120.1µs
http_req_tls_handshaking.......: avg=5.83µs  min=0s      med=0s       max=51.92ms  p(90)=0s      p(95)=0s
http_req_waiting...............: avg=14.71ms min=0s      med=11.67ms  max=222.78ms p(90)=28.88ms p(95)=36.06ms
http_reqs......................: 225040  2738.92674/s
iteration_duration.............: avg=26.14s  min=22.07s  med=26.88s   max=28.52s   p(90)=28.02s  p(95)=28.16s
iterations.....................: 145     1.764772/s
vus............................: 7       min=7        max=50
vus_max........................: 50      min=50       max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 33, 1444, 2295, 2566, 2449, 2563, 2918, 2891, 2886, 2676, 2674, 2943, 2975, 2971, 2757, 2645, 2966, 2967, 2925, 2622, 2678, 2888, 2984, 2972, 2667, 2639, 2896, 2974, 2853, 2760, 2747, 2897, 2952, 3026, 2633, 2569, 2890, 3020, 2701, 2521, 2680, 2922, 3013, 2983, 2674, 2622, 2790, 2463, 2693, 2352, 2853, 2762, 3000, 2960, 2653, 2615, 2987, 2870, 2875, 2536, 2593, 2903, 2990, 2712, 2550, 2532, 2903, 2955, 2683, 2485, 2626, 2930, 3004, 2858, 2664, 2596, 2937, 2942, 2899, 2517, 2436, 2577, 2012 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 10, 20, 30, 40, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 49, 50, 50, 48, 50, 50, 50, 50, 48, 49, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 49, 49, 49, 50, 50, 50, 50, 47, 48, 49, 46, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 43, 38, 37, 30, 23, 7 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 7, 5, 7, 10, 15, 18, 17, 17, 17, 19, 19, 17, 17, 17, 18, 19, 17, 17, 17, 19, 19, 17, 17, 17, 19, 19, 17, 17, 17, 18, 18, 17, 16, 16, 19, 19, 17, 16, 19, 20, 19, 17, 16, 17, 19, 19, 18, 20, 19, 21, 17, 18, 16, 17, 19, 19, 17, 17, 17, 19, 19, 16, 15, 16, 18, 18, 15, 15, 17, 18, 17, 16, 15, 15, 17, 17, 15, 15, 13, 15, 13, 10, 6 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.03, 0.08, 0.37, 0.41, 0.42, 0.41, 0.42, 0.43, 0.41, 0.41, 0.4, 0.38, 0.41, 0.4, 0.42, 0.39, 0.41, 0.41 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.01, 0.05, 0.26, 0.26, 0.27, 0.25, 0.27, 0.26, 0.26, 0.26, 0.26, 0.26, 0.26, 0.27, 0.27, 0.27, 0.26, 0.24 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.1, 0.52, 0.56, 0.57, 0.58, 0.57, 0.57, 0.55, 0.56, 0.56, 0.52, 0.57, 0.55, 0.52, 0.55, 0.56, 0.54, 0.12 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.04, 0.26, 0.3, 0.31, 0.28, 0.3, 0.3, 0.31, 0.3, 0.31, 0.28, 0.3, 0.3, 0.31, 0.3, 0.31, 0.29, 0.07 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

Java is maybe not the best DX experience for me, but it's a beast in terms of raw performance. Besides, we'll again have database bottleneck, which is the only case seen in this scenario on every framework tested ! Impossible to reach 100% java runtime CPU usage, even with 4 CPU cores, staying only at 60-70% overall...

ASP.NET Core

ASP.NET Core PgSQL scenario 1

Iteration creation rate = 20/s

checks.........................: 100.00% ✓ 59109      ✗ 0
data_received..................: 1.3 GB  22 MB/s
data_sent......................: 5.2 MB  86 kB/s
dropped_iterations.............: 42      0.685501/s
http_req_blocked...............: avg=25.19µs  min=212ns    med=505ns    max=57.73ms  p(90)=940ns    p(95)=1.13µs
http_req_connecting............: avg=1.4µs    min=0s       med=0s       max=19.45ms  p(90)=0s       p(95)=0s
http_req_duration..............: avg=43.01ms  min=2.75ms   med=36.68ms  max=278.96ms p(90)=83.54ms  p(95)=99.8ms
{ expected_response:true }...: avg=43.01ms  min=2.75ms   med=36.68ms  max=278.96ms p(90)=83.54ms  p(95)=99.8ms
http_req_failed................: 0.00%   ✓ 0          ✗ 59109
http_req_receiving.............: avg=1.6ms    min=16.85µs  med=545.28µs max=53.91ms  p(90)=4.05ms   p(95)=6.85ms
http_req_sending...............: avg=164.54µs min=13.27µs  med=62.93µs  max=35.42ms  p(90)=166.95µs p(95)=376.28µs
http_req_tls_handshaking.......: avg=22.43µs  min=0s       med=0s       max=52.29ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=41.24ms  min=0s       med=34.73ms  max=275.91ms p(90)=81.52ms  p(95)=97.35ms
http_reqs......................: 59109   964.745322/s
iteration_duration.............: avg=2.22s    min=759.35ms med=2.33s    max=3.24s    p(90)=2.74s    p(95)=2.85s
iterations.....................: 1159    18.916575/s
vus............................: 21      min=17       max=50
vus_max........................: 50      min=50       max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 242, 879, 870, 1010, 1007, 1026, 960, 941, 994, 1037, 1020, 990, 884, 997, 1001, 1022, 1005, 921, 941, 979, 984, 969, 891, 966, 978, 1018, 942, 902, 955, 998, 994, 1009, 907, 969, 991, 975, 999, 925, 960, 975, 999, 1001, 917, 967, 977, 1004, 1000, 919, 954, 1001, 992, 996, 936, 963, 999, 994, 944, 922, 958, 999, 1002, 632 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 17, 22, 27, 26, 27, 26, 30, 33, 32, 32, 31, 34, 36, 43, 42, 39, 42, 43, 46, 50, 49, 48, 49, 50, 46, 44, 47, 49, 49, 48, 50, 43, 50, 45, 43, 44, 47, 49, 47, 42, 48, 48, 49, 47, 49, 45, 45, 47, 48, 48, 49, 49, 49, 48, 49, 47, 46, 48, 45, 44, 21 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 12, 18, 23, 25, 25, 26, 28, 31, 31, 31, 31, 32, 38, 37, 39, 39, 39, 45, 45, 48, 49, 49, 53, 51, 49, 43, 47, 51, 50, 48, 46, 46, 50, 50, 44, 46, 45, 50, 49, 50, 44, 45, 50, 49, 48, 47, 47, 48, 50, 47, 46, 48, 51, 51, 48, 49, 49, 51, 49, 47, 41, 24 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.02, 0.03, 0.34, 0.47, 0.51, 0.5, 0.49, 0.48, 0.48, 0.49, 0.49, 0.48, 0.48, 0.49, 0.26, 0.03, 0.03, 0.02, 0.02 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.02, 0.13, 0.21, 0.18, 0.19, 0.19, 0.19, 0.2, 0.19, 0.19, 0.2, 0.2, 0.18, 0.1, 0.01, 0.01, 0.02, 0.01 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.04, 0.43, 0.76, 0.75, 0.77, 0.78, 0.79, 0.79, 0.78, 0.78, 0.77, 0.78, 0.77, 0.47, 0.03, 0.03, 0.03, 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.15, 0.2, 0.22, 0.23, 0.22, 0.2, 0.21, 0.21, 0.21, 0.22, 0.22, 0.23, 0.13, 0.01, 0.02, 0.01, 0.01, 0.01 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

ASP.NET Core is performing well here. EF Core is incredibly efficient by default without any tuning headaches.

ASP.NET Core PgSQL scenario 2

Iteration creation rate = 10/s

checks.........................: 100.00% ✓ 155200      ✗ 0
data_received..................: 939 MB  14 MB/s
data_sent......................: 14 MB   202 kB/s
dropped_iterations.............: 500     7.323443/s
http_req_blocked...............: avg=10.06µs min=181ns   med=418ns    max=74.82ms  p(90)=812ns    p(95)=997ns
http_req_connecting............: avg=437ns   min=0s      med=0s       max=9.65ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=20.44ms min=1.5ms   med=15.71ms  max=278.6ms  p(90)=41.79ms  p(95)=52.23ms
{ expected_response:true }...: avg=20.44ms min=1.5ms   med=15.71ms  max=278.6ms  p(90)=41.79ms  p(95)=52.23ms
http_req_failed................: 0.00%   ✓ 0           ✗ 155200
http_req_receiving.............: avg=1.52ms  min=14.25µs med=653.02µs max=205.85ms p(90)=3.58ms   p(95)=5.77ms
http_req_sending...............: avg=89.63µs min=12.82µs med=50.62µs  max=36.57ms  p(90)=102.45µs p(95)=140.6µs
http_req_tls_handshaking.......: avg=8.79µs  min=0s      med=0s       max=73.62ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=18.82ms min=0s      med=14.01ms  max=278.29ms p(90)=39.41ms  p(95)=49.76ms
http_reqs......................: 155200  2273.196786/s
iteration_duration.............: avg=31.96s  min=28.57s  med=32.03s   max=34.2s    p(90)=33.6s    p(95)=33.75s
iterations.....................: 100     1.464689/s
vus............................: 8       min=8         max=50
vus_max........................: 50      min=50        max=50

{{< tabs >}} {{< tab tabName="Req/s" >}}

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 143, 1375, 1715, 1720, 1816, 1913, 2037, 2110, 2211, 2142, 2381, 2448, 2417, 2323, 2194, 2387, 2478, 2279, 1817, 2181, 2463, 2412, 2454, 2260, 2247, 2405, 2495, 2530, 2173, 2184, 2480, 2476, 2423, 2146, 2209, 2382, 2513, 2344, 2336, 2266, 2513, 2446, 2496, 2382, 2214, 2371, 2477, 2444, 2269, 2210, 2439, 2498, 2450, 2349, 2195, 2448, 2508, 2406, 2316, 2219, 2528, 2503, 2470, 2285, 2179, 2424, 2374, 2224, 1278 ] } ] {{< /chart >}}

{{< /tab >}}

{{< tab tabName="Req duration" >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 10, 20, 30, 40, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 49, 50, 50, 49, 50, 47, 49, 48, 48, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 49, 48, 46, 45, 41, 36, 30, 8 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 5, 6, 11, 17, 21, 25, 24, 24, 22, 23, 21, 20, 21, 21, 23, 21, 20, 22, 27, 23, 20, 21, 20, 23, 22, 21, 20, 20, 22, 23, 20, 20, 20, 23, 22, 20, 19, 21, 20, 22, 20, 21, 20, 21, 23, 21, 20, 20, 22, 23, 20, 20, 20, 21, 23, 20, 20, 21, 21, 22, 20, 19, 19, 20, 20, 17, 15, 13, 9 ] } ] {{< /chart >}}

{{< /tab >}} {{< tab tabName="CPU load" >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.02, 0.04, 0.42, 0.54, 0.6, 0.6, 0.63, 0.63, 0.62, 0.63, 0.63, 0.64, 0.64, 0.62, 0.62, 0.31, 0.02, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.01, 0.03, 0.19, 0.25, 0.3, 0.3, 0.31, 0.31, 0.3, 0.31, 0.3, 0.31, 0.31, 0.31, 0.31, 0.16, 0.01, 0.01 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.03, 0.4, 0.49, 0.56, 0.55, 0.56, 0.56, 0.54, 0.55, 0.56, 0.55, 0.54, 0.55, 0.55, 0.31, 0.03, 0.04, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.02, 0.23, 0.3, 0.32, 0.29, 0.32, 0.32, 0.32, 0.34, 0.32, 0.34, 0.34, 0.33, 0.32, 0.18, 0.01, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< /tab >}} {{< /tabs >}}

Not that far to Java variant, just a bit behind. But as workers are fully loaded here, contrary to Spring Boot which is limited by database, Java stays by far the clear winner for raw performance (in sacrifice of some memory obviously).

Conclusion

Here are the final req/s results for each framework against PgSQL database.

{{< chart type="timeseries" title="Scenario 1" >}} [ { label: 'Laravel Octane', borderColor: '#c2410c', backgroundColor: '#c2410c', data: [ 68, 243, 275, 268, 243, 277, 229, 261, 255, 263, 270, 245, 261, 258, 227, 281, 242, 238, 279, 257, 249, 248, 235, 281, 255, 259, 227, 239, 296, 256, 265, 248, 240, 276, 244, 266, 244, 273, 243, 256, 239, 213, 222, 285, 255, 297, 242, 249, 282, 264, 269, 235, 273, 253, 245, 265, 210, 281, 276, 242, 203, 11 ] }, { label: 'Symfony', borderColor: '#ffffff', backgroundColor: '#ffffff', data: [ 122, 341, 348, 346, 359, 338, 361, 359, 345, 360, 330, 352, 361, 344, 359, 334, 362, 323, 346, 358, 335, 357, 352, 348, 358, 337, 359, 357, 345, 357, 339, 358, 350, 333, 355, 332, 359, 359, 349, 358, 333, 356, 359, 341, 352, 333, 355, 360, 344, 356, 340, 360, 360, 348, 355, 337, 346, 354, 345, 359, 337, 354, 341, 338, 179 ] }, { label: 'FastAPI', borderColor: '#0f766e', backgroundColor: '#0f766e', data: [ 270, 514, 530, 535, 552, 541, 524, 493, 545, 555, 560, 539, 519, 545, 540, 531, 525, 514, 547, 540, 537, 533, 485, 511, 534, 525, 508, 500, 550, 527, 538, 516, 500, 542, 532, 530, 504, 508, 540, 538, 553, 537, 497, 560, 517, 578, 559, 487, 551, 546, 538, 531, 517, 518, 578, 559, 521, 516, 556, 567, 517, 517, 351 ] }, { label: 'NestJS', borderColor: '#b91c1c', backgroundColor: '#b91c1c', data: [ 273, 496, 577, 588, 599, 624, 621, 610, 641, 638, 641, 614, 585, 580, 600, 604, 601, 571, 606, 663, 643, 572, 596, 585, 616, 650, 680, 615, 612, 611, 617, 572, 587, 593, 605, 633, 633, 573, 646, 645, 650, 570, 629, 653, 691, 650, 580, 555, 590, 646, 565, 585, 638, 594, 567, 555, 602, 570, 641, 648, 601, 630, 8 ] }, { label: 'Spring Boot', borderColor: '#15803d', backgroundColor: '#15803d', data: [ 353, 1407, 1575, 1522, 1483, 1562, 1587, 1578, 1491, 1521, 1545, 1561, 1523, 1493, 1392, 1604, 1609, 1526, 1554, 1493, 1547, 1558, 1531, 1484, 1511, 1530, 1606, 1548, 1479, 1459, 1574, 1582, 1575, 1481, 1439, 1615, 1304, 1567, 1571, 1530, 1610, 1604, 1516, 1523, 1433, 1630, 1503, 1532, 1557, 1492, 1559, 1577, 1521, 1497, 1446, 1583, 1566, 1509, 1424, 1514, 1385 ] }, { label: 'ASP.NET Core', borderColor: '#6d28d9', backgroundColor: '#6d28d9', data: [ 242, 879, 870, 1010, 1007, 1026, 960, 941, 994, 1037, 1020, 990, 884, 997, 1001, 1022, 1005, 921, 941, 979, 984, 969, 891, 966, 978, 1018, 942, 902, 955, 998, 994, 1009, 907, 969, 991, 975, 999, 925, 960, 975, 999, 1001, 917, 967, 977, 1004, 1000, 919, 954, 1001, 992, 996, 936, 963, 999, 994, 944, 922, 958, 999, 1002, 632 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Scenario 2" >}} [ { label: 'Laravel Octane', borderColor: '#c2410c', backgroundColor: '#c2410c', data: [ 59, 137, 245, 368, 393, 519, 334, 446, 490, 576, 526, 397, 379, 482, 597, 623, 652, 626, 631, 612, 631, 637, 608, 630, 651, 625, 637, 632, 671, 622, 683, 631, 655, 641, 632, 673, 659, 603, 650, 652, 642, 623, 668, 678, 642, 640, 498, 628, 638, 665, 624, 665, 630, 650, 669, 692, 628, 588, 650, 628, 687, 687, 639, 629, 632, 649, 657, 608, 669, 648, 658, 601, 618, 654, 654, 654, 521, 508, 317, 634, 685, 608, 662, 593, 675 ] }, { label: 'Symfony', borderColor: '#ffffff', backgroundColor: '#ffffff', data: [ 91, 207, 385, 748, 820, 959, 1075, 964, 1238, 1212, 1320, 1289, 1154, 1361, 1365, 1307, 1346, 1115, 1328, 1461, 1238, 1417, 1237, 953, 1350, 1272, 1331, 1155, 1105, 1017, 1196, 1108, 815, 824, 1327, 1224, 1334, 1213, 1410, 1430, 1365, 1394, 1299, 1165, 1427, 1419, 1341, 1234, 1418, 957, 1035, 1028, 929, 880, 1414, 1491, 1388, 1226, 1510, 1408, 1366, 1571, 1048, 1439, 1486, 1360, 1401, 1216, 1392, 1408, 1486, 1440, 1260, 1312, 1341, 1271, 1250, 964, 628, 4 ] }, { label: 'FastAPI', borderColor: '#0f766e', backgroundColor: '#0f766e', data: [ 19, 168, 465, 631, 720, 792, 752, 758, 757, 763, 839, 819, 777, 770, 821, 828, 760, 792, 741, 869, 862, 831, 846, 811, 820, 878, 792, 811, 815, 829, 804, 807, 842, 819, 791, 804, 744, 839, 810, 828, 841, 890, 841, 834, 804, 829, 821, 837, 852, 853, 853, 884, 871, 773, 774, 825, 794, 832, 825, 787, 807, 872, 837, 815, 826, 778, 811, 810, 823, 807, 786, 872, 886, 810, 808, 831, 824, 853, 770, 818, 793, 827, 795, 813, 795, 858, 869, 805, 846, 823, 563 ] }, { label: 'NestJS', borderColor: '#b91c1c', backgroundColor: '#b91c1c', data: [ 121, 508, 755, 899, 1012, 1007, 1156, 1177, 1096, 1180, 1224, 1237, 1208, 1244, 1295, 1437, 1445, 1345, 1338, 1405, 1378, 1380, 1411, 1293, 1420, 1441, 1451, 1365, 1264, 1439, 1384, 1584, 1241, 1361, 1319, 1427, 1398, 1362, 1320, 1448, 1482, 1458, 1311, 1256, 1399, 1363, 1345, 1259, 1346, 1443, 1499, 1445, 1438, 1451, 1425, 1472, 1479, 1367, 1322, 1450, 1414, 1360, 1355, 1457, 1326, 1411, 1363, 1350, 1277, 1279, 1168, 1216, 1198, 1256, 1314, 1248, 1236, 1192, 1183, 1227, 1263, 1357, 1148, 1141, 1168, 1127, 900, 25 ] }, { label: 'Spring Boot', borderColor: '#15803d', backgroundColor: '#15803d', data: [ 33, 1444, 2295, 2566, 2449, 2563, 2918, 2891, 2886, 2676, 2674, 2943, 2975, 2971, 2757, 2645, 2966, 2967, 2925, 2622, 2678, 2888, 2984, 2972, 2667, 2639, 2896, 2974, 2853, 2760, 2747, 2897, 2952, 3026, 2633, 2569, 2890, 3020, 2701, 2521, 2680, 2922, 3013, 2983, 2674, 2622, 2790, 2463, 2693, 2352, 2853, 2762, 3000, 2960, 2653, 2615, 2987, 2870, 2875, 2536, 2593, 2903, 2990, 2712, 2550, 2532, 2903, 2955, 2683, 2485, 2626, 2930, 3004, 2858, 2664, 2596, 2937, 2942, 2899, 2517, 2436, 2577, 2012 ] }, { label: 'ASP.NET Core', borderColor: '#6d28d9', backgroundColor: '#6d28d9', data: [ 143, 1375, 1715, 1720, 1816, 1913, 2037, 2110, 2211, 2142, 2381, 2448, 2417, 2323, 2194, 2387, 2478, 2279, 1817, 2181, 2463, 2412, 2454, 2260, 2247, 2405, 2495, 2530, 2173, 2184, 2480, 2476, 2423, 2146, 2209, 2382, 2513, 2344, 2336, 2266, 2513, 2446, 2496, 2382, 2214, 2371, 2477, 2444, 2269, 2210, 2439, 2498, 2450, 2349, 2195, 2448, 2508, 2406, 2316, 2219, 2528, 2503, 2470, 2285, 2179, 2424, 2374, 2224, 1278 ] } ] {{< /chart >}}

To resume, compiled languages have always a clear advantage when it comes to raw performance. But do you really need it ?

Keep in mind that it shouldn't be the only criteria to choose a web framework. The DX is also very important, for exemple Laravel stays a very nice candidate in this regard.

In PHP, Laravel Octane seems the be the most performant solution, by skipping the framework bootstrap most of the time.

When it comes to compiled languages, I still personally prefer ASP.NET Core over Spring Boot because of the DX. The performance gap is negligible, and it hasn't this warmup Java feeling and keeps a raisonable memory footprint.

I'm stay open to any suggestions to improve my tests, especially on PHP side. If you have any tips to improve performance by some Framework or PHP low level tuning, let me a comment below !