blog/index.md at 8df54d049597fc36fe01053e45b8bbf446482bd7

okami101/blog

Fork 0

Files

Adrien Beaudouin a17c02f82f ccl

2024-04-03 21:43:02 +02:00

100 KiB

Raw Blame History

title, date, tags

title

date

The contenders

We'll be using the very last up-to-date stable versions of each frameworks, and the latest stable version of the runtime.

I give you all source code as well as public OCI artifacts of each project, so you can test it by yourself quickly.

Framework & Source code	Runtime	ORM	Database
Laravel 11 (api / image)	FrankenPHP 8.3	Eloquent	MySQL & PostgreSQL
Symfony 7 (api / image)	FrankenPHP 8.3	Doctrine	MySQL & PostgreSQL
FastAPI (api / image)	Python 3.12	SQLAlchemy 2.0	PostgreSQL
NestJS 10 (api / image)	Node 20	Prisma 5	PostgreSQL
Spring Boot 3.2 (api / image)	Java 21	Hibernate 6	PostgreSQL
ASP.NET Core 8 (api / image)	.NET 8.0	EF Core 8	PostgreSQL

Each project are:

Using the same OpenAPI contract
Fully tested and fonctional against same Postman collection
Highly tooled with high code quality in mind (static analyzers, formatter, linters, good code coverage, etc.)
Share roughly the same amount of DB datasets, 50 users, 500 articles, 5000 comments, generated by faker-like library for each language
Avoiding N+1 queries with eager loading (normally)
Containerized with Docker, and deployed on a monitored Docker Swarm cluster

Side note on PHP configuration

Even if PostgreSQL is the mainly tested database, I added MySQL for PHP frameworks, because of simplicity of PHP for switching database without changing binaries, as both DB drivers integrated into base PHP Docker image. It allows to have an interesting Eloquent VS Doctrine ORM comparison for each database.

The Swarm cluster for testing

We'll running all Web APIs project on a Docker swarm cluster, where each node are composed of 2 dedicated CPUs for stable performance and 8 GB of RAM. I'll use 4 CCX13 instances from Hetzner.

Traefik will be used as a reverse proxy, load balancing the requests to the replicas of each node.

{{< mermaid >}} flowchart TD client((k6)) client -- Port 80 443 --> traefik-01 subgraph manager-01 traefik-01{Traefik SSL} end subgraph worker-01 app-01([Conduit replica 1]) traefik-01 --> app-01 end subgraph worker-02 app-02([Conduit replica 2]) traefik-01 --> app-02 end subgraph storage-01 DB[(MySQL or PostgreSQL)] app-01 --> DB app-02 --> DB end {{< /mermaid >}}

The Swarm cluster is fully monitored with Prometheus and Grafana, allowing to get relevant performance result.

Here is the complete terraform swarm bootstrap if you want to reproduce the same setup.

The deployment configuration

Following is the deployment configuration for each framework, using Docker Swarm stack file.

version: "3.8"

services:
  app:
    image: gitea.okami101.io/conduit/laravel:latest
    environment:
      - SERVER_NAME=:80
      - APP_KEY=base64:nltxnFb9OaSAr4QcCchy8dG1QXUbc2+2tsXpzN9+ovg=
      - DB_CONNECTION=pgsql
      - DB_HOST=postgres_db
      # - DB_CONNECTION=mysql
      # - DB_HOST=mysql_db
      - DB_USERNAME=okami
      - DB_PASSWORD=okami
      - DB_DATABASE=conduit_laravel
      - JWT_SECRET_KEY=c2b344e1-1a20-47fc-9aef-55b0c0d568a7
    entrypoint: php artisan octane:frankenphp
    networks:
      - postgres_db
      - mysql_db
      - traefik_public
    deploy:
      labels:
        - traefik.enable=true
        - traefik.http.routers.laravel.entrypoints=websecure
        - traefik.http.services.laravel.loadbalancer.server.port=8000
      replicas: 2
      placement:
        max_replicas_per_node: 1
        constraints:
          - node.labels.run == true

networks:
  postgres_db:
    external: true
  mysql_db:
    external: true
  traefik_public:
    external: true

version: "3.8"

services:
  app:
    image: gitea.okami101.io/conduit/symfony:latest
    environment:
      - SERVER_NAME=:80
      - APP_SECRET=ede04f29dd6c8b0e404581d48c36ec73
      - DATABASE_DRIVER=pdo_pgsql
      - DATABASE_URL=postgresql://okami:okami@postgres_db/conduit_symfony
      - DATABASE_RO_URL=postgresql://okami:okami@postgres_db/conduit_symfony
      # - DATABASE_DRIVER=pdo_mysql
      # - DATABASE_URL=mysql://okami:okami@mysql_db/conduit_symfony
      # - DATABASE_RO_URL=mysql://okami:okami@mysql_db/conduit_symfony
      - JWT_PASSPHRASE=c2b344e1-1a20-47fc-9aef-55b0c0d568a7
      - FRANKENPHP_CONFIG=worker ./public/index.php
      - APP_RUNTIME=Runtime\FrankenPhpSymfony\Runtime
    networks:
      - postgres_db
      - mysql_db
      - traefik_public
    deploy:
      labels:
        - traefik.enable=true
        - traefik.http.routers.symfony.entrypoints=websecure
        - traefik.http.services.symfony.loadbalancer.server.port=80
      replicas: 2
      placement:
        max_replicas_per_node: 1
        constraints:
          - node.labels.run == true

networks:
  postgres_db:
    external: true
  mysql_db:
    external: true
  traefik_public:
    external: true

version: "3.8"

services:
  app:
    image: gitea.okami101.io/conduit/fastapi:latest
    environment:
      - DB_HOST=postgres_db
      - DB_RO_HOST=postgres_db
      - DB_PORT=5432
      - DB_USERNAME=okami
      - DB_PASSWORD=okami
      - DB_DATABASE=conduit_fastapi
      - JWT_PASSPHRASE=c2b344e1-1a20-47fc-9aef-55b0c0d568a7
    networks:
      - postgres_db
      - traefik_public
    deploy:
      labels:
        - traefik.enable=true
        - traefik.http.routers.fastapi.entrypoints=websecure
        - traefik.http.services.fastapi.loadbalancer.server.port=8000
      replicas: 4
      placement:
        max_replicas_per_node: 2
        constraints:
          - node.labels.run == true

networks:
  postgres_db:
    external: true
  traefik_public:
    external: true

version: "3.8"

services:
  app:
    image: gitea.okami101.io/conduit/nestjs:latest
    environment:
      - NODE_ENV=production
      - DATABASE_URL=postgres://okami:okami@postgres_db/conduit_nestjs
      - JWT_SECRET=c2b344e1-1a20-47fc-9aef-55b0c0d568a7
    networks:
      - postgres_db
      - traefik_public
    deploy:
      labels:
        - traefik.enable=true
        - traefik.http.routers.nestjs.entrypoints=websecure
        - traefik.http.services.nestjs.loadbalancer.server.port=3000
      replicas: 2
      placement:
        max_replicas_per_node: 1
        constraints:
          - node.labels.run == true

networks:
  postgres_db:
    external: true
  traefik_public:
    external: true

version: "3.8"

services:
  app:
    image: gitea.okami101.io/conduit/spring-boot:latest
    environment:
      - SPRING_PROFILES_ACTIVE=production
      - DB_HOST=postgres_db
      - DB_PORT=5432
      - DB_RO_HOST=postgres_db
      - DB_USERNAME=okami
      - DB_PASSWORD=okami
      - DB_DATABASE=conduit_springboot
      - JWT_SECRET_KEY=YzJiMzQ0ZTEtMWEyMC00N2ZjLTlhZWYtNTViMGMwZDU2OGE3
    networks:
      - postgres_db
      - traefik_public
    deploy:
      labels:
        - traefik.enable=true
        - traefik.http.routers.springboot.entrypoints=websecure
        - traefik.http.services.springboot.loadbalancer.server.port=8080
      replicas: 2
      placement:
        max_replicas_per_node: 1
        constraints:
          - node.labels.run == true

networks:
  postgres_db:
    external: true
  traefik_public:
    external: true

version: "3.8"

services:
  app:
    image: gitea.okami101.io/conduit/symfony:latest
    environment:
      - SERVER_NAME=:80
      - APP_SECRET=ede04f29dd6c8b0e404581d48c36ec73
      - DATABASE_DRIVER=pdo_pgsql
      - DATABASE_URL=postgresql://okami:okami@postgres_db/conduit_symfony
      - DATABASE_RO_URL=postgresql://okami:okami@postgres_db/conduit_symfony
      # - DATABASE_DRIVER=pdo_mysql
      # - DATABASE_URL=mysql://okami:okami@mysql_db/conduit_symfony
      # - DATABASE_RO_URL=mysql://okami:okami@mysql_db/conduit_symfony
      - JWT_PASSPHRASE=c2b344e1-1a20-47fc-9aef-55b0c0d568a7
      - FRANKENPHP_CONFIG=worker ./public/index.php
      - APP_RUNTIME=Runtime\FrankenPhpSymfony\Runtime
    networks:
      - postgres_db
      - mysql_db
      - traefik_public
    deploy:
      labels:
        - traefik.enable=true
        - traefik.http.routers.symfony.entrypoints=websecure
        - traefik.http.services.symfony.loadbalancer.server.port=80
      replicas: 2
      placement:
        max_replicas_per_node: 1
        constraints:
          - node.labels.run == true

networks:
  postgres_db:
    external: true
  mysql_db:
    external: true
  traefik_public:
    external: true

Once the Swarm cluster is ready with all proxy, monitoring, database tools initialized, create all associated databases and let's deploy each project as following :

docker stack deploy laravel -c deploy-laravel.yml
docker stack deploy symfony -c deploy-symfony.yml
docker stack deploy fastapi -c deploy-fastapi.yml
docker stack deploy nestjs -c deploy-nestjs.yml
docker stack deploy spring-boot -c deploy-spring-boot.yml
docker stack deploy aspnet-core -c deploy-aspnet-core.yml

Once deployed you must migrate and seed the database according each project, check associated README for info.

The k6 scenarios

We'll be using k6 to run the tests, with constant-arrival-rate executor for progressive load testing, following 2 different scenarios :

Scenario 1 : fetch all articles, following the pagination
Scenario 2 : fetch all articles, calling each single article with slug, fetch associated comments for each article, and fetch profile of each related author

Duration of each scenario is 1 minute, with a 30 seconds graceful for finishing last started iterations. Results with one single test failures, i.e. any response status different than 200 or any response json error parsing, are not accepted.

The Iteration creation rate (rate / timeUnit) will be choosen in order to obtain the highest possible request rate, without any test failures.

Scenario 1 - Database intensive

The interest of this scenario is to be very database intensive, by fetching all articles, authors, and favorites, following the pagination, with a couple of SQL queries. Note as each code implementation normally use eager loading to avoid N+1 queries, which can have high influence in this test.

import http from "k6/http";
import { check } from "k6";

export const options = {
    scenarios: {
        articles: {
            env: { CONDUIT_URL: '<framework_url>' },
            duration: '1m',
            executor: 'constant-arrival-rate',
            rate: '<rate>',
            timeUnit: '1s',
            preAllocatedVUs: 50,
        },
    },
};

export default function () {
    const apiUrl = `https://${__ENV.CONDUIT_URL}/api`;

    const limit = 10;
    let offset = 0;

    let articles = []

    do {
        const articlesResponse = http.get(`${apiUrl}/articles?limit=${limit}&offset=${offset}`);
        check(articlesResponse, {
            "status is 200": (r) => r.status == 200,
        });

        articles = articlesResponse.json().articles;

        offset += limit;
    }
    while (articles && articles.length >= limit);
}

Here the expected JSON response format:

{
    "articles": [
        {
            "title": "Laboriosam aliquid dolore sed dolore",
            "slug": "laboriosam-aliquid-dolore-sed-dolore",
            "description": "Rerum beatae est enim cum similique.",
            "body": "Voluptas maxime incidunt...",
            "createdAt": "2023-12-23T16:02:03.000000Z",
            "updatedAt": "2023-12-23T16:02:03.000000Z",
            "author": {
                "username": "Devin Swift III",
                "bio": "Nihil impedit totam....",
                "image": "https:\/\/randomuser.me\/api\/portraits\/men\/47.jpg",
                "following": false
            },
            "tagList": [
                "aut",
                "cumque"
            ],
            "favorited": false,
            "favoritesCount": 5
        }
    ],
    //...
    "articlesCount": 500
}

The expected pseudocode SQL queries to build this response:

SELECT * FROM articles LIMIT 10 OFFSET 0;
SELECT count(*) FROM articles;
SELECT * FROM users WHERE id IN (<articles.author_id...>);
SELECT * FROM article_tag WHERE article_id IN (<articles.id...>);
SELECT * FROM favorites WHERE article_id IN (<articles.id...>);

{{< alert >}} It can highly differ according to each ORM, as few of them can prefer to reduce the queries by using subselect, but it's a good approximation. {{< /alert >}}

Scenario 2 - Runtime intensive

The interest of this scenario is to be mainly runtime intensive, by calling each endpoint of the API.

import http from "k6/http";
import { check } from "k6";

export const options = {
    scenarios: {
        articles: {
            env: { CONDUIT_URL: '<framework_url>' },
            duration: '1m',
            executor: 'constant-arrival-rate',
            rate: '<rate>',
            timeUnit: '1s',
            preAllocatedVUs: 50,
        },
    },
};

export default function () {
    const apiUrl = `https://${__ENV.CONDUIT_URL}.sw.okami101.io/api`;

    const limit = 10;
    let offset = 0;

    const tagsResponse = http.get(`${apiUrl}/tags`);
    check(tagsResponse, {
        "status is 200": (r) => r.status == 200,
    });

    let articles = []

    do {
        const articlesResponse = http.get(`${apiUrl}/articles?limit=${limit}&offset=${offset}`);
        check(articlesResponse, {
            "status is 200": (r) => r.status == 200,
        });

        articles = articlesResponse.json().articles;

        for (let i = 0; i < articles.length; i++) {
            const article = articles[i];
            const articleResponse = http.get(`${apiUrl}/articles/${article.slug}`);
            check(articleResponse, {
                "status is 200": (r) => r.status == 200,
            });

            const commentsResponse = http.get(`${apiUrl}/articles/${article.slug}/comments`);
            check(commentsResponse, {
                "status is 200": (r) => r.status == 200,
            });

            const authorsResponse = http.get(`${apiUrl}/profiles/${article.author.username}`);
            check(authorsResponse, {
                "status is 200": (r) => r.status == 200,
            });
        }
        offset += limit;
    }
    while (articles && articles.length >= limit);
}

The results

Laravel (Octane)

Laravel Octane will be enabled with FrankenPHP runtime.

Laravel MySQL scenario 1

Iteration creation rate = 10/s

checks.........................: 100.00% ✓ 7548      ✗ 0
data_received..................: 81 MB   1.1 MB/s
data_sent......................: 718 kB  9.4 kB/s
dropped_iterations.............: 153     2.011418/s
http_req_blocked...............: avg=227.08µs min=271ns   med=1.14µs   max=56.05ms  p(90)=1.69µs   p(95)=1.9µs
http_req_connecting............: avg=7.95µs   min=0s      med=0s       max=3.72ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=435.71ms min=5.62ms  med=205.37ms max=1.16s    p(90)=927.75ms p(95)=974.12ms
  { expected_response:true }...: avg=435.71ms min=5.62ms  med=205.37ms max=1.16s    p(90)=927.75ms p(95)=974.12ms
http_req_failed................: 0.00%   ✓ 0         ✗ 7548
http_req_receiving.............: avg=1.23ms   min=40.25µs med=700.17µs max=208.48ms p(90)=1.7ms    p(95)=2.67ms
http_req_sending...............: avg=231.75µs min=32.25µs med=132.92µs max=60.91ms  p(90)=228.32µs p(95)=316.55µs
http_req_tls_handshaking.......: avg=212.68µs min=0s      med=0s       max=54.71ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=434.24ms min=5.41ms  med=203.73ms max=1.16s    p(90)=925.77ms p(95)=972.64ms
http_reqs......................: 7548    99.229974/s
iteration_duration.............: avg=22.28s   min=3.08s   med=23.16s   max=29.88s   p(90)=27.92s   p(95)=28.86s
iterations.....................: 148     1.945686/s
vus............................: 3       min=3       max=50
vus_max........................: 50      min=50      max=50

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 17, 102, 87, 98, 98, 112, 101, 96, 106, 96, 99, 104, 94, 91, 81, 89, 98, 111, 102, 100, 93, 107, 107, 94, 95, 102, 91, 106, 97, 102, 97, 101, 97, 108, 106, 89, 116, 104, 96, 99, 101, 95, 110, 91, 94, 106, 109, 91, 103, 99, 98, 111, 109, 112, 97, 92, 102, 104, 98, 97, 96, 98, 91, 99, 102, 97, 95, 102, 104, 108, 104, 96, 104 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 5, 10, 15, 19, 24, 28, 32, 36, 41, 46, 50, 50, 50, 50, 49, 49, 50, 50, 50, 50, 50, 50, 50, 49, 50, 50, 50, 50, 48, 49, 49, 50, 48, 49, 49, 50, 50, 50, 50, 50, 48, 50, 49, 50, 50, 49, 50, 50, 50, 50, 49, 49, 50, 48, 50, 50, 50, 49, 50, 49, 47, 46, 44, 43, 43, 42, 42, 41, 38, 36, 30, 29 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 30, 47, 110, 148, 179, 196, 270, 330, 282, 423, 421, 451, 503, 547, 586, 573, 526, 493, 464, 452, 593, 506, 448, 492, 497, 507, 531, 493, 483, 484, 516, 496, 465, 495, 465, 494, 477, 454, 525, 486, 498, 538, 479, 488, 531, 516, 440, 523, 516, 467, 542, 430, 455, 458, 471, 484, 525, 480, 509, 507, 511, 516, 510, 472, 433, 432, 483, 423, 369, 376, 369, 349, 284 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.04, 0.35, 0.35, 0.35, 0.36, 0.35, 0.35, 0.34, 0.36, 0.35, 0.36, 0.35, 0.35, 0.35, 0.37, 0.36, 0.1, 0.04 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.02, 0.07, 0.06, 0.06, 0.06, 0.06, 0.06, 0.05, 0.07, 0.06, 0.06, 0.07, 0.07, 0.06, 0.06, 0.06, 0.03, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.02, 0.11, 0.82, 0.8, 0.82, 0.81, 0.8, 0.79, 0.81, 0.83, 0.82, 0.81, 0.82, 0.83, 0.8, 0.8, 0.77, 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.03, 0.04, 0.05, 0.03, 0.04, 0.04, 0.04, 0.04, 0.04, 0.05, 0.05, 0.04, 0.04, 0.05, 0.05, 0.05, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

We are database limited.

Laravel MySQL scenario 2

Iteration creation rate = 1/s

checks.........................: 100.00% ✓ 60530      ✗ 0
data_received..................: 140 MB  1.6 MB/s
data_sent......................: 5.0 MB  56 kB/s
dropped_iterations.............: 4       0.044438/s
http_req_blocked...............: avg=21.5µs   min=191ns   med=895ns    max=52.03ms  p(90)=1.38µs   p(95)=1.56µs
http_req_connecting............: avg=1.46µs   min=0s      med=0s       max=10.41ms  p(90)=0s       p(95)=0s
http_req_duration..............: avg=50.82ms  min=3.01ms  med=46.99ms  max=323.69ms p(90)=94.36ms  p(95)=109.42ms
  { expected_response:true }...: avg=50.82ms  min=3.01ms  med=46.99ms  max=323.69ms p(90)=94.36ms  p(95)=109.42ms
http_req_failed................: 0.00%   ✓ 0          ✗ 60530
http_req_receiving.............: avg=700.42µs min=16.66µs med=199.85µs max=208.14ms p(90)=1.03ms   p(95)=1.92ms
http_req_sending...............: avg=179.82µs min=16.56µs med=99.19µs  max=122.8ms  p(90)=186.74µs p(95)=258.75µs
http_req_tls_handshaking.......: avg=18.4µs   min=0s      med=0s       max=47.62ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=49.94ms  min=0s      med=46.2ms   max=323.42ms p(90)=93.11ms  p(95)=107.84ms
http_reqs......................: 60530   672.458785/s
iteration_duration.............: avg=53.18s   min=30.44s  med=54.75s   max=1m15s    p(90)=1m13s    p(95)=1m14s
iterations.....................: 13      0.144424/s
vus............................: 43      min=1        max=50
vus_max........................: 50      min=50       max=50

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 65, 149, 216, 476, 507, 578, 561, 625, 668, 647, 623, 611, 626, 666, 677, 707, 671, 665, 652, 686, 699, 711, 722, 667, 694, 698, 727, 712, 729, 677, 730, 750, 694, 736, 676, 700, 727, 704, 701, 712, 705, 646, 713, 734, 699, 747, 668, 714, 728, 721, 695, 741, 710, 664, 692, 752, 712, 707, 722, 692, 739, 648, 745, 616, 666, 686, 736, 704, 721, 716, 732, 686, 712, 710, 682, 719, 720, 723, 696, 690, 734, 724, 696, 670, 705, 698, 739, 706 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 30, 31, 32, 33, 33, 34, 35, 36, 37, 37, 38, 39, 40, 41, 42, 43, 44, 44, 45, 46, 47, 48, 49, 49, 50, 50, 50, 50, 50, 49, 49, 49, 49, 49, 49, 48, 48, 48, 48, 48, 48, 47, 47, 46, 46, 46, 46, 46, 46, 46, 45, 45, 45, 45, 44, 43 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 14, 13, 13, 8, 9, 10, 12, 13, 13, 15, 17, 19, 20, 21, 22, 22, 25, 26, 30, 29, 30, 30, 30, 37, 36, 37, 37, 39, 39, 44, 43, 41, 43, 44, 48, 49, 46, 48, 52, 52, 53, 57, 54, 56, 57, 56, 62, 60, 61, 62, 64, 65, 65, 74, 73, 67, 70, 69, 70, 73, 65, 77, 63, 79, 76, 72, 63, 68, 68, 67, 65, 68, 63, 69, 68, 64, 63, 62, 69, 66, 62, 62, 61, 69, 64, 61, 61, 61 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.04, 0.08, 0.44, 0.58, 0.61, 0.64, 0.67, 0.69, 0.7, 0.66, 0.67, 0.7, 0.68, 0.68, 0.66, 0.69, 0.66, 0.69, 0.66 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.03, 0.09, 0.13, 0.14, 0.16, 0.15, 0.16, 0.15, 0.15, 0.15, 0.15, 0.16, 0.17, 0.15, 0.16, 0.14, 0.16, 0.15 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.07, 0.2, 0.27, 0.3, 0.3, 0.31, 0.29, 0.3, 0.27, 0.32, 0.3, 0.31, 0.3, 0.3, 0.3, 0.33, 0.3, 0.32 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.04, 0.12, 0.14, 0.14, 0.13, 0.14, 0.14, 0.14, 0.14, 0.14, 0.14, 0.14, 0.15, 0.14, 0.13, 0.15, 0.13, 0.13 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

This is where Laravel Octane really shines, previously we had less than 300 req/s with Apache.

Laravel PgSQL scenario 1

Iteration creation rate = 10/s

checks.........................: 100.00% ✓ 22542      ✗ 0
data_received..................: 241 MB  3.8 MB/s
data_sent......................: 2.0 MB  31 kB/s
dropped_iterations.............: 159     2.509297/s
http_req_blocked...............: avg=98.83µs  min=214ns   med=955ns    max=190.07ms p(90)=1.48µs   p(95)=1.68µs
http_req_connecting............: avg=9µs      min=0s      med=0s       max=71.9ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=127.36ms min=4ms     med=82.05ms  max=644.29ms p(90)=247.64ms p(95)=269.15ms
  { expected_response:true }...: avg=127.36ms min=4ms     med=82.05ms  max=644.29ms p(90)=247.64ms p(95)=269.15ms
http_req_failed................: 0.00%   ✓ 0          ✗ 22542
http_req_receiving.............: avg=938.08µs min=26.46µs med=340.84µs max=212.74ms p(90)=1.26ms   p(95)=2.26ms
http_req_sending...............: avg=212.82µs min=14.12µs med=110.91µs max=67.83ms  p(90)=202.2µs  p(95)=308.11µs
http_req_tls_handshaking.......: avg=80.92µs  min=0s      med=0s       max=124.88ms p(90)=0s       p(95)=0s
http_req_waiting...............: avg=126.21ms min=0s      med=80.18ms  max=643.75ms p(90)=246.31ms p(95)=267.53ms
http_reqs......................: 22542   355.752049/s
iteration_duration.............: avg=6.55s    min=2.25s   med=6.66s    max=9.52s    p(90)=7.88s    p(95)=8.26s
iterations.....................: 442     6.97553/s
vus............................: 15      min=10       max=50
vus_max........................: 50      min=50       max=50

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 48, 263, 335, 331, 355, 373, 373, 376, 379, 359, 383, 382, 386, 388, 351, 387, 374, 367, 378, 348, 396, 367, 352, 350, 324, 344, 388, 346, 330, 368, 383, 376, 383, 372, 350, 391, 380, 365, 367, 340, 380, 375, 381, 383, 354, 380, 382, 384, 372, 345, 379, 342, 370, 357, 321, 304, 306, 275, 301, 318, 360, 360, 365, 240 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 10, 20, 26, 32, 38, 42, 49, 50, 47, 50, 48, 49, 46, 49, 49, 50, 48, 50, 49, 46, 45, 49, 50, 47, 50, 50, 48, 50, 48, 49, 50, 49, 50, 48, 48, 47, 48, 50, 46, 48, 49, 50, 50, 48, 49, 47, 47, 48, 50, 50, 48, 50, 47, 48, 50, 50, 50, 48, 49, 50, 43, 25, 15 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 38, 40, 59, 75, 89, 100, 115, 129, 128, 133, 125, 129, 127, 122, 137, 129, 130, 134, 127, 143, 117, 121, 138, 142, 147, 146, 129, 139, 143, 134, 127, 131, 127, 131, 141, 124, 126, 130, 140, 138, 130, 130, 130, 128, 138, 127, 123, 125, 132, 141, 132, 140, 134, 131, 147, 162, 160, 157, 190, 156, 133, 118, 69, 45 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.03, 0.35, 0.72, 0.7, 0.69, 0.69, 0.66, 0.7, 0.68, 0.68, 0.67, 0.66, 0.56, 0.62, 0.05, 0.03, 0.02, 0.04 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.02, 0.07, 0.12, 0.13, 0.12, 0.11, 0.11, 0.12, 0.13, 0.12, 0.13, 0.13, 0.1, 0.12, 0.02, 0.01, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.02, 0.02, 0.1, 0.22, 0.22, 0.22, 0.21, 0.2, 0.21, 0.23, 0.23, 0.22, 0.22, 0.2, 0.21, 0.07, 0.02, 0.03, 0.02 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.02, 0.07, 0.15, 0.14, 0.13, 0.13, 0.13, 0.15, 0.13, 0.13, 0.13, 0.14, 0.13, 0.13, 0.04, 0.02, 0.01, 0.01 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

Way better than with MySQL and no database limited.

Laravel PgSQL scenario 2

Iteration creation rate = 1/s

checks.........................: 100.00% ✓ 53448      ✗ 0
data_received..................: 122 MB  1.4 MB/s
data_sent......................: 4.4 MB  49 kB/s
dropped_iterations.............: 8       0.088883/s
http_req_blocked...............: avg=28.83µs  min=188ns   med=900ns    max=130.65ms p(90)=1.4µs    p(95)=1.57µs
http_req_connecting............: avg=3.26µs   min=0s      med=0s       max=40.47ms  p(90)=0s       p(95)=0s
http_req_duration..............: avg=59.09ms  min=3.07ms  med=49.91ms  max=426.79ms p(90)=121.42ms p(95)=137.82ms
  { expected_response:true }...: avg=59.09ms  min=3.07ms  med=49.91ms  max=426.79ms p(90)=121.42ms p(95)=137.82ms
http_req_failed................: 0.00%   ✓ 0          ✗ 53448
http_req_receiving.............: avg=1.3ms    min=17.89µs med=202.25µs max=277.84ms p(90)=1.17ms   p(95)=2.63ms
http_req_sending...............: avg=271.85µs min=22.02µs med=101.15µs max=269.13ms p(90)=197.28µs p(95)=281.99µs
http_req_tls_handshaking.......: avg=23.83µs  min=0s      med=0s       max=89.85ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=57.51ms  min=0s      med=48.78ms  max=423.57ms p(90)=119.42ms p(95)=135.29ms
http_reqs......................: 53448   593.825611/s
iteration_duration.............: avg=1m3s     min=45.9s   med=1m4s     max=1m19s    p(90)=1m17s    p(95)=1m18s
iterations.....................: 9       0.099993/s
vus............................: 44      min=1        max=50
vus_max........................: 50      min=50       max=50

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 59, 137, 245, 368, 393, 519, 334, 446, 490, 576, 526, 397, 379, 482, 597, 623, 652, 626, 631, 612, 631, 637, 608, 630, 651, 625, 637, 632, 671, 622, 683, 631, 655, 641, 632, 673, 659, 603, 650, 652, 642, 623, 668, 678, 642, 640, 498, 628, 638, 665, 624, 665, 630, 650, 669, 692, 628, 588, 650, 628, 687, 687, 639, 629, 632, 649, 657, 608, 669, 648, 658, 601, 618, 654, 654, 654, 521, 508, 317, 634, 685, 608, 662, 593, 675 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 1, 2, 3, 4, 5, 6, 8, 8, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 45, 46, 47, 47, 48, 49, 50, 50, 50, 50, 49, 50, 50, 50, 50, 49, 49, 49, 49, 49, 49, 49, 49, 48, 48, 48, 48, 47, 47, 47, 47, 47, 47, 47, 47, 46, 46, 45, 45 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 16, 14, 12, 11, 12, 11, 20, 18, 18, 17, 21, 29, 31, 30, 25, 25, 26, 27, 30, 33, 33, 34, 37, 38, 39, 41, 42, 43, 44, 48, 46, 48, 50, 53, 56, 54, 54, 61, 61, 62, 63, 66, 64, 63, 72, 71, 92, 71, 75, 73, 77, 74, 76, 79, 75, 70, 78, 82, 80, 81, 70, 71, 72, 81, 78, 75, 75, 78, 75, 75, 73, 77, 76, 75, 72, 72, 89, 89, 147, 74, 67, 77, 67, 80, 65 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.08, 0.37, 0.46, 0.51, 0.59, 0.58, 0.6, 0.62, 0.61, 0.6, 0.59, 0.6, 0.59, 0.58, 0.61, 0.58, 0.51, 0.64 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.03, 0.08, 0.11, 0.12, 0.13, 0.13, 0.14, 0.13, 0.16, 0.14, 0.14, 0.14, 0.14, 0.15, 0.16, 0.15, 0.12, 0.14 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.02, 0.08, 0.22, 0.22, 0.25, 0.25, 0.27, 0.26, 0.27, 0.26, 0.27, 0.26, 0.28, 0.27, 0.26, 0.26, 0.26, 0.26, 0.27 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.05, 0.13, 0.14, 0.16, 0.16, 0.16, 0.17, 0.16, 0.16, 0.16, 0.17, 0.16, 0.16, 0.15, 0.17, 0.15, 0.16, 0.18 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

Very close to MySQL equivalent.

Symfony (FrankenPHP)

Symfony MySQL scenario 1

Iteration creation rate = 5/s

checks.........................: 100.00% ✓ 11679     ✗ 0
data_received..................: 106 MB  1.6 MB/s
data_sent......................: 1.1 MB  16 kB/s
dropped_iterations.............: 72      1.058701/s
http_req_blocked...............: avg=127.87µs min=180ns   med=1.12µs   max=48.28ms  p(90)=1.59µs   p(95)=1.78µs
http_req_connecting............: avg=5.13µs   min=0s      med=0s       max=5.29ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=241.86ms min=10.69ms med=165.34ms max=935.06ms p(90)=495.74ms p(95)=513.79ms
  { expected_response:true }...: avg=241.86ms min=10.69ms med=165.34ms max=935.06ms p(90)=495.74ms p(95)=513.79ms
http_req_failed................: 0.00%   ✓ 0         ✗ 11679
http_req_receiving.............: avg=745.15µs min=27.68µs med=385.42µs max=207.74ms p(90)=1.02ms   p(95)=1.58ms
http_req_sending...............: avg=171.74µs min=27.68µs med=124.35µs max=23.65ms  p(90)=202.8µs  p(95)=258.4µs
http_req_tls_handshaking.......: avg=118.83µs min=0s      med=0s       max=47.04ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=240.95ms min=10.4ms  med=164.75ms max=934.19ms p(90)=495.14ms p(95)=513.11ms
http_reqs......................: 11679   171.73007/s
iteration_duration.............: avg=12.38s   min=1.78s   med=13.5s    max=17.82s   p(90)=15.61s   p(95)=16.21s
iterations.....................: 229     3.367256/s
vus............................: 1       min=1       max=50
vus_max........................: 50      min=50      max=50

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 46, 172, 170, 177, 165, 173, 176, 168, 170, 162, 174, 171, 168, 177, 169, 178, 176, 170, 116, 174, 179, 180, 172, 181, 172, 180, 178, 171, 179, 168, 172, 177, 172, 175, 170, 181, 176, 172, 180, 170, 178, 182, 171, 179, 171, 176, 177, 167, 181, 170, 177, 175, 173, 179, 171, 177, 179, 172, 178, 172, 170, 179, 172, 179, 167, 174, 177, 165, 34 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 5, 9, 11, 14, 17, 20, 23, 25, 28, 31, 34, 37, 40, 42, 45, 49, 49, 50, 48, 50, 50, 49, 48, 48, 50, 49, 49, 47, 49, 44, 49, 50, 49, 50, 50, 49, 50, 49, 50, 48, 48, 50, 47, 47, 48, 49, 49, 49, 50, 49, 49, 48, 50, 49, 49, 48, 50, 50, 47, 47, 47, 43, 42, 35, 30, 23, 16, 1 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 29, 35, 55, 66, 89, 101, 115, 136, 149, 179, 183, 201, 222, 225, 253, 254, 274, 246, 454, 293, 279, 268, 290, 274, 286, 277, 277, 288, 276, 290, 260, 275, 280, 286, 282, 276, 278, 277, 273, 293, 266, 276, 296, 268, 277, 275, 279, 293, 286, 293, 274, 288, 280, 275, 288, 274, 275, 286, 269, 289, 269, 268, 251, 232, 214, 170, 131, 80, 25 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.1, 0.04, 0.25, 0.37, 0.39, 0.33, 0.38, 0.36, 0.36, 0.37, 0.32, 0.37, 0.39, 0.37, 0.36, 0.3, 0.04, 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.03, 0.02, 0.04, 0.05, 0.06, 0.04, 0.07, 0.06, 0.07, 0.07, 0.05, 0.07, 0.06, 0.07, 0.06, 0.05, 0.02, 0.02, 0.01 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.03, 0.33, 0.9, 0.87, 0.9, 0.88, 0.93, 0.94, 0.94, 0.94, 0.93, 0.94, 0.94, 0.94, 0.93, 0.15, 0.03, 0.02 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.02, 0.03, 0.03, 0.04, 0.03, 0.04, 0.04, 0.04, 0.04, 0.04, 0.04, 0.03, 0.05, 0.04, 0.04, 0.02, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

Database limited too, but almost twice better than Laravel for MySQL.

Symfony MySQL scenario 2

Iteration creation rate = 1/s

checks.........................: 100.00% ✓ 94672       ✗ 0
data_received..................: 171 MB  2.1 MB/s
data_sent......................: 7.5 MB  94 kB/s
http_req_blocked...............: avg=14.88µs  min=182ns   med=643ns    max=93.7ms   p(90)=1.17µs   p(95)=1.34µs
http_req_connecting............: avg=945ns    min=0s      med=0s       max=9.67ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=14.55ms  min=1.89ms  med=10.7ms   max=318.64ms p(90)=27.5ms   p(95)=35.48ms
  { expected_response:true }...: avg=14.55ms  min=1.89ms  med=10.7ms   max=318.64ms p(90)=27.5ms   p(95)=35.48ms
http_req_failed................: 0.00%   ✓ 0           ✗ 94672
http_req_receiving.............: avg=895.52µs min=16.57µs med=242.05µs max=212.14ms p(90)=1.51ms   p(95)=2.88ms
http_req_sending...............: avg=172.15µs min=15.82µs med=77.99µs  max=151.35ms p(90)=163.13µs p(95)=228.85µs
http_req_tls_handshaking.......: avg=12.6µs   min=0s      med=0s       max=92.96ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=13.48ms  min=0s      med=9.93ms   max=317.4ms  p(90)=25.91ms  p(95)=32.93ms
http_reqs......................: 94672   1191.229807/s
iteration_duration.............: avg=22.99s   min=10.92s  med=23.68s   max=31.86s   p(90)=30.33s   p(95)=31.11s
iterations.....................: 61      0.767545/s
vus............................: 2       min=1         max=30
vus_max........................: 50      min=50        max=50

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 77, 197, 295, 628, 880, 1008, 939, 1206, 1212, 1197, 1128, 1171, 1326, 1229, 778, 1325, 1201, 1321, 1315, 1316, 1314, 1217, 1359, 1275, 1335, 1364, 1199, 1418, 1350, 1368, 1327, 1253, 983, 1417, 1405, 1338, 1248, 1403, 1455, 1384, 823, 853, 1121, 815, 961, 1082, 1243, 1474, 1332, 1311, 1256, 555, 1016, 1241, 1055, 1218, 1259, 1447, 1492, 1485, 1487, 1289, 1481, 1403, 1506, 1399, 1250, 1433, 1460, 1157, 1386, 1231, 1471, 1318, 1377, 1332, 1127, 1233, 903, 229 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 10, 11, 11, 12, 12, 13, 13, 14, 14, 14, 15, 15, 16, 16, 16, 16, 17, 17, 17, 17, 18, 18, 19, 19, 19, 20, 20, 20, 20, 20, 21, 22, 22, 23, 23, 23, 24, 24, 26, 25, 25, 25, 26, 27, 27, 28, 29, 29, 30, 28, 27, 27, 26, 25, 24, 23, 23, 22, 21, 18, 18, 15, 14, 13, 10, 6, 2 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 9, 9, 9, 6, 5, 6, 7, 6, 7, 8, 9, 10, 8, 9, 14, 9, 10, 9, 10, 10, 11, 12, 11, 12, 12, 12, 13, 11, 12, 12, 13, 14, 18, 12, 13, 15, 15, 14, 13, 15, 24, 24, 18, 27, 23, 21, 19, 15, 18, 18, 19, 45, 24, 21, 24, 22, 21, 19, 19, 19, 20, 22, 18, 19, 17, 18, 19, 16, 16, 20, 15, 16, 12, 12, 10, 10, 10, 7, 5, 5 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.04, 0.31, 0.52, 0.53, 0.59, 0.6, 0.58, 0.59, 0.61, 0.47, 0.57, 0.48, 0.64, 0.64, 0.62, 0.61, 0.39, 0.04 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.01, 0.07, 0.11, 0.1, 0.11, 0.12, 0.12, 0.12, 0.13, 0.1, 0.14, 0.1, 0.13, 0.14, 0.13, 0.13, 0.08, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.04, 0.16, 0.3, 0.34, 0.35, 0.38, 0.39, 0.36, 0.38, 0.34, 0.38, 0.32, 0.39, 0.41, 0.39, 0.39, 0.32, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.02, 0.05, 0.06, 0.09, 0.09, 0.08, 0.08, 0.09, 0.08, 0.08, 0.09, 0.08, 0.1, 0.1, 0.09, 0.09, 0.07, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

Huge gap in performance against Laravel Octane here, about twice better ! Without FrankenPHP, we were capping to previously about 300 req/s...

Symfony PgSQL scenario 1

Iteration creation rate = 10/s

checks.........................: 100.00% ✓ 22287      ✗ 0
data_received..................: 201 MB  3.1 MB/s
data_sent......................: 1.9 MB  30 kB/s
dropped_iterations.............: 164     2.548995/s
http_req_blocked...............: avg=61.02µs  min=234ns   med=1.01µs   max=131.72ms p(90)=1.48µs   p(95)=1.68µs
http_req_connecting............: avg=6.79µs   min=0s      med=0s       max=62.5ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=130.52ms min=6.1ms   med=129.59ms max=404.9ms  p(90)=199.67ms p(95)=213.58ms
  { expected_response:true }...: avg=130.52ms min=6.1ms   med=129.59ms max=404.9ms  p(90)=199.67ms p(95)=213.58ms
http_req_failed................: 0.00%   ✓ 0          ✗ 22287
http_req_receiving.............: avg=880.47µs min=20.89µs med=297.79µs max=216.18ms p(90)=904.22µs p(95)=1.78ms
http_req_sending...............: avg=214.44µs min=29.19µs med=114.19µs max=92.18ms  p(90)=204.97µs p(95)=297.4µs
http_req_tls_handshaking.......: avg=50.13µs  min=0s      med=0s       max=54.56ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=129.43ms min=0s      med=128.65ms max=396.55ms p(90)=198.43ms p(95)=212.35ms
http_reqs......................: 22287   346.399123/s
iteration_duration.............: avg=6.7s     min=1.75s   med=7.06s    max=8.21s    p(90)=7.47s    p(95)=7.63s
iterations.....................: 437     6.79214/s
vus............................: 14      min=11       max=50
vus_max........................: 50      min=50       max=50

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 122, 341, 348, 346, 359, 338, 361, 359, 345, 360, 330, 352, 361, 344, 359, 334, 362, 323, 346, 358, 335, 357, 352, 348, 358, 337, 359, 357, 345, 357, 339, 358, 350, 333, 355, 332, 359, 359, 349, 358, 333, 356, 359, 341, 352, 333, 355, 360, 344, 356, 340, 360, 360, 348, 355, 337, 346, 354, 345, 359, 337, 354, 341, 338, 179 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 11, 18, 23, 28, 34, 38, 44, 49, 49, 50, 50, 48, 49, 47, 48, 49, 49, 50, 48, 45, 50, 48, 49, 50, 50, 49, 49, 47, 46, 46, 50, 50, 48, 49, 47, 48, 47, 47, 50, 50, 48, 50, 48, 48, 48, 50, 50, 48, 50, 50, 47, 49, 49, 50, 48, 50, 47, 46, 48, 49, 47, 39, 31, 14 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 25, 37, 54, 69, 82, 104, 111, 128, 142, 136, 149, 141, 134, 142, 130, 142, 137, 154, 141, 134, 137, 136, 138, 141, 137, 146, 138, 136, 138, 133, 140, 137, 142, 148, 140, 144, 135, 134, 141, 137, 147, 138, 137, 141, 134, 151, 139, 137, 144, 138, 145, 136, 136, 143, 137, 146, 141, 136, 137, 134, 144, 127, 109, 85, 40 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.03, 0.28, 0.33, 0.34, 0.33, 0.34, 0.35, 0.34, 0.34, 0.35, 0.34, 0.33, 0.34, 0.32, 0.03, 0.04, 0.04, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.02, 0.05, 0.05, 0.06, 0.06, 0.05, 0.05, 0.06, 0.05, 0.05, 0.06, 0.05, 0.07, 0.05, 0.01, 0.02, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.03, 0.54, 0.93, 0.93, 0.91, 0.92, 0.93, 0.91, 0.92, 0.93, 0.93, 0.93, 0.92, 0.93, 0.21, 0.03, 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.02, 0.05, 0.07, 0.07, 0.07, 0.08, 0.07, 0.09, 0.08, 0.07, 0.07, 0.07, 0.08, 0.07, 0.03, 0.02, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

Twice better than MySQL, performing same as Laravel but database limited.

Symfony PgSQL scenario 2

Iteration creation rate = 1/s

checks.........................: 100.00% ✓ 94672       ✗ 0
data_received..................: 171 MB  2.2 MB/s
data_sent......................: 7.5 MB  95 kB/s
http_req_blocked...............: avg=17.77µs  min=172ns   med=626ns    max=132.86ms p(90)=1.19µs   p(95)=1.36µs
http_req_connecting............: avg=2.07µs   min=0s      med=0s       max=62.8ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=14.36ms  min=1.87ms  med=10.33ms  max=358.31ms p(90)=26.54ms  p(95)=36.05ms
  { expected_response:true }...: avg=14.36ms  min=1.87ms  med=10.33ms  max=358.31ms p(90)=26.54ms  p(95)=36.05ms
http_req_failed................: 0.00%   ✓ 0           ✗ 94672
http_req_receiving.............: avg=946.99µs min=15.83µs med=251.63µs max=213.84ms p(90)=1.58ms   p(95)=3ms
http_req_sending...............: avg=180.14µs min=14.98µs med=75.49µs  max=199.41ms p(90)=161.75µs p(95)=230.84µs
http_req_tls_handshaking.......: avg=14.39µs  min=0s      med=0s       max=132.05ms p(90)=0s       p(95)=0s
http_req_waiting...............: avg=13.23ms  min=0s      med=9.58ms   max=278.53ms p(90)=24.63ms  p(95)=33.11ms
http_reqs......................: 94672   1199.909891/s
iteration_duration.............: avg=22.69s   min=10.8s   med=23.88s   max=30.11s   p(90)=29.54s   p(95)=29.85s
iterations.....................: 61      0.773138/s
vus............................: 5       min=1         max=29
vus_max........................: 50      min=50        max=50

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 91, 207, 385, 748, 820, 959, 1075, 964, 1238, 1212, 1320, 1289, 1154, 1361, 1365, 1307, 1346, 1115, 1328, 1461, 1238, 1417, 1237, 953, 1350, 1272, 1331, 1155, 1105, 1017, 1196, 1108, 815, 824, 1327, 1224, 1334, 1213, 1410, 1430, 1365, 1394, 1299, 1165, 1427, 1419, 1341, 1234, 1418, 957, 1035, 1028, 929, 880, 1414, 1491, 1388, 1226, 1510, 1408, 1366, 1571, 1048, 1439, 1486, 1360, 1401, 1216, 1392, 1408, 1486, 1440, 1260, 1312, 1341, 1271, 1250, 964, 628, 4 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 10, 11, 11, 11, 11, 12, 12, 13, 13, 13, 13, 14, 14, 15, 15, 16, 15, 16, 16, 16, 17, 18, 19, 19, 20, 20, 20, 20, 21, 21, 22, 21, 22, 22, 23, 23, 24, 24, 25, 25, 25, 26, 27, 27, 27, 28, 28, 28, 28, 29, 27, 26, 26, 26, 23, 23, 22, 21, 20, 19, 18, 18, 15, 13, 10, 8, 5 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 8, 8, 7, 5, 6, 6, 6, 8, 7, 8, 8, 8, 9, 8, 8, 9, 8, 11, 9, 9, 11, 10, 11, 15, 11, 12, 12, 14, 14, 15, 14, 15, 21, 22, 14, 16, 15, 17, 14, 14, 15, 15, 17, 19, 16, 16, 18, 19, 17, 26, 24, 26, 24, 33, 19, 17, 20, 24, 19, 20, 21, 18, 25, 18, 17, 18, 16, 19, 15, 15, 13, 13, 14, 12, 10, 10, 8, 7, 5, 9 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.07, 0.37, 0.54, 0.58, 0.58, 0.55, 0.5, 0.52, 0.6, 0.6, 0.56, 0.55, 0.6, 0.6, 0.61, 0.58, 0.21, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.02, 0.07, 0.11, 0.12, 0.12, 0.12, 0.11, 0.12, 0.12, 0.12, 0.12, 0.12, 0.13, 0.13, 0.13, 0.11, 0.05, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.03, 0.19, 0.27, 0.33, 0.33, 0.3, 0.3, 0.3, 0.33, 0.32, 0.33, 0.3, 0.35, 0.34, 0.35, 0.34, 0.19, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.02, 0.07, 0.09, 0.1, 0.1, 0.1, 0.09, 0.09, 0.11, 0.1, 0.09, 0.11, 0.11, 0.11, 0.1, 0.11, 0.06, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

Same results than MySQL, no database limit.

FastAPI

As a side note here, uvicorn is limited to 1 CPU core, so I use 2 replicas on each worker to use all CPU cores.

FastAPI scenario 1

Iteration creation rate = 15/s

checks.........................: 100.00% ✓ 33048     ✗ 0
data_received..................: 272 MB  4.3 MB/s
data_sent......................: 2.9 MB  46 kB/s
dropped_iterations.............: 253     4.042284/s
http_req_blocked...............: avg=44.03µs  min=197ns   med=873ns    max=51.66ms  p(90)=1.31µs   p(95)=1.48µs
http_req_connecting............: avg=1.59µs   min=0s      med=0s       max=3.33ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=87.55ms  min=5.74ms  med=79.16ms  max=449.45ms p(90)=160.7ms  p(95)=187.45ms
{ expected_response:true }...: avg=87.55ms  min=5.74ms  med=79.16ms  max=449.45ms p(90)=160.7ms  p(95)=187.45ms
http_req_failed................: 0.00%   ✓ 0         ✗ 33048
http_req_receiving.............: avg=809.01µs min=18.38µs med=273.27µs max=53.15ms  p(90)=1.95ms   p(95)=3.12ms
http_req_sending...............: avg=156.85µs min=23.21µs med=95.41µs  max=45.6ms   p(90)=181.16µs p(95)=248.5µs
http_req_tls_handshaking.......: avg=40.32µs  min=0s      med=0s       max=44.77ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=86.59ms  min=0s      med=78.14ms  max=448.54ms p(90)=159.53ms p(95)=186.05ms
http_reqs......................: 33048   528.02138/s
iteration_duration.............: avg=4.49s    min=1.14s   med=4.65s    max=6.25s    p(90)=5.17s    p(95)=5.3s
iterations.....................: 648     10.35336/s
vus............................: 22      min=15      max=50
vus_max........................: 50      min=50      max=50

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 270, 514, 530, 535, 552, 541, 524, 493, 545, 555, 560, 539, 519, 545, 540, 531, 525, 514, 547, 540, 537, 533, 485, 511, 534, 525, 508, 500, 550, 527, 538, 516, 500, 542, 532, 530, 504, 508, 540, 538, 553, 537, 497, 560, 517, 578, 559, 487, 551, 546, 538, 531, 517, 518, 578, 559, 521, 516, 556, 567, 517, 517, 351 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 15, 23, 31, 41, 46, 50, 50, 50, 48, 49, 50, 48, 48, 45, 49, 50, 49, 48, 50, 49, 50, 49, 49, 50, 50, 50, 50, 48, 50, 47, 49, 48, 48, 49, 49, 50, 48, 48, 50, 49, 49, 50, 48, 48, 49, 48, 45, 50, 48, 49, 49, 48, 47, 50, 49, 50, 48, 47, 50, 48, 39, 22 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 20, 33, 45, 65, 74, 88, 93, 96, 90, 89, 90, 92, 94, 91, 89, 91, 91, 95, 90, 93, 92, 93, 101, 93, 90, 94, 96, 101, 87, 92, 89, 96, 98, 91, 91, 93, 94, 97, 92, 90, 88, 91, 95, 91, 93, 86, 88, 94, 89, 91, 90, 92, 93, 92, 87, 86, 94, 90, 91, 86, 86, 63, 36 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.02, 0.52, 0.71, 0.74, 0.73, 0.71, 0.69, 0.71, 0.73, 0.73, 0.75, 0.71, 0.74, 0.5, 0.04, 0.03, 0.03, 0.02 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.01, 0.12, 0.15, 0.15, 0.15, 0.15, 0.17, 0.16, 0.15, 0.15, 0.15, 0.15, 0.15, 0.11, 0.02, 0.01, 0.01, 0.01 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.3, 0.46, 0.47, 0.47, 0.48, 0.47, 0.48, 0.49, 0.49, 0.49, 0.46, 0.49, 0.35, 0.03, 0.03, 0.03, 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.11, 0.25, 0.25, 0.3, 0.27, 0.27, 0.25, 0.29, 0.28, 0.28, 0.26, 0.29, 0.19, 0.02, 0.02, 0.02, 0.01, 0.01 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

FastAPI outperforms above PHP frameworks in this specific scenario, and database isn't the bottleneck anymore.

FastAPI scenario 2

Iteration creation rate = 2/s

checks.........................: 100.00% ✓ 72075     ✗ 0
data_received..................: 147 MB  1.6 MB/s
data_sent......................: 5.4 MB  60 kB/s
dropped_iterations.............: 68      0.755514/s
http_req_blocked...............: avg=20.35µs  min=213ns    med=903ns    max=52.94ms p(90)=1.31µs   p(95)=1.48µs
http_req_connecting............: avg=909ns    min=0s       med=0s       max=10.34ms p(90)=0s       p(95)=0s
http_req_duration..............: avg=51.06ms  min=3.06ms   med=32.91ms  max=1.07s   p(90)=117.46ms p(95)=138.49ms
{ expected_response:true }...: avg=51.06ms  min=3.06ms   med=32.91ms  max=1.07s   p(90)=117.46ms p(95)=138.49ms
http_req_failed................: 0.00%   ✓ 0         ✗ 72075
http_req_receiving.............: avg=301.26µs min=17.32µs  med=125.14µs max=26.38ms p(90)=678.97µs p(95)=1.14ms
http_req_sending...............: avg=118.25µs min=21.81µs  med=94.18µs  max=20.22ms p(90)=163.96µs p(95)=204.06µs
http_req_tls_handshaking.......: avg=17.89µs  min=0s       med=0s       max=37.55ms p(90)=0s       p(95)=0s
http_req_waiting...............: avg=50.64ms  min=911.97µs med=32.54ms  max=1.07s   p(90)=116.83ms p(95)=137.77ms
http_reqs......................: 72075   800.78886/s
iteration_duration.............: avg=1m10s    min=51.94s   med=1m14s    max=1m21s   p(90)=1m20s    p(95)=1m21s
iterations.....................: 20      0.22221/s
vus............................: 33      min=2       max=50
vus_max........................: 50      min=50      max=50

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 19, 168, 465, 631, 720, 792, 752, 758, 757, 763, 839, 819, 777, 770, 821, 828, 760, 792, 741, 869, 862, 831, 846, 811, 820, 878, 792, 811, 815, 829, 804, 807, 842, 819, 791, 804, 744, 839, 810, 828, 841, 890, 841, 834, 804, 829, 821, 837, 852, 853, 853, 884, 871, 773, 774, 825, 794, 832, 825, 787, 807, 872, 837, 815, 826, 778, 811, 810, 823, 807, 786, 872, 886, 810, 808, 831, 824, 853, 770, 818, 793, 827, 795, 813, 795, 858, 869, 805, 846, 823, 563 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 49, 50, 50, 50, 50, 49, 50, 50, 50, 49, 49, 49, 48, 48, 48, 47, 46, 46, 46, 46, 46, 46, 44, 44, 44, 44, 43, 43, 43, 42, 42, 40, 39, 39, 39, 37, 37, 36, 33 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 13, 12, 8, 9, 11, 12, 16, 18, 21, 23, 24, 27, 30, 34, 34, 36, 42, 42, 48, 44, 46, 51, 51, 56, 59, 56, 63, 59, 64, 59, 63, 62, 58, 61, 62, 62, 68, 60, 61, 60, 58, 57, 59, 59, 60, 61, 61, 58, 60, 58, 59, 56, 56, 65, 64, 58, 66, 61, 60, 62, 61, 57, 58, 61, 55, 62, 60, 59, 57, 57, 58, 53, 51, 56, 56, 53, 52, 52, 56, 51, 56, 51, 51, 52, 46, 48, 44, 46, 44, 43, 44 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.02, 0.02, 0.03, 0.54, 0.65, 0.68, 0.69, 0.72, 0.71, 0.7, 0.71, 0.71, 0.72, 0.69, 0.68, 0.72, 0.69, 0.71, 0.7 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.01, 0.01, 0.11, 0.18, 0.18, 0.16, 0.17, 0.17, 0.16, 0.18, 0.18, 0.18, 0.18, 0.18, 0.15, 0.18, 0.17, 0.17 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.03, 0.22, 0.22, 0.25, 0.28, 0.31, 0.29, 0.31, 0.3, 0.3, 0.32, 0.33, 0.3, 0.28, 0.3, 0.31, 0.3, 0.28 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.01, 0.11, 0.14, 0.18, 0.2, 0.23, 0.23, 0.22, 0.24, 0.26, 0.24, 0.25, 0.21, 0.2, 0.23, 0.24, 0.22, 0.19 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

FastAPI fall behind Symfony but ahead of Laravel.

NestJS

NestJS scenario 1

Iteration creation rate = 15/s

checks.........................: 100.00% ✓ 37434      ✗ 0
data_received..................: 648 MB  11 MB/s
data_sent......................: 3.5 MB  57 kB/s
dropped_iterations.............: 166     2.680206/s
http_req_blocked...............: avg=35.23µs  min=216ns   med=702ns    max=49.57ms  p(90)=1.19µs   p(95)=1.33µs
http_req_connecting............: avg=1.44µs   min=0s      med=0s       max=5.62ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=75.64ms  min=3.2ms   med=70.44ms  max=346.41ms p(90)=134.43ms p(95)=146.32ms
{ expected_response:true }...: avg=75.64ms  min=3.2ms   med=70.44ms  max=346.41ms p(90)=134.43ms p(95)=146.32ms
http_req_failed................: 0.00%   ✓ 0          ✗ 37434
http_req_receiving.............: avg=408.39µs min=19.38µs med=219.61µs max=42.89ms  p(90)=653.52µs p(95)=1.28ms
http_req_sending...............: avg=134.99µs min=18.84µs med=83.94µs  max=26.64ms  p(90)=156.84µs p(95)=222.84µs
http_req_tls_handshaking.......: avg=31.9µs   min=0s      med=0s       max=40.37ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=75.09ms  min=2.92ms  med=69.92ms  max=345.47ms p(90)=133.87ms p(95)=145.68ms
http_reqs......................: 37434   604.402491/s
iteration_duration.............: avg=3.89s    min=1.2s    med=4s       max=5.16s    p(90)=4.41s    p(95)=4.54s
iterations.....................: 734     11.851029/s
vus............................: 31      min=15       max=50
vus_max........................: 50      min=50       max=50

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 273, 496, 577, 588, 599, 624, 621, 610, 641, 638, 641, 614, 585, 580, 600, 604, 601, 571, 606, 663, 643, 572, 596, 585, 616, 650, 680, 615, 612, 611, 617, 572, 587, 593, 605, 633, 633, 573, 646, 645, 650, 570, 629, 653, 691, 650, 580, 555, 590, 646, 565, 585, 638, 594, 567, 555, 602, 570, 641, 648, 601, 630, 8 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 15, 23, 32, 39, 44, 50, 49, 50, 46, 47, 49, 49, 48, 49, 49, 49, 49, 49, 50, 50, 49, 50, 48, 50, 49, 50, 49, 50, 46, 48, 46, 49, 50, 46, 47, 47, 45, 49, 47, 45, 49, 49, 49, 48, 45, 48, 50, 50, 49, 46, 45, 48, 50, 50, 49, 45, 48, 50, 49, 50, 31 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 21, 38, 45, 58, 65, 74, 79, 79, 75, 74, 75, 78, 85, 80, 80, 82, 81, 86, 78, 74, 76, 85, 81, 83, 77, 76, 72, 75, 78, 77, 81, 82, 83, 83, 79, 76, 76, 84, 75, 74, 73, 81, 81, 74, 68, 72, 81, 88, 82, 77, 75, 81, 76, 83, 86, 85, 74, 84, 80, 73, 74, 33, 6 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.02, 0.03, 0.27, 0.49, 0.46, 0.42, 0.45, 0.47, 0.42, 0.42, 0.46, 0.45, 0.42, 0.43, 0.35, 0.02, 0.02, 0.02, 0.02 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.01, 0.29, 0.48, 0.51, 0.55, 0.52, 0.49, 0.54, 0.54, 0.49, 0.51, 0.55, 0.53, 0.4, 0.01, 0.01, 0.01, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.04, 0.12, 0.21, 0.24, 0.21, 0.22, 0.22, 0.21, 0.21, 0.22, 0.23, 0.21, 0.21, 0.18, 0.03, 0.04, 0.03, 0.03, 0.04 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.08, 0.17, 0.16, 0.16, 0.15, 0.18, 0.17, 0.17, 0.17, 0.17, 0.17, 0.16, 0.16, 0.02, 0.02, 0.02, 0.01, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

It's very close to FastAPI, and database is strangely sleeping far more than with FastAPI, let's keep up on scenario 2.

NestJS scenario 2

Iteration creation rate = 3/s

checks.........................: 99.93% ✓ 111672      ✗ 72
data_received..................: 530 MB 6.1 MB/s
data_sent......................: 8.9 MB 103 kB/s
dropped_iterations.............: 109    1.255685/s
http_req_blocked...............: avg=13.34µs  min=209ns   med=701ns    max=48.9ms  p(90)=1.15µs   p(95)=1.3µs
http_req_connecting............: avg=721ns    min=0s      med=0s       max=18.51ms p(90)=0s       p(95)=0s
http_req_duration..............: avg=29.19ms  min=1.38ms  med=26.19ms  max=247.1ms p(90)=53.2ms   p(95)=61.63ms
{ expected_response:true }...: avg=29.2ms   min=2.01ms  med=26.21ms  max=247.1ms p(90)=53.21ms  p(95)=61.65ms
http_req_failed................: 0.06%  ✓ 72          ✗ 111672
http_req_receiving.............: avg=501.09µs min=16.93µs med=189.85µs max=46.9ms  p(90)=1.11ms   p(95)=1.88ms
http_req_sending...............: avg=107.93µs min=13.6µs  med=76.9µs   max=27.43ms p(90)=147.26µs p(95)=188.72µs
http_req_tls_handshaking.......: avg=11.39µs  min=0s      med=0s       max=39.98ms p(90)=0s       p(95)=0s
http_req_waiting...............: avg=28.58ms  min=0s      med=25.61ms  max=246.9ms p(90)=52.33ms  p(95)=60.73ms
http_reqs......................: 111744 1287.295935/s
iteration_duration.............: avg=45.63s   min=26.8s   med=50.21s   max=55.45s  p(90)=54.19s   p(95)=54.62s
iterations.....................: 72     0.829443/s
vus............................: 8      min=3         max=50
vus_max........................: 50     min=50        max=50

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 121, 508, 755, 899, 1012, 1007, 1156, 1177, 1096, 1180, 1224, 1237, 1208, 1244, 1295, 1437, 1445, 1345, 1338, 1405, 1378, 1380, 1411, 1293, 1420, 1441, 1451, 1365, 1264, 1439, 1384, 1584, 1241, 1361, 1319, 1427, 1398, 1362, 1320, 1448, 1482, 1458, 1311, 1256, 1399, 1363, 1345, 1259, 1346, 1443, 1499, 1445, 1438, 1451, 1425, 1472, 1479, 1367, 1322, 1450, 1414, 1360, 1355, 1457, 1326, 1411, 1363, 1350, 1277, 1279, 1168, 1216, 1198, 1256, 1314, 1248, 1236, 1192, 1183, 1227, 1263, 1357, 1148, 1141, 1168, 1127, 900, 25 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 3, 6, 9, 12, 15, 18, 21, 24, 27, 30, 33, 36, 39, 42, 45, 48, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 49, 50, 49, 50, 50, 49, 49, 50, 50, 50, 50, 50, 50, 49, 50, 49, 50, 49, 49, 47, 45, 42, 40, 38, 32, 28, 25, 22, 22, 22, 22, 22, 22, 22, 22, 22, 22, 20, 20, 18, 16, 15, 11, 8 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 8, 8, 9, 11, 13, 16, 16, 18, 23, 23, 25, 27, 30, 32, 33, 32, 34, 37, 37, 35, 36, 36, 35, 38, 35, 35, 34, 36, 39, 34, 36, 31, 40, 37, 38, 35, 36, 36, 36, 35, 33, 34, 38, 39, 36, 36, 37, 40, 37, 34, 33, 34, 35, 34, 35, 34, 33, 37, 37, 35, 35, 35, 34, 30, 32, 28, 27, 23, 21, 19, 18, 18, 18, 17, 17, 17, 18, 18, 19, 18, 16, 14, 15, 14, 12, 9, 7, 6 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.02, 0.03, 0.34, 0.45, 0.47, 0.48, 0.46, 0.49, 0.47, 0.48, 0.46, 0.48, 0.51, 0.49, 0.46, 0.46, 0.43, 0.42, 0.41 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.01, 0.5, 0.53, 0.51, 0.5, 0.53, 0.49, 0.51, 0.51, 0.52, 0.51, 0.47, 0.49, 0.52, 0.52, 0.54, 0.57, 0.56 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.16, 0.23, 0.26, 0.29, 0.29, 0.29, 0.29, 0.28, 0.29, 0.27, 0.3, 0.3, 0.3, 0.27, 0.24, 0.24, 0.25, 0.1 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.08, 0.12, 0.13, 0.16, 0.14, 0.15, 0.16, 0.16, 0.15, 0.15, 0.16, 0.15, 0.15, 0.15, 0.14, 0.14, 0.13, 0.06 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

Now NestJS match the high performance of Symfony. The native even loop system is very efficient. It's time to test it against compiled language.

Spring Boot

Spring Boot scenario 1

Iteration creation rate = 30/s

checks.........................: 100.00% ✓ 91851       ✗ 0
data_received..................: 1.6 GB  26 MB/s
data_sent......................: 7.8 MB  129 kB/s
http_req_blocked...............: avg=16.33µs  min=191ns    med=419ns    max=71.26ms  p(90)=723ns    p(95)=925ns
http_req_connecting............: avg=978ns    min=0s       med=0s       max=19.89ms  p(90)=0s       p(95)=0s
http_req_duration..............: avg=14.04ms  min=2.37ms   med=12.32ms  max=223.11ms p(90)=24.19ms  p(95)=28.67ms
{ expected_response:true }...: avg=14.04ms  min=2.37ms   med=12.32ms  max=223.11ms p(90)=24.19ms  p(95)=28.67ms
http_req_failed................: 0.00%   ✓ 0           ✗ 91851
http_req_receiving.............: avg=1.76ms   min=19.18µs  med=758.76µs max=63.89ms  p(90)=4.48ms   p(95)=6.73ms
http_req_sending...............: avg=147.52µs min=21.29µs  med=51.49µs  max=43.07ms  p(90)=130.77µs p(95)=286.91µs
http_req_tls_handshaking.......: avg=14.4µs   min=0s       med=0s       max=47.91ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=12.12ms  min=0s       med=10.43ms  max=220.97ms p(90)=21.23ms  p(95)=24.99ms
http_reqs......................: 91851   1518.447027/s
iteration_duration.............: avg=741.16ms min=485.58ms med=732.29ms max=1.18s    p(90)=865.14ms p(95)=909.61ms
iterations.....................: 1801    29.773471/s
vus............................: 25      min=17        max=29
vus_max........................: 50      min=50        max=50

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 353, 1407, 1575, 1522, 1483, 1562, 1587, 1578, 1491, 1521, 1545, 1561, 1523, 1493, 1392, 1604, 1609, 1526, 1554, 1493, 1547, 1558, 1531, 1484, 1511, 1530, 1606, 1548, 1479, 1459, 1574, 1582, 1575, 1481, 1439, 1615, 1304, 1567, 1571, 1530, 1610, 1604, 1516, 1523, 1433, 1630, 1503, 1532, 1557, 1492, 1559, 1577, 1521, 1497, 1446, 1583, 1566, 1509, 1424, 1514, 1385 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 20, 23, 24, 25, 25, 24, 20, 20, 22, 21, 19, 18, 21, 22, 26, 20, 21, 22, 20, 21, 20, 18, 20, 21, 21, 21, 18, 21, 21, 23, 22, 19, 19, 21, 22, 19, 29, 26, 27, 25, 22, 18, 21, 24, 24, 21, 21, 23, 21, 23, 20, 17, 19, 23, 22, 20, 21, 22, 25, 25 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 11, 14, 15, 15, 16, 16, 14, 13, 14, 15, 13, 12, 12, 14, 16, 15, 14, 13, 14, 14, 13, 13, 12, 14, 15, 14, 13, 12, 14, 15, 14, 13, 12, 13, 15, 14, 16, 18, 17, 17, 15, 13, 12, 14, 15, 14, 13, 13, 14, 14, 13, 12, 12, 13, 15, 14, 13, 13, 16, 17, 14 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.02, 0.02, 0.06, 0.29, 0.29, 0.29, 0.29, 0.28, 0.3, 0.27, 0.28, 0.28, 0.29, 0.29, 0.26, 0.03, 0.02, 0.02, 0.02 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.01, 0.04, 0.22, 0.22, 0.21, 0.21, 0.21, 0.21, 0.22, 0.22, 0.21, 0.21, 0.21, 0.21, 0.01, 0.01, 0.02, 0.01 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.05, 0.56, 0.6, 0.58, 0.59, 0.58, 0.58, 0.56, 0.59, 0.59, 0.59, 0.57, 0.57, 0.04, 0.03, 0.03, 0.04, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.03, 0.28, 0.29, 0.28, 0.28, 0.28, 0.3, 0.3, 0.27, 0.28, 0.27, 0.29, 0.3, 0.02, 0.02, 0.02, 0.01, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

End of debate, Spring Boot destroys competition for 1st scenario. Moreover, database is the bottleneck, and java runtime is clearly sleeping here. But JPA Hibernate was difficult to tune for optimal performance, and finally the magic @BatchSize annotation was the key, allowing to merge n+1 queries into 1+1 queries. Without it, Spring Boot was performing 3 times slower !

Spring Boot scenario 2

Iteration creation rate = 10/s

checks.........................: 100.00% ✓ 225040     ✗ 0
data_received..................: 921 MB  11 MB/s
data_sent......................: 19 MB   235 kB/s
dropped_iterations.............: 456     5.549905/s
http_req_blocked...............: avg=6.76µs  min=202ns   med=389ns    max=52.58ms  p(90)=662ns   p(95)=859ns
http_req_connecting............: avg=275ns   min=0s      med=0s       max=8.36ms   p(90)=0s      p(95)=0s
http_req_duration..............: avg=16.7ms  min=1.6ms   med=13.62ms  max=237.91ms p(90)=32.02ms p(95)=39.52ms
{ expected_response:true }...: avg=16.7ms  min=1.6ms   med=13.62ms  max=237.91ms p(90)=32.02ms p(95)=39.52ms
http_req_failed................: 0.00%   ✓ 0          ✗ 225040
http_req_receiving.............: avg=1.91ms  min=16.76µs med=886.69µs max=211.62ms p(90)=4.52ms  p(95)=7.22ms
http_req_sending...............: avg=85.38µs min=17.86µs med=45.75µs  max=77.73ms  p(90)=86.49µs p(95)=120.1µs
http_req_tls_handshaking.......: avg=5.83µs  min=0s      med=0s       max=51.92ms  p(90)=0s      p(95)=0s
http_req_waiting...............: avg=14.71ms min=0s      med=11.67ms  max=222.78ms p(90)=28.88ms p(95)=36.06ms
http_reqs......................: 225040  2738.92674/s
iteration_duration.............: avg=26.14s  min=22.07s  med=26.88s   max=28.52s   p(90)=28.02s  p(95)=28.16s
iterations.....................: 145     1.764772/s
vus............................: 7       min=7        max=50
vus_max........................: 50      min=50       max=50

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 33, 1444, 2295, 2566, 2449, 2563, 2918, 2891, 2886, 2676, 2674, 2943, 2975, 2971, 2757, 2645, 2966, 2967, 2925, 2622, 2678, 2888, 2984, 2972, 2667, 2639, 2896, 2974, 2853, 2760, 2747, 2897, 2952, 3026, 2633, 2569, 2890, 3020, 2701, 2521, 2680, 2922, 3013, 2983, 2674, 2622, 2790, 2463, 2693, 2352, 2853, 2762, 3000, 2960, 2653, 2615, 2987, 2870, 2875, 2536, 2593, 2903, 2990, 2712, 2550, 2532, 2903, 2955, 2683, 2485, 2626, 2930, 3004, 2858, 2664, 2596, 2937, 2942, 2899, 2517, 2436, 2577, 2012 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 10, 20, 30, 40, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 49, 50, 50, 48, 50, 50, 50, 50, 48, 49, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 49, 49, 49, 50, 50, 50, 50, 47, 48, 49, 46, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 43, 38, 37, 30, 23, 7 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 7, 5, 7, 10, 15, 18, 17, 17, 17, 19, 19, 17, 17, 17, 18, 19, 17, 17, 17, 19, 19, 17, 17, 17, 19, 19, 17, 17, 17, 18, 18, 17, 16, 16, 19, 19, 17, 16, 19, 20, 19, 17, 16, 17, 19, 19, 18, 20, 19, 21, 17, 18, 16, 17, 19, 19, 17, 17, 17, 19, 19, 16, 15, 16, 18, 18, 15, 15, 17, 18, 17, 16, 15, 15, 17, 17, 15, 15, 13, 15, 13, 10, 6 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.03, 0.08, 0.37, 0.41, 0.42, 0.41, 0.42, 0.43, 0.41, 0.41, 0.4, 0.38, 0.41, 0.4, 0.42, 0.39, 0.41, 0.41 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.01, 0.05, 0.26, 0.26, 0.27, 0.25, 0.27, 0.26, 0.26, 0.26, 0.26, 0.26, 0.26, 0.27, 0.27, 0.27, 0.26, 0.24 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.1, 0.52, 0.56, 0.57, 0.58, 0.57, 0.57, 0.55, 0.56, 0.56, 0.52, 0.57, 0.55, 0.52, 0.55, 0.56, 0.54, 0.12 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.04, 0.26, 0.3, 0.31, 0.28, 0.3, 0.3, 0.31, 0.3, 0.31, 0.28, 0.3, 0.3, 0.31, 0.3, 0.31, 0.29, 0.07 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

Java is maybe not the best DX experience for me, but it's a beast in terms of raw performance. Besides, we'll again have database bottleneck, which is the only case seen in this scenario on every framework tested ! Impossible to reach 100% java runtime CPU usage, even with 4 CPU cores, staying only at 60-70% overall...

ASP.NET Core

ASP.NET Core scenario 1

Iteration creation rate = 20/s

checks.........................: 100.00% ✓ 59109      ✗ 0
data_received..................: 1.3 GB  22 MB/s
data_sent......................: 5.2 MB  86 kB/s
dropped_iterations.............: 42      0.685501/s
http_req_blocked...............: avg=25.19µs  min=212ns    med=505ns    max=57.73ms  p(90)=940ns    p(95)=1.13µs
http_req_connecting............: avg=1.4µs    min=0s       med=0s       max=19.45ms  p(90)=0s       p(95)=0s
http_req_duration..............: avg=43.01ms  min=2.75ms   med=36.68ms  max=278.96ms p(90)=83.54ms  p(95)=99.8ms
{ expected_response:true }...: avg=43.01ms  min=2.75ms   med=36.68ms  max=278.96ms p(90)=83.54ms  p(95)=99.8ms
http_req_failed................: 0.00%   ✓ 0          ✗ 59109
http_req_receiving.............: avg=1.6ms    min=16.85µs  med=545.28µs max=53.91ms  p(90)=4.05ms   p(95)=6.85ms
http_req_sending...............: avg=164.54µs min=13.27µs  med=62.93µs  max=35.42ms  p(90)=166.95µs p(95)=376.28µs
http_req_tls_handshaking.......: avg=22.43µs  min=0s       med=0s       max=52.29ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=41.24ms  min=0s       med=34.73ms  max=275.91ms p(90)=81.52ms  p(95)=97.35ms
http_reqs......................: 59109   964.745322/s
iteration_duration.............: avg=2.22s    min=759.35ms med=2.33s    max=3.24s    p(90)=2.74s    p(95)=2.85s
iterations.....................: 1159    18.916575/s
vus............................: 21      min=17       max=50
vus_max........................: 50      min=50       max=50

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 242, 879, 870, 1010, 1007, 1026, 960, 941, 994, 1037, 1020, 990, 884, 997, 1001, 1022, 1005, 921, 941, 979, 984, 969, 891, 966, 978, 1018, 942, 902, 955, 998, 994, 1009, 907, 969, 991, 975, 999, 925, 960, 975, 999, 1001, 917, 967, 977, 1004, 1000, 919, 954, 1001, 992, 996, 936, 963, 999, 994, 944, 922, 958, 999, 1002, 632 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 17, 22, 27, 26, 27, 26, 30, 33, 32, 32, 31, 34, 36, 43, 42, 39, 42, 43, 46, 50, 49, 48, 49, 50, 46, 44, 47, 49, 49, 48, 50, 43, 50, 45, 43, 44, 47, 49, 47, 42, 48, 48, 49, 47, 49, 45, 45, 47, 48, 48, 49, 49, 49, 48, 49, 47, 46, 48, 45, 44, 21 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 12, 18, 23, 25, 25, 26, 28, 31, 31, 31, 31, 32, 38, 37, 39, 39, 39, 45, 45, 48, 49, 49, 53, 51, 49, 43, 47, 51, 50, 48, 46, 46, 50, 50, 44, 46, 45, 50, 49, 50, 44, 45, 50, 49, 48, 47, 47, 48, 50, 47, 46, 48, 51, 51, 48, 49, 49, 51, 49, 47, 41, 24 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.02, 0.03, 0.34, 0.47, 0.51, 0.5, 0.49, 0.48, 0.48, 0.49, 0.49, 0.48, 0.48, 0.49, 0.26, 0.03, 0.03, 0.02, 0.02 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.01, 0.02, 0.13, 0.21, 0.18, 0.19, 0.19, 0.19, 0.2, 0.19, 0.19, 0.2, 0.2, 0.18, 0.1, 0.01, 0.01, 0.02, 0.01 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.04, 0.43, 0.76, 0.75, 0.77, 0.78, 0.79, 0.79, 0.78, 0.78, 0.77, 0.78, 0.77, 0.47, 0.03, 0.03, 0.03, 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.15, 0.2, 0.22, 0.23, 0.22, 0.2, 0.21, 0.21, 0.21, 0.22, 0.22, 0.23, 0.13, 0.01, 0.02, 0.01, 0.01, 0.01 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

ASP.NET Core is performing well here. EF Core is incredibly efficient by default without any tuning headaches as it was with Sping Boot.

ASP.NET Core scenario 2

Iteration creation rate = 10/s

checks.........................: 100.00% ✓ 155200      ✗ 0
data_received..................: 939 MB  14 MB/s
data_sent......................: 14 MB   202 kB/s
dropped_iterations.............: 500     7.323443/s
http_req_blocked...............: avg=10.06µs min=181ns   med=418ns    max=74.82ms  p(90)=812ns    p(95)=997ns
http_req_connecting............: avg=437ns   min=0s      med=0s       max=9.65ms   p(90)=0s       p(95)=0s
http_req_duration..............: avg=20.44ms min=1.5ms   med=15.71ms  max=278.6ms  p(90)=41.79ms  p(95)=52.23ms
{ expected_response:true }...: avg=20.44ms min=1.5ms   med=15.71ms  max=278.6ms  p(90)=41.79ms  p(95)=52.23ms
http_req_failed................: 0.00%   ✓ 0           ✗ 155200
http_req_receiving.............: avg=1.52ms  min=14.25µs med=653.02µs max=205.85ms p(90)=3.58ms   p(95)=5.77ms
http_req_sending...............: avg=89.63µs min=12.82µs med=50.62µs  max=36.57ms  p(90)=102.45µs p(95)=140.6µs
http_req_tls_handshaking.......: avg=8.79µs  min=0s      med=0s       max=73.62ms  p(90)=0s       p(95)=0s
http_req_waiting...............: avg=18.82ms min=0s      med=14.01ms  max=278.29ms p(90)=39.41ms  p(95)=49.76ms
http_reqs......................: 155200  2273.196786/s
iteration_duration.............: avg=31.96s  min=28.57s  med=32.03s   max=34.2s    p(90)=33.6s    p(95)=33.75s
iterations.....................: 100     1.464689/s
vus............................: 8       min=8         max=50
vus_max........................: 50      min=50        max=50

{{< chart type="timeseries" title="Req/s count" >}} [ { label: 'Req/s', data: [ 143, 1375, 1715, 1720, 1816, 1913, 2037, 2110, 2211, 2142, 2381, 2448, 2417, 2323, 2194, 2387, 2478, 2279, 1817, 2181, 2463, 2412, 2454, 2260, 2247, 2405, 2495, 2530, 2173, 2184, 2480, 2476, 2423, 2146, 2209, 2382, 2513, 2344, 2336, 2266, 2513, 2446, 2496, 2382, 2214, 2371, 2477, 2444, 2269, 2210, 2439, 2498, 2450, 2349, 2195, 2448, 2508, 2406, 2316, 2219, 2528, 2503, 2470, 2285, 2179, 2424, 2374, 2224, 1278 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="VUs count" >}} [ { label: 'VUs', data: [ 10, 20, 30, 40, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 49, 50, 50, 49, 50, 47, 49, 48, 48, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 49, 48, 46, 45, 41, 36, 30, 8 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Request duration in ms" >}} [ { label: 'Duration (ms)', data: [ 5, 6, 11, 17, 21, 25, 24, 24, 22, 23, 21, 20, 21, 21, 23, 21, 20, 22, 27, 23, 20, 21, 20, 23, 22, 21, 20, 20, 22, 23, 20, 20, 20, 23, 22, 20, 19, 21, 20, 22, 20, 21, 20, 21, 23, 21, 20, 20, 22, 23, 20, 20, 20, 21, 23, 20, 20, 21, 21, 22, 20, 19, 19, 20, 20, 17, 15, 13, 9 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU runtime load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.02, 0.04, 0.42, 0.54, 0.6, 0.6, 0.63, 0.63, 0.62, 0.63, 0.63, 0.64, 0.64, 0.62, 0.62, 0.31, 0.02, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.01, 0.03, 0.19, 0.25, 0.3, 0.3, 0.31, 0.31, 0.3, 0.31, 0.3, 0.31, 0.31, 0.31, 0.31, 0.16, 0.01, 0.01 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

{{< chart type="timeseries" title="CPU database load" stacked="true" max="1" step="5" >}} [ { label: 'User', data: [ 0.03, 0.03, 0.4, 0.49, 0.56, 0.55, 0.56, 0.56, 0.54, 0.55, 0.56, 0.55, 0.54, 0.55, 0.55, 0.31, 0.03, 0.04, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', fill: true }, { label: 'System', data: [ 0.02, 0.02, 0.23, 0.3, 0.32, 0.29, 0.32, 0.32, 0.32, 0.34, 0.32, 0.34, 0.34, 0.33, 0.32, 0.18, 0.01, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', fill: true } ] {{< /chart >}}

Not that far to Java variant, just a bit behind. But as workers are fully loaded here, contrary to Spring Boot which is limited by database, Java stays by far the clear winner for raw performance (in sacrifice of some memory obviously).

Conclusion

Here are the final req/s results for each framework against PgSQL database.

{{< chart type="timeseries" title="Scenario 1" >}} [ { label: 'Laravel', borderColor: '#c2410c', backgroundColor: '#c2410c', data: [ 48, 263, 335, 331, 355, 373, 373, 376, 379, 359, 383, 382, 386, 388, 351, 387, 374, 367, 378, 348, 396, 367, 352, 350, 324, 344, 388, 346, 330, 368, 383, 376, 383, 372, 350, 391, 380, 365, 367, 340, 380, 375, 381, 383, 354, 380, 382, 384, 372, 345, 379, 342, 370, 357, 321, 304, 306, 275, 301, 318, 360, 360, 365, 240 ] }, { label: 'Symfony', borderColor: '#ffffff', backgroundColor: '#ffffff', data: [ 122, 341, 348, 346, 359, 338, 361, 359, 345, 360, 330, 352, 361, 344, 359, 334, 362, 323, 346, 358, 335, 357, 352, 348, 358, 337, 359, 357, 345, 357, 339, 358, 350, 333, 355, 332, 359, 359, 349, 358, 333, 356, 359, 341, 352, 333, 355, 360, 344, 356, 340, 360, 360, 348, 355, 337, 346, 354, 345, 359, 337, 354, 341, 338, 179 ] }, { label: 'FastAPI', borderColor: '#0f766e', backgroundColor: '#0f766e', data: [ 270, 514, 530, 535, 552, 541, 524, 493, 545, 555, 560, 539, 519, 545, 540, 531, 525, 514, 547, 540, 537, 533, 485, 511, 534, 525, 508, 500, 550, 527, 538, 516, 500, 542, 532, 530, 504, 508, 540, 538, 553, 537, 497, 560, 517, 578, 559, 487, 551, 546, 538, 531, 517, 518, 578, 559, 521, 516, 556, 567, 517, 517, 351 ] }, { label: 'NestJS', borderColor: '#b91c1c', backgroundColor: '#b91c1c', data: [ 273, 496, 577, 588, 599, 624, 621, 610, 641, 638, 641, 614, 585, 580, 600, 604, 601, 571, 606, 663, 643, 572, 596, 585, 616, 650, 680, 615, 612, 611, 617, 572, 587, 593, 605, 633, 633, 573, 646, 645, 650, 570, 629, 653, 691, 650, 580, 555, 590, 646, 565, 585, 638, 594, 567, 555, 602, 570, 641, 648, 601, 630, 8 ] }, { label: 'Spring Boot', borderColor: '#15803d', backgroundColor: '#15803d', data: [ 353, 1407, 1575, 1522, 1483, 1562, 1587, 1578, 1491, 1521, 1545, 1561, 1523, 1493, 1392, 1604, 1609, 1526, 1554, 1493, 1547, 1558, 1531, 1484, 1511, 1530, 1606, 1548, 1479, 1459, 1574, 1582, 1575, 1481, 1439, 1615, 1304, 1567, 1571, 1530, 1610, 1604, 1516, 1523, 1433, 1630, 1503, 1532, 1557, 1492, 1559, 1577, 1521, 1497, 1446, 1583, 1566, 1509, 1424, 1514, 1385 ] }, { label: 'ASP.NET Core', borderColor: '#6d28d9', backgroundColor: '#6d28d9', data: [ 242, 879, 870, 1010, 1007, 1026, 960, 941, 994, 1037, 1020, 990, 884, 997, 1001, 1022, 1005, 921, 941, 979, 984, 969, 891, 966, 978, 1018, 942, 902, 955, 998, 994, 1009, 907, 969, 991, 975, 999, 925, 960, 975, 999, 1001, 917, 967, 977, 1004, 1000, 919, 954, 1001, 992, 996, 936, 963, 999, 994, 944, 922, 958, 999, 1002, 632 ] } ] {{< /chart >}}

{{< chart type="timeseries" title="Scenario 2" >}} [ { label: 'Laravel', borderColor: '#c2410c', backgroundColor: '#c2410c', data: [ 59, 137, 245, 368, 393, 519, 334, 446, 490, 576, 526, 397, 379, 482, 597, 623, 652, 626, 631, 612, 631, 637, 608, 630, 651, 625, 637, 632, 671, 622, 683, 631, 655, 641, 632, 673, 659, 603, 650, 652, 642, 623, 668, 678, 642, 640, 498, 628, 638, 665, 624, 665, 630, 650, 669, 692, 628, 588, 650, 628, 687, 687, 639, 629, 632, 649, 657, 608, 669, 648, 658, 601, 618, 654, 654, 654, 521, 508, 317, 634, 685, 608, 662, 593, 675 ] }, { label: 'Symfony', borderColor: '#ffffff', backgroundColor: '#ffffff', data: [ 91, 207, 385, 748, 820, 959, 1075, 964, 1238, 1212, 1320, 1289, 1154, 1361, 1365, 1307, 1346, 1115, 1328, 1461, 1238, 1417, 1237, 953, 1350, 1272, 1331, 1155, 1105, 1017, 1196, 1108, 815, 824, 1327, 1224, 1334, 1213, 1410, 1430, 1365, 1394, 1299, 1165, 1427, 1419, 1341, 1234, 1418, 957, 1035, 1028, 929, 880, 1414, 1491, 1388, 1226, 1510, 1408, 1366, 1571, 1048, 1439, 1486, 1360, 1401, 1216, 1392, 1408, 1486, 1440, 1260, 1312, 1341, 1271, 1250, 964, 628, 4 ] }, { label: 'FastAPI', borderColor: '#0f766e', backgroundColor: '#0f766e', data: [ 19, 168, 465, 631, 720, 792, 752, 758, 757, 763, 839, 819, 777, 770, 821, 828, 760, 792, 741, 869, 862, 831, 846, 811, 820, 878, 792, 811, 815, 829, 804, 807, 842, 819, 791, 804, 744, 839, 810, 828, 841, 890, 841, 834, 804, 829, 821, 837, 852, 853, 853, 884, 871, 773, 774, 825, 794, 832, 825, 787, 807, 872, 837, 815, 826, 778, 811, 810, 823, 807, 786, 872, 886, 810, 808, 831, 824, 853, 770, 818, 793, 827, 795, 813, 795, 858, 869, 805, 846, 823, 563 ] }, { label: 'NestJS', borderColor: '#b91c1c', backgroundColor: '#b91c1c', data: [ 121, 508, 755, 899, 1012, 1007, 1156, 1177, 1096, 1180, 1224, 1237, 1208, 1244, 1295, 1437, 1445, 1345, 1338, 1405, 1378, 1380, 1411, 1293, 1420, 1441, 1451, 1365, 1264, 1439, 1384, 1584, 1241, 1361, 1319, 1427, 1398, 1362, 1320, 1448, 1482, 1458, 1311, 1256, 1399, 1363, 1345, 1259, 1346, 1443, 1499, 1445, 1438, 1451, 1425, 1472, 1479, 1367, 1322, 1450, 1414, 1360, 1355, 1457, 1326, 1411, 1363, 1350, 1277, 1279, 1168, 1216, 1198, 1256, 1314, 1248, 1236, 1192, 1183, 1227, 1263, 1357, 1148, 1141, 1168, 1127, 900, 25 ] }, { label: 'Spring Boot', borderColor: '#15803d', backgroundColor: '#15803d', data: [ 33, 1444, 2295, 2566, 2449, 2563, 2918, 2891, 2886, 2676, 2674, 2943, 2975, 2971, 2757, 2645, 2966, 2967, 2925, 2622, 2678, 2888, 2984, 2972, 2667, 2639, 2896, 2974, 2853, 2760, 2747, 2897, 2952, 3026, 2633, 2569, 2890, 3020, 2701, 2521, 2680, 2922, 3013, 2983, 2674, 2622, 2790, 2463, 2693, 2352, 2853, 2762, 3000, 2960, 2653, 2615, 2987, 2870, 2875, 2536, 2593, 2903, 2990, 2712, 2550, 2532, 2903, 2955, 2683, 2485, 2626, 2930, 3004, 2858, 2664, 2596, 2937, 2942, 2899, 2517, 2436, 2577, 2012 ] }, { label: 'ASP.NET Core', borderColor: '#6d28d9', backgroundColor: '#6d28d9', data: [ 143, 1375, 1715, 1720, 1816, 1913, 2037, 2110, 2211, 2142, 2381, 2448, 2417, 2323, 2194, 2387, 2478, 2279, 1817, 2181, 2463, 2412, 2454, 2260, 2247, 2405, 2495, 2530, 2173, 2184, 2480, 2476, 2423, 2146, 2209, 2382, 2513, 2344, 2336, 2266, 2513, 2446, 2496, 2382, 2214, 2371, 2477, 2444, 2269, 2210, 2439, 2498, 2450, 2349, 2195, 2448, 2508, 2406, 2316, 2219, 2528, 2503, 2470, 2285, 2179, 2424, 2374, 2224, 1278 ] } ] {{< /chart >}}

To resume, compiled languages have always a clear advantage when it comes to raw performance. But do you really need it ?

Performance isn't the main criteria for a web framework. The DX is also very important, in that regards Laravel stays a very nice candidate, and you always have Octane for high performance if needed.

As we have seen with Symfony, PHP is now really back in the game in term of raw performance, competing against NodeJS. And no more any headaches for worker configuration thanks to the excellent FrankenPHP runtime which provides production optimized docker images.

When it comes to compiled languages, I still personally prefer the DX of ASP.NET Core over Spring Boot. The performance gap is negligible, and it hasn't this warmup Java feeling and keeps a raisonable memory footprint.

100 KiB Raw Blame History

The contenders

Side note on PHP configuration

The Swarm cluster for testing

The deployment configuration

The k6 scenarios

Scenario 1 - Database intensive

Scenario 2 - Runtime intensive

The results

Laravel (Octane)

Laravel MySQL scenario 1

Laravel MySQL scenario 2

Laravel PgSQL scenario 1

Laravel PgSQL scenario 2

Symfony (FrankenPHP)

Symfony MySQL scenario 1

Symfony MySQL scenario 2

Symfony PgSQL scenario 1

Symfony PgSQL scenario 2

FastAPI

FastAPI scenario 1

FastAPI scenario 2

NestJS

NestJS scenario 1

NestJS scenario 2

Spring Boot

Spring Boot scenario 1

Spring Boot scenario 2

ASP.NET Core

ASP.NET Core scenario 1

ASP.NET Core scenario 2

Conclusion

100 KiB

Raw Blame History