diff --git a/content/posts/22-web-api-benchmarks-2024/index.md b/content/posts/22-web-api-benchmarks-2024/index.md index 7b8da66..376aecc 100644 --- a/content/posts/22-web-api-benchmarks-2024/index.md +++ b/content/posts/22-web-api-benchmarks-2024/index.md @@ -5,7 +5,7 @@ tags: ["kubernetes", "docker", "load-testing", "k6", "webapi"] --- {{< lead >}} -We'll be comparing the read performance of 6 Web APIs frameworks, sharing the same OpenAPI contract from [realworld app](https://github.com/gothinkster/realworld), a medium-like clone, implemented under multiple languages (PHP, Python, Javascript, Java and C#). +We'll be comparing the read performance of 6 Web APIs frameworks, sharing the same OpenAPI contract from [realworld app](https://github.com/gothinkster/realworld), a medium-like clone, implemented under multiple languages (PHP, Python, Javascript, Java and C#). Updated in 09/2024. {{< /lead >}} {{< alert >}} @@ -14,7 +14,7 @@ Update April 2024 for PHP: I replaced previous Apache results by new [FrankenPHP This is not a basic synthetic benchmark, but a real world benchmark with DB data tests, and multiple scenarios. This post may be updated when new versions of frameworks will be released or any suggestions for performance related improvement in below commentary section. -A state of the art of real world benchmarks comparison of Web APIs is difficult to achieve and very time-consuming as it forces to master each framework. As performance can highly dependent of: +A state of the art of real world benchmarks comparison of Web APIs is difficult to achieve and very time-consuming as it forces mastering each framework. As performance can highly dependent of: - Code implementation, all made by my own - Fine-tuning for each runtime, so I mostly take the default configuration @@ -23,7 +23,7 @@ Now that's said, let's fight ! ## The contenders -We'll be using the very last up-to-date stable versions of each frameworks, and the latest stable version of the runtime. +We'll be using the very last up-to-date stable versions of each framework, and the latest stable version of the runtime. I give you all source code as well as public OCI artifacts of each project, so you can test it by yourself quickly. @@ -33,14 +33,14 @@ I give you all source code as well as public OCI artifacts of each project, so y | [Symfony 7](https://github.com/adr1enbe4udou1n/symfony-realworld-example-app) ([api](https://symfonyrealworld.okami101.io/api/) / [image](https://gitea.okami101.io/conduit/-/packages/container/symfony/latest)) | FrankenPHP 8.3 | Doctrine | | [FastAPI](https://github.com/adr1enbe4udou1n/fastapi-realworld-example-app) ([api](https://fastapirealworld.okami101.io/api/) / [image](https://gitea.okami101.io/conduit/-/packages/container/fastapi/latest)) | Python 3.12 | SQLAlchemy 2.0 | | [NestJS 10](https://github.com/adr1enbe4udou1n/nestjs-realworld-example-app) ([api](https://nestjsrealworld.okami101.io/api/) / [image](https://gitea.okami101.io/conduit/-/packages/container/nestjs/latest)) | Node 20 | Prisma 5 | -| [Spring Boot 3.2](https://github.com/adr1enbe4udou1n/spring-boot-realworld-example-app) ([api](https://springbootrealworld.okami101.io/api/) / [image](https://gitea.okami101.io/conduit/-/packages/container/spring-boot/latest)) | Java 21 | Hibernate 6 | +| [Spring Boot 3.3](https://github.com/adr1enbe4udou1n/spring-boot-realworld-example-app) ([api](https://springbootrealworld.okami101.io/api/) / [image](https://gitea.okami101.io/conduit/-/packages/container/spring-boot/latest)) | Java 21 | Hibernate 6 | | [ASP.NET Core 8](https://github.com/adr1enbe4udou1n/aspnetcore-realworld-example-app) ([api](https://aspnetcorerealworld.okami101.io/api/) / [image](https://gitea.okami101.io/conduit/-/packages/container/aspnet-core/latest)) | .NET 8.0 | EF Core 8 | Each project are: - Using PostgreSQL - Using the same OpenAPI contract -- Fully tested and fonctional against same [Postman collection](https://github.com/gothinkster/realworld/blob/main/api/Conduit.postman_collection.json) +- Fully tested and functional against same [Postman collection](https://github.com/gothinkster/realworld/blob/main/api/Conduit.postman_collection.json) - Highly tooled with high code quality in mind (static analyzers, formatter, linters, good code coverage, etc.) - Share roughly the same amount of DB datasets, 50 users, 500 articles, 5000 comments, generated by faker-like library for each language - Avoiding N+1 queries with eager loading (normally) @@ -48,7 +48,7 @@ Each project are: ## The Swarm cluster for testing -We'll running all Web APIs project on a Docker swarm cluster, where each node are composed of 2 dedicated CPUs for stable performance and 8 GB of RAM. I'll use 4 CCX13 instances from Hetzner. +We'll be running all Web APIs project on a Docker swarm cluster, where each node are composed of 2 dedicated CPUs for stable performance and 8 GB of RAM. I'll use 4 CCX13 instances from Hetzner. Traefik will be used as a reverse proxy, load balancing the requests to the replicas of each node. @@ -355,9 +355,9 @@ We'll be using [k6](https://k6.io/) to run the tests, with [constant-arrival-rat - **Scenario 1** : fetch all articles, following the pagination - **Scenario 2** : fetch all articles, calling each single article with slug, fetch associated comments for each article, and fetch profile of each related author -Duration of each scenario is 1 minute, with a 30 seconds graceful for finishing last started iterations. Results with one single test failures, i.e. any response status different than 200 or any response json error parsing, are not accepted. +Duration of each scenario is 1 minute, with a 30 seconds graceful for finishing last started iterations. Results with one single test failures, i.e. any response status different from 200 or any response JSON error parsing, are not accepted. -The **Iteration creation rate** (rate / timeUnit) will be choosen in order to obtain the highest possible request rate, without any test failures. +The **Iteration creation rate** (rate / time unit) will be chosen in order to obtain the highest possible request rate, without any test failures. ### Scenario 1 - Database intensive @@ -444,7 +444,7 @@ SELECT * FROM favorites WHERE article_id IN (); ``` {{< alert >}} -It can highly differ according to each ORM, as few of them can prefer to reduce the queries by using subselect, but it's a good approximation. +It can highly differ according to each ORM, as few of them can prefer to reduce the queries by using sub select, but it's a good approximation. {{< /alert >}} ### Scenario 2 - Runtime intensive @@ -523,23 +523,23 @@ Laravel Octane will be enabled with FrankenPHP runtime. Iteration creation rate = **10/s** ```txt -checks.........................: 100.00% ✓ 22542 ✗ 0 -data_received..................: 241 MB 3.8 MB/s -data_sent......................: 2.0 MB 31 kB/s -dropped_iterations.............: 159 2.509297/s -http_req_blocked...............: avg=98.83µs min=214ns med=955ns max=190.07ms p(90)=1.48µs p(95)=1.68µs -http_req_connecting............: avg=9µs min=0s med=0s max=71.9ms p(90)=0s p(95)=0s -http_req_duration..............: avg=127.36ms min=4ms med=82.05ms max=644.29ms p(90)=247.64ms p(95)=269.15ms - { expected_response:true }...: avg=127.36ms min=4ms med=82.05ms max=644.29ms p(90)=247.64ms p(95)=269.15ms -http_req_failed................: 0.00% ✓ 0 ✗ 22542 -http_req_receiving.............: avg=938.08µs min=26.46µs med=340.84µs max=212.74ms p(90)=1.26ms p(95)=2.26ms -http_req_sending...............: avg=212.82µs min=14.12µs med=110.91µs max=67.83ms p(90)=202.2µs p(95)=308.11µs -http_req_tls_handshaking.......: avg=80.92µs min=0s med=0s max=124.88ms p(90)=0s p(95)=0s -http_req_waiting...............: avg=126.21ms min=0s med=80.18ms max=643.75ms p(90)=246.31ms p(95)=267.53ms -http_reqs......................: 22542 355.752049/s -iteration_duration.............: avg=6.55s min=2.25s med=6.66s max=9.52s p(90)=7.88s p(95)=8.26s -iterations.....................: 442 6.97553/s -vus............................: 15 min=10 max=50 +checks.........................: 100.00% ✓ 22287 ✗ 0 +data_received..................: 236 MB 3.7 MB/s +data_sent......................: 1.9 MB 31 kB/s +dropped_iterations.............: 164 2.570365/s +http_req_blocked...............: avg=40.43µs min=272ns med=699ns max=64ms p(90)=1.11µs p(95)=1.29µs +http_req_connecting............: avg=2.37µs min=0s med=0s max=9.31ms p(90)=0s p(95)=0s +http_req_duration..............: avg=129.05ms min=4.37ms med=99.97ms max=338.95ms p(90)=244.5ms p(95)=254.83ms + { expected_response:true }...: avg=129.05ms min=4.37ms med=99.97ms max=338.95ms p(90)=244.5ms p(95)=254.83ms +http_req_failed................: 0.00% ✓ 0 ✗ 22287 +http_req_receiving.............: avg=381.18µs min=34.13µs med=297.73µs max=15.25ms p(90)=588µs p(95)=780.82µs +http_req_sending...............: avg=112.76µs min=36.13µs med=99.03µs max=10.11ms p(90)=154.25µs p(95)=183.11µs +http_req_tls_handshaking.......: avg=34.66µs min=0s med=0s max=29.3ms p(90)=0s p(95)=0s +http_req_waiting...............: avg=128.55ms min=4.17ms med=99.52ms max=338.41ms p(90)=243.94ms p(95)=254.31ms +http_reqs......................: 22287 349.303183/s +iteration_duration.............: avg=6.62s min=1.33s med=6.87s max=9.38s p(90)=7.77s p(95)=7.99s +iterations.....................: 437 6.849082/s +vus............................: 25 min=10 max=50 vus_max........................: 50 min=50 max=50 ``` @@ -551,14 +551,14 @@ vus_max........................: 50 min=50 max=50 { label: 'Req/s', data: [ - 48, 263, 335, 331, 355, 373, 373, 376, 379, - 359, 383, 382, 386, 388, 351, 387, 374, 367, - 378, 348, 396, 367, 352, 350, 324, 344, 388, - 346, 330, 368, 383, 376, 383, 372, 350, 391, - 380, 365, 367, 340, 380, 375, 381, 383, 354, - 380, 382, 384, 372, 345, 379, 342, 370, 357, - 321, 304, 306, 275, 301, 318, 360, 360, 365, - 240 + 8, 273, 355, 346, 357, 345, 361, 355, 350, + 351, 335, 345, 362, 342, 356, 347, 362, 363, + 350, 359, 349, 361, 348, 350, 358, 342, 364, + 366, 346, 362, 348, 355, 360, 336, 349, 333, + 353, 348, 349, 345, 332, 367, 362, 348, 351, + 331, 351, 351, 348, 359, 353, 356, 357, 346, + 363, 346, 349, 360, 351, 359, 345, 349, 351, + 353, 205 ] } ] @@ -573,12 +573,12 @@ vus_max........................: 50 min=50 max=50 { label: 'VUs', data: [ - 10, 20, 26, 32, 38, 42, 49, 50, 47, 50, 48, 49, - 46, 49, 49, 50, 48, 50, 49, 46, 45, 49, 50, 47, - 50, 50, 48, 50, 48, 49, 50, 49, 50, 48, 48, 47, - 48, 50, 46, 48, 49, 50, 50, 48, 49, 47, 47, 48, - 50, 50, 48, 50, 47, 48, 50, 50, 50, 48, 49, 50, - 43, 25, 15 + 10, 15, 21, 26, 32, 38, 44, 49, 50, 49, 49, 46, + 50, 48, 46, 48, 48, 49, 47, 49, 48, 49, 43, 49, + 49, 49, 50, 50, 50, 50, 48, 44, 50, 50, 50, 48, + 50, 49, 50, 48, 50, 49, 50, 50, 49, 50, 49, 49, + 47, 49, 49, 50, 50, 50, 49, 48, 48, 49, 50, 50, + 43, 40, 25 ] } ] @@ -589,13 +589,13 @@ vus_max........................: 50 min=50 max=50 { label: 'Duration (ms)', data: [ - 38, 40, 59, 75, 89, 100, 115, 129, 128, 133, - 125, 129, 127, 122, 137, 129, 130, 134, 127, 143, - 117, 121, 138, 142, 147, 146, 129, 139, 143, 134, - 127, 131, 127, 131, 141, 124, 126, 130, 140, 138, - 130, 130, 130, 128, 138, 127, 123, 125, 132, 141, - 132, 140, 134, 131, 147, 162, 160, 157, 190, 156, - 133, 118, 69, 45 + 25, 25, 41, 56, 70, 87, 100, 116, 133, 139, + 145, 144, 134, 145, 138, 133, 136, 136, 141, 134, + 136, 138, 140, 130, 129, 136, 140, 135, 141, 134, + 135, 143, 122, 145, 143, 135, 141, 140, 144, 140, + 141, 136, 135, 146, 133, 144, 144, 139, 142, 136, + 130, 142, 138, 145, 136, 137, 142, 133, 139, 136, + 136, 139, 120, 91, 41 ] } ] @@ -609,11 +609,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.03, 0.03, 0.35, 0.72, - 0.7, 0.69, 0.69, 0.66, - 0.7, 0.68, 0.68, 0.67, - 0.66, 0.56, 0.62, 0.05, - 0.03, 0.02, 0.04 + 0.03, 0.03, 0.64, 0.68, + 0.69, 0.69, 0.67, 0.68, + 0.66, 0.65, 0.68, 0.68, + 0.67, 0.68, 0.5, 0.04, + 0.04 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -622,11 +622,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.01, 0.02, 0.07, 0.12, - 0.13, 0.12, 0.11, 0.11, - 0.12, 0.13, 0.12, 0.13, - 0.13, 0.1, 0.12, 0.02, - 0.01, 0.02, 0.02 + 0.02, 0.02, 0.15, 0.17, + 0.16, 0.16, 0.18, 0.15, + 0.18, 0.17, 0.18, 0.15, + 0.17, 0.16, 0.13, 0.03, + 0.03 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -640,11 +640,10 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.02, 0.02, 0.1, 0.22, - 0.22, 0.22, 0.21, 0.2, - 0.21, 0.23, 0.23, 0.22, - 0.22, 0.2, 0.21, 0.07, - 0.02, 0.03, 0.02 + 0.03, 0.02, 0.08, 0.2, 0.2, + 0.2, 0.2, 0.2, 0.21, 0.2, + 0.2, 0.21, 0.2, 0.2, 0.2, + 0.1, 0.02, 0.02 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -653,11 +652,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.02, 0.02, 0.07, 0.15, - 0.14, 0.13, 0.13, 0.13, - 0.15, 0.13, 0.13, 0.13, - 0.14, 0.13, 0.13, 0.04, - 0.02, 0.01, 0.01 + 0.02, 0.02, 0.07, 0.16, + 0.15, 0.17, 0.17, 0.16, + 0.16, 0.16, 0.15, 0.16, + 0.17, 0.15, 0.16, 0.08, + 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -676,23 +675,23 @@ We are not database limited. Iteration creation rate = **1/s** ```txt -checks.........................: 100.00% ✓ 53448 ✗ 0 -data_received..................: 122 MB 1.4 MB/s +checks.........................: 100.00% ✓ 53163 ✗ 0 +data_received..................: 120 MB 1.3 MB/s data_sent......................: 4.4 MB 49 kB/s -dropped_iterations.............: 8 0.088883/s -http_req_blocked...............: avg=28.83µs min=188ns med=900ns max=130.65ms p(90)=1.4µs p(95)=1.57µs -http_req_connecting............: avg=3.26µs min=0s med=0s max=40.47ms p(90)=0s p(95)=0s -http_req_duration..............: avg=59.09ms min=3.07ms med=49.91ms max=426.79ms p(90)=121.42ms p(95)=137.82ms - { expected_response:true }...: avg=59.09ms min=3.07ms med=49.91ms max=426.79ms p(90)=121.42ms p(95)=137.82ms -http_req_failed................: 0.00% ✓ 0 ✗ 53448 -http_req_receiving.............: avg=1.3ms min=17.89µs med=202.25µs max=277.84ms p(90)=1.17ms p(95)=2.63ms -http_req_sending...............: avg=271.85µs min=22.02µs med=101.15µs max=269.13ms p(90)=197.28µs p(95)=281.99µs -http_req_tls_handshaking.......: avg=23.83µs min=0s med=0s max=89.85ms p(90)=0s p(95)=0s -http_req_waiting...............: avg=57.51ms min=0s med=48.78ms max=423.57ms p(90)=119.42ms p(95)=135.29ms -http_reqs......................: 53448 593.825611/s -iteration_duration.............: avg=1m3s min=45.9s med=1m4s max=1m19s p(90)=1m17s p(95)=1m18s -iterations.....................: 9 0.099993/s -vus............................: 44 min=1 max=50 +dropped_iterations.............: 6 0.066664/s +http_req_blocked...............: avg=17.49µs min=252ns med=678ns max=76.67ms p(90)=1.09µs p(95)=1.27µs +http_req_connecting............: avg=732ns min=0s med=0s max=3.15ms p(90)=0s p(95)=0s +http_req_duration..............: avg=59.48ms min=3.41ms med=49.42ms max=336.19ms p(90)=121.32ms p(95)=134.81ms + { expected_response:true }...: avg=59.48ms min=3.41ms med=49.42ms max=336.19ms p(90)=121.32ms p(95)=134.81ms +http_req_failed................: 0.00% ✓ 0 ✗ 53164 +http_req_receiving.............: avg=219.66µs min=23.57µs med=142.94µs max=15.94ms p(90)=400.62µs p(95)=567.21µs +http_req_sending...............: avg=108.62µs min=31.16µs med=95.84µs max=13.53ms p(90)=153.78µs p(95)=181.93µs +http_req_tls_handshaking.......: avg=14.78µs min=0s med=0s max=29.33ms p(90)=0s p(95)=0s +http_req_waiting...............: avg=59.15ms min=3.25ms med=49.07ms max=335.49ms p(90)=120.95ms p(95)=134.56ms +http_reqs......................: 53164 590.690863/s +iteration_duration.............: avg=58.57s min=40.08s med=1m0s max=1m19s p(90)=1m16s p(95)=1m17s +iterations.....................: 9 0.099997/s +vus............................: 45 min=1 max=50 vus_max........................: 50 min=50 max=50 ``` @@ -704,14 +703,15 @@ vus_max........................: 50 min=50 max=50 { label: 'Req/s', data: [ - 59, 137, 245, 368, 393, 519, 334, 446, 490, 576, 526, - 397, 379, 482, 597, 623, 652, 626, 631, 612, 631, 637, - 608, 630, 651, 625, 637, 632, 671, 622, 683, 631, 655, - 641, 632, 673, 659, 603, 650, 652, 642, 623, 668, 678, - 642, 640, 498, 628, 638, 665, 624, 665, 630, 650, 669, - 692, 628, 588, 650, 628, 687, 687, 639, 629, 632, 649, - 657, 608, 669, 648, 658, 601, 618, 654, 654, 654, 521, - 508, 317, 634, 685, 608, 662, 593, 675 + 8, 83, 208, 275, 431, 479, 510, 536, 513, 589, 600, + 578, 584, 589, 616, 611, 581, 621, 597, 608, 649, 579, + 607, 594, 635, 646, 570, 622, 576, 626, 641, 607, 623, + 603, 614, 623, 602, 614, 617, 645, 628, 570, 603, 599, + 646, 632, 622, 614, 592, 591, 647, 614, 613, 623, 612, + 610, 624, 627, 613, 640, 623, 574, 584, 586, 630, 613, + 608, 634, 603, 640, 643, 616, 627, 607, 629, 631, 628, + 624, 626, 640, 622, 576, 636, 619, 647, 646, 596, 625, + 606, 634, 361 ] } ] @@ -726,13 +726,14 @@ vus_max........................: 50 min=50 max=50 { label: 'VUs', data: [ - 1, 2, 3, 4, 5, 6, 8, 8, 9, 10, 11, 12, - 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, - 26, 27, 28, 29, 30, 31, 33, 34, 35, 36, 37, 38, - 39, 40, 41, 42, 43, 44, 45, 45, 46, 47, 47, 48, - 49, 50, 50, 50, 50, 49, 50, 50, 50, 50, 49, 49, - 49, 49, 49, 49, 49, 49, 48, 48, 48, 48, 47, 47, - 47, 47, 47, 47, 47, 47, 46, 46, 45, 45 + 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, + 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, + 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, + 37, 38, 39, 40, 40, 41, 42, 42, 43, 43, 44, 45, + 46, 47, 48, 49, 50, 50, 50, 49, 50, 50, 50, 50, + 50, 50, 50, 50, 49, 49, 49, 49, 48, 48, 48, 48, + 48, 48, 48, 48, 47, 47, 47, 47, 47, 47, 46, 46, + 46, 46, 46, 45, 45, 45 ] } ] @@ -743,14 +744,14 @@ vus_max........................: 50 min=50 max=50 { label: 'Duration (ms)', data: [ - 16, 14, 12, 11, 12, 11, 20, 18, 18, 17, 21, 29, - 31, 30, 25, 25, 26, 27, 30, 33, 33, 34, 37, 38, - 39, 41, 42, 43, 44, 48, 46, 48, 50, 53, 56, 54, - 54, 61, 61, 62, 63, 66, 64, 63, 72, 71, 92, 71, - 75, 73, 77, 74, 76, 79, 75, 70, 78, 82, 80, 81, - 70, 71, 72, 81, 78, 75, 75, 78, 75, 75, 73, 77, - 76, 75, 72, 72, 89, 89, 147, 74, 67, 77, 67, 80, - 65 + 39, 16, 11, 12, 10, 11, 12, 14, 16, 16, 17, 19, + 21, 22, 23, 25, 28, 28, 30, 32, 31, 36, 37, 39, + 39, 39, 46, 43, 50, 47, 47, 51, 52, 53, 58, 56, + 61, 60, 62, 60, 63, 71, 68, 71, 66, 66, 71, 71, + 73, 80, 74, 78, 81, 78, 82, 82, 81, 78, 81, 79, + 80, 87, 84, 82, 82, 81, 79, 78, 80, 76, 75, 78, + 76, 79, 75, 76, 76, 76, 75, 73, 75, 81, 74, 74, + 72, 71, 75, 74, 73, 72, 71 ] } ] @@ -764,11 +765,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.03, 0.08, 0.37, 0.46, - 0.51, 0.59, 0.58, 0.6, - 0.62, 0.61, 0.6, 0.59, - 0.6, 0.59, 0.58, 0.61, - 0.58, 0.51, 0.64 + 0.03, 0.15, 0.47, 0.56, + 0.6, 0.59, 0.61, 0.63, + 0.6, 0.59, 0.61, 0.58, + 0.58, 0.59, 0.59, 0.62, + 0.62, 0.65, 0.64 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -777,11 +778,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.02, 0.03, 0.08, 0.11, - 0.12, 0.13, 0.13, 0.14, - 0.13, 0.16, 0.14, 0.14, - 0.14, 0.14, 0.15, 0.16, - 0.15, 0.12, 0.14 + 0.03, 0.06, 0.14, 0.17, + 0.18, 0.18, 0.19, 0.18, + 0.2, 0.21, 0.19, 0.18, + 0.2, 0.18, 0.19, 0.19, + 0.19, 0.17, 0.18 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -795,11 +796,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.02, 0.08, 0.22, 0.22, - 0.25, 0.25, 0.27, 0.26, - 0.27, 0.26, 0.27, 0.26, - 0.28, 0.27, 0.26, 0.26, - 0.26, 0.26, 0.27 + 0.02, 0.02, 0.13, 0.22, + 0.24, 0.26, 0.25, 0.22, + 0.25, 0.25, 0.24, 0.25, + 0.25, 0.25, 0.25, 0.25, + 0.25, 0.25, 0.25 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -808,11 +809,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.02, 0.05, 0.13, 0.14, - 0.16, 0.16, 0.16, 0.17, - 0.16, 0.16, 0.16, 0.17, - 0.16, 0.16, 0.15, 0.17, - 0.15, 0.16, 0.18 + 0.02, 0.02, 0.1, 0.17, + 0.18, 0.18, 0.19, 0.2, + 0.19, 0.18, 0.19, 0.19, + 0.19, 0.19, 0.17, 0.2, + 0.19, 0.19, 0.19 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -833,23 +834,23 @@ This is where Laravel Octane really shines, previously we had less than 300 req/ Iteration creation rate = **10/s** ```txt -checks.........................: 100.00% ✓ 22287 ✗ 0 -data_received..................: 201 MB 3.1 MB/s +checks.........................: 100.00% ✓ 21930 ✗ 0 +data_received..................: 198 MB 3.1 MB/s data_sent......................: 1.9 MB 30 kB/s -dropped_iterations.............: 164 2.548995/s -http_req_blocked...............: avg=61.02µs min=234ns med=1.01µs max=131.72ms p(90)=1.48µs p(95)=1.68µs -http_req_connecting............: avg=6.79µs min=0s med=0s max=62.5ms p(90)=0s p(95)=0s -http_req_duration..............: avg=130.52ms min=6.1ms med=129.59ms max=404.9ms p(90)=199.67ms p(95)=213.58ms - { expected_response:true }...: avg=130.52ms min=6.1ms med=129.59ms max=404.9ms p(90)=199.67ms p(95)=213.58ms -http_req_failed................: 0.00% ✓ 0 ✗ 22287 -http_req_receiving.............: avg=880.47µs min=20.89µs med=297.79µs max=216.18ms p(90)=904.22µs p(95)=1.78ms -http_req_sending...............: avg=214.44µs min=29.19µs med=114.19µs max=92.18ms p(90)=204.97µs p(95)=297.4µs -http_req_tls_handshaking.......: avg=50.13µs min=0s med=0s max=54.56ms p(90)=0s p(95)=0s -http_req_waiting...............: avg=129.43ms min=0s med=128.65ms max=396.55ms p(90)=198.43ms p(95)=212.35ms -http_reqs......................: 22287 346.399123/s -iteration_duration.............: avg=6.7s min=1.75s med=7.06s max=8.21s p(90)=7.47s p(95)=7.63s -iterations.....................: 437 6.79214/s -vus............................: 14 min=11 max=50 +dropped_iterations.............: 171 2.679253/s +http_req_blocked...............: avg=40.29µs min=266ns med=720ns max=65.15ms p(90)=1.15µs p(95)=1.34µs +http_req_connecting............: avg=2.38µs min=0s med=0s max=6.02ms p(90)=0s p(95)=0s +http_req_duration..............: avg=130.84ms min=7.15ms med=85.37ms max=337.78ms p(90)=242.21ms p(95)=252.98ms + { expected_response:true }...: avg=130.84ms min=7.15ms med=85.37ms max=337.78ms p(90)=242.21ms p(95)=252.98ms +http_req_failed................: 0.00% ✓ 0 ✗ 21930 +http_req_receiving.............: avg=428.79µs min=28.72µs med=294.3µs max=26.69ms p(90)=717.72µs p(95)=1.05ms +http_req_sending...............: avg=116.86µs min=34.94µs med=100.46µs max=16.13ms p(90)=157.81µs p(95)=185.95µs +http_req_tls_handshaking.......: avg=34.42µs min=0s med=0s max=29.11ms p(90)=0s p(95)=0s +http_req_waiting...............: avg=130.3ms min=6.88ms med=84.81ms max=337.45ms p(90)=241.7ms p(95)=252.54ms +http_reqs......................: 21930 343.602476/s +iteration_duration.............: avg=6.7s min=1.41s med=7.02s max=9.09s p(90)=7.96s p(95)=8.17s +iterations.....................: 430 6.737303/s +vus............................: 22 min=10 max=50 vus_max........................: 50 min=50 max=50 ``` @@ -861,14 +862,14 @@ vus_max........................: 50 min=50 max=50 { label: 'Req/s', data: [ - 122, 341, 348, 346, 359, 338, 361, 359, 345, - 360, 330, 352, 361, 344, 359, 334, 362, 323, - 346, 358, 335, 357, 352, 348, 358, 337, 359, - 357, 345, 357, 339, 358, 350, 333, 355, 332, - 359, 359, 349, 358, 333, 356, 359, 341, 352, - 333, 355, 360, 344, 356, 340, 360, 360, 348, - 355, 337, 346, 354, 345, 359, 337, 354, 341, - 338, 179 + 75, 346, 355, 351, 351, 331, 347, 356, 349, + 355, 327, 343, 338, 352, 349, 326, 356, 349, + 350, 354, 327, 350, 345, 347, 341, 329, 352, + 351, 352, 353, 331, 357, 355, 350, 351, 314, + 350, 355, 351, 357, 330, 354, 355, 350, 353, + 323, 351, 346, 344, 356, 323, 355, 343, 351, + 355, 322, 354, 353, 336, 354, 328, 352, 354, + 346, 64 ] } ] @@ -883,12 +884,12 @@ vus_max........................: 50 min=50 max=50 { label: 'VUs', data: [ - 11, 18, 23, 28, 34, 38, 44, 49, 49, 50, 50, 48, - 49, 47, 48, 49, 49, 50, 48, 45, 50, 48, 49, 50, - 50, 49, 49, 47, 46, 46, 50, 50, 48, 49, 47, 48, - 47, 47, 50, 50, 48, 50, 48, 48, 48, 50, 50, 48, - 50, 50, 47, 49, 49, 50, 48, 50, 47, 46, 48, 49, - 47, 39, 31, 14 + 10, 16, 21, 26, 33, 39, 43, 50, 48, 49, 49, 50, + 50, 48, 49, 48, 50, 49, 50, 49, 50, 50, 49, 50, + 49, 49, 50, 50, 47, 49, 50, 49, 50, 50, 50, 49, + 47, 48, 46, 50, 50, 50, 50, 47, 48, 49, 50, 50, + 49, 50, 49, 49, 48, 49, 47, 49, 49, 50, 46, 49, + 38, 31, 22 ] } ] @@ -899,13 +900,13 @@ vus_max........................: 50 min=50 max=50 { label: 'Duration (ms)', data: [ - 25, 37, 54, 69, 82, 104, 111, 128, 142, 136, - 149, 141, 134, 142, 130, 142, 137, 154, 141, 134, - 137, 136, 138, 141, 137, 146, 138, 136, 138, 133, - 140, 137, 142, 148, 140, 144, 135, 134, 141, 137, - 147, 138, 137, 141, 134, 151, 139, 137, 144, 138, - 145, 136, 136, 143, 137, 146, 141, 136, 137, 134, - 144, 127, 109, 85, 40 + 24, 31, 46, 61, 76, 98, 107, 120, 139, 136, + 146, 150, 145, 139, 136, 151, 140, 140, 141, 137, + 146, 143, 140, 140, 144, 149, 140, 140, 138, 132, + 149, 137, 139, 141, 140, 156, 142, 135, 136, 131, + 149, 141, 139, 142, 132, 150, 140, 144, 143, 138, + 154, 139, 141, 139, 138, 151, 140, 141, 145, 136, + 144, 113, 89, 60, 22 ] } ] @@ -919,11 +920,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.03, 0.03, 0.28, 0.33, - 0.34, 0.33, 0.34, 0.35, - 0.34, 0.34, 0.35, 0.34, - 0.33, 0.34, 0.32, 0.03, - 0.04, 0.04, 0.03 + 0.04, 0.11, 0.39, 0.39, + 0.36, 0.35, 0.35, 0.37, + 0.34, 0.34, 0.32, 0.34, + 0.33, 0.35, 0.18, 0.03, + 0.04, 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -932,11 +933,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.01, 0.02, 0.05, 0.05, - 0.06, 0.06, 0.05, 0.05, - 0.06, 0.05, 0.05, 0.06, - 0.05, 0.07, 0.05, 0.01, - 0.02, 0.02, 0.02 + 0.02, 0.04, 0.08, 0.09, + 0.1, 0.08, 0.09, 0.08, + 0.1, 0.09, 0.11, 0.08, + 0.09, 0.08, 0.05, 0.03, + 0.03, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -950,11 +951,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.03, 0.03, 0.54, 0.93, - 0.93, 0.91, 0.92, 0.93, - 0.91, 0.92, 0.93, 0.93, - 0.93, 0.92, 0.93, 0.21, - 0.03, 0.03, 0.03 + 0.03, 0.02, 0.51, 0.91, + 0.91, 0.92, 0.91, 0.9, + 0.92, 0.91, 0.92, 0.92, + 0.91, 0.92, 0.92, 0.15, + 0.02, 0.02, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -963,11 +964,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.02, 0.02, 0.05, 0.07, - 0.07, 0.07, 0.08, 0.07, - 0.09, 0.08, 0.07, 0.07, - 0.07, 0.08, 0.07, 0.03, - 0.02, 0.02, 0.02 + 0.02, 0.02, 0.05, 0.08, + 0.08, 0.08, 0.08, 0.09, + 0.08, 0.08, 0.08, 0.08, + 0.08, 0.08, 0.08, 0.03, + 0.02, 0.02, 0.03 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -983,25 +984,26 @@ We are database limited, performing same as Laravel. #### Symfony scenario 2 -Iteration creation rate = **1/s** +Iteration creation rate = **2/s** ```txt -checks.........................: 100.00% ✓ 94672 ✗ 0 -data_received..................: 171 MB 2.2 MB/s -data_sent......................: 7.5 MB 95 kB/s -http_req_blocked...............: avg=17.77µs min=172ns med=626ns max=132.86ms p(90)=1.19µs p(95)=1.36µs -http_req_connecting............: avg=2.07µs min=0s med=0s max=62.8ms p(90)=0s p(95)=0s -http_req_duration..............: avg=14.36ms min=1.87ms med=10.33ms max=358.31ms p(90)=26.54ms p(95)=36.05ms - { expected_response:true }...: avg=14.36ms min=1.87ms med=10.33ms max=358.31ms p(90)=26.54ms p(95)=36.05ms -http_req_failed................: 0.00% ✓ 0 ✗ 94672 -http_req_receiving.............: avg=946.99µs min=15.83µs med=251.63µs max=213.84ms p(90)=1.58ms p(95)=3ms -http_req_sending...............: avg=180.14µs min=14.98µs med=75.49µs max=199.41ms p(90)=161.75µs p(95)=230.84µs -http_req_tls_handshaking.......: avg=14.39µs min=0s med=0s max=132.05ms p(90)=0s p(95)=0s -http_req_waiting...............: avg=13.23ms min=0s med=9.58ms max=278.53ms p(90)=24.63ms p(95)=33.11ms -http_reqs......................: 94672 1199.909891/s -iteration_duration.............: avg=22.69s min=10.8s med=23.88s max=30.11s p(90)=29.54s p(95)=29.85s -iterations.....................: 61 0.773138/s -vus............................: 5 min=1 max=29 +checks.........................: 100.00% ✓ 110192 ✗ 0 +data_received..................: 195 MB 2.3 MB/s +data_sent......................: 8.7 MB 105 kB/s +dropped_iterations.............: 49 0.588079/s +http_req_blocked...............: avg=9.46µs min=237ns med=582ns max=59.73ms p(90)=890ns p(95)=1.03µs +http_req_connecting............: avg=378ns min=0s med=0s max=3.44ms p(90)=0s p(95)=0s +http_req_duration..............: avg=27.36ms min=2.09ms med=22.91ms max=534.89ms p(90)=53.87ms p(95)=61.96ms + { expected_response:true }...: avg=27.36ms min=2.09ms med=22.91ms max=534.89ms p(90)=53.87ms p(95)=61.96ms +http_req_failed................: 0.00% ✓ 0 ✗ 110192 +http_req_receiving.............: avg=393.34µs min=20.27µs med=169.55µs max=44.39ms p(90)=667.69µs p(95)=1.17ms +http_req_sending...............: avg=96.96µs min=21.03µs med=82.98µs max=11.97ms p(90)=133.54µs p(95)=160.59µs +http_req_tls_handshaking.......: avg=8.02µs min=0s med=0s max=58.6ms p(90)=0s p(95)=0s +http_req_waiting...............: avg=26.87ms min=1.57ms med=22.42ms max=534.69ms p(90)=53.21ms p(95)=61.26ms +http_reqs......................: 110192 1322.481752/s +iteration_duration.............: avg=42.8s min=19.32s med=46.97s max=53.14s p(90)=51.96s p(95)=52.6s +iterations.....................: 71 0.852115/s +vus............................: 2 min=2 max=50 vus_max........................: 50 min=50 max=50 ``` @@ -1013,15 +1015,16 @@ vus_max........................: 50 min=50 max=50 { label: 'Req/s', data: [ - 91, 207, 385, 748, 820, 959, 1075, 964, 1238, - 1212, 1320, 1289, 1154, 1361, 1365, 1307, 1346, 1115, - 1328, 1461, 1238, 1417, 1237, 953, 1350, 1272, 1331, - 1155, 1105, 1017, 1196, 1108, 815, 824, 1327, 1224, - 1334, 1213, 1410, 1430, 1365, 1394, 1299, 1165, 1427, - 1419, 1341, 1234, 1418, 957, 1035, 1028, 929, 880, - 1414, 1491, 1388, 1226, 1510, 1408, 1366, 1571, 1048, - 1439, 1486, 1360, 1401, 1216, 1392, 1408, 1486, 1440, - 1260, 1312, 1341, 1271, 1250, 964, 628, 4 + 162, 449, 974, 1254, 1075, 1260, 1289, 1267, 1306, + 1347, 1403, 1335, 1225, 1431, 1295, 1330, 1365, 1344, + 1461, 1369, 1461, 1398, 1310, 1460, 1412, 1433, 1475, + 1373, 1393, 1342, 1481, 1446, 1364, 1386, 1359, 1353, + 1368, 1316, 1386, 1258, 1451, 1502, 1327, 1419, 1327, + 1339, 1389, 1373, 1388, 1361, 1494, 1454, 1337, 1420, + 1374, 1440, 1408, 1319, 1484, 1370, 1433, 1329, 1334, + 1393, 1415, 1347, 1399, 1412, 1320, 1290, 1508, 1401, + 1312, 1454, 1315, 1347, 1452, 1221, 1262, 1221, 1259, + 1306, 676, 95 ] } ] @@ -1036,13 +1039,13 @@ vus_max........................: 50 min=50 max=50 { label: 'VUs', data: [ - 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 10, - 11, 11, 11, 11, 12, 12, 13, 13, 13, 13, 14, 14, - 15, 15, 16, 15, 16, 16, 16, 17, 18, 19, 19, 20, - 20, 20, 20, 21, 21, 22, 21, 22, 22, 23, 23, 24, - 24, 25, 25, 25, 26, 27, 27, 27, 28, 28, 28, 28, - 29, 27, 26, 26, 26, 23, 23, 22, 21, 20, 19, 18, - 18, 15, 13, 10, 8, 5 + 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, + 26, 28, 30, 32, 34, 36, 38, 39, 41, 41, 43, 45, + 46, 48, 49, 50, 50, 50, 49, 50, 50, 50, 49, 50, + 50, 50, 49, 50, 50, 50, 50, 50, 50, 50, 50, 50, + 49, 50, 48, 50, 48, 50, 50, 50, 49, 50, 50, 50, + 49, 46, 44, 44, 43, 42, 40, 39, 36, 36, 33, 30, + 27, 24, 22, 17, 14, 14, 13, 12, 9, 7, 2 ] } ] @@ -1053,13 +1056,13 @@ vus_max........................: 50 min=50 max=50 { label: 'Duration (ms)', data: [ - 8, 8, 7, 5, 6, 6, 6, 8, 7, 8, 8, 8, - 9, 8, 8, 9, 8, 11, 9, 9, 11, 10, 11, 15, - 11, 12, 12, 14, 14, 15, 14, 15, 21, 22, 14, 16, - 15, 17, 14, 14, 15, 15, 17, 19, 16, 16, 18, 19, - 17, 26, 24, 26, 24, 33, 19, 17, 20, 24, 19, 20, - 21, 18, 25, 18, 17, 18, 16, 19, 15, 15, 13, 13, - 14, 12, 10, 10, 8, 7, 5, 9 + 9, 8, 5, 6, 8, 9, 10, 12, 13, 14, 15, 17, + 21, 19, 22, 23, 25, 26, 25, 28, 28, 28, 34, 30, + 32, 33, 30, 39, 36, 34, 36, 34, 37, 36, 35, 34, + 40, 38, 35, 39, 35, 33, 37, 35, 36, 38, 36, 36, + 35, 37, 33, 34, 37, 35, 36, 34, 35, 38, 33, 36, + 35, 34, 35, 32, 30, 32, 29, 28, 28, 27, 24, 23, + 20, 19, 17, 15, 11, 11, 11, 10, 8, 6, 5, 5 ] } ] @@ -1073,11 +1076,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.03, 0.07, 0.37, 0.54, - 0.58, 0.58, 0.55, 0.5, - 0.52, 0.6, 0.6, 0.56, - 0.55, 0.6, 0.6, 0.61, - 0.58, 0.21, 0.03 + 0.03, 0.4, 0.6, 0.61, + 0.61, 0.64, 0.67, 0.68, + 0.64, 0.66, 0.65, 0.7, + 0.66, 0.66, 0.65, 0.67, + 0.58, 0.34, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -1086,11 +1089,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.02, 0.02, 0.07, 0.11, - 0.12, 0.12, 0.12, 0.11, - 0.12, 0.12, 0.12, 0.12, - 0.12, 0.13, 0.13, 0.13, - 0.11, 0.05, 0.02 + 0.02, 0.14, 0.17, 0.17, + 0.19, 0.17, 0.19, 0.19, + 0.19, 0.21, 0.19, 0.18, + 0.18, 0.18, 0.18, 0.18, + 0.16, 0.11, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -1104,11 +1107,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.03, 0.03, 0.19, 0.27, - 0.33, 0.33, 0.3, 0.3, - 0.3, 0.33, 0.32, 0.33, - 0.3, 0.35, 0.34, 0.35, - 0.34, 0.19, 0.03 + 0.02, 0.06, 0.31, 0.33, + 0.32, 0.35, 0.34, 0.35, + 0.33, 0.34, 0.34, 0.34, + 0.34, 0.34, 0.34, 0.34, + 0.33, 0.32, 0.1 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -1117,11 +1120,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.02, 0.02, 0.07, 0.09, - 0.1, 0.1, 0.1, 0.09, - 0.09, 0.11, 0.1, 0.09, - 0.11, 0.11, 0.11, 0.1, - 0.11, 0.06, 0.02 + 0.02, 0.03, 0.1, 0.13, + 0.11, 0.13, 0.13, 0.12, + 0.13, 0.13, 0.12, 0.13, + 0.13, 0.12, 0.12, 0.12, + 0.13, 0.12, 0.05 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -1133,7 +1136,7 @@ vus_max........................: 50 min=50 max=50 {{< /tab >}} {{< /tabs >}} -Huge gap in performance against Laravel Octane here, about twice better ! Without FrankenPHP, we were capping to previously about 300 req/s... +Huge gap in performance against Laravel Octane here, about twice better ! Without FrankenPHP, we were capping to previously about 300 req/s on Apache... ### FastAPI @@ -1144,24 +1147,24 @@ As a side note here, uvicorn is limited to 1 CPU core, so I use 2 replicas on ea Iteration creation rate = **15/s** ```txt -checks.........................: 100.00% ✓ 33048 ✗ 0 -data_received..................: 272 MB 4.3 MB/s -data_sent......................: 2.9 MB 46 kB/s -dropped_iterations.............: 253 4.042284/s -http_req_blocked...............: avg=44.03µs min=197ns med=873ns max=51.66ms p(90)=1.31µs p(95)=1.48µs -http_req_connecting............: avg=1.59µs min=0s med=0s max=3.33ms p(90)=0s p(95)=0s -http_req_duration..............: avg=87.55ms min=5.74ms med=79.16ms max=449.45ms p(90)=160.7ms p(95)=187.45ms -{ expected_response:true }...: avg=87.55ms min=5.74ms med=79.16ms max=449.45ms p(90)=160.7ms p(95)=187.45ms -http_req_failed................: 0.00% ✓ 0 ✗ 33048 -http_req_receiving.............: avg=809.01µs min=18.38µs med=273.27µs max=53.15ms p(90)=1.95ms p(95)=3.12ms -http_req_sending...............: avg=156.85µs min=23.21µs med=95.41µs max=45.6ms p(90)=181.16µs p(95)=248.5µs -http_req_tls_handshaking.......: avg=40.32µs min=0s med=0s max=44.77ms p(90)=0s p(95)=0s -http_req_waiting...............: avg=86.59ms min=0s med=78.14ms max=448.54ms p(90)=159.53ms p(95)=186.05ms -http_reqs......................: 33048 528.02138/s -iteration_duration.............: avg=4.49s min=1.14s med=4.65s max=6.25s p(90)=5.17s p(95)=5.3s -iterations.....................: 648 10.35336/s -vus............................: 22 min=15 max=50 -vus_max........................: 50 min=50 max=50 +checks.........................: 100.00% ✓ 29835 ✗ 0 +data_received..................: 241 MB 3.9 MB/s +data_sent......................: 2.6 MB 42 kB/s +dropped_iterations.............: 315 5.03316/s +http_req_blocked...............: avg=30.38µs min=227ns med=651ns max=52.06ms p(90)=1.02µs p(95)=1.18µs +http_req_connecting............: avg=1.98µs min=0s med=0s max=5.76ms p(90)=0s p(95)=0s +http_req_duration..............: avg=97.85ms min=6.31ms med=83.03ms max=500.66ms p(90)=192.52ms p(95)=226.66ms + { expected_response:true }...: avg=97.85ms min=6.31ms med=83.03ms max=500.66ms p(90)=192.52ms p(95)=226.66ms +http_req_failed................: 0.00% ✓ 0 ✗ 29835 +http_req_receiving.............: avg=570.36µs min=28.53µs med=260.32µs max=21.18ms p(90)=1.23ms p(95)=1.94ms +http_req_sending...............: avg=112.03µs min=32.86µs med=95.56µs max=16.17ms p(90)=150.61µs p(95)=181.13µs +http_req_tls_handshaking.......: avg=26.39µs min=0s med=0s max=31.75ms p(90)=0s p(95)=0s +http_req_waiting...............: avg=97.17ms min=6.14ms med=82.29ms max=497.14ms p(90)=191.78ms p(95)=225.93ms +http_reqs......................: 29835 476.712194/s +iteration_duration.............: avg=5.02s min=1.44s med=5.12s max=6.71s p(90)=5.77s p(95)=6s +iterations.....................: 585 9.347298/s +vus............................: 21 min=15 max=50 +vus_max........................: 50 min=50 max=50 ``` {{< tabs >}} @@ -1172,13 +1175,13 @@ vus_max........................: 50 min=50 max=50 { label: 'Req/s', data: [ - 270, 514, 530, 535, 552, 541, 524, 493, 545, - 555, 560, 539, 519, 545, 540, 531, 525, 514, - 547, 540, 537, 533, 485, 511, 534, 525, 508, - 500, 550, 527, 538, 516, 500, 542, 532, 530, - 504, 508, 540, 538, 553, 537, 497, 560, 517, - 578, 559, 487, 551, 546, 538, 531, 517, 518, - 578, 559, 521, 516, 556, 567, 517, 517, 351 + 345, 451, 480, 469, 476, 507, 461, 476, 523, + 494, 467, 493, 476, 501, 483, 502, 456, 467, + 502, 465, 480, 476, 460, 488, 513, 507, 449, + 477, 467, 488, 496, 461, 467, 468, 508, 515, + 444, 470, 496, 495, 496, 490, 453, 513, 491, + 481, 441, 471, 520, 463, 495, 457, 484, 454, + 475, 467, 462, 479, 459, 481, 504, 470, 210 ] } ] @@ -1193,12 +1196,12 @@ vus_max........................: 50 min=50 max=50 { label: 'VUs', data: [ - 15, 23, 31, 41, 46, 50, 50, 50, 48, 49, 50, 48, - 48, 45, 49, 50, 49, 48, 50, 49, 50, 49, 49, 50, - 50, 50, 50, 48, 50, 47, 49, 48, 48, 49, 49, 50, - 48, 48, 50, 49, 49, 50, 48, 48, 49, 48, 45, 50, - 48, 49, 49, 48, 47, 50, 49, 50, 48, 47, 50, 48, - 39, 22 + 15, 25, 34, 46, 49, 50, 50, 49, 46, 50, 49, 50, + 50, 47, 49, 50, 50, 50, 50, 50, 50, 50, 50, 49, + 49, 50, 50, 49, 50, 50, 48, 50, 50, 49, 49, 50, + 50, 49, 48, 50, 49, 49, 49, 50, 49, 50, 50, 50, + 49, 48, 50, 49, 50, 50, 50, 49, 49, 50, 50, 50, + 37, 21 ] } ] @@ -1209,12 +1212,12 @@ vus_max........................: 50 min=50 max=50 { label: 'Duration (ms)', data: [ - 20, 33, 45, 65, 74, 88, 93, 96, 90, 89, 90, 92, - 94, 91, 89, 91, 91, 95, 90, 93, 92, 93, 101, 93, - 90, 94, 96, 101, 87, 92, 89, 96, 98, 91, 91, 93, - 94, 97, 92, 90, 88, 91, 95, 91, 93, 86, 88, 94, - 89, 91, 90, 92, 93, 92, 87, 86, 94, 90, 91, 86, - 86, 63, 36 + 24, 48, 62, 83, 104, 97, 104, 101, 93, 99, 103, + 100, 102, 91, 109, 99, 108, 104, 96, 102, 105, 106, + 105, 102, 99, 93, 110, 104, 104, 100, 98, 107, 106, + 106, 96, 96, 103, 108, 96, 99, 102, 98, 108, 93, + 104, 99, 110, 100, 101, 102, 101, 104, 107, 105, 107, + 105, 102, 105, 103, 103, 85, 66, 29 ] } ] @@ -1228,11 +1231,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.03, 0.02, 0.52, 0.71, - 0.74, 0.73, 0.71, 0.69, - 0.71, 0.73, 0.73, 0.75, - 0.71, 0.74, 0.5, 0.04, - 0.03, 0.03, 0.02 + 0.03, 0.48, 0.71, 0.7, + 0.7, 0.67, 0.69, 0.68, + 0.7, 0.69, 0.7, 0.68, + 0.69, 0.5, 0.03, 0.03, + 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -1241,11 +1244,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.01, 0.01, 0.12, 0.15, - 0.15, 0.15, 0.15, 0.17, - 0.16, 0.15, 0.15, 0.15, - 0.15, 0.15, 0.11, 0.02, - 0.01, 0.01, 0.01 + 0.02, 0.14, 0.2, 0.19, + 0.19, 0.19, 0.19, 0.19, + 0.19, 0.19, 0.2, 0.2, + 0.19, 0.13, 0.02, 0.02, + 0.03, 0.03 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -1259,11 +1262,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.03, 0.3, 0.46, 0.47, - 0.47, 0.48, 0.47, 0.48, - 0.49, 0.49, 0.49, 0.46, - 0.49, 0.35, 0.03, 0.03, - 0.03, 0.03, 0.03 + 0.02, 0.05, 0.36, 0.4, + 0.41, 0.4, 0.41, 0.38, + 0.42, 0.41, 0.42, 0.37, + 0.4, 0.39, 0.16, 0.02, + 0.02, 0.02 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -1272,11 +1275,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.01, 0.11, 0.25, 0.25, - 0.3, 0.27, 0.27, 0.25, - 0.29, 0.28, 0.28, 0.26, - 0.29, 0.19, 0.02, 0.02, - 0.02, 0.01, 0.01 + 0.02, 0.04, 0.25, 0.3, + 0.29, 0.3, 0.31, 0.32, + 0.33, 0.32, 0.37, 0.26, + 0.3, 0.31, 0.11, 0.02, + 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -1295,24 +1298,24 @@ FastAPI outperforms above PHP frameworks in this specific scenario, and database Iteration creation rate = **2/s** ```txt -checks.........................: 100.00% ✓ 72075 ✗ 0 -data_received..................: 147 MB 1.6 MB/s -data_sent......................: 5.4 MB 60 kB/s -dropped_iterations.............: 68 0.755514/s -http_req_blocked...............: avg=20.35µs min=213ns med=903ns max=52.94ms p(90)=1.31µs p(95)=1.48µs -http_req_connecting............: avg=909ns min=0s med=0s max=10.34ms p(90)=0s p(95)=0s -http_req_duration..............: avg=51.06ms min=3.06ms med=32.91ms max=1.07s p(90)=117.46ms p(95)=138.49ms -{ expected_response:true }...: avg=51.06ms min=3.06ms med=32.91ms max=1.07s p(90)=117.46ms p(95)=138.49ms -http_req_failed................: 0.00% ✓ 0 ✗ 72075 -http_req_receiving.............: avg=301.26µs min=17.32µs med=125.14µs max=26.38ms p(90)=678.97µs p(95)=1.14ms -http_req_sending...............: avg=118.25µs min=21.81µs med=94.18µs max=20.22ms p(90)=163.96µs p(95)=204.06µs -http_req_tls_handshaking.......: avg=17.89µs min=0s med=0s max=37.55ms p(90)=0s p(95)=0s -http_req_waiting...............: avg=50.64ms min=911.97µs med=32.54ms max=1.07s p(90)=116.83ms p(95)=137.77ms -http_reqs......................: 72075 800.78886/s -iteration_duration.............: avg=1m10s min=51.94s med=1m14s max=1m21s p(90)=1m20s p(95)=1m21s -iterations.....................: 20 0.22221/s -vus............................: 33 min=2 max=50 -vus_max........................: 50 min=50 max=50 +checks.........................: 100.00% ✓ 64223 ✗ 0 +data_received..................: 128 MB 1.4 MB/s +data_sent......................: 4.8 MB 53 kB/s +dropped_iterations.............: 71 0.788854/s +http_req_blocked...............: avg=16.09µs min=242ns med=634ns max=71.12ms p(90)=1µs p(95)=1.18µs +http_req_connecting............: avg=936ns min=0s med=0s max=11.27ms p(90)=0s p(95)=0s +http_req_duration..............: avg=58.19ms min=4.23ms med=33.6ms max=430.25ms p(90)=146.63ms p(95)=177.64ms + { expected_response:true }...: avg=58.19ms min=4.23ms med=33.6ms max=430.25ms p(90)=146.63ms p(95)=177.64ms +http_req_failed................: 0.00% ✓ 0 ✗ 64223 +http_req_receiving.............: avg=224.16µs min=20.82µs med=120.98µs max=11.92ms p(90)=456.57µs p(95)=713.27µs +http_req_sending...............: avg=98.3µs min=28.53µs med=85.53µs max=15.47ms p(90)=136.99µs p(95)=162.39µs +http_req_tls_handshaking.......: avg=13.73µs min=0s med=0s max=44.98ms p(90)=0s p(95)=0s +http_req_waiting...............: avg=57.87ms min=4.04ms med=33.29ms max=429.5ms p(90)=146.17ms p(95)=177.16ms +http_reqs......................: 64223 713.557494/s +iteration_duration.............: avg=1m15s min=1m6s med=1m15s max=1m23s p(90)=1m23s p(95)=1m23s +iterations.....................: 11 0.122217/s +vus............................: 39 min=2 max=50 +vus_max........................: 50 min=50 max=50 ``` {{< tabs >}} @@ -1323,15 +1326,15 @@ vus_max........................: 50 min=50 max=50 { label: 'Req/s', data: [ - 19, 168, 465, 631, 720, 792, 752, 758, 757, 763, 839, - 819, 777, 770, 821, 828, 760, 792, 741, 869, 862, 831, - 846, 811, 820, 878, 792, 811, 815, 829, 804, 807, 842, - 819, 791, 804, 744, 839, 810, 828, 841, 890, 841, 834, - 804, 829, 821, 837, 852, 853, 853, 884, 871, 773, 774, - 825, 794, 832, 825, 787, 807, 872, 837, 815, 826, 778, - 811, 810, 823, 807, 786, 872, 886, 810, 808, 831, 824, - 853, 770, 818, 793, 827, 795, 813, 795, 858, 869, 805, - 846, 823, 563 + 45, 176, 457, 647, 698, 657, 670, 671, 738, 801, 676, + 783, 692, 700, 760, 688, 743, 676, 698, 740, 695, 726, + 763, 759, 723, 684, 774, 731, 740, 778, 685, 792, 746, + 755, 733, 682, 753, 730, 763, 735, 707, 792, 744, 695, + 764, 679, 778, 692, 775, 715, 675, 774, 712, 714, 742, + 699, 719, 684, 762, 789, 681, 752, 646, 722, 769, 651, + 778, 743, 692, 764, 701, 736, 719, 816, 768, 707, 765, + 735, 762, 725, 701, 714, 652, 769, 784, 713, 703, 735, + 767, 754, 225 ] } ] @@ -1350,10 +1353,10 @@ vus_max........................: 50 min=50 max=50 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, - 50, 50, 50, 49, 50, 50, 50, 50, 49, 50, 50, 50, - 49, 49, 49, 48, 48, 48, 47, 46, 46, 46, 46, 46, - 46, 44, 44, 44, 44, 43, 43, 43, 42, 42, 40, 39, - 39, 39, 37, 37, 36, 33 + 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, + 50, 50, 50, 50, 50, 50, 50, 49, 48, 48, 47, 47, + 46, 46, 46, 45, 45, 44, 44, 44, 43, 43, 43, 43, + 42, 42, 41, 41, 39, 39 ] } ] @@ -1364,14 +1367,14 @@ vus_max........................: 50 min=50 max=50 { label: 'Duration (ms)', data: [ - 13, 12, 8, 9, 11, 12, 16, 18, 21, 23, 24, 27, - 30, 34, 34, 36, 42, 42, 48, 44, 46, 51, 51, 56, - 59, 56, 63, 59, 64, 59, 63, 62, 58, 61, 62, 62, - 68, 60, 61, 60, 58, 57, 59, 59, 60, 61, 61, 58, - 60, 58, 59, 56, 56, 65, 64, 58, 66, 61, 60, 62, - 61, 57, 58, 61, 55, 62, 60, 59, 57, 57, 58, 53, - 51, 56, 56, 53, 52, 52, 56, 51, 56, 51, 51, 52, - 46, 48, 44, 46, 44, 43, 44 + 17, 16, 10, 10, 12, 16, 18, 22, 23, 23, 30, 30, + 34, 39, 38, 44, 44, 52, 52, 52, 58, 60, 57, 62, + 65, 72, 64, 68, 69, 65, 73, 62, 63, 71, 67, 74, + 67, 68, 65, 67, 72, 63, 66, 72, 65, 72, 66, 71, + 65, 67, 72, 68, 70, 69, 68, 69, 72, 72, 65, 64, + 72, 68, 76, 69, 65, 74, 66, 65, 72, 60, 69, 64, + 65, 57, 59, 65, 56, 63, 56, 59, 64, 59, 67, 56, + 53, 60, 58, 53, 56, 51, 47 ] } ] @@ -1385,11 +1388,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.02, 0.02, 0.03, 0.54, - 0.65, 0.68, 0.69, 0.72, - 0.71, 0.7, 0.71, 0.71, - 0.72, 0.69, 0.68, 0.72, - 0.69, 0.71, 0.7 + 0.03, 0.18, 0.58, 0.63, + 0.65, 0.67, 0.66, 0.68, + 0.63, 0.67, 0.67, 0.66, + 0.66, 0.68, 0.69, 0.65, + 0.69, 0.66, 0.64 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -1398,11 +1401,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.01, 0.01, 0.01, 0.11, - 0.18, 0.18, 0.16, 0.17, - 0.17, 0.16, 0.18, 0.18, - 0.18, 0.18, 0.18, 0.15, - 0.18, 0.17, 0.17 + 0.03, 0.05, 0.17, 0.2, + 0.19, 0.19, 0.21, 0.2, + 0.21, 0.19, 0.19, 0.21, + 0.21, 0.21, 0.2, 0.19, + 0.21, 0.19, 0.2 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -1416,11 +1419,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.03, 0.03, 0.22, 0.22, - 0.25, 0.28, 0.31, 0.29, - 0.31, 0.3, 0.3, 0.32, - 0.33, 0.3, 0.28, 0.3, - 0.31, 0.3, 0.28 + 0.02, 0.02, 0.14, 0.21, + 0.21, 0.24, 0.27, 0.25, + 0.26, 0.26, 0.26, 0.26, + 0.26, 0.25, 0.25, 0.26, + 0.26, 0.26, 0.22 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -1429,11 +1432,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.01, 0.01, 0.11, 0.14, - 0.18, 0.2, 0.23, 0.23, - 0.22, 0.24, 0.26, 0.24, - 0.25, 0.21, 0.2, 0.23, - 0.24, 0.22, 0.19 + 0.02, 0.03, 0.08, 0.14, + 0.19, 0.22, 0.28, 0.23, + 0.25, 0.22, 0.26, 0.28, + 0.26, 0.24, 0.28, 0.26, + 0.25, 0.24, 0.25 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -1449,28 +1452,28 @@ FastAPI fall behind Symfony but ahead of Laravel. ### NestJS +Note that we're using **Fastify** adapter instead of Express in order to maximize performance. + #### NestJS scenario 1 -Iteration creation rate = **15/s** - ```txt -checks.........................: 100.00% ✓ 37434 ✗ 0 -data_received..................: 648 MB 11 MB/s -data_sent......................: 3.5 MB 57 kB/s -dropped_iterations.............: 166 2.680206/s -http_req_blocked...............: avg=35.23µs min=216ns med=702ns max=49.57ms p(90)=1.19µs p(95)=1.33µs -http_req_connecting............: avg=1.44µs min=0s med=0s max=5.62ms p(90)=0s p(95)=0s -http_req_duration..............: avg=75.64ms min=3.2ms med=70.44ms max=346.41ms p(90)=134.43ms p(95)=146.32ms -{ expected_response:true }...: avg=75.64ms min=3.2ms med=70.44ms max=346.41ms p(90)=134.43ms p(95)=146.32ms -http_req_failed................: 0.00% ✓ 0 ✗ 37434 -http_req_receiving.............: avg=408.39µs min=19.38µs med=219.61µs max=42.89ms p(90)=653.52µs p(95)=1.28ms -http_req_sending...............: avg=134.99µs min=18.84µs med=83.94µs max=26.64ms p(90)=156.84µs p(95)=222.84µs -http_req_tls_handshaking.......: avg=31.9µs min=0s med=0s max=40.37ms p(90)=0s p(95)=0s -http_req_waiting...............: avg=75.09ms min=2.92ms med=69.92ms max=345.47ms p(90)=133.87ms p(95)=145.68ms -http_reqs......................: 37434 604.402491/s -iteration_duration.............: avg=3.89s min=1.2s med=4s max=5.16s p(90)=4.41s p(95)=4.54s -iterations.....................: 734 11.851029/s -vus............................: 31 min=15 max=50 +checks.........................: 100.00% ✓ 46104 ✗ 0 +data_received..................: 791 MB 13 MB/s +data_sent......................: 4.4 MB 72 kB/s +dropped_iterations.............: 296 4.79183/s +http_req_blocked...............: avg=22.48µs min=250ns med=610ns max=56.72ms p(90)=954ns p(95)=1.1µs +http_req_connecting............: avg=1.12µs min=0s med=0s max=6.32ms p(90)=0s p(95)=0s +http_req_duration..............: avg=62.1ms min=2.94ms med=52.66ms max=322.4ms p(90)=115.17ms p(95)=120.56ms + { expected_response:true }...: avg=62.1ms min=2.94ms med=52.66ms max=322.4ms p(90)=115.17ms p(95)=120.56ms +http_req_failed................: 0.00% ✓ 0 ✗ 46104 +http_req_receiving.............: avg=388.24µs min=25.63µs med=233.32µs max=205.79ms p(90)=555.24µs p(95)=925.96µs +http_req_sending...............: avg=119.09µs min=29.63µs med=92.44µs max=41.89ms p(90)=148.13µs p(95)=183.72µs +http_req_tls_handshaking.......: avg=20µs min=0s med=0s max=55.7ms p(90)=0s p(95)=0s +http_req_waiting...............: avg=61.59ms min=0s med=52.07ms max=171.12ms p(90)=114.69ms p(95)=119.96ms +http_reqs......................: 46104 746.359903/s +iteration_duration.............: avg=3.21s min=966.59ms med=3.26s max=4.4s p(90)=3.7s p(95)=3.83s +iterations.....................: 904 14.634508/s +vus............................: 31 min=20 max=50 vus_max........................: 50 min=50 max=50 ``` @@ -1482,13 +1485,13 @@ vus_max........................: 50 min=50 max=50 { label: 'Req/s', data: [ - 273, 496, 577, 588, 599, 624, 621, 610, 641, - 638, 641, 614, 585, 580, 600, 604, 601, 571, - 606, 663, 643, 572, 596, 585, 616, 650, 680, - 615, 612, 611, 617, 572, 587, 593, 605, 633, - 633, 573, 646, 645, 650, 570, 629, 653, 691, - 650, 580, 555, 590, 646, 565, 585, 638, 594, - 567, 555, 602, 570, 641, 648, 601, 630, 8 + 209, 657, 702, 731, 710, 709, 685, 759, 770, + 744, 743, 767, 751, 750, 755, 755, 738, 771, + 764, 735, 751, 725, 750, 759, 750, 762, 733, + 758, 778, 755, 739, 745, 787, 767, 755, 765, + 736, 790, 786, 760, 754, 745, 776, 781, 766, + 768, 698, 791, 755, 744, 749, 741, 790, 762, + 775, 781, 756, 752, 769, 753, 724, 714, 104 ] } ] @@ -1503,11 +1506,11 @@ vus_max........................: 50 min=50 max=50 { label: 'VUs', data: [ - 15, 23, 32, 39, 44, 50, 49, 50, 46, 47, 49, 49, - 48, 49, 49, 49, 49, 49, 50, 50, 49, 50, 48, 50, - 49, 50, 49, 50, 46, 48, 46, 49, 50, 46, 47, 47, - 45, 49, 47, 45, 49, 49, 49, 48, 45, 48, 50, 50, - 49, 46, 45, 48, 50, 50, 49, 45, 48, 50, 49, 50, + 20, 30, 42, 50, 49, 50, 49, 49, 50, 49, 49, 50, + 50, 49, 49, 50, 45, 50, 50, 49, 47, 50, 49, 46, + 46, 50, 50, 48, 46, 50, 50, 48, 49, 47, 48, 50, + 50, 47, 49, 49, 49, 48, 49, 49, 45, 50, 49, 49, + 49, 49, 50, 50, 49, 49, 47, 49, 47, 49, 48, 49, 31 ] } @@ -1519,12 +1522,12 @@ vus_max........................: 50 min=50 max=50 { label: 'Duration (ms)', data: [ - 21, 38, 45, 58, 65, 74, 79, 79, 75, 74, 75, 78, - 85, 80, 80, 82, 81, 86, 78, 74, 76, 85, 81, 83, - 77, 76, 72, 75, 78, 77, 81, 82, 83, 83, 79, 76, - 76, 84, 75, 74, 73, 81, 81, 74, 68, 72, 81, 88, - 82, 77, 75, 81, 76, 83, 86, 85, 74, 84, 80, 73, - 74, 33, 6 + 15, 27, 42, 57, 68, 68, 69, 65, 63, 65, 65, 60, + 66, 65, 65, 64, 63, 62, 63, 67, 65, 63, 66, 65, + 64, 62, 65, 66, 60, 64, 66, 64, 61, 63, 65, 63, + 65, 62, 60, 65, 64, 62, 63, 63, 64, 59, 67, 63, + 63, 64, 65, 64, 63, 64, 63, 59, 62, 64, 63, 64, + 62, 39, 14 ] } ] @@ -1538,11 +1541,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.02, 0.03, 0.27, 0.49, - 0.46, 0.42, 0.45, 0.47, - 0.42, 0.42, 0.46, 0.45, - 0.42, 0.43, 0.35, 0.02, - 0.02, 0.02, 0.02 + 0.03, 0.14, 0.53, 0.49, + 0.5, 0.49, 0.49, 0.48, + 0.5, 0.49, 0.5, 0.5, + 0.49, 0.49, 0.05, 0.03, + 0.04, 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -1551,11 +1554,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.01, 0.01, 0.29, 0.48, - 0.51, 0.55, 0.52, 0.49, - 0.54, 0.54, 0.49, 0.51, - 0.55, 0.53, 0.4, 0.01, - 0.01, 0.01, 0.02 + 0.03, 0.07, 0.22, 0.23, + 0.23, 0.23, 0.23, 0.23, + 0.22, 0.23, 0.22, 0.21, + 0.24, 0.21, 0.02, 0.02, + 0.03, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -1569,11 +1572,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.04, 0.12, 0.21, 0.24, - 0.21, 0.22, 0.22, 0.21, - 0.21, 0.22, 0.23, 0.21, - 0.21, 0.18, 0.03, 0.04, - 0.03, 0.03, 0.04 + 0.02, 0.02, 0.15, 0.24, + 0.26, 0.26, 0.25, 0.25, + 0.28, 0.26, 0.25, 0.24, + 0.25, 0.28, 0.2, 0.02, + 0.02, 0.02, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -1582,11 +1585,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.02, 0.08, 0.17, 0.16, - 0.16, 0.15, 0.18, 0.17, - 0.17, 0.17, 0.17, 0.17, - 0.16, 0.16, 0.02, 0.02, - 0.02, 0.01, 0.02 + 0.02, 0.02, 0.14, 0.25, + 0.25, 0.24, 0.25, 0.26, + 0.24, 0.25, 0.26, 0.26, + 0.26, 0.25, 0.18, 0.02, + 0.02, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -1598,31 +1601,31 @@ vus_max........................: 50 min=50 max=50 {{< /tab >}} {{< /tabs >}} -It's very close to FastAPI, and database is strangely sleeping far more than with FastAPI, let's keep up on scenario 2. +Far ahead of FastAPI, let's keep up on scenario 2. #### NestJS scenario 2 Iteration creation rate = **3/s** ```txt -checks.........................: 99.93% ✓ 111672 ✗ 72 -data_received..................: 530 MB 6.1 MB/s -data_sent......................: 8.9 MB 103 kB/s -dropped_iterations.............: 109 1.255685/s -http_req_blocked...............: avg=13.34µs min=209ns med=701ns max=48.9ms p(90)=1.15µs p(95)=1.3µs -http_req_connecting............: avg=721ns min=0s med=0s max=18.51ms p(90)=0s p(95)=0s -http_req_duration..............: avg=29.19ms min=1.38ms med=26.19ms max=247.1ms p(90)=53.2ms p(95)=61.63ms -{ expected_response:true }...: avg=29.2ms min=2.01ms med=26.21ms max=247.1ms p(90)=53.21ms p(95)=61.65ms -http_req_failed................: 0.06% ✓ 72 ✗ 111672 -http_req_receiving.............: avg=501.09µs min=16.93µs med=189.85µs max=46.9ms p(90)=1.11ms p(95)=1.88ms -http_req_sending...............: avg=107.93µs min=13.6µs med=76.9µs max=27.43ms p(90)=147.26µs p(95)=188.72µs -http_req_tls_handshaking.......: avg=11.39µs min=0s med=0s max=39.98ms p(90)=0s p(95)=0s -http_req_waiting...............: avg=28.58ms min=0s med=25.61ms max=246.9ms p(90)=52.33ms p(95)=60.73ms -http_reqs......................: 111744 1287.295935/s -iteration_duration.............: avg=45.63s min=26.8s med=50.21s max=55.45s p(90)=54.19s p(95)=54.62s -iterations.....................: 72 0.829443/s -vus............................: 8 min=3 max=50 -vus_max........................: 50 min=50 max=50 +checks.........................: 100.00% ✓ 141232 ✗ 0 +data_received..................: 651 MB 7.4 MB/s +data_sent......................: 12 MB 131 kB/s +dropped_iterations.............: 89 1.010713/s +http_req_blocked...............: avg=7.74µs min=200ns med=577ns max=64.13ms p(90)=896ns p(95)=1.04µs +http_req_connecting............: avg=335ns min=0s med=0s max=6.16ms p(90)=0s p(95)=0s +http_req_duration..............: avg=24.88ms min=1.94ms med=20.22ms max=300.63ms p(90)=49.61ms p(95)=62.06ms + { expected_response:true }...: avg=24.88ms min=1.94ms med=20.22ms max=300.63ms p(90)=49.61ms p(95)=62.06ms +http_req_failed................: 0.00% ✓ 0 ✗ 141232 +http_req_receiving.............: avg=415.42µs min=20.02µs med=172.2µs max=210.76ms p(90)=753.01µs p(95)=1.3ms +http_req_sending...............: avg=94.98µs min=21.16µs med=78.33µs max=25.47ms p(90)=129.63µs p(95)=158.24µs +http_req_tls_handshaking.......: avg=6.32µs min=0s med=0s max=32.94ms p(90)=0s p(95)=0s +http_req_waiting...............: avg=24.37ms min=0s med=19.73ms max=295.3ms p(90)=48.93ms p(95)=61.29ms +http_reqs......................: 141232 1603.877186/s +iteration_duration.............: avg=38.96s min=27.77s med=40.36s max=47s p(90)=45.89s p(95)=46.29s +iterations.....................: 91 1.033426/s +vus............................: 3 min=3 max=50 +vus_max........................: 50 min=50 max=50 ``` {{< tabs >}} @@ -1633,16 +1636,16 @@ vus_max........................: 50 min=50 max=50 { label: 'Req/s', data: [ - 121, 508, 755, 899, 1012, 1007, 1156, 1177, 1096, - 1180, 1224, 1237, 1208, 1244, 1295, 1437, 1445, 1345, - 1338, 1405, 1378, 1380, 1411, 1293, 1420, 1441, 1451, - 1365, 1264, 1439, 1384, 1584, 1241, 1361, 1319, 1427, - 1398, 1362, 1320, 1448, 1482, 1458, 1311, 1256, 1399, - 1363, 1345, 1259, 1346, 1443, 1499, 1445, 1438, 1451, - 1425, 1472, 1479, 1367, 1322, 1450, 1414, 1360, 1355, - 1457, 1326, 1411, 1363, 1350, 1277, 1279, 1168, 1216, - 1198, 1256, 1314, 1248, 1236, 1192, 1183, 1227, 1263, - 1357, 1148, 1141, 1168, 1127, 900, 25 + 18, 297, 824, 1140, 1286, 1523, 1425, 1587, 1485, + 1517, 1692, 1669, 1610, 1651, 1677, 1725, 1693, 1590, + 1501, 1574, 1592, 1775, 1535, 1640, 1710, 1633, 1699, + 1524, 1680, 1653, 1787, 1752, 1658, 1680, 1655, 1754, + 1763, 1669, 1567, 1616, 1758, 1600, 1637, 1609, 1740, + 1810, 1660, 1567, 1721, 1510, 1771, 1786, 1631, 1634, + 1671, 1767, 1754, 1649, 1682, 1500, 1708, 1676, 1687, + 1641, 1572, 1707, 1667, 1534, 1546, 1571, 1711, 1618, + 1670, 1703, 1613, 1784, 1677, 1713, 1672, 1679, 1716, + 1698, 1369, 1662, 1637, 1746, 1629, 1491, 852 ] } ] @@ -1659,12 +1662,12 @@ vus_max........................: 50 min=50 max=50 data: [ 3, 6, 9, 12, 15, 18, 21, 24, 27, 30, 33, 36, 39, 42, 45, 48, 50, 50, 50, 50, 50, 50, 50, 50, - 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, - 50, 50, 50, 50, 50, 50, 49, 50, 49, 50, 50, 49, - 49, 50, 50, 50, 50, 50, 50, 49, 50, 49, 50, 49, - 49, 47, 45, 42, 40, 38, 32, 28, 25, 22, 22, 22, - 22, 22, 22, 22, 22, 22, 22, 20, 20, 18, 16, 15, - 11, 8 + 50, 50, 50, 49, 50, 49, 50, 49, 50, 50, 49, 50, + 50, 49, 50, 50, 49, 50, 49, 50, 49, 49, 49, 50, + 49, 50, 50, 50, 49, 49, 50, 49, 47, 48, 49, 50, + 47, 42, 41, 41, 41, 41, 41, 41, 41, 41, 41, 40, + 40, 38, 36, 35, 34, 33, 32, 31, 30, 29, 27, 24, + 21, 17, 12, 3 ] } ] @@ -1675,14 +1678,14 @@ vus_max........................: 50 min=50 max=50 { label: 'Duration (ms)', data: [ - 8, 8, 9, 11, 13, 16, 16, 18, 23, 23, 25, 27, - 30, 32, 33, 32, 34, 37, 37, 35, 36, 36, 35, 38, - 35, 35, 34, 36, 39, 34, 36, 31, 40, 37, 38, 35, - 36, 36, 36, 35, 33, 34, 38, 39, 36, 36, 37, 40, - 37, 34, 33, 34, 35, 34, 35, 34, 33, 37, 37, 35, - 35, 35, 34, 30, 32, 28, 27, 23, 21, 19, 18, 18, - 18, 17, 17, 17, 18, 18, 19, 18, 16, 14, 15, 14, - 12, 9, 7, 6 + 11, 9, 6, 8, 9, 10, 12, 13, 16, 17, 17, 20, + 22, 23, 25, 26, 28, 31, 34, 32, 31, 28, 32, 30, + 29, 30, 29, 33, 29, 30, 28, 28, 30, 30, 30, 28, + 28, 30, 31, 30, 28, 31, 30, 31, 28, 28, 30, 31, + 29, 33, 28, 28, 30, 30, 29, 28, 28, 29, 29, 32, + 30, 28, 26, 25, 25, 24, 24, 27, 26, 26, 24, 25, + 24, 23, 23, 20, 21, 20, 20, 19, 18, 18, 21, 16, + 15, 12, 11, 9, 6 ] } ] @@ -1696,11 +1699,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.02, 0.03, 0.34, 0.45, - 0.47, 0.48, 0.46, 0.49, - 0.47, 0.48, 0.46, 0.48, - 0.51, 0.49, 0.46, 0.46, - 0.43, 0.42, 0.41 + 0.03, 0.28, 0.48, 0.49, + 0.48, 0.48, 0.48, 0.5, + 0.49, 0.48, 0.48, 0.48, + 0.48, 0.5, 0.49, 0.5, + 0.49, 0.48, 0.35 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -1709,11 +1712,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.01, 0.01, 0.5, 0.53, - 0.51, 0.5, 0.53, 0.49, - 0.51, 0.51, 0.52, 0.51, - 0.47, 0.49, 0.52, 0.52, - 0.54, 0.57, 0.56 + 0.02, 0.11, 0.21, 0.22, + 0.23, 0.24, 0.23, 0.23, + 0.23, 0.25, 0.23, 0.24, + 0.24, 0.21, 0.23, 0.23, + 0.24, 0.22, 0.17 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -1727,11 +1730,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.03, 0.16, 0.23, 0.26, - 0.29, 0.29, 0.29, 0.29, - 0.28, 0.29, 0.27, 0.3, - 0.3, 0.3, 0.27, 0.24, - 0.24, 0.25, 0.1 + 0.02, 0.03, 0.25, 0.32, + 0.32, 0.33, 0.33, 0.34, + 0.35, 0.35, 0.34, 0.34, + 0.34, 0.34, 0.33, 0.33, + 0.34, 0.35, 0.33 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -1740,11 +1743,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.02, 0.08, 0.12, 0.13, - 0.16, 0.14, 0.15, 0.16, - 0.16, 0.15, 0.15, 0.16, - 0.15, 0.15, 0.15, 0.14, - 0.14, 0.13, 0.06 + 0.02, 0.02, 0.16, 0.19, + 0.24, 0.21, 0.23, 0.23, + 0.22, 0.22, 0.24, 0.23, + 0.24, 0.23, 0.22, 0.21, + 0.24, 0.24, 0.21 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -1756,31 +1759,32 @@ vus_max........................: 50 min=50 max=50 {{< /tab >}} {{< /tabs >}} -Now NestJS match the high performance of Symfony. The native even loop system is very efficient. It's time to test it against compiled language. +Now NestJS exceeds the performance of Symfony and FrankenPHP. The native even loop system is very efficient. Note as it's configured to use Fastify under the hood instead of Express. With Express it match 1200 req/s, so Fastify really shines here. It's time to test it against compiled language. ### Spring Boot #### Spring Boot scenario 1 -Iteration creation rate = **30/s** +Iteration creation rate = **40/s** ```txt -checks.........................: 100.00% ✓ 91851 ✗ 0 -data_received..................: 1.6 GB 26 MB/s -data_sent......................: 7.8 MB 129 kB/s -http_req_blocked...............: avg=16.33µs min=191ns med=419ns max=71.26ms p(90)=723ns p(95)=925ns -http_req_connecting............: avg=978ns min=0s med=0s max=19.89ms p(90)=0s p(95)=0s -http_req_duration..............: avg=14.04ms min=2.37ms med=12.32ms max=223.11ms p(90)=24.19ms p(95)=28.67ms -{ expected_response:true }...: avg=14.04ms min=2.37ms med=12.32ms max=223.11ms p(90)=24.19ms p(95)=28.67ms -http_req_failed................: 0.00% ✓ 0 ✗ 91851 -http_req_receiving.............: avg=1.76ms min=19.18µs med=758.76µs max=63.89ms p(90)=4.48ms p(95)=6.73ms -http_req_sending...............: avg=147.52µs min=21.29µs med=51.49µs max=43.07ms p(90)=130.77µs p(95)=286.91µs -http_req_tls_handshaking.......: avg=14.4µs min=0s med=0s max=47.91ms p(90)=0s p(95)=0s -http_req_waiting...............: avg=12.12ms min=0s med=10.43ms max=220.97ms p(90)=21.23ms p(95)=24.99ms -http_reqs......................: 91851 1518.447027/s -iteration_duration.............: avg=741.16ms min=485.58ms med=732.29ms max=1.18s p(90)=865.14ms p(95)=909.61ms -iterations.....................: 1801 29.773471/s -vus............................: 25 min=17 max=29 +checks.........................: 100.00% ✓ 95676 ✗ 0 +data_received..................: 1.8 GB 30 MB/s +data_sent......................: 8.2 MB 135 kB/s +dropped_iterations.............: 525 8.61974/s +http_req_blocked...............: avg=13.02µs min=197ns med=519ns max=66.13ms p(90)=752ns p(95)=866ns +http_req_connecting............: avg=940ns min=0s med=0s max=7.96ms p(90)=0s p(95)=0s +http_req_duration..............: avg=29.35ms min=2.59ms med=26.27ms max=184.74ms p(90)=49.8ms p(95)=59.97ms + { expected_response:true }...: avg=29.35ms min=2.59ms med=26.27ms max=184.74ms p(90)=49.8ms p(95)=59.97ms +http_req_failed................: 0.00% ✓ 0 ✗ 95676 +http_req_receiving.............: avg=2.31ms min=24.31µs med=991.85µs max=87.57ms p(90)=5.43ms p(95)=9.12ms +http_req_sending...............: avg=186.7µs min=24.8µs med=81.4µs max=56.81ms p(90)=159.29µs p(95)=243.03µs +http_req_tls_handshaking.......: avg=10.78µs min=0s med=0s max=54.77ms p(90)=0s p(95)=0s +http_req_waiting...............: avg=26.85ms min=0s med=23.8ms max=180.66ms p(90)=46.18ms p(95)=55.49ms +http_reqs......................: 95676 1570.861508/s +iteration_duration.............: avg=1.55s min=775.41ms med=1.56s max=2.12s p(90)=1.71s p(95)=1.76s +iterations.....................: 1876 30.801206/s +vus............................: 49 min=33 max=50 vus_max........................: 50 min=50 max=50 ``` @@ -1792,15 +1796,15 @@ vus_max........................: 50 min=50 max=50 { label: 'Req/s', data: [ - 353, 1407, 1575, 1522, 1483, 1562, 1587, - 1578, 1491, 1521, 1545, 1561, 1523, 1493, - 1392, 1604, 1609, 1526, 1554, 1493, 1547, - 1558, 1531, 1484, 1511, 1530, 1606, 1548, - 1479, 1459, 1574, 1582, 1575, 1481, 1439, - 1615, 1304, 1567, 1571, 1530, 1610, 1604, - 1516, 1523, 1433, 1630, 1503, 1532, 1557, - 1492, 1559, 1577, 1521, 1497, 1446, 1583, - 1566, 1509, 1424, 1514, 1385 + 48, 1270, 1532, 1558, 1625, 1613, 1419, + 1497, 1628, 1514, 1605, 1489, 1575, 1653, + 1590, 1640, 1483, 1568, 1638, 1623, 1629, + 1524, 1641, 1517, 1616, 1592, 1525, 1613, + 1712, 1674, 1598, 1531, 1554, 1584, 1616, + 1606, 1457, 1539, 1606, 1568, 1606, 1532, + 1533, 1664, 1601, 1574, 1483, 1591, 1665, + 1662, 1590, 1537, 1585, 1624, 1603, 1585, + 1514, 1552, 1681, 1592, 1645, 987 ] } ] @@ -1815,11 +1819,11 @@ vus_max........................: 50 min=50 max=50 { label: 'VUs', data: [ - 20, 23, 24, 25, 25, 24, 20, 20, 22, 21, 19, 18, - 21, 22, 26, 20, 21, 22, 20, 21, 20, 18, 20, 21, - 21, 21, 18, 21, 21, 23, 22, 19, 19, 21, 22, 19, - 29, 26, 27, 25, 22, 18, 21, 24, 24, 21, 21, 23, - 21, 23, 20, 17, 19, 23, 22, 20, 21, 22, 25, 25 + 33, 50, 49, 49, 50, 49, 50, 50, 50, 48, 50, 49, + 49, 49, 48, 49, 50, 49, 46, 49, 47, 50, 50, 48, + 50, 48, 49, 45, 50, 50, 48, 49, 49, 49, 49, 50, + 49, 50, 49, 50, 50, 50, 48, 47, 46, 49, 48, 49, + 48, 50, 48, 49, 50, 48, 50, 49, 46, 48, 49, 49 ] } ] @@ -1830,12 +1834,12 @@ vus_max........................: 50 min=50 max=50 { label: 'Duration (ms)', data: [ - 11, 14, 15, 15, 16, 16, 14, 13, 14, 15, 13, 12, - 12, 14, 16, 15, 14, 13, 14, 14, 13, 13, 12, 14, - 15, 14, 13, 12, 14, 15, 14, 13, 12, 13, 15, 14, - 16, 18, 17, 17, 15, 13, 12, 14, 15, 14, 13, 13, - 14, 14, 13, 12, 12, 13, 15, 14, 13, 13, 16, 17, - 14 + 11, 19, 28, 31, 29, 28, 33, 31, 29, 30, 30, 32, + 30, 29, 29, 29, 32, 30, 29, 29, 29, 31, 28, 32, + 29, 29, 29, 28, 27, 28, 29, 31, 30, 30, 29, 30, + 31, 30, 30, 30, 29, 31, 30, 29, 29, 29, 31, 30, + 28, 28, 30, 31, 30, 29, 30, 29, 31, 30, 28, 30, + 29, 21 ] } ] @@ -1849,11 +1853,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.02, 0.02, 0.06, 0.29, - 0.29, 0.29, 0.29, 0.28, - 0.3, 0.27, 0.28, 0.28, - 0.29, 0.29, 0.26, 0.03, - 0.02, 0.02, 0.02 + 0.03, 0.26, 0.35, 0.32, + 0.32, 0.34, 0.32, 0.32, + 0.32, 0.34, 0.32, 0.33, + 0.33, 0.25, 0.04, 0.03, + 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -1862,11 +1866,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.01, 0.01, 0.04, 0.22, - 0.22, 0.21, 0.21, 0.21, - 0.21, 0.22, 0.22, 0.21, - 0.21, 0.21, 0.21, 0.01, - 0.01, 0.02, 0.01 + 0.03, 0.13, 0.32, 0.32, + 0.32, 0.31, 0.33, 0.31, + 0.32, 0.3, 0.32, 0.33, + 0.32, 0.26, 0.02, 0.02, + 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -1880,11 +1884,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.03, 0.05, 0.56, 0.6, - 0.58, 0.59, 0.58, 0.58, - 0.56, 0.59, 0.59, 0.59, - 0.57, 0.57, 0.04, 0.03, - 0.03, 0.04, 0.03 + 0.02, 0.02, 0.39, 0.59, + 0.59, 0.61, 0.61, 0.61, + 0.6, 0.6, 0.59, 0.59, + 0.57, 0.6, 0.29, 0.02, + 0.03, 0.02, 0.02 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -1893,11 +1897,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.02, 0.03, 0.28, 0.29, - 0.28, 0.28, 0.28, 0.3, - 0.3, 0.27, 0.28, 0.27, - 0.29, 0.3, 0.02, 0.02, - 0.02, 0.01, 0.02 + 0.02, 0.03, 0.23, 0.35, + 0.37, 0.36, 0.35, 0.34, + 0.34, 0.36, 0.38, 0.37, + 0.39, 0.36, 0.16, 0.02, + 0.02, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -1916,24 +1920,24 @@ End of debate, Spring Boot destroys competition for 1st scenario. Moreover, data Iteration creation rate = **10/s** ```txt -checks.........................: 100.00% ✓ 225040 ✗ 0 -data_received..................: 921 MB 11 MB/s -data_sent......................: 19 MB 235 kB/s -dropped_iterations.............: 456 5.549905/s -http_req_blocked...............: avg=6.76µs min=202ns med=389ns max=52.58ms p(90)=662ns p(95)=859ns -http_req_connecting............: avg=275ns min=0s med=0s max=8.36ms p(90)=0s p(95)=0s -http_req_duration..............: avg=16.7ms min=1.6ms med=13.62ms max=237.91ms p(90)=32.02ms p(95)=39.52ms -{ expected_response:true }...: avg=16.7ms min=1.6ms med=13.62ms max=237.91ms p(90)=32.02ms p(95)=39.52ms -http_req_failed................: 0.00% ✓ 0 ✗ 225040 -http_req_receiving.............: avg=1.91ms min=16.76µs med=886.69µs max=211.62ms p(90)=4.52ms p(95)=7.22ms -http_req_sending...............: avg=85.38µs min=17.86µs med=45.75µs max=77.73ms p(90)=86.49µs p(95)=120.1µs -http_req_tls_handshaking.......: avg=5.83µs min=0s med=0s max=51.92ms p(90)=0s p(95)=0s -http_req_waiting...............: avg=14.71ms min=0s med=11.67ms max=222.78ms p(90)=28.88ms p(95)=36.06ms -http_reqs......................: 225040 2738.92674/s -iteration_duration.............: avg=26.14s min=22.07s med=26.88s max=28.52s p(90)=28.02s p(95)=28.16s -iterations.....................: 145 1.764772/s -vus............................: 7 min=7 max=50 -vus_max........................: 50 min=50 max=50 +checks.........................: 100.00% ✓ 173824 ✗ 0 +data_received..................: 774 MB 11 MB/s +data_sent......................: 15 MB 215 kB/s +dropped_iterations.............: 489 6.809786/s +http_req_blocked...............: avg=7.54µs min=212ns med=529ns max=52.17ms p(90)=742ns p(95)=852ns +http_req_connecting............: avg=292ns min=0s med=0s max=4.99ms p(90)=0s p(95)=0s +http_req_duration..............: avg=17.72ms min=1.88ms med=14.67ms max=246.82ms p(90)=33.14ms p(95)=41.12ms + { expected_response:true }...: avg=17.72ms min=1.88ms med=14.67ms max=246.82ms p(90)=33.14ms p(95)=41.12ms +http_req_failed................: 0.00% ✓ 0 ✗ 173824 +http_req_receiving.............: avg=2.56ms min=23.95µs med=1.45ms max=58.29ms p(90)=6.13ms p(95)=8.97ms +http_req_sending...............: avg=102.03µs min=22.45µs med=76.81µs max=42.01ms p(90)=127.42µs p(95)=163.59µs +http_req_tls_handshaking.......: avg=6.34µs min=0s med=0s max=50.96ms p(90)=0s p(95)=0s +http_req_waiting...............: avg=15.06ms min=0s med=12ms max=244.1ms p(90)=29.02ms p(95)=36.66ms +http_reqs......................: 173824 2420.663008/s +iteration_duration.............: avg=27.87s min=12s med=29.74s max=31.63s p(90)=30.76s p(95)=31.06s +iterations.....................: 112 1.559706/s +vus............................: 12 min=10 max=50 +vus_max........................: 50 min=50 max=50 ``` {{< tabs >}} @@ -1944,16 +1948,16 @@ vus_max........................: 50 min=50 max=50 { label: 'Req/s', data: [ - 33, 1444, 2295, 2566, 2449, 2563, 2918, 2891, 2886, - 2676, 2674, 2943, 2975, 2971, 2757, 2645, 2966, 2967, - 2925, 2622, 2678, 2888, 2984, 2972, 2667, 2639, 2896, - 2974, 2853, 2760, 2747, 2897, 2952, 3026, 2633, 2569, - 2890, 3020, 2701, 2521, 2680, 2922, 3013, 2983, 2674, - 2622, 2790, 2463, 2693, 2352, 2853, 2762, 3000, 2960, - 2653, 2615, 2987, 2870, 2875, 2536, 2593, 2903, 2990, - 2712, 2550, 2532, 2903, 2955, 2683, 2485, 2626, 2930, - 3004, 2858, 2664, 2596, 2937, 2942, 2899, 2517, 2436, - 2577, 2012 + 419, 1778, 2087, 2261, 2215, 2426, 2458, 2483, + 2402, 2500, 2525, 2763, 2426, 2634, 2587, 2568, + 2740, 2495, 2510, 2458, 2533, 2615, 2392, 2553, + 2577, 2566, 2756, 2424, 2489, 2428, 2515, 2690, + 2526, 2512, 2500, 2511, 2658, 2473, 2468, 2473, + 2593, 2633, 2481, 2558, 2527, 2491, 2672, 2613, + 2447, 2417, 2479, 2589, 2492, 2500, 2558, 2544, + 2604, 2557, 2593, 2452, 2635, 2598, 2361, 2414, + 2066, 2025, 1951, 1988, 1869, 1881, 2023, 1734, + 85 ] } ] @@ -1969,12 +1973,11 @@ vus_max........................: 50 min=50 max=50 label: 'VUs', data: [ 10, 20, 30, 40, 50, 50, 50, 50, 50, 50, 50, 50, - 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 49, 50, - 50, 48, 50, 50, 50, 50, 48, 49, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, - 50, 50, 49, 49, 49, 50, 50, 50, 50, 47, 48, 49, - 46, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, 45, - 45, 45, 45, 45, 43, 38, 37, 30, 23, 7 + 50, 50, 50, 50, 50, 50, 50, 49, 50, 48, 49, 50, + 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, + 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, + 45, 40, 33, 21, 12, 12, 12, 12, 12, 12, 12 ] } ] @@ -1985,13 +1988,13 @@ vus_max........................: 50 min=50 max=50 { label: 'Duration (ms)', data: [ - 7, 5, 7, 10, 15, 18, 17, 17, 17, 19, 19, 17, - 17, 17, 18, 19, 17, 17, 17, 19, 19, 17, 17, 17, - 19, 19, 17, 17, 17, 18, 18, 17, 16, 16, 19, 19, - 17, 16, 19, 20, 19, 17, 16, 17, 19, 19, 18, 20, - 19, 21, 17, 18, 16, 17, 19, 19, 17, 17, 17, 19, - 19, 16, 15, 16, 18, 18, 15, 15, 17, 18, 17, 16, - 15, 15, 17, 17, 15, 15, 13, 15, 13, 10, 6 + 6, 7, 10, 14, 19, 20, 20, 20, 20, 20, 20, 18, + 20, 19, 19, 19, 18, 20, 20, 20, 20, 19, 21, 19, + 19, 20, 18, 20, 20, 20, 20, 18, 19, 19, 19, 20, + 18, 20, 20, 20, 19, 19, 20, 19, 19, 20, 18, 19, + 20, 20, 20, 19, 20, 20, 19, 20, 19, 19, 19, 20, + 19, 17, 16, 13, 9, 6, 6, 6, 6, 6, 6, 5, + 3 ] } ] @@ -2005,11 +2008,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.03, 0.03, 0.08, 0.37, - 0.41, 0.42, 0.41, 0.42, - 0.43, 0.41, 0.41, 0.4, - 0.38, 0.41, 0.4, 0.42, - 0.39, 0.41, 0.41 + 0.03, 0.51, 0.5, 0.37, + 0.39, 0.39, 0.36, 0.37, + 0.37, 0.38, 0.39, 0.36, + 0.38, 0.37, 0.27, 0.13, + 0.04, 0.03, 0.02 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -2018,11 +2021,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.01, 0.01, 0.05, 0.26, - 0.26, 0.27, 0.25, 0.27, - 0.26, 0.26, 0.26, 0.26, - 0.26, 0.26, 0.27, 0.27, - 0.27, 0.26, 0.24 + 0.02, 0.27, 0.37, 0.36, + 0.34, 0.35, 0.36, 0.35, + 0.36, 0.35, 0.35, 0.34, + 0.35, 0.34, 0.26, 0.12, + 0.03, 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -2036,11 +2039,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.03, 0.1, 0.52, 0.56, - 0.57, 0.58, 0.57, 0.57, - 0.55, 0.56, 0.56, 0.52, - 0.57, 0.55, 0.52, 0.55, - 0.56, 0.54, 0.12 + 0.02, 0.07, 0.42, 0.47, + 0.5, 0.5, 0.49, 0.48, + 0.48, 0.48, 0.49, 0.51, + 0.51, 0.51, 0.44, 0.37, + 0.06, 0.02, 0.02 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -2049,11 +2052,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.02, 0.04, 0.26, 0.3, - 0.31, 0.28, 0.3, 0.3, - 0.31, 0.3, 0.31, 0.28, - 0.3, 0.3, 0.31, 0.3, - 0.31, 0.29, 0.07 + 0.03, 0.05, 0.29, 0.33, + 0.33, 0.32, 0.34, 0.33, + 0.34, 0.34, 0.33, 0.31, + 0.32, 0.31, 0.28, 0.25, + 0.04, 0.02, 0.03 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -2074,24 +2077,24 @@ Java is maybe not the best DX experience for me, but it's a beast in terms of ra Iteration creation rate = **20/s** ```txt -checks.........................: 100.00% ✓ 59109 ✗ 0 -data_received..................: 1.3 GB 22 MB/s -data_sent......................: 5.2 MB 86 kB/s -dropped_iterations.............: 42 0.685501/s -http_req_blocked...............: avg=25.19µs min=212ns med=505ns max=57.73ms p(90)=940ns p(95)=1.13µs -http_req_connecting............: avg=1.4µs min=0s med=0s max=19.45ms p(90)=0s p(95)=0s -http_req_duration..............: avg=43.01ms min=2.75ms med=36.68ms max=278.96ms p(90)=83.54ms p(95)=99.8ms -{ expected_response:true }...: avg=43.01ms min=2.75ms med=36.68ms max=278.96ms p(90)=83.54ms p(95)=99.8ms -http_req_failed................: 0.00% ✓ 0 ✗ 59109 -http_req_receiving.............: avg=1.6ms min=16.85µs med=545.28µs max=53.91ms p(90)=4.05ms p(95)=6.85ms -http_req_sending...............: avg=164.54µs min=13.27µs med=62.93µs max=35.42ms p(90)=166.95µs p(95)=376.28µs -http_req_tls_handshaking.......: avg=22.43µs min=0s med=0s max=52.29ms p(90)=0s p(95)=0s -http_req_waiting...............: avg=41.24ms min=0s med=34.73ms max=275.91ms p(90)=81.52ms p(95)=97.35ms -http_reqs......................: 59109 964.745322/s -iteration_duration.............: avg=2.22s min=759.35ms med=2.33s max=3.24s p(90)=2.74s p(95)=2.85s -iterations.....................: 1159 18.916575/s -vus............................: 21 min=17 max=50 -vus_max........................: 50 min=50 max=50 +checks.........................: 100.00% ✓ 55590 ✗ 0 +data_received..................: 1.3 GB 21 MB/s +data_sent......................: 5.1 MB 83 kB/s +dropped_iterations.............: 110 1.793989/s +http_req_blocked...............: avg=17.61µs min=206ns med=575ns max=65.29ms p(90)=888ns p(95)=1.03µs +http_req_connecting............: avg=806ns min=0s med=0s max=3.77ms p(90)=0s p(95)=0s +http_req_duration..............: avg=48.38ms min=2.81ms med=43.08ms max=286.36ms p(90)=94.36ms p(95)=110.84ms + { expected_response:true }...: avg=48.38ms min=2.81ms med=43.08ms max=286.36ms p(90)=94.36ms p(95)=110.84ms +http_req_failed................: 0.00% ✓ 0 ✗ 55590 +http_req_receiving.............: avg=1.46ms min=22.08µs med=596.35µs max=51.59ms p(90)=3.1ms p(95)=5.66ms +http_req_sending...............: avg=141.6µs min=29.36µs med=88.9µs max=36.89ms p(90)=156.02µs p(95)=214.06µs +http_req_tls_handshaking.......: avg=15.21µs min=0s med=0s max=34.23ms p(90)=0s p(95)=0s +http_req_waiting...............: avg=46.77ms min=0s med=41.21ms max=280.31ms p(90)=92.33ms p(95)=108.67ms +http_reqs......................: 55590 906.61695/s +iteration_duration.............: avg=2.52s min=647.26ms med=2.6s max=3.5s p(90)=2.97s p(95)=3.05s +iterations.....................: 1090 17.776803/s +vus............................: 19 min=14 max=50 +vus_max........................: 50 min=50 max=50 ``` {{< tabs >}} @@ -2102,13 +2105,13 @@ vus_max........................: 50 min=50 max=50 { label: 'Req/s', data: [ - 242, 879, 870, 1010, 1007, 1026, 960, 941, 994, - 1037, 1020, 990, 884, 997, 1001, 1022, 1005, 921, - 941, 979, 984, 969, 891, 966, 978, 1018, 942, - 902, 955, 998, 994, 1009, 907, 969, 991, 975, - 999, 925, 960, 975, 999, 1001, 917, 967, 977, - 1004, 1000, 919, 954, 1001, 992, 996, 936, 963, - 999, 994, 944, 922, 958, 999, 1002, 632 + 275, 849, 897, 867, 871, 966, 947, 937, 863, + 897, 905, 921, 908, 835, 914, 936, 927, 915, + 866, 924, 921, 928, 927, 875, 925, 933, 901, + 915, 874, 923, 943, 931, 933, 841, 884, 948, + 929, 939, 886, 925, 947, 938, 938, 841, 912, + 929, 935, 925, 895, 951, 927, 932, 917, 857, + 901, 926, 914, 928, 882, 912, 955, 627 ] } ] @@ -2123,12 +2126,12 @@ vus_max........................: 50 min=50 max=50 { label: 'VUs', data: [ - 17, 22, 27, 26, 27, 26, 30, 33, 32, 32, 31, 34, - 36, 43, 42, 39, 42, 43, 46, 50, 49, 48, 49, 50, - 46, 44, 47, 49, 49, 48, 50, 43, 50, 45, 43, 44, - 47, 49, 47, 42, 48, 48, 49, 47, 49, 45, 45, 47, - 48, 48, 49, 49, 49, 48, 49, 47, 46, 48, 45, 44, - 21 + 14, 22, 26, 30, 36, 38, 43, 46, 50, 47, 46, 48, + 48, 46, 46, 49, 50, 48, 47, 49, 47, 47, 50, 49, + 48, 48, 47, 50, 47, 41, 46, 49, 48, 47, 49, 48, + 44, 44, 45, 50, 48, 44, 46, 50, 49, 46, 49, 48, + 47, 48, 47, 47, 49, 49, 48, 48, 47, 46, 49, 50, + 19 ] } ] @@ -2139,12 +2142,12 @@ vus_max........................: 50 min=50 max=50 { label: 'Duration (ms)', data: [ - 12, 18, 23, 25, 25, 26, 28, 31, 31, 31, 31, 32, - 38, 37, 39, 39, 39, 45, 45, 48, 49, 49, 53, 51, - 49, 43, 47, 51, 50, 48, 46, 46, 50, 50, 44, 46, - 45, 50, 49, 50, 44, 45, 50, 49, 48, 47, 47, 48, - 50, 47, 46, 48, 51, 51, 48, 49, 49, 51, 49, 47, - 41, 24 + 11, 17, 23, 30, 35, 37, 40, 44, 53, 53, 53, 50, + 51, 58, 51, 50, 52, 51, 53, 52, 52, 50, 52, 55, + 51, 51, 50, 50, 55, 49, 45, 49, 51, 56, 52, 50, + 51, 46, 48, 49, 50, 51, 48, 54, 50, 50, 50, 51, + 53, 49, 48, 49, 51, 55, 52, 52, 52, 50, 53, 51, + 46, 23 ] } ] @@ -2158,11 +2161,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.02, 0.03, 0.34, 0.47, - 0.51, 0.5, 0.49, 0.48, - 0.48, 0.49, 0.49, 0.48, - 0.48, 0.49, 0.26, 0.03, - 0.03, 0.02, 0.02 + 0.02, 0.34, 0.44, 0.4, + 0.42, 0.43, 0.43, 0.45, + 0.44, 0.43, 0.43, 0.42, + 0.43, 0.19, 0.04, 0.03, + 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -2171,11 +2174,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.01, 0.02, 0.13, 0.21, - 0.18, 0.19, 0.19, 0.19, - 0.2, 0.19, 0.19, 0.2, - 0.2, 0.18, 0.1, 0.01, - 0.01, 0.02, 0.01 + 0.01, 0.25, 0.27, 0.27, + 0.29, 0.28, 0.28, 0.26, + 0.27, 0.27, 0.28, 0.28, + 0.26, 0.12, 0.03, 0.02, + 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -2189,11 +2192,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.04, 0.43, 0.76, 0.75, - 0.77, 0.78, 0.79, 0.79, - 0.78, 0.78, 0.77, 0.78, - 0.77, 0.47, 0.03, 0.03, - 0.03, 0.03, 0.03 + 0.02, 0.12, 0.72, 0.75, + 0.74, 0.75, 0.74, 0.73, + 0.74, 0.75, 0.74, 0.74, + 0.75, 0.74, 0.05, 0.02, + 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -2202,11 +2205,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.02, 0.15, 0.2, 0.22, - 0.23, 0.22, 0.2, 0.21, - 0.21, 0.21, 0.22, 0.22, - 0.23, 0.13, 0.01, 0.02, - 0.01, 0.01, 0.01 + 0.02, 0.06, 0.24, 0.25, + 0.25, 0.25, 0.24, 0.25, + 0.25, 0.24, 0.26, 0.25, + 0.24, 0.25, 0.03, 0.02, + 0.02, 0.02 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -2225,24 +2228,24 @@ ASP.NET Core is performing well here. EF Core is incredibly efficient by default Iteration creation rate = **10/s** ```txt -checks.........................: 100.00% ✓ 155200 ✗ 0 -data_received..................: 939 MB 14 MB/s -data_sent......................: 14 MB 202 kB/s -dropped_iterations.............: 500 7.323443/s -http_req_blocked...............: avg=10.06µs min=181ns med=418ns max=74.82ms p(90)=812ns p(95)=997ns -http_req_connecting............: avg=437ns min=0s med=0s max=9.65ms p(90)=0s p(95)=0s -http_req_duration..............: avg=20.44ms min=1.5ms med=15.71ms max=278.6ms p(90)=41.79ms p(95)=52.23ms -{ expected_response:true }...: avg=20.44ms min=1.5ms med=15.71ms max=278.6ms p(90)=41.79ms p(95)=52.23ms -http_req_failed................: 0.00% ✓ 0 ✗ 155200 -http_req_receiving.............: avg=1.52ms min=14.25µs med=653.02µs max=205.85ms p(90)=3.58ms p(95)=5.77ms -http_req_sending...............: avg=89.63µs min=12.82µs med=50.62µs max=36.57ms p(90)=102.45µs p(95)=140.6µs -http_req_tls_handshaking.......: avg=8.79µs min=0s med=0s max=73.62ms p(90)=0s p(95)=0s -http_req_waiting...............: avg=18.82ms min=0s med=14.01ms max=278.29ms p(90)=39.41ms p(95)=49.76ms -http_reqs......................: 155200 2273.196786/s -iteration_duration.............: avg=31.96s min=28.57s med=32.03s max=34.2s p(90)=33.6s p(95)=33.75s -iterations.....................: 100 1.464689/s -vus............................: 8 min=8 max=50 -vus_max........................: 50 min=50 max=50 +checks.........................: 100.00% ✓ 155200 ✗ 0 +data_received..................: 946 MB 13 MB/s +data_sent......................: 14 MB 191 kB/s +dropped_iterations.............: 501 6.896565/s +http_req_blocked...............: avg=7.71µs min=194ns med=532ns max=58.81ms p(90)=794ns p(95)=926ns +http_req_connecting............: avg=342ns min=0s med=0s max=8.25ms p(90)=0s p(95)=0s +http_req_duration..............: avg=21.65ms min=1.63ms med=15.74ms max=322.81ms p(90)=45.46ms p(95)=58.96ms + { expected_response:true }...: avg=21.65ms min=1.63ms med=15.74ms max=322.81ms p(90)=45.46ms p(95)=58.96ms +http_req_failed................: 0.00% ✓ 0 ✗ 155200 +http_req_receiving.............: avg=1.78ms min=19.8µs med=848.9µs max=68.46ms p(90)=4.39ms p(95)=6.92ms +http_req_sending...............: avg=105.11µs min=17.95µs med=80.74µs max=44.98ms p(90)=133.41µs p(95)=169.13µs +http_req_tls_handshaking.......: avg=6.35µs min=0s med=0s max=37.09ms p(90)=0s p(95)=0s +http_req_waiting...............: avg=19.76ms min=0s med=13.7ms max=320.85ms p(90)=42.87ms p(95)=56.35ms +http_reqs......................: 155200 2136.42093/s +iteration_duration.............: avg=33.97s min=29.96s med=34.26s max=36.95s p(90)=35.59s p(95)=35.78s +iterations.....................: 100 1.37656/s +vus............................: 15 min=10 max=50 +vus_max........................: 50 min=50 max=50 ``` {{< tabs >}} @@ -2253,15 +2256,16 @@ vus_max........................: 50 min=50 max=50 { label: 'Req/s', data: [ - 143, 1375, 1715, 1720, 1816, 1913, 2037, 2110, - 2211, 2142, 2381, 2448, 2417, 2323, 2194, 2387, - 2478, 2279, 1817, 2181, 2463, 2412, 2454, 2260, - 2247, 2405, 2495, 2530, 2173, 2184, 2480, 2476, - 2423, 2146, 2209, 2382, 2513, 2344, 2336, 2266, - 2513, 2446, 2496, 2382, 2214, 2371, 2477, 2444, - 2269, 2210, 2439, 2498, 2450, 2349, 2195, 2448, - 2508, 2406, 2316, 2219, 2528, 2503, 2470, 2285, - 2179, 2424, 2374, 2224, 1278 + 152, 1336, 1692, 1841, 2152, 2221, 2151, 2160, + 2149, 2278, 2271, 2110, 2120, 1955, 2258, 2304, + 2194, 2034, 2295, 2273, 2222, 2189, 2153, 2166, + 2190, 2206, 2020, 2213, 2182, 2205, 2365, 2172, + 2177, 2060, 2274, 2279, 2183, 2060, 2085, 2237, + 2292, 2221, 2196, 1923, 2298, 2210, 2104, 2181, + 2044, 2285, 2364, 2224, 2054, 2178, 2226, 2405, + 2215, 2137, 2115, 2255, 2155, 2200, 2125, 2131, + 2308, 2303, 2130, 2105, 2092, 2251, 2222, 1987, + 1633, 47 ] } ] @@ -2278,10 +2282,10 @@ vus_max........................: 50 min=50 max=50 data: [ 10, 20, 30, 40, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, - 50, 50, 50, 50, 50, 49, 50, 50, 49, 50, 47, 49, - 48, 48, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, + 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 49, 50, + 49, 50, 47, 49, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, - 49, 48, 46, 45, 41, 36, 30, 8 + 50, 50, 50, 50, 50, 48, 45, 42, 41, 35, 27, 15 ] } ] @@ -2292,12 +2296,13 @@ vus_max........................: 50 min=50 max=50 { label: 'Duration (ms)', data: [ - 5, 6, 11, 17, 21, 25, 24, 24, 22, 23, 21, 20, - 21, 21, 23, 21, 20, 22, 27, 23, 20, 21, 20, 23, - 22, 21, 20, 20, 22, 23, 20, 20, 20, 23, 22, 20, - 19, 21, 20, 22, 20, 21, 20, 21, 23, 21, 20, 20, - 22, 23, 20, 20, 20, 21, 23, 20, 20, 21, 21, 22, - 20, 19, 19, 20, 20, 17, 15, 13, 9 + 7, 7, 11, 16, 18, 22, 23, 23, 23, 22, 22, 23, + 23, 25, 22, 21, 22, 24, 21, 22, 23, 22, 23, 23, + 23, 22, 24, 22, 23, 23, 21, 23, 23, 23, 23, 22, + 23, 24, 23, 21, 22, 22, 22, 26, 21, 22, 24, 22, + 24, 22, 21, 22, 24, 22, 23, 21, 22, 23, 23, 22, + 23, 23, 24, 22, 22, 21, 22, 21, 20, 18, 15, 13, + 9, 4 ] } ] @@ -2311,11 +2316,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.03, 0.02, 0.04, 0.42, - 0.54, 0.6, 0.6, 0.63, - 0.63, 0.62, 0.63, 0.63, - 0.64, 0.64, 0.62, 0.62, - 0.31, 0.02, 0.03 + 0.03, 0.28, 0.51, 0.52, + 0.53, 0.54, 0.52, 0.53, + 0.53, 0.54, 0.52, 0.52, + 0.52, 0.54, 0.54, 0.44, + 0.04, 0.03, 0.02 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -2324,11 +2329,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.02, 0.01, 0.03, 0.19, - 0.25, 0.3, 0.3, 0.31, - 0.31, 0.3, 0.31, 0.3, - 0.31, 0.31, 0.31, 0.31, - 0.16, 0.01, 0.01 + 0.02, 0.22, 0.4, 0.39, + 0.38, 0.37, 0.39, 0.37, + 0.39, 0.38, 0.42, 0.4, + 0.4, 0.38, 0.38, 0.31, + 0.03, 0.02, 0.03 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -2342,11 +2347,11 @@ vus_max........................: 50 min=50 max=50 { label: 'User', data: [ - 0.03, 0.03, 0.4, 0.49, - 0.56, 0.55, 0.56, 0.56, - 0.54, 0.55, 0.56, 0.55, - 0.54, 0.55, 0.55, 0.31, - 0.03, 0.04, 0.03 + 0.03, 0.02, 0.38, 0.52, + 0.51, 0.52, 0.52, 0.52, + 0.53, 0.52, 0.52, 0.51, + 0.52, 0.51, 0.54, 0.52, + 0.25, 0.03, 0.03 ], borderColor: '#4bc0c0', backgroundColor: '#4bc0c0', @@ -2355,11 +2360,11 @@ vus_max........................: 50 min=50 max=50 { label: 'System', data: [ - 0.02, 0.02, 0.23, 0.3, - 0.32, 0.29, 0.32, 0.32, - 0.32, 0.34, 0.32, 0.34, - 0.34, 0.33, 0.32, 0.18, - 0.01, 0.02, 0.02 + 0.02, 0.02, 0.32, 0.38, + 0.37, 0.36, 0.37, 0.36, + 0.37, 0.36, 0.35, 0.39, + 0.38, 0.39, 0.37, 0.38, + 0.17, 0.02, 0.03 ], borderColor: '#ff6384', backgroundColor: '#ff6384', @@ -2377,205 +2382,185 @@ Not that far to Java variant, just a bit behind. But as workers are fully loaded Here are the final req/s results for each framework against PgSQL database. -{{< chart type="timeseries" title="Scenario 1" >}} +{{< chart type="timeseries" title="Database intensive Scenario" >}} [ - { - label: 'Laravel', - borderColor: '#c2410c', - backgroundColor: '#c2410c', - data: [ - 48, 263, 335, 331, 355, 373, 373, 376, 379, - 359, 383, 382, 386, 388, 351, 387, 374, 367, - 378, 348, 396, 367, 352, 350, 324, 344, 388, - 346, 330, 368, 383, 376, 383, 372, 350, 391, - 380, 365, 367, 340, 380, 375, 381, 383, 354, - 380, 382, 384, 372, 345, 379, 342, 370, 357, - 321, 304, 306, 275, 301, 318, 360, 360, 365, - 240 - ] - }, - { - label: 'Symfony', - borderColor: '#ffffff', - backgroundColor: '#ffffff', - data: [ - 122, 341, 348, 346, 359, 338, 361, 359, 345, - 360, 330, 352, 361, 344, 359, 334, 362, 323, - 346, 358, 335, 357, 352, 348, 358, 337, 359, - 357, 345, 357, 339, 358, 350, 333, 355, 332, - 359, 359, 349, 358, 333, 356, 359, 341, 352, - 333, 355, 360, 344, 356, 340, 360, 360, 348, - 355, 337, 346, 354, 345, 359, 337, 354, 341, - 338, 179 - ] - }, - { - label: 'FastAPI', - borderColor: '#0f766e', - backgroundColor: '#0f766e', - data: [ - 270, 514, 530, 535, 552, 541, 524, 493, 545, - 555, 560, 539, 519, 545, 540, 531, 525, 514, - 547, 540, 537, 533, 485, 511, 534, 525, 508, - 500, 550, 527, 538, 516, 500, 542, 532, 530, - 504, 508, 540, 538, 553, 537, 497, 560, 517, - 578, 559, 487, 551, 546, 538, 531, 517, 518, - 578, 559, 521, 516, 556, 567, 517, 517, 351 - ] - }, - { - label: 'NestJS', - borderColor: '#b91c1c', - backgroundColor: '#b91c1c', - data: [ - 273, 496, 577, 588, 599, 624, 621, 610, 641, - 638, 641, 614, 585, 580, 600, 604, 601, 571, - 606, 663, 643, 572, 596, 585, 616, 650, 680, - 615, 612, 611, 617, 572, 587, 593, 605, 633, - 633, 573, 646, 645, 650, 570, 629, 653, 691, - 650, 580, 555, 590, 646, 565, 585, 638, 594, - 567, 555, 602, 570, 641, 648, 601, 630, 8 - ] - }, - { - label: 'Spring Boot', - borderColor: '#15803d', - backgroundColor: '#15803d', - data: [ - 353, 1407, 1575, 1522, 1483, 1562, 1587, - 1578, 1491, 1521, 1545, 1561, 1523, 1493, - 1392, 1604, 1609, 1526, 1554, 1493, 1547, - 1558, 1531, 1484, 1511, 1530, 1606, 1548, - 1479, 1459, 1574, 1582, 1575, 1481, 1439, - 1615, 1304, 1567, 1571, 1530, 1610, 1604, - 1516, 1523, 1433, 1630, 1503, 1532, 1557, - 1492, 1559, 1577, 1521, 1497, 1446, 1583, - 1566, 1509, 1424, 1514, 1385 - ] - }, - { - label: 'ASP.NET Core', - borderColor: '#6d28d9', - backgroundColor: '#6d28d9', - data: [ - 242, 879, 870, 1010, 1007, 1026, 960, 941, 994, - 1037, 1020, 990, 884, 997, 1001, 1022, 1005, 921, - 941, 979, 984, 969, 891, 966, 978, 1018, 942, - 902, 955, 998, 994, 1009, 907, 969, 991, 975, - 999, 925, 960, 975, 999, 1001, 917, 967, 977, - 1004, 1000, 919, 954, 1001, 992, 996, 936, 963, - 999, 994, 944, 922, 958, 999, 1002, 632 - ] - } + { + label: "Laravel", + borderColor: "#c2410c", + backgroundColor: "#c2410c", + data: [ + 8, 83, 208, 275, 431, 479, 510, 536, 513, 589, 600, 578, 584, 589, 616, + 611, 581, 621, 597, 608, 649, 579, 607, 594, 635, 646, 570, 622, 576, 626, + 641, 607, 623, 603, 614, 623, 602, 614, 617, 645, 628, 570, 603, 599, 646, + 632, 622, 614, 592, 591, 647, 614, 613, 623, 612, 610, 624, 627, 613, 640, + 623, 574, 584, 586, 630, 613, 608, 634, 603, 640, 643, 616, 627, 607, 629, + 631, 628, 624, 626, 640, 622, 576, 636, 619, 647, 646, 596, 625, 606, 634, + 361, + ], + }, + { + label: "Symfony", + borderColor: "#ffffff", + backgroundColor: "#ffffff", + data: [ + 75, 346, 355, 351, 351, 331, 347, 356, 349, 355, 327, 343, 338, 352, 349, + 326, 356, 349, 350, 354, 327, 350, 345, 347, 341, 329, 352, 351, 352, 353, + 331, 357, 355, 350, 351, 314, 350, 355, 351, 357, 330, 354, 355, 350, 353, + 323, 351, 346, 344, 356, 323, 355, 343, 351, 355, 322, 354, 353, 336, 354, + 328, 352, 354, 346, 64, + ], + }, + { + label: "FastAPI", + borderColor: "#0f766e", + backgroundColor: "#0f766e", + data: [ + 345, 451, 480, 469, 476, 507, 461, 476, 523, 494, 467, 493, 476, 501, 483, + 502, 456, 467, 502, 465, 480, 476, 460, 488, 513, 507, 449, 477, 467, 488, + 496, 461, 467, 468, 508, 515, 444, 470, 496, 495, 496, 490, 453, 513, 491, + 481, 441, 471, 520, 463, 495, 457, 484, 454, 475, 467, 462, 479, 459, 481, + 504, 470, 210, + ], + }, + { + label: "NestJS", + borderColor: "#b91c1c", + backgroundColor: "#b91c1c", + data: [ + 209, 657, 702, 731, 710, 709, 685, 759, 770, 744, 743, 767, 751, 750, 755, + 755, 738, 771, 764, 735, 751, 725, 750, 759, 750, 762, 733, 758, 778, 755, + 739, 745, 787, 767, 755, 765, 736, 790, 786, 760, 754, 745, 776, 781, 766, + 768, 698, 791, 755, 744, 749, 741, 790, 762, 775, 781, 756, 752, 769, 753, + 724, 714, 104, + ], + }, + { + label: "Spring Boot", + borderColor: "#15803d", + backgroundColor: "#15803d", + data: [ + 48, 1270, 1532, 1558, 1625, 1613, 1419, 1497, 1628, 1514, 1605, 1489, + 1575, 1653, 1590, 1640, 1483, 1568, 1638, 1623, 1629, 1524, 1641, 1517, + 1616, 1592, 1525, 1613, 1712, 1674, 1598, 1531, 1554, 1584, 1616, 1606, + 1457, 1539, 1606, 1568, 1606, 1532, 1533, 1664, 1601, 1574, 1483, 1591, + 1665, 1662, 1590, 1537, 1585, 1624, 1603, 1585, 1514, 1552, 1681, 1592, + 1645, 987, + ], + }, + { + label: "ASP.NET Core", + borderColor: "#6d28d9", + backgroundColor: "#6d28d9", + data: [ + 275, 849, 897, 867, 871, 966, 947, 937, 863, + 897, 905, 921, 908, 835, 914, 936, 927, 915, + 866, 924, 921, 928, 927, 875, 925, 933, 901, + 915, 874, 923, 943, 931, 933, 841, 884, 948, + 929, 939, 886, 925, 947, 938, 938, 841, 912, + 929, 935, 925, 895, 951, 927, 932, 917, 857, + 901, 926, 914, 928, 882, 912, 955, 627 + ], + }, ] {{< /chart >}} -{{< chart type="timeseries" title="Scenario 2" >}} +{{< chart type="timeseries" title="Runtime intensive Scenario" >}} [ - { - label: 'Laravel', - borderColor: '#c2410c', - backgroundColor: '#c2410c', - data: [ - 59, 137, 245, 368, 393, 519, 334, 446, 490, 576, 526, - 397, 379, 482, 597, 623, 652, 626, 631, 612, 631, 637, - 608, 630, 651, 625, 637, 632, 671, 622, 683, 631, 655, - 641, 632, 673, 659, 603, 650, 652, 642, 623, 668, 678, - 642, 640, 498, 628, 638, 665, 624, 665, 630, 650, 669, - 692, 628, 588, 650, 628, 687, 687, 639, 629, 632, 649, - 657, 608, 669, 648, 658, 601, 618, 654, 654, 654, 521, - 508, 317, 634, 685, 608, 662, 593, 675 - ] - }, - { - label: 'Symfony', - borderColor: '#ffffff', - backgroundColor: '#ffffff', - data: [ - 91, 207, 385, 748, 820, 959, 1075, 964, 1238, - 1212, 1320, 1289, 1154, 1361, 1365, 1307, 1346, 1115, - 1328, 1461, 1238, 1417, 1237, 953, 1350, 1272, 1331, - 1155, 1105, 1017, 1196, 1108, 815, 824, 1327, 1224, - 1334, 1213, 1410, 1430, 1365, 1394, 1299, 1165, 1427, - 1419, 1341, 1234, 1418, 957, 1035, 1028, 929, 880, - 1414, 1491, 1388, 1226, 1510, 1408, 1366, 1571, 1048, - 1439, 1486, 1360, 1401, 1216, 1392, 1408, 1486, 1440, - 1260, 1312, 1341, 1271, 1250, 964, 628, 4 - ] - }, - { - label: 'FastAPI', - borderColor: '#0f766e', - backgroundColor: '#0f766e', - data: [ - 19, 168, 465, 631, 720, 792, 752, 758, 757, 763, 839, - 819, 777, 770, 821, 828, 760, 792, 741, 869, 862, 831, - 846, 811, 820, 878, 792, 811, 815, 829, 804, 807, 842, - 819, 791, 804, 744, 839, 810, 828, 841, 890, 841, 834, - 804, 829, 821, 837, 852, 853, 853, 884, 871, 773, 774, - 825, 794, 832, 825, 787, 807, 872, 837, 815, 826, 778, - 811, 810, 823, 807, 786, 872, 886, 810, 808, 831, 824, - 853, 770, 818, 793, 827, 795, 813, 795, 858, 869, 805, - 846, 823, 563 - ] - }, - { - label: 'NestJS', - borderColor: '#b91c1c', - backgroundColor: '#b91c1c', - data: [ - 121, 508, 755, 899, 1012, 1007, 1156, 1177, 1096, - 1180, 1224, 1237, 1208, 1244, 1295, 1437, 1445, 1345, - 1338, 1405, 1378, 1380, 1411, 1293, 1420, 1441, 1451, - 1365, 1264, 1439, 1384, 1584, 1241, 1361, 1319, 1427, - 1398, 1362, 1320, 1448, 1482, 1458, 1311, 1256, 1399, - 1363, 1345, 1259, 1346, 1443, 1499, 1445, 1438, 1451, - 1425, 1472, 1479, 1367, 1322, 1450, 1414, 1360, 1355, - 1457, 1326, 1411, 1363, 1350, 1277, 1279, 1168, 1216, - 1198, 1256, 1314, 1248, 1236, 1192, 1183, 1227, 1263, - 1357, 1148, 1141, 1168, 1127, 900, 25 - ] - }, - { - label: 'Spring Boot', - borderColor: '#15803d', - backgroundColor: '#15803d', - data: [ - 33, 1444, 2295, 2566, 2449, 2563, 2918, 2891, 2886, - 2676, 2674, 2943, 2975, 2971, 2757, 2645, 2966, 2967, - 2925, 2622, 2678, 2888, 2984, 2972, 2667, 2639, 2896, - 2974, 2853, 2760, 2747, 2897, 2952, 3026, 2633, 2569, - 2890, 3020, 2701, 2521, 2680, 2922, 3013, 2983, 2674, - 2622, 2790, 2463, 2693, 2352, 2853, 2762, 3000, 2960, - 2653, 2615, 2987, 2870, 2875, 2536, 2593, 2903, 2990, - 2712, 2550, 2532, 2903, 2955, 2683, 2485, 2626, 2930, - 3004, 2858, 2664, 2596, 2937, 2942, 2899, 2517, 2436, - 2577, 2012 - ] - }, - { - label: 'ASP.NET Core', - borderColor: '#6d28d9', - backgroundColor: '#6d28d9', - data: [ - 143, 1375, 1715, 1720, 1816, 1913, 2037, 2110, - 2211, 2142, 2381, 2448, 2417, 2323, 2194, 2387, - 2478, 2279, 1817, 2181, 2463, 2412, 2454, 2260, - 2247, 2405, 2495, 2530, 2173, 2184, 2480, 2476, - 2423, 2146, 2209, 2382, 2513, 2344, 2336, 2266, - 2513, 2446, 2496, 2382, 2214, 2371, 2477, 2444, - 2269, 2210, 2439, 2498, 2450, 2349, 2195, 2448, - 2508, 2406, 2316, 2219, 2528, 2503, 2470, 2285, - 2179, 2424, 2374, 2224, 1278 - ] - } + { + label: "Laravel", + borderColor: "#c2410c", + backgroundColor: "#c2410c", + data: [ + 8, 83, 208, 275, 431, 479, 510, 536, 513, 589, 600, 578, 584, 589, 616, + 611, 581, 621, 597, 608, 649, 579, 607, 594, 635, 646, 570, 622, 576, 626, + 641, 607, 623, 603, 614, 623, 602, 614, 617, 645, 628, 570, 603, 599, 646, + 632, 622, 614, 592, 591, 647, 614, 613, 623, 612, 610, 624, 627, 613, 640, + 623, 574, 584, 586, 630, 613, 608, 634, 603, 640, 643, 616, 627, 607, 629, + 631, 628, 624, 626, 640, 622, 576, 636, 619, 647, 646, 596, 625, 606, 634, + 361, + ], + }, + { + label: "Symfony", + borderColor: "#ffffff", + backgroundColor: "#ffffff", + data: [ + 162, 449, 974, 1254, 1075, 1260, 1289, 1267, 1306, 1347, 1403, 1335, 1225, + 1431, 1295, 1330, 1365, 1344, 1461, 1369, 1461, 1398, 1310, 1460, 1412, + 1433, 1475, 1373, 1393, 1342, 1481, 1446, 1364, 1386, 1359, 1353, 1368, + 1316, 1386, 1258, 1451, 1502, 1327, 1419, 1327, 1339, 1389, 1373, 1388, + 1361, 1494, 1454, 1337, 1420, 1374, 1440, 1408, 1319, 1484, 1370, 1433, + 1329, 1334, 1393, 1415, 1347, 1399, 1412, 1320, 1290, 1508, 1401, 1312, + 1454, 1315, 1347, 1452, 1221, 1262, 1221, 1259, 1306, 676, 95, + ], + }, + { + label: "FastAPI", + borderColor: "#0f766e", + backgroundColor: "#0f766e", + data: [ + 45, 176, 457, 647, 698, 657, 670, 671, 738, 801, 676, 783, 692, 700, 760, + 688, 743, 676, 698, 740, 695, 726, 763, 759, 723, 684, 774, 731, 740, 778, + 685, 792, 746, 755, 733, 682, 753, 730, 763, 735, 707, 792, 744, 695, 764, + 679, 778, 692, 775, 715, 675, 774, 712, 714, 742, 699, 719, 684, 762, 789, + 681, 752, 646, 722, 769, 651, 778, 743, 692, 764, 701, 736, 719, 816, 768, + 707, 765, 735, 762, 725, 701, 714, 652, 769, 784, 713, 703, 735, 767, 754, + 225, + ], + }, + { + label: "NestJS", + borderColor: "#b91c1c", + backgroundColor: "#b91c1c", + data: [ + 18, 297, 824, 1140, 1286, 1523, 1425, 1587, 1485, 1517, 1692, 1669, 1610, + 1651, 1677, 1725, 1693, 1590, 1501, 1574, 1592, 1775, 1535, 1640, 1710, + 1633, 1699, 1524, 1680, 1653, 1787, 1752, 1658, 1680, 1655, 1754, 1763, + 1669, 1567, 1616, 1758, 1600, 1637, 1609, 1740, 1810, 1660, 1567, 1721, + 1510, 1771, 1786, 1631, 1634, 1671, 1767, 1754, 1649, 1682, 1500, 1708, + 1676, 1687, 1641, 1572, 1707, 1667, 1534, 1546, 1571, 1711, 1618, 1670, + 1703, 1613, 1784, 1677, 1713, 1672, 1679, 1716, 1698, 1369, 1662, 1637, + 1746, 1629, 1491, 852, + ], + }, + { + label: "Spring Boot", + borderColor: "#15803d", + backgroundColor: "#15803d", + data: [ + 419, 1778, 2087, 2261, 2215, 2426, 2458, 2483, 2402, 2500, 2525, 2763, + 2426, 2634, 2587, 2568, 2740, 2495, 2510, 2458, 2533, 2615, 2392, 2553, + 2577, 2566, 2756, 2424, 2489, 2428, 2515, 2690, 2526, 2512, 2500, 2511, + 2658, 2473, 2468, 2473, 2593, 2633, 2481, 2558, 2527, 2491, 2672, 2613, + 2447, 2417, 2479, 2589, 2492, 2500, 2558, 2544, 2604, 2557, 2593, 2452, + 2635, 2598, 2361, 2414, 2066, 2025, 1951, 1988, 1869, 1881, 2023, 1734, + 85, + ], + }, + { + label: "ASP.NET Core", + borderColor: "#6d28d9", + backgroundColor: "#6d28d9", + data: [ + 152, 1336, 1692, 1841, 2152, 2221, 2151, 2160, + 2149, 2278, 2271, 2110, 2120, 1955, 2258, 2304, + 2194, 2034, 2295, 2273, 2222, 2189, 2153, 2166, + 2190, 2206, 2020, 2213, 2182, 2205, 2365, 2172, + 2177, 2060, 2274, 2279, 2183, 2060, 2085, 2237, + 2292, 2221, 2196, 1923, 2298, 2210, 2104, 2181, + 2044, 2285, 2364, 2224, 2054, 2178, 2226, 2405, + 2215, 2137, 2115, 2255, 2155, 2200, 2125, 2131, + 2308, 2303, 2130, 2105, 2092, 2251, 2222, 1987, + 1633, 47 + ], + }, ] {{< /chart >}} -To resume, compiled languages have always a clear advantage when it comes to raw performance. But do you really need it ? +To resume, compiled languages have always a clear advantage when it comes to raw performance. -Performance isn't the main criteria for a web framework. The DX is also very important, in that regards Laravel stays a very nice candidate, and you always have Octane for high performance if needed. +Performance isn't the main criteria for a web framework. The DX is also very important, in that regard Laravel stays a very nice candidate, and you always have Octane for high performance if needed, while it's far less behind than Symfony. -As we have seen with Symfony, PHP is now really back in the game in term of raw performance, competing against NodeJS. And no more any headaches for worker configuration thanks to the excellent FrankenPHP runtime which provides production optimized docker images. +As we have seen with Symfony, PHP is now really back in the game in terms of raw performance, almost competing against NodeJS and outperforming Python. And no more any headaches for worker configuration thanks to the excellent FrankenPHP runtime which provides production optimized docker images. -When it comes to compiled languages, I still personally prefer the DX of ASP.NET Core over Spring Boot. The performance gap is negligible, and it hasn't this warmup Java feeling and keeps a raisonable memory footprint. +When it comes to compiled languages, I still personally prefer the DX of ASP.NET Core over Spring Boot. The performance gap is negligible, and it hasn't this warm up Java feeling and keeps a reasonable memory footprint.