okami101/blog
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 46 Wiki Activity

label

Browse Source
This commit is contained in:
Adrien Beaudouin
2023-12-27 15:23:59 +01:00
parent 375aec9175
commit 3ddc9cf9a6
1 changed files with 113 additions and 113 deletions
Download Patch File Download Diff File Expand all files Collapse all files

226
content/posts/22-web-api-benchmarks-2024/index.md
View File

@ -86,7 +86,7 @@ We'll be using [k6](https://k6.io/) to run the tests, with [constant-arrival-rat
Duration of each scenario is 1 minute, with a 30 seconds graceful for finishing last started iterations. Results with one single test failures, i.e. any response status different than 200 or any response json error parsing, are not accepted.
The **iteration rate** (rate / timeUnit) will be choosen in order to obtain the highest possible request rate, without any test failures.
The **Iteration creation rate** (rate / timeUnit) will be choosen in order to obtain the highest possible request rate, without any test failures.
### Scenario 1
@ -250,13 +250,13 @@ export default function () {
{{< tabs >}}
{{< tab tabName="Counters & Req/s" >}}
| Metric | Value |
| ------------------ | --------- |
| Iteration rate | **3/s** |
| Total requests | **8007** |
| Total iterations | **157** |
| Average max req/s | **140** |
| p(90) req duration | **544ms** |
| Metric | Value |
| ----------------------- | --------- |
| Iteration creation rate | **3/s** |
| Total requests | **8007** |
| Total iterations | **157** |
| Average max req/s | **140** |
| p(90) req duration | **544ms** |
{{< chart type="timeseries" title="Req/s count" >}}
[
@ -340,13 +340,13 @@ As expected here, database is the bottleneck. We'll get slow response time at fu
{{< tabs >}}
{{< tab tabName="Counters & Req/s" >}}
| Metric | Value |
| ------------------ | --------- |
| Iteration rate | **1/2/s** |
| Total requests | **29015** |
| Total iterations | **5** |
| Average max req/s | **360** |
| p(90) req duration | **117ms** |
| Metric | Value |
| ----------------------- | --------- |
| Iteration creation rate | **1/2/s** |
| Total requests | **29015** |
| Total iterations | **5** |
| Average max req/s | **360** |
| p(90) req duration | **117ms** |
{{< chart type="timeseries" title="Req/s count" >}}
[
@ -430,13 +430,13 @@ Now we have a very runtime intensive scenario, with workers as bottleneck, API i
{{< tabs >}}
{{< tab tabName="Counters & Req/s" >}}
| Metric | Value |
| ------------------ | --------- |
| Iteration rate | **2/s** |
| Total requests | **4386** |
| Total iterations | **86** |
| Average max req/s | **70** |
| p(90) req duration | **1.24s** |
| Metric | Value |
| ----------------------- | --------- |
| Iteration creation rate | **2/s** |
| Total requests | **4386** |
| Total iterations | **86** |
| Average max req/s | **70** |
| p(90) req duration | **1.24s** |
{{< chart type="timeseries" title="Req/s count" >}}
[
@ -520,13 +520,13 @@ Now it seems interesting, Laravel performs literally about 2x slower with Postgr
{{< tabs >}}
{{< tab tabName="Counters & Req/s" >}}
| Metric | Value |
| ------------------ | --------- |
| Iteration rate | **1/3/s** |
| Total requests | **16219** |
| Total iterations | **0** |
| Average max req/s | **220** |
| p(90) req duration | **128ms** |
| Metric | Value |
| ----------------------- | --------- |
| Iteration creation rate | **1/3/s** |
| Total requests | **16219** |
| Total iterations | **0** |
| Average max req/s | **220** |
| p(90) req duration | **128ms** |
{{< chart type="timeseries" title="Req/s count" >}}
[
@ -612,13 +612,13 @@ Laravel seems less limited by database performance, but still slower than MySQL.
{{< tabs >}}
{{< tab tabName="Counters & Req/s" >}}
| Metric | Value |
| ------------------ | --------- |
| Iteration rate | **5/s** |
| Total requests | **10302** |
| Total iterations | **202** |
| Average max req/s | **160** |
| p(90) req duration | **399s** |
| Metric | Value |
| ----------------------- | --------- |
| Iteration creation rate | **5/s** |
| Total requests | **10302** |
| Total iterations | **202** |
| Average max req/s | **160** |
| p(90) req duration | **399s** |
{{< chart type="timeseries" title="Req/s count" >}}
[
@ -702,13 +702,13 @@ It's very similar to Laravel in this case.
{{< tabs >}}
{{< tab tabName="Counters & Req/s" >}}
| Metric | Value |
| ------------------ | --------- |
| Iteration rate | **1/2/s** |
| Total requests | **32086** |
| Total iterations | **18** |
| Average max req/s | **410** |
| p(90) req duration | **41ms** |
| Metric | Value |
| ----------------------- | --------- |
| Iteration creation rate | **1/2/s** |
| Total requests | **32086** |
| Total iterations | **18** |
| Average max req/s | **410** |
| p(90) req duration | **41ms** |
{{< chart type="timeseries" title="Req/s count" >}}
[
@ -792,13 +792,13 @@ Symfony is able to handle the load, slightly than Laravel in the same context, w
{{< tabs >}}
{{< tab tabName="Counters & Req/s" >}}
| Metric | Value |
| ------------------ | --------- |
| Iteration rate | **5/s** |
| Total requests | **10302** |
| Total iterations | **160** |
| Average max req/s | **160** |
| p(90) req duration | **379ms** |
| Metric | Value |
| ----------------------- | --------- |
| Iteration creation rate | **5/s** |
| Total requests | **10302** |
| Total iterations | **160** |
| Average max req/s | **160** |
| p(90) req duration | **379ms** |
{{< chart type="timeseries" title="Req/s count" >}}
[
@ -882,13 +882,13 @@ Symfony performs same with PostgreSQL and MySQL, contrary to Laravel which perfo
{{< tabs >}}
{{< tab tabName="Counters & Req/s" >}}
| Metric | Value |
| ------------------ | --------- |
| Iteration rate | **1/3/s** |
| Total requests | **19633** |
| Total iterations | **4** |
| Average max req/s | **250** |
| p(90) req duration | **95ms** |
| Metric | Value |
| ----------------------- | --------- |
| Iteration creation rate | **1/3/s** |
| Total requests | **19633** |
| Total iterations | **4** |
| Average max req/s | **250** |
| p(90) req duration | **95ms** |
{{< chart type="timeseries" title="Req/s count" >}}
[
@ -976,13 +976,13 @@ As a side note here, uvicorn is limited to 1 CPU core, so I use 2 replicas on ea
{{< tabs >}}
{{< tab tabName="Counters & Req/s" >}}
| Metric | Value |
| ------------------ | --------- |
| Iteration rate | **10/s** |
| Total requests | **30651** |
| Total iterations | **601** |
| Average max req/s | **550** |
| p(90) req duration | **49ms** |
| Metric | Value |
| ----------------------- | --------- |
| Iteration creation rate | **10/s** |
| Total requests | **30651** |
| Total iterations | **601** |
| Average max req/s | **550** |
| p(90) req duration | **49ms** |
{{< chart type="timeseries" title="Req/s count" >}}
[
@ -1066,13 +1066,13 @@ Now we are talking, FastAPI outperforms above PHP frameworks, and database isn't
{{< tabs >}}
{{< tab tabName="Counters & Req/s" >}}
| Metric | Value |
| ------------------ | --------- |
| Iteration rate | **2/s** |
| Total requests | **71394** |
| Total iterations | **16** |
| Average max req/s | **870** |
| p(90) req duration | **113ms** |
| Metric | Value |
| ----------------------- | --------- |
| Iteration creation rate | **2/s** |
| Total requests | **71394** |
| Total iterations | **16** |
| Average max req/s | **870** |
| p(90) req duration | **113ms** |
{{< chart type="timeseries" title="Req/s count" >}}
[
@ -1158,13 +1158,13 @@ FastAPI performs around at least twice better than PHP main frameworks in every
{{< tabs >}}
{{< tab tabName="Counters & Req/s" >}}
| Metric | Value |
| ------------------ | --------- |
| Iteration rate | **15/s** |
| Total requests | **37281** |
| Total iterations | **731** |
| Average max req/s | **700** |
| p(90) req duration | **Xms** |
| Metric | Value |
| ----------------------- | --------- |
| Iteration creation rate | **15/s** |
| Total requests | **37281** |
| Total iterations | **731** |
| Average max req/s | **700** |
| p(90) req duration | **Xms** |
{{< chart type="timeseries" title="Req/s count" >}}
[
@ -1248,13 +1248,13 @@ It's slightly better than FastAPI, let's keep up on scenario 2.
{{< tabs >}}
{{< tab tabName="Counters & Req/s" >}}
| Metric | Value |
| ------------------ | ---------- |
| Iteration rate | **3/s** |
| Total requests | **105536** |
| Total iterations | **68** |
| Average max req/s | **1400** |
| p(90) req duration | **53ms** |
| Metric | Value |
| ----------------------- | ---------- |
| Iteration creation rate | **3/s** |
| Total requests | **105536** |
| Total iterations | **68** |
| Average max req/s | **1400** |
| p(90) req duration | **53ms** |
{{< chart type="timeseries" title="Req/s count" >}}
[
@ -1340,13 +1340,13 @@ Huge gap now, NestJS is the clear winner so far. The native even loop system see
{{< tabs >}}
{{< tab tabName="Counters & Req/s" >}}
| Metric | Value |
| ------------------ | --------- |
| Iteration rate | **30/s** |
| Total requests | **91851** |
| Total iterations | **1801** |
| Average max req/s | **1600** |
| p(90) req duration | **33ms** |
| Metric | Value |
| ----------------------- | --------- |
| Iteration creation rate | **30/s** |
| Total requests | **91851** |
| Total iterations | **1801** |
| Average max req/s | **1600** |
| p(90) req duration | **33ms** |
{{< chart type="timeseries" title="Req/s count" >}}
[
@ -1430,13 +1430,13 @@ End of debate, Spring Boot destroys competition for 1st scenario. Moreover, data
{{< tabs >}}
{{< tab tabName="Counters & Req/s" >}}
| Metric | Value |
| ------------------ | ---------- |
| Iteration rate | **10/s** |
| Total requests | **197104** |
| Total iterations | **127** |
| Average max req/s | **2900** |
| p(90) req duration | **33ms** |
| Metric | Value |
| ----------------------- | ---------- |
| Iteration creation rate | **10/s** |
| Total requests | **197104** |
| Total iterations | **127** |
| Average max req/s | **2900** |
| p(90) req duration | **33ms** |
{{< chart type="timeseries" title="Req/s count" >}}
[
@ -1523,13 +1523,13 @@ Java is maybe not the best DX experience for me, but it's a beast in terms of ra
{{< tabs >}}
{{< tab tabName="Counters & Req/s" >}}
| Metric | Value |
| ------------------ | --------- |
| Iteration rate | **20** |
| Total requests | **57936** |
| Total iterations | **1136** |
| Average max req/s | **980** |
| p(90) req duration | **87ms** |
| Metric | Value |
| ----------------------- | --------- |
| Iteration creation rate | **20** |
| Total requests | **57936** |
| Total iterations | **1136** |
| Average max req/s | **980** |
| p(90) req duration | **87ms** |
{{< chart type="timeseries" title="Req/s count" >}}
[
@ -1613,13 +1613,13 @@ ASP.NET Core is performing well here. EF Core is incredibly efficient by default
{{< tabs >}}
{{< tab tabName="Counters & Req/s" >}}
| Metric | Value |
| ------------------ | ---------- |
| Iteration rate | **5** |
| Total requests | **167616** |
| Total iterations | **108** |
| Average max req/s | **2500** |
| p(90) req duration | **38ms** |
| Metric | Value |
| ----------------------- | ---------- |
| Iteration creation rate | **5** |
| Total requests | **167616** |
| Total iterations | **108** |
| Average max req/s | **2500** |
| p(90) req duration | **38ms** |
{{< chart type="timeseries" title="Req/s count" >}}
[