Add Aero

[proyb6]

Language : Go
Repo : https://github.com/aerogo/aero
Author : Unknown

[waghanza]

Hi @akyoto,

Would you mind to add (or consent that we add) aero in here ?

This project compare performance based on 3 routes :

• GET /, SHOULD return a successful status code and an empty string
• GET /user/:id, SHOULD return a successful status code and the id
• POST /user, SHOULD return a successful status code and an empty string

Regards,

[proyb6]

@waghanza Forgot to add in my 1st comment, for your useful reference which I came across Aero from this repo
https://github.com/julienschmidt/go-http-routing-benchmark

[akyoto]

@waghanza Is the reason you are only testing a 3-routes scenario the reduction in complexity when implementing a lot of routers for the benchmark?

I can see that limiting the testing scenario to just 3 routes probably makes the implementation easier, but the downside of it is that it's not a realistic scenario (no real-world API or website has 3 routes only) and you're basically testing nearly empty routing trees.

With only 3 routes, I can imagine that routers in this benchmark will perform very similar to each other, because they all have nearly empty trees and perform nearly the same tasks. The benchmark doesn't reflect a real-world scenario like Julien Schmidt's benchmark.

I can respect the decision to make the benchmark implementation easier, however I also think that loading a few routes from a file isn't particularly hard to implement with the frameworks tested here.

This is all just constructive feedback, please let me know about your thoughts on the matter.

[waghanza]

@akyoto In deed, there is a reason why we are using a simple scenario (a 3 routes one) -> history

However, it will change #1728 (at least when I have enough time, to do it myself, or if someone start with it ❤️ )

The purpose, is to have a more complex / real-world scenario, before considering this project are reliable, now it's more about research and fun, we are not at the same matutity level than https://www.techempower.com/benchmarks/

[akyoto]

According to the benchmarks listed in the README, chi (6.77 ms) outperforms echo (8.50 ms) with very similar throughput which is a misleading result because the routing trees are almost empty.

People will think "aha, chi is pretty fast so let's use it for our next project!" and once they start using it they will suddenly notice that Chi is about 6x slower than Echo for real work loads according to my own results.

This is just one example of how misleading the information in this benchmark is.

The issue #1728 mentions more HTTP verbs and JSON replies, however those are not the main problem. The main problem is the lack of sufficient routes. 50 routes seems like the minimum for a non-trivial web application. Most web applications I worked with use 100-200 routes.

Focusing on automated loading of routes from a single file would put this benchmark ahead of many others, from my perspective.

[waghanza]

@akyoto I understand what you are saying, and I'll be glad you join this task force ❤️

The more routes we have, the more complex is the maintenance, thus I'm not convinced with the fact number of routes could reflect a real-word application

I know this project is not perfect, but none benchmark are 😛

[akyoto]

I see, I can't make any promises but if I have some free time left I'll try to help with the implementation.

[createconnect]

Hello, when can we see Aero result up?

[akyoto]

@createconnect

It will be exactly the same speed as every other net/http based lib for the reasons I mentioned in this thread.

---

I sincerely think this benchmark should put more weight into testing the actual router instead of the HTTP stack of the language.

At this rate we're testing 95% Go's HTTP stack and only 5% actual routing speed.

@waghanza already mentioned that no benchmark is perfect but also said that the maintenance of more routes is too difficult.

My counter-argument is that routes can be loaded from a single definition file which would automate the whole process. We could load 10, 20, ...50 routes, as many as you specify in the definition file for the benchmark. All frameworks use the same file, hence same routes. This would shift the weights in favor of testing the actual framework/router.

However, even if we all agree that testing the actual package is more important than testing the language's HTTP stack, somebody still needs to volunteer and rewrite the benchmarks for all languages. This is an immense effort and I can sympathize with the decision to keep it simple.

[createconnect]

@akyoto Could you compare with fasthttp? I can present the result for our next project.

[akyoto]

To do a comparison, you would actually need to compare 2 things:

• fasthttp needs to be compared with net/http.
• fasthttprouter needs to be compared with Aero.

---

• I know that fasthttp is multiple times faster than net/http because it has a much better API.
• I have no numbers regarding fasthttprouter vs Aero but I suspect that fasthttprouter would perform better simply because the fasthttp stack is so great.

---

Fun fact: Aero was fasthttp based 2 years ago but I switched to net/http for HTTP/2 support.

I would go back to using fasthttp in Aero any day (I love it) but so far only net/http supports HTTP/2.

---

Conclusion:

• If you or your company only needs HTTP/1.1 support, I recommend using fasthttp(router). It has amazing performance.
• If you need HTTP/2, then Aero will do its job among the net/http based packages.