Add language statistics API endpoint

[lafriks]

Fixes #10232

[CirnoT]

Why use custom and not json?

[6543]

I dont know how hard this is - but tests would be nice

[lafriks]

@CirnoT to return languages in descending order by respective code size in bytes. Map serialization would return them in alphabetical order

[CirnoT]

JSON map objects have undefined order, no one should depend on what order we return them in - if they do, their code is bound to break on something at some point, 100% guaranteed.

[lafriks]

that's why I don't use map and use custom json serialization

[CirnoT]

You don't understand me, it's not Go spec, its JSON spec. You shall not return ordered or depend on order of JSON objects, EVER.

In fact you can not even rely on what order you will get after you use some language built-in function to fetch that JSON string from our API endpoint.

[CirnoT]

In other words, If you have code like this:

JSON.parse('{"PHP":23334, "Go":6754}')

And rely or expect that PHP will always be first on iteration, you are doing things wrong.

[CirnoT]

The reason why GitHub returns them ordered is most likely because they internally iterate in that way, not because they TRY to present such order.

[lafriks]

But if you use JSON reader like parsing than you will get primary language first. imho github specifically returns them it this order not just by coincidence

[lafriks]

Test added

[CirnoT]

I can not agree with explicitly suggesting order and encouraging developers to rely on it; it's just wrong.

[CirnoT]

But if you use JSON reader like parsing than you will get primary language first

Today it works for you, tomorrow you will update unrelated dependency and it won't work anymore because you were relying on something that is strictly against JSON spec.

[lafriks]

It's not against spec, against spec would be if json was invalid. There is no reason why this code would start returning them in different order. There are languages that has ordered map classes that would decode this structure just fine, if you just use default JSON.parse it will work same no matter in what order we return them

[CirnoT]

https://www.json.org/json-en.html

An object is an unordered set of name/value pairs

As a consequence, JSON libraries are free to rearrange the order of the elements as they see fit

[CirnoT]

Your code attempts to explicitly return object in specific order, and by your explanation I fully expect that you believe you can write code that will assume first element to be always primary language. That is incorrect. Some libraries will preserve input order, some will re-arrange it alphabetically, others will try to move integer values above other values.

I don't mind any specific order, what I have issue is that you are forcing specific order and to do that, you go to insane lengths of writing own MarshalJSON.

[CirnoT]

For example, if someone were to naively use our API in Go and expect first element to by primary language, he would be very surprised by the result: https://play.golang.org/p/XYUrtJlEG_b

As such, I don't see any point on keeping any order for what we return, especially when it requires additional code that is subject to breakage.

[lafriks]

I understand all that but that does not mean we can not return it using same order as github to be as compatible with them as possible. It's up to the API consumer to either to depend on it or not

[lafriks]

As for MarshalJSON length it's not that insanely big or complex

[CirnoT]

to be as compatible with them as possible

It's compatible either way

It's up to the API consumer to either to depend on it or not

No, it's not. Attempting to depend on it is incorrect.

[lafriks]

I do understand your point but I don't see much of the problem returning exactly like github does and I can bet they do that order intentionally.
Also not that it changes much but this way it is less memory consuming :]

[6543]

beside the none default way how it is converted ... I'm ok with this

works smothly 🙃

[CirnoT]

I am not convinced that your arguments warrant introducing custom JSON marshal function; I'll let someone else review it.

[zeripath]

Although I take @CirnoT's point that if we want order we should use a list - I suspect that we need to do this to keep compatibility with GH.

[techknowlogick]

Like @CirnoT I also would prefer the built-in marshal, and that even though Github's list is sorted by us doing it then that creates a guarantee that the first elem will always be primary, however Github doesn't state that as a promise in their documentation. So I don't think we should make that promise either to encourage devs working with API to not take data at face value.

If we put it in without the sorting, then later we decide to presort it, then that wouldn't be a breaking change, but if we start out sorting the list and decide to switch that would be breaking.

Otherwise, this PR LGTM and so I won't block it. Sidenote: @lafriks I can click "update branch" are you able to check "accept edits from maintainers"?

[lafriks]

I can't seem to find such option for existing PR :)

[6543]

Ping bot