synapse.metrics: implement detailed memory usage reporting on PyPy

[intelfx]

PyPy's gc.get_stats() returns an object containing detailed allocator statistics
which could be beneficial to collect as metrics.

Signed-off-by: Ivan Shapovalov intelfx@intelfx.name

Pull Request Checklist

• Pull request is based on the develop branch
• Pull request includes a changelog file. The entry should:
  • Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
  • Use markdown where necessary, mostly for code blocks.
  • End with either a period (.) or an exclamation mark (!).
  • Start with a capital letter.
• Pull request includes a sign off
• Code style is correct (run the linters)

[intelfx]

Should I add sample panes to the contrib Grafana dashboard?

[richvdh]

Should I add sample panes to the contrib Grafana dashboard?

I'd rather not, as it will mean that everybody else who uses the dashboard ends up with useless panes. I'm not quite sure of the best way forward here though.

In any case, please can you fix the CI failures here?

[intelfx]

@richvdh I'm not sure what I should do wrt CI failures.

The style check wants me to reformat this:

        pypy_gc_time = CounterMetricFamily(
            "pypy_gc_time_seconds_total",
            "Total time spent in PyPy GC",
            labels=[],
        )

        <...>

        pypy_mem = GaugeMetricFamily(
            "pypy_memory_bytes",
            "Memory tracked by PyPy allocator",
            labels=["state", "class", "kind"],
        )

Into this:

        pypy_gc_time = CounterMetricFamily(
            "pypy_gc_time_seconds_total", "Total time spent in PyPy GC", labels=[]
        )

        <...>

        pypy_mem = GaugeMetricFamily(
            "pypy_memory_bytes",
            "Memory tracked by PyPy allocator",
            labels=["state", "class", "kind"],
        )

Losing style consistency in the process. As far as I can see, my style is permitted per the style guide. Do I have to strictly obey the linter's suggestion?

Other two build failures complain that PyPy-specific methods and fields don't exist, but I'm fairly sure none of the new code normally gets executed under CPython.

[richvdh]

As far as I can see, my style is permitted per the style guide. Do I have to strictly obey the linter's suggestion?

yes. The style guide mostly says "do what black tells you to do". Just run black on your changes.

you can add # type: ignore to stop mypy complaining about pypy-only interfaces, if you can't find a better solution.

[intelfx]

yes. The style guide mostly says "do what black tells you to do". Just run black on your changes.

Hmph. I must've been looking at an older version of the style guide. I don't like the inconsistency, but fine.

you can add # type: ignore to stop mypy complaining about pypy-only interfaces, if you can't find a better solution.

I can't, but then again, I didn't write a synapse in python :) Can you? I'd gladly do something more appropriate, if said something exists.

[erikjohnston]

Looks great! Thanks!