Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release v3.5.0 #170

Merged
merged 29 commits into from
Feb 21, 2023
Merged

Release v3.5.0 #170

merged 29 commits into from
Feb 21, 2023

Conversation

P403n1x87
Copy link
Owner

See the changelog for details.

P403n1x87 and others added 29 commits October 29, 2022 12:05
When trying to profile a child process that is not recognised as
Python process, Austin tries to resolve the interpreter state too
many times, causing poor overall performance. This change ensures
that if we fail to attach to a child Process we give up quickly to
allow for samples to be collected efficiently for the processes
that are correctly identified as Python processes.
During testing, some minor multiprocess issues were identified and
resolved.
This change introduces the code and frame header files with content
taken out of the stack header for better source code modularity.
The architectures build and release workflow share the same build
script. This has been factored out into a script to reduce
duplication and improve maintenance.
The libbfd project is affected by a known issue that is marked as
WONTFIX. However, we can easily implement a workaround to allow
austinp to be built on Linux distros that don't ship a patched
version of the libbfd sources where the issue is removed.

Fixes #152.
The alternative output format was a hack to allow for a quick
visualisation with line breakdown using existing flame graph
visualisation tools. If required, the same effect can be obtained
with ad-hoc tools and therefore it is not necessary to offer this
option from Austin. The benefit is a slight simplification of the
CLI interface.
We use a hash lookup structure instead of plain lookup arrays in
multiprocessing support to reduce the overall memory requirements
for Austin when sampling child processes.
This change allows Austin to infer the PID of a process running in
a different PID namespace (e.g. containers) so that the pthread TID
field offset can be computed reliably in this case. Previously, the
inference would fail and cause an infinite loop.
GCC provides the cleanup attribute which can be used to run a cleanup
function when an auto function local variables goes out of scope. We
make use of this to refactor and simplify the code in places where
resources are used locally but not returned to the caller.
A previous change has introduced a helper to open PID-based files from
procfs. This change makes use of the helper in other places too for code
reuse.
This change mimics the improvements done for Linux for detecting the
correct binary files to analyse for symbols and memory maps.
This change aligns the Python detection algorithm to the
implementation for the other supported platforms.
This change adds support for the extra column-level information
exported by the location table of Python 3.11 code objects. The
finer-graied location data is made available though the MOJO
binary format only.
When the -Werror option was added, it caused clang to error on the use
of the ignored -s option. This resulted in build failures on macOS for
the conda-forge package (see conda-forge/austin-feedstock#21). This
change adds the -Wno-unused-command-line-argument option to prevent the
compiler from treating unused command line options as errors while still
treating all other warnings as errors.
@P403n1x87 P403n1x87 added the release New release label Feb 21, 2023
@P403n1x87 P403n1x87 added this to the Austin 3.5 milestone Feb 21, 2023
@codecov
Copy link

codecov bot commented Feb 21, 2023

Codecov Report

Base: 71.63% // Head: 70.29% // Decreases project coverage by -1.35% ⚠️

Coverage data is based on head (2c9ad18) compared to base (4139481).
Patch coverage: 62.11% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #170      +/-   ##
==========================================
- Coverage   71.63%   70.29%   -1.35%     
==========================================
  Files          23       25       +2     
  Lines        2454     2474      +20     
  Branches      716      730      +14     
==========================================
- Hits         1758     1739      -19     
- Misses        387      410      +23     
- Partials      309      325      +16     
Impacted Files Coverage Δ
src/argparse.c 65.06% <ø> (+3.06%) ⬆️
src/linux/addr2line.h 53.12% <0.00%> (ø)
src/mem.h 85.71% <ø> (ø)
src/mojo.h 100.00% <ø> (ø)
src/py_proc_list.h 0.00% <0.00%> (ø)
src/stack.h 78.12% <ø> (-9.54%) ⬇️
src/py_proc_list.c 74.79% <43.75%> (-3.07%) ⬇️
src/linux/common.h 60.00% <50.00%> (-40.00%) ⬇️
src/resources.h 58.33% <58.33%> (ø)
src/cache.c 81.57% <63.41%> (-3.99%) ⬇️
... and 15 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@github-actions
Copy link

Running Austin benchmarks with Python 3.10.10

Wall time [sampling interval: 1]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
===================================================================
         Sample Rate   Saturation    Error Rate     Sampling Speed 
-------------------------------------------------------------------
 3.2.0   24600 ± 900     1 ± 0     3.3e-05 ± 1e-06      66 ± 3     
 3.3.0  116000 ± 8000    1 ± 0     0.0001 ± 0.0001      13 ± 1     
 3.4.1  118000 ± 6000    1 ± 0     2.1e-05 ± 7e-06    13.4 ± 0.7   
  dev   122000 ± 4000    1 ± 0      2e-05 ± 1e-05     12.8 ± 0.6   
===================================================================

Wall time [sampling interval: 10]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
=========================================================================
         Sample Rate      Saturation       Error Rate     Sampling Speed 
-------------------------------------------------------------------------
 3.2.0   23000 ± 1000   0.99845 ± 8e-05   2e-05 ± 2e-05       69 ± 4     
 3.3.0  110000 ± 10000   0.544 ± 0.001    5e-06 ± 3e-06       14 ± 1     
 3.4.1  118000 ± 6000    0.545 ± 0.001   1.8e-05 ± 9e-06    13.3 ± 0.7   
  dev   121000 ± 3000    0.545 ± 0.001    2e-05 ± 8e-06       13 ± 0     
=========================================================================

Wall time [sampling interval: 100]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
====================================================================
        Sample Rate    Saturation      Error Rate    Sampling Speed 
--------------------------------------------------------------------
 3.2.0  8200 ± 300    0.023 ± 0.006   1e-05 ± 3e-05      89 ± 3     
 3.3.0  10700 ± 600  0.0008 ± 0.0002  1e-05 ± 3e-05    14.3 ± 0.7   
 3.4.1  10300 ± 400  0.0006 ± 0.0002  6e-05 ± 7e-05    14.5 ± 0.5   
  dev   10300 ± 300  0.0009 ± 0.0006  5e-05 ± 5e-05    14.5 ± 0.5   
====================================================================

Wall time [sampling interval: 1000]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
======================================================================
        Sample Rate    Saturation       Error Rate     Sampling Speed 
----------------------------------------------------------------------
 3.2.0   1870 ± 20   0.0003 ± 0.0006       0 ± 0           71 ± 2     
 3.3.0   1880 ± 5    0.0002 ± 0.0004       0 ± 0         17.4 ± 0.7   
 3.4.1   1870 ± 10        0 ± 0       0.0001 ± 0.0002    17.4 ± 0.5   
  dev    1869 ± 5         0 ± 0       0.0001 ± 0.0002    17.5 ± 0.5   
======================================================================

CPU time [sampling interval: 1]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
================================================================
        Sample Rate   Saturation   Error Rate    Sampling Speed 
----------------------------------------------------------------
 3.2.0  19000 ± 700     1 ± 0         0 ± 0          66 ± 1     
 3.3.0  45000 ± 7000    1 ± 0         0 ± 0          24 ± 2     
 3.4.1  51000 ± 4000    1 ± 0     3e-05 ± 2e-05    22.7 ± 0.8   
  dev   50000 ± 6000    1 ± 0     3e-05 ± 2e-05      23 ± 2     
================================================================

CPU time [sampling interval: 10]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
=====================================================================
        Sample Rate     Saturation      Error Rate    Sampling Speed 
---------------------------------------------------------------------
 3.2.0  18900 ± 700   0.9886 ± 0.0006      0 ± 0          66 ± 1     
 3.3.0  46000 ± 6000  0.9956 ± 0.0006      0 ± 0          24 ± 2     
 3.4.1  50000 ± 4000  0.9962 ± 0.0005  2e-05 ± 2e-05      23 ± 1     
  dev   52000 ± 5000  0.9956 ± 0.0003  3e-05 ± 1e-05      22 ± 1     
=====================================================================

CPU time [sampling interval: 100]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
======================================================================
        Sample Rate    Saturation       Error Rate     Sampling Speed 
----------------------------------------------------------------------
 3.2.0  4800 ± 900     0.01 ± 0.01         0 ± 0           65 ± 3     
 3.3.0  5400 ± 300   0.0012 ± 0.0004   1e-05 ± 4e-05     22.2 ± 0.9   
 3.4.1  5400 ± 100   0.0011 ± 0.0003  0.0001 ± 0.0001    22.3 ± 0.7   
  dev   5400 ± 100   0.0009 ± 0.0004  0.0001 ± 0.0002    22.0 ± 0.8   
======================================================================

CPU time [sampling interval: 1000]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
======================================================================
        Sample Rate    Saturation       Error Rate     Sampling Speed 
----------------------------------------------------------------------
 3.2.0    948 ± 2    0.0006 ± 0.0009       0 ± 0           60 ± 5     
 3.3.0    954 ± 5         0 ± 0            0 ± 0         24.9 ± 0.6   
 3.4.1    952 ± 2         0 ± 0       0.0002 ± 0.0004    25.5 ± 0.5   
  dev     952 ± 4         0 ± 0       0.0002 ± 0.0005    25.9 ± 1.0   
======================================================================

RSA keygen [sampling interval: 1]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
===================================================================
         Sample Rate   Saturation    Error Rate     Sampling Speed 
-------------------------------------------------------------------
 3.2.0  20000 ± 20000    1 ± 0          0 ± 0          80 ± 30     
 3.3.0  43000 ± 1000     1 ± 0      1e-05 ± 3e-05     22.9 ± 0.9   
 3.4.1  41000 ± 1000     1 ± 0     0.0002 ± 0.0001    23.9 ± 0.6   
  dev   42000 ± 1000     1 ± 0     0.0002 ± 0.0001    23.4 ± 0.7   
===================================================================

RSA keygen [sampling interval: 10]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
=====================================================================
        Sample Rate    Saturation      Error Rate     Sampling Speed 
---------------------------------------------------------------------
 3.2.0  13000 ± 8000    0.9 ± 0.2         0 ± 0          90 ± 20     
 3.3.0  42000 ± 1000  0.982 ± 0.01    1e-05 ± 3e-05     23.3 ± 0.7   
 3.4.1  41000 ± 1000  0.988 ± 0.009  0.0002 ± 0.0001    23.6 ± 0.5   
  dev   41000 ± 1000   0.98 ± 0.01   0.0002 ± 0.0001    23.8 ± 0.6   
=====================================================================

RSA keygen [sampling interval: 100]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
====================================================================
        Sample Rate   Saturation      Error Rate     Sampling Speed 
--------------------------------------------------------------------
 3.2.0  9300 ± 400     0.6 ± 0.2         0 ± 0          104 ± 4     
 3.3.0  6200 ± 100   0.002 ± 0.001       0 ± 0           28 ± 2     
 3.4.1  6200 ± 100   0.004 ± 0.003  0.0009 ± 0.0008      28 ± 2     
  dev   6200 ± 100   0.004 ± 0.003   0.001 ± 0.001       28 ± 2     
====================================================================

RSA keygen [sampling interval: 1000]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
======================================================================
        Sample Rate    Saturation       Error Rate     Sampling Speed 
----------------------------------------------------------------------
 3.2.0    935 ± 9    0.0002 ± 0.0006       0 ± 0          108 ± 4     
 3.3.0    940 ± 4    0.0001 ± 0.0003       0 ± 0           31 ± 2     
 3.4.1    936 ± 7      0.0 ± 0.001     0.002 ± 0.002       37 ± 4     
  dev     942 ± 3         0 ± 0       0.0005 ± 0.0009      32 ± 1     
======================================================================

Full metrics [sampling interval: 1]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
================================================================
        Sample Rate   Saturation   Error Rate    Sampling Speed 
----------------------------------------------------------------
 3.2.0  18700 ± 1000    1 ± 0      0.0 ± 1e-05       86 ± 5     
 3.3.0  48000 ± 3000    1 ± 0         0 ± 0          32 ± 2     
 3.4.1  52000 ± 2000    1 ± 0     5e-05 ± 3e-05      30 ± 1     
  dev   50000 ± 3000    1 ± 0     3e-05 ± 2e-05      30 ± 2     
================================================================

Full metrics [sampling interval: 10]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
==================================================================
        Sample Rate   Saturation    Error Rate     Sampling Speed 
------------------------------------------------------------------
 3.2.0  18800 ± 900     1 ± 0      1e-05 ± 2e-05       85 ± 5     
 3.3.0  49000 ± 4000    1 ± 0     0.0001 ± 0.0002      32 ± 3     
 3.4.1  51000 ± 3000    1 ± 0      5e-05 ± 2e-05       30 ± 2     
  dev   52600 ± 800     1 ± 0      4e-05 ± 1e-05     29.2 ± 0.4   
==================================================================

Full metrics [sampling interval: 100]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
======================================================================
        Sample Rate    Saturation       Error Rate     Sampling Speed 
----------------------------------------------------------------------
 3.2.0  17600 ± 900    0.44 ± 0.03      0.0 ± 1e-05        87 ± 4     
 3.3.0  10000 ± 200  0.0015 ± 0.0004       0 ± 0           32 ± 1     
 3.4.1  9700 ± 300   0.0018 ± 0.0004   9e-05 ± 7e-05     32.8 ± 0.6   
  dev   9700 ± 300   0.0016 ± 0.0003  0.00013 ± 9e-05    32.7 ± 0.8   
======================================================================

Full metrics [sampling interval: 1000]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
======================================================================
        Sample Rate    Saturation       Error Rate     Sampling Speed 
----------------------------------------------------------------------
 3.2.0   1860 ± 20   0.0004 ± 0.0004       0 ± 0           90 ± 2     
 3.3.0   1860 ± 30   0.0001 ± 0.0003       0 ± 0         36.5 ± 1.0   
 3.4.1   1860 ± 20        0 ± 0       0.0002 ± 0.0003    36.1 ± 0.3   
  dev    1850 ± 30   0.0001 ± 0.0003   0.0 ± 0.0001      36.5 ± 0.7   
======================================================================

Multiprocess wall time [sampling interval: 1]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
=================================================================
        Sample Rate  Saturation    Error Rate     Sampling Speed 
-----------------------------------------------------------------
 3.2.0   510 ± 30      1 ± 0     0.00014 ± 5e-05    1310 ± 60    
 3.3.0   1510 ± 50     1 ± 0      3e-05 ± 2e-05      450 ± 50    
 3.4.1  3100 ± 200     1 ± 0     0.00013 ± 4e-05     300 ± 10    
  dev   3500 ± 100     1 ± 0     0.00014 ± 4e-05     280 ± 10    
=================================================================

Multiprocess wall time [sampling interval: 10]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
=================================================================
        Sample Rate  Saturation    Error Rate     Sampling Speed 
-----------------------------------------------------------------
 3.2.0   570 ± 50      1 ± 0     0.00012 ± 5e-05    1200 ± 100   
 3.3.0  1500 ± 100     1 ± 0      5e-05 ± 3e-05      430 ± 90    
 3.4.1  3000 ± 200     1 ± 0     0.00018 ± 6e-05     310 ± 20    
  dev   3700 ± 200     1 ± 0     0.00013 ± 5e-05     260 ± 10    
=================================================================

Multiprocess wall time [sampling interval: 100]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
====================================================================
        Sample Rate   Saturation      Error Rate     Sampling Speed 
--------------------------------------------------------------------
 3.2.0   550 ± 40    0.996 ± 0.006  0.00013 ± 5e-05    1180 ± 80    
 3.3.0  1500 ± 100    0.05 ± 0.02    3e-05 ± 4e-05      440 ± 70    
 3.4.1  3000 ± 200    0.05 ± 0.01    9e-05 ± 3e-05      310 ± 20    
  dev   3400 ± 200    0.04 ± 0.02    7e-05 ± 3e-05      290 ± 20    
====================================================================

Multiprocess wall time [sampling interval: 1000]
> Running with Austin 3.2.0 ...    
> Running with Austin 3.3.0 ...    
> Running with Austin 3.4.1 ...    
> Running with Austin dev ...    
=======================================================================
        Sample Rate    Saturation        Error Rate     Sampling Speed 
-----------------------------------------------------------------------
 3.2.0   540 ± 30      0.11 ± 0.03    0.000107 ± 7e-06    1200 ± 100   
 3.3.0   1390 ± 50     0.03 ± 0.01     1e-05 ± 2e-05       220 ± 30    
 3.4.1  2800 ± 100    0.012 ± 0.004    1e-05 ± 1e-05       100 ± 20    
  dev   3200 ± 200   0.0054 ± 0.0008   1e-05 ± 1e-05        56 ± 3     
=======================================================================

@P403n1x87 P403n1x87 merged commit b48d980 into master Feb 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release New release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant