[pull] master from DataDog:master #2
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
* Baby steps for quote-unquote ultimate pipeline. Managed unit tests first * Use an install-dotnet pipeline to simplify dotnet installation. Test usage on the managed pools first before trying to use on arm64 unit test job * Add install-dotnet template to build step * Remove parameter from install-dotnet.yml * Fix indentation in ultimate-pipeline.yml * Add no build argument to the unit test runs * Build the windows profiler after running the managed build * Run the windows native unit tests on the same machine for x86 and x64 * Set buildConfiguration to Release. Remove --no-build argument on unit tests for now * Use one Datadog.Trace.proj per build step. For build_managed, that is BuildCsharp. For unit_tests_managed, that is BuildCsharpTests. * Add --no-build argument to dotnet pack. This means we'll build our binaries once for the BuildCsharp target and then pack the result of that * Update publish command in build_managed job * Add succeededOrFailed condition to native unit tests * Fix native test that only throws after my pipeline changes * Attempt to fix managed unit tests * Try and get Windows build running * Small native unit test fix * Add the alpine and linux profiler * Combine managed and native unit tests into same build stage * Publish linux packages from source build * Modify intermediate artifacts to more clearly identify some artifacts as build-only * Attempt to add benchmarks stage with "condition: true" so that, if it works, we can disable it until it's needed * Modify build and windows build to make it easier to build without passing custom properties everywhere * Modify package.sh to rely on src/bin/tracer-home * Fix benchmarks stage * Turn off benchmark runs because they seem to run just fine * Fix packaging build for linux containers * Rename default tracer home directory to windows-tracer-home * Add S3 upload * Build ARM64 linux profiler * Try to fix flaky unit test build... * Build the macOS profiler in the build phase * Actually do a git checkout in the upload step and add debugging info * Try to make the build-linux-tracer-home-arm64 as fully fleshed out as other linux tracer homes * Try to make the build-macos-tracer-home as fully fleshed out as other linux tracer homes * Try to add runner tool to pipeline * Fix tool build...hopefully * Fix all the dotnet CLI tasks to correctly pass the build configuration * Modify hard-coded release to instead use buildConfiguration variable * Copy over the windows tracer home, including the .NET assemblies, into the dotnet tool * Try windows copy again * Last time I swear * Get linux integration tests up and running in the unified pipeline * Add Alpine Linux now * Revise Linux and Alpine Linux integration tests to not checkout repo * Attempt linux arm64 integration test run * Add initial set of Windows integration-tests. Get rid of duplication later... * Do explicit restore to try and fix framework reproductions * Upload artifacts for integration-test directories to help diagnose the issue with them * Point the managed profiler directory to Windows Tracer Home. This should work! * Fix the TestAllPackageVersions: true to only apply to non-Windows integration test runs * Need to install the earlier .NET Core SDKs in integration test jobs The runtime doesn't install the asp.net crore runtime, whereas the ask does * Extract isMainBranch variable to global * Pass $(tracerHome) variable to MSI builder * Add Crank stage to pipelin * Only run pipeline on master for now * Disable runs on PRs * Undo changes to clr_helper_test.cpp * Restore the publishing of managed profiler assets in the 'docker-compose run Profiler' step to fix the main integration-tests build Co-authored-by: Zach Montoya <zach.montoya@datadoghq.com>
pull bot
pushed a commit
that referenced
this pull request
Oct 7, 2021
* Fix buffer overflow reported by Clang Address Sanitizer
using memcmp, we limited the count by taking the min between NameBuffer (1024)
and the size of the functionName (functionNameLength). But, if the SpecificMethodToInjectName
(and SpecificTypeToInjectName) is shorter than the max count, we will read memory beyond.
==21348==ERROR: AddressSanitizer: global-buffer-overflow on address 0x7fff40460ab2 at pc 0x7fff402c3682 bp 0x00c823dfc460 sp 0x00c823dfbbe8
READ of size 74 at 0x7fff40460ab2 thread T0
#0 0x7fff402c36b0 in MemcmpInterceptorCommon(void *, int (__cdecl *)(void const *, void const *, unsigned __int64), void const *, void const *, unsigned __int64) D:\a01\_work\2\s\src\vctools\crt\asan\llvm\compiler-rt\lib\sanitizer_common\sanitizer_common_interceptors.inc:851
#1 0x7fff402c5b71 in __asan_wrap_memcmp D:\a01\_work\2\s\src\vctools\crt\asan\llvm\compiler-rt\lib\sanitizer_common\sanitizer_common_interceptors.inc:882
#2 0x7fff4027e8d1 in shared::Loader::HandleJitCachedFunctionSearchStarted(unsigned __int64, int *) C:\Users\gregory.leocadie\repos\dd-trace-dotnet\shared\src\native-src\loader.cpp:670
#3 0x7fff401ea649 in CorProfilerCallback::JITCachedFunctionSearchStarted(unsigned __int64, int *) C:\Users\gregory.leocadie\repos\dd-continuous-profiler-dotnet\src\ProfilerEngine\Datadog.AutoInstrumentation.Profiler.Native.Shared\CorProfilerCallback.cpp:647
#4 0x7fffd09799ae (C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll+0x1805299ae)
#5 0x7fffd06620b0 (C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll+0x1802120b0)
#6 0x7fffd045f05b (C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll+0x18000f05b)
#7 0x7fffd0454854 (C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll+0x180004854)
#8 0x7fffcb70ac4c (C:\Windows\assembly\NativeImages_v4.0.30319_64\mscorlib\16234675ede351917e6b94c968a734df\mscorlib.ni.dll+0x6447857ac4c)
#9 0x7fffcb70a88d (C:\Windows\assembly\NativeImages_v4.0.30319_64\mscorlib\16234675ede351917e6b94c968a734df\mscorlib.ni.dll+0x6447857a88d)
#10 0x7fffcb713b70 (C:\Windows\assembly\NativeImages_v4.0.30319_64\mscorlib\16234675ede351917e6b94c968a734df\mscorlib.ni.dll+0x64478583b70)
#11 0x7fffd0456952 (C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll+0x180006952)
#12 0x7fffd0456857 (C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll+0x180006857)
#13 0x7fffd0457117 (C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll+0x180007117)
#14 0x7fffd0581bf9 (C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll+0x180131bf9)
#15 0x7fffd0590970 (C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll+0x180140970)
#16 0x7fffd0592176 (C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll+0x180142176)
#17 0x7fffd0591f63 (C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll+0x180141f63)
#18 0x7fffd0591cbc (C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll+0x180141cbc)
#19 0x7fffd0592ea3 (C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll+0x180142ea3)
#20 0x7fffd13f8c00 (C:\Windows\Microsoft.NET\Framework64\v4.0.30319\mscoreei.dll+0x180008c00)
#21 0x7fffd17dac41 (C:\Windows\SYSTEM32\MSCOREE.DLL+0x18000ac41)
#22 0x7fffe7947033 (C:\Windows\System32\KERNEL32.dll+0x180017033)
#23 0x7fffe92a2650 (C:\Windows\SYSTEM32\ntdll.dll+0x180052650)
pull bot
pushed a commit
that referenced
this pull request
Jan 19, 2024
…tion.Metadata) (DataDog#4782) * Initial commit for upload symbols * Add sym db api endpoint and remove blockingcollection * Tests WIP * Add tests and fixed issue after Oskar's review * Add service version and environment * Fix NRE and reset byte index after clearing array * PR comments and more fixes * Add discovery service sand environment variable to enable\disable symbols upload * Tests WIP * Refactor SymbolExtractor class and support of compiler generated methods and types * Change keys to snake_case * Change the way we are waiting for a discovery service notification about symbol database endpoint * Extract ShouldRetry logic and simplify get delay time * Fix build and update tests * Partial class test * Update local function test * revert SourceLinkInformationExtractor changes * PR comments * Change symbol batch size from MB to Bytes * Fix build * Fix brolen json and NRE * Fix json, tests and enum string representation * Fix missing method attributes and add class source file * Fix assembly name * Make sure to set Unknown in case there is not source info and set -1 for missing line info * enum names should be SnakeCaseNamingStrategy * update approvals tests * change unknown line numbers to 0 or int.Max * Set unknown for missing assembly path * Unknown method end line should be 0 * Fix missing pdb info in async methods and update approvals * Vendor System.Reflection.Metadata and dependencies * WIP - Replace dnlib with System.Reflection.Metadata * WIP - Replace dnlib with System.Reflection.Metadata #2 * Everything compiles * Now some of the tests work * More work on closures and update tests approvals * All tests are passed * Some strings to spans conversation * Wrap dnlib and reflection metadata pdb usage in DatadogMetadataReader Add more tests * Update vendored code * Revert "Update vendored code" This reverts commit 39efa76. * Update vendored code * Naming and logs * Fix Span and ReadOnlySpan vendored code * Add PrivateAssets="All" * Skip interfaces, methods without IL and more not relevant types * Handle properties and hoisted arguments * Fix pdb related info issues * Fox array out of range in argument symbols * Fix build error * Fixed arguments duplication * Fixed arguments duplication * PR comments * Change log level and text * Fix typo in configuration key * PR comments: configuration keys and error handling * Add test for async method with different kind of locals * Configuration keys * * Add missing closures * Handle hoisted arg as arg and not as local * Change async method locals line number to 0 * * Add LanguageSpecific to field symbols * Add local Scopes * Use memory and span instead of allocating list inside loop * Support local constants * More tests * * Add PdbExist to language specific of class * Fix source start and end column * Improve performance * Add some missing vendored code * Fix CI build * Fix symbol extractor tests to pass CI * Add System.IO.MemoryMappedFiles and System.Resources.ResourceManager to AssemblyReferencesHaveNotChanged test * Update Trimming.xml * Sanitize more symbol fields * Fix tests * Fix vendor version * Change vendored libs version * revert me * Revert "revert me" This reverts commit b98445b. * Try run symbol extractor tests only on windows * Update approvals * Enable nullable * Add SymbolDatabaseIncludes EV * Add `readonly`, `const` and `async` annotations * Symbol DB enablement WIP * Update assembly filter * Update missing-nullability-files * Add VendoredCodeTests * Fix CI tests * Skip AsyncVoid test in linux * Update depenabot versions * Fix compilation error * Update DefinitionPaths * Fix missing field attributes * Fix typo in upload symbols enablement * update nullability --------- Co-authored-by: Matan Green <matangrn@gmail.com>
pull bot
pushed a commit
that referenced
this pull request
Feb 20, 2024
* DSM checkpoint for SQS + pathway context propagation * fix possible nullrefs * missing nullability file * fix tostring on mockspan without delimiter, it means all spans are mixed when printed in a list * fix a couple issues with the ducktyping * missing nullability * read timestamp from message + DSM ITest * fix missing null file * fix indentation * log info about error before exiting in SQS sample program * use a collection to prevent parallel execution * improve diff printing in CI * probably need different snapshots depending on fwk * remove silent catch clause ? * wait for datastream points * make test more stable * add sent time in sync receive * cleanup * putting some changes aside for an other PR * address easy comments * comment to explain a bit what happens in the sample app * unify datastreams stats normalization between kafka and sqs * make sure we don't add requested attributes twice to receive requests * fix compilaiton for netframework * actually, keep the same logic as before when normalizing datastreams * pull object creation out of the loop * snapshot update after fix * bugfix * bugfix #2 where I was querying the wrong type * nit: WaitForDataStreamsPoints should stop if it has more points than expected * factorize SQS send/receive instrumentation code * replace extra calls to GetQueueName * support null DSM in Inject * fix static values that were recreated every time * expand string formating in stringbuilder Co-authored-by: Andrew Lock <andrew.lock@datadoghq.com> * nullable enable some files * put adapters in their own file + named methods to acess them * add a bunch of nullchecks * parse headers json only once in ctor --------- Co-authored-by: Andrew Lock <andrew.lock@datadoghq.com>
pull bot
pushed a commit
that referenced
this pull request
Jul 28, 2024
…ataDog#5808) ## Summary of changes Prevent deadlock betwen signal-based profilers (walltime/manual cpu profilers) and non-signal based profilers (exception, contention....) ## Reason for change When an exception occurs, the thread can be interrupted by a signal-based profiler (walltime/manual cpu). It can be interrupted while holding the lock used to update the `dl-iterate-phdr` cache. ``` Thread 18 (LWP 995): #0 __syscall_cp_c (nr=202, u=140244538814536, v=128, w=-1, x=0, y=0, z=0) at ./arch/x86_64/syscall_arch.h:61 #1 0x00007f8dba343ccd in __futex4_cp (to=0x0, val=-1, op=128, addr=0x7f8d39eaf048 <LibrariesInfoCache::Get()::Instance>) at src/thread/__timedwait.c:24 #2 __timedwait_cp (addr=addr@entry=0x7f8d39eaf048 <LibrariesInfoCache::Get()::Instance>, val=val@entry=-1, clk=clk@entry=0, at=at@entry=0x0, priv=priv@entry=128) at src/thread/__timedwait.c:52 #3 0x00007f8dba343d74 in __timedwait (addr=addr@entry=0x7f8d39eaf048 <LibrariesInfoCache::Get()::Instance>, val=-1, clk=clk@entry=0, at=at@entry=0x0, priv=128) at src/thread/__timedwait.c:68 #4 0x00007f8dba3463e6 in __pthread_rwlock_timedrdlock (at=<optimized out>, rw=<optimized out>) at src/thread/pthread_rwlock_timedrdlock.c:18 #5 __pthread_rwlock_timedrdlock (rw=0x7f8d39eaf048 <LibrariesInfoCache::Get()::Instance>, at=0x0) at src/thread/pthread_rwlock_timedrdlock.c:3 #6 0x00007f8d398f3ca8 in std::__glibcxx_rwlock_rdlock (__rwlock=0x7f8d39eaf048 <LibrariesInfoCache::Get()::Instance>) at /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../include/c++/10.3.1/shared_mutex:73 #7 std::__shared_mutex_pthread::lock_shared (this=0x7f8d39eaf048 <LibrariesInfoCache::Get()::Instance>) at /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../include/c++/10.3.1/shared_mutex:224 #8 std::shared_mutex::lock_shared (this=0x7f8d39eaf048 <LibrariesInfoCache::Get()::Instance>) at /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../include/c++/10.3.1/shared_mutex:421 #9 std::shared_lock<std::shared_mutex>::shared_lock (this=0x7f4ca05a2ac0, __m=...) at /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../include/c++/10.3.1/shared_mutex:722 #10 LibrariesInfoCache::DlIteratePhdrImpl (this=0x7f8d39eaf048 <LibrariesInfoCache::Get()::Instance>, callback=0x7f8d3997d900 <_Ux86_64_dwarf_callback>, data=0x7f4ca05a2b20) at /project/profiler/src/ProfilerEngine/Datadog.Profiler.Native.Linux/LibrariesInfoCache.cpp:104 #11 0x00007f8d3997e4ee in _Ux86_64_dwarf_find_proc_info (as=0x7f8d39eb2a00 <local_addr_space>, ip=140246691112115, pi=0x7f4ca05a3170, need_unwind_info=1, arg=0x7f4ca05a3411) at /project/obj/libunwind-prefix/src/libunwind/src/dwarf/Gfind_proc_info-lsb.c:807 #12 0x00007f8d3997e690 in fetch_proc_info (c=0x7f4ca05a3018, ip=140246691112115) at /project/obj/libunwind-prefix/src/libunwind/src/dwarf/Gparser.c:473 #13 0x00007f8d3998113d in find_reg_state (sr=0x7f4ca05a2dc0, c=0x7f4ca05a3018) at /project/obj/libunwind-prefix/src/libunwind/src/dwarf/Gparser.c:1024 #14 _Ux86_64_dwarf_step (c=c@entry=0x7f4ca05a3018) at /project/obj/libunwind-prefix/src/libunwind/src/dwarf/Gparser.c:1069 #15 0x00007f8d3997d13a in _Ux86_64_step (cursor=0x7f4ca05a3018) at /project/obj/libunwind-prefix/src/libunwind/src/x86_64/Gstep.c:75 #16 0x00007f8d398f55c8 in LinuxStackFramesCollector::CollectStackManually (this=this@entry=0x7f8d392dc6d0, ctx=ctx@entry=0x7f4ca05a3880) at /project/profiler/src/ProfilerEngine/Datadog.Profiler.Native.Linux/LinuxStackFramesCollector.cpp:288 #17 0x00007f8d398f53dc in LinuxStackFramesCollector::CollectCallStackCurrentThread (this=this@entry=0x7f8d392dc6d0, ctx=ctx@entry=0x7f4ca05a3880) at /project/profiler/src/ProfilerEngine/Datadog.Profiler.Native.Linux/LinuxStackFramesCollector.cpp:227 #18 0x00007f8d398f4672 in LinuxStackFramesCollector::CollectStackSampleSignalHandler (signal=<optimized out>, info=<optimized out>, context=0x7f4ca05a3880) at /project/profiler/src/ProfilerEngine/Datadog.Profiler.Native.Linux/LinuxStackFramesCollector.cpp:373 #19 0x00007f8d398fb871 in ProfilerSignalManager::CallCustomHandler (this=0x7f8d39eaf928 <ProfilerSignalManager::Get(int)::signalManagers+1944>, signal=10, info=0x7f4ca05a39b0, context=0x7f4ca05a3880) at /project/profiler/src/ProfilerEngine/Datadog.Profiler.Native.Linux/ProfilerSignalManager.cpp:197 #20 ProfilerSignalManager::SignalHandler (signal=10, info=0x7f4ca05a39b0, context=0x7f4ca05a3880) at /project/profiler/src/ProfilerEngine/Datadog.Profiler.Native.Linux/ProfilerSignalManager.cpp:188 #21 <signal handler called> #22 __pthread_rwlock_unlock (rw=0x7f8d39eaf048 <LibrariesInfoCache::Get()::Instance>) at src/thread/pthread_rwlock_unlock.c:5 #23 0x00007f8d398f3bf9 in std::__glibcxx_rwlock_unlock (__rwlock=0x7f8d39eaf048 <LibrariesInfoCache::Get()::Instance>) at /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../include/c++/10.3.1/shared_mutex:77 #24 std::__shared_mutex_pthread::unlock (this=0x7f8d39eaf048 <LibrariesInfoCache::Get()::Instance>) at /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../include/c++/10.3.1/shared_mutex:208 #25 std::shared_mutex::unlock (this=0x7f8d39eaf048 <LibrariesInfoCache::Get()::Instance>) at /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../include/c++/10.3.1/shared_mutex:417 #26 std::unique_lock<std::shared_mutex>::unlock (this=0x7f4ca05a3e20) at /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../include/c++/10.3.1/bits/unique_lock.h:194 #27 std::unique_lock<std::shared_mutex>::~unique_lock (this=0x7f4ca05a3e20) at /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../include/c++/10.3.1/bits/unique_lock.h:103 #28 LibrariesInfoCache::UpdateCache (this=0x7f8d39eaf048 <LibrariesInfoCache::Get()::Instance>) at /project/profiler/src/ProfilerEngine/Datadog.Profiler.Native.Linux/LibrariesInfoCache.cpp:88 #29 0x00007f8d398f4e59 in LinuxStackFramesCollector::CollectStackSampleImplementation (this=0x7f8d3b91bc90, pThreadInfo=0x7f4ca06b9900, pHR=0x7f8d3a63c510, selfCollect=true) at /p--Type <RET> for more, q to quit, c to continue without paging-- roject/profiler/src/ProfilerEngine/Datadog.Profiler.Native.Linux/LinuxStackFramesCollector.cpp:100 #30 0x00007f8d399637ba in StackFramesCollectorBase::CollectStackSample (this=0x7f8d3b91bc90, pThreadInfo=0x7f4ca06b9900, pHR=0x7f4ca05a3fdc) at /project/profiler/src/ProfilerEngine/Datadog.Profiler.Native/StackFramesCollectorBase.cpp:185 #31 0x00007f8d3992acb9 in ExceptionsProvider::OnExceptionThrown (this=0x7f8d392a7160, thrownObjectId=139969739182080) at /project/profiler/src/ProfilerEngine/Datadog.Profiler.Native/ExceptionsProvider.cpp:149 #32 0x00007f8d39917045 in CorProfilerCallback::ExceptionThrown (this=0x7f8d392c0d20, thrownObjectId=139969739182080) at /project/profiler/src/ProfilerEngine/Datadog.Profiler.Native/CorProfilerCallback.cpp:1734 ``` ## Implementation details - move the call which updates the cache after acquiring the thread lock - call Update before sending signal ## Test coverage ## Other details <!-- Fixes #{issue} --> <!--⚠️ Note: where possible, please obtain 2 approvals prior to merging. Unless CODEOWNERS specifies otherwise, for external teams it is typically best to have one review from a team member, and one review from apm-dotnet. Trivial changes do not require 2 reviews. -->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )