Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Process tries to allocate 133TB of memory #852

Open
yanschor opened this issue Dec 6, 2024 · 4 comments
Open

Process tries to allocate 133TB of memory #852

yanschor opened this issue Dec 6, 2024 · 4 comments

Comments

@yanschor
Copy link

yanschor commented Dec 6, 2024

While processing the attached SVG file the process fails with the following error message:

memory allocation of 133671936000000 bytes failed

We know that limiting the memory allocation is already the topic of issue #815. But considering the scale of the allocated memory we thought that it might be interesting anyway.

Our team discovered the issue during a current research project focusing on the schema-based generation of test-data for different XML specification formats.

Steps to Reproduce

  1. Execute resvg <filepath> test.png

Environment:

• OS: macOS 15.1.1

• reSVG version: 0.44.0

SVG File:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><tg:svg xmlns:tg="http://www.w3.org/2000/svg" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xml="http://www.w3.org/XML/1998/namespace" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  id = "id0"     xml:lang="" xml:space ="preserve" externalResourcesRequired = "true"   color ="navy" color-interpolation ="auto"   flood-color="" flood-opacity= "
"  fill-opacity= "&#x10CfFe;&#xED9f;" fill-rule="nonzero" stroke = "" stroke-dasharray = "" stroke-dashoffset = "" stroke-linecap = "butt"   stroke-opacity = "&#xC84b6;
&lt;"  color-interpolation-filters= "sRGB" font-family = ""   font-stretch="extra-expanded" font-style = "italic"  font-weight="700"  stop-opacity ="	"  clip-rule = "inherit"  display="table-caption"  image-rendering = "optimizeQuality" mask ="("  pointer-events = "visibleFill"     lighting-color ="" marker-start= "none" marker-mid = "inherit" marker-end =":"   direction="ltr"   glyph-orientation-vertical="inherit" letter-spacing = "auto" text-anchor ="end"  unicode-bidi = "embed"     viewBox= "a"  zoomAndPan ="disable" onfocusin = "" onfocusout="&#242;&#x66Fb8;" onactivate ="" onclick= "" onmousedown= "" onmouseup ="
" onmouseover ="&#246;" onmousemove ="" onmouseout= "

" onload ="	" onunload=""    onscroll=""  x = "363in" y="-7in" width="8210" height ="+424E05in"   ><tg:polyline   id ="id1" xml:base="-"   systemLanguage = "WI-ZwK-UnAZ-9c58-xT9dR-x-C22" xml:lang =""       color-rendering ="optimizeQuality"    stroke ="
"  stroke-dashoffset = ""  stroke-linejoin= "round" stroke-miterlimit = "	" stroke-opacity = "
" stroke-width=""   cursor="n-resize" display="none"  image-rendering ="optimizeQuality"   pointer-events ="visible" shape-rendering= "geometricPrecision" text-rendering= "inherit"   marker-mid="w+"  transform= "&#xcCc7b;" onfocusin= "" onfocusout ="" onactivate ="
"  onmousedown = "" onmouseup= "
"   onmouseout = "&#xF0c0;" onload= "	" points = ""> <desc  id= "id2" xml:base ="&quot;" xml:lang= "hBaVhOA-se-Q-0GB7-H-T1Nq06-B-Mj-x-86464Yk4-93qKF1-Uo238-3n006S-b2"   style="&#xfA2e;	&#xe06d;
" content="structured text"></desc> </tg:polyline></tg:svg>
@yanschor
Copy link
Author

yanschor commented Dec 6, 2024

We also found a similar issue where the process is killed by the OS after starting to allocate tons of memory.

SVG:

<?xml version="1.0" encoding="ISO-8859-1" standalone="yes"?><tg:svg xmlns:tg="http://www.w3.org/2000/svg" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xml="http://www.w3.org/XML/1998/namespace" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  id = "id0" xml:base ="&amp;"    xml:lang ="" xml:space= "preserve" externalResourcesRequired = "true" class ="" style = "" color="WcyqYi"  color-rendering = "optimizeSpeed" enable-background="accumulate"   fill= "" fill-opacity= ""    stroke-dashoffset= ""  stroke-linejoin = "miter" stroke-miterlimit=""  stroke-width ="&#xEa14;" color-interpolation-filters ="sRGB"    font-stretch="ultra-condensed" font-style = "italic"  font-weight="bolder"   clip-path="GC" clip-rule ="inherit"   filter="x@[QI"       visibility ="hidden"  lighting-color ="&#xb995c;&#xe2DE;
"  marker-mid="inherit" marker-end= "W"  baseline-shift = "5.e5"  dominant-baseline= "auto"  glyph-orientation-vertical="229"   text-decoration= "none" unicode-bidi="normal"   clip= ""   preserveAspectRatio= "xMinYMin  "  onfocusin= "
" onfocusout =""  onclick= "&#163;" onmousedown ="f" onmouseup ="" onmouseover= ""  onmouseout ="" onload = "&#248;
&#142;W"  onabort =""  onresize= "
&#x10d015;
k"   x="+1"  width="+93in"   contentStyleType="message" > <tg:circle   id = "id1" xml:base= "q9a\m"          color-interpolation ="auto"      stroke-dasharray= "" stroke-dashoffset = "&#x10836e;"  stroke-linejoin = "round"    clip-path ="inherit" clip-rule="nonzero"  display ="table-cell"   mask = "inherit"  pointer-events = "visibleStroke"   visibility= "hidden" transform = "	" onfocusin="	"  onactivate ="" onclick ="
" onmousedown = "@
"   onmousemove="		"  onload = "" cx= "4872em" cy= "+2e+6" r ="9mm"> <metadata  id ="id2" xml:base= "s">  </metadata>  <tg:animateColor   id = "id3" xml:base="$%l(" requiredFeatures = ""     onend="	"     xlink:title= "" xlink:show= "replace"   attributeName = "
&#204;&#xf8FB;"  begin= "&#xCF7A0;.	" dur =""       fill ="remove" calcMode ="discrete"   keySplines = "&#194;" from ="

"   additive="sum" accumulate ="sum"><desc  id = "id5"  xml:lang = "i-bnn"   style ="	&#xEdaa;&#xEdc5;" >  </desc></tg:animateColor> <tg:animateMotion   id = "id4" xml:base = "IN]&apos;T" requiredFeatures= ""   externalResourcesRequired="1"  onend = "&#x101e6f;" onrepeat ="	&apos;&#xfa7a;" xlink:type="simple" xlink:role = "&#251;"  xlink:title =""  xlink:actuate = "none" xlink:href =":@\F" begin= "" dur = ""   max = "" restart ="always" repeatCount ="&#215;"  fill ="remove" calcMode= "paced" values="
"  keySplines="&#x10cE1d;	"  to ="|
"   accumulate ="sum" path="" keyPoints= ""  ><metadata  id="id6" xml:base= "Nr@"> 
: &#xf0c9;</metadata>   </tg:animateMotion>  </tg:circle></tg:svg>

@waywardmonkeys
Copy link
Collaborator

The first of these is the same as #853, but with a different failure mode but the same solution.

The stack trace for it is:

  * frame #0: 0x000000019d762600 libsystem_kernel.dylib`__pthread_kill + 8
    frame #1: 0x000000019d79af70 libsystem_pthread.dylib`pthread_kill + 288
    frame #2: 0x000000019d6a7908 libsystem_c.dylib`abort + 128
    frame #3: 0x0000000100476c70 resvg`std::sys::pal::unix::abort_internal::h857d2ad66020786f at mod.rs:373:14 [opt]
    frame #4: 0x000000010049bbd8 resvg`std::process::abort::hefc10ade1c26ea0e at process.rs:2374:5 [opt]
    frame #5: 0x00000001004779b4 resvg`std::alloc::rust_oom::ha54d832161426ec7 at alloc.rs:376:5 [opt]
    frame #6: 0x00000001004779cc resvg`__rg_oom at alloc.rs:371:1 [opt]
    frame #7: 0x000000010049c580 resvg`alloc::alloc::handle_alloc_error::h0a3050efcb156a76 [inlined] alloc::alloc::handle_alloc_error::rt_error::h7cf12a9711aa7428 at alloc.rs:403:13 [opt]
    frame #8: 0x000000010049c574 resvg`alloc::alloc::handle_alloc_error::h0a3050efcb156a76 at alloc.rs:409:9 [opt]
    frame #9: 0x000000010049c568 resvg`alloc::raw_vec::handle_error::hbe5ce02d989aad7d at raw_vec.rs:776:38 [opt]
    frame #10: 0x000000010018fa8c resvg`_$LT$u8$u20$as$u20$alloc..vec..spec_from_elem..SpecFromElem$GT$::from_elem::h319437aabc5370ff at raw_vec.rs:438:25
    frame #11: 0x000000010018fa78 resvg`_$LT$u8$u20$as$u20$alloc..vec..spec_from_elem..SpecFromElem$GT$::from_elem::h319437aabc5370ff [inlined] alloc::raw_vec::RawVec$LT$T$C$A$GT$::with_capacity_zeroed_in::h05559af22ea2d781(capacity=133671936000000, alloc=Global @ 0x000000016fdfcc97) at raw_vec.rs:210:20
    frame #12: 0x000000010018fa78 resvg`_$LT$u8$u20$as$u20$alloc..vec..spec_from_elem..SpecFromElem$GT$::from_elem::h319437aabc5370ff(elem='\0', n=133671936000000, alloc=Global @ 0x000000016fdfcc97) at spec_from_elem.rs:51:31
    frame #13: 0x0000000100199590 resvg`alloc::vec::from_elem::h66e9710abb14d595(elem='\0', n=133671936000000) at mod.rs:3155:5
    frame #14: 0x00000001001496d8 resvg`tiny_skia::pixmap::Pixmap::new::h146ab0adf8252f4a(width=8210, height=4070400000) at pixmap.rs:51:19
    frame #15: 0x000000010000d794 resvg`resvg::render_svg::hd32c288490d7a725(args=0x000000016fdfd6b0, tree=0x000000016fdfe2e0) at main.rs:734:26
    frame #16: 0x0000000100007f48 resvg`resvg::process::hebc78e1620b3f18b at main.rs:103:15
    ```

So you were creating a pixmap with a width of 8210 and height of 4070400000.

@waywardmonkeys
Copy link
Collaborator

The second one is more interesting and I'd have to look more:

* thread #1, name = 'main', queue = 'com.apple.main-thread', stop reason = signal SIGKILL
  * frame #0: 0x000000019d7cfe08 libsystem_platform.dylib`_platform_memmove + 88
    frame #1: 0x0000000100271ec8 resvg`_$LT$T$u20$as$u20$alloc..slice..hack..ConvertVec$GT$::to_vec::h983a9aca5d2b815e [inlined] core::intrinsics::copy_nonoverlapping::h619d1df2b6a5f8a0(src="", dst="", count=467991955728) at intrinsics.rs:3375:14
    frame #2: 0x0000000100271e90 resvg`_$LT$T$u20$as$u20$alloc..slice..hack..ConvertVec$GT$::to_vec::h983a9aca5d2b815e [inlined] core::ptr::const_ptr::_$LT$impl$u20$$BP$const$u20$T$GT$::copy_to_nonoverlapping::hd8d070cc2d0304ce(self="", dest="", count=467991955728) at const_ptr.rs:1310:18
    frame #3: 0x0000000100271e90 resvg`_$LT$T$u20$as$u20$alloc..slice..hack..ConvertVec$GT$::to_vec::h983a9aca5d2b815e(s=(data_ptr = "", length = 467991955728), alloc=Global @ 0x000000016fdfca47) at slice.rs:164:28
    frame #4: 0x0000000100179f8c resvg`alloc::slice::_$LT$impl$u20$$u5b$T$u5d$$GT$::to_vec::h47e5b12a08d8d4f9 [inlined] alloc::slice::hack::to_vec::h74723452fa1ea3d2(s=(data_ptr = "", length = 467991955728)) at slice.rs:108:9
    frame #5: 0x0000000100179f88 resvg`alloc::slice::_$LT$impl$u20$$u5b$T$u5d$$GT$::to_vec::h47e5b12a08d8d4f9 [inlined] alloc::slice::_$LT$impl$u20$$u5b$T$u5d$$GT$::to_vec_in::h9755591e1d912c54(self=(data_ptr = "", length = 467991955728)) at slice.rs:502:9
    frame #6: 0x0000000100179f88 resvg`alloc::slice::_$LT$impl$u20$$u5b$T$u5d$$GT$::to_vec::h47e5b12a08d8d4f9(self=(data_ptr = "", length = 467991955728)) at slice.rs:477:14
    frame #7: 0x000000010014ab10 resvg`tiny_skia::pixmap::PixmapRef::to_owned::h382a16fce2f306bb(self=0x000000016fdfd220) at pixmap.rs:314:19
    frame #8: 0x000000010014b150 resvg`tiny_skia::pixmap::PixmapRef::encode_png::ha43f4456a32599f5(self=0x000000016fdfd220) at pixmap.rs:397:30
    frame #9: 0x000000010000ea6c resvg`tiny_skia::pixmap::PixmapRef::save_png::h49e4a2e78c4dfeeb(self=0x000000016fdfd220, path=0x000000016fdfe9e0) at pixmap.rs:424:20
    frame #10: 0x000000010000ea14 resvg`tiny_skia::pixmap::Pixmap::save_png::heb9d440ae3de44f0(self=0x000000016fdfe940, path=0x000000016fdfe9e0) at pixmap.rs:179:9
    frame #11: 0x00000001000060a0 resvg`resvg::process::_$u7b$$u7b$closure$u7d$$u7d$::h0ed5e24227c737b0 at main.rs:113:17
    frame #12: 0x00000001000052a0 resvg`resvg::timed::h8b4859e0f6f53d62(perf=false, name=(data_ptr = "Saving\nresvg is an SVG rendering application.\n\nUSAGE:\n  resvg [OPTIONS] <in-svg> <out-png>  # from file to file\n  resvg [OPTIONS] <in-svg> -c         # from file to stdout\n  resvg [OPTIONS] - <out-png>         # from stdin to file\n  resvg [OPTIONS] - -c                # from stdin to stdout\n\n  resvg in.svg out.png\n  resvg -z 4 in.svg out.png\n  resvg --query-all in.svg\n\nOPTIONS:\n      --help                    Prints this help\n  -V, --version                 Prints version\n  -c                            Prints the output PNG to the stdout\n\n  -w, --width LENGTH            Sets the width in pixels\n  -h, --height LENGTH           Sets the height in pixels\n  -z, --zoom FACTOR             Zooms the image by a factor\n      --dpi DPI                 Sets the resolution\n                                [default: 96] [possible values: 10..4000 (inclusive)]\n  --background COLOR            Sets the background color\n                                Examples: red, #fff, #fff000\n  --stylesheet PATH             Inject a style"..., length = 6), f={closure_env#8} @ 0x000000016fdfd310) at main.rs:23:18
    frame #13: 0x00000001000080dc resvg`resvg::process::hebc78e1620b3f18b at main.rs:112:13
    frame #14: 0x00000001000078a8 resvg`resvg::main::h7007c36a68505103 at main.rs:12:21
    frame #15: 0x00000001000039d4 resvg`core::ops::function::FnOnce::call_once::h02d9d4c4ea691fcf((null)=(resvg`resvg::main::h7007c36a68505103 at main.rs:11), (null)=<unavailable>) at function.rs:250:5
    frame #16: 0x0000000100012914 resvg`std::sys::backtrace::__rust_begin_short_backtrace::h12394669fe5d34f5(f=(resvg`resvg::main::h7007c36a68505103 at main.rs:11)) at backtrace.rs:154:18

@waywardmonkeys
Copy link
Collaborator

@yanschor If you could include the stack traces on future reports, that'd be great ... and if you can isolate what the actual issue is, that'd be good too. The test cases are already pretty minimal, but could be slightly more so with a bit of hand editing, probably!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants