memory leak on accessing params, while sending post-request

[profundus]

Hello! I'm not sure, but it really looks like memory leak in Mojo.

The demo application is:

use Mojolicious::Lite;

app->routes->post('/foo', sub {
  my $c = shift;
  $c->param('ab');
  $c->rendered(200);
});

app->start;

I start it in command-line via:

morbo mojo-app-ml.pl -w . -l 'http://*:3000' \
 > /dev/null 2&> /dev/null

Then in separate terminal I start loading mojo with requests

for i (`seq 10000`); do \
 curl -s --data-binary @/usr/share/webkitgtk-1.0/images/urlIcon.png \
 'http://localhost:3000/foo?ab=x.png&size=1200,1200'; \
done

Then I switch to another terminal and attentively look at memory usage via htop utility... and both "virtual" and "resident" memory are very slowly but constantly increasing.

I have created test plan in apache-jmeter which leads to the same results, but a little bit faster

<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="2.2">
  <hashTree>
    <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Test Plan" enabled="true">
      <stringProp name="TestPlan.comments"></stringProp>
      <boolProp name="TestPlan.functional_mode">false</boolProp>
      <boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
      <elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
        <collectionProp name="Arguments.arguments">
          <elementProp name="" elementType="Argument">
            <stringProp name="Argument.name"></stringProp>
            <stringProp name="Argument.value"></stringProp>
            <stringProp name="Argument.metadata">=</stringProp>
          </elementProp>
        </collectionProp>
      </elementProp>
      <stringProp name="TestPlan.user_define_classpath"></stringProp>
    </TestPlan>
    <hashTree>
      <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Thread Group" enabled="true">
        <stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
        <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
          <boolProp name="LoopController.continue_forever">false</boolProp>
          <stringProp name="LoopController.loops">10</stringProp>
        </elementProp>
        <stringProp name="ThreadGroup.num_threads">30</stringProp>
        <stringProp name="ThreadGroup.ramp_time">1</stringProp>
        <longProp name="ThreadGroup.start_time">1362307199000</longProp>
        <longProp name="ThreadGroup.end_time">1362307199000</longProp>
        <boolProp name="ThreadGroup.scheduler">false</boolProp>
        <stringProp name="ThreadGroup.duration"></stringProp>
        <stringProp name="ThreadGroup.delay"></stringProp>
      </ThreadGroup>
      <hashTree>
        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="HTTP Request (/foo)" enabled="true">
          <boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments">
            <collectionProp name="Arguments.arguments">
              <elementProp name="" elementType="HTTPArgument">
                <boolProp name="HTTPArgument.always_encode">false</boolProp>
                <stringProp name="Argument.value">la-la-la</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
              </elementProp>
            </collectionProp>
          </elementProp>
          <stringProp name="HTTPSampler.domain">localhost</stringProp>
          <stringProp name="HTTPSampler.port">3000</stringProp>
          <stringProp name="HTTPSampler.connect_timeout"></stringProp>
          <stringProp name="HTTPSampler.response_timeout"></stringProp>
          <stringProp name="HTTPSampler.protocol"></stringProp>
          <stringProp name="HTTPSampler.contentEncoding"></stringProp>
          <stringProp name="HTTPSampler.path">/foo?ab=x.png&amp;size=1200,1200</stringProp>
          <stringProp name="HTTPSampler.method">POST</stringProp>
          <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
          <boolProp name="HTTPSampler.auto_redirects">false</boolProp>
          <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
          <boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
          <boolProp name="HTTPSampler.monitor">false</boolProp>
          <stringProp name="HTTPSampler.embedded_url_re"></stringProp>
        </HTTPSamplerProxy>
        <hashTree/>
      </hashTree>
    </hashTree>
  </hashTree>
</jmeterTestPlan>

The software versions are:

$cpanm --info Mojolicious
SRI/Mojolicious-3.87.tar.gz
#perl -v

This is perl 5, version 16, subversion 2 (v5.16.2) built for x86_64-linux-thread-multi
(with 22 registered patches, see perl -V for more detail)

Copyright 1987-2012, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

[kraih]

I can confirm the leak, but so far i've been unable to track it down and wouldn't be surprised if it was actually a Perl bug.

[kraih]

I've been able to reduce the leak to a minimal test case, so far it does look like a Perl (or Encode) bug.

use 5.16.0;
use Encode 'find_encoding';

my $encoding = find_encoding 'UTF-8';
my $invalid  = "\x89";

for (1 .. 100000000) {
  say $_;
  eval { $encoding->decode($invalid, 1); 1 };
}

[kraih]

I've opened an issue for Encode too. dankogai/p5-encode#8

[kraih]

Closing this issue since we have a first confirmation that it is in fact an Encode bug.