Skip to content

Commit

Permalink
jemalloc: Downgrade to 3.6.0 to reduce memory usage
Browse files Browse the repository at this point in the history
See the following issue for more detail:
#305

Signed-off-by: Takuro Ashie <ashie@clear-code.com>
  • Loading branch information
ashie committed Jul 14, 2021
1 parent 050f9e2 commit 647a536
Show file tree
Hide file tree
Showing 2 changed files with 24 additions and 16 deletions.
35 changes: 20 additions & 15 deletions td-agent/Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -631,21 +631,25 @@ class BuildTask
configure_opts = [
"--prefix=#{install_prefix}",
]
if ENV["TD_AGENT_STAGING_PATH"] and
(ENV["TD_AGENT_STAGING_PATH"].end_with?("el8.aarch64") or
ENV["TD_AGENT_STAGING_PATH"].end_with?("el7.aarch64"))
# NOTE: There is a case that PAGE_SIZE detection on
# CentOS 7 CentOS 8 with aarch64 AWS ARM instance.
# So, explicitly set PAGE_SIZE by with-lg-page 16 (2^16 = 65536)
configure_opts.concat(["--with-lg-page=16"])
end
if ENV["TD_AGENT_STAGING_PATH"] and
ENV["TD_AGENT_STAGING_PATH"].end_with?("el8.ppc64le")
# NOTE: There is a case that PAGE_SIZE detection on
# CentOS 8 with ppc64le.
# So, explicitly set PAGE_SIZE by with-lg-page 16 (2^16 = 65536)
configure_opts.concat(["--with-lg-page=16"])

if JEMALLOC_VERSION.split('.')[0].to_i >= 4
if ENV["TD_AGENT_STAGING_PATH"] and
(ENV["TD_AGENT_STAGING_PATH"].end_with?("el8.aarch64") or
ENV["TD_AGENT_STAGING_PATH"].end_with?("el7.aarch64"))
# NOTE: There is a case that PAGE_SIZE detection on
# CentOS 7 CentOS 8 with aarch64 AWS ARM instance.
# So, explicitly set PAGE_SIZE by with-lg-page 16 (2^16 = 65536)
configure_opts.concat(["--with-lg-page=16"])
end
if ENV["TD_AGENT_STAGING_PATH"] and
ENV["TD_AGENT_STAGING_PATH"].end_with?("el8.ppc64le")
# NOTE: There is a case that PAGE_SIZE detection on
# CentOS 8 with ppc64le.
# So, explicitly set PAGE_SIZE by with-lg-page 16 (2^16 = 65536)
configure_opts.concat(["--with-lg-page=16"])
end
end

cd(source_dir) do
sh("./configure", *configure_opts)
sh("make", "install", "-j#{Etc.nprocessors}", "DESTDIR=#{STAGING_DIR}")
Expand Down Expand Up @@ -1045,7 +1049,8 @@ class BuildTask
end

def remove_needless_files
remove_files("#{td_agent_staging_dir}/bin/jeprof", true)
remove_files("#{td_agent_staging_dir}/bin/jeprof", true) # jemalloc 4 or later
remove_files("#{td_agent_staging_dir}/bin/pprof", true) # jemalloc 3
remove_files("#{td_agent_staging_dir}/share/doc", true) # Contains only jemalloc.html
remove_files("#{td_agent_staging_dir}/share/ri", true)
cd("#{gem_staging_dir}/cache") do
Expand Down
5 changes: 4 additions & 1 deletion td-agent/config.rb
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@
FLUENTD_LOCAL_GEM_REPO = "file://" + File.expand_path(File.join(__dir__, "local_gem_repo"))

# https://github.com/jemalloc/jemalloc/releases
JEMALLOC_VERSION = "5.2.1"
# Use jemalloc 3.x to reduce memory usage
# See https://github.com/fluent-plugins-nursery/td-agent-builder/issues/305
JEMALLOC_VERSION = "3.6.0"
#JEMALLOC_VERSION = "5.2.1"

# https://www.openssl.org/source/
OPENSSL_VERSION = "1.1.1k"
Expand Down

0 comments on commit 647a536

Please sign in to comment.