Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(PUP-9997) Avoid Dir.chdir #9387

Merged
merged 4 commits into from
Jul 1, 2024
Merged

Conversation

joshcooper
Copy link
Contributor

@joshcooper joshcooper commented Jun 7, 2024

Dir.chdir is problematic because it affects all threads in the current process
and if puppet is started with a current working directory it doesn't have
traverse/execute permission to, then it won't be able to restore the cwd at the
end of the Dir.chdir block.

Puppet supports two execution implementations (posix and windows), both of which
already support the cwd option. Puppetserver also injects its external
implementation using the execution_stub. Its execution implementation now
supports the cwd option, see SERVER-3051.

Also pass the directory in which to extract modules to the tar command.

This should not be backported to 7.x

@joshcooper joshcooper force-pushed the dir_chdir_9997 branch 3 times, most recently from f9b945c to 6b900b6 Compare June 12, 2024 17:23
@joshcooper joshcooper marked this pull request as ready for review June 12, 2024 22:10
@joshcooper joshcooper requested review from a team as code owners June 12, 2024 22:10
@joshcooper joshcooper added the bug Something isn't working label Jun 13, 2024
spec/unit/module_tool/tar/gnu_spec.rb Show resolved Hide resolved
lib/puppet/module_tool/tar/gnu.rb Outdated Show resolved Hide resolved
lib/puppet/module_tool/tar/gnu.rb Show resolved Hide resolved
Copy link
Contributor

@ekohl ekohl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get you're not going to rename Puppet::ModuleTool::Tar::Gnu to Puppet::ModuleTool::Tar::Unix, but it would be nice if that was done eventually. Luckily the module tool isn't all that intensively used and I don't think the performance of executing a shell is a significant part of its overall performance.

On a related note: I have thought about a more generic gem/utility that tools like r10k can use to decouple from Puppet itself. For example, in our Puppet module acceptance tests I'd love to be able to trim the whole dependency tree down to remove Puppet.

But for now this looks like a good improvement. I just question if it's complete.

For example, there is:

# We need to chdir to our cwd before changing privileges as there's a
# chance that the user may not have permissions to access the cwd, which
# would cause execute_posix to fail.
cwd = options[:cwd]
Dir.chdir(cwd) if cwd

This end up calling Kernel.exec but if I understand the documentation correctly that has the same signature as spawn, which takes a current_directory option. And an env option to remove the withenv option, but that's maybe a separate issue.

Then there's:

# Now convert to pdf
Dir.chdir("/tmp") do
%x(texi2pdf puppetdoc.tex >/dev/null 2>/dev/null)
end

This should probably use some execution helper instead of %x() (and respect TMPDIR).

The daemonize code is probably the most legitimate use of chdir and can stay:

Dir.chdir("/")

Last one is very questionable:

Dir.chdir(File.dirname(@resource[:path])) do

Because if you make target absolute then there's no need to chdir. Even more questionable: @resource.remove_existing(target) above it implies target is already absolute making the chdir redundant.

@joshcooper joshcooper marked this pull request as draft June 24, 2024 16:04
@joshcooper
Copy link
Contributor Author

I just question if it's complete.

Thanks @ekohl, I appreciate your feedback. I'm thinking of next steps:

  • The call to Kernel.exec happens inside a forked child. I'll look into passing the directory to the exec method
  • The Puppet::Util::Reference code needs updating, I'll fix that
  • Agree about daemonizing code
  • The file target can be relative but the path should be absolute. Let me see if there's a way to resolve this

@joshcooper
Copy link
Contributor Author

joshcooper commented Jun 28, 2024

This end up calling Kernel.exec but if I understand the documentation correctly that has the same signature as spawn, which takes a current_directory option.

There's a comment that explains why we need to Dir.chdir before dropping privileges before calling Kernel.exec. The call also happens in a forked child, so won't affect the parent process.

This should probably use some execution helper instead of %x() (and respect TMPDIR).

This code was dead, so I deleted it

The daemonize code is probably the most legitimate use of chdir and can stay:

Yes, agreed. From https://man7.org/linux/man-pages/man7/daemon.7.html, "In the daemon process, change the current directory to the root directory (/), in order to avoid that the daemon involuntarily blocks mount points from being unmounted."

Because if you make target absolute then there's no need to chdir.

The Dir.chdir was introduced as a way to validate relative symlinks, but is no longer necessary. I deleted it in my PR. See commit message for details.

When the Dir.chdir block ends we may not have permission to switch our cwd back
to where we started. So keep the cwd as is and pass the dest dir to the
tar/find/chown commands. Since we're passing arbitrary destination directories,
escape it.
Dir.chdir is problematic because it affects all threads in the current process
and if puppet is started with a current working directory it doesn't have
traverse/execute permission to, then it won't be able to restore the cwd at the
end of the Dir.chdir block.

Puppet supports three execution implementations: posix, windows and stub. The
first two already support the `cwd` option. Puppetserver injects its stub
implementation and it recently added support for `cwd`, see SERVER-3051.
The Puppet::Util::Reference.pdf method has been broken since 2012 due to
commit f0c9995, because it was calling
Puppet::Util.replace_file with 1 argumentm, but it takes 2, resulting
in:

    $ puppet doc --mode pdf --reference type
    creating pdf
    Error: Could not run: wrong number of arguments (given 1, expected 2)

So delete the code and remove `pdf` as a valid reference doc mode.
@joshcooper joshcooper force-pushed the dir_chdir_9997 branch 2 times, most recently from 2e5078d to 66c4a25 Compare June 28, 2024 17:32
@joshcooper joshcooper closed this Jun 28, 2024
@joshcooper joshcooper reopened this Jun 28, 2024
@joshcooper joshcooper closed this Jun 28, 2024
@joshcooper joshcooper reopened this Jun 28, 2024
Symlinks can refer to their target using a relative or absolute path. If a
relative path is used, then it's assumed to be relative to the symlink's parent
directory. For example, /tmp/link refers to /tmp/target using a relative path:

    $ ls -l /tmp/link
    lrwxrwxrwx 1 josh 6 Jun 27 23:26 /tmp/link -> target

In commit 02f91fc, symlinks were merged into the file (aka pfile) type. The
`ensure` property was modified to validate whether the target existed or was a
directory[1] In order for relative symlinks to be validated, it was necessary
to call `Dir.chdir`.

Later the call to `Dir.chdir` was moved to `target.rb`[2], and then the
`FileTest.exists?(target)` check was deleted[3].

This commit removes the call to `Dir.chdir`. This means we no longer change our
cwd prior to dropping privileges and creating the symlink. This should be ok
because none of the code within the Dir.chdir block is sensitive to cwd. Also
when we reach this block of code, we're committed to creating the symlink given
an absolute `@resource[:path]`, so we don't need to be concerned about TOCTOU
race conditions.

[1] https://github.com/puppetlabs/puppet/blob/02f91fcd55aefe63a11a29c5608c0738867b8130/lib/puppet/type/pfile/ensure.rb#L76-L86
[2] https://github.com/puppetlabs/puppet/blob/2cd67ad843409a49747748a1278e5ef788dd97bd/lib/puppet/type/pfile/target.rb#L41-L45
[3] puppetlabs@662fcaf#diff-03f0464c0f42c4b979f6150ec2f7e8043ec964fe4d401cd0cd57a82f43a856ba
@joshcooper joshcooper marked this pull request as ready for review June 28, 2024 20:15
@mhashizume mhashizume merged commit 36bdec0 into puppetlabs:main Jul 1, 2024
9 checks passed
ekohl added a commit to ekohl/puppet-archive that referenced this pull request Sep 13, 2024
The use of chdir is problematic in threaded environments (only one
thread may chdir) and Puppet has a native method for this.

Link: puppetlabs/puppet#9387
ekohl added a commit to ekohl/puppet-archive that referenced this pull request Sep 13, 2024
The use of chdir is problematic in threaded environments (only one
thread may chdir) and Puppet has a native method for this.

Link: puppetlabs/puppet#9387
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants