CA-364138 XSI-1217: fix FD leak, Unix.EMFILE

[lindig]

• Fix file descriptor leak
• Log the number of file descriptors in use when serving a new VM - this is optional but helps to verify the fix
• Currently testing this (after having removed the debugging code that I used to find the leak)

[lindig]

Processing  1/1: [default: git]
Error:  Could not update repository "default": "/usr/bin/git fetch -q" exited with code 128 "fatal: unable to access 'https://github.com/xapi-project/xs-opam.git/': The requested URL returned error: 429"
Error:  Initial download of repository failed
'/opt/hostedtoolcache/opam/2.0.8/x64/opam init --bare -yav https://github.com/xapi-project/xs-opam.git' failed.
Error: Error: The process '/opt/hostedtoolcache/opam/2.0.8/x64/opam' failed with exit code 40

HTTP error 429 would indicate a rate limit. So: not a code problem so far.

[edwintorok]

You can use Lwt_unix.files_of_directory and then count the number of entries in the Lwt stream if we want to avoid blocking here (Sys.readdir would be a blocking call that blocks all Lwt promises)

[edwintorok]

let fds = Lwt_unix.files_of_directory "/proc/self/fd" |> Lwt_stream.to_list >|= List.length in

or

let count_stream s = Lwt_stream.fold (fun _ acc -> acc + 1) s 0 in
let fds = Lwt_unix.files_of_directory "/proc/self/fd" |> Lwt_stream.to_list >|= List.length in

[edwintorok]

(there are a few more places in varstored_guard that use blocking calls like Sys.file_exists, don't worry about those I'll try to fix those in my PR)

[psafont]

I think you meant

let stream_length s = Lwt_stream.fold (fun _ acc -> acc + 1) s 0 in
let fds = Lwt_unix.files_of_directory "/proc/self/fd" |> stream_length

[lindig]

The point about blocking is good. On the other hand: the /proc file system is implemented in the kernel and this is called only once we serve a new VM.

[lindig]

Small addendum: the change to use Lwt_unix.files_of_directory leads to a slight inaccuracy because it reports . and .. as well, which the previous code did not report.

+let log_fds () =
+  let count stream = Lwt_stream.fold (fun _ n -> n + 1) stream 0 in
+  Lwt_unix.files_of_directory "/proc/self/fd" |> count >>= fun fds ->
+  D.info "file descriptors in use: %d" fds ;
+  Lwt.return_unit