Emacs freezes at the compiling step while installing vterm-module via straight's use-package integration.

[hongyi-zhao]

On Ubuntu 20.04.3 LTS, I use the following version of self-compiled GNU Emacs:

GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.20, cairo version 1.16.0) of 2021-11-14

At the same time, the following command was used to install emacs-libvterm with straight's use-package integration:

(use-package vterm)

But Emacs will freeze at the following step:

For more related discussions, please see here. Any hints for fixing this problem?

Regards,
HZ

[progfolio]

Yes. We don't handle user input during package installation. Because you aren't deferring the loading of vterm, it is immediately required. The package (reasonably) checks to see if the module it relies on is available.

One solution is to defer the loading of vterm and you will be prompted in your session to compile the module when vterm is loaded. This can be achieved by several use-package keywords (:defer, :commands, etc).

The second is to use straight's :post-build keyword to compile the module. That's what I do in my config:
https://github.com/progfolio/.emacs.d#vterm

[hongyi-zhao]

I inspected your configurartion here, as follows:

(use-package vterm
  :straight (:post-build (cl-letf (((symbol-function #'pop-to-buffer)
                                    (lambda (buffer) (with-current-buffer buffer (message (buffer-string))))))
                           (setq vterm-always-compile-module t)
                           (require 'vterm)))
  :commands (vterm vterm-other-window)
  :general
  (+general-global-application
    "t" '(:ignore t :which-key "terminal")
    "tt" 'vterm-other-window
    "t." 'vterm)
  :config
  (evil-set-initial-state 'vterm-mode 'emacs))

I've some questions for adapting it to my scenario:

1. Why do you use cl-letf, and what's the meaning?
2. The + maybe indicate that you are using doom-emacs based configuration, which I don't use. So, how to adjust it for my case?
3. I don't use evil. Can I omit the last line configuration you use?

[progfolio]

1. The cl-letfwas intended to temporarily remap pop-to-buffer because the compilation process would pop to another buffer.
   You can ignore it. I don't recall if it actually did the trick or not. What's important is setting vterm-always-compile-module non-nil. Then the first time the package is required it should compile the module without prompting (and freezing your installation).

2. I don't use Doom. That's just a naming convention.

3. The only relevant section is the :pre-build section. Modify as you see fit.

[hongyi-zhao]

2. I don't use Doom. That's just a naming convention.

If I understand correctly, you use general here to define key definitions.

3. The only relevant section is the :pre-build section. Modify as you see fit.

:pre-build or :post-build? Your configuration uses :post-build, but you say :pre-build here.

[progfolio]

Apologies. :post-build in this case.

[hongyi-zhao]

I tried the following, and it seems to work:

(use-package vterm
  :straight (:post-build (cl-letf (((symbol-function #'pop-to-buffer)
                                    (lambda (buffer) (with-current-buffer buffer (message (buffer-string))))))
                           (setq vterm-always-compile-module t)
                           (require 'vterm)))
  :commands (vterm vterm-other-window)
  :general
  (+general-global-application
   "t" '(:ignore t :which-key "terminal")
   "tt" 'vterm-other-window
   "t." 'vterm))

[progfolio]

That's good. Though I don't see how the :general bit would apply to your config unless you've copied over my key binding macros.

[hongyi-zhao]

This is just what puzzles me. Where is the definition of this key binding macro? What additional bits should be added to my above configuration in order to use the general settings you customized?

[progfolio]

https://gist.github.com/progfolio/1c96a67fcec7584b31507ef664de36cc

[hongyi-zhao]

Thank you again. The following macro needs to be put into my Emacs initialization file before the above configuration mentioned in this issue:

(defmacro +general-global-menu! (name infix-key &rest body)
  "Create a definer named +general-global-NAME wrapping global-definer.
Create prefix map: +general-global-NAME. Prefix bindings in BODY with INFIX-KEY."
  (declare (indent 2))
  `(progn
     (general-create-definer ,(intern (concat "+general-global-" name))
       :wrapping global-definer
       :prefix-map (quote ,(intern (concat "+general-global-" name "-map")))
       :infix ,infix-key
       :wk-full-keys nil
       "" '(:ignore t :which-key ,name))
     (,(intern (concat "+general-global-" name))
      ,@body)))

So, the complete configuration used by me is as follows:

(defmacro +general-global-menu! (name infix-key &rest body)
  "Create a definer named +general-global-NAME wrapping global-definer.
Create prefix map: +general-global-NAME. Prefix bindings in BODY with INFIX-KEY."
  (declare (indent 2))
  `(progn
     (general-create-definer ,(intern (concat "+general-global-" name))
       :wrapping global-definer
       :prefix-map (quote ,(intern (concat "+general-global-" name "-map")))
       :infix ,infix-key
       :wk-full-keys nil
       "" '(:ignore t :which-key ,name))
     (,(intern (concat "+general-global-" name))
      ,@body)))

(use-package vterm
  :straight (:post-build (cl-letf (((symbol-function #'pop-to-buffer)
                                    (lambda (buffer) (with-current-buffer buffer (message (buffer-string))))))
                           (setq vterm-always-compile-module t)
                           (require 'vterm)))
  :commands (vterm vterm-other-window)
  :general
  (+general-global-application
   "t" '(:ignore t :which-key "terminal")
   "tt" 'vterm-other-window
   "t." 'vterm))

But for the above configuration, when Emacs started, I didn't notice the installation of vterm.

[progfolio]

You still need to define +general-global-application as shown in the gist I linked earlier.
But this is beyond the scope of the original issue.

when Emacs started, I didn't notice the installation of vterm.

The compilation should only run after the build steps are run. So unless the package needs to be rebuilt or you trigger that manually via straight-rebuild-package, the module compilation should not be triggered.

[hongyi-zhao]

You still need to define +general-global-application as shown in the gist I linked earlier.
But this is beyond the scope of the original issue.

I follow up this question here. Hope to get some further help and clues.

The compilation should only run after the build steps are run.

With the configuration discussed here, the emacs-libvterm can't be cloned to my machine at all:

$ ls ~/.emacs.d/straight/repos/emacs-libvterm
ls: cannot access '.emacs.d/straight/repos/emacs-libvterm': No such file or directory

So unless the package needs to be rebuilt or you trigger that manually via straight-rebuild-package, the module compilation should not be triggered.

Due to the emacs-libvterm repository doesn't exist on my machine, it's impossible for the straight-rebuild-package to find this package for compilation.

[progfolio]

What happens with M-x straight-rebuild-package vterm?

[hongyi-zhao]

See the following:

Another very strange problem will occur when I try to use the following configuration according to the comment you have given on the gist:

(defmacro +general-global-menu! (name infix-key &rest body)
  "Create a definer named +general-global-NAME wrapping global-definer.
Create prefix map: +general-global-NAME. Prefix bindings in BODY with INFIX-KEY."
  (declare (indent 2))
  `(progn
     (general-create-definer ,(intern (concat "+general-global-" name))
       :wrapping global-definer
       :prefix-map (quote ,(intern (concat "+general-global-" name "-map")))
       :infix ,infix-key
       :wk-full-keys nil
       "" '(:ignore t :which-key ,name))
     (,(intern (concat "+general-global-" name))
      ,@body)))


;; https://gist.github.com/progfolio/1c96a67fcec7584b31507ef664de36cc#gistcomment-3978515
(+general-global-menu! "application" "a")

(use-package vterm
  :straight (:post-build (cl-letf (((symbol-function #'pop-to-buffer)
                                    (lambda (buffer) (with-current-buffer buffer (message (buffer-string))))))
                           (setq vterm-always-compile-module t)
                           (require 'vterm)))
  :commands (vterm vterm-other-window)
  :general
  (+general-global-application
   "t" '(:ignore t :which-key "terminal")
   "tt" 'vterm-other-window
   "t." 'vterm))

With the above configuration, i.e., by adding the following line:

(+general-global-menu! "application" "a")

My Emacs will only load part of the initialization file.

If I commented out the above line, Emacs will clone the vterm repository and build it, but still will freeze:

[progfolio]

I think it's likely that you are experiencing a configuration issue that isn't straight specific.
Please try to resolve that first. I can't offer support for that gist at the moment, but you can see a working example in my configuration:

https://github.com/progfolio/.emacs.d

[hongyi-zhao]

I think it's likely that you are experiencing a configuration issue that isn't straight specific. Please try to resolve that first. I can't offer support for that gist at the moment, but you can see a working example in my configuration:

https://github.com/progfolio/.emacs.d

I try to do the test with a separate, almost the same configuration (/home/werner/.emacs.d/debug/.emacs.d/init.el) as you use:

#!/usr/bin/env bash
:;#/home/werner/.emacs.d/debug/.emacs.d/init.el
:; realpath $0 

:;# Use the following command to test:
:;#bash /home/werner/.emacs.d/debug/.emacs.d/init.el

:;#https://groups.google.com/g/comp.unix.shell/c/krxzfCd_8qM/m/ryoNs9AGCQAJ
:;# https://groups.google.com/g/comp.unix.shell/c/3vrq7pqoG-A/m/Hnk0lsKfAwAJ
:;# https://lists.gnu.org/archive/html/help-gnu-emacs/2021-10/msg00698.html
:;# https://github.com/company-mode/company-mode/discussions/1248#discussioncomment-1535692
:; HOME=$(dirname $(dirname $(realpath -e $0))) proxychains-ng-socks5 /usr/local/bin/emacs -- "$@"; exit
:;# or
:;# HOME=$(dirname $(dirname $(realpath -e $0))) exec proxychains-ng-socks5 /usr/local/bin/emacs -- "$@"


;;Bootstrap straight
(defvar bootstrap-version)
(let ((bootstrap-file
       (expand-file-name "straight/repos/straight.el/bootstrap.el" user-emacs-directory))
      (bootstrap-version 5))
  (unless (file-exists-p bootstrap-file)
    (with-current-buffer
        (url-retrieve-synchronously
         "https://raw.githubusercontent.com/raxod502/straight.el/develop/install.el"
         'silent 'inhibit-cookies)
      (goto-char (point-max))
      (eval-print-last-sexp)))
  (load bootstrap-file nil 'nomessage))
(straight-use-package 'use-package)
(setq straight-use-package-by-default t)

;; https://github.com/progfolio/.emacs.d#general-key-bindings
;; general.el provides a more convenient method for binding keys in emacs (for both evil and non-evil users).
;; https://github.com/noctuid/general.el#about
;; Load general before the remaining packages so they can make use of the :general keyword in their declarations.
(use-package general
  :demand t
  :config
  (general-override-mode)
  (general-auto-unbind-keys)
  <<general-config>>)


;; https://github.com/progfolio/.emacs.d#config
;; The global definer allows me to use a leader key in most states.
(general-create-definer global-definer
  :keymaps 'override
  :states '(insert normal hybrid motion visual operator)
  :prefix "SPC"
  :non-normal-prefix "S-SPC")

(global-definer
 "!"   'shell-command
 ":"   'eval-expression
 "."   'repeat)

;; We define a global-leader definer to access major-mode specific bindings:
(general-create-definer global-leader
  :keymaps 'override
  :states '(insert normal hybrid motion visual operator)
  :prefix "SPC m"
  :non-normal-prefix "S-SPC m"
  "" '( :ignore t
        :which-key
        (lambda (arg)
          (cons (cadr (split-string (car arg) " "))
                (replace-regexp-in-string "-mode$" "" (symbol-name major-mode))))))

;; And a macro to ease the creation of nested menu bindings:
;; https://github.com/raxod502/straight.el/issues/891#issuecomment-982384771
;; https://lists.gnu.org/archive/html/help-gnu-emacs/2021-11/msg00404.html
(defmacro +general-global-menu! (name infix-key &rest body)
  "Create a definer named +general-global-NAME wrapping global-definer.
Create prefix map: +general-global-NAME. Prefix bindings in BODY with INFIX-KEY."
  (declare (indent 2))
  `(progn
     (general-create-definer ,(intern (concat "+general-global-" name))
       :wrapping global-definer
       :prefix-map (quote ,(intern (concat "+general-global-" name "-map")))
       :infix ,infix-key
       :wk-full-keys nil
       "" '(:ignore t :which-key ,name))
     (,(intern (concat "+general-global-" name))
      ,@body)))


;; https://github.com/progfolio/.emacs.d#applications
;; https://gist.github.com/progfolio/1c96a67fcec7584b31507ef664de36cc#gistcomment-3978515
(+general-global-menu! "application" "a")

;; https://github.com/progfolio/.emacs.d#vterm
;; Emacs-libvterm (vterm) is fully-fledged terminal emulator inside GNU Emacs based on libvterm, a C library.
;; https://github.com/akermu/emacs-libvterm
(use-package vterm
  :straight (:post-build (cl-letf (((symbol-function #'pop-to-buffer)
                                    (lambda (buffer) (with-current-buffer buffer (message (buffer-string))))))
                           (setq vterm-always-compile-module t)
                           (require 'vterm)))
  :commands (vterm vterm-other-window)
  :general
  (+general-global-application
   "t" '(:ignore t :which-key "terminal")
   "tt" 'vterm-other-window
   "t." 'vterm))

But Emacs still freezes at the compiling step. After I cancelled the compilation with C-g, the following message is observed:

[progfolio]

Please use straight-bug-report to reproduce any issues you are having. It ensures the testing environment is clean and consistent and provides us with other useful information.

Unless the issue is graphical, please share the text of a buffer instead of a screenshot of the text of a buffer. Text is easier to search and copy.

The main issue here is that you're adding in too many variables at once to diagnose the issue. You want to start from the minimal configuration which exhibits the issue and work from there. This following test works on my system:

Yodel Report (2021-11-30 14:59:31):

(yodel
  :save "vterm.straight"
  :formatter yodel-format-as-github-markdown
  :packages*
  (vterm
   :post-build
   (progn
     (setq vterm-always-compile-module t)
     (require 'vterm)))
  :post*
  (let
      ((default-directory
        (straight--build-dir "vterm")))
    (print
     (straight--process-output "ls" "-lah" "vterm-module.so"))))

STDOUT:

Loading /tmp/vterm.straight/straight-bootstrap-snippet.el (source)...

"-rwxr-xr-x 1 n n 134K Nov 30 14:57 vterm-module.so"

Environment

• emacs version: GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo version 1.17.4, Xaw3d scroll bars)
  of 2021-11-22
• system type: gnu/linux

Packages

Name	Branch	Commit	Date	Source
vterm	master	akermu/emacs-libvterm@2681120	2021-09-08	melpa

The gist I made is a rough guide, but anyone using it should read over general's documentation and understand how to use that package first.

Copying bits and pieces of someone's configuration is not guaranteed to work for various reasons. As you've seen here, functions may rely on other functions. There may be some configuration context missing, etc. For example, the last test you've posted contains Org noweb syntax:

<<general-config>>

This is not valid elisp. It's a syntax used in Org src blocks which only results in a valid elisp file after tangling.

See if the issue persists with the following test case:

Test Case

(straight-bug-report
  :pre-bootstrap 
  (setq straight-repository-branch "develop")
  :post-bootstrap 
  (straight-use-package
   '(vterm :post-build
           (progn
             (setq vterm-always-compile-module t)
             (require 'vterm))))
  (let
      ((default-directory
        (straight--build-dir "vterm")))
    (print
     (straight--process-output "ls" "-lah" "vterm-module.so"))))

• Test run at: 2021-11-30 15:13:21
• system-type: gnu/linux
• straight-version: prerelease (HEAD -> develop, origin/develop) 92d4153 2021-11-23
• emacs-version: GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo version 1.17.4, Xaw3d scroll bars) of 2021-11-22

Output

Bootstrapping straight.el...
Bootstrapping straight.el...done
Rebuilding all packages due to build cache schema change
Looking for gnu-elpa-mirror recipe -> Cloning melpa...
Looking for gnu-elpa-mirror recipe -> Cloning melpa...done
Looking for emacsmirror-mirror recipe -> Cloning gnu-elpa-mirror...
Looking for emacsmirror-mirror recipe -> Cloning gnu-elpa-mirror...done
Looking for emacsmirror-mirror recipe -> Cloning el-get...
Looking for emacsmirror-mirror recipe -> Cloning el-get...done
Looking for straight recipe -> Cloning emacsmirror-mirror...
Looking for straight recipe -> Cloning emacsmirror-mirror...done
Building straight...
Building straight...done

Test run with version: prerelease (HEAD -> develop, origin/develop) 92d4153 2021-11-23
Cloning emacs-libvterm (for vterm)...
Cloning emacs-libvterm (for vterm)...done
Building vterm...

Compilation of `emacs-libvterm' module succeeded
Building vterm...done

"-rwxr-xr-x 1 n n 134K Nov 30 15:13 vterm-module.so"

Packages:
"org-elpa"                n/a                  n/a
"melpa"                   n/a                  master 76e7a6c9 2021-11-29
"gnu-elpa-mirror"         n/a                  master 98cfebc 2021-11-30
"el-get"                  melpa                master 960f3fb9 2021-10-31
"emacsmirror-mirror"      n/a                  master 659f28e 2021-11-30
"straight"                n/a                  develop 92d4153 2021-11-23
"vterm"                   melpa                master 2681120 2021-09-08

If it does, we can continue troubleshooting.
If it does not, it would be more appropriate to ask general elisp/configuration questions on a separate forum (e.g. reddit.com/r/emacs, emacs IRC channel, the help-gnu-emacs mailing list).

[hongyi-zhao]

OK. I try to reproduce your above test as follows:

1. Install yodel with the following configuration:

(use-package yodel
  :straight (:host github :repo "progfolio/yodel"))

2. Start Emacs, and C-x C-e the following code snippet in scratch buffer:

(yodel
  :save "vterm.straight"
  :formatter yodel-format-as-github-markdown
  :packages*
  (vterm
   :post-build
   (progn
     (setq vterm-always-compile-module t)
     (require 'vterm)))
  :post*
  (let
      ((default-directory
        (straight--build-dir "vterm")))
    (print
     (straight--process-output "ls" "-lah" "vterm-module.so"))))

Emacs will freeze at the following step, as shown in the yodel buffer:

[progfolio]

This sounds a lot like an earlier issue you submitted about auctex:

#836

Which, unless I'm mistaken, ended up being a configuration issue and not an issue with straight.
I would follow the same process outlined in that issue. Try to eliminate as many variables as possible and then add one variable at a time to see if/when the issue crops up. I would start by seeing if you can manually compile the module from the command line first.

[hongyi-zhao]

I would start by seeing if you can manually compile the module from the command line first.

$ cd ~/.emacs.d/straight/repos/emacs-libvterm/
$ git log -1
commit 2681120b770573044832ba8c22ccbac192e1a294 (HEAD -> master, origin/master, origin/HEAD)
Author: jixiuf <jixiuf@qq.com>
Date:   Wed Sep 8 14:40:05 2021 +0800

    close #529  #496 fix Laggy scrolling in terminal programs.

$ cmake .
-- The C compiler identification is GNU 9.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- No build type selected, defaulting to RelWithDebInfo
-- System libvterm detected
-- Configuring done
-- Generating done
-- Build files have been written to: /home/werner/.emacs.d/straight/repos/emacs-libvterm

$ make
Scanning dependencies of target vterm-module
[ 25%] Building C object CMakeFiles/vterm-module.dir/vterm-module.c.o
[ 50%] Building C object CMakeFiles/vterm-module.dir/utf8.c.o
[ 75%] Building C object CMakeFiles/vterm-module.dir/elisp.c.o
[100%] Linking C shared module vterm-module.so
[100%] Built target vterm-module

$ ls *.so
vterm-module.so

[progfolio]

Ok. Then I would remove the module, and atttempt to compile it via the commands we were executing via the :pre-build step in the recipe from within Emacs.

[hongyi-zhao]

I tried with the following configuration:

(use-package vterm
  :straight (
	     :pre-build (
			 ("rm -fr build")
			 ("mkdir build")
			 ("cd build")
			 ("cmake ..")
			 ("make")
			 )

	     :post-build (cl-letf (((symbol-function #'pop-to-buffer)
                                    (lambda (buffer) (with-current-buffer buffer (message (buffer-string))))))
                           (setq vterm-always-compile-module t)
                           (require 'vterm)))
  :commands (vterm vterm-other-window)
  :general
  (+general-global-application
    "t" '(:ignore t :which-key "terminal")
    "tt" 'vterm-other-window
    "t." 'vterm))

But the following warnings/errors are triggered while Emacs is starting:

Warning (initialization): An error occurred while loading ‘/home/werner/.emacs.d/init.el’:

• Warnings buffer:

error: :pre-build command error in "vterm" recipe in command "(\"rm -fr build\")"

To ensure normal operation, you should investigate and remove the
cause of the error in your initialization file.  Start Emacs with
the ‘--debug-init’ option to view a complete error backtrace. Disable showing Disable logging

• straight-process buffer:

$ cd /home/werner/.emacs.d/straight/repos/emacs-libvterm/
$ rm\ -fr\ build

[File error while Searching for program]

But if I use the following configuration, though the compilation process will not be triggered automatically during the Emacs' initialization, it will be done by manually run M-x straight-rebuild-package RET vterm RET:

(use-package vterm
  :straight (
	     :pre-build (
			 (shell-command "rm -fr build && mkdir build && cd $_ && cmake .. && make")
			 )

	     :post-build (cl-letf (((symbol-function #'pop-to-buffer)
                                    (lambda (buffer) (with-current-buffer buffer (message (buffer-string))))))
                           (setq vterm-always-compile-module t)
                           (require 'vterm)))
  :commands (vterm vterm-other-window)
  :general
  (+general-global-application
    "t" '(:ignore t :which-key "terminal")
    "tt" 'vterm-other-window
    "t." 'vterm))

I also tried by running M-x vterm-module-compile RET, but it will freeze at the following step:

[progfolio]

Your :pre-build in the first test is incorrect. If you want a shell command it has to be a list of strings of the form (PROGRAM [ARG...]). You have one string that includes the args and the program. The error is telling you there is no program called "rm -fr build". Please see the documentation for :build/:pre-build in the recipe format section.

In the second test, you've added a :pre-build before confirming whether or not the :post-build would work on its own. Setting vterm-always-compile-module non-nil and requiring vterm should obviate the need for your :pre-build. Again, you should only add one variable at a time so you can determine what is causing a change in behavior.

I also tried by running M-x vterm-module-compile RET, but it will freeze at the following step

This indicates that the issue is most likely not due to straight.el. I would seek support on one of the appropriate channels I mentioned (reddit, IRC, mailing list). Again, this looks similar to the issue you were having with auctex. If you were able to fix that issue, I would follow a similar process. Unfortunately, since I am unable to reproduce this issue I don't think I'll be able to offer any more support.

[hongyi-zhao]

I now try with the following recipe:

(use-package vterm
  :straight (
	     :pre-build (
			 ("rm" "-fr" "build")
			 ("mkdir" "build")
			 ("cd" "build")
			 ("cmake" "..")
			 ("make"))))

But meet the following error while starting Emacs:

$ cd /home/werner/.emacs.d/straight/repos/emacs-libvterm/
$ rm -fr build

[Return code: 0]

$ cd /home/werner/.emacs.d/straight/repos/emacs-libvterm/
$ mkdir build


[Return code: 0]

$ cd /home/werner/.emacs.d/straight/repos/emacs-libvterm/
$ cd build

[File error while Searching for program]

Do you mean that only one of the :pre-build and :post-build instructions is required here? In this specific example, are they doing the same thing? If so, which one is preferable and should be used?

Another thing puzzles me most is that even I manually compile the vterm-module.so, this library file is only located under the straight/repos/emacs-libvterm, instead of located at straight/build/emacs-libvterm, and what surprised me more was that the emacs-libvterm package directory doesn't exist at all.

$ ls ~/.emacs.d/straight/build/emacs-libvterm
ls: cannot access '/home/werner/.emacs.d/straight/build/emacs-libvterm': No such file or directory

But vterm still can work smoothly by run M-x vterm.

[progfolio]

$ cd build

[File error while Searching for program]

"cd" is not a program. It is a shell command. See previous discussion here for why this also won't work the way you think it will:

#800 (comment)

But vterm still can work smoothly by run M-x vterm.

I would stick with this then.

[hongyi-zhao]

#800 (comment)

Yes. As the comment above points out, even the following command cannot affect any subsequent commands:

("bash" "-c" "cd" "build")

So, how can I do the cd operation correctly in :pre-build?

OTOH, I also tried without creating the build folder as follows:

 :pre-build (("cmake" ".")
	         ("make"))

But Emacs will freeze too, as shown below:

I would stick with this then.

I just confused about how it works.

[progfolio]

So, how can I do the cd operation correctly in :pre-build?

Two methods were explained in later comments in that same thread.

But Emacs will freeze too, as shown below:

I understand. As I mentioned, I'm unable to reproduce this. So unfortunately, I can't offer any new help. I have provided several ways to diagnose this type of problem in the issue you filed about auctex freezing. Please try those methods.

[hongyi-zhao]

Two methods were explained in later comments in that same thread.

According to the method suggested here, I tried the following two methods, but both failed:
1.

`("bash" "-c" "cd \"$1\" &&  "cmake" ".." && "make" "--"  ,(concat (straight--repos-dir "emacs-libvterm") "build"))

  ("bash" "-c" "cd" "build" && "cmake" ".." && "make")

As for another method proposed by you, I still can't figure out how to use it to this example.

If the problem discussed here can't be solved on my scenario, I must manually compile the vterm-module.so outside of Emacs. This is somewhat cumbersome.

[progfolio]

The :pre-build commands aren't your issue. The issue seems to be that your system freezes when trying to execute certain programs through call-process (assuming the information you gave in the thread about auctex freezing is accurate).
Figuring out the root cause of that would solve the problem.

("bash" "-c" "cd" "build" && "cmake" ".." && "make")

Probably doesn't work because bash's "-c" arg is expected to be a commad string.
e.g.

("bash" "-c" "build && cmake .. && make")

Again, the problem seems to be when call-process is invoked from within your Emacs session.
Please seek advice for that problem on a more general forum.

[hongyi-zhao]

("bash" "-c" "build && cmake .. && make")

You're right. I confirmed the following two equivalent working forms when using in straight's :pre-build directive when integrated with use-package:

 ("rm" "-fr" "build")
 ("mkdir" "build")
 ("bash" "-c" "cd build && cmake .. && make")

and

(shell-command "rm -fr build && mkdir build && cd $_ && cmake .. && make")

Both of the above forms work. But I observed the following strange phenomena:

After the vterm-module.so has been compiled by the above methods, if I first run git clean -xdf under the emacs-libvterm repository to remove the already generated vterm-module.so, then restart Emacs, the following window will be triggered and if I hit y Emacs will freeze at the compiling step again:

In this case, if I hit C-g twice, and then run M-x straight-rebuild-package RET vterm RET will work.

If I hit n, then run M-x straight-rebuild-package RET vterm RET will work.

I also tried the following lisp code, but it didn't work:

`("bash" "-c" "cd \"$1\" && cmake .. && make" "--"  ,(concat (straight--repos-dir "emacs-libvterm") "build"))

Based on my tries, when putting the backquote symbol at the beginning of the whole :pre-build block works as shown below:

(use-package vterm
  :straight `(
	     :pre-build (
			 ("rm" "-fr" "build")
			 ("mkdir" "build")
			 ("bash" "-c" "cd \"$1\" && cmake .. && make" "--"  ,(concat (straight--repos-dir "emacs-libvterm") "build"))
			 )))

See here and here for the relevant discussions.

Again, the problem seems to be when call-process is invoked from within your Emacs session.
Please seek advice for that problem on a more general forum.

Here I try to understand the problem that I cannot understand as far as possible, which will help me to further describe and locate the problem in other places. I would like to thank you again for your help and careful analysis.

[hongyi-zhao]

I have provided several ways to diagnose this type of problem in the issue you filed about auctex freezing. Please try those methods.

The :pre-build commands aren't your issue. The issue seems to be that your system freezes when trying to execute certain programs through call-process (assuming the information you gave in the thread about auctex freezing is accurate).

This indicates that the issue is most likely not due to straight.el. I would seek support on one of the appropriate channels I mentioned (reddit, IRC, mailing list). Again, this looks similar to the issue you were having with auctex. If you were able to fix that issue, I would follow a similar process. Unfortunately, since I am unable to reproduce this issue I don't think I'll be able to offer any more support.

This sounds a lot like an earlier issue you submitted about auctex:

#836

Which, unless I'm mistaken, ended up being a configuration issue and not an issue with straight. I would follow the same process outlined in that issue. Try to eliminate as many variables as possible and then add one variable at a time to see if/when the issue crops up. I would start by seeing if you can manually compile the module from the command line first.

Really. I confirmed your judgement. The problem discussed here belongs to the same type of the auctex installation with straight. The configuration I've posted here also has the same problem as this issue.

I verified this conclusion by the following method: First run the following commands under ~/.emacs.d/straight/repos/auctex:

$ git reset --hard
$ git clean -xdf
$ git pull

Then restart Emacs with the auctex configuration commented here. As a result, Emacs won't compile auctex automatically, till I run the following command: M-x straight-rebuild-package RET auctex RET.

P.S. All the above testings are done with the git master version of straight.el.

[hongyi-zhao]

(yodel
  :save "vterm.straight"
  :formatter yodel-format-as-github-markdown
  :packages*
  (vterm
   :post-build
   (progn
     (setq vterm-always-compile-module t)
     (require 'vterm)))
  :post*
  (let
      ((default-directory
        (straight--build-dir "vterm")))
    (print
     (straight--process-output "ls" "-lah" "vterm-module.so"))))

What's the meaning of :post* used above?

[progfolio]

P.S. All the above testings are done with the git master version of straight.el.

In general, please follow the issue template's instructions when filing an issue.
You should always try to reproduce errors on the "develop" branch because it is ahead of the master branch.
The fix may already be present on "devleop". This saves everyone involved time.

What's the meaning of :post* used above?

Off topic, but see the documentation of the yodel macro. M-x describe-function yodel.

[hongyi-zhao]

In general, please follow the issue template's instructions when filing an issue.

Thank you for your suggestion. I'll try to do so and using your yodel package if possible.

You should always try to reproduce errors on the "develop" branch because it is ahead of the master branch.
The fix may already be present on "devleop". This saves everyone involved time.

But it doesn't have the "develop" branch at all:

werner@X10DAi-00:~/.emacs.d/straight/repos/straight.el$ git log -1
commit af5437f2afd00936c883124d6d3098721c2d306c (HEAD -> master, origin/master, origin/HEAD)
Author: Nicholas Vollmer <44036031+progfolio@users.noreply.github.com>
Date:   Mon Sep 27 13:53:05 2021 -0400

    Document straight-base-dir and friends (#853)
    
    See: #852
werner@X10DAi-00:~/.emacs.d/straight/repos/straight.el$ git branch -v
* master af5437f Document straight-base-dir and friends (#853)

[progfolio]

I'll try to do so and using your yodel package if possible.

Stick with straight-bug-report. Yodel uses my custom fork of straight.el.

But it doesn't have the "develop" branch at all:

You need to (setq straight-repository-branch "develop") prior to the bootstrap code.
In a straight-bug-report, this would be after the :pre-bootstrap keyword.

[hongyi-zhao]

Stick with straight-bug-report. Yodel uses my custom fork of straight.el.

Thank you for your comment.

You need to (setq straight-repository-branch "develop") prior to the bootstrap code.

I now use the following configuration immediately prior to the bootstrap code of straight.el:

(setq straight-check-for-modifications '(check-on-save find-when-checking)
         straight-repository-branch "develop")

Then I remove the already cloned straight.el repository:

$ rm -fr ~/.emacs.d/straight/repos/straight.el

Finally, I restart Emacs to re-install the "develop" branch of straight.el, and see the following for the ultimate result:

werner@X10DAi-00:~/.emacs.d/straight/repos/straight.el$ git branch -v
* develop 92d4153 Remove alias for straight-package-neutering-mode (#887)
  master  af5437f Document straight-base-dir and friends (#853)

BTW, the following command can also do the same job:

$ git switch develop

In a straight-bug-report, this would be after the :pre-bootstrap keyword.

Thank you for your hint.

[progfolio]

straight-check-for-modifications '(check-on-save find-when-checking)

This is also probably why straight is not automatically rebuilding packages unless you call straight-rebuild-package. See the documentation of the straight-check-for-modifications variable.

[hongyi-zhao]

Wonderful. I really admire your insight. Using the default value of straight-check-for-modifications, i.e., (find-at-startup find-when-checking), the package auto-rebuilding now works properly when Emacs starts.