default directory changed?

[jabranham]

Before the commits a few days ago, ESS would start the R process in the directory of the file (with ess-ask-for-ess-directory set to nil). Now, it starts the R process wherever a .git folder is found, it seems? This causes issues for me because $HOME is version-controlled, so ESS will only start the R process in $HOME (or a git-controlled subfolder).

Is it possible to restrict this new behavior to only set the working directory to the project root for R packages, not random R scripts?

[vspinu]

@lionel- shall we restrict the change to R packages only? I think this issue proves the point. You just cannot anticipate users' patterns. Nor you can anticipate what projects "project.el" will detect in the future. Introducing new custom variable for this is not a good idea either.

On a general note, users of ESS and emacs are so used to the idea of navigating to a dir and starting process there that this "auto-pwd" behavior is likely to be a real nuisance. We don't really fix anything by this new "feature", but are very likely to break things.

[richardsc]

I second the comment by @vspinu -- I haven't updated ESS in a few weeks, but I can see this behavior driving me nuts for non-R package development scenarios (probably 85% of my ESS/R use cases), as I often work in subfolders of a git controlled directory.

[lionel-]

yeah, I guess that's something to consider, at least making the project thing optional.

You can also add projectile to project-find-functions and then make sure that all your projects have a .projectile file. It seems that would improve the workflow of users. Also you can advice project-current so that $HOME is not considered a project. Finally, the project module is in its early stages so you can be sure that there will be user variables in the future to exclude paths from projects just like projectile has.

I think that developing a project oriented workflow in ESS, especially for process management, will help users more than hinder them in the long term. I know of users who just quit using ESS because project workflow wasn't working out for them. So while we have to hear out users who have to change their workflow or habits, we also have to think of users who quit using ESS because of this. I feel this issue smells a bit like that Wald story about returning aircrafts during world war II: https://en.wikipedia.org/wiki/Abraham_Wald

Now personally I only use ESS for package development and I'm very happy if other people use RStudio. I just think that project-oriented workflow is the right fit for ESS and Emacs. I couldn't use Emacs without projectile nowadays.

Of course, I have to finish this comment by mentioning the obligatory https://xkcd.com/1172/

[lionel-]

You can also set project-find-functions to nil in your init file by the way. It just happens to have the version control (git, svn) finder by default.

Also as more of Emacs integrates with projects you'll find that $HOME problem reoccuring (and as mentioned above that problem has solutions).

Edit: That said we should probably make that feature optional while both Emacs and ESS project management are maturing.

[vspinu]

I think that developing a project oriented workflow in ESS, especially for process management, will help users more than hinder them in the long term

No-one questions that. We are not talking about project management in general, just about this particular feature. It's just that this particular features does a bit too much. Most of the file and directory specific tools assume current directory and we should do the same, unless there is a compelling reason to do otherwise. R packages is such a reason, arbitrary .git roots is not.

'm very happy if other people use RStudio.

Does RStudio start processes in arbitrary root project, or only in its own projects?

Now personally I only use ESS for package development

R package development not arbitrary packages, right?

We are spending too much time on this feature. It's a minor tweak and it's not worth it, especially that we don't have a consensus on it. Let's just add a variable for this and set it to 'r-packages-only by default. If someone needs a more generic project support they will find it.

[vspinu]

BTW, Bozhidar, the author of projectile is not particularly happy with project.el, so there is no consensus about that project either.

[lionel-]

I think you are wrong that it is a minor tweak. It is a big deal to have ESS start the process in the right directory (after you have set up your project detection correctly, e.g. create subprojects where needed if the R project is included in a wider project). I see it as the start of a more integrated notion of project in ESS. It needs two sides: folder startup selection and process management after that. These two things are small tweaks in a sense, but they have big consequences (positive imo) in terms of workflow.

arbitrary .git roots is not.

I think they are. It makes sense to version control your data analysis projects. The alternative is to come up with our own notion of projects in ESS and I think that's not the right direction.

Bozhidar, the author of projectile is not particularly happy with project.el, so there is no consensus about that project either.

I know that not everyone is happy about project.el but Emacs needs a generic way of dealing with projects and project.el is what we have.

I agree that the project integration should be optional and disabled by default for now.

cc @dgutov who I know has been wondering about wider usage of Emacs projects.

[vspinu]

The alternative is to come up with our own notion of projects in ESS and I think that's not the right direction.

No, there is no such alternative. You are constantly stirring the discussion in an unrelated direction. Project management is a hard topic and we are not discussing it here. So, let's please stick to the topic - "In which directory should R start by default?"

No other emacs based IDE that I am aware of (slime, cider, geiser, run-python, run-xyz etc) starts its process in arbitrary vc root. Nor does RStudio start in arbitrary vc root. So let's not do it in ESS either. I can only guess where you got this idea from, as it's not even required by your own workflow.

I am not asking you to revert the change, just please add a variable to customize the behavior. If other IDE's start doing that, we will consider it. Till then, I propose to stick to the standard emacs-wide behavior and to what people expect most. You are in a minority regarding this issue. So, can we please go with the democratic vote on this?

[dgutov]

the author of projectile is not particularly happy with project.el

If people are content with "not being happy" and doing nothing about it, there is not much I, as an Emacs developer, can do. You can notice that Bozhidar doesn't offer any particular critique.

[vspinu]

It is a big deal to have ESS start

It's not. If ESS does the right thing 99% of the time and in 1% of cases you need to press once backspace to reach parent directory, it's not such a big deal at all. In most cases you will navigate to the directory or file first and then start the process there. This is the current, intuitive, the simplest and recommended use pattern. No need to outsmart the user here.

[dgutov]

@vspinu

I do believe third-party packages should start using project.el, but maybe you don't need the project root here anyway. Or you can detect it more reliably knowing R's particular project structure.

In the latter case, ESS can define its own project type and add to project-find-functions. Any function in ESS, however, can call its project-finding code directly, without going through that hook. This way, you can will always be sure which project implementation is used.

No other emacs based IDE that I am aware of (slime, cider, geiser, run-python, run-xyz etc)

Since you're wondering, inf-ruby-console-auto does start in a project root (not a VC root, though).

[lionel-]

You are constantly stirring the discussion in an unrelated direction. Project management is a hard topic and we are not discussing it here.

Vitalie, it has been about projects from the start, why else would I integrate with project.el?

Also please remain considerate, you are being overly dismissive and frankly snarky. Your comments about being in the minority and democratic votes do not make sense unless you have been conducting polls that I am not aware of. Finally, note that I agreed from the start of this thread that these new features should be optional. They should also be marked as experimental.

Back to the point, project management is indeed hard so the crux of the matter is how do we want to deal with project genericity in Emacs. The way I see it, project managers such as projectile can hook up into project.el. Then as an ESS user that happens to use projectile you can add a .projectile file in a project and subproject and have it work in projectile of course, but also in ESS. That leads to a simple workflow and improves the Emacs experience.

Maybe the right way of dealing with genericity is the other direction, that is to develop a notion of ESS project and hooking that up into project.el. Then projectile would make requests to project.el? It seems to make less sense to me. That said I think we should deal with projects in ESS in such a way that both approaches are feasible so we can take a final decision later on.

No need to outsmart the user here.

Maybe you find the current state of things satisfying but I don't. To me the major principles of Emacs workflow are mode-wise genericity and doing the right thing most of the time. Currently ESS process management gets in my way. I am trying to fix this in a way that makes sense for data analysis users as well as for package authors. If that does not work out for everyone then I agree it should be optional. I do not have malicious intents so please take it easy.

[vspinu]

Since you're wondering, inf-ruby-console-auto does start in a project root (not a VC root, though).

@dgutov, Yes. Thats our behavior by default and we had a project detection since long. We will also do our best to integrate with package.el. The change that we are discussing here is to generalize the behavior of process start-up to arbitrary project roots (not just R).

@lionel- It's not my intend to harm your feelings by all means. I do really appreciate all your contributions to this project and I think we are all very happy of having you here. I am just sensing a pushy trend in your recent proposals and commits that are overly-centric on your use patterns but interfere with ages of usage of Emacs and ESS. All I want is to be "considerate" to our users and add breaking changes only if we have a consensus on that.

I can assure you that issues that ESS or RStudio are facing with regard to process management are relatively simple ones. RStudio cannot even run multiple processes in one session;) In context of clojure/clojurescript projects one needs to deal with multiple session, each with multiple connections/repls, each potentially dealing with multiple project directories. A generic process management solution should take into account those scenarios and we won't be reaching one here.

Back to the feature. I never liked it and I gave concrete and clear-cut arguments against it. But those seem to be flying past you. By "minority" I meant people in this thread and emacs-ide tooling, none of which does what you have implemented in ESS.

note that I agreed from the start of this thread that these new features should be optional. They should also be marked as experimental.

This wasn't clear to me. All I sensed is a pushy desire for change and I am sorry if I misread your wording. We are on the brink of new release and any sort of experimental feature should not be in the master at this moment.

[lionel-]

I didn't know this feature would cause such a rufus and in retrospect it was indeed too late in the release process to introduce this, I should have marked it as experimental from the beginning. I thought it would be non-intrusive because it's only enabled on Emacs 25.1 and only concerns process startup, but I was mistaken.

But those seem to be flying past you

You also said that I had a good point regarding project roots, e.g. #457 (comment)

that are overly-centric on your use patterns

This is unfair. You said yourself that as a package-only user I don't need a broader notion of project. I am basing these changes on my experience of Emacs but also because I think it is the right direction for ESS. Emacs project management has been lacking for decades (except for third party packages like projectile) so I don't think it is an argument to bring up how it's done by all other tools. This is an area that needs to be improved.

I'll disable this feature and remove its mention from the news file and we can discuss this further later on.

[mmaechler]

I strongly concur with @vspinu notably because we are aiming for a release in one week or so !!!
If you really want you create a branch that is not going to be merged into master before release.

I really really would be much much much more happy if someone would address issue #427 (and yes, I know it's not exactly netiquette to misuse one issue for raising another unrelated one..)

[vspinu]

You also said that I had a good point regarding project roots, e.g. #457 (comment)

Well, I liked your point on "good practice for scripts to execute relative to the project root" . Nudging people into organizing data science projects hierarchically is not a bad idea. But it turned to be a bit too much of a nudge I am afraid. Plus I have now extra doubts about the UI because I have now spent some extra brain cycles on it. But well, let's discuss it elsewhere.

Maybe the right way of dealing with genericity is the other direction, that is to develop a notion of ESS project and hooking that up into project.el.

Are we picking Rproj somewhere? How about instead of VC roots we pick Rproj files only (and DESCRIPTION)? Such a change would be entirely backward compatible. It might mess up Rstudio though.

[lionel-]

How about instead of VC roots we pick Rproj files only

Maybe. Again my goal was not to pick up VC roots per se but whatever the user uses for generic project detection. It just happens that project.el picks up VC roots by default (which I think makes sense).

My first intuition is that ESS should not be creating Rproj files. But maybe it'd make sense to have our own project config file? I think that existing project management facilities (e.g. .projectile file) + dir-local config files is all ESS needs. Let's keep thinking about this.

[vspinu]

My first intuition is that ESS should not be creating Rproj files. But maybe it'd make sense to have our own project config file?

No, no. ESS should never create it. Just pick the location. So, if ESS sees such a marker in the path it knows to auto-start there. I think that if we try to piggyback on markers which were not designed specifically for process-auto-start purpose, we are bound to run in issues as the OP. So, there might not be other way but to introduce an ".Rroot" marker specifically intended for the auto-start. I think this might work even right now by adding emtpy DESCRIPTION in a directory.

[Currerius]

Is it still true that ESS no longer opens the current directory as working directory? ESS opens the parent directory when opened in a .git repo - kind of strange.
(setq project-find-functions nil )
did not change that.

[lionel-]

Current issue at #468

[Currerius]

Wow, that was quick. Thx.
I followed #468 and try ess-ask-for-ess-directory t in the mean time and it works!
Good enought

BR