Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: multiple root folders #61

Closed
trollusk opened this issue Jul 22, 2022 · 26 comments · Fixed by #93 or #105
Closed

Feature request: multiple root folders #61

trollusk opened this issue Jul 22, 2022 · 26 comments · Fixed by #93 or #105
Assignees
Labels
enhancement New feature or request settings
Milestone

Comments

@trollusk
Copy link

Is there a reason that workspaceSidebar.folder only allows a single folder in which to look for workspaces? I'm sure lots of people don't have every project they work on located in a single root folder. Could this setting be converted to a list, or could it accept multiple paths separated by semicolons?

@sketchbuch
Copy link
Owner

sketchbuch commented Jul 27, 2022

Yes, the reason is you set depth from a root project folder and find workspaces that way, I also do not have all my workspaces in one folder. The idea is to set a root folder where all your projects are within, what folder structure you use or how deep you go is up to you.

Could you explain why this is not sufficient for you? What is your usecase?

@trollusk
Copy link
Author

They just aren't, and that is my choice. I suppose I could make a bunch of symlinks to work around this limitation. I don't think setting my root folder as c:/ would work out very well.

@AtomicRobotMan0101
Copy link

AtomicRobotMan0101 commented Jun 11, 2023

I'm the same. My example:

~/
~/python
~/pi
~/bash
~/sites
~/ChatGPT

Each is, and has, its own project tree. I'd hoped this is what the extension was actually for :) :)

There is no way in hell I'm giving it ~/

(even if I do give it ~/, or ANY directory, the extension gives me the "Folder path is not a directory" error reported in Error #68)

Otherwise, I'm actually STUNNED that this extension isnt actually 98% popular... or even a feature of VSCodium.

@sketchbuch sketchbuch self-assigned this Jun 11, 2023
@sketchbuch
Copy link
Owner

I will consider if I do this. It is a breaking design change and would require a considerable rewrite.

@Rykarix
Copy link

Rykarix commented Oct 5, 2023

I'd also like to +1 request this.

My projects folder is split up into sub-folders. It's not designed like a git repo

@sketchbuch
Copy link
Owner

OK, as several users want this I will try and impliment it. Can't say how quickly this gets done, but now that file icons are finished I could devote more time to it.

@sketchbuch
Copy link
Owner

@Rykarix Would you be able to test a new version with multiple root folders?

@Rykarix
Copy link

Rykarix commented Oct 30, 2023

@sketchbuch Sorry for the delay. Do you still need me to test?

If so send me the vsix of this fork and I'll try it :)

@sketchbuch
Copy link
Owner

sketchbuch commented Oct 30, 2023

@Rykarix Please download 2.0.0-beta

The folder setting is deprecated and will no longer be visible in settings. There is a new setting that needs adding to the settings json.

The older folder will be used, if the old setting is found and the new setting has not been set.

@Rykarix
Copy link

Rykarix commented Oct 31, 2023

@Rykarix Please download 2.0.0-beta

The folder setting is deprecated and will no longer be visible in settings. There is a new setting that needs adding to the settings json.

The older folder will be used, if the old setting is found and the new setting has not been set.

Summary

Doesn't work if workspaceSidebar.rootFolders array < 2
Has issues, see below

Settings used:

settings in user settings.json:

    "workspaceSidebar.rootFolders": [
        "C:\\Users\\Rykari\\PyProj",
        // "C:/Users/Rykari/PyProj",
    ],
    "workspaceSidebar.depth": 25,

Testing

Testing yields the following issues:

  • Default depth is set to 0 which may confuse some users. I'd recommend setting it to 5 or something
  • As you can see I tried different string formats.
  • When only 1 element in the array it doesn't work:
    • Tried with both / and \.
    • Tried with a leading / and \ in the path string, IE "..PyProj/" and "..PyProj\"
    • Tried with and without a comma
  • When More than 1 element is in the workspaceSidebar.folders array it works however:
    • Does not detect deeper levels once it has found a workspace file.

Additional testing

Since I know python, I threw together a highly inefficient but working function to detect all occurrences of extension ".code-workspace" using os.walk:

import os

def find_code_workspaces(directory, depth):
    workspace_files = []
    root_depth = directory.rstrip(os.path.sep).count(
        os.path.sep
    ) 
    max_depth = root_depth + depth

    for dirpath, dirnames, filenames in os.walk(directory):
        current_depth = dirpath.rstrip(os.path.sep).count(os.path.sep)
        if current_depth <= max_depth:
            for filename in filenames:
                if filename.endswith(".code-workspace"):
                    full_path = os.path.join(dirpath, filename)
                    workspace_files.append(full_path)
        else:
            del dirnames[:]

    return workspace_files

directory_to_search = "C:\\Users\\Rykari\\PyProj"
max_depth = 5

list_of_workspaces = find_code_workspaces(directory_to_search, max_depth)
print(list_of_workspaces)

Output:

['C:\\Users\\Rykari\\PyProj\\.DATASCIENCE\\ds-databricks\\ds-databricks.code-workspace',
 'C:\\Users\\Rykari\\PyProj\\.LEARNING\\UDEMY\\Python SDK for Azure Bootcamp\\.vscode\\Python SDK for Azure Bootcamp.code-workspace',
 'C:\\Users\\Rykari\\PyProj\\[workspace]\\projects - Copy.code-workspace',
 'C:\\Users\\Rykari\\PyProj\\[workspace]\\projects.code-workspace',
 'C:\\Users\\Rykari\\PyProj\\[workspace] - Copy\\projects - Copy.code-workspace',
 'C:\\Users\\Rykari\\PyProj\\[workspace] - Copy\\projects.code-workspace']

Your current code only returns:

 'C:\\Users\\Rykari\\PyProj\\[workspace]\\projects - Copy.code-workspace',
 'C:\\Users\\Rykari\\PyProj\\[workspace]\\projects.code-workspace',
 'C:\\Users\\Rykari\\PyProj\\[workspace] - Copy\\projects - Copy.code-workspace',
 'C:\\Users\\Rykari\\PyProj\\[workspace] - Copy\\projects.code-workspace']

Even though "workspaceSidebar.depth": 5 is set.

Hope this helps

@Rykarix
Copy link

Rykarix commented Oct 31, 2023

Also, Had to manually click "Reload Workspaces" instead it refreshing when a new setting has been updated or saved.

@sketchbuch
Copy link
Owner

sketchbuch commented Oct 31, 2023

It is manual refreshing now, the cache timeout has been removed. But there are one or two config properties that cause recollection automatically - see readme.

I think your issues maybe windows related. One or more root folders works under linux/mac. I haven't tested yet with windows and it is slow as I only have a vm but I will start checking and see why it is not working for you.

So finding folders excludes hidden folders - ones starting with a dot which is why you can't see some of your workspaces. I had to allow .vscode as some users added workspaces there. Now that exclude folders are exposed in config I could remove this filtering and leave it up to users to decide if they want to exclude hidden folders with the exclude folders config.

@Rykarix
Copy link

Rykarix commented Nov 3, 2023

It is manual refreshing now, the cache timeout has been removed. But there are one or two config properties that cause recollection automatically - see readme.

I think your issues maybe windows related. One or more root folders works under linux/mac. I haven't tested yet with windows and it is slow as I only have a vm but I will start checking and see why it is not working for you.

So finding folders excludes hidden folders - ones starting with a dot which is why you can't see some of your workspaces. I had to allow .vscode as some users added workspaces there. Now that exclude folders are exposed in config I could remove this filtering and leave it up to users to decide if they want to exclude hidden folders with the exclude folders config.

Sounds good. Reply to any of my threads with a link to the vsix you want me to test when you think you have a working build & I'll try to be as helpful as I can :)

@sketchbuch
Copy link
Owner

sketchbuch commented Nov 16, 2023

@Rykarix please test https://github.com/sketchbuch/vsc-workspace-sidebar/files/13374439/vsc-workspace-sidebar-2.0.0-beta-2.zip

This should fix the issue with hidden folders not being looked into and not working if only 1 root folder. There are some issues to fix to do with closing folders but basically the job is done. I tested in my windows vm and it worked fine.

Screenshot from 2023-11-13 20-14-01

@sketchbuch
Copy link
Owner

Will be in version 2.0.0

@Rykarix
Copy link

Rykarix commented Nov 27, 2023

@Rykarix please test https://github.com/sketchbuch/vsc-workspace-sidebar/files/13374439/vsc-workspace-sidebar-2.0.0-beta-2.zip

This should fix the issue with hidden folders not being looked into and not working if only 1 root folder. There are some issues to fix to do with closing folders but basically the job is done. I tested in my windows vm and it worked fine.

Screenshot from 2023-11-13 20-14-01

Still doesn't work.

Also throws an error if settings is:

    "workspaceSidebar.rootFolders": [
        "${env:USERPROFILE}\\PyProj",
    ],
    "workspaceSidebar.depth": 5,
    "workspaceSidebar.showFolderHierarchy": true,

Let me know if there's any log file I can gen for you

@sketchbuch
Copy link
Owner

sketchbuch commented Nov 27, 2023

env:USERPROFILE will not work, unless vscode itself internally does something with that variable. I don't do anything with them as I never expected them to be there. You can use ~/ for your home directory.

I will create a new version of the vsix. I have tested on mac,linux and windows and it is working with 1 or more paths. Why it isn't for your paths I can't say at the moment.

@sketchbuch sketchbuch reopened this Nov 27, 2023
@Rykarix
Copy link

Rykarix commented Nov 27, 2023

Ah, fair enough, I modified it to ~/PyProj

However it's still not getting all of the .code-workspace files unfortunately

If there's a quick way to generate log files I'll try to look into it?

EDIT
I think the issue is related to the . that I added to some of the folders.

So for eg,

    "workspaceSidebar.rootFolders": [
        "~/PyProj",
        "~/PyProj/.DATASCIENCE"
    ],

I have 2 code-workspace files in .DATASCIENCE

When I add this folder like this it finds them.

So I think this goes back to what you mentioned before about if there's a '.' in the folder string somewhere

@sketchbuch
Copy link
Owner

I have identified the issue, and will release a fix by the end of the week. Maybe you can test a new version then? I shall update this ticket when ready.

@Rykarix
Copy link

Rykarix commented Dec 7, 2023

No rush :)

@sketchbuch
Copy link
Owner

sketchbuch commented Dec 8, 2023

@Rykarix Please try: https://github.com/sketchbuch/vsc-workspace-sidebar/files/13614381/vsc-workspace-sidebar-2.0.0-beta-4.zip

@sketchbuch
Copy link
Owner

Screenshot from 2023-12-08 14-18-22

@sketchbuch sketchbuch added this to the v2.0 milestone Dec 12, 2023
@Rykarix
Copy link

Rykarix commented Dec 18, 2023

@Rykarix
Copy link

Rykarix commented Dec 18, 2023

@Rykarix Please try: https://github.com/sketchbuch/vsc-workspace-sidebar/files/13614381/vsc-workspace-sidebar-2.0.0-beta-4.zip

Still doesn't work and still gives different results depending on the string passed into rootFolders:

    "workspaceSidebar.rootFolders": [
        // "~/PyProj",
        "C:\\Users\\Rykari\\PyProj",
    ],

To help you out, I've created some python code and made it efficient (because that's the language I'm far more experienced with). Once I was comfortable with the efficiency I then converted it to javascript

My python code:

import json
import logging
import os
from pathlib import Path

from rich.logging import RichHandler

logging.basicConfig(
    level=logging.INFO,
    format="%(message)s",
    handlers=[RichHandler(show_level=False, show_path=False, show_time=False)],
)


def log_pretty_list(data: list):
    formatted_data = json.dumps(data, indent=4)
    logging.info(f"\n{formatted_data}")


def limited_walk(path: Path, max_depth: int):
    path = Path(path)
    root_depth = len(path.parts)
    for dirpath, dirnames, filenames in os.walk(path):
        current_depth = len(Path(dirpath).parts)
        if current_depth <= root_depth + max_depth:
            yield dirpath, dirnames, filenames
        else:
            dirnames.clear()  # Prevent descending into subdirectories


def find_code_workspaces(directory: Path, depth: int) -> list:
    if not directory.is_dir():
        raise ValueError(f"Provided path '{directory}' is not a valid directory.")

    workspace_files = []
    for dirpath, _, filenames in limited_walk(directory, depth):
        for filename in filenames:
            if filename.endswith(".code-workspace"):
                full_path = Path(dirpath, filename)
                relative_path = full_path.relative_to(directory)
                workspace_files.append(
                    (Path("~/PyProj") / relative_path).as_posix(),
                )

    return workspace_files


directory_to_search = Path("~/PyProj").expanduser()
max_depth = 5

list_of_workspaces = find_code_workspaces(directory_to_search, max_depth)
log_pretty_list(list_of_workspaces)

Converted to JS:

const fs = require('fs/promises');
const path = require('path');
const os = require('os');

function normalizePath(userPath) {
    if (userPath.startsWith('~')) {
        return path.join(os.homedir(), userPath.slice(1));
    }
    return path.resolve(userPath);
}

async function* limitedWalk(dir, maxDepth) {
    async function* walk(currentPath, currentDepth) {
        if (currentDepth > maxDepth) return;
        const files = await fs.readdir(currentPath, { withFileTypes: true });

        for (const file of files) {
            const resolvedPath = path.resolve(currentPath, file.name);
            if (file.isDirectory()) {
                yield* walk(resolvedPath, currentDepth + 1);
            } else if (file.isFile() && file.name.endsWith('.code-workspace')) {
                yield resolvedPath;
            }
        }
    }
    yield* walk(dir, 0);
}

async function findCodeWorkspaces(directory, depth) {
    if (!(await fs.stat(directory)).isDirectory()) {
        throw new Error(`Provided path '${directory}' is not a valid directory.`);
    }

    const workspaceFiles = [];
    const isAbsolutePath = path.isAbsolute(directory);

    for await (const filePath of limitedWalk(directory, depth)) {
        let formattedPath;
        if (isAbsolutePath) {
            formattedPath = filePath.replace(/\\/g, '/');
        } else {
            const relativePath = path.relative(directory, filePath);
            formattedPath = relativePath.replace(/\\/g, '/');
        }
        if (formattedPath.endsWith('.code-workspace')) {
            workspaceFiles.push(formattedPath);
        }
    }
    return workspaceFiles;
}


async function main(directoryInput, maxDepth) {
    const directoryToSearch = normalizePath(directoryInput);

    try {
        const listOfWorkspaces = await findCodeWorkspaces(directoryToSearch, maxDepth);
        console.log(JSON.stringify(listOfWorkspaces, null, 4));
    } catch (error) {
        console.error(error.message);
    }
}

main('~/PyProj', 5); // 200 - 300ms on my laptop
// main('C:\\Users\\Rykari\\PyProj', 8); // 2 - 3s on my laptop
// main('C:/Users/Rykari/PyProj', 15); // 4 - 6s on my laptop

This should give you a starting point for solving this back and forth.

You now have a function that

  1. Correctly handles multiple path strings 'styles'
  2. Should work regardless of operating system
  3. Is fairly performant

I'm sure you could improve the JS performance but honestly I think it'd be fair to set a hard limit of 10 or 15 because I don't think it should be on you to support depths at this level. It's unecessary.

@sketchbuch
Copy link
Owner

sketchbuch commented Dec 18, 2023

Thank for the reply. I will have to think about how to reproduce your issue. The version above worked on linux, mac, and windows (vm) when I was testing it (with workspaces in hidden folders).

As I don't have access to Windows I use a VM but if it works in a VM I expect it to work on the real OS. If anyone else reading this uses Windows, feel free to test and give feedback here.

I have noticed since getting rid of the check that filters out hidden folders that performance is slow but this can be fixed by adding to the excluded folders.

It was so much slower that I added a checkbox setting to exclude all hidden folders which will be enabled by default as most users will not be putting workspaces in hidden folders.

Since this is difficult to see what folders it is checking I have new issue to make some kind of logger when collecting files so that people can see what folders are being collected and see potential folders that could be excluded.

I will update the readme to include better documentation on the folder paths. They should be absolute (relative was never supported), not include any variables. MAybe this is why it is not working for you?

I will keep you updated, there are several new features that will make it into 2.0.0

Re the depth, unfortunately I can't lower the depth as several users were not happy with the old max (10) and asked for it to be increased, so I know some users do want more than 10 levels searched. I can't see why either, but appearently they do.

I will release a point upgrade to fix an issue with excluded files in the current marketplace version. I was hoping to release the 2.0.0 (which has the fix) but it is taking longer than I expected as there are several other issues to add.

@sketchbuch
Copy link
Owner

Done in v2.0.0

  • More robust checking of folders
  • excludeHiddenFolders and depth overridable per folder
  • Better error messages
  • Handles symlinks which were ignored before

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request settings
Projects
None yet
4 participants