Consider deleting our old kernelspecs

[joyceerhl]

Some of them are generated incorrectly. We should clean them all up for good.

[greazer]

Make sure we address #7179, before we do this.

[greazer]

Suggested course of action, since it's impossible for us to tell whether a kernel is validly created without running it.

1. For this release don't delete them but move them to a place we know about but that Jupyter notebook/lab doesn't. Would be best if this could be a subfolder of the kernels folder called something like "VSCodeJupyterTemp". That way if somebody found it they could delete it themselves.
2. If a user was exclusively using VS Code to do their notebook work, then when they open a notebook that was created with one of those kernels, it will "just work". Whether we create a corresponding kernelspec.json (due to RAW vs non-RAW) won't matter.
3. If a user was using the kernel in Jupyter notebook and opens a notebook that was set to it, one of two things can happen:
   a. User doesn't really notice or care. They select an appropriate kernel that IS there and go on their merry way. It could be they end up selecting a kernel/environment that isn't set up for the given notebook. They might grumble a bit but will work through the problem on their own.
   b. User TOTALLY notices and enters an angry issue.

For occurrence b, is there a proposed workaround for how they can get back to a working state?

[DonJayamanne]

b. User TOTALLY notices and enters an angry issue.

Question: How would the user know the VSCode Jupyter extension removed this & file an angry issue.
If they are now using Jupyter notebook/lab, they'd grumble and complain about that application.
I'm not saying this is a good thing at all, but saying that its a big leap to make.

Again, we're deleting kernels that we created in jupyter extesnion to get things working in VS Code (in non-raw, which is a very small % of users).

My assumption is users don't need those kernels in jupyter notebook/lab (its purely accidental that they happen to use them. I.e. if they didn't have VS Code, such kernels wouldn't exist). If they did then would they complain if they don't have the same kernels appearing in jupyter lab/notebook on another machine (which supports raw). I don't think so, & if they did, then we need to explain that these kernels are created only in a specific scenario (non-raw) on specific machines.

[rchiodo]

I'm not sure this worked. I still have the directories in my 'old' kernelspecs.

[DonJayamanne]

I still have the directories in my 'old' kernelspecs.

What do you mean by this. PLease could you share the folder structure.

[rchiodo]

Jupyter kernelspec list still lists one of them:

Available kernels:
  python3                                                                                  C:\Users\aku91\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\ipykernel\resources
  .net-csharp                                                                              C:\Users\aku91\AppData\Roaming\jupyter\kernels\.net-csharp
  .net-fsharp                                                                              C:\Users\aku91\AppData\Roaming\jupyter\kernels\.net-fsharp
  .net-powershell                                                                          C:\Users\aku91\AppData\Roaming\jupyter\kernels\.net-powershell
  .venvkernel                                                                              C:\Users\aku91\AppData\Roaming\jupyter\kernels\.venvkernel
  .venvnokernel                                                                            C:\Users\aku91\AppData\Roaming\jupyter\kernels\.venvnokernel
  ir                                                                                       C:\Users\aku91\AppData\Roaming\jupyter\kernels\ir
  julia-1.6                                                                                C:\Users\aku91\AppData\Roaming\jupyter\kernels\julia-1.6
  python395jvsc74a57bd0d02b32c6f124efe7f1cc215e800019f4c678af31d66bc7ea9b433a707274158e    C:\Users\aku91\AppData\Roaming\jupyter\kernels\python395jvsc74a57bd0d02b32c6f124efe7f1cc215e800019f4c678af31d66bc7ea9b433a707274158e

The last one in the list looks like one of ours.

That directory has this in it:

Directory of C:\Users\aku91\AppData\Roaming\jupyter\kernels\python395jvsc74a57bd0d02b32c6f124efe7f1cc215e800019f4c678af31d66bc7ea9b433a707274158e

06/02/2021  02:42 PM    <DIR>          .
06/02/2021  02:42 PM    <DIR>          ..
06/22/2021  03:12 PM             6,911 kernel.json
03/08/2021  05:36 PM             1,084 logo-32x32.png
03/08/2021  05:36 PM             2,180 logo-64x64.png
               3 File(s)         10,175 bytes
               2 Dir(s)  735,681,708,032 bytes free

That's definitely an 'old' one as it has an interpreter in it:

(release-2021.10.100) C:\Users\aku91\AppData\Roaming\jupyter\kernels\python395jvsc74a57bd0d02b32c6f124efe7f1cc215e800019f4c678af31d66bc7ea9b433a707274158e>type kernel.json
{
  "specFile": "d:\\Source\\Testing_3\\testEnv\\share\\jupyter\\kernels\\python3\\kernel.json",
  "interpreterPath": "d:\\Source\\Testing_3\\testEnv\\Scripts\\python.exe",
  "name": "python395jvsc74a57bd0d02b32c6f124efe7f1cc215e800019f4c678af31d66bc7ea9b433a707274158e",
  "argv": [
    "d:\\Source\\Testing_3\\testEnv\\Scripts\\python.exe",
    "-m",
    "ipykernel_launcher",
    "-f",
    "{connection_file}"
  ],
  "language": "python",
  "path": "python",
  "display_name": "Python 3.9.2  ('testEnv': venv)",
  "metadata": {
    "originalSpecFile": "d:\\Source\\Testing_3\\testEnv\\share\\jupyter\\kernels\\python3\\kernel.json",
    "interpreter": {
      "sysPrefix": "",
      "envType": "Venv",
      "envName": "testEnv",
      "envPath": "d:\\Source\\Testing_3\\testEnv",
      "path": "d:\\Source\\Testing_3\\testEnv\\Scripts\\python.exe",
      "architecture": 1,
      "sysVersion": "",
      "version": {
        "raw": "3.9.2",
        "major": 3,
        "minor": 9,
        "patch": 2,
        "build": [],
        "prerelease": []
      },
      "displayName": "Python 3.9.5 64-bit ('testEnv': venv)"
    }
  },

I also still have all of the old kernelspec directories:

(release-2021.10.100) C:\Users\aku91\AppData\Roaming\jupyter\kernels>dir /ad
 Volume in drive C is Windows
 Volume Serial Number is 9C59-9A5C

 Directory of C:\Users\aku91\AppData\Roaming\jupyter\kernels

10/20/2021  08:56 AM    <DIR>          .
10/20/2021  08:56 AM    <DIR>          ..
09/21/2021  02:17 PM    <DIR>          .net-csharp
09/21/2021  02:17 PM    <DIR>          .net-fsharp
09/21/2021  02:17 PM    <DIR>          .net-powershell
09/20/2021  04:20 PM    <DIR>          .venvkernel
09/20/2021  04:21 PM    <DIR>          .venvnokernel
07/01/2021  04:18 PM    <DIR>          ir
04/28/2021  12:45 PM    <DIR>          julia-1.6
09/23/2021  01:57 PM    <DIR>          python380jvsc74a57bd08d06b578fee6feaf7c36758381f71ee046b0b7658f0309c0328fc2e4c7ac3eb9
09/23/2021  01:57 PM    <DIR>          python388jvsc74a57bd05e8eefcd3fe358d9c8f573088a953aaf26c823c16859eee39d47a8065f309cbc
09/23/2021  01:57 PM    <DIR>          python395jvsc74a57bd080f97fa3b34780e22298c6b9d6b34f546adccdc25ebd9f49472acdb98af000e0
06/02/2021  02:42 PM    <DIR>          python395jvsc74a57bd0d02b32c6f124efe7f1cc215e800019f4c678af31d66bc7ea9b433a707274158e
09/23/2021  01:57 PM    <DIR>          pythonjvsc74a57bd09815dc2182e388e549254302fc1da5e478ed2cd2a907ea93363f940da7dbd695
10/20/2021  08:56 AM    <DIR>          __old_vscode_kernelspecs
               0 File(s)              0 bytes

[DonJayamanne]

@rchiodo Looks like the __old_vscode_kernelspecs directory is on your machine.
That means the kernel specs were deleted at one point. You may have uses non-raw kernels which resulted in those kernels getting re-created.

Note: We don't delete the kernels everytime, we only do it once. After that if it gets re-created, thats fine.
Please could you deltete your global cache and verify this again, i'm certian thats the problem.

[rchiodo]

Yeah I'm afraid that will mess up the whole thing.

From the dir data you can see that it was an old kernel spec:

Directory of C:\Users\aku91\AppData\Roaming\jupyter\kernels\python395jvsc74a57bd0d02b32c6f124efe7f1cc215e800019f4c678af31d66bc7ea9b433a707274158e

06/02/2021  02:42 PM    <DIR>          .
06/02/2021  02:42 PM    <DIR>          ..
06/22/2021  03:12 PM             6,911 kernel.json
03/08/2021  05:36 PM             1,084 logo-32x32.png
03/08/2021  05:36 PM             2,180 logo-64x64.png
               3 File(s)         10,175 bytes
               2 Dir(s)  735,681,708,032 bytes free

[DonJayamanne]

From the dir data you can see that it was an old kernel spec:

So what you have is, some kernelspecs have been deleted, and others haven't. Is that correct.
Please could you confirm the folder __old_vscode_kernelspecs is not empty.

Also what's interesting is the fact that we don't add those png files and the like.

[rchiodo]

Yes that is correct. Some were moved, one was not.

[DonJayamanne]

I don't think we need this anymore.
Suggestion we not focus on this anymore. We have 2 release that does this and that should be sufficient.

[greazer]

We decided we don't need to resolve this at all, per @DonJayamanne's suggestion.