VS Code: How to upload File

[xwcoder]

Code Server Web: How to upload file

[toc]

结论

根据 VS Code - transport protocol 的介绍, VS Code 支持大文件上传。
在支持 File.stream 的浏览器上采用分片上传。由于浏览器端 WebSocket 不支持帧级别的操作，
所以使用 WebSocket message 进行分片上传。

出于性能和稳定性考虑，VS Code 打开大文件需要用户确认。

可以设置需要确认的最小尺寸。

下面我们分析上传文件的过程，这里不会详细介绍代码级别的整条链路，也会忽略一些处理细节，比如缓存安全处理等。 原因如下：

• VS Code 项目本身很复杂，需要处理的情况非常多，所以调用链很长，
  详细梳理调用链路可能会迷失在细节里, 且容易造成混乱。
• 这里主要目的并不是分析 VS Code 的架构。

浏览器端选择文件的方式有两种：Input Element 和 Drag & Drop. 对于两种方式，后续的处理方式基本相同。

两条 WebSocket 连接

通常情况下 Code Server Web 启动时会创建两条 WebSocket 连接。代码级别分别命名为 Management 和 ExtensionHost.

// vs/platform/remote/common/remoteAgentConnection.ts
export const enum ConnectionType {
  Management = 1,
  ExtensionHost = 2,
  Tunnel = 3,
}

根据命名推断 Management 用于核心工作，如文件上传; ExtensionHost 用于扩展相关的工作。

Management 连接创建

// vs/workbench/browser/web.main.ts
export class BrowserMain extends Disposable {
  private async initServices(): Promise<{ serviceCollection: ServiceCollection; configurationService: IWorkbenchConfigurationService; logService: ILogService }> {
    // Remote Agent
    const remoteSocketFactoryService = new RemoteSocketFactoryService();
    remoteSocketFactoryService.register(RemoteConnectionType.WebSocket, new BrowserSocketFactory(this.configuration.webSocketFactory));
    serviceCollection.set(IRemoteSocketFactoryService, remoteSocketFactoryService);
    const remoteAgentService = this._register(new RemoteAgentService(remoteSocketFactoryService, userDataProfileService, environmentService, productService, remoteAuthorityResolverService, signService, logService));
    serviceCollection.set(IRemoteAgentService, remoteAgentService);
    this._register(RemoteFileSystemProviderClient.register(remoteAgentService, fileService, logService));
  }
}

// vs/platform/remote/browser/browserSocketFactory.ts
export class BrowserSocketFactory implements ISocketFactory<RemoteConnectionType.WebSocket> {
  connect({ host, port }: WebSocketRemoteConnection, path: string, query: string, debugLabel: string): Promise<ISocket> {
    return new Promise<ISocket>((resolve, reject) => {
      const webSocketSchema = (/^https:/.test(window.location.href) ? 'wss' : 'ws');
      path = (window.location.pathname + '/' + path).replace(/\/\/+/g, '/');
      const socket = this._webSocketFactory.create(`${webSocketSchema}://${(/:/.test(host) && !/\[/.test(host)) ? `[${host}]` : host}:${port}${path}?${query}&skipWebSocketFrames=false`, debugLabel);
      const errorListener = socket.onError(reject);
      socket.onOpen(() => {
        errorListener.dispose();
        resolve(new BrowserSocket(socket, debugLabel));
      });
    });
  }
}

UI 启动时会初始化一系列 service，之前介绍 VS Code Ioc 和 startup 时有相关介绍。其中会创建 Management WebSocket 连接。

上传入口

上传入口在 vs/workbench/contrib/files/browser/fileImportExport.ts

private async doUpload(target: ExplorerItem, source: IWebkitDataTransfer, progress: IProgress<IProgressStep>, token: CancellationToken): Promise<void> {
  // Open uploaded file in editor only if we upload just one
  const firstUploadedFile = results[0];
  if (!token.isCancellationRequested && firstUploadedFile?.isFile) {
    await this.editorService.openEditor({ resource: firstUploadedFile.resource, options: { pinned: true } });
  }
}

当只有一个上传文件时，上传后才会在编辑器打开。

当浏览器支持 File.stream 并且文件大于 1M 时使用 buffer 方式(分段)上传。

// Chrome/Edge/Firefox support stream method, but only use it for
// larger files to reduce the overhead of the streaming approach
if (typeof file.stream === 'function' && file.size > ByteSize.MB) {
  await this.doUploadFileBuffered(resource, file, reportProgress, token);
}

最终使用 FileService 进行上传

private async doUploadFileBuffered(resource: URI, file: File, progressReporter: (fileSize: number, bytesUploaded: number) => void, token: CancellationToken): Promise<void> {
  const writeableStream = newWriteableBufferStream({
    // Set a highWaterMark to prevent the stream
    // for file upload to produce large buffers
    // in-memory
    highWaterMark: 10
  });
  const writeFilePromise = this.fileService.writeFile(resource, writeableStream);
}

FileService

vs/platform/files/common/fileService.ts.

FileService 作为通用的文件服务，注册有很多具体服务的提供者，称为 provider。策略模式。

export class FileService extends Disposable implements IFileService {
  private readonly provider = new Map<string, IFileSystemProvider>();

  registerProvider(scheme: string, provider: IFileSystemProvider): IDisposable {
    this.provider.set(scheme, provider);
  }
}

vscode-remote 用于处理远程文件系统。

private async doWriteBuffered(provider: IFileSystemProviderWithOpenReadWriteCloseCapability, resource: URI, options: IWriteFileOptions | undefined, readableOrStreamOrBufferedStream: VSBufferReadable | VSBufferReadableStream | VSBufferReadableBufferedStream): Promise<void> {
  return this.writeQueue.queueFor(resource, this.getExtUri(provider).providerExtUri).queue(async () => {
    // open handle
    const handle = await provider.open(resource, { create: true, unlock: options?.unlock ?? false });
    await this.doWriteStreamBufferedQueued(provider, handle, readableOrStreamOrBufferedStream);
  });
}

private async doWriteBuffer(provider: IFileSystemProviderWithOpenReadWriteCloseCapability, handle: number, buffer: VSBuffer, length: number, posInFile: number, posInBuffer: number): Promise<void> {
  let totalBytesWritten = 0;
  while (totalBytesWritten < length) {

    // Write through the provider
    const bytesWritten = await provider.write(handle, posInFile + totalBytesWritten, buffer.buffer, posInBuffer + totalBytesWritten, length - totalBytesWritten);
    totalBytesWritten += bytesWritten;
  }
}

根据 resource(包含远程文件路径和文件名), 使用 provider 打开或创建远程文件，并获取文件句柄。然后使用 provider.write 写入文件。

provider- vscode-remote: RemoteFileSystemProviderClient

// vs/workbench/services/remote/common/remoteFileSystemProviderClient.ts

export const REMOTE_FILE_SYSTEM_CHANNEL_NAME = 'remoteFilesystem';

export class RemoteFileSystemProviderClient extends DiskFileSystemProviderClient {
  private constructor(remoteAgentEnvironment: IRemoteAgentEnvironment, connection: IRemoteAgentConnection) {
    super(connection.getChannel(REMOTE_FILE_SYSTEM_CHANNEL_NAME), { pathCaseSensitive: remoteAgentEnvironment.os === OperatingSystem.Linux });
  }
}

RemoteFileSystemProviderClient 实例化时会创建名为 remoteFilesystem 的 channel.

Channel 本质上是返回的一个对象，提供 call 和 listen 两个方法，使用闭包锁定 channelName。

// vs/base/parts/ipc/common/ipc.ts
export class ChannelClient implements IChannelClient, IDisposable {
  getChannel<T extends IChannel>(channelName: string): T {
    const that = this;

    return {
      call(command: string, arg?: any, cancellationToken?: CancellationToken) {
        if (that.isDisposed) {
          return Promise.reject(new CancellationError());
        }
        return that.requestPromise(channelName, command, arg, cancellationToken);
      },
      listen(event: string, arg: any) {
        if (that.isDisposed) {
          return Event.None;
        }
        return that.requestEvent(channelName, event, arg);
      }
    } as T;
  }
}

RemoteFileSystemProviderClient 的 write 方法继承自 DiskFileSystemProviderClient.

// vs/platform/files/common/diskFileSystemProviderClient.ts

write(fd: number, pos: number, data: Uint8Array, offset: number, length: number): Promise<number> {
  return this.channel.call('write', [fd, pos, VSBuffer.wrap(data), offset, length]);
}

序列化&传输

最终按 VS Code - transport protocol 描述的格式进行传输。

服务端处理

VS Code 从零实现的 WebSocket 协议。

所以在服务端需要解析出 WebSocket 帧， 然后按 VS Code - transport protocol 解析消息，
针对不同类型的消息和请求支持响应的处理。

具体到文件上传。
// TODO