Repository navigation
Provide guidance how language servers should support Virtual Workspaces #123025
Description
The new Virtual Workspace feature is great! Could you provide guidance how we might support it from our language server?
At the moment, our LSP implementation reads files from disc all the time. I've long hoped that instead of using direct file IO we could just do calls from the server to the client via LSP to read file content, and if that supported custom URI schemes, I guess this would work. But at the moment it is not clear to me how we could get our LS to support a virtual workspace.
Activity
tamuratak commentedon May 6, 2021
Related to microsoft/language-server-protocol/issues/336
jrieken commentedon May 6, 2021
The VSCode API has this since a while. There is
vscode.workspace.fswhich should allow you to do the basic file operations like read, write, stat, readdir, delete etc.davidanthoff commentedon May 6, 2021
Yes, I'm aware of the
vscode.workspace.fsAPI. But I can only use that from within the extension, right? Not from within our language server implementation. Essentially I'm thinking that we need the equivalent ofvscode.workspace.fsin the language server, i.e. a LSP version that has JSON RPC methods for exactly those kind of file IO operations?jrieken commentedon May 6, 2021
Yes - only through some custom protocol messages it can be used
davidanthoff commentedon May 6, 2021
Would it make sense to extend the LSP to include those kind of protocol messages, and have the
LanguageClientclass provide a default implementation on the extension side that usesvscode.workspace.fs? It seems that almost any language server implementation that wants to work in a virtual workspace would need this, right?dbaeumer commentedon May 10, 2021
There is work under way that will add FS support to LSP.
heejaechang commentedon Jun 14, 2021
Dirk Bäumer (@dbaeumer) I hope that includes fs access to outside of workspace root files (it is common libraries to be installed outside of workspace root) and file watcher support for those files as well. also, not all LSP server is asynchronous, so hopefully, those API support synchronous API as well like fs.
dbaeumer commentedon Jun 15, 2021
Heejae Chang (@heejaechang) what range of files the provider will support depends on the provider itself and is out of the control of the protocol or VS Code API. For example the file system provider for
filesupports files outside the workspace since it can read it. Thevscode-vfscan only provide access to the files on GitHub for example and therefore under the workspace root. It can't support reading files outside the repository.As all process to process API we can only make this async.
jakebailey commentedon Jun 15, 2021
That's going to end up being quite problematic for Pylance/pyright, where we are deeply synchronous during the analysis. I'm wondering how TypeScript is able to handle this, as they have a very similar architecture and are unlikely to be able to rearchitect to be async any time soon.
(Further, our cancellation itself is based on FS accesses to deal with the lack of async message handling.)
dbaeumer commentedon Jun 16, 2021
I do have a similar problem with ESLint and I solved it by yielding as much as possible. This can be done without making all the code async (ESLint itself was sync until version 7). Doing this allows for example for normal cancellation and the json-rpc libraries do buffer all messages internally to make this easier. We even do the yielding approach in VS Code itself. When we colorize a file we yield every n lines so that events and other things can be processed. In Pylance/pyright you might be able to easily yield after every file which would only make the top level code async.
I haven't looked at TypeScript since quite some time but I do know that they introduced the concept of workers to be able to yield more efficiently. But you need to talk to the TS people.
dbaeumer commentedon Jun 16, 2021
Another approach could be https://github.com/abbr/deasync. But I also heard about problems using it (e.g. total dead locks).
jakebailey commentedon Jun 16, 2021
The problem is twofold; the plumbing (as async/await is infectious), and the assumptions we have made about reentrancy.
Libraries like deasync would yield back to the event loop, so it's possible that we then handle some other request that reenters the critical analysis code and breaks our assumptions, so that wouldn't be something I'd want to use.
(deasync and libs like it are also a compiled module so they can hack into V8, which are not so fun to manage for VS Code extensions, as compared to things that users
npm install.)heejaechang commentedon Jun 16, 2021
Dirk Bäumer (@dbaeumer) won't yield cause new message in the event loop to be picked up before finishing processing current message? that will introduce re-entrance and can cause problem for us. (such as updating parse tree in the middle of processing them in the other task). I believe it has same blocking issue on why it is so hard for us to move to async (we need to make our LS able to handle multiple requests)
and unfortunately, any lib that involves native node lib is problem for us since we need to run our LS in browser as well in near future.
that being said, can you show me how you yield in ESLint? I am looking this (https://github.com/microsoft/vscode-eslint/blob/main/server/src/eslintServer.ts) but not sure where I should look.
dbaeumer commentedon Jun 17, 2021
The trick is here: https://github.com/microsoft/vscode-eslint/blob/main/server/src/eslintServer.ts#L1029
I take every message from JSON-RPC and but them into my own queue. In that queue I know better when to execute the messages. I even drop message if I know that I can skip the result. https://github.com/microsoft/vscode-eslint/blob/main/server/src/eslintServer.ts#L1130
I do agree that this is not for free but it does allow for more yielding than executing everything sync.
Regarding deasync: I fully agree with the concerns and I kept my hands off something like this since it does ask for problems. I simply wanted to point out the possibility.
vscode-triage-bot commentedon Oct 18, 2021
This feature request is now a candidate for our backlog. The community has 60 days to upvote the issue. If it receives 20 upvotes we will move it to our backlog. If not, we will close it. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!
vscode-triage-bot commentedon Dec 8, 2021
This feature request has not yet received the 20 community upvotes it takes to make to our backlog. 10 days to go. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!
vscode-triage-bot commentedon Dec 18, 2021
🙁 In the last 60 days, this feature request has received less than 20 community upvotes and we closed it. Still a big Thank You to you for taking the time to create this issue! To learn more about how we handle feature requests, please see our documentation.
Happy Coding!