Improve logging system using separate output channels #659

dhruvmanila · 2025-01-03T04:05:50Z

Summary

The basic idea is that there will be two channels where the logs will go:

Ruff: Here, all the logs for the extension will go
Ruff Language Server: Here, all the server log messages will go and are captured from the stderr. It will also contain the trace communications between the client and the server when it's enabled.
Ruff Language Server Trace: Here, all the communication messages between the client and the server will go. This channel is created lazily when the value of ruff.trace.server is changed to "verbose" or "messages" for the first time.

(2) is controlled by the ruff.logLevel configuration which defaults to info.

(3) is controlled by the ruff.trace.server configuration which defaults to off and can be changed to messages and verbose. The difference between them is that the former will only log the method name for both the request and response while the latter will also log the request parameters and the response result.

Red Knot

The end goal is to merge the tracing system with Red knot. Currently, Red knot's tracing system utilizes the -v, -vv and -vvv flags from the command-line and RED_KNOT_LOG environment variable for configuration. For the red knot server specifically, we will need to provide an additional configuration parameter like ruff.server.extraEnv which allows user to fine tune the logging using the mentioned environment variable. The ruff.logLevel will by default pass RED_KNOT_LOG=<logLevel> to allow for easy configuration.

Why not `window/logMessage` ?

The window/logMessage request can be used by the server to notify the client to log a message at a certain log level. There are a few disadvantages of using this as oppose to just using stderr:

It needs to go through the RPC which requires serializing and de-serializing the messages
The output format depends on how client handles the message
The trace channel will get populated with "Received notification 'window/logMessage'" lines
There's no 'debug' and 'trace' message type the LSP protocol although the next version is going to include debug type

Additionally, putting the message onto stderr also allows us to use a custom formatter which is what is being done in red knot and would prove to be beneficial when merging the tracing setup.

For posterity, here's the patch to use window/logMessage:

diff --git a/crates/ruff_server/src/trace.rs b/crates/ruff_server/src/trace.rs
index 7b367c780..ceedeff21 100644
--- a/crates/ruff_server/src/trace.rs
+++ b/crates/ruff_server/src/trace.rs
@@ -15,6 +15,9 @@
 //! Tracing will write to `stderr` by default, which should appear in the logs for most LSP clients.
 //! A `logFile` path can also be specified in the settings, and output will be directed there instead.
 use core::str;
+use lsp_server::{Message, Notification};
+use lsp_types::notification::{LogMessage, Notification as _};
+use lsp_types::{LogMessageParams, MessageType};
 use serde::Deserialize;
 use std::{
     path::PathBuf,
@@ -22,6 +25,7 @@ use std::{
     sync::{Arc, OnceLock},
 };
 use tracing::level_filters::LevelFilter;
+use tracing_subscriber::fmt::MakeWriter;
 use tracing_subscriber::{
     fmt::{time::Uptime, writer::BoxMakeWriter},
     layer::SubscriberExt,
@@ -73,7 +77,7 @@ pub(crate) fn init_tracing(
 
     let logger = match log_file {
         Some(file) => BoxMakeWriter::new(Arc::new(file)),
-        None => BoxMakeWriter::new(std::io::stderr),
+        None => BoxMakeWriter::new(LogMessageMakeWriter),
     };
     let subscriber = tracing_subscriber::Registry::default().with(
         tracing_subscriber::fmt::layer()
@@ -135,3 +139,61 @@ impl<S> tracing_subscriber::layer::Filter<S> for LogLevelFilter {
         Some(LevelFilter::from_level(self.filter.trace_level()))
     }
 }
+
+struct LogMessageMakeWriter;
+
+impl<'a> MakeWriter<'a> for LogMessageMakeWriter {
+    type Writer = LogMessageWriter;
+
+    fn make_writer(&'a self) -> Self::Writer {
+        LogMessageWriter {
+            level: tracing::Level::INFO,
+        }
+    }
+
+    fn make_writer_for(&'a self, meta: &tracing::Metadata<'_>) -> Self::Writer {
+        LogMessageWriter {
+            level: *meta.level(),
+        }
+    }
+}
+
+struct LogMessageWriter {
+    level: tracing::Level,
+}
+
+impl LogMessageWriter {
+    const fn message_type(&self) -> MessageType {
+        match self.level {
+            tracing::Level::ERROR => MessageType::ERROR,
+            tracing::Level::WARN => MessageType::WARNING,
+            tracing::Level::INFO => MessageType::INFO,
+            tracing::Level::DEBUG => MessageType::LOG,
+            tracing::Level::TRACE => MessageType::LOG,
+        }
+    }
+}
+
+impl std::io::Write for LogMessageWriter {
+    fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
+        let message = str::from_utf8(buf)
+            .map_err(|e| std::io::Error::new(std::io::ErrorKind::InvalidData, e))?
+            .to_owned();
+        LOGGING_SENDER
+            .get()
+            .expect("logging sender should be initialized at this point")
+            .send(Message::Notification(Notification {
+                method: LogMessage::METHOD.to_owned(),
+                params: serde_json::to_value(LogMessageParams {
+                    typ: self.message_type(),
+                    message,
+                })?,
+            }))
+            .map_err(|e| std::io::Error::new(std::io::ErrorKind::Other, e))?;
+        Ok(buf.len())
+    }
+
+    fn flush(&mut self) -> std::io::Result<()> {
+        Ok(())
+    }
+}

Test Plan

Using the following VS Code settings:

{
  "ruff.nativeServer": "on",
  "ruff.path": ["/Users/dhruv/work/astral/ruff/target/debug/ruff"],
  "ruff.trace.server": "verbose",
  "ruff.logLevel": "debug"
}

The following video showcases the three channels in VS Code:

Screen.Recording.2025-01-07.at.6.20.54.PM.mov

Using the following Zed settings:

{
  "lsp": {
    "ruff": {
      "binary": {
        "path": "/Users/dhruv/work/astral/ruff/target/debug/ruff",
        "arguments": [
          "server"
        ]
      },
      "initialization_options": {
        "settings": {
          "logLevel": "debug"
        }
      }
    }
  }
}

The following video showcases that the stderr messages are captured in the "Server Logs" window and the RPC window provides the raw request and response messages:

Screen.Recording.2025-01-03.at.5.44.23.PM.mov

For Neovim, there's an open issue (neovim/neovim#16807) to send stderr messages to a separate file but currently it gets logged as "ERROR" in the main LSP log file. There doesn't seem to be any way to hook into the stderr of the Neovim LSP client for us to provide a solution for Ruff users.

MichaReiser · 2025-01-03T12:55:11Z

Thanks for working on this. I haven't reviewed the code yet because I've some UX questions.

As a user, I think I'd find it confusing that the extension and server settings are separated. For example, Red Knot logs:

INFO Python version: Python 3.9
INFO Found 8 files in package `test`
error[lint:unresolved-import] /Users/micha/astral/test/distribution/distribution.py:1:8 Cannot resolve import `seaborn`
error[lint:unresolved-import] /Users/micha/astral/test/distribution/distribution.ipynb:1:8 Cannot resolve import `seaborn`
error[lint:unresolved-import] /Users/micha/astral/test/distribution/distribution.ipynb:2:8 Cannot resolve import `pandas`
error[lint:unresolved-import] /Users/micha/astral/test/other.py:2:1 Cannot resolve import `c`

I would find it most useful as a user when I see this in the same panel as the main extension logs. Having to switch between the two panels will make it difficult to match the log lines (what's the order of the events?)

As a developer, I think the separation of the server and tracing output might be problematic because it makes it harder to correlate events. Did the panic happen before or after this specific LSP message? I'm worried that not having this knowledge will make debugging errors harder.

Can you tell me more for the motivation for separating the two outputs?

dhruvmanila · 2025-01-06T10:18:15Z

My main motivation behind having separate channels to so that it's easy to go through the logs which are specific to either the client (editor) or the server. Additionally, the format of the log messages in the extension channel is different because it's a log channel while the server messages uses a normal channel. The difference between a log and a normal channel in VS Code is that the former will add the timestamp and the log level while the latter will add the messages as is. This separation is useful especially for the server because the logs from both the server and the CLI output will have exactly the same format and it allows the server to be in control of the message format.

The kind of log messages between the editor and the server are also quite different considering that once the server starts, there's almost no involvement from the client side apart from notifying the user for any issues regarding the server.

As a developer, I think the separation of the server and tracing output might be problematic because it makes it harder to correlate events. Did the panic happen before or after this specific LSP message? I'm worried that not having this knowledge will make debugging errors harder.

Yeah, I think I see your point and it makes sense to have one channel for both the server logs and the trace logs at least.

I'm interested in your opinion regarding whether to have a separate channel for client and server logs. Currently, my main worry is about the message format.

MichaReiser · 2025-01-06T13:30:23Z

Thanks. The explanation about the different channel styles is helpful, and I can see how that can motivate having separate server and client traces.

Looking at Red Knot, all logs with info are targeted at end users. To some degree, this even includes debug messages. It's really only trace messages that are only intended for developers. The same is true for the extension logs, which is why it would be nice if all of them could be found in a single place. I'd also say that multiple output panels are probably harder to understand than just one. E.g., as a user, I don't think I'd know which output channel I should look at if something isn't working. That's why I'd say that the fewer channels the better (as long as we find a good UX)

But then, having two channels seems to be fairly common: Both Rust analyzer and Python have separate output channels (ESLint, TypeScript have just one). Biome uses Biome and tracing

The kind of log messages between the editor and the server are also quite different considering that once the server starts, there's almost no involvement from the client side apart from notifying the user for any issues regarding the server.

That's fair, although I don't think this matters much from a user perspective. I just want to see the logs.

server to be in control of the message format.

This is nice, although I think we could customize the tracing output to have a format that's optimized for the VS code output.

Either way. I think two channels, one for the client and one for the server, seems a reasonable start.

MichaReiser · 2025-01-07T12:46:05Z

We discussed this via chat and decided to keep the three output channels because:

Combining the VS Code tracing output channel and the client channel has the downside that trace messages end up with two time stamps: one from the log output panel and one from the trace
We create the trace output panel lazily. There's no point in having an empty output panel for most users. It's primarily a development feature
We improve the server spans to create a span for each request/response that includes the request method and request ID. This allows correlating logs between the tracing and server panels.

dhruvmanila · 2025-01-07T12:56:23Z

.github/workflows/ci.yaml

-    runs-on: ubuntu-latest
+    runs-on: ubuntu-22.04
    env:


This is because the latest Ubuntu version doesn't provide Python 3.7

MichaReiser

Thanks. This is great.

For Neovim. I assume the proposed solution is to set logFile?

src/common/logger.ts

README.md

src/common/server.ts

src/extension.ts

src/common/logger.ts

dhruvmanila · 2025-01-08T03:44:36Z

For Neovim. I assume the proposed solution is to set logFile?

Yes, ideally that's the best solution for all editors that uses a single file for logs from all the language server. I'll update the Ruff documentation to add a tip for this.

## Summary Refer to the VS Code PR (astral-sh/ruff-vscode#659) for details on the change. This PR changes the following: 1. Add tracing span for both request (request id and method name) and notification (method name) handler 2. Remove the `RUFF_TRACE` environment variable. This was being used to turn on / off logging for the server 3. Similarly, remove reading the `trace` value from the initialization options 4. Remove handling the `$/setTrace` notification 5. Remove the specialized `TraceLogWriter` used for Zed and VS Code (#12564) Regarding the (5) for the Zed editor, the reason that was implemented was because there was no way of looking at the stderr messages in the editor which has been changed. Now, it captures the stderr as part of the "Server Logs". (https://github.com/zed-industries/zed/blob/82492d74a8d0350cba66671c27e282a928fd4c85/crates/language_tools/src/lsp_log.rs#L548-L552) ### Question Regarding (1), I think having just a simple trace level message should be good for now as the spans are not hierarchical. This could be tackled with #12744. The difference between the two: <details><summary>Using <code>tracing::trace</code></summary> <p> ``` 0.019243416s DEBUG ThreadId(08) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/playground/ruff/.vscode 0.026398750s INFO main ruff_server::session::index: Registering workspace: /Users/dhruv/playground/ruff 0.026802125s TRACE ruff:main ruff_server::server::api: Received notification "textDocument/didOpen" 0.026930666s TRACE ruff:main ruff_server::server::api: Received notification "textDocument/didOpen" 0.026962333s TRACE ruff:main ruff_server::server::api: Received request "textDocument/diagnostic" (1) 0.027042875s TRACE ruff:main ruff_server::server::api: Received request "textDocument/diagnostic" (2) 0.027097500s TRACE ruff:main ruff_server::server::api: Received request "textDocument/codeAction" (3) 0.027107458s DEBUG ruff:worker:0 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py 0.027123541s DEBUG ruff:worker:3 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/organize_imports.py 0.027514875s INFO ruff:main ruff_server::server: Configuration file watcher successfully registered 0.285689833s TRACE ruff:main ruff_server::server::api: Received request "textDocument/codeAction" (4) 45.741101666s TRACE ruff:main ruff_server::server::api: Received notification "textDocument/didClose" 47.108745500s TRACE ruff:main ruff_server::server::api: Received notification "textDocument/didOpen" 47.109802041s TRACE ruff:main ruff_server::server::api: Received request "textDocument/diagnostic" (5) 47.109926958s TRACE ruff:main ruff_server::server::api: Received request "textDocument/codeAction" (6) 47.110027791s DEBUG ruff:worker:6 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py 51.863679125s TRACE ruff:main ruff_server::server::api: Received request "textDocument/hover" (7) ``` </p> </details> <details><summary>Using <code>tracing::trace_span</code></summary> <p> Only logging the enter event: ``` 0.018638750s DEBUG ThreadId(11) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/playground/ruff/.vscode 0.025895791s INFO main ruff_server::session::index: Registering workspace: /Users/dhruv/playground/ruff 0.026378791s TRACE ruff:main notification{method="textDocument/didOpen"}: ruff_server::server::api: enter 0.026531208s TRACE ruff:main notification{method="textDocument/didOpen"}: ruff_server::server::api: enter 0.026567583s TRACE ruff:main request{id=1 method="textDocument/diagnostic"}: ruff_server::server::api: enter 0.026652541s TRACE ruff:main request{id=2 method="textDocument/diagnostic"}: ruff_server::server::api: enter 0.026711041s DEBUG ruff:worker:2 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/organize_imports.py 0.026729166s DEBUG ruff:worker:1 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py 0.027023083s INFO ruff:main ruff_server::server: Configuration file watcher successfully registered 5.197554750s TRACE ruff:main notification{method="textDocument/didClose"}: ruff_server::server::api: enter 6.534458000s TRACE ruff:main notification{method="textDocument/didOpen"}: ruff_server::server::api: enter 6.535027958s TRACE ruff:main request{id=3 method="textDocument/diagnostic"}: ruff_server::server::api: enter 6.535271166s DEBUG ruff:worker:3 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/organize_imports.py 6.544240583s TRACE ruff:main request{id=4 method="textDocument/codeAction"}: ruff_server::server::api: enter 7.049692458s TRACE ruff:main request{id=5 method="textDocument/codeAction"}: ruff_server::server::api: enter 7.508142541s TRACE ruff:main request{id=6 method="textDocument/hover"}: ruff_server::server::api: enter 7.872421958s TRACE ruff:main request{id=7 method="textDocument/hover"}: ruff_server::server::api: enter 8.024498583s TRACE ruff:main request{id=8 method="textDocument/codeAction"}: ruff_server::server::api: enter 13.895063666s TRACE ruff:main request{id=9 method="textDocument/codeAction"}: ruff_server::server::api: enter 14.774706083s TRACE ruff:main request{id=10 method="textDocument/hover"}: ruff_server::server::api: enter 16.058918958s TRACE ruff:main notification{method="textDocument/didChange"}: ruff_server::server::api: enter 16.060562208s TRACE ruff:main request{id=11 method="textDocument/diagnostic"}: ruff_server::server::api: enter 16.061109083s DEBUG ruff:worker:8 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py 21.561742875s TRACE ruff:main notification{method="textDocument/didChange"}: ruff_server::server::api: enter 21.563573791s TRACE ruff:main request{id=12 method="textDocument/diagnostic"}: ruff_server::server::api: enter 21.564206750s DEBUG ruff:worker:4 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py 21.826691375s TRACE ruff:main request{id=13 method="textDocument/codeAction"}: ruff_server::server::api: enter 22.091080125s TRACE ruff:main request{id=14 method="textDocument/codeAction"}: ruff_server::server::api: enter ``` </p> </details> **Todo** - [x] Update documentation (I'll be adding a troubleshooting section under "Editors" as a follow-up which is for all editors) - [x] Check for backwards compatibility. I don't think this should break backwards compatibility as it's mainly targeted towards improving the debugging experience. ~**Before I go on to updating the documentation, I'd appreciate initial review on the chosen approach.**~ resolves: #14959 ## Test Plan Refer to the test plan in astral-sh/ruff-vscode#659. Example logs at `debug` level: ``` 0.010770083s DEBUG ThreadId(15) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/playground/ruff/.vscode 0.018101916s INFO main ruff_server::session::index: Registering workspace: /Users/dhruv/playground/ruff 0.018559916s DEBUG ruff:worker:4 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py 0.018992375s INFO ruff:main ruff_server::server: Configuration file watcher successfully registered 23.408802375s DEBUG ruff:worker:11 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py 24.329127416s DEBUG ruff:worker:6 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py ``` Example logs at `trace` level: ``` 0.010296375s DEBUG ThreadId(13) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/playground/ruff/.vscode 0.017422583s INFO main ruff_server::session::index: Registering workspace: /Users/dhruv/playground/ruff 0.018034458s TRACE ruff:main notification{method="textDocument/didOpen"}: ruff_server::server::api: enter 0.018199708s TRACE ruff:worker:0 request{id=1 method="textDocument/diagnostic"}: ruff_server::server::api: enter 0.018251167s DEBUG ruff:worker:0 request{id=1 method="textDocument/diagnostic"}: ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py 0.018528708s INFO ruff:main ruff_server::server: Configuration file watcher successfully registered 1.611798417s TRACE ruff:worker:1 request{id=2 method="textDocument/codeAction"}: ruff_server::server::api: enter 1.861757542s TRACE ruff:worker:4 request{id=3 method="textDocument/codeAction"}: ruff_server::server::api: enter 7.027361792s TRACE ruff:worker:2 request{id=4 method="textDocument/codeAction"}: ruff_server::server::api: enter 7.851361500s TRACE ruff:worker:5 request{id=5 method="textDocument/codeAction"}: ruff_server::server::api: enter 7.901690875s TRACE ruff:main notification{method="textDocument/didChange"}: ruff_server::server::api: enter 7.903063167s TRACE ruff:worker:10 request{id=6 method="textDocument/diagnostic"}: ruff_server::server::api: enter 7.903183500s DEBUG ruff:worker:10 request{id=6 method="textDocument/diagnostic"}: ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py 8.702385292s TRACE ruff:main notification{method="textDocument/didChange"}: ruff_server::server::api: enter 8.704106625s TRACE ruff:worker:3 request{id=7 method="textDocument/diagnostic"}: ruff_server::server::api: enter 8.704304875s DEBUG ruff:worker:3 request{id=7 method="textDocument/diagnostic"}: ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/lsp/play.py 8.966853458s TRACE ruff:worker:9 request{id=8 method="textDocument/codeAction"}: ruff_server::server::api: enter 9.229622792s TRACE ruff:worker:6 request{id=9 method="textDocument/codeAction"}: ruff_server::server::api: enter 10.513111583s TRACE ruff:worker:7 request{id=10 method="textDocument/codeAction"}: ruff_server::server::api: enter ```

Improve logging system

3119625

dhruvmanila mentioned this pull request Jan 3, 2025

Improve logging system using logLevel, avoid trace value astral-sh/ruff#15232

Merged

2 tasks

dhruvmanila changed the title ~~Improve logging system~~ Improve logging system using separate output channels Jan 3, 2025

dhruvmanila marked this pull request as ready for review January 3, 2025 12:29

dhruvmanila added 5 commits January 7, 2025 10:03

Merge trace channel into the server channel

42c9c14

Run CI on Ubuntu 22.04

671dc4d

Update docs

f860c8e

Use lazy output channel for server trace

4bf8470

Update docs

2ce6c32

Update tsconfig.json

84f005f

dhruvmanila commented Jan 7, 2025

View reviewed changes

MichaReiser approved these changes Jan 7, 2025

View reviewed changes

Review feedback

a1f4192

dhruvmanila merged commit 0b4696e into main Jan 8, 2025
6 checks passed

dhruvmanila deleted the dhruv/server-logs branch January 8, 2025 03:44

dhruvmanila mentioned this pull request Jan 9, 2025

Error messages are not logged by default in the output channel #642

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve logging system using separate output channels #659

Improve logging system using separate output channels #659

dhruvmanila commented Jan 3, 2025 •

edited

Loading

MichaReiser commented Jan 3, 2025

dhruvmanila commented Jan 6, 2025

MichaReiser commented Jan 6, 2025

MichaReiser commented Jan 7, 2025

dhruvmanila Jan 7, 2025

MichaReiser left a comment

dhruvmanila commented Jan 8, 2025

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Improve logging system using separate output channels #659

Improve logging system using separate output channels #659

Conversation

dhruvmanila commented Jan 3, 2025 • edited Loading

Summary

Red Knot

Why not window/logMessage ?

Test Plan

MichaReiser commented Jan 3, 2025

dhruvmanila commented Jan 6, 2025

MichaReiser commented Jan 6, 2025

MichaReiser commented Jan 7, 2025

dhruvmanila Jan 7, 2025

Choose a reason for hiding this comment

MichaReiser left a comment

Choose a reason for hiding this comment

dhruvmanila commented Jan 8, 2025

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

dhruvmanila commented Jan 3, 2025 •

edited

Loading

Why not `window/logMessage` ?