Skip to content

[feat] syntactic semantic tokens #4672

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 7 commits into
base: master
Choose a base branch
from
Draft

[feat] syntactic semantic tokens #4672

wants to merge 7 commits into from

Conversation

MangoIV
Copy link
Contributor

@MangoIV MangoIV commented Jul 22, 2025 •
edited
Loading

Use the GHC AST and lsp semantic tokens to convince the language server to give highlighting even without any editor highlighting plugins.

Fixes #4501

  • check that this works on all ghc versions
  • check which AST nodes I might have overlooked that are important. (add a tests that checks that all source ranges are covered (except space)

@MangoIV MangoIV mentioned this pull request Jul 22, 2025
Draft
@MangoIV MangoIV force-pushed the mangoiv/syntactic-tokens branch from e6621bc to 1bde91a Compare July 22, 2025 18:14
@MangoIV MangoIV changed the title [feat] syntactic syntactic tokens [feat] syntactic semantic tokens Jul 22, 2025
@soulomoon
Copy link
Collaborator

soulomoon commented Jul 22, 2025 •
edited
Loading

Some keypoints beyond the tokens retrieval:

  1. UTF8 vs UTF16 differences for the token range between haskell and LSP spec. see focusTokenAt.
  2. Consider position mappings so we have correct highlighting even the file could not even parse, see toCurrentRange.
  3. we need to consider delta between each result, see semanticTokensFullDelta. This is to minimize the bandwith between LSP server and clients.

Post it here as a reminder.

@MangoIV
Copy link
Contributor Author

MangoIV commented Jul 22, 2025

Thank you. I will go through all of these.

@MangoIV
[feat] syntactic syntactic tokens
bc4ed9a 
Use the GHC AST and lsp semantic tokens to convince the language server
to give highlighting even without any editor highlighting plugins.
@MangoIV MangoIV force-pushed the mangoiv/syntactic-tokens branch from 1bde91a to bc4ed9a Compare July 22, 2025 18:47
@MangoIV MangoIV mentioned this pull request Jul 23, 2025
Open
9 tasks
@MangoIV
Copy link
Contributor Author

MangoIV commented Jul 26, 2025

@soulomoon I'm currently struggling a bit with the tests - the ASTs are different enough that making the syntax highlighting consistent between ghc versions is probably a big pain. Should I introduce more goldens for all supported ghc versions?

@MangoIV
Copy link
Contributor Author

MangoIV commented Jul 26, 2025

also stylish-haskell's parser seems to be confused about the CPP :/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@soulomoon soulomoon soulomoon left review comments

@berberman berberman Awaiting requested review from berberman berberman will be requested when the pull request is marked ready for review berberman is a code owner

@michaelpj michaelpj Awaiting requested review from michaelpj michaelpj will be requested when the pull request is marked ready for review michaelpj is a code owner

@guibou guibou Awaiting requested review from guibou guibou will be requested when the pull request is marked ready for review guibou is a code owner

@fendor fendor Awaiting requested review from fendor fendor will be requested when the pull request is marked ready for review fendor is a code owner

@wz1000 wz1000 Awaiting requested review from wz1000 wz1000 will be requested when the pull request is marked ready for review wz1000 is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

the syntax highlighting of a field name used as a function and a field used with -XOverloadedRecordDot should look the same
2 participants
@MangoIV @soulomoon
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy