tidy: move rustdoc js stuff into a tidy extra check #142924

lolbinarycat · 2025-06-23T18:44:23Z

Most of these were factored out of CI scripts, but eslint in particular was previously implemented with its own special cased logic.

A new option has been added to bootstrap, build.tidy-extra-checks, which serves as a default value for the --extra-checks flag. This is mostly for the benefit of rustdoc js maintainers, but should also help bootstrap py maintainers.

Additionally, --extra-checks=cpp has been documented.

I'm not super happy with how long the extra check names are in comparison to the others (in particular typecheck), but I couldn't think of anything better (I didn't want to name it tsc on the off chance we want to switch to a different typechecking engine in the future).

It would be nice to convert the extra checks arg into a proper enum, both for warning on unknown values and to provide better shell completion.

r? @GuillaumeGomez

rustbot · 2025-06-23T18:44:30Z

This PR modifies bootstrap.example.toml.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

There are changes to the tidy tool.

cc @jieyouxu

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

bors · 2025-06-24T05:46:56Z

☔ The latest upstream changes (presumably #142929) made this pull request unmergeable. Please resolve the merge conflicts.

Kobzol

Thanks! Moving this from CI scripts to tidy is definitely an improvement. Left a few comments.

Kobzol · 2025-06-24T07:20:34Z

src/bootstrap/src/core/config/flags.rs

@@ -382,7 +382,7 @@ pub enum Subcommand {
        bless: bool,
        #[arg(long)]
        /// comma-separated list of other files types to check (accepts py, py:lint,
-        /// py:fmt, shell)
+        /// py:fmt, shell, cpp, cpp:fmt, js, js:lint, js:typecheck, js:es-check)


Do we need to separate type checking and eslint? It's essentially the same category of "checking code" to me, at least in the world of JavaScript. Unless it runs for too long, I'd just collapse both into "lint" or "check". When you develop on JS, you'd set your bootstrap.toml file to contain both anyway, I imagine.

tsc generally checks correctness, while eslint mostly checks style... thought i suppose if you want fast typechecking, it would be best to run tsc directly...

also calling an extra check "check" feels unhelpful.

Kobzol · 2025-06-24T07:21:21Z

src/ci/docker/host-x86_64/mingw-check-1/Dockerfile

-           # Runs checks to ensure that there are no issues in our JS code.
-           es-check es2019 ../src/librustdoc/html/static/js/*.js && \
-           tsc --project ../src/librustdoc/html/static/js/tsconfig.json
+	   src/tools/tidy --extra-checks=js


Since this is now part of tidy, I think that we can move it out of this CI runner and just put it into mingw-check-tidy, to have centralized execution of all tidy checks on one place.

src/bootstrap/src/core/config/toml/build.rs

GuillaumeGomez · 2025-06-24T09:49:14Z

src/tools/tidy/src/ext_tool_checks.rs

@@ -224,6 +248,18 @@ fn check_impl(
        shellcheck_runner(&merge_args(&cfg_args, &file_args_shc))?;
    }

+    if js_lint {
+        rustdoc_js::lint(librustdoc_path, tools_path, src_path)?;


Didn't we say that the checks would be run if the JS files were modified too? Having them never run unless an extra arguments is passed is very inconvenient.

it wasn't too long ago that they were never run locally.

i was gonna have a warning for modifying files and not enabling their extra checks....

i suppose i could add a new "auto" value to --extra-checks, and make that the default.

i would prefer a design which still lets people manually disable these checks if they want.

Well, if you modify JS files, seems like you do want these checks by default, no?

What if you don't have the tools installed, and have limited or no network connectivity, but still want to run tidy?

I agree that running the tools based on modification is a good default, but there's very little cost to letting people disable it.

I think I would want to add 3 new special values, actually:

all, run everything, possibly used in CI (currently shellcheck isn't run in CI, so we would need to do that first).

auto, run checks for modified files

none, run no extra checks, same as the empty string

This would give a robust system that would be easy to add any future checks into, and would also help improve the existing checks, instead of just improving js checks.

the8472 · 2025-06-24T17:17:48Z

src/tools/tidy/src/main.rs

@@ -171,7 +170,17 @@ fn main() {
        };
        check!(unstable_book, &src_path, collected);

-        check!(ext_tool_checks, &root_path, &output_directory, bless, extra_checks, pos_args);
+        check!(
+            ext_tool_checks,


Why is this being added to a single check module? AFAICS ext_tools is running external programs serially, so the wall-time of all those tools adds up. The check! macro exists to run things in threads so they can run in parallel to cut down on walltime.

It's in ext_tools because that's where --extra-checks is parsed. I definitely agree that the parsing logic and stuff could use a big upgrade, and that tools should be run concurrently, but I think that can be done within ext_tools. It doesn't even technically need threads, just to spawn processes then wait for them all simultaneously. I could get working on a system for that if you want. It would probably involve making a struct or trait to represent an extra-check instead of having it be ad-hoc. This would also be a good opportunity to refactor the parsing.

Well, the macro also limits concurrency to obey -j, so having thread per check is the easiest way to get that.

So, should I inline every --extra-check as its own call to check!, probably doubling the size of main() in the process, or do you have a better idea?

Well, before doing more work I think it'd make sense to check how long it takes to run all the commands and compare that to the overall runtime of tidy. If it's the longest chain then parallelizing makes sense.

#143032 happens to have some numbers. We should probably move some of the longer-running tests to the front of main so that other ones get a chance to execute in parallel with them.

bors · 2025-06-26T13:22:40Z

☔ The latest upstream changes (presumably #142581) made this pull request unmergeable. Please resolve the merge conflicts.

yotamofek · 2025-06-26T19:29:37Z

src/tools/tidy/src/ext_tool_checks/rustdoc_js.rs

+}
+
+pub(super) fn typecheck(librustdoc_path: &Path) -> Result<(), super::Error> {
+    // use npx to ensure correct version


I'm not sure this comment makes much sense :)

yotamofek · 2025-06-26T19:32:32Z

src/tools/tidy/src/ext_tool_checks/rustdoc_js.rs

+    // Having the correct `eslint` version installed via `npm` isn't strictly necessary, since we're invoking it via `npx`,
+    // but this check allows the vast majority that is not working on the rustdoc frontend to avoid the penalty of running
+    // `eslint` in tidy. See also: https://github.com/rust-lang/rust/pull/142851
+    match get_eslint_version() {


I do think we can get rid of this check. I wanted to remove it in #142851 (in the first iteration, before I moved everything to be gated under the --extra-checks flag), but it would've made skipping eslint harder.
If we're going in the direction of only running eslint when needed, not just for everyone, then there's absolutely no reason to ask the user to explicitly install it, and not just use npx as it was intended to be used.
(just take care to run npx eslint@<version we pinned>)

npx might also not be installed, though.

Yeah, but then most likely npm isn't either (unless someone is running a really cursed environment [with s/python/node/] 😁)

Oh, this PR anyway makes it a hard error to not have npm/npx when requesting JS checks? I guess that requires explicit opt-in?

RalfJung · 2025-06-26T19:55:58Z

src/tools/tidy/src/ext_tool_checks/rustdoc_js.rs

+    let mut child = Command::new("npx")
+        .arg("tsc")
+        .arg("-p")
+        .arg(librustdoc_path.join("html/static/js/tsconfig.json"))
+        .spawn()?;


So, uh, do all of these npx invocations basically mean "download random JS from the web and run it locally"? Without any kind of lockfile ensuring integrity of the download? That seems... bad? Definitely not something I want to see happen on my dev machine.

Well, npx will first look for an eslint installed with npm install [-g], but otherwise yeah. See #142902 (which I intend to get to, but am waiting for this PR to be merged first, since it will probably conflict with the PR I'll open for that).

I mean, npm install eslint@some.specific.version && npx eslint is the same as npx eslint@some.specific.version,
in the sense that without a package-lock.json file (and an --frozen flag, I think?) both will run a certain version of eslint, but might run any semver-compatible version of the direct and transient deps.

Yeah, I don't think we should do any of that without explicit consent from the user. Running arbitrary code without integrity checks is a complete no-go.

Also, as far as I can see the commands here don't do the @some-version part.

See #142902 (which I intend to get to, but am waiting for this PR to be merged first, since it will probably conflict with the PR I'll open for that).

I think it is crucial that we start having proper lockfiles before we do do any sort of remote code execution from npm.

The state right now, before this PR, is that running tidy will ask you to run npm install eslint@some-version, and then will run that version of eslint the next time.

That is a way of asking for explicit consent, and very different from running npm install on the user's behalf.

Yeah, but since npx won't be invoked unless the correct (i.e. rustc-repo-specified) version of eslint is installed, it has the same effect.

That is because of the check which you suggested could be removed, right? Also, this comment here is attached to something doing npx tsc, not eslint, so -- I don't follow.

whether we check if that version is already installed and then run it with npx, or just run it with npx eslint@that-version doesn't make that much of a difference IMHO.

It makes a huge difference whether you ask the user to do something vs whether you just do it without asking them. Asking for consent is different from just assuming consent!

The state right now, before this PR, is that running tidy will ask you to run npm install eslint@some-version, and then will run that version of eslint the next time.

I would say the state before the PR is that tidy asks me to install eslint, and then I just don't do that, and everything is fine. I don't touch the JS files in this repo anyway, so I will happily ignore tidy's warning.

If even with this PR eslint is only run if it is already installed, then why use npx at all? Without a lockfile I feel like every single npx invocation needs a comment explaining why this is not remote code execution without integrity checks but just running something the user has already installed -- which defeats the entire purpose of npx as I understand it (but I may well misunderstand it, I am not familiar with the JS ecosystem).

That is because of the check which you #142924 (comment) could be removed, right? Also, this comment here is attached to something doing npx tsc, not eslint, so -- I don't follow.

Oh woops, completely missed that this was a different npx invocation. Sorry about the confusion! So yeah, the versoin of tsc that npx will run here is completely arbitrary (tsc and all its transitive dependencies).

It makes a huge difference whether you ask the user to do something vs whether you just do it without asking them. Asking for consent is different from just assuming consent!

Absolutely. I just meant that asking for consent does not absolve "us" (as in, the rustc repo) from also including a package-lock.json file so that the user is downloading/installing, either implicitly or explicitly, a pre-determined set of packages that won't change just because some left-pad maintainer decided to release a new patch version of his npm package.

If we had that, I think that a comment about what it means to run --extra-checks=eslint would suffice, in terms of getting the user's permission to run npx install --frozen eslint (not sure if --frozen is the correct flag but consider it pseudo-shell)

The state right now, before this PR, is that running tidy will ask you to run npm install eslint@some-version, and then will run that version of eslint the next time.

I would say the state before the PR is that tidy asks me to install eslint, and then I just don't do that, and everything is fine. I don't touch the JS files in this repo anyway, so I will happily ignore tidy's warning.

If even with this PR eslint is only run if it is already installed, then why use npx at all? Without a lockfile I feel like every single npx invocation needs a comment explaining why this is not remote code execution without integrity checks but just running something the user has already installed -- which defeats the entire purpose of npx as I understand it (but I may well misunderstand it, I am not familiar with the JS ecosystem).

Oh yeah, absolutely. Having the user run npm install and then running npx doesn't make a lot of sense. I was suggesting just doing the latter, but I agree that it might be better to just do the former (and then run it with npm run instead of npx).

(sorry for the multiple comments, I didn't see that you had already replied when I posted my previous ones)

this makes us less vulnerable to MITM and supply chain attacks. it also means that the CI scripts are no longer responsible for tracking the versions of these tools. it should also avoid the situation where local tsc and CI disagree on the presense of errors due to them being different versions.

rustbot assigned GuillaumeGomez Jun 23, 2025

lolbinarycat mentioned this pull request Jun 23, 2025

tidy: only run eslint on JS files if specified with "--extra-checks" #142851

Closed

This comment has been minimized.

Sign in to view

Kobzol reviewed Jun 24, 2025

View reviewed changes

GuillaumeGomez reviewed Jun 24, 2025

View reviewed changes

the8472 reviewed Jun 24, 2025

View reviewed changes

lolbinarycat force-pushed the tidy-js-extra-checks branch from 575be8c to b023137 Compare June 24, 2025 19:36

lolbinarycat added 4 commits June 26, 2025 12:50

tidy: running of eslint, tsc, and es-check are now an extra check

3f2661f

update CI to use new tidy extra checks for rustdoc js

adbad90

bootstrap: add build.tidy-extra-checks option

17aab7b

bootstrap: add CONFIG_CHANGE_HISTORY entry for build.tidy-extra-checks

36cd096

lolbinarycat force-pushed the tidy-js-extra-checks branch from b023137 to 36cd096 Compare June 26, 2025 17:54

yotamofek reviewed Jun 26, 2025

View reviewed changes

the8472 mentioned this pull request Jun 26, 2025

Performance issue for x test tidy #143032

Open

RalfJung reviewed Jun 26, 2025

View reviewed changes

This comment has been minimized.

Sign in to view

lolbinarycat force-pushed the tidy-js-extra-checks branch from 1e602f3 to ec37d89 Compare June 27, 2025 20:31

tidy: move rustdoc js stuff into a tidy extra check #142924

Are you sure you want to change the base?

tidy: move rustdoc js stuff into a tidy extra check #142924

Conversation

lolbinarycat commented Jun 23, 2025

Uh oh!

rustbot commented Jun 23, 2025

Uh oh!

This comment has been minimized.

bors commented Jun 24, 2025

Uh oh!

Kobzol left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lolbinarycat Jun 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bors commented Jun 26, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RalfJung Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yotamofek Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RalfJung Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RalfJung Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

lolbinarycat Jun 24, 2025 •

edited

Loading

RalfJung Jun 26, 2025 •

edited

Loading

yotamofek Jun 26, 2025 •

edited

Loading

RalfJung Jun 26, 2025 •

edited

Loading

RalfJung Jun 26, 2025 •

edited

Loading

yotamofek Jun 26, 2025 •

edited

Loading