Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(turbopack-ecmascript): Make expand_star_exports more deterministic #75383

Merged
merged 1 commit into from
Feb 4, 2025

Conversation

bgw
Copy link
Member

@bgw bgw commented Jan 28, 2025

Closes PACK-3867

@ijjk ijjk added the created-by: Turbopack team PRs by the Turbopack team. label Jan 28, 2025
Copy link
Member Author

bgw commented Jan 28, 2025

This stack of pull requests is managed by Graphite. Learn more about stacking.

@ijjk
Copy link
Member

ijjk commented Jan 28, 2025

Stats from current PR

Default Build (Increase detected ⚠️)
General
vercel/next.js canary vercel/next.js bgw/expand-star-exports-non-deterministic Change
buildDuration 21s 19.8s N/A
buildDurationCached 18.7s 15.7s N/A
nodeModulesSize 391 MB 391 MB
nextStartRea..uration (ms) 478ms 484ms N/A
Client Bundles (main, webpack)
vercel/next.js canary vercel/next.js bgw/expand-star-exports-non-deterministic Change
5306-HASH.js gzip 54 kB 54 kB N/A
8276.HASH.js gzip 169 B 168 B N/A
8377-HASH.js gzip 5.46 kB 5.46 kB N/A
bccd1874-HASH.js gzip 52.9 kB 52.9 kB N/A
framework-HASH.js gzip 57.5 kB 57.5 kB N/A
main-app-HASH.js gzip 240 B 243 B N/A
main-HASH.js gzip 34.5 kB 34.4 kB N/A
webpack-HASH.js gzip 1.71 kB 1.71 kB N/A
Overall change 0 B 0 B
Legacy Client Bundles (polyfills)
vercel/next.js canary vercel/next.js bgw/expand-star-exports-non-deterministic Change
polyfills-HASH.js gzip 39.4 kB 39.4 kB
Overall change 39.4 kB 39.4 kB
Client Pages
vercel/next.js canary vercel/next.js bgw/expand-star-exports-non-deterministic Change
_app-HASH.js gzip 193 B 193 B
_error-HASH.js gzip 193 B 193 B
amp-HASH.js gzip 512 B 510 B N/A
css-HASH.js gzip 343 B 342 B N/A
dynamic-HASH.js gzip 1.84 kB 1.84 kB
edge-ssr-HASH.js gzip 265 B 265 B
head-HASH.js gzip 363 B 362 B N/A
hooks-HASH.js gzip 393 B 392 B N/A
image-HASH.js gzip 4.59 kB 4.58 kB N/A
index-HASH.js gzip 268 B 268 B
link-HASH.js gzip 2.35 kB 2.35 kB N/A
routerDirect..HASH.js gzip 328 B 328 B
script-HASH.js gzip 397 B 397 B
withRouter-HASH.js gzip 323 B 326 B N/A
1afbb74e6ecf..834.css gzip 106 B 106 B
Overall change 3.59 kB 3.59 kB
Client Build Manifests
vercel/next.js canary vercel/next.js bgw/expand-star-exports-non-deterministic Change
_buildManifest.js gzip 748 B 747 B N/A
Overall change 0 B 0 B
Rendered Page Sizes
vercel/next.js canary vercel/next.js bgw/expand-star-exports-non-deterministic Change
index.html gzip 524 B 522 B N/A
link.html gzip 539 B 538 B N/A
withRouter.html gzip 520 B 520 B
Overall change 520 B 520 B
Edge SSR bundle Size
vercel/next.js canary vercel/next.js bgw/expand-star-exports-non-deterministic Change
edge-ssr.js gzip 129 kB 129 kB N/A
page.js gzip 210 kB 210 kB N/A
Overall change 0 B 0 B
Middleware size
vercel/next.js canary vercel/next.js bgw/expand-star-exports-non-deterministic Change
middleware-b..fest.js gzip 670 B 666 B N/A
middleware-r..fest.js gzip 155 B 156 B N/A
middleware.js gzip 31.3 kB 31.3 kB N/A
edge-runtime..pack.js gzip 844 B 844 B
Overall change 844 B 844 B
Next Runtimes
vercel/next.js canary vercel/next.js bgw/expand-star-exports-non-deterministic Change
app-page-exp...dev.js gzip 385 kB 385 kB
app-page-exp..prod.js gzip 132 kB 132 kB
app-page-tur..prod.js gzip 145 kB 145 kB
app-page-tur..prod.js gzip 141 kB 141 kB
app-page.run...dev.js gzip 372 kB 372 kB
app-page.run..prod.js gzip 128 kB 128 kB
app-route-ex...dev.js gzip 39.3 kB 39.3 kB
app-route-ex..prod.js gzip 25.6 kB 25.6 kB
app-route-tu..prod.js gzip 25.6 kB 25.6 kB
app-route-tu..prod.js gzip 25.4 kB 25.4 kB
app-route.ru...dev.js gzip 40.8 kB 40.8 kB
app-route.ru..prod.js gzip 25.4 kB 25.4 kB
dist_client_...dev.js gzip 356 B 356 B
dist_client_...dev.js gzip 349 B 349 B
pages-api-tu..prod.js gzip 9.69 kB 9.69 kB
pages-api.ru...dev.js gzip 11.8 kB 11.8 kB
pages-api.ru..prod.js gzip 9.68 kB 9.68 kB
pages-turbo...prod.js gzip 21.9 kB 21.9 kB
pages.runtim...dev.js gzip 31.5 kB 31.5 kB
pages.runtim..prod.js gzip 21.9 kB 21.9 kB
server.runti..prod.js gzip 60.2 kB 60.2 kB
Overall change 1.65 MB 1.65 MB
build cache Overall increase ⚠️
vercel/next.js canary vercel/next.js bgw/expand-star-exports-non-deterministic Change
0.pack gzip 2.1 MB 2.1 MB N/A
index.pack gzip 74.9 kB 75.6 kB ⚠️ +675 B
Overall change 74.9 kB 75.6 kB ⚠️ +675 B
Diff details
Diff for main-HASH.js

Diff too large to display

Commit: 17f5dfe

@mischnic
Copy link
Contributor

In the rest of the codebase, we usually use IndexSet for this.
https://doc.rust-lang.org/std/collections/struct.BTreeMap.html claims to work well if you have cheap comparisons, which is not the case here with RcStr? Though I'm not sure myself which one to choose @sokra

@sokra
Copy link
Member

sokra commented Jan 28, 2025

Yeah I would also prefer IndexSet here for performance reasons. BTreeMap has O(log n) as it sorts it, while IndexSet has O(1) for inserts.

Copy link
Member Author

bgw commented Jan 30, 2025

I'll switch it to IndexMap, but to clarify a few things:

  • O(log n) vs O(1) only matters for very large collections (e.g. >1000 items).

  • Most of the real-world benefit of a HashMap versus a BTreeMap comes from the improved cache locality of a HashMap. That comes into play mostly when your data structure is too big to fit in the L1 cache.

  • There's some theoretical benefit to BTreeMap in turbo-tasks because sorting might give more opportunities for cache hits. That's probably not happening here because the cell containing the Vec is created by the same function that constructs it, but it could be in many places.

RcStr is cheap to compare:

  • In most cases comparisons will only need to look at the first couple characters.

  • Only one full equality comparison will happen per lookup. These are export keys which are short strings. Long keys would encounter similar issues with HashMap where equality and hashing would take O(key size) time.

  • The only time RcStr comparisons would be expensive is with long strings that have common prefixes.

@bgw bgw force-pushed the bgw/expand-star-exports-non-deterministic branch from 921e484 to 951dcc6 Compare January 31, 2025 02:41
@bgw bgw marked this pull request as ready for review January 31, 2025 03:31
@bgw bgw requested review from mischnic and sokra January 31, 2025 07:53
@ijjk
Copy link
Member

ijjk commented Feb 3, 2025

Tests Passed

@bgw bgw force-pushed the bgw/expand-star-exports-non-deterministic branch from 951dcc6 to 17f5dfe Compare February 3, 2025 22:26
@bgw bgw merged commit 4787e5c into canary Feb 4, 2025
131 checks passed
Copy link
Member Author

bgw commented Feb 4, 2025

Merge activity

  • Feb 3, 7:13 PM EST: A user merged this pull request with Graphite.

@bgw bgw deleted the bgw/expand-star-exports-non-deterministic branch February 4, 2025 00:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
created-by: Turbopack team PRs by the Turbopack team.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy