add sticky cache pgo #154098

bobrenjc93 · 2025-05-22T04:44:30Z

Stack from ghstack (oldest at bottom):

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @amjames

[ghstack-poisoned]

pytorch-bot · 2025-05-22T04:44:34Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/154098

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 61 New Failures, 1 Unrelated Failure

As of commit 45511a6 with merge base 0087f5f ():

NEW FAILURES - The following jobs have failed:

Check Labels / Check labels (gh)
RuntimeError: Error checking labels: PR does not have required labels
inductor / cuda12.6-py3.10-gcc9-sm86 / test (inductor_huggingface, 1, 1, linux.g5.4xlarge.nvidia.gpu) (gh)
BartForConditionalGeneration
inductor / cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
'Test'
inductor / cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
'Test'
inductor / cuda12.6-py3.10-gcc9-sm86 / test (inductor_torchbench, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
'Test'
inductor / cuda12.6-py3.10-gcc9-sm86 / test (inductor_torchbench, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
'Test'
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (cpu_inductor_torchbench, 1, 2, linux.8xlarge.amx) (gh)
'Test'
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (cpu_inductor_torchbench, 2, 2, linux.8xlarge.amx) (gh)
'Test'
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (dynamic_cpu_inductor_huggingface, 1, 1, linux.8xlarge.amx) (gh)
BartForConditionalGeneration
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (dynamic_cpu_inductor_timm, 1, 2, linux.8xlarge.amx) (gh)
'Test'
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (dynamic_cpu_inductor_timm, 2, 2, linux.8xlarge.amx) (gh)
'Test'
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (dynamic_cpu_inductor_torchbench, 1, 2, linux.8xlarge.amx) (gh)
'Test'
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (dynamic_cpu_inductor_torchbench, 2, 2, linux.8xlarge.amx) (gh)
'Test'
inductor / linux-jammy-cpu-py3.9-gcc11-inductor / test (inductor_torchbench_cpu_smoketest_perf, 1, 1, linux.24xl.spr-metal) (gh)
Process completed with exit code 1.
inductor / unit-test / cuda12.6-py3.10-gcc9-sm86 / test (inductor_cpp_wrapper, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
inductor/test_cpu_repro.py::CPUReproTests::test_horizontal_fusion
inductor / unit-test / cuda12.6-py3.10-gcc9-sm86 / test (inductor_cpp_wrapper, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_linalg_cholesky_ex_cuda_float32
inductor / unit-test / cuda12.6-py3.10-gcc9-sm86 / test (inductor, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___rmatmul___cuda_float32
inductor / unit-test / cuda12.6-py3.10-gcc9-sm86 / test (inductor, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_amin_cuda_float32
inductor / unit-test / cuda12.6-py3.12-gcc9-sm86 / test (inductor, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___rdiv___cuda_float32
inductor / unit-test / cuda12.6-py3.12-gcc9-sm86 / test (inductor, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
'test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_max_pool3d_cuda_float32'
inductor / unit-test / cuda12.6-py3.13-gcc9-sm86 / test (inductor, 1, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___rmatmul___cuda_float16
inductor / unit-test / cuda12.6-py3.13-gcc9-sm86 / test (inductor, 2, 2, linux.g5.4xlarge.nvidia.gpu) (gh)
'test/test_ops.py::TestCommonCUDA::test_variant_consistency_eager_nn_functional_max_pool3d_cuda_float32'
inductor / unit-test / linux-jammy-cpu-py3.12-gcc11-inductor-halide / test (inductor-halide, 1, 1, linux.12xlarge) (gh)
inductor/test_halide.py::HalideCpuTests::test_resize_cpu_halide
inductor / unit-test / linux-jammy-cpu-py3.12-gcc11-inductor-triton-cpu / test (inductor-triton-cpu, 1, 1, linux.12xlarge) (gh)
inductor/test_triton_cpu_backend.py::CpuTritonTests::test_resize_cpu
inductor / unit-test / linux-jammy-cpu-py3.9-gcc11-inductor / test (inductor_amx, 1, 2, linux.8xlarge.amx) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive_amax_cpu_int32
inductor / unit-test / linux-jammy-cpu-py3.9-gcc11-inductor / test (inductor_amx, 2, 2, linux.8xlarge.amx) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive_masked_amax_cpu_int32
inductor / unit-test / linux-jammy-cpu-py3.9-gcc11-inductor / test (inductor_avx2, 1, 2, linux.10xlarge.avx2) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive_cov_cpu_float32
inductor / unit-test / linux-jammy-cpu-py3.9-gcc11-inductor / test (inductor_avx2, 2, 2, linux.10xlarge.avx2) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive_as_strided_partial_views_cpu_int64
inductor-rocm / rocm-py3.10-inductor / test (inductor, 1, 2, linux.rocm.gpu.2) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___rmatmul___cuda_float32
inductor-rocm / rocm-py3.10-inductor / test (inductor, 2, 2, linux.rocm.gpu.2) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive___rmatmul___cuda_float16
Lint / lintrunner-noclang / linux-job (gh)
>>> Lint for torch/compiler/config.py:
pull / linux-focal-cuda12.6-py3.10-gcc11 / test (default, 1, 5, linux.4xlarge.nvidia.gpu) (gh)
inductor/test_cpu_repro.py::CPUReproTests::test_horizontal_fusion
pull / linux-focal-cuda12.6-py3.10-gcc11 / test (default, 4, 5, linux.4xlarge.nvidia.gpu) (gh)
dynamo/test_recompiles.py::RecompileTests::test_automatic_dynamic_reduce_recompiles
pull / linux-focal-cuda12.6-py3.10-gcc11-sm89 / test (default, 1, 5, linux.g6.4xlarge.experimental.nvidia.gpu) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive__softmax_backward_data_cuda_float64
pull / linux-focal-cuda12.6-py3.10-gcc11-sm89 / test (default, 2, 5, linux.g6.4xlarge.experimental.nvidia.gpu) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCUDA::test_comprehensive_amax_cuda_float32
pull / linux-focal-cuda12.6-py3.10-gcc11-sm89 / test (default, 3, 5, linux.g6.4xlarge.experimental.nvidia.gpu) (gh)
inductor/test_torchinductor.py::GPUTests::test_resize_cuda
pull / linux-focal-cuda12.6-py3.10-gcc11-sm89 / test (default, 4, 5, linux.g6.4xlarge.experimental.nvidia.gpu) (gh)
inductor/test_cpu_repro.py::CPUReproTests::test_horizontal_fusion
pull / linux-focal-cuda12.6-py3.10-gcc11-sm89 / test (default, 5, 5, linux.g6.4xlarge.experimental.nvidia.gpu) (gh)
inductor/test_codecache.py::TestFxGraphCache::test_cache_hot_load_device_cpu_bfloat16_dynamic_False
pull / linux-focal-py3.13-clang10 / test (default, 1, 5, linux.4xlarge) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive_as_strided_partial_views_cpu_float32
pull / linux-focal-py3.13-clang10 / test (default, 2, 5, linux.4xlarge) (gh)
inductor/test_torchinductor.py::CpuTests::test_resize_cpu
pull / linux-focal-py3.13-clang10 / test (default, 3, 5, linux.4xlarge) (gh)
dynamo/test_recompiles.py::RecompileTests::test_automatic_dynamic_reduce_recompiles
pull / linux-focal-py3.13-clang10 / test (default, 4, 5, linux.4xlarge) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive_combinations_cpu_float64
pull / linux-focal-py3.13-clang10 / test (default, 5, 5, linux.4xlarge) (gh)
inductor/test_cpu_repro.py::CPUReproTests::test_horizontal_fusion
pull / linux-focal-py3.13-clang10 / test (dynamo_wrapped, 1, 3, linux.2xlarge) (gh)
test_nn.py::TestNNDeviceTypeCPU::test_upsamplingBiMode2d_consistency_memory_format1_mode_bicubic_antialias_False_align_corners_False_num_channels_3_output_size_32_check_as_unsqueezed_3d_tensor_False_non_contig_False_batch_size_1_cpu
pull / linux-focal-py3.13-clang10 / test (dynamo_wrapped, 2, 3, linux.2xlarge) (gh)
test_nn.py::TestNNDeviceTypeCPU::test_upsamplingBiMode2d_consistency_memory_format1_mode_bicubic_antialias_False_align_corners_False_num_channels_3_output_size_32_check_as_unsqueezed_3d_tensor_False_non_contig_False_batch_size_5_cpu
pull / linux-focal-py3.13-clang10 / test (dynamo_wrapped, 3, 3, linux.2xlarge) (gh)
functorch/test_eager_transforms.py::TestJacCPU::test_argnums_tuple_jacfwd_cpu
pull / linux-focal-py3.9-clang10 / test (default, 1, 5, linux.4xlarge) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive_combinations_cpu_float16
pull / linux-focal-py3.9-clang10 / test (default, 2, 5, linux.4xlarge) (gh)
inductor/test_torchinductor.py::CpuTests::test_resize_cpu
pull / linux-focal-py3.9-clang10 / test (default, 3, 5, linux.4xlarge) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive_cov_cpu_float32
pull / linux-focal-py3.9-clang10 / test (default, 4, 5, linux.4xlarge) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive_as_strided_partial_views_cpu_int64
pull / linux-focal-py3.9-clang10 / test (default, 5, 5, linux.4xlarge) (gh)
inductor/test_cpu_repro.py::CPUReproTests::test_horizontal_fusion
pull / linux-focal-py3.9-clang10 / test (dynamo_wrapped, 1, 3, linux.2xlarge) (gh)
test_nn.py::TestNNDeviceTypeCPU::test_upsamplingBiMode2d_consistency_memory_format1_mode_bicubic_antialias_False_align_corners_False_num_channels_3_output_size_32_check_as_unsqueezed_3d_tensor_False_non_contig_False_batch_size_1_cpu
pull / linux-focal-py3.9-clang10 / test (dynamo_wrapped, 2, 3, linux.2xlarge) (gh)
'test/profiler/test_profiler.py::TestProfiler::test_source'
pull / linux-focal-py3.9-clang10 / test (dynamo_wrapped, 3, 3, linux.2xlarge) (gh)
functorch/test_eager_transforms.py::TestJacCPU::test_argnums_tuple_jacfwd_cpu
pull / linux-jammy-py3.10-clang15-asan / test (default, 2, 6, linux.4xlarge) (gh)
inductor/test_cpu_repro.py::CPUReproTests::test_horizontal_fusion
pull / linux-jammy-py3.10-clang15-asan / test (default, 4, 6, linux.4xlarge) (gh)
inductor/test_torchinductor.py::CpuTests::test_resize_cpu
pull / linux-jammy-py3.9-gcc11 / test (default, 1, 5, linux.2xlarge) (gh)
inductor/test_cpu_repro.py::CPUReproTests::test_horizontal_fusion
pull / linux-jammy-py3.9-gcc11 / test (default, 2, 5, linux.2xlarge) (gh)
inductor/test_torchinductor.py::CpuTests::test_resize_cpu
pull / linux-jammy-py3.9-gcc11 / test (default, 3, 5, linux.2xlarge) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive_combinations_cpu_float64
pull / linux-jammy-py3.9-gcc11 / test (default, 4, 5, linux.2xlarge) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive_as_strided_partial_views_cpu_float64
pull / linux-jammy-py3.9-gcc11 / test (default, 5, 5, linux.2xlarge) (gh)
inductor/test_torchinductor_opinfo.py::TestInductorOpInfoCPU::test_comprehensive_combinations_cpu_bool

FLAKY - The following job failed but was likely due to flakiness present on trunk:

pull / cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu) (gh) (similar failure)
MISSING REGRESSION TEST

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: 4ee406c Pull-Request-resolved: #154098

github-actions · 2025-05-22T04:46:04Z

This PR needs a `release notes:` label

If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[ghstack-poisoned]

ghstack-source-id: 228eb51 Pull-Request-resolved: #154098

[ghstack-poisoned]

ghstack-source-id: e2f8ea7 Pull-Request-resolved: #154098

laithsakka · 2025-05-22T17:13:49Z

I like this

Update

da86d22

[ghstack-poisoned]

bobrenjc93 added a commit that referenced this pull request May 22, 2025

add sticky cache pgo

88865bc

ghstack-source-id: 4ee406c Pull-Request-resolved: #154098

bobrenjc93 mentioned this pull request May 22, 2025

catch CSE recursion depth errors #154039

Closed

pytorch-bot bot added ciflow/inductor module: dynamo labels May 22, 2025

Update

b6159dc

[ghstack-poisoned]

bobrenjc93 added a commit that referenced this pull request May 22, 2025

add sticky cache pgo

eff83f4

ghstack-source-id: 228eb51 Pull-Request-resolved: #154098

Update

45511a6

[ghstack-poisoned]

bobrenjc93 added a commit that referenced this pull request May 22, 2025

add sticky cache pgo

2d0a287

ghstack-source-id: e2f8ea7 Pull-Request-resolved: #154098

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

add sticky cache pgo #154098

add sticky cache pgo #154098

bobrenjc93 commented May 22, 2025 •

edited by pytorch-bot bot

Loading

Uh oh!

pytorch-bot bot commented May 22, 2025 •

edited

Loading

Uh oh!

github-actions bot commented May 22, 2025

Uh oh!

laithsakka commented May 22, 2025

Uh oh!

Uh oh!

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

add sticky cache pgo #154098

Are you sure you want to change the base?

add sticky cache pgo #154098

Conversation

bobrenjc93 commented May 22, 2025 • edited by pytorch-bot bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented May 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/154098

❌ 61 New Failures, 1 Unrelated Failure

Uh oh!

github-actions bot commented May 22, 2025

This PR needs a release notes: label

Uh oh!

laithsakka commented May 22, 2025

Uh oh!

Uh oh!

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

bobrenjc93 commented May 22, 2025 •

edited by pytorch-bot bot

Loading

pytorch-bot bot commented May 22, 2025 •

edited

Loading

This PR needs a `release notes:` label