Skip to content

[cp] dispatch flex_attention on DTensor to cp implementation #151900

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 1 commit into
base: gh/XilunWu/136/base
Choose a base branch
from
Draft

[cp] dispatch flex_attention on DTensor to cp implementation #151900

wants to merge 1 commit into from

Conversation

XilunWu
Copy link
Contributor

@XilunWu XilunWu commented Apr 22, 2025 •
edited
Loading

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Apr 22, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/151900

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 1 Unrelated Failure

As of commit 11c155d with merge base b7c7000 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added ciflow/inductor oncall: distributed Add this issue/PR to distributed oncall triage queue labels Apr 22, 2025
This was referenced Apr 22, 2025
Closed
Open
Closed
XilunWu added a commit that referenced this pull request Apr 22, 2025
@XilunWu
[cp] Add e2e flex_attention test w/ causal masking
56bf490 
ghstack-source-id: af830b8
Pull Request resolved: #151900
@github-actions GitHub Actions
Copy link
Contributor

This PR needs a release notes: label

If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@XilunWu XilunWu marked this pull request as draft April 22, 2025 09:36
This was referenced Apr 22, 2025
Closed
Closed
@XilunWu XilunWu changed the title [cp] Add e2e flex_attention test w/ causal masking [cp] dispatch flex_attention on DTensor to cp implementation Apr 24, 2025
XilunWu added a commit that referenced this pull request Apr 24, 2025
@XilunWu
[cp] dispatch flex_attention to CP impl in TorchDispatchMode
063d63c 
ghstack-source-id: 9f052d6
Pull Request resolved: #151497

DTensor HOP call in TorchDispatchMode

ghstack-source-id: 9f052d6
Pull Request resolved: #151685

[cp] Add e2e flex_attention test w/ causal masking

ghstack-source-id: 9f052d6
Pull Request resolved: #151900

[cp] Context Parallel: dispatch flex_attention to CP impl in TorchDispatchMode

ghstack-source-id: 9f052d6
Pull Request resolved: #151903
XilunWu added a commit that referenced this pull request Apr 24, 2025
@XilunWu
[cp] dispatch flex_attention to CP impl in TorchDispatchMode
86cf8ad 
ghstack-source-id: e098b23
Pull Request resolved: #151497

DTensor HOP call in TorchDispatchMode

ghstack-source-id: e098b23
Pull Request resolved: #151685

[cp] Add e2e flex_attention test w/ causal masking

ghstack-source-id: e098b23
Pull Request resolved: #151900

[cp] Context Parallel: dispatch flex_attention to CP impl in TorchDispatchMode

ghstack-source-id: e098b23
Pull Request resolved: #151903
XilunWu added a commit that referenced this pull request Apr 27, 2025
@XilunWu
[cp] dispatch flex_attention to CP impl in TorchDispatchMode
1a440d1 
ghstack-source-id: 4524be1
Pull Request resolved: #151497

DTensor HOP call in TorchDispatchMode

ghstack-source-id: 4524be1
Pull Request resolved: #151685

[cp] Add e2e flex_attention test w/ causal masking

ghstack-source-id: 4524be1
Pull Request resolved: #151900

[cp] Context Parallel: dispatch flex_attention to CP impl in TorchDispatchMode

ghstack-source-id: 4524be1
Pull Request resolved: #151903
@github-actions GitHub Actions
Copy link
Contributor

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

@github-actions github-actions bot added the Stale label Jun 23, 2025
superiwan pushed a commit to superiwan/pytorch that referenced this pull request Jul 14, 2025
@XilunWu
[cp] Add e2e flex_attention test w/ causal masking
2feae0a 
ghstack-source-id: af830b8
Pull Request resolved: pytorch/pytorch#151900
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
ciflow/inductor oncall: distributed Add this issue/PR to distributed oncall triage queue Stale
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@XilunWu
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy