Skip to content

[fake tensor] fix issue of no attribute tags #156689

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 2 commits into from
Closed

[fake tensor] fix issue of no attribute tags #156689

wants to merge 2 commits into from

Conversation

Valentine233
Copy link
Collaborator

Fixes #156688

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jun 24, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/156689

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit b4cbcc8 with merge base 0364db7 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@leslie-fang-intel leslie-fang-intel added ciflow/trunk Trigger trunk jobs on your pull request topic: not user facing topic category labels Jun 24, 2025
atalman
atalman reviewed Jun 24, 2025
View reviewed changes
torch/_subclasses/fake_tensor.py
@@ -2774,7 +2774,7 @@ def validate(x: T) -> Union[T, FakeTensor]:

nonlocal flat_arg_fake_tensors
if not self.is_our_fake(x):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Valentine233 could you please also add unit test for this change ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @atalman , the UT has been added.

@Valentine233 Valentine233 requested a review from atalman June 25, 2025 05:22
@mikaylagawarecki mikaylagawarecki added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Jun 26, 2025
leslie-fang-intel
leslie-fang-intel reviewed Jun 27, 2025
View reviewed changes
torch/_subclasses/fake_tensor.py
@@ -2774,7 +2774,7 @@ def validate(x: T) -> Union[T, FakeTensor]:

nonlocal flat_arg_fake_tensors
if not self.is_our_fake(x):
if torch.Tag.inplace_view in func.tags:
if hasattr(func, "tags") and torch.Tag.inplace_view in func.tags:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes in this PR looks good to me. But
when auditing the code, I noticed many instances where func.tags is used without checking whether func actually has a tags attribute. Do we need to review other parts of the code for similar issues or should we add the tags attribute to HigherOrderOperator? also cc @bdhirsh

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, good point! We can discuss more in this thread, and do the follow-up in the future.
But I suppose we can firstly land this PR, as urgently required in v2.8 release.

Copy link
Collaborator

@leslie-fang-intel leslie-fang-intel Jul 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, good point! We can discuss more in this thread, and do the follow-up in the future. But I suppose we can firstly land this PR, as urgently required in v2.8 release.

Looks good to me given the changes in this PR should be no side effect. Will you further work on for a formal solution?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe needs some inputs from @bdhirsh about why HigherOrderOperator does not have tags attribute. Is this part of design or should we add tags for HigherOrderOperator?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @drisspg @zou3519 - this looks like it's because of #151719

We now plumb FlexAttention through FakeTensorMode.__torch_dispatch__, but it looks like FakeTensorMode expects HigherOrderOperators that we plumb through to have tags.

My vote would probably just be to add a .tag field to FlexAttention to make it more OpOverload-like, what do you both think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH I dont have the best context on when tags are used, do all HOPs not have tags or just FlexAttention?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all HOPs don't have tags right now

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bdhirsh @zou3519 @drisspg Thanks, would you like to add tags for HOPs as a future step?

@Valentine233
Copy link
Collaborator Author

@atalman @bdhirsh Could you please help review? This needs to be cherry-picked in release v2.8.0.

@Valentine233 Valentine233 force-pushed the fix_fake_tensor_issue branch from 91c5925 to b4cbcc8 Compare July 2, 2025 07:45
leslie-fang-intel
leslie-fang-intel approved these changes Jul 2, 2025
View reviewed changes
Copy link
Collaborator

@leslie-fang-intel leslie-fang-intel left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, since the changes in this PR should have no side effects. I'm not sure, but perhaps we could consider adding the tags attribute to the higher-order operator as a future improvement.

atalman
atalman approved these changes Jul 2, 2025
View reviewed changes
Copy link
Contributor

@atalman atalman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@Valentine233
Copy link
Collaborator Author

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

Valentine233 added a commit that referenced this pull request Jul 3, 2025
@Valentine233
[fake tensor] fix issue of no attribute tags (#156689)
2d9f970 
Fixes #156688

Pull Request resolved: #156689
Approved by: https://github.com/leslie-fang-intel, https://github.com/atalman

(cherry picked from commit 7597988)
@Valentine233 Valentine233 mentioned this pull request Jul 3, 2025
Open
malfet pushed a commit that referenced this pull request Jul 7, 2025
@Valentine233
[cherry-pick] [fake tensor] fix issue of no attribute tags (#156689) (#…
b10409a 
…157519)

[fake tensor] fix issue of no attribute tags (#156689)

Fixes #156688

Pull Request resolved: #156689
Approved by: https://github.com/leslie-fang-intel, https://github.com/atalman

(cherry picked from commit 7597988)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zou3519 zou3519 zou3519 left review comments

@drisspg drisspg drisspg left review comments

@atalman atalman atalman approved these changes

@leslie-fang-intel leslie-fang-intel leslie-fang-intel approved these changes

@bdhirsh bdhirsh Awaiting requested review from bdhirsh

Assignees
No one assigned
Labels
ciflow/inductor ciflow/trunk Trigger trunk jobs on your pull request Merged open source topic: not user facing topic category triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[CPU][flex attention] Llama 3 failed on CPU with PyTorch 2025-06-22 nightly wheel
9 participants
@Valentine233 @pytorchmergebot @zou3519 @atalman @bdhirsh @drisspg @leslie-fang-intel @pytorchbot @mikaylagawarecki
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy