Skip to content

Rust: extract source files of dependencies #19506

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 32 commits into from
May 24, 2025
Merged

Rust: extract source files of dependencies #19506

Changes from 1 commit
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
dd5c487
Rust: extract source files of depdendencies
aibaars May 15, 2025
f05bed6
Rust: remove module data from Crate elements
aibaars May 15, 2025
980cebe
Rust: fix QL code after removing Crate::getModule()
aibaars May 16, 2025
0bb0a70
Rust: add upgrade/downgrade scripts
aibaars May 16, 2025
8996f9e
Rust: Follow-up work to make path resolution and type inference tests…
hvitved May 19, 2025
1269a2e
Rust: fix extractor-tests
aibaars May 19, 2025
456a4b2
Rust: Make `dataflow/modeled` pass by not using `#[derive(Clone)]`
hvitved May 20, 2025
44a4045
Rust: fixes
aibaars May 19, 2025
643059e
Rust: fix type-interence file paths
aibaars May 20, 2025
67846f1
fixup TestUtils
aibaars May 20, 2025
3761099
Rust: drop Param::pat when extracting libraries
aibaars May 20, 2025
5ee7658
Rust: update DataFlowStep.expected
aibaars May 20, 2025
457632e
Rust: update UncontrolledAllocationSize.expected
aibaars May 20, 2025
e90ab7b
Rust: fix diagnostics tests
aibaars May 20, 2025
76da2e4
Rust: drop crate_graph/modules.ql test
aibaars May 20, 2025
81f0e42
Rust: improve ExtractionConsistency.ql
aibaars May 20, 2025
f093c49
Rust: normalize file paths for PathResolutionConsistency.ql
aibaars May 20, 2025
9ee0d2e
Rust: Exclude flow summary nodes from `DataFlowStep.ql`
hvitved May 21, 2025
c69aa22
Rust: restrict to library files
aibaars May 21, 2025
1eaa491
Rust: update integration tests
aibaars May 21, 2025
2a93b2a
Rust: integration-tests: update output
aibaars May 21, 2025
fa1a21b
Rust: reduce log-level of diagnostics when extracting library files
aibaars May 21, 2025
a6cd60f
Rust: address comments
aibaars May 21, 2025
28be208
Rust: drop too noisy log statements
aibaars May 21, 2025
76737cb
Rust: Follow-up changes after rebase
hvitved May 22, 2025
7e5f652
Rust: disable ResolvePaths when extracting library source files
aibaars May 22, 2025
a4788fd
Rust: update expected output
aibaars May 22, 2025
df99e06
Rust: temporarily disable attribute macro expansion in library mode
aibaars May 23, 2025
b62d52e
Rust: prevent source files from being extracted in both source and li…
aibaars May 23, 2025
23b4e50
Rust: update expected output
aibaars May 23, 2025
c8ff69a
Rust: Fix bad join
hvitved May 23, 2025
5b28ff1
Merge branch 'main' into aibaars/rust-extract-libs
aibaars May 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Rust: address comments
  • Loading branch information
@aibaars @hvitved
aibaars authored and hvitved committed May 22, 2025
commit a6cd60f20e647116292d2048a96343a63e5af935
48 changes: 28 additions & 20 deletions rust/extractor/src/translate/base.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -640,33 +640,41 @@ impl<'a> Translator<'a> {
pub(crate) fn should_be_excluded(&self, item: &impl ast::AstNode) -> bool {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot shake the feeling this is a bit a roundabout way of doing it: what I mean, is reconstructing the context around the AstNode by visiting the parent, when the emission was requested by emitting the parent already.

I wonder if we shouldn't do this by generating a new exclusion point where the emission of the body takes place. What about having this in the template for ast node emission:

    pub(crate) fn emit_{{snake_case_name}}(&mut self, node: &ast::{{ast_name}}) -> Option<Label<generated::{{name}}>> {
        pre_emit!({{name}}, self, node);
        {{#has_attrs}}
        if self.should_be_excluded(node) { return None; }
        {{/has_attrs}}
        {{#fields}}
        let {{name}} =
            {{#is_body}}
            if self.should_skip_bodies() { None } else {
            {{/is_body}}
            {{#predicate}}
            node.{{method}}().is_some()
            {{/predicate}}
            {{#string}}
            node.try_get_text()
            {{/string}}
            {{#list}}
            node.{{method}}().filter_map(|x| self.emit_{{snake_case_ty}}(&x)).collect()
            {{/list}}
            {{#optional}}
            node.{{method}}().and_then(|x| self.emit_{{snake_case_ty}}(&x))
            {{/optional}}
            {{#is_body}}
            }
            {{/is_body}}
            ;
        {{/fields}}
        ...

(and adding is_body: field.name == "body", to the FieldType::Optional(ty) => ExtractorNodeFieldInfo creation). This will cover all possible bodies, and avoid their emission at the sources without working up the AST. For example:

        let body = if self.should_skip_bodies() {
            None
        } else {
            node.body().and_then(|x| self.emit_expr(&x))
        };

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, the above does not also skip pat, but something could be done there at generation level as well

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here's what I mean: #19559 shifts the logic from climbing up the AST in the base extractor to generating the property skipping at field emission level, with the logic in the ast-generator instead.

if self.source_kind == SourceKind::Library {
let syntax = item.syntax();
if let Some(body) = syntax.parent().and_then(Fn::cast).and_then(|x| x.body()) {
if body.syntax() == syntax {
tracing::debug!("Skipping Fn body");
return true;
}
if syntax
.parent()
.and_then(Fn::cast)
.and_then(|x| x.body())
.is_some_and(|body| body.syntax() == syntax)
{
tracing::debug!("Skipping Fn body");
return true;
}
if let Some(body) = syntax.parent().and_then(Const::cast).and_then(|x| x.body()) {
if body.syntax() == syntax {
tracing::debug!("Skipping Const body");
return true;
}
if syntax
.parent()
.and_then(Const::cast)
.and_then(|x| x.body())
.is_some_and(|body| body.syntax() == syntax)
{
tracing::debug!("Skipping Const body");
return true;
}
if let Some(body) = syntax
if syntax
.parent()
.and_then(Static::cast)
.and_then(|x| x.body())
.is_some_and(|body| body.syntax() == syntax)
{
if body.syntax() == syntax {
tracing::debug!("Skipping Static body");
return true;
}
tracing::debug!("Skipping Static body");
return true;
}
if let Some(pat) = syntax.parent().and_then(Param::cast).and_then(|x| x.pat()) {
if pat.syntax() == syntax {
tracing::debug!("Skipping parameter");
return true;
}
if syntax
.parent()
.and_then(Param::cast)
.and_then(|x| x.pat())
.is_some_and(|pat| pat.syntax() == syntax)
{
tracing::debug!("Skipping parameter");
return true;
}
}
false
Expand Down
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy