Skip to content

Feat: Lazy-load relationships #9669

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 15 commits into
base: 1.7.x
Choose a base branch
from
Open

Conversation

ItzNotABug
Copy link
Member

@ItzNotABug ItzNotABug commented Apr 19, 2025

What does this PR do?

Skips fetching related documents unless explicitly requested via select queries.

  1. Default behavior

    GET

    /v1/databases/theatre/collections/movies/documents/avengers
    

    Response

    {
        "title": "Avengers",
        "$id": "avengers",
        "$permissions": [],
        "$databaseId": "theatre",
        "$collectionId": "movies",
        "$createdAt": "2025-04-19T11:59:18.339+00:00",
        "$updatedAt": "2025-04-19T11:59:18.339+00:00"
    }
  2. With nested relationship selects

    GET

    /v1/databases/theatre/collections/movies/documents/avengers
    

    PARAMS

    queries[0]: {"method":"select","values":["ratings.score"]}
    queries[1]: {"method":"select","values":["actors.name","actors.role"]}
    

    Response

    {
        "title": "Avengers",
        "actors": [
            {
                "name": "Chris Evans",
                "role": "Captain America",
                "$databaseId": "theatre",
                "$collectionId": "actors"
            },
            ...
        ],
        "ratings": [
            {
                "score": 9,
                "$databaseId": "theatre",
                "$collectionId": "ratings"
            }
        ],
        "$databaseId": "theatre",
        "$collectionId": "movies"
    }

Test Plan

Manual.

Related PRs and Issues

Checklist

  • Have you read the Contributing Guidelines on issues?
  • If the PR includes a change to an API's metadata (desc, label, params, etc.), does it also include updated API specs and example docs?

Summary by CodeRabbit

  • New Features
    • Improved handling of related documents in database queries, ensuring backward compatibility for older request formats by automatically including related documents when needed.
  • Bug Fixes
    • Enhanced query parsing to better manage selection clauses and related data retrieval.
  • Chores
    • Updated dependencies for improved stability and compatibility with newer versions.

@ItzNotABug ItzNotABug self-assigned this Apr 19, 2025
Copy link

coderabbitai bot commented Apr 19, 2025

Walkthrough

The changes implement conditional relationship traversal in database document queries, based on the presence of selection clauses. In the API controller, the code now checks for selection queries and disables recursive relationship loading when none are present. The request filter system is enhanced to support dependency injection of the project-specific database and route, allowing the filter for version 1.9 to access related collection keys and modify select queries for backward compatibility. Composer dependencies are updated to use a development branch for the database package and a new cache version. The abstract filter and V19 filter classes are extended with new methods and logic to support these features.

Changes

File(s) Change Summary
app/controllers/api/databases.php Added logic to conditionally disable relationship traversal in document queries by checking for selection clauses and using skipRelationships when absent.
app/controllers/general.php Modified filter instantiation to inject the project database and route into the RequestV19 filter for requests with format version < 1.7.0.
composer.json Updated utopia-php/cache to 0.13.*, switched utopia-php/database to a development branch, and added a repositories entry for the database package.
src/Appwrite/Utopia/Request/Filter.php Extended the abstract Filter class with a constructor, private properties for project database and route, and new public methods for accessing them and retrieving parameter values.
src/Appwrite/Utopia/Request/Filters/V19.php Refactored the V19 filter to delegate query management to a new method, which augments select queries for backward compatibility; added methods to retrieve related collection keys and manage selection logic.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant API_Controller
    participant Filter_V19
    participant ProjectDB

    Client->>API_Controller: Request documents (list/get)
    API_Controller->>Filter_V19: Apply request filter (with db and route)
    Filter_V19->>ProjectDB: Retrieve related collection keys (if needed)
    Filter_V19-->>API_Controller: Return possibly modified queries
    API_Controller->>ProjectDB: Fetch documents (skipRelationships if no selections)
    ProjectDB-->>API_Controller: Return documents
    API_Controller-->>Client: Respond with documents
Loading

Poem

In the warren of code, a new path we hop,
Where wildcards and queries tell us when to stop.
With filters now wiser, and dependencies neat,
Our database fetches are nimble and fleet.
Composer brings freshness, the cache is up-to-date—
Oh, what a fine update for us rabbits to celebrate!
🐇✨

✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

github-actions bot commented Apr 19, 2025

Security Scan Results for PR

Docker Image Scan Results

Package Version Vulnerability Severity
binutils 2.42-r0 CVE-2025-0840 HIGH
libexpat 2.6.4-r0 CVE-2024-8176 HIGH
libxml2 2.12.7-r0 CVE-2024-56171 HIGH
libxml2 2.12.7-r0 CVE-2025-24928 HIGH
libxml2 2.12.7-r0 CVE-2025-27113 HIGH
sqlite-libs 3.45.3-r1 CVE-2025-29087 HIGH
xz 5.6.2-r0 CVE-2025-31115 HIGH
xz-libs 5.6.2-r0 CVE-2025-31115 HIGH
golang.org/x/crypto v0.31.0 CVE-2025-22869 HIGH

Source Code Scan Results

🎉 No vulnerabilities found!

Copy link

github-actions bot commented Apr 19, 2025

✨ Benchmark results

  • Requests per second: 981
  • Requests with 200 status code: 176,692
  • P99 latency: 0.188673985

⚡ Benchmark Comparison

Metric This PR Latest version
RPS 981 1,398
200 176,692 251,705
P99 0.188673985 0.130505873

@ItzNotABug ItzNotABug marked this pull request as ready for review May 3, 2025 05:04
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (3)
src/Appwrite/Utopia/Request/Filter.php (2)

10-12: Consider giving typed properties an explicit default value

Although the constructor normally initialises $route and $dbForProject, it is still possible for an extending class to forget calling parent::__construct(), leaving the properties un-initialised and triggering a Typed property must not be accessed before initialization error. Adding a = null default makes the class more defensive at virtually zero cost.

-    private ?Route $route;
-    private ?Database $dbForProject;
+    private ?Route $route = null;
+    private ?Database $dbForProject = null;

47-56: Broaden the caught throwable & remove redundant null-coalescing

Route::getParamValue() may throw any Throwable, not only Exception (e.g. Error). Catching \Throwable is safer.
Also, the null-coalescing operator already covers the route is null case, so the surrounding try/​catch only needs to protect against the method throwing. This tiny refactor keeps the intent but avoids double work:

-        try {
-            $value = $this->route?->getParamValue($key) ?? $default;
-        } catch (\Exception $e) {
-            $value = $default;
-        }
+        try {
+            return $this->route?->getParamValue($key) ?? $default;
+        } catch (\Throwable) {
+            return $default;
+        }
-
-        return $value;
src/Appwrite/Utopia/Request/Filters/V19.php (1)

30-36: Redundant prefix check

$isDatabaseModel is always true when $model equals either databases.listDocuments or databases.getDocument. Removing the extra str_starts_with simplifies the guard clause:

-        $isDatabaseModel = \str_starts_with($model, 'databases.');
-        $isTargetOperation = \in_array($model, ['databases.listDocuments', 'databases.getDocument']);
-
-        if (! $isDatabaseModel || ! $isTargetOperation) {
+        if (!\in_array($model, ['databases.listDocuments', 'databases.getDocument'], true)) {
             return $content;
         }
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f76aff1 and 04539bf.

⛔ Files ignored due to path filters (1)
  • composer.lock is excluded by !**/*.lock
📒 Files selected for processing (5)
  • app/controllers/api/databases.php (2 hunks)
  • app/controllers/general.php (1 hunks)
  • composer.json (3 hunks)
  • src/Appwrite/Utopia/Request/Filter.php (2 hunks)
  • src/Appwrite/Utopia/Request/Filters/V19.php (1 hunks)
🔇 Additional comments (7)
app/controllers/general.php (1)

833-834:

✅ Verification successful

Dependency injection improves request filter capabilities for handling relationship queries.

The change injects the project database instance and route object into the RequestV19 filter, enabling it to access collection metadata and modify query selections to control relationship loading behavior. This aligns with the PR objective of implementing lazy loading for relationships, ensuring backward compatibility for versions before 1.7.0.


🏁 Script executed:

#!/bin/bash
# Verify how the V19 filter class uses these dependencies

# Look for the V19 filter class implementation
echo "Searching for V19 filter implementation..."
rg --type php -A 15 "class V19" src/

Length of output: 3774


🏁 Script executed:

#!/bin/bash
# Display the full V19 request filter implementation to verify constructor and manageSelectQueries usage
rg --type php -A200 "class V19 extends Filter" src/Appwrite/Utopia/Request/Filters/V19.php

Length of output: 3500


🏁 Script executed:

#!/bin/bash
# Verify Filter base class constructor signature and dependency handling
rg --type php -A200 "class Filter" src/Appwrite/Utopia/Request/Filters/Filter.php

Length of output: 170


🏁 Script executed:

#!/bin/bash
# Search for base Filter class definition across the src directory
rg --type php -n "class Filter" src/

Length of output: 165


🏁 Script executed:

#!/bin/bash
# Show the Request Filter base class implementation
sed -n '1,200p' src/Appwrite/Utopia/Request/Filter.php

# Check for constructor definition in Filter
rg --type php -n "__construct" src/Appwrite/Utopia/Request/Filter.php

Length of output: 1457


Dependency injection correctly enhances RequestV19 filter

Injecting the project’s Database instance and the Route into the RequestV19 filter aligns perfectly with how the base Filter constructor is defined (__construct(Database $dbForProject = null, Route $route = null)). This grants the filter access to:

  • getDbForProject() for resolving collection metadata
  • getParamValue() for extracting databaseId and collectionId from the route

which it uses in manageSelectQueries() to preserve 1.6.x behavior (including related documents by default) while maintaining lazy-loading support for 1.7.x+. This change meets the PR’s goal of backward-compatible relationship loading.

composer.json (3)

52-52: Cache dependency updated.

The utopia-php/cache package has been updated to version 0.13.*, which likely includes improvements or features needed to support the lazy-loading relationship functionality.


97-102: Added custom repository source for the database package.

This configuration directs Composer to fetch the utopia-php/database package directly from the GitHub repository instead of Packagist, which is necessary to use the development branch specified above.


56-56:

❓ Verification inconclusive

Using a development branch for database package.

The dependency on utopia-php/database has been changed from a stable version to a development branch specifically created for handling wildcards in queries. This branch is required for implementing the relationship lazy-loading functionality described in the PR.


🏁 Script executed:

#!/bin/bash
# Check if there's a reference to a related PR for the database package

# Look for references to the related database PR in the current repo
echo "Searching for references to the utopia-php/database PR..."
rg -i "utopia.*database.*PR" --type md

Length of output: 165


Please verify the “dev-manage-wildcards” alias for utopia-php/database

It looks like you’ve switched utopia-php/database to a development branch alias:

• composer.json: line 56

"utopia-php/database": "dev-manage-wildcards as 0.66.0",

Before we merge, confirm that:

  • The dev-manage-wildcards branch exists and is up-to-date in the utopia-php/database repo.
  • That branch has been published (or is accessible) via Packagist under the correct version alias.
  • Any consumers of this package in the repo will still resolve correctly to the expected API.
app/controllers/api/databases.php (2)

3513-3522: Well-implemented optimization of database queries!

This change implements the lazy-loading of relationships by checking for selection queries before deciding how to fetch documents. When no selections are present, the code skips loading relationships, which should significantly improve performance for documents with many relationships.


3673-3684: Consistent implementation of lazy-loading in the getDocument endpoint.

Similar to the change in listDocuments, this implements lazy-loading for single document retrieval. The conditional relationship loading is well-structured and mirrors the implementation in the list endpoint, ensuring consistent behavior throughout the API.

src/Appwrite/Utopia/Request/Filters/V19.php (1)

43-45: Verify key used in Query::groupByType() result

You access the grouped array via ['selections'].
In Utopia Database, the key is usually derived from the constant (Query::TYPE_SELECT), so it may be 'selects'. A mismatch yields an empty $selections, making the wildcard-detection loop ineffective when explicit selects are supplied.

Please double-check the constant name or update the index:

$groups = Query::groupByType($parsed);
$selections = $groups[Query::TYPE_SELECT.'s'] ?? [];

(Adjust if the library already uses the plural form you expect.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy