pFad - Phone/Frame/Anonymizer/Declutterfier! Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

URL: http://github.com/izure1/document-dataply

ssorigen="anonymous" media="all" rel="stylesheet" href="https://github.githubassets.com/assets/repository-6534fbc3f5e83ac0.css" /> GitHub - izure1/document-dataply: Simple and powerful JSON document database supporting complex queries and flexible indexing policies. Β· GitHub
Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

168 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

node.js workflow Performance Benchmark License

Document-Dataply

Warning

This project is currently in the Alpha stage. API structures may change in future updates. Please ensure sufficient testing before deploying to production environments.

πŸ“– Introduction

document-dataply is a high-performance Document Database implemented in pure JavaScript. It prevents server memory (RAM) exhaustion even when handling millions of records, and supports ultra-fast searching and batch processing systems. It can be used intuitively without the complex tuning required by RDBMS.

Live Demo

https://document-dataply-demo.onrender.com/

Note

The demo is hosted on a free-tier server, so response times and benchmark results may be noticeably slower than in a production environment. Performance may also vary depending on the hosting server's current load. The demo is intended to showcase the library's functionality rather than provide representative performance measurements.

✨ Key Features

  • JSON Document Based: Reads and writes data in raw JavaScript Object (JSON) format. Easily supports querying deeply nested object arrays (a.b.c).
  • B+Tree Indexing: Built-in indexing system to drastically improve query speed (O(log N)).
  • Cost-Based Optimizer: The engine automatically analyzes data distribution during query requests to find the most optimal execution path.
  • Full-Text Search (FTS): Provides a real-time search engine to find specific keywords within long text, such as article contents.
  • Safe Transactions: If an error occurs during an operation, all modifications are fully rolled back, completely preventing data inconsistency.

πŸ’» Installation

npm install document-dataply
  • Supported Environments: Node.js, Electron, NW.js (Experimental support for Bun, Deno)

πŸš€ Quick Start

Here is the most basic guide for creating a database, registering an index, and querying data.

import { DocumentDataply } from 'document-dataply';

// 1. Define Document Type (Schema)
type UserProfile = {
  name: string;
  age: number;
  tags: string[];
}

async function main() {
  // 2. Create instance and connect to file
  const db = DocumentDataply.Define<UserProfile>()
    .Options({ wal: 'my-database.wal' })
    .Open('my-database.db');
  
  await db.init();

  // 3. Create Index (Specify 'name' field to improve search speed)
  await db.migration(1, async (tx) => {
    await db.createIndex('idx_name', { type: 'btree', fields: ['name'] }, tx);
    console.log('Index created successfully');
  });

  // 4. Insert Single Document
  await db.insert({
    name: 'IU',
    age: 30,
    tags: ['Singer', 'Actor']
  });

  // 5. Query Data
  const query = db.select({
    name: 'IU' 
  });

  // Fetch results as an array at once
  const results = await query.drain();
  console.log(results); 
  // Output: [{ _id: 1, name: 'IU', age: 30, tags: ['Singer', 'Actor'] }]

  // 6. Close the connection
  await db.close();
}

main();

πŸ’‘ Common Patterns

Batch Processing for Performance

const largeData = Array.from({ length: 10000 }, (_, i) => ({
  name: `User_${i}`,
  age: Math.floor(Math.random() * 50),
  tags: ['imported']
}));

// Much faster than individual inserts
await db.insertBatch(largeData);

Complex Queries with Operators

const activeSeniors = await db.select({
  age: { gte: 65 },
  status: { equal: 'active' },
  name: { like: 'John%' }
}).drain();

Upsert Operations (Insert or Full Update)

// Single Upsert: Inserts if _id is missing/not found, updates if _id exists
await db.upsert({ _id: 1, name: 'John Doe', age: 31 });

// Batch Upsert: Process multiple inserts and updates in a single batch
await db.upsertBatch([
  { name: 'New User', age: 20 },                    // Inserted with auto _id
  { _id: 1, name: 'John Doe Updated', age: 32 }    // Updated existing document
]);

πŸ“š Detailed Manual

Please refer to the following documents for detailed usage of each feature and the library's core architecture.

  1. Query & Index Guide
    • Explains various search operators (lt, gt, etc.) and how to configure composite indices.
  2. Mutation & Transaction
    • Covers batch insertion techniques and transaction rollback principles.
  3. Stream Architecture (Memory Optimization)
    • Analyzes the secrets of the stream-based architecture that prevents OOM (Out Of Memory) crashes.
  4. Architecture Overview
    • Deeply explores the internal workings of the database core, such as the optimizer.

πŸ“Š Library Benchmark

document-dataply strictly measures speed upon every code merge to defend against performance degradation.

Check Real-time Performance on Main Branch

πŸ“œ License

MIT License

About

Simple and powerful JSON document database supporting complex queries and flexible indexing policies.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

pFad - Phonifier reborn

Pfad - The Proxy pFad © 2024 Your Company Name. All rights reserved.





Check this box to remove all script contents from the fetched content.



Check this box to remove all images from the fetched content.


Check this box to remove all CSS styles from the fetched content.


Check this box to keep images inefficiently compressed and original size.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy