StackQL Provider Utils

A comprehensive toolkit for transforming OpenAPI specs into StackQL providers. Includes parsing, mapping, validation, testing, and documentation generation utilities. Compatible with both Node.js and Deno.

Table of Contents

• Prerequisites
• Installation
• Local Development Setup
• Testing with Node.js
• Testing with Deno
• Using the Documentation Generator
• API Reference
• Contributing

Prerequisites

For Node.js

• Node.js >= 20
• npm or yarn
• StackQL server (for documentation generation)

For Deno

• Deno >= 1.30.0
• StackQL server (for documentation generation)

Installing StackQL

Download and install StackQL from stackql.io/downloads

# macOS
brew install stackql

# Linux
curl -L https://bit.ly/stackql-zip -O && unzip stackql-zip

# Windows
# Download from https://stackql.io/downloads

Installation

For Node.js Projects

npm install @stackql/provider-utils
# or
yarn add @stackql/provider-utils

For Deno Projects

import { docgen } from "https://deno.land/x/stackql_provider_utils/mod.ts";

Local Development Setup

1. Clone the repository:

git clone https://github.com/stackql/stackql-provider-utils.git
cd stackql-provider-utils

2. Install dependencies (Node.js):

npm install

Testing with Node.js

1. Create a Test Script

Create a file test-docgen.js:

import { docgen } from './src/index.js';

// Test the documentation generator
async function testDocGen() {
    try {
        const result = await docgen.generateDocs({
            providerName: 'myservice',
            providerDir: './test-data/output/src/myservice/v00.00.00000',
            outputDir: './test-output',
            providerDataDir: './test-data/provider-data',
            stackqlConfig: {
                host: 'localhost',
                port: 5444,
                user: 'stackql',
                database: 'stackql'
            }
        });
        
        console.log('Documentation generated successfully:', result);
    } catch (error) {
        console.error('Error generating documentation:', error);
    }
}

testDocGen();

2. Set Up Test Data

Create the required directory structure:

mkdir -p test-data/output/src/myservice/v00.00.00000/services
mkdir -p test-data/provider-data

Add test files:

test-data/provider-data/headerContent1.txt:

---
title: myservice
hide_title: false
hide_table_of_contents: false
keywords:
  - myservice
  - stackql
  - infrastructure-as-code
  - configuration-as-data
description: Query and manage myservice resources using SQL
---

# myservice Provider

The myservice provider for StackQL allows you to query, deploy, and manage myservice resources using SQL.

test-data/provider-data/headerContent2.txt:

See the [myservice provider documentation](https://myservice.com/docs) for more information.

test-data/output/src/myservice/v00.00.00000/services/example.yaml:

openapi: 3.0.0
info:
  title: Example Service
  version: 1.0.0
paths:
  /examples:
    get:
      operationId: listExamples
      responses:
        '200':
          description: Success
components:
  x-stackQL-resources:
    examples:
      id: myservice.example.examples
      name: examples
      title: Examples
      methods:
        list:
          operation:
            $ref: '#/paths/~1examples/get'
          response:
            mediaType: application/json
            openAPIDocKey: '200'
      sqlVerbs:
        select:
          - $ref: '#/components/x-stackQL-resources/examples/methods/list'

3. Start StackQL Server

# In a separate terminal
stackql srv \
  --pgsrv.port=5444 \
  --pgsrv.tls=false \
  --loglevel=INFO

4. Run the Test

node test-docgen.js

Testing with Deno

1. Create a Test Script

Create a file test-docgen.ts:

import { docgen } from './src/mod.ts';

// Test the documentation generator
async function testDocGen() {
    try {
        const result = await docgen.generateDocs({
            providerName: 'myservice',
            providerDir: './test-data/output/src/myservice/v00.00.00000',
            outputDir: './test-output',
            providerDataDir: './test-data/provider-data',
            stackqlConfig: {
                host: 'localhost',
                port: 5444,
                user: 'stackql',
                database: 'stackql'
            }
        });
        
        console.log('Documentation generated successfully:', result);
    } catch (error) {
        console.error('Error generating documentation:', error);
    }
}

testDocGen();

2. Run the Test

# With permissions
deno run --allow-read --allow-write --allow-net test-docgen.ts

Using the Documentation Generator

Basic Example

import { docgen } from '@stackql/provider-utils';

const options = {
    providerName: 'github',
    providerDir: './output/src/github/v00.00.00000',
    outputDir: './docs',
    providerDataDir: './config/provider-data',
    stackqlConfig: {
        host: 'localhost',
        port: 5444,
        user: 'stackql',
        database: 'stackql'
    }
};

const result = await docgen.generateDocs(options);
console.log(`Generated docs for ${result.totalServices} services and ${result.totalResources} resources`);
console.log(`Output location: ${result.outputPath}`);

Options

Option	Type	Description	Default
providerName	string	Name of the provider (e.g., 'github', 'aws')	Required
providerDir	string	Path to provider spec directory	Required
outputDir	string	Directory for generated documentation	Required
providerDataDir	string	Directory containing provider header files	Required
stackqlConfig	object	StackQL server connection configuration	See below

StackQL Config Options

{
    host: 'localhost',  // StackQL server host
    port: 5444,         // StackQL server port
    user: 'stackql',    // Database user
    database: 'stackql' // Database name
}

Directory Structure Requirements

Provider Data Directory

provider-data/
├── headerContent1.txt    # Provider introduction
├── headerContent2.txt    # Additional provider info
└── stackql-provider-registry.mdx (optional)

Provider Spec Directory

output/src/{provider}/v00.00.00000/
├── provider.yaml
└── services/
    ├── service1.yaml
    ├── service2.yaml
    └── ...

Generated Output

docs/{provider}-docs/
├── index.md
├── stackql-provider-registry.mdx
└── providers/
    └── {provider}/
        └── {service}/
            ├── index.md
            └── {resource}/
                └── index.md

Running Tests

Node.js

npm test

Deno

deno task test

Troubleshooting

StackQL Server Connection Issues

• Ensure StackQL server is running: stackql srv --pgsrv.port=5444
• Check if port 5444 is available
• Verify connection settings in stackqlConfig

Missing Provider Data

• Ensure headerContent1.txt and headerContent2.txt exist in provider data directory
• Check file permissions

Empty Documentation

• Verify provider specs have x-stackQL-resources components
• Check that resources have proper method definitions

API Reference

docgen.generateDocs(options)

Generates documentation for a StackQL provider.

Parameters:

• options (Object): Configuration options

Returns:

• Promise: Result object containing:
  • totalServices: Number of services processed
  • totalResources: Number of resources documented
  • outputPath: Path to generated documentation

  Example:

  const result = await docgen.generateDocs({
      providerName: 'aws',
      providerDir: './providers/src/aws/v00.00.00000',
      outputDir: './documentation',
      providerDataDir: './config/aws',
      stackqlConfig: {
          host: 'localhost',
          port: 5444
      }
  });

  docgen.createResourceIndexContent(...)

  Creates markdown content for a single resource. This is a lower-level function used internally by generateDocs.

  Contributing

  1. Fork the repository
  2. Create your feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add some amazing feature')
  4. Push to the branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

  License

  MIT

  Support

  • StackQL Documentation
  • GitHub Issues
  • StackQL Discord