Poster
Poster
Introduction Methodology
LLVM (Low Level Virtual Machine) is a collection of 1.Pattern Extraction: Extract OpenMP test patterns
modular and reusable compiler and toolchain from LLVM repository using Clang AST analysis to
technologies. It provides a modern, open-source identify directives, clauses, error patterns, and test
infrastructure for building compilers, optimizers, and structures, storing them in SQLite database with
code generation tools. compiler stage classification and complexity scoring.
In the context of this robust compiler infrastructure, this 2.GitHub Integration: Fetch PR metadata, diffs, and
project presents an AI-driven OpenMP test case modified files via GitHub API to extract feature
generation framework that addresses the critical keywords, specification details, and change context
challenge of manual test creation in OpenMP compiler for targeted test generation.
development by leveraging LLVM's capabilities for 3.AI-Powered Generation: Query database for similar
pattern extraction from existing test suites and patterns based on compiler stage and PR context,
integrating with GitHub pull request analysis to construct detailed prompts with extracted patterns
automatically generate contextually relevant compiler and PR information, then use Groq API to generate 1-
test cases. 4 contextually relevant test variants.
4.Multi-Stage Validation: Validate generated tests
through compilation checking, content analysis for
Problem Statement OpenMP directive relevance, quality scoring based
on complexity and coverage, and relevance
OpenMP compiler development within the LLVM
assessment against PR changes.
ecosystem faces a significant challenge in maintaining
5.Output Management: Save generated tests with
comprehensive and relevant test coverage, as the
structured naming (pr_NUMBER_STAGE_test_N.cpp),
manual creation of test cases for new features and bug
generate validation reports with quality metrics, and
fixes is time-consuming, error-prone, and often results in
provide comprehensive feedback on test relevance
inconsistent test quality across different compiler stages
and compilation success rates.
(Parse, Sema, and CodeGen), leading to potential gaps in
6.Documentation: Document the entire project and
validation that can compromise compiler reliability and
also create shell files to install dependencies
slow down the development cycle.
automatically
Objectives Results
Automate OpenMP Test Case Generation: Develop
a comprehensive system to automatically create
contextually relevant OpenMP compiler test cases,
eliminating manual effort and significantly reducing
development time for new OpenMP features and
bug fixes.
Extract and Analyze Existing Test Patterns: Build
an intelligent pattern extraction system that mines
the LLVM OpenMP test repository using Clang AST
matchers to identify, categorize, and store reusable
testing structures and methodologies across
different compiler stages.
Implement AI-Powered Test Synthesis: Leverage
large language models through the Groq API to
generate semantically meaningful test skeletons
based on GitHub pull request analysis and extracted
patterns, ensuring relevance to ongoing
development activities.
Create Multi-Stage Compiler Support: Generate
appropriate test cases for Parse (syntax validation),
Sema (semantic analysis), and CodeGen (LLVM IR
generation) stages, providing comprehensive testing
coverage throughout the compilation pipeline.