Content-Length: 338752 | pFad | https://github.com/open-telemetry/opentelemetry-java-instrumentation/issues/13468

88 Instrumentation Metadata System · Issue #13468 · open-telemetry/opentelemetry-java-instrumentation · GitHub
Skip to content

Instrumentation Metadata System #13468

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
11 of 24 tasks
jaydeluca opened this issue Mar 6, 2025 · 5 comments
Open
11 of 24 tasks

Instrumentation Metadata System #13468

jaydeluca opened this issue Mar 6, 2025 · 5 comments
Assignees
Labels
documentation Improvements or additions to documentation enhancement New feature or request

Comments

@jaydeluca
Copy link
Member

jaydeluca commented Mar 6, 2025

Is your feature request related to a problem? Please describe.

The java instrumentation project currently includes 250 individual instrumentations. There are 242 javaagent instrumentations - 15 of which currently have readmes, and 59 library instrumentations - 35 of which currently have readmes. These readmes can have varying depth in terms of their contents. Going through one by one and creating and updating documentation for each instrumentation manually would require considerable effort and toil.

Describe the solution you'd like

Having a system to manage and generate meta data about each instrumentation can unlock automated documentation as well as unlock other tooling capabilities. This information will give users much better insight into what they can expect when using the instrumentation, as well as more information around changes between releases.

We will establish a standard set of metadata we want to track for each instrumentation, and design a system that leverages automated gathering of information that is then augmented by some human maintained metadata file per instrumentation to generate an instrumentation list yaml file that can then be used to feed other workflows.

Metadata that would be useful to have available:

  • Classification of the instrumentation
    • library - instrumentation for particular libraries
    • internal - instrumentation used internally within the agent
    • custom - instrumentation associated with supporting or generating custom instrumentation
  • Some description of what an instrumentation provides
    • Ability add descriptions via the metadata.yaml file
    • Descriptions added for all modules
  • A breakdown of the library versions that are supported, broken down by javaagent vs library support
  • Minimum Java version supported (if not 8+)
  • Whether the instrumentation is enabled or disabled by default
  • The key to use to disable/enable
  • The configuration options and defaults
  • Scope information
    • name
    • schemaUrl
    • attributes
  • Semantic conventions
  • Span attributes
  • Metrics

There are various ways we can attempt to obtain some of this information, and for other pieces we will create a metadata.yaml file for each instrumentation, similar to the collector

Ideas

Things to explore:

Other notes:

Additional context

If we complete this issue, I believe it will also solve the following:


See latest instrument list output here

@jaydeluca jaydeluca added enhancement New feature or request needs triage New issue that requires triage labels Mar 6, 2025
@jaydeluca jaydeluca self-assigned this Mar 6, 2025
@jaydeluca jaydeluca added documentation Improvements or additions to documentation and removed needs triage New issue that requires triage labels Mar 6, 2025
@jaydeluca
Copy link
Member Author

jaydeluca commented Mar 8, 2025

Capturing input from @jackshirazi around scope information:

Having all of the fields from the InstrumentationScopeInfo object available to be able to re-construct it is useful because it is used as the key in a map to the tracer, and if you want to let users dynamically adjust the scope, that's how they would access it.

@jaydeluca
Copy link
Member Author

Update on the current state of things:

  • Automated target version discovery for both javaagent and library instrumentation types
  • Instrumentation classification via metadata.yaml
  • Whether instrumentation is disabled by default via metadata.yaml
  • Starting to add some basic descriptions (see stats on coverage below)
  • Starting to document configuration options (see stats on coverage below)
Total Modules: 252
By classification:
	library: 226
	custom: 5
	internal: 21
metadata.yaml contents:
	descriptions: 12 (4.76%)
	configurations: 5 (1.98%)

Work in flight:

I am working through each module adding descriptions and configurations manually, while in parallel investigating ways we might be able to do some of this programmatically.

I am also still experimenting with using test runs in order to attain emitted span and metric data.

@jaydeluca
Copy link
Member Author

adding a note that @laurit brought up here about certain modules containing multiple instrumentations:

An alternative to this would be to allow metadata.yaml to describe multiple instrumentations. I guess it is tricky, in some cases one module contains multiple instrumentation modules that operate separately like here the jdbc and datasource instrumentation or kafka and kafka-metrics while in other cases multiple modules could be merged like opensearch-rest-1.0 and opensearch-rest-3.0.

I think it does probably make sense to consider further breaking down the modules to not just differentiate by file structure, but to also be able to break down further by InstrumentationModule classes and use those to identify each instrumentation. And then the metadata.yaml might need a way to specify attributes that apply to specific instrumentations within a module.

Identifying modules that could be merged will need further thought.

I will start experimenting

@jaydeluca
Copy link
Member Author

jaydeluca commented May 29, 2025

Capturing some notes from the discussion at todays SIG meeting:

  • We should target eventually publishing this information (or parts of it) to the registry in opentelemetry.io, this will allow embedding the information in other areas in the site (there is an existing example within the "instrumentation ecosystem" page)
  • This system doesn't do a good job at documenting and covering native instrumentation.
    • What native instrumentation emits can be a moving target
    • We do already have some native instrumentation wired up in test modules that we run tests against, and we can collect and document that telemetry information.
  • This automated system could be used to validate documentation and identify drift, but the final documentation should be managed and maintained by humans.
    • The documentation could initially be bootstrapped with a one time automated generation and then continually updated/validated going forward
  • Declarative configuration project is doing something where they are trying to track information about the various types associated with the data model. They want to be able to generate yaml based on code, and then markdown based on yaml.
  • If we were to embed information into the code, how could that work? annotations?
  • We still have some instrumentations that are outliers in some ways, there are instrumentations with more than one InstrumentationModule (jdbc), or some instrumentations that are split across modules but represent the same thing.
    • We could incorporate some context about these outliers into the instrumentation analyzer code and handle them accordingly if needed

@jaydeluca
Copy link
Member Author

Making note that @zeitlinger had a question about the spring starter configuration metadata file:

do you have an idea if we can also generate the spring metadata out of this?

I am adding an item to this issue to investigate how this system might be able to integrate with managing and/or documenting those configuration options as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant








ApplySandwichStrip

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier!      Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

Fetched URL: https://github.com/open-telemetry/opentelemetry-java-instrumentation/issues/13468

Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy