Thanks for the easy to follow reproduction repo. Can confirm, I'm observing the same outcomes with the http integration added as the only integration. I also tried production mode with the same results.

When only using the httpIntegration and disabling everything that's configurable inside it, I got up to 27k rps and 3ms avg response time on my machine. As a baseline I'm getting 85k rps tho.

Sentry.httpIntegration({
   breadcrumbs: false,
   spans: false,
   trackIncomingRequestsAsSessions: false,
   disableIncomingRequestSpans: true,
}),

cc @chargome @mydea

Hey @ReneGreen27, we have several PRs in place that should slightly improve performance, but generally the instrumentation will cause some overhead. Current recommendation is to use the setup posted above by @andreiborza and update to latest versions as we'll try to further optimize this.

Here's a flamegraph of an extremely simple node.js server load tested:

https://drive.google.com/file/d/19CC_zFcRhVme_exz1gpthNv6fh8tNcKe/view?usp=sharing

const Sentry = require("@sentry/node");

Sentry.init({
  dsn: "....",

  // Add Tracing by setting tracesSampleRate
  // We recommend adjusting this value in production
  tracesSampleRate: 1.0,

  // Set sampling rate for profiling
  // This is relative to tracesSampleRate
  profilesSampleRate: 1.0,

  debug: true,

  environment: "node-test-app",
});

const http = require("http");

const hostname = "127.0.0.1";
const port = 3000;

const server = http.createServer((req, res) => {
  res.statusCode = 200;
  res.setHeader("Content-Type", "text/plain");
  res.end("Hello, World!\n");
});

server.listen(port, hostname, () => {
  console.log(`Server running at http://${hostname}:${port}/`);
});

Some offenders I could make out:

• httpRequestToRequestData - or maybe drop undefined keys
• processEvent
• promiseBuffer is using an enormous amount of self-time
• normalize
• otel’s getIncomingRequestAttributes
• for whatever reason some anonymous function in the SentryHttpInstrumentation
• processEvent in eventFilters.js
• createTransactionForOtelSpan
• I think logger we can ignore, I had debug: true in the test

Found another one - looks like the spanStart hooks we run in integrations can be avoided

I know we're improving dropUndefinedKeys but we should look at if we can avoid adding client hooks if no instrumentation was added.

For example with tedious, we should not even register client.on('spanStart' if tedious is not instrumented (which for vast majority of people it wont be

We should also avoid dropUndefinedKeys in spanToJson

Object.defineProperty is def expensive. We should aim to avoid this as much as possible. I actually see a lot of use cases where we can replace this with a weakmap instead.

addChildSpanToSpan, setContextOnScope, and setCapturedScopesOnSpan

shouldSample

uuid4

Why is getCrypto getter so expensive???

#15767

Note: After profiling the changes we made in:

• perf(core): Add short-circuits to eventFilters integration #15752
• ref: Avoid some usage of dropUndefinedKeys() #15757
• feat(core): Optimize dropUndefinedKeys #15760
• perf(nestjs): Remove usage of addNonEnumerableProperty #15766

We seem to have made some improvements!

We've dropped dropUndefinedKey usage total time by 50%, isPojo has pretty much disappeared off the list,

Before:

After:

Now we gotta make spanToJson avoid dropUndefinedKeys, and fix SentryError usage: