diff --git a/.github/workflows/scorecards-analysis.yml b/.github/workflows/scorecards-analysis.yml new file mode 100644 index 0000000..2bd24ef --- /dev/null +++ b/.github/workflows/scorecards-analysis.yml @@ -0,0 +1,53 @@ +name: Scorecards supply-chain security +on: + # Only the default branch is supported. + branch_protection_rule: + push: + branches: [ main ] + +# Declare default permissions as read only. +permissions: read-all + +jobs: + analysis: + name: Scorecards analysis + runs-on: ubuntu-latest + permissions: + # Needed to upload the results to code-scanning dashboard. + security-events: write + actions: read + contents: read + + steps: + - name: "Checkout code" + uses: actions/checkout@ec3a7ce113134d7a93b817d10a8272cb61118579 # v2.4.0 + with: + persist-credentials: false + + - name: "Run analysis" + uses: ossf/scorecard-action@c8416b0b2bf627c349ca92fc8e3de51a64b005cf # v1.0.2 + with: + results_file: results.sarif + results_format: sarif + # Read-only PAT token. To create it, + # follow the steps in https://github.com/ossf/scorecard-action#pat-token-creation. + repo_token: ${{ secrets.SCORECARD_READ_TOKEN }} + # Publish the results to enable scorecard badges. For more details, see + # https://github.com/ossf/scorecard-action#publishing-results. + # For private repositories, `publish_results` will automatically be set to `false`, + # regardless of the value entered here. + publish_results: true + + # Upload the results as artifacts (optional). + - name: "Upload artifact" + uses: actions/upload-artifact@82c141cc518b40d92cc801eee768e7aafc9c2fa2 # v2.3.1 + with: + name: SARIF file + path: results.sarif + retention-days: 5 + + # Upload the results to GitHub's code scanning dashboard. + - name: "Upload to code-scanning" + uses: github/codeql-action/upload-sarif@5f532563584d71fdef14ee64d17bafb34f751ce5 # v1.0.26 + with: + sarif_file: results.sarif diff --git a/README.md b/README.md index 507ca4f..06bc753 100644 --- a/README.md +++ b/README.md @@ -1,42 +1,45 @@ -# Github App for Splunk +# GitHub App for Splunk -The Github App for Splunk is a collection of out of the box dashboards and Splunk knowledge objects designed to give Github Admins and platform owners immediate visibility into Github. +The GitHub App for Splunk is a collection of out of the box dashboards and Splunk knowledge objects designed to give GitHub Admins and platform owners immediate visibility into GitHub. -This App is designed to work across multiple Github data sources however not all all required. You may choose to only collect a certain set of data and the parts of this app that utilize that set will function, while those that use other data sources will not function correctly, so please only use the Dashboards that relate to the data you are collecting. +This App is designed to work across multiple GitHub data sources however not all all required. You may choose to only collect a certain set of data and the parts of this app that utilize that set will function, while those that use other data sources will not function correctly, so please only use the Dashboards that relate to the data you are collecting. -The Github App for Splunk is designed to work with the following data sources: +The GitHub App for Splunk is designed to work with the following data sources: -* [Github Audit Log Monitoring Add-On For Splunk](./docs/ghe_audit_logs.MD): Audit logs from Github Enterprise Cloud. -* [Github.com Webhooks](./docs/github_webhooks.MD): A select set of webhook events like Push, PullRequest, and Repo. -* [Github Enterprise Server Syslog Forwarder](https://docs.github.com/en/enterprise-server/admin/user-management/monitoring-activity-in-your-enterprise/log-forwarding): Audit and Application logs from Github Enterprise Server. +* [GitHub Audit Log Monitoring Add-On For Splunk](./docs/ghe_audit_logs.MD): Audit logs from GitHub Enterprise Cloud. +* [Github.com Webhooks](./docs/github_webhooks.MD): A select set of webhook events like Push, PullRequest, Code Scanning and Repo. +* [Github Enterprise Server Syslog Forwarder](./docs/ghes_syslog_setup.MD): Audit and Application logs from Github Enterprise Server. * [Github Enterprise Collectd monitoring](./docs/splunk_collectd_forwarding_for_ghes.MD): Performance and Infrastructure metrics from Github Enterprise Server. ## Dashboard Instructions ### Installation -The Github App for Splunk is available for download from [Splunkbase](https://splunkbase.splunk.com/app/5596/). For Splunk Cloud, refer to [Install apps in your Splunk Cloud deployment](https://docs.splunk.com/Documentation/SplunkCloud/latest/Admin/SelfServiceAppInstall). For non-Splunk Cloud deployments, refer to the standard methods for Splunk Add-on installs as documented for a [Single Server Install](http://docs.splunk.com/Documentation/AddOns/latest/Overview/Singleserverinstall) or a [Distributed Environment Install](http://docs.splunk.com/Documentation/AddOns/latest/Overview/Distributedinstall). +The GitHub App for Splunk is available for download from [Splunkbase](https://splunkbase.splunk.com/app/5596/). For Splunk Cloud, refer to [Install apps in your Splunk Cloud deployment](https://docs.splunk.com/Documentation/SplunkCloud/latest/Admin/SelfServiceAppInstall). For non-Splunk Cloud deployments, refer to the standard methods for Splunk Add-on installs as documented for a [Single Server Install](http://docs.splunk.com/Documentation/AddOns/latest/Overview/Singleserverinstall) or a [Distributed Environment Install](http://docs.splunk.com/Documentation/AddOns/latest/Overview/Distributedinstall). **This app should be installed on both your search head tier as well as your indexer tier.** - + ### Configuration ![Settings>Advanced Search>Search macros](./docs/images/macros.png) -1. The Github App for Splunk uses macros so that index and `sourcetype` names don't need to be updated in each dashboard panel. You'll need to update the macros to account for your selected indexes. -1. The macro `github_source` is the macro for all audit log events, whether from Github Enterprise Cloud or Server. The predefined macro includes examples of **BOTH**. Update to account for your specific needs. +1. The GitHub App for Splunk uses macros so that index and `sourcetype` names don't need to be updated in each dashboard panel. You'll need to update the macros to account for your selected indexes. +1. The macro `github_source` is the macro for all audit log events, whether from GitHub Enterprise Cloud or Server. The predefined macro includes examples of **BOTH**. Update to account for your specific needs. 1. The macro `github_webhooks` is the macro used for all webhook events. Since it is assuming a single index for all webhook events, that is the predefined example, but update as needed. -1. Finally, the macro `github_collectd` is the macro used for all `collectd` metrics sent from Github Enterprise Server. Please update accordingly. +1. Finally, the macro `github_collectd` is the macro used for all `collectd` metrics sent from GitHub Enterprise Server. Please update accordingly. ### Integration Overview dashboard -There is an *Integration Overview* dashboard listed under *Dashboards* that allows you to monitor API rate limits, audit events fetched, or webhooks received. This dashboard is primarily meant to be used with the `Github Audit Log Monitoring Add-On for Splunk` and uses internal Splunk logs. To be able to view them you will probably need elevated privileges in Splunk that include access to the `_internal` index. Please coordinate with your Splunk team if that dashboard is desired. +There is an *Integration Overview* dashboard listed under *Dashboards* that allows you to monitor API rate limits, audit events fetched, or webhooks received. This dashboard is primarily meant to be used with the `GitHub Audit Log Monitoring Add-On for Splunk` and uses internal Splunk logs. To be able to view them you will probably need elevated privileges in Splunk that include access to the `_internal` index. Please coordinate with your Splunk team if that dashboard is desired. ### Examples
Expand for screenshots +#### Code Scanning Alerts + ![Code Scanning Dashboard](./docs/images/code_scanning_dashboard.png) + #### Audit Log Dashboard ![Audit Log Dashboard](./docs/images/9F8E9A89-1203-4C0A-B227-C2FD1E17C8B0.jpg) @@ -59,4 +62,4 @@ There is an *Integration Overview* dashboard listed under *Dashboards* that allo ## Support -Support for Github App for Splunk is run through [Github Issues](https://github.com/splunk/github_app_for_splunk/issues). Please open a new issue for any support issues or for feature requests. You may also open a Pull Request if you'd like to contribute additional dashboards, eventtypes for webhooks, or enhancements you may have. +Support for GitHub App for Splunk is run through [GitHub Issues](https://github.com/splunk/github_app_for_splunk/issues). Please open a new issue for any support issues or for feature requests. You may also open a Pull Request if you'd like to contribute additional dashboards, eventtypes for webhooks, or enhancements you may have. diff --git a/docs/ghe_audit_logs.MD b/docs/ghe_audit_logs.MD index a75747b..c9eed4b 100644 --- a/docs/ghe_audit_logs.MD +++ b/docs/ghe_audit_logs.MD @@ -125,7 +125,7 @@ This modular input fetches events by calling the [Enterprise Audit Log API](http ### Activity dashboard example -Along with this modular input we're providing a [Github App for Splunk](https://github.com/splunk/github_app_for_splunk) that makes use of the collected audit log events to give you an overview of the activities across your enterprise. +Along with this modular input we're providing a [GitHub App for Splunk](https://github.com/splunk/github_app_for_splunk) that makes use of the collected audit log events to give you an overview of the activities across your enterprise. You can install it via the [Manage Apps page](https://docs.splunk.com/Documentation/Splunk/8.2.0/Admin/Deployappsandadd-ons). diff --git a/docs/ghes_syslog_setup.MD b/docs/ghes_syslog_setup.MD new file mode 100644 index 0000000..f084386 --- /dev/null +++ b/docs/ghes_syslog_setup.MD @@ -0,0 +1,25 @@ +# Sending GitHub Enterprise Server Logs to Splunk + +GitHub Enterprise Server comes with syslog-ng built in to send data to platforms like Splunk: https://docs.github.com/en/enterprise-server@3.3/admin/user-management/monitoring-activity-in-your-enterprise/log-forwarding. Following those directions will allow you to easily onboard logs to Splunk. However, The GitHub App for Splunk comes with enhancements for those logs that will allow you to search more efficently. + +## Sources and Transformations + + The syslog feed from GitHub Enterprise Server contains ALL application logs including audit logs, web server logs, database logs, etc. Being able to differentiate the logs is critical. This app includes the ability to overwrite the source of events with the log type out of the box. However, for this to happen, you must use the sourcetype of `GithubEnterpriseServerLog` or duplicate that stanza from the default `props.conf` file into a custom stanza in your local copy. When setting up a TCP input you have the ability to force that specific sourcetype. This will enable easy filtering of log files to their specific process. + +## Default `props.conf` + +``` +[GithubEnterpriseServerLog] +DATETIME_CONFIG = +LINE_BREAKER = ([\r\n]+) +NO_BINARY_CHECK = true +category = Application +pulldown_type = true +TIME_FORMAT = +TZ = +EXTRACT-audit_event = github_audit\[\d+\]\:\s(?.*) +EXTRACT-audit_fields = \"(?<_KEY_1>.*?)\"\:\"*(?<_VAL_1>.*?)\"*, +EXTRACT-github_log_type = \d+\:\d+\:\d+\s\d+\-\d+\-\d+\-\d+\s(?.*?)\: +EXTRACT-github_document_id = \"_document_id\"\:\"(?.*?)\" +FIELDALIAS-source = github_log_type AS source +``` diff --git a/docs/github_webhooks.MD b/docs/github_webhooks.MD index 12ea7d1..140d3ae 100644 --- a/docs/github_webhooks.MD +++ b/docs/github_webhooks.MD @@ -1,10 +1,10 @@ -# Using Github Webhooks +# Using GitHub Webhooks -Github Webhooks are a great way to collect rich information as it occurs. You can easily enable webhooks within the Github UI and can even select specific actions on which to trigger a webhook call to Splunk. This is only available at the Organization level and will require this to be done for each Org as desired. To do so, you'll need to configure Splunk as a receiver and then setup the webhooks within Github. +GitHub Webhooks are a great way to collect rich information as it occurs. You can easily enable webhooks within the GitHub UI and can even select specific actions on which to trigger a webhook call to Splunk. This is only available at the Organization level and will require this to be done for each Org as desired. To do so, you'll need to configure Splunk as a receiver and then setup the webhooks within GitHub. ## Configuring Splunk to receive Webhooks -Splunk's HTTP Event Collector (HEC) is a quick and easy endpoint built to receive data from other producers like Github. +Splunk's HTTP Event Collector (HEC) is a quick and easy endpoint built to receive data from other producers like GitHub. ### Setting Up Splunk to Listen for Webhooks 1. Under Settings > Data Inputs, click **HTTP Event Collector** @@ -13,19 +13,19 @@ Splunk's HTTP Event Collector (HEC) is a quick and easy endpoint built to receiv 1. Unless required by your Splunk administrator, the rest of this page can be left as is and continue onto the next step. 1. You'll want to click `select` for Source Type, and a new selection box will appear below that. 1. Under the Application option, there should be an entry for `github_json`, however you may need to use the little search bar to find it. -1. For App Context, you'll want to select **Splunk App for Github** +1. For App Context, you'll want to select **Splunk App for GitHub** 1. Next select the index created for this data. If none exist, create a new Index. Names like `github` or the like are recommended, depending on corporate naming conventions. 1. Lastly, click the Review button and confirm the data is correct and hit Submit. Your token is now available to collect data, however we'll need to enable that token to allow Query String Authentication using that token. For this, you'll need command line access to your Splunk environment or be using a deployment server to deploy apps to Splunk. -To enable Query String Authentication, you'll need to update the `inputs.conf` file within the Splunk App for Github local directory. In that file, there will be a stanza with the name and value of the token you created. At the end of that stanza, you'll need to add `allowQueryStringAuth = true` and then restart Splunk. This is best done with the help of your Splunk team, so please reach out to them for assistance on this step. +To enable Query String Authentication, you'll need to update the `inputs.conf` file within the Splunk App for GitHub local directory. In that file, there will be a stanza with the name and value of the token you created. At the end of that stanza, you'll need to add `allowQueryStringAuth = true` and then restart Splunk. This is best done with the help of your Splunk team, so please reach out to them for assistance on this step. -### Setting Up Github Webhooks +### Setting Up GitHub Webhooks Webhooks are a simple push mechanism that will send an event each time the webhook is triggered. Unfortunately, Webhooks are unique to each Organization and will need to be setup for each Org as desired. To do this, a user will need to be an Admin for the Org. -1. In your Github Organization Settings page, select Webhooks from the menu on the left. +1. In your GitHub Organization Settings page, select Webhooks from the menu on the left. 1. On this page, you'll see all the existing Webhooks, click the **Add webhook** button to add one to send data to Splunk. 1. The Payload URL will be the Splunk HTTP Event Collector endpoint that was enabled above. It should look something like: `https://YOUR SPLUNK URL:8088/services/collector/raw?token=THE TOKEN FROM ABOVE`. The default port of 8088 may be different for your Splunk Environment, so please confirm the HEC port with your Splunk Admin team. 1. For Content Type, you'll want to select `application/json` as the best option. @@ -41,27 +41,33 @@ Once that is complete and webhooks are triggering, you'll want to update the mac - + - + - + - + - + + + + + + +
Splunk EventtypeGithub Webhook EventGitHub Webhook Event Description
Github::RepoGitHub::Repo Repositories Repository created, deleted, archived, unarchived, publicized, privatized, edited, renamed, or transferred.
Github::PushGitHub::Push Pushes Git push to a repository.
Github::PullRequestGitHub::PullRequest Pull requests Pull request opened, closed, reopened, edited, assigned, unassigned, review requested, review request removed, labeled, unlabeled, synchronized, ready for review, converted to draft, locked, unlocked, auto merge enabled, auto merge disabled, milestoned, or demilestoned.
Github::PullRequest::ReviewGitHub::PullRequest::Review Pull request reviews Pull request review submitted, edited, or dismissed.
GitHub::CodeScanningCode scanning alertsAlerts identified by CodeQL and other 3rd party/OSS scanning tools.
diff --git a/docs/images/code_scanning_dashboard.png b/docs/images/code_scanning_dashboard.png new file mode 100644 index 0000000..92135b9 Binary files /dev/null and b/docs/images/code_scanning_dashboard.png differ diff --git a/docs/splunk_collectd_forwarding_for_ghes.MD b/docs/splunk_collectd_forwarding_for_ghes.MD index 0273458..d491b05 100644 --- a/docs/splunk_collectd_forwarding_for_ghes.MD +++ b/docs/splunk_collectd_forwarding_for_ghes.MD @@ -1,4 +1,4 @@ -# Splunk Collectd Forwarding for Github Enterprise Server +# Splunk Collectd Forwarding for GitHub Enterprise Server This guide describes how to enable collectd forwarding on GitHub Enterprise Server (GHES) using Splunk Enterprise (v8.0+). diff --git a/github_app_for_splunk/README.md b/github_app_for_splunk/README.md index e36796a..d6c1918 100644 --- a/github_app_for_splunk/README.md +++ b/github_app_for_splunk/README.md @@ -1,30 +1,30 @@ -# Github App for Splunk +# GitHub App for Splunk -The Github App for Splunk is a collection of out of the box dashboards and Splunk knowledge objects designed to give Github Admins and platform owners immediate visibility into Github. +The GitHub App for Splunk is a collection of out of the box dashboards and Splunk knowledge objects designed to give GitHub Admins, platform owners, and Security Engineers immediate visibility into GitHub. -This App is designed to work across multiple Github data sources however not all all required. You may choose to only collect a certain set of data and the parts of this app that utilize that set will function, while those that use other data sources will not function correctly, so please only use the Dashboards that relate to the data you are collecting. +This App is designed to work across multiple GitHub data sources however not all all required. You may choose to only collect a certain set of data and the parts of this app that utilize that set will function, while those that use other data sources will not function correctly, so please only use the Dashboards that relate to the data you are collecting. -The Github App for Splunk is designed to work with the following data sources: +The GitHub App for Splunk is designed to work with the following data sources: -* [Github Audit Log Monitoring Add-On For Splunk](./docs/ghe_audit_logs.MD): Audit logs from Github Enterprise Cloud. -* [Github.com Webhooks]((./docs/github_webhooks.MD)): A select set of webhook events like Push, PullRequest, and Repo. -* [Github Enterprise Server Syslog Forwarder](https://docs.github.com/en/enterprise-server@3.0/admin/user-management/monitoring-activity-in-your-enterprise/log-forwarding): Audit and Application logs from Github Enterprise Server. -* [Github Enterprise Collectd monitoring](./docs/splunk_collectd_forwarding_for_ghes.MD): Performance and Infrastructure metrics from Github Enterprise Server. +* [GitHub Audit Log Monitoring Add-On For Splunk](./docs/ghe_audit_logs.MD): Audit logs from GitHub Enterprise Cloud. +* [Github.com Webhooks]((./docs/github_webhooks.MD)): A select set of webhook events like Push, PullRequest, Repo, and Code Scanning alerts. +* [GitHub Enterprise Server Syslog Forwarder](https://docs.github.com/en/enterprise-server@3.0/admin/user-management/monitoring-activity-in-your-enterprise/log-forwarding): Audit and Application logs from GitHub Enterprise Server. +* [GitHub Enterprise Collectd monitoring](./docs/splunk_collectd_forwarding_for_ghes.MD): Performance and Infrastructure metrics from GitHub Enterprise Server. ## Dashboard Instructions -The Github App for Splunk is available for download from [Splunkbase](https://splunkbase.splunk.com/app/5596/). Once installed there are a couple steps needed to light up all the dashboards. +The GitHub App for Splunk is available for download from [Splunkbase](https://splunkbase.splunk.com/app/5596/). Once installed there are a couple steps needed to light up all the dashboards. ![Settings>Advanced Search>Search macros](./docs/images/macros.png) -1. The Github App for Splunk uses macros so that index and sourcetype names don't need to be updated in each dashboard panel. You'll need to update the macros to account for your selected indexes. -1. The macro `github_source` is the macro for all audit log events, whether from Github Enterprise Cloud or Server. The predefined maco includes examples of **BOTH**. Update to account for your specific needs. +1. The GitHub App for Splunk uses macros so that index and sourcetype names don't need to be updated in each dashboard panel. You'll need to update the macros to account for your selected indexes. +1. The macro `github_source` is the macro for all audit log events, whether from GitHub Enterprise Cloud or Server. The predefined maco includes examples of **BOTH**. Update to account for your specific needs. 1. The macro `github_webhooks` is the macro used for all webhook events. Since it is assuming a single index for all webhook events, that is the predefined example, but update as needed. -1. Finally, the macro `github_collectd` is the macro used for all collectd metrics sent from Github Enterprise Server. Please update accordingly. +1. Finally, the macro `github_collectd` is the macro used for all collectd metrics sent from GitHub Enterprise Server. Please update accordingly. ### Integration Overview dashboard -There is an *Integration Overview* dashboard listed under *Dashboards* that allows you to monitor API rate limits, audit events fetched, or webhooks received. This dashboard is primarily meant to be used with the `Github Audit Log Monitoring Add-On for Splunk` and uses internal Splunk logs. To be able to view them you will probably need elevated privileges in Splunk that include access to the `_internal` index. Please coordinate with your Splunk team if that dashboard is desired. +There is an *Integration Overview* dashboard listed under *Dashboards* that allows you to monitor API rate limits, audit events fetched, or webhooks received. This dashboard is primarily meant to be used with the `GitHub Audit Log Monitoring Add-On for Splunk` and uses internal Splunk logs. To be able to view them you will probably need elevated privileges in Splunk that include access to the `_internal` index. Please coordinate with your Splunk team if that dashboard is desired. ## Support -Support for Github App for Splunk is run through [Github Issues](https://github.com/splunk/github_app_for_splunk/issues). Please open a new issue for any support issues or for feature requests. You may also open a Pull Request if you'd like to contribute additional dashboards, eventtypes for webhooks, or enhancements you may have. +Support for GitHub App for Splunk is run through [GitHub Issues](https://github.com/splunk/github_app_for_splunk/issues). Please open a new issue for any support issues or for feature requests. You may also open a Pull Request if you'd like to contribute additional dashboards, eventtypes for webhooks, or enhancements you may have. diff --git a/github_app_for_splunk/default/app.conf b/github_app_for_splunk/default/app.conf index 022cf47..57e3596 100644 --- a/github_app_for_splunk/default/app.conf +++ b/github_app_for_splunk/default/app.conf @@ -7,11 +7,11 @@ version = X.Y.Z [ui] is_visible = 1 -label = Github App for Splunk +label = GitHub App for Splunk [launcher] author = Doug Erkkila -description = Report on Activity and Audit Data from Github +description = Report on Activity and Audit Data from GitHub version = X.Y.Z [package] diff --git a/github_app_for_splunk/default/data/ui/nav/default.xml b/github_app_for_splunk/default/data/ui/nav/default.xml index 6837a78..c537b06 100644 --- a/github_app_for_splunk/default/data/ui/nav/default.xml +++ b/github_app_for_splunk/default/data/ui/nav/default.xml @@ -8,8 +8,10 @@ - + + + diff --git a/github_app_for_splunk/default/data/ui/views/1_system_health_monitor.xml b/github_app_for_splunk/default/data/ui/views/1_system_health_monitor.xml index a8c7154..b8d12c4 100644 --- a/github_app_for_splunk/default/data/ui/views/1_system_health_monitor.xml +++ b/github_app_for_splunk/default/data/ui/views/1_system_health_monitor.xml @@ -1,4 +1,4 @@ -
+
diff --git a/github_app_for_splunk/default/data/ui/views/2_process_monitor.xml b/github_app_for_splunk/default/data/ui/views/2_process_monitor.xml index 8c12cf1..a9e8fbe 100644 --- a/github_app_for_splunk/default/data/ui/views/2_process_monitor.xml +++ b/github_app_for_splunk/default/data/ui/views/2_process_monitor.xml @@ -1,4 +1,4 @@ - +
diff --git a/github_app_for_splunk/default/data/ui/views/3_authentication_monitor.xml b/github_app_for_splunk/default/data/ui/views/3_authentication_monitor.xml index 6f198a8..f35698d 100644 --- a/github_app_for_splunk/default/data/ui/views/3_authentication_monitor.xml +++ b/github_app_for_splunk/default/data/ui/views/3_authentication_monitor.xml @@ -1,4 +1,4 @@ - +
@@ -311,4 +311,4 @@ - \ No newline at end of file + diff --git a/github_app_for_splunk/default/data/ui/views/8_storage_monitor.xml b/github_app_for_splunk/default/data/ui/views/8_storage_monitor.xml index 291b885..96ecbbb 100644 --- a/github_app_for_splunk/default/data/ui/views/8_storage_monitor.xml +++ b/github_app_for_splunk/default/data/ui/views/8_storage_monitor.xml @@ -1,4 +1,4 @@ -
+
@@ -125,4 +125,4 @@ - \ No newline at end of file + diff --git a/github_app_for_splunk/default/data/ui/views/api_config.xml b/github_app_for_splunk/default/data/ui/views/api_config.xml index c101422..6e20b1f 100644 --- a/github_app_for_splunk/default/data/ui/views/api_config.xml +++ b/github_app_for_splunk/default/data/ui/views/api_config.xml @@ -12,7 +12,7 @@

Installation

  1. -

    Download the latest release of the Splunk Add-On for Github Enterprise Audit Logs from SplunkBase

    +

    Download the latest release of the Splunk Add-On for GitHub Enterprise Audit Logs from SplunkBase

  2. Go to Apps > Manage Apps in the toolbar menu.

    @@ -24,7 +24,7 @@

    Generate a Personal Access Token in GitHub Enterprise with the site_admin scope.

  3. -

    Under Settings > Data inputs, there should be a new option called Github Audit Log Monitoring, click "+ Add new"

    +

    Under Settings > Data inputs, there should be a new option called GitHub Audit Log Monitoring, click "+ Add new"

  4. Configure the Input by entering the necessary information in the input fields. Don't forget to define the Index for the data to be stored in. This option is under the "More settings" option.

    diff --git a/github_app_for_splunk/default/data/ui/views/code_scanning_overview.xml b/github_app_for_splunk/default/data/ui/views/code_scanning_overview.xml new file mode 100644 index 0000000..9522135 --- /dev/null +++ b/github_app_for_splunk/default/data/ui/views/code_scanning_overview.xml @@ -0,0 +1,192 @@ +
    + + + + `github_webhooks` (eventtype="GitHub::CodeScanning" OR eventtype="GitHub::Push") | eval action='action', tool=if(isnotnull('alert.tool.name'),'alert.tool.name','unknown'), repository=if(isnotnull('repository.name'),'repository.name','unknown'), severity=if(isnotnull('alert.rule.security_severity_level'),'alert.rule.security_severity_level','none'), create_time=if(isnotnull('alert.created_at'),'alert.created_at','unknown'), received_time='_time', alert_url=if(isnotnull('alert.html_url'),'alert.html_url','unknown'), eventtype='eventtype', created=strptime(create_time, "%Y-%m-%dT%H:%M:%S%Z"), duration=received_time - created, duration_str=tostring(avg(duration), "duration") + + $timeTkn.earliest$ + $timeTkn.latest$ + 1 + +
    + + + + -24h@h + now + + + + + tool + tool + " + " + + | table tool | dedup tool + + All + * + * + + + + All + * + * + " + " + , + repository + repository + + | dedup repository | table repository + + +
    + + + + Created + + | search tool=$tool_name$ repository=$repoTkn$ action="created" | stats count + + + + + + + + + + Fixed + + | search tool=$tool_name$ repository=$repoTkn$ action="fixed" | stats count + + + + + + + + + Reopened + + | search tool=$tool_name$ repository=$repoTkn$ action="reopened" | stats count + + + + + + + + + + + Alert Found/Fixed Ratio + + | search tool=$tool_name$ repository=$repoTkn$ (action=created OR action=fixed) +| timechart count(_raw) by action +| accum created +| accum fixed +| rename created as "Found" +| rename fixed as "Fixed" + + + + + + + + + + + Commit/Alert Ratio + + | search (eventtype="GitHub::Push" repository=$repoTkn$) OR ((action=created OR action=reopened) tool=$tool_name$ repository=$repoTkn$ ) +| timechart count(_raw) by eventtype +| accum "GitHub::Push" +| accum "GitHub::CodeScanning" +| rename GitHub::Push as "Pushes" +| rename GitHub::CodeScanning as "Code Scanning Alerts" + + + + + + + + + + + + + + + New Alerts by Tool + + | search tool=$tool_name$ repository=$repoTkn$ (action=created OR action=appeared_in_branch) | timechart count(_raw) by tool + + + + + + + + + + + + + Fixed Alerts + + | search (action=fixed OR action=closed_by_user) repository=$repoTkn$ tool=$tool_name$ +| table repository, tool, alert_url,duration_str +| rename repository AS "Repository" duration_str AS "Time to Resolution",tool AS "Tool", alert_url AS "Alert URL" +| sort -"Time to Resolution" + + + +
    +     +     + + + + Alerts by Severity + + | search (action=created OR action=reopened) repository=$repoTkn$ tool=$tool_name$ | chart usenull=f count over repository by severity + + + + + + + + + + + + + + +
    +     + + + Alerts by Repo + + | search (action=created OR action=reopened) repository=$repoTkn$ tool=$tool_name$| chart usenull=f count over repository by tool + + + + + + + + + +
    +     +     +
    diff --git a/github_app_for_splunk/default/data/ui/views/repository_audit.xml b/github_app_for_splunk/default/data/ui/views/repository_audit.xml index 9ba39f7..df556cd 100644 --- a/github_app_for_splunk/default/data/ui/views/repository_audit.xml +++ b/github_app_for_splunk/default/data/ui/views/repository_audit.xml @@ -97,7 +97,7 @@ Repository Workflow Details - Clicking an Workflow run will take you to Github to view the Workflow + Clicking an Workflow run will take you to GitHub to view the Workflow `github_source` action IN("workflows.completed*") repo="*" | stats latest(conclusion) as status by org, actor, name, repo, head_branch, workflow_run_id $timeRng.earliest$ diff --git a/github_app_for_splunk/default/data/ui/views/secret_scanning_overview.xml b/github_app_for_splunk/default/data/ui/views/secret_scanning_overview.xml new file mode 100644 index 0000000..5fc7164 --- /dev/null +++ b/github_app_for_splunk/default/data/ui/views/secret_scanning_overview.xml @@ -0,0 +1,148 @@ +
    + + + + `github_webhooks` eventtype="GitHub::SecretScanning" | eval action='action', enterprise=if(isnotnull('enterprise.name'),'enterprise.name','unknown'), organization=if(isnotnull('organization.login'),'organization.login','unknown'), repository=if(isnotnull('repository.name'),'repository.name','unknown'), secret_type=if(isnotnull('alert.secret_type'),'alert.secret_type','unknown'), resolution=if(isnotnull('alert.resolution'),'alert.resolution','unknown'), resolved_at=if(isnotnull('alert.resolved_at'),'alert.resolved_at','unknown'), resolved_by=if(isnotnull('alert.resolved_by.login'),'alert.resolved_by.login','unknown') + + $timeTkn.earliest$ + $timeTkn.latest$ + 1 + +
    + + + + -24h@h + now + + + + + secret_type + secret_type + " + " + + | table secret_type | dedup secret_type + + All + * + * + + + + All + * + * + " + " + , + organization + organization + + | dedup organization | table organization + + + + + All + * + * + " + " + , + repository + repository + + | dedup repository | table repository + + +
    + + + + Found Secrets + + | search repository=$repoTkn$ organization=$orgTkn$ secret_type=$secret_type$ action="created" | stats count + + + + + + + + + + Fixed Secrets + + | search repository=$repoTkn$ organization=$orgTkn$ secret_type=$secret_type$ action="resolved" | stats count + + + + + + + + + Secret Types + + | search repository=$repoTkn$ organization=$orgTkn$ secret_type=$secret_type$ | chart count by secret_type + + + + + + + + + Secrets Found/Fixed Ratio + + | search repository=$repoTkn$ organization=$orgTkn$ secret_type=$secret_type$ (action=created OR action=resolved) +| timechart count(_raw) by action +| accum created +| accum resolved +| rename created as "Found" +| rename resolved as "Fixed" + + + + + + + + + + + +
    + Fixed Secrets + + | search action=resolved repository=$repoTkn$ organization=$orgTkn$ secret_type=$secret_type$ | table secret_type, organization, repository, resolution, resolved_by, _time + | rename secret_type as "Secret Type" + | rename organization as "Organization" + | rename repository as "Repository" + | rename resolution as "Resolution" + | rename resolved_by as "Resolved By" + + + +
    +     + + + + + Found Secrets + + | search action=created repository=$repoTkn$ organization=$orgTkn$ secret_type=$secret_type$ | table secret_type, organization, repository, action, _time + | rename secret_type as "Secret Type" + | rename organization as "Organization" + | rename repository as "Repository" + | rename action as "Action" + + + +
    +     +     + diff --git a/github_app_for_splunk/default/data/ui/views/security_alert_overview.xml b/github_app_for_splunk/default/data/ui/views/security_alert_overview.xml index a19e82f..7433349 100644 --- a/github_app_for_splunk/default/data/ui/views/security_alert_overview.xml +++ b/github_app_for_splunk/default/data/ui/views/security_alert_overview.xml @@ -1,8 +1,8 @@ -
    - + + - index="github_webhook" alert.created_at=* | eval reason=if(isnotnull('alert.affected_package_name'),'alert.affected_package_name','alert.rule.name'), id=if(isnotnull('alert.external_identifier'),'alert.external_identifier','alert.rule.id'), severity=if(isnotnull('alert.severity'),'alert.severity','alert.rule.security_severity_level'), type=if(isnotnull('alert.external_identifier'),"vulnerability alert","code scanning alert") | stats latest(action) as status, earliest(alert.created_at) as created_at by repository.name, reason, id, type, severity | eval age = toString(round(now() - strptime(created_at, "%Y-%m-%dT%H:%M:%S")),"Duration") + `github_webhooks` alert.created_at=* | eval reason=if(isnotnull('alert.affected_package_name'),'alert.affected_package_name','alert.rule.name'), id=if(isnotnull('alert.external_identifier'),'alert.external_identifier','alert.rule.id'), severity=if(isnotnull('alert.severity'),'alert.severity','alert.rule.security_severity_level'), type=if(isnotnull('alert.external_identifier'),"Dependabot Alert","Code Scanning Alert") | stats latest(action) as status, earliest(alert.created_at) as created_at by repository.name, reason, id, type, severity | eval age = toString(round(now() - strptime(created_at, "%Y-%m-%dT%H:%M:%S")),"Duration") $timeTkn.earliest$ $timeTkn.latest$ @@ -240,4 +240,4 @@ - \ No newline at end of file + diff --git a/github_app_for_splunk/default/data/ui/views/value_stream_analytics.xml b/github_app_for_splunk/default/data/ui/views/value_stream_analytics.xml index cb72211..70a1de8 100644 --- a/github_app_for_splunk/default/data/ui/views/value_stream_analytics.xml +++ b/github_app_for_splunk/default/data/ui/views/value_stream_analytics.xml @@ -1,4 +1,4 @@ -
    + index=github_webhook (eventtype="GitHub::Issue" (action IN("opened","milestoned")) OR (action="labeled" AND label.name IN("in progress","to do"))) OR (eventtype="GitHub::Push" issueNumber=*) OR (eventtype="GitHub::PullRequest" action IN("opened","closed") issueNumber=*) repository.name IN("$repoTkn$") issueNumber!=9 | eval openTime=if(action=="opened",_time,NULL) | eval inProgressTime=if(action=="labeled",if('label.name'=="to do",_time,NULL),if(action=="milestoned",_time,NULL)) | eval workTime=if(action="labeled",if('label.name'="in progress",_time,NULL),if(eventtype=="GitHub::Push",_time,NULL)) | eval mergeTime=if(eventtype="GitHub::PullRequest",if(action=="opened",_time,NULL),if(eventtype="GitHub::Push",if(ref="refs/heads/main",_time,NULL),NULL)) | eval reviewTime=if(eventtype="GitHub::PullRequest",if('pull_request.merged'="true",_time,NULL),if(eventtype="GitHub::Push",if(ref="refs/heads/main",_time,NULL),NULL)) | eval sha=after | join type=left max=0 sha [ search index="github_webhook" eventtype="GitHub::Workflow" | eval sha='workflow_job.head_sha' | stats min(_time) as startTestTime, max(_time) as endTestTime by sha, workflow_job.id | eval testTimeDiff=endTestTime-startTestTime] | eval release='milestone.title' | join type=left release [search index=github_webhook eventtype="GitHub::Release" | eval release='release.tag_name' | stats max(_time) as releaseTime by release] | stats max(issue.title) as issue.title, latest(milestone.title) as release, min(openTime) as opened, min(inProgressTime) as in_progress, min(workTime) as working , max(mergeTime) as merge, max(reviewTime) as review, avg(testTimeDiff) as avgTestDuration, max(endTestTime) as endTestTime, max(releaseTime) as releaseTime by repository.name,issueNumber | search opened=* diff --git a/github_app_for_splunk/default/data/ui/views/webhook_config.xml b/github_app_for_splunk/default/data/ui/views/webhook_config.xml index e72e7c9..4a41155 100644 --- a/github_app_for_splunk/default/data/ui/views/webhook_config.xml +++ b/github_app_for_splunk/default/data/ui/views/webhook_config.xml @@ -3,10 +3,10 @@ -

    Using Github Webhooks

    -

    Github Webhooks are a great way to collect rich information as it occurs. You can easily enable webhooks within the Github UI and can even select specific actions on which to trigger a webhook call to Splunk. This is only available at the Organization level and will require this to be done for each Org as desired. To do so, you'll need to configure Splunk as a receiver and then setup the webhooks within Github.

    +

    Using GitHub Webhooks

    +

    GitHub Webhooks are a great way to collect rich information as it occurs. You can easily enable webhooks within the GitHub UI and can even select specific actions on which to trigger a webhook call to Splunk. This is only available at the Organization level and will require this to be done for each Org as desired. To do so, you'll need to configure Splunk as a receiver and then setup the webhooks within GitHub.

    Configuring Splunk to receive Webhooks

    -

    Splunk's HTTP Event Collector (HEC) is a quick and easy endpoint built to receive data from other producers like Github.

    +

    Splunk's HTTP Event Collector (HEC) is a quick and easy endpoint built to receive data from other producers like GitHub.

    Steps

    1. Under Settings > Data Inputs, click HTTP Event Collector
      2. @@ -15,13 +15,13 @@
    2. Unless required by your SPlunk administrator, the rest of this page can be left as is and continue onto the next step.
    3. You'll want to click select for Source Type, and a new selection box will appear below that.
    4. Under the Application option, there should be an entry for github_json, however you may need to use the little search bar to find it.
      5. -
    5. For App Context, you'll want to select Splunk App for Github
      6. +
    6. For App Context, you'll want to select Splunk App for GitHub
    7. Next select the index created for this data. If none exist, create a new Index. Names like github or the like are recommended, depending on corporate naming conventions.
    8. Lastly, click the Review button and confirm the data is correct and hit Submit.

    Your token is now available to collect data, however we'll need to enable that token to allow Query String Authentication using that token. For this, you'll need command line access to your Splunk environment or be using a deployment server to deploy apps to Splunk.

    -

    To enable Query String Authentication, you'll need to update the inputs.conf file within the Splunk App for Github local directory. In that file, there will be a stanza with the name and value of the token you created. At the end of that stanza, you'll need to add allowQueryStringAuth = true and then restart Splunk. This is best done with the help of your Splunk team, so please reach out to them for assistance on this step.

    -

    Setting Up Github Webhooks

    +

    To enable Query String Authentication, you'll need to update the inputs.conf file within the Splunk App for GitHub local directory. In that file, there will be a stanza with the name and value of the token you created. At the end of that stanza, you'll need to add allowQueryStringAuth = true and then restart Splunk. This is best done with the help of your Splunk team, so please reach out to them for assistance on this step.

    +

    Setting Up GitHub Webhooks

    Webhooks are a simple push mechanism that will send an event each time the webhook is triggered. Unfortunately, Webhooks are unique to each Organization and will need to be setup for each Org as desired. To do this, a user will need to be an Admin for the Org.

    Steps

      @@ -41,29 +41,35 @@ - + - + - + - + - + + + + + + +
      Splunk EventtypeGithub Webhook EventGitHub Webhook Event Description
      Github::RepoGitHub::Repo Repositories Repository created, deleted, archived, unarchived, publicized, privatized, edited, renamed, or transferred.
      Github::PushGitHub::Push Pushes Git push to a repository.
      Github::PullRequestGitHub::PullRequest Pull requests Pull request opened, closed, reopened, edited, assigned, unassigned, review requested, review request removed, labeled, unlabeled, synchronized, ready for review, converted to draft, locked, unlocked, auto merge enabled, auto merge disabled, milestoned, or demilestoned.
      Github::PullRequest::ReviewGitHub::PullRequest::Review Pull request reviews Pull request review submitted, edited, or dismissed.
      GitHub::CodeScanningCode Scanning alertsAlert created, fixed, reopened, appeared in branch, closed by user, or reopened by user.
      diff --git a/github_app_for_splunk/default/data/ui/views/welcome_page.xml b/github_app_for_splunk/default/data/ui/views/welcome_page.xml index 183c0d5..70d7d04 100644 --- a/github_app_for_splunk/default/data/ui/views/welcome_page.xml +++ b/github_app_for_splunk/default/data/ui/views/welcome_page.xml @@ -1,4 +1,4 @@ - + @@ -16,7 +16,7 @@

      This Splunk app is meant to be your single pane of glass for anything GitHub. Whether you're looking for audit log analytics, GitHub Enterprise Server monitoring, or other GitHub metrics, you're in the right place!

      - +       @@ -39,9 +39,12 @@
    1. The "Enterprise Server Monitor" drop down has several dashboards that report on the health and performance of your GHES environment
    2. "Audit" drop down works for GHES as well as GitHub.com audit logs
    3. The Alerts menu item contains all GitHub recommended alerts
      4. +
    4. Audit Log Activity provides a highlevel overview of what activity is going on in GitHub.
      5. +
    5. Repository Audit lets you review changes to individual or groups of repositories
      6. +
    6. User Change Audit is the best place to review actions taken by or made to individual users.

    - +     @@ -55,16 +58,16 @@ }
    -

    Audit and Security

    +

    GitHub Advanced Security

    - Security users will probably be most interested in the Audit dashboards + Open Source repositories and customers of GitHub Advanced Security have access to application security tooling such as Code Scanning, Secret Scanning, and Dependency Review.

      -
    1. Audit Log Activity provides a highlevel overview of what activity is going on in GitHub.
      2. -
    2. Repository Audit lets you review changes to individual or groups of repositories
      3. -
    3. User Change Audit is the best place to review actions taken by or made to individual users.
      4. +
    4. The Advanced Security Overview dashboard gives insight into the security posture of your GitHub Organization
      5. +
    5. The Code Scanning Alerts dashboard gives you access to alerts created by Code Scanning within your Organization
      6. +
    6. The Secret Scanning Alerts dashboard provides visibility into secrets like API keys and personal access tokens that have been checked into your repositories

    -
    +     @@ -85,11 +88,11 @@ GitHub has several ways to collect data from their services depending on your needs. Information is available within the App on how to collect different types of data from GitHub:
    1. Audit Log data is available through a Splunk Add-On
      2. -
    2. Rich commit and pull request data is available through GitHub Webhooks
      3. +
    3. Rich commit, pull request, and Code Scanning data is available through GitHub Webhooks

    - + - \ No newline at end of file + diff --git a/github_app_for_splunk/default/data/ui/views/workflow_analytics.xml b/github_app_for_splunk/default/data/ui/views/workflow_analytics.xml index 1db6ce3..b070ee7 100644 --- a/github_app_for_splunk/default/data/ui/views/workflow_analytics.xml +++ b/github_app_for_splunk/default/data/ui/views/workflow_analytics.xml @@ -1,4 +1,4 @@ - +
    @@ -102,4 +102,4 @@ - \ No newline at end of file + diff --git a/github_app_for_splunk/default/eventtypes.conf b/github_app_for_splunk/default/eventtypes.conf index 71e1e06..591e352 100644 --- a/github_app_for_splunk/default/eventtypes.conf +++ b/github_app_for_splunk/default/eventtypes.conf @@ -14,7 +14,7 @@ search = `github_webhooks` action IN ("submitted","edited","dismissed") pull_req search = `github_webhooks` after=* before=* "commits{}.id"=* ref=* "pusher.name"=* [GitHub::Repo] -search = `github_webhooks` action IN ("created","deleted","archived","unarchived","edited","renamed","transferred","publicized","privatized") "repository.name"=* NOT "pull_request.id"=* NOT "project_card.id"=* NOT "project.number"=* NOT "project_column.id"=* NOT "check_run.id"=* +search = `github_webhooks` action IN ("created","deleted","archived","unarchived","edited","renamed","transferred","publicized","privatized") "repository.name"=* NOT "pull_request.id"=* NOT "project_card.id"=* NOT "project.number"=* NOT "project_column.id"=* NOT "check_run.id"=* NOT "alert.created_at"=* NOT "alert.number"=* [GitHub::Project] search = `github_webhooks` action IN ("created","edited","closed","reopenend","deleted") "project.number"=* @@ -27,3 +27,15 @@ search = `github_webhooks` action IN ("created","edited","moved","deleted") "pr [GitHub::Workflow] search = `github_webhooks` action IN ("queued","created","started","completed") workflow_job.id=* + +[GitHub::CodeScanning] +search = `github_webhooks` action IN ("appeared_in_branch", "closed_by_user", "created", "fixed", "reopened", "reopened_by_user") "alert.created_at"=* + +[GitHub::SecretScanning] +search = `github_webhooks` action IN ("created", "resolved") "alert.secret_type"=* + +[GitHub::VulnerabilityAlert] +search = `github_webhooks` action IN ("create", "dismiss", "resolve") "alert.external_identifier"=* + +[GitHub::Release] +search = `github_webhooks` action IN ("released","published") release.id=* diff --git a/github_app_for_splunk/default/props.conf b/github_app_for_splunk/default/props.conf index ad76534..b6fcbb1 100644 --- a/github_app_for_splunk/default/props.conf +++ b/github_app_for_splunk/default/props.conf @@ -1,5 +1,5 @@ [default] -[GithubEnterpriseServerAuditLog] +[GithubEnterpriseServerLog] DATETIME_CONFIG = LINE_BREAKER = ([\r\n]+) NO_BINARY_CHECK = true @@ -9,7 +9,7 @@ TIME_FORMAT = TZ = EXTRACT-audit_event = github_audit\[\d+\]\:\s(?.*) EXTRACT-audit_fields = \"(?<_KEY_1>.*?)\"\:\"*(?<_VAL_1>.*?)\"*, -EXTRACT-github_log_type = \d+\:\d+\:\d+\s\d+\-\d+\-\d+\-\d+\s(?.*?)\: +EXTRACT-github_log_type = \d+\:\d+\:\d+\s[\d\w\-]+\s(?.*?)\: EXTRACT-github_document_id = \"_document_id\"\:\"(?.*?)\" FIELDALIAS-source = github_log_type AS source @@ -29,7 +29,7 @@ disabled = false pulldown_type = 1 [github_json] -DATETIME_CONFIG = +DATETIME_CONFIG = CURRENT LINE_BREAKER = ([\r\n]+) NO_BINARY_CHECK = true TRUNCATE = 250000 diff --git a/github_app_for_splunk/default/savedsearches.conf b/github_app_for_splunk/default/savedsearches.conf index 132168e..8161ab4 100644 --- a/github_app_for_splunk/default/savedsearches.conf +++ b/github_app_for_splunk/default/savedsearches.conf @@ -38,7 +38,8 @@ search = | mstats avg(_value) as "Avg" WHERE `github_collectd` AND metric_name=" | eval metric_name=mvindex(split(metric_name,"."),2)\ | stats avg("disk_gb") as "Avg" by metric_name, host\ | xyseries host metric_name Avg\ -| eval disk_util=(used/free)*100\ +| eval disk_total=used+free\ +| eval disk_util=(used/disk_total)*100\ | fields host disk_util [GitHub Disk Utilization Over 85%] @@ -82,7 +83,8 @@ search = | mstats avg(_value) as "Avg" WHERE `github_collectd` AND metric_name=" | eval metric_name=mvindex(split(metric_name,"."),2)\ | stats avg("disk_gb") as "Avg" by metric_name, host\ | xyseries host metric_name Avg\ -| eval disk_util=(used/free)*100\ +| eval disk_total=used+free\ +| eval disk_util=(used/disk_total)*100\ | fields host disk_util [GitHub Load Average Above 1] @@ -119,6 +121,6 @@ display.visualizations.custom.type = sunburst_viz.sunburst_viz enableSched = 1 request.ui_dispatch_app = github_app_for_splunk request.ui_dispatch_view = search -search = | mstats max(_value) as "Max" WHERE `github_collectd` AND metric_name="load.longterm" AND host="*" span=10s BY metric_name, host\ -| stats max(Max) as "Load" by metric_name, host\ +search = | mstats avg(_value) as "Avg" WHERE `github_collectd` AND metric_name="load.longterm" AND host="*" span=10s BY metric_name, host\ +| stats avg(Avg) as "Load" by metric_name, host\ | xyseries host metric_name Load pFad - Phonifier reborn

    Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

    Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


    Alternative Proxies:

    Alternative Proxy

    pFad Proxy

    pFad v3 Proxy

    pFad v4 Proxy