Contact Us

For free technical help, visit the Percona Community Forum.
-

To report bugs or submit feature requests, open a JIRA ticket.
-

For paid support and managed or consulting services , contact Percona Sales.

404 - Not found

+We can't find the page you are looking for. Try using the Search or return to homepage .

For help, click the link below to get free database assistance or contact our experts for personalized support.

+ +

+ + Get help from Percona +

+ Percona LLC and/or its affiliates, © {{ build_date_utc.strftime('%Y') }} — Cookie Preferences +

+ {% if not config.extra.generator == false %} + Made with + + Material for MkDocs + + {% endif %} +

-multi + $ docker run --name container-name -e POSTGRES_PASSWORD=secret -d percona/percona-distribution-postgresql:{{dockertag}} ``` Where: * `container-name` is the name you assign to your container * `POSTGRES_PASSWORD` is the superuser password - * `tag-multi` is the tag specifying the version you need. For example, `{{dockertag}}-multi`. The `multi` part of the tag serves to identify the architecture (x86_64 or ARM64) and pull the respective image. See the [full list of tags :octicons-link-external-16:](https://hub.docker.com/r/percona/percona-distribution-postgresql/tags/). + * `{{dockertag}}` is the tag specifying the version you need. Docker identifies the architecture (x86_64 or ARM64) and pulls the respective image. See the [full list of tags :octicons-link-external-16:](https://hub.docker.com/r/percona/percona-distribution-postgresql/tags/). !!! tip @@ -56,7 +56,7 @@ For more information about using Docker, see the [Docker Docs :octicons-link-ext 2. Start the container: ```{.bash data-prompt="$"} - $ docker run --name container-name --env-file ./.my-pg.env -d percona/percona-distribution-postgresql:-multi + $ docker run --name container-name --env-file ./.my-pg.env -d percona/percona-distribution-postgresql:{{dockertag}} ``` 2. Connect to the container's interactive terminal: @@ -87,14 +87,14 @@ where: The following command starts another container instance and runs the `psql` command line client against your original container, allowing you to execute SQL statements against your database: ```{.bash data-prompt="$"} -$ docker run -it --network container:db-container-name --name container-name percona/percona-distribution-postgresql:-multi psql -h address -U postgres +$ docker run -it --network container:db-container-name --name container-name percona/percona-distribution-postgresql:{{dockertag}} psql -h address -U postgres ``` Where: * `db-container-name` is the name of your database container * `container-name` is the name of your container that you will use to connect to the database container using the `psql` command line client -`tag-multi` is the tag specifying the version you need. For example, `{{dockertag}}-multi`. The `multi` part of the tag serves to identify the architecture (x86_64 or ARM64) and pull the respective image. +* `{{dockertag}}` is the tag specifying the version you need. Docker identifies the architecture (x86_64 or ARM64) and pulls the respective image. * `address` is the network address where your database container is running. Use 127.0.0.1, if the database container is running on the local machine/host. ## Enable `pg_stat_monitor` diff --git a/docs/enable-extensions.md b/docs/enable-extensions.md index 76c386a41..31cee70e0 100644 --- a/docs/enable-extensions.md +++ b/docs/enable-extensions.md @@ -4,7 +4,7 @@ Some components require additional configuration before using them with Percona ## Patroni -Patroni is the third-party high availability solution for PostgreSQL. The [High Availability in PostgreSQL with Patroni](solutions/high-availability.md) chapter provides details about the solution overview and architecture deployment. +Patroni is the high availability solution for PostgreSQL. The [High Availability in PostgreSQL with Patroni](solutions/high-availability.md) chapter provides details about the solution overview and architecture deployment. While setting up a high availability PostgreSQL cluster with Patroni, you will need the following components: @@ -14,9 +14,9 @@ While setting up a high availability PostgreSQL cluster with Patroni, you will n - [HAProxy :octicons-link-external-16:](http://www.haproxy.org/). -If you install the software fom packages, all required dependencies and service unit files are included. If you [install the software from the tarballs](tarball.md), you must first enable `etcd`. See the steps in the [etcd](#etcd) section if this document. +If you install the software fom packages, all required dependencies and service unit files are included. If you [install the software from the tarballs](tarball.md), you must first enable `etcd`. See the steps in the [etcd](#etcd) section in this document. -See the configuration guidelines for [Debian and Ubuntu](solutions/ha-setup-apt.md) and [RHEL and CentOS](solutions/ha-setup-yum.md). +See the configuration guidelines for [Patroni](solutions/ha-patroni.md) and [etcd](solutions/ha-etcd-config.md). ## etcd @@ -60,7 +60,7 @@ For details about each option, see [pdBadger documentation :octicons-link-extern ## pgaudit -Add the `pgaudit` to `shared_preload_libraries` in `postgresql.conf`. The recommended way is to use the [ALTER SYSTEM](https://www.postgresql.org/docs/16/sql-altersystem.html) command. [Connect to psql](#connect-to-the-postgresql-server) and use the following command: +Add the `pgaudit` to `shared_preload_libraries` in `postgresql.conf`. The recommended way is to use the [ALTER SYSTEM](https://www.postgresql.org/docs/16/sql-altersystem.html) command. [Connect to psql](connect.md) and use the following command: ```sql ALTER SYSTEM SET shared_preload_libraries = 'pgaudit'; @@ -125,7 +125,7 @@ $ pgpool -f /pgpool.conf ## pg_stat_monitor -Please refer to [`pg_stat_monitor`](pg-stat-monitor.md#setup) for setup steps. +Please refer to [`pg_stat_monitor`](https://docs.percona.com/pg-stat-monitor/setup.html) for setup steps. ## wal2json @@ -137,6 +137,14 @@ wal_level = logical Start / restart the server to apply the changes. +## pgvector + +To get started, enable the extension for the database where you want to use it: + +```sql +CREATE EXTENSION vector; +``` + ## Next steps [Connect to PostgreSQL :material-arrow-right:](connect.md){.md-button} diff --git a/docs/extensions.md b/docs/extensions.md index 4c7dd7d87..9ce4a189d 100644 --- a/docs/extensions.md +++ b/docs/extensions.md @@ -4,12 +4,10 @@ Percona Distribution for PostgreSQL is not only the PostgreSQL server. It also i Percona Distribution for PostgreSQL includes the extensions that have been tested to work together. These extensions encompass the following: -* [PostgreSQL contrib modules and utilities](contrib.md) -* Extensions authored by Percona: - - * [`pg_stat_monitor`](pg-stat-monitor.md) - * [`pg_tde`](pg-tde.md) +Percona Distribution for PostgreSQL includes the extensions that have been tested to work together. These extensions encompass the following: +* [PostgreSQL contrib modules and utilities](contrib.md) +* [Extensions authored by Percona](percona-ext.md) * [Third-party components](third-party.md) Percona also supports [extra modules](https://repo.percona.com/ppg-16-extras/), not included in Percona Distribution for PostgreSQL but tested to work with it. diff --git a/docs/get-help.md b/docs/get-help.md new file mode 100644 index 000000000..f5b0420be --- /dev/null +++ b/docs/get-help.md @@ -0,0 +1,27 @@ +# Get help from Percona + +Our documentation guides are packed with information, but they can’t cover everything you need to know about Percona Distribution for PostgreSQL. They also won’t cover every scenario you might come across. Don’t be afraid to try things out and ask questions when you get stuck. + +## Percona's Community Forum + +Be a part of a space where you can tap into a wealth of knowledge from other database enthusiasts and experts who work with Percona’s software every day. While our service is entirely free, keep in mind that response times can vary depending on the complexity of the question. You are engaging with people who genuinely love solving database challenges. + +We recommend visiting our [Community Forum](https://forums.percona.com/t/welcome-to-perconas-community-forum/7){:target="_blank"}. It’s an excellent place for discussions, technical insights, and support around Percona database software. If you’re new and feeling a bit unsure, our [FAQ](https://forums.percona.com/faq){:target="_blank"} and [Guide for New Users](https://forums.percona.com/t/faq-guide-for-new-users/8562){:target="_blank"} ease you in. + +If you have thoughts, feedback, or ideas, the community team would like to hear from you at [Any ideas on how to make the forum better?](https://forums.percona.com/t/any-ideas-on-how-to-make-the-forum-better/11522){:target="blank"}. We’re always excited to connect and improve everyone's experience. + +## Percona experts + +Percona experts bring years of experience in tackling tough database performance issues and design challenges. + +

+We understand your challenges when managing complex database environments. That's why we offer various services to help you simplify your operations and achieve your goals. + +| Service | Description | +|----------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| 24/7 Expert Support | Our dedicated team of database experts is available 24/7 to assist you with any database issues. We provide flexible support plans tailored to your specific needs. | +| Hands-On Database Management | Our managed services team can take over the day-to-day management of your database infrastructure, freeing up your time to focus on other priorities. | +| Expert Consulting | Our experienced consultants provide guidance on database topics like architecture design, migration planning, performance optimization, and security best practices. | +| Comprehensive Training | Our training programs help your team develop skills to manage databases effectively, offering virtual and in-person courses. | + +We're here to help you every step of the way. Whether you need a quick fix or a long-term partnership, we're ready to provide our expertise and support. diff --git a/docs/how-to.md b/docs/how-to.md deleted file mode 100644 index 86acdd79e..000000000 --- a/docs/how-to.md +++ /dev/null @@ -1,75 +0,0 @@ -# How to - -## How to configure etcd nodes simultaneously - -!!! note - - We assume you have a deeper knowledge of how etcd works. Otherwise, refer to the configuration where you add etcd nodes one by one. - -Instead of adding `etcd` nodes one by one, you can configure and start all nodes in parallel. - -1. Create the etcd configuration file on every node. You can edit the sample configuration file `/etc/etcd/etcd.conf.yaml` or create your own one. Replace the node names and IP addresses with the actual names and IP addresses of your nodes. - - === "node1" - - ```yaml title="/etc/etcd/etcd.conf.yaml" - name: 'node1' - initial-cluster-token: PostgreSQL_HA_Cluster_1 - initial-cluster-state: new - initial-cluster: node1=http://10.104.0.1:2380,node2=http://10.104.0.2:2380,node3=http://10.104.0.3:2380 - data-dir: /var/lib/etcd - initial-advertise-peer-urls: http://10.104.0.1:2380 - listen-peer-urls: http://10.104.0.1:2380 - advertise-client-urls: http://10.104.0.1:2379 - listen-client-urls: http://10.104.0.1:2379 - ``` - - === "node2" - - ```yaml title="/etc/etcd/etcd.conf.yaml" - name: 'node2' - initial-cluster-token: PostgreSQL_HA_Cluster_1 - initial-cluster-state: new - initial-cluster: node1=http://10.104.0.1:2380,node2=http://10.104.0.2:2380, node3=http://10.104.0.3:2380 - data-dir: /var/lib/etcd - initial-advertise-peer-urls: http://10.104.0.2:2380 - listen-peer-urls: http://10.104.0.2:2380 - advertise-client-urls: http://10.104.0.2:2379 - listen-client-urls: http://10.104.0.2:2379 - ``` - - === "node3" - - ```yaml title="/etc/etcd/etcd.conf.yaml" - name: 'node1' - initial-cluster-token: PostgreSQL_HA_Cluster_1 - initial-cluster-state: new - initial-cluster: node1=http://10.104.0.1:2380,node2=http://10.104.0.2:2380, node3=http://10.104.0.3:2380 - data-dir: /var/lib/etcd - initial-advertise-peer-urls: http://10.104.0.3:2380 - listen-peer-urls: http://10.104.0.3:2380 - advertise-client-urls: http://10.104.0.3:2379 - listen-client-urls: http://10.104.0.3:2379 - ``` - -2. Enable and start the `etcd` service on all nodes: - - ```{.bash data-prompt="$"} - $ sudo systemctl enable --now etcd - ``` - - During the node start, etcd searches for other cluster nodes defined in the configuration. If the other nodes are not yet running, the start may fail by a quorum timeout. This is expected behavior. Try starting all nodes again at the same time for the etcd cluster to be created. - -3. Check the etcd cluster members. Connect to one of the nodes and run the following command: - - ```{.bash data-prompt="$"} - $ sudo etcdctl member list - ``` - - The output resembles the following: - - ``` - 2d346bd3ae7f07c4: name=node2 peerURLs=http://10.104.0.2:2380 clientURLs=http://10.104.0.2:2379 isLeader=false - 8bacb519ebdee8db: name=node3 peerURLs=http://10.104.0.3:2380 clientURLs=http://10.104.0.3:2379 isLeader=false - c5f52ea2ade25e1b: name=node1 peerURLs=http://10.104.0.1:2380 clientURLs=http://10.104.0.1:2379 isLeader=true - ``` diff --git a/docs/index.md b/docs/index.md index 1a9085ceb..b468921a9 100644 --- a/docs/index.md +++ b/docs/index.md @@ -47,11 +47,6 @@ Our comprehensive resources will help you overcome challenges, from everyday iss Learn about the releases and changes in the Distribution. -[Release notes :material-arrow-right:](release-notes.md){.md-button} +[Release notes :material-arrow-right:](release-notes/{{release}}.md){.md-button} - - - - - diff --git a/docs/installing.md b/docs/installing.md index 738877148..27c95e567 100644 --- a/docs/installing.md +++ b/docs/installing.md @@ -4,14 +4,14 @@ Percona Distribution for PostgreSQL is the PostgreSQL server with the collection This document aims to guide database application developers and DevOps engineer in getting started with Percona Distribution for PostgreSQL. Upon completion of this guide, you’ll have Percona Distribution for PostgreSQL installed and operational, and you’ll be able to: -* Connect to PostgreSQL using the `psql` interactive terminal +* Connect to PostgreSQL using the `psql` interactive terminal * Interact with PostgreSQL with basic psql commands -* Manipulate data in PostgreSQL +* Manipulate data in PostgreSQL * Understand the next steps you can take as a database application developer or administrator to expand your knowledge of Percona Distribution for PostgreSQL ## Install Percona Distribution for PostgreSQL -You can select from multiple easy-to-follow installation options, but **we recommend using a Package Manager** for a convenient and quick way to try the software first. +You can select from multiple easy-to-follow installation options, however **we strongly recommend using a Package Manager** for a convenient and quick way to try the software first. === ":octicons-terminal-16: Package manager" @@ -24,7 +24,6 @@ You can select from multiple easy-to-follow installation options, but **we recom [Install via apt :material-arrow-right:](apt.md){.md-button} [Install via yum :material-arrow-right:](yum.md){.md-button} - === ":simple-docker: Docker" Get our image from Docker Hub and spin up a cluster on a Docker container for quick evaluation. @@ -41,15 +40,13 @@ You can select from multiple easy-to-follow installation options, but **we recom [Get started with Percona Operator :octicons-link-external-16:](https://docs.percona.com/percona-operator-for-postgresql/2.0/quickstart.html){.md-button} -=== ":octicons-download-16: Manual download" +=== ":octicons-download-16: Tar download (not recommended)" - If you need to install Percona Distribution for PostgreSQL offline or as a non-superuser, check out the link below for a step-by-step guide and get access to the downloads directory. + If installing the package (the **recommended** method for a safe, secure, and reliable setup) is not an option, refer to the link below for step-by-step instructions on installing from tarballs using the provided download links. - Note that for this scenario you must make sure that all dependencies are satisfied. + In this scenario, you must ensure that all dependencies are met. Failure to do so may result in errors or crashes. + + !!! note + This method is **not recommended** for mission-critical environments. [Install from tarballs :material-arrow-right:](tarball.md){.md-button} - - - - - diff --git a/docs/js/version-select.js b/docs/js/version-select.js index dd66d6b4a..b24febf38 100644 --- a/docs/js/version-select.js +++ b/docs/js/version-select.js @@ -1,120 +1,64 @@ -setTimeout(() => { - const asideMenu = document.getElementsByClassName('sphinxsidebarwrapper')[0]; - hideSubMenus(); - asideMenu.style.display = 'block'; -}, 500); - -function hideSubMenus() { - const asideMenu = document.getElementsByClassName('sphinxsidebarwrapper')[0]; - const activeCheckboxClass = 'custom-button--active'; - const activeBackgroundClass = 'custom-button--main-active'; - const links = Array.from(asideMenu.getElementsByTagName('a')); - const accordionLinks = links.filter(links => links.nextElementSibling && links.nextElementSibling.localName === 'ul'); - const simpleLinks = links.filter(links => !links.nextElementSibling && links.parentElement.localName === 'li'); - - simpleLinks.forEach(simpleLink => { - simpleLink.parentElement.style.listStyleType = 'disc'; - simpleLink.parentElement.style.marginLeft = '20px'; +/* + * Custom version of same taken from mike code for injecting version switcher into percona.com + */ + +window.addEventListener('DOMContentLoaded', function () { + // This is a bit hacky. Figure out the base URL from a known CSS file the + // template refers to... + var ex = new RegExp('/?css/version-select.css$'); + var sheet = document.querySelector('link[href$="version-select.css"]'); + + if (!sheet) { + return; + } + + var ABS_BASE_URL = sheet.href.replace(ex, ''); + var CURRENT_VERSION = ABS_BASE_URL.split('/').pop(); + + function makeSelect(options, selected) { + var select = document.createElement('select'); + select.classList.add('btn'); + select.classList.add('btn-primary'); + + options.forEach(function (i) { + var option = new Option(i.text, i.value, undefined, i.value === selected); + select.add(option); }); - accordionLinks.forEach((link, index) => { - insertButton(link, index); + return select; + } + + var xhr = new XMLHttpRequest(); + xhr.open('GET', ABS_BASE_URL + '/../versions.json'); + xhr.onload = function () { + var versions = JSON.parse(this.responseText); + + var realVersion = versions.find(function (i) { + return ( + i.version === CURRENT_VERSION || i.aliases.includes(CURRENT_VERSION) + ); + }).version; + + var select = makeSelect( + versions.map(function (i) { + return { text: i.title, value: i.version }; + }), + realVersion + ); + select.addEventListener('change', function (event) { + window.location.href = ABS_BASE_URL + '/../' + this.value; }); - const buttons = Array.from(document.getElementsByClassName('custom-button')); - - buttons.forEach(button => button.addEventListener('click', event => { - event.preventDefault(); - const current = event.currentTarget; - const parent = current.parentElement; - const isMain = Array.from(parent.classList).includes('toctree-l1'); - const isMainActive = Array.from(parent.classList).includes(activeBackgroundClass); - const targetClassList = Array.from(current.classList); - - toggleElement(targetClassList.includes(activeCheckboxClass), current, activeCheckboxClass); - if (isMain) { - toggleElement(isMainActive, parent, activeBackgroundClass); - } - })); - -// WIP var toctree_heading = document.getElementById("toctree-heading"); -// NOT NEEDED? asideMenu.parentNode.insertBefore(styleDomEl, asideMenu); -} - -function toggleElement(condition, item, className) { - const isButton = item.localName === 'button'; - - if (!condition) { - const previousActive = Array.from(item.parentElement.parentElement.getElementsByClassName('list-item--active')); - if (isButton) { - localStorage.setItem(item.id, 'true'); + var container = document.createElement('div'); + container.id = 'custom_select'; + container.classList.add('side-column-block'); - if (previousActive.length) { - previousActive.forEach(previous => { + // Add menu + container.appendChild(select); - const previousActiveButtons = Array.from(previous.getElementsByClassName('custom-button--active')); - removeClass(previous, ['list-item--active', 'custom-button--main-active']); + var sidebar = document.querySelector('#version-select-wrapper'); // Inject menu into element with this ID + sidebar.appendChild(container); + }; - if (previousActiveButtons.length) { - previousActiveButtons.forEach(previousButton => { - - removeClass(previousButton, 'custom-button--active'); - localStorage.removeItem(previousButton.id); - }); - } - }) - } - } - addClass(item, className); - addClass(item.parentElement, 'list-item--active'); - } else { - removeClass(item, className); - removeClass(item.parentElement, 'list-item--active'); - - if (isButton) { - localStorage.removeItem(item.id); - } - } -} -function addClass(item, classes) { - item.classList.add(...Array.isArray(classes) ? classes : [classes]); -} -function removeClass(item, classes) { - item.classList.remove(...Array.isArray(classes) ? classes : [classes]); -} -function insertButton(element, id) { - const button = document.createElement('button'); - const isMain = Array.from(element.parentElement.classList).includes('toctree-l1'); - button.id = id; - addClass(button, 'custom-button'); - if (localStorage.getItem(id)) { - addClass(button, 'custom-button--active'); - addClass(element.parentElement, 'list-item--active'); - if (isMain) { - addClass(element.parentElement, 'custom-button--main-active'); - } - } - element.insertAdjacentElement('beforebegin', button); -} -function makeSelect() { - const custom_select = document.getElementById('custom_select'); - const select_active_option = custom_select.getElementsByClassName('select-active-text')[0]; - const custom_select_list = document.getElementById('custom_select_list'); - - select_active_option.innerHTML = window.location.href.includes('') ? - custom_select_list.getElementsByClassName('custom-select__option')[1].innerHTML : - custom_select_list.getElementsByClassName('custom-select__option')[0].innerHTML; - - document.addEventListener('click', event => { - if (event.target.parentElement.id === 'custom_select' || event.target.id === 'custom_select') { - custom_select_list.classList.toggle('select-hidden') - } - if (Array.from(event.target.classList).includes('custom-select__option')) { - select_active_option.innerHTML = event.target.innerHTML; - } - if (event.target.id !== 'custom_select' && event.target.parentElement.id !== 'custom_select') { - custom_select_list.classList.add('select-hidden') - } - - }); -} \ No newline at end of file + xhr.send(); +}); \ No newline at end of file diff --git a/docs/ldap.md b/docs/ldap.md index 45e24eba1..03ae6c1ba 100644 --- a/docs/ldap.md +++ b/docs/ldap.md @@ -2,6 +2,6 @@ When a client application or a user that runs the client application connects to the database, it must identify themselves. The process of validating the client's identity and determining whether this client is permitted to access the database it has requested is called **authentication**. -Percona Distribution for PortgreSQL supports several [authentication methods :octicons-link-external-16:](https://www.postgresql.org/docs/15/auth-methods.html), including the [LDAP authentication :octicons-link-external-16:](https://www.postgresql.org/docs/14/auth-ldap.html). The use of LDAP is to provide a central place for authentication - meaning the LDAP server stores usernames and passwords and their resource permissions. +Percona Distribution for PortgreSQL supports several [authentication methods :octicons-link-external-16:](https://www.postgresql.org/docs/{{pgversion}}/auth-methods.html), including the [LDAP authentication :octicons-link-external-16:](https://www.postgresql.org/docs/{{pgversion}}/auth-ldap.html). The use of LDAP is to provide a central place for authentication - meaning the LDAP server stores usernames and passwords and their resource permissions. The LDAP authentication in Percona Distribution for PortgreSQL is implemented the same way as in upstream PostgreSQL. \ No newline at end of file diff --git a/docs/major-upgrade.md b/docs/major-upgrade.md index 46fca6a82..ebe842801 100644 --- a/docs/major-upgrade.md +++ b/docs/major-upgrade.md @@ -2,16 +2,15 @@ This document describes the in-place upgrade of Percona Distribution for PostgreSQL using the `pg_upgrade` tool. -!!! important +To ensure a smooth upgrade path, follow these steps: - When running a major upgrade on **RHEL 8 and compatible derivatives**, consider the following: - - Percona Distribution for PostgreSQL 16.3, 15.7, 14.12, 13.15 and 12.18 include `llvm` packages 16.0.6, while its previous versions 16.2, 15.6, 14.11, 13.14, and 12.17 include `llvm` 12.0.1. Since `llvm` libraries differ and are not compatible, the direct major version upgrade from 15.6 to 16.3 may cause issues. +* Upgrade to the latest minor version within your current major version (e.g., from 15.11 to 15.13). +* Then, perform the major upgrade to your desired version (e.g., from 15.13 to 16.9). - To ensure a smooth upgrade path, follow these steps: +!!! Note + When running a major upgrade for **RHEL 8 and compatible derivatives**, consider the following: - * Upgrade to the latest minor version within your current major version (e.g., from 15.6 to 15.7). - * Then, perform the major upgrade to your desired version (e.g., from 15.7 to 16.3). + Percona Distribution for PostgreSQL 16.3, 15.7, 14.12, 13.15 and 12.18 include `llvm` packages 16.0.6, while its previous versions 16.2, 15.6, 14.11, 13.14, and 12.17 include `llvm` 12.0.1. Since `llvm` libraries differ and are not compatible, the direct major version upgrade from 15.6 to 16.3 may cause issues. The in-place upgrade means installing a new version without removing the old version and keeping the data files on the server. @@ -58,17 +57,18 @@ Run **all** commands as root or via **sudo**: 1. Install Percona Distribution for PostgreSQL 16 packages. + !!! note + When installing version 16, if prompted via a pop-up to upgrade to the latest available version, select **No**. * [Install percona-release :octicons-link-external-16:](https://docs.percona.com/percona-software-repositories/installing.html). If you have installed it before, [update it to the latest version](https://docs.percona.com/percona-software-repositories/updating.html) - - * Enable Percona repository: + + * Enable Percona repository ```{.bash data-prompt="$"} $ sudo percona-release setup ppg-16 ``` - - * Install Percona Distribution for PostgreSQL 16 package: + * Install Percona Distribution for PostgreSQL 16 package ```{.bash data-prompt="$"} $ sudo apt install percona-postgresql-16 @@ -82,94 +82,113 @@ Run **all** commands as root or via **sudo**: This stops both Percona Distribution for PostgreSQL 15 and 16. - 3. Run the database upgrade. + * Log in as the `postgres` user - * Log in as the `postgres` user. - - ```{.bash data-prompt="$"} - $ sudo su postgres - ``` - - - * Change the current directory to the `tmp` directory where logs and some scripts will be recorded: - - ```{.bash data-prompt="$"} - $ cd tmp/ - ``` - - - * Check the ability to upgrade Percona Distribution for PostgreSQL from 15 to 16: - - ```{.bash data-prompt="$"} - $ /usr/lib/postgresql/16/bin/pg_upgrade \ - --old-datadir=/var/lib/postgresql/15/main \ - --new-datadir=/var/lib/postgresql/16/main \ - --old-bindir=/usr/lib/postgresql/15/bin \ - --new-bindir=/usr/lib/postgresql/16/bin \ - --old-options '-c config_file=/etc/postgresql/15/main/postgresql.conf' \ - --new-options '-c config_file=/etc/postgresql/16/main/postgresql.conf' \ - --check - ``` - - The `--check` flag here instructs `pg_upgrade` to only check the upgrade without changing any data. + ```{.bash data-prompt="$"} + $ sudo su postgres + ``` - **Sample output** + * Check if you can upgrade Percona Distribution for PostgreSQL from 15 to 16 - ``` - Performing Consistency Checks - ----------------------------- - Checking cluster versions ok - Checking database user is the install user ok - Checking database connection settings ok - Checking for prepared transactions ok - Checking for reg* data types in user tables ok - Checking for contrib/isn with bigint-passing mismatch ok - Checking for tables WITH OIDS ok - Checking for invalid "sql_identifier" user columns ok - Checking for presence of required libraries ok - Checking database user is the install user ok - Checking for prepared transactions ok - - *Clusters are compatible* - ``` + ```bash + $ pg_upgradecluster 15 main --check + # Sample output: pg_upgradecluster pre-upgrade checks ok + ``` + The `--check` flag here instructs `pg_upgrade` to only check the upgrade without changing any data. * Upgrade the Percona Distribution for PostgreSQL - ```{.bash data-prompt="$"} - $ /usr/lib/postgresql/16/bin/pg_upgrade \ - --old-datadir=/var/lib/postgresql/15/main \ - --new-datadir=/var/lib/postgresql/16/main \ - --old-bindir=/usr/lib/postgresql/15/bin \ - --new-bindir=/usr/lib/postgresql/16/bin \ - --old-options '-c config_file=/etc/postgresql/15/main/postgresql.conf' \ - --new-options '-c config_file=/etc/postgresql/16/main/postgresql.conf' \ - --link - ``` - - The `--link` flag creates hard links to the files on the old version cluster so you don’t need to copy data. - - If you don’t wish to use the `--link` option, make sure that you have enough disk space to store 2 copies of files for both old version and new version clusters. - - - * Go back to the regular user: - - ```{.bash data-prompt="$"} - $ exit - ``` - - - * The Percona Distribution for PostgreSQL 15 uses the `5432` port while the Percona Distribution for PostgreSQL 16 is set up to use the `5433` port by default. To start the Percona Distribution for PostgreSQL 15, swap ports in the configuration files of both versions. - - ```{.bash data-prompt="$"} - $ sudo vim /etc/postgresql/16/main/postgresql.conf - $ port = 5433 # Change to 5432 here - $ sudo vim /etc/postgresql/15/main/postgresql.conf - $ port = 5432 # Change to 5433 here - ``` + ```bash + $ pg_upgradecluster 15 main + ``` +

Sample output (click to expand)

+ ```bash + Upgrading cluster 15/main to 16/main ... + Stopping old cluster... + Restarting old cluster with restricted connections... + ... + Success. Please check that the upgraded cluster works. If it does, + you can remove the old cluster with: + pg_dropcluster 15 main + + Ver Cluster Port Status Owner Data directory Log file + 16 main 5432 online postgres /var/lib/postgresql/16/main /var/log/postgresql/postgresql-16-main.log + + Sample output: + Upgrading cluster 15/main to 16/main ... + Stopping old cluster... + Restarting old cluster with restricted connections... + Notice: extra pg_ctl/postgres options given, bypassing systemctl for start operation + Creating new PostgreSQL cluster 16/main ... + /usr/lib/postgresql/16/bin/initdb -D /var/lib/postgresql/16/main --auth-local peer --auth-host scram-sha-256 --no-instructions --encoding UTF8 --lc-collate C.UTF-8 --lc-ctype C.UTF-8 --locale-provider libc + The files belonging to this database system will be owned by user "postgres". + This user must also own the server process. + + The database cluster will be initialized with locale "C.UTF-8". + The default text search configuration will be set to "english". + + Data page checksums are disabled. + + fixing permissions on existing directory /var/lib/postgresql/16/main ... ok + creating subdirectories ... ok + selecting dynamic shared memory implementation ... posix + selecting default max_connections ... 100 + selecting default shared_buffers ... 128MB + selecting default time zone ... Etc/UTC + creating configuration files ... ok + running bootstrap script ... ok + performing post-bootstrap initialization ... ok + syncing data to disk ... ok + + Copying old configuration files... + Copying old start.conf... + Copying old pg_ctl.conf... + Starting new cluster... + Notice: extra pg_ctl/postgres options given, bypassing systemctl for start operation + Running init phase upgrade hook scripts ... + + Roles, databases, schemas, ACLs... + set_config + ------------ + + (1 row) + + set_config + ------------ + + (1 row) + + Fixing hardcoded library paths for stored procedures... + Upgrading database template1... + Fixing hardcoded library paths for stored procedures... + Upgrading database postgres... + Stopping target cluster... + Stopping old cluster... + Disabling automatic startup of old cluster... + Starting upgraded cluster on port 5432... + Running finish phase upgrade hook scripts ... + vacuumdb: processing database "postgres": Generating minimal optimizer statistics (1 target) + vacuumdb: processing database "template1": Generating minimal optimizer statistics (1 target) + vacuumdb: processing database "postgres": Generating medium optimizer statistics (10 targets) + vacuumdb: processing database "template1": Generating medium optimizer statistics (10 targets) + vacuumdb: processing database "postgres": Generating default (full) optimizer statistics + vacuumdb: processing database "template1": Generating default (full) optimizer statistics + + Success. Please check that the upgraded cluster works. If it does, + you can remove the old cluster with + pg_dropcluster 15 main + + Ver Cluster Port Status Owner Data directory Log file + 15 main 5433 down postgres /var/lib/postgresql/15/main /var/log/postgresql/postgresql-15-main.log + Ver Cluster Port Status Owner Data directory Log file + 16 main 5432 online postgres /var/lib/postgresql/16/main /var/log/postgresql/postgresql-16-main.log + ``` +

4. Start the `postgreqsl` service. @@ -177,38 +196,29 @@ Run **all** commands as root or via **sudo**: $ sudo systemctl start postgresql.service ``` - 5. Check the `postgresql` version. * Log in as a postgres user - + ```{.bash data-prompt="$"} $ sudo su postgres ``` * Check the database version - + ```{.bash data-prompt="$"} $ psql -c "SELECT version();" ``` +6. Delete the old cluster's data files. -6. After the upgrade, the Optimizer statistics are not transferred to the new cluster. Run the `vaccumdb` command to analyze the new cluster: - - ```{.bash data-prompt="$"} - $ /usr/lib/postgresql/16/bin/vacuumdb --all --analyze-in-stages - ``` + !!! note + Before deleting the old cluster, verify that the newly upgraded cluster is fully operational. Keeping the old cluster does not negatively affect the functionality or performance of your upgraded cluster. -7. Delete the old cluster's data files: - ```{.bash data-prompt="$"} - $ ./delete_old_cluster.sh - $ sudo rm -rf /etc/postgresql/15/main - $ #Logout - $ exit + $ pg_dropcluster 15 main ``` - ## On Red Hat Enterprise Linux and CentOS using `yum` Run **all** commands as root or via **sudo**: @@ -216,7 +226,6 @@ Run **all** commands as root or via **sudo**: 1. Install Percona Distribution for PostgreSQL 16 packages - * [Install percona-release :octicons-link-external-16:](https://docs.percona.com/percona-software-repositories/installing.html) * Enable Percona repository: @@ -225,7 +234,6 @@ Run **all** commands as root or via **sudo**: $ sudo percona-release setup ppg-16 ``` - * Install Percona Distribution for PostgreSQL 16: ```{.bash data-prompt="$"} @@ -253,24 +261,20 @@ Run **all** commands as root or via **sudo**: $ /usr/pgsql-16/bin/initdb -D /var/lib/pgsql/16/data ``` - 3. Stop the `postgresql` 15 service ```{.bash data-prompt="$"} $ systemctl stop postgresql-15 ``` - 4. Run the database upgrade. - * Log in as the `postgres` user ```{.bash data-prompt="$"} $ sudo su postgres ``` - * Check the ability to upgrade Percona Distribution for PostgreSQL from 15 to 16: ```{.bash data-prompt="$"} @@ -304,7 +308,6 @@ Run **all** commands as root or via **sudo**: *Clusters are compatible* ``` - * Upgrade the Percona Distribution for PostgreSQL ```{.bash data-prompt="$"} @@ -319,7 +322,6 @@ Run **all** commands as root or via **sudo**: The `--link` flag creates hard links to the files on the old version cluster so you don’t need to copy data. If you don’t wish to use the `--link` option, make sure that you have enough disk space to store 2 copies of files for both old version and new version clusters. - 5. Start the `postgresql` 16 service. ```{.bash data-prompt="$"} @@ -332,10 +334,8 @@ Run **all** commands as root or via **sudo**: $ systemctl status postgresql-16 ``` - 7. After the upgrade, the Optimizer statistics are not transferred to the new cluster. Run the `vaccumdb` command to analyze the new cluster: - * Log in as the postgres user ```{.bash data-prompt="$"} @@ -348,14 +348,12 @@ Run **all** commands as root or via **sudo**: $ /usr/pgsql-16/bin/vacuumdb --all --analyze-in-stages ``` - 8. Delete Percona Distribution for PostgreSQL 15 configuration files ```{.bash data-prompt="$"} $ ./delete_old_cluster.sh ``` - 9. Delete Percona Distribution old data files ```{.bash data-prompt="$"} diff --git a/docs/percona-ext.md b/docs/percona-ext.md index a68a188f5..0fc67434b 100644 --- a/docs/percona-ext.md +++ b/docs/percona-ext.md @@ -16,7 +16,7 @@ A query performance monitoring tool for PostgreSQL that brings more insight and An open-source extension designed to enhance PostgreSQL’s security by encrypting data files on disk. The encryption is transparent for users allowing them to access and manipulate the data and not to worry about the encryption process. -[pg_tde documentation :octicons-link-external-16:](https://percona-lab.github.io/pg_tde/main/){.md-button} +[pg_tde documentation :octicons-link-external-16:](https://percona.github.io/pg_tde/main/index.html){.md-button} \ No newline at end of file diff --git a/docs/pg-stat-monitor.md b/docs/pg-stat-monitor.md deleted file mode 100644 index 7793c5fed..000000000 --- a/docs/pg-stat-monitor.md +++ /dev/null @@ -1,277 +0,0 @@ -# pg_stat_monitor - -!!! note - - This document describes the functionality of pg_stat_monitor {{pgsmversion}}. - -## Overview - -`pg_stat_monitor` is a Query Performance Monitoring -tool for PostgreSQL. It collects various statistics data such as query statistics, query plan, SQL comments and other performance insights. The collected data is aggregated and presented in a single view. This allows you to view queries from performance, application and analysis perspectives. - -`pg_stat_monitor` groups statistics data and writes it in a storage unit called *bucket*. The data is added and stored in a bucket for the defined period – the bucket lifetime. This allows you to identify performance issues and patterns based on time. - -You can specify the following: - - -* The number of buckets. Together they form a bucket chain. -* Bucket size. This is the amount of shared memory allocated for buckets. Memory is divided equally among buckets. -* Bucket lifetime. - -When a bucket lifetime expires, `pg_stat_monitor` resets all statistics and writes the data in the next bucket in the chain. When the last bucket’s lifetime expires, `pg_stat_monitor` returns to the first bucket. - -!!! important - - The contents of the bucket will be overwritten. In order not to lose the data, make sure to read the bucket before `pg_stat_monitor` starts writing new data to it. - - -### Views - -#### pg_stat_monitor view - -The `pg_stat_monitor` view contains all the statistics collected and aggregated by the extension. This view contains one row for each distinct combination of metrics and whether it is a top-level statement or not (up to the maximum number of distinct statements that the module can track). For details about available metrics, refer to the [`pg_stat_monitor` view reference :octicons-link-external-16:](https://docs.percona.com/pg-stat-monitor/reference.html). - -The following are the primary keys for pg_stat_monitor: - -* `bucket` -* `userid` -* `datname` -* `queryid` -* `client_ip` -* `planid` -* `application_name` - -A new row is created for each key in the `pg_stat_monitor` view. - -For security reasons, only superusers and members of the `pg_read_all_stats` role are allowed to see the SQL text, `client_ip` and `queryid` of queries executed by other users. Other users can see the statistics, however, if the view has been installed in their database. - -#### pg_stat_monitor_settings view (dropped) - -Starting with version 2.0.0, the `pg_stat_monitor_settings` view is deprecated and removed. All `pg_stat_monitor` configuration parameters are now available though the `pg_settings` view using the following query: - -```sql -SELECT name, setting, unit, context, vartype, source, min_val, max_val, enumvals, boot_val, reset_val, pending_restart FROM pg_settings WHERE name LIKE '%pg_stat_monitor%'; -``` - -For backward compatibility, you can create the `pg_stat_monitor_settings` view using the following SQL statement: - -```sql -CREATE VIEW pg_stat_monitor_settings - -AS - -SELECT * - -FROM pg_settings - -WHERE name like 'pg_stat_monitor.%'; -``` - -In `pg_stat_monitor` version 1.1.1 and earlier, the `pg_stat_monitor_settings` view shows one row per `pg_stat_monitor` configuration parameter. It displays configuration parameter name, value, default value, description, minimum and maximum values, and whether a restart is required for a change in value to be effective. - -To learn more, see the [Changing the configuration](#changing-the-configuration) section. - -## Installation - -This section describes how to install `pg_stat_monitor` from Percona repositories. To learn about other installation methods, see the [Installation :octicons-link-external-16:](https://docs.percona.com/pg-stat-monitor/install.html) section in the `pg_stat_monitor` documentation. - -**Preconditions**: - -To install `pg_stat_monitor` from Percona repositories, you need to subscribe to them. To do this, you must have the [`percona-release` repository management tool :octicons-link-external-16:](https://www.percona.com/doc/percona-repo-config/installing.html) up and running. - -To install `pg_stat_monitor`, run the following commands: - -=== ":material-debian: On Debian and Ubuntu" - - 1. Enable the repository - - ```{.bash data-prompt="$"} - $ sudo percona-release setup ppg16 - ``` - - 2. Install the package: - - ```{.bash data-prompt="$"} - $ sudo apt-get install percona-pg-stat-monitor16 - ``` - -=== ":material-redhat: On Red Hat Enterprise Linux and derivatives" - - 1. Enable the repository - - ```{.bash data-prompt="$"} - $ sudo percona-release setup ppg16 - ``` - - 2. Install the package: - - ```{.bash data-prompt="$"} - $ sudo yum install percona-pg-stat-monitor16 - ``` - -## Setup - -`pg_stat_monitor` requires additional setup in order to use it with PostgreSQL. The setup steps are the following: - - -1. Add `pg_stat_monitor` in the `shared_preload_libraries` configuration parameter. - - The recommended way to modify PostgreSQL configuration file is using the [ALTER SYSTEM :octicons-link-external-16:](https://www.postgresql.org/docs/15/sql-altersystem.html) command. [Connect to psql](connect.md) and use the following command: - - ```sql - ALTER SYSTEM SET shared_preload_libraries = 'pg_stat_monitor'; - ``` - - The parameter value is written to the `postgresql.auto.conf` file which is read in addition with `postgresql.conf` file. - - !!! note - - To use `pg_stat_monitor` together with `pg_stat_statements`, specify both modules separated by commas for the `ALTER SYSTEM SET` command. - - The order of modules is important: `pg_stat_monitor` must be specified **after** `pg_stat_statements`: - - ```sql - ALTER SYSTEM SET shared_preload_libraries = ‘pg_stat_statements, pg_stat_monitor’ - ``` - -2. Start or restart the `postgresql` instance to enable `pg_stat_monitor`. Use the following command for restart: - - - === ":material-debian: On Debian and Ubuntu" - - ```{.bash data-prompt="$"} - $ sudo systemctl restart postgresql.service - ``` - - - === ":material-redhat: On Red Hat Enterprise Linux and derivatives" - - ```{.bash data-prompt="$"} - $ sudo systemctl restart postgresql-16 - ``` - - -3. Create the extension. Connect to `psql` and use the following command: - - ```sql - CREATE EXTENSION pg_stat_monitor; - ``` - - By default, the extension is created against the `postgres` database. You need to create the extension on every database where you want to collect statistics. - -!!! tip - - To check the version of the extension, run the following command in the `psql` session: - - ```sql - SELECT pg_stat_monitor_version(); - ``` - -## Usage - -For example, to view the IP address of the client application that made the query, run the following command: - -```sql -SELECT DISTINCT userid::regrole, pg_stat_monitor.datname, substr(query,0, 50) AS query, calls, bucket, bucket_start_time, queryid, client_ip -FROM pg_stat_monitor, pg_database -WHERE pg_database.oid = oid; -``` - -Output: - -``` - userid | datname | query | calls | bucket | bucket_start_time | queryid | client_ip -----------+----------+---------------------------------------------------+-------+--------+---------------------+------------------+----------- - postgres | postgres | SELECT name,description FROM pg_stat_monitor_sett | 1 | 9 | 2022-10-24 07:29:00 | AD536A8DEA7F0C73 | 127.0.0.1 - postgres | postgres | SELECT c.oid, +| 1 | 9 | 2022-10-24 07:29:00 | 34B888E5C844519C | 127.0.0.1 - | | n.nspname, +| | | | | - | | c.relname +| | | | | - | | FROM pg_ca | | | | | - postgres | postgres | SELECT DISTINCT userid::regrole, pg_stat_monitor. | 1 | 1 | 2022-10-24 07:31:00 | 6230793895381F1D | 127.0.0.1 - postgres | postgres | SELECT pg_stat_monitor_version() | 1 | 9 | 2022-10-24 07:29:00 | B617F5F12931F388 | 127.0.0.1 - postgres | postgres | CREATE EXTENSION pg_stat_monitor | 1 | 8 | 2022-10-24 07:28:00 | 14B98AF0776BAF7B | 127.0.0.1 - postgres | postgres | SELECT a.attname, +| 1 | 9 | 2022-10-24 07:29:00 | 96F8E4B589EF148F | 127.0.0.1 - | | pg_catalog.format_type(a.attt | | | | | - postgres | postgres | SELECT c.relchecks, c.relkind, c.relhasindex, c.r | 1 | 9 | 2022-10-24 07:29:00 | CCC51D018AC96A25 | 127.0.0.1 - -``` - - -Find more usage examples in the [`pg_stat_monitor` user guide :octicons-link-external-16:](https://docs.percona.com/pg-stat-monitor/user_guide.html). - -## Changing the configuration - -Run the following query to list available configuration parameters. - -```sql -SELECT name, short_desc FROM pg_settings WHERE name LIKE '%pg_stat_monitor%'; -``` - -**Output** - -``` - name | short_desc --------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------- - pg_stat_monitor.pgsm_bucket_time | Sets the time in seconds per bucket. - pg_stat_monitor.pgsm_enable_overflow | Enable/Disable pg_stat_monitor to grow beyond shared memory into swap space. - pg_stat_monitor.pgsm_enable_pgsm_query_id | Enable/disable PGSM specific query id calculation which is very useful in comparing same query across databases and clusters.. - pg_stat_monitor.pgsm_enable_query_plan | Enable/Disable query plan monitoring. - pg_stat_monitor.pgsm_extract_comments | Enable/Disable extracting comments from queries. - pg_stat_monitor.pgsm_histogram_buckets | Sets the maximum number of histogram buckets. - pg_stat_monitor.pgsm_histogram_max | Sets the time in millisecond. - pg_stat_monitor.pgsm_histogram_min | Sets the time in millisecond. - pg_stat_monitor.pgsm_max | Sets the maximum size of shared memory in (MB) used for statement's metadata tracked by pg_stat_monitor. - pg_stat_monitor.pgsm_max_buckets | Sets the maximum number of buckets. - pg_stat_monitor.pgsm_normalized_query | Selects whether save query in normalized format. - pg_stat_monitor.pgsm_overflow_target | Sets the overflow target for pg_stat_monitor. (Deprecated, use pgsm_enable_overflow) - pg_stat_monitor.pgsm_query_max_len | Sets the maximum length of query. - pg_stat_monitor.pgsm_query_shared_buffer | Sets the maximum size of shared memory in (MB) used for query tracked by pg_stat_monitor. - pg_stat_monitor.pgsm_track | Selects which statements are tracked by pg_stat_monitor. - pg_stat_monitor.pgsm_track_planning | Selects whether planning statistics are tracked. - pg_stat_monitor.pgsm_track_utility | Selects whether utility commands are tracked. -``` - -You can change a parameter by setting a new value in the configuration file. Some parameters require server restart to apply a new value. For others, configuration reload is enough. Refer to the [configuration parameters :octicons-link-external-16:](https://docs.percona.com/pg-stat-monitor/configuration.html) of the `pg_stat_monitor` documentation for the parameters’ description, how you can change their values and if the server restart is required to apply them. - -As an example, let’s set the bucket lifetime from default 60 seconds to 40 seconds. Use the **ALTER SYSTEM** command: - -```sql -ALTER SYSTEM set pg_stat_monitor.pgsm_bucket_time = 40; -``` - -Restart the server to apply the change: - - -=== ":material-debian: On Debian and Ubuntu" - - ```{.bash data-prompt="$"} - $ sudo systemctl restart postgresql.service - ``` - -=== "On Red Hat Enterprise Linux and derivatives" - - ```{.bash data-prompt="$"} - $ sudo systemctl restart postgresql-15 - ``` - -Verify the updated parameter: - -```sql -SELECT name, setting -FROM pg_settings -WHERE name = 'pg_stat_monitor.pgsm_bucket_time'; - - name | setting - ----------------------------------+--------- - pg_stat_monitor.pgsm_bucket_time | 40 -``` - -!!! admonition "See also" - - [`pg_stat_monitor` Documentation :octicons-link-external-16:](https://docs.percona.com/pg-stat-monitor/index.html) - - - Percona Blog: - - * [pg_stat_monitor: A New Way Of Looking At PostgreSQL Metrics :octicons-link-external-16:](https://www.percona.com/blog/2021/01/19/pg_stat_monitor-a-new-way-of-looking-at-postgresql-metrics/) - * [Improve PostgreSQL Query Performance Insights with pg_stat_monitor :octicons-link-external-16:](https://www.percona.com/blog/improve-postgresql-query-performance-insights-with-pg_stat_monitor/) diff --git a/docs/pg-tde.md b/docs/pg-tde.md deleted file mode 100644 index dd78a43b3..000000000 --- a/docs/pg-tde.md +++ /dev/null @@ -1,197 +0,0 @@ -# pg_tde - -!!! note - - This is the Beta version of the extension and is not recommended for production use yet. Please use it in testing environments only. - -## Overview - -`pg_tde` stands for Transparent Data Encryption for PostgreSQL. This is an open-source extension designed to enhance PostgreSQL’s security by encrypting data files on disk. The encryption is transparent for users allowing them to access and manipulate the data and not to worry about the encryption process. - -Unlike traditional encryption methods that require significant changes to database schemas and applications, `pg_tde` seamlessly integrates with PostgreSQL, encrypting data at the table level without disrupting existing workflows. It uses the Advanced Encryption Standard (AES) encryption algorithm. - -### Key features: - -* Encryption of heap tables, including TOAST. -* Storage of encryption keys in either a Hashicorp Vault server or a local keyring file (primarily for development purposes). -* Configurable encryption settings per database: you can choose which tables to encrypt, achieving granular control over data protection. -* Replication support. -* Enhanced security through the ability to rotate principal keys used for data encryption, reducing the risk of long-term exposure to potential attacks and aiding compliance with security standards like GDPR, HIPAA, and PCI DSS. - -## Installation - -This section provides instructions how to install `pg_tde` from Percona repositories using the package manager of your operating system. For other installation methods, refer to the [`pg_tde` documentation :octicons-link-external-16:](https://percona-lab.github.io/pg_tde/main/install.html#procedure). - -=== ":material-debian: :material-debian: On Debian and Ubuntu" - - `pg_tde` packages are available for the following Linux distributions: - - * Ubuntu 20.04 (Focal Fossa) - * Ubuntu 22.04 (Jammy Jellyfish) - * Debian 10 (Buster) - * Debian 11 (Bullseye) - * Debian 12 (Bookworm) - - To install `pg_tde`, run the following commands as the root user or with the `sudo` privileges: - - 1. [Install `percona-release` :octicons-link-external-16:](https://docs.percona.com/percona-software-repositories/installing.html) repository management tool. - - 2. Enable the repository: - - ```{.bash data-prompt="$"} - $ sudo percona-release enable-only ppg-16.3 testing - ``` - - 3. Install the package: - - ```{.bash data-prompt="$"} - $ sudo apt-get install percona-postgresql-16-pg-tde - ``` - -=== ":material-redhat: On Red Hat Enterprise Linux and compatible derivatives" - - `pg_tde` packages are available for the following Linux distributions: - - * Red Hat Enterprise Linux and CentOS 7 - * Red Hat Enterprise Linux 8 and compatible derivatives - * Red Hat Enterprise Linux 9 and compatible derivatives - - To install `pg_tde`, run the following commands as the root user or with the `sudo` privileges: - - 1. Enable / disable modules: - - === "CentOS 7" - - Install the `epel-release` package: - - ```{.bash data-prompt="$"} - $ sudo yum -y install epel-release - $ sudo yum repolist - ``` - - === "RHEL8/Oracle Linux 8/Rocky Linux 8" - - Disable the ``postgresql`` and ``llvm-toolset``modules: - - ```{.bash data-prompt="$"} - $ sudo dnf module disable postgresql llvm-toolset - ``` - - 2. [Install `percona-release` :octicons-link-external-16:](https://docs.percona.com/percona-software-repositories/ installing.html) repository management tool. - - 3. Enable the repository: - - ```{.bash data-prompt="$"} - $ sudo percona-release enable-only ppg-16.3 testing - ``` - - 4. Install the package: - - ```{.bash data-prompt="$"} - $ sudo yum install percona-pg_tde_16 - ``` - -## Setup - -`pg_tde` requires additional setup steps in order to use it with PostgreSQL. -This section provides setup using the HashiCorp Vault server for storing encryption key as the recommended approach. Please see [`pg_tde` documentation :octicons-link-external-16:] for alternative configuration using a keyfile. - -The setup of the Vault server is out of scope of this document. We're assuming you have the Vault server up and running and have the following information required for the setup: - -* The secret access token to the Vault server -* The URL to access the Vault server -* (Optional) The CA file used for SSL verification - - -### Install the extension in PostgreSQL - -1. Add `pg_tde` to `shared_preload_libraries`. - - The recommended way to modify PostgreSQL configuration file is using the [ALTER SYSTEM :octicons-external-link-16:](https://www.postgresql.org/docs/15/sql-altersystem.html) command. [Connect to psql](connect.md) and use the following command: - - ```sql - ALTER SYSTEM SET shared_preload_libraries = 'pg_tde'; - ``` - -2. Start or restart the `postgresql` instance to enable `pg_tde`. Use the following command for restart: - - - === ":material-debian: On Debian and Ubuntu" - - ```{.bash data-prompt="$"} - $ sudo systemctl restart postgresql.service - ``` - - - === ":material-redhat: On Red Hat Enterprise Linux and derivatives" - - ```{.bash data-prompt="$"} - $ sudo systemctl restart postgresql-{{pgversion}} - ``` - -3. Install the extension in your PostgreSQL using the CREATE EXTENSION command. [CREATE EXTENSION](https://www.postgresql.org/docs/current/sql-createextension.html) command. You must have the privileges of a superuser or a database owner to use this command. Connect to `psql` as a superuser for a database and run the following command: - - ```sql - CREATE EXTENSION pg_tde; - ``` - - By default, the `pg_tde` extension is created for the currently used database. To enable data encryption in other databases, you must explicitly run the `CREATE EXTENSION` command against them. - - !!! tip - - You can have the `pg_tde` extension automatically enabled for every newly created database. Modify the template `template1` database as follows: - - ``` - psql -d template1 -c 'CREATE EXTENSION pg_tde;' - ``` - - After you enabled `pg_tde`, the [access method :octicons-external-link-16:](https://www.postgresql.org/docs/current/tableam.html) `pg_tde` is created for that database. - -### Key configuration - -1. Set up a key provider for for the database where you have enabled the extension - - ```sql - SELECT pg_tde_add_key_provider_vault_v2('provider-name',:'secret_token','url','mount','ca_path'); - ``` - - where: - - * `url` is the URL of the Vault server - * `mount` is the mount point where the keyring should store the keys - * `secret_token` is an access token with read and write access to the above mount point - * [optional] `ca_path` is the path of the CA file used for SSL verification - -2. Add a principal key - - ```sql - SELECT pg_tde_set_principal_key('name-of-the-principal-key', 'provider-name'); - ``` - -## Usage - -To check if the data is encrypted, do the following: - -1. Create a table for the database where you have enabled `pg_tde` using the `pg_tde` access method: - - ```sql - CREATE TABLE my_encrypted_table ( - id SERIAL PRIMARY KEY, - sensitive_data TEXT - ) USING pg_tde; - ``` - -2. Insert some data ito it: - - ```sql - INSERT INTO my_encrypted_table (sensitive_data) - VALUES ('Sensitive data 1'), ('Sensitive data 2'), ('Sensitive data 3'); - ``` - -3. Check if the data is encrypted: - - ```sql - SELECT pg_tde_is_encrypted('my_encrypted_table'); - ``` - - The function returns `t` if the table is encrypted and `f` - if not. \ No newline at end of file diff --git a/docs/release-notes.md b/docs/release-notes.md deleted file mode 100644 index 7fae13f6f..000000000 --- a/docs/release-notes.md +++ /dev/null @@ -1,15 +0,0 @@ -# Percona Distribution for PostgreSQL release notes - -* [Percona Distribution for PostgreSQL 16.4](release-notes-v16.4.md) ({{date.16_4}}) - -* [Percona Distribution for PostgreSQL 16.3](release-notes-v16.3.md) (2024-06-06) - -* [Percona Distribution for PostgreSQL 16.2](release-notes-v16.2.md) (2024-02-27) - -* [Percona Distribution for PostgreSQL 16.1 Update](release-notes-v16.1.upd.md) (2024-01-18) - -* [Percona Distribution for PostgreSQL 16.1](release-notes-v16.1.md) (2023-11-29) - -* [Percona Distribution for PostgreSQL 16.0 Update](release-notes-v16.0.upd.md) (2023-11-02) - -* [Percona Distribution for PostgreSQL 16](release-notes-v16.0.md) (2023-09-19) diff --git a/docs/release-notes-v16.0.md b/docs/release-notes/release-notes-v16.0.md similarity index 98% rename from docs/release-notes-v16.0.md rename to docs/release-notes/release-notes-v16.0.md index 844c5a685..8e0889a3e 100644 --- a/docs/release-notes-v16.0.md +++ b/docs/release-notes/release-notes-v16.0.md @@ -1,7 +1,7 @@ # Percona Distribution for PostgreSQL 16.0 (2023-09-19) -[Installation](installing.md){.md-button} -[Upgrade](major-upgrade.md){.md-button} +[Installation](../installing.md){.md-button} +[Upgrade](../major-upgrade.md){.md-button} We are pleased to announce the launch of Percona Distribution for PostgreSQL 16.0 - a solution with the collection of tools from PostgreSQL community that are tested to work together and serve to assist you in deploying and managing PostgreSQL. The aim of Percona Distribution for PostgreSQL is to address the operational issues like High-Availability, Disaster Recovery, Security, Observability, Spatial data handling, Performance and Scalability and others that enterprises are facing. diff --git a/docs/release-notes-v16.0.upd.md b/docs/release-notes/release-notes-v16.0.upd.md similarity index 57% rename from docs/release-notes-v16.0.upd.md rename to docs/release-notes/release-notes-v16.0.upd.md index 61f2b8415..80e60b0a1 100644 --- a/docs/release-notes-v16.0.upd.md +++ b/docs/release-notes/release-notes-v16.0.upd.md @@ -1,7 +1,6 @@ # Percona Distribution for PostgreSQL 16.0 Update (2023-11-02) -[Installation](installing.md){.md-button} -[Upgrade](major-upgrade.md){.md-button} +[Installation](../installing.md){.md-button} +[Upgrade](../major-upgrade.md){.md-button} - -This update to the release of Percona Distribution for PostgreSQL 16.0 includes the Docker images for x86_64 architectures. It aims to simplify the developers' experience with the Distribution. Refer to the [Docker guide](docker.md) for how to run Percona Distribution for PostgreSQL in Docker. \ No newline at end of file +This update to the release of Percona Distribution for PostgreSQL 16.0 includes the Docker images for x86_64 architectures. It aims to simplify the developers' experience with the Distribution. Refer to the [Docker guide](../docker.md) for how to run Percona Distribution for PostgreSQL in Docker. diff --git a/docs/release-notes-v16.1.md b/docs/release-notes/release-notes-v16.1.md similarity index 95% rename from docs/release-notes-v16.1.md rename to docs/release-notes/release-notes-v16.1.md index 900f15e4d..a147948cd 100644 --- a/docs/release-notes-v16.1.md +++ b/docs/release-notes/release-notes-v16.1.md @@ -1,6 +1,6 @@ # Percona Distribution for PostgreSQL 16.1 (2023-11-29) -[Installation](installing.md){.md-button} +[Installation](../installing.md){.md-button} --8<-- "release-notes-intro.md" @@ -8,7 +8,7 @@ This release of Percona Distribution for PostgreSQL is based on [PostgreSQL 16.1 ## Release Highlights -* Telemetry is now enabled in Percona Distribution for PostgreSQL to fill in the gaps in our understanding of how you use it and help us improve our products. Participation in the anonymous program is optional. You can opt-out if you prefer not to share this information. Find more information in the [Telemetry on Percona Distribution for PostgreSQL](telemetry.md) document. +* Telemetry is now enabled in Percona Distribution for PostgreSQL to fill in the gaps in our understanding of how you use it and help us improve our products. Participation in the anonymous program is optional. You can opt-out if you prefer not to share this information. Find more information in the [Telemetry on Percona Distribution for PostgreSQL](../telemetry.md) document. * The `percona-postgis33` and `percona-pgaudit` packages on YUM-based operating systems are renamed `percona-postgis33_{{pgversion}}` and `percona-pgaudit{{pgversion}}` respectively @@ -41,8 +41,6 @@ Percona Distribution for PostgreSQL also includes the following packages: | RHEL 8 | `etcd` | 3.3.11 | A consistent, distributed key-value store| | | `python3-etcd`| 0.4.5 | A Python client for etcd | - - Percona Distribution for PostgreSQL is also shipped with the [libpq :octicons-link-external-16:](https://www.postgresql.org/docs/16/libpq.html) library. It contains "a set of library functions that allow client programs to pass queries to the PostgreSQL -backend server and to receive the results of these queries." +backend server and to receive the results of these queries." diff --git a/docs/release-notes-v16.1.upd.md b/docs/release-notes/release-notes-v16.1.upd.md similarity index 90% rename from docs/release-notes-v16.1.upd.md rename to docs/release-notes/release-notes-v16.1.upd.md index 6f6788c68..7bddf0ef2 100644 --- a/docs/release-notes-v16.1.upd.md +++ b/docs/release-notes/release-notes-v16.1.upd.md @@ -1,6 +1,6 @@ # Percona Distribution for PostgreSQL 16.1 Update (2024-01-18) -[Installation](installing.md){.md-button} +[Installation](../installing.md){.md-button} --8<-- "release-notes-intro.md" diff --git a/docs/release-notes-v16.2.md b/docs/release-notes/release-notes-v16.2.md similarity index 96% rename from docs/release-notes-v16.2.md rename to docs/release-notes/release-notes-v16.2.md index d8ce16b83..152ade38f 100644 --- a/docs/release-notes-v16.2.md +++ b/docs/release-notes/release-notes-v16.2.md @@ -1,6 +1,6 @@ # Percona Distribution for PostgreSQL 16.2 (2024-02-27) -[Installation](installing.md){.md-button} +[Installation](../installing.md){.md-button} --8<-- "release-notes-intro.md" @@ -40,8 +40,6 @@ Percona Distribution for PostgreSQL also includes the following packages: | RHEL 8 and derivatives| `etcd` | 3.5.12 | A consistent, distributed key-value store| | | `python3-etcd`| 0.4.5 | A Python client for etcd | - - Percona Distribution for PostgreSQL is also shipped with the [libpq :octicons-link-external-16:](https://www.postgresql.org/docs/16/libpq.html) library. It contains "a set of library functions that allow client programs to pass queries to the PostgreSQL -backend server and to receive the results of these queries." +backend server and to receive the results of these queries." diff --git a/docs/release-notes-v16.3.md b/docs/release-notes/release-notes-v16.3.md similarity index 96% rename from docs/release-notes-v16.3.md rename to docs/release-notes/release-notes-v16.3.md index 790949c67..deff6a4bb 100644 --- a/docs/release-notes-v16.3.md +++ b/docs/release-notes/release-notes-v16.3.md @@ -1,6 +1,6 @@ # Percona Distribution for PostgreSQL 16.3 (2024-06-06) -[Installation](installing.md){.md-button} +[Installation](../installing.md){.md-button} --8<-- "release-notes-intro.md" @@ -41,8 +41,6 @@ Percona Distribution for PostgreSQL Red Hat Enterprise Linux 8 and compatible de * `llvm` 16.0.6 packages. This fixes compatibility issues with LLVM from upstream. * supplemental `python3-etcd` packages, which can be used for setting up Patroni clusters. - - Percona Distribution for PostgreSQL is also shipped with the [libpq](https://www.postgresql.org/docs/16/libpq.html) library. It contains "a set of library functions that allow client programs to pass queries to the PostgreSQL -backend server and to receive the results of these queries." +backend server and to receive the results of these queries." diff --git a/docs/release-notes-v16.4.md b/docs/release-notes/release-notes-v16.4.md similarity index 98% rename from docs/release-notes-v16.4.md rename to docs/release-notes/release-notes-v16.4.md index a8de8257e..09756d608 100644 --- a/docs/release-notes-v16.4.md +++ b/docs/release-notes/release-notes-v16.4.md @@ -1,6 +1,6 @@ # Percona Distribution for PostgreSQL 16.4 ({{date.16_4}}) -[Installation](installing.md){.md-button} +[Installation](../installing.md){.md-button} --8<-- "release-notes-intro.md" @@ -20,7 +20,7 @@ This release of Percona Distribution for PostgreSQL is based on [PostgreSQL 16.4 * Debian 11 * Debian 12 -* Percona Distribution for PostgreSQL includes the enhanced telemetry feature and provides comprehensive information about how telemetry works, its components and metrics as well as updated methods how to disable telemetry. Read more in [Telemetry and data collection](telemetry.md) +* Percona Distribution for PostgreSQL includes the enhanced telemetry feature and provides comprehensive information about how telemetry works, its components and metrics as well as updated methods how to disable telemetry. Read more in [Telemetry and data collection](../telemetry.md) * Percona Distribution for PostgreSQL includes `pg_stat_monitor` 2.1.0 that provides the ability to [disable the application name tracking for a query](https://docs.percona.com/pg-stat-monitor/configuration.html#pg_stat_monitorpgsm_track_application_names). This way you can optimize query execution performance. diff --git a/docs/release-notes/release-notes-v16.6.md b/docs/release-notes/release-notes-v16.6.md new file mode 100644 index 000000000..c94d9a3f9 --- /dev/null +++ b/docs/release-notes/release-notes-v16.6.md @@ -0,0 +1,46 @@ +# Percona Distribution for PostgreSQL 16.6 ({{date.16_6}}) + +[Installation](../installing.md){.md-button} + +--8<-- "release-notes-intro.md" + +This release of Percona Distribution for PostgreSQL is based on [PostgreSQL 16.6](https://www.postgresql.org/docs/16/release-16-6.html). + +## Release Highlights + +* This release includes fixes for [CVE-2024-10978](https://www.postgresql.org/support/security/CVE-2024-10978/) and for certain PostgreSQL extensions that break because they depend on the modified Application Binary Interface (ABI). These regressions were introduced in PostgreSQL 17.1, 16.5, 15.9, 14.14, 13.17, and 12.21. For this reason, the release of Percona Distribution for PostgreSQL 16.5 has been skipped. + +* Percona Distribution for PostgreSQL includes [`pgvector` :octicons-link-external-16:](https://github.com/pgvector/pgvector) - an open source extension that enables you to use PostgreSQL as a vector database. It brings vector data type and vector operations (mainly similarity search) to PosgreSQL. You can install `pgvector` from repositories, tarballs, and it is also available as a Docker image. + +* Percona Distribution for PostgreSQL now statically links `llvmjit.so` library for Red Hat Enterprise Linux 8 and 9 and compatible derivatives. This resolves the conflict between the LLVM version required by Percona Distribution for PostgreSQL and the one supplied with the operating system. This also enables you to use the LLVM modules supplied with the operating system for other software you require. + +## Supplied third-party extensions + +Review each extension’s release notes for What’s new, improvements, or bug fixes. The following is the list of extensions available in Percona Distribution for PostgreSQL. + +The following is the list of extensions available in Percona Distribution for PostgreSQL. + +| Extension | Version | Description | +| ------------------- | -------------- | ---------------------------- | +| [etcd](https://etcd.io/)| 3.5.16 | A distributed, reliable key-value store for setting up high available Patroni clusters | +| [HAProxy](http://www.haproxy.org/) | 2.8.11 | a high-availability and load-balancing solution | +| [Patroni](https://patroni.readthedocs.io/en/latest/) | 4.0.3 | a HA (High Availability) solution for PostgreSQL | +| [PgAudit](https://www.pgaudit.org/) | 16 | provides detailed session or object audit logging via the standard logging facility provided by PostgreSQL | +| [pgAudit set_user](https://github.com/pgaudit/set_user)| 4.1.0 | provides an additional layer of logging and control when unprivileged users must escalate themselves to superusers or object owner roles in order to perform needed maintenance tasks.| +| [pgBackRest](https://pgbackrest.org/) | 2.54.0 | a backup and restore solution for PostgreSQL | +|[pgBadger](https://github.com/darold/pgbadger) | 12.4 | a fast PostgreSQL Log Analyzer.| +|[PgBouncer](https://www.pgbouncer.org/) |1.23.1 | a lightweight connection pooler for PostgreSQL| +| [pg_gather](https://github.com/jobinau/pg_gather)| v28 | an SQL script for running the diagnostics of the health of PostgreSQL cluster | +| [pgpool2](https://git.postgresql.org/gitweb/?p=pgpool2.git;a=summary) | 4.5.4 | a middleware between PostgreSQL server and client for high availability, connection pooling and load balancing.| +| [pg_repack](https://github.com/reorg/pg_repack) | 1.5.1 | rebuilds PostgreSQL database objects | +| [pg_stat_monitor](https://github.com/percona/pg_stat_monitor)|{{pgsmversion}} | collects and aggregates statistics for PostgreSQL and provides histogram information.| +| [pgvector](https://github.com/pgvector/pgvector)| v0.8.0 | A vector similarity search for PostgreSQL| +| [PostGIS](https://github.com/postgis/postgis) | 3.3.7 | a spatial extension for PostgreSQL.| +| [PostgreSQL Commons](https://salsa.debian.org/postgresql/postgresql-common)| 266 | PostgreSQL database-cluster manager. It provides a structure under which multiple versions of PostgreSQL may be installed and/or multiple clusters maintained at one time.| +| [wal2json](https://github.com/eulerto/wal2json) |2.6 | a PostgreSQL logical decoding JSON output plugin| + +For Red Hat Enterprise Linux 8 and 9 and compatible derivatives, Percona Distribution for PostgreSQL also includes the supplemental `python3-etcd` 0.4.5 packages, which are used for setting up Patroni clusters. + +Percona Distribution for PostgreSQL is also shipped with the [libpq](https://www.postgresql.org/docs/16/libpq.html) library. It contains "a set of +library functions that allow client programs to pass queries to the PostgreSQL +backend server and to receive the results of these queries." diff --git a/docs/release-notes/release-notes-v16.8.md b/docs/release-notes/release-notes-v16.8.md new file mode 100644 index 000000000..2c2a27c95 --- /dev/null +++ b/docs/release-notes/release-notes-v16.8.md @@ -0,0 +1,56 @@ +# Percona Distribution for PostgreSQL 16.8 ({{date.16_8}}) + +[Installation](../installing.md){.md-button} + +--8<-- "release-notes-intro.md" + +This release of Percona Distribution for PostgreSQL is based on [PostgreSQL 16.7](https://www.postgresql.org/docs/16/release-16-7.html) and PostgreSQL 16.8. + +## Release Highlights + +This release fixes [CVE-2025-1094](https://www.postgresql.org/support/security/CVE-2025-1094/), which closed a vulnerability in the `libpq` PostgreSQL client library but introduced a regression related to string handling for non-null terminated strings. The error would be visible based on how a PostgreSQL client implemented this behavior. This regression affects versions 17.3, 16.7, 15.11, 14.16, and 13.19. For this reason, version 16.7 was skipped. + +### Improved security and user experience for Docker images + +* Percona Distribution for PostgreSQL Docker image is now based on Universal Base Image (UBI) version 9, which includes the latest security fixes. This makes the image compliant with the Red Hat certification and ensures the seamless work of containers on Red Hat OpenShift Container Platform. + +* You no longer have to specify the `{{dockertag}}-multi` tag when you run Percona Distribution for PostgreSQL in Docker. Instead, use the `percona/percona-distribution-postgresql:{{dockertag}}`. Docker automatically identifies the architecture of your operating system and pulls the corresponding image. Refer to [Run in Docker](../docker.md) for how to get started. + +### PostGIS is included into tarballs + +We have extended Percona Distribution for PostgreSQL tarballs with PostGIS - an open-source extension to handle spacial data. This way you can install and run PostgreSQL as a geospatial database on hosts without a direct access to the Internet. Learn more about [installing from tarballs](../tarball.md) and [Spacial data manipulation](../solutions/postgis.md) + +### Deprecation of meta packages + +[Meta-packages for Percona Distribution for PostgreSQL](../repo-overview.md#repository-contents) are deprecated and will be removed in future releases. + +## Supplied third-party extensions + +Review each extension’s release notes for What’s new, improvements, or bug fixes. The following is the list of extensions available in Percona Distribution for PostgreSQL. + +The following is the list of extensions available in Percona Distribution for PostgreSQL. + +| Extension | Version | Description | +| ------------------- | -------------- | ---------------------------- | +| [etcd](https://etcd.io/)| 3.5.18 | A distributed, reliable key-value store for setting up high available Patroni clusters | +| [HAProxy](http://www.haproxy.org/) | 2.8.13 | a high-availability and load-balancing solution | +| [Patroni](https://patroni.readthedocs.io/en/latest/) | 4.0.4 | a HA (High Availability) solution for PostgreSQL | +| [PgAudit](https://www.pgaudit.org/) | 16 | provides detailed session or object audit logging via the standard logging facility provided by PostgreSQL | +| [pgAudit set_user](https://github.com/pgaudit/set_user)| 4.1.0 | provides an additional layer of logging and control when unprivileged users must escalate themselves to superusers or object owner roles in order to perform needed maintenance tasks.| +| [pgBackRest](https://pgbackrest.org/) | 2.54.2 | a backup and restore solution for PostgreSQL | +| [pgBadger](https://github.com/darold/pgbadger) | 13.0 | a fast PostgreSQL Log Analyzer. | +| [PgBouncer](https://www.pgbouncer.org/) | 1.24.0 | a lightweight connection pooler for PostgreSQL | +| [pg_gather](https://github.com/jobinau/pg_gather) | v29 | an SQL script for running the diagnostics of the health of PostgreSQL cluster | +| [pgpool2](https://git.postgresql.org/gitweb/?p=pgpool2.git;a=summary) | 4.5.5 | a middleware between PostgreSQL server and client for high availability, connection pooling, and load balancing. | +| [pg_repack](https://github.com/reorg/pg_repack) | 1.5.2 | rebuilds PostgreSQL database objects | +| [pg_stat_monitor](https://github.com/percona/pg_stat_monitor) | {{pgsmversion}} | collects and aggregates statistics for PostgreSQL and provides histogram information. | +| [pgvector](https://github.com/pgvector/pgvector) | v0.8.0 | A vector similarity search for PostgreSQL | +| [PostGIS](https://github.com/postgis/postgis) | 3.3.8 | a spatial extension for PostgreSQL. | +| [PostgreSQL Commons](https://salsa.debian.org/postgresql/postgresql-common) | 267 | PostgreSQL database-cluster manager. It provides a structure under which multiple versions of PostgreSQL may be installed and/or multiple clusters maintained at one time. | +| [wal2json](https://github.com/eulerto/wal2json) | 2.6 | a PostgreSQL logical decoding JSON output plugin | + +For Red Hat Enterprise Linux 8 and compatible derivatives, Percona Distribution for PostgreSQL also includes the supplemental `python3-etcd` 0.4.5 packages, which are used for setting up Patroni clusters. + +Percona Distribution for PostgreSQL is also shipped with the [libpq](https://www.postgresql.org/docs/16/libpq.html) library. It contains "a set of +library functions that allow client programs to pass queries to the PostgreSQL +backend server and to receive the results of these queries." diff --git a/docs/release-notes/release-notes-v16.9.md b/docs/release-notes/release-notes-v16.9.md new file mode 100644 index 000000000..65ed3d161 --- /dev/null +++ b/docs/release-notes/release-notes-v16.9.md @@ -0,0 +1,44 @@ +# Percona Distribution for PostgreSQL 16.9 ({{date.16_9}}) + +[Installation](../installing.md){.md-button} + +--8<-- "release-notes-intro.md" + +This release of Percona Distribution for PostgreSQL is based on [PostgreSQL 16.9](https://www.postgresql.org/docs/16/release-16-9.html). + +## Release Highlights + +### Updated Major upgrade topic in documentation + +The [Upgrading Percona Distribution for PostgreSQL from 15 to 16](../major-upgrade.md) guide has been updated with revised steps for the [On Debian and Ubuntu using `apt`](../major-upgrade.md/#on-debian-and-ubuntu-using-apt) section, improving clarity and reliability of the upgrade process. + +## Supplied third-party extensions + +Review each extension’s release notes for What’s new, improvements, or bug fixes. + +The following is the list of extensions available in Percona Distribution for PostgreSQL. + +| Extension | Version | Description | +| ------------------- | -------------- | ---------------------------- | +| [etcd](https://etcd.io/)| 3.5.21 | A distributed, reliable key-value store for setting up high available Patroni clusters | +| [HAProxy](http://www.haproxy.org/) | 2.8.15 | a high-availability and load-balancing solution | +| [Patroni](https://patroni.readthedocs.io/en/latest/) | 4.0.5 | a HA (High Availability) solution for PostgreSQL | +| [PgAudit](https://www.pgaudit.org/) | 16.0 | provides detailed session or object audit logging via the standard logging facility provided by PostgreSQL | +| [pgAudit set_user](https://github.com/pgaudit/set_user)| 4.1.0 | provides an additional layer of logging and control when unprivileged users must escalate themselves to superusers or object owner roles in order to perform needed maintenance tasks.| +| [pgBackRest](https://pgbackrest.org/) | 2.55.0 | a backup and restore solution for PostgreSQL | +| [pgBadger](https://github.com/darold/pgbadger) | 13.1 | a fast PostgreSQL Log Analyzer. | +| [PgBouncer](https://www.pgbouncer.org/) | 1.24.1 | a lightweight connection pooler for PostgreSQL | +| [pg_gather](https://github.com/jobinau/pg_gather) | v30 | an SQL script for running the diagnostics of the health of PostgreSQL cluster | +| [pgpool2](https://git.postgresql.org/gitweb/?p=pgpool2.git;a=summary) | 4.6.0 | a middleware between PostgreSQL server and client for high availability, connection pooling, and load balancing. | +| [pg_repack](https://github.com/reorg/pg_repack) | 1.5.2 | rebuilds PostgreSQL database objects | +| [pg_stat_monitor](https://github.com/percona/pg_stat_monitor) | 2.1.1 | collects and aggregates statistics for PostgreSQL and provides histogram information. | +| [pgvector](https://github.com/pgvector/pgvector) | v0.8.0 | A vector similarity search for PostgreSQL | +| [PostGIS](https://github.com/postgis/postgis) | 3.3.8 | a spatial extension for PostgreSQL. | +| [PostgreSQL Commons](https://salsa.debian.org/postgresql/postgresql-common) | 277 | PostgreSQL database-cluster manager. It provides a structure under which multiple versions of PostgreSQL may be installed and/or multiple clusters maintained at one time. | +| [wal2json](https://github.com/eulerto/wal2json) | 2.6 | a PostgreSQL logical decoding JSON output plugin | + +For Red Hat Enterprise Linux 8 and compatible derivatives, Percona Distribution for PostgreSQL also includes the supplemental `python3-etcd` 0.4.5 packages, which are used for setting up Patroni clusters. + +Percona Distribution for PostgreSQL is also shipped with the [libpq](https://www.postgresql.org/docs/16/libpq.html) library. It contains "a set of +library functions that allow client programs to pass queries to the PostgreSQL +backend server and to receive the results of these queries." diff --git a/docs/release-notes/release-notes-v16.9.upd.md b/docs/release-notes/release-notes-v16.9.upd.md new file mode 100644 index 000000000..b44578904 --- /dev/null +++ b/docs/release-notes/release-notes-v16.9.upd.md @@ -0,0 +1,7 @@ +# Percona Distribution for PostgreSQL 16.9 Update ({{date.16_9_1}}) + +[Installation](../installing.md){.md-button} + +--8<-- "release-notes-intro.md" + +This update of Percona Distribution for PostgreSQL includes the new version of [`pg_stat_monitor` 2.2.0 :octicons-link-external-16:](https://docs.percona.com/pg-stat-monitor/release-notes/2.2.0.html) that improves query annotation parsing, enhances SQL error visibility, and fixes diagnostic issues with command types, improving performance. diff --git a/docs/release-notes/release-notes.md b/docs/release-notes/release-notes.md new file mode 100644 index 000000000..bfffa6b66 --- /dev/null +++ b/docs/release-notes/release-notes.md @@ -0,0 +1,31 @@ +# Percona Distribution for PostgreSQL release notes + +This page lists all release notes for Percona Distribution for PostgreSQL 16, organized by year and version. Use it to track new features, fixes, and updates across major and minor versions. + +## 2025 + +* [16.9 Update](release-notes-v16.9.upd.md) ({{date.16_9_1}}) + +* [16.9](release-notes-v16.9.md) ({{date.16_9}}) + +* [16.8](release-notes-v16.8.md) ({{date.16_8}}) + +## 2024 + +* [16.6](release-notes-v16.6.md) ({{date.16_6}}) + +* [16.4](release-notes-v16.4.md) ({{date.16_4}}) + +* [16.3](release-notes-v16.3.md) (2024-06-06) + +* [16.2](release-notes-v16.2.md) (2024-02-27) + +* [16.1 Update](release-notes-v16.1.upd.md) (2024-01-18) + +## 2023 + +* [16.1](release-notes-v16.1.md) (2023-11-29) + +* [16.0 Update](release-notes-v16.0.upd.md) (2023-11-02) + +* [16](release-notes-v16.0.md) (2023-09-19) diff --git a/docs/repo-overview.md b/docs/repo-overview.md index cd7abae19..c9127039e 100644 --- a/docs/repo-overview.md +++ b/docs/repo-overview.md @@ -12,6 +12,10 @@ Percona Distribution for PostgreSQL provides individual packages for its compone Using a meta-package, you can install all components it contains in one go. +!!! note + + Meta packages are deprecated and will be removed in future releases. + ### `percona-ppg-server` === "Package name on Debian/Ubuntu" diff --git a/docs/solutions/backup-recovery.md b/docs/solutions/backup-recovery.md index 57a1da194..edfcd3165 100644 --- a/docs/solutions/backup-recovery.md +++ b/docs/solutions/backup-recovery.md @@ -21,9 +21,9 @@ A Disaster Recovery (DR) solution ensures that a system can be quickly restored
PostgreSQL offers multiple options for setting up database disaster recovery. - - **[pg_dump :octicons-link-external-16:](https://www.postgresql.org/docs/15/app-pgdump.html) or the [pg_dumpall :octicons-link-external-16:](https://www.postgresql.org/docs/15/app-pg-dumpall.html) utilities** + - **[pg_dump :octicons-link-external-16:](https://www.postgresql.org/docs/{{pgversion}}/app-pgdump.html) or the [pg_dumpall :octicons-link-external-16:](https://www.postgresql.org/docs/{{pgversion}}/app-pg-dumpall.html) utilities** - This is the basic backup approach. These tools can generate the backup of one or more PostgreSQL databases (either just the structure, or both the structure and data), then restore them through the [pg_restore :octicons-link-external-16:](https://www.postgresql.org/docs/15/app-pgrestore.html) command. + This is the basic backup approach. These tools can generate the backup of one or more PostgreSQL databases (either just the structure, or both the structure and data), then restore them through the [pg_restore :octicons-link-external-16:](https://www.postgresql.org/docs/{{pgversion}}/app-pgrestore.html) command. | Advantages | Disadvantages | | ------------ | --------------- | @@ -37,7 +37,7 @@ A Disaster Recovery (DR) solution ensures that a system can be quickly restored | ------------ | --------------- | | Consistent snapshot of the data directory or the whole data disk volume | 1. Requires stopping PostgreSQL in order to copy the files. This is not practical for most production setups.
2. No backup of individual databases or tables.| - - **PostgreSQL [pg_basebackup :octicons-link-external-16:](https://www.postgresql.org/docs/15/app-pgbasebackup.html)** + - **PostgreSQL [pg_basebackup :octicons-link-external-16:](https://www.postgresql.org/docs/{{pgversion}}/app-pgbasebackup.html)** This backup tool is provided by PostgreSQL. It is used to back up data when the database instance is running. `pgasebackup` makes a binary copy of the database cluster files, while making sure the system is put in and out of backup mode automatically. diff --git a/docs/solutions/dr-pgbackrest-setup.md b/docs/solutions/dr-pgbackrest-setup.md index 1e89747e4..5146f9942 100644 --- a/docs/solutions/dr-pgbackrest-setup.md +++ b/docs/solutions/dr-pgbackrest-setup.md @@ -239,7 +239,7 @@ log-level-console=info log-level-file=debug [prod_backup] -pg1-path=/var/lib/postgresql/14/main +pg1-path=/var/lib/postgresql/{{pgversion}}/main ``` diff --git a/docs/solutions/etcd-info.md b/docs/solutions/etcd-info.md new file mode 100644 index 000000000..dd1ddb993 --- /dev/null +++ b/docs/solutions/etcd-info.md @@ -0,0 +1,67 @@ +# ETCD + +`etcd` is one of the key components in high availability architecture, therefore, it's important to understand it. + +`etcd` is a distributed key-value consensus store that helps applications store and manage cluster configuration data and perform distributed coordination of a PostgreSQL cluster. + +`etcd` runs as a cluster of nodes that communicate with each other to maintain a consistent state. The primary node in the cluster is called the "leader", and the remaining nodes are the "followers". + +## How `etcd` works + +Each node in the cluster stores data in a structured format and keeps a copy of the same data to ensure redundancy and fault tolerance. When you write data to `etcd`, the change is sent to the leader node, which then replicates it to the other nodes in the cluster. This ensures that all nodes remain synchronized and maintain data consistency. + +When a client wants to change data, it sends the request to the leader. The leader accepts the writes and proposes this change to the followers. The followers vote on the proposal. If a majority of followers agree (including the leader), the change is committed, ensuring consistency. The leader then confirms the change to the client. + +This flow corresponds to the Raft consensus algorithm, based on which `etcd` works. Read morea bout it the [`ectd` Raft consensus](#etcd-raft-consensus) section. + +## Leader election + +An `etcd` cluster can have only one leader node at a time. The leader is responsible for receiving client requests, proposing changes, and ensuring they are replicated to the followers. When an `etcd` cluster starts, or if the current leader fails, the nodes hold an election to choose a new leader. Each node waits for a random amount of time before sending a vote request to other nodes, and the first node to get a majority of votes becomes the new leader. The cluster remains available as long as a majority of nodes (quorum) are still running. + +### How many members to have in a cluster + +The recommended approach is to deploy an odd-sized cluster (e.g., 3, 5, or 7 nodes). The odd number of nodes ensures that there is always a majority of nodes available to make decisions and keep the cluster running smoothly. This majority is crucial for maintaining consistency and availability, even if one node fails. For a cluster with `n` members, the majority is `(n/2)+1`. + +To better illustrate this concept, take an example of clusters with 3 nodes and 4 nodes. In a 3-node cluster, if one node fails, the remaining 2 nodes still form a majority (2 out of 3), and the cluster can continue to operate. In a 4-node cluster, if one node fails, there are only 3 nodes left, which is not enough to form a majority (3 out of 4). The cluster stops functioning. + +## `etcd` Raft consensus + +The heart of `etcd`'s reliability is the Raft consensus algorithm. Raft ensures that all nodes in the cluster agree on the same data. This ensures a consistent view of the data, even if some nodes are unavailable or experiencing network issues. + +An example of the Raft's role in `etcd` is the situation when there is no majority in the cluster. If a majority of nodes can't communicate (for example, due to network partitions), no new leader can be elected, and no new changes can be committed. This prevents the system from getting into an inconsistent state. The system waits for the network to heal and a majority to be re-established. This is crucial for data integrity. + +You can also check [this resource :octicons-link-external-16:](https://thesecretlivesofdata.com/raft/) to learn more about Raft and understand it better. + +## `etcd` logs and performance considerations + +`etcd` keeps a detailed log of every change made to the data. These logs are essential for several reasons, including the ensurance of consistency, fault tolerance, leader elections, auditing, and others, maintaining a consistent state across nodes. For example, if a node fails, it can use the logs to catch up with the other nodes and restore its data. The logs also provide a history of all changes, which can be useful for debugging and security analysis if needed. + +### Slow disk performance + +`etcd` is very sensitive to disk I/O performance. Writing to the logs is a frequent operation and will be slow if the disk is slow. This can lead to timeouts, delaying consensus, instability, and even data loss. In extreme cases, slow disk performance can cause a leader to fail health checks, triggering unnecessary leader elections. Always use fast, reliable storage for `etcd`. + +### Slow or high-latency networks + +Communication between `etcd` nodes is critical. A slow or unreliable network can cause delays in replicating data, increasing the risk of stale reads. This can trigger premature timeouts leading to leader elections happening more frequently, and even delays in leader elections in some cases, impacting performance and stability. Also keep in mind that if nodes cannot reach each other in a timely manner, the cluster may lose quorum and become unavailable. + +## etcd Locks + +`etcd` provides a distributed locking mechanism, which helps applications coordinate actions across multiple nodes and access to shared resources preventing conflicts. Locks ensure that only one process can hold a resource at a time, avoiding race conditions and inconsistencies. Patroni is an example of an application that uses `etcd` locks for primary election control in the PostgreSQL cluster. + +### Deployment considerations + +Running `etcd` on separate hosts has the following benefits: + +* Both PostgreSQL and `etcd` are highly dependant on I/O. And running them on the separate hosts improves performance. + +* Higher resilience. If one or even two PostgreSQL node crash, the `etcd` cluster remains healthy and can trigger a new primary election. + +* Scalability and better performance. You can scale the `etcd` cluster separately from PostgreSQL based on the load and thus achieve better performance. + +Note that separate deployment increases the complexity of the infrastructure and requires additional effort on maintenance. Also, pay close attention to network configuration to eliminate the latency that might occur due to the communication between `etcd` and Patroni nodes over the network. + +If a separate dedicated host for 1 is not a viable option, you can use the same host machines used for Patroni and PostgreSQL. + +## Next steps + +[Patroni](patroni-info.md){.md-button} \ No newline at end of file diff --git a/docs/solutions/ha-architecture.md b/docs/solutions/ha-architecture.md new file mode 100644 index 000000000..c3a9c743c --- /dev/null +++ b/docs/solutions/ha-architecture.md @@ -0,0 +1,60 @@ +# Architecture + +In the [overview of high availability](high-availability.md), we discussed the required components to achieve high-availability. + +Our recommended minimalistic approach to a highly-available deployment is to have a three-node PostgreSQL cluster with the cluster management and failover mechanisms, load balancer and a backup / restore solution. + +The following diagram shows this architecture, including all additional components. If you are considering a simple and cost-effective setup, refer to the [Bare-minimum architecture](#bare-minimum-architecture) section. + +![Architecture of the three-node, single primary PostgreSQL cluster](../_images/diagrams/ha-recommended.svg) + +## Components + +The components in this architecture are: + +### Database layer + +- PostgreSQL nodes bearing the user data. + +- [Patroni](patroni-info.md) - an automatic failover system. Patroni requires and uses the Distributed Configuration Store to store the cluster configuration, health and status. + +- watchdog - a mechanism that will reset the whole system when they do not get a keepalive heartbeat within a specified timeframe. This adds an additional layer of fail safe in case usual Patroni split-brain protection mechanisms fail. + +### DCS layer + +- [etcd](etcd-info.md) - a Distributed Configuration Store. It stores the state of the PostgreSQL cluster and handles the election of a new primary. The odd number of nodes (minimum three) is required to always have the majority to agree on updates to the cluster state. + +### Load balancing layer + +- [HAProxy](haproxy-info.md) - the load balancer and the single point of entry to the cluster for client applications. Minimum two instances are required for redundancy. + +- keepalived - a high-availability and failover solution for HAProxy. It provides a virtual IP (VIP) address for HAProxy and prevents its single point of failure by failing over the services to the operational instance + +- (Optional) pgbouncer - a connection pooler for PostgreSQL. The aim of pgbouncer is to lower the performance impact of opening new connections to PostgreSQL. + +### Services layer + +- [pgBackRest](pgbackrest-info.md) - the backup and restore solution for PostgreSQL. It should also be redundant to eliminate a single point of failure. + +- (Optional) Percona Monitoring and Management (PMM) - the solution to monitor the health of your cluster + +## Bare-minimum architecture + +There may be constraints to use the [reference architecture with all additional components](#architecture), like the number of available servers or the cost for additional hardware. You can still achieve high-availability with the minimum two database nodes and three `etcd` instances. The following diagram shows this architecture: + +![Bare-minimum architecture of the PostgreSQL cluster](../_images/diagrams/HA-basic.svg) + +Using such architecture has the following limitations: + +* This setup only protects against a one node failure, either a database or a etcd node. Losing more than one node results in the read-only database. +* The application must be able to connect to multiple database nodes and fail over to the new primary in the case of outage. +* The application must act as the load-balancer. It must be able to determine read/write and read-only requests and distribute them across the cluster. +- The `pbBackRest` component is optional as it doesn't server the purpose of high-availability. But it is highly-recommended for disaster recovery and is a must fo production environments. [Contact us](https://www.percona.com/about/contact) to discuss backup configurations and retention policies. + +## Additional reading + +[How components work together](ha-components.md){.md-button} + +## Next steps + +[Deployment - initial setup :material-arrow-right:](ha-init-setup.md){.md-button} \ No newline at end of file diff --git a/docs/solutions/ha-components.md b/docs/solutions/ha-components.md new file mode 100644 index 000000000..3b7f24a81 --- /dev/null +++ b/docs/solutions/ha-components.md @@ -0,0 +1,53 @@ +# How components work together + +This document explains how components of the proposed [high-availability architecture](ha-architecture.md) work together. + +## Database and DSC layers + +Let's start with the database and DCS layers as they are interconnected and work closely together. + +Every database node hosts PostgreSQL and Patroni instances. + +Each PostgreSQL instance in the cluster maintains consistency with other members through streaming replication. Streaming replication is asynchronous by default, meaning that the primary does not wait for the secondaries to acknowledge the receipt of the data to consider the transaction complete. + +Each Patroni instance manages its own PostgreSQL instance. This means that Patroni starts and stops PostgreSQL and manages its configuration, being a sophisticated service manager for a PostgreSQL cluster. + +Patroni also can make an initial cluster initialization, monitor the cluster state and take other automatic actions if needed. To do so, Patroni relies on and uses the Distributed Configuration Store (DCS), represented by `etcd` in our architecture. + +Though Patroni supports various Distributed Configuration Stores like ZooKeeper, etcd, Consul or Kubernetes, we recommend and support `etcd` as the most popular DCS due to its simplicity, consistency and reliability. + +Note that the PostgreSQL high availability (HA) cluster and Patroni cluster are the same thing, and we will use these names interchangeably. + +When you start Patroni, it writes the cluster configuration information in `etcd`. During the initial cluster initialization, Patroni uses the `etcd` locking mechanism to ensure that only one instance becomes the primary. This mechanism ensures that only a single process can hold a resource at a time avoiding race conditions and inconsistencies. + +You start Patroni instances one by one so the first instance acquires the lock with a lease in `etcd` and becomes the primary PostgreSQL node. The other instances join the primary as replicas, waiting for the lock to be released. + +If the current primary node crashes, its lease on the lock in `etcd` expires. The lock is automatically released after its expiration time. `etcd` the starts a new election and a standby node attempts to acquire the lock to become the new primary. + +Patroni uses not only `etcd` locking mechanism. It also uses `etcd` to store the current state of the cluster, ensuring that all nodes are aware of the latest topology and status. + +Another important component is the watchdog. It runs on each database node. The purpose of watchdog is to prevent split-brain scenarios, where multiple nodes might mistakenly think they are the primary node. The watchdog monitors the node's health by receiving periodic "keepalive" signals from Patroni. If these signals stop due to a crash, high system load or any other reason, the watchdog resets the node to ensure it does not cause inconsistencies. + +## Load balancing layer + +This layer consists of HAProxy as the connection router and load balancer. + +HAProxy acts as a single point of entry to your cluster for client applications. It accepts all requests from client applications and distributes the load evenly across the cluster nodes. It can route read/write requests to the primary and read-only requests to the secondary nodes. This behavior is defined within HAProxy configuration. To determine the current primary node, HAProxy queries the Patroni REST API. + +HAProxy must be also redundant. Each application server or Pod can have its own HAProxy. If it cannot have own HAProxy, you can deploy HAProxy outside the application layer. This may introduce additional network hops and a failure point. + +If you are deploying HAProxy outside the application layer, you need a minimum of 2 HAProxy nodes (one is active and another one standby) to avoid a single point of failure. These instances share a floating virtual IP address using Keepalived. + +Keepalived acts as the failover tool for HAProxy. It provides the virtual IP address (VIP) for HAProxy and monitors its state. When the current active HAProxy node is down, it transfers the VIP to the remaining node and fails over the services there. + +## Services layer + +Finally, the services layer is represented by `pgBackRest` and PMM. + +`pgBackRest` can manage a dedicated backup server or make backups to the cloud. `pgBackRest` agent are deployed on every database node. `pgBackRest` can utilize standby nodes to offload the backup load from the primary. However, WAL archiving is happening only from the primary node. By communicating with its agents,`pgBackRest` determines the current cluster topology and uses the nodes to make backups most effectively without any manual reconfiguration at the event of a switchover or failover. + +The monitoring solution is optional but nice to have. It enables you to monitor the health of your high-availability architecture, receive timely alerts should performance issues occur and proactively react to them. + +## Next steps + +[Deployment - initial setup :material-arrow-right:](ha-init-setup.md){.md-button} diff --git a/docs/solutions/ha-etcd-config.md b/docs/solutions/ha-etcd-config.md new file mode 100644 index 000000000..9b95b3493 --- /dev/null +++ b/docs/solutions/ha-etcd-config.md @@ -0,0 +1,170 @@ +# Etcd setup + +In our solutions, we use etcd distributed configuration store. [Refresh your knowledge about etcd](etcd-info.md). + +## Install etcd + +Install etcd on all PostgreSQL nodes: `node1`, `node2` and `node3`. + +=== ":material-debian: On Debian / Ubuntu" + + 1. Install etcd: + + ```{.bash data-prompt="$"} + $ sudo apt install etcd etcd-server etcd-client + ``` + + 3. Stop and disable etcd: + + ```{.bash data-prompt="$"} + $ sudo systemctl stop etcd + $ sudo systemctl disable etcd + ``` + +=== ":material-redhat: On RHEL and derivatives" + + + 1. Install etcd. + + ```{.bash data-prompt="$"} + $ sudo yum install etcd python3-python-etcd + ``` + + 3. Stop and disable etcd: + + ```{.bash data-prompt="$"} + $ sudo systemctl stop etcd + $ sudo systemctl disable etcd + ``` + +!!! note + + If you [installed etcd from tarballs](../tarball.md), you must first [enable it](../enable-extensions.md#etcd) before configuring it. + +## Configure etcd + +To get started with `etcd` cluster, you need to bootstrap it. This means setting up the initial configuration and starting the etcd nodes so they can form a cluster. There are the following bootstrapping mechanisms: + +* Static in the case when the IP addresses of the cluster nodes are known +* Discovery service - for cases when the IP addresses of the cluster are not known ahead of time. + +Since we know the IP addresses of the nodes, we will use the static method. For using the discovery service, please refer to the [etcd documentation :octicons-link-external-16:](https://etcd.io/docs/v3.5/op-guide/clustering/#etcd-discovery){:target="_blank"}. + +We will configure and start all etcd nodes in parallel. This can be done either by modifying each node's configuration or using the command line options. Use the method that you prefer more. + +### Method 1. Modify the configuration file + +1. Create the etcd configuration file on every node. You can edit the sample configuration file `/etc/etcd/etcd.conf.yaml` or create your own one. Replace the node names and IP addresses with the actual names and IP addresses of your nodes. + + === "node1" + + ```yaml title="/etc/etcd/etcd.conf.yaml" + name: 'node1' + initial-cluster-token: PostgreSQL_HA_Cluster_1 + initial-cluster-state: new + initial-cluster: node1=http://10.104.0.1:2380,node2=http://10.104.0.2:2380,node3=http://10.104.0.3:2380 + data-dir: /var/lib/etcd + initial-advertise-peer-urls: http://10.104.0.1:2380 + listen-peer-urls: http://10.104.0.1:2380 + advertise-client-urls: http://10.104.0.1:2379 + listen-client-urls: http://10.104.0.1:2379 + ``` + + === "node2" + + ```yaml title="/etc/etcd/etcd.conf.yaml" + name: 'node2' + initial-cluster-token: PostgreSQL_HA_Cluster_1 + initial-cluster-state: new + initial-cluster: node1=http://10.104.0.1:2380,node2=http://10.104.0.2:2380, node3=http://10.104.0.3:2380 + data-dir: /var/lib/etcd + initial-advertise-peer-urls: http://10.104.0.2:2380 + listen-peer-urls: http://10.104.0.2:2380 + advertise-client-urls: http://10.104.0.2:2379 + listen-client-urls: http://10.104.0.2:2379 + ``` + + === "node3" + + ```yaml title="/etc/etcd/etcd.conf.yaml" + name: 'node3' + initial-cluster-token: PostgreSQL_HA_Cluster_1 + initial-cluster-state: new + initial-cluster: node1=http://10.104.0.1:2380,node2=http://10.104.0.2:2380, node3=http://10.104.0.3:2380 + data-dir: /var/lib/etcd + initial-advertise-peer-urls: http://10.104.0.3:2380 + listen-peer-urls: http://10.104.0.3:2380 + advertise-client-urls: http://10.104.0.3:2379 + listen-client-urls: http://10.104.0.3:2379 + ``` + +2. Enable and start the `etcd` service on all nodes: + + ```{.bash data-prompt="$"} + $ sudo systemctl enable --now etcd + $ sudo systemctl status etcd + ``` + + During the node start, etcd searches for other cluster nodes defined in the configuration. If the other nodes are not yet running, the start may fail by a quorum timeout. This is expected behavior. Try starting all nodes again at the same time for the etcd cluster to be created. + +--8<-- "check-etcd.md" + +### Method 2. Start etcd nodes with command line options + +1. On each etcd node, set the environment variables for the cluster members, the cluster token and state: + + ``` + TOKEN=PostgreSQL_HA_Cluster_1 + CLUSTER_STATE=new + NAME_1=node1 + NAME_2=node2 + NAME_3=node3 + HOST_1=10.104.0.1 + HOST_2=10.104.0.2 + HOST_3=10.104.0.3 + CLUSTER=${NAME_1}=http://${HOST_1}:2380,${NAME_2}=http://${HOST_2}:2380,${NAME_3}=http://${HOST_3}:2380 + ``` + +2. Start each etcd node in parallel using the following command: + + === "node1" + + ```{.bash data-prompt="$"} + THIS_NAME=${NAME_1} + THIS_IP=${HOST_1} + etcd --data-dir=data.etcd --name ${THIS_NAME} \ + --initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://${THIS_IP}:2380 \ + --advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://${THIS_IP}:2379 \ + --initial-cluster ${CLUSTER} \ + --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN} & + ``` + + === "node2" + + ```{.bash data-prompt="$"} + THIS_NAME=${NAME_2} + THIS_IP=${HOST_2} + etcd --data-dir=data.etcd --name ${THIS_NAME} \ + --initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://${THIS_IP}:2380 \ + --advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://${THIS_IP}:2379 \ + --initial-cluster ${CLUSTER} \ + --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN} & + ``` + + === "node3" + + ```{.bash data-prompt="$"} + THIS_NAME=${NAME_3} + THIS_IP=${HOST_3} + etcd --data-dir=data.etcd --name ${THIS_NAME} \ + --initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://${THIS_IP}:2380 \ + --advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://${THIS_IP}:2379 \ + --initial-cluster ${CLUSTER} \ + --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN} & + ``` + +--8<-- "check-etcd.md" + +## Next steps + +[Patroni setup :material-arrow-right:](ha-patroni.md){.md-button} \ No newline at end of file diff --git a/docs/solutions/ha-haproxy.md b/docs/solutions/ha-haproxy.md new file mode 100644 index 000000000..e89957216 --- /dev/null +++ b/docs/solutions/ha-haproxy.md @@ -0,0 +1,269 @@ +# Configure HAProxy + +HAproxy is the connection router and acts as a single point of entry to your PostgreSQL cluster for client applications. Additionally, HAProxy provides load-balancing for read-only connections. + +A client application connects to HAProxy and sends its read/write requests there. You can provide different ports in the HAProxy configuration file so that the client application can explicitly choose between read-write (primary) connection or read-only (replica) connection using the right port number to connect. In this deployment, writes are routed to port 5000 and reads - to port 5001. + +The client application doesn't know what node in the underlying cluster is the current primary. But it must connect to the HAProxy read-write connection to send all write requests. This ensures that HAProxy correctly routes all write load to the current primary node. Read requests are routed to the secondaries in a round-robin fashion so that no secondary instance is unnecessarily loaded. + +When you deploy HAProxy outside the application layer, you must deploy multiple instances of it and have the automatic failover mechanism to eliminate a single point of failure for HAProxy. + +For this document we focus on deployment on premises and we use `keepalived`. It monitors HAProxy state and manages the virtual IP for HAProxy. + +If you use a cloud infrastructure, it may be easier to use the load balancer provided by the cloud provider to achieve high-availability with HAProxy. + +## HAProxy setup + +1. Install HAProxy on the HAProxy nodes: `HAProxy1`, `HAProxy2` and `HAProxy3`: + + ```{.bash data-prompt="$"} + $ sudo apt install percona-haproxy + ``` + +2. The HAProxy configuration file path is: `/etc/haproxy/haproxy.cfg`. Specify the following configuration in this file for every node. + + ``` + global + maxconn 100 # Maximum number of concurrent connections + + defaults + log global # Use global logging configuration + mode tcp # TCP mode for PostgreSQL connections + retries 2 # Number of retries before marking a server as failed + timeout client 30m # Maximum time to wait for client data + timeout connect 4s # Maximum time to establish connection to server + timeout server 30m # Maximum time to wait for server response + timeout check 5s # Maximum time to wait for health check response + + listen stats # Statistics monitoring + mode http # The protocol for web-based stats UI + bind *:7000 # Port to listen to on all network interfaces + stats enable # Statistics reporting interface + stats uri /stats # URL path for the stats page + stats auth percona:myS3cr3tpass # Username:password authentication + + listen primary + bind *:5000 # Port for write connections + option httpchk /primary + http-check expect status 200 + default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions # Server health check parameters + server node1 node1:5432 maxconn 100 check port 8008 + server node2 node2:5432 maxconn 100 check port 8008 + server node3 node3:5432 maxconn 100 check port 8008 + + listen standbys + balance roundrobin # Round-robin load balancing for read connections + bind *:5001 # Port for read connections + option httpchk /replica + http-check expect status 200 + default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions # Server health check parameters + server node1 node1:5432 maxconn 100 check port 8008 + server node2 node2:5432 maxconn 100 check port 8008 + server node3 node3:5432 maxconn 100 check port 8008 + ``` + + HAProxy will use the REST APIs hosted by Patroni to check the health status of each PostgreSQL node and route the requests appropriately. + + To monitor HAProxy stats, create the user who has the access to it. Read more about statistics dashboard in [HAProxy documentation :octicons-link-external-16:](https://www.haproxy.com/documentation/haproxy-configuration-tutorials/alerts-and-monitoring/statistics/) + +3. Restart HAProxy: + + ```{.bash data-prompt="$"} + $ sudo systemctl restart haproxy + ``` + +4. Check the HAProxy logs to see if there are any errors: + + ```{.bash data-prompt="$"} + $ sudo journalctl -u haproxy.service -n 100 -f + ``` + +## Keepalived setup + +The HAproxy instances will share a virtual IP address `203.0.113.1` as the single point of entry for client applications. + +In this setup we define the basic health check for HAProxy. You may want to use a more sophisticated check. You can do this by writing a script and referencing it in the `keeplaived` configuration. See the [Example of HAProxy health check](#example-of-haproxy-health-check) section for details. + +1. Install `keepalived` on all HAProxy nodes: + + === ":material-debian: On Debian and Ubuntu" + + ```{.bash data-prompt="$"} + $ sudo apt install keepalived + ``` + + === ":material-redhat: On RHEL and derivatives" + + ```{.bash data-prompt="$"} + $ sudo yum install keepalived + ``` + +2. Create the `keepalived` configuration file at `/etc/keepalived/keepalived.conf` with the following contents for each node: + + === "Primary HAProxy (HAProxy1)" + + ```ini + vrrp_script chk_haproxy { + script "killall -0 haproxy" # Basic check if HAProxy process is running + interval 3 # Check every 2 seconds + fall 3 # The number of failures to mark the node as down + rise 2 # The number of successes to mark the node as up + weight -11 # Reduce priority by 2 on failure + } + + vrrp_instance CLUSTER_1 { # The name of Patroni cluster + state MASTER # Initial state for the primary node + interface eth1 # Network interface to bind to + virtual_router_id 99 # Unique ID for this VRRP instance + priority 110 # The priority for the primary must be the highest + advert_int 1 # Advertisement interval + authentication { + auth_type PASS + auth_pass myS3cr3tpass # Authentication password + } + virtual_ipaddress { + 203.0.113.1/24 # The virtual IP address + } + track_script { + chk_haproxy + } + } + ``` + + === "HAProxy2" + + ```ini + vrrp_script chk_haproxy { + script "killall -0 haproxy" # Basic check if HAProxy process is running + interval 2 # Check every 2 seconds + fall 2 # The number of failures to mark the node as down + rise 2 # The number of successes to mark the node as up + weight 2 # Reduce priority by 2 on failure + } + + vrrp_instance CLUSTER_1 { + state BACKUP # Initial state for backup node + interface eth1 # Network interface to bind to + virtual_router_id 99 # Same ID as primary + priority 100 # Lower priority than primary + advert_int 1 # Advertisement interval + authentication { + auth_type PASS + auth_pass myS3cr3tpass # Same password as primary + } + virtual_ipaddress { + 203.0.113.1/24 + } + track_script { + chk_haproxy + } + } + ``` + + === "HAProxy3" + + ```ini + vrrp_script chk_haproxy { + script "killall -0 haproxy" # Basic check if HAProxy process is running + interval 2 # Check every 2 seconds + fall 3 # The number of failures to mark the node as down + rise 2 # The number of successes to mark the node as up + weight 6 # Reduce priority by 2 on failure + } + + vrrp_instance CLUSTER_1 { + state BACKUP # Initial state for backup node + interface eth1 # Network interface to bind to + virtual_router_id 99 # Same ID as primary + priority 105 # Lowest priority + advert_int 1 # Advertisement interval + authentication { + auth_type PASS + auth_pass myS3cr3tpass # Same password as primary + } + virtual_ipaddress { + 203.0.113.1/24 + } + track_script { + chk_haproxy + } + } + ``` + +3. Start `keepalived`: + + ```{.bash data-prompt="$"} + $ sudo systemctl start keepalived + ``` + +4. Check the `keepalived` status: + + ```{.bash data-prompt="$"} + $ sudo systemctl status keepalived + ``` + +!!! note + + The basic health check (`killall -0 haproxy`) only verifies that the HAProxy process is running. For production environments, consider implementing more comprehensive health checks that verify the node's overall responsiveness and HAProxy's ability to handle connections. + +### Example of HAProxy health check + +Sometimes checking only the running haproxy process is not enough. The process may be running while HAProxy is in a degraded state. A good practice is to make additional checks to ensure HAProxy is healthy. + +Here's an example health check script for HAProxy. It performs the following checks: + +1. Verifies that the HAProxy process is running +2. Tests if the HAProxy admin socket is accessible +3. Confirms that HAProxy is binding to the default port `5432` + +```bash +#!/bin/bash + +# Exit codes: +# 0 - HAProxy is healthy +# 1 - HAProxy is not healthy + +# Check if HAProxy process is running +if ! pgrep -x haproxy > /dev/null; then + echo "HAProxy process is not running" + exit 1 +fi + +# Check if HAProxy socket is accessible +if ! socat - UNIX-CONNECT:/var/run/haproxy/admin.sock > /dev/null 2>&1; then + echo "HAProxy socket is not accessible" + exit 1 +fi + +# Check if HAProxy is binding to port 5432 +if ! netstat -tuln | grep -q ":5432 "; then + exit 1 +fi + +# All checks passed +exit 0 +``` + +Save this script as `/usr/local/bin/check_haproxy.sh` and make it executable: + +```{.bash data-prompt="$"} +$ sudo chmod +x /usr/local/bin/check_haproxy.sh +``` + +Then define this script in Keepalived configuration on each node: + +```ini +vrrp_script chk_haproxy { + script "/usr/local/bin/check_haproxy.sh" + interval 2 + fall 3 + rise 2 + weight -10 +} +``` + +Congratulations! You have successfully configured your HAProxy solution. Now you can proceed to testing it. + +## Next steps + +[Test Patroni PostgreSQL cluster :material-arrow-right:](ha-test.md){.md-button} diff --git a/docs/solutions/ha-init-setup.md b/docs/solutions/ha-init-setup.md new file mode 100644 index 000000000..6d8d5ee53 --- /dev/null +++ b/docs/solutions/ha-init-setup.md @@ -0,0 +1,81 @@ +# Initial setup for high availability + +This guide provides instructions on how to set up a highly available PostgreSQL cluster with Patroni. This guide relies on the provided [architecture](ha-architecture.md) for high-availability. + +## Considerations + +1. This is an example deployment where etcd runs on the same host machines as the Patroni and PostgreSQL and there is a single dedicated HAProxy host. Alternatively etcd can run on different set of nodes. + + If etcd is deployed on the same host machine as Patroni and PostgreSQL, separate disk system for etcd and PostgreSQL is recommended due to performance reasons. + +2. For this setup, we will use the nodes that have the following IP addresses: + + + | Node name | Public IP address | Internal IP address + |---------------|-------------------|-------------------- + | node1 | 157.230.42.174 | 10.104.0.7 + | node2 | 68.183.177.183 | 10.104.0.2 + | node3 | 165.22.62.167 | 10.104.0.8 + | HAProxy1 | 112.209.126.159 | 10.104.0.6 + | HAProxy2 | 134.209.111.138 | 10.104.0.5 + | HAProxy3 | 134.60.204.27 | 10.104.0.3 + | backup | 97.78.129.11 | 10.104.0.9 + + We also need a virtual IP address for HAProxy: `203.0.113.1` + + +!!! important + + We recommend not to expose the hosts/nodes where Patroni / etcd / PostgreSQL are running to public networks due to security risks. Use Firewalls, Virtual networks, subnets or the like to protect the database hosts from any kind of attack. + +## Configure name resolution + +It’s not necessary to have name resolution, but it makes the whole setup more readable and less error prone. Here, instead of configuring a DNS, we use a local name resolution by updating the file `/etc/hosts`. By resolving their hostnames to their IP addresses, we make the nodes aware of each other’s names and allow their seamless communication. + +Run the following commands on each node. + +1. Set the hostname for nodes. Change the node name to `node1`, `node2`, `node3`, `HAProxy1`, `HAProxy2` and `backup`, respectively: + + ```{.bash data-prompt="$"} + $ sudo hostnamectl set-hostname node1 + ``` + +2. Modify the `/etc/hosts` file of each node to include the hostnames and IP addresses of the remaining nodes. Add the following at the end of the `/etc/hosts` file on all nodes: + + ```text + # Cluster IP and names + + 10.104.0.7 node1 + 10.104.0.2 node2 + 10.104.0.8 node3 + 10.104.0.6 HAProxy1 + 10.104.0.5 HAProxy2 + 10.104.0.3 HAProxy3 + 10.104.0.9 backup + ``` + +## Configure Percona repository + +To install the software from Percona, you need to subscribe to Percona repositories. To do this, you require `percona-release` - the repository management tool. + +Run the following commands on each node as the root user or with `sudo` privileges. + +1. Install `percona-release` + + === ":material-debian: On Debian and Ubuntu" + + --8<-- "percona-release-apt.md" + + === ":material-redhat: On RHEL and derivatives" + + --8<-- "percona-release-yum.md" + +2. Enable the repository: + + ```{.bash data-prompt="$"} + $ sudo percona-release setup ppg{{pgversion}} + ``` + +## Next steps + +[Set up etcd :material-arrow-right:](ha-etcd-config.md){.md-button} \ No newline at end of file diff --git a/docs/solutions/ha-measure.md b/docs/solutions/ha-measure.md new file mode 100644 index 000000000..058350022 --- /dev/null +++ b/docs/solutions/ha-measure.md @@ -0,0 +1,39 @@ +# Measuring high availability + +The need for high availability is determined by the business requirements, potential risks, and operational limitations. For example, the more components you add to your infrastructure, the more complex and time-consuming it is to maintain. Moreover, it may introduce extra failure points. The recommendation is to follow the principle "The simpler the better". + +The level of high availability depends on the following: + +* how frequently you may encounter an outage or a downtime. +* how much downtime you can bear without negatively impacting your users for every outage, and +* how much data loss you can tolerate during the outage. + + +When you evaluate high-availability, consider these two aspects: + +* Expected level of availability. +* Actual availability level of your infrastructure. + +### Expected level of availability + +It is measured by establishing a measurement time frame and dividing it by the time that it was available. This ratio will rarely be one, which is equal to 100% availability. At Percona, we don't consider a solution to be highly available if it is not at least 99% or two nines available. + +The following table shows the amount of downtime for each level of availability from two to five nines. + +| Availability % | Downtime per year | Downtime per month | Downtime per week | Downtime per day | +|--------------------------|-------------------|--------------------|-------------------|-------------------| +| 99% (“two nines”) | 3.65 days | 7.31 hours | 1.68 hours | 14.40 minutes | +| 99.5% (“two nines five”) | 1.83 days | 3.65 hours | 50.40 minutes | 7.20 minutes | +| 99.9% (“three nines”) | 8.77 hours | 43.83 minutes | 10.08 minutes | 1.44 minutes | +| 99.95% (“three nines five”) | 4.38 hours | 21.92 minutes | 5.04 minutes | 43.20 seconds | +| 99.99% (“four nines”) | 52.60 minutes | 4.38 minutes | 1.01 minutes | 8.64 seconds | +| 99.995% (“four nines five”) | 26.30 minutes | 2.19 minutes | 30.24 seconds | 4.32 seconds | +| 99.999% (“five nines”) | 5.26 minutes | 26.30 seconds | 6.05 seconds | 864.00 milliseconds | + +### Actual level of availability + +Measuring the real level of high availability (HA) in your system is key to making sure your investment in HA infrastructure pays off. Instead of relying on assumptions or expectations, you should base your availability insights on incident management data. This is the information collected during service disruptions, failures, or outages that affect the normal functioning of the setup. With this data, you can track metrics like uptime, Mean Time to Recovery (MTTR), and Mean Time Between Failures (MTBF). + +MTBF gives you a picture of how reliable your infrastructure really is. In well-designed high-availability environment, the incidents should be rare, typically occurring no more than once every 2 to 4 years. This assumes a robust infrastructure, as not all systems equally suit for handling database load. + +Recovery speed matters too. For example, a typical Patroni-based cluster can fail over to a new primary node within 30 to 50 seconds. However, note that database availability metrics typically don't consider the application's ability to detect the failover and reconnect. Some applications recover seamlessly, while others may require a restart. diff --git a/docs/solutions/ha-patroni.md b/docs/solutions/ha-patroni.md new file mode 100644 index 000000000..d0516ef61 --- /dev/null +++ b/docs/solutions/ha-patroni.md @@ -0,0 +1,371 @@ +# Patroni setup + +## Install Percona Distribution for PostgreSQL and Patroni + +Run the following commands as root or with `sudo` privileges on `node1`, `node2` and `node3`. + +=== ":material-debian: On Debian / Ubuntu" + + 1. Disable the upstream `postgresql-{{pgversion}}` package. + + 2. Install Percona Distribution for PostgreSQL package + + ```{.bash data-prompt="$"} + $ sudo apt install percona-postgresql-{{pgversion}} + ``` + + 3. Install some Python and auxiliary packages to help with Patroni + + ```{.bash data-prompt="$"} + $ sudo apt install python3-pip python3-dev binutils + ``` + + 4. Install Patroni + + ```{.bash data-prompt="$"} + $ sudo apt install percona-patroni + ``` + + 5. Stop and disable all installed services: + + ```{.bash data-prompt="$"} + $ sudo systemctl stop {patroni,postgresql} + $ sudo systemctl disable {patroni,postgresql} + ``` + + 6. Even though Patroni can use an existing Postgres installation, our recommendation for a **new cluster that has no data** is to remove the data directory. This forces Patroni to initialize a new Postgres cluster instance. + + ```{.bash data-prompt="$"} + $ sudo systemctl stop postgresql + $ sudo rm -rf /var/lib/postgresql/{{pgversion}}/main + ``` + +=== ":material-redhat: On RHEL and derivatives" + + 1. Install Percona Distribution for PostgreSQL package + + ```{.bash data-prompt="$"} + $ sudo yum install percona-postgresql{{pgversion}}-server + ``` + + 2. Check the [platform specific notes for Patroni](../yum.md#for-percona-distribution-for-postgresql-packages) + + 3. Install some Python and auxiliary packages to help with Patroni and etcd + + ```{.bash data-prompt="$"} + $ sudo yum install python3-pip python3-devel binutils + ``` + + 4. Install Patroni + + ```{.bash data-prompt="$"} + $ sudo yum install percona-patroni + ``` + + 3. Stop and disable all installed services: + + ```{.bash data-prompt="$"} + $ sudo systemctl stop {patroni,postgresql-{{pgversion}}} + $ sudo systemctl disable {patroni,postgresql-{{pgversion}}} + ``` + + !!! important + + **Don't** initialize the cluster and start the `postgresql` service. The cluster initialization and setup are handled by Patroni during the bootsrapping stage. + +## Configure Patroni + +Run the following commands on all nodes. You can do this in parallel: + +### Create environment variables + +Environment variables simplify the config file creation: + +1. Node name: + + ```{.bash data-prompt="$"} + $ export NODE_NAME=`hostname -f` + ``` + +2. Node IP: + + ```{.bash data-prompt="$"} + $ export NODE_IP=`getent hosts $(hostname -f) | awk '{ print $1 }' | grep -v grep | grep -v '127.0.1.1'` + ``` + + * Check that the correct IP address is defined: + + ```{.bash data-prompt="$"} + $ echo $NODE_IP + ``` + + ??? example "Sample output `node1`" + + ```{text .no-copy} + 10.104.0.7 + ``` + + If you have multiple IP addresses defined on your server and the environment variable contains the wrong one, you can manually redefine it. For example, run the following command for `node1`: + + ```{.bash data-prompt="$"} + $ NODE_IP=10.104.0.7 + ``` + +3. Create variables to store the `PATH`. Check the path to the `data` and `bin` folders on your operating system and change it for the variables accordingly: + + === ":material-debian: Debian and Ubuntu" + + ```bash + DATA_DIR="/var/lib/postgresql/{{pgversion}}/main" + PG_BIN_DIR="/usr/lib/postgresql/{{pgversion}}/bin" + ``` + + === ":material-redhat: RHEL and derivatives" + + ```bash + DATA_DIR="/var/lib/pgsql/data/" + PG_BIN_DIR="/usr/pgsql-{{pgversion}}/bin" + ``` + +4. Patroni information: + + ```bash + NAMESPACE="percona_lab" + SCOPE="cluster_1" + ``` + +### Create the directories required by Patroni + +Create the directory to store the configuration file and make it owned by the `postgres` user. + +```{.bash data-prompt="$"} +$ sudo mkdir -p /etc/patroni/ +$ sudo chown -R postgres:postgres /etc/patroni/ +``` + +### Patroni configuration file + +Use the following command to create the `/etc/patroni/patroni.yml` configuration file and add the following configuration for every node: + +```bash +echo " +namespace: ${NAMESPACE} +scope: ${SCOPE} +name: ${NODE_NAME} + +restapi: + listen: 0.0.0.0:8008 + connect_address: ${NODE_IP}:8008 + +etcd3: + host: ${NODE_IP}:2379 + +bootstrap: + # this section will be written into Etcd:///config after initializing new cluster + dcs: + ttl: 30 + loop_wait: 10 + retry_timeout: 10 + maximum_lag_on_failover: 1048576 + + postgresql: + use_pg_rewind: true + use_slots: true + parameters: + wal_level: replica + hot_standby: "on" + wal_keep_segments: 10 + max_wal_senders: 5 + max_replication_slots: 10 + wal_log_hints: "on" + logging_collector: 'on' + max_wal_size: '10GB' + archive_mode: "on" + archive_timeout: 600s + archive_command: "cp -f %p /home/postgres/archived/%f" + + pg_hba: # Add following lines to pg_hba.conf after running 'initdb' + - host replication replicator 127.0.0.1/32 trust + - host replication replicator 0.0.0.0/0 md5 + - host all all 0.0.0.0/0 md5 + - host all all ::0/0 md5 + recovery_conf: + restore_command: cp /home/postgres/archived/%f %p + + # some desired options for 'initdb' + initdb: # Note: It needs to be a list (some options need values, others are switches) + - encoding: UTF8 + - data-checksums + + +postgresql: + cluster_name: cluster_1 + listen: 0.0.0.0:5432 + connect_address: ${NODE_IP}:5432 + data_dir: ${DATA_DIR} + bin_dir: ${PG_BIN_DIR} + pgpass: /tmp/pgpass0 + authentication: + replication: + username: replicator + password: replPasswd + superuser: + username: postgres + password: qaz123 + parameters: + unix_socket_directories: "/var/run/postgresql/" + create_replica_methods: + - basebackup + basebackup: + checkpoint: 'fast' + + watchdog: + mode: required # Allowed values: off, automatic, required + device: /dev/watchdog + safety_margin: 5 + +tags: + nofailover: false + noloadbalance: false + clonefrom: false + nosync: false +" | sudo tee /etc/patroni/patroni.yml +``` + +??? admonition "Patroni configuration file" + + Let’s take a moment to understand the contents of the `patroni.yml` file. + + The first section provides the details of the node and its connection ports. After that, we have the `etcd` service and its port details. + + Following these, there is a `bootstrap` section that contains the PostgreSQL configurations and the steps to run once + +### Systemd configuration + +1. Check that the systemd unit file `percona-patroni.service` is created in `/etc/systemd/system`. If it is created, skip this step. + + If it's **not created**, create it manually and specify the following contents within: + + ```ini title="/etc/systemd/system/percona-patroni.service" + [Unit] + Description=Runners to orchestrate a high-availability PostgreSQL + After=syslog.target network.target + + [Service] + Type=simple + + User=postgres + Group=postgres + + # Start the patroni process + ExecStart=/bin/patroni /etc/patroni/patroni.yml + + # Send HUP to reload from patroni.yml + ExecReload=/bin/kill -s HUP $MAINPID + + # only kill the patroni process, not its children, so it will gracefully stop postgres + KillMode=process + + # Give a reasonable amount of time for the server to start up/shut down + TimeoutSec=30 + + # Do not restart the service if it crashes, we want to manually inspect database on failure + Restart=no + + [Install] + WantedBy=multi-user.target + ``` + +2. Make `systemd` aware of the new service: + + ```{.bash data-prompt="$"} + $ sudo systemctl daemon-reload + ``` + +3. Make sure you have the configuration file and the `systemd` unit file created on every node. + +### Start Patroni + +Now it's time to start Patroni. You need the following commands on all nodes but **not in parallel**. + +1. Start Patroni on `node1` first, wait for the service to come to live, and then proceed with the other nodes one-by-one, always waiting for them to sync with the primary node: + + ```{.bash data-prompt="$"} + $ sudo systemctl enable --now percona-patroni + ``` + + When Patroni starts, it initializes PostgreSQL (because the service is not currently running and the data directory is empty) following the directives in the bootstrap section of the configuration file. + +2. Check the service to see if there are errors: + + ```{.bash data-prompt="$"} + $ sudo journalctl -fu percona-patroni + ``` + + See [Troubleshooting Patroni startup](#troubleshooting-patroni-startup) for guidelines in case of errors. + + If Patroni has started properly, you should be able to locally connect to a PostgreSQL node using the following command: + + ```{.bash data-prompt="$"} + $ sudo psql -U postgres + + psql ({{dockertag}}) + Type "help" for help. + + postgres=# + ``` + +9. When all nodes are up and running, you can check the cluster status using the following command: + + ```{.bash data-prompt="$"} + $ sudo patronictl -c /etc/patroni/patroni.yml list + ``` + + The output resembles the following: + + ??? example "Sample output node1" + + ```{.text .no-copy} + + Cluster: cluster_1 (7440127629342136675) -----+----+-------+ + | Member | Host | Role | State | TL | Lag in MB | + +--------+------------+---------+-----------+----+-----------+ + | node1 | 10.0.100.1 | Leader | running | 1 | | + ``` + + ??? example "Sample output node3" + + ```{.text .no-copy} + + Cluster: cluster_1 (7440127629342136675) -----+----+-------+ + | Member | Host | Role | State | TL | Lag in MB | + +--------+------------+---------+-----------+----+-----------+ + | node1 | 10.0.100.1 | Leader | running | 1 | | + | node2 | 10.0.100.2 | Replica | streaming | 1 | 0 | + | node3 | 10.0.100.3 | Replica | streaming | 1 | 0 | + +--------+------------+---------+-----------+----+-----------+ + ``` + +### Troubleshooting Patroni startup + + A common error is Patroni complaining about the lack of proper entries in the `pg_hba.conf` file. If you see such errors, you must manually add or fix the entries in that file and then restart the service. + +An example of such an error is `No pg_hba.conf entry for replication connection from host to , user replicator, no encryption`. This means that Patroni cannot connect to the node you're adding to the cluster. To resolve this issue, add the IP addresses of the nodes to the `pg_hba:` section of the Patroni configuration file. + +``` +pg_hba: # Add following lines to pg_hba.conf after running 'initdb' +- host replication replicator 127.0.0.1/32 trust +- host replication replicator 0.0.0.0/0 md5 +- host replication replicator 10.0.100.2/32 trust +- host replication replicator 10.0.100.3/32 trust +- host all all 0.0.0.0/0 md5 +- host all all ::0/0 md5 +recovery_conf: + restore_command: cp /home/postgres/archived/%f %p +``` + +For production use, we recommend adding nodes individually as the more secure way. However, if your network is secure and you trust it, you can add the whole network these nodes belong to as the trusted one to bypass passwords use during authentication. Then all nodes from this network can connect to Patroni cluster. + +Changing the `patroni.yml` file and restarting the service will not have any effect here because the bootstrap section specifies the configuration to apply when PostgreSQL is first started in the node. It will not repeat the process even if the Patroni configuration file is modified and the service is restarted. + +## Next steps + +[pgBackRest setup :material-arrow-right:](pgbackrest.md){.md-button} diff --git a/docs/solutions/ha-setup-apt.md b/docs/solutions/ha-setup-apt.md deleted file mode 100644 index c3baa590a..000000000 --- a/docs/solutions/ha-setup-apt.md +++ /dev/null @@ -1,569 +0,0 @@ -# Deploying PostgreSQL for high availability with Patroni on Debian or Ubuntu - -This guide provides instructions on how to set up a highly available PostgreSQL cluster with Patroni on Debian or Ubuntu. - - -## Preconditions - -1. This is an example deployment where etcd runs on the same host machines as the Patroni and PostgreSQL and there is a single dedicated HAProxy host. Alternatively etcd can run on different set of nodes. - - If etcd is deployed on the same host machine as Patroni and PostgreSQL, separate disk system for etcd and PostgreSQL is recommended due to performance reasons. - -2. For this setup, we will use the nodes running on Ubuntu 22.04 as the base operating system: - - -| Node name | Public IP address | Internal IP address -|---------------|-------------------|-------------------- -| node1 | 157.230.42.174 | 10.104.0.7 -| node2 | 68.183.177.183 | 10.104.0.2 -| node3 | 165.22.62.167 | 10.104.0.8 -| HAProxy-demo | 134.209.111.138 | 10.104.0.6 - - -!!! note - - We recommend not to expose the hosts/nodes where Patroni / etcd / PostgreSQL are running to public networks due to security risks. Use Firewalls, Virtual networks, subnets or the like to protect the database hosts from any kind of attack. - -## Initial setup - -It’s not necessary to have name resolution, but it makes the whole setup more readable and less error prone. Here, instead of configuring a DNS, we use a local name resolution by updating the file `/etc/hosts`. By resolving their hostnames to their IP addresses, we make the nodes aware of each other’s names and allow their seamless communication. - -1. Run the following command on each node. Change the node name to `node1`, `node2` and `node3` respectively: - - ```{.bash data-prompt="$"} - $ sudo hostnamectl set-hostname node-1 - ``` - -2. Modify the `/etc/hosts` file of each PostgreSQL node to include the hostnames and IP addresses of the remaining nodes. Add the following at the end of the `/etc/hosts` file on all nodes: - - === "node1" - - ```text hl_lines="3 4" - # Cluster IP and names - 10.104.0.1 node1 - 10.104.0.2 node2 - 10.104.0.3 node3 - ``` - - === "node2" - - ```text hl_lines="2 4" - # Cluster IP and names - 10.104.0.1 node1 - 10.104.0.2 node2 - 10.104.0.3 node3 - ``` - - === "node3" - - ```text hl_lines="2 3" - # Cluster IP and names - 10.104.0.1 node1 - 10.104.0.2 node2 - 10.104.0.3 node3 - ``` - - === "HAproxy-demo" - - The HAProxy instance should have the name resolution for all the three nodes in its `/etc/hosts` file. Add the following lines at the end of the file: - - ```text hl_lines="4 5 6" - # Cluster IP and names - 10.104.0.6 HAProxy-demo - 10.104.0.1 node1 - 10.104.0.2 node2 - 10.104.0.3 node3 - ``` - - -### Install the software - -Run the following commands on `node1`, `node2` and `node3`: - -1. Install Percona Distribution for PostgreSQL - - * [Install `percona-release` :octicons-link-external-16:](https://www.percona.com/doc/percona-repo-config/installing.html). - - * Enable the repository: - - ```{.bash data-prompt="$"} - $ sudo percona-release setup ppg16 - ``` - - * [Install Percona Distribution for PostgreSQL packages](../apt.md). - -2. Install some Python and auxiliary packages to help with Patroni and etcd - - ```{.bash data-prompt="$"} - $ sudo apt install python3-pip python3-dev binutils - ``` - -3. Install etcd, Patroni, pgBackRest packages: - - - ```{.bash data-prompt="$"} - $ sudo apt install percona-patroni \ - etcd etcd-server etcd-client \ - percona-pgbackrest - ``` - -4. Stop and disable all installed services: - - ```{.bash data-prompt="$"} - $ sudo systemctl stop {etcd,patroni,postgresql} - $ systemctl disable {etcd,patroni,postgresql} - ``` - -5. Even though Patroni can use an existing Postgres installation, remove the data directory to force it to initialize a new Postgres cluster instance. - - ```{.bash data-prompt="$"} - $ sudo systemctl stop postgresql - $ sudo rm -rf /var/lib/postgresql/16/main - ``` - -## Configure etcd distributed store - -The distributed configuration store provides a reliable way to store data that needs to be accessed by large scale distributed systems. The most popular implementation of the distributed configuration store is etcd. etcd is deployed as a cluster for fault-tolerance and requires an odd number of members (n/2+1) to agree on updates to the cluster state. An etcd cluster helps establish a consensus among nodes during a failover and manages the configuration for the three PostgreSQL instances. - -This document provides configuration for etcd version 3.5.x. For how to configure etcd cluster with earlier versions of etcd, read the blog post by _Fernando Laudares Camargos_ and _Jobin Augustine_ [PostgreSQL HA with Patroni: Your Turn to Test Failure Scenarios](https://www.percona.com/blog/postgresql-ha-with-patroni-your-turn-to-test-failure-scenarios/) - -If you [installed the software from tarballs](../tarball.md), check how you [enable etcd](../enable-extensions.md#etcd). - -The `etcd` cluster is first started in one node and then the subsequent nodes are added to the first node using the `add `command. - -!!! note - - Users with deeper understanding of how etcd works can configure and start all etcd nodes at a time and bootstrap the cluster using one of the following methods: - - * Static in the case when the IP addresses of the cluster nodes are known - * Discovery service - for cases when the IP addresses of the cluster are not known ahead of time. - - See the [How to configure etcd nodes simultaneously](../how-to.md#how-to-configure-etcd-nodes-simultaneously) section for details. - -### Configure `node1` - -1. Create the configuration file. You can edit the sample configuration file `/etc/etcd/etcd.conf.yaml` or create your own one. Replace the node name and IP address with the actual name and IP address of your node. - - ```yaml title="/etc/etcd/etcd.conf.yaml" - name: 'node1' - initial-cluster-token: PostgreSQL_HA_Cluster_1 - initial-cluster-state: new - initial-cluster: node1=http://10.104.0.1:2380 - data-dir: /var/lib/etcd - initial-advertise-peer-urls: http://10.104.0.1:2380 - listen-peer-urls: http://10.104.0.1:2380 - advertise-client-urls: http://10.104.0.1:2379 - listen-client-urls: http://10.104.0.1:2379 - ``` - -2. Start the `etcd` service to apply the changes on `node1`. - - ```{.bash data-prompt="$"} - $ sudo systemctl enable --now etcd - $ sudo systemctl start etcd - $ sudo systemctl status etcd - ``` - -3. Check the etcd cluster members on `node1`: - - ```{.bash data-prompt="$"} - $ sudo etcdctl member list - ``` - - Sample output: - - ```{.text .no-copy} - 21d50d7f768f153a: name=default peerURLs=http://10.104.0.1:2380 clientURLs=http://10.104.0.1:2379 isLeader=true - ``` - -4. Add the `node2` to the cluster. Run the following command on `node1`: - - ```{.bash data-prompt="$"} - $ sudo etcdctl member add node2 http://10.104.0.2:2380 - ``` - - ??? example "Sample output" - - ```{.text .no-copy} - Added member named node2 with ID 10042578c504d052 to cluster - - etcd_NAME="node2" - etcd_INITIAL_CLUSTER="node2=http://10.104.0.2:2380,node1=http://10.104.0.1:2380" - etcd_INITIAL_CLUSTER_STATE="existing" - ``` - -### Configure `node2` - -1. Create the configuration file. You can edit the sample configuration file `/etc/etcd/etcd.conf.yaml` or create your own one. Replace the node names and IP addresses with the actual names and IP addresses of your nodes. - - ```yaml title="/etc/etcd/etcd.conf.yaml" - name: 'node2' - initial-cluster-token: PostgreSQL_HA_Cluster_1 - initial-cluster-state: existing - initial-cluster: node1=http://10.104.0.1:2380,node2=http://10.104.0.2:2380 - data-dir: /var/lib/etcd - initial-advertise-peer-urls: http://10.104.0.2:2380 - listen-peer-urls: http://10.104.0.2:2380 - advertise-client-urls: http://10.104.0.2:2379 - listen-client-urls: http://10.104.0.2:2379 - ``` - -3. Start the `etcd` service to apply the changes on `node2`: - - ```{.bash data-prompt="$"} - $ sudo systemctl enable --now etcd - $ sudo systemctl start etcd - $ sudo systemctl status etcd - ``` - -### Configure `node3` - -1. Add `node3` to the cluster. **Run the following command on `node1`** - - ```{.bash data-prompt="$"} - $ sudo etcdctl member add node3 http://10.104.0.3:2380 - ``` - -2. On `node3`, create the configuration file. You can edit the sample configuration file `/etc/etcd/etcd.conf.yaml` or create your own one. Replace the node names and IP addresses with the actual names and IP addresses of your nodes. - - ```yaml title="/etc/etcd/etcd.conf.yaml" - name: 'node1' - initial-cluster-token: PostgreSQL_HA_Cluster_1 - initial-cluster-state: existing - initial-cluster: node1=http://10.104.0.1:2380,node2=http://10.104.0.2:2380,node3=http://10.104.0.3:2380 - data-dir: /var/lib/etcd - initial-advertise-peer-urls: http://10.104.0.3:2380 - listen-peer-urls: http://10.104.0.3:2380 - advertise-client-urls: http://10.104.0.3:2379 - listen-client-urls: http://10.104.0.3:2379 - ``` - -3. Start the `etcd` service to apply the changes. - - ```{.bash data-prompt="$"} - $ sudo systemctl enable --now etcd - $ sudo systemctl start etcd - $ sudo systemctl status etcd - ``` - -4. Check the etcd cluster members. - - ```{.bash data-prompt="$"} - $ sudo etcdctl member list - ``` - - ??? example "Sample output" - - ``` - 2d346bd3ae7f07c4: name=node2 peerURLs=http://10.104.0.2:2380 clientURLs=http://10.104.0.2:2379 isLeader=false - 8bacb519ebdee8db: name=node3 peerURLs=http://10.104.0.3:2380 clientURLs=http://10.104.0.3:2379 isLeader=false - c5f52ea2ade25e1b: name=node1 peerURLs=http://10.104.0.1:2380 clientURLs=http://10.104.0.1:2379 isLeader=true - ``` - -## Configure Patroni - -Run the following commands on all nodes. You can do this in parallel: - -1. Export and create environment variables to simplify the config file creation: - - * Node name: - - ```{.bash data-prompt="$"} - $ export NODE_NAME=`hostname -f` - ``` - - * Node IP: - - ```{.bash data-prompt="$"} - $ export NODE_IP=`hostname -i | awk '{print $1}'` - ``` - - * Create variables to store the PATH: - - ```bash - DATA_DIR="/var/lib/postgresql/16/main" - PG_BIN_DIR="/usr/lib/postgresql/16/bin" - ``` - - **NOTE**: Check the path to the data and bin folders on your operating system and change it for the variables accordingly. - - * Patroni information: - - ```bash - NAMESPACE="percona_lab" - SCOPE="cluster_1" - ``` - -2. Create the `/etc/patroni/patroni.yml` configuration file. Add the following configuration for `node1`: - - ```bash - echo " - namespace: ${NAMESPACE} - scope: ${SCOPE} - name: ${NODE_NAME} - - restapi: - listen: 0.0.0.0:8008 - connect_address: ${NODE_IP}:8008 - - etcd3: - host: ${NODE_IP}:2379 - - bootstrap: - # this section will be written into Etcd:///config after initializing new cluster - dcs: - ttl: 30 - loop_wait: 10 - retry_timeout: 10 - maximum_lag_on_failover: 1048576 - - postgresql: - use_pg_rewind: true - use_slots: true - parameters: - wal_level: replica - hot_standby: "on" - wal_keep_segments: 10 - max_wal_senders: 5 - max_replication_slots: 10 - wal_log_hints: "on" - logging_collector: 'on' - max_wal_size: '10GB' - archive_mode: "on" - archive_timeout: 600s - archive_command: "cp -f %p /home/postgres/archived/%f" - - # some desired options for 'initdb' - initdb: # Note: It needs to be a list (some options need values, others are switches) - - encoding: UTF8 - - data-checksums - - pg_hba: # Add following lines to pg_hba.conf after running 'initdb' - - host replication replicator 127.0.0.1/32 trust - - host replication replicator 0.0.0.0/0 md5 - - host all all 0.0.0.0/0 md5 - - host all all ::0/0 md5 - - # Some additional users which needs to be created after initializing new cluster - users: - admin: - password: qaz123 - options: - - createrole - - createdb - percona: - password: qaz123 - options: - - createrole - - createdb - - postgresql: - cluster_name: cluster_1 - listen: 0.0.0.0:5432 - connect_address: ${NODE_IP}:5432 - data_dir: ${DATA_DIR} - bin_dir: ${PG_BIN_DIR} - pgpass: /tmp/pgpass0 - authentication: - replication: - username: replicator - password: replPasswd - superuser: - username: postgres - password: qaz123 - parameters: - unix_socket_directories: "/var/run/postgresql/" - create_replica_methods: - - basebackup - basebackup: - checkpoint: 'fast' - - tags: - nofailover: false - noloadbalance: false - clonefrom: false - nosync: false - " | sudo tee -a /etc/patroni/patroni.yml - ``` - - ??? admonition "Patroni configuration file" - - Let’s take a moment to understand the contents of the `patroni.yml` file. - - The first section provides the details of the node and its connection ports. After that, we have the `etcd` service and its port details. - - Following these, there is a `bootstrap` section that contains the PostgreSQL configurations and the steps to run once the database is initialized. The `pg_hba.conf` entries specify all the other nodes that can connect to this node and their authentication mechanism. - - -3. Check that the systemd unit file `patroni.service` is created in `/etc/systemd/system`. If it is created, skip this step. - - If it's **not created**, create it manually and specify the following contents within: - - ```ini title="/etc/systemd/system/patroni.service" - [Unit] - Description=Runners to orchestrate a high-availability PostgreSQL - After=syslog.target network.target - - [Service] - Type=simple - - User=postgres - Group=postgres - - # Start the patroni process - ExecStart=/bin/patroni /etc/patroni/patroni.yml - - # Send HUP to reload from patroni.yml - ExecReload=/bin/kill -s HUP $MAINPID - - # only kill the patroni process, not its children, so it will gracefully stop postgres - KillMode=process - - # Give a reasonable amount of time for the server to start up/shut down - TimeoutSec=30 - - # Do not restart the service if it crashes, we want to manually inspect database on failure - Restart=no - - [Install] - WantedBy=multi-user.target - ``` - -4. Make systemd aware of the new service: - - ```{.bash data-prompt="$"} - $ sudo systemctl daemon-reload - ``` - -5. Now it's time to start Patroni. You need the following commands on all nodes but not in parallel. Start with the `node1` first, wait for the service to come to live, and then proceed with the other nodes one-by-one, always waiting for them to sync with the primary node: - - ```{.bash data-prompt="$"} - $ sudo systemctl enable --now patroni - $ sudo systemctl restart patroni - ``` - -When Patroni starts, it initializes PostgreSQL (because the service is not currently running and the data directory is empty) following the directives in the bootstrap section of the configuration file. - -6. Check the service to see if there are errors: - - ```{.bash data-prompt="$"} - $ sudo journalctl -fu patroni - ``` - - A common error is Patroni complaining about the lack of proper entries in the pg_hba.conf file. If you see such errors, you must manually add or fix the entries in that file and then restart the service. - - Changing the patroni.yml file and restarting the service will not have any effect here because the bootstrap section specifies the configuration to apply when PostgreSQL is first started in the node. It will not repeat the process even if the Patroni configuration file is modified and the service is restarted. - -7. Check the cluster: - - ```{.bash data-prompt="$"} - $ patronictl -c /etc/patroni/patroni.yml list $SCOPE - ``` - - The output on `node1` resembles the following: - - ```{.text .no-copy} - + Cluster: cluster_1 --+---------+---------+----+-----------+ - | Member | Host | Role | State | TL | Lag in MB | - +--------+-------------+---------+---------+----+-----------+ - | node-1 | 10.0.100.1 | Leader | running | 1 | | - +--------+-------------+---------+---------+----+-----------+ - ``` - - On the remaining nodes: - - ```{.text .no-copy} - + Cluster: cluster_1 --+---------+---------+----+-----------+ - | Member | Host | Role | State | TL | Lag in MB | - +--------+-------------+---------+---------+----+-----------+ - | node-1 | 10.0.100.1 | Leader | running | 1 | | - | node-2 | 10.0.100.2 | Replica | running | 1 | 0 | - +--------+-------------+---------+---------+----+-----------+ - ``` - -If Patroni has started properly, you should be able to locally connect to a PostgreSQL node using the following command: - -```{.bash data-prompt="$"} -$ sudo psql -U postgres -``` - -The command output is the following: - -``` -psql (16.0) -Type "help" for help. - -postgres=# -``` - -## Configure HAProxy - -HAproxy is the load balancer and the single point of entry to your PostgreSQL cluster for client applications. A client application accesses the HAPpoxy URL and sends its read/write requests there. Behind-the-scene, HAProxy routes write requests to the primary node and read requests - to the secondaries in a round-robin fashion so that no secondary instance is unnecessarily loaded. To make this happen, provide different ports in the HAProxy configuration file. In this deployment, writes are routed to port 5000 and reads - to port 5001 - -This way, a client application doesn’t know what node in the underlying cluster is the current primary. HAProxy sends connections to a healthy node (as long as there is at least one healthy node available) and ensures that client application requests are never rejected. - -1. Install HAProxy on the `HAProxy-demo` node: - - ```{.bash data-prompt="$"} - $ sudo apt install percona-haproxy - ``` - -2. The HAProxy configuration file path is: `/etc/haproxy/haproxy.cfg`. Specify the following configuration in this file. - - ``` - global - maxconn 100 - - defaults - log global - mode tcp - retries 2 - timeout client 30m - timeout connect 4s - timeout server 30m - timeout check 5s - - listen stats - mode http - bind *:7000 - stats enable - stats uri / - - listen primary - bind *:5000 - option httpchk /primary - http-check expect status 200 - default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions - server node1 node1:5432 maxconn 100 check port 8008 - server node2 node2:5432 maxconn 100 check port 8008 - server node3 node3:5432 maxconn 100 check port 8008 - - listen standbys - balance roundrobin - bind *:5001 - option httpchk /replica - http-check expect status 200 - default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions - server node1 node1:5432 maxconn 100 check port 8008 - server node2 node2:5432 maxconn 100 check port 8008 - server node3 node3:5432 maxconn 100 check port 8008 - ``` - - - HAProxy will use the REST APIs hosted by Patroni to check the health status of each PostgreSQL node and route the requests appropriately. - -3. Restart HAProxy: - - ```{.bash data-prompt="$"} - $ sudo systemctl restart haproxy - ``` - -4. Check the HAProxy logs to see if there are any errors: - - ```{.bash data-prompt="$"} - $ sudo journalctl -u haproxy.service -n 100 -f - ``` - -## Next steps - -[Configure pgBackRest](pgbackrest.md){.md-button} diff --git a/docs/solutions/ha-setup-yum.md b/docs/solutions/ha-setup-yum.md deleted file mode 100644 index d88fe1595..000000000 --- a/docs/solutions/ha-setup-yum.md +++ /dev/null @@ -1,571 +0,0 @@ -# Deploying PostgreSQL for high availability with Patroni on RHEL or CentOS - -This guide provides instructions on how to set up a highly available PostgreSQL cluster with Patroni on Red Hat Enterprise Linux or CentOS. - - -## Considerations - -1. This is an example deployment where etcd runs on the same host machines as the Patroni and PostgreSQL and there is a single dedicated HAProxy host. Alternatively etcd can run on different set of nodes. - - If etcd is deployed on the same host machine as Patroni and PostgreSQL, separate disk system for etcd and PostgreSQL is recommended due to performance reasons. - -2. For this setup, we use the nodes running on Red Hat Enterprise Linux 8 as the base operating system: - - | Node name | Application | IP address - |---------------|-------------------|-------------------- - | node1 | Patroni, PostgreSQL, etcd | 10.104.0.1 - | node2 | Patroni, PostgreSQL, etcd | 10.104.0.2 - | node3 | Patroni, PostgreSQL, etcd | 10.104.0.3 - | HAProxy-demo | HAProxy | 10.104.0.6 - - -!!! note - - We recommend not to expose the hosts/nodes where Patroni / etcd / PostgreSQL are running to public networks due to security risks. Use Firewalls, Virtual networks, subnets or the like to protect the database hosts from any kind of attack. - -## Initial setup - -### Set up hostnames in the `/etc/hosts` file - -It's not necessary to have name resolution, but it makes the whole setup more readable and less error prone. Here, instead of configuring a DNS, we use a local name resolution by updating the file `/etc/hosts`. By resolving their hostnames to their IP addresses, we make the nodes aware of each other's names and allow their seamless communication. - -1. Run the following command on each node. Change the node name to `node1`, `node2` and `node3` respectively: - - ```{.bash data-prompt="$"} - $ sudo hostnamectl set-hostname node-1 - ``` - -2. Modify the `/etc/hosts` file of each PostgreSQL node to include the hostnames and IP addresses of the remaining nodes. Add the following at the end of the `/etc/hosts` file on all nodes: - - === "node1" - - ```text hl_lines="3 4" - # Cluster IP and names - 10.104.0.1 node1 - 10.104.0.2 node2 - 10.104.0.3 node3 - ``` - - === "node2" - - ```text hl_lines="2 4" - # Cluster IP and names - 10.104.0.1 node1 - 10.104.0.2 node2 - 10.104.0.3 node3 - ``` - - === "node3" - - ```text hl_lines="2 3" - # Cluster IP and names - 10.104.0.1 node1 - 10.104.0.2 node2 - 10.104.0.3 node3 - ``` - - === "HAproxy-demo" - - The HAProxy instance should have the name resolution for all the three nodes in its `/etc/hosts` file. Add the following lines at the end of the file: - - ```text hl_lines="4 5 6" - # Cluster IP and names - 10.104.0.6 HAProxy-demo - 10.104.0.1 node1 - 10.104.0.2 node2 - 10.104.0.3 node3 - ``` - -### Install the software - -1. Install Percona Distribution for PostgreSQL on `node1`, `node2` and `node3` from Percona repository: - - * [Install `percona-release` :octicons-link-external-16:](https://www.percona.com/doc/percona-repo-config/installing.html). - * Enable the repository: - - ```{.bash data-prompt="$"} - $ sudo percona-release setup ppg16 - ``` - - * [Install Percona Distribution for PostgreSQL packages](../yum.md). - - !!! important - - **Don't** initialize the cluster and start the `postgresql` service. The cluster initialization and setup are handled by Patroni during the bootsrapping stage. - -2. Install some Python and auxiliary packages to help with Patroni and etcd - - ```{.bash data-prompt="$"} - $ sudo yum install python3-pip python3-devel binutils - ``` - -3. Install etcd, Patroni, pgBackRest packages. Check [platform specific notes for Patroni](../yum.md#for-percona-patroni-package): - - ```{.bash data-prompt="$"} - $ sudo yum install percona-patroni \ - etcd python3-python-etcd\ - percona-pgbackrest - ``` - -4. Stop and disable all installed services: - - ```{.bash data-prompt="$"} - $ sudo systemctl stop {etcd,patroni,postgresql} - $ systemctl disable {etcd,patroni,postgresql} - ``` - -## Configure etcd distributed store - -The distributed configuration store provides a reliable way to store data that needs to be accessed by large scale distributed systems. The most popular implementation of the distributed configuration store is etcd. etcd is deployed as a cluster for fault-tolerance and requires an odd number of members (n/2+1) to agree on updates to the cluster state. An etcd cluster helps establish a consensus among nodes during a failover and manages the configuration for the three PostgreSQL instances. - -This document provides configuration for etcd version 3.5.x. For how to configure etcd cluster with earlier versions of etcd, read the blog post by _Fernando Laudares Camargos_ and _Jobin Augustine_ [PostgreSQL HA with Patroni: Your Turn to Test Failure Scenarios](https://www.percona.com/blog/postgresql-ha-with-patroni-your-turn-to-test-failure-scenarios/). - -If you [installed the software from tarballs](../tarball.md), check how you [enable etcd](../enable-extensions.md#etcd). - -The `etcd` cluster is first started in one node and then the subsequent nodes are added to the first node using the `add `command. - -!!! note - - Users with deeper understanding of how etcd works can configure and start all etcd nodes at a time and bootstrap the cluster using one of the following methods: - - * Static in the case when the IP addresses of the cluster nodes are known - * Discovery service - for cases when the IP addresses of the cluster are not known ahead of time. - - See the [How to configure etcd nodes simultaneously](../how-to.md#how-to-configure-etcd-nodes-simultaneously) section for details. - -### Configure `node1` - -1. Create the configuration file. You can edit the sample configuration file `/etc/etcd/etcd.conf.yaml` or create your own one. Replace the node name and IP address with the actual name and IP address of your node. - - ```yaml title="/etc/etcd/etcd.conf.yaml" - name: 'node1' - initial-cluster-token: PostgreSQL_HA_Cluster_1 - initial-cluster-state: new - initial-cluster: node1=http://10.104.0.1:2380 - data-dir: /var/lib/etcd - initial-advertise-peer-urls: http://10.104.0.1:2380 - listen-peer-urls: http://10.104.0.1:2380 - advertise-client-urls: http://10.104.0.1:2379 - listen-client-urls: http://10.104.0.1:2379 - ``` - -2. Enable and start the `etcd` service to apply the changes on `node1`. - - ```{.bash data-prompt="$"} - $ sudo systemctl enable --now etcd - $ sudo systemctl status etcd - ``` - -3. Check the etcd cluster members on `node1`: - - ```{.bash data-prompt="$"} - $ sudo etcdctl member list - ``` - - Sample output: - - ```{.text .no-copy} - 21d50d7f768f153a: name=default peerURLs=http://10.104.0.1:2380 clientURLs=http://10.104.0.1:2379 isLeader=true - ``` - -4. Add the `node2` to the cluster. Run the following command on `node1`: - - ```{.bash data-prompt="$"} - $ sudo etcdctl member add node2 http://10.104.0.2:2380 - ``` - - ??? example "Sample output" - - ```{.text .no-copy} - Added member named node2 with ID 10042578c504d052 to cluster - - etcd_NAME="node2" - etcd_INITIAL_CLUSTER="node2=http://10.104.0.2:2380,node1=http://10.104.0.1:2380" - etcd_INITIAL_CLUSTER_STATE="existing" - ``` - -### Configure `node2` - -1. Create the configuration file. You can edit the sample configuration file `/etc/etcd/etcd.conf.yaml` or create your own one. Replace the node names and IP addresses with the actual names and IP addresses of your nodes. - - ```yaml title="/etc/etcd/etcd.conf.yaml" - name: 'node2' - initial-cluster-token: PostgreSQL_HA_Cluster_1 - initial-cluster-state: existing - initial-cluster: node1=http://10.104.0.1:2380,node2=http://10.104.0.2:2380 - data-dir: /var/lib/etcd - initial-advertise-peer-urls: http://10.104.0.2:2380 - listen-peer-urls: http://10.104.0.2:2380 - advertise-client-urls: http://10.104.0.2:2379 - listen-client-urls: http://10.104.0.2:2379 - ``` - -3. Enable and start the `etcd` service to apply the changes on `node2`: - - ```{.bash data-prompt="$"} - $ sudo systemctl enable --now etcd - $ sudo systemctl status etcd - ``` - -### Configure `node3` - -1. Add `node3` to the cluster. **Run the following command on `node1`** - - ```{.bash data-prompt="$"} - $ sudo etcdctl member add node3 http://10.104.0.3:2380 - ``` - -2. On `node3`, create the configuration file. You can edit the sample configuration file `/etc/etcd/etcd.conf.yaml` or create your own one. Replace the node names and IP addresses with the actual names and IP addresses of your nodes. - - ```yaml title="/etc/etcd/etcd.conf.yaml" - name: 'node1' - initial-cluster-token: PostgreSQL_HA_Cluster_1 - initial-cluster-state: existing - initial-cluster: node1=http://10.104.0.1:2380,node2=http://10.104.0.2:2380,node3=http://10.104.0.3:2380 - data-dir: /var/lib/etcd - initial-advertise-peer-urls: http://10.104.0.3:2380 - listen-peer-urls: http://10.104.0.3:2380 - advertise-client-urls: http://10.104.0.3:2379 - listen-client-urls: http://10.104.0.3:2379 - ``` - -3. Enable and start the `etcd` service to apply the changes. - - ```{.bash data-prompt="$"} - $ sudo systemctl enable --now etcd - $ sudo systemctl status etcd - ``` - -4. Check the etcd cluster members. - - ```{.bash data-prompt="$"} - $ sudo etcdctl member list - ``` - - ??? example "Sample output" - - ``` - 2d346bd3ae7f07c4: name=node2 peerURLs=http://10.104.0.2:2380 clientURLs=http://10.104.0.2:2379 isLeader=false - 8bacb519ebdee8db: name=node3 peerURLs=http://10.104.0.3:2380 clientURLs=http://10.104.0.3:2379 isLeader=false - c5f52ea2ade25e1b: name=node1 peerURLs=http://10.104.0.1:2380 clientURLs=http://10.104.0.1:2379 isLeader=true - ``` - -## Configure Patroni - -Run the following commands on all nodes. You can do this in parallel: - -1. Export and create environment variables to simplify the config file creation: - - * Node name: - - ```{.bash data-prompt="$"} - $ export NODE_NAME=`hostname -f` - ``` - - * Node IP: - - ```{.bash data-prompt="$"} - $ export NODE_IP=`hostname -i | awk '{print $1}'` - ``` - - * Create variables to store the PATH: - - ```bash - DATA_DIR="/var/lib/pgsql/data/" - PG_BIN_DIR="/usr/pgsql-16/bin" - ``` - - **NOTE**: Check the path to the data and bin folders on your operating system and change it for the variables accordingly. - - * Patroni information: - - ```bash - NAMESPACE="percona_lab" - SCOPE="cluster_1 - ``` - -2. Create the directories required by Patroni - - * Create the directory to store the configuration file and make it owned by the `postgres` user. - - ```{.bash data-prompt="$"} - $ sudo mkdir -p /etc/patroni/ - $ sudo chown -R postgres:postgres /etc/patroni/ - ``` - - * Create the data directory to store PostgreSQL data. Change its ownership to the `postgres` user and restrict the access to it - - ```{.bash data-prompt="$"} - $ sudo mkdir /data/pgsql -p - $ sudo chown -R postgres:postgres /data/pgsql - $ sudo chmod 700 /data/pgsql - ``` - -3. Create the `/etc/patroni/patroni.yml` configuration file. Add the following configuration: - - ```bash - echo " - namespace: ${NAMESPACE} - scope: ${SCOPE} - name: ${NODE_NAME} - - restapi: - listen: 0.0.0.0:8008 - connect_address: ${NODE_IP}:8008 - - etcd3: - host: ${NODE_IP}:2379 - - bootstrap: - # this section will be written into Etcd:///config after initializing new cluster - dcs: - ttl: 30 - loop_wait: 10 - retry_timeout: 10 - maximum_lag_on_failover: 1048576 - - postgresql: - use_pg_rewind: true - use_slots: true - parameters: - wal_level: replica - hot_standby: "on" - wal_keep_segments: 10 - max_wal_senders: 5 - max_replication_slots: 10 - wal_log_hints: "on" - logging_collector: 'on' - max_wal_size: '10GB' - archive_mode: "on" - archive_timeout: 600s - archive_command: "cp -f %p /home/postgres/archived/%f" - - # some desired options for 'initdb' - initdb: # Note: It needs to be a list (some options need values, others are switches) - - encoding: UTF8 - - data-checksums - - pg_hba: # Add following lines to pg_hba.conf after running 'initdb' - - host replication replicator 127.0.0.1/32 trust - - host replication replicator 0.0.0.0/0 md5 - - host all all 0.0.0.0/0 md5 - - host all all ::0/0 md5 - - # Some additional users which needs to be created after initializing new cluster - users: - admin: - password: qaz123 - options: - - createrole - - createdb - percona: - password: qaz123 - options: - - createrole - - createdb - - postgresql: - cluster_name: cluster_1 - listen: 0.0.0.0:5432 - connect_address: ${NODE_IP}:5432 - data_dir: ${DATA_DIR} - bin_dir: ${PG_BIN_DIR} - pgpass: /tmp/pgpass0 - authentication: - replication: - username: replicator - password: replPasswd - superuser: - username: postgres - password: qaz123 - parameters: - unix_socket_directories: "/var/run/postgresql/" - create_replica_methods: - - basebackup - basebackup: - checkpoint: 'fast' - - tags: - nofailover: false - noloadbalance: false - clonefrom: false - nosync: false - " | sudo tee -a /etc/patroni/patroni.yml - ``` - -4. Check that the systemd unit file `patroni.service` is created in `/etc/systemd/system`. If it is created, skip this step. - - If it's **not** created, create it manually and specify the following contents within: - - ```ini title="/etc/systemd/system/patroni.service" - [Unit] - Description=Runners to orchestrate a high-availability PostgreSQL - After=syslog.target network.target - - [Service] - Type=simple - - User=postgres - Group=postgres - - # Start the patroni process - ExecStart=/bin/patroni /etc/patroni/patroni.yml - - # Send HUP to reload from patroni.yml - ExecReload=/bin/kill -s HUP $MAINPID - - # only kill the patroni process, not its children, so it will gracefully stop postgres - KillMode=process - - # Give a reasonable amount of time for the server to start up/shut down - TimeoutSec=30 - - # Do not restart the service if it crashes, we want to manually inspect database on failure - Restart=no - - [Install] - WantedBy=multi-user.target - ``` - -5. Make `systemd` aware of the new service: - - ```{.bash data-prompt="$"} - $ sudo systemctl daemon-reload - ``` - -6. Now it's time to start Patroni. You need the following commands on all nodes but not in parallel. Start with the `node1` first, wait for the service to come to live, and then proceed with the other nodes one-by-one, always waiting for them to sync with the primary node: - - ```{.bash data-prompt="$"} - $ sudo systemctl enable --now patroni - $ sudo systemctl restart patroni - ``` - - When Patroni starts, it initializes PostgreSQL (because the service is not currently running and the data directory is empty) following the directives in the bootstrap section of the configuration file. - -7. Check the service to see if there are errors: - - ```{.bash data-prompt="$"} - $ sudo journalctl -fu patroni - ``` - - A common error is Patroni complaining about the lack of proper entries in the pg_hba.conf file. If you see such errors, you must manually add or fix the entries in that file and then restart the service. - - Changing the patroni.yml file and restarting the service will not have any effect here because the bootstrap section specifies the configuration to apply when PostgreSQL is first started in the node. It will not repeat the process even if the Patroni configuration file is modified and the service is restarted. - - If Patroni has started properly, you should be able to locally connect to a PostgreSQL node using the following command: - - ```{.bash data-prompt="$"} - $ sudo psql -U postgres - - psql (16.0) - Type "help" for help. - - postgres=# - ``` - -8. When all nodes are up and running, you can check the cluster status using the following command: - - ```{.bash data-prompt="$"} - $ sudo patronictl -c /etc/patroni/patroni.yml list - ``` - - The output on `node1` resembles the following: - - ```{.text .no-copy} - + Cluster: cluster_1 --+---------+---------+----+-----------+ - | Member | Host | Role | State | TL | Lag in MB | - +--------+-------------+---------+---------+----+-----------+ - | node-1 | 10.0.100.1 | Leader | running | 1 | | - +--------+-------------+---------+---------+----+-----------+ - ``` - - On the remaining nodes: - - ```{.text .no-copy} - + Cluster: cluster_1 --+---------+---------+----+-----------+ - | Member | Host | Role | State | TL | Lag in MB | - +--------+-------------+---------+---------+----+-----------+ - | node-1 | 10.0.100.1 | Leader | running | 1 | | - | node-2 | 10.0.100.2 | Replica | running | 1 | 0 | - +--------+-------------+---------+---------+----+-----------+ - ``` - -## Configure HAProxy - -HAproxy is the load balancer and the single point of entry to your PostgreSQL cluster for client applications. A client application accesses the HAPpoxy URL and sends its read/write requests there. Behind-the-scene, HAProxy routes write requests to the primary node and read requests - to the secondaries in a round-robin fashion so that no secondary instance is unnecessarily loaded. To make this happen, provide different ports in the HAProxy configuration file. In this deployment, writes are routed to port 5000 and reads - to port 5001 - -This way, a client application doesn’t know what node in the underlying cluster is the current primary. HAProxy sends connections to a healthy node (as long as there is at least one healthy node available) and ensures that client application requests are never rejected. - -1. Install HAProxy on the `HAProxy-demo` node: - - ```{.bash data-prompt="$"} - $ sudo yum install percona-haproxy - ``` - -2. The HAProxy configuration file path is: `/etc/haproxy/haproxy.cfg`. Specify the following configuration in this file. - - ``` - global - maxconn 100 - - defaults - log global - mode tcp - retries 2 - timeout client 30m - timeout connect 4s - timeout server 30m - timeout check 5s - - listen stats - mode http - bind *:7000 - stats enable - stats uri / - - listen primary - bind *:5000 - option httpchk /primary - http-check expect status 200 - default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions - server node1 node1:5432 maxconn 100 check port 8008 - server node2 node2:5432 maxconn 100 check port 8008 - server node3 node3:5432 maxconn 100 check port 8008 - - listen standbys - balance roundrobin - bind *:5001 - option httpchk /replica - http-check expect status 200 - default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions - server node1 node1:5432 maxconn 100 check port 8008 - server node2 node2:5432 maxconn 100 check port 8008 - server node3 node3:5432 maxconn 100 check port 8008 - ``` - - - HAProxy will use the REST APIs hosted by Patroni to check the health status of each PostgreSQL node and route the requests appropriately. - -3. Enable a SELinux boolean to allow HAProxy to bind to non standard ports: - - ```{.bash data-prompt="$"} - $ sudo setsebool -P haproxy_connect_any on - ``` - -4. Restart HAProxy: - - ```{.bash data-prompt="$"} - $ sudo systemctl restart haproxy - ``` - -5. Check the HAProxy logs to see if there are any errors: - - ```{.bash data-prompt="$"} - $ sudo journalctl -u haproxy.service -n 100 -f - ``` - -## Next steps - -[Configure pgBackRest](pgbackrest.md){.md-button} - - diff --git a/docs/solutions/haproxy-info.md b/docs/solutions/haproxy-info.md new file mode 100644 index 000000000..8c2ae5c89 --- /dev/null +++ b/docs/solutions/haproxy-info.md @@ -0,0 +1,77 @@ +# HAProxy + +HAProxy (High Availability Proxy) is a powerful, open-source load balancer and +proxy server used to improve the performance and reliability of web services by +distributing network traffic across multiple servers. It is widely used to enhance the scalability, availability, and reliability of web applications by balancing client requests among backend servers. + +HAProxy architecture is +optimized to move data as fast as possible with the least possible operations. +It focuses on optimizing the CPU cache's efficiency by sticking connections to +the same CPU as long as possible. + +## How HAProxy works + +HAProxy operates as a reverse proxy, which means it accepts client requests and distributes them to one or more backend servers using the configured load-balancing algorithm. This ensures efficient use of server resources and prevents any single server from becoming overloaded. + +- **Client request processing**: + + 1. A client application connects to HAProxy instead of directly to the server. + 2. HAProxy analyzes the requests and determines what server to route it to for further processing. + 3. HAProxy forwards the request to the selected server using the routing algorithm defined in its configuration. It can be round robin, least connections, and others. + 4. HAProxy receives the response from the server and forwards it back to the client. + 5. After sending the response, HAProxy either closes the connection or keeps it open, depending on the configuration. + +- **Load balancing**: HAProxy distributes incoming traffic using various algorithms such as round-robin, least connections, and IP hash. +- **Health checks**: HAProxy continuously monitors the health of backend servers to ensure requests are only routed to healthy servers. +- **SSL termination**: HAProxy offloads SSL/TLS encryption and decryption, reducing the workload on backend servers. +- **Session persistence**: HAProxy ensures that requests from the same client are routed to the same server for session consistency. +- **Traffic management**: HAProxy supports rate limiting, request queuing, and connection pooling for optimal resource utilization. +- **Security**: HAProxy supports SSL/TLS, IP filtering, and integration with Web Application Firewalls (WAF). + +## Role in a HA Patroni cluster + +HAProxy plays a crucial role in managing PostgreSQL high availability in a Patroni cluster. Patroni is an open-source tool that automates PostgreSQL cluster management, including failover and replication. HAProxy acts as a load balancer and proxy, distributing client connections across the cluster nodes. + +Client applications connect to HAProxy, which transparently forwards their requests to the appropriate PostgreSQL node. This ensures that clients always connect to the active primary node without needing to know the cluster's internal state and topology. + +HAProxy monitors the health of PostgreSQL nodes using Patroni's API and routes traffic to the primary node. If the primary node fails, Patroni promotes a secondary node to a new primary, and HAProxy updates its routing to reflect the change. You can configure HAProxy to route write requests to the primary node and read requests - to the secondary nodes. + +## Redundancy for HAProxy + +A single HAProxy node creates a single point of failure. If HAProxy goes down, clients lose connection to the cluster. To prevent this, set up multiple HAProxy instances with a failover mechanism. This way, if one instance fails, another takes over automatically. + +To implement HAProxy redundancy: + +1. Set up a virtual IP address that can move between HAProxy instances. + +2. Install and configure a failover mechanism to monitor HAProxy instances and move the virtual IP to a backup if the primary fails. + +3. Keep HAProxy configurations synchronized across all instances. + +!!! note + + In this reference architecture we focus on the on-premises deployment and use Keepalived as the failover mechanism. + + If you use a cloud infrastructure, it may be easier to use the load balancer provided by the cloud provider to achieve high-availability for HAProxy. + +## How Keepalived works + +Keepalived manages failover by moving the virtual IP to a backup HAProxy node when the primary fails. + +No matter how many HAProxy nodes you have, only one of them can be a primary and have the MASTER state. All other nodes are BACKUP nodes. They monitor the MASTER state and take over when it is down. + +To determine the MASTER, Keepalived uses the `priority` setting. Every node must have a different priority. + +The node with the highest priority becomes the MASTER. Keepalived periodically checks every node's health. + +When the MASTER node is down or unavailable, it's priority is lowered so that the next highest priority node becomes the new MASTER and takes over. The priority is adjusted by the value you define in the `weight` setting. + +You must carefully define the `priority` and `weight` values in the configuration. When a primary node is down, its priority must be adjusted to be lower than the active node with the lowest priority by at least 1. + +For example, your nodes have priority 110 and 100. The node with priority 110 is MASTER. When it is down, its priority must be lower than the priority of the remaining node (100). + +When a failed node restores, its priority adjusts again. If it is the highest one among the nodes, this node restores its MASTER state, holds the virtual IP address and handles the client connections. + +## Next steps + +[pgBackRest](pgbackrest-info.md){.md-button} \ No newline at end of file diff --git a/docs/solutions/high-availability.md b/docs/solutions/high-availability.md index f79e3a1b5..b9bc79502 100644 --- a/docs/solutions/high-availability.md +++ b/docs/solutions/high-availability.md @@ -1,95 +1,119 @@ # High Availability in PostgreSQL with Patroni -PostgreSQL has been widely adopted as a modern, high-performance transactional database. A highly available PostgreSQL cluster can withstand failures caused by network outages, resource saturation, hardware failures, operating system crashes or unexpected reboots. Such cluster is often a critical component of the enterprise application landscape, where [four nines of availability :octicons-link-external-16:](https://en.wikipedia.org/wiki/High_availability#Percentage_calculation) is a minimum requirement. +Whether you are a small startup or a big enterprise, downtime of your services may cause severe consequences, such as loss of customers, impact on your reputation, and penalties for not meeting the Service Level Agreements (SLAs). That’s why ensuring a highly-available deployment is crucial. -There are several methods to achieve high availability in PostgreSQL. This solution document provides [Patroni](#patroni) - the open-source extension to facilitate and manage the deployment of high availability in PostgreSQL. +But what does it mean, high availability (HA)? And how to achieve it? This document answers these questions. -??? admonition "High availability methods" +After reading this document, you will learn the following: - There are several native methods for achieving high availability with PostgreSQL: +* [what is high availability](#what-is-high-availability) +* the recommended [reference architecture](ha-architecture.md) to achieve it +* how to deploy it using our step-by-step deployment guides for each component. The deployment instructions focus on the minimalistic approach to high availability that we recommend. It also gives instructions how to deploy additional components that you can add when your infrastructure grows. +* how to verify that your high availability deployment works as expected, providing replication and failover with the [testing guidelines](ha-test.md) +* additional components that you can add to address existing limitations on to your infrastructure. An example of such limitations can be the ones on application driver/connectors, or the lack of the connection pooler at the application framework. - - shared disk failover, - - file system replication, - - trigger-based replication, - - statement-based replication, - - logical replication, - - Write-Ahead Log (WAL) shipping, and - - [streaming replication](#streaming-replication) +## What is high availability +High availability (HA) is the ability of the system to operate continuously without the interruption of services. During the outage, the system must be able to transfer the services from the failed component to the healthy ones so that they can take over its responsibility. The system must have sufficient automation to perform this transfer without the need of human intervention, minimizing disruption and avoiding the need for human intervention. - ## Streaming replication +Overall, High availability is about: - Streaming replication is part of Write-Ahead Log shipping, where changes to the WALs are immediately made available to standby replicas. With this approach, a standby instance is always up-to-date with changes from the primary node and can assume the role of primary in case of a failover. +1. Reducing the chance of failures +2. Elimination of single-point-of-failure (SPOF) +3. Automatic detection of failures +4. Automatic action to reduce the impact +### How to achieve it? - ### Why native streaming replication is not enough +A short answer is: add redundancy to your deployment, eliminate a single point of failure (SPOF) and have the mechanism to transfer the services from a failed member to the healthy one. - Although the native streaming replication in PostgreSQL supports failing over to the primary node, it lacks some key features expected from a truly highly-available solution. These include: +* Although the native streaming replication in PostgreSQL supports failing over to the primary node, it lacks some key features expected from a truly highly-available solution. These include: +For a long answer, let's break it down into steps. +#### Step 1. Replication - * No consensus-based promotion of a “leader” node during a failover - * No decent capability for monitoring cluster status - * No automated way to bring back the failed primary node to the cluster - * A manual or scheduled switchover is not easy to manage +First, you should have more than one copy of your data. This means, you need to have several instances of your database where one is the primary instance that accepts reads and writes. Other instances are replicas – they must have an up-to-date copy of the data from the primary and remain in sync with it. They may also accept reads to offload your primary. - To address these shortcomings, there are a multitude of third-party, open-source extensions for PostgreSQL. The challenge for a database administrator here is to select the right utility for the current scenario. +You must deploy these instances on separate hardware (servers or nodes) and use a separate storage for storing the data. This way you eliminate a single point of failure for your database. - Percona Distribution for PostgreSQL solves this challenge by providing the [Patroni :octicons-link-external-16:](https://patroni.readthedocs.io/en/latest/) extension for achieving PostgreSQL high availability. +The minimum number of database nodes is two: one primary and one replica. -## Patroni +The recommended deployment is a three-instance cluster consisting of one primary and two replica nodes. The replicas receive the data via the replication mechanism. -[Patroni :octicons-link-external-16:](https://patroni.readthedocs.io/en/latest/) is a template for you to create your own customized, high-availability solution using Python and - for maximum accessibility - a distributed configuration store like ZooKeeper, etcd, Consul or Kubernetes. +![Primary-replica setup](../_images/diagrams/ha-overview-replication.svg) -### Key benefits of Patroni: +PostgreSQL natively supports logical and streaming replication. To achieve high availability, use streaming replication to ensure an exact copy of data is maintained and is ready to take over, while reducing the delay between primary and replica nodes to prevent data loss. -* Continuous monitoring and automatic failover -* Manual/scheduled switchover with a single command -* Built-in automation for bringing back a failed node to cluster again. -* REST APIs for entire cluster configuration and further tooling. -* Provides infrastructure for transparent application failover -* Distributed consensus for every action and configuration. -* Integration with Linux watchdog for avoiding split-brain syndrome. +#### Step 2. Switchover and Failover +You may want to transfer the primary role from one machine to another. This action is called a **manual switchover**. A reason for that could be the following: -!!! admonition "See also" +* a planned maintenance on the OS level, like applying quarterly security updates or replacing some of the end-of-life components from the server +* troubleshooting some of the problems, like high network latency. - - [Patroni documentation :octicons-link-external-16:](https://patroni.readthedocs.io/en/latest/SETTINGS.html#settings) +Switchover is a manual action performed when you decide to transfer the primary role to another node. The high-availability framework makes this process easier and helps minimize downtime during maintenance, thereby improving overall availability. - - Percona Blog: +There could be an unexpected situation where a primary node is down or not responding. Reasons for that can be different, from hardware or network issues to software failures, power outages and the like. In such situations, the high-availability solution should automatically detect the problem, find out a suitable candidate from the remaining nodes and transfer the primary role to the best candidate (promote a new node to become a primary). Such automatic remediation is called **Failover**. - - [PostgreSQL HA with Patroni: Your Turn to Test Failure Scenarios :octicons-link-external-16:](https://www.percona.com/blog/2021/06/11/postgresql-ha-with-patroni-your-turn-to-test-failure-scenarios/) +![Failover](../_images/diagrams/ha-overview-failover.svg) -## Architecture layout +You can do a manual failover when automatic remediation fails, for example, due to: -The following diagram shows the architecture of a three-node PostgreSQL cluster with a single-leader node. +* a complete network partitioning +* high-availability framework not being able to find a good candidate +* the insufficient number of nodes remaining for a new primary election. -![Architecture of the three-node, single primary PostgreSQL cluster](../_images/diagrams/ha-architecture-patroni.png) +The high-availability framework allows a human operator / administrator to take control and do a manual failover. -### Components +#### Step 3. Connection routing and load balancing -The components in this architecture are: +Instead of a single node you now have a cluster. How to enable users to connect to the cluster and ensure they always connect to the correct node, especially when the primary node changes? -- PostgreSQL nodes -- Patroni - a template for configuring a highly available PostgreSQL cluster. +One option is to configure a DNS resolution that resolves the IPs of all cluster nodes. A drawback here is that only the primary node accepts all requests. When your system grows, so does the load and it may lead to overloading the primary node and result in performance degradation. -- etcd - a Distributed Configuration store that stores the state of the PostgreSQL cluster. +You can write your application to send read/write requests to the primary and read-only requests to the secondary nodes. This requires significant programming experience. -- HAProxy - the load balancer for the cluster and is the single point of entry to client applications. +![Load-balancer](../_images/diagrams/ha-overview-load-balancer.svg) -- pgBackRest - the backup and restore solution for PostgreSQL +Another option is to use a load-balancing proxy. Instead of connecting directly to the IP address of the primary node, which can change during a failover, you use a proxy that acts as a single point of entry for the entire cluster. This proxy provides the IP address visible for user applications. It also knows which node is currently the primary and directs all incoming write requests to it. At the same time, it can distribute read requests among the replicas to evenly spread the load and improve performance. -- Percona Monitoring and Management (PMM) - the solution to monitor the health of your cluster +To eliminate a single point of failure for a load balancer, we recommend to deploy multiple connection routers/proxies for redundancy. Each application server can have its own connection router whose task is to identify the cluster topology and route the traffic to the current primary node. -### How components work together +Alternatively you can deploy a redundant load balancer for the whole cluster. The load balancer instances share the public IP address so that it can "float" from one instance to another in the case of a failure. To control the load balancer's state and transfer the IP address to the active instance, you also need the failover solution for load balancers. -Each PostgreSQL instance in the cluster maintains consistency with other members through streaming replication. Each instance hosts Patroni - a cluster manager that monitors the cluster health. Patroni relies on the operational etcd cluster to store the cluster configuration and sensitive data about the cluster health there. +The use of a load balancer is optional. If your application implements the logic of connection routing and load-balancing, it is a highly-recommended approach. -Patroni periodically sends heartbeat requests with the cluster status to etcd. etcd writes this information to disk and sends the response back to Patroni. If the current primary fails to renew its status as leader within the specified timeout, Patroni updates the state change in etcd, which uses this information to elect the new primary and keep the cluster up and running. +#### Step 4. Backups -The connections to the cluster do not happen directly to the database nodes but are routed via a connection proxy like HAProxy. This proxy determines the active node by querying the Patroni REST API. +Even with replication and failover mechanisms in place, it’s crucial to have regular backups of your data. Backups provide a safety net for catastrophic failures that affect both the primary and replica nodes. While replication ensures data is synchronized across multiple nodes, it does not protect against data corruption, accidental deletions, or malicious attacks that can affect all nodes. -## Next steps +![Backup tool](../_images/diagrams/ha-overview-backup.svg) + +Having regular backups ensures that you can restore your data to a previous state, preserving data integrity and availability even in the worst-case scenarios. Store your backups in separate, secure locations and regularly test them to ensure that you can quickly and accurately restore them when needed. This additional layer of protection is essential to maintaining continuous operation and minimizing data loss. + +The backup tool is optional but highly-recommended for data corruption recovery. Additionally, backups protect against human error, when a user can accidentally drop a table or make another mistake. + +As a result, you end up with the following components for a minimalistic highly-available deployment: + +* A minimum two-node PostgreSQL cluster with the replication configured among nodes. The recommended minimalistic cluster is a three-node one. +* A solution to manage the cluster and perform automatic failover when the primary node is down. +* (Optional but recommended) A load-balancing proxy that provides a single point of entry to your cluster and distributes the load across cluster nodes. You need at least two instances of a load-balancing proxy and a failover tool to eliminate a single point of failure. +* (Optional but recommended) A backup and restore solution to protect data against loss, corruption and human error. + +Optionally, you can add a monitoring tool to observe the health of your deployment, receive alerts about performance issues and timely react to them. + +### What tools to use? -[Deploy on Debian or Ubuntu](ha-setup-apt.md){.md-button} -[Deploy on RHEL or derivatives](ha-setup-yum.md){.md-button} +The PostgreSQL ecosystem offers many tools for high availability, but choosing the right ones can be challenging. At Percona, we have carefully selected and tested open-source tools to ensure they work well together and help you achieve high availability. + +In our [reference architecture](ha-architecture.md) section we recommend a combination of open-source tools, focusing on a minimalistic three-node PostgreSQL cluster. + +Note that the tools are recommended but not mandatory. You can use your own solutions and alternatives if they better meet your business needs. However, in this case, we cannot guarantee their compatibility and smooth operation. + +### Additional reading + +[Measuring high availability](ha-measure.md){.md-button} + +## Next steps +[Architecture :material-arrow-right:](ha-architecture.md){.md-button} diff --git a/docs/solutions/patroni-info.md b/docs/solutions/patroni-info.md new file mode 100644 index 000000000..b88d0cfa7 --- /dev/null +++ b/docs/solutions/patroni-info.md @@ -0,0 +1,84 @@ +# Patroni + +Patroni is an open-source tool designed to manage and automate the high availability (HA) of PostgreSQL clusters. It ensures that your PostgreSQL database remains available even in the event of hardware failures, network issues or other disruptions. Patroni achieves this by using distributed consensus stores like ETCD, Consul, or ZooKeeper to manage cluster state and automate failover processes. We'll use [`etcd`](etcd-info.md) in our architecture. + +## Key benefits of Patroni for high availability + +- Automated failover and promotion of a new primary in case of a failure; +- Prevention and protection from split-brain scenarios (where two nodes believe they are the primary and both accept transactions). Split-brain can lead to serious logical corruptions such as wrong, duplicate data or data loss, and to associated business loss and risk of litigation; +- Simplifying the management of PostgreSQL clusters across multiple data centers; +- Self-healing via automatic restarts of failed PostgreSQL instances or reinitialization of broken replicas. +- Integration with tools like `pgBackRest`, `HAProxy`, and monitoring systems for a complete HA solution. + +## How Patroni works + +Patroni uses the `etcd` distributed consensus store to coordinate the state of a PostgreSQL cluster for the following operations: + +1. Cluster state management: + + - After a user installs and configures Patroni, Patroni takes over the PostgreSQL service administration and configuration; + - Patroni maintains the cluster state data such as PostgreSQL configuration, information about which node is the primary and which are replicas, and their health status. + - Patroni manages PostgreSQL configuration files such as`postgresql.conf` and `pg_hba.conf` dynamically, ensuring consistency across the cluster. + - A Patroni agent runs on each cluster node and communicates with `etcd` and other nodes. + +2. Primary node election: + + - Patroni initiates a primary election process after the cluster is initialized; + - Patroni initiates a failover process if the primary node fails; + - When the old primary is recovered, it rejoins the cluster as a new replica; + - Every new node added to the cluster joins it as a new replica; + - `etcd` and the Raft consensus algorithm ensures that only one node is elected as the new primary, preventing split-brain scenarios. + +3. Automatic failover: + + - If the primary node becomes unavailable, Patroni initiates a new primary election process with the most up-to-date replicas; + - When a node is elected it is automatically promoted to primary; + - Patroni updates the `etcd` consensus store and reconfigures the remaining replicas to follow the new primary. + +4. Health checks: + + - Patroni continuously monitors the health of all PostgreSQL instances; + - If a node fails or becomes unreachable, Patroni takes corrective actions by restarting PostgreSQL or initiating a failover process. + +## Split-brain prevention + +Split-brain is an issue, which occurs when two or more nodes believe they are the primary, leading to data inconsistencies. + +Patroni prevents split-brain by using a three-layer protection and prevention mechanism where the `etcd` distributed locking mechanism plays a key role: + +* At the Patroni layer, a node needs to acquire a leader key in the race before promoting itself as the primary. If the node cannot to renew its leader key, Patroni demotes it to a replica. +* The `etcd` layer uses the Raft consensus algorithm to allow only one node to acquire the leader key. +- At the OS and hardware layers, Patroni uses Linux Watchdog to perform [STONITH](https://en.wikipedia.org/wiki/Fencing_(computing)#STONITH) / fencing and terminate a PostgreSQL instance to prevent a split-brain scenario. + +One important aspect of how Patroni works is that it requires a quorum (the majority) of nodes to agree on the cluster state, preventing isolated nodes from becoming a primary. The quorum strengthens Patroni's capabilities of preventing split-brain. + +## Watchdog + +Patroni can use a watchdog mechanism to improve resilience. But what is watchdog? + +A watchdog is a mechanism that ensures a system can recover from critical failures. In the context of Patroni, a watchdog is used to forcibly restart the node and terminate a failed primary node to prevent split-brain scenarios. + +While Patroni itself is designed for high availability, a watchdog provides an extra layer of protection against system-level failures that Patroni might not be able to detect, such as kernel panics or hardware lockups. If the entire operating system becomes unresponsive, Patroni might not be able to function correctly. The watchdog operates independently so it can detect that the server is unresponsive and reset it, bringing it back to a known good state. + +Watchdog adds an extra layer of safety, because it helps protecting against scenarios where the `etcd` consensus store is unavailable or network partitions occur. + +There are 2 types of watchdogs: + +- Hardware watchdog: A physical device that reboots the server if the operating system becomes unresponsive. +- Software watchdog (also called a softdog): A software-based watchdog timer tha emulates the functionality of a hardware watchdog but is implemented entirely in software. It is part of the Linux kernel's watchdog infrastructure and is useful in systems that lack dedicated hardware watchdog timers. The softdog monitors the system and takes corrective actions such as killing processes or rebooting the node. + +Most of the servers in the cloud nowadays use a softdog. + +## Integration with other tools + +Patroni integrates well with other tools to create a comprehensive high-availability solution. In our architecture, such tools are: + +* HAProxy to check the current topology and route the traffic to both the primary and replica nodes, balancing the load among them, +* pgBackRest to help to ensure robust backup and restore, +* PMM for monitoring. + +Patroni provides hooks that allow you to customize its behavior. You can use hooks to execute custom scripts or commands at various stages of Patroni lifecycle, such as before and after failover, or when a new instance joins the cluster. Thereby you can integrate Patroni with other systems and automate various tasks. For example, use a hook to update the monitoring system when a failover occurs. + +## Next steps + +[HAProxy](haproxy-info.md){.md-button} \ No newline at end of file diff --git a/docs/solutions/pgbackrest-info.md b/docs/solutions/pgbackrest-info.md new file mode 100644 index 000000000..e94d1d9c5 --- /dev/null +++ b/docs/solutions/pgbackrest-info.md @@ -0,0 +1,41 @@ +# pgBackRest + +`pgBackRest` is an advanced backup and restore tool designed specifically for PostgreSQL databases. `pgBackRest` emphasizes simplicity, speed, and scalability. Its architecture is focused on minimizing the time and resources required for both backup and restoration processes. + +`pgBackRest` uses a custom protocol, which allows for more flexibility compared to traditional tools like `tar` and `rsync` and limits the types of connections that are required to perform a backup, thereby increasing security. `pgBackRest` is a simple, but feature-rich, reliable backup and restore system that can seamlessly scale up to the largest databases and workloads. + +## Key features of `pgBackRest` + +1. **Full, differential, and incremental backups (at file or block level)**: `pgBackRest` supports various types of backups, including full, differential, and incremental, providing efficient storage and recovery options. Block-level backups save space by only copying the parts of files that have changed. + +2. **Point-in-Time recovery (PITR)**: `pgBackRest` enables restoring a PostgreSQL database to a specific point in time, crucial for disaster recovery scenarios. + +3. **Parallel backup and restore**: `pgBackRest` can perform backups and restores in parallel, utilizing multiple CPU cores to significantly reduce the time required for these operations. + +4. **Local or remote operation**: A custom protocol allows `pgBackRest` to backup, restore, and archive locally or remotely via TLS/SSH with minimal configuration. This allows for flexible deployment options. + +5. **Backup rotation and archive expiration**: You can set retention policies to manage backup rotation and WAL archive expiration automatically. + +6. **Backup integrity and verification**: `pgBackRest` performs integrity checks on backup files, ensuring they are consistent and reliable for recovery. + +7. **Backup resume**: `pgBackRest` can resume an interrupted backup from the point where it was stopped. Files that were already copied are compared with the checksums in the manifest to ensure integrity. This operation can take place entirely on the repository host, therefore, it reduces load on the PostgreSQL host and saves time since checksum calculation is faster than compressing and retransmitting data. + +8. **Delta restore**: This feature allows pgBackRest to quickly apply incremental changes to an existing database, reducing restoration time. + +9. **Compression and encryption**: `pgBackRest` offers options for compressing and encrypting backup data, enhancing security and reducing storage requirements. + +## How `pgBackRest` works + +`pgBackRest` supports a backup server (or a dedicated repository host in `pgBackRest` terminology). This repository host acts as the centralized backup storage. Multiple PostgreSQL clusters can use the same repository host. + +In addition to a repository host with `pgBackRest` installed, you also need `pgBackRest` agents running on the database nodes. The backup server has the information about a PostgreSQL cluster, where it is located, how to back it up and where to store backup files. This information is defined within a configuration section called a *stanza*. + +The storage location where `pgBackRest` stores backup data and WAL archives is called the repository. It can be a local directory, a remote server, or a cloud storage service like AWS S3, S3-compatible storages or Azure blob storage. `pgBackRest` supports up to 4 repositories, allowing for redundancy and flexibility in backup storage. + +When you create a stanza, it initializes the repository and prepares it for storing backups. During the backup process, `pgBackRest` reads the data from the PostgreSQL cluster and writes it to the repository. It also performs integrity checks and compresses the data if configured. + +Similarly, during the restore process, `pgBackRest` reads the backup data from the repository and writes it to the PostgreSQL data directory. It also verifies the integrity of the restored data. + +## Next steps + +[How components work together :material-arrow-right:](ha-components.md){.md-button} \ No newline at end of file diff --git a/docs/solutions/pgbackrest.md b/docs/solutions/pgbackrest.md index 5b147f3f0..e921ce709 100644 --- a/docs/solutions/pgbackrest.md +++ b/docs/solutions/pgbackrest.md @@ -1,49 +1,45 @@ # pgBackRest setup -[pgBackRest :octicons-link-external-16:](https://pgbackrest.org/) is a backup tool used to perform PostgreSQL database backup, archiving, restoration, and point-in-time recovery. While it can be used for local backups, this procedure shows how to deploy a [pgBackRest server running on a dedicated host :octicons-link-external-16:](https://pgbackrest.org/user-guide-rhel.html#repo-host) and how to configure PostgreSQL servers to use it for backups and archiving. +[pgBackRest :octicons-link-external-16:](https://pgbackrest.org/) is a backup tool used to perform PostgreSQL database backup, archiving, restoration, and point-in-time recovery. -You also need a backup storage to store the backups. It can either be a remote storage such as AWS S3, S3-compatible storages or Azure blob storage, or a filesystem-based one. +In our solution we deploy a [pgBackRest server on a dedicated host :octicons-link-external-16:](https://pgbackrest.org/user-guide-rhel.html#repo-host) and also deploy pgBackRest on the PostgreSQL servers. Them we configure PostgreSQL servers to use it for backups and archiving. -## Configure backup server +You also need a backup storage to store the backups. It can either be a remote storage such as AWS S3, S3-compatible storages or Azure blob storage, or a filesystem-based one. -To make things easier when working with some templates, run the commands below as the root user. Run the following command to switch to the root user: - -```{.bash data-prompt="$"} -$ sudo su - -``` +## Preparation + +Make sure to complete the [initial setup](ha-init-setup.md) steps. -### Install pgBackRest +## Install pgBackRest -1. Enable the repository with [percona-release :octicons-link-external-16:](https://www.percona.com/doc/percona-repo-config/index.html) +Install pgBackRest on the following nodes: `node1`, `node2`, `node3`, `backup` + +=== ":material-debian: On Debian/Ubuntu" ```{.bash data-prompt="$"} - $ percona-release setup ppg-16 + $ sudo apt install percona-pgbackrest ``` -2. Install pgBackRest package - - === ":material-debian: On Debian/Ubuntu" +=== ":material-redhat: On RHEL/derivatives" - ```{.bash data-prompt="$"} - $ apt install percona-pgbackrest - ``` + ```{.bash data-prompt="$"} + $ sudo yum install percona-pgbackrest + ``` - === ":material-redhat: On RHEL/derivatives" +## Configure a backup server - ```{.bash data-prompt="$"} - $ yum install percona-pgbackrest - ``` +Do the following steps on the `backup` node. ### Create the configuration file 1. Create environment variables to simplify the config file creation: ```{.bash data-prompt="$"} - export SRV_NAME="bkp-srv" - export NODE1_NAME="node-1" - export NODE2_NAME="node-2" - export NODE3_NAME="node-3" - export CA_PATH="/etc/ssl/certs/pg_ha" + $ export SRV_NAME="backup" + $ export NODE1_NAME="node1" + $ export NODE2_NAME="node2" + $ export NODE3_NAME="node3" + $ export CA_PATH="/etc/ssl/certs/pg_ha" ``` 2. Create the `pgBackRest` repository, *if necessary* @@ -53,25 +49,25 @@ $ sudo su - This directory is usually created during pgBackRest's installation process. If it's not there already, create it as follows: ```{.bash data-prompt="$"} - $ mkdir -p /var/lib/pgbackrest - $ chmod 750 /var/lib/pgbackrest - $ chown postgres:postgres /var/lib/pgbackrest + $ sudo mkdir -p /var/lib/pgbackrest + $ sudo chmod 750 /var/lib/pgbackrest + $ sudo chown postgres:postgres /var/lib/pgbackrest ``` 3. The default `pgBackRest` configuration file location is `/etc/pgbackrest/pgbackrest.conf`, but some systems continue to use the old path, `/etc/pgbackrest.conf`, which remains a valid alternative. If the former is not present in your system, create the latter. - Access the file's parent directory (either `cd /etc/` or `cd /etc/pgbackrest/`), and make a backup copy of it: + Go to the file's parent directory (either `cd /etc/` or `cd /etc/pgbackrest/`), and make a backup copy of it: ```{.bash data-prompt="$"} - $ cp pgbackrest.conf pgbackrest.conf.bak + $ sudo cp pgbackrest.conf pgbackrest.conf.orig ``` - Then use the following command to create a basic configuration file using the environment variables we created in a previous step: +4. Then use the following command to create a basic configuration file using the environment variables we created in a previous step. This example command adds the configuration file at the path `/etc/pgbackrest.conf`. Make sure to specify the correct path for the configuration file on your system: === ":material-debian: On Debian/Ubuntu" ``` - cat < pgbackrest.conf + echo " [global] # Server repo details @@ -96,7 +92,7 @@ $ sudo su - repo1-retention-full=4 # Server general options - process-max=12 + process-max=4 # This depends on the number of CPU resources your server has. The recommended value should equal or be less than the number of CPUs. While more processes can speed up backups, they will also consume additional system resources. log-level-console=info #log-level-file=debug log-level-file=info @@ -120,7 +116,7 @@ $ sudo su - pg1-host=${NODE1_NAME} pg1-host-port=8432 pg1-port=5432 - pg1-path=/var/lib/postgresql/16/main + pg1-path=/var/lib/postgresql/{{pgversion}}/main pg1-host-type=tls pg1-host-cert-file=${CA_PATH}/${SRV_NAME}.crt pg1-host-key-file=${CA_PATH}/${SRV_NAME}.key @@ -130,7 +126,7 @@ $ sudo su - pg2-host=${NODE2_NAME} pg2-host-port=8432 pg2-port=5432 - pg2-path=/var/lib/postgresql/16/main + pg2-path=/var/lib/postgresql/{{pgversion}}/main pg2-host-type=tls pg2-host-cert-file=${CA_PATH}/${SRV_NAME}.crt pg2-host-key-file=${CA_PATH}/${SRV_NAME}.key @@ -140,19 +136,20 @@ $ sudo su - pg3-host=${NODE3_NAME} pg3-host-port=8432 pg3-port=5432 - pg3-path=/var/lib/postgresql/16/main + pg3-path=/var/lib/postgresql/{{pgversion}}/main pg3-host-type=tls pg3-host-cert-file=${CA_PATH}/${SRV_NAME}.crt pg3-host-key-file=${CA_PATH}/${SRV_NAME}.key pg3-host-ca-file=${CA_PATH}/ca.crt pg3-socket-path=/var/run/postgresql - EOF + + " | sudo tee /etc/pgbackrest.conf ``` === ":material-redhat: On RHEL/derivatives" ``` - cat < pgbackrest.conf + echo " [global] # Server repo details @@ -177,7 +174,7 @@ $ sudo su - repo1-retention-full=4 # Server general options - process-max=12 + process-max=4 # This depends on the number of CPU resources your server has. The recommended value should equal or be less than the number of CPUs. While more processes can speed up backups, they will also consume additional system resources. log-level-console=info #log-level-file=debug log-level-file=info @@ -201,7 +198,7 @@ $ sudo su - pg1-host=${NODE1_NAME} pg1-host-port=8432 pg1-port=5432 - pg1-path=/var/lib/pgsql/16/data + pg1-path=/var/lib/postgresql/{{pgversion}}/main pg1-host-type=tls pg1-host-cert-file=${CA_PATH}/${SRV_NAME}.crt pg1-host-key-file=${CA_PATH}/${SRV_NAME}.key @@ -211,7 +208,7 @@ $ sudo su - pg2-host=${NODE2_NAME} pg2-host-port=8432 pg2-port=5432 - pg2-path=/var/lib/pgsql/16/data + pg2-path=/var/lib/postgresql/{{pgversion}}/main pg2-host-type=tls pg2-host-cert-file=${CA_PATH}/${SRV_NAME}.crt pg2-host-key-file=${CA_PATH}/${SRV_NAME}.key @@ -221,56 +218,70 @@ $ sudo su - pg3-host=${NODE3_NAME} pg3-host-port=8432 pg3-port=5432 - pg3-path=/var/lib/pgsql/16/data + pg3-path=/var/lib/postgresql/{{pgversion}}/main pg3-host-type=tls pg3-host-cert-file=${CA_PATH}/${SRV_NAME}.crt pg3-host-key-file=${CA_PATH}/${SRV_NAME}.key pg3-host-ca-file=${CA_PATH}/ca.crt pg3-socket-path=/var/run/postgresql - EOF + + " | sudo tee /etc/pgbackrest.conf ``` *NOTE*: The option `backup-standby=y` above indicates the backups should be taken from a standby server. If you are operating with a primary only, or if your secondaries are not configured with `pgBackRest`, set this option to `n`. ### Create the certificate files - + +Run the following commands as a root user or with `sudo` privileges + 1. Create the folder to store the certificates: ```{.bash data-prompt="$"} - $ mkdir -p ${CA_PATH} + $ sudo mkdir -p /etc/ssl/certs/pg_ha ``` - -2. Create the certificates and keys + +2. Create the environment variable to simplify further configuration + + ```{.bash data-prompt="$"} + $ export CA_PATH="/etc/ssl/certs/pg_ha" + ``` + +3. Create the CA certificates and keys ```{.bash data-prompt="$"} - $ openssl req -new -x509 -days 365 -nodes -out ${CA_PATH}/ca.crt -keyout ${CA_PATH}/ca.key -subj "/CN=root-ca" + $ sudo openssl req -new -x509 -days 365 -nodes -out ${CA_PATH}/ca.crt -keyout ${CA_PATH}/ca.key -subj "/CN=root-ca" ``` -3. Create the certificate for the backup and the PostgreSQL servers +3. Create the certificate and keys for the backup server ```{.bash data-prompt="$"} - $ for node in ${SRV_NAME} ${NODE1_NAME} ${NODE2_NAME} ${NODE3_NAME} - do - openssl req -new -nodes -out ${CA_PATH}/$node.csr -keyout ${CA_PATH}/$node.key -subj "/CN=$node"; - done + $ sudo openssl req -new -nodes -out ${CA_PATH}/${SRV_NAME}.csr -keyout ${CA_PATH}/${SRV_NAME}.key -subj "/CN=${SRV_NAME}" ``` -4. Sign the certificates with the `root-ca` key +4. Create the certificates and keys for each PostgreSQL node ```{.bash data-prompt="$"} - $ for node in ${SRV_NAME} ${NODE1_NAME} ${NODE2_NAME} ${NODE3_NAME} - do - openssl x509 -req -in ${CA_PATH}/$node.csr -days 365 -CA ${CA_PATH}/ca.crt -CAkey ${CA_PATH}/ca.key -CAcreateserial -out ${CA_PATH}/$node.crt; - done + $ sudo openssl req -new -nodes -out ${CA_PATH}/${NODE1_NAME}.csr -keyout ${CA_PATH}/${NODE1_NAME}.key -subj "/CN=${NODE1_NAME}" + $ sudo openssl req -new -nodes -out ${CA_PATH}/${NODE2_NAME}.csr -keyout ${CA_PATH}/${NODE2_NAME}.key -subj "/CN=${NODE2_NAME}" + $ sudo openssl req -new -nodes -out ${CA_PATH}/${NODE3_NAME}.csr -keyout ${CA_PATH}/${NODE3_NAME}.key -subj "/CN=${NODE3_NAME}" + ``` + +4. Sign all certificates with the `root-ca` key + + ```{.bash data-prompt="$"} + $ sudo openssl x509 -req -in ${CA_PATH}/${SRV_NAME}.csr -days 365 -CA ${CA_PATH}/ca.crt -CAkey ${CA_PATH}/ca.key -CAcreateserial -out ${CA_PATH}/${SRV_NAME}.crt + $ sudo openssl x509 -req -in ${CA_PATH}/${NODE1_NAME}.csr -days 365 -CA ${CA_PATH}/ca.crt -CAkey ${CA_PATH}/ca.key -CAcreateserial -out ${CA_PATH}/${NODE1_NAME}.crt + $ sudo openssl x509 -req -in ${CA_PATH}/${NODE2_NAME}.csr -days 365 -CA ${CA_PATH}/ca.crt -CAkey ${CA_PATH}/ca.key -CAcreateserial -out ${CA_PATH}/${NODE2_NAME}.crt + $ sudo openssl x509 -req -in ${CA_PATH}/${NODE3_NAME}.csr -days 365 -CA ${CA_PATH}/ca.crt -CAkey ${CA_PATH}/ca.key -CAcreateserial -out ${CA_PATH}/${NODE3_NAME}.crt ``` 5. Remove temporary files, set ownership of the remaining files to the `postgres` user, and restrict their access: ```{.bash data-prompt="$"} - $ rm -f ${CA_PATH}/*.csr - $ chown postgres:postgres -R ${CA_PATH} - $ chmod 0600 ${CA_PATH}/* - ``` + $ sudo rm -f ${CA_PATH}/*.csr + $ sudo chown postgres:postgres -R ${CA_PATH} + $ sudo chmod 0600 ${CA_PATH}/* + ``` ### Create the `pgbackrest` daemon service @@ -294,60 +305,71 @@ $ sudo su - [Install] WantedBy=multi-user.target ``` - -2. Reload, start, and enable the service + +2. Make `systemd` aware of the new service: ```{.bash data-prompt="$"} - $ systemctl daemon-reload - $ systemctl start pgbackrest.service - $ systemctl enable pgbackrest.service + $ sudo systemctl daemon-reload + ``` + +3. Enable `pgBackRest`: + + ```{.bash data-prompt="$"} + $ sudo systemctl enable --now pgbackrest.service ``` ## Configure database servers Run the following commands on `node1`, `node2`, and `node3`. -1. Install pgBackRest package +1. Install `pgBackRest` package === ":material-debian: On Debian/Ubuntu" ```{.bash data-prompt="$"} - $ apt install percona-pgbackrest + $ sudo apt install percona-pgbackrest ``` === ":material-redhat: On RHEL/derivatives" ```{.bash data-prompt="$"} - $ yum install percona-pgbackrest - + $ sudo yum install percona-pgbackrest + ``` + 2. Export environment variables to simplify the config file creation: ```{.bash data-prompt="$"} $ export NODE_NAME=`hostname -f` - $ export SRV_NAME="bkp-srv" + $ export SRV_NAME="backup" $ export CA_PATH="/etc/ssl/certs/pg_ha" ``` - + 3. Create the certificates folder: ```{.bash data-prompt="$"} - $ mkdir -p ${CA_PATH} + $ sudo mkdir -p ${CA_PATH} ``` 4. Copy the `.crt`, `.key` certificate files and the `ca.crt` file from the backup server where they were created to every respective node. Then change the ownership to the `postgres` user and restrict their access. Use the following commands to achieve this: ```{.bash data-prompt="$"} - $ scp ${SRV_NAME}:${CA_PATH}/{$NODE_NAME.crt,$NODE_NAME.key,ca.crt} ${CA_PATH}/ - $ chown postgres:postgres -R ${CA_PATH} - $ chmod 0600 ${CA_PATH}/* + $ sudo scp ${SRV_NAME}:${CA_PATH}/{$NODE_NAME.crt,$NODE_NAME.key,ca.crt} ${CA_PATH}/ + $ sudo chown postgres:postgres -R ${CA_PATH} + $ sudo chmod 0600 ${CA_PATH}/* ``` - -5. Edit or create the configuration file which, as explained above, can be either at the `/etc/pgbackrest/pgbackrest.conf` or `/etc/pgbackrest.conf` path: + +5. Make a copy of the configuration file. The path to it can be either `/etc/pgbackrest/pgbackrest.conf` or `/etc/pgbackrest.conf`: + + ```{.bash data-prompt="$"} + $ sudo cp pgbackrest.conf pgbackrest.conf.orig + ``` + +6. Create the configuration file. This example command adds the configuration file at the path `/etc/pgbackrest.conf`. Make sure to specify the correct path for the configuration file on your system: === ":material-debian: On Debian/Ubuntu" ```ini title="pgbackrest.conf" - cat < pgbackrest.conf + echo " [global] repo1-host=${SRV_NAME} repo1-host-user=postgres @@ -357,7 +379,7 @@ Run the following commands on `node1`, `node2`, and `node3`. repo1-host-ca-file=${CA_PATH}/ca.crt # general options - process-max=16 + process-max=6 log-level-console=info log-level-file=debug @@ -369,15 +391,14 @@ Run the following commands on `node1`, `node2`, and `node3`. tls-server-auth=${SRV_NAME}=cluster_1 [cluster_1] - pg1-path=/var/lib/postgresql/16/main - EOF + pg1-path=/var/lib/postgresql/{{pgversion}}/main + " | sudo tee /etc/pgbackrest.conf ``` - === ":material-redhat: On RHEL/derivatives" ```ini title="pgbackrest.conf" - cat < pgbackrest.conf + echo " [global] repo1-host=${SRV_NAME} repo1-host-user=postgres @@ -387,7 +408,7 @@ Run the following commands on `node1`, `node2`, and `node3`. repo1-host-ca-file=${CA_PATH}/ca.crt # general options - process-max=16 + process-max=6 log-level-console=info log-level-file=debug @@ -399,11 +420,11 @@ Run the following commands on `node1`, `node2`, and `node3`. tls-server-auth=${SRV_NAME}=cluster_1 [cluster_1] - pg1-path=/var/lib/pgsql/16/data - EOF + pg1-path=/var/lib/pgsql/{{pgversion}}/data + " | sudo tee /etc/pgbackrest.conf ``` -6. Create the pgbackrest `systemd` unit file at the path `/etc/systemd/system/pgbackrest.service` +7. Create the pgbackrest `systemd` unit file at the path `/etc/systemd/system/pgbackrest.service` ```ini title="/etc/systemd/system/pgbackrest.service" [Unit] @@ -424,71 +445,79 @@ Run the following commands on `node1`, `node2`, and `node3`. WantedBy=multi-user.target ``` -7. Reload, start, and enable the service +8. Reload the `systemd`, the start the service ```{.bash data-prompt="$"} - $ systemctl daemon-reload - $ systemctl start pgbackrest - $ systemctl enable pgbackrest + $ sudo systemctl daemon-reload + $ sudo systemctl enable --now pgbackrest ``` The pgBackRest daemon listens on port `8432` by default: ```{.bash data-prompt="$"} - $ netstat -taunp - Active Internet connections (servers and established) - Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name - tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/systemd - tcp 0 0 0.0.0.0:8432 0.0.0.0:* LISTEN 40224/pgbackrest + $ netstat -taunp | grep '8432' ``` -8. If you are using Patroni, change its configuration to use `pgBackRest` for archiving and restoring WAL files. Run this command only on one node, for example, on `node1`: + ??? example "Sample output" + + ```{text .no-copy} + Active Internet connections (servers and established) + Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name + tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/systemd + tcp 0 0 0.0.0.0:8432 0.0.0.0:* LISTEN 40224/pgbackrest + ``` + +9. If you are using Patroni, change its configuration to use `pgBackRest` for archiving and restoring WAL files. Run this command only on one node, for example, on `node1`: ```{.bash data-prompt="$"} $ patronictl -c /etc/patroni/patroni.yml edit-config ``` - - === ":material-debian: On Debian/Ubuntu" - - ```yaml title="/etc/patroni/patroni.yml" - postgresql: - (...) - parameters: - (...) - archive_command: pgbackrest --stanza=cluster_1 archive-push /var/lib/postgresql/{{pgversion}}/main/pg_wal/%f - (...) - recovery_conf: - (...) - restore_command: pgbackrest --config=/etc/pgbackrest.conf --stanza=cluster_1 archive-get %f %p - (...) - ``` - === ":material-redhat: On RHEL/derivatives" + This opens the editor for you. + +10. Change the configuration as follows: + + ```yaml title="/etc/patroni/patroni.yml" + postgresql: + parameters: + archive_command: pgbackrest --stanza=cluster_1 archive-push /var/lib/postgresql/{{pgversion}}/main/pg_wal/%f + archive_mode: true + archive_timeout: 600s + hot_standby: true + logging_collector: 'on' + max_replication_slots: 10 + max_wal_senders: 5 + max_wal_size: 10GB + wal_keep_segments: 10 + wal_level: logical + wal_log_hints: true + recovery_conf: + recovery_target_timeline: latest + restore_command: pgbackrest --config=/etc/pgbackrest.conf --stanza=cluster_1 archive-get %f "%p" + use_pg_rewind: true + use_slots: true + retry_timeout: 10 + slots: + percona_cluster_1: + type: physical + ttl: 30 + ``` - ```yaml title="/etc/patroni/patroni.yml" - postgresql: - (...) - parameters: - archive_command: pgbackrest --stanza=cluster_1 archive-push /var/lib/pgsql/{{pgversion}}/data/pg_wal/%f - (...) - recovery_conf: - restore_command: pgbackrest --config=/etc/pgbackrest.conf --stanza=cluster_1 archive-get %f %p - (...) - ``` - - Reload the changed configurations: +11. Reload the changed configurations. Provide the cluster name or the node name for the following command. In our example we use the `cluster_1` cluster name: ```{.bash data-prompt="$"} - $ patronictl -c /etc/patroni/postgresql.yml reload + $ patronictl -c /etc/patroni/patroni.yml restart cluster_1 ``` - :material-information: Note: When configuring a PostgreSQL server that is not managed by Patroni to archive/restore WALs from the `pgBackRest` server, edit the server's main configuration file directly and adjust the `archive_command` and `restore_command` variables as shown above. + It may take a while to reload the new configuration. + + *NOTE*: When configuring a PostgreSQL server that is not managed by Patroni to archive/restore WALs from the `pgBackRest` server, edit the server's main configuration file directly and adjust the `archive_command` and `restore_command` variables as shown above. ## Create backups Run the following commands on the **backup server**: -1. Create the stanza. A stanza is the configuration for a PostgreSQL database cluster that defines where it is located, how it will be backed up, archiving options, etc. +1. Create the stanza. A stanza is the configuration for a PostgreSQL database cluster that defines where it is located, how it will be backed up, archiving options, etc. ```{.bash data-prompt="$"} $ sudo -iu postgres pgbackrest --stanza=cluster_1 stanza-create @@ -501,7 +530,7 @@ Run the following commands on the **backup server**: ``` 3. Check backup info - + ```{.bash data-prompt="$"} $ sudo -iu postgres pgbackrest --stanza=cluster_1 info ``` @@ -512,4 +541,6 @@ Run the following commands on the **backup server**: $ sudo -iu postgres pgbackrest --stanza=cluster_1 expire --set= ``` -[Test PostgreSQL cluster](ha-test.md){.md-button} +## Next steps + +[Configure HAProxy :material-arrow-right:](ha-haproxy.md){.md-button} diff --git a/docs/solutions/postgis-deploy.md b/docs/solutions/postgis-deploy.md index 77fb41892..ceb0e8731 100644 --- a/docs/solutions/postgis-deploy.md +++ b/docs/solutions/postgis-deploy.md @@ -66,6 +66,10 @@ The following document provides guidelines how to install PostGIS and how to run FROM pg_available_extensions WHERE name LIKE 'postgis%' or name LIKE 'address%'; ``` +=== ":octicons-download-16: From tarballs" + + PostGIS is included into binary tarball and is a part of the `percona-postgresql{{pgversion}}` binary. Use the [install from tarballs](../tarball.md) tutorial to install it. + ## Enable PostGIS extension diff --git a/docs/tarball.md b/docs/tarball.md index fdc05b36e..556d4d290 100644 --- a/docs/tarball.md +++ b/docs/tarball.md @@ -1,13 +1,21 @@ -# Install Percona Distribiution for PostgreSQL from binary tarballs +# Install Percona Distribution for PostgreSQL from binary tarballs -You can find the binary tarballs on the [Percona website](https://www.percona.com/downloads). Select the desired version from a version dropdown and _All_ from the Select Platform dropdown. +You can download the tarballs using the links below. -There are the following tarballs available both for x86_64 and ARM64 architectures: +!!! note -* percona-postgresql-{{dockertag}}-ssl1.1-linux-.tar.gz - for operating systems that run OpenSSL version 1.x -* percona-postgresql-{{dockertag}}-ssl3-linux-.tar.gz - for operating systems that run OpenSSL version 3.x + Unlike package managers, a tarball installation does **not** provide mechanisms to ensure that all dependencies are resolved to the correct library versions. There is no built-in method to verify that required libraries are present or to prevent them from being removed. As a result, unresolved or broken dependencies may lead to errors, crashes, or even data corruption. + + For this reason, tarball installations are **not recommended** for environments where safety, security, reliability, or mission-critical stability are required. + +The following tarballs are available for the x86_64 and ARM64 architectures: + +* [percona-postgresql-{{dockertag}}-ssl1.1-linux-aarch64.tar.gz](https://downloads.percona.com/downloads/postgresql-distribution-16/{{dockertag}}/binary/tarball/percona-postgresql-{{dockertag}}-ssl1.1-linux-aarch64.tar.gz) - for operating systems on ARM64 architecture that run OpenSSL version 1.x +* [percona-postgresql-{{dockertag}}-ssl1.1-linux-x86_64.tar.gz](https://downloads.percona.com/downloads/postgresql-distribution-16/{{dockertag}}/binary/tarball/percona-postgresql-{{dockertag}}-ssl1.1-linux-x86_64.tar.gz) - for operating systems on x86_64 architecture that run OpenSSL version 1.x +* [percona-postgresql-{{dockertag}}-ssl3-linux-aarch64.tar.gz](https://downloads.percona.com/downloads/postgresql-distribution-16/{{dockertag}}/binary/tarball/percona-postgresql-{{dockertag}}-ssl3-linux-aarch64.tar.gz) - for operating systems on ARM64 architecture that run OpenSSL version 3.x +* [percona-postgresql-{{dockertag}}-ssl3-linux-x86_64.tar.gz](https://downloads.percona.com/downloads/postgresql-distribution-16/{{dockertag}}/binary/tarball/percona-postgresql-{{dockertag}}-ssl3-linux-x86_64.tar.gz) - for operating systems on x86_64 architecture that run OpenSSL version 3.x -To check what OpenSSL version you have, run the following command: +To check what OpenSSL version you have, run the following command: ```{.bash data-prompt="$"} $ openssl version @@ -19,7 +27,7 @@ The tarballs include the following components: | Component | Description | |-----------|-------------| -| percona-postgresql{{pgversion}}| The latest version of PostgreSQL server and the following extensions:
- `pgaudit`
- `pgAudit_set_user`
- `pg_repack`
- `pg_stat_monitor`
- `pg_gather`
- `wal2json`
- the set of [contrib extensions](contrib.md)| +| percona-postgresql{{pgversion}}| The latest version of PostgreSQL server and the following extensions:
- `pgaudit`
- `pgAudit_set_user`
- `pg_repack`
- `pg_stat_monitor`
- `pg_gather`
- `wal2json`
- `postGIS`
- `pgvector`
- the set of [contrib extensions](contrib.md)| | percona-haproxy | A high-availability solution and load-balancing solution | | percona-patroni | A high-availability solution for PostgreSQL | | percona-pgbackrest| A backup and restore tool | @@ -35,8 +43,14 @@ The tarballs include the following components: === "Debian and Ubuntu" - 1. Uninstall the upstream PostgreSQL package. - 2. Create the user to own the PostgreSQL process. For example, `mypguser`. Run the following command: + 1. Uninstall the upstream PostgreSQL package. + 2. Ensure that the `libreadline` is present on the system, as it is **required** for tarballs to work correctly: + + ```{.bash data-prompt="$"} + $ sudo apt install -y libreadline-dev + ``` + + 3. Create the user to own the PostgreSQL process. For example, `mypguser`. Run the following command: ```{.bash data-prompt="$"} $ sudo useradd -m mypguser @@ -50,7 +64,13 @@ The tarballs include the following components: === "RHEL and derivatives" - Create the user to own the PostgreSQL process. For example, `mypguser`, Run the following command: + Ensure that the `libreadline` is present on the system, as it is **required** for tarballs to work correctly: + + ```{.bash data-prompt="$"} + $ sudo yum install -y readline-devel + ``` + + Create the user to own the PostgreSQL process. For example, `mypguser`, Run the following command: ```{.bash data-prompt="$"} $ sudo useradd mypguser -m @@ -66,6 +86,8 @@ The tarballs include the following components: The steps below install the tarballs for OpenSSL 3.x on x86_64 architecture. Use another tarball if your operating system has OpenSSL version 1.x and / or has the ARM64 architecture. +Before step 1 please perform the following steps: + 1. Create the directory where you will store the binaries. For example, `/opt/pgdistro` 2. Grant access to this directory for the `mypguser` user. @@ -74,7 +96,7 @@ The steps below install the tarballs for OpenSSL 3.x on x86_64 architecture. Use $ sudo chown mypguser:mypguser /opt/pgdistro/ ``` -3. Fetch the binary tarball. +3. Fetch the binary tarball. ```{.bash data-prompt="$"} $ wget https://downloads.percona.com/downloads/postgresql-distribution-16/{{dockertag}}/binary/tarball/percona-postgresql-{{dockertag}}-ssl3-linux-x86_64.tar.gz @@ -83,15 +105,15 @@ The steps below install the tarballs for OpenSSL 3.x on x86_64 architecture. Use 4. Extract the tarball to the directory for binaries that you created on step 1. ```{.bash data-prompt="$"} - $ sudo tar -xfv percona-postgresql-{{dockertag}}-ssl3-linux-x86_64.tar.gz -C /opt/pgdistro/ + $ sudo tar -xvf percona-postgresql-{{dockertag}}-ssl3-linux-x86_64.tar.gz -C /opt/pgdistro/ ``` -5. If you extracted the tarball in a directory other than `/opt`, copy `percona-python3`, `percona-tcl` and `percona-perl` to the `/opt` directory. This is required for the correct run of libraries that require those modules. - +5. If you extracted the tarball in a directory other than `/opt`, copy `percona-python3`, `percona-tcl` and `percona-perl` to the `/opt` directory. This is required for the correct run of libraries that require those modules. + ```{.bash data-prompt="$"} $ sudo cp /percona-perl /percona-python3 /percona-tcl /opt/ ``` - + 6. Add the location of the binaries to the PATH variable: ```{.bash data-prompt="$"} @@ -112,12 +134,11 @@ The steps below install the tarballs for OpenSSL 3.x on x86_64 architecture. Use ``` 9. Initiate the PostgreSQL data directory: - + ```{.bash data-prompt="$"} $ /opt/pgdistro/percona-postgresql{{pgversion}}/bin/initdb -D /usr/local/pgsql/data ``` - ??? example "Sample output" ```{.text .no-copy} @@ -133,27 +154,27 @@ The steps below install the tarballs for OpenSSL 3.x on x86_64 architecture. Use ``` ??? example "Sample output" - + ```{.text .no-copy} waiting for server to start.... done server started ``` 9. Connect to `psql` - + ```{.bash data-prompt="$"} - $ /opt/pgdistro/percona-postgresql{{pgversion}}/bin/psql + $ /opt/pgdistro/percona-postgresql{{pgversion}}/bin/psql -d postgres ``` ??? example "Sample output" - + ```{.text .no-copy} psql ({{dockertag}}) Type "help" for help. postgres=# ``` - + ### Start the components After you unpacked the tarball and added the location of the components' binaries to the `$PATH` variable, the components are available for use. You can invoke a component by running its command-line tool. @@ -165,4 +186,3 @@ $ haproxy version ``` Some components require additional setup. Check the [Enabling extensions](enable-extensions.md) page for details. - diff --git a/docs/telemetry.md b/docs/telemetry.md index 864c76e54..40dc55c9a 100644 --- a/docs/telemetry.md +++ b/docs/telemetry.md @@ -67,7 +67,7 @@ The telemetry also uses the Percona Platform with the following components: `percona_pg_telemetry` is an extension to collect telemetry data in PostgreSQL. It is added to Percona Distribution for PostgreSQL and is automatically loaded when you install a PostgreSQL server. -`percona_pg_telemetry` collects metrics from the database instance daily to the Metrics File. It creates a new Metrics File for each collection. You can find the Metrics File in its [location](#location) to inspect what data is collected. +`percona_pg_telemetry` collects metrics from the database instance daily to the Metrics File. It creates a new Metrics File for each collection. You can find the Metrics File in its [location](#locations) to inspect what data is collected. Before generating a new file, the `percona_pg_telemetry` deletes the Metrics Files that are older than seven days. This process ensures that only the most recent week's data is maintained. @@ -102,74 +102,74 @@ The Metrics File uses the Javascript Object Notation (JSON) format. Percona rese The following is an example of the collected data generated by the `percona_pg_telemetry` extension: - ```json +```json + { + "db_instance_id": "7310358902660071382", + "pillar_version": "16.3", + "uptime": "36", + "databases_count": "2", + "settings": [ { - "db_instance_id": "7310358902660071382", - "pillar_version": "16.3", - "uptime": "36", - "databases_count": "2", - "settings": [ + "key": "setting", + "value": [ + { + "key": "name", + "value": "allow_in_place_tablespaces" + }, + { + "key": "unit", + "value": "NULL" + }, { "key": "setting", - "value": [ - { - "key": "name", - "value": "allow_in_place_tablespaces" - }, - { - "key": "unit", - "value": "NULL" - }, - { - "key": "setting", - "value": "off" - }, - { - "key": "reset_val", - "value": "off" - }, - { - "key": "boot_val", - "value": "off" - } - ] + "value": "off" + }, + { + "key": "reset_val", + "value": "off" + }, + { + "key": "boot_val", + "value": "off" + } + ] + }, + ... + ], + "databases": [ + { + "key": "database", + "value": [ + { + "key": "database_oid", + "value": "5" }, - ... - ], - "databases": [ { - "key": "database", + "key": "database_size", + "value": "7820895" + }, + { + "key": "active_extensions", "value": [ { - "key": "database_oid", - "value": "5" + "key": "extension_name", + "value": "plpgsql" }, { - "key": "database_size", - "value": "7820895" + "key": "extension_name", + "value": "pg_tde" }, { - "key": "active_extensions", - "value": [ - { - "key": "extension_name", - "value": "plpgsql" - }, - { - "key": "extension_name", - "value": "pg_tde" - }, - { - "key": "extension_name", - "value": "percona_pg_telemetry" - } - ] + "key": "extension_name", + "value": "percona_pg_telemetry" } ] } ] } - ``` + ] + } +``` ### Telemetry Agent diff --git a/docs/templates/pdf_cover_page.tpl b/docs/templates/pdf_cover_page.tpl new file mode 100644 index 000000000..83dadd948 --- /dev/null +++ b/docs/templates/pdf_cover_page.tpl @@ -0,0 +1,12 @@ + +{{ config.extra.added_key }} +

+ +

Distribution for PostgreSQL

+{% if config.site_description %} +

+{% endif %} +

16.9 (May 29, 2025)

+ + diff --git a/docs/third-party.md b/docs/third-party.md index b9e3253af..0cadeee57 100644 --- a/docs/third-party.md +++ b/docs/third-party.md @@ -18,4 +18,5 @@ Percona Distribution for PostgreSQL is supplied with the set of third-party open | [pg_repack](https://github.com/reorg/pg_repack) | Required | Rebuilds PostgreSQL database objects | | [pg_stat_monitor](https://github.com/percona/pg_stat_monitor) | Required | Collects and aggregates statistics for PostgreSQL and provides histogram information | | [PostGIS](http://postgis.net/) | Required | Allows storing and manipulating spacial data in PostgreSQL | +|[pgvector :octicons-link-external-16:](https://github.com/pgvector/pgvector)| Required | An extension that enables you to use PostgreSQL as a vector database| |[wal2json](https://github.com/eulerto/wal2json)|Required| A PostgreSQL logical decoding JSON output plugin.| \ No newline at end of file diff --git a/docs/yum.md b/docs/yum.md index 884961a70..3d7b60409 100644 --- a/docs/yum.md +++ b/docs/yum.md @@ -19,10 +19,10 @@ Depending on what operating system you are using, you may need to enable or disa === "RHEL8/Oracle Linux 8/Rocky Linux 8" - Disable the ``postgresql`` and ``llvm-toolset``modules: + Disable the ``postgresql`` module: ```{.bash data-prompt="$"} - $ sudo dnf module disable postgresql llvm-toolset + $ sudo dnf module disable postgresql ``` ### For `percona-postgresql{{pgversion}}-devel` package @@ -32,28 +32,28 @@ You may need to install the `percona-postgresql{{pgversion}}-devel` package when === "RHEL8" ```{.bash data-prompt="$"} - $ sudo yum --enablerepo=codeready-builder-for-rhel-8-rhui-rpms install perl-IPC-Run -y + $ sudo yum --enablerepo=codeready-builder-for-rhel-8-rhui-rpms + $ sudo dnf install perl-IPC-Run -y ``` === "Rocky Linux 8" ```{.bash data-prompt="$"} $ sudo dnf install dnf-plugins-core - $ sudo dnf module enable llvm-toolset $ sudo dnf config-manager --set-enabled powertools ``` === "Oracle Linux 8" ```{.bash data-prompt="$"} - $ sudo dnf config-manager --set-enabled ol8_codeready_builder install perl-IPC-Run -y + $ sudo dnf config-manager --set-enabled ol8_codeready_builder + $ sudo dnf install perl-IPC-Run -y ``` === "Rocky Linux 9" ```{.bash data-prompt="$"} $ sudo dnf install dnf-plugins-core - $ sudo dnf module enable llvm-toolset $ sudo dnf config-manager --set-enabled crb $ sudo dnf install perl-IPC-Run -y ``` @@ -61,7 +61,8 @@ You may need to install the `percona-postgresql{{pgversion}}-devel` package when === "Oracle Linux 9" ```{.bash data-prompt="$"} - $ sudo dnf config-manager --set-enabled ol9_codeready_builder install perl-IPC-Run -y + $ sudo dnf config-manager --set-enabled ol9_codeready_builder + $ sudo dnf install perl-IPC-Run -y ``` ### For `percona-patroni` package @@ -96,55 +97,95 @@ The following are commands for Red Hat Enterprise Linux 9 and derivatives. For R $ sudo dnf config-manager --set-enabled ol9_codeready_builder ``` -### For PostGIS +### For PostGIS For Red Hat Enterprise Linux 8 and derivatives, replace the operating system version in the following commands accordingly. -=== "RHEL 9" +=== "RHEL 8" Run the following commands: {.power-number} - 1. Install `epel` repository + 1. Install DNF plugin utilities ```{.bash data-prompt="$"} - $ sudo yum install epel-release + $ sudo dnf install dnf-plugins-core ``` - 2. Enable the `llvm-toolset dnf` module + 2. Install the EPEL repository - ```{.bash data-prompt="$"} - $ sudo dnf module enable llvm-toolset - ``` + ```{.bash data-prompt="$"} + $ sudo dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm + ``` + + 3. Enable the CodeReady Builder repository to resolve dependency conflicts - 3. Enable the codeready builder repository to resolve dependencies conflict. + ```{.bash data-prompt="$"} + $ sudo dnf config-manager --set-enabled codeready-builder-for-rhel-8-rhui-rpms + ``` + + 4. Disable the default PostgreSQL module ```{.bash data-prompt="$"} - $ sudo dnf config-manager --set-enabled codeready-builder-for-rhel-9-x86_64-rpms + $ sudo dnf module disable postgresql ``` -=== "Rocky Linux 9" +=== "RHEL 9" Run the following commands: {.power-number} - 1. Install `epel` repository + 1. Install DNF plugin utilities ```{.bash data-prompt="$"} - $ sudo yum install epel-release + $ sudo dnf install dnf-plugins-core + ``` + + 2. Install the EPEL repository + + ```{.bash data-prompt="$"} + $ sudo dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm + ``` + + 3. Enable the CodeReady Builder repository to resolve dependency conflicts + + ```{.bash data-prompt="$"} + $ sudo dnf config-manager --set-enabled codeready-builder-for-rhel-9-rhui-rpms ``` - 2. Enable the `llvm-toolset dnf` module +=== "Oracle Linux 8" + + Run the following commands: + {.power-number} + + 1. Install the EPEL repository ```{.bash data-prompt="$"} - $ sudo dnf module enable llvm-toolset + $ sudo dnf install -y epel-release ``` - 3. Enable the codeready builder repository to resolve dependencies conflict. + 2. Install DNF plugin utilities ```{.bash data-prompt="$"} $ sudo dnf install dnf-plugins-core - $ sudo dnf config-manager --set-enabled crb + ``` + + 3. Enable the CodeReady Builder repository to resolve dependency conflicts + + ```{.bash data-prompt="$"} + $ sudo dnf config-manager --set-enabled ol8_codeready_builder + ``` + + 4. (Alternative) Install the latest EPEL release + + ```{.bash data-prompt="$"} + $ sudo dnf -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm + ``` + + 5. Disable the default PostgreSQL module + + ```{.bash data-prompt="$"} + $ sudo dnf module disable postgresql ``` === "Oracle Linux 9" @@ -152,24 +193,76 @@ For Red Hat Enterprise Linux 8 and derivatives, replace the operating system ver Run the following commands: {.power-number} - 1. Install `epel` repository + 1. Install the EPEL repository ```{.bash data-prompt="$"} - $ sudo yum install epel-release + $ sudo dnf install -y epel-release ``` - 2. Enable the `llvm-toolset dnf` module + 2. Install DNF plugin utilities ```{.bash data-prompt="$"} - $ sudo dnf module enable llvm-toolset + $ sudo dnf install dnf-plugins-core ``` - 3. Enable the codeready builder repository to resolve dependencies conflict. + 3. Enable the CodeReady Builder repository to resolve dependency conflicts ```{.bash data-prompt="$"} $ sudo dnf config-manager --set-enabled ol9_codeready_builder ``` +=== "Rocky Linux 8" + + Run the following commands: + {.power-number} + + 1. Install the EPEL release package + + ```{.bash data-prompt="$"} + $ sudo dnf install -y epel-release + ``` + + 2. Install DNF plugin utilities + + ```{.bash data-prompt="$"} + $ sudo dnf install dnf-plugins-core + ``` + + 3. Enable the PowerTools repository + + ```{.bash data-prompt="$"} + $ sudo dnf config-manager --set-enabled powertools + ``` + + 4. Disable the default PostgreSQL module + + ```{.bash data-prompt="$"} + $ sudo dnf module disable postgresql + ``` + +=== "Rocky Linux 9" + + Run the following commands: + {.power-number} + + 1. Install the EPEL repository + + ```{.bash data-prompt="$"} + $ sudo dnf install -y epel-release + ``` + + 2. Install DNF plugin utilities + + ```{.bash data-prompt="$"} + $ sudo dnf install dnf-plugins-core + ``` + + 3. Enable the CodeReady Builder repository to resolve dependency conflicts + + ```{.bash data-prompt="$"} + $ sudo dnf config-manager --set-enabled crb + ``` + === "RHEL UBI 9" Run the following commands: @@ -248,7 +341,7 @@ $ sudo yum -y install curl ### Install packages -=== "Install using meta-package" +=== "Install using meta-package (deprecated)" The [meta package](repo-overview.md#percona-ppg-server){:target=”_blank”} enables you to install several components of the distribution in one go. @@ -293,7 +386,7 @@ $ sudo yum -y install curl $ sudo yum install percona-patroni ``` - [Install `pg_stat_monitor`](pg-stat-monitor.md): + [Install `pg_stat_monitor` :octicons-external-link-16:](https://docs.percona.com/pg-stat-monitor/install.html): Install `pgBouncer`: @@ -347,6 +440,12 @@ $ sudo yum -y install curl $ sudo yum install percona-pgpool-II-pg{{pgversion}} ``` + Install pgvector package suite: + + ```{.bash data-prompt="$"} + $ sudo yum install percona-pgvector_{{pgversion}} percona-pgvector_{{pgversion}}-debuginfo percona-pgvector_{{pgversion}}-debugsource percona-pgvector_{{pgversion}}-llvmjit + ``` + Some extensions require additional setup in order to use them with Percona Distribution for PostgreSQL. For more information, refer to [Enabling extensions](enable-extensions.md). ### Start the service diff --git a/mkdocs-base.yml b/mkdocs-base.yml index 74c73544e..dd2614a84 100644 --- a/mkdocs-base.yml +++ b/mkdocs-base.yml @@ -98,8 +98,8 @@ markdown_extensions: pymdownx.inlinehilite: {} pymdownx.snippets: base_path: ["snippets"] - auto_append: - - services-banner.md + # auto_append: + # - services-banner.md pymdownx.emoji: emoji_index: !!python/name:material.extensions.emoji.twemoji emoji_generator: !!python/name:material.extensions.emoji.to_svg @@ -112,6 +112,7 @@ plugins: section-index: {} search: separator: '[\s\-,:!=\[\]()"`/]+|\.(?!\d)|&[lg]t;|(?!\b)(?=[A-Z][a-z])' + open-in-new-tab: {} git-revision-date-localized: enable_creation_date: true enabled: !ENV [ENABLED_GIT_REVISION_DATE, True] @@ -124,24 +125,29 @@ plugins: macros: include_yaml: - 'variables.yml' # Use in markdown as '{{ VAR }}' -# exclude: # Don't process these files -# glob: -# - file.md - with-pdf: # https://github.com/orzih/mkdocs-with-pdf - output_path: '_pdf/PerconaDistributionPostgreSQL-16.pdf' - cover_title: 'Distribution for PostgreSQL Documentation' - - cover_subtitle: 16.4 (September 10, 2024) - author: 'Percona Technical Documentation Team' - cover_logo: docs/_images/Percona_Logo_Color.png - debug_html: false - custom_template_path: _resource/templates - enabled_if_env: ENABLE_PDF_EXPORT mike: version_selector: true css_dir: css javascript_dir: js canonical_version: null + print-site: + add_to_navigation: false + print_page_title: 'Percona Distribution for PostgreSQL documentation' + add_print_site_banner: false + # Table of contents + add_table_of_contents: true + toc_title: 'Table of Contents' + toc_depth: 2 + # Content-related + add_full_urls: false + enumerate_headings: false + enumerate_headings_depth: 1 + enumerate_figures: true + add_cover_page: true + cover_page_template: "docs/templates/pdf_cover_page.tpl" + path_to_pdf: "" + include_css: true + enabled: true extra: version: @@ -152,7 +158,7 @@ extra: nav: - 'Home': 'index.md' - + - get-help.md - Get started: - Quickstart guide: installing.md - 1. Install: @@ -173,11 +179,22 @@ nav: - Solutions: - Overview: solutions.md - High availability: - - 'High availability': 'solutions/high-availability.md' - - 'Deploying on Debian or Ubuntu': 'solutions/ha-setup-apt.md' - - 'Deploying on RHEL or derivatives': 'solutions/ha-setup-yum.md' - - solutions/pgbackrest.md - - solutions/ha-test.md + - 'Overview': 'solutions/high-availability.md' + - solutions/ha-measure.md + - 'Architecture': solutions/ha-architecture.md + - Components: + - 'ETCD': 'solutions/etcd-info.md' + - 'Patroni': 'solutions/patroni-info.md' + - 'HAProxy': 'solutions/haproxy-info.md' + - 'pgBackRest': 'solutions/pgbackrest-info.md' + - solutions/ha-components.md + - Deployment: + - 'Initial setup': 'solutions/ha-init-setup.md' + - 'etcd setup': 'solutions/ha-etcd-config.md' + - 'Patroni setup': 'solutions/ha-patroni.md' + - solutions/pgbackrest.md + - 'HAProxy setup': 'solutions/ha-haproxy.md' + - 'Testing': solutions/ha-test.md - Backup and disaster recovery: - 'Overview': 'solutions/backup-recovery.md' - solutions/dr-pgbackrest-setup.md @@ -193,10 +210,12 @@ nav: - minor-upgrade.md - migration.md - Troubleshooting guide: troubleshooting.md - - How to: how-to.md - Uninstall: uninstalling.md - Release Notes: - "Release notes index": "release-notes.md" + - release-notes-v16.9.md + - release-notes-v16.8.md + - release-notes-v16.6.md - release-notes-v16.4.md - release-notes-v16.3.md - release-notes-v16.2.md diff --git a/mkdocs-pdf.yml b/mkdocs-pdf.yml deleted file mode 100644 index c6ffae69c..000000000 --- a/mkdocs-pdf.yml +++ /dev/null @@ -1,17 +0,0 @@ -# MkDocs configuration for PDF builds -# Usage: ENABLE_PDF_EXPORT=1 mkdocs build -f mkdocs-pdf.yml - -INHERIT: mkdocs-base.yml - -copyright: Percona LLC, © 2024 - -extra_css: - - https://unicons.iconscout.com/release/v3.0.3/css/line.css - - https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.4.0/css/font-awesome.min.css - - css/percona.css - - css/extra.css - - css/osano.css - -markdown_extensions: - pymdownx.tabbed: {} - admonition: {} diff --git a/mkdocs.yml b/mkdocs.yml index a91b409f6..b839a860f 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -6,7 +6,7 @@ site_url: "https://docs.percona.com/postgresql/" theme: name: material - custom_dir: _resource/overrides/ + custom_dir: _resourcepdf/overrides/ extra: analytics: @@ -28,6 +28,79 @@ extra: feedback form. -#markdown_extensions: -# - pymdownx.tabbed: -# alternate_style: true +nav: + - 'Home': 'index.md' + - get-help.md + - Get started: + - Quickstart guide: installing.md + - 1. Install: + - Via apt: apt.md + - Via yum: yum.md + - From tarballs: tarball.md + - Run in Docker: docker.md + - enable-extensions.md + - repo-overview.md + - 2. Connect to PostgreSQL: connect.md + - 3. Manipulate data in PostgreSQL: crud.md + - 4. What's next: whats-next.md + - Extensions: + - 'Extensions': extensions.md + - contrib.md + - Percona-authored extensions: percona-ext.md + - third-party.md + - Solutions: + - Overview: solutions.md + - High availability: + - 'Overview': 'solutions/high-availability.md' + - solutions/ha-measure.md + - 'Architecture': solutions/ha-architecture.md + - Components: + - 'ETCD': 'solutions/etcd-info.md' + - 'Patroni': 'solutions/patroni-info.md' + - 'HAProxy': 'solutions/haproxy-info.md' + - 'pgBackRest': 'solutions/pgbackrest-info.md' + - solutions/ha-components.md + - Deployment: + - 'Initial setup': 'solutions/ha-init-setup.md' + - 'etcd setup': 'solutions/ha-etcd-config.md' + - 'Patroni setup': 'solutions/ha-patroni.md' + - solutions/pgbackrest.md + - 'HAProxy setup': 'solutions/ha-haproxy.md' + - 'Testing': solutions/ha-test.md + - Backup and disaster recovery: + - 'Overview': 'solutions/backup-recovery.md' + - solutions/dr-pgbackrest-setup.md + - Spatial data handling: + - Overview: solutions/postgis.md + - Deployment: solutions/postgis-deploy.md + - Query spatial data: solutions/postgis-testing.md + - Upgrade spatial database: solutions/postgis-upgrade.md + - LDAP authentication: + - ldap.md + - Upgrade: + - "Major upgrade": major-upgrade.md + - minor-upgrade.md + - migration.md + - Troubleshooting guide: troubleshooting.md + - Uninstall: uninstalling.md + - Release notes: + - "Release notes index": release-notes/release-notes.md + - "2025": + - "16.9 Update": release-notes/release-notes-v16.9.upd.md + - "16.9": release-notes/release-notes-v16.9.md + - "16.8": release-notes/release-notes-v16.8.md + - "2024 (versions 16.6 - 16.1 Update)": + - "16.6": release-notes/release-notes-v16.6.md + - "16.4": release-notes/release-notes-v16.4.md + - "16.3": release-notes/release-notes-v16.3.md + - "16.2": release-notes/release-notes-v16.2.md + - "16.1 Update": release-notes/release-notes-v16.1.upd.md + - "2023 (versions 16.1 - 16.0)": + - "16.1": release-notes/release-notes-v16.1.md + - "16.0 Update": release-notes/release-notes-v16.0.upd.md + - "16.0": release-notes/release-notes-v16.0.md + - Reference: + - Telemetry: telemetry.md + - Licensing: licensing.md + - Trademark policy: trademark-policy.md + diff --git a/requirements.txt b/requirements.txt index f1d3d82d1..d21b46b7b 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,4 +1,13 @@ - +# This file is used to install the required packages for the doc project. +# Ensure you are in the same location/root that the requirements.txt file is in. +# It is recommended to use Windows Powershell in Administrator mode or Linux Terminal to run the commands. +# You can install the required packages using the following command: +# pip install -r requirements.txt +# This will install all the packages listed in this file. +# To update the packages, run the following command: +# pip install --upgrade -r requirements.txt +# To check for outdated packages, run the following command: +# pip list --outdated Markdown mkdocs mkdocs-versioning @@ -14,4 +23,6 @@ mkdocs-section-index mkdocs-htmlproofer-plugin mkdocs-meta-descriptions-plugin mike -Pillow > 10.1.0 \ No newline at end of file +Pillow > 10.1.0 +mkdocs-open-in-new-tab +mkdocs-print-site-plugin diff --git a/snippets/check-etcd.md b/snippets/check-etcd.md new file mode 100644 index 000000000..1bd516fd2 --- /dev/null +++ b/snippets/check-etcd.md @@ -0,0 +1,47 @@ +3. Check the etcd cluster members. Use `etcdctl` for this purpose. Ensure that `etcdctl` interacts with etcd using API version 3 and knows which nodes, or endpoints, to communicate with. For this, we will define the required information as environment variables. Run the following commands on one of the nodes: + + ``` + export ETCDCTL_API=3 + HOST_1=10.104.0.1 + HOST_2=10.104.0.2 + HOST_3=10.104.0.3 + ENDPOINTS=$HOST_1:2379,$HOST_2:2379,$HOST_3:2379 + ``` + +4. Now, list the cluster members and output the result as a table as follows: + + ```{.bash data-prompt="$"} + $ sudo etcdctl --endpoints=$ENDPOINTS -w table member list + ``` + + ??? example "Sample output" + + ``` + +------------------+---------+-------+------------------------+----------------------------+------------+ + | ID | STATUS | NAME | PEER ADDRS | CLIENT ADDRS | IS LEARNER | + +------------------+---------+-------+------------------------+----------------------------+------------+ + | 4788684035f976d3 | started | node2 | http://10.104.0.2:2380 | http://192.168.56.102:2379 | false | + | 67684e355c833ffa | started | node3 | http://10.104.0.3:2380 | http://192.168.56.103:2379 | false | + | 9d2e318af9306c67 | started | node1 | http://10.104.0.1:2380 | http://192.168.56.101:2379 | false | + +------------------+---------+-------+------------------------+----------------------------+------------+ + ``` + +5. To check what node is currently the leader, use the following command + + ```{.bash data-prompt="$"} + $ sudo etcdctl --endpoints=$ENDPOINTS -w table endpoint status + ``` + + ??? example "Sample output" + + ```{.text .no-copy} + +-----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+ + | ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS | + +-----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+ + | 10.104.0.1:2379 | 9d2e318af9306c67 | 3.5.16 | 20 kB | true | false | 2 | 10 | 10 | | + | 10.104.0.2:2379 | 4788684035f976d3 | 3.5.16 | 20 kB | false | false | 2 | 10 | 10 | | + | 10.104.0.3:2379 | 67684e355c833ffa | 3.5.16 | 20 kB | false | false | 2 | 10 | 10 | | + +-----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+ + ``` + + \ No newline at end of file diff --git a/snippets/percona-release-apt.md b/snippets/percona-release-apt.md new file mode 100644 index 000000000..c3a80d194 --- /dev/null +++ b/snippets/percona-release-apt.md @@ -0,0 +1,24 @@ +1. Install the `curl` download utility if it's not installed already: + + ```{.bash data-prompt="$"} + $ sudo apt update + $ sudo apt install curl + ``` + +2. Download the `percona-release` repository package: + + ```{.bash data-prompt="$"} + $ curl -O https://repo.percona.com/apt/percona-release_latest.generic_all.deb + ``` + +3. Install the downloaded repository package and its dependencies using `apt`: + + ```{.bash data-prompt="$"} + $ sudo apt install gnupg2 lsb-release ./percona-release_latest.generic_all.deb + ``` + +4. Refresh the local cache to update the package information: + + ```{.bash data-prompt="$"} + $ sudo apt update + ``` \ No newline at end of file diff --git a/snippets/percona-release-yum.md b/snippets/percona-release-yum.md new file mode 100644 index 000000000..05d669385 --- /dev/null +++ b/snippets/percona-release-yum.md @@ -0,0 +1,5 @@ +Run the following command as the `root` user or with `sudo` privileges: + +```{.bash data-prompt="$"} +$ sudo yum install -y https://repo.percona.com/yum/percona-release-latest.noarch.rpm +``` \ No newline at end of file diff --git a/snippets/services-banner.md b/snippets/services-banner.md deleted file mode 100644 index 2bb683aac..000000000 --- a/snippets/services-banner.md +++ /dev/null @@ -1,13 +0,0 @@ - -

- -## Get expert help { .title } - -If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services. - -

- -[:material-forum-outline: Community Forum](https://forums.percona.com/c/postgresql/25?utm_campaign=Doc%20pages) [:percona-logo: Get a Percona Expert](https://www.percona.com/about/contact) - -

- diff --git a/variables.yml b/variables.yml index c726ba15a..add67c4de 100644 --- a/variables.yml +++ b/variables.yml @@ -1,13 +1,16 @@ # PG Variables set for HTML output # See also mkdocs.yml plugins.with-pdf.cover_subtitle and output_path -release: 'release-notes-v16.4' -dockertag: '16.4' +release: 'release-notes-v16.9' +dockertag: '16.9' pgversion: '16' -pgsmversion: '2.1.0' - +pgsmversion: '2.1.1' date: + 16_9_1: 2025-07-10 + 16_9: 2025-05-29 + 16_8: 2025-02-27 + 16_6: 2024-12-03 16_4: 2024-09-10 pFad - Phonifier reborn

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.

Alternative Proxies:

Contact Us

Contact Us

404 - Not found

Distribution for PostgreSQL

{{ config.site_description }}

16.9 (May 29, 2025)

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.