if: steps.pgml_extension_changed.outputs.PGML_EXTENSION_CHANGED_FILES != '0'

run: |

- name: Run tests

if: steps.pgml_extension_changed.outputs.PGML_EXTENSION_CHANGED_FILES != '0'

run: |

* **Model Serving** - _**GPU accelerated**_ inference engine for interactive applications, with no additional networking latency or reliability costs.

* **Model Store** - Download _**open-source**_ models including state of the art LLMs from HuggingFace, and track changes in performance between versions.

* **Model Training** - Train models with _**your application data**_ using more than 50 algorithms for regression, classification or clustering tasks. Fine tune pre-trained models like LLaMA and BERT to improve performance.

<figure><img src=".gitbook/assets/ml_system.svg" alt="Machine Learning Infrastructure (2.0) by a16z"><figcaption><p>PostgresML handles all of the functions typically performed by a cacophony of services, <a href="https://a16z.com/emerging-architectures-for-modern-data-infrastructure/">described by a16z</a></p></figcaption></figure>

* **pgml** is an open source extension for PostgreSQL. It adds support for GPUs and the latest ML & AI algorithms _**inside**_ the database with a SQL API and no additional infrastructure, networking latency, or reliability costs.

* **PgCat** is an open source proxy pooler for PostgreSQL. It abstracts the scalability and reliability concerns of managing a distributed cluster of Postgres databases. Client applications connect only to the proxy, which handles load balancing and failover, _**outside**_ of any single database.

<figure><img src=".gitbook/assets/architecture.png" alt="PostgresML architectural diagram" width="275"><figcaption><p>A PostgresML deployment at scale</p></figcaption></figure>

SDK clients can perform advanced machine learning tasks in a single SQL request, without having to transfer additional data, models, hardware or dependencies to the client application. For example:

* Forecasting timeseries data for key metrics with complex metadata

* Fraud and anomaly detection with application data

We can host your AI database in our cloud, or you can run our Docker image locally with PostgreSQL, pgml, pgvector and NVIDIA drivers included.

* [Overview](introduction/apis/client-sdks/getting-started.md)

* [Collections](introduction/apis/client-sdks/collections.md)

* [Pipelines](introduction/apis/client-sdks/pipelines.md)

* [Tutorials](introduction/apis/client-sdks/tutorials/README.md)

* [Semantic Search](introduction/apis/client-sdks/tutorials/semantic-search.md)

There are 3 different deployment strategies available:

| Strategy | Description |

| `most_recent` | The most recently trained model for this project is immediately deployed, regardless of metrics. |

| `best_score` | The model that achieved the best key metric score is immediately deployed. |

| `rollback` | The model that was deployed before to the current one is deployed. |

(1 row)

In case the new model isn't performing well in production, it's easy to rollback to the previous version. A rollback creates a new deployment for the old model. Multiple rollbacks in a row will oscillate between the two most recently deployed models, making rollbacks a safe and reversible operation.

Embeddings are a numeric representation of text. They are used to represent words and sentences as vectors, an array of numbers. Embeddings can be used to find similar pieces of text, by comparing the similarity of the numeric vectors using a distance measure, or they can be used as input features for other machine learning models, since most algorithms can't use text directly.

description: >-

| Parameter | Example | Description |

| --------------- | ----------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |

| `project_name` | `'Search Results Ranker'` | An easily recognizable identifier to organize your work. |

| `relation_name` | `'public.search_logs'` | The Postgres table or view where the training data is stored or defined. |

| `y_column_name` | `'clicked'` | The name of the label (aka "target" or "unknown") column in the training table. |

| `algorithm` | `'xgboost'` | <p>The algorithm to train on the dataset, see the task specific pages for available algorithms:<br><a data-mention href="regression.md">regression.md</a></p><p><a data-mention href="classification.md">classification.md</a><br><a data-mention href="clustering.md">clustering.md</a></p> |

layout:

title:

visible: true

Quantization allows PostgresML to fit larger models in less RAM. These algorithms perform inference significantly faster on NVIDIA, Apple and Intel hardware. Half-precision floating point and quantized optimizations are now available for your favorite LLMs downloaded from Huggingface.

Going beyond 16-bit down to 8 or 4 bits is possible, but not with hardware accelerated floating point operations. If we want hardware acceleration for smaller types, we'll need to use small integers w/ vectorized instruction sets. This is the process of _quantization_. Quantization can be applied to existing models trained with 32-bit floats, by converting the weights to smaller integer primitives that will still benefit from hardware accelerated instruction sets like Intel's [AVX](https://en.wikipedia.org/wiki/Advanced\_Vector\_Extensions). A simple way to quantize a model can be done by first finding the maximum and minimum values of the weights, then dividing the range of values into the number of buckets available in your integer type, 256 for 8-bit, 16 for 4-bit. This is called _post-training quantization_, and it's the simplest way to quantize a model.