First steps

After going through the Installation section and having installed all the operators, deploy a Hive metastore cluster and it’s dependencies. Afterward you can verify that it works.

Setup

Several requirements should have already been installed in the Installation guide:

S3Connection

In order to connect Hive to MinIO we need to create several files (or concat in one file).

An S3Connection to connect to MinIO (hive-minio-s3-connection.yaml)

---
apiVersion: s3.stackable.tech/v1alpha1
kind: S3Connection
metadata:
  name: minio
spec:
  host: minio
  port: 9000
  accessStyle: Path
  credentials:
    secretClass: hive-s3-secret-class

Credentials for the S3Connection to log into MinIO (hive-minio-credentials.yaml)

---
apiVersion: v1
kind: Secret
metadata:
  name: hive-s3-secret
  labels:
    secrets.stackable.tech/class: hive-s3-secret-class
stringData:
  accessKey: hive
  secretKey: hivehive

A SecretClass for the credentials to the Minio. The credentials were defined in the installation of Minio (hive-minio-credentials-secret-class.yaml)

---
apiVersion: secrets.stackable.tech/v1alpha1
kind: SecretClass
metadata:
  name: hive-s3-secret-class
spec:
  backend:
    k8sSearch:
      searchNamespace:
        pod: {}

And lastly the actual Apache Hive cluster definition (hive-postgres-s3.yaml). Note how it references the minio S3 object

---
apiVersion: hive.stackable.tech/v1alpha1
kind: HiveCluster
metadata:
  name: hive-postgres-s3
spec:
  image:
    productVersion: 4.0.1
  clusterConfig:
    database:
      connString: jdbc:postgresql://postgresql:5432/hive
      credentialsSecret: hive-credentials
      dbType: postgres
    s3:
      reference: minio
  metastore:
    roleGroups:
      default:
        replicas: 1
---
apiVersion: v1
kind: Secret
metadata:
  name: hive-credentials
type: Opaque
stringData:
  username: hive
  password: hive

Once the files are created apply them to the cluster:

kubectl apply -f hive-minio-credentials.yaml
kubectl apply -f hive-minio-credentials-secret-class.yaml
kubectl apply -f hive-minio-s3-connection.yaml
kubectl apply -f hive-postgres-s3.yaml

Verify that it works

Make sure that all the Pods in the StatefulSets are ready:

kubectl get statefulset

The output should show all pods ready:

NAME                                 READY   AGE
hive-postgres-s3-metastore-default   1/1     48s

For further testing we recommend to use e.g. the python hive-metastore-client in order to e.g. create tables or connect a product like Trino via the Stackable Trino Operator.

What’s next

Have a look at the usage guide to find out more about the features of the Operator.

pFad - Phonifier reborn

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.

Alternative Proxies:

First steps

Setup

S3Connection

Verify that it works

What’s next

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.