Skip to main content
Version: v0.14 🚧

Managed Databases

You could also specify a database needed for the application. That can be achieved via a mysql or a postgres module (or bring-your-own-module) in the accessories field in AppConfiguration to achieve that.

You can currently have several databases with different database names for an application at the same time.

Import​

In the examples below, we are using schemas defined in the kam package and the mysql Kusion Module. For more details on KCL package and module import, please refer to the Configuration File Overview.

The import statements needed for the following walkthrough:

import kam.v1.app_configuration as ac
import service
import service.container as c
import mysql
import postgres

The kcl.mod must contain reference to the mysql module or postgres module:

#...

[dependencies]
mysql = { oci = "oci://ghcr.io/kusionstack/mysql", tag = "0.2.0" }
postgres = { oci = "oci://ghcr.io/kusionstack/postgres", tag = "0.2.0" }
#...

Types of Database offerings​

As of version 0.11.0, Kusion supports the following database offerings on the cloud:

  • MySQL and PostgreSQL Relational Database Service (RDS) on AWS
  • MySQL and PostgreSQL Relational Database Service (RDS) on AliCloud

More database types on more cloud vendors will be added in the future.

Alternatively, Kusion also supports creating a database at localhost for local testing needs. A local database is quicker to stand up and easier to manage. It also eliminates the need for an account and any relevant costs with the cloud providers in the case that a local testing environment is sufficient.

info

You do need a local Kubernetes cluster to run the local database workloads. You can refer to Minikube or Kind to get started. To see an end-to-end use case for standing up a local testing environment including a local database, please refer to the Kusion Quickstart.

Cloud Credentials and Permissions​

Kusion provisions databases on the cloud via terraform providers. For it to create any cloud resources, it requires a set of credentials that belongs to an account that has the appropriate write access so the terraform provider can be initialized properly.

For AWS, the environment variables needed:

export AWS_REGION=us-east-1            # replace it with your region
export AWS_ACCESS_KEY_ID="xxxxxxxxxxx" # replace it with your AccessKey
export AWS_SECRET_ACCESS_KEY="xxxxxxx" # replace it with your SecretKey

For AliCloud, the environment variables needed:

export ALICLOUD_REGION=cn-shanghai     # replace it with your region
export ALICLOUD_ACCESS_KEY="xxxxxxxxx" # replace it with your AccessKey
export ALICLOUD_SECRET_KEY="xxxxxxxxx" # replace it with your SecretKey

The user account that owns these credentials would need to have the proper permission policies attached to create databases and security groups. If you are using the cloud-managed policies, the policies needed to provision a database and configure firewall rules are listed below.

For AWS:

  • AmazonVPCFullAccess for creating and managing database firewall rules via security group
  • AmazonRDSFullAccess for creating and managing RDS instances

For AliCloud:

  • AliyunVPCFullAccess for creating and managing database firewall rules via security group
  • AliyunRDSFullAccess for creating and managing RDS instances

Alternatively, you can use customer managed policies if the cloud provider built-in policies don't meet your needs. The list of permissions needed are in the AmazonRDSFullAccess Policy Document and AmazonVPCFullAccess Policy Document. It will most likely be a subset of the permissions in the policy documents.

Configure Database​

Provision a Cloud Database​

Assuming the steps in the Cloud Credentials and Permissions section is setup properly, you can now provision cloud databases via Kusion.

AWS RDS Instance​

To provision an AWS RDS instance with MySQL v8.0 or PostgreSQL v14.0, you can append the following YAML file to your own workspace configurations and update the corresponding workspace with command kusion workspace update. 

runtimes: 
  terraform: 
    random: 
      version: 3.5.1
      source: hashicorp/random
    aws: 
      version: 5.0.1
      source: hashicorp/aws
      region: us-east-1 # Please replace with your own aws provider region

# MySQL configurations for AWS RDS
modules: 
  kusionstack/mysql@0.1.0:
    default: 
      cloud: aws
      size: 20
      instanceType: db.t3.micro
      securityIPs: 
        - 0.0.0.0/0
      suffix: "-mysql"

For KCL configuration file declarations: 

wordpress: ac.AppConfiguration {
    # ...
    accessories: {
        "mysql": mysql.MySQL {
            type:   "cloud"
            version: "8.0"
        }
    }
}

It's highly recommended to replace 0.0.0.0/0 and closely manage the whitelist of IPs that can access the database for security purposes. The 0.0.0.0/0 in the example above or if securityIPs is omitted altogether will allow connections from anywhere which would typically be a security bad practice.

The instanceType field determines the computation and memory capacity of the RDS instance. The db.t3.micro instance type in the example above represents the db.t3 instance class with a size of micro. In the same db.t3 instance family there are also db.t3.small, db.t3.medium, db.t3.2xlarge, etc.

The full list of supported instanceType values can be found here.

You can also adjust the storage capacity for the database instance by changing the size field which is storage size measured in gigabytes. The minimum is 20. More details can be found here.

AliCloud RDS Instance​

To provision an Alicloud RDS instance with MySQL or PostgreSQL, you can append the following YAML file to your own workspace configurations and update the corresponding workspace with command kusion workspace update. Note that AliCloud RDS has several additional fields such as category, subnetID and privateRouting:

runtimes: 
  terraform: 
    random: 
      version: 3.5.1
      source: hashicorp/random
    alicloud:
      version: 1.209.1
      source: aliyun/alicloud
      region: cn-beijing # Please replace with your own alicloud provider region

# MySQL configurations for Alicloud RDS
modules: 
  kusionstack/mysql@0.1.0: 
    default: 
      cloud: alicloud
      size: 20
      instanceType: mysql.n2.serverless.1c
      category: serverless_basic
      privateRouting: false
      subnetID: [your-subnet-id]
      securityIPs: 
        - 0.0.0.0/0
      suffix: "-mysql"

For KCL configuration file declarations: 

wordpress: ac.AppConfiguration {
    # ...
    accessories: {
        "mysql": mysql.MySQL {
            type:   "cloud"
            version: "8.0"
        }
    }
}

We will walkthrough subnetID and privateRouting in the Configure Network Access section.

The full list of supported instanceType values can be found in:

Local Database​

To deploy a local database with MySQL v8.0 or PostgreSQL v14.0:

wordpress: ac.AppConfiguration {
    # ...
    accessories: {
        "mysql": mysql.MySQL {
            type: "local"
            version: "8.0"
        }
    }
}

Database Credentials​

There is no need to manage the database credentials manually. Kusion will automatically generate a random password, set it as the credential when creating the database, and then inject the hostname, username and password into the application runtime.

You have the option to BYO (Bring Your Own) username for the database credential by specifying the username attribute in the workspace.yaml:

modules:
  kusionstack/mysql@0.1.0:
    default:
      # ...
      username: "my_username"

You cannot bring your own password. The password will always be managed by Kusion automatically.

The database credentials are injected into the environment variables of the application container. You can access them via the following env vars:

# env | grep KUSION_DB
KUSION_DB_HOST_WORDPRESS_MYSQL=wordpress.xxxxxxxx.us-east-1.rds.amazonaws.com
KUSION_DB_USERNAME_WORDPRESS_MYSQL=xxxxxxxxx
KUSION_DB_PASSWORD_WORDPRESS_MYSQL=xxxxxxxxx
info

More details about the environment of database credentials injected by Kusion can be found at mysql credentials and connectivity and postgres credentials and connectivity

You can use these environment variables out of the box. Or most likely, your application might retrieve the connection details from a different set of environment variables. In that case, you can map the kusion environment variables to the ones expected by your application using the $() expression.

This example below will assign the value of KUSION_DB_HOST_WORDPRESS_MYSQL into WORDPRESS_DB_HOST, KUSION_DB_USERNAME_WORDPRESS_MYSQL into WORDPRESS_DB_USER, likewise for KUSION_DB_PASSWORD_WORDPRESS_MYSQL and WORDPRESS_DB_PASSWORD:

wordpress: ac.AppConfiguration {
    workload: service.Service {
        containers: {
            wordpress: c.Container {
                image = "wordpress:6.3-apache"
                env: {
                    "WORDPRESS_DB_HOST": "$(KUSION_DB_HOST_WORDPRESS_MYSQL)"
                    "WORDPRESS_DB_USER": "$(KUSION_DB_USERNAME_WORDPRESS_MYSQL)"
                    "WORDPRESS_DB_PASSWORD": "$(KUSION_DB_PASSWORD_WORDPRESS_MYSQL)"
                }
                # ...
            }
        }
        # ...
    }
    accessories: {
        # ...
    }
}

Configure Network Access​

You can also optionally configure the network access to the database as part of the AppConfiguration. This is highly recommended because it dramatically increases the security posture of your cloud environment in the means of least privilege principle.

The securityIPs field in the Database schema declares the list of network addresses that are allowed to access the database. The network addresses are in the CIDR notation and can be either a private IP range (RFC-1918 and RFC-6598 address) or a public one.

If the database need to be accessed from a public location (which should most likely not be the case in a production environment), securityIPs need to include the public IP address of the traffic source (For instance, if the RDS database needs to be accessed from your computer).

To configure AWS RDS to restrict network access from a VPC with a CIDR of 10.0.1.0/24 and a public IP of 103.192.227.125:

modules: 
  kusionstack/mysql@0.1.0: 
    default: 
      cloud: aws
      # ...
      securityIPs: 
        - "10.0.1.0/24"
        - "103.192.227.125/32"

Depending on the cloud provider, the default behavior of the database firewall settings may differ if omitted.

Subnet ID​

On AWS, you have the option to launch the RDS instance inside a specific VPC if a subnetID is present in the application configuration. By default, if subnetID is not provided, the RDS will be created in the default VPC for that account. However, the recommendation is to self-manage your VPCs to provider better isolation from a network security perspective.

On AliCloud, the subnetID is required. The concept of subnet maps to VSwitch in AliCloud.

To place the RDS instance into a specific VPC on Alicloud:

modules: 
  kusionstack/mysql@0.1.0: 
    default: 
      cloud: alicloud
      # ...
      subnetID: "subnet-xxxxxxxxxxxxxxxx"

Private Routing​

There is an option to enforce private routing on certain cloud providers if both the workload and the database are running on the cloud.

On AliCloud, you can set the privateRouting flag to True. The database host generated will be a private FQDN that is only resolvable and accessible from within the AliCloud VPCs. Setting privateRouting flag to True when type is aws is a no-op.

To enforce private routing on AliCloud:

modules: 
  kusionstack/mysql@0.1.0: 
    default: 
      cloud: alicloud
      # ...
      privateRouting: true

Kusion will then generate a private FQDN and inject it into the application runtime as the environment variable KUSION_DB_HOST_<DATABASE_NAME> for the application to use. A complete list of Kusion-managed environment variables for mysql database can be found here.

Otherwise when using the public FQDN to connect to a database from the workload, the route will depend on cloud provider's routing preference. The options are generally either:

  • Travel as far as possible on the cloud provider's global backbone network, or also referred to as cold potato routing, or
  • Egress as early as possible to the public Internet and re-enter the cloud provider's datacenter later, or also referred to as hot potato routing

The prior generally has better performance but is also more expensive.

You can find a good read on the AWS Blog or the Microsoft Learn.