Allow Multi-AZ deployments in your Amazon Redshift knowledge warehouse

Amazon Redshift is a totally managed, petabyte scale cloud knowledge warehouse that lets you analyze massive datasets utilizing commonplace SQL. Knowledge warehouse workloads are more and more getting used with business-critical analytics functions that require the very best ranges of availability and resiliency. Amazon Redshift is a cloud-based knowledge warehouse that already helps many restoration capabilities to deal with unexpected outages and decrease downtime. Amazon Redshift RA3 occasion sorts retailer their knowledge in Redshift Managed Storage (RMS), which is backed by Amazon Easy Storage Service (Amazon S3), which is extremely out there and sturdy by default. Amazon Redshift additionally helps automated backups that can be utilized to get well a knowledge warehouse, automated remediation of failures, and the power to relocate a cluster to a different Availability Zone with out modifications to functions. Though many purchasers profit from these options, enterprise knowledge warehouse prospects require a low RTO and better availability to help their enterprise continuity with minimal impression to functions.

Amazon Redshift now helps Multi-AZ deployments (Preview) for provisioned RA3 clusters. Multi-AZ deployments help working your knowledge warehouse in a number of Availability Zones concurrently and might proceed working in unexpected failure eventualities. A Multi-AZ deployment is meant for purchasers with business-critical analytics functions that require the very best ranges of availability and resiliency.

A Redshift Multi-AZ deployment leverages compute assets in a number of AZs to scale knowledge warehouse workload processing.  In conditions the place there’s a excessive stage or concurrency Redshift will mechanically leverage the assets in each AZs to scale the workload for each learn and write requests utilizing active-active processing.

On this put up, we present the right way to configure an Amazon Redshift Multi-AZ deployment in a number of Availability Zones.

Overview of answer

We offer a walkthrough of the right way to carry out a Multi-AZ deployment for an Amazon Redshift cluster utilizing the AWS Administration Console. We additionally present a walkthrough on the right way to take a look at fault tolerance of an Amazon Redshift Multi-AZ knowledge warehouse and monitor queries in your Multi-AZ deployment.

Single-AZ vs. Multi-AZ deployment

Amazon Redshift requires a cluster subnet group to create a cluster in your VPC. The cluster subnet group consists of details about the VPC ID and an inventory of subnets in your VPC. Whenever you launch a cluster, Amazon Redshift both creates a default cluster subnet group mechanically otherwise you select a cluster subnet group of your alternative in order that Amazon Redshift can provision your cluster in one of many subnets within the VPC. You’ll be able to configure your cluster subnet group so as to add subnets from completely different Availability Zones that you really want Amazon Redshift to make use of for cluster deployment.

All Amazon Redshift clusters at the moment are created and located in a specific Availability Zone inside an AWS Area and thus known as Single-AZ deployments. For a Single-AZ deployment, Amazon Redshift selects the subnet from one of many Availability Zones inside a Area and deploys the cluster there. You’ll be able to select an Availability Zone for deployment, and Amazon Redshift will deploy your cluster within the chosen Availability Zone primarily based on the subnets supplied.

However, a multi-AZ deployment is provisioned in a number of Availability Zones concurrently. For a Multi-AZ deployment, Amazon Redshift mechanically selects two subnets from two completely different Availability Zones and deploys an equal variety of compute nodes in every Availability Zone. All these compute nodes are utilized through a single endpoint as compute nodes from each Availability Zones are used for workload processing. 

As proven within the following diagrams, Amazon Redshift deploys a cluster in a single Availability Zone for Single-AZ deployment, and a number of Availability Zones for Multi-AZ deployment.

Auto restoration of multi-AZ deployment

Within the unlikely occasion of an Availability Zone failure, Amazon Redshift Multi-AZ deployments proceed to serve your workloads by mechanically utilizing assets within the different Availability Zone. You aren’t required to make any software modifications to keep up enterprise continuity throughout unexpected outages since a multi-AZ deployment is accessed as a single knowledge warehouse with one endpoint. Amazon Redshift Multi-AZ deployments are designed to make sure there isn’t a knowledge loss, and you’ll question all knowledge dedicated up till the purpose of failure.

As proven within the under diagram, if there’s an unlikely occasion that causes compute nodes in AZ1 to fail, then a multi-AZ deployment mechanically recovers to make use of compute assets in AZ2. Amazon Redshift may also mechanically provision similar compute nodes in one other availability zone (AZ3) to proceed working concurrently in two Availability zones (AZ2 and AZ3).

Amazon Redshift Multi-AZ deployment is just not solely used for cover in opposition to the potential of Availability Zone failures, however it will probably additionally maximize your knowledge warehouse efficiency by mechanically distributing workload processing throughout a number of Availability Zones. A Multi-AZ deployment will at all times course of a person question utilizing compute assets solely from one Availability Zone, however it will probably mechanically distribute processing of a number of simultaneous queries to each Availability Zones to extend total efficiency for prime concurrency workloads.

It’s a great apply to arrange automated retries in your extract, rework, and cargo (ETL) processes and dashboards in order that they are often reissued and served by the cluster within the secondary Availability Zone when an unlikely failure occurs within the major Availability Zone. If a connection is dropped, it will probably then be retried or reestablished instantly. As well as, queries and hundreds that have been working within the failed Availability Zone will likely be aborted. New queries issued at or after a failure happens might expertise run delays whereas the multi-AZ knowledge warehouse is being are recovered to a two AZ setup.

Create a brand new Multi-AZ deployment from the console

You’ll be able to simply create a brand new multi-AZ deployments via Amazon Redshift console. Amazon Redshift will deploy the identical variety of nodes in every of the 2 Availability Zones for a Multi-AZ deployment. All nodes of a multi-AZ deployment can carry out learn and write workload processing throughout regular operation. A Multi-AZ deployment will help solely provisioned RA3 clusters.

Observe these steps to create an Amazon Redshift provisioned cluster in a number of Availability Zones:

1. On the Amazon Redshift console, within the navigation pane, select Clusters.
2. A banner shows on the Clusters checklist web page that introduces preview mode. Select the button Create preview cluster to open the create cluster web page.
3. For Preview monitor, select preview_2022.
4. We suggest getting into a reputation for the cluster that signifies that it’s on a preview monitor. Select choices in your cluster, together with choices labeled as -preview, for the options you wish to take a look at.

For common details about creating clusters, see Making a cluster.

5. Select one of many RA3 node sorts on the Node kind drop-down menu. The Multi-AZ deployment possibility solely turns into out there if you select an RA3 node kind.
6. For Multi-AZ deployment, choose Sure.
7. For Variety of nodes per AZ, enter the variety of nodes that you simply want in your cluster.

8. Underneath the Database configurations, select Admin consumer identify and Admin consumer password.
9. Flip Use defaults on subsequent to Further configurations to switch the default settings.
10. Underneath Community and safety, specify the next:
    1. For Digital personal cloud (VPC), select the VPC you wish to deploy the cluster in.
    2. For VPC safety teams, both depart as default or add the safety teams of your alternative.
    3. For Cluster subnet group, both depart as default or add a cluster subnet group of your alternative. For a Multi-AZ deployment, a cluster subnet group should embody one subnet every from no less than three or extra completely different Availability Zones.

For common details about managing cluster subnet teams, see Cluster subnet teams

11. Underneath Database configuration, for Database port, you both use the default worth 5439 or select a worth from the vary of 5431–5455 and 8191–8215.
12. Underneath Database configuration, within the Database encryption part, to make use of a customized AWS Key Administration Service (AWS KMS) key aside from the default KMS key, select Customise encryption settings. This feature is deselected by default.
13. Underneath Select an AWS KMS key, you possibly can both select an current KMS key, or select Create an AWS KMS key to create a brand new KMS key.

For extra data to create key utilizing KMS, consult with Creating keys.

14. Select Create cluster.

When the cluster creation succeeds, you possibly can view the small print on the cluster particulars web page.

Underneath Common data, you possibly can see Multi-AZ as Sure.

On the Properties tab, below Community and safety settings, you could find the small print on the first and secondary Availability Zone.

Convert a Single-AZ deployment to Multi-AZ deployment

To transform an current Single-AZ deployment to a Multi-AZ deployment, you possibly can restore from a snapshot to configure it right into a Multi-AZ knowledge warehouse. When migrating to a Multi-AZ deployment from an current Single-AZ deployment, sustaining efficiency of a single question might require the identical variety of nodes used within the present Single-AZ deployment to be provisioned in each Availability Zones, leading to doubling the quantity of cluster nodes wanted when migrating to Multi-AZ to make sure that single question efficiency is maintained.

Full the next steps to create a Multi-AZ deployment restored from a snapshot:

1. On the Amazon Redshift console, within the navigation pane below Clusters, select Snapshots.
2. Choose the snapshot to make use of.
3. The snapshot must be encrypted with a view to restore to a Multi-AZ deployment.
4. On the Restore snapshot menu, select Restore to provisioned cluster.

5. Select the Preview mode.
6. For Preview monitor, select preview_2022
7. We suggest getting into a reputation for the cluster that signifies that it’s on a preview monitor. Select choices in your cluster, together with choices labeled as -preview, for the options you wish to take a look at.

For common details about creating clusters, see Making a cluster.

8. Just be sure you select one of many RA3 node sorts on the Node kind drop-down menu. The Multi-AZ deployment possibility solely turns into out there if you selected an RA3 node kind.
9. For Multi-AZ deployment, choose Sure.
10. For Variety of nodes per AZ, enter the variety of nodes that you simply want in your cluster.

11. Scroll all the way down to Further configurations, broaden Community and safety, just remember to both settle for the default for Cluster subnet group or select one other certainly one of your alternative. For a Multi-AZ deployment, a cluster subnet group should embody one subnet every from no less than three or extra completely different Availability Zones.
12. Underneath Further configurations, broaden Database configurations.
13. Underneath Database encryption, to make use of a customized KMS key aside from the default KMS key, select Customise encryption settings. This feature is deselected by default.
14. Underneath Select an AWS KMS key, you possibly can both select a KMS key or enter an ARN. Or, you possibly can select Create an AWS Key Administration Service key to create a key.

15. Select Restore cluster from snapshot.

When the cluster restoration succeeds, you possibly can view the small print on the cluster particulars web page.

Check fault tolerance of your multi-AZ knowledge warehouse

You’ll be able to take a look at the fault tolerance of your Amazon Redshift Multi-AZ deployment by injecting a failure that causes compute nodes in a single Availability Zone to develop into unavailable. Amazon Redshift detects this occasion and triggers an automated restoration. When the cluster efficiently recovers, Multi-AZ deployment turns into out there. Your Multi-AZ deployment additionally mechanically provisions new compute nodes in one other Availability Zone as quickly as it’s out there.

Let’s take a look at the fault tolerance of the Amazon Redshift Multi-AZ deployment.

1. On the Amazon Redshift console, select Clusters within the navigation pane.
2. Navigate to the cluster element web page
3. On the Actions menu, select Inject Failure (Public Preview).

4. When prompted, select Verify.

After the cluster is again to Out there standing, you possibly can observe that the first and secondary Availability Zones have modified.

The next screenshot exhibits the standing earlier than injecting failure.

The next screenshot exhibits the standing after injecting failure.

Monitor queries for Multi-AZ deployments

A Multi-AZ deployment makes use of compute assets which might be deployed in each Availability Zones and might proceed working within the occasion that the assets in a given Availability Zone aren’t out there. All of the compute assets are used always, which permits full operation throughout two Availability Zones in each learn and write operations.

You’ll be able to question SYS_ views within the pg_catalog schema to observe Multi-AZ question runs. The SYS_ views cowl question run actions and stats from major and secondary clusters.

The next are the system tables within the SYS_ view checklist:

Observe these steps to observe the question run on Multi-AZ deployment from the Amazon Redshift Console:

1. On the Amazon Redshift console, hook up with the database in your Multi-AZ deployment and run queries via the question editor.
2. Run any pattern question on the Multi-AZ Redshift deployment.
3. For a Multi-AZ deployment, you possibly can determine a question and the Availability Zone the place it’s being run (working on the first cluster or secondary availability zone) through the use of the compute_type column within the SYS_QUERY_HISTORY desk. The legitimate values for the compute kind column are as follows:
   1. major – When run on major availability zone within the Multi-AZ deployment.
   2. secondary – When run on secondary availability zone within the Multi-AZ deployment.

The next is a pattern question utilizing the compute_type column to observe a question:

dev=# choose (compute_type) as compute_type, left(query_text, 50) query_text from sys_query_history order by start_time desc;

 compute_type | query_text
--------------+----------------------------------------------------
 secondary    | choose rely(*) from t1;

You too can entry the question historical past from the console to investigate your question diagnostics.

4. On the Question monitoring tab, select Hook up with database.

5. For Authentication, select Non permanent credentials
6. For Database identify, enter the database identify (for instance, dev).
7. For Database consumer, enter the database consumer identify (for instance, awsuser).
8. Select Join.

After you’re linked, below Question Monitoring, on the Question historical past tab, you possibly can view all of the queries and hundreds, as proven within the following screenshot.

Underneath Metric filters, you should utilize the varied filters within the Further filtering choices part to view question historical past primarily based on Time interval, Customers, Databases, or SQL instructions.

There are a number of limitations when working with Amazon Redshift Multi-AZ in preview mode, refer right here for the restrictions.

Buyer suggestions

Janssen Prescribed drugs, a subsidiary of Johnson & Johnson, researches and manufactures medicines with a give attention to the altering wants of sufferers and the healthcare business. “Janssen Pharmaceutical makes use of Amazon Redshift to allow vital insights that drive necessary enterprise choices for our knowledge scientists, knowledge stewards, enterprise customers, and exterior stakeholders. With Amazon Redshift Multi-AZ, we could be assured that our knowledge warehouse will at all times be out there with none disruptions that may delay impression our capability to make vital enterprise choices.” – Shyam Mohapatra, Director of Info Expertise – Janssen Pharmaceutical Corporations of Johnson & Johnson

Conclusion

This put up demonstrated the right way to configure an Amazon Redshift Multi-AZ deployment in a number of Availability Zones and take a look at the fault tolerance of your workloads throughout an unlikely failure of an Availability Zone. Amazon Redshift Multi-AZ deployment additionally helps enhance total efficiency of your knowledge warehouse as a result of compute nodes in each Availability Zones are used for learn and write operations. Amazon Redshift Multi-AZ knowledge warehouse helps meets the calls for of consumers with enterprise vital analytics functions that require the very best ranges of availability and resiliency.

For extra particulars, refer Configuring Multi-AZ deployment.

Concerning the Authors

Ranjan Burman is an Analytics Specialist Options Architect at AWS. He focuses on Amazon Redshift and helps prospects construct scalable analytical options. He has greater than 16 years of expertise in several database and knowledge warehousing applied sciences. He’s keen about automating and fixing buyer issues with cloud options.

Jeff Sosa leads the Redshift product administration group answerable for the core redshift compute and storage platform, availability, backup/restoration and catastrophe restoration areas. Jeff has been at AWS for over 3 years and has centered on high-scale distributed methods processing and storage all through his 20 12 months profession in product administration.

Saurav Das is a part of the Amazon Redshift Product Administration group. He has greater than 16 years of expertise in working with relational databases applied sciences and knowledge safety. He has a deep curiosity in fixing buyer challenges centered round excessive availability and catastrophe restoration.

Anusha Challa is a Senior Analytics Specialist Options Architect centered on Amazon Redshift. She has helped many purchasers construct large-scale knowledge warehouse options within the cloud and on premises. She is keen about knowledge analytics and knowledge science.

Nita Shah is an Analytics Specialist Options Architect at AWS primarily based out of New York. She has been constructing knowledge warehouse options for over 20 years and focuses on Amazon Redshift. She is concentrated on serving to prospects design and construct enterprise-scale well-architected analytics and resolution help platforms.

Suresh Patnam is a Principal BDM – GTM AI/ML Chief at AWS. He works with prospects to construct IT technique, making digital transformation via the cloud extra accessible through the use of knowledge and AI/ML. In his spare time, Suresh enjoys enjoying tennis and spending time together with his household.