A data warehouse is a centralized repository of business data coming from varied sources. It’s a process of collecting and managing huge amounts of data into one comprehensive database. The database size grows every day with increasing demand of integrated and analytical data to provide end-to-end business insights.
But users have had a data realization: some data is mission critical and need to be available constantly while other data may not be as importance to business and is available only for reporting and historical requirements.
In response to the unprecedented supply of workloads and the challenge of ever-changing & diverse data, either on premises or in the cloud, many customers have chosen to deal with a unified multi-system environment, depending on their data and workload requirements, to enhance business value with transparent and available systems. When users are analyzing business-critical applications, maintaining two copies are must-have for high availability and disaster recovery protection in order to run daily business operations.
Here, the challenge is not managing the various combinations of data in collaboration, but how to keep those huge amounts of data in sync to ensure that all records are consistent and available all the time in the case of a multi-system environment.
This is where unity comes into the picture. Unity is a strong product which integrates with multisystem environments and turns them into an orchestrated analytic environment. It provides the critical role of managing multi-system Teradata environments. It enables multiple active systems to work together with proper consistency and durability. It provides all features and capabilities required to keep the Teradata systems in sync which helps us to load balance and maximize utilization of both systems simultaneously. Organizations with multiple systems understand that integration and centralized management provides value.
The Purpose of Unity
Teradata Unity has power to parse the queries and determine the system with the most updated data, which could satisfy the incoming request’s needs via workload balancing and the user/query routing rule. It provides end-to-end monitoring of jobs and workflow via ecosystem manager and moves the data across systems intelligently via data mover.
To automate the process and simplify the multiple system tasks, the Teradata Unity portfolio comprises of Unity, Unity Data Mover, and Unity Ecosystem Manager. Altogether, these products provide greater integration and value to the customer to enable the effective management of multiple systems.
Unity: Teradata Dual Systems
It provides an environment that promises to fulfill high availability and disaster recovery requirements of a company to ensure users’ SLAs are met.
Unity has the capability to route and re-route queries intelligently across the systems. It is important to route the users and sessions to the correct system. In case of a managed routing, a user’s requirement is to maintain consistent data across the systems. If one system becomes non-functional, users can get the same output with an alternate system via routing. This process is completely oblivious to the end users. Routing will be managed internally through routing rules.
There are two different scenarios to consider when routing queries:
Most of users are ETL users where data synchronization is needed across the system, or
Most of the users are reporting users where synchronization is not required.
The key differences between Managed Routing and Passive Routing are:
Unity uses different synchronizing methods to keep the multisystem data in sync.
It sends the SQL code (DDL, DCL and DML) to all participating system (2 systems, for active-active scenario). Unity maintains the data sequence and automatically dispatches the queries as they arrive, maintaining the same order (e.g., an Update/insert query will make changes on all participating systems at the same time to make a consistent data model).
Change Data Multicast (CDM):
It’s a new capability of the Teradata Vantage Advanced SQL Engine and only works if the system is CDM enabled. CDM also replicates the changes like SQL Multicast but only for non-deterministic functions (e.g., Random, Sample, Identity Columns, Top, Updated Cursor) which can produce different results if it runs at the same time.
Unity Ecosystem Manager:
Teradata Unity works with Ecosystem Manager to facilitate the monitoring of each component of the Ecosystem and troubleshoot any issues. Ecosystem manager enable us to monitor servers, tables, daemons, jobs, workflows, applications and even management dashboards. The DBA can easily access the ecosystem manager to check the status of the overall system. Unity Ecosystem Manager ensures that data is processed in a cohesive manner to deliver the highest-quality results.
Unity Bulk Load:
It provides the bulk loading capability to one or more Teradata systems using TPT Load, Teradata Fast Load, Fast Export, Multiload, and JDBC Fast load. It automatically figures out the system to sends the loads to and maintains the data sequencing to keep the data accurate and up to date.
Unity Data Mover:
Data Mover is ideal for moving data between multiple systems at high speed. It can copy or move full data or partial data for synchronizing tables. It uses Teradata load utilities, Arc, DSA etc. to perform it. Before copying any data, changed data needs to be identified which can be easily done by the timestamp column.
Why is Unity Needed?
In case of failure or a restart on a system, unity can route the sessions and users to another system so users can have uninterrupted and transparent access to ready, consistent, up-to-date analytic data at ease. Unity enables efficient and high-value data synchronization across multiple Teradata systems, providing high data availability and protection against data loss during system or data center failures.
Without Unity, users will not be able to access data in case of any data failure. Data recovery is possible only if a backup is taken. Restoring data from backup files will take time depending on the size of the backup. Considering the size of data in a data warehouse, it may take several hours to days of downtime in the case of a major data failure. Business users may be fraught with delays and inevitable data loss. It may lead to loss of a customer’s trust and revenue. So, if you have strict SLAs or if you cannot afford any downtime, Unity is the product you should opt for.