Modern Data Architecture - 7 Key Characteristics

Modern Data Architecture – 7 Key Characteristics

Venkat Obillaneni

Cloud Data Engineer at Aezion, Inc. | + posts

Venkat Obillaneni is Aezion Cloud Data Engineer and Data Practice Lead.

With modern data architecture (MDA), a company can be empowered to fully exploit its data through the use of emerging technologies, including artificial intelligence (AI), automation, blockchain, and Internet of Things (IoT). Alongside modern data architecture, an organization should pursue a business model that supports processes, people, and technology in a way that is optimized for the company’s goals.

Ultimately, implementing modern data architecture is admirable, but it’s no easy feat. The marker of good modern data architecture is the ability to keep up with evolving and emerging technologies, including those of tomorrow, and that’s difficult to do.

Staying ahead of the latest technologies is hard enough, but try to implement MDA in a way that enables efficient frameworks for making use of those technologies and you’re going to run into a number of roadblocks along the way. So, let’s explore the characteristics that make modern data architecture so powerful with advice to make achieving MDA possible at your organization.

#1 Cloud-native isn’t optional.

MDA relies on a cloud-native environment in order to leverage new technologies without limitation. A cloud-native environment seeks to work around the challenges of standard cloud environments, giving way to a highly available, highly secure, highly scalable solution for all of your company’s data.

#2 Robust data pipelines are a must.

Your modern data architecture has to be capable of supporting real-time data streams while being able to batch process bursts of data consisting of events, cognitive analytics, intelligent workflows, and so on. This requires robust, highly scalable data pipelines in order to allow business processes to be efficiently managed. All the while, fundamentals like cost, security, and compliance need to be accommodated from the start.

#3 Seamless integrations should be easy.

Modern data architecture does not necessarily replace data, functionality, or services that you have already found a good solution for from another vendor or even in the form of a legacy solution. That’s why being able to seamlessly integrate with existing solutions is a must for MDA. Your MDA should be optimized to support data sharing across locations and systems without having to create a new interface for each one.

#4 Designed to enforce data governance.

Most companies have adopted the concept that whoever creates the data owns it, which gives rise to the need for self-service data access. Your MDA needs to be able to actively and automatically validate, classify, manage, and govern all of its data while ensuring complete visibility into all data and metadata. This often comes in the form of a searchable data catalog that’s being constantly updated, allowing for instant data discovery across your organization.

Ultimately, your MDA should enforce your data governance strategy to make integrating, curating, and utilizing data effortless for all users. Fail to design governance enforcement into your MDA and you’ll begin to see data swamps and costly post-processing cleanup initiatives that drain resources, compromise data quality and hinder data accessibility.

#5 Removes dependencies with loose coupling.

Modern data architecture should be designed in a way that minimizes dependencies. This means each service has a set of functions it needs to perform, and it should be able to do it without depending on any other service in the architecture. This loose coupling helps ensure that services conform to portability standards while allowing new functionality to be added rapidly. The extensible framework behind MDA allows companies to make use of the latest technology, like machine learning, robotic process automation, and more.

#6 Centers on a common business information model.

Your MDA should center around a common business information model (BIM). This means that every artifact, topic, and microservice within your MDA should be based on a business data domain that your BIM defines. Additionally, your MDA should be triggered when certain business events occur, automatically initiating processes and workflows. By following these best practices, you’ll end up with a consistent and sound architecture that supports rapid adoption.

#7 Balances cost and purpose.

One of the most important characteristics of modern data architecture is that it does not try to use every technology. Rather, MDA enables a business to apply the best technology for each problem it is facing, ensuring complete visibility and control while helping to balance cost with purpose. This level of flexibility where a business is able to handpick a solution for an individual problem and seamlessly integrate it into the tech stack demonstrates where the power of MDA truly lies.

Visualizing an MDA

Now that we’ve outlined the 7 characteristics of an MDA, what does one look like? Let’s take a look!

Modern data architecture will always begin with data sources. The path from there varies from one provider to the next, but Kappa offers one of the simplest constructs. In Kappa, every data point is treated like a streaming event and it all flows down the same path. This approach allows you to travel to any point in time and analyze the state of your data at that moment.

A Kappa-powered MDA collects event data from your various data sources (i.e., CRM, HR, operations) and sends it to a streaming ingestion service, like Apache Kafka or Azure Events Hub. This acts as a checkpoint, helping to prevent data loss, and allows data to be queried merely seconds after it has been published. From there, the data moves into an analytics service, like Python or Apache Storm.

The next layer in an MDA is the storage layer. In the case of Kappa, the storage layer consists simply of a data lake, like Hadoop or Amazon S3. Once stored, the data can be pulled into the visualization and query layer, moving into a data warehouse (e.g., RedShift, Snowflake, etc.). This is where data finally takes separate paths.

Some data leaves the warehouse and goes straight to visualization tools like Power BI or Tableau. Other data moves through a virtualization layer and then moves into applications. Data may also be sent to an app after visualization, or vice versa. Visualization and applications represent the “consumption layer,” and the end of the MDA pathway.

Towards your own MDA

So, how can your business design the best MDA for its needs? It can be difficult to know where to begin, but Aezion can help you take the first step. Contact us today to discuss how we can help your business solve its current challenges and implement MDA in a way that supports future innovation and growth.

Venkat Obillaneni

Cloud Data Engineer at Aezion, Inc. | + posts

Venkat Obillaneni is Aezion Cloud Data Engineer and Data Practice Lead.

Blog