CAI SIX PART WEBCAST SERIES | STEP 4
About 7 in 10 business organizations view data as an essential asset in their operations, yet only about 15% of business managers trust the quality of their data.
At the same time, consumers, governments, and corporations all worry about the security of their data—with good reason.
Rapid advances in digital technologies only magnify the urgent need to resolve these and similar concerns. How do savvy IT leaders increase trust, bolster security, assure accessibility, support business plans, and stimulate innovation?
The answer rests in creating a future-proof data architecture that blends the potentials of people, processes, new tools, and sound governance in ways that result in better business analytics. CAI, which has provided business technology services for two decades, explored this topic as part of its six-part series, "Activating Data & Analytics for Real Business Value Creation.” This summary highlights key points from the fourth session “Envisioning Your Ideal Modern-day Architecture – What to Consider.”
The series features Steven Stone, CEO & Founder of NSU Technologies, and Tom Villani, Senior Vice President, Digital Innovation, at CAI. Stone, a former CIO at Lowe’s and L Brands, shares real-world examples and proven solutions to help companies accelerate their own transformations. We invite you to register for the full series for free at www.cai.io and to contact Tom.Villani@cai.io to complete a complimentary Data & Analytics Readiness Assessment.
Begin with the End in Mind
Simply put, data architecture is the blueprint for managing an organization’s data assets. It starts with a strategy that aligns data with the needs of the organization, then blends in governance as a holistic framework of policies and procedures. Finally, it incorporates high-level data models for the critical data elements like physical representation models and patterns that will be used to address all organizational needs.
On average, 12% of revenue is wasted because of poor quality of data. (Cerasis)
It’s essential that a data architecture leans toward the future because it must enable the achievement of mid-to-long-term business goals. In a recent survey, a number of business objectives were identified as enabled by data architecture. Six notable objectives from this survey included: insights from analytics and reporting; cost savings and efficiencies; mitigation of risk; improving customer satisfaction; stimulating revenue and growth, and supporting a digital transformation.
Beginning with the end in mind starts with understanding your organization’s data obligations. These obligations are mandated by industry or geopolitical structures and are often manifested in security, privacy, and compliance requirements. Standards like GDPR and the newer CCPA have only begun to outline some of the consumer privacy issues involved; many countries have requirements that coexist or go beyond these. Of course, there are also sector-specific security frameworks like HIPPA in healthcare or PCI in credit card transactions. The US federal government often requires the use of the FISMA or FEDRAMP frameworks. In addition, the National Institute of Standards and Technology (NIST) has published its cybersecurity framework that is often recognized as the gold standard. To comply with any of these standards may require an organization to adopt practices like encryption, tokenization, and data masking where appropriate.
A good next step is to prioritize the types of data, which tend to fall into one of four major categories. Restricted data includes proprietary information that could expose your company to risk, such as intellectual property or Social Security numbers. Another area is customer data, such as patient information in the health sector or consumer credit details, that must be protected. A third is confidential information needed by those within a business such as business plans, inventory reports, salaries, contracts, and so on. Finally, there is public data that is freely disclosed inside or outside of the organization.
The level of control and support needed for restricted data is much different than those needed for public data. In building a robust data architecture it is important to match the level of architecture to the level of data prioritization.
Finally, plans for a data architecture should be built on a solid technical foundation that looks to the future. This raises a series of questions such as how will architecture be maintained, or which operations will shift to the cloud over time? The plan should take extensibility and scalability into consideration, as well. Lowe’s, for example, grew from $3 billion to $50 billion over 18 years. The data architecture for the $50B entity was markedly more complex.
The Role of Governance
If we look at data architecture as a blueprint for a building, governance is the building code. Often when an analytics project fails, it is due to a failure in governance. Governance connects the four Ps: people, policy, process, and practice. Governance establishes the rules the architecture will employ and the roles of people who enforce them. It also ensures data security, reliability, and availability. Finally, it sets the foundation for data stewardship, quality, and Master Data Management, which we’ll discuss shortly.
We need governance to tie the data architecture to business goals and operational enhancements. It will define who’ll build and manage the architecture, and it sets metrics and measurements to help assure those roles stay in line with the needs of the data architecture.
31% Greater confidence in data accuracy when companies have a Data Governance program. (Dataversity)
Beyond the what and why of governance, there are some basic goals it must accomplish. The first is to establish the rules and define the controls that will become part of the data architecture. Another is to ensure the safety and integrity of data assets in line with the organizational requirements that we discussed earlier. A third is to enhance the value of data assets by ensuring they are trusted.
Trust is a foundational concept that must be addressed by a data governance program. Consider this: Would you get on a plane if a pilot told you there was an 85% chance it would arrive safely? Well then, would you base an important business decision on data that was only 85% accurate? As we noted in the introduction, only about 15% of business managers trust their data.
Master Data Management is a specific type of governance that defines the technologies and processes that ensure data consistency and accuracy by establishing data architecture principles. There are four basic models: The registry model matches and cleans duplicates from multiple sources without changing the data in the source systems. The consolidation model merges data from multiple systems to create a single version of the truth. A co-existence model allows changes in both the source and master system. Finally, the transactional model would be integral to a data hub system where the data is moved closer to the edge of federated systems, as it is in, say, e-commerce.
Future-proofing Your Data Architecture
We already touched on the need for a data architecture to be flexible, extensible, and scalable so that an organization can grow, evolve, and change as internal and external needs shift over time. That concept needs to be baked into thinking about data architecture.
Let’s revisit the topic of trust, for example. Trust demands both high quality and high accessibility. Those elements, in turn, involve five factors: accuracy, completeness, reliability, relevance, and timeliness. Even if you have good data, it’s no help if you can’t access it, it’s incomplete, or it isn’t relevant to what you need at that moment. Data architecture should ensure consistency and accuracy during data collection and curation as sources and business needs change. And it should allow immediate access to all who may develop a business need, including third-party partners or customers.
More companies are moving data storage and analytics to the cloud, so the expanding role of cloud and hybrid models should be part of forward-leaning data architecture. This will require strong integration layers. Fortunately, there are new tools to support business intelligence apps as well as more sophisticated needs like artificial intelligence, machine learning, and augmented reality. Accordingly, these tools help to flesh-out data architecture.
New DBs and Tools
Relational databases still account for 70% of the DB marketplace, but there’s a rapid shift to other technologies like graph databases that store relationships as part of the data itself. Time-series databases are built specifically for handling metrics, events, or other types of measurement that are time-stamped. Multi-value databases use lists of attributes instead of viewing them as a single value. Object-oriented databases are particularly helpful with the growing number of IoT devices. Document databases help with semi-structured data associated with XML and other document types. And multi-model databases enable queries across different database types, like those mentioned above.
There are also many new tools coming into common use that may become part of data architecture. Data wrangling is an obvious choice for those performing analytics, with more sophisticated tools helping with data cleansing, bad data, missing data, or duplicates. Some use tools like R, Python, Ultra-X, and Paxata. Different BI tools may be great for enterprise-class reporting, visualization, or quick ad-hoc analysis. Data architecture may have multiple BI tools. There are also processes within popular discovery tools for data prep, visual analysis, or guided analytics.
Decision-makers who use data visualization tools are 28% more likely to discover relevant information. (MIT)
New tools are also coming into the marketplace that allows analysts and others without extensive statistical backgrounds to develop predictive models. AI and machine learning studios have evolved from some of the industry leaders that allow non-data scientists to build very sophisticated AI models.
As it is evident there will be multiple analytic tools in most organizations a key aspect of a successful data architecture is accounting for the interdependencies and overlaps between the tools. Managing these relationships is paramount to achieve consistency in data analysis and reporting.
Data-as-a-Service demands a set of services that connect and expose the data using various analytic tools. It requires a strong integration layer, particularly when you have AI and machine learning running complex operations against the data. Once you’ve generated meaningful insights, you’ll want to make it easy for others to consume, thereby heightening the value of the data assets.
We'd love to talk.Get in Touch