How to Build Your First Data Warehouse Like a Boss. Part 3/3 The How’s and Why’s of a Data Warehouse
“He who has a why can deal with any what or how.” –Stephen Covey
In this last instalment of this series, we focus on the raison d’être of your data warehouse. We’ll walk you through a series of questions to help you design your data warehouse tech stack better.
What You’ll Learn
- Necessity of Data Warehouse Gap Assessments
- Assessment Quadrants
- Data Warehouse roadmap as an output of a gap assessment>
- Necessity of periodic reassessments
Aside from time constraints, gap reports can be messy and overwhelming. Some people like to throw around blame, and things can get political. It is, therefore, vital for stakeholders to approach an assessment activity such as this with neutral objectivity.
At the end of the day, everyone is on the same team, and we all want to see your organisation succeed. The mindset should be this: information is always an asset. Regardless of what the assessment reveals, it’s a good thing. If we can recognise the problem, we can always fix it.
A gap report presents our goals and aspirations for our project, honest recognition of where we currently are, and the steps to reach our goals. For a data warehouse, there are four quadrants to consider when creating a gap assessment:
- Business Needs
- Data Architecture
- Technical Architecture
Let’s dive in.
- What questions do we need to answer to improve our processes?
- How are these related to our company’s objectives?
- How often do we need to answer these questions?
- Are we able to answer those questions given the data that we have? If it isn’t available, where can we get it?
- What analyses can we perform on our data to reveal the answers we need?
- Do the information needs to identify the key performance indicators (business metrics) and business perspectives (dimensions, descriptive attributes) needed to measure, analyse, and
- optimise the targeted business processes?
- Who needs access to the answers to these questions?
- How have these requirements been documented?
A conversation can begin with these prompts:
- I’d like it if we could…
- So that…
- We’re successful if…
- When we don’t succeed ….
- Our priorities are…
- Concretely, a client could reply:
- I’d like it if we could develop a report that highlights our top 50 clients who have purchased an item from us in the past year days, but not recently.
- So that we can pinpoint who these customers are and target them with some incentives to retain them.
- Currently, the reports we have show us our top 50 customers and their average order value, but we want to improve customer retention.
- We’re successful if we can find these customers and track how many of them actually came back to purchase more from us.
- When we don’t succeed, we could lose these top clients
- Our priorities are requirement 1 > requirement 2 > requirement 3
- Do the information requirements line up with the business requirements?
- Does the information exist in other systems? How will they be aggregated and standardised?
- What business, operational, and technical metadata requirements have been identified? How have these requirements been addressed?
- Do users understand the purpose of specific data? Are they using it correctly?
- Are there issues with data quality?
- Do users trust the data being delivered?
- Do the users understand what data they have access to and how to use it properly?
- How flexible is the current data structure? Can new information be incorporated easily?
- Can it support multiple analytical needs? Does it readily allow for the integration of new data?
- Are data transformations fully documented?
This quadrant looks at hardware, software and network infrastructure and examines physical database designs. Technical architecture assessment seeks to identify performance, maintenance, scalability, data distribution, disaster recovery, and sizing issues. This assessment also aims to identify opportunities to leverage the value of existing technology. Existing software and tools are re-examined to determine their overall fit to the business and technical environments.
Some of the key questions related to technical architecture include:
- Does the technical architecture provide for a suitable range of data delivery services? (e.g. publishing of regular reports; ad hoc queries; analytical models, etc.)
- Do the data and information services reach the right people? Do data warehouse services integrate with other business processes where needed?
- How fast can users perform their queries?
- Is data refreshed in a timely manner?
- Will the technical architecture scale to increasing demands to support more processing power, more users, more data, more frequent and higher volume queries, new applications and front ends, and new technology components?
- Is sensitive data adequately secured from unauthorised access and alteration? Is there a proper balance between security and accessibility?
- What tools are implemented? Are they deployed effectively to support environment-specific needs?
- How much can we spend on technical resources like servers, network, etc?
- What technical skills does our workforce have?
- How are these technical details documented?
Methodology and Project Management
This quadrant assesses the ability of an organisation to begin a project and ensure its delivery. It looks into the procedures and process to manage requirements, changes and issues.
- Are deliverables and artefacts identified and managed?
- Are frequent checkpoints and walkthroughs included?
- How are issues or exceptions resolved?
- How is testing conducted?
- How are changes and upgrades introduced?
- Are the issue and change management processes effective?
- Do we have a workforce with the necessary skill set?
- Can we deliver the data warehouse in the desired timeframe?
- Does release planning align with business need?
- How is the performance of the data warehouse assessed?
This quadrant involves the assessment of the necessary roles and responsibilities of IT and business users. It reviews the readiness of the organisation to assume responsibility of the operation, maintenance and continuous upgrades required for the data warehouse to continually bring value to the organisation over time.
- Who are your stakeholders?
- Are project roles and responsibilities well defined?
- Have project communication lines been defined and established?
- Have these roles and responsibilities for the data warehouse been documented?
- Who is responsible for the quality and accuracy of the information in the data warehouse?
- Who is responsible for the availability of the data warehouse?
- Do the assigned users have the necessary skills to perform their functions properly?
- Who should have access to the data warehouse?
- Does the plan provide for incremental delivery of business value according to business priorities, budget, and technical feasibility?
- Does the project have access to the necessary business stakeholders, subject matter experts, technical assets and technical support?
By answering these questions, you’re well on your way to understanding your organisation’s data warehouse requirements.
By identifying the gaps between your company’s current state and where it wants to be, a roadmap can be created to reach your goals in a reasonable timeframe. Choices on tools and the right data warehouse components should be easier to decide on now that you’ve considered these questions.
We recommend trying to build demos or proof of concepts on a small scale to see if a certain technology does meet your requirements.
Selecting the right components of a data warehouse is not easy. A thorough assessment could prevent you from making wrong decisions that could cost you down the line. Regardless of your data warehouse’s maturity – if you’re just starting or are years into it – it’s a good idea to assess and reassess your data warehouse’s design and performance periodically.
This is something that we don’t take lightly at MyDataforce. We want you to take the necessary time to understand your underlying needs fully. We aren’t just a vendor; we are your strategic partner to help you understand your business data needs and manage them more efficiently. The design of your data warehouse is more than half of the work. Implementation should be the easy part.
At MyDataforce, our years of experience has allowed us to create the necessary tools and templates to complete a thorough assessment of your company’s data warehousing requirements.
Contact us today to start your data warehouse conversation.
The post How to Build Your First Data Warehouse Like a Boss Part 3 appeared first on MyDataforce.