Governance Lineage During Disruption

Key Question Would you consider taking steps during this disruption to strengthen your “IT offensive and defensive lines”—give managers more control over information, document information flows, and manage IT projects better—at a time of rapid business change?

Disruption The sudden business reorientation of March 2020 has business scrambling to redefine ways to do business. Managers have expressed surprise at the level of disruption:

“We don’t have nearly enough VPN connections to allow our staff to work from home.”
“I don’t have access to the people who can get me answers. The whole organization is upended.”
“Our IT [outsourcing] vendor staff isn’t familiar enough with our current systems to rapidly make changes important to the business.”

Preparation Coach Bill Belichick talks a lot about “situational football” and his favorite Sun Tzu quote: “Every battle is won before it is fought.” The ability to find and track how data flows in your organization—share that knowledge—then act as a team with common knowledge—is situational business.

Prepare Day-to-Day For Crisis We understand the disruption your business faces because ours has been disrupted as well. On the other hand, we believe that lineage can help build and govern both existing and new systems more rapidly. The reasons:

Organizations absolutely rely on their computer systems to survive. Many organiations have lost detailed knowledge of their existing IT infrastructure through attrition or outsourcing. Lineage helps document both existing systems and changes.
We see customers who cut the time to respond to government regulator requests for audit trails by over 80%.
Organizations spend millions on IT upgrade projects, migration, or cloud initiatives. Statistics say over half fail to deliver expected results, run over time, and over budget. Lineage can provide new management tools for large projects.
Lineage can help diagnose problems during internal audits and remediate problems involving inconsistent data flows, to increase assurance and mitigate risk and loss.
Access to accurate, easily-accessible lineage is rapidly becoming table stakes to remain competitive

How often do you get lost these days? Back when there were paper maps and AAA printed out TripTiks, we used to get lost regularly or purchase newer, updated maps. Almost everyone has developed reliance on electronic maps and directions. We have applied the same concepts to governance.

What we do and how we do it is different from other governance. We use powerful software to quickly, automatically build comprehensive, vendor-agnostic dataflow maps—sort of like road mapping for IT:

On the right are reports¹ or other people-facing applications²
In the middle are data stores³, transformation scripts⁴, and ETL tools⁵ that manage and transform data. This includes detailed transformation logic, models⁶, and multiple branches of transformation built up over, for some, more than 40 years (One client has 20,000 Teradata scripts). This is a pain point with multiple vendors and developer/analyst skill sets.
At the left side are your operations: systems where the sales are tracked, the service is performed, and the $Money$ is made.
An important point is that we track lineage for all of the “cities” and “roads” in the map, but people only need one route at a time. What a savings in research and expertise!

Got It Covered Sometimes we hear “Our people understand all of that IT stuff.” Most IT people admit they know the part of the system with which they have experience, but a comprehensive map would be as valuable to them as it would be to business people. Automated lineage can also track modifications and additions through point-in-time (PIT) lineage. As a bonus, defining business glossary concepts/hierarchies tied to IT assets has proven ROI.

Knowledge Has ROI Companies have become used to minor IT disruption, data fixes, and promises that new architectures or technologies will “fix everything.” Lineage can allow rapid access to the one data path in millions that causes a problem. One of our financial customers experienced a miscalculation affecting all of their customers’ bottom lines; they used lineage to isolate the problem quickly, fix it, and restate the correct information within 24 hours.

Cost “What’s my cost, ROI, and effort?” Cost is surprisingly low, starting at less than one FTE per year, and ROI depends upon how much you leverage it for project planning/tracking, root cause analysis, and day-to-day analyst support and education. The automated base product does not require a lot of effort, and the organization can use lineage data to support governance frameworks, stewardship campaigns, and quality monitoring.

Next Step Let’s have a conversation. We can provide a more detailed ROI analysis of your world with automated lineage versus your world without it. We work internationally, have ISO certifications for quality, and serve some of the largest banks, financial companies, health care organizations, and other industries on earth.

Footnotes (contact us for other vendors or custom solutions)

¹ Reporting tools: Tableau, SSRS, PowerBI, Qlik, Oracle BI, Cognos, and others

²Customized outputs such as Excel

³Relational or file stores such as SQL Server, SSAS, Oracle, DB2, Teradata, Redshift, Azure, Netezza, Snowflake, and others

⁴Scripting languages such as PL/SQL, Java, Hive, BTEQ, Pig, Sqoop, and others

⁵ETL tools such as SSIS, Informatica, DataStage, Talend, and others

⁶Modeling tools such as Erwin, ER/Studio, and PowerDesigner