IDG Contributor Network: Shopping for data: what’s fit for your purpose?


There’s been a shift in data strategy from defensive to offensive. Historically, data governance focused on compliance and security. It still does, but it’s expanding to address data accessibility—getting data to the people that need it to solve business problems, drive new revenue, create value, and even monetize their data.

But how do you accomplish this? In the past, I’d receive a report with data deemed relevant to my tasks, and if I needed more I’d ask someone who had the tools to get me the data. But the dramatic increase in data volume (much of it produced by automated devices) renders that method obsolete. Enter the data catalog.

Great expectations

With the rise of ecommerce, we’ve gone from browsing physical stores to browsing online catalogs. On Amazon, by setting a few simple search criteria, I can find everything from the novels of Murakami to All-Clad cookware. I can see what’s in stock and from whom. I can see how others rated the item, and what else they’ve purchased. This rich user experience has permanently changed how we shop.

It’s not surprising, then, that we expect that same experience when searching for data. Access to and understanding of data has become indispensable; jobs across all industries demand insights from data in some form. In my last post I addressed the importance of data democratization across organizations and in society at large, and that’s why what began as the data dictionary and transformed into the metadata repository is no longer adequate. Whether you call it the data catalog or the data marketplace, people now want an online location to “shop for data,” and there are characteristics that these tools must provide to be satisfactory.

as the No. 1 barrier to their work. When browsing the data catalog, being able to articulate such questions, and see how the set of available data satisfies them, is critical if we are to work through hundreds or thousands of data sources. It’s not the rating of data, but the context of how well the data fits differing requirements that allows us to gauge whether it is useful or not.

The data catalog addresses the first barrier toward data democratization: finding and accessing data. A familiar, consumer-style search capability is foundational, but the ability to apply questions to the data for a given business problem is central to reduce the time required to wade through the range of data sets and quickly get to those of highest value and interest. If you’re exploring a data catalog solution, ensure that it not only captures metadata but also provides business semantics, context, and a means to evaluate the data content against your requirements.

This article is published as part of the IDG Contributor Network.