Combining metadata, information, and user interaction to enhance the user’s data journey
 
Joseph True
Data Catalog Specialist
August 19, 2021
What’s the ‘data experience’ (DX) for users in your data catalog?
Can users easily navigate their way around and find what they’re looking for?
Can the catalog help them answer the big questions like:
  • What is this data and where does it come from?
  • Who owns this data and how is it used?
Data catalogs continue to evolve, becoming more mainstream and morphing into something bigger than just a data inventory for IT. Catalogs now provide a gateway and destination for all types of data-consumers and providers. They have become an important part of the modern data technology stack: providing a self-service, data discovery platform. These platforms can represent the front-page and central repository for all metadata and related knowledge in your organization. They can also serve as the first touch point where everyday users can search, browse, and interact with an inventory of data.
 
Due to this growing popularity and mainstream acceptance, data catalog solutions must now target the end-user and focus on the overall data and user experience. Whether you’re using an open-source package, buying a commercially available solution, or building your own catalog, here are some essential DX features to consider.
 

Mashing Up Data, Information, and Interaction

Modern data catalogs combine elements from different software, design, and data concepts. They’re part search engine, part information and knowledge base, and part data engineering and analysis tool. To be successful and useful to all kinds of users, the mashup of data, information, and user interaction must combine into a seamless data discovery experience.

The Essential Elements

You need all the data catalog and discovery pieces to create and support the essential UI elements (see diagram). Combining these elements in the right way makes the catalog a more effective and user-friendly environment.

  • Homepage

  • Search

  • Navigation and browsing

  • Documentation and content

  • Data and users

  • Metadata summary

 

There’s No Place Like Home

The homepage is the front-face to your catalog and a central component for both new and returning users. It contains the essential parts to ground your users and entice them into exploring more. Key elements include:

  • Catalog name and tagline

  • Images and links related to main resources

  • Quick search capability

Giving your catalog a name and tagline sounds like a basic item, but it lets users know that they’re in the right place. Here’s a quick sample:

Acme Company Data Discovery Platform

Quickly search, find and understand company data assets and resources

 

The bulk of the homepage should provide links to main resources; primary data assets, important articles, and other basics like that. Adding images here makes the space more inviting and visually appealing.

A dedicated search box on the homepage gives users access to one of the main tools they’ll use in the catalog. In some cases, the homepage design puts search front and center.

Where’s My Data? 

People often describe a data catalog as a ‘search engine for your data’. Searching is one of the main ways that users will interact with the catalog and find what they’re looking for.

Just as the internet is an ‘exo-brain’ where we lookup topics we don’t know about, a data catalog can serve as your organization’s ‘exo-brain about data’. It becomes a collection of curated information stored in a central location for everyone one to share and use. Of course, this only works if you can efficiently search the catalog in a simple, logical manner. For example:“Show me all columns named “Account ID” in data source ABC.”

Search features to include:

  • Raw text search

  • Filter options to narrow and refine your search. For example, filtering by object type (table, column …)

  • Facets to find data from many different angles; technical name, business name, data source, endorsed objects …

After searching, the output should not be just a plain list of results, but an interactive summary of what’s available. For example, Google often throws in added context for common searches; weather, locations, peoples, concepts. This added information can make a difference and provide more value to the results.

Navigation, Browsing and a Sprinkle of Breadcrumbs

In addition to searching the catalog, many users will want to just browse through the available resources. Finding your way around the catalog in a logical manner should be another central focus of the catalog. This means providing an interactive and intuitive inventory of data assets that users can easily navigate, browse, and know where they are.

Navigation

Navigation includes mapping the overall UI to the user’s mental model of data resources at their company. Users should be able to easily navigate to the core components of the catalog; data assets, documentation, and users. For visual reference, each area of the catalog should be represented by a dedicated icon or image. This helps emphasize the objects as the user moves through the catalog.

Browsing

Users should be able to easily click their way through the hierarchy of data assets and objects in the catalog. For example, when browsing the hierarchy of scanned data assets, users can expand a tree list from data source to database, schema, table, and column names.

Breadcrumbs

Breadcrumbs are a UI feature that provide users an interactive and visual reference to where they are in the app. Breadcrumbs remind users: “Where am I?” and “Where did I come from?”. They provide a visual cue of the catalog page they are currently viewing or where they are in a hierarchy of objects.

Here are some simple examples of breadcrumbs:

  • Data Source > Database > Schema > Table > Column

  • BI Source > Project > Folder > Dashboard > Report

  • Settings > Users

Give Me Some Context with that Metadata

Beyond scanning and building an inventory of data sources, the next important aspect of a data catalog is documentation. The metadata inventory is the foundation of the catalog, but for users to understand the data, catalogs must provide a space for context and documentation.

This context typically comes in two flavors:

  • Documentation for data objects (tables, columns …)

  • Additional documentation (separate from the data objects)

Data object documentation equates to a dedicated UI element to describe each part of the metadata (schemas, tables, columns …). This can be a simple chunk of text describing that object or a more dedicated rich-text, documentation page for each specific data object.

In addition to documentation for data objects, catalogs are also becoming a repository for general documentation related to data governance, policies, data analysis and data science. All this additional content requires features that support information design (visual format) and information architecture (structure and organization).

The visual format should provide rich text viewing and editing. Documentation organization should provide ways to group documents into hierarchies and collections, such as glossaries for business terms and other common definitions.

Metadata Summaries, Data Previews and Data Profiles

A simple set of data visualizations in the catalog UI can provide users with quick previews and insights of the metadata and underlying data.

Metadata summaries offer a quick compass to the data inventory. How many tables in this schema? How many columns? Which table has the most columns? Basic stats like that go a long way.

Previews of the actual data (where user permissions allow) give users a quick snapshot into the actual content of each table without having to switch tools or run SQL statements.

Similar to data previews, profiles of column data values provide users even more insight into the data before they start their data analysis journey.

Connecting Data to Actual Users

Discovering the data is only part of the story. Users also need to know the people connected to the data. The catalog should help answer questions like “Who owns this data?” and “Who uses this data?”.

To support this connection, the catalog can display UI elements that tie the data resources back to actual users. Fields for data owner, steward, and frequent users help make this connection.

User profile pages offer another way to support the data-to-people connection. Profile pages can provide a dedicated space per user that lists the data and assets associated with each specific person.

Looking Forward

Data catalogs will continue to evolve as a cornerstone of the data technology stack, but the ones that keep the user’s overall data journey in mind will be the most successful. Blending data, information, and interaction in just the right way is the key.

Whether it’s a data platform, data knowledge base or ‘Galactic Data Portal’ … an intuitive front-end and rich user experience will support the user’s data discovery journey and improve their overall data interaction experience.