As companies grow from SMB to Enterprise size, a lot changes across the organization. These changes affect everything from technology used to organizational hierarchy to even mission statements. From a technology standpoint, we see more security and compliance requirements. We also tend to get more people asking for access to data. Making things tricky due to the sheer number of systems that might be in place from data warehousing, ETL, and BI/analytics tools.
One of our clients was going through this transition recently and decided to implement a hub and spoke model in their Looker instance. While the concept for hub and spoke is not new, it has been something new in Looker modeling. The general concept is that there is a hub model which contains validated central business logic that can then be imported and further built upon in the spoke models. The spokes themselves utilize the central hub for their own liking and have the flexibility to quick build and prototype their own models while keeping the main hub under control and unaffected by their changes.
This has allowed them to have a central Enterprise Reporting hub which supports company wide metrics, reporting, and dashboarding. Alongside that, specific verticals/squads have their own spokes where they can extend files from the hub changing them as needed. Looker helps accomplish this architecture in a unique way but it does require some foundational setup to work properly. Here are a few key points regarding the setup:
Each spoke has it’s own LookML project with it’s own Github Repository for proper version control
All LookML developers would have access to the hub project but you should restrict who can push changes to the hub through a PR process. This way you can grant Github repo access to only specific developers. You can go to the project settings in Looker to enable this
Model files can not be extended but Explore, Views, other files can be
Why & When To Utilize This Framework
Having done built this framework for a variety of clients, here are some common problems and ways in which a hub and spoke framework can be utilized.
Confusion around how metrics are calculated
Repetition of views across projects
PR review process takes too long
Explore menu has too many options
Several LookML Developers working within the same models and projects
Above are just a few of the problems that can be solved using the hub and spoke framework. Further considerations are deciding what should absolutely belong in the hub. Like we mentioned earlier, the hub should contain the following: standard field definitions, core explores with common joins already defined, dimensions and measures that would be used company wide (e.g. avoid one offs), and multi-use PDTs.
The department spokes on the other hand will contain the only: department specific models (i.e. Sales performance - quotas, revenue generated, etc.), views designed by and for that specific department, extending hub views but creating your own dimensions/measures for the department, and specific schemas that can only be accessed by that department.
If you'd like to discuss in further detail, kindly reference the button below.
Data Clymer is a premier boutique consulting firm specializing in data culture transformation. Our proven methodology includes full data stack implementation, data democratization, custom training, and analytics to enable data-driven decisions across the organization. We have curated a set of best practices from our deep expertise in Looker, Tableau, Snowflake, Redshift, BigQuery, Panoply, Matillon, DBT, and Fivetran.
We just sent you an email. Please click the link in the email to confirm your subscription!