Hiring Analytics Tools - Is Building or Buying the Best Choice for You?

Should you cook dinner or order out? Should you buy a turn-key house, or have one designed for you? Should you buy a ready-made software or build the system in-house?

Both building and buying can have benefits and downsides, and it can be hard to know what the right choice is for you and your business. If you’re considering building a reporting solution for your talent and recruiting team, you probably have a lot of questions. What will it look like? Who will have access and will they be able to update the reports themselves? Does every team member get the same reports, or can you customize results depending on the function? Who will build the reports, and how will they get built? Can you change a dashboard after it’s been made? How often does the data update? Do you get reports for sourcing activity? How about coordinator scheduling activity?

This is a two-part series, and this article centers on building. We’ll answer the questions above, as well as go over a few key areas like:

• The elements of a standard data reporting solution
• The the cost of a reporting solution
• The biggest benefits that come from an in-house build
• Potential complications and downsides to building your reporting software

Though we’re using Ashby Analytics as our primary example, the information in this article is just as relevant to other products. Of course we’re Ashby Analytics fans, but we’re examining the “build vs buy” decision as a whole. You might be just as interested in comparing Looker or tableau to building an in-house tool.

Let’s begin by breaking down the upsides, complications, and costs of an in-house building solution.

The traditional data team solution

When you’re looking at improving how data-driven your business is, your first stop is often to build out your data team. Here, you have two immediate choices: do you have a centralized data team or an embedded data team?

The centralized data team tends to take on a service model, where a core team of data engineers and analysts maintain both the data infrastructure and data analysis requests that are fielded in an approach similar to support tickets. The core data team is often responsible for standardizing corporate metrics as well as maintaining systems that make analysis results and reports available across the organization’s various teams.

Although some very large organizations maintain and operate under a centralized data team model, after a certain size (typically at least four or five team members) some organizations opt for an embedded analyst model. In the embedded approach, each analyst will work within a given product area or function (e.g. the marketing analyst, the core product analyst, etc.)

If your organization is large enough to have a data team, one of these approaches may sound familiar. Which one works for your company will differ, and doesn’t always impact the build or buy decision. But regardless, neither tends to serve the recruiting function well. The centralized data team model by its nature (and to its critique) is request-based and results in only periodic, intermittent reports delivered. And even for large organizations that can afford many embedded data analysts, it’s rare that one is dedicated to the recruiting team exclusively.

In the rare and fortunate case that your recruiting team has a dedicated analyst, that’s great! It makes building an in-house recruiting analysis system a much more feasible and attractive option. At this point, you’ll need to make decisions about the analysis tooling and infrastructure itself, which brings us to our next topic.

Breaking down cost

The elements of building out an analytics tool is completely inextricable from the cost. Every element that goes into the design, construction, and upkeep of a large, complicated internal application adds to the cost. We’ve broken down what you’ll need to account for by stage, so start here, then jump to the stage that best describes your business.

Which data maturity stage is your organization in? Here’s a rough guide to check against:

• Early stage (~seed to series A startup): No central data warehouse, no standardized reporting infrastructure or tooling, reports are done in a one-off basis, reports are hard to share broadly and require a lot of custom rework if the results are to be updated
• Beginning stage (~series A to B): Central data warehouse and standard data sources are integrated across business and product systems, typically one data engineer and at least one data analyst, a standard reporting tool used to regulate and share common reports
• Established stage (~series C and beyond): Your data team has a manager, two data engineers, and a number of analysts such that most corporate functions are covered through a service or embedded model, standard metrics and regular reporting are mostly standardized, your product is mature and growing with a majority of analysis efforts spent on optimization and new feature development

Got your stage? Then let’s break down what you’ll need to build out effectively.

Early stage considerations

If you’re at the early stage you’ll be facing the challenging of starting from scratch. These foundational efforts will be tied to the time scale of hiring your first data expert, but it will also bring up a number of technology decisions. At the very least, you’ll likely be faced with the following steps:

• Hiring your first data domain expert - Founding hire; salary may range between $66k - $200k
• Purchasing a data warehouse - For data storage; may range from $10k - $90k yearly
• Purchasing data integration technology - To continuously sync and maintain data records across business and product systems; may range from $2k - $90k yearly
• Purchasing your reporting infrastructure - There are open source options here which come with the cost of building and maintaining; off-the-shelf reporting solutions may range from $50k - $100k yearly

In addition to the starting costs, there is still the time cost of establishing each of these components

Beginning stage considerations

At this point you likely have the foundations of data team and reporting in place. The question at hand typically becomes where is the data team’s time and effort applied? As mentioned above, small data teams have to operate on a request basis, as they are not large enough to pursue an embedded analyst model. At this point it is likely that some departments or specific challenges receive solutions, while others are left unattended.

The questions most pertinent at this stage are:

• Can your recruiting team get dedicated attention from the data team?
• Do you have reporting infrastructure built? If so, does your team have access to it, to ensure everyone is looking at the same results and metrics?
• Is your ATS data integrated into your data warehouse and has it been modeled by analysts already?
• How urgent are your reporting needs? Do you need results next week, or is a month or longer acceptable?

It’s common at this stage to see a company investing the data team resources into a backlog of corporate and product reporting. It’s also normal to see bandwidth divided between multiple areas, such as product usage and engagement across your customer base, marketing efforts, sales and success planning.

You’ll want to make a sober assessment of your recruiting analytics needs relative to the demand the data team is under to assess whether you’ll get the assistance you need at the cadence you need it. While it’s normal to see a queue forming, that doesn’t mean the queue is unavoidable or that it’s the best option.

Established stage considerations

If you’re at a large organization with a well-established data team there is more of an opportunity for talent and recruiting teams to get dedicated analysts. At this point, it becomes an internal conversation about setting bandwidth and budget. Many of the beginning stage considerations above do still persist, and implementation still takes time, but an enterprise-level business may have more ability to address the costs and complications.

At the established stage, the scale and importance of a reliable solution becomes more evident. In the absence of standard-built solutions, teams have likely learned to live on their own, using partial solutions and spreadsheets. Though the point solutions may exist for various needs, the ability to train new team members or perform broader analysis across teams veers towards the impossible.

At this scale, the important thing is finding a strong solution quickly, be it built or bought, as the time and energy atrophy of these ad hoc solutions can create some serious havoc in the talent acquisition process.

“Hidden” cost

There’s a baseline cost to any data solution that comes from hiring a data professional and investing in the infrastructure technologies they require to do their job. Those initial investments really are just the beginning of a data solution.

Businesses of all sizes should keep an eye out for different pricing structures when comparing in-house vs. purchased options. For instance, Looker charges a $5k/month fee that may be inaccessible to a small business, while tableau's $70/user/month rate may be unfriendly for large companies. When crunching the cost of a potential build, make sure you're running an accurate comparison!

Here are some common additional in-house build costs to consider:

• What is the time to your solution? This is an opportunity cost. Some Ashby customers have reported taking more than six months to get a reliable reporting system established prior to making the switch to Ashby.
• What is the cost of ongoing knowledge transfer? Recruiting is a complex business, and you need to ensure that your data analyst understands it. Many teams find one of the primary time delays is having to explain the recruiting business and operations to non-recruiter analysis.
• Is your solution resilient? Recruiting is a dynamic field where the strategies and tactics often change. One of the most common outcomes we’ve seen is a business intelligence-based solution that’s brittle: it reports useful metrics but doesn’t afford slight modifications that are required to drive further insight. Further insight demands additional requests and more time from your data team.
• Are you able to self-serve your own reporting needs? Many recruiting teams are handed reports they’re unable to adjust as needed. This relates to the brittle BI solution above, but it’s important to address whether you have the autonomy to drive your own reporting needs, not just adjust pre-existing reports.

You may already be familiar with some of these situations. In any in-house software build out, hidden costs are completely normal, and are really only a big problem when they take you by surprise.

What you get for the price

In-house software is expensive, but what else is it? There are several benefits to building an in-house solution.

• Project control: Building in-house means that your business is in control of every single aspect of the solution. You get the final say on where to focus resources, who is managing the data, what’s done with the data, and what the user interface ends up being. This might be a requirement for businesses with extremely high security or privacy demands.
• Pivot: Due to that level of control, you’ll be able to pivot your methodology as quickly as the data itself will allow. The learning curve is built along side the system itself.
• Specificity: Having your own personal analysis system means you can get heavily granular with your information, data, and reporting in a way a purchased ready-made solution often isn’t. If there’s something especially specific that you need to dig into, that’s unique to your business’s recruiting, this alone may make it worth it.
• Long-term cost: If you already have a robust data integration tool, warehouse, and data experts on staff, it may be worth the time cost to see the long-term savings an in-house built system will offer you.

Recruiting analytics-specific complications

If you’ve made it this far, you might be wondering why typical data solutions seem to fail to address recruiting analytics needs. That’s a good question, and the short answer is that recruiting analytics done well is a difficult data problem. But there’s more to it than that simple statement.

The underlying data is dynamic in ways that reflect the real world. Unlike, say, a product analytics environment where engagement measures are well defined, recruiting touches the real world. Interviews are created, cancelled, rescheduled, canceled again, and rescheduled. Then one of the three interviewers doesn’t show up, and only one of the two submits feedback. The candidate takes a vacation, so you get back to them at a later date, and when they come back they want to apply for a different job so you transfer them. A different hiring manager takes over, your in-house recruiter goes on leave, so a different recruiter gets involved. This is a very real portrayal of the paper trail of data that gets created in the recruiting environment. Answering a simple question like "How many people did recruiter X hire?" gets complicated pretty quickly.

Recruiting data is prone to quality issues and requires a lot of data modeling and filtering. It’s always necessary to clean and prepare data. Recruiting data and the complexities of the real world are particularly prone to quality issues. This is where powerful filtering, data exploration, and underlying data modeling comes in. Building this in-house takes a lot of domain familiarity, and maintaining the system as recruiting efforts evolve is far from trivial. New sources, agencies, team members and recruiting efforts all act to continuously complicate the reporting landscape.

The scope of analysis focuses on the individual all the way up to broad, long-term trends. Recruiters need to pay attention to the details, know where individual candidates are, and get timely alerts and reports for day-to-day operations. Managers like to see weekly summaries, while executives need to see broader aggregates and be able to drill down into the details. There are variety of stakeholders, each of whom have separate requests. This isn’t hard per se, but it is a lot of work! This is where the amount of time to stand up a recruiting analytics solution and ongoing maintenance costs factor in.

Self-serving reporting is critical to maintaining recruiting velocity. Some of the most critical metrics in reporting relate to time: time to fill a position, time in process, and time to hire. When a recruiter needs to make an external request they’re almost certainly looking at additional days of delay. Those days add up and hiring efforts slow down in aggregate, which trickles down to your organizations progress against the hiring plan. Self-service is a mission critical aspect of any recruiting analytics solution. The requisite filtering, grouping and customization needs to be readily available to the end user: a recruiting team member, not a data analyst. This is a hard data design problem, and the standard BI dashboard with fixed reports does not meet recruiting team needs.

Recruiting analytics are simply different from other types of data analysis. This alone complicates recruiting data system significantly. While building in-house isn’t impossible, and may remain the right choice for your business, it’s uniquely difficult in a way that other in-house systems are not.

The bottom line

Building an in-house data analytic system for recruiting is a massive undertaking. Once you pull it off, it can offer a lot of additional control, precision, and specificity that many bought solutions can’t match. However, it’s time-consuming, expensive, and may prioritize what stakeholders think they need over what they actually need.

Building in-house is most likely right for you if:

• The type of work your business does and the type of recruiting you undertake is exceptionally specific
• Intense recruiting data privacy is a must in your field
• You’re at an enterprise level or are established enough that building more data systems isn’t a large additional cost or lift
• You’re willing to wait longer and spend more money to create something completely unique to your business

Even if this sounds like you, building still isn’t your only choice. Join us in part two of our article to learn more about the pros and cons of buying recruiting analytics tools.