The Emergence of Data Generalists
Merriam-Webster defines a generalist as one whose skills, interests, or habits are varied or unspecialized. Having varied and unspecialized skills seems like an undesirable trait for job applicants. Recruiters are expected to find candidates with specific skill sets based on a defined job description. Meanwhile, candidates are expected to tailor their resume to a specific job, rather than submit a general one for each application. With these competing priorities, a generalist has not been a desirable trait. I believe the future will create opportunities for both generalists and specialists.
The Cause
This reality will emerge because of the adoption of the cloud, which is expected to exceed a global market of 330 billion dollars this year. The cloud is an innovation accelerator. It reduces the cost for building software and speeds up development by outsourcing much of the infrastructure work. As software development becomes easier, niche products will be developed to tackle every business problem that you could imagine. If there is money to be made by solving a business problem, then there will be a software created. This competitive race will drive prices down for software products, increase automation of work, and reduce costs for businesses. This is consistent with a 2019 survey where 87 percent of organizations claimed to experience business acceleration from their use of cloud services.
Two of the most important byproducts of the cloud acceleration are more APIs and better user authentication services. Software tools with APIs (e.g. Stripe, Twilio, DataRobot) allow software engineers to connect capabilities to different applications with only a few lines of code. Meanwhile, better user authentication services for cloud-hosted applications will allow non-tech organizations to easily partner with third party vendors for different tasks. Once it becomes easier to outsource third party work, non-tech organizations will have a difficult decision on their hands. Will they outsource more of their information technology and data analytics expertise? Theoretically, an organization should outsource when it is cheaper than producing the same capabilities in-house.
The Effect
I believe these factors will all contribute to a major change in hiring behaviors. The ease of integrating capabilities across organizations will encourage many to specialize in certain skills. However, the growth in specialization will leave a gap of knowledge. This gap will exist at the edge of the specializations. Who will be responsible for decisions that cross multiple specializations within an organization? One obvious example will be the need for liaisons or translators between non-tech organizations and their third party vendors. There needs to be an employee with a high level understanding of technical concepts, coupled with strong domain knowledge within their organization’s core competencies. Let’s call these individuals data generalists.
I envision two common paths for people to become data generalists.
Path 1: Tech First, Domain Later
The first path will be individuals who have a wide breadth of knowledge across information technology or data analytics. They have an intermediate level of understanding across several technical areas, such as cybersecurity, digital transformations, data engineering, or data science. They enjoy learning new things, rather than optimizing one specific technical skill. Typically, these individuals will join a non-tech organization and specialize in a specific domain or industry. They will acquire this additional expertise through work experience, additional education, or personal hobbies.
Path 2: Domain First, Tech Later
The second path will be individuals who studied a subject in a non-tech area (e.g. accounting, finance, chemistry, history). They are considered a subject matter expert in their specific field or industry. Whether it is through personal hobbies or a work project, this individual will discover an interest in data analytics or information technology. In time, they will acquire a enough competency of technical concepts to be the equivalent of a project manager.
Final Thoughts
Personally, I know individuals who have followed both paths outlined above. In fact, I consider myself a data generalist who might be on the first path. It is a constant struggle to choose whether to specialize or broaden your skill set throughout your career. My bet is that the emergence of data generalists will occur sooner than you think. This is why I created The Data Generalist.
~ The Data Generalist
Data Science Career Advisor