US companies will face persistent shortages of experienced data specialists as well as data engineers and data architects as digitalization becomes the new normal across the industry.
A new study by management consultants Bain & Company concludes that many companies are having trouble finding big data experts. The only short-term solution to the problem is stepping up internal re-education and training programs and contracting with third-party service providers. The use of datahubs and crowdsourcing can also help.
The findings show that growing efforts to educate data specialists cannot keep pace with the sharply rising demand for these experts. That endangers companies‘ effort to quickly and comprehensiely introduce modern analysis methods, Bain warns.
The consultants predict that, worldwide, the number of advanced analytics experts will double to around 1 million between 2018 and 2020, but they say that will be insufficient to meet demand.
While the number of data experts is growing globally, India is taking the lead in efforts to provide more advanced data expertise. By the end of 2020, Bain says, the country will have tripled its advanced analytics talent pool from 2018 levels. The number of experts will grow to 200,000 from 65,000. In western Europe, by comparison, data analysis experts will number 170,000 in 2020, up from 125,000.
“This development is good news for all companies that are proactively looking ahead,” said Florian Mueller, who heads Bain’s advanced analytics practice in Europe, the Middle East and Africa. For these companies, he added “innovative data analysis is of critical importance.
Bain says the availability of fresh graduates will help, but won’t address the need for more experienced specialists. In the US, for example, Bain says there may be an oversupply of data scientists, but a shortage of advanced data engineers and architects.