IDG Contributor Network: The 80/20 data science dilemma


The emergence of cloud has led to an explosion of data that has left data scientists in high demand. A job that didn’t exist a decade ago has topped , based on salary, job satisfaction, and number of job openings. It was even dubbed by the Harvard Business Review.

Though growing in population, data scientists are scarce and busy. A recent shows that demand for data scientists and analysts is projected to grow by 28 percent by 2020. This is on top of the current market need. According to LinkedIn, there are more than . Unless something changes, this skills gap will continue to widen.

Against this backdrop, helping data scientists work more efficiently should be a key priority. Which is why it’s an issue that currently, most data scientists spend only 20 percent of their time on actual data analysis.

The reason data scientists are hired in the first place is to develop algorithms and build machine learning models—and these are typically the parts of the job that they enjoy most. Yet in most companies today, 80 percent of a data scientist’s valuable time is spent simply finding, cleaning and reorganizing huge amounts of data. Without the right cloud tools, this task is insurmountable.