Why pressure will increase for agencies to handle data lakes?

Data lakes are becoming the preferred platform for advancing the data settings, but several agencies sustain to struggle with managing them.

Rising issues in managing a swelling tide of information for several organizations is resulting in something more closely resembling a data swamp in contrast to a data lake, several data experts believe.

An increasing number of agencies are using data lakes to either augment data warehouses or to serve as the enterprise data hub, claims Ben Sharma, chief executive officer at Zaloni, a data management company based in Durham, NC.

“Unmanaged, poorly thought via data lakes are generally data swamps, and their usefulness decays over time,” Sharma asserts.

“Organizations are realizing that they require more agile data platforms and deeper analytical capabilities to compete effectively in their market,” Sharma claims. “The huge trend we see is agencies moving from sandbox or single-purpose big data applications to enterprise wide governed data lake implementations.”

Numerous other trends are emerging in various data-intensive industries, Sharma adds.

“The Internet of Things (IoT) is a big topic. Machine learning is also on everyone’s list. It is early stage, but as an industry, we’re all searching for ways to leverage automated algorithms to make better our understanding of our data and to get faster insight,” Sharma states.

“We’re also seeing a real emergence of IT in the big data landscape,” he adds. “As data lakes become more mission critical, organizations are looking to IT to give the governance, security and automation needed for these applications.”

Perhaps the greatest challenge agencies are facing is “finding, rationalizing and curating the data from across an enterprise for analytics solutions,” Sharma elaborates. “The capability to easily access data, refine data and collaborate on data needs sustains to be a large roadblock for many analytic applications.”

The requirement for improved analytics is increasing within several industries, involving healthcare, and that will force improvements in data management capabilities.

“While there are increasingly strong and effective analytics applications, the data management, integration and governance activities sustain to be a key hurdle in rapidly making effective use of scale out architectures. For this reason, several agencies are still slow to adopt big data technologies in a production capacity,” Sharma states.