Big data management is the organization, administration and governance of large volumes of both structured and unstructured data.
The goal of big data management is to ensure a high level of data quality and accessibility for business intelligence and big data analytics applications. Corporations, government agencies and other organizations employ big datamanagement strategies to help them contend with fast-growing pools of data, typically involving many terabytes or even petabytes of information saved in a variety of file formats. Effective big data management helps companies locate valuable information in large sets of unstructured data and semi-structured data from a variety of sources, including call detail records, system logs and social media sites.
Most big data environments go beyond relational databases and traditional data warehouseplatforms to incorporate technologies that are suited to processing and storing nontransactional forms of data. The increasing focus on collecting and analyzing big data is shaping new platforms that combine the traditional data warehouse with big data systems in a logical data warehousing architecture. As part of the process, the must decide what data must be kept for compliance reasons, what data can be disposed of and what data should be kept and analyzed in order to improve current business processes or provide a business with a competitive advantage. This process requires careful data classification so that ultimately, smaller sets of data can be analyzed quickly and productively
‘Big data’ management changing the face of data warehousing
Gartner says the onslaught of “big data” is turning data warehouses into distributed data processing platforms. But Hadoop and other big data tools aren’t as widespread as you might think — yet.
When the phrase “big data” management hit the data management and business intelligence (BI) industry, it had many IT professionals wondering if it would be the real deal or just another buzzword that faded away as quickly as it arrived. Well, it looks like that question has been answered: real deal. Or, at least, data warehouse vendors are banking on it being answered that way.