De-duplication

It is important to look at what data is being created, how it is being created and stored and what steps can be taken to provide a more streamlined and manageable environment. There are two major ways to approach the problem: master data management (MDM) and data deduplication.
Let’s look at basic deduplication approaches. If a means of identifying when identical documents are being stored can be put in place, a virtual pointer to a single copy of the document can be created, saving the overhead of storing multiple copies. How about saving the changes when they are made, rather than the whole changed document? Such approaches can drastically reduce storage requirements. Email vaulting solutions from vendors such as Symantec and CA can really help with managing the main cause of this - the overuse of email as a document workflow and review system.
However, deduplication can be taken further. The majority of storage management vendors, such as Symantec, IBM and EMC, now provide capabilities to look at data at a binary code level and identify where blocks of data are identical. These blocks can then be stored as single master records, and only where changes are noticed are they stored - but they are stored as delta changes, rather than new full data records.
Such an approach can collapse data storage needs by 60 percent or more, and this can be magnified when you look at backup storage requirements. After all, when you back up your system as a complete image, it is unlikely that more than 10 percent of that will have changed when you next back up. If you have applied deduplication techniques to the original data as well, then everything becomes far more compact. Even with the overhead of rebuilding data sets from the initial master and applying the changes, response times are improved, due to the much smaller data sets involved.
Bringing together MDM and deduplication gives organizations just what is needed in today’s markets - a far more responsive and manageable data environment for supporting the business.