How to Manage Data at the Edge

Data is the lifeblood of any business.  Consequently, the availability of that data is critical.  The Internet of Things (IOT) produces a tremendous amount of data that a business needs to understand in order to make decisions.  Additionally, this data must be secured. 

All of this leads to the requirement to capture and then process data from source locations.  This is referred to as “Data at the Edge”.

Ideally, if the data is captured and analyzed at the edge, then only the result sets would need to be captured and stored.  This is a much lower volume than all of the data that is being generated at the edge.  This process is critical for such things as self-driving cars, oxygen supplies for mines, for ensuring the quality and safety of manufacturing processes, etc.

Benefits of capturing data at the edge:  low latency, decreases the workload on networks and bandwidth, and higher security due to data transfers occurring over company networks rather than public ones.  The higher security also protects any sensitive data that is being collected.

By consolidating workloads at the edge, companies are able to more effectively manage and store the data and this reduces costs.

The challenge for businesses is managing the data at the edge. This is due to the fact that there are many sources of data, and this volume of data needs to be analyzed at the edge in order to find the key information the business needs within the massive volume of data. Here are a few ways to analyze and manage data at the edge:

1. A business should determine what data needs to be analyzed, and then store that data securely and at a reasonable cost. Other aspects regarding data at the edge are regarding backup and recovery as well as archiving.
2. An enterprise should have a secure and readily accessible backup process for the data. This includes the ability to have it accessible for those who need to retrieve it.  There should be routine testing of this process.  Additionally, the data should be routinely archived with date and time stamps and follow Enterprise legal standards for its retention.
3. It is important to choose the correct medium for archiving. Currently, these are tape, disc and cloud storage.  Before any archiving is attempted, it is important to ensure that the data is deduped so that costs are more manageable, and it is easier to search the data. 
4. Whatever method of archiving is chosen, that it be periodically updated to ensure that there is no loss of data due to decay of the medium chosen. Along these lines it is also important to remember to upgrade your archiving environment whenever you upgrade your systems to ensure that the data will be able to be retrieved and read on the new system. 
5. Finally, preventive measures need to be taken to ensure that a data file cannot be overwritten or accidently deleted.

With the correct precautions, Data at the Edge can be a powerful tool for most organizations.