The growth of data volume is like a tsunami, rapidly swallowing up all kinds of storage devices and devices in data centers around the world. Data centers are spending huge amounts of money to keep acquiring large amounts of storage devices in order to try to meet the growing data volume storage demand, which puts great cost pressure on data centers and reduces the profitability of data centers. The data tsunami has become a problem that every data center cannot avoid, and data centers must make effective countermeasures in order not to be knocked down by the data tsunami.

Data center construction is expanding and growing at a long-term rate, which is inextricably linked to the high growth of data volumes. Today, the total amount of global data is doubling every eighteen months, a "scary" growth rate that is far beyond anyone's imagination. If we follow IDC's forecast, by 2025, there will be 163 ZB of data worldwide, an amount equivalent to four Atlantic Ocean waters, and a 3.3-fold increase in global data center traffic. The growth of data volume is like a tsunami, rapidly swallowing up all kinds of storage devices and devices in data centers around the world. Data centers are spending huge amounts of money to keep acquiring large amounts of storage devices in order to try to meet the growing data volume storage demand, which puts great cost pressure on data centers and reduces the profitability of data centers. The data tsunami has become a problem that every data center cannot avoid, and data centers must make effective countermeasures in order not to be knocked down by the data tsunami. So how do you effectively respond to a data tsunami? This article gives some remedies for the symptoms.

Deleting useless data

On the surface the volume of data is growing at a rapid rate, but much of it is in some ways duplicative, useless, unused and completely discardable. When storing data, be sure to filter it to stop junk data from entering the storage device. Many of the data are also highly repetitive. Perhaps everyone collects a classic movie "Ghost in the Shell" and puts it in their personal storage space in the data center so that they can log on and play it when they want to watch it, which spends most of its time sitting in the data center storage device, pointlessly taking up huge storage space. So the data center could store just one copy of this movie and then store data links for countless individual users, and when people need to watch this movie, they ostensibly click on this movie to play it, but the backend is actually just a data link that then plays the stored copy of the movie to them, which would greatly reduce the storage space for the data, and could be implemented this way for other repetitive data as well. We know that the general use of an application software in twenty years, many software has actually rarely been accessed, the large amount of data stored have become useless data, the data center to clean up these data in a timely manner to make storage space to save the use of storage equipment, data center storage resources are limited, sooner or later useful to run out of this day, the later this day comes, the more cost savings to the data center, bringing superior competitiveness of the data center. Some intermediate computational data also takes up temporary storage space and should be cleaned up in a timely manner. Also, for this data center, the data may be useful and can be analyzed by big data technology to get something useful, while these data centers may be useless for other data, and no further analysis can get beneficial results, these data may be deleted directly and do not occupy data center storage space. In fact, ninety percent of the data in a data center is arguably useless and should be removed in a timely manner.

Use of compression technology

It is well known that data can be compressed. We can compress the data before saving it to a storage device to minimize the space it takes up. Data compression technology is advancing with higher and higher compression ratios, slowing down the consumption of data for storage space. There are lossless and lossy compression techniques, lossless compression is mainly some coding algorithms, such as subband coding, differential coding, Huffman coding and other algorithms; Lossy compression is mainly some quantization algorithms, such as a-rate, u-rate, and lloyds optimal quantization algorithm. For some unimportant audio and video resources, the appropriate use of some lossy compression techniques does not affect the use of the data and can greatly improve the compression effect. For some files of this paper, even using lossless compression can reduce the file by tens, if not hundreds, of times. The popular use of compression technology can effectively relieve the pressure on data storage and is to be promoted in data centers. Of course, for those accessing real-time data, it is not suitable to use compressed storage, because it takes time to decompress the data, the higher the compression ratio the more time consuming to decompress the data, so that when accessing such compressed data, it takes more waiting time, reducing the access efficiency, for such access is not suitable to use the compression algorithm to save the data. Compression technology is also advancing and decompression times are being reduced, which will continue to expand the widespread use of compression technology.

Expanding network bandwidth

The larger the amount of data, the more network bandwidth it takes up. It is because of the rising volume of data that we need higher bandwidth, and in some cases, even the level of network bandwidth has become the most critical factor affecting the development of data center business deployment. The higher the network bandwidth, the faster the data can be transferred, preventing network congestion that can affect data center business operations. Data center network design should be kept simple, with fewer network layers and a flat network design, with two levels generally designed from access to core egress, so that there are fewer points in the network where congestion occurs. Don't design your network traffic model to be teapot-shaped, with a small mouth and a big belly. It has to be designed in a cylinder shape with large entrances and exits so that the data loaded into the data center is fast enough. Expanding the network bandwidth does not only refer to the place of network entrances and exits, but also to plan and design from the network as a whole, including connecting the server and storage side should be considered to improve the speed between the top-of-rack switches and server NICs and storage devices in the data center to avoid the situation of big horse-drawn car or small horse-drawn car.

With the arrival of massive amounts of data, the existing data center storage capacity is far from adequate. Data centers are going to have to meet this data tsunami on multiple fronts. Global data growth will remain high for at least nearly a decade, and the data tsunami is putting tremendous storage pressure on data centers. Massive data has been growing at a high rate and we have to find ways to bring it under control, and the data centers of the future will work on how to shrink the massive amounts of data and not allow the tsunami of data to expand. In addition to sniping the data tsunami from the above three areas, data centers can also invest more in improving storage utilization, optimizing storage resource allocation, invalid data screening capabilities, and the application of cloud computing and big data. These are great ways to deal with the data tsunami and need to be applied and practiced in the data centers of the future.

Copyright statement.This article is originally published by EnterpriseNet D1Net, and the reproduction must indicate the source as: EnterpriseNet D1Net, if the source is not indicated, EnterpriseNet D1Net will reserve the right to pursue its legal responsibility.

(Source: EnterpriseNet D1Net)

EnterpriseNet D1net has launched Enterprise App Store (www.enappstore.com), for providers of enterprise software, SaaS, etc., offering display, review capabilities, not involved in transactions and delivery. Now you can join for free, and after you join, you can get the opportunity to be recommended in the corresponding public number of Enterprise D1net. Welcome aboard.

Recommended>>
1、CCsizeof below
2、Modifications to JbuilderX
3、Machine Learning 2 logisticregression
4、Java learning of deep copy shallow copy and object copy of two ways
5、Shanghai officially opens credit data platform to get credit cards

已发送

发送中