GITEX GLOBAL 2023: Huawei Launches Deep Learning Data Lake Storage

(CTN News) – As Huawei’s OceanStor A310 provides optimal storage capability for basic model training, industry model training, and inference in segmented scenario models, Huawei’s OceanStor A310 offers a unique solution to global industry challenges and extends the horizons of artificial intelligence.

It is clear from Huawei’s launch that the company is aligned with the latest trends in AI development in the era of large model phones.

It is known that the OceanStor A310 deep learning data lake storage is suited for different industries and scenarios in large AI model applications to mitigate the most significant concerns enterprises have with the development and implementation of large model applications.

It currently takes approximately ten days for about a hundred terabytes of data to be pre-processed, as data preparation takes a long time with dispersed data sources and a slow aggregation process.

The current loading speed for a large number of small files for multi-modal large models that use massive text and images as training sets is less than 100MB/s, resulting in a low efficiency for loading large numbers of training sets for such multi-modal large models.

Due to the frequent tuning of large model parameters and the instability of the training platform, further delays occur as a result of the frequent interruptions and recovery processes caused by tuning large model parameters.

As a rule of thumb, the threshold for the successful implementation of large models, the creation of complex systems, the scheduling of challenging resources, and the use of GPU resources is usually below 40%.

In addition to this, the Huawei OceanStor A310’s deep learning data lake storage can be adapted to various industries and scenarios when it comes to large model applications.

It is important to emphasize that OceanStor A310 is a deep learning data lake storage solution that offers unlimited horizontal scalability and high performance in the context of basic and industry large model scenarios.

As a result, it provides end-to-end management of massive amounts of data for AI processes, including data collection, preprocessing, training, and inference.

This system possesses data analysis capabilities in line with high-performance computing (HPC) and big data, as well as the capability of processing data from similar sources.

The OceanStor A310 will enable Huawei to offer customers and partners a cutting-edge AI storage solution for unlocking new AI capabilities and creating value for a variety of industries, said Peter Zhou, President of Huawei’s Data Storage Product Line.

It is our hope that visitors to our GITEX booth will see how Huawei is committed to accelerating the intelligent era through leveraging the OceanStor platform that is trusted, secure, and reliable by some of the world’s leading businesses.”

SEE ALSO:

NVIDIA Chips Will Not Be Supplied To China By Biden