Collective Communication - AlltoAllVC

【免费下载链接】hccl集合通信库（Huawei Collective Communication Library，简称HCCL）是基于昇腾AI处理器的高性能集合通信库，为计算集群提供高性能、高可靠的通信方案项目地址: https://gitcode.com/cann/hccl

Sample Description

This sample demonstrates how to use theHcclAlltoAllVC()API to perform the AlltoAllVC operation. It covers the following functions:

• CallaclrtGetDeviceCount()to detect devices and query the number of available devices.

• CallHcclGetRootInfo()and userank 0as the root rank to generate the rootinfo identifier.

  The rootinfo identifier contains the device IP address and device ID. This information must be broadcast to all ranks in the cluster to initialize the communicator.

• In each thread, callHcclCommInitRootInfo()to initialize the communicator based on the rootinfo identifier.

• CallHcclAlltoAllVC()to split the input data into a specific number of blocks along a given dimension, send the blocks sequentially to other ranks, receive data from other ranks, concatenate the received data along the specific dimension in order, and display the result.

Directory Structure

├── main.cc # Sample source file ├── Makefile # Compilation and build configuration file └── alltoallvc # Compiled executable file

Environment Preparation

Environment Requirements

This sample supports the following products in a single-server N-card configuration (N >= 2):

• Ascend 950PR / Ascend 950DT
• Atlas A3 Training Series Products / Atlas A3 Inference Series Products
• Atlas A2 Training Series Products

Setting Environment Variables

# Set CANN environment variables. The following uses the root user default installation path as an example. source /usr/local/Ascend/cann/set_env.sh

Compiling and Running the Sample

Run the following commands in the sample code directory:

make make test

Note: You can set theHCCL_OP_EXPANSION_MODEenvironment variable to configure the expansion mode of communication operators. For the supported ranges for different product models, see the usage instructions for this environment variable in the Environment Variable List.

# Set the communication operator expansion mode to the AI CPU communication engine export HCCL_OP_EXPANSION_MODE=AI_CPU

Sample Output

The data of each rank is initialized to the corresponding rank ID. After the AlltoAllVC operation, the content of each node is the concatenation of the input data of all nodes.

Found 8 NPU device(s) available rankId: 0, output: [ 0 1 2 3 4 5 6 7 ] rankId: 1, output: [ 0 1 2 3 4 5 6 7 ] rankId: 2, output: [ 0 1 2 3 4 5 6 7 ] rankId: 3, output: [ 0 1 2 3 4 5 6 7 ] rankId: 4, output: [ 0 1 2 3 4 5 6 7 ] rankId: 5, output: [ 0 1 2 3 4 5 6 7 ] rankId: 6, output: [ 0 1 2 3 4 5 6 7 ] rankId: 7, output: [ 0 1 2 3 4 5 6 7 ]

【免费下载链接】hccl集合通信库（Huawei Collective Communication Library，简称HCCL）是基于昇腾AI处理器的高性能集合通信库，为计算集群提供高性能、高可靠的通信方案项目地址: https://gitcode.com/cann/hccl