Communicator Management - One NPU Device per Thread

【免费下载链接】hcommHCOMM（Huawei Communication）是HCCL的通信基础库，提供通信域以及通信资源的管理能力。项目地址: https://gitcode.com/cann/hcomm

Sample Overview

This sample demonstrates how to use theHcclCommInitRootInfoConfig()API to initialize a communicator in a single process, with each thread managing one NPU device. It covers the following features:

• Device detection: Query the number of available devices using theaclrtGetDeviceCount()API.

• Use rank0 as the root node and generate rootinfo identification information through theHcclGetRootInfo()API.

  The rootinfo identification information mainly includes the Device IP, Device ID, and so on. This information must be broadcast to all ranks in the cluster to initialize the communicator.

• Initialize the communicator in each thread based on the rootinfo identification information using theHcclCommInitRootInfoConfig()API.

• Call theHcclAllReduce()operator and print the results.

Directory Structure

├── main.cc # Sample source file ├── Makefile # Build or configuration file └── one_device_per_pthread # Compiled executable

Environment Preparation

Requirements

This sample supports the following products:

• Ascend 950PR / Ascend 950DT
• Atlas A3 training series products / Atlas A3 inference series products
• Atlas A2 training series products
• Atlas training series products / Atlas inference series products

Software Dependencies

Running this sample requires the CANN ops operator package to be installed. For detailed installation steps, see the "Install the CANN Software Package" section in Source Code Build.

Disabling Signature Verification

Thecann-hcomm_<version>_linux-<arch>.runsoftware package generated from this source repository contains the following tar.gz subpackages:

• cann-hcomm-compat.tar.gz: HCOMM compatibility upgrade package.
• cann-hccd-compat.tar.gz: DataFlow compatibility upgrade package.
• aicpu_hcomm.tar.gz: AI CPU communication base package.

These tar.gz packages are loaded to the Device when the service starts. During the loading process, the driver performs security signature verification by default to ensure the package is trusted. Because the tar.gz packages compiled from this source repository do not contain a signature header, the driver security signature verification mechanism needs to be disabled. For the method to disable signature verification, refer to the "Disable Signature Verification" section in Source Code Build.

Configuring Environment Variables

# Set CANN environment variables, using the root user default installation path as an example source /usr/local/Ascend/cann/set_env.sh

Compiling and Running the Sample

Execute the following commands in the sample code directory:

make make test

Note: You can set theHCCL_OP_EXPANSION_MODEenvironment variable to configure the expansion mode of communication operators. For the range supported by different product models, refer to the usage of this environment variable in the Environment Variable List.

# Set the expansion mode of communication operators to AI CPU communication engine export HCCL_OP_EXPANSION_MODE=AI_CPU

Sample Output

The data for each rank is initialized to 0 to 7. After the AllReduce operation, the result for each rank is the sum of the data at the corresponding positions across all ranks (the data from 8 ranks is summed).

Found 8 NPU device(s) available rankId: 0, output: [ 0 8 16 24 32 40 48 56 ] rankId: 1, output: [ 0 8 16 24 32 40 48 56 ] rankId: 2, output: [ 0 8 16 24 32 40 48 56 ] rankId: 3, output: [ 0 8 16 24 32 40 48 56 ] rankId: 4, output: [ 0 8 16 24 32 40 48 56 ] rankId: 5, output: [ 0 8 16 24 32 40 48 56 ] rankId: 6, output: [ 0 8 16 24 32 40 48 56 ] rankId: 7, output: [ 0 8 16 24 32 40 48 56 ]

【免费下载链接】hcommHCOMM（Huawei Communication）是HCCL的通信基础库，提供通信域以及通信资源的管理能力。项目地址: https://gitcode.com/cann/hcomm