Description: Guide for selecting and configuring distributed training strategies in NeMo AutoModel, including FSDP2, Megatron FSDP, DDP, and parallelism settings.
Lines: 612 | Code: 177 | Dir: nemo-automodel-distributed-training
name: nemo-automodel-distributed-training description: Guide for selecting and configuring distributed training strategies in NeMo AutoModel, including FSDP2, Megatron FSDP, DDP, and parallelism s…