According to the NVIDIA Technology Blog, NVIDIA has introduced significant improvements to Federated XGBoost with the Federated Learning Application Runtime Environment (FLARE). This integration aims to make federated learning more practical and productive, especially for machine learning tasks such as regression, classification, and ranking.
Key features of Federated XGBoost
XGBoost, a machine learning algorithm known for its scalability and effectiveness, has been widely used in a variety of data science tasks. The introduction of Federated XGBoost in version 1.7.0 allowed multiple organizations to jointly train XGBoost models without having to share data. Later version 2.0.0 further enhanced this feature to support vertically federated learning, allowing for more complex data structures.
Starting in 2023, NVIDIA FLARE has built-in integration with these Federated XGBoost features, including horizontal histogram-based and tree-based XGBoost, and vertical XGBoost. We’ve also added support for Private Set Intersection (PSI) for sample alignment, allowing you to perform federated learning without extensive coding requirements.
Run multiple experiments simultaneously
One of the great features of NVIDIA FLARE is the ability to run multiple simultaneous XGBoost training experiments. This feature allows data scientists to test different hyperparameters or feature combinations simultaneously, reducing overall training time. NVIDIA FLARE manages communication multiplexing so you don’t have to open a new port for each task.
Fault-tolerant XGBoost training
In cross-regional or cross-border training scenarios, network stability can be a critical issue. NVIDIA FLARE addresses this with fault tolerance features that automatically handle message retries during network outages. This ensures resilience and maintains data integrity throughout the training process.
Tracking the Union Experiment
Monitoring training and evaluation metrics is especially important in distributed settings such as federated learning. NVIDIA FLARE integrates with a variety of experiment tracking systems, including MLflow, Weights & Biases, and TensorBoard, to provide comprehensive monitoring capabilities. Users can choose between distributed and centralized tracking configurations, depending on their needs.
Adding tracking to your experiment is simple and requires minimal code changes. For example, integrating MLflow tracking requires just three lines of code.
from nvflare.client.tracking import MLflowWriter
mlflow = MLflowWriter()
mlflow.log_metric("loss", running_loss / 2000, global_step)
summary
NVIDIA FLARE 2.4.x provides powerful support for Federated XGBoost, making federated learning more efficient and reliable. For more information, see the NVIDIA FLARE 2.4 branch and the NVIDIA FLARE 2.4 documentation on GitHub.
Image source: Shutterstock