Horovod Installation Guide

Horovod is a library for distributed training of deep learning models. It is designed to work with TensorFlow, Keras, PyTorch, and Apache MXNet. This guide will help you get Horovod up and running on your system.

Prerequisites

Before installing Horovod, ensure that you have the following prerequisites:

• Python 3.6 or higher
• A deep learning framework: TensorFlow, Keras, PyTorch, or Apache MXNet

Installation

To install Horovod, use the following command:

pip install horovod

TensorFlow

To use Horovod with TensorFlow, ensure that you have TensorFlow installed and then run the following command:

pip install horovod-tensorflow

Keras

To use Horovod with Keras, ensure that you have Keras installed and then run the following command:

pip install horovod-keras

PyTorch

To use Horovod with PyTorch, ensure that you have PyTorch installed and then run the following command:

pip install horovod-pytorch

Apache MXNet

To use Horovod with Apache MXNet, ensure that you have Apache MXNet installed and then run the following command:

pip install horovod-mxnet

Example

Here is a simple example of how to use Horovod with TensorFlow:

import tensorflow as tf
from horovod.tensorflow import Horovod
from horovod.tensorflow.keras import hvd

# Initialize Horovod
hvd.init()

# Create a model
model = tf.keras.models.Sequential([
    tf.keras.layers.Flatten(input_shape=(28, 28)),
    tf.keras.layers.Dense(128, activation=tf.nn.relu),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(10, activation=tf.nn.softmax)
])

model.compile(
    optimizer=tf.keras.optimizers.Adam(hvd.DistributedOptimizer(tf.keras.optimizers.Adam())),
    loss='sparse_categorical_crossentropy',
    metrics=['accuracy']
)

# Train the model
model.fit(x_train, y_train, epochs=5, validation_data=(x_test, y_test))

# Save the model
model.save('my_model.h5')

For more information, please visit the Horovod documentation.

Horovod Logo