Tensor
A tensor is a mathematical object that can be represented as a multi-dimensional array of numbers. It is a generalization of vectors and matrices and is widely used in various fields, including machine learning, physics, and engineering. In machine learning, tensors are used to represent data, model parameters, and intermediate computations.
Key Characteristics
- Multi-dimensional: Tensors can have any number of dimensions, making them suitable for representing complex data structures.
- Linear algebra operations: Tensors support various linear algebra operations, such as addition, subtraction, multiplication, and division.
- High-dimensional: Tensors can represent high-dimensional data, which is essential for capturing the complexity of real-world problems.
Types of Tensors
- Scalar: A tensor with zero dimensions. It represents a single value.
- Vector: A tensor with one dimension. It represents a sequence of values.
- Matrix: A tensor with two dimensions. It represents a table of values.
- Higher-order tensor: A tensor with more than two dimensions.
Applications in Machine Learning
- Data representation: Tensors are used to represent input data, model parameters, and intermediate computations in machine learning models.
- Neural networks: Tensors are the fundamental data structure in neural networks, used to store and manipulate weights and activations.
- Convolutional neural networks (CNNs): Tensors are used to represent images and other spatial data in CNNs.
- Recurrent neural networks (RNNs): Tensors are used to represent sequences of data in RNNs.
Example
Consider a simple neural network with one input layer, one hidden layer, and one output layer. The input layer has 10 neurons, the hidden layer has 5 neurons, and the output layer has 3 neurons.
- The input tensor has shape (1, 10), where 1 represents the batch size and 10 represents the number of input neurons.
- The hidden layer weights tensor has shape (10, 5), where 10 represents the number of input neurons and 5 represents the number of hidden neurons.
- The hidden layer activations tensor has shape (1, 5), where 1 represents the batch size and 5 represents the number of hidden neurons.
- The output layer weights tensor has shape (5, 3), where 5 represents the number of hidden neurons and 3 represents the number of output neurons.
- The output layer activations tensor has shape (1, 3), where 1 represents the batch size and 3 represents the number of output neurons.
For more information on tensors in machine learning, please visit our Tensor Basics page.