PyTorch Tensors Explained: A Complete Beginner's Guide (2026)

 


What Is PyTorch? A Complete Beginner's Guide to Deep Learning in Python (2026)

Part 2: Understanding Tensors, Tensor Operations, CPU vs GPU, Autograd, and Practical Code Examples

Introduction

In Part 1, you learned what PyTorch is, why it has become one of the most popular deep learning frameworks, how to install it, and how to write your first PyTorch program.

Now it's time to learn the building blocks of every PyTorch application: tensors.

If neural networks are the brain of an AI model, tensors are the language they speak. Every image, sentence, sound clip, or numerical dataset processed by PyTorch is represented as one or more tensors.

Understanding tensors and how to manipulate them is one of the most important skills for anyone learning deep learning. In this chapter, we'll also explore how PyTorch performs calculations on CPUs and GPUs, introduce automatic differentiation with Autograd, and walk through practical code examples you can run on your own computer.

What Is a Tensor?

A tensor is the fundamental data structure in PyTorch. You can think of it as a multi-dimensional array that stores numerical data.

If you've used Python lists or NumPy arrays before, tensors will feel familiar. However, tensors have two major advantages:

  • They can run efficiently on GPUs, making deep learning much faster.
  • They support automatic differentiation (Autograd), which is essential for training neural networks.

Almost everything in PyTorch—from images and text to neural network weights—is stored as tensors.

Tensor Dimensions

Tensors are categorized by the number of dimensions they have.

0-D Tensor (Scalar)

A scalar contains a single value.

import torch

x = torch.tensor(10)
print(x)

Output:

tensor(10)

Examples:

  • Temperature
  • Age
  • A single prediction score

1-D Tensor (Vector)

A vector is a list of values.

import torch

x = torch.tensor([1, 2, 3, 4])
print(x)

Output:

tensor([1, 2, 3, 4])

Examples:

  • Student marks
  • Daily temperatures
  • Sensor readings

2-D Tensor (Matrix)

A matrix contains rows and columns.

matrix = torch.tensor([
    [1, 2],
    [3, 4]
])

print(matrix)

Output:

tensor([[1, 2],
        [3, 4]])

Examples:

  • Spreadsheet data
  • Grayscale images
  • Tables

3-D Tensor and Higher

Higher-dimensional tensors are commonly used in deep learning.

tensor3d = torch.rand(2, 3, 4)

This creates a tensor with:

  • 2 blocks
  • 3 rows
  • 4 columns

Examples:

  • RGB images
  • Video frames
  • Medical scans
  • Mini-batches of training data

Creating Tensors

PyTorch offers several ways to create tensors.

From a Python List

import torch

numbers = torch.tensor([5, 10, 15])

print(numbers)

Output:

tensor([ 5, 10, 15])

Create a Tensor Filled with Zeros

zeros = torch.zeros(3, 4)

print(zeros)

Output:

tensor([[0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.]])

This is useful when initializing data structures or neural network parameters.

Create a Tensor Filled with Ones

ones = torch.ones(2, 5)

print(ones)

Output:

tensor([[1., 1., 1., 1., 1.],
        [1., 1., 1., 1., 1.]])

Random Numbers

Random tensors are extremely common when initializing neural networks.

random_tensor = torch.rand(3, 3)

print(random_tensor)

Every execution generates different values.

Creating a Range

numbers = torch.arange(1, 11)

print(numbers)

Output:

tensor([1,2,3,4,5,6,7,8,9,10])

Tensor Shapes

Every tensor has a shape.

Example:

x = torch.rand(3, 4)

print(x.shape)

Output:

torch.Size([3, 4])

This means:

  • 3 rows
  • 4 columns

Shape information is very important because neural networks expect input data in specific dimensions.

Tensor Data Types

Just like Python variables, tensors have data types.

x = torch.tensor([1,2,3], dtype=torch.float32)

print(x.dtype)

Output:

torch.float32

Common types include:

  • torch.int32
  • torch.int64
  • torch.float16
  • torch.float32
  • torch.float64
  • torch.bool

Choosing the correct data type affects both memory usage and performance.

Basic Tensor Operations

PyTorch supports many mathematical operations.

Addition

import torch

a = torch.tensor([1,2,3])
b = torch.tensor([4,5,6])

print(a + b)

Output:

tensor([5,7,9])

Subtraction

print(b - a)

Output:

tensor([3,3,3])

Multiplication

print(a * b)

Output:

tensor([4,10,18])

This is element-wise multiplication.

Division

print(b / a)

Output:

tensor([4.0000,2.5000,2.0000])

Matrix Multiplication

Matrix multiplication is different from element-wise multiplication.

A = torch.tensor([
    [1,2],
    [3,4]
])

B = torch.tensor([
    [5,6],
    [7,8]
])

print(torch.matmul(A, B))

Output:

tensor([[19,22],
        [43,50]])

Matrix multiplication is used extensively in neural networks.

Tensor Indexing

Accessing individual values is straightforward.

numbers = torch.tensor([10,20,30,40])

print(numbers[0])

Output:

tensor(10)

For matrices:

matrix = torch.tensor([
    [1,2],
    [3,4]
])

print(matrix[1][0])

Output:

tensor(3)

Tensor Slicing

You can retrieve portions of a tensor.

numbers = torch.tensor([5,10,15,20,25])

print(numbers[1:4])

Output:

tensor([10,15,20])

Slicing is useful when selecting batches of data.

Reshaping Tensors

Sometimes data must be rearranged.

x = torch.arange(12)

print(x.reshape(3,4))

Output:

tensor([[0,1,2,3],
        [4,5,6,7],
        [8,9,10,11]])

Reshaping changes the organization of data without changing its values.

CPU vs GPU

One of PyTorch's greatest strengths is its ability to use GPUs for computation.

CPU

The Central Processing Unit (CPU) is the main processor in your computer.

Advantages:

  • Works on all computers
  • Good for small projects
  • Simple debugging

Disadvantages:

  • Slower for deep learning
  • Limited parallel processing

GPU

The Graphics Processing Unit (GPU) contains thousands of small cores capable of performing many calculations simultaneously.

Advantages:

  • Extremely fast for matrix operations
  • Ideal for training neural networks
  • Significantly reduces training time

Disadvantages:

  • Requires compatible hardware (typically NVIDIA GPUs with CUDA support)
  • Consumes more power
  • Higher cost

Large AI models often train much faster on GPUs than on CPUs.

Checking GPU Availability

You can check if PyTorch detects a CUDA-compatible GPU.

import torch

print(torch.cuda.is_available())

Output:

True

or

False

If the result is False, PyTorch will use the CPU.

Moving Tensors to the GPU

device = torch.device("cuda")

x = torch.tensor([1,2,3])

x = x.to(device)

print(x)

The output indicates that the tensor resides on the GPU.

To move it back:

x = x.cpu()

This flexibility allows you to choose the best device for your workload.

What Is Autograd?

One of PyTorch's most powerful features is Autograd, short for automatic differentiation.

Training a neural network requires calculating how each parameter influences the final prediction. This involves computing gradients—something that would be tedious and error-prone to do by hand.

Autograd automates this process. As operations are performed on tensors, PyTorch records them in a computation graph. When you ask for gradients, it applies the chain rule from calculus to compute them automatically.

This makes it possible to train neural networks efficiently without manually deriving mathematical formulas.

Enabling Gradient Tracking

To allow PyTorch to compute gradients, create a tensor with requires_grad=True.

import torch

x = torch.tensor(2.0, requires_grad=True)

Now, any operation involving x will be tracked.

A Simple Autograd Example

import torch

x = torch.tensor(2.0, requires_grad=True)

y = x ** 2

y.backward()

print(x.grad)

Output:

tensor(4.)

Here's what happened:

  • We defined y=x2y = x^2.
  • The derivative of x2x^2 is 2x2x.
  • Since x=2x = 2, the gradient is 2×2=42 \times 2 = 4.

PyTorch computed this automatically.

A More Complex Example

import torch

x = torch.tensor(3.0, requires_grad=True)

y = x * x + 2 * x + 1

y.backward()

print(x.grad)

Output:

tensor(8.)

The function is:

y=x2+2x+1y = x^2 + 2x + 1

Its derivative is:

dydx=2x+2\frac{dy}{dx} = 2x + 2

When x=3x = 3:

2(3)+2=82(3) + 2 = 8

Autograd performs this calculation for you.

Why Autograd Matters

Every deep learning model improves by adjusting its parameters based on gradients.

Autograd enables:

  • Efficient training
  • Automatic gradient computation
  • Backpropagation
  • Optimization using algorithms such as Stochastic Gradient Descent (SGD) and Adam

Without automatic differentiation, training modern neural networks would be far more difficult.

Practical Example: Tensor Operations

Let's combine several concepts.

import torch

a = torch.tensor([1.0, 2.0, 3.0])
b = torch.tensor([4.0, 5.0, 6.0])

c = a + b
d = c * 2

print("A:", a)
print("B:", b)
print("C:", c)
print("D:", d)

Output:

A: tensor([1., 2., 3.])
B: tensor([4., 5., 6.])
C: tensor([5., 7., 9.])
D: tensor([10., 14., 18.])

This example demonstrates how tensor operations can be chained together to build more complex computations.

Summary

In this chapter, you learned the core concepts that every PyTorch developer needs:

  • Tensors are the primary data structure in PyTorch.
  • Tensors can represent data of various dimensions, from scalars to high-dimensional arrays.
  • PyTorch provides powerful tensor creation, indexing, reshaping, and mathematical operations.
  • GPUs dramatically accelerate deep learning by performing parallel computations.
  • Autograd automatically computes gradients, enabling efficient training of neural networks.
  • These building blocks form the foundation for creating and optimizing AI models.

In Part 3, we'll build on this knowledge by exploring neural networks, the torch.nn module, loss functions, optimizers, the training loop, and how to train your first deep learning model from start to finish.

Continue to Part #3:
PyTorch Tensors Explained: A Complete Beginner's Guide (2026)
https://khayyamshah2007.blogspot.com/2026/08/what-is-pytorch-complete-beginners_01450564732.html

Comments

Post a Comment