Data augmentation: new views of an example

A model learns whatever its examples have in common

A machine-learning model is a program whose settings are learned from training examples: inputs paired with the right answers. It picks up any pattern that helps it give those answers, including patterns you did not intend.

Suppose you train a model to recognize chairs, and every chair in your examples faces the same way. The model may learn "a chair has its back on the left." Shown a chair turned around, it fails. It never saw evidence that the facing direction does not matter.

Data augmentation supplies that evidence. Each time the model sees an example, you first change the example at random, in a way that keeps the answer the same. A chair turned around is still a chair. Over many passes, the model sees each chair facing many ways and learns to ignore the direction.

For point clouds: rotate the whole cloud, then jitter each point

Two changes are common for a point cloud.

• A random rotation. Draw one random angle (see random draws) and turn every point by it, using the same rotation matrix. A rotation keeps every distance between points, so the shape itself does not change.
• Jitter. Add a small random offset to each point separately. Each offset is drawn from a bell curve centered on zero, with a small spread such as 0.01 of the shape's size. Programs often clip each offset, for example to at most 0.05, so that no point moves far.

Before you use the figure, predict: how large can the jitter get before you can no longer read the letter?

The strokes are 0.12 wide. Once the spread is a large share of that width, the strokes blur into each other. A spread of 0.01 changes almost nothing you can see, and that is the point: the model learns not to depend on exact positions. The figure leaves out clipping, so that large spreads show their full effect.

Each change teaches the model to ignore one thing

Choose each change by asking what the answer should not depend on.

One more rule matters as much as the choice of change. If the answer is itself a shape, apply the same rotation to the input and to the answer. For example, if the model must predict the missing part of a shape, rotate the part you give it and the part it must predict with the same matrix. Otherwise the copies no longer match.

In NumPy, rotation is one matrix product and jitter is one random draw

Each row of cloud is a point. A rotation matrix R turns a column vector p into Rp. For points stored as rows, the same rotation is the product cloud @ R.T, with R transposed.

augment.py

import numpy as np

rng = np.random.default_rng(7)
cloud = np.array([[2.0, 0.0, 1.0],
                  [0.0, 1.0, 0.0],
                  [1.0, 1.0, 3.0]])

theta = rng.uniform(0, 2 * np.pi)          # a random angle about the z axis
c, s = np.cos(theta), np.sin(theta)
R = np.array([[c, -s, 0],
              [s,  c, 0],
              [0,  0, 1]])
rotated = cloud @ R.T                      # each row is a point, so multiply by R transposed
# np.clip keeps each jitter value between -0.05 and 0.05.
jitter = np.clip(rng.normal(0, 0.01, cloud.shape), -0.05, 0.05)
augmented = rotated + jitter

def pairwise(p):                           # every distance between two points of p
    return np.linalg.norm(p[:, None] - p[None, :], axis=-1)

print(round(np.degrees(theta), 1))
print(np.allclose(pairwise(cloud), pairwise(rotated)))   # all equal, up to rounding?
print(np.abs(pairwise(augmented) - pairwise(cloud)).max().round(3))

225.0
True
0.012

The random angle is 225°. The rotation keeps every distance between points exactly, and the jitter changes them by at most 0.012. Inside pairwise, p[:, None] - p[None, :] subtracts every point from every other point at once (see broadcasting).