Data Augmentation

MixUp — counter-intuitive কিন্তু empirically powerful। Multiple theoretical perspective।

Vicinal risk minimization:

• Standard ERM — sample point-এ optimize।
• VRM — sample-এর neighborhood-এ optimize।
• MixUp — linear neighborhood সংজ্ঞা।
• Smooth decision boundary।

Linear interpolation interpretation:

• $\tilde{x} = \lambda x_i + (1-\lambda) x_j$।
• Class boundary linear hint।
• Network linearly interpolate predict।
• Smooth manifold encourage।

Regularization view:

• Network noisy label train।
• Confidence reduce।
• Calibration improve।
• Memorization prevent।

Adversarial robustness:

• MixUp-trained network — adversarial attack-এ robust।
• Decision boundary far from data points।
• Implicit max-margin।

Feature smoothness:

• Embedding space smooth।
• Classes linearly separable।
• Better generalization।

Empirical benefits:

• ImageNet — ResNet-50 +1.5% accuracy।
• CIFAR — significant boost।
• NLP — text mixing too works।
• Multi-modal applicable।

Hyperparameter $\alpha$:

• $\alpha = 0$ — no MixUp।
• $\alpha = 1.0$ — uniform interpolation।
• $\alpha = 0.2$ — typical, slight blending।
• $\alpha \to \infty$ — $\lambda = 0.5$ always।

Variants:

• Manifold MixUp: hidden layer-এ mix।
• CutMix: spatial patch mix।
• AugMix: diverse augmentation chain।
• FMix: Fourier-domain mask।

When MixUp helps most:

• Small/medium dataset।
• Class imbalance।
• Noisy labels।
• Strong overfitting tendency।

When MixUp hurts:

• Very small data (under 100/class)।
• Domain where mixing meaningless (text, structured data)।
• Already strong augmentation।
• Fine-grained classification।

Modern NLP — MixUp:

• Word/sentence embedding mix।
• Attention scores mix।
• Text classification regularization।

Bangladesh applications:

• Bangla character recognition — MixUp at hidden layer।
• Medical imaging — disease detection।
• Crop disease classification।
• Sentiment classification।

Implementation tip:

• Validation — no MixUp।
• Loss adjusted (linear combination)।
• Combine standard augmentation।
• $\alpha = 0.2$ default।

Theoretical depth:

• Zhang-Cisse-Dauphin-Lopez-Paz (২০১৭) original।
• Multiple follow-up theoretical paper।
• Connections to data augmentation theory।
• Active research।

মূল উপলব্ধি: MixUp — counter-intuitive কিন্তু theoretically grounded। Vicinal risk + linear smoothness + adversarial robust। Standard augmentation-এর সাথে combine — strong। Bangladesh-এ small data scenario — particularly useful। ML-এ "irrational" idea sometimes most effective। Empirical exploration encourage।

RandAugment-এর simplicity beat AutoAugment — DL-এ "simpler often better" lesson।

AutoAugment (২০১৮):

• RL-based search — best augmentation policy।
• Sub-policy: 5-stage augmentation chain।
• Search space — operations + magnitudes।
• 15,000 GPU hour search!
• Per-dataset optimal।

RandAugment (২০২০):

• $N$ random ops, magnitude $M$।
• No search — 2 hyperparameter total।
• Simple grid search।
• Compute negligible।
• Per-dataset tune $N, M$।

Why RandAugment beats:

• Search space too big — local optima।
• Random sufficient diversity।
• Tuning effort — practitioner ROI।
• Reproducibility better।

Empirical comparison:

• ImageNet — RandAugment ~ AutoAugment।
• CIFAR — same।
• Compute — 1000x less।
• Easy adoption।

RandAugment hyperparameters:

• $N$: number ops applied (1-3 typical)।
• $M$: magnitude (0-30 scale, 7-15 typical)।
• Larger model — bigger M helpful।
• Larger data — bigger M too।

Implementation:

from torchvision.transforms import RandAugment

# Default values
ra = RandAugment(num_ops=2, magnitude=9)

# Apply to PIL image
augmented = ra(pil_image)

Available operations:

• Identity, AutoContrast, Equalize, Rotate।
• Solarize, Color, Contrast, Brightness, Sharpness।
• ShearX, ShearY, TranslateX, TranslateY।
• Posterize।

Why "learned" augmentation overhyped:

• Search overhead massive।
• Improvement marginal vs random।
• Practical adoption barrier।
• "AutoML" tendency — over-engineering।

TrivialAugment (২০২১):

• Single random op, random magnitude।
• Even simpler!
• Comparable performance।
• Müller-Hutter paper।

Lesson — Occam's razor in ML:

• Simple methods often surprisingly strong।
• Complex methods marginal gain often।
• Engineering cost matters।
• Reproducibility valued।

Domain-specific:

• Medical — domain-aware ops add।
• Bangla text — character-specific care।
• Audio — different op set।
• Customize, not search।

RandAugment for transfer learning:

• Pretrained model + RandAugment — strong।
• Magnitude moderate (M=5-9)।
• Standard recipe।

Bangladesh adoption:

• Default — RandAugment।
• 2 hyperparameter — easy tune।
• Strong baseline immediately।
• Compute-friendly।

Modern best practice:

• RandAugment + horizontal flip + RandomErasing।
• MixUp/CutMix for additional gain।
• Domain-specific tweak।
• Hyperparameter tune।

Failure modes:

• Magnitude too high — performance drop।
• Inappropriate ops (text vertical flip)।
• Augmentation excess — under-fit।

মূল উপলব্ধি: RandAugment beat AutoAugment — simplicity > learned policy। 2 hyperparameter — tune easy। 15K GPU hour search avoid। Bangladesh practical adoption easier। ML "Occam's razor" — সরল idea অসহায় powerful। Engineering cost reproducibility critical। Default recipe modern training pipeline।

Bangla script-এ augmentation careful। Symmetry assumption ভেঙে যায় — character semantic বদলায়।

Bangla script peculiarity:

• Character — left-right asymmetric।
• Conjunct character (যুক্তাক্ষর) — complex structure।
• Vowel modifier (kar) — directional।
• Numeric ০-৯ — unique shape।

Standard augmentation problem:

• Horizontal flip — "ক" → mirror character (different meaning)।
• Vertical flip — meaningless in Bangla।
• 180° rotation — "৬" → "৯" issue।
• Random rotation large — lossy।

Bangla-safe augmentations:

• Translation: ±5-10 pixel — position invariance।
• Small rotation: ±5° (limit slight variation)।
• Scale: 0.9-1.1 — handwriting size variation।
• Brightness/contrast: mild — paper variation।
• Gaussian noise: sensor variation।
• Elastic deformation: handwriting style।

Bangla-unsafe:

• Horizontal flip ❌
• Vertical flip ❌
• Large rotation ❌
• Severe shear ❌

Recommended pipeline:

import albumentations as A

bangla_safe = A.Compose([
    A.Resize(40, 40),
    A.RandomCrop(32, 32),
    A.Rotate(limit=5, p=0.5),  # very limited
    A.Affine(translate_percent={'x': 0.1, 'y': 0.1},
             scale=(0.9, 1.1),
             p=0.5),
    A.ElasticTransform(alpha=1, sigma=5, p=0.3),
    A.GaussNoise(var_limit=(5, 15), p=0.3),
    A.RandomBrightnessContrast(0.2, 0.2, p=0.3),
    A.Normalize(mean=0.5, std=0.5),
    ToTensorV2(),
])

Synthetic data generation:

# Bangla font diverse — synthetic data
from PIL import Image, ImageFont, ImageDraw
import random

bangla_fonts = [
    "Kalpurush.ttf", "Nikosh.ttf",
    "Ekushey-Lohit.ttf", "MitraMono.ttf",
]

def generate_bangla_char(char, size=32):
    font_path = random.choice(bangla_fonts)
    font = ImageFont.truetype(font_path, size)
    img = Image.new('L', (size, size), 255)
    draw = ImageDraw.Draw(img)
    draw.text((random.randint(-2, 2),
               random.randint(-2, 2)),
              char, fill=0, font=font)
    return img

Class-specific augmentation:

• "৬" vs "৯" — separate strict augmentation।
• Confused pair — careful।
• Conjunct vs simple — different range।

Conjunct character handling:

• Variable size — adaptive crop।
• Multiple components — preserve।
• Recognition harder — more augmentation।

Real-world data variation:

• Handwriting — elastic deform।
• Print — clean, less aug।
• Scanned — JPEG artifact, blur।
• Mobile photo — perspective, light।

MixUp Bangla concerns:

• Two character mix — meaningless visual।
• Embedding-level MixUp better।
• Manifold MixUp helpful।
• Test before adopting।

OCR-specific:

• Random crop sequence preserve।
• Background augment — paper texture।
• Color/grayscale conversion।
• Resolution variation।

Validation strategy:

• Augmentation effect measure individually।
• Combination tune।
• Holdout — pure data validation।
• Real-world test critical।

Bangla-specific challenges:

• Limited labeled data।
• Handwriting variation extreme।
• Compound character complexity।
• Font diversity scarce।

Synthetic data generation:

• Multiple Bangla font।
• Random background।
• Distortion simulate।
• Augment further on synthetic।

Active learning:

• Confused predictions — manual label।
• Real handwriting samples।
• Iterative dataset growth।
• Quality > quantity।

Validation real-world:

• Bangladesh different region handwriting।
• Education level variation।
• Age group difference।
• Test set diverse।

মূল উপলব্ধি: Bangla character augmentation — domain-specific care। Standard ImageNet augmentation copy-paste — fail। "৬"-"৯" issue, flip dangers। Safe set: small rotation, translation, scale, elastic deform। Synthetic data + real handwriting + careful aug — Bangla OCR success। Cultural/script awareness — ML quality determinant।

Crop disease + drone image + small data — Bangladesh-এর realistic agri-tech project।

Domain analysis:

• Drone aerial — top-down view।
• Disease — leaf-level pattern।
• Multi-scale — close + distant।
• Lighting — sun, shadow, time।
• Crop variety — Boro, Aman, Aush।

Disease pattern variation:

• Brown spot — discoloration patch।
• Stem borer — visible damage।
• Leaf hopper — yellowing।
• Blast — fungal lesion।

Augmentation challenges:

• 2000 sample — augmentation crucial।
• Class imbalance — some disease rare।
• Real-world variation — extreme।
• Labeling noise possible।

Recommended pipeline:

import albumentations as A

train_transform = A.Compose([
    # Spatial - drone aerial flexibility
    A.Resize(640, 640),
    A.RandomResizedCrop(512, 512,
                         scale=(0.5, 1.0)),
    A.HorizontalFlip(p=0.5),
    A.VerticalFlip(p=0.5),  # aerial — okay
    A.RandomRotate90(p=0.5),
    A.Rotate(limit=45, p=0.5),

    # Lighting - field variation
    A.RandomBrightnessContrast(
        brightness_limit=0.3,
        contrast_limit=0.3, p=0.7),
    A.HueSaturationValue(
        hue_shift_limit=20,
        sat_shift_limit=30,
        val_shift_limit=20, p=0.5),

    # Weather - Bangladesh monsoon
    A.OneOf([
        A.RandomRain(p=0.5),
        A.RandomFog(p=0.5),
        A.RandomShadow(p=0.5),
    ], p=0.3),

    # Quality - drone variation
    A.OneOf([
        A.GaussianBlur(blur_limit=5, p=0.5),
        A.MotionBlur(blur_limit=7, p=0.5),
        A.Defocus(p=0.3),
    ], p=0.3),

    # Noise
    A.GaussNoise(var_limit=(10, 50), p=0.3),
    A.ISONoise(p=0.3),

    # Disease-relevant
    A.ChannelShuffle(p=0.1),  # Carefully
    A.Normalize(),
    ToTensorV2(),
])

Mosaic augmentation (YOLOv4):

def mosaic_4(images, bboxes, size=512):
    """4 image combine in 2x2 grid"""
    canvas = np.zeros((size, size, 3), dtype=np.uint8)
    cx, cy = np.random.randint(size//4, 3*size//4)
    for idx, (img, bb) in enumerate(zip(images, bboxes)):
        # Place each image in quadrant
        if idx == 0:  # top-left
            x1a, y1a, x2a, y2a = 0, 0, cx, cy
        elif idx == 1:  # top-right
            x1a, y1a, x2a, y2a = cx, 0, size, cy
        elif idx == 2:  # bottom-left
            x1a, y1a, x2a, y2a = 0, cy, cx, size
        else:  # bottom-right
            x1a, y1a, x2a, y2a = cx, cy, size, size
        canvas[y1a:y2a, x1a:x2a] = resize(img,
            (y2a - y1a, x2a - x1a))
    return canvas

Class balance:

from torch.utils.data import WeightedRandomSampler

class_counts = compute_per_class()
weights = 1.0 / np.sqrt(class_counts)  # sqrt smooth
sample_weights = weights[targets]
sampler = WeightedRandomSampler(
    sample_weights,
    len(sample_weights),
    replacement=True)

MixUp + CutMix combined:

• 50% standard augmentation।
• 25% MixUp।
• 25% CutMix।
• Diversity maximize।

Synthetic data:

• GAN-based generation।
• Style transfer healthy → disease।
• Limited but supplementary।
• Validate real-world transfer।

Test-time augmentation:

def tta_disease(model, image, n=8):
    preds = []
    for _ in range(n):
        # Different augmentation
        x_aug = drone_test_aug(image)
        with torch.no_grad():
            preds.append(F.softmax(model(x_aug), -1))
    return torch.stack(preds).mean(0)

Bangladesh-specific augmentations:

• Monsoon rain: A.RandomRain — common scenario।
• Hot sun: brightness extreme।
• Dust: low contrast।
• Crop stage: growth phase variation।

Validation strategy:

• Field-wise hold-out।
• Season-wise split।
• Variety-wise generalization।
• Real-world deployment test।

Domain expert validation:

• Agronomist review augmentation।
• Visually inspect synthetic।
• Disease-specific consistency।
• False positive analysis।

Practical deployment:

• Drone onboard inference।
• Bangla farmer app।
• GPS-tagged disease map।
• Treatment recommendation।

Continuous improvement:

• New season — new data।
• New disease emergence track।
• Active learning loop।
• Regional adaptation।

Edge cases:

• Mixed disease (multiple)।
• Early-stage subtle।
• Healthy crop confusion (autumn yellowing)।
• Weather damage vs disease।

Dataset growth:

• Crowdsource farmer photo।
• University research collaboration।
• BARI (Bangladesh Agricultural Research Institute) partnership।
• Open dataset contribution।

Realistic expectation:

• 2000 sample — 75-85% achievable।
• With ensemble + TTA — 88%।
• 10000 sample target — 95%+।
• Production threshold — 90%।

Economic impact:

• Early detection — yield save।
• Targeted spray — pesticide reduce।
• Cost — farmer benefit।
• Sustainable agriculture।

মূল উপলব্ধি: Crop disease detection — augmentation extensive use। Drone-aware (rotation, vertical flip ok), weather-aware (rain, fog), quality-aware (blur)। Mosaic + MixUp + CutMix combined। Bangladesh-এ — monsoon, sun, dust simulate। Active learning + farmer crowdsource। 2000 sample augmentation দিয়ে production-quality achievable। Domain-specific aug — ML practical impact। Agriculture AI Bangladesh transformation potential।