Research Archive — GAL RESEARCH

This is the complete GAL RESEARCH index. Every article is a working research note rather than a final textbook chapter. Entries will be updated as experiments, sources and implementations improve.

★ Scaling & Compute ★

COVER STORY

Can Smaller Models Think Bigger?

Model size, data, compute budgets and why efficiency matters for independent AI research.

STATUS: PUBLISHED

SYSTEMS

Training With Limited Compute

Mixed precision, accumulation, checkpointing and practical experiments on a consumer GPU.

STATUS: PUBLISHED

♥ Architecture & Papers ♥

PAPER NOTES

Attention Is All You Need

Self-attention, positional encoding, multi-head attention and the original Transformer.

STATUS: PUBLISHED

COMING SOON

Mixture of Experts

Sparse routing, expert specialization and scaling capacity without activating every parameter.

STATUS: PLANNED

★ Data Quality ★

DATA

When More Data Stops Helping

Duplication, contamination, filtering and the difference between dataset size and dataset value.

STATUS: PUBLISHED

COMING SOON

Synthetic Data Without Delusion

When generated training data helps, when it collapses and how to evaluate it honestly.

STATUS: PLANNED

♥ Inference & Reasoning ♥

INFERENCE

Test-Time Compute Is the New Black

Multiple candidates, search, verifiers and the systems cost of spending more compute after training.

STATUS: PUBLISHED

COMING SOON

Quantization for Gal Scientists

Lower precision, memory savings, inference speed and the accuracy tradeoffs behind smaller deployment formats.

STATUS: PLANNED

★ Lab Diary ★

EXPERIMENT 001

Ayda's Tiny AI Lab

Run logs, model configurations, failures, memory measurements and tiny language-model experiments.

STATUS: ACTIVE

COMING SOON

Experiment 002

A controlled comparison of raw data and filtered data under the same training-token budget.

STATUS: PLANNED