Modern C++ neural-network engineering

Understand the math. Inspect the code. Benchmark the system.

A from-scratch C++17 neural-network library for learning, experimentation, numerical validation, and systems-level ML engineering — with reusable CMake targets and optional OpenMP, BLAS, and CUDA backends.

View repository CI pipeline

C++17 Core implementation

7 tests CTest baseline

4 backends CPU, OpenMP, BLAS, CUDA

CMake Installable package

Current capabilities

Small enough to inspect, structured enough to extend

The project keeps neural-network mechanics explicit while still treating build, packaging, testing, numerical validation, and backend selection as first-class engineering concerns.

LayersDense, ReLU, Sigmoid, Tanh

LossBinary Cross-Entropy

OptimizersSGD, Momentum, NAG, Adam, AdamW, Nadam, RMSProp, AdaGrad, AdaDelta, Lion

ExecutionCPU, OpenMP, BLAS, experimental CUDA

ValidationUnit tests, finite-difference gradient checks, Valgrind CI

PackagingCMake install, exported targets, find_package(mlp)

Architecture

Small modules, clear data flow

Sequential owns the layer stack. Forward propagation produces predictions, the loss starts backpropagation, and optimizers update trainable parameters through explicit parameter/gradient references.

Input→ Dense→ Activation→ Dense→ Prediction

Target→ Loss→ Backward gradients

Parameter refs→ Optimizer→ Updated weights

C++ API

Build a network with explicit components

The public surface stays deliberately small while the training mechanics remain inspectable.

Minimal model

std::mt19937 rng(42);

mlp::Sequential model;
model.add(std::make_unique<mlp::Dense>(2, 16, rng));
model.add(std::make_unique<mlp::Tanh>());
model.add(std::make_unique<mlp::Dense>(16, 1, rng));
model.add(std::make_unique<mlp::Sigmoid>());

mlp::Adam optimizer(0.01);

Local workflow

Build, run, validate

Quick start

cmake -S . -B build
cmake --build build
./build/mlp
ctest --test-dir build --output-on-failure

BLAS benchmark

cmake -S . -B build-blas -DMLP_ENABLE_BLAS=ON
cmake --build build-blas
./build-blas/mlp_dense_benchmark --min-speedup 2.0

Roadmap

A focused path toward v1.0

The next milestones prioritize practical MLP training and multiclass support before broader framework scope.

Foundation

Reliability baseline

Unit tests, gradient checks, Valgrind CI, BLAS acceleration, architecture docs, and reproducible builds.

Now

Training foundations

#8 mini-batches, #55 initialization, and #18 multiclass loss/activation.

Multiclass & validation

#59 stable logits CE, #56 metrics, and #57 reference parity.

Documentation

Everything needed to explore and contribute

Start

Pick a focused issue and make the project sharper

Good contributions include tests, docs, optimizer improvements, backend benchmarking, package polish, and small refactors that keep the math easy to inspect.

Browse issues Open pull requests