Optuna

✓ Editorially verified

Open-source Python framework for automated hyperparameter optimization across any ML stack.

Free· Free and open source (MIT)Fine-tuning

Best for

Pick Optuna if you want a reproducible, code-first way to tune hyperparameters for any ML or LLM fine-tuning pipeline without locking into a vendor.

Skip if

Skip it if you want a one-click hosted AutoML product or a no-code interface for non-engineers.

Optuna is an open-source hyperparameter optimization framework built in Python and maintained by Preferred Networks. It uses define-by-run search spaces (so you can express conditionals and loops in standard Python), pairs that with modern samplers like TPE, CMA-ES and Gaussian-process Bayesian optimization, and prunes unpromising trials early to save compute. A study object tracks all trials, and results can be inspected through the bundled Optuna Dashboard with Jupyter Lab and VS Code extensions.

It's framework-agnostic: PyTorch, TensorFlow/Keras, scikit-learn, XGBoost, LightGBM and basically anything you can wrap in an objective function. Parallel and distributed search works without code changes, and OptunaHub lets the community publish custom samplers and pruners. Pricing is simple, it's free and MIT-licensed. The audience is ML engineers and researchers who want a serious tuner without committing to a closed AutoML platform.

Fine-tuning teams typically reach for Optuna when they need to sweep learning rates, LoRA ranks, batch sizes or architecture choices and want reproducible studies backed by a SQL or in-memory storage. It is a library, not a hosted service, so you bring your own compute and orchestration.

Editor's take

Optuna is the default we'd reach for when a fine-tuning run needs real hyperparameter search rather than a hand-tuned guess. The define-by-run API and pruning support make it noticeably more pleasant than older grid-search tooling, and the dashboard finally makes long studies readable.

— The AI Tool Bible editorial team