WhiteLightning - LLM Distillation Tool
WhiteLightning is a powerful tool designed to distill large language models (LLMs) into lightweight, efficient text classifiers. By leveraging advanced techniques, it simplifies the process of creating text classifiers that can run anywhere, from cloud environments to edge devices, using the ONNX format for cross-platform compatibility.
whitelightning identify toxic comments in online discussions WL identify toxic comments!
WhiteLightning > I'll help you create a binary classifier for toxic comments detection. Creating a binary classifier...
WhiteLightning > First, I'll generate synthetic training data using LLM... Generating synthetic data...
WhiteLightning > Created 1000 examples (500 spam, 500 ham) Created 1000 examples
WhiteLightning > Training lightweight classifier model... Training model...
WhiteLightning > Model trained successfully. Accuracy: 94.2% Model trained. Accuracy: 94.2%
WhiteLightning > Saved model to toxic_comments_classifier.onnx (2.1MB) Model saved
What is LLM Distillation?
LLM distillation is the process of transforming large, complex language models into smaller, task-specific models. WhiteLightning focuses on text classification, enabling users to create efficient classifiers for various use cases. This approach ensures high performance while significantly reducing computational requirements.
Why ONNX?
WhiteLightning uses ONNX (Open Neural Network Exchange) to export trained models, making them deployable across a wide range of platforms and programming languages. With ONNX, you can run your models in Python, JavaScript, C++, Rust, and more, ensuring flexibility and scalability for your applications.
Key Features
Multiple Model Types
Support for binary and multiclass classification with different activation types
Cross-Platform Deployment
Export models to ONNX for use in diverse environments
Lightweight and Fast
Optimized for performance with minimal resource usage
Customizable
Supports multiple machine learning frameworks (TensorFlow, PyTorch, Scikit-learn)
Multilingual Support
Generate training data in multiple languages
Automatic Configuration
Smart prompt generation and refinement based on your task
Model Types
Binary Classifier
Simple yes/no or true/false classification with probability output for single class. Ideal for sentiment analysis, spam detection, and content moderation.
Learn MoreMulticlass Classifier
Single-label classification with mutually exclusive class probabilities.Ideal for news categorization, intent classification, and language detection.
Learn MoreQuick Start
Pull the Docker image:
docker pull ghcr.io/whitelightning-ai/whitelightning:latest
docker pull ghcr.io/whitelightning-ai
/whitelightning:latest
Get your OpenRouter API key:
# Visit https://openrouter.ai
# Sign up and create an API key
Run the classifier agent:
docker run --rm -v $(pwd):/app/models -e
OPEN_ROUTER_API_KEY="YOUR_OPEN_ROUTER_KEY_HERE"
ghcr.io/whitelightning-ai/whitelightning:latest
-p="Classify customer feedback as positive or negative
sentiment"
docker run --rm -v $(pwd):/app/models
-e OPEN_ROUTER_API_KEY="YOUR_OPEN_ROUTER_KEY_HERE".
env ghcr.io/whitelightning-ai/
/whitelightning:latest -p=
"Classify customer feedback as
positive or negative sentiment"
Parameters
Parameter Description Default
------------------------ ------------------------------------------------ ----------------------
-p --problem_description Description of your classification problem (Required)
--model-type ML library to use for the classifier model tensorflow
tensorflow | torch | sklearn
--data-gen-model OpenRouter model name for bulk data generation openai/gtp-4o-mini
--config-llm-model OpenRouter model for config, refinement, x-ai/grok-3-beta
edge cases, analysis (should be powerful,
e.g., GPT-4, Claude Opus).",
--refinement-cycles Number of prompt refinement cycles (0 to disable). 1
--generate-edge-cases Generate challenging edge case data (true/false) true
--batch-size Number of parallel LLM API requests for bulk data
generation batches. 10
--lang Primary language for the generated dataset english
(e.g., english, german, spanish, etc.).
Advanced Parameters
Parameter Description
------------------------ ----------------------------------------------------------------------------
--config-path Continue from a previous configuration file. Can be used for additional data
generation or retraining with user data or another library.
--skip-data-gen Skip data generation and only train the model from existing data. Works only
if training_data.csv is present in the model directory and --config-path is
specified.
--skip-model-training Skip model training and only generate data
Output
The trained model and associated files will be saved in the specified output directory:
.[model_prefix]/
βββ api_requests/ # API requests used for training data generation
βββ generation_config.json # Configuration and analysis
βββ edge_case_data.csv # Sophisticated data used for model evaluation by LLM
βββ model.onnx # ONNX model file
βββ scaler.json # StandardScaler parameters
βββ tokenizer.json # Tokenizer configuration (for softmax models)
βββ training_data.csv # Training data used for model training
βββ vocab.json # TF-IDF vocabulary (for sigmoid models)
+ ... native model files .h5, .keras
Deployment Guide
Model Deployment
Learn how to deploy your ONNX models across different platforms and environments for maximum flexibility.
Binary Multiclass Demos