WhiteLightning - LLM Distillation Tool

WhiteLightning is a powerful tool designed to distill large language models (LLMs) into lightweight, efficient text classifiers. By leveraging advanced techniques, it simplifies the process of creating text classifiers that can run anywhere, from cloud environments to edge devices, using the ONNX format for cross-platform compatibility.

whitelightning identify toxic comments in online discussions WL identify toxic comments!

WhiteLightning > I'll help you create a binary classifier for toxic comments detection. Creating a binary classifier...

WhiteLightning > First, I'll generate synthetic training data using LLM... Generating synthetic data...

WhiteLightning > Created 1000 examples (500 spam, 500 ham) Created 1000 examples

WhiteLightning > Training lightweight classifier model... Training model...

WhiteLightning > Model trained successfully. Accuracy: 94.2% Model trained. Accuracy: 94.2%

WhiteLightning > Saved model to toxic_comments_classifier.onnx (2.1MB) Model saved

🧠

What is LLM Distillation?

LLM distillation is the process of transforming large, complex language models into smaller, task-specific models. WhiteLightning focuses on text classification, enabling users to create efficient classifiers for various use cases. This approach ensures high performance while significantly reducing computational requirements.

πŸ”„

Why ONNX?

WhiteLightning uses ONNX (Open Neural Network Exchange) to export trained models, making them deployable across a wide range of platforms and programming languages. With ONNX, you can run your models in Python, JavaScript, C++, Rust, and more, ensuring flexibility and scalability for your applications.

Key Features

πŸ”’

Multiple Model Types

Support for binary and multiclass classification with different activation types

🌐

Cross-Platform Deployment

Export models to ONNX for use in diverse environments

⚑

Lightweight and Fast

Optimized for performance with minimal resource usage

πŸ”§

Customizable

Supports multiple machine learning frameworks (TensorFlow, PyTorch, Scikit-learn)

🌍

Multilingual Support

Generate training data in multiple languages

πŸ€–

Automatic Configuration

Smart prompt generation and refinement based on your task

Model Types

Binary Classifier

Simple yes/no or true/false classification with probability output for single class. Ideal for sentiment analysis, spam detection, and content moderation.

Learn More

Multiclass Classifier

Single-label classification with mutually exclusive class probabilities.Ideal for news categorization, intent classification, and language detection.

Learn More

Quick Start

Pull the Docker image:

docker pull ghcr.io/whitelightning-ai/whitelightning:latest docker pull ghcr.io/whitelightning-ai /whitelightning:latest

Get your OpenRouter API key:

# Visit https://openrouter.ai # Sign up and create an API key

Run the classifier agent:

docker run --rm -v $(pwd):/app/models -e OPEN_ROUTER_API_KEY="YOUR_OPEN_ROUTER_KEY_HERE" ghcr.io/whitelightning-ai/whitelightning:latest -p="Classify customer feedback as positive or negative sentiment" docker run --rm -v $(pwd):/app/models -e OPEN_ROUTER_API_KEY="YOUR_OPEN_ROUTER_KEY_HERE". env ghcr.io/whitelightning-ai/ /whitelightning:latest -p= "Classify customer feedback as positive or negative sentiment"

Parameters


Parameter                   Description                                          Default
------------------------    ------------------------------------------------     ----------------------
-p --problem_description    Description of your classification problem           (Required)

--model-type                ML library to use for the classifier model           tensorflow
                            tensorflow | torch | sklearn

--data-gen-model            OpenRouter model name for bulk data generation       openai/gtp-4o-mini

--config-llm-model          OpenRouter model for config, refinement,             x-ai/grok-3-beta
                            edge cases, analysis (should be powerful,
                            e.g., GPT-4, Claude Opus).",

--refinement-cycles         Number of prompt refinement cycles (0 to disable).   1

--generate-edge-cases       Generate challenging edge case data (true/false)     true

--batch-size                Number of parallel LLM API requests for bulk data
                            generation batches.                                  10

--lang                      Primary language for the generated dataset           english
                            (e.g., english, german, spanish, etc.).

Advanced Parameters


Parameter                   Description
------------------------    ----------------------------------------------------------------------------
--config-path               Continue from a previous configuration file. Can be used for additional data
                            generation or retraining with user data or another library.

--skip-data-gen             Skip data generation and only train the model from existing data. Works only
                            if training_data.csv is present in the model directory and --config-path is
                            specified.

--skip-model-training       Skip model training and only generate data

Output

The trained model and associated files will be saved in the specified output directory:

                    
.[model_prefix]/
    β”œβ”€β”€ api_requests/                  # API requests used for training data generation
    β”œβ”€β”€ generation_config.json         # Configuration and analysis
    β”œβ”€β”€ edge_case_data.csv             # Sophisticated data used for model evaluation by LLM
    β”œβ”€β”€ model.onnx                     # ONNX model file
    β”œβ”€β”€ scaler.json                    # StandardScaler parameters
    β”œβ”€β”€ tokenizer.json                 # Tokenizer configuration (for softmax models)
    β”œβ”€β”€ training_data.csv              # Training data used for model training
    └── vocab.json                     # TF-IDF vocabulary (for sigmoid models)
    + ... native model files .h5, .keras
                    
                

Deployment Guide

Model Deployment

Learn how to deploy your ONNX models across different platforms and environments for maximum flexibility.

Binary Multiclass Demos