Running Multiclass Classifier Models

WhiteLightning.ai Intro

🔄 Preprocessing: Tokenizing the Text

1

Text Input

Start with a string (e.g., "The government announced new policies")

2

Tokenization

Convert words to lowercase and map them to integer IDs using a tokenizer vocabulary (from _tokenizer.json). Unknown words use the <OOV> token (default ID: 1)

3

Padding/Truncation

Truncate to 30 tokens and pad with zeros to ensure a fixed-length sequence

4

Output

A 30-element int32 array, ready for the ONNX model

💻 Running Guide

🧠 How to Run an ONNX Model with Python: Full Beginner-Friendly Guide

📌 What is this?

This guide walks you through running an ONNX model for text classification using Python, starting from scratch — including Python installation, setting up dependencies, and running the model.

Choose your path:

✅ 1. Install Python

🔷 Windows
  1. Go to: https://www.python.org/downloads/windows
  2. Download the latest Python 3.11+ installer
  3. During installation, check ✅ Add Python to PATH
  4. After installation, check if it worked:
python --version
🍏 macOS

You have two options to install Python:

  1. Option 1 - Official Website:
  2. Option 2 - Homebrew:
    • Install Homebrew (if you don't have it):
    /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

    Then install Python:

    brew install python@3.11

After installation, check if it worked:

python3 --version

Note: macOS uses python3, not python.

🐧 Linux (Ubuntu/Debian)

You have two options to install Python:

  1. Option 1 - Package Manager: 
    sudo apt update
sudo apt install python3 python3-pip
  2. Option 2 - Official Website: 
    tar -xf Python-3.11.x.tar.xz
cd Python-3.11.x
./configure
make
sudo make install

After installation, check if it worked:

python3 --version

✅ 2. Get Your Model

🔄 Download Repository

Clone our repository to get started:

git clone https://github.com/whitelightning-ai/whitelightning.git
cd whitelightning.ai
📦 Choose Your Model

You have two options:

  1. Use Pre-trained Model:
    • Navigate to the models directory
    • Copy the multiclass classifier model files to your project:
      • model.onnx - The ONNX model file
      • model_vocab.json - Tokenizer vocabulary
      • model_labels.json - Class labels mapping
  2. Train Your Own Model:
    • Follow the training guide in the repository
    • Use the provided scripts to train a custom multiclass classifier
    • Export your model to ONNX format

✅ 3. Set Up Your Project Folder

mkdir onnx_python_demo
cd onnx_python_demo

Folder structure:

onnx_python_demo/
├── model.onnx
├── vocab.json
├── scaler.json
└── run_onnx.py

✅ 4. Install Required Python Libraries

pip install onnxruntime numpy

On macOS or Linux, you might need to run pip3 install instead.

✅ 5. Prepare Supporting Files

🔹 vocab.json (tokenizer dictionary)
{
  "": 1,
  "the": 2,
  "government": 3,
  "announced": 4,
  "new": 5,
  "policies": 6,
  "to": 7,
  "boost": 8,
  "economy": 9
}
🔹 scaler.json (label map)
{
  "0": "Politics",
  "1": "Sports",
  "2": "Technology"
}

Adjust to match your model's output classes.

🔹 model.onnx

Place your trained ONNX model here. It should accept a (1, 30) input tensor of int32.

✅ 6. Create the Python Script run_onnx.py

Use the code example below:

import json
import numpy as np
import onnxruntime as ort

def preprocess_text(text, tokenizer_file):
    with open(tokenizer_file, 'r') as f:
        tokenizer = json.load(f)
    
    oov_token = '<OOV>'
    words = text.lower().split()
    sequence = [tokenizer.get(word, tokenizer.get(oov_token, 1)) for word in words]
    sequence = sequence[:30]  # Truncate to max_len
    padded = np.zeros(30, dtype=np.int32)
    padded[:len(sequence)] = sequence  # Pad with zeros
    return padded

# Test
text = "The government announced new policies to boost the economy"
vector = preprocess_text(text, 'vocab.json')

session = ort.InferenceSession('model.onnx')
input_name = session.get_inputs()[0].name
output_name = session.get_outputs()[0].name
input_data = vector.reshape(1, 30)
outputs = session.run([output_name], {input_name: input_data})

# Load label map
with open('scaler.json', 'r') as f:
    label_map = json.load(f)

probabilities = outputs[0][0]
predicted_idx = np.argmax(probabilities)
label = label_map[str(predicted_idx)]
score = probabilities[predicted_idx]
print(f'Python ONNX output: {label} (Score: {score:.4f})')

✅ 7. Run the Script

🔷 Windows
python run_onnx.py
🍏 macOS/Linux
python3 run_onnx.py

✅ 8. Expected Output

Prediction: Score: 0.9732

The output shows the predicted class and its confidence score. In this example, the model predicted "Politics" with a confidence of 97.32%.

🧠 How to Run an ONNX Model with JavaScript (Browser or Node.js)

📌 What is this?

This guide explains how to load and run ONNX models using JavaScript and ONNX Runtime Web, covering both browser and Node.js environments.

Choose your path:

✅ 1. Choose Your Runtime

You can run ONNX models in JavaScript in two ways:

Environment Description Recommended For
✅ Browser Uses WebAssembly or WebGL Web apps, frontend demos
✅ Node.js Uses Node runtime (CPU only) Backend/CLI usage

✅ 2. Requirements

🔷 For browser

No install — just include the library from a CDN or bundle via npm.

🟩 For Node.js

Install Node.js:

  1. Download from: https://nodejs.org/
  2. Check installation:
node -v
npm -v

Then install ONNX Runtime:

npm install onnxruntime-web

✅ 3. Get Your Model

🔄 Download Repository

Clone our repository to get started:

git clone https://github.com/whitelightning-ai/whitelightning.git
cd whitelightning.ai
📦 Choose Your Model

You have two options:

  1. Use Pre-trained Model:
    • Navigate to the models directory
    • Copy the multiclass classifier model files to your project:
      • model.onnx - The ONNX model file
      • model_vocab.json - Tokenizer vocabulary
      • model_labels.json - Class labels mapping
  2. Train Your Own Model:
    • Follow the training guide in the repository
    • Use the provided scripts to train a custom multiclass classifier
    • Export your model to ONNX format

✅ 4. Folder Setup

mkdir onnx_js_demo
cd onnx_js_demo

Files you'll need:

onnx_js_demo/
├── index.html            # For browser use
├── run.js                # Main logic
├── model.onnx
├── vocab.json
├── scaler.json

✅ 5. Sample vocab.json

{
  "": 1,
  "the": 2,
  "government": 3,
  "announced": 4,
  "new": 5,
  "policies": 6,
  "to": 7,
  "boost": 8,
  "economy": 9
}

✅ 6. Sample scaler.json

{
  "0": "Politics",
  "1": "Sports",
  "2": "Technology"
}

✅ 7. JavaScript Code (run.js)

Works in both browser and Node.js (with minor changes)

async function preprocessText(text, tokenizerUrl) {
const tokenizerResp = await fetch(tokenizerUrl);
const tokenizer = await tokenizerResp.json();

const oovToken = '<OOV>';
const words = text.toLowerCase().split(/\s+/);
const sequence = words.map(word => tokenizer[word] || tokenizer[oovToken] || 1).slice(0, 30);
const padded = new Int32Array(30).fill(0);
sequence.forEach((val, idx) => padded[idx] = val);
return padded;
}

async function runModel(text) {
const session = await ort.InferenceSession.create('model.onnx');
const vector = await preprocessText(text, 'vocab.json');
const tensor = new ort.Tensor('int32', vector, [1, 30]);
const feeds = { input: tensor };
const output = await session.run(feeds);

const labelResp = await fetch('scaler.json');
const labelMap = await labelResp.json();

const probabilities = output[Object.keys(output)[0]].data;
const predictedIdx = probabilities.reduce((maxIdx, val, idx) => val > probabilities[maxIdx] ? idx : maxIdx, 0);
const label = labelMap[predictedIdx];
const score = probabilities[predictedIdx];
console.log(`JS ONNX output: ${label} (Score: ${score.toFixed(4)})`);
}

runModel('The government announced new policies to boost the economy');

✅ 8. Run in Browser (option A)

🔹 index.html
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <title>ONNX JS Inference</title>
</head>
<body>
  <h1>Running ONNX Model...</h1>
  <script src="https://cdn.jsdelivr.net/npm/onnxruntime-web/dist/ort.min.js"></script>
  <script type="module" src="run.js"></script>
</body>
</html>

📦 Start a local server (required due to fetch):

npx serve .
# OR
python3 -m http.server

Visit: http://localhost:3000

✅ 9. Run with Node.js (option B)

🔹 Modify run.js for Node
import * as ort from 'onnxruntime-node';
import fs from 'fs/promises';

async function loadJSON(path) {
  const data = await fs.readFile(path, 'utf-8');
  return JSON.parse(data);
}

// Keep rest of logic same from previous example

📦 Run it:

node run.js

✅ 10. Expected Output

JS ONNX output: Score: 0.9732

🧠 How to Run an ONNX Model with C using ONNX Runtime and cJSON

📌 What is this?

This guide explains how to load and run ONNX models using C, ONNX Runtime C API, and cJSON for JSON parsing.

Choose your path:

✅ 1. Prerequisites

🔷 C Compiler

macOS: clang comes with Xcode Command Line Tools

xcode-select --install

Linux: install gcc

sudo apt install build-essential
🟩 ONNX Runtime C Library

Download ONNX Runtime C API from the official website:

👉 https://github.com/microsoft/onnxruntime/releases

Choose:

onnxruntime-osx-universal2-.tgz   # For macOS
onnxruntime-linux-x64-.tgz        # For Linux
📦 Install cJSON

macOS:

brew install cjson

Linux:

sudo apt install libcjson-dev

✅ 2. Choose Your Model

🔄 Download Repository

Clone our repository to get started:

git clone https://github.com/whitelightning-ai/whitelightning.git
cd whitelightning.ai
📦 Choose Your Model

You have two options:

  1. Use Pre-trained Model:
    • Navigate to the models directory
    • Copy the multiclass classifier model files to your project:
      • model.onnx - The ONNX model file
      • model_vocab.json - Tokenizer vocabulary
      • model_labels.json - Class labels mapping
  2. Train Your Own Model:
    • Follow the training guide in the repository
    • Use the provided scripts to train a custom multiclass classifier
    • Export your model to ONNX format

✅ 3. Folder Structure

project/
├── ONNX_test.c               ← your C code
├── vocab.json                ← tokenizer
├── scaler.json               ← label map
├── model.onnx                ← ONNX model
├── onnxruntime-osx-universal2-1.22.0/
│   ├── include/
│   └── lib/

✅ 4. Build Command

🔷 macOS
gcc ONNX_test.c \
  -I./onnxruntime-osx-universal2-1.22.0/include \
  -L./onnxruntime-osx-universal2-1.22.0/lib \
  -lonnxruntime \
  -lcjson \
  -o onnx_test
🐧 Linux

Replace the onnxruntime-osx-... path with onnxruntime-linux-x64-....

✅ 5. Run the Executable

🔷 macOS

Important: You must set the library path.

export DYLD_LIBRARY_PATH=./onnxruntime-osx-universal2-1.22.0/lib:$DYLD_LIBRARY_PATH
./onnx_test
🐧 Linux
export LD_LIBRARY_PATH=./onnxruntime-linux-x64-1.22.0/lib:$LD_LIBRARY_PATH
./onnx_test

✅ 6. C Code Example

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <ctype.h>
#include "onnxruntime-osx-universal2-1.22.0/include/onnxruntime_c_api.h"
#include <cjson/cJSON.h>

const OrtApi* g_ort = NULL;

int32_t* preprocess_text(const char* text, const char* tokenizer_file) {
    int32_t* vector = calloc(30, sizeof(int32_t));

    FILE* f = fopen(tokenizer_file, "r");
    if (!f) return NULL;
    
    fseek(f, 0, SEEK_END);
    long len = ftell(f);
    fseek(f, 0, SEEK_SET);
    char* json_str = malloc(len + 1);
    fread(json_str, 1, len, f);
    json_str[len] = 0;
    fclose(f);

    cJSON* tokenizer = cJSON_Parse(json_str);
    if (!tokenizer) {
        free(json_str);
        return NULL;
    }

    char* text_copy = strdup(text);
    for (char* p = text_copy; *p; p++) *p = tolower(*p);

    char* word = strtok(text_copy, " \t\n");
    int idx = 0;
    while (word && idx < 30) {
        cJSON* token = cJSON_GetObjectItem(tokenizer, word);
        vector[idx++] = token ? token->valueint : (cJSON_GetObjectItem(tokenizer, "<OOV>") ? cJSON_GetObjectItem(tokenizer, "<OOV>")->valueint : 1);
        word = strtok(NULL, " \t\n");
    }

    free(text_copy);
    free(json_str);
    cJSON_Delete(tokenizer);
    return vector;
}

int main() {
    g_ort = OrtGetApiBase()->GetApi(ORT_API_VERSION);
    if (!g_ort) return 1;

    const char* text = "That's really thoughtful feedback — thank you.";
    int32_t* vector = preprocess_text(text, "hate_speech(English)/vocab.json");
    if (!vector) return 1;

    OrtEnv* env;
    OrtStatus* status = g_ort->CreateEnv(ORT_LOGGING_LEVEL_WARNING, "test", &env);
    if (status) return 1;

    OrtSessionOptions* session_options;
    status = g_ort->CreateSessionOptions(&session_options);
    if (status) return 1;

    OrtSession* session;
    status = g_ort->CreateSession(env, "hate_speech(English)/model.onnx", session_options, &session);
    if (status) return 1;

    OrtMemoryInfo* memory_info;
    status = g_ort->CreateCpuMemoryInfo(OrtArenaAllocator, OrtMemTypeDefault, &memory_info);
    if (status) return 1;

    int64_t input_shape[] = {1, 30};
    OrtValue* input_tensor;
    status = g_ort->CreateTensorWithDataAsOrtValue(memory_info, vector, 30 * sizeof(int32_t), input_shape, 2, ONNX_TENSOR_ELEMENT_DATA_TYPE_INT32, &input_tensor);
    if (status) return 1;

    const char* input_names[] = {"input"};
    const char* output_names[] = {"sequential"};
    OrtValue* output_tensor = NULL;
    status = g_ort->Run(session, NULL, input_names, (const OrtValue* const*)&input_tensor, 1, output_names, 1, &output_tensor);
    if (status) return 1;

    float* output_data;
    status = g_ort->GetTensorMutableData(output_tensor, (void**)&output_data);
    if (status) return 1;

    FILE* f = fopen("hate_speech(English)/scaler.json", "r");
    if (!f) return 1;

    fseek(f, 0, SEEK_END);
    long len = ftell(f);
    fseek(f, 0, SEEK_SET);
    char* json_str = malloc(len + 1);
    fread(json_str, 1, len, f);
    json_str[len] = 0;
    fclose(f);

    cJSON* label_map = cJSON_Parse(json_str);
    if (!label_map) {
        free(json_str);
        return 1;
    }

    int predicted_idx = 0;
    float max_prob = output_data[0];
    int num_classes = cJSON_GetArraySize(label_map);
    for (int i = 1; i < num_classes; i++) {
        if (output_data[i] > max_prob) {
            max_prob = output_data[i];
            predicted_idx = i;
        }
    }

    char idx_str[16];
    snprintf(idx_str, sizeof(idx_str), "%d", predicted_idx);
    cJSON* label = cJSON_GetObjectItem(label_map, idx_str);
    if (!label) return 1;
    
    printf("C ONNX output: %s (Score: %.4f)\n", label->valuestring, max_prob);

    g_ort->ReleaseValue(input_tensor);
    g_ort->ReleaseValue(output_tensor);
    g_ort->ReleaseMemoryInfo(memory_info);
    g_ort->ReleaseSession(session);
    g_ort->ReleaseSessionOptions(session_options);
    g_ort->ReleaseEnv(env);

    free(vector);
    free(json_str);
    cJSON_Delete(label_map);

    return 0;
}

C ONNX output: Not_hate (Score: 0.9971)

✅ 7. Expected Output

C ONNX output: Not_hate (Score: 0.9971)

🧠 How to Run an ONNX Model with C++ using ONNX Runtime and nlohmann/json

📌 What is this?

This guide explains how to load and run ONNX models using C++, ONNX Runtime C++ API, and nlohmann/json for JSON parsing.

Choose your path:

✅ 1. Prerequisites

🔷 C++ Compiler

macOS: clang++ comes with Xcode Command Line Tools

xcode-select --install

Linux: install g++

sudo apt install build-essential
🟩 ONNX Runtime C++ Library

Download ONNX Runtime C++ API from the official website:

👉 https://github.com/microsoft/onnxruntime/releases

Choose:

onnxruntime-osx-universal2-.tgz   # For macOS
onnxruntime-linux-x64-.tgz        # For Linux
📦 Install nlohmann/json

macOS:

brew install nlohmann-json

Linux:

sudo apt install nlohmann-json3-dev

✅ 2. Choose Your Model

🔄 Download Repository

Clone our repository to get started:

git clone https://github.com/whitelightning-ai/whitelightning.git
cd whitelightning.ai
📦 Choose Your Model

You have two options:

  1. Use Pre-trained Model:
    • Navigate to the models directory
    • Copy the multiclass classifier model files to your project:
      • model.onnx - The ONNX model file
      • model_vocab.json - Tokenizer vocabulary
      • model_labels.json - Class labels mapping
  2. Train Your Own Model:
    • Follow the training guide in the repository
    • Use the provided scripts to train a custom multiclass classifier
    • Export your model to ONNX format

✅ 3. Folder Structure

project/
├── main.cpp                ← your C++ code
├── vocab.json             ← tokenizer
├── scaler.json            ← label map
├── model.onnx             ← ONNX model
├── onnxruntime-osx-universal2-1.22.0/
│   ├── include/
│   └── lib/

✅ 4. Build Command

🔷 macOS
g++ -std=c++17 main.cpp \
  -I./onnxruntime-osx-universal2-1.22.0/include \
  -L./onnxruntime-osx-universal2-1.22.0/lib \
  -lonnxruntime \
  -o onnx_test
🐧 Linux

Replace the onnxruntime-osx-... path with onnxruntime-linux-x64-....

✅ 5. Run the Executable

🔷 macOS

Important: You must set the library path.

export DYLD_LIBRARY_PATH=./onnxruntime-osx-universal2-1.22.0/lib:$DYLD_LIBRARY_PATH
./onnx_test
🐧 Linux
export LD_LIBRARY_PATH=./onnxruntime-linux-x64-1.22.0/lib:$LD_LIBRARY_PATH
./onnx_test

✅ 6. C++ Code Example

#include <onnxruntime_cxx_api.h>
#include <fstream>
#include <nlohmann/json.hpp>
#include <iostream>
#include <iomanip>
#include <algorithm>
#include <string>
#include <vector>

using json = nlohmann::json;

std::vector<int32_t> preprocess_text(const std::string& text, const std::string& tokenizer_file) {
    std::vector<int32_t> vector(30, 0);
    
    std::ifstream tf(tokenizer_file);
    json tokenizer; tf >> tokenizer;
    
    std::string text_lower = text;
    std::transform(text_lower.begin(), text_lower.end(), text_lower.begin(), ::tolower);
    std::vector<std::string> words;
    size_t start = 0, end;
    while ((end = text_lower.find(' ', start)) != std::string::npos) {
        if (end > start) words.push_back(text_lower.substr(start, end - start));
        start = end + 1;
    }
    if (start < text_lower.length()) words.push_back(text_lower.substr(start));
    
    for (size_t i = 0; i < std::min(words.size(), size_t(30)); i++) {
        auto it = tokenizer.find(words[i]);
        if (it != tokenizer.end()) {
            vector[i] = it->get();
        } else {
            auto oov = tokenizer.find("");
            vector[i] = oov != tokenizer.end() ? oov->get() : 1;
        }
    }
    return vector;
}

int main() {
    std::string text = "I hate you";
    auto vector = preprocess_text(text, "hate_speech(English)/vocab.json");
    
    Ort::Env env(ORT_LOGGING_LEVEL_WARNING, "test");
    Ort::SessionOptions session_options;
    Ort::Session session(env, "hate_speech(English)/model.onnx", session_options);
    
    std::vector<int64_t> input_shape = {1, 30};
    Ort::MemoryInfo memory_info("Cpu", OrtDeviceAllocator, 0, OrtMemTypeDefault);
    Ort::Value input_tensor = Ort::Value::CreateTensor(memory_info, vector.data(), vector.size(), 
                                                     input_shape.data(), input_shape.size());
    
    std::vector<const char*> input_names = {"input"};
    std::vector<const char*> output_names = {"sequential"};
    auto output_tensors = session.Run(Ort::RunOptions{nullptr}, input_names.data(), &input_tensor, 1, 
                                    output_names.data(), 1);
    
    float* output_data = output_tensors[0].GetTensorMutableData();
    size_t output_size = output_tensors[0].GetTensorTypeAndShapeInfo().GetElementCount();
    
    std::ifstream lf("hate_speech(English)/scaler.json");
    json label_map; lf >> label_map;
    
    auto max_it = std::max_element(output_data, output_data + output_size);
    int predicted_idx = std::distance(output_data, max_it);
    std::string label = label_map[std::to_string(predicted_idx)];
    float score = *max_it;
    
    std::cout << "C++ ONNX output: " << label << " (Score: " << std::fixed << std::setprecision(4) 
              << score << ")" << std::endl;
    return 0;
}

C++ ONNX output: Hate (Score: 0.9876)

✅ 7. Expected Output

C++ ONNX output: Hate (Score: 0.9876)

🧠 How to Run an ONNX Model with Rust: Full Beginner-Friendly Guide

📌 What is this?

This guide walks you through running an ONNX model for text classification using Rust, starting from scratch — including Rust installation, setting up dependencies, and running the model.

Choose your path:

✅ 1. Install Rust

🔷 Windows
  1. Download and run rustup-init.exe from: https://rustup.rs/
  2. Follow the installation prompts
  3. After installation, open a new terminal and verify:
rustc --version
cargo --version
🍏 macOS

Open Terminal and run:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

After installation, verify:

rustc --version
cargo --version
🐧 Linux

Open Terminal and run:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

After installation, verify:

rustc --version
cargo --version

✅ 2. Get Your Model

🔄 Download Repository

Clone our repository to get started:

git clone https://github.com/whitelightning-ai/whitelightning.git
cd whitelightning.ai
📦 Choose Your Model

You have two options:

  1. Use Pre-trained Model:
    • Navigate to the models directory
    • Copy the multiclass classifier model files to your project:
      • model.onnx - The ONNX model file
      • model_vocab.json - Tokenizer vocabulary
      • model_labels.json - Class labels mapping
  2. Train Your Own Model:
    • Follow the training guide in the repository
    • Use the provided scripts to train a custom multiclass classifier
    • Export your model to ONNX format

✅ 3. Create a New Rust Project

cargo new onnx_rust_demo
cd onnx_rust_demo

✅ 4. Add Dependencies

Edit Cargo.toml to add required dependencies:

[package]
name = "onnx_rust_demo"
version = "0.1.0"
edition = "2021"

[dependencies]
ort = "1.16.0"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
anyhow = "1.0"
thiserror = "1.0"

✅ 5. Project Structure

onnx_rust_demo/
├── src/
│   └── main.rs
├── model.onnx
├── vocab.json
├── scaler.json
└── Cargo.toml

✅ 6. Create Supporting Files

🔹 vocab.json (tokenizer dictionary)
{
  "": 1,
  "the": 2,
  "government": 3,
  "announced": 4,
  "new": 5,
  "policies": 6,
  "to": 7,
  "boost": 8,
  "economy": 9
}
🔹 scaler.json (label map)
{
  "0": "Politics",
  "1": "Sports",
  "2": "Technology"
}

✅ 7. Create the Rust Code (src/main.rs)

use anyhow::Result;
use ort::{Environment, Session, SessionBuilder, Value};
use serde_json::Value as JsonValue;
use std::fs::File;
use std::io::Read;
use std::collections::HashMap;

struct Tokenizer {
    vocab: HashMap<String, i32>,
}

impl Tokenizer {
    fn new(vocab_file: &str) -> Result<Self> {
        let mut file = File::open(vocab_file)?;
        let mut contents = String::new();
        file.read_to_string(&mut contents)?;
        let vocab: HashMap<String, i32> = serde_json::from_str(&contents)?;
        Ok(Tokenizer { vocab })
    }

    fn tokenize(&self, text: &str) -> Vec<i32> {
        let words: Vec<&str> = text.to_lowercase().split_whitespace().collect();
        let mut tokens = Vec::with_capacity(30);
        
        for word in words.iter().take(30) {
            let token = self.vocab.get(*word)
                .or_else(|| self.vocab.get(""))
                .copied()
                .unwrap_or(1);
            tokens.push(token);
        }

        while tokens.len() < 30 {
            tokens.push(0);
        }

        tokens
    }
}

fn load_label_map(file_path: &str) -> Result<HashMap<String, String>> {
    let mut file = File::open(file_path)?;
    let mut contents = String::new();
    file.read_to_string(&mut contents)?;
    let label_map: HashMap<String, String> = serde_json::from_str(&contents)?;
    Ok(label_map)
}

fn main() -> Result<()> {
    // Initialize tokenizer and load label map
    let tokenizer = Tokenizer::new("vocab.json")?;
    let label_map = load_label_map("scaler.json")?;

    // Create ONNX Runtime environment and session
    let environment = Environment::builder()
        .with_name("onnx-rust-demo")
        .build()?;
    
    let session = SessionBuilder::new(&environment)?
        .with_model_from_file("model.onnx")?;

    // Prepare input text
    let text = "The government announced new policies to boost the economy";
    let tokens = tokenizer.tokenize(text);

    // Create input tensor
    let input_tensor = Value::from_array(([1, 30], tokens))?;

    // Run inference
    let outputs = session.run(vec![input_tensor])?;
    let output: &Value = outputs[0].downcast_ref().unwrap();
    let probabilities = output.as_slice::<f32>()?;

    // Find predicted class
    let (predicted_idx, &score) = probabilities.iter().enumerate()
        .max_by(|a, b| a.1.partial_cmp(b.1).unwrap())
        .unwrap();

    let label = label_map.get(&predicted_idx.to_string())
        .ok_or_else(|| anyhow::anyhow!("Label not found"))?;

    println!("Rust ONNX output: {} (Score: {:.4})", label, score);
    Ok(())
}

Rust ONNX output: Politics (Score: 0.9123)

✅ 8. Build and Run

cargo build --release
cargo run --release

✅ 9. Expected Output

Rust ONNX output: Politics (Score: 0.9123)

✅ 10. Troubleshooting

Common Issues and Solutions:
  • Error: "Could not find ONNX Runtime"

    Solution: Make sure you have the correct version of ONNX Runtime installed and the library path is set correctly.

  • Error: "Failed to load model"

    Solution: Verify that the model file path is correct and the file exists.

  • Error: "Invalid input shape"

    Solution: Ensure your input tensor has the correct shape [1, 30] and contains valid token IDs.

🧠 How to Run an ONNX Model with Java using ONNX Runtime and org.json

📌 What is this?

This guide explains how to load and run ONNX models using Java, ONNX Runtime Java API, and org.json for JSON parsing.

Choose your path:

✅ 1. Prerequisites

🔷 Java Development Kit (JDK)

Install JDK 17 or later:

🐧 Linux

✅ Installation via package manager (Ubuntu/Debian):

sudo apt update
sudo apt install openjdk-17-jdk -y

📦 Download from Oracle website:

  • .tar.gz archive: jdk-17.0.15_linux-x64_bin.tar.gz
  • .deb package: jdk-17.0.15_linux-x64_bin.deb
  • .rpm package: jdk-17.0.15_linux-x64_bin.rpm

🔗 Alternative sources:

  • Adoptium (Temurin)
  • OpenLogic
  • Liberica JDK
🪟 Windows

📥 Download from Oracle:

  • .exe installer: jdk-17.0.15_windows-x64_bin.exe
  • .msi installer: jdk-17.0.15_windows-x64_bin.msi
  • .zip archive: jdk-17.0.15_windows-x64_bin.zip

🔗 Alternative sources:

  • Adoptium (Temurin)
  • Microsoft Build of OpenJDK
🍏 macOS

📥 Download from Oracle:

For Intel (x64):

  • .dmg installer: jdk-17.0.15_macos-x64_bin.dmg
  • .tar.gz archive: jdk-17.0.15_macos-x64_bin.tar.gz

For Apple Silicon (ARM64):

  • .dmg installer: jdk-17.0.15_macos-aarch64_bin.dmg
  • .tar.gz archive: jdk-17.0.15_macos-aarch64_bin.tar.gz

🔗 Alternative sources:

  • Adoptium (Temurin)
  • Liberica JDK

Verify installation:

java -version
javac -version
🟩 Maven

Install Maven for dependency management:

  1. Download from: https://maven.apache.org/download.cgi
  2. Verify installation:
mvn -version

✅ 2. Choose Your Model

🔄 Download Repository

Clone our repository to get started:

git clone https://github.com/whitelightning-ai/whitelightning.git
cd whitelightning.ai
📦 Choose Your Model

You have two options:

  1. Use Pre-trained Model:
    • Navigate to the models directory
    • Copy the multiclass classifier model files to your project:
      • model.onnx - The ONNX model file
      • model_vocab.json - Tokenizer vocabulary
      • model_labels.json - Class labels mapping
  2. Train Your Own Model:
    • Follow the training guide in the repository
    • Use the provided scripts to train a custom multiclass classifier
    • Export your model to ONNX format

✅ 3. Create a New Maven Project

mvn archetype:generate -DgroupId=com.example -DartifactId=onnx-java-demo -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false
cd onnx-java-demo

✅ 4. Add Dependencies

Edit pom.xml to add required dependencies:

<dependencies>
    <dependency>
        <groupId>com.microsoft.onnxruntime</groupId>
        <artifactId>onnxruntime</artifactId>
        <version>1.16.3</version>
    </dependency>
    <dependency>
        <groupId>org.json</groupId>
        <artifactId>json</artifactId>
        <version>20231013</version>
    </dependency>
</dependencies>

✅ 5. Project Structure

onnx-java-demo/
├── src/
│   └── main/
│       ├── java/
│       │   └── com/
│       │       └── example/
│       │           └── ONNXModelRunner.java
│       └── resources/
│           ├── model.onnx
│           ├── vocab.json
│           └── scaler.json
└── pom.xml

✅ 6. Java Code Example

import ai.onnxruntime.*;
import org.json.JSONObject;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.*;

public class ONNXModelRunner {
    public static void main(String[] args) {
        try {
            LabelVocabLoader loader = new LabelVocabLoader("resources/labelMap.json", "resources/vocab.json");
            Map<Integer, String> labelMap = loader.getLabelMap();
            Map<String, Integer> vocab = loader.getVocab();

            String modelPath = "resources/model.onnx";
            OrtEnvironment env = OrtEnvironment.getEnvironment();
            OrtSession session = env.createSession(modelPath, new OrtSession.SessionOptions());

            String inputText = "The government announced new policies to boost the economy";

            Tokenizer tokenizer = new Tokenizer(vocab);
            int maxLen = 30;
            int[] tokenizedInput = tokenizer.tokenize(inputText);
            int[] paddedInput = new int[maxLen];
            for (int i = 0; i < maxLen; i++) {
                if (i < tokenizedInput.length) {
                    paddedInput[i] = tokenizedInput[i];
                } else {
                    paddedInput[i] = 0;
                }
            }

            int[][] inputData = new int[1][maxLen];
            inputData[0] = paddedInput;

            OnnxTensor inputTensor = OnnxTensor.createTensor(env, inputData);

            String inputName = session.getInputNames().iterator().next();
            OrtSession.Result result = session.run(Collections.singletonMap(inputName, inputTensor));

            float[][] outputArray = (float[][]) result.get(0).getValue();
            int maxIndex = 0;
            float maxScore = outputArray[0][0];
            for (int i = 1; i < outputArray[0].length; i++) {
                if (outputArray[0][i] > maxScore) {
                    maxScore = outputArray[0][i];
                    maxIndex = i;
                }
            }

            System.out.println("Java ONNX output: " + labelMap.get(maxIndex) + 
                             " (Score: " + String.format("%.4f", maxScore) + ")");

            session.close();
            env.close();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    static class Tokenizer {
        private Map<String, Integer> vocab;

        public Tokenizer(Map<String, Integer> vocab) {
            this.vocab = vocab;
        }

        public int[] tokenize(String text) {
            String[] words = text.toLowerCase().split("\\s+");
            int[] tokenized = new int[words.length];
            for (int i = 0; i < words.length; i++) {
                Integer token = vocab.getOrDefault(words[i], vocab.get(""));
                tokenized[i] = token;
            }
            return tokenized;
        }
    }

    static class LabelVocabLoader {
        private Map<Integer, String> labelMap;
        private Map<String, Integer> vocab;

        public LabelVocabLoader(String labelMapPath, String vocabPath) throws Exception {
            String labelMapJson = new String(Files.readAllBytes(Paths.get(labelMapPath)));
            JSONObject labelMapObject = new JSONObject(labelMapJson);
            this.labelMap = new HashMap<>();
            for (String key : labelMapObject.keySet()) {
                this.labelMap.put(Integer.parseInt(key), labelMapObject.getString(key));
            }

            String vocabJson = new String(Files.readAllBytes(Paths.get(vocabPath)));
            JSONObject vocabObject = new JSONObject(vocabJson);
            this.vocab = new HashMap<>();
            for (String key : vocabObject.keySet()) {
                this.vocab.put(key, vocabObject.getInt(key));
            }
        }

        public Map<Integer, String> getLabelMap() {
            return labelMap;
        }

        public Map<String, Integer> getVocab() {
            return vocab;
        }
    }
}

✅ 7. Build and Run

mvn clean package
java -cp target/onnx-java-demo-1.0-SNAPSHOT.jar com.example.ONNXModelRunner

✅ 8. Expected Output

Java ONNX output: Politics (Score: 0.9123)

🧠 How to Run an ONNX Model with Dart/Flutter: Complete Mobile App Guide

📌 What is this?

This guide shows you how to create a Flutter mobile app that runs ONNX multiclass text classification models directly on Android and iOS devices.

Choose your path:

✅ 1. Install Flutter and Dart

🔷 Windows
  1. Download Flutter SDK from: https://docs.flutter.dev/get-started/install/windows
  2. Extract to C:\flutter
  3. Add C:\flutter\bin to your PATH
  4. Verify installation:
flutter --version
dart --version
🍏 macOS

Install via Homebrew (recommended):

brew install --cask flutter

Or download manually:

  1. Download from: https://docs.flutter.dev/get-started/install/macos
  2. Extract and add to PATH:
export PATH="$PATH:`pwd`/flutter/bin"

Verify installation:

flutter --version
dart --version
🐧 Linux

Install Flutter:

  1. Download from: https://docs.flutter.dev/get-started/install/linux
  2. Extract and add to PATH:
tar xf flutter_linux_*-stable.tar.xz
export PATH="$PATH:`pwd`/flutter/bin"

Verify installation:

flutter --version
dart --version

✅ 2. Get Your Model

🔄 Download Repository

Clone our repository to get started:

git clone https://github.com/whitelightning-ai/whitelightning.git
cd whitelightning.ai
📦 Choose Your Model

Navigate to the multiclass models directory and choose your model:

  • hate_speech(English)/ - English hate speech detection
  • news_classifier(English)/ - English news classification
  • news_classifier(Spanish)/ - Spanish news classification
  • And many more language variants...

Each directory contains:

  • model.onnx - The ONNX model file
  • vocab.json - Tokenizer vocabulary
  • scaler.json - Class labels mapping

✅ 3. Create Flutter Project

flutter create multiclass_classifier_flutter
cd multiclass_classifier_flutter

✅ 4. Add Dependencies

Edit pubspec.yaml:

name: flutter_application_1
description: "A new Flutter project."
publish_to: 'none'
version: 1.0.0+1

environment:
  sdk: ^3.8.0

dependencies:
  flutter:
    sdk: flutter
  onnxruntime: ^1.4.1
  cupertino_icons: ^1.0.8

dev_dependencies:
  flutter_test:
    sdk: flutter
  flutter_lints: ^5.0.0

flutter:
  assets:
    - assets/models/

Install dependencies:

flutter pub get

✅ 5. Project Structure

multiclass_classifier_flutter/
├── lib/
│   └── main.dart
├── assets/
│   └── models/
│       ├── model.onnx
│       ├── vocab.json
│       └── scaler.json
└── pubspec.yaml

✅ 6. Flutter App Code (lib/main.dart)

import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'dart:convert';
import 'package:onnxruntime/onnxruntime.dart';
import 'dart:typed_data';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Multiclass Text Classifier',
      theme: ThemeData(
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
      ),
      home: const MyHomePage(title: 'Text Classifier'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({super.key, required this.title});
  final String title;

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _counter = 0;
  String _prediction = '';
  String _inputText = '';
  bool _isLoading = false;

  void _incrementCounter() {
    setState(() {
      _counter++;
    });
  }

  Future<void> _classifyText() async {
    setState(() {
      _isLoading = true;
      _prediction = '';
    });
    print('Starting classification...');
    final prediction = await runTextClassifierWithResult(_inputText);
    print('Prediction result: ' + prediction);
    setState(() {
      _prediction = prediction;
      _isLoading = false;
    });
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        backgroundColor: Theme.of(context).colorScheme.inversePrimary,
        title: Text(widget.title),
      ),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            Padding(
              padding: const EdgeInsets.symmetric(horizontal: 24.0),
              child: TextField(
                decoration: InputDecoration(
                  labelText: 'Enter text to classify',
                ),
                onChanged: (value) {
                  setState(() {
                    _inputText = value;
                  });
                },
              ),
            ),
            const SizedBox(height: 16),
            ElevatedButton(
              onPressed: _isLoading || _inputText.isEmpty
                  ? null
                  : _classifyText,
              child: _isLoading
                  ? CircularProgressIndicator()
                  : Text('Classify'),
            ),
            const SizedBox(height: 24),
            Text(
              _prediction.isEmpty
                  ? 'Prediction will appear here.'
                  : 'Prediction: $_prediction',
              style: TextStyle(fontSize: 18, fontWeight: FontWeight.bold),
            ),
            const SizedBox(height: 40),
            const Text('I love you this much:'),
            Text(
              '$_counter',
              style: Theme.of(context).textTheme.headlineMedium,
            ),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: _incrementCounter,
        tooltip: 'Increment',
        child: const Icon(Icons.add),
      ),
    );
  }
}

Future<String> runTextClassifierWithResult(String text) async {
  print('Initializing ONNX...');
  OrtEnv.instance.init();
  final sessionOptions = OrtSessionOptions();

  print('Loading model...');
  final rawModel = await rootBundle.load('assets/models/model.onnx');
  final session = OrtSession.fromBuffer(
    rawModel.buffer.asUint8List(),
    sessionOptions,
  );

  print('Loading vocab...');
  final vocabJson = await rootBundle.loadString('assets/models/vocab.json');
  final vocab = json.decode(vocabJson) as Map<String, dynamic>;

  print('Loading scaler...');
  final scalerJson = await rootBundle.loadString('assets/models/scaler.json');
  final scaler = json.decode(scalerJson) as Map<String, dynamic>;

  print('Tokenizing...');
  final words = text.toLowerCase().split(' ');
  final sequence = List<int>.filled(30, 0);
  for (int i = 0; i < words.length && i < 30; i++) {
    sequence[i] = vocab[words[i]] ?? vocab['<OOV>'] ?? 1;
  }

  print('Running inference...');
  final inputTensor = OrtValueTensor.createTensorWithDataList(
    Int32List.fromList(sequence),
    [1, 30],
  );
  final result = await session.runAsync(OrtRunOptions(), {
    'input': inputTensor,
  });

  String resultString = 'No prediction';
  final resultList = result?.toList();
  if (resultList != null && resultList.isNotEmpty && resultList[0] != null) {
    final outputTensor = resultList[0] as OrtValueTensor;
    final List<dynamic> probabilities = outputTensor.value as List<dynamic>;
    print('Probabilities: ' + probabilities.toString());
    // Flatten if needed
    final List<dynamic> flatProbs =
        (probabilities.isNotEmpty && probabilities.first is List)
        ? probabilities.first as List<dynamic>
        : probabilities;
    if (flatProbs.isNotEmpty) {
      int maxIndex = 0;
      for (int i = 1; i < flatProbs.length; i++) {
        if (flatProbs[i] > flatProbs[maxIndex]) {
          maxIndex = i;
        }
      }
      final label = scaler[maxIndex.toString()];
      final score = flatProbs[maxIndex];
      resultString = '$label (Score: ${score.toStringAsFixed(4)})';
    }
  }
  inputTensor.release();
  OrtEnv.instance.release();
  print('Returning result: ' + resultString);
  return resultString;
}

✅ 7. Add Model Files

Copy your model files to the assets directory:

# Copy from your chosen model directory
cp models/news_classifier\(English\)/model.onnx assets/models/
cp models/news_classifier\(English\)/vocab.json assets/models/
cp models/news_classifier\(English\)/scaler.json assets/models/

✅ 8. Run the App

📱 On Device/Emulator
flutter run
🌐 On Web (for testing)
flutter run -d web

✅ 9. Expected Output

📊 RESULTS:
🏆 Predicted Category: Politics
📈 Confidence: 94.56% (0.9456)
    
(Real-time classification in mobile app)

This Flutter app provides a beautiful, responsive mobile interface for multiclass text classification. Users can input text and get instant predictions with confidence scores.

✅ 10. Key Features

  • 📱 Cross-Platform: Runs on iOS, Android, and Web
  • 🎯 Real-time Classification: Instant text classification with ONNX
  • 🎨 Modern UI: Beautiful Material Design interface
  • ⚡ Fast Inference: Local on-device processing
  • 🔒 Privacy-First: No data sent to servers
  • 📊 Multiple Categories: Supports any multiclass ONNX model