Fix skl2onnx Accuracy Drop in RandomForestClassifier

In a machine learning engineering forum I participate in, a developer recently reported a baffling issue: their RandomForestClassifier had 98% accuracy in Scikit-Learn, but after converting it to ONNX using skl2onnx, the accuracy plummeted to 57%.

The culprit was identified as a “list unpacking shape mismatch” during the inference phase, where the ONNX Runtime was receiving data in a format that caused the forest to interpret features as separate samples or vice versa.

The Original Question

“I converted my RandomForestClassifier to ONNX using to_onnx. In Python, clf.predict(X_test) works perfectly. In ONNX Runtime, my predictions are garbage. I noticed that if I pass a list of inputs, the internal zipmap or the tensor shape seems to shift, leading to a 41% drop in accuracy. How do I force the ONNX model to respect the (N, Features) shape without the list unpacking error?”

---

The Immediate Fix

The most common cause for this specific accuracy drop is a mismatch between the initial_types defined during conversion and the shape of the NumPy array passed to the InferenceSession. Additionally, the default ZipMap operator in skl2onnx can produce a list of dictionaries that many developers unpack incorrectly.

Solution A: Disable ZipMap and Enforce Explicit Shapes

This is the cleanest approach for modern production environments. It forces the output to be a standard NumPy tensor rather than a list of dictionaries.

# Illustrative example — verify in your environment (Python 3.10+, skl2onnx 1.15+)
import numpy as np
from skl2onnx import to_onnx
from onnxruntime import InferenceSession

# 1. Convert with ZipMap disabled
# This ensures the output is a consistent [Batch, Classes] tensor
options = {type(model): {'zipmap': False}}
onnx_model = to_onnx(
    model,
    X_train[:1].astype(np.float32),
    options=options
)

with open("model.onnx", "wb") as f:
    f.write(onnx_model.SerializeToString())

# 2. Strict Inference Loading
sess = InferenceSession("model.onnx", providers=["CPUExecutionProvider"])
input_name = sess.get_inputs()[0].name

# Ensure X_test is 2D: (Samples, Features)
# A common mistake is passing a 1D array for a single prediction
X_test_fixed = X_test.astype(np.float32)
if len(X_test_fixed.shape) == 1:
    X_test_fixed = X_test_fixed.reshape(1, -1)

label, probabilities = sess.run(None, {input_name: X_test_fixed})

---

Detailed Explanation: Why the Accuracy Drops

The “41% accuracy drop” is rarely about the weights of the trees being wrong; it is almost always a data alignment issue. There are three layers where this “list unpacking” failure happens:

1. The Flattening Trap: Scikit-learn is often “too helpful.” If you pass a 1D array to predict(), it might warn you but still work. ONNX is strict. If the model expects [None, 10] (a batch of samples with 10 features) and you provide a 1D list of 10 elements, ONNX may interpret this as 10 samples with 1 feature each if the dimensions aren’t explicitly handled.
2. ZipMap Overhead: By default, skl2onnx adds a ZipMap node at the end of the graph. This converts the probability tensor into a “List of Maps” (e.g., [{0: 0.1, 1: 0.9}, ...]). If your post-processing code expects a NumPy array and tries to slice it like probs[:, 1], it will fail or return unintended data, making it appear as though the model is inaccurate.
3. Type Silently Casting: If your X_train was float64 but you converted the model using a float32 initial type, the precision loss—combined with shape coercion—can lead to the model making decisions at the wrong split points in the trees.

---

Solution B: Using update_initial_types for Dynamic Batching

If you cannot change how the data is fed into the model (e.g., it comes from a legacy pipeline as a list), you must define the FloatTensorType explicitly to handle the variable dimension.

# Illustrative example — verify in your environment (Python 3.11, skl2onnx 1.16)
from bignum import FloatTensorType # logic helper
from skl2onnx.common.data_types import FloatTensorType

# Explicitly define the input shape
# [None, X.shape[1]] allows for a variable number of rows (batch size)
initial_type = [('float_input', FloatTensorType([None, X_train.shape[1]]))]

onnx_model = to_onnx(
    model,
    initial_types=initial_type,
    target_opset=17 # Use a recent opset for better compatibility
)

# When running inference, explicitly cast to the name defined above
results = sess.run(None, {'float_input': X_test.astype(np.float32)})

---

Edge Cases and Troubleshooting

• Categorical Features: If your Random Forest was trained on a DataFrame with category types or object strings, to_onnx requires a StringTensorType. If you unpack these into a float tensor, accuracy will drop to random chance levels.
• The “Double” Precision Issue: If your model relies on very fine-grained thresholds, float32 might not be enough. Use DoubleTensorType during conversion, though note that not all ONNX runtimes (like some mobile versions) support float64 efficiently.

---

Related Follow-up Questions

1. Is zipmap=False always recommended? Yes, for almost all new projects. ZipMap was designed to make ONNX outputs look more like Scikit-learn’s predict_proba (which returns classes), but in practice, it adds unnecessary computation overhead and makes the output format harder to work with in high-performance C++ or Rust environments.

2. Does this affect RandomForestRegressor too? Regressors don’t use ZipMap because they don’t return class probabilities. However, the shape mismatch issue remains. If you provide a 1D array to a regressor, it may return a single value or an error, but the 2D reshape (.reshape(1, -1)) is still the standard safety measure for single-row inference.

3. Why did my accuracy drop only slightly instead of 41%? If the drop is small (1-2%), it is likely not a shape mismatch but a floating-point precision difference. Scikit-learn uses double (64-bit) by default, while ONNX often defaults to float (32-bit). You can fix this by ensuring both the conversion and the input data use np.float32.