HomeTechniques and TipsNeuralToolsInterpreting Incorrect% in Testing Report

# 15.24. Interpreting Incorrect% in Testing Report

My question is about the Detailed Report that NeuralTools produces when testing category predictions in a PNN. With a Good result, the Incorrect% is 100 minus the Prediction%, which makes sense to me. But with a Bad result, there doesn't seem to be a relationship between the Incorrect% and the Prediction%. What does the Incorrect% mean?

The Incorrect% is the sum of the probabilities for all the incorrect categories.

This is easier to understand if you "change Detailed Report settings to show the probabilities assigned by a Probabilistic Neural Net to every possible category for the dependent variable." Click Utilities » Application Settings. In the Reports section, click the drop-down menu at the end of the Columns in Detailed Reports row. A dialog box titled NeuralTools — Columns to Display in Detailed Reports will open. "In that dialog, select Probabilities of All Categories (for PNN) for Testing. Then train a PN Net on a data set with at least 3 categories in the dependent variable." (Source: NeuralTools 7 user manual, page 51, or topic "Training Reports" in the help file.)

Attached is part of the results from our standard example "Auto Loans 1a" (Help » Example Spreadsheets) with that application setting in effect. (Click the image to open it full size, in a new tab.)

Row 5 is probably the clearest: The correct answer was "late payments", but NeuralTools had computed there was only a 43.12% chance it was correct. NeuralTools picked "timely payments" because it thought there was a 54.69% chance that was correct. It computed a 2.19% probability for the third category, "default". The Incorrect% is 100% minus the probability of the correct answer, 100–43.12 = 56.88%. That same 56.88% is the sum of the probabilities of the incorrect answers, 54.69+2.19%. By contrast, for a Good result like row 6, Incorrect% is always 100% minus Prediction%, since the predicted answer was correct and all the incorrect answers were also not predicted.