An algorithm is a process or set of rules to be followed in calculations or other problem-solving operations, especially by a computer; Oxford says so.
In the realm of Machine Learning, we have two basic classifications for algorithms. Texts often represent the following, in a simplified fashion:
- Supervised Learning: uses labeled data to correct and optimize a model during training.
- Unsupervised Learning: uses unlabeled data, with nothing to compare predictions against.
If you explore that mechanically, the implication seems obvious:
for an algorithm to be supervised, it must be trained and optimized using known outcomes. That’s what “supervision” means. You know the correct answer, you compare the model’s guess to that answer, you measure error, and you adjust until the model improves.
That’s clean. That’s logical. That’s how supervision works everywhere else in life.
So… is that actually the case? No.
Somewhere along the way, the neat binary of supervised vs. unsupervised stopped being as crisp as the definitions suggest. What should have been a simple True/False distinction drifted into a fuzzy “maybe,” “it depends,” and “kind of, sort of.”
And that’s where things get interesting. Let’s dive into why the terminology doesn’t always match the mechanics, and how we ended up with algorithms that are supervised… except not really.
If the world followed the neat definitions, the boundary would be obvious:
- Supervised: You know the correct answers, your data is labeled, and you leverage them to supervise and train the model.
- Unsupervised: You don’t have the correct answers, your data is not labeled.
Simple. Clean. Binary. Supervised has a better ring to it. It sounds more rigorous, methodical, and of course, accurate.
Then models started being applied in new, inventive, and one could argue, questionable ways:
- Using labeled data from one task to “evaluate” a model trained for a completely different task.
- Training a model with no true labels, then calling it “supervised” simply because a supervised model appeared somewhere upstream.
- Generating labels automatically from the data itself, then presenting the process as supervised learning.
- Evaluating an unsupervised model using a labeled dataset that has nothing to do with the model’s actual objective.
- Using labeled predictors and an output variable, but the output variable represents something entirely different, related, perhaps, but not the true target.
Each of these practices bends the original logic of supervision, blurring the line between what is genuinely supervised and what is merely borrowing the appearance of supervision. Is borrowing the appearance an accurate description?
If the process does not adhere to the complete methodological rigors of supervised learning, it should not be classified as such in any quantifiable sense.
The nomenclature has begun to blossom in more recent times, we now have:
- Semi-Supervised Learning: Some labeled data with plenty of unlabeled data.
- Self-Supervised Learning: The model generates its own labels from the data structure.
- Weakly Supervised Learning: Labels exist, but in some state of disarray.
- Transfer Learning: Supervised on one task, reused for another.
- Representation learning: Supervised somewhere, somehow, at some time.
In the tech world, we already drown in acronyms and elaborate terminology, yet in this case, I actually welcome the added complexity.
It’s far easier to treat these as distinct categories than to pretend that something like Transfer Learning cleanly fits under the banner of “Supervised Learning.” Logically, if we do not know the true outcome for the task at hand, we cannot supervise the model’s training. In that sense, it resembles Unsupervised Learning far more closely: we may have labeled data somewhere in the pipeline, but without known outputs for the actual task, we cannot satisfy the full criteria required to call the process supervised.
Now, I think this would be much more proactive and acceptable if we were to iron out a few other terms to make up for the added load.
Take the Output Variable. It is the Output of your model. One or more Input Variables flow into your model, which produces the resultant Output Variable.
Output Variable, aka:
- Criterion Variable
- Dependent Variable
- Endogenous Variable
- Endpoint
- Explain Variable
- Explained Variable
- KPI
- Measured Effect
- Metric
- Objective Function Output
- Outcome
- Outcome Variable
- Output Signal
- Performance Measure
- Result Variable
- Return Value
- Score Variable
- System Output
- Target Metric
- Target Variable
- Y Variable
Despite the variety of names, each term describes the same mechanical role: the variable produced by the model, the object you’re ultimately trying to predict or evaluate.
If we eliminated half of these terms, it would more than compensate for the five new categories of algorithmic learning we’ve invented to patch the supervised/unsupervised divide.
Somewhat tongue in cheek, yet factually evident….
Additionally, maybe we should stop changing definitions to fit alternative ideas and methods.
Missouri dBA 