Is there a built-in mechanic in D&D5e for when the players choose to disable or neutralize a threat as opposed to kill outright?

You can knock a creature out with a melee attack

Knocking a Creature Out

Sometimes an attacker wants to incapacitate a foe, rather than deal a
killing blow. When an attacker reduces a creature to 0 hit points with
a melee attack, the attacker can knock the creature out. The attacker
can make this choice the instant the damage is dealt. The creature
falls unconscious and is stable. (basic rules)

Note that this only works with melee attacks. So ranged attacks will still kill. Though it is good to note that melee spell attacks do still count under the rule.

This is the only rule that provides a mechanic for knocking out an opponent with an attack instead of doing lethal damage.

1. Deal nonlethal damage, as Rubiksmoose pointed out.



2. 
   

   Disarm

   
   
   

   
   

   A creature can use a weapon attack to knock a weapon or another item from a target's grasp. The attacker makes an attack roll contested by the target's Strength (Athletics) check or Dexterity (Acrobatics) check. If the attacker wins the contest, the attack causes no damage or other ill effect, but the defender drops the item.

   
   
   

   The attacker has disadvantage on its attack roll if the target is holding the item with two or more hands. The target has advantage on its ability check if it is larger than the attacking creature, or disadvantage if it is smaller.

   
   

   
   


3. 
   

   Grapple

   
   
   

   
   

   A grappled creature's speed becomes 0, and it can't benefit from any bonus to its speed.

   
   

   
   


4. 
   

   Use the Grapple feat to restrain the target (thanks Carcer). Please note that this actually imposes the restrained condition, not the grappled condition.

   
   
   

   
   

   You can use your action to try to pin a creature grappled by you. To do so, make another grapple check. If you succeed, you and the creature are both restrained until the grapple ends.

   
   

   
   


5. 
   

   Although it doesn't disable the threat, you can also knock the target prone.

   
   
   

   
   

   A prone creature’s only Movement option is to crawl, unless it stands up and thereby ends the condition.
   The creature has disadvantage on Attack rolls.
   An Attack roll against the creature has advantage if the attacker is within 5 feet of the creature. Otherwise, the Attack roll has disadvantage.

   
   

   
   

Demand Surrender

It's not a hard and fast rule, like grappling or knocking a creature out, but you can always demand an enemy in a loosing situation surrender in exchange for their life. If your are handily wiping the floor with some bandits, they're likely to accept, barring some sort of code of honor.

Besides, there's practically no cost to trying; in the worst case, they keep fighting a loosing battle, and you use the more direct "knock them out" option. Though really, why risk a continued fight, when you could end it here and now with a polite request.

Listen up, kitty cat. Unless you lay down your weapons in the next 5 seconds, Dellan here is going to blast you with fire so hot, you won't be able to tell yourself from your campfire. Drop that sword, and we can settle this more cool headedly.

Disclaimer

Unlike the other answer, whether this attempt works outright, fails outright, or requires an ability check is up to the DM. Since speaking is a free action, it's likely a demand for surrender wouldn't take an action, but there's no specific rule, so again it's up to the DM.