Fix “TypeError: Unhashable Type” in Python

What Causes the Unhashable Type Error in Python?

The “unhashable type” error surfaces when:

• Attempting to use a mutable object like a list or dict as a dictionary key.
• Adding a mutable object to a set.
• Passing a mutable value to a function expecting a hashable object.

This fails because mutable objects can change value, breaking their hash value consistency.

For example:

my_dict = {}
my_dict[[1,2]] = 'value' 

# TypeError: unhashable type: 'list'

Here the list [1,2] cannot be hashed as a key.

Sets have the same constraint:

my_set = {[1,2]}

# TypeError: unhashable type: 'list'

So attempting to use mutable objects in hashes causes these errors.

Why Python Requires Hashable Keys

Python raises unhashable type errors because its hash-based data structures assume immutable keys:

• Dictionaries hash keys to lookup values quickly.
• Hashing requires that keys have a consistent hash value.
• Mutable objects can change, so their hashes are not reliable.

For example, in this flawed dictionary:

So attempting to use mutable objects in hashes causes these errors.

Why Python Requires Hashable Keys
Python raises unhashable type errors because its hash-based data structures assume immutable keys:

Dictionaries hash keys to lookup values quickly.
Hashing requires that keys have a consistent hash value.
Mutable objects can change, so their hashes are not reliable.
For example, in this flawed dictionary:

Modifying the list key breaks the dictionary lookup since its hash changed.

So Python disallows mutable types as keys to prevent this class of bugs.

How to Fix Unhashable Type Errors

Fixing unhashable type errors involves:

1. Making keys immutable
2. Properly handling mutable keys

Let’s explore easy solutions for both cases:

Fixing By Making Keys Immutable

Convert mutable keys like lists to immutable:

# Bad
dict[[1,2]] = 'value'

# Good 
key = tuple([1,2])
dict[key] = 'value'

So focus on using immutable keys to avoid the issue entirely.

Properly Handling Mutable Keys

If you must use a mutable key, tell Python explicitly to hash it:

So focus on using immutable keys to avoid the issue entirely.

Properly Handling Mutable Keys
If you must use a mutable key, tell Python explicitly to hash it:

This forces Python to hash the key. But you must ensure the key does not then change.

Another option is wrapping in a tuple:

dict[(mutable_key,)] = 'value'

This creates an immutable tuple with the contents of the mutable key.

So in summary:

1. Favor immutable keys like tuples or frozensets.
2. If you must use a mutable key, hash it or wrap in a tuple first.

Following these simple fixes will avoid unhashable type errors.

Comprehensive Dictionary Example

Let’s walk through a full dictionary example demonstrating solutions:

from functools import hash

# Define mutable keys
key1 = [1,2]  
key2 = {'a': 1}

# OPTION 1: Use immutable keys
immutable_key1 = tuple(key1)
immutable_key2 = frozenset(key2)  

dict[immutable_key1] = 'value1'
dict[immutable_key2] = 'value2'

# OPTION 2: Hash the mutable keys
dict[hash(key1)] = 'value3'
dict[hash(key2)] = 'value4'

# OPTION 3: Wrap in tuple
dict[(key1,)] = 'value5'
dict[(key2,)] = 'value6'

This shows converting to immutables, hashing, and tuple wrapping in action.

Comprehensive Set Example

Similarly, fixing unhashable errors when adding to sets:

# Define mutable values
value1 = [1,2]
value2 = {'a': 1}

# OPTION 1: Convert to immutable
immutable_value1 = tuple(value1)
immutable_value2 = frozenset(value2)

my_set = {immutable_value1, immutable_value2}

# OPTION 2: Hash the mutable values 
my_set = {hash(value1), hash(value2)}

# OPTION 3: Wrap in tuple
my_set = {(value1,), (value2,)}

The same immutability, hashing, and tuple wrapping principles apply.

Key Benefits of Resolving Unhashable Errors

Fixing unhashable type errors:

• Enables using lists, dicts, and other mutable types as keys.
• Avoids unexpected bugs if keys change values.
• Makes code more generic by allowing flexible key types.
• Results in standard dictionary and set behavior.
• Reduces time wasted debugging cryptic TypeErrors.
• Improves confidence that data structures will work predictably.

Overall, properly handling mutable keys makes code safer and more robust.

Common Pitfalls When Using Mutable Keys

Avoid these pitfalls when dealing with mutable keys:

• Simply ignoring errors and carrying on with buggy code.
• Attempting to re-define built-in hash() or __hash__() on mutable types.
• Disabling hash randomization which can cause security issues.
• Using mutable values as keys without hashing or tuple wrapping.
• Changing mutable keys after inserting into dictionaries or sets.

Proactively managing mutability is essential for correctness.

Best Practices for Using Mutable Keys

To safely leverage mutable keys:

• Strongly prefer immutable keys like strings or tuples for simplcity.
• Wrap mutable keys in hash() or tuples at dictionary insertion time.
• Document keys requiring special handling.
• Raise custom exceptions if problematic mutable keys are used unsafely.
• Use FrozenDict or other frozen wrappers to make dicts immutable.
• Validate invariants on mutable keys to ensure no changes after insertion.

These best practices prevent nasty surprises down the line.

Alternatives to Dictionaries for Mutable Keys

For frequently changing keys, dictionaries may not be most suitable. Alternatives include:

Using Lists of Tuples

Store key-value pairs in an immutable list:

mutable_dict = [[(1,2), 'value1'], [{(2,3): 'value2'}]

This avoids dictionaries entirely.

Custom Classes

For advanced use cases, implement a custom MutableKeyDict class overriding handling to permit mutable keys.

Database Tables

Store mutable keys in a database schema rather than dictionaries.

Caching Layers

Some cache implementations permit mutable keys by segregating logic.

Consider these options when mutability is core to your domain.

Common Sources of Unhashable Types in Python

Unhashable type errors frequently arise when:

• Accidentally using lists as keys instead of tuples.
• Attempting to use dictionaries as keys.
• Combining dicts and lists together into unhashable structures.
• Passing nested mutable structures to functions expecting hashable objects.
• Inserting mutable default arguments into sets or dictionaries.
• Fetching data from external sources into unhashable in-memory structures.

Being aware of these common pitfalls helps avoid them.

Origin of Python’s Unhashable Type Error

The unhashable type error arises from the TypeError class in Python’s exceptions module.

The full error hierarchy is:

• Exception (base class for all errors)
  • BaseException
    • TypeError
      • UnhashableTypeError (the unhashable type error)

So UnhashableTypeError extends TypeError pertaining specifically to failures hashing objects.

The naming encodes important semantic context about the nature of the problem for debugging.

Examples of Hashable and Unhashable Types

Built-in Python types like strings, integers, floats, bools, tuples are all hashable (immutable).

While lists, sets, dicts are unhashable (mutable).

Some types like NumPy arrays are unhashable by default but can be made hashable with flags.

Sets can become immutable (hashable) when converted to frozensets.

And Python classes can implement __hash__ and optionally freeze state to enable hashing.

So hashability centers around whether the object has fixed immutable state.

Unhashable Types and Dictionaries vs Sets

The unhashable type error arises in both Python dictionaries and sets:

# Dictionaries
my_dict = {}
my_dict[[1,2]] = 'value' # TypeError

# Sets 
my_set = {[1,2]} # TypeError

This is because both data structures rely on hashing under the hood:

• Dictionaries hash keys to map them to values.
• Sets hash elements to ensure uniqueness.

Requiring hashability allows efficient implementations.

So both dictionaries and sets give this error to maintain their performance guarantees.

Immutable Data Structures in Python

When hashability is required, prefer immutable (hashable) data structures:

• Strings: Immutable sequences of unicode characters.
• Tuples: Immutable sequences of values.
• Integers: Immutable numeric values.
• Floats: Immutable numeric values.
• Booleans: True and False boolean values.
• Frozenset: Immutable form of sets.
• Namedtuple: Immutable object tuples.
• Dates: Immutable date/time values like datetime.date.

Choosing these immutable types simplifies hashing.

Making Types Immutable in Python

To make a mutable object immutable (and therefore hashable):

• Call tuple on lists, sets, dicts.
• Use frozenset on sets.
• Call dict.copy to create a shallow immutable dictionary copy.
• Set the writeable flag to False on NumPy arrays.
• Use deepcopy to create an immutable deep copy of an object.
• Assign object attributes to __slots__ to prevent new attributes.
• Monkey-patch mutable classes to throw errors on state changes.

Immutability makes keys reliable.

Mutability Can Be Necessary

Despite issues with hashing, sometimes mutability is required:

• When objects intrinsically can change state.
• Data structures with different semantics than dicts or sets.
• Interfaces requiring accepting mutable types.
• Integrating with legacy systems built around mutable data.

In these cases, hash correctly with hash() or tuple wrap.

Do not compromise code correctness just to avoid mutability.

Sets vs Frozensets in Python

Sets are mutable while frozensets are immutable:

normal_set = {1, 2, 3}
frozen_set = frozenset({1, 2, 3}) 

normal_set.add(4) # Ok
frozen_set.add(4) # AttributeError

Methods like .add() on frozensets will error since they are immutable.

Hashing implications:

• Sets are unhashable, can’t be dict keys.
• Frozensets are hashable, can be dict keys.

Convert with frozenset(my_set) if hashability is needed.

The Hash(object) Function in Python

The built-in hash() function in Python computes an object’s hash:

hash('string') 
# 4661577028822941000

hash((1,2))
# 3713081631934410656

This allows forcibly hashing mutable objects:

mutable = [1,2]
hash(mutable) # Compute hash anyway

Hashes are randomized but stable within one execution.

Note that hash(obj) relies on obj.__hash__() being implemented.

How Dictionaries and Sets Use Hashing in Python

Dictionaries and sets are optimized for performance using hashing:

Dictionaries

• Dicts hash keys with a fast hash function to locate them.
• Keys must be immutable so their hashes don’t change.

This enables O(1) key lookups since hashes are stable.

Sets

• Sets hash elements and store them in a hash table.
• Hashing ensures uniqueness as duplicate elements have identical hashes.

This allows lightning-fast O(1) set operations like unions and intersections.

So both data structures leverage hashing internally for speed.

Python key-sharing Dictionaries

Python has dict key sharing optimization:

dict1 = {'a': 1}
dict2 = {'a': 2}

dict1 and dict2 actually share the same key object 'a'.

This works because strings are immutable so it’s safe for multiple dicts.

It saves memory by reusing shared immutable keys.

hash vs eq in Python

__hash__ and __eq__ both relate to hashing in Python:

• __hash__ returns an object’s hash value.
• __eq__ defines comparison and equality checking behavior.

These dunder methods enable custom hashable behavior on classes:

class MyClass:
  def __init__(self,val):
    self.val = val

  def __eq__(self, other):
    return self.val == other.val
  
  def __hash__(self):
    return hash(self.val)

Now instances can be put in dicts and sets.

Hash Randomization in Python

Python randomizes hash values to avoid denial of service attacks:

hash('string')
# -415270070575732742 

hash('string') 
# 332384537058561894

The hash changes across runs.

But it remains consistent within a single execution:

value = 'string'
hash(value) == hash(value) # True

Randomization makes hashes unpredictable across processes.

Hashing and Entropy in Python

Good hash functions have high entropy so outputs appear random:

import hashlib

hash_md5 = hashlib.md5(b'hello').hexdigest() 
# 5d41402abc4b2a76b9719d911017c592

hash_sha1 = hashlib.sha1(b'hello').hexdigest()
# aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d

These cryptographic hashes are unpredictable.

In contrast, simple linear hashing has poor entropy:

def bad_string_hash(s):
    value = 0
    for c in s:
        value += ord(c)
    return value

This is easy for attackers to reverse engineer hashes.

Good hash distributions are key to security and performance.

Salting Hash Functions in Python

Salts strengthen hash functions against reversal attacks:

import hashlib

password = 'password123'

# Repeatable hash 
hash_md5 = hashlib.md5(password.encode()).hexdigest()
# 5f4dcc3b5aa765d61d8327deb882cf99

# Salted hash
salted = password.encode() + b'saltysalt'
hash_md5 = hashlib.md5(salted).hexdigest()  
# e5c9da11c85c4938140b29540f336439

The added random salt value defeats precomputed lookup tables.

Salting is a best practice for user password hashing.

Setting Environment Variables in Python

Environment variables can be set to configure application and debugging behavior:

import os

os.environ['DEBUG'] = 'true'

if os.environ['DEBUG'] == 'true':
  print('In debug mode')

Now the application can check the custom env var.

Setting user environment variables:

Linux/macOS:

export DEBUG=true

Windows:

set DEBUG=true

Debugging Unhashable Type Errors

To debug unhashable type issues:

• Inspect the full error traceback for offending object types.
• Enable outputting variable types during debugging with type(var).
• Methodically check types flowing into hash operations.
• Consider implicit type changes like views or generators.
• Trace code to origin points mutating previously hashable objects.
• Leverage IDE type hinting to surface mismatches.
• Catch and log hash errors to diagnose common failure points.

Slowing down and checking types thoroughly avoids losing time to trial-and-error.

Conclusion

In conclusion, encountering a “TypeError: unhashable type” error in Python can be a frustrating experience for developers. This error typically arises when attempting to use mutable objects like lists and dictionaries as keys in dictionaries or sets. Python requires keys to be hashable, meaning they have a consistent hash value, which mutable objects lack due to their ability to change.