Python Quirks: Understand How a Variable Can Be Modified by a Function That Doesn’t Return Anything

A deep dive into how Python passes arguments and mutability to prevent unexpected bugs

Mike Huls

Apr 13, 2023 — 8 min read

Tracking down the unexpected bugs (image by cottonbro studio on Pexels)

In this article we’ll put on our detective hat and solve a “Python Mystery”. In this episode we’ll find out how a function that doesn’t return a value can alter a variable. (see below for example). This is not all: it only ‘works’ on some types of variables. Additionally it’s pretty easy to fall into the trap of this behaviour so it’s important to know what causes it.

We’ll focus on understanding the mechanics beneath the mystery. Not only will a better understanding of Python make you a better developer, it will also save you a lot of frustration trying to solve incomprehensible bugs. Let’s code!

The mystery — an example

Let’s first analyze our “Python Mystery” a bit more: suppose we have two functions that:

accept a variable
modify that variable
don’t return the variable

def change_string(input_string:str) -> None: 
    """ Notice that this functions doesn't return anything! """ 
    input_string += 'a' 
 
 
def change_list(input_list:list) -> None: 
    """ Notice that this functions doesn't return anything! """ 
    input_list.append('a')

For both functions we define a variable, print it out, call the function with the variable and then print it out again

my_str = 'hello' 
print(my_str)                        # 'hello' 
change_string(input_string=my_str) 
print(my_str)                        # 'hello' 
 
my_list = ['hello'] 
print(my_list)                       # ['hello']  
change_list(input_list=my_list) 
print(my_list)                       # ['hello', 'a'] !?

What happened? Why has the my_list variable changed while the my_str variable hasn’t? This is despite the functions not returning anything! Three questions arise, which we will anser in three corresponding chapters:

How does the function “have access” to the variable?
Why is the list altered while the string is unchanged?
How can we prevent this behaviour?

1. How the function accesses the variable

To find this out we need to understand how the variable ends up in the function: we need to know how Python passes variables to functions. There are many ways this can be done. In order to understand how Python passes variables to functions, we first need to look into how Python stores values in memory.

1.1 How Python stores variables

You might think that when we define a variable like this: person = 'mike' that there is an object in memory named ‘person’ with the value ‘mike’ (see the images below). This is only party true.

How variables get stored in memory in Python vs other languages (think C .e.g.) (Expertly drawn by author)

Python works with references. It creates an object in memory and then it creates a reference called ‘person’ that points to the object in memory, at a specific memory address, with the value ‘mike’. Think of it as hanging a label on the object where the object in memory gets a label with the name of the variable on it.

If we then do something like this: person2 = person we do not create a new object in memory, merely a new reference called ‘person2’ to the object in memory that already exists:

Create a new reference that points to the same object (image by author)

Redefining person2 = ‘bert' will cause Python to create a new object in memory and point the reference named “person2” to there:

1.2 Does Python pass the object or the reference to the function?

It’s pretty key to understand that when we call somefunction(person) we don’t give the function an object in memory but merely the reference to that object.

Python passes variables “by reference” as opposed to “by value”.

This is the first answer to solve the mystery: we give the function a reference to a value in memory in stead of providing the function a copy of the object. This is why we can alter the value without the function returning anything.

Let’s now look at the other part of the solution: why some variables can be altered while others cannot.

2. Why can some values be altered while others cannot? — mutability

Mutability is the ability of objects to change their values after they are created. Let’s start with an overview of mutable variables:

IMMUTABLE                                  MUTABLE 
int, float, decimal, complex (numbers)     list 
bool                                       set 
str                                        dict 
tuple 
frozenset

As you see the str is immutable; this means that it cannot change after it’s initialized. Then how is it possible that we’ve ‘modified’ our string in our earlier example (something like this: input_string += ‘a'). The next few parts explain what happens when we try to alter and overwrite mutable and immutable values.

2.1 What happens when we try to alter immutable values?

We create a variable called my_str with the value 'a'. Next we use the id function to print the variable’s memory address. This is the location in memory where the reference(s) point to.
To re-iterate: in the example below we create a reference called my_str that points to an object in memory that has the value 'a'`. and is located at memory address 1988650365763.

my_str = 'a' 
print(id(my_str))    # 1988650365763 
my_str += 'b' 
print(id(my_str))    # 1988650363313

Next, on line 3, we add 'b' to my_str and print the memory location again. As you see, by the changing memory location, the my_str is different after you’ve added 'b' to it. This means that a new object is created in memory.

It may seem like Python alters the string but under the hood it just creates a new object in memory and ponts the reference called my_str to that new object. The old object with value 'a’ will get get removed. Check out this article to read more about why Python doesn’t just overwrite object in memory and how the old values get removed.

2.2 What happens when we try to alter mutable values?

Let’s do the same experiment with a mutable variable:

my_list= ['a'] 
print(id(my_list))    # 1988503659344 
my_list.append('b') 
print(id(my_list))    # 1988503659344

So the reference called my_list still points to the same location in memory where the object resides. This proves that the object in memory has changed! Also notice that elements inside the list can contain immutable types. If we try to alter these variables the same happens as described in the previous section.

2.3 What happens when we try to overwrite variables?

As we’ve seen in the previous part Python doesn’t overwrite objects in memory. Let’s see this in action:

# Immutable var: string 
my_str = 'a' 
print(id(my_str))            # 1988650365936 
my_str = 'b' 
print(id(my_str))            # 1988650350704 
 
# Mutable var: list 
my_lst = ['a', 'list'] 
print(id(my_lst))            # 1988659494080 
my_lst = ['other', 'list'] 
print(id(my_lst))            # 1988659420608

As you see all memory locations are changed, both mutable and immutable variables. This is the default way Python works with variables. Notice that we don’t try to change the contents of the mutable list: we’re defining a new list; we’re not mutating it but assigning completely new data to my_lst.

2.4 Why are some values mutable while others aren’t?

The mutability is often a design choice; some variables guarantee that the contents remain unchanged and ordered.

Solution: Pass-by-reference and mutability in action

In this part we’ll take our new-found knowledge and solve the mystery. In the code below we declare a (mutable) list and pass it (by reference) to a function. The function is then able to alter the contents of the list. We can see this by the fact that the memory address is the same on the 3rd line and the last one while the content has changed:

# 1. Define list and check out the memory-address and content 
my_list = ['a', 'list'] 
print(id(my_list), my_list)            # 2309673102336 ['a', 'list'] 
 
def change_list(input_list:list): 
    """ Adds value to the list but don't return the list """ 
    print(id(input_list), input_list)  # 2309673102336 ['a', 'list'] 
    input_list.append('b') 
    print(id(input_list))              # 2309673102336 ['a', 'list', 'b'] 
 
# 2. Pass the list into our function (function doesn't return anything) 
change_list(input_list=my_list)      
 
# 3. Notice that the memory location is the same and the list has changed 
print(id(my_list), my_list)            # 2309673102336 ['a', 'list', 'b']

How does this work with immutable values?

Good question. Let’s check it out with a tuple that’s immutable:

# 1. Define a tuple, check out memory address and content 
my_tup = {'a', 'tup'} 
print(id(immutable_string), my_tup)        # 2560317441984, {'a', 'tup'} 
 
def change_tuple(input_tuple:tuple): 
    """ 'overwrites' the tuple we received, don't return anything """ 
    print(id(input_tuple))                 # 2560317441984, {'a', 'tup'} 
    input_tuple = ('other', 'tuple') 
    print(id(input_tuple))                 # 2560317400064, {'other', 'tup'} 
 
# 2. Pass the list into our function (nothing is returned from function) 
change_tuple(input_tuple=immutable_tuple)  
 
# 3. Print out memory location and content again 
print(id(my_tup), my_tup)                  # 2560317441984, {'a', 'tup'}

Since we cannot mutate the value we have to ‘overwrite ’ the input_tuple in the change_tuple function. This doesn’t mean that the object in memory gets overwritten but that a new object gets created in memory.

Then we modify the reference input_tuple that exists in the scope of the change_tuple function so that it now points to this new object. When we exit the function this reference gets cleaned up and in the outer scope the my_tup reference still points to the old object’s memory address.
In short: the ‘new’ tuple only exists in the scope of the function.

3. How to prevent unwanted behaviour

You can prevent this behaviour by giving the function a my_list.copy(). This creates a copy of the list first and provides the function with a reference to that copy so that all changes alter the copy in stead of my_list:

# 2. Pass the list into our function (nothing is returned from function) 
change_list(input_list=my_list.copy())

Conclusion

We’ve discussed mutability and the way Python passes variables to function; two important concepts to understand when designing your Python code. With this article I hope you avoid incomprehensible errors and a lot of time debugging.

I hope this article was as clear as I hope it to be but if this is not the case please let me know what I can do to clarify further. In the meantime, check out my other articles on all kinds of programming-related topics like these:

Happy coding!

— Mike

P.S: like what I’m doing? Follow me!