How to Remove Duplicates from a List in Python

Are you a Python programmer and sometimes find yourself dealing with a list that contains duplicate elements? While this might not be a problem in some situations, in most scenarios it’s more efficient to have a list without duplicates.

In this tutorial, we will guide you on how to remove duplicates from a list in Python. Python offers multiple ways to remove duplicates from a list and we are going to explore three main methods: using a for loop, using list comprehension, and using the built-in Python function set().

Method 1: Using a for loop

One way to remove duplicates from a list in Python is by using a for loop. In this method, you can use a for loop and an if statement to append unique elements to a new list.

1
2
3
4
5
6
initialList = [1, 2, 2, 3, 3, 4, 4, 5, 6, 6]
 
duplicatesRemoved = []
for i in initialList:
    if i not in duplicatesRemoved:
        duplicatesRemoved.append(i)

The output for this code will display a list without duplicates:

[1, 2, 3, 4, 5, 6]

Method 2: Using list comprehension

Another method to remove duplicates from a list is by using list comprehension. List comprehension is an elegant way to define and create lists based on existing lists.

1
2
3
4
initialList = [1, 2, 2, 3, 3, 4, 4, 5, 6, 6]
 
duplicatesRemoved = []
[duplicatesRemoved.append(i) for i in initialList if i not in duplicatesRemoved]

The output will be the same as the first method, which is a list without any duplicate values:

[1, 2, 3, 4, 5, 6]

Method 3: Using the set() function

The most efficient way to remove duplicates from a list in Python is to convert the list to a set. The set() function in Python automatically removes any duplicates because sets cannot contain duplicate values.

1
2
initialList = [1, 2, 2, 3, 3, 4, 4, 5, 6, 6]
duplicatesRemoved = list(set(initialList))

Again, the output will be a list without any duplicate values:

[1, 2, 3, 4, 5, 6]

Full Code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
#Method 1
initialList = [1, 2, 2, 3, 3, 4, 4, 5, 6, 6]
duplicatesRemoved = []
for i in initialList:
    if i not in duplicatesRemoved:
        duplicatesRemoved.append(i)
 
print(duplicatesRemoved)
 
#Method 2
initialList = [1, 2, 2, 3, 3, 4, 4, 5, 6, 6]
duplicatesRemoved = []
[duplicatesRemoved.append(i) for i in initialList if i not in duplicatesRemoved]
 
print(duplicatesRemoved)
 
#Method 3
initialList = [1, 2, 2, 3, 3, 4, 4, 5, 6, 6]
duplicatesRemoved = list(set(initialList))
 
print(duplicatesRemoved)

Output:

[1, 2, 3, 4, 5, 6]
[1, 2, 3, 4, 5, 6]
[1, 2, 3, 4, 5, 6]

Conclusion

In this tutorial, we covered three methods for removing duplicates from a list in Python. We hope this helps boost your efficiency in programming with Python as you will no longer be burdened by unwanted duplicate values in your lists. Happy coding!