REMOVING DUPLICATES USING PYTHON SET

Table of Contents

INTRODUCTION

We can remove duplicate elements by using python set() operator.

REMOVING DUPLICATES IN STRING LIST

We can remove duplicates using set() operator in Python.

# Removing duplicates using set() operator example 1

str_list = ['Python', 'Java', 'C++', 'C', 'PHP', 'XML', 'Python', 'Python'] 
print("\nElement of original list ->", str_list)

print("\nElement of list after removing duplicates ->", set(str_list))

REMOVING DUPLICATES USING PYTHON SET : Output

Element of original list -> ['Python', 'Java', 'C++', 'C', 'PHP', 'XML', 'Python', 'Python']

Element of list after removing duplicates -> {'PHP', 'Python', 'C', 'Java', 'C++', 'XML'}

In the above example we can see that the element “Python” was appeared thrice in the list. But after using set() operator, “Python” appeared only once. We can also see that the output of set operation is enclosed by curly brackets. This is indicates that the result is a set.

# Removing duplicates using set() operator example 2

str_list = ['Python', 'Java', 'C++', 'C', 'PHP', 'XML', 'Python', 'Python', 'Java', 'PHP'] 
print("\nElement of original list ->", str_list)

print("\nElement of list after removing duplicates ->", set(str_list))

REMOVING DUPLICATES USING PYTHON SET : Output

Element of original list -> ['Python', 'Java', 'C++', 'C', 'PHP', 'XML', 'Python', 'Python', 'Java', 'PHP']

Element of list after removing duplicates -> {'C', 'PHP', 'Java', 'XML', 'C++', 'Python'}

In the above example we have seen that in the original list Python was appeared thrice, Java and PHP both were appeared twice. But after using the set() operator all the duplicates are removed and each element in the list have distinct values.

REMOVING DUPLICATES IN INTEGER LIST

# Removing duplicates using set() operator example 3

int_list = [400, 200, 600, 100, 150, 600, 500, 600, 600] 
print("\nElement of original list ->", int_list)

print("\nElement of list after removing duplicates ->", set(int_list))

REMOVING DUPLICATES USING PYTHON SET : Output

Element of original list -> [400, 200, 600, 100, 150, 600, 500, 600, 600]

Element of list after removing duplicates -> {100, 200, 400, 500, 150, 600}

In the above example we have seen that the element 600 is appeared 4 times in the original list. But after using the set() operator the element 600 is appeared only once.

# Removing duplicates using set() operator example 4

int_list = [400, 200, 600, 100, 150, 600, 500, 600, 200, 150] 
print("\nElement of original list ->", int_list)

print("\nElement of list after removing duplicates ->", set(int_list))

REMOVING DUPLICATES USING PYTHON SET : Output

Element of original list -> [400, 200, 600, 100, 150, 600, 500, 600, 200, 150]

Element of list after removing duplicates -> {100, 200, 400, 500, 150, 600}

In the above example we have seen that the element 600 appeared thrice and both 200 and 150 were appeared twice. But after using set() operator all the elements have the distinct values.

REMOVING DUPLICATES IN MIXED DATA TYPE LIST

# Removing duplicates using set() operator example 5

int_list = [400, 200,  'PHP', 'XML', 'Python', 600, 5.8, 600, 5.8, 150, 'Python'] 
print("\nElement of original list ->", int_list)

print("\nElement of list after removing duplicates ->", set(int_list))

REMOVING DUPLICATES USING PYTHON SET : Output

Element of original list -> [400, 200, 'PHP', 'XML', 'Python', 600, 5.8, 600, 5.8, 150, 'Python']

Element of list after removing duplicates -> {5.8, 200, 'XML', 'Python', 'PHP', 400, 150, 600}

In the above example, we have seen that the original list is containing mixed data typed elements. Out of that the elements ‘Python‘, 600, 5.8 are appeared twice. But after using set() operator all the duplicates are removed.

If we put a child list in the outer list elements then set() operator will not work.

# Removing duplicates using set() operator example 6

mixed_list = [400, 200,  'PHP', 'XML', 'Python', [1, 2, 3], 600, 5.8, 600, 5.8, 150, 'Python', [1, 2, 3]] 
print("\nElement of original list ->", mixed_list)

print("\nElement of list after removing duplicates ->", set(mixed_list))

REMOVING DUPLICATES USING PYTHON SET : Output

Element of original list -> [400, 200, 'PHP', 'XML', 'Python', [1, 2, 3], 600, 5.8, 600, 5.8, 150, 'Python', [1, 2, 3]]
Traceback (most recent call last):
  File "d:\PYTHON\PROGRAM\tempCodeRunnerFile.python", line 9, in <module>
    print("\nElement of list after removing duplicates ->", set(mixed_list))
                                                            ^^^^^^^^^^^^^^^
TypeError: unhashable type: 'list'

In the above example we can see that we have used a nested list [1, , 3] as an element and python has raised the Type-error in this case.

REMOVING DUPLICATES USING PYTHON SET : Output

RELATED TOPICS:

Sayantan's blog on Python Programming

Learn Python with Sayantan

REMOVING DUPLICATES USING PYTHON SET