Question

So I am fairly new to python. But would like your help solving this minor issue I having removing similar duplicates out of a list.

So I have a list of urls: myList = ['http://www.mywebsite.com/shoes', 'http://wwww.yourwebsite.com/', 'http://www.mywebsite.com/shoes/']

I want to remove similar url’s as you can see http://www.mywebsite.com/shoes and http://www.mywebsite.com/shoes/ are pretty much the same. I would like to remove one of them (I don’t care which one) But keep the other. Essentially removing the duplicate from the list. I would give an example. But I don’t even know where to begin.

Any insight would great help.

Answer 1

You could do:

1. First, remove last slash
2. List item

Remove duplicates:

set(map(lambda url: url.rstrip('/'), myList))