def remove_duplicates(seq, idfun=None):  
    # order preserving 
    if idfun is None: 
        def idfun(x): return x 
    seen = {} 
    result = [] 
    for item in seq: 
        marker = idfun(item) 
        # in old Python versions: 
        # if seen.has_key(marker) 
        # but in new ones: 
        if marker in seen: continue 
        seen[marker] = 1 
        result.append(item) 
    return result



>>> a=list('ABeeE') 
>>> f5(a) 
['A','B','e','E'] 
>>> f5(a, lambda x: x.lower()) 
['A','B','e']

http://www.peterbe.com/plog/uniqifiers-benchmark

Not only is it really really fast; it's also order preserving and supports an optional transform function

Remove duplicates from a list

list, python

Django