Remove Duplicates in String

Remove duplicate in a string - javascript

With Set and Array.from this is pretty easy:

Array.from(new Set(x.split(','))).toString()

var x = "Int32,Int32,Int32,Int32,Int32,Int32,Int32,Int32,Int32,Double,Double,Double"x = Array.from(new Set(x.split(','))).toString();document.write(x);

How to remove duplicate chars in a string?

It seems from your example that you want to remove REPEATED SEQUENCES of characters, not duplicate chars across the whole string. So this is what I'm solving here.

You can use a regular expression.. not sure how horribly inefficient it is but it
works.

>>> import re
>>> phrase = str("oo rarato roeroeu aa rouroupa dodo rerei dde romroma")
>>> re.sub(r'(.+?)\1+', r'\1', phrase)
'o rato roeu a roupa do rei de roma'

How this substitution proceeds down the string:

oo -> o
" " -> " "
rara -> ra
to -> to
" "-> " "
roeroe -> roe

etc..

Edit: Works for the other example string which should not be modified:

>>> phrase = str("Barbara Bebe com Bernardo")
>>> re.sub(r'(.+?)\1+', r'\1', phrase)
'Barbara Bebe com Bernardo'

Remove duplicates in string

You were almost there. The only thing is that you need to split with ",\\s*" instead of just ",". In the latter case, calling unique won't produce the wanted output, since some string may differ for the number of blank spaces. If you remove them when you split, you solve this issue.

On another note, since you used setDT(df), I guess you are using data.table. If so, you need to use proper data.table grammar to avoid copies:

df[,path:=sapply(
   strsplit(as.character(df$path ), split=",\\s*"), 
    function(x) {paste(unique(x), collapse = ', ')})]

will modify the path column by reference.

Removing duplicate characters from a string

If order does not matter, you can use

"".join(set(foo))

set() will create a set of unique letters in the string, and "".join() will join the letters back to a string in arbitrary order.

If order does matter, you can use a dict instead of a set, which since Python 3.7 preserves the insertion order of the keys. (In the CPython implementation, this is already supported in Python 3.6 as an implementation detail.)

foo = "mppmt"
result = "".join(dict.fromkeys(foo))

resulting in the string "mpt". In earlier versions of Python, you can use collections.OrderedDict, which has been available starting from Python 2.7.

How to remove duplicate values in string which has delimiters

You could use a LinkedHashSet to preserve insertion order. Once you splitted the String by "||" just add the delimiters when constructing back the String.

 String s = "||HelpDesk||IT Staff||IT Staff||Admin||Audit||HelpDesk||";
 Set<String> set = new LinkedHashSet<>(Arrays.asList(s.split(Pattern.quote("||"))));
 String noDup = "||";
 for(String st : set) {
     if(st.isEmpty()) continue;
     noDup += st+"||";
 }

Or using the new java 8 Stream API :

 String noDup = "||"+
     Arrays.stream(s.split(Pattern.quote("||")))
           .distinct()
           .filter(st -> !st.isEmpty()) //we need to remove the empty String produced by the split
           .collect(Collectors.joining("||"))+"||";

Both approaches yield the same result (||HelpDesk||IT Staff||Admin||Audit||).

How to remove duplicate letters in string in PHP

I tend to avoid regex when possible. Here, I'd just split all the letters into one big array and then use array_unique() to de-duplicate:

$unique = array_unique(str_split(implode('', $elem)));

That gives you an array of the unique characters, one character per array element. If you'd prefer those as a string, just implode the array:

$unique = implode('', array_unique(str_split(implode('', $elem))));

How to remove duplicates strings or int from Slice in Go

I found Burak's and Fazlan's solution helpful. Based on that, I implemented the simple functions that help to remove or filter duplicate data from slices of strings, integers, or any other types with generic approach.

Here are my three functions, first is generic, second one for strings and last one for integers of slices. You have to pass your data and return all the unique values as a result.

Generic solution: => Go v1.18

func removeDuplicate[T string | int](sliceList []T) []T {
    allKeys := make(map[T]bool)
    list := []T{}
    for _, item := range sliceList {
        if _, value := allKeys[item]; !value {
            allKeys[item] = true
            list = append(list, item)
        }
    }
    return list
}

To remove duplicate strings from slice:

func removeDuplicateStr(strSlice []string) []string {
    allKeys := make(map[string]bool)
    list := []string{}
    for _, item := range strSlice {
        if _, value := allKeys[item]; !value {
            allKeys[item] = true
            list = append(list, item)
        }
    }
    return list
}

To remove duplicate integers from slice:

func removeDuplicateInt(intSlice []int) []int {
    allKeys := make(map[int]bool)
    list := []int{}
    for _, item := range intSlice {
        if _, value := allKeys[item]; !value {
            allKeys[item] = true
            list = append(list, item)
        }
    }
    return list
}

You can update the slice type, and it will filter out all duplicates data for all types of slices.

Here is the GoPlayground link: https://go.dev/play/p/iyb97KcftMa

Removing duplicates from a String in Java

Convert the string to an array of char, and store it in a LinkedHashSet. That will preserve your ordering, and remove duplicates. Something like:

String string = "aabbccdefatafaz";

char[] chars = string.toCharArray();
Set<Character> charSet = new LinkedHashSet<Character>();
for (char c : chars) {
    charSet.add(c);
}

StringBuilder sb = new StringBuilder();
for (Character character : charSet) {
    sb.append(character);
}
System.out.println(sb.toString());