Java Remove Duplicates from an Array

How to efficiently remove duplicates from an array without using Set

Since this question is still getting a lot of attention, I decided to answer it by copying this answer from Code Review.SE:

You're following the same philosophy as the bubble sort, which is
very, very, very slow. Have you tried this?:

• Sort your unordered array with quicksort. Quicksort is much faster
  than bubble sort (I know, you are not sorting, but the algorithm you
  follow is almost the same as bubble sort to traverse the array).

• Then start removing duplicates (repeated values will be next to each
  other). In a for loop you could have two indices: source and
  destination. (On each loop you copy source to destination unless they
  are the same, and increment both by 1). Every time you find a
  duplicate you increment source (and don't perform the copy).
  @morgano

Removing left sided duplicates from array

A simple way will be to

1. Add the numbers of the array in reverse order to a LinkedHashSet in order to keep only unique numbers as well as to preserve the last entry of each unique number.

2. Create a List out of the LinkedHashSet.

3. Reverse the List using Collections#reverse.

   import java.util.ArrayList;
   import java.util.Collections;
   import java.util.LinkedHashSet;
   import java.util.List;
   import java.util.Set;
   
   public class Main {
       public static void main(String[] args) {
           Set<Integer> set = new LinkedHashSet<>();
           int[] arr = { 7, 7, 4, 5, 7, 2, 7 };
   
           // Add the numbers of the array in reverse order to a LinkedHashSet in order to remove
           // duplicates as well as to preserve the last entry of each unique number.
           for (int i = arr.length - 1; i >= 0; i--) {
               set.add(arr[i]);
           }
   
           // Create a List out of the LinkedHashSet
           List<Integer> list = new ArrayList<>(set);
   
           // Reverse the List
           Collections.reverse(list);
   
           // Display the result
           System.out.println(list);
       }
   }
   

Output:

[4, 5, 2, 7]

Remove duplicates in array, zero padding at end

@artshakhov:
Here is my approach, which is pretty much close enough to what you've found but using a bit fewer operations...

private static int[] deleteArrayDuplicates(int[] array) {
    for (int i = 0; i < array.length - 1; i++) {
        if (array[i] == NEUTRAL) continue; //if zero is a valid input value then don't waste time with it
        int idx = i + 1;  //no need for third cycle, just use memorization for current shifting index.
        for (int j = i + 1; j < array.length; j++) {
            if (array[i] == array[j]) {
                array[j] = NEUTRAL;
            } else {
                array[idx++] = array[j];
            }
        }
    }
    return array;
}

Remove duplicates from a list of String Array

You can use the toMap collector to provide a custom keyMapper function which serves as a uniqueness test, then simply use the values of the map as your result.

For your uniqueness test, I think it makes more sense to use index 1 (the userID) instead of index 0 (the userName). However, if you wish to change it back, use arr[0] instead of arr[1] below:

List<String[]> userList = new ArrayList<>();
userList.add(new String[]{"George","123"});
userList.add(new String[]{"George","123"});
userList.add(new String[]{"George","456"});
List<String[]> userListNoDupes = new ArrayList<>(userList.stream()
    .collect(Collectors.toMap(arr-> arr[1], Function.identity(), (a,b)-> a)).values());
for(String[] user: userListNoDupes) {
    System.out.println(Arrays.toString(user));
}

Output:

[George, 123]

[George, 456]

How do I remove repeated elements from ArrayList?

If you don't want duplicates in a Collection, you should consider why you're using a Collection that allows duplicates. The easiest way to remove repeated elements is to add the contents to a Set (which will not allow duplicates) and then add the Set back to the ArrayList:

Set<String> set = new HashSet<>(yourList);
yourList.clear();
yourList.addAll(set);

Of course, this destroys the ordering of the elements in the ArrayList.

Remove duplicates from array by shifting elements to the end

When u get match just swap with latest last position. Try with this:

public static String[] remove(String[] array) {
    int len = array.length;
    for (int i = 0; i < len; i++) {
        for (int j = i+1; j < len && j > 0; j++) {
            if(array[i].equals(array[j])){
                String tmp = array[len -1];
                array[len - 1] = array[j];
                array[j] = tmp;
                len--;
                j--;
            }
        }
    }
    return Arrays.copyOf(array, len);
}

Remove duplicates from an array in Java

A nice way to do this is to utilize a Set. That's a structure, that contains only unique values.

Set<Integer> set = new HashSet<Integer>();
int[] array = {1,1,2,2,2,3,3,4,5,6,8};

for (int num : array) {
    set.add(num);
}

System.out.println(set);

Outputs:

[1, 2, 3, 4, 5, 6, 8]

To convert the set to an array you can use set.toArray().

Remove Duplicate from 2D array or Matrix

You have to use distinct() for each array, you can solve your problem using :

String[][] result = Arrays.stream(finalArr)
        .map(s -> Arrays.stream(s).distinct().toArray(String[]::new))
        .toArray(String[][]::new);

Outputs

[[5.0, 100, 99], [5.5, 101, 100], [6.0, 102, 101]]

---

Related Topics