Get a JavaScript array with unique values

Get a JavaScript array with unique values

save running time with new JavaScript functions

Since I need to update an array in Firebase Cloud Firestore with duplicate-free values, let’s go through the methods I found and see how they perform in action.

Preparation

Let’s say we have a randomArray with 100,000 random float values and log the execution time in Node.js (v10.0.0) with:

const randomArray = Array.from({length: 1e5}, () => Math.random() * 100);

const start = new Date();
// run scripts
const elapse = new Date() - start;

console.info("Execution time: %d ms", elapse);

indexOf and includes

function onlyUnique(value, index, self) { 
  return self.indexOf(value) === index;
}

const uniqueValues = randomArray.filter(onlyUnique);

This method uses the inbuilt Array.prototype.filter method available from ES5.

The filter method goes through each value in the array, and passes (currentValue, index of currentValue in the array, and the array itself) as arguments. It returns an array of values at which the passed callback returns a true. 1

The onlyUnique method gets those arguments for each iteration. It relies on the fact that the indexOf method would go through the entire array and return the index of the first match; the equality condition would thus only be true for the first occurrence of a value in the array.

This method takes about 3755 ms on my MacBook.

It’s the same with a explicit for loop using the indexOf method as:

let result = [];

for (let index = 0; index < randomArray.length; index++) {
  let el = randomArray[index];
  if (result.indexOf(el) === -1) result.push(el);
  // if (!result.includes(el)) result.push(el);
}

The execution time is about the same with onlyUnique function above (i.e. 3736 ms here). Let’s check the Array.includes() method instead of indexOf. Well, it’s almost the same, with execution time of 3733 ms.

forEach & reduce

Utilise the forEach() or reduce() methods to check through the array again:

let result = [];
randomArray.forEach((el) => {
  if (!result.includes(el)) result.push(el);
});

// let result = randomArray.reduce((array, el) => {
//   if (!array.includes(el)) array.push(el);
//   return array;
// }, []);

3739 ms and 3734 ms for each method, no much difference.

Set

The Set is a new built-in object available in ES6 that lets you store unique values of any type, whether primitive values or object references. Using a Set in conjunction with the Array.from() method we can quickly remove duplicated values from source array.

let result = Array.from(new Set(randomArray));

Well, let’s check the execution time, 38 ms! That’s 100x faster than loops with indexOf or includes.

Instead of the Array.from() method, we can use splat operator (...):2

let result = [...new Set(randomArray)];

But be careful about the browser support for the new Set data structure and the equality rules it follows to calculate the equality between values.

Also, it seems that Array.from() is not supported on old Android webview.3

Set.has()

The Set.has() method allows to check if an element exists in a Set in similar way as Array.includes() or Array.indexOf(). Let’s try it in a for loop together with Set.add() method:

let result = [];
let seen = new Set();

for (let index = 0; index < randomArray.length; index++) {
  let value = randomArray[index];
  if (seen.has(value)) continue;
  seen.add(value);
  result.push(value);
}

This time, the execution time is only 29 ms, a little bit faster. From 3000+ ms to ~30 ms, we got much better performance. Check the compatibility if you’re going to use new Set() in web browsers.

New onlyUnique function

Now, let’s compose the above method into a function that can be reused:

function onlyUnique(array) { 
  const seen = new Set();
  const tempArray = [];
  for (let index = 0; index < array.length; index++) {
    const value = array[index];
    if (seen.has(value)) continue;
    seen.add(value);
    tempArray.push(value);
  }
  return tempArray
}

Simply check with a small array with our new onlyUnique function:

const randomArray = [1, 4, 5, 2, 5, 2, 2, 4, 'h', 'e', 'l', 'l', 'o', 'w', 'o', 'r', 'l', 'd'];
const result = onlyUnique(randomArray);

// [ 1, 4, 5, 2, 5, 2, 2, 4, 'h', 'e', 'l', 'l', 'o', 'w', 'o', 'r', 'l', 'd' ]
// [ 1, 4, 5, 2, 'h', 'e', 'l', 'o', 'w', 'r', 'd' ]

Fine, our new onlyUnique works as expected.

Any problems or suggestions? Please let me know!

Ads by Google

林宏

Frank Lin

Hey, there! This is Frank Lin (@flinhong), one of the 1.41 billion 🇨🇳. This 'inDev. Journal' site holds the exploration of my quirky thoughts and random adventures through life. Hope you enjoy reading and perusing my posts.

YOU MAY ALSO LIKE

Practising closures in JavaScript

JavaScript Notes

2018.12.17

Practising closures in JavaScript

JavaScript is a very function-oriented language. As we know, functions are first class objects and can be easily assigned to variables, passed as arguments, returned from another function invocation, or stored into data structures. A function can access variable outside of it. But what happens when an outer variable changes? Does a function get the most recent value or the one that existed when the function was created? Also, what happens when a function invoked in another place - does it get access to the outer variables of the new place?

Using Liquid in Jekyll - Live with Demos

Web Notes

2016.08.20

Using Liquid in Jekyll - Live with Demos

Liquid is a simple template language that Jekyll uses to process pages for your site. With Liquid you can output complex contents without additional plugins.

Hands on IBM Cloud Functions with CLI

Tools

2020.10.20

Hands on IBM Cloud Functions with CLI

IBM Cloud CLI allows complete management of the Cloud Functions system. You can use the Cloud Functions CLI plugin-in to manage your code snippets in actions, create triggers, and rules to enable your actions to respond to events, and bundle actions into packages.

TOC