Prevent Duplicate Submission of Data - Overview
How to prevent duplicate submission of data? What is the easiest solution? When asked these questions, the effective way I can think of is to intercept through the front and back ends separately. They can effectively solve the problem of data duplication.
1. Front-end Interception
Front-end interception refers to intercepting repeated requests through HTML pages. For example, after the user clicks the "Submit" button, we can make the button unavailable or hidden. The implementation code of front-end interception is as follows.
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RequestMapping("/user")
@RestController
public class UserController {
/**
* Method that is repeatedly requested
*/
@RequestMapping("/add")
public String addUser(String id) {
// Business code...
System.out.println("Add user ID:" + id);
return "Successful execution!";
}
}
But front-end interception has a fatal problem. If you are a knowledgeable programmer or an illegal user, you can bypass the front-end page directly and submit the request repeatedly by simulating the request. For example, if you recharged $100 and resubmitted it 10 times, it became $1000.
Therefore, in addition to the front-end interception of some normal misoperations, the back-end interception is also essential.
2. Back-end Interception
The implementation idea of back-end interception is to determine whether the business has been executed before the method is executed. If it has been executed, it will not be executed, otherwise, it will be executed normally.
How to Prevent Duplicate Submission of Data
We store the requested business ID in memory and add a mutex to ensure program execution security under multi-threading. However, the easiest way to store data in memory is to use a HashMap store. Using Guava Cache has the same effect. But obviously, HashMap can implement functions faster, so let's first implement a duplication-proof (anti-duplication) version of HashMap.
1. Basic Version - HashMap
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.util.HashMap;
import java.util.Map;
/**
* Normal Map version
*/
@RequestMapping("/user")
@RestController
public class UserController3 {
// cache ID collection
private Map<String, Integer> reqCache = new HashMap<>();
@RequestMapping("/add")
public String addUser(String id) {
// non-null judgment (ignored)...
synchronized (this.getClass()) {
// Repeat request judgment
if (reqCache.containsKey(id)) {
// repeat the request
System.out.println("Do not submit again!!!" + id);
return "Failed to execute";
}
// store request id
reqCache.put(id, 1);
}
// Business code...
System.out.println("Add user ID:" + id);
return "Successful execution!";
}
}
This implementation has a fatal problem because the HashMap grows infinitely. So it will take up more and more memory, and the lookup speed will decrease as the number of HashMap increases. Therefore, we need to implement an implementation that can automatically "clear" expired data.
2. Optimized Version - Fixed Size Array
This release solves the problem of the infinite growth of HashMap. It uses the method of adding a subscript counter (reqCacheCounter) to the array to realize the circular storage of a fixed array. When the array is stored to the last bit, set the storage subscript of the array to 0, and then store the data from the beginning. The implementation code is shown below.
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.util.Arrays;
@RequestMapping("/user")
@RestController
public class UserController {
private static String[] reqCache = new String[100]; // Request ID storage collection
private static Integer reqCacheCounter = 0; // request counter (indicates where the ID is stored)
@RequestMapping("/add")
public String addUser(String id) {
// non-null judgment (ignored)...
synchronized (this.getClass()) {
// Repeat request judgment
if (Arrays.asList(reqCache).contains(id)) {
// repeat the request
System.out.println("Do not submit again!!!" + id);
return "Failed to execute";
}
// record request id
if (reqCacheCounter >= reqCache.length) reqCacheCounter = 0; // reset the counter
reqCache[reqCacheCounter] = id; // save ID to cache
reqCacheCounter++; // move the subscript back one place
}
// Business code...
System.out.println("Add user ID:" + id);
return "Successful execution!";
}
}
3. Extended Version - Double Detect Lock (DCL)
In the previous implementation method, the judgment and addition of business are put into synchronized for locking operation. Obviously, the performance is not very high, so we can use the well-known DCL (Double Checked Locking) in the singleton to optimize the execution efficiency of the code. The implementation code is as follows.
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.util.Arrays;
@RequestMapping("/user")
@RestController
public class UserController {
private static String[] reqCache = new String[100]; // Request ID storage collection
private static Integer reqCacheCounter = 0; // request counter (indicates where the ID is stored)
@RequestMapping("/add")
public String addUser(String id) {
// non-null judgment (ignored)...
// Repeat request judgment
if (Arrays.asList(reqCache).contains(id)) {
// repeat the request
System.out.println("Do not submit again!!!" + id);
return "Failed to execute";
}
synchronized (this.getClass()) {
// Double checked locking (DCL, double checked locking) improves the execution efficiency of the program
if (Arrays.asList(reqCache).contains(id)) {
// repeat the request
System.out.println("Do not submit again!!!" + id);
return "Failed to execute";
}
// record request id
if (reqCacheCounter >= reqCache.length) reqCacheCounter = 0; // reset the counter
reqCache[reqCacheCounter] = id; // save ID to cache
reqCacheCounter++; // move the subscript back one place
}
// Business code...
System.out.println("Add user ID:" + id);
return "Successful execution!";
}
}
Please note that DCL is suitable for business scenarios with high frequency of repeated submissions. DCL is not applicable for the opposite business scenario.
4. Full Version - LRUMap
The above code has basically implemented the interception of repeated data, but it is obviously not concise and elegant, such as the declaration and business processing of the subscript counter. But fortunately, Apache provides us with a commons-collections framework, which has a very useful data structure LRUMap that can save a specified amount of fixed data. And it will follow the LRU algorithm to help you clear the least commonly used data.
Tips: LRU is the abbreviation of Least Recently Used, which is the least recently used.
First, let's add a reference to the Apache commons-collections.
<!-- Collection tool class apache commons collections --> <!-- https://mvnrepository.com/artifact/org.apache.commons/commons-collections4 --> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-collections4</artifactId> <version>4.4</version> </dependency>
The implementation code is shown below. After using LRUMap, the code is obviously much simpler.
import org.apache.commons.collections4.map.LRUMap;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RequestMapping("/user")
@RestController
public class UserController {
// The maximum capacity is 100, and the Map collection of data is eliminated according to the LRU algorithm
private LRUMap<String, Integer> reqCache = new LRUMap<>(100);
@RequestMapping("/add")
public String addUser(String id) {
// non-null judgment (ignored)...
synchronized (this.getClass()) {
// Repeat request judgment
if (reqCache.containsKey(id)) {
// repeat the request
System.out.println("Do not submit again!!!" + id);
return "Failed to execute";
}
// store request id
reqCache.put(id, 1);
}
// Business code...
System.out.println("Add user ID:" + id);
return "Successful execution!";
}
}
5. Final Version - Packaging
All of the above are method-level implementations. However, in actual business, we may have many ways to prevent weight. Then, let's encapsulate a public method for all classes to use.
import org.apache.commons.collections4.map.LRUMap;
/**
* Idempotency judgment
*/
public class IdempotentUtils {
// According to the LRU (Least Recently Used, least recently used) algorithm to eliminate the map set of data, the maximum capacity is 100
private static LRUMap<String, Integer> reqCache = new LRUMap<>(100);
/**
* Idempotency judgment
* @return
*/
public static boolean judge(String id, Object lockClass) {
synchronized (lockClass) {
// Repeat request judgment
if (reqCache.containsKey(id)) {
// repeat the request
System.out.println("Do not submit again!!!" + id);
return false;
}
// Non-repeated request, store the request ID
reqCache.put(id, 1);
}
return true;
}
}
The calling code is as follows.
import com.example.idempote.util.IdempotentUtils;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RequestMapping("/user")
@RestController
public class UserController4 {
@RequestMapping("/add")
public String addUser(String id) {
// non-null judgment (ignored)...
// -------------- idempotent call (start) --------------
if (!IdempotentUtils.judge(id, this.getClass())) {
return "Failed to execute";
}
// -------------- idempotent call (end) --------------
// Business code...
System.out.println("Add user ID:" + id);
return "Successful execution!";
}
}
In general, the code is written here to end. But if you want to be more concise, you can also write business code into annotations through custom annotations. The method that needs to be called only needs to write a line of annotation to prevent repeated submission of data.
Prevent Duplicate Submission of Data - Summary
This article discusses several effective ways to prevent data duplication. The first is the front-end interception, which can be used to block repeated submissions under normal operation by hiding and setting the unavailability of the button. However, in order to avoid repeated submissions through abnormal channels, we have implemented 5 versions of back-end interception, including the HashMap version, fixed array version, array version of double detection lock, LRUMap version, and LRUMap packaged version.
Please note that all the content in this article only applies to duplicate data interception in a stand-alone environment. If it is a distributed environment, it needs to be implemented with a database or Redis.