Multiple Insert Statements Vs. Single Insert With Multiple Values

Which is faster: multiple single INSERTs or one multiple-row INSERT?

https://dev.mysql.com/doc/refman/8.0/en/insert-optimization.html

The time required for inserting a row is determined by the following factors, where the numbers indicate approximate proportions:

• Connecting: (3)
• Sending query to server: (2)
• Parsing query: (2)
• Inserting row: (1 × size of row)
• Inserting indexes: (1 × number of indexes)
• Closing: (1)

From this it should be obvious, that sending one large statement will save you an overhead of 7 per insert statement, which in further reading the text also says:

If you are inserting many rows from the same client at the same time, use INSERT statements with multiple VALUES lists to insert several rows at a time. This is considerably faster (many times faster in some cases) than using separate single-row INSERT statements.

Multiple inserts vs one insert with multiple values

One insert versus multiple inserts:

• One insert is going to be a wee bit faster because the query only needs to be parsed once.
• If each statement is committed independently (the default), then a single insert is going to be much faster, because it only has one commit.
• If a record fails to insert, all records fail in a single insert. With multiple inserts, some may succeed (.
• Insert triggers on the table should be built to handle multiple rows at the same time. This is a comment, but wouldn't affect performance.

My guess is that the difference in performance will be pretty minor in your case, but it is worth measuring. It can become important if you have lots of rows at the same time and your table structure includes multiple indexes.

Multiple INSERT statements vs. single INSERT with multiple VALUES

Addition: SQL Server 2012 shows some improved performance in this area but doesn't seem to tackle the specific issues noted below. This
should apparently be fixed in the next major version after
SQL Server 2012!

Your plan shows the single inserts are using parameterised procedures (possibly auto parameterised) so parse/compile time for these should be minimal.

I thought I'd look into this a bit more though so set up a loop (script) and tried adjusting the number of VALUES clauses and recording the compile time.

I then divided the compile time by the number of rows to get the average compile time per clause. The results are below

Up until 250 VALUES clauses present the compile time / number of clauses has a slight upward trend but nothing too dramatic.

But then there is a sudden change.

That section of the data is shown below.

+------+----------------+-------------+---------------+---------------+
| Rows | CachedPlanSize | CompileTime | CompileMemory | Duration/Rows |
+------+----------------+-------------+---------------+---------------+
|  245 |            528 |          41 |          2400 | 0.167346939   |
|  246 |            528 |          40 |          2416 | 0.162601626   |
|  247 |            528 |          38 |          2416 | 0.153846154   |
|  248 |            528 |          39 |          2432 | 0.157258065   |
|  249 |            528 |          39 |          2432 | 0.156626506   |
|  250 |            528 |          40 |          2448 | 0.16          |
|  251 |            400 |         273 |          3488 | 1.087649402   |
|  252 |            400 |         274 |          3496 | 1.087301587   |
|  253 |            400 |         282 |          3520 | 1.114624506   |
|  254 |            408 |         279 |          3544 | 1.098425197   |
|  255 |            408 |         290 |          3552 | 1.137254902   |
+------+----------------+-------------+---------------+---------------+

The cached plan size which had been growing linearly suddenly drops but CompileTime increases 7 fold and CompileMemory shoots up. This is the cut off point between the plan being an auto parametrized one (with 1,000 parameters) to a non parametrized one. Thereafter it seems to get linearly less efficient (in terms of number of value clauses processed in a given time).

Not sure why this should be. Presumably when it is compiling a plan for specific literal values it must perform some activity that does not scale linearly (such as sorting).

It doesn't seem to affect the size of the cached query plan when I tried a query consisting entirely of duplicate rows and neither affects the order of the output of the table of the constants (and as you are inserting into a heap time spent sorting would be pointless anyway even if it did).

Moreover if a clustered index is added to the table the plan still shows an explicit sort step so it doesn't seem to be sorting at compile time to avoid a sort at run time.

I tried to look at this in a debugger but the public symbols for my version of SQL Server 2008 don't seem to be available so instead I had to look at the equivalent UNION ALL construction in SQL Server 2005.

A typical stack trace is below

sqlservr.exe!FastDBCSToUnicode()  + 0xac bytes  
sqlservr.exe!nls_sqlhilo()  + 0x35 bytes    
sqlservr.exe!CXVariant::CmpCompareStr()  + 0x2b bytes   
sqlservr.exe!CXVariantPerformCompare<167,167>::Compare()  + 0x18 bytes  
sqlservr.exe!CXVariant::CmpCompare()  + 0x11f67d bytes  
sqlservr.exe!CConstraintItvl::PcnstrItvlUnion()  + 0xe2 bytes   
sqlservr.exe!CConstraintProp::PcnstrUnion()  + 0x35e bytes  
sqlservr.exe!CLogOp_BaseSetOp::PcnstrDerive()  + 0x11a bytes    
sqlservr.exe!CLogOpArg::PcnstrDeriveHandler()  + 0x18f bytes    
sqlservr.exe!CLogOpArg::DeriveGroupProperties()  + 0xa9 bytes   
sqlservr.exe!COpArg::DeriveNormalizedGroupProperties()  + 0x40 bytes    
sqlservr.exe!COptExpr::DeriveGroupProperties()  + 0x18a bytes   
sqlservr.exe!COptExpr::DeriveGroupProperties()  + 0x146 bytes   
sqlservr.exe!COptExpr::DeriveGroupProperties()  + 0x146 bytes   
sqlservr.exe!COptExpr::DeriveGroupProperties()  + 0x146 bytes   
sqlservr.exe!CQuery::PqoBuild()  + 0x3cb bytes  
sqlservr.exe!CStmtQuery::InitQuery()  + 0x167 bytes 
sqlservr.exe!CStmtDML::InitNormal()  + 0xf0 bytes   
sqlservr.exe!CStmtDML::Init()  + 0x1b bytes 
sqlservr.exe!CCompPlan::FCompileStep()  + 0x176 bytes   
sqlservr.exe!CSQLSource::FCompile()  + 0x741 bytes  
sqlservr.exe!CSQLSource::FCompWrapper()  + 0x922be bytes    
sqlservr.exe!CSQLSource::Transform()  + 0x120431 bytes  
sqlservr.exe!CSQLSource::Compile()  + 0x2ff bytes   

So going off the names in the stack trace it appears to spend a lot of time comparing strings.

This KB article indicates that DeriveNormalizedGroupProperties is associated with what used to be called the normalization stage of query processing

This stage is now called binding or algebrizing and it takes the expression parse tree output from the previous parse stage and outputs an algebrized expression tree (query processor tree) to go forward to optimization (trivial plan optimization in this case) [ref].

I tried one more experiment (Script) which was to re-run the original test but looking at three different cases.

1. First Name and Last Name Strings of length 10 characters with no duplicates.
2. First Name and Last Name Strings of length 50 characters with no duplicates.
3. First Name and Last Name Strings of length 10 characters with all duplicates.

It can clearly be seen that the longer the strings the worse things get and that conversely the more duplicates the better things get. As previously mentioned duplicates don't affect the cached plan size so I presume that there must be a process of duplicate identification when constructing the algebrized expression tree itself.

Edit

One place where this information is leveraged is shown by @Lieven here

SELECT * 
FROM (VALUES ('Lieven1', 1),
             ('Lieven2', 2),
             ('Lieven3', 3))Test (name, ID)
ORDER BY name, 1/ (ID - ID) 

Because at compile time it can determine that the Name column has no duplicates it skips ordering by the secondary 1/ (ID - ID) expression at run time (the sort in the plan only has one ORDER BY column) and no divide by zero error is raised. If duplicates are added to the table then the sort operator shows two order by columns and the expected error is raised.

Performance Multiple inserts or multiple values single insert

Yes, generally bulk insertion is faster than single insert at a time as it avoids intermediate communication which was occurring for every insert statement.

But sometimes it leads to problems as insert/update statements acquire exclusive locks on tables/rows which means no other process/connection can use table at that time.

If you have multiple processes using DB at the same some of which are reading from table and some are writing then whole operations will be locked/stopped at the time of your bulk insertion

so bulk insertion will lock the table for more time than single insert which can cause issues for other processes if lock time is more and DB is not tuned for it.

If you are just inserting into the DB with no other operation, then go for bulk insert from a file (which is much faster) and if you have other processes in place adjust bulk insert frequency considering locking.

Possible duplicate is of update statements Is bulk update faster than single update in db2?

Insert multiple rows using single insert statement

Your syntax is slightly off:

Insert Into @AddressRecordsToPurge (GuidValue, GuidColumn, GuidTable)
SELECT EMPLOYMENTSEQUENCENUMBER, 'EMPLOYMENTSEQUENCENUMBER', 'ACCOUNTANTSREFERENCE'
FROM ACCOUNTANTSREFERENCE
WHERE CustomerNumber =  @CustomerNumber
    AND Customerversionnumber = @CustomerVersionNumber
    AND EMPLOYMENTSEQUENCENUMBER IS NOT NULL; 

---

Related Topics