Hello everyone, my question is in regards to a bulkcopy insertion in sql.
My Table is very simple:
ID (uniqueidentifier, set as primary key)
Data (nvarchar(MAX))
INDEX: On ID (clustered, unique, hosted on primary key)
I'm running a small script in C# that uses a bulkcopy to take a massive amount of excel files (approximately 30,000 rows in each file) and copies them into the table.
At first I was copying files insanely fast, and was very happy with the results.
Here's a snippet of the current code for reference
DataTable ColumnData = new DataTable(); ColumnData.Columns.Add("ID", typeof(Guid)); ColumnData.Columns.Add("Data", typeof(string)); //Add Data try { using (SqlBulkCopy bulkCopy = new SqlBulkCopy("CONNECTIONSTRING", SqlBulkCopyOptions.TableLock | SqlBulkCopyOptions.KeepIdentity | SqlBulkCopyOptions.KeepNulls)) { bulkCopy.BatchSize = 10000; bulkCopy.DestinationTableName = "dbo.TestData"; try { CentralDatabase.Open(); bulkCopy.BulkCopyTimeout = 0; bulkCopy.WriteToServer(ColumnData); CentralDatabase.Close(); Console.WriteLine("Data Inserted."); ColumnData.Rows.Clear(); } catch (Exception ex) { Console.WriteLine(ex.Message); CentralDatabase.Close(); }; } } catch (Exception ex) { CentralDatabase.Close(); };
After about 15,000,000 rows of data however everything slowed down to a crawl.
I'm certain it's coming from SQL, but I'm not sure where I can go from here.
Is it my index? Should I change it to a nonclustered type?