Quantcast
Channel: SQL Server Database Engine forum
Viewing all articles
Browse latest Browse all 5123

BulkCopy Issues with SQL

$
0
0

Hello everyone, my question is in regards to a bulkcopy insertion in sql. 

My Table is very simple: 

ID (uniqueidentifier, set as primary key) 

Data (nvarchar(MAX)) 

INDEX: On ID (clustered, unique, hosted on primary key) 

I'm running a small script in C# that uses a bulkcopy to take a massive amount of excel files (approximately 30,000 rows in each file) and copies them into the table. 

At first I was copying files insanely fast, and was very happy with the results. 

Here's a snippet of the current code for reference 

 DataTable ColumnData = new DataTable();
                    ColumnData.Columns.Add("ID", typeof(Guid));
                    ColumnData.Columns.Add("Data", typeof(string));

//Add Data

 try
                    {

                        using (SqlBulkCopy bulkCopy = new SqlBulkCopy("CONNECTIONSTRING", SqlBulkCopyOptions.TableLock | SqlBulkCopyOptions.KeepIdentity | SqlBulkCopyOptions.KeepNulls))
                        {
                            bulkCopy.BatchSize = 10000;
                            bulkCopy.DestinationTableName = "dbo.TestData";
                            try
                            {
                                CentralDatabase.Open();
                                bulkCopy.BulkCopyTimeout = 0;
                                bulkCopy.WriteToServer(ColumnData);
                                CentralDatabase.Close();
                                Console.WriteLine("Data Inserted.");
                                ColumnData.Rows.Clear();
                            }
                            catch (Exception ex) { Console.WriteLine(ex.Message); CentralDatabase.Close();  };

                        }
                    }
                    catch (Exception ex) { CentralDatabase.Close(); };

After about 15,000,000 rows of data however everything slowed down to a crawl. 

I'm certain it's coming from SQL, but I'm not sure where I can go from here.

Is it my index? Should I change it to a nonclustered type? 


Viewing all articles
Browse latest Browse all 5123

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>