Want to improve your insert records? In this article, you can learn how to improve bulk insert performance by 100x using Spring Data JPA.
Join the DZone community and get the full member experience. I was facing a problem where I wanted to insert millions of records into the database, which needed to be imported from the file. So, I did some research around this, and I would like to share with you what I found which helped me improve the insert records throughput by nearly 100 times. Initially, when I was just trying to do bulk insert using spring JPA’s saveAll method, I was getting a performance of about 185 seconds per 10,000 records. After doing the following changes below, the performance to insert 10,000 records was just in 4.3 seconds. Yes,4.3 Seconds for 10k records. So, to achieve this, I had to change the way I was inserting data. When I was inserting initially, I was pushing all the 10k records from the list directly by calling the saveAll method. I changed this to the batch size of 30. You could also increase the batch size to even 60, but it doesn’t half the time taken to insert records.
