Forums | Admin

Discussion Forums: help

Start New Thread Start New Thread
Message: 91316
BY: Gall Anonim (ggolebio)
DATE: 2009-10-31 11:44
SUBJECT: Rails + sqlite2/3 - very slow data saving

 

Hi,

I start my adventure with rails and I meet seriously problem.
When I try save data to DB - save operation take ~0.5 second per row.

I generate 5000 records with 6 numbers and later every position from my array is save to db.

My table schema is :
CREATE TABLE "results" ("id" INTEGER PRIMARY KEY AUTOINCREMENT NOT
NULL, "restime" date, "no1" byte, "no2" byte, "no3" byte, "no4" byte,
"no5" byte, "no6" byte, "created_at" datetime, "updated_at" datetime);

This is controller

class AllresultsController < ApplicationController
def index
@result = Result.new #create model class
@results = @result.get_all #get 5000 instances result class as an
array

timest = Time.now
puts "start saving to db " + timest.to_s

@results.each do |res| #for every generated result save it to
db
print "Saving " + res.to_s
sstart = Time.now
res.save
puts " (#{(Time.now - sstart).to_s}s)"
end
puts "finish in " + (Time.now - timest).to_s + "seconds"
end
end

example output on server console is :
Saving 2002-09-04 - 2, 9, 11, 22, 35, 45 (0.481178s)
Saving 2002-08-31 - 3, 18, 26, 29, 45, 48 (0.455533s)
Saving 2002-09-11 - 7, 16, 17, 25, 38, 44 (0.464589s)
Saving 2002-09-07 - 4, 5, 10, 17, 30, 40 (0.472471s)

but in db log i see that save operation do not take that amount of
time :
[4;35;1mResult Create (0.7ms)[0m [0mINSERT INTO
"results" ("created_at", "updated_at", "no1", "restime", "no2", "no3",
"no4", "no5", "no6") VALUES('2009-10-29 21:57:08', '2009-10-29
21:57:08', 1, '2004-01-21', 6, 28, 38, 39, 48)[0m
[4;36;1mResult Create (0.8ms)[0m [0;1mINSERT INTO
"results" ("created_at", "updated_at", "no1", "restime", "no2", "no3",
"no4", "no5", "no6") VALUES('2009-10-29 21:57:09', '2009-10-29
21:57:09', 6, '2004-01-24', 13, 16, 20, 36, 44)[0m
[4;35;1mResult Create (0.8ms)[0m [0mINSERT INTO
"results" ("created_at", "updated_at", "no1", "restime", "no2", "no3",
"no4", "no5", "no6") VALUES('2009-10-29 21:57:09', '2009-10-29
21:57:09', 1, '2004-01-28', 3, 11, 17, 24, 35)[0m

Finally save all records takes :
"finish in 2237.331956seconds" ~= 37 minutes for sqlite3 ;/
Sqlite2 is little faster and save all records take ~30 minutes.
I'm try do this on webrick server.
My rails version is : 2.3.4
My ruby version is : ruby 1.8.7 (2008-08-11 patchlevel 72) [x86_64-linux]
My system is : Linux ggolebio-lnx 2.6.28-11-generic #42-Ubuntu SMP Fri Apr 17 01:58:03 UTC 2009 x86_64 GNU/Linux

My database.yml for sqlite3 development db :

development:
adapter: sqlite3
database: db/development.sqlite3
pool: 5
timeout: 5000

Why it is so slow? What I must do to made this faster?

Thanks
Grzegorz


Thread View

Thread Author Date
Rails + sqlite2/3 - very slow data saving Gall Anonim2009-10-31 11:44

Post a followup to this message