From liming.lian at oracle.com Fri Dec 7 07:20:38 2007 From: liming.lian at oracle.com (Liming Lian) Date: Fri, 07 Dec 2007 20:20:38 +0800 Subject: [Ruby-oci8-devel] alloc_sz Message-ID: <47593A96.7040100@oracle.com> Hi Kubo, Maybe I have missed some knowledge so that I am a little confused about how the alloc_sz is calculated during binding phase. Look at following code snippet for initializing String binding: File: bind.c Function: bind_string_init sz += sizeof(sb4); obind->value_sz = sz; obind->alloc_sz = (sz + (sizeof(sb4) - 1)) & ~(sizeof(sb4) - 1); I am wondering why the alloc_sz is calculated from the above expression, any hints? I want to make this clear because this attribute will be used for allocating memory for binding value and passed to OCIBindArrayOfStruct as skip parameter later. Thanks in advance! Liming From kubo at jiubao.org Fri Dec 7 11:42:19 2007 From: kubo at jiubao.org (KUBO Takehiro) Date: Sat, 8 Dec 2007 01:42:19 +0900 Subject: [Ruby-oci8-devel] alloc_sz In-Reply-To: <47593A96.7040100@oracle.com> References: <47593A96.7040100@oracle.com> Message-ID: <5d847bcd0712070842h414064aew164279ef4486f0a7@mail.gmail.com> Hi Liming, > sz += sizeof(sb4); > obind->value_sz = sz; > obind->alloc_sz = (sz + (sizeof(sb4) - 1)) & ~(sizeof(sb4) - 1); This round up to sz to 4-byte boundary. If sz is 5~8, alloz_sz is 8. If 9~12, 12. (sz + (sizeof(sb4) - 1)) & ~(sizeof(sb4) - 1) => (sz + (4 - 1) & ~(4 - 1) => (sz + 3) & ~3 => (sz + 3) & 0xFFFFFFFC round-up is done by one addition and one bit-mask. This is most efficient way. sizeof(sb4) is 4 in any OS and CPU. We can write more simply as "(sz + 3) & 0xFFFFFFFC". But if sizeof(x)'s value depends on OS or CPU, "(sz + (sizeof(x) - 1)) & ~(sizeof(x) - 1)" is most portable. This is an idiom in C. > I am wondering why the alloc_sz is calculated from the above expression, > any hints? I want to make this clear because this attribute will be used > for allocating memory for binding value and passed to > OCIBindArrayOfStruct as skip parameter later. The first member of oci8_vstr_t is sb4. It should be aligned to 4-byte boundary in memory on x86 CPU. If it is not aligned, it causes memory access penalty. As for SPARC CPU, it must be aligned, otherwise it causes BUS error. If alloc_sz is not a multiple of 4, the second element is not on 4-byte boundary. -- KUBO Takehiro From kubo at jiubao.org Fri Dec 7 22:16:08 2007 From: kubo at jiubao.org (KUBO Takehiro) Date: Sat, 8 Dec 2007 12:16:08 +0900 Subject: [Ruby-oci8-devel] test Message-ID: <5d847bcd0712071916g4b2b1b93uda1d82168825fb42@mail.gmail.com> Sorry, ignore this mail. This is a test. My previous post looks like not to be delivered. It would be caused by my changing of ML setting... From kubo at jiubao.org Sat Dec 8 08:39:11 2007 From: kubo at jiubao.org (KUBO Takehiro) Date: Sat, 8 Dec 2007 08:39:11 -0500 (EST) Subject: [ruby-oci8-devel] In-reply-to header test Message-ID: <20071208133926.8497918585CC@rubyforge.org> In-reply-to header test Please ignore this mail From liming.lian at oracle.com Mon Dec 10 05:19:52 2007 From: liming.lian at oracle.com (Liming Lian) Date: Mon, 10 Dec 2007 18:19:52 +0800 Subject: [ruby-oci8-devel] [Ruby-oci8-devel] alloc_sz In-Reply-To: <5d847bcd0712070842h414064aew164279ef4486f0a7@mail.gmail.com> References: <47593A96.7040100@oracle.com> <5d847bcd0712070842h414064aew164279ef4486f0a7@mail.gmail.com> Message-ID: <475D12C8.6090201@oracle.com> Hi Kubo, Thanks for your reply! By hacking the code, I am glad to see a lot of efforts have been done for preparing the implementation of Array DML. I notice following variable: max_array_size, the magic switch for entering array insert mode. By passing a non-null max_array_size parameter to oci8_bind_initialize, the array insert mode is switched on. The program will check the input binding variable of array type, and then allocate necessary memory, assign element values to the right positions by using oci8_set_data and oci8_set_data_at, and finally call the OCIBindByPos or OCIBindByName to finish the binding task. One thing I want to make clear is, kubo, which kind of binds are you initially considering to support for Array DML? OCI supports two kinds of binds for array DML, PL/SQL binds and non-PL/SQL binds. PHP has implemented a function, oci_bind_array_by_name, which implements the PL/SQL binds for array DML. Check more at http://us2.php.net/manual/en/function.oci-bind-array-by-name.php. While Perl's bind_param_array is to implement non-PL/SQL binds. If I don't remember wrong, for OCIBindByPos or OCIBindByName, once you have passed non-null values to parameters maxarr_len and curelep, OCI assumes it is a PL/SQL bind. So if we want to implement non-PL/SQL, both maxarr_len and curelep should be set NULL. Otherwise OCI will report following error: ORA-01484: arrays can only be bound to PL/SQL statements. Please look at the function call for OCIBindByPos in oci8_bind of file stmt.c. if (obind->maxar_sz == 0) { curelep = NULL; } else { curelep = &obind->curar_sz; } And, status = OCIBindByPos(stmt->base.hp.stmt, &obind->base.hp.bnd, oci8_errhp, position, obind->valuep, obind->value_sz, bind_class->dty, indp, NULL, 0, obind->maxar_sz, curelep, mode); It means that once we have set maxar_sz, both parameters: maxarr_len and curelep will be enabled in the OCIBindByPos call. So from the source code, I guess it is to implement the PL/SQL binds. > Hi Liming, > > >> sz += sizeof(sb4); >> obind->value_sz = sz; >> obind->alloc_sz = (sz + (sizeof(sb4) - 1)) & ~(sizeof(sb4) - 1); >> > > This round up to sz to 4-byte boundary. > If sz is 5~8, alloz_sz is 8. If 9~12, 12. > > (sz + (sizeof(sb4) - 1)) & ~(sizeof(sb4) - 1) > => (sz + (4 - 1) & ~(4 - 1) > => (sz + 3) & ~3 > => (sz + 3) & 0xFFFFFFFC > > round-up is done by one addition and one bit-mask. This is most > efficient way. > > sizeof(sb4) is 4 in any OS and CPU. We can write more simply as > "(sz + 3) & 0xFFFFFFFC". But if sizeof(x)'s value depends on OS or > CPU, "(sz + (sizeof(x) - 1)) & ~(sizeof(x) - 1)" is most portable. > This is an idiom in C. > > >> I am wondering why the alloc_sz is calculated from the above expression, >> any hints? I want to make this clear because this attribute will be used >> for allocating memory for binding value and passed to >> OCIBindArrayOfStruct as skip parameter later. >> > > The first member of oci8_vstr_t is sb4. It should be aligned to 4-byte > boundary in memory on x86 CPU. If it is not aligned, it causes memory > access penalty. As for SPARC CPU, it must be aligned, otherwise it > causes BUS error. If alloc_sz is not a multiple of 4, the second > element is not on 4-byte boundary. > > From liming.lian at oracle.com Tue Dec 11 01:50:20 2007 From: liming.lian at oracle.com (Liming Lian) Date: Tue, 11 Dec 2007 14:50:20 +0800 Subject: [ruby-oci8-devel] Proposal for bind_param_array function prototype Message-ID: <475E332C.5090305@oracle.com> Hi, I have proposed the function prototype for bind_param_array. Please let me know what's your opinion on it. ------------- bind_param_array ------------- ## Description bind_param_array(key, var_array, type = nil, max_item_length = nil, max_array_size =nil) bind array explicitly ## Parameters * key Same as key parameter in bind_param * var_array An array to insert * type The type of elements in parameter var_array. If not specified, the type will be determined by checking the first element in var_array? * max_item_length Maximum length of the item in var_array. If not specified, a default size will be used. The default sizes vary from the different data types of var_array. * max_array_size Maximum length for incoming array. If not specified, a default size will be used. (Need to discuss: how large the default value should be proper for max_array_size. The maximum number of rows allowed in an array DML statements is 4 gigabytes -1, but obviously we can't use that number here) From kubo at jiubao.org Tue Dec 11 10:19:29 2007 From: kubo at jiubao.org (KUBO Takehiro) Date: Wed, 12 Dec 2007 00:19:29 +0900 Subject: [ruby-oci8-devel] [Ruby-oci8-devel] alloc_sz In-Reply-To: <475D12C8.6090201@oracle.com> References: <47593A96.7040100@oracle.com> <5d847bcd0712070842h414064aew164279ef4486f0a7@mail.gmail.com> <475D12C8.6090201@oracle.com> Message-ID: <5d847bcd0712110719j32d7485ao587fd66e264e4c14@mail.gmail.com> Hi Liming, On Dec 10, 2007 7:19 PM, Liming Lian wrote: > One thing I want to make clear is, kubo, which kind of binds are you > initially considering to support for Array DML? OCI supports two kinds > of binds for array DML, PL/SQL binds and non-PL/SQL binds. (snip) > It means that once we have set maxar_sz, both parameters: maxarr_len and > curelep will be enabled in the OCIBindByPos call. So from the source > code, I guess it is to implement the PL/SQL binds. Yes. Current source code is for the PL/SQL binds. I had started to implement PL/SQL binds initially. Please change the code or add code to support non-PL/SQL binds. We can delete the PL/SQL binds support if the following URL's problem can be fixed. http://forums.oracle.com/forums/thread.jspa?messageID=23975 If we can get the TDO of the collection type, we can create an instance of the collection type by OCIObjectNew(). If the package local types can be described, we have no need to support PL/SQL binds. From kubo at jiubao.org Tue Dec 11 11:37:02 2007 From: kubo at jiubao.org (KUBO Takehiro) Date: Wed, 12 Dec 2007 01:37:02 +0900 Subject: [ruby-oci8-devel] Proposal for bind_param_array function prototype In-Reply-To: <475E332C.5090305@oracle.com> References: <475E332C.5090305@oracle.com> Message-ID: <5d847bcd0712110837y6343b788r9acb504b944aa376@mail.gmail.com> Hi Liming, On Dec 11, 2007 3:50 PM, Liming Lian wrote: > I have proposed the function prototype for bind_param_array. Please let > me know what's your opinion on it. Is this prototype is for PL/SQL binds or non-PL/SQL binds? If it is for PL/SQL binds, > * max_array_size > > Maximum length for incoming array. If not specified, a default size will > be used. (Need to discuss: how large the default value should be proper > for max_array_size. The maximum number of rows allowed in an array DML > statements is 4 gigabytes -1, but obviously we can't use that number here) IMO, the default value should the length of var_array. If the parameter is used only for input, var_array's length is the exact required size. If the parameter is used for output, we cannot know correct size. Users only know it. Setting proper size is a task of them. If it is for non-PL/SQL binds, that's another story. All max_array_size should be same for one OCI8::Cursor. That's why one of the values is passed to OCIStmtExecute()'s iters argument. Consider a sample case inserting csv data to an Oracle table. Without non-PL/SQL binds: execute a SQL per one line. conn = OCI8.new(user, pass) cursor = conn.parse('insert into csv_data values(:1, :2, :3)') cursor.bind_param(1, nil, String, 30) cursor.bind_param(2, nil, String, 30) cursor.bind_param(3, nil, String, 30) open('cvs_file', 'r') do |f| while line = f.gets col1, co2, col3 = line.split(',') cursor[1] = col1 cursor[2] = col2 cursor[3] = col3 cursor.exec end end conn.commit With non-PL/SQL binds: execute a SQL per 100 lines. conn = OCI8.new(user, pass) cursor = conn.parse('insert into csv_data values(:1, :2, :3)') max_array_size = 100 cursor.bind_param_array(1, nil, String, 30, max_array_size) cursor.bind_param_array(2, nil, String, 30, max_array_size) cursor.bind_param_array(3, nil, String, 30, max_array_size) open('cvs_file', 'r') do |f| ary1 = Array.new(max_array_size) ary2 = Array.new(max_array_size) ary3 = Array.new(max_array_size) nitems = 0 while line = f.gets col1, co2, col3 = line.split(',') ary1[nitems] = col1 ary2[nitems] = col2 ary3[nitems] = col3 nitems += 1 if nitems == max_array_size cursor[1] = ary1 cursor[2] = ary2 cursor[3] = ary3 cursor.exec nitems = 0 end end ... We have to call cursor.exec whose iteration count is nitems. ... ... But how to set the iteration count? ... end conn.commit Hmm, it needs more time for us to think about this. From liming.lian at oracle.com Wed Dec 12 07:38:36 2007 From: liming.lian at oracle.com (Liming Lian) Date: Wed, 12 Dec 2007 20:38:36 +0800 Subject: [ruby-oci8-devel] Proposal for bind_param_array function prototype In-Reply-To: <5d847bcd0712110837y6343b788r9acb504b944aa376@mail.gmail.com> References: <475E332C.5090305@oracle.com> <5d847bcd0712110837y6343b788r9acb504b944aa376@mail.gmail.com> Message-ID: <475FD64C.3060100@oracle.com> Hi Kubo, > Is this prototype is for PL/SQL binds or non-PL/SQL binds? > Sorry for forgetting to declare it, this is for non-PL/SQL binds. Another update for parameter "type", if it isn't specified, it should use the first *non-nil* element's type, not just the first element's. > If it is for non-PL/SQL binds, that's another story. > All max_array_size should be same for one OCI8::Cursor. > That's why one of the values is passed to OCIStmtExecute()'s iters > argument. > Maybe we have different views on the attribute "max_array_size". In my proposal, it is a pre-defined value for the limit of maximum array size. Users can bind arrays with sizes not greater than max_array_size freely. At the same time they should make sure all the bound arrays in an cursor are the same size. But I think you may treat this attribute as number of array elements for a single statement execution. It is hard to implement array insert if we have to call cursor.exec whose iteration count is fixed value: max_array_size. This is because, once user has bound an array with size less than max_array_size and call cursor.exec, the iteration count will be greater than the bound array size. This will lead an error: "ORA-12899: value too large for column xxx." So in my opinion, the iteration count for cursor.exec should be equal to the size of the array user has bound, not the max_array_size. Then the problem comes to me: how can I know the exact size of the array user has bound at the calling of cursor.exec? I think this is a little bit difficult because OCI8::Cursor support method OCI8::Cursor[key] = val. I have managed to insert arrays through OCI8 with following assumption and modifications. (I don't mean this is a proper and reasonable approach to implement array DML for OCI8. I only want to address the difficult point more clear through this example.) *** Assumption Users should bind incoming array only by bind_param_array and should never use "Cursor[Key] = val" later to update the array binding. *** Modifications I added an instance variable "exec_iters" to OCI8::Cursor with default value "nil". When binding normal parameter through bind_param, we keep the exec_iters nil. Once user calls bind_param_array, we assign the array's size to the exec_iters. Since we have assumed that user should never call "Cursor[Key] = val" after bind_param_array, the exec_iters should exactly indicate the iteration count for cursor.exec. Another modification is, I added a new parameter "VALUE exec_iters" to function oci8_stmt_execute in stmt.c. The updated function oci8_stmt_execute in stmt.c is: static VALUE oci8_stmt_execute(VALUE self, VALUE exec_iters) { ....... if (oci8_get_ub2_attr(&stmt->base, OCI_ATTR_STMT_TYPE) == INT2FIX(OCI_STMT_SELECT)) { iters = 0; mode = OCI_DEFAULT; } else { if(!NIL_P(exec_iters)) { iters=NUM2INT(exec_iters); } else { iters = 1; } mode = svcctx->is_autocommit ? OCI_COMMIT_ON_SUCCESS : OCI_DEFAULT; } .... } Meanwhile, the updated function OCI8::Cursor#exec in oci8.rb is: def exec(*bindvars) bind_params(*bindvars) __execute(@exec_iters) ........ end I haven't fully tested the above modifications but I can insert String arrays without any problem. The "Cursor[Key] = val" doesn't cause problem with bind_param, because the iteration count for a normal insert is always 1. In case of array insert, the iteration count changes with calling of "Cursor[Key] = val". Obviously it is unrealistic to make assumption as what I mentioned above. So I am thinking other ways to get the exact size of the array user has bound at the calling of cursor.exec. Hope I am clear. Any ideas? From liming.lian at oracle.com Mon Dec 17 01:55:59 2007 From: liming.lian at oracle.com (Liming Lian) Date: Mon, 17 Dec 2007 14:55:59 +0800 Subject: [ruby-oci8-devel] Proposal for bind_param_array function prototype In-Reply-To: <475FD64C.3060100@oracle.com> References: <475E332C.5090305@oracle.com> <5d847bcd0712110837y6343b788r9acb504b944aa376@mail.gmail.com> <475FD64C.3060100@oracle.com> Message-ID: <47661D7F.2050100@oracle.com> Hi, I have a workaround for the problem of execution count. Following is my proposal. We can add a function OCI8::Cursor#set_batch_size to set the iteration count for statement execution. This function needs a parameter "size" like following: set_batch_size(size). When users want to bind array, they *must* call OCI8::Cursor#set_batch_size first. And they must also make sure the size of the binding array is equal to the size declared at set_batch_size. Otherwise, an error will be raised to prompt users incoming array with incorrect size. Besides set_batch_size, like the implementation of array DML in Perl DBD::ORACLE, we should also add another function OCI8::Cursor#exec_array. Bind_param_array and bind_param cannot be mixed in the same statement execution, and bind_param_array *must* be used with OCI8::Cursor#exec_array; using bind_param_array will have no effect for OCI8::Cursor#exec. Users can call set_batch_size many times during Cursor process. But every time at calling set_batch_size, if the new size isn't equal to the old one, all the binds are clear from the Cursor. In that situation, they have to re-bind all the parameter array. Look at following usage scenarios for demonstration. 1) Correct usage Cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") Cursor.set_batch_size(3) Cursor.bind_param_array(1, ["test1", "test2", "test3"]) Cursor.bind_param_array(2, [20, 21, 22]) Cursor.exec_array 2) Correct usage Cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") Cursor.set_batch_size(3) Cursor.bind_param_array(1, nil, String) Cursor.bind_param_array(2, nil, Fixnum) Cursor[1] = ["test1", "test2", "test3"] Cursor[2] = [20, 21, 22] Cursor.exec_array 3) Correct usage Cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") Cursor.set_batch_size(3) Cursor.bind_param_array(1, ["test1", "test2", "test3"]) Cursor.bind_param_array(2, [20, 21, 22]) Cursor.set_batch_size(2) Cursor.bind_param_array(1, ["test4", "test5"]) Cursor.bind_param_array(2, [23, 24]) Cursor.exec_array 4) Incorrect usage Cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") Cursor.bind_param_array(1, ["test1", "test2"]) Cursor.bind_param_array(2, [20, 21]) Cursor.exec_array == Error raised: should call set_batch_size first 5) Incorrect usage Cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") Cursor.set_batch_size(3) Cursor.bind_param_array(1, ["test1", "test2"]) Cursor.bind_param_array(2, [20, 21]) Cursor.exec_array == Error raised: incorrect incoming bound array size 6) Incorrect usage Cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") Cursor.set_batch_size(3) Cursor.bind_param_array(1, ["test1", "test2"]) Cursor.bind_param_array(2, [20, 21]) Cursor.exec == Error raised: should call exec_array to process array DML 7) Incorrect usage Cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") Cursor.set_batch_size(3) Cursor.bind_param_array(1, ["test1", "test2"]) Cursor.bind_param_array(2, [20, 21]) Cursor.set_batch_size(2) Cursor.exec_array == Error raised: OCIError: ORA-01008: not all variables bound == Reason: didn't re-bind the array after calling set_batch_size with a new batch size With the new function set_batch_size, the function bind_param_array doesn't need a parameter "max_array_size" any more. The parameter size passed to set_batch_size will be used as max_array_size internally. So the prototype of bind_param_array has been updated to: bind_param_array(key, var_array, type = nil, max_item_length = nil) Liming > I haven't fully tested the above modifications but I can insert String > arrays without any problem. The "Cursor[Key] = val" doesn't cause > problem with bind_param, because the iteration count for a normal insert > is always 1. In case of array insert, the iteration count changes with > calling of "Cursor[Key] = val". Obviously it is unrealistic to make > assumption as what I mentioned above. So I am thinking other ways to get > the exact size of the array user has bound at the calling of cursor.exec. > > Hope I am clear. Any ideas? > > _______________________________________________ > ruby-oci8-devel mailing list > ruby-oci8-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/ruby-oci8-devel > From liming.lian at oracle.com Mon Dec 17 20:44:24 2007 From: liming.lian at oracle.com (Liming Lian) Date: Tue, 18 Dec 2007 09:44:24 +0800 Subject: [ruby-oci8-devel] Proposal for bind_param_array function prototype In-Reply-To: <47661D7F.2050100@oracle.com> References: <475E332C.5090305@oracle.com> <5d847bcd0712110837y6343b788r9acb504b944aa376@mail.gmail.com> <475FD64C.3060100@oracle.com> <47661D7F.2050100@oracle.com> Message-ID: <476725F8.1080805@oracle.com> Sorry for the typo, the variable for cursor in code sample should be "cursor" instead of "Cursor". > 1) Correct usage > > Cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") > Should be: cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") Liming From kubo at jiubao.org Tue Dec 18 09:16:15 2007 From: kubo at jiubao.org (KUBO Takehiro) Date: Tue, 18 Dec 2007 23:16:15 +0900 Subject: [ruby-oci8-devel] Proposal for bind_param_array function prototype In-Reply-To: <47661D7F.2050100@oracle.com> (Liming Lian's message of "Mon\, 17 Dec 2007 14\:55\:59 +0800") References: <475E332C.5090305@oracle.com> <5d847bcd0712110837y6343b788r9acb504b944aa376@mail.gmail.com> <475FD64C.3060100@oracle.com> <47661D7F.2050100@oracle.com> Message-ID: <87lk7say1c.fsf_-_@lilliput.jiubao.org> Hi Liming, Sorry too late to reply you. Liming Lian writes: > Hi, > > I have a workaround for the problem of execution count. Following is my > proposal. > > We can add a function OCI8::Cursor#set_batch_size to set the iteration > count for statement execution. This function needs a parameter "size" > like following: set_batch_size(size). When users want to bind array, > they *must* call OCI8::Cursor#set_batch_size first. And they must also > make sure the size of the binding array is equal to the size declared at > set_batch_size. Otherwise, an error will be raised to prompt users > incoming array with incorrect size. Besides set_batch_size, like the > implementation of array DML in Perl DBD::ORACLE, we should also add > another function OCI8::Cursor#exec_array. Bind_param_array and > bind_param cannot be mixed in the same statement execution, and > bind_param_array *must* be used with OCI8::Cursor#exec_array; using > bind_param_array will have no effect for OCI8::Cursor#exec. > > Users can call set_batch_size many times during Cursor process. But > every time at calling set_batch_size, if the new size isn't equal to the > old one, all the binds are clear from the Cursor. In that situation, > they have to re-bind all the parameter array. How about the following proposal? --------------- 1. OCI8::Cursor#max_array_size = positive_number Users can call max_array_size= many times during Cursor process. But every time at calling max_array_size= all the binds are clear from the Cursor. In that situation, they have to re-bind all the parameter array. 2. OCI8::Cursor#bind_param_array(key, var_array, type = nil, max_item_length = nil) If batch_size= is not called, an error is raised. If the size of var_array is less than max_array_size, the rest values are NULL. If the size of var_arary is greater than max_array_size, an error is raised. 3. OCI8::Cursor[key] = var_array If bind_param_array(key, ...) is not called, an error is raised. If the size of var_array is less than max_array_size, the rest values are not changed. If the size of var_arary is greater than max_array_size, an error is raised. 4. OCI8::Cursor#exec_array(iteration_count = nil) If iteration_count is nil, max_array_size is used instead. If iteration_count > max_array_size, an error is raised. --------------- What I mind about your proposal is: > every time at calling set_batch_size, if the new size isn't equal to the > old one, all the binds are clear from the Cursor. In that situation, > they have to re-bind all the parameter array. I imagine what time users execute array DML. IMO, it is when inserting many rows from CSV files or from other database tables. Getting all rows and inserting them at once is a bad way. It takes large memory for many rows. I think fetching 100 rows or so and inserting them repeatedly will be most efficient. But the last execution's array size will not equal to the size of set_batch_size. Thus, I want to add a parameter iteration_count to exec_array. set_batch_size, bind_param_array and [key]= are changed as a result of a change in exec_array. > Look at following usage scenarios for demonstration. > > 1) Correct usage > > cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") > cursor.set_batch_size(3) > cursor.bind_param_array(1, ["test1", "test2", "test3"]) > cursor.bind_param_array(2, [20, 21, 22]) > cursor.exec_array 1) Correct usage cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") cursor.max_array_size = 3 cursor.bind_param_array(1, ["test1", "test2", "test3"]) cursor.bind_param_array(2, [20, 21, 22]) cursor.exec_array > 2) Correct usage > > cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") > cursor.set_batch_size(3) > cursor.bind_param_array(1, nil, String) > cursor.bind_param_array(2, nil, Fixnum) > cursor[1] = ["test1", "test2", "test3"] > cursor[2] = [20, 21, 22] > cursor.exec_array 2) Correct usage cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") cursor.max_array_size = 3 cursor.bind_param_array(1, nil, String) cursor.bind_param_array(2, nil, Fixnum) cursor[1] = ["test1", "test2", "test3"] cursor[2] = [20, 21, 22] cursor.exec_array > 3) Correct usage > > cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") > cursor.set_batch_size(3) > cursor.bind_param_array(1, ["test1", "test2", "test3"]) > cursor.bind_param_array(2, [20, 21, 22]) > cursor.set_batch_size(2) > cursor.bind_param_array(1, ["test4", "test5"]) > cursor.bind_param_array(2, [23, 24]) > cursor.exec_array 3) Correct usage cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") cursor.max_array_size = 3 cursor.bind_param_array(1, ["test1", "test2", "test3"]) cursor.bind_param_array(2, [20, 21, 22]) cursor.max_array_size = 2 cursor.bind_param_array(1, ["test4", "test5"]) cursor.bind_param_array(2, [23, 24]) cursor.exec_array > 4) Incorrect usage > > cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") > cursor.bind_param_array(1, ["test1", "test2"]) > cursor.bind_param_array(2, [20, 21]) > cursor.exec_array > > == Error raised: should call set_batch_size first 4) Incorrect usage cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") cursor.bind_param_array(1, ["test1", "test2"]) cursor.bind_param_array(2, [20, 21]) cursor.exec_array == Error raised: should call max_array_size first > 5) Incorrect usage > > cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") > cursor.set_batch_size(3) > cursor.bind_param_array(1, ["test1", "test2"]) > cursor.bind_param_array(2, [20, 21]) > cursor.exec_array > > == Error raised: incorrect incoming bound array size 5) Correct usage (different with yours) cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") cursor.max_array_size = 3 cursor.bind_param_array(1, ["test1", "test2"]) cursor.bind_param_array(2, [20, 21]) cursor.exec_array(2) > 6) Incorrect usage > > cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") > cursor.set_batch_size(3) > cursor.bind_param_array(1, ["test1", "test2"]) > cursor.bind_param_array(2, [20, 21]) > cursor.exec > > == Error raised: should call exec_array to process array DML > 6) Incorrect usage cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") cursor.max_array_size = 3 cursor.bind_param_array(1, ["test1", "test2"]) cursor.bind_param_array(2, [20, 21]) cursor.exec == Error raised: should call exec_array to process array DML > 7) Incorrect usage > > cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") > cursor.set_batch_size(3) > cursor.bind_param_array(1, ["test1", "test2"]) > cursor.bind_param_array(2, [20, 21]) > cursor.set_batch_size(2) > cursor.exec_array > > == Error raised: OCIError: ORA-01008: not all variables bound > == Reason: didn't re-bind the array after calling set_batch_size with a > new batch size > 7) Incorrect usage cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") cursor.max_array_size = 3 cursor.bind_param_array(1, ["test1", "test2"]) cursor.bind_param_array(2, [20, 21]) cursor.max_array_size = 2 cursor.exec_array == Error raised: OCIError: ORA-01008: not all variables bound == Reason: didn't re-bind the array after calling set_batch_size with a new batch size -- KUBO Takehiro From liming.lian at oracle.com Wed Dec 19 01:29:41 2007 From: liming.lian at oracle.com (Liming Lian) Date: Wed, 19 Dec 2007 14:29:41 +0800 Subject: [ruby-oci8-devel] Proposal for bind_param_array function prototype In-Reply-To: <87lk7say1c.fsf_-_@lilliput.jiubao.org> References: <475E332C.5090305@oracle.com> <5d847bcd0712110837y6343b788r9acb504b944aa376@mail.gmail.com> <475FD64C.3060100@oracle.com> <47661D7F.2050100@oracle.com> <87lk7say1c.fsf_-_@lilliput.jiubao.org> Message-ID: <4768BA55.9020408@oracle.com> Hi Kubo, > 1. OCI8::Cursor#max_array_size = positive_number > > Users can call max_array_size= many times during Cursor process. But > every time at calling max_array_size= all the binds are clear from the > Cursor. In that situation, they have to re-bind all the parameter array. Do you mean, the binds should be clear even the new value of max_array_size is equal to the old one? Take a look at following example: cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") cursor.max_array_size = 3 cursor.bind_param_array(1, ["test1", "test2", "test3"]) cursor.bind_param_array(2, [20, 21, 22]) cursor.max_array_size = 3 cursor.exec_array IMO, this should be a correct usage. Users don't need to re-bind the parameter since the max_array_size doesn't change. > 2. OCI8::Cursor#bind_param_array(key, var_array, type = nil, max_item_length = nil) > > If batch_size= is not called, an error is raised. I guess you mean "max_array_size=" here, right? > If the size of var_array is less than max_array_size, the rest values > are NULL. > I am considering if it is proper to insert nil values that users may not want. Example: cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") cursor.max_array_size = 3 cursor.bind_param_array(1, ["test1"]) cursor.bind_param_array(2, [20]) cursor.exec_array The rows inserted to "test_table" are row1: test1, 20 row2: NULL, NULL row3: NULL, NULL I am not sure if users would be happy on those rows full of useless NULL values. > 4. OCI8::Cursor#exec_array(iteration_count = nil) > > If iteration_count is nil, max_array_size is used instead. > If iteration_count > max_array_size, an error is raised. > > Thanks kubo, here you gave me the answer to my previous worry. If users don't want to insert rows with full NULL values, they can add a parameter iteration_count to exec_array. Liming From liming.lian at oracle.com Thu Dec 20 03:31:35 2007 From: liming.lian at oracle.com (Liming Lian) Date: Thu, 20 Dec 2007 16:31:35 +0800 Subject: [ruby-oci8-devel] Clear binds Message-ID: <476A2867.2020006@oracle.com> Hi, As we discussed at the proposal, all existing binds should be clear from cursor when new value of max_array_size is set. I have implemented this by adding functions to stmt.c. I am not sure if I have fully understood the mechanism of how Ruby-OCI8 works. So I want to hear any comments on my implementation. Following is my code added to stmt.c: static ID id_empty; static ID id_clear; static VALUE clear_binds_iterator_proc(VALUE pair, VALUE arg, VALUE self) { VALUE value = RARRAY(pair)->ptr[1]; if(!NIL_P(value)) { oci8_base_free((oci8_base_t*)oci8_get_bind(value)); } return self; } static VALUE oci8_stmt_clear_binds(VALUE self) { oci8_stmt_t *stmt = DATA_PTR(self); if(!RTEST(rb_funcall(stmt->binds, id_empty, 0))) { rb_iterate(rb_each, stmt->binds, clear_binds_iterator_proc, (VALUE) NULL); rb_funcall(stmt->binds,id_clear,0); } return self; } void Init_oci8_stmt(VALUE cOCI8) { ........ id_empty = rb_intern("empty?"); id_clear = rb_intern("clear"); ........ rb_define_private_method(cOCIStmt, "__clearBinds", oci8_stmt_clear_binds, 0); ........ } From kubo at jiubao.org Thu Dec 20 11:33:52 2007 From: kubo at jiubao.org (KUBO Takehiro) Date: Fri, 21 Dec 2007 01:33:52 +0900 Subject: [ruby-oci8-devel] Proposal for bind_param_array function prototype In-Reply-To: <4768BA55.9020408@oracle.com> (Liming Lian's message of "Wed\, 19 Dec 2007 14\:29\:41 +0800") References: <475E332C.5090305@oracle.com> <5d847bcd0712110837y6343b788r9acb504b944aa376@mail.gmail.com> <475FD64C.3060100@oracle.com> <47661D7F.2050100@oracle.com> <87lk7say1c.fsf_-_@lilliput.jiubao.org> <4768BA55.9020408@oracle.com> Message-ID: <87lk7pqqa7.fsf@lilliput.jiubao.org> Hi Liming, Liming Lian writes: > Hi Kubo, >> 1. OCI8::Cursor#max_array_size = positive_number >> >> Users can call max_array_size= many times during Cursor process. But >> every time at calling max_array_size= all the binds are clear from the >> Cursor. In that situation, they have to re-bind all the parameter array. > > Do you mean, the binds should be clear even the new value of > max_array_size is equal to the old one? Yes. > Take a look at following example: > > cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") > cursor.max_array_size = 3 > cursor.bind_param_array(1, ["test1", "test2", "test3"]) > cursor.bind_param_array(2, [20, 21, 22]) > cursor.max_array_size = 3 > cursor.exec_array > > IMO, this should be a correct usage. Users don't need to re-bind the > parameter since the max_array_size doesn't change. In that case, users don't need to call max_array_size=. I can't imagine the merit that users set same size without re-binding. If it is rare and users have a workaround, I prefer to cut off specs for special cases. IMO, we can disallow users to re-set max_array_size. Once the size is set, setting again raises an error. If users want to change the size, they need to create a new cursor for a workaround. But I'm not sure that re-setting of max_array_size is rare. So I don't presist in this idea. >> 2. OCI8::Cursor#bind_param_array(key, var_array, type = nil, max_item_length = nil) >> >> If batch_size= is not called, an error is raised. > > I guess you mean "max_array_size=" here, right? > >> If the size of var_array is less than max_array_size, the rest values >> are NULL. >> > > I am considering if it is proper to insert nil values that users may not > want. Example: > > cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") > cursor.max_array_size = 3 > cursor.bind_param_array(1, ["test1"]) > cursor.bind_param_array(2, [20]) > cursor.exec_array > > The rows inserted to "test_table" are > row1: test1, 20 > row2: NULL, NULL > row3: NULL, NULL > > I am not sure if users would be happy on those rows full of useless NULL > values. > >> 4. OCI8::Cursor#exec_array(iteration_count = nil) >> >> If iteration_count is nil, max_array_size is used instead. >> If iteration_count > max_array_size, an error is raised. >> >> > Thanks kubo, here you gave me the answer to my previous worry. If users > don't want to insert rows with full NULL values, they can add a > parameter iteration_count to exec_array. Yes. Yet another idea is the iteration_count is the size of bind values. cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") cursor.max_array_size = 3 cursor.bind_param_array(1, ["test1"]) cursor.bind_param_array(2, [20]) cursor.exec_array # iteration_count is 1 because parameter's # array sizes are all 1. cursor.bind_param_array(1, ["test2", "test3"]) cursor.bind_param_array(2, [20, 30]) cursor.exec_array # iteration_count is 2 because parameter's # array sizes are all 2. cursor.bind_param_array(1, ["test4", "test5"]) cursor.bind_param_array(2, [40]) cursor.exec_array # raise an error because array sizes are not same. This may be usable than my previous idea. -- KUBO Takehiro From kubo at jiubao.org Thu Dec 20 18:24:23 2007 From: kubo at jiubao.org (KUBO Takehiro) Date: Fri, 21 Dec 2007 08:24:23 +0900 Subject: [ruby-oci8-devel] Clear binds In-Reply-To: <476A2867.2020006@oracle.com> (Liming Lian's message of "Thu\, 20 Dec 2007 16\:31\:35 +0800") References: <476A2867.2020006@oracle.com> Message-ID: <87d4t1q7a0.fsf@lilliput.jiubao.org> Hi Liming, Liming Lian writes: > Hi, > > As we discussed at the proposal, all existing binds should be clear from > cursor when new value of max_array_size is set. I have implemented this > by adding functions to stmt.c. I am not sure if I have fully understood > the mechanism of how Ruby-OCI8 works. So I want to hear any comments on > my implementation. I'll check your C code tomorrow. I have no time now. From liming.lian at oracle.com Fri Dec 21 01:55:56 2007 From: liming.lian at oracle.com (Liming Lian) Date: Fri, 21 Dec 2007 14:55:56 +0800 Subject: [ruby-oci8-devel] Clear binds In-Reply-To: <87d4t1q7a0.fsf@lilliput.jiubao.org> References: <476A2867.2020006@oracle.com> <87d4t1q7a0.fsf@lilliput.jiubao.org> Message-ID: <476B637C.7020606@oracle.com> Hi Kubo, > > I'll check your C code tomorrow. I have no time now. > _______________________________________________ Thanks in advance! And please take it easy, no hurry at all. Liming From shiwei.zhang at oracle.com Fri Dec 21 04:29:52 2007 From: shiwei.zhang at oracle.com (shiwei zhang) Date: Fri, 21 Dec 2007 17:29:52 +0800 Subject: [ruby-oci8-devel] propose to implement pooling technology for Ruby-OCI8 In-Reply-To: <476A3101.2030103@oracle.com> References: <47622312.8070500@oracle.com> <87tzmgj71k.fsf@lilliput.jiubao.org> <476A3101.2030103@oracle.com> Message-ID: <476B8790.4080409@oracle.com> Hi, shiwei zhang wrote: > Hi Kubo, > > Yes, it's better to make our discussion public on this ML. > > KUBO Takehiro wrote: >> Hi shiwei, >> >> Sorry too late to reply you. >> >> shiwei zhang writes: >> >> >>> How are you? This is Shiwei from Oracle Asia R&D Center, I hope you >>> still remember me:) >>> >> >> Yes, of course. >> >> >>> Last time I submitted through Christ Jones to you a patch about Easy >>> Connection String in Ruby-OCI8. Now I want to continue contributing >>> some work for Ruby-OCI8, in order to make it further satisfy Ruby >>> users. >>> >>> These days I did some tentative research about pooling technology for >>> Ruby-OCI8. Database pooling is a top-notch technology that can improve >>> the performance of application programs, especially for those >>> large-scale ones handling many DB requests. >>> >> >> In general, agree. Connection pooling is good for web applications. >> PHP offers persistent connections for the purpose. >> http://php.net/manual/en/features.persistent-connections.php >> >> IMO, most ruby's web applications are built on rails. But as for >> rails, it is another story. Rails uses only one connection for all >> requests. Pooling will not improve the performance. >> >> BTW go on. >> > Agree. Haha, ruby shouldn't be always limited to Rails, and even > shouldn't always limited to web application. Furthermore Rails might > change its principle to hold more connections in its pool. :-) >>> There are two kinds of pooling at the client side: one is connection >>> pooling, which can create & destroy stateful sessions; the other is >>> session pooling, which can reuse stateless sessions. JDBC has supportd >>> both connection pooling and session pooling, it has classes >>> OracleOCIConnectionPool and OracleConnectionPoolDataSource. Now I >>> propose to implement silimar connection pooling and session pooling >>> for Ruby-OCI8, do you agree? What's your opinion? >>> >> >> Surely. >> > Great, so I'd like to carry on, both connection pooling and session > pooling. >> >>> I've read the related codes in RubyOCI8_1.0.0-rc3, I think we can work >>> it out by either of the two methods: >>> >> >> I want to add new features to ruby-oci8 svn trunk, not to ruby-oci8 >> 1.0 seriese. Svn trunk will be ruby-oci8 2.0. And its internal >> structure is thoroughly changed. >> >> The svn trunk code can be checked out with the following command. >> >> svn checkout http://ruby-oci8.rubyforge.org/svn/trunk/ruby-oci8 >> or >> svn checkout svn://rubyforge.org/var/svn/ruby-oci8/trunk/ruby-oci8 >> > This morning I downloaded the SVN source codes and glimpsed at it. As > you said the internal structure is changed. Actually I wrote some > codes locally on my pc about pooling for ruby-oci8_1.0, and I think > it's fine for me to move to ruby-oci8_2.0. You transferred more codes > from ruby side to C side in the new version, which I think makes the > logic easier to decide & process the connection parameters, which is good. > BTW, I have a small question: you use "OCISessionBegin()" for > T_EXPLICIT (external credential or OCI_SYSDBA, OCI_SYSOPER); and use > "OCILogon" for T_IMPLICIT (OCI_CRED_RDBMS). *Why don't you use > "OCISessionBegin()" for both T_EXPLICIT and T_IMPLICIT?* I think > "OCISessionBegin()" can work for both of the two conditions, do you > have other concerns about this? > I'm sure connection pooling and session pooling support T_IMPLICIT, > but I'm not sure whether they also support T_EXPLICIT. And I'm not > sure whether it's necessary for T_EXPLICIT to have connection pooling > and session pooling, since you know, T_EXPLICIT is used a lot less > frequently. I need to investigate more on this. Today I wrote a program to confirm that the pooling doesn't support OCI_SYSDBA or OCI_SYSOPER, it returns "ORA-24300: bad value for mode" when using the mode OCI_SYSDBA or OCI_SYSOPER. I will make sure whether OCI_CRED_EXT works in pooling soon. >>> 1. Write sub-classes inherited from the current Ruby class OCI8; and >>> write some C functions to execute the corresponding pooling work. >>> 2. Modify the current Ruby class OCI; also write some C functions to >>> execute the corresponding pooling work. >>> >> >> Can you show me the spec and usage example? >> for example: >> >> # connection pool >> pool = OCI8::ConnectionPool.new(username, password, tns_name, conn_min, conn_max, conn_incr) >> conn = OCI8.new(pool) >> >> # session pool >> pool = OCI8::SessionPool.new(username, password, tns_name, sess_min, sess_max) >> conn = OCI8.new(pool) >> > For now I think: > > # connection pool > pool = OCI8::ConnectionPool.new(username, password, tns_name, > conn_min, conn_max, conn_incr) > conn = OCI8.new(pool, appusername, apppassword) > or > pool = OCI8::ConnectionPool.new(username, password, tns_name, > conn_min, conn_max, conn_incr) > conn = OCI8.new($poolname, appusername, apppassword) > > # session pool > pool = OCI8::SessionPool.new(username, password, tns_name, sess_min, > sess_max) > conn = OCI8.new(pool, appusername, apppassword) > or > pool = OCI8::SessionPool.new(username, password, tns_name, sess_min, > sess_max) > conn = OCI8.new($poolname, appusername, apppassword) > > Notes: > username is used to create the pool, it's the implicit user. > appusername is used to create/get a session from the pool. appusername > could be different with username. > $poolname is generate from "OCI8::ConnectionPool.new()" or > "OCI8::ConnectionPool.new()". If we've gotten the value of $poolname, > it's enough for "OCI8.new()" to create/hand out a session for appusername. >> If you can, please join to ruby-oci8-devel to make our discussion public. >> >> ruby-oci8-devel ML: http://rubyforge.org/mail/?group_id=256 >> >> Thanks & Best Regards, Shiwei -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ruby-oci8-devel/attachments/20071221/79b387c2/attachment.html From shiwei.zhang at oracle.com Thu Dec 20 04:08:17 2007 From: shiwei.zhang at oracle.com (shiwei zhang) Date: Thu, 20 Dec 2007 17:08:17 +0800 Subject: [ruby-oci8-devel] propose to implement pooling technology for Ruby-OCI8 In-Reply-To: <87tzmgj71k.fsf@lilliput.jiubao.org> References: <47622312.8070500@oracle.com> <87tzmgj71k.fsf@lilliput.jiubao.org> Message-ID: <476A3101.2030103@oracle.com> Hi Kubo, Yes, it's better to make our discussion public on this ML. KUBO Takehiro wrote: > Hi shiwei, > > Sorry too late to reply you. > > shiwei zhang writes: > > >> How are you? This is Shiwei from Oracle Asia R&D Center, I hope you >> still remember me:) >> > > Yes, of course. > > >> Last time I submitted through Christ Jones to you a patch about Easy >> Connection String in Ruby-OCI8. Now I want to continue contributing >> some work for Ruby-OCI8, in order to make it further satisfy Ruby >> users. >> >> These days I did some tentative research about pooling technology for >> Ruby-OCI8. Database pooling is a top-notch technology that can improve >> the performance of application programs, especially for those >> large-scale ones handling many DB requests. >> > > In general, agree. Connection pooling is good for web applications. > PHP offers persistent connections for the purpose. > http://php.net/manual/en/features.persistent-connections.php > > IMO, most ruby's web applications are built on rails. But as for > rails, it is another story. Rails uses only one connection for all > requests. Pooling will not improve the performance. > > BTW go on. > Agree. Haha, ruby shouldn't be always limited to Rails, and even shouldn't always limited to web application. Furthermore Rails might change its principle to hold more connections in its pool. :-) >> There are two kinds of pooling at the client side: one is connection >> pooling, which can create & destroy stateful sessions; the other is >> session pooling, which can reuse stateless sessions. JDBC has supportd >> both connection pooling and session pooling, it has classes >> OracleOCIConnectionPool and OracleConnectionPoolDataSource. Now I >> propose to implement silimar connection pooling and session pooling >> for Ruby-OCI8, do you agree? What's your opinion? >> > > Surely. > Great, so I'd like to carry on, both connection pooling and session pooling. > >> I've read the related codes in RubyOCI8_1.0.0-rc3, I think we can work >> it out by either of the two methods: >> > > I want to add new features to ruby-oci8 svn trunk, not to ruby-oci8 > 1.0 seriese. Svn trunk will be ruby-oci8 2.0. And its internal > structure is thoroughly changed. > > The svn trunk code can be checked out with the following command. > > svn checkout http://ruby-oci8.rubyforge.org/svn/trunk/ruby-oci8 > or > svn checkout svn://rubyforge.org/var/svn/ruby-oci8/trunk/ruby-oci8 > This morning I downloaded the SVN source codes and glimpsed at it. As you said the internal structure is changed. Actually I wrote some codes locally on my pc about pooling for ruby-oci8_1.0, and I think it's fine for me to move to ruby-oci8_2.0. You transferred more codes from ruby side to C side in the new version, which I think makes the logic easier to decide & process the connection parameters, which is good. BTW, I have a small question: you use "OCISessionBegin()" for T_EXPLICIT (external credential or OCI_SYSDBA, OCI_SYSOPER); and use "OCILogon" for T_IMPLICIT (OCI_CRED_RDBMS). *Why don't you use "OCISessionBegin()" for both T_EXPLICIT and T_IMPLICIT?* I think "OCISessionBegin()" can work for both of the two conditions, do you have other concerns about this? I'm sure connection pooling and session pooling support T_IMPLICIT, but I'm not sure whether they also support T_EXPLICIT. And I'm not sure whether it's necessary for T_EXPLICIT to have connection pooling and session pooling, since you know, T_EXPLICIT is used a lot less frequently. I need to investigate more on this. >> 1. Write sub-classes inherited from the current Ruby class OCI8; and >> write some C functions to execute the corresponding pooling work. >> 2. Modify the current Ruby class OCI; also write some C functions to >> execute the corresponding pooling work. >> > > Can you show me the spec and usage example? > for example: > > # connection pool > pool = OCI8::ConnectionPool.new(username, password, tns_name, conn_min, conn_max, conn_incr) > conn = OCI8.new(pool) > > # session pool > pool = OCI8::SessionPool.new(username, password, tns_name, sess_min, sess_max) > conn = OCI8.new(pool) > For now I think: # connection pool pool = OCI8::ConnectionPool.new(username, password, tns_name, conn_min, conn_max, conn_incr) conn = OCI8.new(pool, appusername, apppassword) or pool = OCI8::ConnectionPool.new(username, password, tns_name, conn_min, conn_max, conn_incr) conn = OCI8.new($poolname, appusername, apppassword) # session pool pool = OCI8::SessionPool.new(username, password, tns_name, sess_min, sess_max) conn = OCI8.new(pool, appusername, apppassword) or pool = OCI8::SessionPool.new(username, password, tns_name, sess_min, sess_max) conn = OCI8.new($poolname, appusername, apppassword) Notes: username is used to create the pool, it's the implicit user. appusername is used to create/get a session from the pool. appusername could be different with username. $poolname is generate from "OCI8::ConnectionPool.new()" or "OCI8::ConnectionPool.new()". If we've gotten the value of $poolname, it's enough for "OCI8.new()" to create/hand out a session for appusername. > If you can, please join to ruby-oci8-devel to make our discussion public. > > ruby-oci8-devel ML: http://rubyforge.org/mail/?group_id=256 > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ruby-oci8-devel/attachments/20071220/38e3bfd8/attachment-0001.html From kubo at jiubao.org Mon Dec 24 08:23:23 2007 From: kubo at jiubao.org (KUBO Takehiro) Date: Mon, 24 Dec 2007 22:23:23 +0900 Subject: [ruby-oci8-devel] Clear binds In-Reply-To: <476A2867.2020006@oracle.com> (Liming Lian's message of "Thu\, 20 Dec 2007 16\:31\:35 +0800") References: <476A2867.2020006@oracle.com> Message-ID: <87abo01b1w.fsf@lilliput.jiubao.org> Hi, Liming Lian writes: > Hi, > > As we discussed at the proposal, all existing binds should be clear from > cursor when new value of max_array_size is set. I have implemented this > by adding functions to stmt.c. I am not sure if I have fully understood > the mechanism of how Ruby-OCI8 works. So I want to hear any comments on > my implementation. I wrote indent style in C used in ruby-oci8. http://ruby-oci8.rubyforge.org/en/dev_CodingStyle.html I won't check indent style in this mail. > Following is my code added to stmt.c: > > static ID id_empty; > static ID id_clear; > > static VALUE clear_binds_iterator_proc(VALUE pair, VALUE arg, VALUE self) The third argument of rb_iterate() accepts two arguments. Delete last one. > { > VALUE value = RARRAY(pair)->ptr[1]; Use PARRAY_PTR(pair) instead of RARRAY(pair)->ptr. This macro was added in ruby 1.8.6. Your code works fine with ruby 1.9 also. But RArray's internal structure may be changed to improve performace in future, as RString's strcuture was changed in ruby 1.9. > if(!NIL_P(value)) { > oci8_base_free((oci8_base_t*)oci8_get_bind(value)); > } > > return self; > } > > > static VALUE oci8_stmt_clear_binds(VALUE self) > { > oci8_stmt_t *stmt = DATA_PTR(self); > > if(!RTEST(rb_funcall(stmt->binds, id_empty, 0))) > { > rb_iterate(rb_each, stmt->binds, clear_binds_iterator_proc, (VALUE) NULL); stmt->binds is a Hash object. It is better to use each_value instead of each. Add following code and use each_value instead of rb_each. static VALUE each_value(VALUE obj) { return rb_funcall(obj, id_each_value, 0); } > rb_funcall(stmt->binds,id_clear,0); > } > > return self; > } > > void Init_oci8_stmt(VALUE cOCI8) > { > ........ > id_empty = rb_intern("empty?"); Use id_empty_p instead of id_empty. Ruby's C source code uses '_p' for methods which end with '?'. > id_clear = rb_intern("clear"); > ........ > rb_define_private_method(cOCIStmt, "__clearBinds", oci8_stmt_clear_binds, 0); Use __clear_binds instead of __clearBinds. I used CamelCase method names in ruby-oci8 1.0. But now I obey ruby's convention. All method words are lower case and concatenated by '_'. Though, starting '__' is not in ruby's convention.... > ........ > } Here is my code based on yours. static ID id_empty_p; static ID id_clear; static ID id_each_value; static VALUE each_value(VALUE obj) { return rb_funcall(obj, id_each_value, 0); } static VALUE clear_binds_iterator_proc(VALUE val, VALUE unused) { if (!NIL_P(val)) { oci8_base_free((oci8_base_t*)oci8_get_bind(val)); } return Qnil; } static VALUE oci8_stmt_clear_binds(VALUE self) { oci8_stmt_t *stmt = DATA_PTR(self); if (!RTEST(rb_funcall(stmt->binds, id_empty_p, 0))) { rb_iterate(each_value, stmt->binds, clear_binds_iterator_proc, Qnil); rb_funcall(stmt->binds,id_clear,0); } return self; } void Init_oci8_stmt(VALUE cOCI8) { ........ id_empty_p = rb_intern("empty?"); id_clear = rb_intern("clear"); id_each_value = rb_intern("each_value"); ........ rb_define_private_method(cOCIStmt, "__clear_binds", oci8_stmt_clear_binds, 0); ........ } -- KUBO Takehiro From kubo at jiubao.org Mon Dec 24 09:35:44 2007 From: kubo at jiubao.org (KUBO Takehiro) Date: Mon, 24 Dec 2007 23:35:44 +0900 Subject: [ruby-oci8-devel] propose to implement pooling technology for Ruby-OCI8 References: <47622312.8070500@oracle.com> <87tzmgj71k.fsf@lilliput.jiubao.org> <476A3101.2030103@oracle.com> <476B8790.4080409@oracle.com> Message-ID: <873ats17pb.fsf@lilliput.jiubao.org> Hi, shiwei zhang writes: >> BTW, I have a small question: you use "OCISessionBegin()" for >> T_EXPLICIT (external credential or OCI_SYSDBA, OCI_SYSOPER); and use >> "OCILogon" for T_IMPLICIT (OCI_CRED_RDBMS). *Why don't you use >> "OCISessionBegin()" for both T_EXPLICIT and T_IMPLICIT?* I think >> "OCISessionBegin()" can work for both of the two conditions, do you >> have other concerns about this? I agree in theory. I had used OCISessionBegin() for both before 0.1.13. But I received a mail it sometimes causes a segmentation faults in 10 logons. Its environment was as follows. ruby 1.8.2 ruby-oci8 0.1.13 Oracle Client 8.1.7 (Windows) Oracle Server 8.0.5 This would be really rare. But there was no workaround. I changed to use OCILogon for non-privilege logons. If the code become too complex by your changes, you can merge them to OCISessionBegin(). Because it might be caused by a bug in ruby-oci8 and its internal structure was changed. The same problem may not occur in current implementation. >> For now I think: >> >> # connection pool >> pool = OCI8::ConnectionPool.new(username, password, tns_name, conn_min, conn_max, conn_incr) >> conn = OCI8.new(pool, appusername, apppassword) >> or >> pool = OCI8::ConnectionPool.new(username, password, tns_name, conn_min, conn_max, conn_incr) >> conn = OCI8.new($poolname, appusername, apppassword) >> >> # session pool >> pool = OCI8::SessionPool.new(username, password, tns_name, sess_min, sess_max) >> conn = OCI8.new(pool, appusername, apppassword) >> or >> pool = OCI8::SessionPool.new(username, password, tns_name, sess_min, sess_max) >> conn = OCI8.new($poolname, appusername, apppassword) >> >> Notes: >> username is used to create the pool, it's the implicit user. >> appusername is used to create/get a session from the pool. appusername >> could be different with username. Another proposal: pool = OCI8::ConnectionPool.new(username, password, tns_name, conn_min, conn_max, conn_incr) conn = OCI8.new(appusername, apppassword, pool) Change the current spec to: OCI8.new(username, password, dbname_or_pool = nil, privilege = nil) Argument values username and password are appusername and apppassword respectively if the third argument is a connection pool. One question: Are appusername and apppassword required mandatory or optional? >> $poolname is generate from "OCI8::ConnectionPool.new()" or >> "OCI8::ConnectionPool.new()". If we've gotten the value of $poolname, >> it's enough for "OCI8.new()" to create/hand out a session for appusername. What is the data type of $poolname? If it is a string, we cannot distinguish a logon with dbname from a login using connection pool. If is is a connection pool object, when is the pool destroyed? $poolname is set by a side effect of OCI::ConnectionPool.new(). IMO, it is not straightforward. -- KUBO Takehiro From liming.lian at oracle.com Mon Dec 24 22:13:58 2007 From: liming.lian at oracle.com (Liming Lian) Date: Tue, 25 Dec 2007 11:13:58 +0800 Subject: [ruby-oci8-devel] Clear binds In-Reply-To: <87abo01b1w.fsf@lilliput.jiubao.org> References: <476A2867.2020006@oracle.com> <87abo01b1w.fsf@lilliput.jiubao.org> Message-ID: <47707576.7020004@oracle.com> Thanks kubo for your valuable comments. I notice two wiki pages for developers on ruby-oci8 project website. Really helpful~ Merry Christmas and Happy New Year! Liming > Hi, > > Liming Lian writes: > > >> Hi, >> >> As we discussed at the proposal, all existing binds should be clear from >> cursor when new value of max_array_size is set. I have implemented this >> by adding functions to stmt.c. I am not sure if I have fully understood >> the mechanism of how Ruby-OCI8 works. So I want to hear any comments on >> my implementation. >> > > I wrote indent style in C used in ruby-oci8. > http://ruby-oci8.rubyforge.org/en/dev_CodingStyle.html > > I won't check indent style in this mail. > > >> Following is my code added to stmt.c: >> >> static ID id_empty; >> static ID id_clear; >> >> static VALUE clear_binds_iterator_proc(VALUE pair, VALUE arg, VALUE self) >> > > The third argument of rb_iterate() accepts two arguments. Delete last one. > > >> { >> VALUE value = RARRAY(pair)->ptr[1]; >> > > Use PARRAY_PTR(pair) instead of RARRAY(pair)->ptr. > This macro was added in ruby 1.8.6. > Your code works fine with ruby 1.9 also. But RArray's internal > structure may be changed to improve performace in future, as RString's > strcuture was changed in ruby 1.9. > > >> if(!NIL_P(value)) { >> oci8_base_free((oci8_base_t*)oci8_get_bind(value)); >> } >> >> return self; >> } >> >> >> static VALUE oci8_stmt_clear_binds(VALUE self) >> { >> oci8_stmt_t *stmt = DATA_PTR(self); >> >> if(!RTEST(rb_funcall(stmt->binds, id_empty, 0))) >> { >> rb_iterate(rb_each, stmt->binds, clear_binds_iterator_proc, (VALUE) NULL); >> > > stmt->binds is a Hash object. It is better to use each_value instead > of each. > > Add following code and use each_value instead of rb_each. > static VALUE each_value(VALUE obj) > { > return rb_funcall(obj, id_each_value, 0); > } > > >> rb_funcall(stmt->binds,id_clear,0); >> } >> >> return self; >> } >> >> void Init_oci8_stmt(VALUE cOCI8) >> { >> ........ >> id_empty = rb_intern("empty?"); >> > > Use id_empty_p instead of id_empty. > Ruby's C source code uses '_p' for methods which end with '?'. > > >> id_clear = rb_intern("clear"); >> ........ >> rb_define_private_method(cOCIStmt, "__clearBinds", oci8_stmt_clear_binds, 0); >> > > Use __clear_binds instead of __clearBinds. > I used CamelCase method names in ruby-oci8 1.0. > But now I obey ruby's convention. All method words are lower case and > concatenated by '_'. > Though, starting '__' is not in ruby's convention.... > > >> ........ >> } >> > > > Here is my code based on yours. > > static ID id_empty_p; > static ID id_clear; > static ID id_each_value; > > static VALUE each_value(VALUE obj) > { > return rb_funcall(obj, id_each_value, 0); > } > > static VALUE clear_binds_iterator_proc(VALUE val, VALUE unused) > { > if (!NIL_P(val)) { > oci8_base_free((oci8_base_t*)oci8_get_bind(val)); > } > return Qnil; > } > > static VALUE oci8_stmt_clear_binds(VALUE self) > { > oci8_stmt_t *stmt = DATA_PTR(self); > > if (!RTEST(rb_funcall(stmt->binds, id_empty_p, 0))) { > rb_iterate(each_value, stmt->binds, clear_binds_iterator_proc, Qnil); > rb_funcall(stmt->binds,id_clear,0); > } > return self; > } > > void Init_oci8_stmt(VALUE cOCI8) > { > ........ > id_empty_p = rb_intern("empty?"); > id_clear = rb_intern("clear"); > id_each_value = rb_intern("each_value"); > ........ > rb_define_private_method(cOCIStmt, "__clear_binds", oci8_stmt_clear_binds, 0); > ........ > } > > From liming.lian at oracle.com Thu Dec 27 02:08:06 2007 From: liming.lian at oracle.com (Liming Lian) Date: Thu, 27 Dec 2007 15:08:06 +0800 Subject: [ruby-oci8-devel] Proposal for bind_param_array function prototype In-Reply-To: <87lk7pqqa7.fsf@lilliput.jiubao.org> References: <475E332C.5090305@oracle.com> <5d847bcd0712110837y6343b788r9acb504b944aa376@mail.gmail.com> <475FD64C.3060100@oracle.com> <47661D7F.2050100@oracle.com> <87lk7say1c.fsf_-_@lilliput.jiubao.org> <4768BA55.9020408@oracle.com> <87lk7pqqa7.fsf@lilliput.jiubao.org> Message-ID: <47734F56.4080000@oracle.com> Hi Kubo, > Yet another idea is the iteration_count is the size of bind values. > > cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") > cursor.max_array_size = 3 > cursor.bind_param_array(1, ["test1"]) > cursor.bind_param_array(2, [20]) > cursor.exec_array # iteration_count is 1 because parameter's > # array sizes are all 1. > cursor.bind_param_array(1, ["test2", "test3"]) > cursor.bind_param_array(2, [20, 30]) > cursor.exec_array # iteration_count is 2 because parameter's > # array sizes are all 2. > Agree. > cursor.bind_param_array(1, ["test4", "test5"]) > cursor.bind_param_array(2, [40]) > cursor.exec_array # raise an error because array sizes are not same. > > This may be usable than my previous idea. > > For this case, do you think it is better to use following strategic: If all the sizes of bound arrays are less than max_array_size, we will use the largest one as the iteration_count. It is not required that all the arrays are the same size. "nil" elements will be appended to those shorter arrays to make them the size of iteration_count. Example: cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") cursor.max_array_size = 3 cursor.bind_param_array(1, ["test4", "test5"]) cursor.bind_param_array(2, [40]) cursor.exec_array # iteration_count is 2 because the largest parameter's # array size are 2. The rows inserted into the db are: row1: test4 40 row2: test5 nil From liming.lian at oracle.com Thu Dec 27 21:14:41 2007 From: liming.lian at oracle.com (Liming Lian) Date: Fri, 28 Dec 2007 10:14:41 +0800 Subject: [ruby-oci8-devel] Issue of binding nil element for Array DML Message-ID: <47745C11.4030602@oracle.com> Hi, Open this thread to discuss issue of binding nil element for Array DML. An example of normal insert: cursor = conn.parse("INSERT INTO test_table VALUE (:str)") cursor.bind_param(1, nil , String) cursor.exec The result of above example is inserting a row of "nil" to test_table. Let's turn to array insert example: cursor = conn.parse("INSERT INTO test_table VALUE (:str)") cursor.max_array_size = 3 cursor.bind_param(1, nil , String) cursor.exec_array For this binding "nil" elements for array insert, I have several optional results: 1) insert an array of "nil" 2) insert single row of "nil" 3) raise an error Personally, I prefer option 1). Any idea? From shiwei.zhang at oracle.com Sat Dec 29 04:25:45 2007 From: shiwei.zhang at oracle.com (shiwei zhang) Date: Sat, 29 Dec 2007 17:25:45 +0800 Subject: [ruby-oci8-devel] propose to implement pooling technology for Ruby-OCI8 In-Reply-To: <873ats17pb.fsf@lilliput.jiubao.org> References: <47622312.8070500@oracle.com> <87tzmgj71k.fsf@lilliput.jiubao.org> <476A3101.2030103@oracle.com> <476B8790.4080409@oracle.com> <873ats17pb.fsf@lilliput.jiubao.org> Message-ID: <47761299.3060306@oracle.com> Hi, Sorry I had couple days off work, so am replying late. Btw wish everyboday a happy new year :-) . KUBO Takehiro wrote: > Hi, > > shiwei zhang writes: > > >>> BTW, I have a small question: you use "OCISessionBegin()" for >>> T_EXPLICIT (external credential or OCI_SYSDBA, OCI_SYSOPER); and use >>> "OCILogon" for T_IMPLICIT (OCI_CRED_RDBMS). *Why don't you use >>> "OCISessionBegin()" for both T_EXPLICIT and T_IMPLICIT?* I think >>> "OCISessionBegin()" can work for both of the two conditions, do you >>> have other concerns about this? >>> > > I agree in theory. I had used OCISessionBegin() for both before 0.1.13. > But I received a mail it sometimes causes a segmentation faults in 10 > logons. Its environment was as follows. > > ruby 1.8.2 > ruby-oci8 0.1.13 > Oracle Client 8.1.7 (Windows) > Oracle Server 8.0.5 > > This would be really rare. But there was no workaround. I changed to > use OCILogon for non-privilege logons. > > If the code become too complex by your changes, you can merge them > to OCISessionBegin(). Because it might be caused by a bug in ruby-oci8 > and its internal structure was changed. The same problem may not occur > in current implementation. > I plan to choose OCISessionGet() for both connection pooling and session pooling, because OCISessionBegin() can't work in session pooling. And for now I plan to provide pooling only for T_IMPLICIT, not for T_EXPLICIT, because: (1) OCI_SYSDBA, OCI_SYSOPER are not supported by pooling. (2) external credential is not supported by session pooling. (3) external credential is probably not supported by connection pooling. >>> For now I think: >>> >>> # connection pool >>> pool = OCI8::ConnectionPool.new(username, password, tns_name, conn_min, conn_max, conn_incr) >>> conn = OCI8.new(pool, appusername, apppassword) >>> or >>> pool = OCI8::ConnectionPool.new(username, password, tns_name, conn_min, conn_max, conn_incr) >>> conn = OCI8.new($poolname, appusername, apppassword) >>> >>> # session pool >>> pool = OCI8::SessionPool.new(username, password, tns_name, sess_min, sess_max) >>> conn = OCI8.new(pool, appusername, apppassword) >>> or >>> pool = OCI8::SessionPool.new(username, password, tns_name, sess_min, sess_max) >>> conn = OCI8.new($poolname, appusername, apppassword) >>> >>> Notes: >>> username is used to create the pool, it's the implicit user. >>> appusername is used to create/get a session from the pool. appusername >>> could be different with username. >>> > > Another proposal: > pool = OCI8::ConnectionPool.new(username, password, tns_name, conn_min, conn_max, conn_incr) > conn = OCI8.new(appusername, apppassword, pool) > > Change the current spec to: > OCI8.new(username, password, dbname_or_pool = nil, privilege = nil) > > Argument values username and password are appusername and apppassword > respectively if the third argument is a connection pool. > > One question: > Are appusername and apppassword required mandatory or optional? > I think this is a good proposal. appusername and apppassword is mandatory for connection pooling, but they are optional for session pooling. >>> $poolname is generate from "OCI8::ConnectionPool.new()" or >>> "OCI8::ConnectionPool.new()". If we've gotten the value of $poolname, >>> it's enough for "OCI8.new()" to create/hand out a session for appusername. >>> > > What is the data type of $poolname? If it is a string, we cannot > distinguish a logon with dbname from a login using connection pool. > If is is a connection pool object, when is the pool destroyed? > $poolname is a string. So let's use the pool object instead of it, in order that we can use the proposal above. > $poolname is set by a side effect of OCI::ConnectionPool.new(). IMO, > it is not straightforward. > Best Regards, Shiwei -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ruby-oci8-devel/attachments/20071229/d4716de9/attachment-0001.html From shiwei.zhang at oracle.com Sat Dec 29 03:40:21 2007 From: shiwei.zhang at oracle.com (shiwei zhang) Date: Sat, 29 Dec 2007 16:40:21 +0800 Subject: [ruby-oci8-devel] propose to implement pooling technology for Ruby-OCI8 In-Reply-To: <476B8790.4080409@oracle.com> References: <47622312.8070500@oracle.com> <87tzmgj71k.fsf@lilliput.jiubao.org> <476A3101.2030103@oracle.com> <476B8790.4080409@oracle.com> Message-ID: <477607F5.308@oracle.com> shiwei zhang wrote: > Hi, > > shiwei zhang wrote: >> Hi Kubo, >> >> Yes, it's better to make our discussion public on this ML. >> >> KUBO Takehiro wrote: >>> Hi shiwei, >>> >>> Sorry too late to reply you. >>> >>> shiwei zhang writes: >>> >>> >>>> How are you? This is Shiwei from Oracle Asia R&D Center, I hope you >>>> still remember me:) >>>> >>> >>> Yes, of course. >>> >>> >>>> Last time I submitted through Christ Jones to you a patch about Easy >>>> Connection String in Ruby-OCI8. Now I want to continue contributing >>>> some work for Ruby-OCI8, in order to make it further satisfy Ruby >>>> users. >>>> >>>> These days I did some tentative research about pooling technology for >>>> Ruby-OCI8. Database pooling is a top-notch technology that can improve >>>> the performance of application programs, especially for those >>>> large-scale ones handling many DB requests. >>>> >>> >>> In general, agree. Connection pooling is good for web applications. >>> PHP offers persistent connections for the purpose. >>> http://php.net/manual/en/features.persistent-connections.php >>> >>> IMO, most ruby's web applications are built on rails. But as for >>> rails, it is another story. Rails uses only one connection for all >>> requests. Pooling will not improve the performance. >>> >>> BTW go on. >>> >> Agree. Haha, ruby shouldn't be always limited to Rails, and even >> shouldn't always limited to web application. Furthermore Rails might >> change its principle to hold more connections in its pool. :-) >>>> There are two kinds of pooling at the client side: one is connection >>>> pooling, which can create & destroy stateful sessions; the other is >>>> session pooling, which can reuse stateless sessions. JDBC has supportd >>>> both connection pooling and session pooling, it has classes >>>> OracleOCIConnectionPool and OracleConnectionPoolDataSource. Now I >>>> propose to implement silimar connection pooling and session pooling >>>> for Ruby-OCI8, do you agree? What's your opinion? >>>> >>> >>> Surely. >>> >> Great, so I'd like to carry on, both connection pooling and session >> pooling. >>> >>>> I've read the related codes in RubyOCI8_1.0.0-rc3, I think we can work >>>> it out by either of the two methods: >>>> >>> >>> I want to add new features to ruby-oci8 svn trunk, not to ruby-oci8 >>> 1.0 seriese. Svn trunk will be ruby-oci8 2.0. And its internal >>> structure is thoroughly changed. >>> >>> The svn trunk code can be checked out with the following command. >>> >>> svn checkout http://ruby-oci8.rubyforge.org/svn/trunk/ruby-oci8 >>> or >>> svn checkout svn://rubyforge.org/var/svn/ruby-oci8/trunk/ruby-oci8 >>> >> This morning I downloaded the SVN source codes and glimpsed at it. As >> you said the internal structure is changed. Actually I wrote some >> codes locally on my pc about pooling for ruby-oci8_1.0, and I think >> it's fine for me to move to ruby-oci8_2.0. You transferred more codes >> from ruby side to C side in the new version, which I think makes the >> logic easier to decide & process the connection parameters, which is >> good. >> BTW, I have a small question: you use "OCISessionBegin()" for >> T_EXPLICIT (external credential or OCI_SYSDBA, OCI_SYSOPER); and use >> "OCILogon" for T_IMPLICIT (OCI_CRED_RDBMS). *Why don't you use >> "OCISessionBegin()" for both T_EXPLICIT and T_IMPLICIT?* I think >> "OCISessionBegin()" can work for both of the two conditions, do you >> have other concerns about this? >> I'm sure connection pooling and session pooling support T_IMPLICIT, >> but I'm not sure whether they also support T_EXPLICIT. And I'm not >> sure whether it's necessary for T_EXPLICIT to have connection pooling >> and session pooling, since you know, T_EXPLICIT is used a lot less >> frequently. I need to investigate more on this. > Today I wrote a program to confirm that the pooling doesn't support > OCI_SYSDBA or OCI_SYSOPER, it returns "ORA-24300: bad value for mode" > when using the mode OCI_SYSDBA or OCI_SYSOPER. I will make sure > whether OCI_CRED_EXT works in pooling soon. Now I'm sure OCI_CRED_EXT can't be supported in session pooling. Regarding connection pooling, from OCI documentation I found no modes like OCI_CRED_EXT available for the function "OCIConnectionPoolCreat()" yet I found the modes OCI_SESSGET_CPOOL and OCI_SESSGET_CREDEXT are available for the function "OCISessionGet()". However when I wrote some programs about connection pooling using the mode OCI_SESSGET_CREDEXT, it always ran to failure. Note: in my testing program OCI_CRED_EXT can work successfully in a usual connection without pooling. >>>> 1. Write sub-classes inherited from the current Ruby class OCI8; and >>>> write some C functions to execute the corresponding pooling work. >>>> 2. Modify the current Ruby class OCI; also write some C functions to >>>> execute the corresponding pooling work. >>>> >>> >>> Can you show me the spec and usage example? >>> for example: >>> >>> # connection pool >>> pool = OCI8::ConnectionPool.new(username, password, tns_name, conn_min, conn_max, conn_incr) >>> conn = OCI8.new(pool) >>> >>> # session pool >>> pool = OCI8::SessionPool.new(username, password, tns_name, sess_min, sess_max) >>> conn = OCI8.new(pool) >>> >> For now I think: >> >> # connection pool >> pool = OCI8::ConnectionPool.new(username, password, tns_name, >> conn_min, conn_max, conn_incr) >> conn = OCI8.new(pool, appusername, apppassword) >> or >> pool = OCI8::ConnectionPool.new(username, password, tns_name, >> conn_min, conn_max, conn_incr) >> conn = OCI8.new($poolname, appusername, apppassword) >> >> # session pool >> pool = OCI8::SessionPool.new(username, password, tns_name, sess_min, >> sess_max) >> conn = OCI8.new(pool, appusername, apppassword) >> or >> pool = OCI8::SessionPool.new(username, password, tns_name, sess_min, >> sess_max) >> conn = OCI8.new($poolname, appusername, apppassword) >> >> Notes: >> username is used to create the pool, it's the implicit user. >> appusername is used to create/get a session from the pool. >> appusername could be different with username. >> $poolname is generate from "OCI8::ConnectionPool.new()" or >> "OCI8::ConnectionPool.new()". If we've gotten the value of $poolname, >> it's enough for "OCI8.new()" to create/hand out a session for >> appusername. >>> If you can, please join to ruby-oci8-devel to make our discussion public. >>> >>> ruby-oci8-devel ML: http://rubyforge.org/mail/?group_id=256 >>> >>> > Thanks & Best Regards, > Shiwei > ------------------------------------------------------------------------ > > _______________________________________________ > ruby-oci8-devel mailing list > ruby-oci8-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/ruby-oci8-devel > Best Rgds, Shiwei -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/ruby-oci8-devel/attachments/20071229/f4bad612/attachment.html From kubo at jiubao.org Sat Dec 29 07:21:56 2007 From: kubo at jiubao.org (KUBO Takehiro) Date: Sat, 29 Dec 2007 21:21:56 +0900 Subject: [ruby-oci8-devel] Proposal for bind_param_array function prototype References: <475E332C.5090305@oracle.com> <5d847bcd0712110837y6343b788r9acb504b944aa376@mail.gmail.com> <475FD64C.3060100@oracle.com> <47661D7F.2050100@oracle.com> <87lk7say1c.fsf_-_@lilliput.jiubao.org> <4768BA55.9020408@oracle.com> <87lk7pqqa7.fsf@lilliput.jiubao.org> <47734F56.4080000@oracle.com> Message-ID: <87r6h5ady3.fsf@lilliput.jiubao.org> Hi, Liming Lian writes: >> cursor.bind_param_array(1, ["test4", "test5"]) >> cursor.bind_param_array(2, [40]) >> cursor.exec_array # raise an error because array sizes are not same. > > For this case, do you think it is better to use following strategic: If > all the sizes of bound arrays are less than max_array_size, we will use > the largest one as the iteration_count. It is not required that all the > arrays are the same size. "nil" elements will be appended to those > shorter arrays to make them the size of iteration_count. > > Example: > > cursor = conn.parse("INSERT INTO test_table VALUES(:name, :age)") > cursor.max_array_size = 3 > cursor.bind_param_array(1, ["test4", "test5"]) > cursor.bind_param_array(2, [40]) > cursor.exec_array # iteration_count is 2 because the largest parameter's > # array size are 2. > > The rows inserted into the db are: > row1: test4 40 > row2: test5 nil I prefer an error in this case. I cannot imagine users try to insert such arrays. If they make the arrays by their hands, their sizes may not be same. But IMO, it is rare to write array data by hands. If a program make them, it will be unexpected case. It should raise an error. -- KUBO Takehiro From kubo at jiubao.org Sat Dec 29 07:51:27 2007 From: kubo at jiubao.org (KUBO Takehiro) Date: Sat, 29 Dec 2007 21:51:27 +0900 Subject: [ruby-oci8-devel] Issue of binding nil element for Array DML References: <47745C11.4030602@oracle.com> Message-ID: <87odc9ackw.fsf@lilliput.jiubao.org> Hi, Liming Lian writes: > Hi, > > Open this thread to discuss issue of binding nil element for Array DML. > > An example of normal insert: > > cursor = conn.parse("INSERT INTO test_table VALUE (:str)") > cursor.bind_param(1, nil , String) > cursor.exec > > The result of above example is inserting a row of "nil" to test_table. > Let's turn to array insert example: > > cursor = conn.parse("INSERT INTO test_table VALUE (:str)") > cursor.max_array_size = 3 > cursor.bind_param(1, nil , String) Isn't this a typo of bind_param_array? Do you use bind_param explicitly? > cursor.exec_array > > For this binding "nil" elements for array insert, I have several > optional results: > > 1) insert an array of "nil" > > 2) insert single row of "nil" > > 3) raise an error > > Personally, I prefer option 1). Any idea? If it is not a typo of bind_param_array, I also 1). In addition, if users set a value, all array elements are the value. cursor = conn.parse("INSERT INTO test_table VALUE (:str)") cursor.max_array_size = 3 cursor.bind_param(1, 'abc', String) cursor.exec_array Inserted rows are: row1: 'abc' row2: 'abc' row3: 'abc' We can implement it easily by just changing code as following. stmt.c line 225-227 from: if (NIL_P(obind->tdo) && obind->maxar_sz > 0) { oci_lc(OCIBindArrayOfStruct(obind->base.hp.bnd, oci8_errhp, obind->alloc_sz, sizeof(sb2), 0, 0)); } to: if (NIL_P(obind->tdo) && obind->maxar_sz > 0) { oci_lc(OCIBindArrayOfStruct(obind->base.hp.bnd, oci8_errhp, obind->alloc_sz, sizeof(sb2), 0, 0)); } else { /* All iterations refer to same position. */ oci_lc(OCIBindArrayOfStruct(obind->base.hp.bnd, oci8_errhp, 0, 0, 0, 0)); } This will be useful when users want to set same values to a parameter. cursor = conn.parse("INSERT INTO test_table VALUE (:name, :sex)") cursor.max_array_size = 3 cursor.bind_param_array(1, ['John', 'James', 'Charlee'], String) cursor.bind_param(2, 'male') cursor.exec_array Inserted rows are: row1: 'John', 'male' row2: 'James', 'male' row3: 'Charlee', 'male' If it is a typo of bind_param_array, '3) raise an error.' -- KUBO Takehiro From kubo at jiubao.org Mon Dec 31 00:02:05 2007 From: kubo at jiubao.org (KUBO Takehiro) Date: Mon, 31 Dec 2007 14:02:05 +0900 Subject: [ruby-oci8-devel] propose to implement pooling technology for Ruby-OCI8 In-Reply-To: <47761299.3060306@oracle.com> References: <47622312.8070500@oracle.com> <87tzmgj71k.fsf@lilliput.jiubao.org> <476A3101.2030103@oracle.com> <476B8790.4080409@oracle.com> <873ats17pb.fsf@lilliput.jiubao.org> <47761299.3060306@oracle.com> Message-ID: <5d847bcd0712302102h43288ac5oe51d0ed1f0534db@mail.gmail.com> Hi, On Dec 29, 2007 6:25 PM, shiwei zhang wrote: > I plan to choose OCISessionGet() for both connection pooling and session > pooling, because OCISessionBegin() can't work in session pooling. OK. > And for now I plan to provide pooling only for T_IMPLICIT, not for > T_EXPLICIT, because: (1) OCI_SYSDBA, OCI_SYSOPER are not supported by > pooling. (2) external credential is not supported by session pooling. (3) > external credential is probably not supported by connection pooling. I think you need to add a enum value T_POOL or so. It may be better to use more explanatory name as T_IMPLICIT -> T_OCI_LOGON, T_EXPLICIT -> T_OCI_SESSION_BEGIN, T_POOL -> T_OCI_SESSION_GET. ruby-oci8 supports Oracle 8.0 or later. Look at the following URL. http://ruby-oci8.rubyforge.org/en/dev_APIWrap.html You may have known already. But I notice to make sure. 1. OCI8::ConnectionPool and OCI8::SessionPool must be defined by oci8_define_class_under not rb_define_class_under because OCICPool and OCISPool are both OCI Handle. 2. When OCI8.new(user, pass, pool) is called, set pool as a parent of an instance of OCI8 by oci8_link_to_parent(). 3. Add a mark function to OCI8 and call rb_gc_mark() to prevent its parent.from being freed. static oci8_base_class_t oci8_svcctx_class = { NULL, <- add a new mark callback function. oci8_svcctx_free, sizeof(oci8_svcctx_t) };