From liming.lian at oracle.com Tue Mar 11 23:25:35 2008 From: liming.lian at oracle.com (Liming Lian) Date: Wed, 12 Mar 2008 11:25:35 +0800 Subject: [ruby-oci8-devel] About XML DB support in Ruby-OCI8 Message-ID: <47D74D2F.4010703@oracle.com> Hi Kubo, I am looking into XML DB support in Ruby-OCI8. It is apparent the existed code in xmldb.c only implements part of this functionality. And it is quite standalone and focuses only on how to construct REXML objects from XML data. The missing parts are: defining new data type of XMLType for binding, operating XML data in database and some other functions related to OCI operations. Another issue I concern is that, the XML DB support depends on XMLDB development toolkit(XDK). We have to check the host environment to determine if the XDK is available in the configuration phase. To check prerequisite of XDK installation, I think we have to change the ext/oci8/extconf.rb or ext/oci8/oraconf.rb a little bit to generate proper Makefile for compiling oci8lib. Before taking further step into improving XML DB support in OCI8, I would like to know your thoughts including how did you previously plan to design xmldb.c, what's the difficulties you have ever met, and does this module depends on any other parts in Ruby-OCI8. I believe those information is a great of importance to me. Thanks in advance! Liming From kubo at jiubao.org Thu Mar 13 10:27:42 2008 From: kubo at jiubao.org (KUBO Takehiro) Date: Thu, 13 Mar 2008 23:27:42 +0900 Subject: [ruby-oci8-devel] About XML DB support in Ruby-OCI8 In-Reply-To: <47D74D2F.4010703@oracle.com> References: <47D74D2F.4010703@oracle.com> Message-ID: <5d847bcd0803130727n11a7a876o2ef663cf1cffa994@mail.gmail.com> Hi Liming, On Wed, Mar 12, 2008 at 12:25 PM, Liming Lian wrote: > Hi Kubo, > > I am looking into XML DB support in Ruby-OCI8. It is apparent the > existed code in xmldb.c only implements part of this functionality. And > it is quite standalone and focuses only on how to construct REXML > objects from XML data. The missing parts are: defining new data type of > XMLType for binding, operating XML data in database and some other > functions related to OCI operations. Yes. I had implemented oci8_make_rexml() to construct REXML objects from XML data in Oracle type attributes. It was called from tdo.c, which was deleted about one year ago. When I reconstructed Oracle object type support, I didn't add code for XML type. That's because I thought I would add it after XML type binding was finished. But it have been postponed. > Another issue I concern is that, the XML DB support depends on XMLDB > development toolkit(XDK). We have to check the host environment to > determine if the XDK is available in the configuration phase. To check > prerequisite of XDK installation, I think we have to change the > ext/oci8/extconf.rb or ext/oci8/oraconf.rb a little bit to generate > proper Makefile for compiling oci8lib. As for oci8_make_rexml(), it doesn't need XMLDB headers. Required function prototypes are copied to xmldb.c. > Before taking further step into improving XML DB support in OCI8, I > would like to know your thoughts including how did you previously plan > to design xmldb.c, what's the difficulties you have ever met, and does > this module depends on any other parts in Ruby-OCI8. I believe those > information is a great of importance to me. * How did you previously plan to design xmldb.c. previous plan: xmldb.c - conversion between REXML objects and XML data tdo.c - binding XML data - get/set XML data in object type attributes * Does this module depends on any other parts in Ruby-OCI8. current plan: xmldb.c - conversion between REXML objects and XML data - binding XML data object.c - get/set XML data in object type attributes Almost XML type support are implemented in xmldb.c. - bind class (OCI8::BindType::REXML?) - conversion between XML data and REXML objects The number of non-static C functions in xmldb.c is only three. void Init_oci_xmldb(void); VALUE oci8_make_rexml(struct xmlctx *xctx, xmlnode *node); construct REXML objects from XML data. void oci8_set_rexml(struct xmlctx *xctx, xmlnode *node, VALUE obj); 1. clear all contents of a XML datum 2. set REXML object's contents to the XML datum. object.c uses oci8_make_rexml and oci8_set_rexml to get and set XML type attributes in Oracle object types. * What's the difficulties you have ever met. 1. whitespaces Whitespaces in XML text may or may not be reserved when converting to DOM objects. I have forgot the details. 2. encoding REXML's encoding must be UTF-8. But XML type's encoding depends on NLS_LANG. From kubo at jiubao.org Thu Mar 13 10:30:40 2008 From: kubo at jiubao.org (KUBO Takehiro) Date: Thu, 13 Mar 2008 23:30:40 +0900 Subject: [ruby-oci8-devel] About XML DB support in Ruby-OCI8 In-Reply-To: <47D74D2F.4010703@oracle.com> References: <47D74D2F.4010703@oracle.com> Message-ID: <5d847bcd0803130730y2fb71cd1tba62d673f5d6060a@mail.gmail.com> Hi Liming, On Wed, Mar 12, 2008 at 12:25 PM, Liming Lian wrote: > Hi Kubo, > > I am looking into XML DB support in Ruby-OCI8. It is apparent the > existed code in xmldb.c only implements part of this functionality. And > it is quite standalone and focuses only on how to construct REXML > objects from XML data. The missing parts are: defining new data type of > XMLType for binding, operating XML data in database and some other > functions related to OCI operations. Yes. I had implemented oci8_make_rexml() to construct REXML objects from XML data in Oracle type attributes. It was called from tdo.c, which was deleted about one year ago. When I reconstructed Oracle object type support, I didn't add code for XML type. That's because I thought I would add it after XML type binding was finished. But it have been postponed. > Another issue I concern is that, the XML DB support depends on XMLDB > development toolkit(XDK). We have to check the host environment to > determine if the XDK is available in the configuration phase. To check > prerequisite of XDK installation, I think we have to change the > ext/oci8/extconf.rb or ext/oci8/oraconf.rb a little bit to generate > proper Makefile for compiling oci8lib. As for oci8_make_rexml(), it doesn't need XMLDB headers. Required function prototypes are copied to xmldb.c. > Before taking further step into improving XML DB support in OCI8, I > would like to know your thoughts including how did you previously plan > to design xmldb.c, what's the difficulties you have ever met, and does > this module depends on any other parts in Ruby-OCI8. I believe those > information is a great of importance to me. * How did you previously plan to design xmldb.c. previous plan: xmldb.c - conversion between REXML objects and XML data tdo.c - binding XML data - get/set XML data in object type attributes * Does this module depends on any other parts in Ruby-OCI8. current plan: xmldb.c - conversion between REXML objects and XML data - binding XML data object.c - get/set XML data in object type attributes Almost XML type support are implemented in xmldb.c. - bind class (OCI8::BindType::REXML?) - conversion between XML data and REXML objects The number of non-static C functions in xmldb.c is only three. void Init_oci_xmldb(void); VALUE oci8_make_rexml(struct xmlctx *xctx, xmlnode *node); construct REXML objects from XML data. void oci8_set_rexml(struct xmlctx *xctx, xmlnode *node, VALUE obj); 1. clear all contents of a XML datum 2. set REXML object's contents to the XML datum. object.c uses oci8_make_rexml and oci8_set_rexml to get and set XML type attributes in Oracle object types. * What's the difficulties you have ever met. 1. whitespaces Whitespaces in XML text may or may not be reserved when converting to DOM objects. I have forgot the details. 2. encoding REXML's encoding must be UTF-8. But XML type's encoding depends on NLS_LANG. From liming.lian at oracle.com Thu Mar 20 02:19:53 2008 From: liming.lian at oracle.com (Liming Lian) Date: Thu, 20 Mar 2008 14:19:53 +0800 Subject: [ruby-oci8-devel] About XML DB support in Ruby-OCI8 In-Reply-To: <5d847bcd0803130730y2fb71cd1tba62d673f5d6060a@mail.gmail.com> References: <47D74D2F.4010703@oracle.com> <5d847bcd0803130730y2fb71cd1tba62d673f5d6060a@mail.gmail.com> Message-ID: <47E20209.2020806@oracle.com> Hi Kubo, Thanks for sharing. As you said, the XML DB implementation depends on object types support. This raised my interest on how object types are supported in Ruby-OCI8. By hacking the code, I am still not sure how to use this feature and whether it has been fully implemented. It will be appreciated if you could show me some Ruby code samples on how to manipulate Object types in database supposing that this feature is available. Thanks in advance! Liming > Hi Liming, > > On Wed, Mar 12, 2008 at 12:25 PM, Liming Lian wrote: > >> Hi Kubo, >> >> I am looking into XML DB support in Ruby-OCI8. It is apparent the >> existed code in xmldb.c only implements part of this functionality. And >> it is quite standalone and focuses only on how to construct REXML >> objects from XML data. The missing parts are: defining new data type of >> XMLType for binding, operating XML data in database and some other >> functions related to OCI operations. >> > > Yes. I had implemented oci8_make_rexml() to construct REXML objects > from XML data in Oracle type attributes. It was called from tdo.c, > which was deleted about one year ago. When I reconstructed Oracle > object type support, I didn't add code for XML type. That's because > I thought I would add it after XML type binding was finished. > But it have been postponed. > > >> Another issue I concern is that, the XML DB support depends on XMLDB >> development toolkit(XDK). We have to check the host environment to >> determine if the XDK is available in the configuration phase. To check >> prerequisite of XDK installation, I think we have to change the >> ext/oci8/extconf.rb or ext/oci8/oraconf.rb a little bit to generate >> proper Makefile for compiling oci8lib. >> > > As for oci8_make_rexml(), it doesn't need XMLDB headers. Required > function prototypes are copied to xmldb.c. > > >> Before taking further step into improving XML DB support in OCI8, I >> would like to know your thoughts including how did you previously plan >> to design xmldb.c, what's the difficulties you have ever met, and does >> this module depends on any other parts in Ruby-OCI8. I believe those >> information is a great of importance to me. >> > > * How did you previously plan to design xmldb.c. > > previous plan: > xmldb.c > - conversion between REXML objects and XML data > > tdo.c > - binding XML data > - get/set XML data in object type attributes > > * Does this module depends on any other parts in Ruby-OCI8. > > current plan: > xmldb.c > - conversion between REXML objects and XML data > - binding XML data > > object.c > - get/set XML data in object type attributes > > Almost XML type support are implemented in xmldb.c. > - bind class (OCI8::BindType::REXML?) > - conversion between XML data and REXML objects > > The number of non-static C functions in xmldb.c is only three. > void Init_oci_xmldb(void); > VALUE oci8_make_rexml(struct xmlctx *xctx, xmlnode *node); > construct REXML objects from XML data. > void oci8_set_rexml(struct xmlctx *xctx, xmlnode *node, VALUE obj); > 1. clear all contents of a XML datum > 2. set REXML object's contents to the XML datum. > > object.c uses oci8_make_rexml and oci8_set_rexml to get and set > XML type attributes in Oracle object types. > > * What's the difficulties you have ever met. > > 1. whitespaces > > Whitespaces in XML text may or may not be reserved when converting > to DOM objects. I have forgot the details. > > 2. encoding > > REXML's encoding must be UTF-8. But XML type's encoding depends on > NLS_LANG. > _______________________________________________ > ruby-oci8-devel mailing list > ruby-oci8-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/ruby-oci8-devel > From kubo at jiubao.org Thu Mar 20 11:08:33 2008 From: kubo at jiubao.org (KUBO Takehiro) Date: Fri, 21 Mar 2008 00:08:33 +0900 Subject: [ruby-oci8-devel] About XML DB support in Ruby-OCI8 In-Reply-To: <47E20209.2020806@oracle.com> (Liming Lian's message of "Thu\, 20 Mar 2008 14\:19\:53 +0800") References: <47D74D2F.4010703@oracle.com> <5d847bcd0803130730y2fb71cd1tba62d673f5d6060a@mail.gmail.com> <47E20209.2020806@oracle.com> Message-ID: <87bq59o2ku.fsf@lilliput.jiubao.org> Hi Liming, Liming Lian writes: > Hi Kubo, > > Thanks for sharing. As you said, the XML DB implementation depends on > object types support. This raised my interest on how object types are > supported in Ruby-OCI8. By hacking the code, I am still not sure how to > use this feature and whether it has been fully implemented. It will be > appreciated if you could show me some Ruby code samples on how to > manipulate Object types in database supposing that this feature is > available. > > Thanks in advance! I'll answer your questions later. It needs a time for me to write complex things in English. It is easy for me to write C source code than English documents. Here is a quick fix to fetch XMLTYPE data as REXML objects. =================================================================== --- ext/oci8/xmldb.c (revision 253) +++ ext/oci8/xmldb.c (working copy) @@ -1,7 +1,8 @@ /* -*- c-file-style: "ruby"; indent-tabs-mode: nil -*- */ #include "oci8.h" -#if 0 /* disabled for a while. */ && (ORACLE_CLIENT_VERSION >= 1000 || defined RUNTIME_API_CHECK) +#if defined RUNTIME_API_CHECK || ORACLE_CLIENT_VERSION >= 1000 +/* Oracle 10g or upper */ #ifndef HAVE_XMLOTN_H /* declarations in xmlproc.h of Oracle XML Development Kit */ @@ -158,6 +159,87 @@ static VALUE add_attributes(VALUE obj, struct xmlctx *xctx, xmlnode *node); static VALUE add_nodemap(VALUE obj, struct xmlctx *xctx, xmlnamedmap *map); +typedef struct { + oci8_bind_t bind; + struct xmlctx *xctx; +} oci8_bind_xmltype_t; + +static void bind_xmltype_free(oci8_base_t *base) +{ + oci8_bind_t *obind = (oci8_bind_t *)base; + oci8_bind_xmltype_t *xmlbind = (oci8_bind_xmltype_t *)base; + xmlnode **nodes = (xmlnode **)obind->valuep; + ub4 idx = 0; + + do { + if (nodes[idx] != NULL) { + OCIObjectFree(oci8_envhp, oci8_errhp, nodes[idx], OCI_DEFAULT); + nodes[idx] = NULL; + } + } while (++idx < obind->maxar_sz); + if (xmlbind->xctx != NULL) { + OCIXmlDbFreeXmlCtx(xmlbind->xctx); + xmlbind->xctx = NULL; + } +} + +static VALUE bind_xmltype_get(oci8_bind_t *obind, void *data, void *null_struct) +{ + oci8_bind_xmltype_t *xmlbind = (oci8_bind_xmltype_t *)obind; + xmlnode *node = *(xmlnode **)data; + return oci8_make_rexml(xmlbind->xctx, node); +} + +static void bind_xmltype_set(oci8_bind_t *obind, void *data, void **null_structp, VALUE val) +{ + rb_raise(rb_eRuntimeError, "not supported"); +} + +static void bind_xmltype_init(oci8_bind_t *obind, VALUE svc, VALUE val, VALUE length) +{ + oci8_bind_xmltype_t *xmlbind = (oci8_bind_xmltype_t *)obind; + oci8_svcctx_t *svcctx = oci8_get_svcctx(svc); + VALUE md = rb_funcall(svc, rb_intern("describe_type"), 1, rb_str_new2("SYS.XMLTYPE")); + VALUE tdo = rb_funcall(svc, rb_intern("get_tdo_by_metadata"), 1, md); + /* + * md = conn.describe_type('SYS.XMLTYPE') # md: metadata + * tdo = conn.get_tdo_by_metadata(md) # tdo: type descriptor object + */ + + obind->value_sz = sizeof(xmlnode *); + obind->alloc_sz = sizeof(xmlnode *); + obind->tdo = tdo; + xmlbind->xctx = OCIXmlDbInitXmlCtx(oci8_envhp, svcctx->base.hp.svc, oci8_errhp, NULL, 0); +} + +static void bind_xmltype_init_elem(oci8_bind_t *obind, VALUE svc) +{ + xmlnode **nodes = (xmlnode **)obind->valuep; + oci8_base_t *tdo = DATA_PTR(obind->tdo); + oci8_svcctx_t *svcctx = oci8_get_svcctx(svc); + ub4 idx = 0; + + do { + oci_lc(OCIObjectNew(oci8_envhp, oci8_errhp, svcctx->base.hp.svc, OCI_TYPECODE_OPAQUE, tdo->hp.tdo, NULL, OCI_DURATION_SESSION, FALSE, (dvoid**)&nodes[idx])); + } while (++idx < obind->maxar_sz); +} + +static oci8_bind_class_t bind_rexml_class = { + { + NULL, + bind_xmltype_free, + sizeof(oci8_bind_xmltype_t) + }, + bind_xmltype_get, + bind_xmltype_set, + bind_xmltype_init, + bind_xmltype_init_elem, + NULL, + NULL, + NULL, + SQLT_NTY +}; + void Init_oci_xmldb(void) { id_add = rb_intern("add"); @@ -172,6 +254,9 @@ REXML_XMLDecl = rb_eval_string("REXML::XMLDecl"); REXML_DocType = rb_eval_string("REXML::DocType"); REXML_NotationDecl = rb_eval_string("REXML::NotationDecl"); + + /* OCI8::BindType::REXML */ + oci8_define_bind_class("REXML", &bind_rexml_class); } VALUE oci8_make_rexml(struct xmlctx *xctx, xmlnode *node) =================================================================== and OCI8::BindType::Mapping[:xmltype] = OCI8::BindType::REXML From liming.lian at oracle.com Thu Mar 20 21:47:35 2008 From: liming.lian at oracle.com (Liming Lian) Date: Fri, 21 Mar 2008 09:47:35 +0800 Subject: [ruby-oci8-devel] About XML DB support in Ruby-OCI8 In-Reply-To: <87bq59o2ku.fsf@lilliput.jiubao.org> References: <47D74D2F.4010703@oracle.com> <5d847bcd0803130730y2fb71cd1tba62d673f5d6060a@mail.gmail.com> <47E20209.2020806@oracle.com> <87bq59o2ku.fsf@lilliput.jiubao.org> Message-ID: <47E313B7.4050105@oracle.com> Hi Kubo, Thanks for your quick update! > I'll answer your questions later. It needs a time for me to write > complex things in English. It is easy for me to write C source code > than English documents. > Just take it easy, no hurry at all! Thanks again! Liming From liming.lian at oracle.com Tue Mar 25 23:16:44 2008 From: liming.lian at oracle.com (Liming Lian) Date: Wed, 26 Mar 2008 11:16:44 +0800 Subject: [ruby-oci8-devel] About XML DB support in Ruby-OCI8 In-Reply-To: <87bq59o2ku.fsf@lilliput.jiubao.org> References: <47D74D2F.4010703@oracle.com> <5d847bcd0803130730y2fb71cd1tba62d673f5d6060a@mail.gmail.com> <47E20209.2020806@oracle.com> <87bq59o2ku.fsf@lilliput.jiubao.org> Message-ID: <47E9C01C.5070300@oracle.com> Hi Kubo, After applying your patch for defining bind type of XMLType, I can successfully retrieve XML data from db by explicitly defining the select data as XMLType. Here is code snippet: ## SQLs for defining XMLType table and inserting XML data to the table CREATE TABLE xml_table (key VARCHAR2(10) PRIMARY KEY, xml_column XMLType); CREATE DIRECTORY xmldir AS path_to_folder_containing_XML_file; INSERT INTO xml_table VALUES (1, XMLType(bfilename('XMLDIR', 'mytest.xml'), nls_charset_id('AL32UTF8'))); ## Ruby code for retrieving XML data conn = OCI8.new(dbuser, dbpass, dbname) cur = conn.parse('select xml_column from xml_table where key = 1') cur.define(1, :xmltype) cur.exec r = cur.fetch xml = r[0] After that, the variable "xml" is a REXML object, so that we can populate it with standard REXML APIs. It is fine with this kind of explicitly defining XMLtype while I also find some existed code in lib/oci8/oci8.rb to deal with XMLType as following: File: lib/oci8/oci8.rb Method: make_bind_object def make_bind_object(param) case param when Hash .......... when OCI8::Metadata::Base key = param.data_type case key when :named_type if param.type_name == 'XMLTYPE' key = :xmltype else param = @con.get_tdo_by_metadata(param.type_metadata) end end else ............. I am only familiar with passing Hash parameter to make_bind_object. Meantime, I notice method make_bind_object also accepts param of OCI8::Metadata::Base type. I am wondering how to construct such OCI8::Metadata::Base object so that we can bind or define XMLType in a manner other than the one I have tried. Any hints? Code sample is appreciated! Thanks in advance! Liming > Here is a quick fix to fetch XMLTYPE data as REXML objects. > > =================================================================== > --- ext/oci8/xmldb.c (revision 253) > From kubo at jiubao.org Thu Mar 27 06:28:19 2008 From: kubo at jiubao.org (KUBO Takehiro) Date: Thu, 27 Mar 2008 19:28:19 +0900 Subject: [ruby-oci8-devel] About XML DB support in Ruby-OCI8 In-Reply-To: <47E9C01C.5070300@oracle.com> References: <47D74D2F.4010703@oracle.com> <5d847bcd0803130730y2fb71cd1tba62d673f5d6060a@mail.gmail.com> <47E20209.2020806@oracle.com> <87bq59o2ku.fsf@lilliput.jiubao.org> <47E9C01C.5070300@oracle.com> Message-ID: <5d847bcd0803270328s110fe466hef0ff814f9fde370@mail.gmail.com> Hi Liming, On Wed, Mar 26, 2008 at 12:16 PM, Liming Lian wrote: > I am only familiar with passing Hash parameter to make_bind_object. > Meantime, I notice method make_bind_object also accepts param of > OCI8::Metadata::Base type. I am wondering how to construct such > OCI8::Metadata::Base object so that we can bind or define XMLType in a > manner other than the one I have tried. Any hints? Code sample is > appreciated! OCI8::Metadata::Base is passed only when a column is not defined explicitly. Its actual type is OCI8::Metadata::Column which is declared in lib/oci8/metadata.rb. File: lib/oci8/oci8.rb def exec(*bindvars) bind_params(*bindvars) __execute(nil) case type when :select_stmt define_columns() .... end # I would refactor this code.... def define_columns num_cols = __param_count 1.upto(num_cols) do |i| parm = __paramGet(i) # => parm is a OCI8::Metadata::Column. define_a_column(i, parm) unless __defiend?(i) # If the column is not explicitly defined, define it. @column_metadata[i - 1] = parm end num_cols end # define_columns