From Dominik.Mayer at BeAgile.de  Thu Apr  3 09:57:12 2008
From: Dominik.Mayer at BeAgile.de (Dominik Mayer)
Date: Thu, 03 Apr 2008 15:57:12 +0200
Subject: [Activewarehouse-discuss] Tests and Rspec
Message-ID: <47F4E238.3040307@BeAgile.de>


Hallo everybody,

I'm new to ActiveWarehouse and just got it to work on my machine (so far 
so good). Next step would be for me to check if all the tests (including 
rspec) are running.

Unfortunately I don't seem to understand how to populate my 
test-database, which I've already specified in a database.yml-file under 
./vendor/plugins/activewarehouse/test.

I keep getting the following error:

Mysql::Error: #42S02Table 'test3.pos_retail_sales_transaction_facts' 
doesn't exist: SHOW FIELDS FROM `pos_retail_sales_transaction_facts` 
(ActiveRecord::StatementInvalid)

As it seems it is at least using hte right db (test3). But I don't find 
a suitable rake-task to prepare this db.

It's obviously just me being blind, so please help me!

Thanks in advance.


Dominik Mayer


From thibaut.barrere at gmail.com  Thu Apr 10 10:21:04 2008
From: thibaut.barrere at gmail.com (=?ISO-8859-1?Q?Thibaut_Barr=E8re?=)
Date: Thu, 10 Apr 2008 16:21:04 +0200
Subject: [Activewarehouse-discuss] A helper to filter out data during
	development or maintenance
Message-ID: <4a68b8cf0804100721p3f9a9892o59f43c56dc166c19@mail.gmail.com>

Hi,

very often while doing maintenance work on a datawarehouse using CSV
files as input, I need to keep only the first line (headers) and any
line matching a given regexp, in order to save some time.

Here's a little helper do to this:

def keep_headers_and_matching_lines(filename,regexp)
  tempfilename = filename + ".tmp"
  FileUtils.mv(filename,tempfilename)
  File.open(tempfilename) do |input|
    File.open(filename,'w') do |output|
      input.each_with_index do |line,index|
        output << line if (line =~ regexp || index == 0)
      end
    end
  end
end

Typical use:

preprocess { keep_headers_and_matching_lines('mydata.csv',/customer/i) }

(sure that can be done also with a grep call - and that would be faster as well)

in case it's useful to someone else !

cheers,

Thibaut Barr?re
--
[blog] http://evolvingworker.com - tools for a better day
[blog] http://blog.logeek.fr - about writing software

From ggeller at heartlandamerica.com  Wed Apr 30 17:21:52 2008
From: ggeller at heartlandamerica.com (Gunner Geller)
Date: Wed, 30 Apr 2008 16:21:52 -0500
Subject: [Activewarehouse-discuss] activewarehouse-etl and sql server from
	leopard
Message-ID: <FCA1485D-D3FA-4C91-BE28-B5A579C66C67@heartlandamerica.com>

Hello,
	Our company is looking to develop all future web sites using ruby on  
rails. I am on a mac running leopard but the data needed for the back  
end of our websites is in sql server. I thought activewarehouse would  
be a logical way to go since we are using ruby on rails(ror). We are  
going to be grabbing data on a continuous basis from sql server and  
putting it in mysql which will power the ror sites.
	After a bit of tinkering I was able to get aw-etl to actually work  
using both a csv file and mysql db as a source and a mysql db as a  
destination. However I cannot connect to sql server. Below is the  
error I get when running my control file.

gunner-gellers-mac-pro:etl gunnergeller$ etl etl_test_db_sql2.ctl
Using AdapterExtensions
Starting ETL process
initializing ETL engine
Processing etl_test_db_sql2.ctl
Source: localhost/etl_development/test
/usr/local/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/ 
base.rb:1532:in `method_missing': undefined method  
`sqlserver_connection' for #<Class:0x1121e88> (NoMethodError)
	from /usr/local/lib/ruby/gems/1.8/gems/activewarehouse-etl-0.9.0/ 
bin/../lib/etl/engine.rb:207:in `send'
.......

Here is my database.yml file:

etl_sql:
     adapter: sqlserver
     mode: odbc
     dsn: DRIVER=/opt/local/lib/ 
libtdsodbc 
.so 
;TDS_Version 
=8.0;SERVER=192.168.xxx.xxx;DATABASE=db;Port=1433;uid=un;pwd=pw;

and my control file:

.....

destination :out, {
   :type => :database,
   :database => 'etl_development',
   :target => 'etl_sql',
   :table => 'test2'
},
{
   :order => [:id, :first_name, :last_name, :email, :loaded_at]
}

....

I can connect to sql server in my rails app using the exact same  
database.yml file. Also if I use mysql as a destination everything  
works fine too. Has anyone tried to use aw-etl with sql server? Or  
does anyone know what  I need to fix? The error says method missing,  
which makes me think I haven't installed something, but the fact that  
I can connect in my rails app just confuses things.

Any help is appreciated. Thanks Gunner.