From thibaut.barrere at gmail.com  Fri Dec 19 11:20:05 2008
From: thibaut.barrere at gmail.com (=?ISO-8859-1?Q?Thibaut_Barr=E8re?=)
Date: Fri, 19 Dec 2008 17:20:05 +0100
Subject: [Activewarehouse-discuss] Keeping state between jobs in
	etl_execution db, your opinion ?
Message-ID: <4a68b8cf0812190820t5207a803r88d33daddc2b7748@mail.gmail.com>

Hello,
Sharing a few thoughts here, I'd love to have your opinion.

I have a datawarehouse that is rebuild totally from scratch each night (as
opposed to incrementally) - basically drop database, db:migrate, then
import. Works pretty well.

For an evolution, I'd like to keep some unimportant state (geocoding cache)
into a database so it can be reused each night (and completed if relevant).
Loosing the cache would not be a problem if it happened, we would just
rebuild it.

Are some of you using the etl_execution database for storing that kind of
things, as I'm thinking of doing it ?

I'm thinking about using an AR record, like:

class CachedAddress < ActiveRecord::Base
  establish_connection :etl_execution
end

I would probably define an additional migration for that as well (like the
ones built-in for :jobs,:records and :batches) by patching
ETL::Execution::Migration migrate.

what do you think ?

thanks,

Thibaut Barr?re
--
LoGeek
[blog] http://evolvingworker.com - tools for a better day
[blog] http://blog.logeek.fr - about writing software
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/activewarehouse-discuss/attachments/20081219/80669aaf/attachment-0001.html>

From thibaut.barrere at gmail.com  Sat Dec 20 14:04:25 2008
From: thibaut.barrere at gmail.com (=?ISO-8859-1?Q?Thibaut_Barr=E8re?=)
Date: Sat, 20 Dec 2008 20:04:25 +0100
Subject: [Activewarehouse-discuss] [patch] file destination: output fields
	names on first row
Message-ID: <4a68b8cf0812201104x11671c78ge14694181f36706b@mail.gmail.com>

Hi,

here's a quick patch that could be useful to others. I had that need to use
the file destination to output a CSV file (with the headers included so that
another process can reliably import it on another machine later on).

Syntax:

destination :out, { :file => output_file, :include_headers => true, :enclose
=> '"', :separator => ";" }, { :order => target_fields }

cheers,

Thibaut

Index:
vendor/activewarehouse-etl/lib/etl/control/destination/file_destination.rb
===================================================================
---
vendor/activewarehouse-etl/lib/etl/control/destination/file_destination.rb
(revision 765)
+++
vendor/activewarehouse-etl/lib/etl/control/destination/file_destination.rb
(working copy)
@@ -35,6 +35,7 @@
       # * <tt>:enclose</tt>: Enclosure character (default is none)
       # * <tt>:unique</tt>: Set to true to only write unique records
       # * <tt>:append_rows</tt>: Array of rows to append
+      # * <tt>:include_headers</tt>: Set to true to output the headers as
the first row (based on :order)
       #
       # Mapping options:
       # * <tt>:order</tt>: The order array
@@ -49,6 +50,8 @@

         @order = mapping[:order] || order_from_source
         raise ControlError, "Order required in mapping" unless @order
+
+        write(Hash[*@order.zip(@order).flatten]) if
configuration[:include_headers] == true
       end

       # Close the destination. This will flush the buffer and close the
underlying stream or connection.

Thibaut Barr?re
--
LoGeek
[tel] +33 6 85 33 40 88
[mail] thibaut.barrere at gmail.com
[blog] http://evolvingworker.com - tools for a better day
[blog] http://blog.logeek.fr - about writing software
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/activewarehouse-discuss/attachments/20081220/2d0c431d/attachment.html>

From jeroen at kanji.nl  Sun Dec 21 13:59:06 2008
From: jeroen at kanji.nl (Jeroen Roodnat)
Date: Sun, 21 Dec 2008 19:59:06 +0100
Subject: [Activewarehouse-discuss] cube detail question
Message-ID: <A9D7BB93-A270-4F03-934C-F9D0E55C6E44@kanji.nl>

Hi,

I have a AW setup as follows:

sales cube:

   reports_on :transaction
   pivots_on :date, :product, :location



I have setup the transaction fact and the dimensions needed. Using the  
table report helpers I can generate a table where I can drill down my  
hierarchies. I
use the default table-report helper in my views.

I would like to stay in group sum tables until I reach the end of the  
hierarchy. Then I would like to click on the last hierarchy and have  
another drill-down that shows not the sum but all the transactions for  
a specific date/product for all locations.

Can this be done ? I can create custom helper but do not know how to  
get this specific data from AW.

regards

Jeroen

From thibaut.barrere at gmail.com  Mon Dec 22 18:48:13 2008
From: thibaut.barrere at gmail.com (=?ISO-8859-1?Q?Thibaut_Barr=E8re?=)
Date: Tue, 23 Dec 2008 00:48:13 +0100
Subject: [Activewarehouse-discuss] Keeping state between jobs in
	etl_execution db, your opinion ?
In-Reply-To: <4a68b8cf0812190820t5207a803r88d33daddc2b7748@mail.gmail.com>
References: <4a68b8cf0812190820t5207a803r88d33daddc2b7748@mail.gmail.com>
Message-ID: <4a68b8cf0812221548g2a99f526x5a4c8aeba919bc07@mail.gmail.com>

Hi,
replying to myself, in case it's useful to someone else - here's the
solution I came up with: http://gist.github.com/39191.

I refactored a bit the way etl_execution migrations are handled, so I can
add custom migrations to the etl_execution database (copy this in
environment.rb):

ETL::Execution::Migration.register_migration do |connection|
  connection.create_table :geocoded_addresses do |t|
    t.column :adresse, :string
    t.column :latitude, :float
    t.column :longitude, :float
  end
end

ETL::Execution::Migration.migrate

The matching model is a regular AR model except that I tell it to use the
etl_execution database:

class GeocodedAddress < ActiveRecord::Base
  establish_connection :etl_execution
end

cheers,

-- Thibaut


On Fri, Dec 19, 2008 at 5:20 PM, Thibaut Barr?re
<thibaut.barrere at gmail.com>wrote:

> Hello,
> Sharing a few thoughts here, I'd love to have your opinion.
>
> I have a datawarehouse that is rebuild totally from scratch each night (as
> opposed to incrementally) - basically drop database, db:migrate, then
> import. Works pretty well.
>
> For an evolution, I'd like to keep some unimportant state (geocoding cache)
> into a database so it can be reused each night (and completed if relevant).
> Loosing the cache would not be a problem if it happened, we would just
> rebuild it.
>
> Are some of you using the etl_execution database for storing that kind of
> things, as I'm thinking of doing it ?
>
> I'm thinking about using an AR record, like:
>
> class CachedAddress < ActiveRecord::Base
>   establish_connection :etl_execution
> end
>
> I would probably define an additional migration for that as well (like the
> ones built-in for :jobs,:records and :batches) by patching
> ETL::Execution::Migration migrate.
>
> what do you think ?
>
> thanks,
>
> Thibaut Barr?re
> --
> LoGeek
> [blog] http://evolvingworker.com - tools for a better day
> [blog] http://blog.logeek.fr - about writing software
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/activewarehouse-discuss/attachments/20081223/c47830f3/attachment.html>

From kidpollo at gmail.com  Fri Dec 26 14:01:28 2008
From: kidpollo at gmail.com (Paco Viramontes)
Date: Fri, 26 Dec 2008 13:01:28 -0600
Subject: [Activewarehouse-discuss] etl command error
Message-ID: <42FDA0EF-2768-45E8-BE4A-165BDCB1A4AB@gmail.com>

Hi

I am new to active warehouse.

I made my first ctl file and I run the etl command and get the  
following error

A source was specified but no matching type was found

This is my ctl file:

source :in, {
     :database => "Delitos_ZM",
     :target => :operational ,
     :table => "General"
},
[
   :AGENCIA,  :AREA,  :MUNICIPIO
]


I am using activewarehous-etl version 0.9.1.2

Thanks

paco


From jmaine at blurb.com  Mon Dec 29 15:58:37 2008
From: jmaine at blurb.com (Jacob Maine)
Date: Mon, 29 Dec 2008 12:58:37 -0800 (PST)
Subject: [Activewarehouse-discuss] etl command error
In-Reply-To: <42FDA0EF-2768-45E8-BE4A-165BDCB1A4AB@gmail.com>
Message-ID: <9186686.226651230584317010.JavaMail.root@kilgore.blurb.com>

It seems redundant, but I believe you need :type => :database in the hash.  I don't have source code on this computer, so you'll have to double-check the syntax.  

----- Paco Viramontes <kidpollo at gmail.com> wrote:
> Hi
> 
> I am new to active warehouse.
> 
> I made my first ctl file and I run the etl command and get the  
> following error
> 
> A source was specified but no matching type was found
> 
> This is my ctl file:
> 
> source :in, {
>      :database => "Delitos_ZM",
>      :target => :operational ,
>      :table => "General"
> },
> [
>    :AGENCIA,  :AREA,  :MUNICIPIO
> ]
> 
> 
> I am using activewarehous-etl version 0.9.1.2
> 
> Thanks
> 
> paco
> 
> _______________________________________________
> Activewarehouse-discuss mailing list
> Activewarehouse-discuss at rubyforge.org
> http://rubyforge.org/mailman/listinfo/activewarehouse-discuss