From andy at chipin.com  Wed Apr 18 22:20:11 2007
From: andy at chipin.com (Andy Triboletti)
Date: Wed, 18 Apr 2007 16:20:11 -1000
Subject: [Activewarehouse-discuss] ETL job stats for a control file with
	dependencies
Message-ID: <893BDA3E-E1D3-4D61-89DE-0895B013C81E@chipin.com>

Hi, I have a control file which has some dependencies called all.ctl,  
and I noticed that the all.ctl row in the jobs table never  
completes.  The status after a successful run is still executing.   
The problem is that the ETL::Engine.job in the process method in  
engine.rb is actually the job for the last dependency on all.

If I change all instances of ETL::Engine.job to just 'job' then the  
job is the right job and the control file with dependencies does get  
marked as completed.  However the Avg after_reads, Avg before_writes,  
Avg etc... stats are for the last dependency --not for all.  So maybe  
there is a better way to fix this.  Ideally the stats would be the  
average for all dependencies. Any suggestions?

Thanks
Andy

From anthonyeden at gmail.com  Thu Apr 19 17:21:36 2007
From: anthonyeden at gmail.com (Anthony Eden)
Date: Thu, 19 Apr 2007 17:21:36 -0400
Subject: [Activewarehouse-discuss] ETL job stats for a control file with
	dependencies
In-Reply-To: <893BDA3E-E1D3-4D61-89DE-0895B013C81E@chipin.com>
References: <893BDA3E-E1D3-4D61-89DE-0895B013C81E@chipin.com>
Message-ID: <bd41ed320704191421o15b16154u17609010e83e323e@mail.gmail.com>

I'll take a look into the dependencies and see if I can improve it.
The first stab at it is probably not robust enough and needs some
work.

V/r
Anthony

On 4/18/07, Andy Triboletti <andy at chipin.com> wrote:
> Hi, I have a control file which has some dependencies called all.ctl,
> and I noticed that the all.ctl row in the jobs table never
> completes.  The status after a successful run is still executing.
> The problem is that the ETL::Engine.job in the process method in
> engine.rb is actually the job for the last dependency on all.
>
> If I change all instances of ETL::Engine.job to just 'job' then the
> job is the right job and the control file with dependencies does get
> marked as completed.  However the Avg after_reads, Avg before_writes,
> Avg etc... stats are for the last dependency --not for all.  So maybe
> there is a better way to fix this.  Ideally the stats would be the
> average for all dependencies. Any suggestions?
>
> Thanks
> Andy
> _______________________________________________
> Activewarehouse-discuss mailing list
> Activewarehouse-discuss at rubyforge.org
> http://rubyforge.org/mailman/listinfo/activewarehouse-discuss
>


-- 
Cell: 808 782-5046
Current Location: Melbourne, FL

From andy at chipin.com  Thu Apr 19 17:44:28 2007
From: andy at chipin.com (Andy Triboletti)
Date: Thu, 19 Apr 2007 11:44:28 -1000
Subject: [Activewarehouse-discuss] ETL job stats for a control file with
	dependencies
In-Reply-To: <bd41ed320704191421o15b16154u17609010e83e323e@mail.gmail.com>
References: <893BDA3E-E1D3-4D61-89DE-0895B013C81E@chipin.com>
	<bd41ed320704191421o15b16154u17609010e83e323e@mail.gmail.com>
Message-ID: <1285A9C5-BF59-4095-9EE1-E7AE6D25D74E@chipin.com>

Great, thanks Anthony

Andy

On Apr 19, 2007, at 11:21 AM, Anthony Eden wrote:

> I'll take a look into the dependencies and see if I can improve it.
> The first stab at it is probably not robust enough and needs some
> work.
>
> V/r
> Anthony
>
> On 4/18/07, Andy Triboletti <andy at chipin.com> wrote:
>> Hi, I have a control file which has some dependencies called all.ctl,
>> and I noticed that the all.ctl row in the jobs table never
>> completes.  The status after a successful run is still executing.
>> The problem is that the ETL::Engine.job in the process method in
>> engine.rb is actually the job for the last dependency on all.
>>
>> If I change all instances of ETL::Engine.job to just 'job' then the
>> job is the right job and the control file with dependencies does get
>> marked as completed.  However the Avg after_reads, Avg before_writes,
>> Avg etc... stats are for the last dependency --not for all.  So maybe
>> there is a better way to fix this.  Ideally the stats would be the
>> average for all dependencies. Any suggestions?
>>
>> Thanks
>> Andy
>> _______________________________________________
>> Activewarehouse-discuss mailing list
>> Activewarehouse-discuss at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/activewarehouse-discuss
>>
>
>
> -- 
> Cell: 808 782-5046
> Current Location: Melbourne, FL


From anthonyeden at gmail.com  Sat Apr 21 15:46:27 2007
From: anthonyeden at gmail.com (Anthony Eden)
Date: Sat, 21 Apr 2007 15:46:27 -0400
Subject: [Activewarehouse-discuss] ETL job stats for a control file with
	dependencies
In-Reply-To: <1285A9C5-BF59-4095-9EE1-E7AE6D25D74E@chipin.com>
References: <893BDA3E-E1D3-4D61-89DE-0895B013C81E@chipin.com>
	<bd41ed320704191421o15b16154u17609010e83e323e@mail.gmail.com>
	<1285A9C5-BF59-4095-9EE1-E7AE6D25D74E@chipin.com>
Message-ID: <bd41ed320704211246p4769125bufb714412c8656522@mail.gmail.com>

I think that the current dependency implementation should probably be
scrapped altogether. For a project I am working on I used a shell
script and it works much better, so I am going to use that for now
until I can get enough of a grasp on the use cases to implement
something in Ruby if necessary.

V/r
Anthony

On 4/19/07, Andy Triboletti <andy at chipin.com> wrote:
> Great, thanks Anthony
>
> Andy
>
> On Apr 19, 2007, at 11:21 AM, Anthony Eden wrote:
>
> > I'll take a look into the dependencies and see if I can improve it.
> > The first stab at it is probably not robust enough and needs some
> > work.
> >
> > V/r
> > Anthony
> >
> > On 4/18/07, Andy Triboletti <andy at chipin.com> wrote:
> >> Hi, I have a control file which has some dependencies called all.ctl,
> >> and I noticed that the all.ctl row in the jobs table never
> >> completes.  The status after a successful run is still executing.
> >> The problem is that the ETL::Engine.job in the process method in
> >> engine.rb is actually the job for the last dependency on all.
> >>
> >> If I change all instances of ETL::Engine.job to just 'job' then the
> >> job is the right job and the control file with dependencies does get
> >> marked as completed.  However the Avg after_reads, Avg before_writes,
> >> Avg etc... stats are for the last dependency --not for all.  So maybe
> >> there is a better way to fix this.  Ideally the stats would be the
> >> average for all dependencies. Any suggestions?
> >>
> >> Thanks
> >> Andy
> >> _______________________________________________
> >> Activewarehouse-discuss mailing list
> >> Activewarehouse-discuss at rubyforge.org
> >> http://rubyforge.org/mailman/listinfo/activewarehouse-discuss
> >>
> >
> >
> > --
> > Cell: 808 782-5046
> > Current Location: Melbourne, FL
>
>


-- 
Cell: 808 782-5046
Current Location: Melbourne, FL