From jim.weirich at gmail.com Thu Jun 4 17:10:52 2009 From: jim.weirich at gmail.com (Jim Weirich) Date: Thu, 4 Jun 2009 17:10:52 -0400 Subject: [Rake-devel] Rake DSL Commands are no long in top level Message-ID: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> I've just pushed a new version of Rake to github under the branch 'env'. This version of rake has a number of changes from the latest released version, including better error reporting, Dave Thomas's "Ruby comments as Rake task comments" patch, task line number reporting and splitting the app out of the monolithic rake.rb file into one class per file. However, the biggest change is that the Rake DSL methods (i.e. "task", "file", "dir", "import", etc) are no longer available at the top level scope of Ruby. For one think, this means that we will no long conflict with JRuby's import statement. Rakefiles (and .rake libraries) will be loaded in a special environment that enables the DSL methods, but regular Ruby code (in .rb files) will not. This has the potential of breaking some rake libraries out there, so I'm leaving this change in the 'env' branch until I get more experience with it. If you wish to check it out, please do so and report back there. WIth this change, if you want to use the Rake DSL in regular Ruby code, you can: (1) Include the module Rake::DSL into your class, or (2) Run the code inside a Rake::DSL.environment do ... end block. Thanks. -- -- Jim Weirich -- jim.weirich at gmail.com From Patrick.Bennett at inin.com Thu Jun 4 17:38:20 2009 From: Patrick.Bennett at inin.com (Bennett, Patrick) Date: Thu, 4 Jun 2009 17:38:20 -0400 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> Message-ID: I think the 'ruby comments as rake task comments' patch is a really bad idea. I really don't like the idea of the task methods going back and reading the rakefile *every* time a task is defined to see if a comment line proceeds the task. For complex rakefiles or for code that dynamically creates tasks (ie: for dependency trees) this seems like a real performance killer for little more than a 'cute' feature IMO. Patrick -----Original Message----- From: rake-devel-bounces at rubyforge.org [mailto:rake-devel-bounces at rubyforge.org] On Behalf Of Jim Weirich Sent: Thursday, June 04, 2009 5:11 PM To: Rake Development and Discussion Subject: [Rake-devel] Rake DSL Commands are no long in top level I've just pushed a new version of Rake to github under the branch 'env'. This version of rake has [...] Dave Thomas's "Ruby comments as Rake task comments" patch [...] From dave at pragprog.com Thu Jun 4 18:11:54 2009 From: dave at pragprog.com (Dave Thomas) Date: Thu, 4 Jun 2009 17:11:54 -0500 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> Message-ID: <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> On Thu, Jun 4, 2009 at 4:38 PM, Bennett, Patrick wrote: > I think the 'ruby comments as rake task comments' patch is a really bad > idea. > I really don't like the idea of the task methods going back and reading the > rakefile *every* time a task is defined to see if a comment line proceeds > the task. > For complex rakefiles or for code that dynamically creates tasks (ie: for > dependency trees) this seems like a real performance killer for little more > than a 'cute' feature IMO. It only reads once per file. Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From Patrick.Bennett at inin.com Thu Jun 4 18:32:48 2009 From: Patrick.Bennett at inin.com (Bennett, Patrick) Date: Thu, 4 Jun 2009 18:32:48 -0400 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> Message-ID: So it caches the entire file? Doesn?t it also have to examine and parse the backtrace to even determine the file to load? Doing any of this on *every* task init bothers me ? particularly when I?m creating thousands upon thousands of task objects. Yeah, still a bad idea imo. From: rake-devel-bounces at rubyforge.org [mailto:rake-devel-bounces at rubyforge.org] On Behalf Of Dave Thomas Sent: Thursday, June 04, 2009 6:12 PM To: Rake Development and Discussion Subject: Re: [Rake-devel] Rake DSL Commands are no long in top level On Thu, Jun 4, 2009 at 4:38 PM, Bennett, Patrick > wrote: I think the 'ruby comments as rake task comments' patch is a really bad idea. I really don't like the idea of the task methods going back and reading the rakefile *every* time a task is defined to see if a comment line proceeds the task. For complex rakefiles or for code that dynamically creates tasks (ie: for dependency trees) this seems like a real performance killer for little more than a 'cute' feature IMO. It only reads once per file. Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From dave at pragprog.com Thu Jun 4 18:54:55 2009 From: dave at pragprog.com (Dave Thomas) Date: Thu, 4 Jun 2009 17:54:55 -0500 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> Message-ID: <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> On Thu, Jun 4, 2009 at 5:32 PM, Bennett, Patrick wrote: > So it caches the entire file? > > Doesn?t it also have to examine and parse the backtrace to even determine > the file to load? Doing any of this on **every** task init bothers me ? > particularly when I?m creating thousands upon thousands of task objects. > > Yeah, still a bad idea imo. > Perhaps try it and see... ? Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From Patrick.Bennett at inin.com Thu Jun 4 19:32:41 2009 From: Patrick.Bennett at inin.com (Bennett, Patrick) Date: Thu, 4 Jun 2009 19:32:41 -0400 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> Message-ID: I run builds with rake where tens if not hundreds of thousands of task objects are created and destroyed. Any additional processing which adds no direct value to the performance of the code is undesirable for me. Having non-obvious side-effects like having the comments in the code which constructs a task automatically applying as a task description is already quite dubious to me. Adding on to that the fact that to do so requires things that are hardly free: reading the entire calling script (for the build infrastructure I have where tasks are dynamically generated, these are not small files), and performing regex matches on the backtrace (again, not free) for every single task instantiation? As far as trying it? How about doing a test with 300 different rakefiles, and 30,000 tasks or so. What?s the timing on it w/o this patch, and what is it with? From: rake-devel-bounces at rubyforge.org [mailto:rake-devel-bounces at rubyforge.org] On Behalf Of Dave Thomas Sent: Thursday, June 04, 2009 6:55 PM To: Rake Development and Discussion Subject: Re: [Rake-devel] Rake DSL Commands are no long in top level On Thu, Jun 4, 2009 at 5:32 PM, Bennett, Patrick > wrote: So it caches the entire file? Doesn?t it also have to examine and parse the backtrace to even determine the file to load? Doing any of this on *every* task init bothers me ? particularly when I?m creating thousands upon thousands of task objects. Yeah, still a bad idea imo. Perhaps try it and see... ? Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From luislavena at gmail.com Thu Jun 4 19:50:58 2009 From: luislavena at gmail.com (Luis Lavena) Date: Thu, 4 Jun 2009 20:50:58 -0300 Subject: [Rake-devel] Rake test (master) - Windows results Message-ID: <71166b3b0906041650p339c8203w54f236edcb21cc4a@mail.gmail.com> Hello everybody, the following are the test results for "rake test" http://gist.github.com/123938 I'm going to test env branch on a few projects that generate on the fly tasks and RubyInstaller one which has almost 200 defined ones. Cheers, -- Luis Lavena AREA 17 - Perfection in design is achieved not when there is nothing more to add, but rather when there is nothing more to take away. Antoine de Saint-Exup?ry From dave at pragprog.com Thu Jun 4 19:51:56 2009 From: dave at pragprog.com (Dave Thomas) Date: Thu, 4 Jun 2009 18:51:56 -0500 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> Message-ID: <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> > As far as trying it? How about doing a test with 300 different rakefiles, > and 30,000 tasks or so. What?s the timing on it w/o this patch, and what is > it with? > I'll be interested to see. (Not trying to be awkward--you have that environment, while I do not). I'm not particularly attached to the patch it was done while sitting next to Jim at SoR. At the same time, I much prefer it to +desc+ in the general case, simply from a readability POV. And I'm happy to concede that, if in the general case, it harms performance, it should probably go. But perhaps some real numbers, rather than some genuine concerns, would give us the data to go on. Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexch at gmail.com Thu Jun 4 21:16:05 2009 From: alexch at gmail.com (Alex Chaffee) Date: Thu, 4 Jun 2009 18:16:05 -0700 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> Message-ID: <7a93bd340906041816g415b806bgd64a37d98c75ee76@mail.gmail.com> > I'm not particularly attached to the patch it was done while sitting next to > Jim at SoR. At the same time, I much prefer it to +desc+ in the general > case, simply from a readability POV. Really? I think the natural semantics for code comments is "these are thrown away" so it's astonishing if they're used for anything important. Having a named method in the DSL that means "this string will be visible to end-users" is much more natural and readable to me. Though "desc" is not the best name for this method -- how about "description"? Or how about an attribute in the default hash for "task", e.g. task :test, :description => "runs all tests" do ... end Speaking of which, it's always baffled and confounded me that only the tasks whose authors had the presence of mind to add a "desc" show up in "rake -T" or "rake -D". Can we make all tasks show up by default, perhaps with an empty description, with perhaps an optional ":private => true" if you want to hide them? - A From luislavena at gmail.com Thu Jun 4 21:26:10 2009 From: luislavena at gmail.com (Luis Lavena) Date: Thu, 4 Jun 2009 22:26:10 -0300 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: <7a93bd340906041816g415b806bgd64a37d98c75ee76@mail.gmail.com> References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> <7a93bd340906041816g415b806bgd64a37d98c75ee76@mail.gmail.com> Message-ID: <71166b3b0906041826k79ea36adkcaa5c1c4316be842@mail.gmail.com> On Thu, Jun 4, 2009 at 10:16 PM, Alex Chaffee wrote: >> I'm not particularly attached to the patch it was done while sitting next to >> Jim at SoR. At the same time, I much prefer it to +desc+ in the general >> case, simply from a readability POV. > > Really? I think the natural semantics for code comments is "these are > thrown away" so it's astonishing if they're used for anything > important. Having a named method in the DSL that means "this string > will be visible to end-users" is much more natural and readable to me. > Though "desc" is not the best name for this method -- how about > "description"? > > Or how about an attribute in the default hash for "task", e.g. > > task :test, :description => "runs all tests" do > ... > end > > Speaking of which, it's always baffled and confounded me that only the > tasks whose authors had the presence of mind to add a "desc" show up > in "rake -T" or "rake -D". Can we make all tasks show up by default, > perhaps with an empty description, with perhaps an optional ":private > => true" if you want to hide them? > -100 on that. By default I have 300 tasks defined in rubyinstaller, listing all them is meaningless since they don't provide a direct benefit to the user. -- Luis Lavena AREA 17 - Perfection in design is achieved not when there is nothing more to add, but rather when there is nothing more to take away. Antoine de Saint-Exup?ry From dave at pragprog.com Thu Jun 4 21:49:03 2009 From: dave at pragprog.com (Dave Thomas) Date: Thu, 4 Jun 2009 20:49:03 -0500 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: <7a93bd340906041816g415b806bgd64a37d98c75ee76@mail.gmail.com> References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> <7a93bd340906041816g415b806bgd64a37d98c75ee76@mail.gmail.com> Message-ID: <28a042850906041849q499b6bfdrc9ea320c84c03527@mail.gmail.com> > > > Really? I think the natural semantics for code comments is "these are > thrown away" so it's astonishing if they're used for anything > important. *cough*rdoc*coudh* -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexch at gmail.com Thu Jun 4 23:37:03 2009 From: alexch at gmail.com (Alex Chaffee) Date: Thu, 4 Jun 2009 20:37:03 -0700 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: <28a042850906041849q499b6bfdrc9ea320c84c03527@mail.gmail.com> References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> <7a93bd340906041816g415b806bgd64a37d98c75ee76@mail.gmail.com> <28a042850906041849q499b6bfdrc9ea320c84c03527@mail.gmail.com> Message-ID: <7a93bd340906042037r38c8e836y19846f910c0cdfcd@mail.gmail.com> Yeah, I agree: rdoc is also astonishing. Why isn't it rdoc """ This method is awesome. """ def awesome 11 end ? --- Alex Chaffee - alex at stinky.com - http://alexch.github.com Stalk me: http://friendfeed.com/alexch | http://twitter.com/alexch | http://alexch.tumblr.com On Thu, Jun 4, 2009 at 6:49 PM, Dave Thomas wrote: >> >> Really? I think the natural semantics for code comments is "these are >> thrown away" so it's astonishing if they're used for anything >> important. > > *cough*rdoc*coudh* > > _______________________________________________ > Rake-devel mailing list > Rake-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/rake-devel > From luislavena at gmail.com Thu Jun 4 23:54:29 2009 From: luislavena at gmail.com (Luis Lavena) Date: Fri, 5 Jun 2009 00:54:29 -0300 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: <7a93bd340906042037r38c8e836y19846f910c0cdfcd@mail.gmail.com> References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> <7a93bd340906041816g415b806bgd64a37d98c75ee76@mail.gmail.com> <28a042850906041849q499b6bfdrc9ea320c84c03527@mail.gmail.com> <7a93bd340906042037r38c8e836y19846f910c0cdfcd@mail.gmail.com> Message-ID: <71166b3b0906042054v64299eb7kb9d536fa3ed64506@mail.gmail.com> On Fri, Jun 5, 2009 at 12:37 AM, Alex Chaffee wrote: > Yeah, I agree: rdoc is also astonishing. Why isn't it > > rdoc """ > This method is awesome. > """ > def awesome > ?11 > end > > ? Maybe because that's not the Ruby syntax for comments? Anyhow, this is going beyond the main topic here, test the branch and send the comments with real life usage. -- Luis Lavena AREA 17 - Perfection in design is achieved not when there is nothing more to add, but rather when there is nothing more to take away. Antoine de Saint-Exup?ry From patrick.bennett at inin.com Fri Jun 5 02:21:10 2009 From: patrick.bennett at inin.com (Patrick Bennett) Date: Fri, 5 Jun 2009 02:21:10 -0400 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: <28a042850906041849q499b6bfdrc9ea320c84c03527@mail.gmail.com> References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> <7a93bd340906041816g415b806bgd64a37d98c75ee76@mail.gmail.com> <28a042850906041849q499b6bfdrc9ea320c84c03527@mail.gmail.com> Message-ID: Yes, an independent process that scans code for generating documentation, not a runtime side-effect that causes source-code to be parsed at runtime to determine something that should be explicit. :\ On Jun 4, 2009, at 9:49 PM, Dave Thomas wrote: > > Really? I think the natural semantics for code comments is "these are > thrown away" so it's astonishing if they're used for anything > important. > > *cough*rdoc*coudh* > > From jim.weirich at gmail.com Fri Jun 5 03:34:12 2009 From: jim.weirich at gmail.com (Jim Weirich) Date: Fri, 5 Jun 2009 03:34:12 -0400 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> Message-ID: <57A0D6F8-CCDE-4185-9D16-1F57C80F7132@gmail.com> On Jun 4, 2009, at 5:10 PM, Jim Weirich wrote: > This has the potential of breaking some rake libraries out there, so > I'm leaving this change in the 'env' branch until I get more > experience with it. If you wish to check it out, please do so and > report back there. And (not really surprised), it breaks on rails Rakefile. Rails uses a standard Ruby load command to explicitly load all its .rake files, which bypasses the setup of the Rake environment. To tired to ponder the implications of this now. More later. -- -- Jim Weirich -- jim.weirich at gmail.com From jim.weirich at gmail.com Fri Jun 5 03:40:46 2009 From: jim.weirich at gmail.com (Jim Weirich) Date: Fri, 5 Jun 2009 03:40:46 -0400 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> Message-ID: On Jun 4, 2009, at 7:51 PM, Dave Thomas wrote: > And I'm happy to concede that, if in the general case, it harms > performance, > it should probably go. If performance turns out to be a real concern, we can disable it unless the two or three options that need it are specified. -- -- Jim Weirich -- jim.weirich at gmail.com From jim.weirich at gmail.com Fri Jun 5 10:38:58 2009 From: jim.weirich at gmail.com (Jim Weirich) Date: Fri, 5 Jun 2009 10:38:58 -0400 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> Message-ID: On Jun 4, 2009, at 7:51 PM, Dave Thomas wrote: > But perhaps some real numbers, rather than some genuine concerns, > would give > us the data to go on. So, here are some numbers. This is based on 30,000 tasks split into 300 files. The tasks do nothing but declare a single dependency. This is against the latest code base that does not do task metadata recording (i.e. comments and line numbers) unless a -T, -D or -W option is given. # USING DESC [env*]$ time ruby -I../lib ../bin/rake -Rdesc (in /Users/jim/working/git/rake/x) real 0m1.779s user 0m1.553s sys 0m0.135s # USING RUBY COMMENTS [env*]$ time ruby -I../lib ../bin/rake -Rcomment (in /Users/jim/working/git/rake/x) real 0m1.623s user 0m1.467s sys 0m0.135s # WITH NO COMMENTS [env*]$ time ruby -I../lib ../bin/rake -Rnone (in /Users/jim/working/git/rake/x) real 0m1.505s user 0m1.365s sys 0m0.120s Turns out the comment version is actually faster (because I'm not processing the desc command). I imagine the speed of the desc version could be improved if I made it obey the no task metadata flag. Running with the -T (and piping all output to /dev/null) shows that the comment version runs about 1/2 second slower than the desc version (3.8 vs 3.3 seconds). Given that these cases will be primarily dominated by the speed of the output, I'm not too worried about a slow down here. I don't see any performance reasons to take out Dave's patch. -- -- Jim Weirich -- jim.weirich at gmail.com From Patrick.Bennett at inin.com Fri Jun 5 10:52:11 2009 From: Patrick.Bennett at inin.com (Bennett, Patrick) Date: Fri, 5 Jun 2009 10:52:11 -0400 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> Message-ID: While I still find it disturbing that comments in code will be assumed to be task comments, if the performance impact is only there if an option is specified requesting task descriptions then my performance concerns are definitely lessened. Still - how is that desc - something that does nothing more than setting a string, take more time more than reading a file (a bunch of two line files isn't the best test, but you did actually test something so I can't complain too much ;>) and running a regexp parse across the caller backtrace? That really doesn't make any sense Jim. Speaking of the patch - how does it handle it when multiple tasks are created on the same line [say in a block iteration]? If there happens to be a comment on the line above they'll all get the same task description? -----Original Message----- From: rake-devel-bounces at rubyforge.org [mailto:rake-devel-bounces at rubyforge.org] On Behalf Of Jim Weirich Sent: Friday, June 05, 2009 10:39 AM To: Rake Development and Discussion Subject: Re: [Rake-devel] Rake DSL Commands are no long in top level On Jun 4, 2009, at 7:51 PM, Dave Thomas wrote: > But perhaps some real numbers, rather than some genuine concerns, > would give > us the data to go on. So, here are some numbers. This is based on 30,000 tasks split into 300 files. The tasks do nothing but declare a single dependency. This is against the latest code base that does not do task metadata recording (i.e. comments and line numbers) unless a -T, -D or -W option is given. # USING DESC [env*]$ time ruby -I../lib ../bin/rake -Rdesc (in /Users/jim/working/git/rake/x) real 0m1.779s user 0m1.553s sys 0m0.135s # USING RUBY COMMENTS [env*]$ time ruby -I../lib ../bin/rake -Rcomment (in /Users/jim/working/git/rake/x) real 0m1.623s user 0m1.467s sys 0m0.135s # WITH NO COMMENTS [env*]$ time ruby -I../lib ../bin/rake -Rnone (in /Users/jim/working/git/rake/x) real 0m1.505s user 0m1.365s sys 0m0.120s Turns out the comment version is actually faster (because I'm not processing the desc command). I imagine the speed of the desc version could be improved if I made it obey the no task metadata flag. Running with the -T (and piping all output to /dev/null) shows that the comment version runs about 1/2 second slower than the desc version (3.8 vs 3.3 seconds). Given that these cases will be primarily dominated by the speed of the output, I'm not too worried about a slow down here. I don't see any performance reasons to take out Dave's patch. -- -- Jim Weirich -- jim.weirich at gmail.com _______________________________________________ Rake-devel mailing list Rake-devel at rubyforge.org http://rubyforge.org/mailman/listinfo/rake-devel From dave at pragprog.com Fri Jun 5 11:39:25 2009 From: dave at pragprog.com (Dave Thomas) Date: Fri, 5 Jun 2009 10:39:25 -0500 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> Message-ID: <28a042850906050839n5dd0d5fcs6c056b11a450de92@mail.gmail.com> > > Speaking of the patch - how does it handle it when multiple tasks are > created on the same line [say in a block iteration]? If there happens to be > a comment on the line above they'll all get the same task description? > If you don't want that behavior, just use desc, which takes priority... Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From Patrick.Bennett at inin.com Fri Jun 5 11:59:15 2009 From: Patrick.Bennett at inin.com (Bennett, Patrick) Date: Fri, 5 Jun 2009 11:59:15 -0400 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: <28a042850906050839n5dd0d5fcs6c056b11a450de92@mail.gmail.com> References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> <28a042850906050839n5dd0d5fcs6c056b11a450de92@mail.gmail.com> Message-ID: I?d rather they didn?t have a description at all. # some rather long comment about this involved set of task creation code xxxxx.each { |y| task ?.} With this patch I?d now have to worry about where comments for the *code* have to be located so that it doesn?t interfere with the tasks I?m programmatically creating? Or sprinkle empty desc ?? calls everywhere I programmatically create tasks that I don?t want to have task descriptions? Yuck. Perhaps I?m just in a very small minority in that I use rake in a complete build system where almost all tasks are programmatically generated. I guess if I had a bunch of rakefiles with simple file, task, etc. calls in it I wouldn?t be so offended, but given that I have huge bodies of code built around programmatically generating various tasks, this patch really bothers me. I?ve more than said what I needed to say. If no one else cares there?s not much else I can say about it. I already have to patch rake locally with some of my own changes whenever a new version comes out anyway so I guess I can remove this patch as well if it?s ever introduced publicly. Cheers? Patrick From: rake-devel-bounces at rubyforge.org [mailto:rake-devel-bounces at rubyforge.org] On Behalf Of Dave Thomas Sent: Friday, June 05, 2009 11:39 AM To: Rake Development and Discussion Subject: Re: [Rake-devel] Rake DSL Commands are no long in top level Speaking of the patch - how does it handle it when multiple tasks are created on the same line [say in a block iteration]? If there happens to be a comment on the line above they'll all get the same task description? If you don't want that behavior, just use desc, which takes priority... Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim.weirich at gmail.com Fri Jun 5 11:59:40 2009 From: jim.weirich at gmail.com (Jim Weirich) Date: Fri, 5 Jun 2009 11:59:40 -0400 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> Message-ID: <29744B71-61F1-4E11-81DA-8639C2516125@gmail.com> On Jun 5, 2009, at 10:52 AM, Bennett, Patrick wrote: > Still - how is that desc - something that does nothing more than > setting a string, take more time more than reading a file Because unless you actually request an option that needs task comments, the comments are not parsed, so nothing is done. Currently, the desc method doesn't look at the "need task metadata" flag, so it still runs and stores the comment. But even if I fix that, you still have a method dispatch (that does nothing). > (a bunch of two line files isn't the best test, but you did actually > test something so I can't complain too much ;>) 300 files, each 400 lines long. Each file defining 100 tasks. Longer more complicated tasks would add to the individual task overhead times without contributing to the comment parse times. So I figure this is a worst case scenario for the Ruby comment runtime overhead. > and running a regexp parse across the caller backtrace? That really > doesn't make any sense Jim. The line number info is needed anyways to support the new -W option. Is there a better way to get the line number info? (If we had macros, we could capture the __FILE__ and __LINE__ values at the point of task definition ... never mind ... too much clojure programming lately :) Actually, another way just occurred to me. We can eval("__FILE__", block) to get the file name (and __LINE__ for the line number). However, that only works if the task supplies a block (and not all tasks do). But I don't know if it will be significantly faster, and even if we used that, we would still need a call stack search fall back for tasks without blocks. > Speaking of the patch - how does it handle it when multiple tasks > are created on the same line [say in a block iteration]? If there > happens to be a comment on the line above they'll all get the same > task description? Right. The desc technique can be customized with string interpolation for each task. The ruby comment version can't do that. Good point. And desc isn't going away, so its still available. -- -- Jim Weirich -- jim.weirich at gmail.com From dave at pragprog.com Fri Jun 5 12:15:05 2009 From: dave at pragprog.com (Dave Thomas) Date: Fri, 5 Jun 2009 11:15:05 -0500 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> <28a042850906050839n5dd0d5fcs6c056b11a450de92@mail.gmail.com> Message-ID: <28a042850906050915u7bb3debai22bb6f6c81dc6ba4@mail.gmail.com> On Fri, Jun 5, 2009 at 10:59 AM, Bennett, Patrick wrote: > I?d rather they didn?t have a description at all. > > # some rather long comment about this involved set of task creation code > > xxxxx.each { |y| task ?.} > > > > With this patch I?d now have to worry about where comments for the **code** > have to be located so that it doesn?t interfere with the tasks I?m > programmatically creating? Or sprinkle empty desc ?? calls everywhere I > programmatically create tasks that I don?t want to have task descriptions? > Actually, in this case, it wouldn't. The comment has to immediately precede the task line. I understand this is troubling to you. But perhaps you could just try it and see what happens. You could then report back the issues you encounter. Regards Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From jim.weirich at gmail.com Fri Jun 5 12:16:40 2009 From: jim.weirich at gmail.com (Jim Weirich) Date: Fri, 5 Jun 2009 12:16:40 -0400 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> <28a042850906050839n5dd0d5fcs6c056b11a450de92@mail.gmail.com> Message-ID: <6F16A9A3-6ECE-4D7C-9AFD-0B7918857E1D@gmail.com> On Jun 5, 2009, at 11:59 AM, Bennett, Patrick wrote: > I?d rather they didn?t have a description at all. > # some rather long comment about this involved set of task creation > code > xxxxx.each { |y| task ?.} yes, that comment would be attached to each task. However, # some rather long comment about this involved set of task creation code xxxxx.each { |y| task ?.} or # some rather long comment about this involved set of task creation code xxxxx.each { |y| task ?. } would not have that problem. -- -- Jim Weirich -- jim.weirich at gmail.com From alexch at gmail.com Fri Jun 5 12:14:59 2009 From: alexch at gmail.com (Alex Chaffee) Date: Fri, 5 Jun 2009 09:14:59 -0700 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> Message-ID: <7a93bd340906050914n5a2f38b5u52da20fe87377c2a@mail.gmail.com> > Anyhow, this is going beyond the main topic here, test the branch and > send the comments with real life usage. My concern with this has nothing to do with performance. I think it's a bad idea to twist the definition of the Ruby language (and programming languages in general), which says that comments are to be seen by people who use the *code*, and not by those who use the *program*. Rdoc is arguably an excusable exception here because the audience for its output is also people who are using the code, not running the program. But I'm responding to Dave's mention of rdoc comments -- I'm not suggesting an rdoc rewrite, but I see that as a slippery slope, not a useful precedent... On Fri, Jun 5, 2009 at 12:37 AM, Alex Chaffee wrote: > Yeah, I agree: rdoc is also astonishing. Why isn't it > > rdoc """ > This method is awesome. > """ > def awesome > 11 > end > > ? On Thu, Jun 4, 2009 at 8:54 PM, Luis Lavena wrote: > Maybe because that's not the Ruby syntax for comments? I think you're missing my point. Code comments have many uses, but rdoc (and rake if this patch stays in) has changed their semantics in a destructive way. If I want to put a comment in the code that's intended to describe the method to other coders -- say, "oy, this method needs to be optimized" -- then I can't put it right above the method, because then it will appear in the public API documentation. (I wonder how many rdoc pages in the world have inadvertently exposed "todo" comments?) --- Alex Chaffee - alex at stinky.com - http://alexch.github.com Stalk me: http://friendfeed.com/alexch | http://twitter.com/alexch | http://alexch.tumblr.com From jim.weirich at gmail.com Fri Jun 5 12:22:08 2009 From: jim.weirich at gmail.com (Jim Weirich) Date: Fri, 5 Jun 2009 12:22:08 -0400 Subject: [Rake-devel] Rake DSL Commands are no long in top level In-Reply-To: References: <705F3181-0BA0-4FA6-BD15-C0227A8B8528@gmail.com> <28a042850906041511o377f6f93ye43e98392ca6ed42@mail.gmail.com> <28a042850906041554s76ca01e4v94ca632b42086560@mail.gmail.com> <28a042850906041651g1b53f933vd70bbf1195d069f1@mail.gmail.com> <28a042850906050839n5dd0d5fcs6c056b11a450de92@mail.gmail.com> Message-ID: On Jun 5, 2009, at 11:59 AM, Bennett, Patrick wrote: > I?d rather they didn?t have a description at all. > # some rather long comment about this involved set of task creation > code > xxxxx.each { |y| task ?.} Actually, this /could/ be a problem with regular, non-generated rake files, where code comments inadvertently get attached to tasks. In order for the Ruby comment to become a task comment, it must immediately precede the task definition with no intervening code or blank lines. So I feel the problem is minimal. If this turns out to not be the case, please feedback to this list. Actually, I suspect the removal of task, file, import, etc. from the top level scope to be a much bigger problem. (and no one seems to be discussing that :) -- -- Jim Weirich -- jim.weirich at gmail.com From mgassmann at amberg.ch Fri Jun 5 16:32:05 2009 From: mgassmann at amberg.ch (mgassmann at amberg.ch) Date: Fri, 5 Jun 2009 22:32:05 +0200 Subject: [Rake-devel] =?iso-8859-1?q?AUTO=3A_Michael_Gassmann_is_abroad=2E?= =?iso-8859-1?q?_=28R=FCckkehr_am_22=2E06=2E2009=29?= Message-ID: Ich bin bis 22.06.2009 abwesend i'll answer your mail as soon as i am back. in urgent matters please contact Gilbert Jeiziner (gjeiziner at amberg.ch). Hinweis: Dies ist eine automatische Antwort auf Ihre Nachricht "Rake-devel Digest, Vol 49, Issue 3" gesendet am 05.06.2009 16:52:15. Diese ist die einzige Benachrichtigung, die Sie empfangen werden, w?hrend diese Person abwesend ist. From jim.weirich at gmail.com Fri Jun 5 20:45:38 2009 From: jim.weirich at gmail.com (Jim Weirich) Date: Fri, 5 Jun 2009 20:45:38 -0400 Subject: [Rake-devel] Rake test (master) - Windows results In-Reply-To: <71166b3b0906041650p339c8203w54f236edcb21cc4a@mail.gmail.com> References: <71166b3b0906041650p339c8203w54f236edcb21cc4a@mail.gmail.com> Message-ID: <81BA7AC5-04BA-49F5-AC5A-39B44A919517@gmail.com> On Jun 4, 2009, at 7:50 PM, Luis Lavena wrote: > Hello everybody, the following are the test results for "rake test" > > http://gist.github.com/123938 > > I'm going to test env branch on a few projects that generate on the > fly tasks and RubyInstaller one which has almost 200 defined ones. Should be fixed in the "env" branch. Do you mind checking? Thanks. -- -- Jim Weirich -- jim.weirich at gmail.com From tphilipp at potion-studios.com Fri Jun 19 06:08:19 2009 From: tphilipp at potion-studios.com (Tassilo Philipp) Date: Fri, 19 Jun 2009 05:08:19 -0500 Subject: [Rake-devel] 'rule' ignored (bump) Message-ID: <20090619050819.ef7d32fa.tphilipp@potion-studios.com> Greetings, I'm sorry to bump my initial question - usually I don't like bumping requests, but I'm still confused about the behaviour mentioned in my original mail, and I would definitely like to hear some opinions about this. I'm happy to come up with a patch if the current behaviour is not the expected one, but in order to do so I need to know if it is correct the way it is, or, what behaviour to expect from the sample rakefile below... Anyhow, here's my initial mail: Hi, I have an issue with a rule in one of my rakefiles and don't really know what behaviour a rake user expects rake to do in that case. I'm working with 0.8.4. I have boiled it down to the following rakefile (which doesn't make too much sense, but well, it's doing fine to illustrate the problem): ------------------------------------------> file 'other.x' file 'bla.o' => ['other.x'] rule '.o' => [ proc { |tn| puts 'proc executed!'; tn+'.c' } ] do |t| puts 'writing .o file' File.open(t.to_s, 'w') {|f| } end task :default => [ "bla.o" ] <------------------------------------------ Let's assume we have a bla.o.c file (just create one for this test, content is unimportant), then, when invoking rake, the advanced rule's proc gets evaluated, and the rule's body called, and we end up with a bla.o file. When invoking rake a second time, not even the rule's proc gets evaluated anymore. However, from what I wanted to do, it should be evaluated - e.g. in order to update bla.o when the timestamp of bla.o.c has changed. In the example above, bla.o only gets created when it doesn't exist beforehand. I tried to trace this down in rake.rb, but I didn't find the exact reason, yet. Anyway, I'm not sure which one of these two possibilities (never create bla.o, or always create/update it) is the expected rake behaviour... so what do you think? Thanks! From jim.weirich at gmail.com Tue Jun 23 09:47:28 2009 From: jim.weirich at gmail.com (Jim Weirich) Date: Tue, 23 Jun 2009 09:47:28 -0400 Subject: [Rake-devel] 'rule' ignored (bump) In-Reply-To: <20090619050819.ef7d32fa.tphilipp@potion-studios.com> References: <20090619050819.ef7d32fa.tphilipp@potion-studios.com> Message-ID: <1F15A402-C3EC-4CFA-8378-9AAFF957C4E1@gmail.com> On Jun 19, 2009, at 6:08 AM, Tassilo Philipp wrote: > Let's assume we have a bla.o.c file (just create one for this test, > content is unimportant), then, when invoking rake, the advanced > rule's proc gets evaluated, and the rule's body called, and we end > up with a bla.o file. > When invoking rake a second time, not even the rule's proc gets > evaluated anymore. However, from what I wanted to do, it should be > evaluated - e.g. in order to update bla.o when the timestamp of > bla.o.c has changed. > In the example above, bla.o only gets created when it doesn't exist > beforehand. That is certainly the current behavior of rake. The rules only get consulted when the file does not currently exist. However, you raise an interesting point. Should the rules be consulted for time dependencies as well?. I'm not sure. One one hand, make checks the rules when just updating a file. Since the rake rules were patterned on make's, this argues that rake should do the same. On the other hand, dynamically generating dependencies is much easier in rake, so perhaps the need for rules to handle this scenario is not as great. I'm open to discussion (and/or patches) on the topic. -- -- Jim Weirich -- jim.weirich at gmail.com From tphilipp at potion-studios.com Wed Jun 24 09:45:54 2009 From: tphilipp at potion-studios.com (Tassilo Philipp) Date: Wed, 24 Jun 2009 15:45:54 +0200 Subject: [Rake-devel] 'rule' ignored (bump) In-Reply-To: <1F15A402-C3EC-4CFA-8378-9AAFF957C4E1@gmail.com> References: <20090619050819.ef7d32fa.tphilipp@potion-studios.com> <1F15A402-C3EC-4CFA-8378-9AAFF957C4E1@gmail.com> Message-ID: Hi, thanks for your answer. ATM I stick to generating the dependencies dynamically, as you pointed out below. However, the rule _does_ compare the timestamps when removing the dependency on other.x (first two lines of my example). I think this is the crucial part of my problem, that the rule itself works just fine, and its proc gets evaluated, except if there is another additional dependency - then it only gets executed when the file is nonexistant. Not even the rule's proc gets executed anymore. The real world problem I have is, that the additional dependency is a C/C++ precompiled header file, so I run into that problem quite a few times. Every translation unit in such a project depends on the PCH, and needs an inference rule. Thanks, Tassilo > > On Jun 19, 2009, at 6:08 AM, Tassilo Philipp wrote: > >> Let's assume we have a bla.o.c file (just create one for this test, >> content is unimportant), then, when invoking rake, the advanced >> rule's proc gets evaluated, and the rule's body called, and we end >> up with a bla.o file. >> When invoking rake a second time, not even the rule's proc gets >> evaluated anymore. However, from what I wanted to do, it should be >> evaluated - e.g. in order to update bla.o when the timestamp of >> bla.o.c has changed. >> In the example above, bla.o only gets created when it doesn't exist >> beforehand. > > > That is certainly the current behavior of rake. The rules only get > consulted when the file does not currently exist. > > However, you raise an interesting point. Should the rules be > consulted for time dependencies as well?. > > I'm not sure. > > One one hand, make checks the rules when just updating a file. Since > the rake rules were patterned on make's, this argues that rake should > do the same. > > On the other hand, dynamically generating dependencies is much easier > in rake, so perhaps the need for rules to handle this scenario is not > as great. > > I'm open to discussion (and/or patches) on the topic. > > -- > -- Jim Weirich > -- jim.weirich at gmail.com > > _______________________________________________ > Rake-devel mailing list > Rake-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/rake-devel > >