 |
History
| Log In
|
|
|
|
why make a separate step?
seems like something that a source control could easily do. forgot to mention that I disagree w/the assumption that an update is always necessary -- when using the buildstatus source control it is quite likely to have a build where no update is required.
I think CruiseControl.NET does what Jeff tried. That is, there is a configurable chain of tasks that occur. I was thinking that SourceControls that don't need to do an update could just have a no-op method, which I think would be simpler to start with. A configurable chain sounds like something worth investigating though.
Maybe you could add an "update=true" attribute to the cvs sourcecontrol?
CC.NET does something along those lines: <sourcecontrol type="cvs"> <executable>C:\cvs\cvs.exe</executable> <workingDirectory>C:\ccnet-projects\MyTests</workingDirectory> <autoGetSource>true</autoGetSource> </sourcecontrol> I'm surprised that no one mentioned a "label" or "tagging" step in the build loop, as this also seems to be pretty common amongst the other CI tools.
For what it's worth, I had to deal with these issues with the CMSynergy plugins. I ended up doing pretty much what was described here: 1. I needed to check and possibly start a session with which to communicate with the SCM tool - *before* the bootstrappers run. I had to do this as a listener. 2. The sourcecontrol does *not* need to do an update to detect changes, but it *does* have an option to perform an update if changes were detected. This was a manditory feature as there are no working Ant or Maven tasks to do an update for CMSynergy (the Ant ones only work under windows - and not very well). 3. I added publishers to make the changes available to developers (CMSynergy equivelent of tagging). Again, no support in Ant or Maven to handle this, so I couldn't push the responsibility away from CC. It works just fine, but the whole time I was writing the code it just felt wrong. I've always wished that CC could take more of a workflow approach - where you can specify the build loop to be an arbitrary set of tasks to perform in a specified order, passing an arbitrary set of properties from one step to the next. Robert said:
"1. I needed to check and possibly start a session with which to communicate with the SCM tool - *before* the bootstrappers run. I had to do this as a listener." why couldn't you have done this step as a bootstrapper or even from your bootstrapper? of the steps you mention that the only one that really seems weird. I don't agree that it "is *always* necessary" to update. In fact, of the 3 main instances of CruiseControl that we run, only one does an update and not even on all of the project's it is running.
For example, one of the instances watches for changes in a specific documentation project and then notifies another team when changes are made via email. At no point is anything ever checked out or updated. Brad C said " Some users even seem to expect the sourcecontrol to do the update already... " I agree that this occurs, but just because some users are confused at first about the purpose of CC does not mean that CC shoudl change what it does. The solution is to update the documentation to make it very clear to new users that CC just watches for changes and then fires of a build script of your choosing when changes are detected and that it is up to you to perform any SCM specific tasks such as checkouts as part of your build. To sum up, I would agree that a majority of users use CC to watch for changes in code and then check those changes out and build the code. However, there are many other use cases that CC can/should be used for. Adding an update feature adds more unnecesary code to CC. Let Ant or other projects maintain code to perform SCM specific checkouts (because they already are). I don't understand the argument that "a majority of users use CC to watch for changes in code and then check those changes out and build the code" but then saying "Adding an update feature adds more unnecesary code to CC". If it's something that the majority of the users are doing then how can that code be unnecesary? Just because it can be done without CC doing it doesn't mean that a large number of users wouldn't benefit greatly from adding this as a feature.
Changing default behavior would be bad in any case. I think simply adding an 'update=true/false' to the source control tags that will support it would be appropriate, defaulting to false. This would also include documentation of the new attribute which would at least be a first step toward educating users about exactly what a source control can and does do. Brad Clarke said "I don't understand the argument that "a majority of users use CC to watch for changes in code and then check those changes out and build the code" but then saying "Adding an update feature adds more unnecessary code to CC"."
I will try to explain it better as I agree I did not explain as well as I could have: Performing an "update" or "checkout" is not trivial - even for users of the same SCM tool, it could be completely different. For example, we use StarTeam. If a file is reported as being in a state of "merge" should it be overwritten with whatever is in the repository? Some users may be forcing checkouts, other users not. What about uncontrolled artifacts? Should they be deleted as part of the "update"? There is no correct answer; it depends on your usage. So what would the new CC code do? My guess is that most users would expect it to match the default behavior of the corresponding Ant task (however the default behavior is not always what is desired with CC). Whether we copy code from Ant or write completely new code to get the same functionality, it is a basic Java and Object Oriented Design concept that duplicating code is bad. In fact one of the main driving forces for OO is code reuse. Adding this feature to CC would be duplicating code that already exists (and has stood the test of time) in Ant (and other build tools). I recently found a subtle yet critical bug in the StarTeam checkout task in Ant. I fixed this and submitted a patch which will likely be included in the next 1.7 release. If this feature were in CC, then the bug might have been present here too and a second patch created (assuming that the developer even knew about CC and its code). (Another good explanation of this can be seen under "Why do I need Simian?" here: http://www.redhillconsulting.com.au/products/simian/) The only time that duplicated code may be preferable is when you need a small piece of functionality but do not want to include a whole other project/library to get it. However, this is not the case with CC - it ships with Ant! Brad Clarke said "If it's something that the majority of the users are doing then how can that code be unnecessary?" Because nearly everyone who is in this majority of users (those who need to do an update/checkout as part of the build that is triggered by CC) are already doing it using existing Ant (or other builder) code. Adding new code to CC when the code already exists and is in use by the majority of users is completely unnecessary. Brad Clarke said "Just because it can be done without CC doing it doesn't mean that a large number of users wouldn't benefit greatly from adding this as a feature." The only possible benefit I see is for new users who have a currently incorrect idea of what CC does. And in my last post I gave a better solution to this problem: make the documentation more clear. (Part of this documentation could include small simple Ant scripts to do an update for each SCM tool that can be used and customized to a new user's needs). Brad Clarke said "Changing default behavior would be bad in any case. I think simply adding an 'update=true/false' to the source control tags that will support it would be appropriate, defaulting to false. This would also include documentation of the new attribute which would at least be a first step toward educating users about exactly what a source control can and does do. " I agree that changing default behavior would be bad (though it really isn't that bad in this case especially if this were included in a major release number increment where BW isn't always preserved). However, again, my main concern is not BW - its unnecessarily adding complexity and duplicating existing code which we are already packaging it with CC anyway. I find it ridiculous to code SCM scripts in Ant. For development I use an IDE with excellent CVS support. The only reason I would write SCM code in Ant is so that CC could do an update or checkout. I would much rather configure that capability than code it (What about all the code duplication in my Ant scripts? Code that isn't reusable and doesn't have unit test coverage...). In fact, I'd like to see the default behavior be to update (and checkout MUST be supported!), but that switch should probably be another change for 3.0.
Jeff Ridiculous was probably an overstatement. I tend towards that sometimes--sorry. :-)
This thread has listed at least three problems that suggest a configurable build loop: 1) Update as build loop step 2) Distributed builds need different build loops on master and worker 3) Labeling There are ways to get around CC's inflexibility on 1 & 3, but 2 has no good solution. Since reworking the code to support a flexible build loop wouldn't have to change the default behavior most people wouldn't notice the difference. I continue to think it's something we should consider. Jeff After re-reading this whole thread a few times and participating in another ongoing discussion of this topic on the development list, it seems there may be a compromise here.
I personally dont care if sourcecontrol is enhanced to optionally perform the actual checkout/update of local code. I think wrapper scripts to perform the update works fine; however, as Jeff points out, not everyone is familiar or already using Ant, and therefore some are forced to learn something new to perform the update. I do think it unwise to unnnecesarily duplicate code; CC shouldn't write new code to perform these updates because this creates a whole new SCM tool intercation code base in the industry when there are already many others in existance (Ant, Maven, thirdparty tools etc..). The compromise would be to implement the update attribute of sourcecontrol to delegate to an AntBuilder which calls an update script that ships with CC and has a target for each sourcecontrol. Seen as how CC already ships with Ant, this shouldn't be an issue; end users don't need to know how it is implemented, but likewise, if they want to change the default update behavior, that can be done by simply changing the update build script rather than java code. There of course are details to be worked out (like the Ant optional jars needed by some source controls etc..) but I think it answers everyone's needs. I've been staring at my build for ages trying to get it to work properly.
It's related to my wanting to update from source control before building, hence why I'm commenting to this issue I build using Maven2, and the maven2 builder works fine on its own. However, Maven2 (unlike Ant) doesn't interact with source control, so I need a means of updating before I run the maven2 builder. When I put an exec before it to update from SubVersion (svn) before the build, I find that the svn runs but the maven2 doesn't. So: This builds (but doesn't update from svn) (example 1): {code:xml} <schedule interval="300"> <maven2 mvnhome="/usr/local/maven/maven-2.0.4" pomfile="../pom.xml" goal="clean install"/> </schedule> {code} but this updates successfully from svn but doesn't do the maven2 build (example 2): {code:xml} <schedule interval="300"> <exec workingdir=".." command="svn" args="update" /> <maven2 mvnhome="/usr/local/maven/maven-2.0.4" pomfile="../pom.xml" goal="clean install"/> </schedule> {code} I can get the build to run by using a delegating Ant script: {code:xml} <schedule interval="300"> <ant buildfile="cc-build.xml"/> </schedule> {code} ... where cc-build.xml contains: {code:xml} <project name="CruiseControl_delegator" default="updateAndBuild" basedir=".."> <target name="updateAndBuild"> <exec executable="svn"> <arg line="update" /> </exec> <exec executable="mvn"> <arg line="clean install" /> </exec> </target> </project> {code} but then I don't get the benefits of using the maven plugin, so then what's the point in having a maven plugin if I'm forced to delegate to ant? Am I missing something? Does example 2 above look like it should work? "Maven2 (unlike Ant) doesn't interact with source control," - either I don't understand your point, or you haven't heard of Maven SCM...
Secondly, you could use a <bootstrapper> for your specific SCM. Mikie,
m2 knows about scm. just do mvn scm:update on the command line. A nice way to integrate maven2 and CruiseControl with svn is to do something like: <build> <maven2 pomfile="${project.dir}/pom.xml" goal="clean scm:update | install"/> </build> Feel free to look at the PDF hosted on http://www.coffeebreaks.org/blogs/?page_id=15 (shameless plug) Thanks for your responses, guys.
I've been otherwise distracted hence the delay in responding. I'm sure I can get this to do what I want by using scm:update (of course! D'oh!). I'm new to Maven and Cruisecontrol, so I'm sorry I'm asking simple questions. Documentation on this is fairly scarce; I'd seen your PDF before, Jerome (love the Austin Powers pic; Yeah, Baby!), and I didn't spot the scm:update in the example. Perhaps some commented example config.xml files using maven2 could be added to the Cruisecontrol site, and liked to from the config reference? There's still an issue here that I don't understand, though. I tried for ages to get the following config to do what it seems to say it does (don't worry - I'm not going to do this now - I'll use maven's scm:update goal!) <schedule interval="300"> <exec workingdir=".." command="svn" args="update" /> <maven2 mvnhome="/usr/local/maven/maven-2.0.4" pomfile="../pom.xml" goal="clean install"/> </schedule> Shouldn't that do the exec then the maven2? I found that it just did the exec then stopped without telling me why. I would expect the above to work or for there to be a message or some documentation saying why it's behaving why it is. Is it a bug or just an "expected behaviour without much documentation or useful-log-output" issue? Does it deserve a new JIRA issue? Thanks again You need to put a <builders> tag around the exec and maven2. Otherwise it chooses one of the two and the exec will always win as it has a default buildstatus of 1 [I think].
I am going to work on this feature. I will probably start with Svn and work my way to the other scms.
Once the structure is defined, patches for the other scms would be greatly appreciated. BTW, you will notice that the SvnBottstrapper doc is slightly incorrect. It already supports update of the full working directory (not just the file as the doc lets one think). So you should be able to reuse it.
| |||||||||||||||||||||||||||||||||||||||||||||
 |
|
New Feature
Open
Major
Incorporate source control update step into "build loop"
Jeff