Ross Patterson [10/Jan/08 09:57 AM] Can you please give us a sample "si mods" command that isn't handled correctly and its output?

Ryan Younker [22/Jan/08 12:31 PM] 2008-01-22 13:25:27,148 [MarginSubsidy:DEBUG] Starting process [D:\Program Files\MKS\IntegrityClient\bin\si.exe] in working directory [] with arguments [mods -R -S D:\Projects\Solutions\MarginSubsidy\project.pj --user=--password=--quiet] 2008-01-22 13:25:27,242 [MarginSubsidy:DEBUG] Subproject changed: /Fluid Connectors/Hose Products/Solutions/MarginSubsidy/Fcg.MarginSubsidy/project.pj was at checkpoint 1.2 2008-01-22 13:25:27,242 [MarginSubsidy:DEBUG] Subproject changed: /Fluid Connectors/Hose Products/Solutions/MarginSubsidy/Hpd.Collections.Generic/project.pj was at checkpoint 1.2 2008-01-22 13:25:27,257 [MarginSubsidy:DEBUG] Subproject changed: /Fluid Connectors/Hose Products/Solutions/MarginSubsidy/MarginSubsidy/project.pj was at checkpoint 1.2 2008-01-22 13:25:27,257 [MarginSubsidy:DEBUG] Member revision changed: hibernate.cfg.xml from 1.4 to 1.5 2008-01-22 13:25:27,257 [MarginSubsidy:INFO] No modifications detected.

Ryan Younker [23/Jan/08 07:00 AM] I've actually been experiencing the same issue with MKS 2006. I've been looking at the source code, and have determined exactly what the issue is. I've made a couple of changes to some of the code, and managed to get it working. Although, I've actually just stubbed in some information to trick it into working for now. First off, MODIFICATION_SEARCH_REGEX in ThoughtWorks.CruiseControl.Core.Sourcecontrol.MksHistoryParser needs to be modified to pick up the changes to members of subprojects. Below is the regex that works: ^\s*(?<ModificationType>(Member\srevision|Revision|Added|Deleted))\s(changed|member):\s+(?<Filename>.*?)(was|now|from).*\s(?<NewRevision>\d+(\.\d+)*)(\r\n|$) The next issue is that ThoughtWorks.CruiseControl.Core.Sourcecontrol.Mks.BuildMemberInfoCommand(Modification modification) doesn't take into account that the modified member doesn't belong to the root project. So, when it tries to execute si memberinfo, it uses the wrong project root. I haven't worked it out yet, but modifications to the MksHistoryParser could be made to pull the previous "Subproject changed: " line and parse it to determine the correct subproject to obtain the memberinfo from. What I did to work around the issue just to get it to work was to modify ThoughtWorks.CruiseControl.Core.Sourcecontrol.Mks.AddMemberInfo(), and commented out the execution of the memberInfoProcess, and passed in an empty StringReader into MksHistoryParser.ParseMemberInfoAndAddToModification Then, I went into ThoughtWorks.CruiseControl.Core.Sourcecontrol.MksHistoryParser.ParseMemberInfoAndAddToModification(), and since the StringReader I'm now passing in is empty, I manually assign the UserName, Comment, and ModifiedTime just to get it to process and build the project. In order for this to work, you need to set the project to checkpointOnSuccess since the ModifiedTime will not be correct. This particular hack has some undesirable effects, such as not reporting the actual username of the person that made the changes, losing the comments, and not providing the correct modified time, but it does allow cc.net to rebuild projects when they're updated in MKS. If I can set aside some time, I'll examine the remaining issues. But this is enough to get it working, if not properly, enough to avoid having to force build everything on an interval.

Ryan Younker [25/Jan/08 02:06 PM] Here are replacements for Mks.cs and MksHistoryParser.cs that have been modified to work with subprojects. They successfully pull all changes for items in subprojects, and correctly identify the modified files, who made the modification, when, and any comments attached. It's admittedly a bit of a hack. I backtracked through the results of the si mods command to find the path of the subproject, and store it in a public Dictionary<modification,string> field to make it accessible from Mks.cs. Then I call a slightly different si memberinfo command to get the member info. If you intend to use these, you may want to look at the MEMBER_INFO_REGEX in MksHistoryParser.cs to ensure that it's pulling the info you want. I was only after the userid of the person that made the change, and our implementation of MKS provides the full name of the user followed by the userid in parens.

Drew Rhoades [04/Mar/08 02:59 PM] We also ran into CC.Net's lack of modified member information when run with MKS. However, Ryan's solution didn't quite address our problem, although it was a good start (it got me looking at the right area of the code to start with). His solution not working for us is probably attributable to several factors: - We're running MKS Integrity 2007 Service Pack 001-03 (Build 4.9.0.4466). There are likely some content differences from earlier versions, particularly in the output for the "si mods" command. Modification dates appear in a different format than was expected in Ryan's code, and "si mods" reports "Dropped" members, not "Deleted" members. - Ryan's solution didn't account for added or dropped subprojects. It also didn't seem to pickup subprojects more than one level deep (only picked up the top-level subprojects). At least in our implementation, every folder under our main project is represented as a subproject, so we have several nesting levels of subprojects. Since these weren't being picked up, the ultimate filepath associated with the changed members was incorrect. I'm attaching the updated MksHistoryParser.cs and Mks.cs files (prefixed with MKS2007), along with the sample output from the "si mods" and "si memberinfo" commands that I tested with (contains adds, changes, and drops).

Ross Patterson [04/Mar/08 03:16 PM] I'm returning this issue to the unassigned pool. It's obviously more complex than it looked at first and I don't have the facilities to properly test any fix I might make.

Drew Rhoades [04/Mar/08 05:15 PM] I just ran my changes against a different project, and I uncovered an additional flaw. There's an issue if two members with the same filename, but under two different projects, both changed. The ModificationPaths collection gets a "this key already exists in the collection" error. I'm chasing it down now. Updates to follow.

Drew Rhoades [04/Mar/08 08:47 PM] Okay, I think I've shaken out the last of the issues (including an additional one I discovered). - In addressing the collection key problem, I ended up completely removing it. I took a different approach to address the problem, and it negated the need for it. In doing so, the ultimate solution is probably a little closer to the original functional intent anyway. - Part of Ryan's changes included using a "si memberinfo" command that references the project (-P) instead of a sandbox (-S). The problem with this approach is that it doesn't consider mutliple development paths on the project. When you use the project-oriented approach, you get member revision information from the normal (trunk) development path. If you're building a variant development path, and let's say you've added a new member on the variant dev path that has not yet been propagated to the trunk, when you try to get the member revision info, you will fail. It's not yet a member of the trunk project. If you use the sandbox-oriented approach, it will give you member information relative to the dev path you're building. I've attached the new-and-improved code changes, binaries, and another sample from the other (much larger) project that uncovered the additional issues. They are prefixed with "MKS2007_Take2". Sorry for pulling the trigger early on my initial changes. If possible, they should probably be removed from this issue (I don't seem to be able to do that).

Drew Rhoades [12/Mar/08 08:58 AM] After running for a week or so, found one last bug that was causing modifications to the first subproject in the "si mods" result to not get the correct project path. This then caused the "si memberinfo" command to fail. It's a one-liner, and I'm attaching the corrections (*Take3").

Daniel Hommel [13/Aug/08 03:42 AM] Attached a patch (MKS_r4005.patch) based on: - MKS2007_Take2_Mks.cs - MKS2007_Take3_MksHistoryParser.cs The patch fixes a few problems with subprojects. As i don't have access to MKS myself i can't test it, but it was testet at a customers site and it worked better than the set of changes mentioned above. MKS users please verify the patch so we can include it in the mainline.

Dave Johnson [10/Nov/08 10:26 AM] I ran into this same issue since starting at a new employer where we use MKS 2007. I found that many of the MKS si.exe commands accept an "xmlapi" parameter. I also found that the "si viewsandbox" command with the xmlapi switch provides all the information needed to see all kinds of changes (adds and deletes included). I was able to put together something that has been working for a few days now. I don't know if this will be backwards compatible with previous MKS versions since I don't have those. Either way, it avoids the use of Regex on what appears to be inconsistent command output. I'm attaching the following files. I'd like to see if these changes work for others. DJ_MKS2007_Mks.cs DJ_MKS2007_MksHistoryParser.cs DJ_MksHistory-viewsandbox-mods.xsl (add to core.csproj as an embedded resource in the same folder as Mks.cs and MksHistoryParser.cs)

Dave Johnson [10/Nov/08 10:26 AM] Update using viewsandbox command with xmlapi switch.

Dave Johnson [10/Nov/08 10:27 AM] Update using viewsandbox command with xmlapi switch.

Dave Johnson [10/Nov/08 10:27 AM] Update using viewsandbox command with xmlapi switch.