History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: CCNET-819
Type: Bug Bug
Status: Open Open
Priority: Major Major
Assignee: Craig Sutherland
Reporter: Vladimir Bliznikov
Votes: 2
Watchers: 4
Operations

If you were logged in you would be able to see more operations.
CruiseControl .NET

System.OutOfMemoryException happens while merging large xml file

Created: 17/Jan/07 09:57 PM   Updated: 13/Jan/10 12:37 PM
Component/s: Task
Affects Version/s: 1.1
Fix Version/s: None

Original Estimate: Unknown Remaining Estimate: Unknown Time Spent: Unknown
Environment: Windows Server 2003, 2GB RAM; .Net 1.1, .Net 2.0 installed.
Issue Links:
Relationship
 
This issue Relates to:
CCNET-1659 Crash when log file is big Critical Closed
CCNET-955 buildLogBuildPlugin: System.OutOfMemo... Major Closed
CCNET-1785 Out of memory exception on viewing bu... Minor Closed


 Description  « Hide
File Merge task failed to merge back large (~300 mb) xml output from MSTest.exe. Bellow is log excerpt:

Publisher threw exception: ThoughtWorks.CruiseControl.Core.CruiseControlException: Unable to read the contents of the file: d:\CCNet\WSB-4.0-AcceptanceTest\working\testresult.xml ---> System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.String.GetStringForStringBuilder(String value, Int32 startIndex, Int32 length, Int32 capacity)
   at System.Text.StringBuilder.GetNewString(String currentString, Int32 requiredLength)
   at System.Text.StringBuilder.Append(Char[] value, Int32 startIndex, Int32 charCount)
   at System.IO.StreamReader.ReadToEnd()
   at ThoughtWorks.CruiseControl.Core.Tasks.FileTaskResult.ReadFileContents(FileInfo file)
   --- End of inner exception stack trace ---
   at ThoughtWorks.CruiseControl.Core.Tasks.FileTaskResult.ReadFileContents(FileInfo file)
   at ThoughtWorks.CruiseControl.Core.Tasks.MergeFilesTask.Run(IIntegrationResult result)
   at ThoughtWorks.CruiseControl.Core.Project.PublishResults(IIntegrationResult result)

 All   Comments   Work Log   Change History      Sort Order:
Chris B [25/Feb/07 12:40 PM]
I got the same error when trying to merge a 50MB MbUnit report.
This was the error left on the ccnet console app:
2007-02-25 12:13:23,020 [V1.5:DEBUG] Exception: System.Xml.XmlException: Unexpected end of file has occurred. The following elements are not closed: Files, file, string, string, string, string, string, string, string. Line 2113, position 1.
   at System.Xml.XmlTextReaderImpl.Throw(Exception e)
   at System.Xml.XmlTextReaderImpl.Throw(String res, String arg)
   at System.Xml.XmlTextReaderImpl.ThrowUnclosedElements()
   at System.Xml.XmlTextReaderImpl.ParseElementContent()
   at System.Xml.XmlTextReaderImpl.Read()
   at System.Xml.XmlTextReader.Read()
   at System.Xml.XmlValidatingReaderImpl.Read()
   at System.Xml.XmlValidatingReader.Read()
   at System.Xml.XmlWriter.WriteNode(XmlReader reader, Boolean defattr)
   at ThoughtWorks.CruiseControl.Core.Util.XmlFragmentWriter.WriteNodeBase(XmlReader reader, Boolean defattr)
   at ThoughtWorks.CruiseControl.Core.Util.XmlFragmentWriter.WriteNode(XmlReader reader, Boolean defattr)
   at ThoughtWorks.CruiseControl.Core.Util.XmlFragmentWriter.WriteNode(String xml)

And the error waiting for me on the dashboard was this:
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown. at System.String.GetStringForStringBuilder(String value, Int32 startIndex, Int32 length, Int32 capacity) at System.Text.StringBuilder.GetNewString(String currentString, Int32 requiredLength) at System.Text.StringBuilder.Append(String value) at ThoughtWorks.CruiseControl.Core.Util.HtmlAwareMultiTransformer.Transform(String input, String[] transformerFileNames, Hashtable xsltArgs) at ThoughtWorks.CruiseControl.WebDashboard.Dashboard.PathMappingMultiTransformer.Transform(String input, String[] transformerFileNames, Hashtable xsltArgs) at ThoughtWorks.CruiseControl.WebDashboard.Dashboard.BuildRequestTransformer.Transform(IBuildSpecifier buildSpecifier, String[] transformerFileNames, Hashtable xsltArgs) at ThoughtWorks.CruiseControl.WebDashboard.Dashboard.Actions.XslReportBuildAction.Execute(ICruiseRequest cruiseRequest) at ThoughtWorks.CruiseControl.WebDashboard.MVC.Cruise.ServerCheckingProxyAction.Execute(ICruiseRequest cruiseRequest) at ThoughtWorks.CruiseControl.WebDashboard.MVC.Cruise.BuildCheckingProxyAction.Execute(ICruiseRequest cruiseRequest) at ThoughtWorks.CruiseControl.WebDashboard.MVC.Cruise.ProjectCheckingProxyAction.Execute(ICruiseRequest cruiseRequest) at ThoughtWorks.CruiseControl.WebDashboard.MVC.Cruise.CruiseActionProxyAction.Execute(IRequest request) at ThoughtWorks.CruiseControl.WebDashboard.MVC.Cruise.CachingActionProxy.Execute(IRequest request) at ThoughtWorks.CruiseControl.WebDashboard.MVC.Cruise.ExceptionCatchingActionProxy.Execute(IRequest request)


Owen Rogers [26/Feb/07 04:51 PM]
there are a number of problems with the file merging approach taken by ccnet (inherited from cruisecontrol-java) and dealing with large artifacts is definitely one of them. for now, i would recommend doing the following:
i) wherever possible, reduce the verbosity of the tools to the minimal level
ii) if feasible, increasing the RAM on the ccnet server
iii) preprocessing the artifacts (through nant or msbuild) to filter out unnecessary detail

Dhananjay Kadam [29/Aug/09 06:54 AM]
I am facing this issue with CruiseControl 1.4 SP1 and need resolution urgently. Please help me..

Dhananjay Kadam [29/Aug/09 07:00 AM]
This is been a major issue as the our client is not able to see any MSTest Results on dashboard. Is there any way from where I can achieve this?


Craig Sutherland [06/Jan/10 02:13 AM]
This is caused by the way CC.NET generates the build log. Unfortunately, there is no easy fix for this (other than the solutions outlined by Own above). There is an experimental branch with a fix in it, but as it requires major rework to the build-log generation mechanism, this will not be added in the short term (e.g. the 1.5 release).

Craig Sutherland [12/Jan/10 03:45 PM]
For the moment I have added a MemoryFailPoint to FileTaskResult (the location of the first exception). This will fail cause the build to fail, but in a much nicer way than an OutOfMemoryException (which tends to break things because there is not enough memory).