Client Management Suite

 View Only
Expand all | Collapse all

Seeing a lot of basexmlhttpcallback exceptions

Migration User

Migration UserNov 10, 2010 11:40 AM

Migration User

Migration UserDec 07, 2010 04:59 PM

Ludovic Ferre

Ludovic FerreMar 14, 2011 09:21 AM

Migration User

Migration UserApr 21, 2011 04:34 PM

Migration User

Migration UserAug 19, 2011 03:44 PM

Migration User

Migration UserAug 19, 2011 03:44 PM

Ludovic Ferre

Ludovic FerreOct 21, 2011 11:52 AM

Migration User

Migration UserNov 03, 2011 10:46 AM

Patrick Schmid

Patrick SchmidFeb 08, 2012 02:21 AM

Migration User

Migration UserFeb 28, 2012 10:00 AM

  • 1.  Seeing a lot of basexmlhttpcallback exceptions

    Posted Nov 09, 2010 11:37 AM

    I see this one:
     <event date="Nov 09 15:54:09 +00:00" severity="1" hostName="DEVICEMGMT" source="Altiris.TaskManagement.ClientTask.*" module="w3wp.exe" process="w3wp" pid="7032" thread="129" tickCount="606113671"><![CDATA[BaseXmlHttpCallback Exception: System.Exception: An exception occurred while executing the request ---> System.Exception: System.Xml.XmlException: The 'statusEvt' start tag on line 1 does not match the end tag of 'st辮'. Line 1, position 532.
       at System.Xml.XmlTextReaderImpl.Throw(Exception e)
       at System.Xml.XmlTextReaderImpl.Throw(String res, String[] args)
       at System.Xml.XmlTextReaderImpl.ThrowTagMismatch(NodeData startTag)
       at System.Xml.XmlTextReaderImpl.ParseEndElement()
       at System.Xml.XmlTextReaderImpl.ParseElementContent()
       at System.Xml.XmlTextReaderImpl.Read()
       at System.Xml.XmlTextReader.Read()
       at Altiris.DotNetLib.Xml.XmlNodeSetReader.Next()
       at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatusXml(String statusXml)
       at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatus(Hashtable input, Hashtable& output)
       at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingServer.ProcessRequestObject(Object work)
       --- End of inner exception stack trace ---
       at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingClient.SendRequestI(String service, Hashtable input)
       at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingClient.SendRequest(String service, Hashtable input)
       at Altiris.ClientTask.Server.HttpHandlers.WebClientApiManager.ProcessPostClientStatus(XmlTextWriter wr, NameValueCollection queryValues, Stream requestStream, Int32 contentLength)
       at Altiris.ClientTask.Server.HttpHandlers.PostStatus.WriteResponse(XmlTextWriter wr)
       at Altiris.TaskManagement.Common.XmlHttp.BaseXmlXmlHttpCallback.WriteResponseRaw(XmlTextWriter xwr)
       at Altiris.TaskManagement.Common.XmlHttp.BaseXmlHttpCallback.ProcessRequest(HttpContext context)]]></event>

     

    And this one over and over:

    <event date="Nov 09 15:54:09 +00:00" severity="1" hostName="DEVICEMGMT" source="Altiris.TaskManagement.ClientTask.*" module="w3wp.exe" process="w3wp" pid="7032" thread="78" tickCount="606113531"><![CDATA[BaseXmlHttpCallback Exception: System.Exception: An exception occurred while executing the request ---> System.Exception: System.Xml.XmlException: Unable to find node named "statusEvt".  Found node "" instead
       at Altiris.DotNetLib.Xml.XmlTextReaderHelper.MoveToExpectedNode(XmlReader r, String nodeName)
       at Altiris.TaskManagement.Common.ClientTask.Events.AltirisServerClientTaskRunEvent.OnFromClientXml(XmlTextReader r, String nodeName, AltirisResourceGuid resourceGuid)
       at Altiris.TaskManagement.Common.ClientTask.Events.AltirisServerClientTaskRunEvent.FromClientXml(XmlTextReader r, String nodeName, AltirisResourceGuid resourceGuid)
       at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatusXml(AltirisResourceGuid resourceGuid, String statusType, XmlTextReader r)
       at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatusXml(String statusXml)
       at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatus(Hashtable input, Hashtable& output)
       at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingServer.ProcessRequestObject(Object work)
       --- End of inner exception stack trace ---
       at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingClient.SendRequestI(String service, Hashtable input)
       at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingClient.SendRequest(String service, Hashtable input)
       at Altiris.ClientTask.Server.HttpHandlers.WebClientApiManager.ProcessPostClientStatus(XmlTextWriter wr, NameValueCollection queryValues, Stream requestStream, Int32 contentLength)
       at Altiris.ClientTask.Server.HttpHandlers.PostStatus.WriteResponse(XmlTextWriter wr)
       at Altiris.TaskManagement.Common.XmlHttp.BaseXmlXmlHttpCallback.WriteResponseRaw(XmlTextWriter xwr)
       at Altiris.TaskManagement.Common.XmlHttp.BaseXmlHttpCallback.ProcessRequest(HttpContext context)]]></event>



  • 2.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Nov 10, 2010 11:09 AM

    is receiving http requests that do not contain the expected information, in some cases bad characters and in other empty one if I see the logs correctly.

    Can you check on the IIS log files what the exact parameters are?

    May be post the files here or mail them to me for a quick review.



  • 3.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Nov 10, 2010 11:40 AM
      |   view attached

    here are those IIS log files.

    Attachment(s)

    zip
    W3SVC1.zip   114.18 MB 1 version


  • 4.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Nov 10, 2010 02:39 PM

    I looked at the log files and it doesn't provide any insights, so this points to a problem in the posted data.

    The data is getting corrupted somehow so it could comes from one of the various elements in the chain:

    • The agent sending back bad information
    • Network issue causing the posted data to become corruption
    • Compression issue causing the posted data to become corrupt
    • File system / OS issue

    I have seen network corruption occur in one of my customers with an ACNS environment in place, with strange char sets being inserted in odd location similar to what you are seeing now.

    But I have also seen problems with the agent sending invalid data or with compression going wild...

    So we can't rule problems on the agent or server out yet, so the best course of action right now would be to run the Altiris profiler to trace operations and see which IP the "bad" posted data is coming from, so we can see if the problem is rooted at a single managed computers or if the error is wiedly seen (pointing to a likely communication / server / setting issue).



  • 5.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Nov 10, 2010 03:14 PM

    All of these clients are out in the wild and the server is in the DMZ.   So my guess would be network....  Thoughts



  • 6.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Nov 10, 2010 04:33 PM

    This should be a quick way to see some of the problems and qualify whether it is serious or something you can live with ;).

    But if you are unfamiliar with the profiler this could be a little tricky to setup. I think I wrote an article on using the profiler a while back... and I am sure you should be able to find some help here on Connect.



  • 7.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Nov 26, 2010 05:20 AM

    Sup Camaroguy.

    I am having the same issue at a client.

    Did you manage to resolve this issue?

    Eugene



  • 8.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Dec 07, 2010 04:59 PM

    Never did.   Still seeing a ton of these.  



  • 9.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Dec 07, 2010 06:13 PM

    Hi Camaroguy, EugeneFT,

    One option that could be worth check is Wireshark or any other network capture tool.

    The invalid data is sent to the server via the post method, so it doesn't appear on the IIS log or agent log, so only the network and application / IIS can get access to it.

    What would be worth is to check from both a client and the server whether the invalid string are crafted by the agent or made up during transmission (unlikely but not totally unheard off).



  • 10.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Jan 03, 2011 08:57 AM

    I'm recieving the exact same problem. At first I though it was an issue with agent installed incorrectly. I have reloaded Altirs CMS 3 times (tried with old and new databases) and still recieve the "statusEvt" error constantly (about every other second). I think it may have something to do with site servers. Does anyone know how to resolve this?



  • 11.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Jan 04, 2011 08:19 AM

    Hello Richard,

    The problem seem to come from the posted data (data send to NS via the post method from the Client Task Agent) that doesn't contain correctly formatted xml (invalid end tag in the first example, or missing event tag in the second one).

    As posted data is not recorded on the IIS logs the only way to see what is sent to the server would be either to run a profiler traceon the NS or to do a wireshark (network) trace.

    I would favor the network trace as it will tell us exactly what is sent to the NS and from which computer / ip. Wireshark is very nice to run and you can specify filters to only capture http traffic or only display post method call.

    From that you should be able to inspect the xml data sent to the NS and check what uis actually invalid.

    Then if you can do the same from one of the faulty agent (if the messages appear to come from only a few agents rather than many random ones) to check whether the outgoing data is correct (indicating a network corruptioon) or not (indicating an agent problem).

    I suspect it'll be an agent issue so you may want to start tracing from there.



  • 12.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Mar 03, 2011 02:54 PM

    I get a ton of these in my log too!

     

    "3/3/2011 1:08:56 PM","BaseXmlHttpCallback Exception: System.Exception: An exception occurred while executing the request ---> System.Exception: System.Xml.XmlException: Unable to find node named ""statusEvt"".  Found node """" instead
       at Altiris.DotNetLib.Xml.XmlTextReaderHelper.MoveToExpectedNode(XmlReader r, String nodeName)
       at Altiris.TaskManagement.Common.ClientTask.Events.AltirisServerClientTaskRunEvent.OnFromClientXml(XmlTextReader r, String nodeName, AltirisResourceGuid resourceGuid)
       at Altiris.TaskManagement.Common.ClientTask.Events.AltirisServerClientTaskRunEvent.FromClientXml(XmlTextReader r, String nodeName, AltirisResourceGuid resourceGuid)
       at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatusXml(AltirisResourceGuid resourceGuid, String statusType, XmlTextReader r)
       at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatusXml(String statusXml)
       at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatus(Hashtable input, Hashtable& output)
       at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingServer.ProcessRequestObject(Object work)
       --- End of inner exception stack trace ---
       at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingClient.SendRequestI(String service, Hashtable input)
       at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingClient.SendRequest(String service, Hashtable input)
       at Altiris.ClientTask.Server.HttpHandlers.WebClientApiManager.ProcessPostClientStatus(XmlTextWriter wr, NameValueCollection queryValues, Stream requestStream, Int32 contentLength)
       at Altiris.ClientTask.Server.HttpHandlers.PostStatus.WriteResponse(XmlTextWriter wr)
       at Altiris.TaskManagement.Common.XmlHttp.BaseXmlXmlHttpCallback.WriteResponseRaw(XmlTextWriter xwr)
       at Altiris.TaskManagement.Common.XmlHttp.BaseXmlHttpCallback.ProcessRequest(HttpContext context)","Altiris.TaskManagement.ClientTask.*","w3wp","24" 



  • 13.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Mar 03, 2011 05:21 PM

    I heard something about it being resolved in 7.1?

    We get it logged as an error and I'm pretty sure someone said that it was just classed incorrectly (should be informational or such).

    That said, it doesn't resolve the issue. I just hate the way it fills up the Log Viewer errors.

     



  • 14.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Mar 14, 2011 09:07 AM

    But the engineering team never managed to reproduce the issue so this was target for 7.1.

    However this doesn't mean it's solved in 7.1, just that it's not known to occur yet...

    I have a couple of customer with this issue now on various use case, so I'm looking into the details and I'm also trying to repdoruce or cpature interesting data (wireshark, taskmanagement logs).

    BTW, defect ID is etrack #1827643.

    I'll try to get this in a KB as I did not find anything on Symwise (may be my search term or brain at fault here).



  • 15.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Mar 14, 2011 09:20 AM
    Not sure if this helps, but I have removed BESR and Mobile Management Suite from my NS.


  • 16.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Mar 14, 2011 09:21 AM

    Did that solve the problem?



  • 17.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Mar 14, 2011 09:21 AM

    http://www.symantec.com/docs/TECH47665

    Probably not the best answer, but it makes sense.

    In short, these problem are seen when other (worse) issues arise on the clients. And fixing the other issues (the real problem) stops the flow of errors on the server.

    The KB marks these events to be downgraded (from error to another logging level) albeit right now I find them useful (it's clear from the red logs that something is wrong, so action must be taken to find out what and why).



  • 18.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Mar 14, 2011 09:26 AM
    No, I think it caused the problem. I am wondering of anyone else started seeing this issue after removing a solution.


  • 19.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Mar 14, 2011 11:14 AM

    I am on a remote session with a customer having these problems.

    We checked the client and server side and one thing we found was 4 xml entries under "%programfiles%\Altiris\Altiris Agent\TaskManagement\status". These xml files should exist only during the excution of a task, until the status is sent to the server.

    The files not only happened to be there, but they were all blank (full of blank spaces), despite their size of 2.7KB.

    So, we stopped the Altiris Agent, deleted the files and the broken client task agent started to work properly again (it was running in a loop before that time).

    No, for information, I traced the http request (get and post) whilst running a taks on a client computer of mine.

    Here's an xml string that is sent to the server in the post method, from the client agent:

    <status>
        <event resourceGuid="e1d4aaf1-7157-4ead-958b-464ebbf50a61" type="empty">
            <statusEvt type="empty" tig="bc5c45a5-e40e-4e76-9925-590734701c72" pig="bc5c45a5-e40e-4e76-9925-590734701c72" jng="00000000-0000-0000-0000-000000000000" tvg="759f10aa-bdb9-4f39-bb29-2be0d1213ad5">
                <output>
                    <param name="@ReturnValue" type="System.Int32">
                        <int>0</int>
                    </param>
                    <param name="@Success" type="System.Boolean">
                        <boolean>true</boolean>
                    </param>
                    <param name="PDS" type="System.String">
                        <string>Package already downloaded on 2011-03-14 15:21:32</string>
                    </param>
                    <param name="TRS" type="System.String">
                        <string> launched successfully</string>
                    </param>
                </output>
            </statusEvt>
        </event>
    </status>
    

    Which contrast from the string xml sent when the server was throwing the BaseXmlHttpCallback errors:

    <status>
        <event resourceGuid="e1d4aaf1-7157-4ead-958b-464ebbf50a61" type="empty">
        </event>
    </status>
    


  • 20.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Mar 14, 2011 11:25 AM

    We checked the client and server side and one thing we found was 4 xml entries under "%programfiles%\Altiris\Altiris Agent\TaskManagement\status.

    Did you find these files on the clients? I don't see anything on my NS server. The folder is empty.  



  • 21.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Mar 14, 2011 11:29 AM

    The problem found on the server log are caused by the information posted by the Client Task Server.



  • 22.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Mar 14, 2011 11:51 AM

    I see an empty file called <GUID>.type.ClientCreated.xml on a few of my clients. I am thinking about creating a task to just find these and remove them from all my machines and then hope these errors go away.

     

    Should I go ahead and do that?



  • 23.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Mar 14, 2011 12:05 PM

    If you are encountering the same problem as my customer then you could find that sending tasks to the computer will not work.

    In the case I worked on the CTA was looping and constantly sending empty status message, thus causing the error on the server.

    I hope you are on the lucky side and can do set the agent back to work form Altiris. I'll suggest running a BAT or VBS to stop the agent and delete the status files.

    Finally, I can comment on the KB and defect: these error messages look perfectly legitimate and downgrading them seem like a bad idea, like sweeping dust under the carpet. I'll go and create a case to pass data to the Task Server team now.



  • 24.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Mar 14, 2011 02:55 PM

    I created the following task and ran it against my environment. The amount of errors have reduceded dramatically. Is there a way to hunt down what clients are generating the error?

     

    @ECHO OFF

    IF EXIST "C:\Program Files (x86)\Altiris\Altiris Agent\TaskManagement\status" (
       CD "C:\Program Files (x86)\Altiris\Altiris Agent\TaskManagement\status"
       DEL *.XML /Q
    )

    IF EXIST "C:\Program Files\Altiris\Altiris Agent\TaskManagement\status" (
       CD "C:\Program Files\Altiris\Altiris Agent\TaskManagement\status"
       DEL *.XML /Q
    )

     

     



  • 25.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Mar 14, 2011 06:24 PM

    We'll try to see form my customer whether it's possible to run something on the affected clients or if they are <<zombies>> as the client we worked on today (the task agent was looping sending empty status events to the NS).

    It's good that you can run this from the CTA without stopping the agent or restarting.

    Actually this is something I noted on another case with the CTA, as the agent fails to refresh updated packages (including the SP5 HOWTO36013 version). Our solution there was to clean the TaskManagement cache of all the xml files in the directory (they are only a compress or encrypted - I doubt that, given it's only a subset of the client policy file - yet they have an xml extension).



  • 26.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Mar 15, 2011 08:38 PM

    %ProgramFiles%\Altiris\Altiris Agent\TaskManagement\cache

    When you open them in notepad, it's just a whole bunch of crazy characters.

    I'll look at cleaning them up and see what happens.



  • 27.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Mar 16, 2011 11:33 AM

    The xml files under the cache folder seem to be compressed entries. Their are legitimate and will never go away, as they are the local CTA copy of the policies it handles (Managed SWD etc).



  • 28.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Mar 16, 2011 06:38 PM

     

    I will refrain from deleting them.

    We get a bunch of the errors too, but I've yet to see a client with bad XML files in the status folder.

    Maybe I've just been unlucky so far.



  • 29.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Apr 05, 2011 02:29 PM

    I was working with a Support Tech and he pointed me to this Tech article..

    http://www.symantec.com/business/support/index?page=content&id=TECH144662

    He told me to run the client cleanup script on all my computers. I had to do this with DS since the task services was having issues on some machines. After running this all my errors went away. However it did add increased load onto my SQL and NS server since inventory tasks started to run again and report in.

    Run a Cleanup Script on Clients

    Create a new run script task with the following syntax:

    ---------------------------------------------------------
    REM Get the Altiris Agent install path
    FOR /F "tokens=2*" %%A IN ('REG.EXE QUERY "HKLM\Software\Altiris\Altiris Agent" /V "installdir"') DO SET AgentDir=%%B
    set tempbat=%temp%\AgentClean.bat"

    REM Create temporary batch file to execute while the agent restarts
    echo "%AgentDir%\aexagentutil" /stop > %tempbat%
    echo rmdir "%AgentDir%\TaskManagement" /s /q >> %tempbat%
    echo ping localhost -n 30 >> %tempbat%
    echo "%AgentDir%\aexagentutil" /start >> %tempbat%
    echo exit >> %tempbat%

    REM Executes temporary batch file
    start "" /MIN %tempbat%
    ---------------------------------------------------------



  • 30.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Apr 08, 2011 07:33 AM

    Hello NoodleNT,

    The script does a full clean up of the directory, so to me it looks like we are treating the symptoms rather than the root cause.

    I'm creating an etrack (defect) today as I received many files from my customer to pass on to engineering, so they can try to prevent the problem from happening.



  • 31.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Apr 20, 2011 05:28 PM

    I am still seeing these come in but it isn't as bad as before. Has anyone figured out how to pin the error message to the client that is causing it?



  • 32.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Apr 21, 2011 06:39 AM

    It's called aila (which stands for the Altiris IIS Log Analyzer) and version 0.5.0 (just released) contains code to handle exactly this (as you know, my customers are affected by this problem).

    Here are some docs to get started on it:

    I'm working on a second sample using the query shell that will address this specific issue.

    It should be on Connect by tomorrow, if all goes well ;-).

    Here's a sample output that shows the list of ip and guids causing the workload on a server:

    Ip cache top 20 entries:
        14763, 192.168.10.51
        10845, 192.168.20.58
        10350, 192.168.2.134
        10162, 192.168.14.154
        10158, 192.168.72.154
        <<data truncated here>>
    
    Guid cache top 20 entries:
        14762, 965c234c-9e45-4d19-8f83-bb9970f401d5
        10845, 62bd92f5-b1bb-41eb-899d-90e08ee2dbcb
        10350, 6a52a043-1594-41d5-bd17-669533ff6e83
        10162, 83cbe1f5-7c2d-4071-976f-fb522f699f23
        10158, 5377e478-49ee-40b3-8c42-e744c5873fc3
        <<data truncated here>>
    
    Enter a command (help or ?) or exit to quit:
    


  • 33.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Apr 21, 2011 02:32 PM

    The sample (or is it a tutorial?) with this problem in mind was made public earlier today.

    Here it is:

    https://www-secure.symantec.com/connect/articles/cwoc-2010-aila-050-samples-using-query-shell-part-ii

    As usual, let me know if you have any issues with the tool or if the articles need clarifications etc.



  • 34.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Apr 21, 2011 04:34 PM

    So how do you fix it?



  • 35.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Apr 21, 2011 05:03 PM

    If the client task agent (the machines found using aila above) is operational you can run the clean up task described earlier:

    https://www-secure.symantec.com/connect/forums/seeing-lot-basexmlhttpcallback-exceptions#comment-5298431

    If the CTA is out-of-order then you can revert to the AClient/DAgent as per earlier again ;):

    https://www-secure.symantec.com/connect/forums/seeing-lot-basexmlhttpcallback-exceptions#comment-5397391



  • 36.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Jul 07, 2011 10:09 AM

    I can't thank you guys enough for this.  My logs are finally clean.  For now...



  • 37.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Jul 07, 2011 11:51 AM
    Mine came back (Worse). Which step did you do to clean yours out? Is the agent getting in a loop because of multiple schedules on a policy?


  • 38.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Aug 18, 2011 02:02 PM

    I was hoping this would go away with 7.1.  It didn't.  I can run the tasks to clean it up, but are we any closer to finding an actual SOLUTION?



  • 39.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Aug 18, 2011 02:56 PM

    Did you do a DB upgrade or new install? I am planning on going new because DMC uninstall really messed up my DB.



  • 40.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Aug 18, 2011 04:45 PM

    ...and confirmed last week that they have a code fix for this specific problem.

    The next step will be for Engineering Management to decide how the fix is delivered to our customer.

    From what I have seen it is unlikely to make it to a point fix, however it is most likely that it will be included in the upcoming SP5 MR5.

    I'll post updates on this thread when I have more information on this.



  • 41.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Aug 19, 2011 03:44 PM

    Its a new clean install.



  • 42.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Aug 19, 2011 03:44 PM

    That doesn't sound like a 7.1 version number...



  • 43.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Oct 04, 2011 05:26 PM

    Still seeing this in my 7.0 MR4 environment. I can get it to stop for a bit if I run a job on all computers that stops the agent service, deletes the TaskManagement folder the client and then starts the Symantec Management agent service again.



  • 44.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Oct 21, 2011 11:51 AM

    Check with your local support team, this is an on-demand rollup much like we did earlier this year (it was Pointfix rollup 18, I can't remember what it applied to - may be pure SP5).



  • 45.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Oct 21, 2011 11:52 AM

    The 7.1 fix should be in SP2.



  • 46.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Oct 21, 2011 04:18 PM

    Is there a KB article pointing to this? I want to see what actually fixes it. Right now in my environment I have a daily task to delete that TASKMANAGEMENT folder else my agents stop working all together.



  • 47.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Nov 03, 2011 10:46 AM

    Is this also fixed in 7.1 somewhere?



  • 48.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Nov 03, 2011 11:19 AM

    Do this step only in DS 6.9 ONLY!

    My agents got stuck rebooting over and over and over. I have now moved over all our servers to a new CLEAN DB with the 7.1 SP1 NS. Unfortunatly I will have to manually migrate each software package over and move Recovery solution over to the new environment before I can get my clients switched over. At least I don't have my servers going crazzy and patch managment actually works now.

    At this point, I suggest all of you with this issue to create a new 7.1 environment. Don't try doing a migration. My logs look clean on the new environment. I still see these basexml errors in my old 7.0 environment.



  • 49.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Nov 07, 2011 10:52 AM

    Is there possibly a hotfix or rollup that resolves the issue???



  • 50.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Dec 21, 2011 03:49 PM

    still having this problem on 7.1 with sp2 installed



  • 51.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Dec 21, 2011 04:39 PM

    I ran this from the post above:

    @ECHO OFF

    IF EXIST "C:\Program Files (x86)\Altiris\Altiris Agent\TaskManagement\status" (
       CD "C:\Program Files (x86)\Altiris\Altiris Agent\TaskManagement\status"
       DEL *.XML /Q
    )

    IF EXIST "C:\Program Files\Altiris\Altiris Agent\TaskManagement\status" (
       CD "C:\Program Files\Altiris\Altiris Agent\TaskManagement\status"
       DEL *.XML /Q
    )

    and it looks like the error has stopped. It may come back but at least I can do something about it. The logs were being deluged with the error. Unfortunately I ran the packet capture after the script so I could not confirm that there were any files in the directory above. If I get the error again I will confirm whether there are files in the status directory prior to running the job to delete them.



  • 52.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Jan 04, 2012 08:27 AM

    In my environment I had this error, but it wasn't frequently. Now after application of SP2 and the relative upgrade of the agent and plug-in I receive a lot of this error.
    I have tried to ran the batch for cleaning the "TaskManagement\status"  directory but I have not seen any improvment.



  • 53.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Jan 06, 2012 02:09 PM

    Could you paste some error message samples here? I'm currious to see to it is exactly the same problem.



  • 54.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Jan 25, 2012 10:36 AM

    Hello,

     

    I too am seeing hundreds of these errors in my log files since upgrading the NS to 7.1 SP2, appearing at a rate of 3-4 per second.  This NS has about 600 clients and is installed on Windows Server 2008 R2 x64

     

    <event date="Jan 25 15:05:33 +00:00" severity="1" hostName="myServerName" source="Altiris.TaskManagement.ClientTask.*" module="w3wp.exe" process="w3wp" pid="10000" thread="230" tickCount="87106824"><![CDATA[BaseXmlHttpCallback Exception: System.Exception: An exception occurred while executing the request ---> System.Exception: System.Xml.XmlException: Unable to find node named "statusEvt".  Found node "" instead
       at Altiris.DotNetLib.Xml.XmlTextReaderHelper.MoveToExpectedNode(XmlReader r, String nodeName)
       at Altiris.TaskManagement.Common.ClientTask.Events.AltirisServerClientTaskRunEvent.OnFromClientXml(XmlTextReader r, String nodeName, AltirisResourceGuid resourceGuid)
       at Altiris.TaskManagement.Common.ClientTask.Events.AltirisServerClientTaskRunEvent.FromClientXml(XmlTextReader r, String nodeName, AltirisResourceGuid resourceGuid)
       at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatusXml(AltirisResourceGuid resourceGuid, String statusType, XmlTextReader r)
       at Altiris.ClientTask.Server.ClientTaskServer.ProcessPostClientStatusXml(String statusXml)
       at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingServer.ProcessRequestObject(Object work)
       --- End of inner exception stack trace ---
       at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingClient.SendRequestI(String service, Hashtable input)
       at Altiris.DotNetLib.Sockets.SecureRemoting.SecureRemotingClient.SendRequest(String service, Hashtable input)
       at Altiris.ClientTask.Server.HttpHandlers.WebClientApiManager.ProcessPostClientStatus(XmlTextWriter wr, NameValueCollection queryValues, Stream requestStream, Int32 contentLength)
       at Altiris.ClientTask.Server.HttpHandlers.PostStatus.WriteResponse(XmlTextWriter wr)
       at Altiris.TaskManagement.Common.XmlHttp.BaseXmlXmlHttpCallback.WriteResponseRaw(XmlTextWriter xwr)
       at Altiris.TaskManagement.Common.XmlHttp.BaseXmlHttpCallback.ProcessRequest(HttpContext context)]]></event>

     

    In addition, I see about a dozen errors in Windows Server Management under Roles - Web Server ....  I am not sure if they are related, but thought I'd include them here since they also come from IIS

     

    Log Name:      System
    Source:        Microsoft-Windows-WAS
    Date:          1/25/2012 3:59:14 AM
    Event ID:      5053
    Task Category: None
    Level:         Error
    Keywords:      Classic
    User:          N/A
    Computer:      myServerNameDescription:
    The Windows Process Activation Service received a change notification, but was unable to process it correctly.  The data field contains the error number.
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="Microsoft-Windows-WAS" Guid="{524B5D04-133C-4A62-8362-64E8EDB9CE40}" EventSourceName="WAS" />
        <EventID Qualifiers="49152">5053</EventID>
        <Version>0</Version>
        <Level>2</Level>
        <Task>0</Task>
        <Opcode>0</Opcode>
        <Keywords>0x80000000000000</Keywords>
        <TimeCreated SystemTime="2012-01-25T06:59:14.000000000Z" />
        <EventRecordID>13800</EventRecordID>
        <Correlation />
        <Execution ProcessID="0" ThreadID="0" />
        <Channel>System</Channel>
        <Computer>ARKMAASRV747.ark.xstrata.int</Computer>
        <Security />
      </System>
      <EventData>
        <Binary>0D000780</Binary>
      </EventData>
    </Event>

    Log Name:      System
    Source:        Microsoft-Windows-WAS
    Date:          1/25/2012 3:59:14 AM
    Event ID:      5189
    Task Category: None
    Level:         Error
    Keywords:      Classic
    User:          N/A
    Computer:      ARKMAASRV747.ark.xstrata.int
    Description:
    The Windows Process Activation Service failed to generate an application pool config file for application pool '*'. The error type is '0'. To resolve this issue, please ensure that the applicationhost.config file is correct and recommit the last configuration changes made. The data field contains the error number.
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="Microsoft-Windows-WAS" Guid="{524B5D04-133C-4A62-8362-64E8EDB9CE40}" EventSourceName="WAS" />
        <EventID Qualifiers="49152">5189</EventID>
        <Version>0</Version>
        <Level>2</Level>
        <Task>0</Task>
        <Opcode>0</Opcode>
        <Keywords>0x80000000000000</Keywords>
        <TimeCreated SystemTime="2012-01-25T06:59:14.000000000Z" />
        <EventRecordID>13799</EventRecordID>
        <Correlation />
        <Execution ProcessID="0" ThreadID="0" />
        <Channel>System</Channel>
        <Computer>ARKMAASRV747.ark.xstrata.int</Computer>
        <Security />
      </System>
      <EventData>
        <Data Name="AppPoolID">*</Data>
        <Data Name="ErrorType">0</Data>
        <Binary>0D000780</Binary>
      </EventData>
    </Event>



  • 55.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Jan 25, 2012 10:49 AM

    Did you try deleting any files in %program files%\Altiris\Altiris Agent\TaskManagement\status? That worked for me and the errors haven't come back.

    I don't think the other errors are related unless IIS is not functional on the server in question. It is hard to wade through that much noise so if you haven't tried deleting the files I would suggest trying that first to see if you can tone down how much is being reported in the log and then choose another error to work on.



  • 56.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Jan 25, 2012 10:55 AM

    But then was working with Support on an issue with RMS agents crashing. They had me uninstall PPA and then we had to re-install ITMS. After that I am seeing these alerts again.



  • 57.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Jan 25, 2012 11:17 AM

    Did you do that on the clients? It would be great if the log actually had the IP of the machine that caused the error!



  • 58.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Jan 25, 2012 11:26 AM

    I ran a script through DS, we are still runing 6.9, which deleted any files in those directories. I have not had an issue with that error since I did that and I too have upgraded to SP2 since then.



  • 59.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Jan 27, 2012 10:00 AM

    or I dunno the GUID, anything.... cause hey guess what I did not get this error until sp2 was install but now im getting tons...yet another problem after sp2 lets make this like number 8.



  • 60.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Feb 03, 2012 03:13 AM

    I had these errors too, they were logged every 2 seconds.

    In my environment it was 1 notebook that did problems after the redirection from a 7.0 to a 7.1 Server.

     

    I determined which computer it was by querying against the SQL DB and checked which client had an old Agent. After that it was easy, just hit "Update" on the client and it worked (the Client did that not for 10 hours, settings are every hour once).

     

    Maybe this helps



  • 61.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Trusted Advisor
    Posted Feb 03, 2012 03:28 PM

    we also are having the issue... every 2 seconds error in logs.  CMS SP2



  • 62.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Feb 06, 2012 06:08 PM

    Just wondering if your solution is still working for you?



  • 63.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Feb 08, 2012 02:21 AM

    yes no more errors about this in the logs



  • 64.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Feb 10, 2012 06:43 AM

    Hallo,

    a Symantec Engineer give me the follow tip to stop the flood of this message.

    1. Open IIS console, right click on the NS application pools and click recycle.
    2. Restart the following services: Altiris Client Task Data Loader, Altiris Object Host Service.



  • 65.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Trusted Advisor
    Posted Feb 10, 2012 04:31 PM

    This help anyone else?  Either I didn't do it right or it didn't help me.  Putting in a support ticket.



  • 66.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Feb 10, 2012 04:41 PM

    Restarting the data loader service and Host service were key to get mine to stop after cleaning up the status folder on all my computers. I have not recieved anymore base.xml errors since doing this.



  • 67.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Feb 13, 2012 04:16 PM

    I am using this batch file in DS to find the computers that have the status files queued. This helped me ID the comptuers that needed their agents fixed. The script will exit with an error code with the number of files found. If the agent isn't installed on the client computer you will get a -1 error code. I hope this helps you guys as much as it helped me. I found 3 machines causing all my problems.

     

    @ECHO OFF
    SET STATUS=0

    @ECHO OFF
    REM Get the Altiris Agent install path
    FOR /F "tokens=2*" %%A IN ('REG.EXE QUERY "HKLM\Software\Altiris\Altiris Agent" /V "installdir"') DO (SET AgentDir=%%B)

    FOR /F %%a in ('DIR /B "%AgentDir%\TaskManagement\status\*" ^|Find /V /C""') do (SET /a STATUS=%%a-1)

    IF %STATUS% GTR 0 (
     ECHO Files Queued:%Status%
     EXIT %Status%
    ) ELSE (
     ECHO Files Queued:%Status%
     EXIT %Status%
    )



  • 68.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Trusted Advisor
    Posted Feb 14, 2012 09:33 AM

    Anyone have tips for running something similar in 7.x?

    If I run as a quick delivery or even schedule the task, it seems to run but times out on machines that are powered off.  

    I miss the days of 6.x where I could schedule a task and it would queue until a machine came online.

    If I run as a managed software policy, I'm not seeing a good way to see results of the script?

    Support pointed me to this, which is less than helpful

    http://www.symantec.com/business/support/index?page=content&id=TECH47665



  • 69.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Feb 14, 2012 10:29 AM

    Untill Symantec comes around and splits their DS agent out of the NS agent in 7.X you will need to keep a DS box around. The number of times this thing has saved my butt is astonishing. The NS agent is no where near as stable as the DS agent. Besides you get online and offline status and queued tasks that you can't get in 7.X. (btw..I have heard that online offline status will never come to the new 7 Action Center console) You should check with your rep to see if you already own a license for DS6.9. It's also super easy to get up and running compaired to NS.



  • 70.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Feb 25, 2012 11:37 AM

    Our engineering team has done some great work correcting the behaviour of the Task Agent. The problem was fixed in 7.0 but only for a fail case, and as you all probably noticed the coming back of this issue in 7.1 was rather unpleasant (and surprising).

    The fix this time around is comprehensive, including task server switching if the current TS fails to respond and of course verification of the return code from the TS (so that we don't try to send some empty xml files for ever and ever and...).

    This code fix is in the current 7.5 base but I have asked for an inclusion to a future poinntfix rollup in the SP2 codebase.



  • 71.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Feb 28, 2012 10:00 AM

    That is good news. Thank you for the update.



  • 72.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Posted Feb 28, 2012 10:12 AM

    I am finding that I have to run the detect and cleanup job on a daily bassis to keep my NS performing at it's best.



  • 73.  RE: Seeing a lot of basexmlhttpcallback exceptions

    Broadcom Employee
    Posted Feb 29, 2012 09:15 AM

    But don't quote me on this ;-).

    I'm providing the infog because I know it's a hot topic for you guys, our beloved customer, but I have to had the usual disclaimer here - anything time line or fix inclusion is tentative... you know the drill.