Deployment Solution

Yesterday we upgraded to 7.5 including DS.  My existing deploy image tasks are failing and images are no longer in the deployment\task handler share. 

In the PE logs i get Message: Invalid URL failed to open image file

I cant find any of my images from before the upgrade.

Also, when i go to Settings | Deployment and Migration | DIsk Images |  <any pre-upgrade image> and click save changes i get an error similar to :

Unable to save the Package item. Reason: Invalid Package location: '\\NotificationServer\deployment\task handler\image\f3ffad95-a0c8-4ba9-b7d4-ba7d7a207f64'.

It looks like the pre-existing images and their location are blown away during the install process.  I didnt have backups of all of them but at least the most important ones I was able to bring back from my dev box.  I hope this was just a one off occurance with me. 

I can confirm this happened to us too. Luckily we tested the upgrade in a test environment first so we didn't lose production images. 

Wow.  I've heard of this, but have no documentation.  I hate to say this folks, but will you please call this in?  Especially at the beginning of an issue, we need numbers, impact, evidence, scenarios, etc., to be able to document correctly what is happening.

For example, we heard a rumor of this, so we tested it - over and over - and we can't make it happen.  So why is it happening to you and not us?  What did you do differently than we did?  Different drive letter for install?  ONLY on the SMP and no site servers?  ONLY on site servers and not on the SMP?  Hierarchy?  etc, etc..

Thanks for bringing it up.

For anyone running into this, call us, for now.  Once we figure out what it is, we'll post it here, AND we'll add  a KB to that recommended reading list.

Thanks!

Case #05320980 submitted. Talk to you in a few days once I get through the lower support tiers, I suppose. I don't know if it's related, but we had another issue with our upgrade where our NS was assigned a new GUID, which messed up our packages (resolved by following http://www.symantec.com/business/support/index?page=content&id=TECH209632). Burndtjammer, did you happen to have this issue as well?

Same thing to me.  Thank goodness that I took a snapshot.  I was able to roll back and get the images.  I still can't deploy them.  After they copy off I'm going to restore to 7.5 snapshot, put them back in place and see if I can deploy them.  This is REALLY bad!

I have adult level support so I suppose I will do the same.  Odds are they will want to look at the SIM logs from the install/upgrade.

I did not have the problem you linked to.

Guys - sorry about the images.  Read the Tech article I just posted at the top of the Forum.

Shawn, I have your case - it came almost directly to me.  I'll call you next week 'cause this week is booked.

Everyone, we're trying as fast as we can to narrow in on this.  It IS already in Dev.  I posted up to them as soon as I had some solid information, and we're chasing this on several fronts.

if you'd like to help behind-the-scenes, there are a few things we can use to help (especially if you call support):

1. Which scenario are you in (look at the tech article sticky at the top of the forum)?
2. When you upgraded, was it ONLY an SMP, or did you have site servers?  If you had site servers, did ONLY the SMP images get removed, or also the site servers?
3. Was your SMP a Task Server prior to upgrade?
4. Did your SMP have the NBS serviices set to Automatic prior to upgrade, or Manual?  Started or Stopped?
5. Did you manually roll-out any policies immediately after upgrade (e.g. NBS service sent to the NS to make it a PXE server)? or did you wait a day or so before turning things on (we think timing MAY be related)?
6. Did you have to troubleshoot anything else prior to seeing this?  Anything?  Reinstall agents?  Anything (specifically on the SMP itself)?  OR did the upgrade work clean and then BAM...  (Example, did you have PXE issues?)
7. Anythign else you can maybe think of?

The faster we can find something common, the faster we can dup it, and fix it.  I need to reiterate that we are NOT seeing this in-house, and other customers are NOT as well, so what makes this happen to some?  We don't yet know.  your help is, I promise, appreciated.

Oh, one more thing, if you call in, please work with the advanced engineers in the US vs asking for me (no prob Shawn).  I can't troubleshoot everyone's cases or no bugs or KB's will be logged.  I'm helping them all anyway, so I'll hear about it.  :P

Thanks!!!

Thank you for your active role in this.  I have my ticket in, 05321665.  Support already gave me a call.  Im on east coast so ill make sure my logs and thoughts are organized as soon as im in before the tech gives me a call back on monday.  I dont feel that my issue is resolvable in its current state, however, if it helps someone else avoid this problem ill do what i can.

Thanks everyone who called, and who did not.  We know that for many, the images are gone.  Others have been able to go to backups and/or snapshots to pull them back.  The reason the callers can possibly help is, again, so we can try to find out WHAT makes them go away, duplicate it, and then STOP it as quickly as possible.  Do you all need to call?  No.  But if anyone here finds a pattern, trust me, we're listening and want to know.

Thanks again.

I upgraded from 7.1 SP2 to 7.5 on Friday (4/4/14).  I checked the Log Viewer this morning only to discover the same issue:  Images are gone.  I have backups of everything, so no big deal, but still an issue.  For reference here's what C:\Program Files\Altiris\Altiris Agent\Agents\Deployment\Task Handler looked like before the upgrade:

Here's an after shot:

So you can see there's a significant difference.

To answer your questions:

1.  Sorry, not sure what situation, as I'm not sure the tech sticky you're referring to.

2.  Only an SMP, no site servers.

3.  We have just one instance doing everything.

4.  If you mean the 4 _Symantec_netboot_* services, all automatic

5.  I finished the upgrade to 7.5, rebooted, and immediately installed the available hotfixes.

6.  Upgrade worked clean.  There were a few of the pre-req warnings, but I can't remember what they were.  Nothing severe enough to prevent installation.

7.  A few possible thoughts:

-  Avast Antivirus was running when I started the upgrade, though I did turn it off midway through.  Nothing was flagged as blocked/malicous though.

- Step 8 of 132? took an extremely long time (1-2 hours).  So much so that I started perusing the forums for complaints.

- In Event Viewer during the above timeframe there are just under 10,000 entries for EventID 10010:  Application 'C:\Program Files\Altiris\Symantec Installation Manager\SymantecInstallationManager.exe' (pid 3232) cannot be restarted - Application SID does not match Conductor SID..

- Event Viewer shows 36 consecutive Warnings for System.ServiceModel.Install 3.0.0.0 during the installation timeframe.  They appear to be an attempt to install some .Net stuff that already existed.  This may have been during the hotfix installation though.

- Event viewer shows shows event 5189 (source:  WAS) about 5 hours into the upgrade:  The Windows Process Activation Service failed to generate an application pool config file for application pool 'Symantec Agent AppPool'. The error type is '7'. To resolve this issue, please ensure that the applicationhost.config file is correct and recommit the last configuration changes made. The data field contains the error number.

prior to above, at the beginning of the upgrade were two other WAS errors: 

• The Windows Process Activation Service received a change notification, but was unable to process it correctly. The data field contains the error number.
• The Windows Process Activation Service failed to generate an application pool config file for application pool '*'. The error type is '0'. To resolve this issue, please ensure that the applicationhost.config file is correct and recommit the last configuration changes made. The data field contains the error number.

- I did not run the "Optional" task in the ITMS 7.5 guide to Run "NS.packagerefresh"

So given the two above screenshots, besides restoring the image folder, should anything else be restored or should it automatically be rebuilt?  I'm just digging into this now, so I'm sure things are missing (PE images probably?)  What's the best advice moving forward?

First, to everyone reading this, it is FULLY EXPLECTED to have the Task Handler folder nearly empty post-upgrade.

Why?

Because all the files moved.

The short version of the answer goes like this: In DS 7.1 we were doing things the DS 6.9 way, not the SMP/NS way.  In DS 7.5 we do things the SMP/NS way - that is, we use package servers, GUIDs, all that stuff, instead of making a share point and simply mapping/calling it.  UNC mapping BTW is a security hole and some companies have UNC blocked completely.

Anyway

The simple truth is that we no longer heavily use the Task Handler folder, so looking there for your images will avail you nothing.

The tech article I posted at one point about loosing images is a bit dated now, but the risk is still there (much much lower).  MOST of what we've seen in recent months were simply a misnaming of GUID folders and if you go LOOK at the images and the image location in the console, and then go LOOK for THAT location (instead of where you memorized it in the past under Task Handler) you'll probably find your images.

GL!!

So... if the images aren't where they used to be (which as you can see I didn't reference a UNC path) and Log Viewer is throwing errors due to them being missing and a search for *.gho on the HDD turns up nothing and the path specified in the SMC is invalid (It's a UNC path pointing to \\{servername}\deployment\task handler\image\{GUID}\{imagename})...  Where did they go and where should they be?

I have backups... great.  But if I don't put them in the .\Task Handler\image\ folder, where should I put them?

That is the location they were on, on site servers.  That's the only location known by the SMP and it can't be moved because why?  Because it was on the site server.  Remember they're NOT replicated to the SMP.

Hence the backups of even site servers recommended.

That said, frankly, you will need to recapture them anyway, so why not just remove them?  Look - any image with the 7.1 agent in it wont work beyond deployment - all post-deployment tasks  will fail until the agent is upgraded.  So pretty much every image has to be updated anyway, so all the worry about WHERE the images are is... sorta wasted, right?

Just recapture them.

Then when you do, look at the new locations, and if you must, put the old images in a location LIKE that and modify the package to point to it.  Pain in the **** but it works.  I prefer just recapturing then personally since it helps everything anyway.

No, the worry isn't wasted... I would prefer to not spend unnecessary time to re-upload all the images.  They should still deploy with the old agent and upon deployment upgrade to the new Agent automatically as my current clients do.  I have Altiris DS 7.5, I have the images, but I'm missing the link between them.  I only have one server, there's no seperate site server.  I do backups anyway, but what's the point of doing them if the images are worthless?

I suppose I can just use the ol' ResourceImportTool.exe and call it a day then work on rebuilding images as I find time.

:(

Yeah, the resource import tool is the best way to recover.  If nothing else, then you get to see the actual location where the images will be.  Because your old images were on the site servers anyway, I highly suggest doing the import tool anyway.  This is actually what we suggested in support as a work-around when this happens, and I think it's what I wrote in the pinned KB that talks about this issue at the top of the DS forum.

GL!!  Remember not to run jobs after.  The clients will upgrade by policy, but the jobs may show as failed post deployment.  Just be warned.  Forward compatibility wasn't handled too well with this.

I just realized you're no longer with Symantec... wow.  Big loss for Symantec.

Thanks for your guidance, I was able to get all my images re-imported and will try deploying one shortly, being mindful of your warning. 

Somehow it wasn't clear to me until your last post that the "Tech article ... posted at the top of the Forum" was the one referenced in the pinned KB in the Deployment Solution forum, which I just found based on your latest comment.  Google fail!  :)

Well, the good news is I'm still playing on their team, just for a partner.  The bad news is that I'm no longer committed to ONLY playing on their team.  LOL

Keep posting if you struggle.  A new thread though might be better if one of these newly imported images then fail to do what you want.

I went the route of recapturing with mine but didnt have the source systems for some of my older but sometimes used images.  A shame, but we will go on.