Deploying software updates
You might not be familiar with a Rube Goldberg machine, a complex machine that is built of chain reactions: a ball rolls, then hits a lever, the lever sets off another reaction and so on. A fun example can be seen by the band OK GO on YouTube:
Deploying software updates to the enterprise is not unlike one of these crazy cool machines. A server downloads all the updates from Microsoft and the clients scan the server for new updates that are required. Then, the server will create a new list of updates to push, the updates are sent to Deployment Points and a new Update Deployment is scheduled to run. There’s a bit more detail in the process but you can get the idea that every domino in the process must line up correctly or the cool trick of patching will not work properly.
System Center Configuration Manager
With the release System Center Configuration Manager 2012 R2 the patching process has become easier, but it’s not an “Easy Button.” Care and preparation must be done to ensure that software patches are deployed successfully to the desired targets at the desired time. To start, we must have a solid understanding of the overall process flow of how SCCM imports, prepares, deploys and monitors software updates (how all the dominos are laid out).
The top of SCCM’s patching infrastructure is the Software Update Point (the SUP). The SUP should be set to download only the patches that meet the Classification and Products that are supported in the given environment. The synchronization schedule should be the same or more frequent as the clients Software Updates scans configure within the SCCM “Client Settings”. The server sync and the client scan work together to get the full picture of what updates are needed. The client scans the SUP database for all the available updates and any required updates are then reported back to the SUP. With the server/client schedule working like a well-timed clock, the next ball in our Rube Goldberg update machine, is Automatic Deployment Rules (ADRs). New to SCCM 2012, ADRs are SCCM’s answer to WSUS’s “automatic approval”. The trick with ADRs is to configure the schedule to run at appropriate time…and then set the deployment schedule to follow based on the desired timing.
An SCCM example
For example: Let’s assume are tasked with patching workstations and are to be deployed no earlier than Thursday at 8pm following “Patch Tuesday”. All patches must be installed to all systems by the following Monday at 6am. With those requirements the ADR should look like:
- Evaluation Schedule – custom scheduled to run Monthly, every 2nd Tuesday at 10pm (patches are released 18:00 UTC)
- Software Updates filters – Required “>0” and Superseded “No”
- Deployment Schedule – Software Available time = Specific Time:2 Days, Installation Deadline = 56 hours
- Deployment Schedule – Based on Client Local Time
- Create a new Software Update Group each time the ADR runs
Some explanation for the scheduling is required. The Evaluation Schedule is when the SUP will run the rule and download the updates that meets the filters. The above filter is set to download any update that has at least 1 client that requires the update and is NOT superseded. The deployment schedule’s “Software Available Time” is based on the time the Evaluation runs. In the above example the Evaluation runs on patch Tuesday at 10pm and the Available time is adding to that time (Tuesday at 10pm plus 2 days = Thursday at 10pm). The Installation Deadline is derived from the Software Available time so Thursday at 10pm plus 56 hours = Monday at 6am.
So it becomes clear that the Rube Goldberg analogy is not too far off.
iVision SCCM Best Practices
Before we leave the topic for today, I should mention a few “Best Practices” that we at iVision recommend:
- Classifications should be configured at the SUP level to only what is needed for the given environment – in general the following Classifications: “Critical Updates, Definition Updates (if SCEP is deployed), and Security Updates”
- Products should be configured at the SUP level to only what is needed – in general only the Operating Systems that are deployed and supported in the environment and the products that are deployed
- Automatic Updates/Deployment should only be done if a non-production “Pilot” deployment can be first to validate the given updates
- Use a single Software Update Deployment for each year – this means that all the updates for workstations and servers are downloaded to the same deployment. And note the recommended limit is 1000 updates so most environments will not hit this limit in a single year
- Leverage custom collections that are just for patching – this will be helpful when reporting on compliance
- Use maintenance windows for servers to ensure the updates happen only when they should – (more on this in a future blog)
- Only suppress reboots for those systems that require manual intervention for a reboot – if a system does not reboot when a patch requires a reboot, then the systems is not really patched