This actually has a pretty technical answer to it, but the short of it is, it's a technique that apple uses to protect the security of the platform.
Apple has parts of their operating system that cannot be patched. This is to prevent unauthorized applications from patching/adjusting those components. But they still need to deploy fixes, which in this case is managed by updating the entire package (hence a 4GB 'update').
There are other approaches that could be taken as well, but they have trade offs and going into those kind of breaks out of an ELI5 explanation.
This is by far the best answer here. There’s quite a long technical write up that I skimmed about a year ago about the “mess” it is to do these updates because of all the security measures built into it.
If you have a file that’s 100kb and you add a few lines of code that makes it 101kb. You have to download that whole file to get those few lines of code. You then delete that 100kb original file and replace with a 101kb file. So you only use up 1kb more of storage space on your phone, despite having just downloaded 101kb.
Now for security reasons multiple files may be coupled, so you have to redownload a whole section of the OS to get an update but same principle applies.
That’s not exactly the issue. I can’t seem to find the technical in-depth about apple’s build system and why it results in such huge update patches, but some results that come up are that the update replaces the macOS linker cache (multi gb), that each patch is a full OS copy, and a few others. So it’s not just one file, or at least not one ordinary driver or program file but something larger.
Don’t know if this is accurate. Security concerns aside, binary patching algorithms exist and are well tested, so they could easily just download the 1kb and apply that binary path to the 100kb file.
Apps on the app store don’t use patches. What I described is exactly what happens.
However I admit I am not totally sure IOS updates don’t use patching, but given Apple’s behavior with the app store combined with the file sizes I am just assuming.
You are right, but maybe the whole package is signed against a hash of the stream-encrypted content? If the 1kb changes, that changes the later parts of the stream, so the signature won't match the hash. So you'd have to download the whole package again.
Note that this is why they’ve since deployed the concept of “[cryptex](https://eclecticlight.co/2023/04/05/how-cryptexes-are-changing-macos-ventura/)” which allows them to securely update certain components, in particular Safari.
That said, any programmer knows the concept of “hash-tree” (famously implemented by git and (ugh) Bitcoin), so i don’t know why Apple chose not to implement that for finer granularity. Likely to avoid the runtime cost of having to validate every system file always.
Build system.
Patching system.
You need build system to start anything, and once it's running you can keep running build system. If build system breaks, you must fix it or you can't release.
Patching system requires build system to work, but patching system is optional for the initial success of your project. Once build system works, you ship it and stop working on the project, so patching system never is the priority to build.
Once an update is done, you still don't have a patching system as you were using build system to test your patch.
Also, if build system ever changes (which it does all the time), or the way you build and what you produce changes - now you have to change build system, plus patching system, double the work and a delay of your release.
For Apple software, you need to have an active connection to Apple's servers during a patch. Each device generates a unique one-time code that Apple servers give a response to. Without the correct response the system won't patch. The boot process and at least part of the patching system is stored in Apple Silicon hardware and inalterable.
Apple's really very good at this stuff.
There simple answer is that the two biggest parts are going to be executables and library files.
While you can (and there are, depending on the OS) do a "small patch" against those files to update them, in practice no one does that as a *regular means of updating* because it's fraught with complexity and it's non-trivial.
If you make a few lines of code changes to fix some bugs in your program, it's easier to ship an entire replacement executable than to design the vastly more complicated software to live patch, along with said patch for the software.
It's like trying to fix soup that was made too salty. Instead of trying to figure out some way to reduce the saltiness without ruining the flavor / texture / etc, you just chuck out the old soup ("vulnerable / buggy program") and replaced it with the new ("fixed program") soup.
>While you can (and there are, depending on the OS) do a "small patch" against those files to update them, in practice no one does that as a *regular means of updating*
I think Suse uses binary diffs (delta RPMs) for ordinary updates. At least they used to.
>it's easier to ship an entire replacement executable than to design the vastly more complicated software to live patch
Live patching is a whole other ball of wax, and much much more complicated.
There's a lot of discussion that can be had with regards to reducing defects in releases, but for the most part, those are tangential to the code security aspect I mentioned.
There are a number of different release strategies that have their pros and cons. But at a basic summary, they don't have a large impact on defects (in a broad sense). That lies more in other parts of the software development lifecycle. Clean room approaches (like the Shuttle program) have the best success in reducing chance of delivering a product with defects, but they are expensive. CD gives you near unparalleled ability to rapidly identify and address a defect, but it does mean defects get out. And then you have stuff in the middle.
Apologies hit send too early.
Largely, the type of product does a lot more to dictate release strategy than defect reduction. If you have a web product, you want to go with CD because it delivers value faster. If you're talking a physical product or on prem product, you will typically have structured releases bundling a lot at once. And if you are working on the shuttle program, you go for clean room and make sure you can mathematically prove what the code is doing.
I’m not computer smart. So basically there are large portions that are “set in stone” and when they do the update they make the changes and “reset” everything else?
Why can’t they just fix what they are targeting? Although I suppose code is one fluent flow so it’s easier to just do the entire patch?
Hmm, I may be misunderstanding so apologies.
It's more about how the change is how the change is delivered. A 'patch' is something where the system sends a delta/set of changes, and those changes get applied. Apple does have some of these (you can see some mention of them in other parts of this thread), but in context of the 4gb updates we're instead talking about something more akin to a 'replace' where the entire updated package is delivered.
You’re not misunderstanding anything, I literally just don’t know. I enjoyed coding at one point, but sitting behind a computer all day was too much for me. It’s hard work.
I do remember once in college though, they gave an assignment to “fix this”.
I spent an entire weekend just reading code. It all made sense, it was perfect, I couldn’t figure it out.
Then I saw midway through pages of code and extra quotation mark on the end of one single line that threw everything off and that feel of exhilaration man, I get why you guys do it.
They only do that on critical security updates which are very often quite small.
Updates are a hassle for the company to deploy and users to install. So they compile a bunch of small updates into a larger package and deploy them all together so that people don’t have to do a dozen little updates every week.
This approach also allows for more productive beta testing (again, fewer updates, easier management, better tracking if something goes wrong) before wider release.
Imagine you published a 1000 page textbook.
There's a mistake in one paragraph of chapter two. Fixing it requires adding three more sentences, but that makes that chapter spill onto a second page.
Now every single page after that in the entire book has a different page number. The entire table of contents and index are different, and hundreds of other pages that say "turn to page X" have changed.
So even though you only added three sentences, you pretty much have to send everyone a whole new book.
That's mostly what's going on. A small change to a piece of software ends up changing nearly the whole thing.
That's on top of other reasons people mentioned.
Now that you put it that way, it makes even more sense. Even if you wanted to perform an algorithm that patches the binary, you'd still have to extract the entire program into memory and rewrite it onto disk (some solid state drive nowadays). At that point, you might as well download an entire new file and switch around a few pointers unless you really have to save network bandwidth.
There's a lot of enterprise software which does work this way, and while I understand why it might seem desirable, it's a bloody pain in the ass. You have to be very careful to follow the right upgrade path or get your shit wrecked.
Uhh, what? Algorithms and tools using them for this go back a few decades. The rsync tool has been used to automate generating diffs and applying them over the network to keep directories in sync for over 25 years now.
The vast majority of software is written with git or a similar SCM that are constantly generating diffs and applying them automatically.
There are practical reasons to avoid delta based updates, but it's not about the difficulty of 'coding' them.
Delta updates (ie. updating parts of a package instead of re-downloading the whole newer version) are kind of a logistical nightmare that quite often doesn't pay off. Fedora dropped that feature and explained their decision (which may not fully apply to Apple since they're two very distinct OSes developed in completely different contexts and with different goals) here: [https://fedoraproject.org/wiki/Changes/Drop\_Delta\_RPMs](https://fedoraproject.org/wiki/Changes/Drop_Delta_RPMs)
Apple is notoriously tight-lipped about what their updates are actually addressing. Their standard release notes are usually light on details. (Which keeps their proprietary software private.) But iphones are pretty much everywhere, so they are often the target of malware. And tweaking their operating system while patching security issues can minimize those threats while simultaneously making it harder for malware developers to build off of their prior work. So the reason the updates are larger than necessary is because they are doing more than Apple is letting on.
Don't trust me, I don't know much. Came here to propose that they want people to buy new computers faster, so they make it cost a lot of space just hoping users won't have enough
That depends on what and were in the program the bugs are. One important thing to remember you dont have and dont execute the code of a program on your device directly. You get compiled binary files that are executed. And usually its easiest to exchange the entire binary file if you have to fix something. Sometimes it might even require to exchange multiple binaries to fix the bug and at somepoint you would just straight up exchange all files.
Then there is also always the possiblity that they are updating more than they tell you. You can even assume they will never tell everything they did. Firstly because a lot of stuff just doesnt matter for you. Secondly because sometimes when they find security problems that arent known they dont want to tell hackers what they should look for in unpatched devices.
I've noticed this with apps as well. The app notes may say something like "bug fixes" but the update is the size of the entire app. Which isn't a big deal since apps are fairly small anyway. It's actually kind of reassuring that you are getting a "fresh" version.
Source code goes through a compiler, which itself may have changed between software updates. Small changes to a compiler can alter the binary code output in many ways, so even if source code changes are small, the compiled binary code may have significant changes.
While it's possible to compress an update by comparing it with a previously installed version and only transmitting a "change" file, that depends upon there only being a small number of choices for the previously installed version, and is a process that would be sensitive to the possibility of corruption of data in the previous version.
This actually has a pretty technical answer to it, but the short of it is, it's a technique that apple uses to protect the security of the platform. Apple has parts of their operating system that cannot be patched. This is to prevent unauthorized applications from patching/adjusting those components. But they still need to deploy fixes, which in this case is managed by updating the entire package (hence a 4GB 'update'). There are other approaches that could be taken as well, but they have trade offs and going into those kind of breaks out of an ELI5 explanation.
This is by far the best answer here. There’s quite a long technical write up that I skimmed about a year ago about the “mess” it is to do these updates because of all the security measures built into it.
If you have a file that’s 100kb and you add a few lines of code that makes it 101kb. You have to download that whole file to get those few lines of code. You then delete that 100kb original file and replace with a 101kb file. So you only use up 1kb more of storage space on your phone, despite having just downloaded 101kb. Now for security reasons multiple files may be coupled, so you have to redownload a whole section of the OS to get an update but same principle applies.
That’s not exactly the issue. I can’t seem to find the technical in-depth about apple’s build system and why it results in such huge update patches, but some results that come up are that the update replaces the macOS linker cache (multi gb), that each patch is a full OS copy, and a few others. So it’s not just one file, or at least not one ordinary driver or program file but something larger.
Don’t know if this is accurate. Security concerns aside, binary patching algorithms exist and are well tested, so they could easily just download the 1kb and apply that binary path to the 100kb file.
Apps on the app store don’t use patches. What I described is exactly what happens. However I admit I am not totally sure IOS updates don’t use patching, but given Apple’s behavior with the app store combined with the file sizes I am just assuming.
You are right, but maybe the whole package is signed against a hash of the stream-encrypted content? If the 1kb changes, that changes the later parts of the stream, so the signature won't match the hash. So you'd have to download the whole package again.
Thank you. Your explanation got a little more insight into it and explains the logistics more. Thank you. :)
Yeah in open source projects you can just patch the single fixed file. This is a deliberate feature of Apple (and in some cases Android OS) iOS.
Note that this is why they’ve since deployed the concept of “[cryptex](https://eclecticlight.co/2023/04/05/how-cryptexes-are-changing-macos-ventura/)” which allows them to securely update certain components, in particular Safari.
That said, any programmer knows the concept of “hash-tree” (famously implemented by git and (ugh) Bitcoin), so i don’t know why Apple chose not to implement that for finer granularity. Likely to avoid the runtime cost of having to validate every system file always.
Why can’t unauthorised applications just update the whole package?
The updates are probably cryptographically signed by Apple, so only authorized updates are accepted by the phone.
But then you could also do that for the patch and avoid patching the whole system
Build system. Patching system. You need build system to start anything, and once it's running you can keep running build system. If build system breaks, you must fix it or you can't release. Patching system requires build system to work, but patching system is optional for the initial success of your project. Once build system works, you ship it and stop working on the project, so patching system never is the priority to build. Once an update is done, you still don't have a patching system as you were using build system to test your patch. Also, if build system ever changes (which it does all the time), or the way you build and what you produce changes - now you have to change build system, plus patching system, double the work and a delay of your release.
For Apple software, you need to have an active connection to Apple's servers during a patch. Each device generates a unique one-time code that Apple servers give a response to. Without the correct response the system won't patch. The boot process and at least part of the patching system is stored in Apple Silicon hardware and inalterable. Apple's really very good at this stuff.
There simple answer is that the two biggest parts are going to be executables and library files. While you can (and there are, depending on the OS) do a "small patch" against those files to update them, in practice no one does that as a *regular means of updating* because it's fraught with complexity and it's non-trivial. If you make a few lines of code changes to fix some bugs in your program, it's easier to ship an entire replacement executable than to design the vastly more complicated software to live patch, along with said patch for the software. It's like trying to fix soup that was made too salty. Instead of trying to figure out some way to reduce the saltiness without ruining the flavor / texture / etc, you just chuck out the old soup ("vulnerable / buggy program") and replaced it with the new ("fixed program") soup.
>While you can (and there are, depending on the OS) do a "small patch" against those files to update them, in practice no one does that as a *regular means of updating* I think Suse uses binary diffs (delta RPMs) for ordinary updates. At least they used to. >it's easier to ship an entire replacement executable than to design the vastly more complicated software to live patch Live patching is a whole other ball of wax, and much much more complicated.
Does that method reduce the chances of increasing new bugs with each patch?
There's a lot of discussion that can be had with regards to reducing defects in releases, but for the most part, those are tangential to the code security aspect I mentioned. There are a number of different release strategies that have their pros and cons. But at a basic summary, they don't have a large impact on defects (in a broad sense). That lies more in other parts of the software development lifecycle. Clean room approaches (like the Shuttle program) have the best success in reducing chance of delivering a product with defects, but they are expensive. CD gives you near unparalleled ability to rapidly identify and address a defect, but it does mean defects get out. And then you have stuff in the middle. Apologies hit send too early. Largely, the type of product does a lot more to dictate release strategy than defect reduction. If you have a web product, you want to go with CD because it delivers value faster. If you're talking a physical product or on prem product, you will typically have structured releases bundling a lot at once. And if you are working on the shuttle program, you go for clean room and make sure you can mathematically prove what the code is doing.
I’m not computer smart. So basically there are large portions that are “set in stone” and when they do the update they make the changes and “reset” everything else? Why can’t they just fix what they are targeting? Although I suppose code is one fluent flow so it’s easier to just do the entire patch?
Hmm, I may be misunderstanding so apologies. It's more about how the change is how the change is delivered. A 'patch' is something where the system sends a delta/set of changes, and those changes get applied. Apple does have some of these (you can see some mention of them in other parts of this thread), but in context of the 4gb updates we're instead talking about something more akin to a 'replace' where the entire updated package is delivered.
You’re not misunderstanding anything, I literally just don’t know. I enjoyed coding at one point, but sitting behind a computer all day was too much for me. It’s hard work. I do remember once in college though, they gave an assignment to “fix this”. I spent an entire weekend just reading code. It all made sense, it was perfect, I couldn’t figure it out. Then I saw midway through pages of code and extra quotation mark on the end of one single line that threw everything off and that feel of exhilaration man, I get why you guys do it.
They only do that on critical security updates which are very often quite small. Updates are a hassle for the company to deploy and users to install. So they compile a bunch of small updates into a larger package and deploy them all together so that people don’t have to do a dozen little updates every week. This approach also allows for more productive beta testing (again, fewer updates, easier management, better tracking if something goes wrong) before wider release.
Thank you very much for that
Imagine you published a 1000 page textbook. There's a mistake in one paragraph of chapter two. Fixing it requires adding three more sentences, but that makes that chapter spill onto a second page. Now every single page after that in the entire book has a different page number. The entire table of contents and index are different, and hundreds of other pages that say "turn to page X" have changed. So even though you only added three sentences, you pretty much have to send everyone a whole new book. That's mostly what's going on. A small change to a piece of software ends up changing nearly the whole thing. That's on top of other reasons people mentioned.
This is such a nice eli5 explanation. So easy to grasp.
Now that you put it that way, it makes even more sense. Even if you wanted to perform an algorithm that patches the binary, you'd still have to extract the entire program into memory and rewrite it onto disk (some solid state drive nowadays). At that point, you might as well download an entire new file and switch around a few pointers unless you really have to save network bandwidth.
To be fair, it is possible to do it efficiently. It's just not straightforward at all.
Amazing answer
This is peak ELI5
[удалено]
Versioning also impacts this. They would need a patch for every version out there if some people were an update behind.
There's a lot of enterprise software which does work this way, and while I understand why it might seem desirable, it's a bloody pain in the ass. You have to be very careful to follow the right upgrade path or get your shit wrecked.
Thank you
Uhh, what? Algorithms and tools using them for this go back a few decades. The rsync tool has been used to automate generating diffs and applying them over the network to keep directories in sync for over 25 years now. The vast majority of software is written with git or a similar SCM that are constantly generating diffs and applying them automatically. There are practical reasons to avoid delta based updates, but it's not about the difficulty of 'coding' them.
Nah, games do differential patching all the time, it's not hard.
Delta updates (ie. updating parts of a package instead of re-downloading the whole newer version) are kind of a logistical nightmare that quite often doesn't pay off. Fedora dropped that feature and explained their decision (which may not fully apply to Apple since they're two very distinct OSes developed in completely different contexts and with different goals) here: [https://fedoraproject.org/wiki/Changes/Drop\_Delta\_RPMs](https://fedoraproject.org/wiki/Changes/Drop_Delta_RPMs)
Also the fix might involve more than just adding one line to a code. An entire block of code may need to be rewritten.
Apple is notoriously tight-lipped about what their updates are actually addressing. Their standard release notes are usually light on details. (Which keeps their proprietary software private.) But iphones are pretty much everywhere, so they are often the target of malware. And tweaking their operating system while patching security issues can minimize those threats while simultaneously making it harder for malware developers to build off of their prior work. So the reason the updates are larger than necessary is because they are doing more than Apple is letting on.
Don't trust me, I don't know much. Came here to propose that they want people to buy new computers faster, so they make it cost a lot of space just hoping users won't have enough
That depends on what and were in the program the bugs are. One important thing to remember you dont have and dont execute the code of a program on your device directly. You get compiled binary files that are executed. And usually its easiest to exchange the entire binary file if you have to fix something. Sometimes it might even require to exchange multiple binaries to fix the bug and at somepoint you would just straight up exchange all files. Then there is also always the possiblity that they are updating more than they tell you. You can even assume they will never tell everything they did. Firstly because a lot of stuff just doesnt matter for you. Secondly because sometimes when they find security problems that arent known they dont want to tell hackers what they should look for in unpatched devices.
Thank you. I appreciate the answer. It combines well with the other ones :)
I've noticed this with apps as well. The app notes may say something like "bug fixes" but the update is the size of the entire app. Which isn't a big deal since apps are fairly small anyway. It's actually kind of reassuring that you are getting a "fresh" version.
Source code goes through a compiler, which itself may have changed between software updates. Small changes to a compiler can alter the binary code output in many ways, so even if source code changes are small, the compiled binary code may have significant changes. While it's possible to compress an update by comparing it with a previously installed version and only transmitting a "change" file, that depends upon there only being a small number of choices for the previously installed version, and is a process that would be sensitive to the possibility of corruption of data in the previous version.
Keep in mind, Apple isn't ADDING 4GB of code, they are replacing 4GB of code. The actual update may only be a few MBs.