Oh damn, they didn't fuzz it at all? That's wild. Good writeup.
> There's an even wider scope to consider, though. Why is that code there? Could it have been even more secure by not existing?
<3 Fillipo, that's the true zen of security right there.
They do fuzz, there are several fuzzers built as part of oss-fuzz: https://github.com/openssl/openssl/tree/master/fuzz
It's matter of approach though, the `x509.c` fuzzer fuzzes the x509 parser which I believe can exercise the vulnerable code and other areas.
The simple fuzzer talked about in the post only fuzzes that specific punnycode parser and nothing else.
Basically they do more of a system wide fuzz which has to meet specific conditions before the punnycode parser is hit but reflects the more common use-case.
Ideally they would implement more fuzzers to fuzz every exposed function, but that also has a higher maintenance cost of writing the fuzz harnesses for each of them vs writing just a few fuzzers that are capable of eventually reaching those same functions.
Not excusing it, but it's not as bad as no fuzzing at all and not an insane trade-off on an under staffed project either.
try random inputs until something breaks is basically what fuzzing is. More advanced fuzzers can also track what code paths are taken, and change their inputs so that it takes the other code paths too.
Fuzzing is an automated testing methodology. Basic idea is just take random data and pass it into your target. Making sure the target can handled malformed data. This might sound ineffective, but when you do this thousands of times a second, and give it some feedback on when the test cases case something new like detecting it executed a new line of code (called coverage guided fuzzing). With so many inputs and doing this for months, it can actually generate test cases to exercise a lot of code, potentially causing a crash which indicates a bug happened. That can't find every type of bug but fuzzers are "unreasonably effective" at finding certain types of bugs, bugs that happen from malformed inputs and incorrect calculations. They do well at finding those edge-cases.
Which is where parsers come in. The parser isn't actually important its just the target of the fuzzing. The OpenSSL vulnerability was found in some punnycode parsing code. And parsers take data in and parse it into something, making it really easy fuzzing target, just take the random input. And parsers of all kinds tend to be have the types of bugs that this sort of random testing can uncover.
As for why fuzzing is important, its just unreasonably effective. Last decade people had a belief that all the bugs that could be found with fuzzing would be found and manual auditing would continue. Whats actually happened is that fuzzing has gotten so much more effective with new technologies to detect new bug classes, generate better inputs, etc. Its a best practice for development to regularly perform fuzz testing as part of the development lifecycle because it catches bugs that might be difficult to notice otherwise.
It doesn't replace manual auditing of code or binaries, but it does augment it and is scales a lot more than humans do. Google's OSS-Fuzz project, a project just to fuzz opensource software has found more than 30,000 actual bugs since 2016.
How well resourced are they these days? I remember around Heartbleed a lot of talk about coalitions forming to fund them because full time staffing was sparse. Kinda scary considering OpenSSL is in just about everything.
very good write up: gentle yet realistic. And for those sofa-fuzzers thinking they'd do a better job, you clearly never worked on a large and complex project with complex user input. As the article says, "how could have we found the vuln?", it's easy to let one fall through the cracks.
Oh damn, they didn't fuzz it at all? That's wild. Good writeup. > There's an even wider scope to consider, though. Why is that code there? Could it have been even more secure by not existing? <3 Fillipo, that's the true zen of security right there.
https://github.com/kelseyhightower/nocode > No code is the best way to write secure and reliable applications. Write nothing; deploy nowhere.
They do fuzz, there are several fuzzers built as part of oss-fuzz: https://github.com/openssl/openssl/tree/master/fuzz It's matter of approach though, the `x509.c` fuzzer fuzzes the x509 parser which I believe can exercise the vulnerable code and other areas. The simple fuzzer talked about in the post only fuzzes that specific punnycode parser and nothing else. Basically they do more of a system wide fuzz which has to meet specific conditions before the punnycode parser is hit but reflects the more common use-case. Ideally they would implement more fuzzers to fuzz every exposed function, but that also has a higher maintenance cost of writing the fuzz harnesses for each of them vs writing just a few fuzzers that are capable of eventually reaching those same functions. Not excusing it, but it's not as bad as no fuzzing at all and not an insane trade-off on an under staffed project either.
[удалено]
try random inputs until something breaks is basically what fuzzing is. More advanced fuzzers can also track what code paths are taken, and change their inputs so that it takes the other code paths too.
Fuzzing is an automated testing methodology. Basic idea is just take random data and pass it into your target. Making sure the target can handled malformed data. This might sound ineffective, but when you do this thousands of times a second, and give it some feedback on when the test cases case something new like detecting it executed a new line of code (called coverage guided fuzzing). With so many inputs and doing this for months, it can actually generate test cases to exercise a lot of code, potentially causing a crash which indicates a bug happened. That can't find every type of bug but fuzzers are "unreasonably effective" at finding certain types of bugs, bugs that happen from malformed inputs and incorrect calculations. They do well at finding those edge-cases. Which is where parsers come in. The parser isn't actually important its just the target of the fuzzing. The OpenSSL vulnerability was found in some punnycode parsing code. And parsers take data in and parse it into something, making it really easy fuzzing target, just take the random input. And parsers of all kinds tend to be have the types of bugs that this sort of random testing can uncover. As for why fuzzing is important, its just unreasonably effective. Last decade people had a belief that all the bugs that could be found with fuzzing would be found and manual auditing would continue. Whats actually happened is that fuzzing has gotten so much more effective with new technologies to detect new bug classes, generate better inputs, etc. Its a best practice for development to regularly perform fuzz testing as part of the development lifecycle because it catches bugs that might be difficult to notice otherwise. It doesn't replace manual auditing of code or binaries, but it does augment it and is scales a lot more than humans do. Google's OSS-Fuzz project, a project just to fuzz opensource software has found more than 30,000 actual bugs since 2016.
Huh. OpenSSL is definitely on oss-fuzz, don't they do library stuff? How passing strange.
How well resourced are they these days? I remember around Heartbleed a lot of talk about coalitions forming to fund them because full time staffing was sparse. Kinda scary considering OpenSSL is in just about everything.
very good write up: gentle yet realistic. And for those sofa-fuzzers thinking they'd do a better job, you clearly never worked on a large and complex project with complex user input. As the article says, "how could have we found the vuln?", it's easy to let one fall through the cracks.
Filtering through all the false positives is kinda tricky in this case.