Final goal of the security code review process is to:
improve the overall security of the product
provide output that can be used by the devs to make changes or mitigations
Each approach to software security analysis has different advantages
Advantages of Static Code Analysis
Access to the actual instructions the software will be executing
No need to guess or interpret behavior
Full access to all the software’s possible behaviors
Can find the exact location of weaknesses in the code
Can be conducted by trained software assurance developers who fully understand code
Allows quick turnaround for fixes
Relatively fast if automated tools are used
Automated tools can scan the entire code base
Automated tools can provide mitigation recommendations, reducing research time
Permits weaknesses to be found earlier in the development lifecycle, reducing the cost to fix
Limitations of Static Code Analysis
Requires access to source code or at least binary code
typically needs access to enough software artifacts to execute a build
Typically requires proficiency in running software builds
Will not find issues related to operational deployment environments
Time-consuming if conducted manually
Automated tools do not support all programming languages
Automate tools produce false positives and false negatives
Are not enough trained personnel to thoroughly conduct static code analysis
Automated tools can provide a false sense of security that everything is being addressed
Automated tools are only as good as the rules they are using to scan with
Does not find vulnerabilities introduced in the runtime environment
Advantages of Dynamic Code Analysis
No access to actual instructions being executed
tool is exercising the application
Pattern matching on requests and responses
Requires only a running system to perform a test
No requirement to have access to source code or binary code
No need to understand how to write software or execute builds
Tests a specific operational deployment
Can find infrastructure, configuration, and patch errors that static analysis tools will miss
Identifies vulnerabilities in a runtime environment
Automated tools provide flexibility on what to scan for
Allows for analysis of applications without access to the actual code
Identifies vulnerabilities that might have been false negatives in the static code analysis
Permits validation of static code analysis findings
Can be conducted on any application
Limitations of Dynamic Code Analysis
Automated tools provide a false sense of security that everything is being addressed
Automated tools produce false positives and false negatives
Automated tools are only as good as the rules they are using to scan with
not enough trained personnel to thoroughly conduct dynamic code analysis
more difficult to trace the vulnerability back to the exact location in the code, taking longer to fix the problem
Advantages of Fuzzing
the test design is extremely simple, and free of preconceptions about system behavior
systematical/random approach allows this method to find bugs that would often be missed by human eyes
Bugs found using fuzz testing are frequently severe, exploitable bugs that could be used by a real attacker
Limitations of Fuzzing
Fuzzers usually tend to find simple bugs
the more a fuzzer is protocol-aware, the fewer weird errors it will find
when you do some black-box testing, you usually attack a closed system, which increases the difficulty of evaluating the danger/impact of the found vulnerability
main problem with fuzzing to find program faults is that it generally finds only very simple fault
problem itself is exponential
every fuzzer takes shortcuts to find something interesting in a timeframe that a human cares about
Advantages of Manual Code Review
Requires no supporting technology
Can be applied to a variety of situations
Flexible
Promotes teamwork
Early in the SDLC
Limitations of Manual Code Review
Can be time-consuming
Supporting material not always available
Requires significant human thought and skill to be effective