Introducing shift smart, a new approach that encourages development and security teams to seek out the best place in the SDLC to test for each type of vulnerability, rather than naively shifting everything as far left as possible.
By Jeff Williams, CTO and Co-Founder, Contrast Security
You’ve heard that fixing bugs early in the Software Development Life Cycle (SDLC) is a hundred times cheaper than bugs fixed in production. Everybody that works in information security knows that to be “true.” Unfortunately, it just isn’t so. It’s a statistic made up long ago for a presentation, later quoted in a book and now widespread. Research shows the cost to fix bugs is about the same no matter where they are fixed. The mythical 100x cost of fixing bugs in production is the basis for shift left, a principle developed more than 20 years ago that says it’s better to screen for security vulnerabilities early in the development cycle.
Shift left is a great idea. It’s better to catch bugs early, rather than wait until vulnerabilities get into production and get exploited. However, the principle of shift left has been twisted, as many helpful security ideas often are.
Good ideas like shift left (along with Zero Trust and DevSecOps) become mindless dogma: once-useful terms that have become nearly worthless buzzwords through overuse and distortion of the initial concept. If we could catch all vulnerabilities in development without wasting anybody’s time, that would be great. But we can’t. Different vulnerabilities are best detected at different points in the SDLC.
Rather than blindly shift left, organizations should deploy a shift smart strategy. Organizations should choose the timing and approach for security testing to match context — details of how an application or application programming interface (API) actually functions in production — as well as speed, accuracy and cost effectiveness.
Different vulnerabilities, different requirements
Let’s consider examples of vulnerabilities and discuss how well they fit with the shift left principle:
Hard-coded passwords. Searching files and looking for secrets in code repositories is fast, easy and accurate. Shift the search for these vulnerabilities as far left as it’ll go — but don’t stop there. Continue to test when you have a more production-like environment to check, to be sure to catch all potential problems.
SQL injection. Perennially at the top of the Open Web Application Security Project (OWASP) Top 10, these dangerous vulnerabilities resist shifting left. The best way to test for SQL injection vulnerabilities is to observe how the actual application runs. That means you need to shift right in the development process, using Interactive Application Security Testing (IAST) with the fully assembled application.
Authentication and authorization. Testing for these vulnerabilities is challenging. These functions almost certainly use custom code unique to the particular application, requiring manual testing such as pen testing. These vulnerabilities should be tested regularly, perhaps every year. Shift right. Far right.
Weak encryption algorithms. These vulnerabilities can generally be searched for in the integrated development environment (IDE) quickly, easily and accurately. Shift left. Far left.
Vulnerabilities in libraries. Open-source code is widely used in most applications, and open-source libraries seem to be good candidates for shift left. After all, it’s easy to scan a code repository.
But the reality is more complicated. Scanning every library wastes time. Many of those libraries don’t make it into the repository; they’re part of the app server or platform the app runs on. And according to Application Security (AppSec) platform metrics, 88% of open-source code never runs at all: For example, data has shown that only 37% of Java applications actually invoked the vulnerable Log4j programming library at the heart of the Log4Shell Common Vulnerability and Exposure (CVE). It’s never even loaded into memory. It’s not attackable and not at real risk.
So you want to avoid making tests on libraries that are in use but not in the repo, and also avoid testing libraries that you will never use. You can do testing early in the process, but you also want to ensure you’re not introducing libraries with known vulnerabilities later. For library vulnerabilities, you want to both shift left and shift right.
Shifting smart gives you context
A more recent buzzword than shift left is “shift everywhere,” which is even worse than blindly shifting left because it suggests that all security activities should be done during all phases of the SDLC. But as we’ve just seen, different types of vulnerabilities require screening at different stages of the development process. Security requires context to accurately identify real, exploitable vulnerabilities. Shifting smart gives you that context.
Shift smart avoids blowback from shift left and developer-first security tools, which are prone to these problems:
- Avalanches of false positives from code repositories that aren’t in use
- Wasting time on code that never runs
- Tools without enough context and generate false positives and false negatives
- Tools that sacrifice accuracy for speed
When you shift smart, make use of these guiding principles:
Bring QA and security testing together. Improve the efficiency of security testing by unifying it with quality assurance testing, which developers already do, avoiding the expense of building separate and unnecessary environments, testing rigs, coverage measures, reporting and metrics.
Modern security testing with instrumentation can add accurate security tests to existing QA activity without any changes. As you do your normal QA testing, security testing happens automatically in the background. That means that the security team and the QA team are now aligned.
Avoid wasting time and resources
Test what matters. Don’t blindly cast a net for everything in every application. Optimize security testing based on your threat model. You only need to test defenses for the threats you actually face. You don’t need to test for SQL injections if your app doesn’t use a database. To determine what you do need to test for, you need a threat model, which drives what defenses you have in place and need to test. You should ensure you have the proper defenses, but testing anything else wastes time.
Test with the best. You don’t build a house with only a screwdriver, and you don’t drive screws with a saw. You need to use the proper tests for vulnerabilities and make sure the tests are fast, complete, accurate, easy, inexpensive and don’t irritate your staff. Rather than using every tool on every vulnerability, organizations should select security testing approaches that deliver the optimal balance of those elements, and you need to put those tests in the places where they’re most efficient. Seek out tools that provide strong evidence of coverage and accuracy for each class of vulnerabilities. Weaker tools are unlikely to strengthen overall results and introduce opportunity costs for your teams.
Notify left. Stamp out PDF reports and avoid gigantic vulnerability databases, which generate backlogs that will slow down your AppSec program. Security dashboards are helpful for managers, but developers shouldn’t have to log in and check a separate system. Vulnerabilities that go into defective databases will likely never be fixed.
Test smart, but notify left for faster remediation and learning
Get security feedback to the folks who need it through the tools they’re already using. Even when you test right — e.g., when you look for vulnerabilities later in the life cycle — you can notify left. Today, code goes into production many times a day, so when you shift left, you’re just shifting earlier in the day, sometimes just by a few minutes.
Optimize for learning. The most cost-effective AppSec programs don’t just find and fix vulnerabilities faster. They prevent those vulnerabilities from existing in the first place.
This doesn’t just mean training, because training is often generic. It lacks specificity. Learning needs to be interpreted. I’ve spent over a decade training developers in secure coding; they learn more by working on their own code than in a generic training session.
The best learning happens when developers get fast, clear and accurate feedback on their code in the context of their day-to-day work: the exact environment, framework, language, coding techniques and particular idioms that developers are using. This just-in-time security training helps developers learn from their mistakes and not repeat them.
This kind of training can dramatically reduce the vulnerability escape rate, which is the rate at which projects introduce new vulnerabilities. An efficient AppSec program can reduce that rate below one per month.
Test smart, but notify left
Shift left is a great idea, but it shouldn’t be followed blindly. Shifting too far left wastes time and risks missing security vulnerabilities that can’t be tested effectively until later in the SDLC. Instead, shift smart, and choose the right security test for each rule and apply it at the points in the SDLC where they’re most cost-effective.
The shift-smart strategy is the way to dramatically reduce the cost of application / API security and improve security at the same time.
Jeff Williams’ bio:
Jeff brings more than 20 years of security leadership experience as Co-Founder and Chief Technology Officer of Contrast. Previously, Jeff was Co-Founder and Chief Executive Officer of Aspect Security, a successful and innovative application security consulting company acquired by Ernst & Young. Jeff is also a founder and major contributor to OWASP, where he served as Global Chairman for eight years and created the OWASP Top 10, OWASP Enterprise Security API, OWASP Application Security Verification Standard, XSS Prevention Cheat Sheet, and many other widely adopted free and open projects. Jeff has a BA from the University of Virginia, an MA from George Mason, and a JD from Georgetown.
