• Share
  • Share

Stop punishing your developers

Enterprises often create hurdles for developers in the name of security. When is this worthwhile—and when does it slow down and frustrate teams unnecessarily?

Alexander Golden | May 15, 2017

Go to any medium-to-large company and you’ll likely see the same story: a complex web of networks, VPNs, environments, access credentials, and domains. Each represents an additional hurdle for developers hired to create software for the company. Access must be granted, credentials issued, environments made accessible, and firewall ports opened before teams have the resources they need to succeed. Weeks may go by before a single line of code can be written.

Does it have to be this way? Let’s look at a common enterprise scenario to understand why this situation occurs—and whether or not it’s truly warranted.

The arguments for an internally managed and secured development pipeline

IT and risk managers everywhere will be quick to point out that development environment complexity is sometimes in place for important reasons, not least of which is software security. After all, we’re not just talking about meaningless 1s and 0s. The software being created may be intellectual property that makes or breaks the company. For software-intensive companies, a source code leakage to competitors or hackers could destroy the company’s competitive advantage and ultimately doom it to failure. Similarly, inadvertent or malicious injection of code could expose the company to catastrophic liability or a loss of consumer confidence.

Having an internally managed and secured development pipeline, proponents argue, is the only way to counter these risks. They insist on two basic tenets. First, all parts of the development pipeline must be secured on company-managed environments. Second, software should be developed on company machines that connect securely to a monitored internal network. In theory, this two-pronged approach helps the company minimize the risk of valuable software being lost, stolen, or manipulated to others’ ends.

A case study in developer frustration

The problem, of course, is that the benefits of “secure” software development come with a price. The following fictitious case study demonstrates the ways in which a strict development pipeline can end up punishing developers and the company at large.

Acme Online is looking to create a cutting-edge redesign of its online shopping experience. The company has hired a crack team of well-paid contract developers for this ambitious project. Team members are scouring the backlog and preparing to start development when roadblocks start popping up fast and furious:

  • Acme corporate security mandates that only company-secured computers can access the network, so the developers’ shiny, latest-generation MacBooks are shelved in favor of company-issued laptops. As contractors, developers get the leftover previous-generation laptops that are large and slow, limiting their productivity.

  • Part of the team is overseas, so some of the Acme-issued laptops will be shipped. Half of the team loses a sprint when the laptops are unexpectedly caught up in customs.

  • The VPN is old and slow, and the connection sometimes drops during times of peak volume. Overseas developers, who rely exclusively on the VPN, are at its mercy.

  • The developers are not given administrative access on their computers, so software tools that the team is accustomed to using must go through a lengthy security review and approval process. Some tools (e.g., Slack) are ultimately rejected entirely because they are hosted outside the Acme company firewall, and the team is forced to find alternatives.

  • Setting up the team’s development environment involves submitting a ticket to a centralized queue. The service level agreement specifies 3–5 days before the request is addressed, and the team will spend several more days in back-and-forth discussions with the systems engineer who didn’t quite understand the original request.

  • Integrations with third-party tools (e.g., remote testing services) require opening ports in the firewall, another submit-ticket-and-wait proposition.

  • The internally managed Acme pipeline doesn’t yet support the latest version of Node.js, which is needed to support some cutting-edge libraries that will allow for a better customer experience. Instead, the user experience is watered down to accommodate the limitation.

Just weeks into the project, the scope has already been cut, the developers are struggling without the tools they need to succeed, the development environment still isn’t fully configured, and the projected cost and duration of the project are exceeding original estimates. Unsurprisingly, the morale of the development team is low.

Was it worth it?

On the plus side, none of the (somewhat crippled) source code created by the developers could leak into the wrong hands—until it’s deployed to production, that is. The front-end source code created by this team consists of HTML and JavaScript libraries that are minimally compiled and can be easily downloaded, read, and appropriated by Acme’s competitors.

Meanwhile, like all good developers, the team sought to speed development and lower costs by leveraging several free, third party–developed user interface components and software development kits. While the use of these libraries helped keep the project on track, it exposed Acme to the potential for malicious code injection, a risk that the company’s “secure” pipeline could not effectively address on its own.

In the end, despite the burdensome process foisted upon the development team, Acme’s security measures failed to provide the intended security benefits.

Protect what matters

The lesson is clear: Security matters, but only when you secure the “right” things. In the case of Acme Online, real competitive value comes not from the front-end user interface code, but from the back-end services code that codifies Acme’s business processes for pricing, inventory, product recommendations, and sales.

In this case, the contractors don’t actually need access to the proprietary back-end source code to achieve their objectives. What they do need, however, is secure, well-documented, and highly accessible development web services. This requirement could have been satisfied with far less onerous security measures, including:

  • A publicly facing set of web services that is accessible outside the company firewall, enabling development from anywhere

  • SSL certificate authentication, ensuring that those services are accessible to only company-approved developers

  • IP whitelisting, to minimize the chances of a denial-of-service attack from unauthorized computers

  • API rate limiting, to ensure that no one developer (or malicious actor that has obtained a developer’s credentials) can attempt to reverse engineer back-end algorithms by issuing thousands of API requests

Finally, a clear company policy on the use of third-party components, rigorous validation of the completed code, and appropriate safeguards for Acme’s IT and API infrastructure could be effective in combating the risks of malicious front-end code injection without impeding the progress of the development team.

With these security measures in place, the developers could have set up their own development pipeline, used the tools they’re comfortable using, and developed software anywhere and anytime they had an internet connection.

This approach would also have significant security benefits by implementing the principle of least privilege: Developers would have access to only those resources necessary to complete their work, with no broader access to internal company systems or networks. The use of external-facing APIs in this manner replaces the outmoded and ultimately ineffective concept of a “company firewall” that does little more than divide network traffic into internal (‘good’) and external (‘bad’) sources.

When red tape is a good thing

The example above is no doubt a simplification of software development. Not all projects produce relatively non-proprietary front-end web code. So, when does a highly secured development pipeline, and the regulated and lengthy processes that come with it, become a necessary evil? Situations that may meet these criteria include those where the code to be developed:

  • Contains highly proprietary algorithms.  Code that codifies an insurer’s rating algorithm or an investment bank’s investing strategy, for example, may call for a tightly controlled development pipeline. In these cases, the algorithm itself is a major source of the company’s business value.

  • Must be strictly traceable. HIPAA-compliant software, for example, must undergo rigorous validation that maps directly to system specifications. A single, highly secure, fully traceable development pipeline may be necessary when the development process must comply with strict laws and regulations.

  • Exposes the company to significant risk. Certain types of software—such as those responsible for managing financial transactions—can have real-world impacts that expose a company to significant liability risk. Managing and effectively delegating accountability for this risk is especially necessary when the risk is high, contractors are involved, and the software to be developed lacks clear boundaries of ownership. A highly regulated development pipeline can help a company effectively assess, manage, and mitigate this risk.

Development ops is not one-size-fits-all

The lesson to be learned is that no single development pipeline is right in every situation. For mission-critical software that requires the highest levels of security, processes, and rules, some inconvenience to developers may be absolutely warranted. For other software, a more hands-off approach gives the team greater flexibility.

Only by truly understanding a company’s competitive advantage and risk profile—and being willing to tailor development processes to the software being created—can an organization effectively balance security and flexibility.

Alexander Golden originally published this post on Medium.

Alexander Golden

Alexander Golden is a principal consultant in Slalom’s Boston office. He is a product owner, software systems architect, and project management professional specializing in web-based and mobile systems strategy, design, development, and deployment/integration. Follow him on LinkedIn.