National Security Innovation: Creating new capabilities for the future

This publication is licensed under the terms of the Creative Commons Attribution License 4.0 which permits unrestricted use, provided the original authors and source are credited.

Introduction

Why innovation is vital for national security – and also really hard in practice

It is part of the responsibility of the State to protect its citizens both at home and abroad, and so national security remains - and will always be - a key concern of Government. But how Governments go about discharging this responsibility has varied over time, as social norms, threats, and technologies have changed. As a liberal democracy, the UK aims to strike a socially acceptable balance between protecting civil liberties, individual rights to privacy, and free speech, and maintaining effective national security practices. As the internet, widespread encryption, and social media have transformed how people live and work, so national security practices have needed to adapt within a clearly defined legal and regulatory framework.

Innovation (by which we mean ‘ideas, successfully applied’) is necessary to counter the threats posed by our adversaries’ use of innovative technologies, and to keep ahead of the game by developing new intelligence and security capabilities as old ones become compromised or outdated. Standing still is not an option. Innovation is therefore not optional: it lies at the cutting edge of what national security organisations should and should not do on our behalf, what is deemed acceptable to a majority, and how we as a country wish to respond to future threats in a ever-changing geopolitical context. But, by its very nature, national security must remain secret (so our adversaries don’t know what we are doing) and operate covertly - making open and informed debate challenging.

Herein lies the paradox at the heart of national security innovation: maintaining secrecy in a world which thrives on open collaborative innovation, and striking the right balance between in-house technical knowledge and development, and relying on skills and knowledge from outside. Staying ahead of the game needs sustained, multi-year, substantial investment and clear goals in order to align people, funding and efforts – with at least as much (or more) attention paid to pull-through and implementation as to finding innovative ideas. Often the biggest impact can be a simple idea, applied well.

This paper discusses the challenges of innovating in national security, considering the ecosystem, funding mechanisms, and skills needed, and suggests a way forward for national security agencies, industry, and academia to work better together towards keeping people safe.

Challenges to effective innovation

Being the same – but different 

The job of the intelligence agencies is to think of things that nobody else has, and they have often had to innovate by necessity. The UK intelligence community (UKIC) has a long history of inventing gadgets and devices. Some of the most revolutionary technological advances in history can be traced back to the intelligence community, such as the invention of the Turing-Welchman Bombe machine and Colossus computer at Bletchley Park during World War 2, beginning the modern computing era.  So innovating is very much in the DNA of UKIC.

The UKIC comprises several organisations, each with their own historical remit, culture and ways of working. They face many of the same challenges and barriers as many other large public sector organisations.  They struggle, like most, to recruit, manage, develop and retain the right talent, put in place the right corporate systems, hardware, software, and processes to manage estates, supply chains and equipment, and to operate cost-effectively at scale across organisational boundaries. On top of these common features, they are unique in needing to protect the identity of their staff, operate covertly and within the law, collect and analyse intelligence and gather evidence to bring people to justice, and to respond at pace to critical incidents, such as planned or actual terrorist attacks.

Many suppliers assume that solutions which have been applied to other organisations will work in the same way for national security agencies, and so they recommend using commercial off-the-shelf (COTS) technologies and business change practices. While the theory may stand up, in practice this is always more difficult than one might expect – especially in national security, thanks to the four horsemen of secrecy, complexity, risk aversion, and groupthink.

Many private and public sector organisations have successfully embraced digital and cloud technologies, for example, in order to become more agile and effective and break away from legacy IT systems. Cloud technologies aim to provide more access and joining up of datasets, shared services and compute power in a way that is resilient, cost effective and time efficient. But classified data requires high standards of encryption and cybersecurity, given the likely interest in accessing the data from hostile states.

The challenge here is identifying the right uses for these modern technologies – for example to collect and analyse open source data – whilst protecting the value-adding, sensitive analysis and interpretation used to make operational decisions. A technology company wishing to work with the national security community needs an excellent understanding of the organisational context to give the right advice and lead to good outcomes. The costs of mistakes are high, and irretrievable.

A complex landscape for innovation

Many talented people work in S&T innovation across UKIC. The Government Communications Headquarters (GCHQ), National Cyber Security Centre (NCSC) and Her Majesty's Government Communications Centre (HMGCC) are deep science and technology (S&T) organisations employing engineers, scientists, cryptographers, cyber and technical experts, and the Defence Science and Technology Laboratory (Dstl) provides research, expertise, S&T and specialist services for both defence and security. The Centre for the Protection of National Infrastructure (CPNI) is an S&T-based organisation which protects the UK’s critical assets and functions as the National Technical Authority for Protective Security. They are supplemented by academic expertise from many universities, and by private sector suppliers.

As UKIC has recognised, it cannot stay ahead of the game by relying on in-house expertise. Since 2017 there has been a focus on innovation with the creation of the Defence and Security Accelerator (DASA), Joint Security and Resilience Centre (JSaRC), Accelerated Capability Environment (ACE), National Security Strategic Investment Fund (NSSIF) and National Security Technology and Innovation Exchange (NSTIx). There is no shortage of innovators and innovation-enablers seeking to join up the ecosystem. But the complexity of the landscape makes it hard for national security agencies and the wider community to keep track of what everyone is doing, leading to duplication, gaps and inefficiency.

Funding for national security innovation has been historically small compared with much larger budgets for defence innovation, despite recent increases as a result of the 2021 Integrated Review’s emphasis on sustaining ‘strategic advantage through science and technology’. The disaggregated nature of national security innovation means that budgets are patchy and fragmented – there is a limit to how much innovation can be achieved by £50k or £100k projects and it is not going to be game-changing – Turing would not have invented the computer on piecemeal grants paid in arrears. Often, projects suffer from public sector spending demands of capital departmental expenditure limits (CDEL) for investment versus resource departmental expenditure limits (RDEL), annualised funding cycles, and payment after delivery.  RDEL is needed to recruit new staff, while annual funding  cycles mean that project starts are highly uneven (with a rush towards the end of the financial year), and must be completed in-year. Commercial processes can be too slow and risk-averse for the pace of innovating, and the needs of commercial businesses for future planning and consistency.

Staying secret in a world of open innovation

Open innovation

Open innovation is a mindset and practice which welcomes external contributors to innovation, looking outside for ideas with the emphasis on collaboration and cross-fertilisation. The rationale in favour is:

1. The cutting edge of modern innovation is in the private sector, with start-ups that have the agility and drive to move fast, so Government needs to use them to get ahead of adversaries.
2. Lots of brilliant people work for industry and in universities, so UKIC should tap into their expertise, and help them to maintain critical skills.
3. Government agencies can’t fund all the technology development they need, so must rely on the market and draw in what they need when they need it. 

But there are three other factors to bear in mind:

1. National security agencies need to own any important IP to protect it from adversaries and use it freely.
2. An idea might start in the open innovation space, but if it will become a cutting-edge national security capability, it will need to be classified.
3. UKIC needs to adapt to the pace of open innovation, and build sustainable external relationships.

Managing large numbers of external innovators relies on relationships, networking, and communication – difficult for organisations which are by their nature secretive. UKIC needs to get comfortable with engaging more widely, and doing unclassified S&T not only for research but also for creating user-ready tools and services. It needs to pull things through into classified spaces at the right time, and manage information about what is and is not public. That needs clarity, processes, consistency and governance.

Buying in equipment and expertise

Given the scale of the challenge, it makes perfect sense for national security agencies to tap into the best commercial industry expertise and capabilities which would be inefficient and costly to replicate from scratch with public funding. Such relationships may help to ensure cooperation and access, given the amount of data now held by private companies. But national security agencies also need to maintain enough in-house expertise and knowledge, even if only to be intelligent customers and to ask the right questions. National security personnel need to manage the projects, ensure value for money to taxpayers, and ensure that the outcomes are fit for purpose for frontline operations. Excessive reliance on a small number of suppliers carries its own risks of deskilling their own people while creating monopolies too important to ever be allowed to fail.

For external suppliers, working with national security agencies is not something to be taken on lightly. It can require at least some of the team to go through some level of vetting under the sponsorship of the agency, which can be a slow and intrusive process. Commercial arrangements may be complicated and the opportunity may not come out to public tender but through a restricted framework. Contract terms and conditions can be onerous and complicated, and it may take a long time to get on contract. And relations ‘on the ground’ can vary, depending on how much the supplier is trusted – they may be seen as having profit-driven motives rather than the purity of the mission. All of which can make it particularly hard for small businesses, who haven’t got the time to wait for contracts or the scale to have enough security-cleared people.

National security agencies are, to their great credit, aware of and seeking to solve these barriers with various initiatives. The NSSIF, run under the umbrella of the British Business Bank, is a great example of bringing start-ups to work in national security, while the DASA invites good ideas through challenges and an Open Call. But many cultural challenges remain.

Making difficult choices

Just because a technology exists, or could exist, which may aid national security, does not mean that pursuing it is the right thing to do. International restrictions exist, rightfully, in chemical and nuclear weapons development – but other areas such as AI, autonomy and IoT are more of a Wild West. We need a clear, socially responsible and cohesive view, based on the core principles of proportionality, necessity and due authority over national security technology development both in-house and by suppliers, with independent and expert oversight – not least so practitioners have a single source of policy advice. 

The skills challenge

There is, as might be expected, a well-trodden path for people from national security to move out to work in industry, which has many benefits for knowledge-sharing as well as building trust, but isn’t for everybody. It’s much less common for people from industry to join national security agencies, especially mid or later career, due to pay differentials and possibly poor promotion prospects. The risk of losing good people to industry in return for better pay is one of the tensions in public/private sector partnership working. Smoothing career paths in and out of national security and industry or academia is one of the ways that  effective partnership working will help.

Next steps for national security innovation

There are no easy answers. We have identified some of the key barriers to overcome:

• Encouraging a traditionally and necessarily secret and risk-averse set of organisations to adopt ‘open innovation’ approaches.
• Navigating a complex, difficult-to-access stakeholder landscape which currently has no end-to-end model for innovating.
• Fixing unhelpful financial, commercial and vetting rules which lead to stop-start projects, making it too hard for some potentially useful suppliers to work in national security.

National security will always need to remain secret – but then many companies want to keep their innovations secret to stay ahead of commercial competitors, so secrecy it not necessarily a barrier to innovation per se. What is necessary is to be intentional, manage flows of information consistently and with purpose in deciding what is and is not secret, and make clear what people can and can’t say. More can be done to resolve some of the challenges around complexity, risk aversion and groupthink, however, which overinflate secrecy and make it hard for others to work with national security teams.

Building the ecosystem

National security organisations must diversify their networks beyond traditional suppliers and engage with the wider innovation ecosystem, across Government, and in industry and academia – noting that each has different incentives and approaches, and can be used according to their strengths. UKIC must reach beyond the same few ‘usual suspects’ who are easy and comfortable to work with (often ex-national security personnel). 

It is a skill to be collaborative, be open to others’ ideas and help the community as a whole to succeed without trying to run the whole show. They need to articulate externally, if not specific applications, the sense of purpose underlying the development of new capabilities: that is how to connect with bright people who care about solving interesting problems, and develop them into useful allies – making use of networks such as RISC and Academic RISC.

A prescription for impact

1. Build trusted, long-term partnerships with a few key partners and empower them to innovate.
2. Communicate priorities with clarity and tell stories about innovation successes.
3. Create a consistent end-to-end innovation framework approach and insist that everyone uses it.

Creating strategic long-term partnerships founded on mutual trust will go a long way towards de-risking innovation, and embracing entrepreneurialism. A partnership agreement with a consortium of universities and businesses, with umbrella funding for innovation over years, not weeks, would provide a solid foundation for companies and universities to co-invest, and help generate and maintain the wider skills base on which future innovation will depend. These partnerships can provide the overarching structure for broadening the supply chain to one-person and small start-ups, helping them to participate without taking on all the risk, and allowing everyone to benefit from a wider pool of security-cleared people with specialist skills and creating opportunities for personal development through secondments and knowledge exchange activities. Over time, this will change the culture towards being more entrepreneurial and confident taking appropriate risk.

Having a clear set of priorities and problem-sets and being able to talk about them in different security settings helps more people to get involved, supporting diversity and inclusion. That means having dedicated resources for external engagement to build the networks and relationships, scout for ideas, speak at events, and champion innovation both externally and internally. Plugging in to the networks which already exist is preferable to creating new ones, but it needs joining up behind the scenes through a shared IT platform to avoid crossed wires and create a shared organisational memory.

Creating a strategic, prioritised end-to-end approach to innovation is key to impact (by which we mean, make choices and make them happen – taking a balanced view across immediate, medium and longer-term challenges). Without a way to identify, develop, exploit and evaluate good ideas, innovating is a waste of time, money and effort. So it is imperative for public spending that investment in finding good ideas is matched or even exceeded by investing in scaling and pull-through – using the right commercial models, such as the Innovation Partnerships introduced in the Public Procurement Regulations 2015 –  to make it work, at the pace needed. Other enablers – such as managing clearances – need to be adapted to the urgent needs of innovating, with more risks taken on engaging without security clearances at least early on in the innovation process.

In conclusion, innovations – either by adversaries or simply people trying to do things better – may one day pose an existential threat to national security, leaving them behind when it comes to spotting and mitigating threats. Staying ahead of the game needs sustained, co-ordinated, and substantial investment of people, funding and effort.

The views expressed in this article are those of the authors, and do not necessarily represent the views of The Alan Turing Institute or any other organisation.