Understanding the risks and benefits of open-source software

Open-source software is the backbone of the internet. The Linux and BSD operating systems are embedded in millions of unique devices, and curl is the most shipped piece of software ever, with an estimated 10 billion installs across computers, phones, consoles, printers, and cars.

Whilst having the source openly available makes it easier for threat actors to find vulnerabilities, it equally provides an opportunity for outside developers to spot and fix them.

A community development model has proven itself to be stable, to ship features, and to provide a strong basis to build upon. Yet in the context of technology provision for goverment, there are legitimate concerns about the security risks posed by software with a non-commercial background.

A common view is that ‘open-source’ means ‘less secure’ because anyone can modify it, and anyone can read and find the vulnerabilities.

This argument isn’t entirely right or wrong. Indeed, whilst having the source openly available makes it easier for threat actors to find vulnerabilities, it equally provides an opportunity for outside developers to spot and fix them.

If we consider the converse, that closed-source software must be more secure, then evidence suggests that this isn’t necessarily the case. The 

 project has been tracking vulnerabilities across the software ecosystem since 1999 and contains over 250,000 entries from both closed and open-source components, most of them open. Although a useful source, it is not entirely representative of the closed-source landscape; whilst open projects must notify MITRE to get a CVE assigned for a bug, private companies are not required to. As such, issues discovered internally are not necessarily publicised and fixes can be pushed out under the heading of "miscellaneous bug fixes and improvements". 

We know these issues exist because when they do become public, they become big news. In 2021, the U.S. Cybersecurity and Infrastructure Security Agency (CISA) 

 tracing the 30 most widely exploited vulnerabilities. Only two of the issues were from open-source projects.

This isn't a matter of reach - the "Log4shell" vulnerability from the Apache log4j project was the single most exploited bug precisely because of its almost ubiquitous usage in both open-source and closed-source Java software.

There is a downside to choosing an open project to build from – you’re not paying anything, so you’re not paying for support and risk management. For small businesses and hobby projects, posting for help on a forum might be sufficient, but this is a significant and understandable risk for government customers: when you’re trying to build something long-lasting and stable, the support lifecycle is critical.

This is where system integrators play a critical role. Underpinned by specialist expertise in software, integrators should take on that risk and provide the requisite support and training to government customers and end users through life.

Importantly, open-source also helps to mitigate vendor lock-in. Even if customers wish to keep using the same database engine under the hood, other integrators or consultants should be able to step in and work with it as the same technology is accessible to everyone.

So, whilst there are legitimate concerns regarding the security and stability of open-source software, there are considerable benefits. With the community development model providing a strong ecosystem of expertise to build upon, open-source software is a critical element of modern, dynamic technology provision.

Synthetic_training_software_Modelling_detection

Synthetic_training_software_Modelling_detection_ROWDEN23_1842

A view on open-source software security

Machine Learning (ML) algorithms typically require large amounts of data to train for object detection. For example, if we wanted to train a computer vision model to detect cars, we’d need to ensure the training data contains images of a large range of different models of car, in different lighting conditions, landscapes, and weather conditions.

In the case of Supervised Learning, the data must be annotated with the correct ground truth labels to provide the necessary feedback during training for the algorithm to improve. The annotation process is incredibly time-consuming and requires extensive resourcing to complete.

At Rowden, we have used 3D modelling engines to generate synthetic training datasets for drone detection in edge environments, allowing us to easily generate the variation in data required to train a robust model, and enabling us to annotate images automatically. Once the model is optimised to the GPU’s architecture, we can demonstrate the utility of synthetic data for deployed MLOps.

Increasingly, Machine Learning engineers are using 3D modelling engines to generate synthetic training datasets. For example, the open-source procedural pipeline 

 harnesses Blender’s modelling and rendering engine to rapidly generate training data annotated with the correct ground truth labels. It does this by using a Ray Tracing algorithm to identify which pixels represent the target object to calculate a corresponding image mask or bounding box.

Generic_drone_bounding_box

To create a full training set, you need to repeatedly vary the position of the drone in a range of different environments and lighting conditions. This can be done using High Dynamic Range Images (HDRIs); HDRIs can be thought of as a 360 degree panoramic picture that covers all possible angles from a single position, whilst also capturing information about the lighting of the scene. Conveniently, there are a large range of HDRIs available for free online. Here is an example from polyhaven.com:

HRDI_image_Spaichingen_Hill_Germany_Polyhaven

In Blender, we can use a HDRI to act as the backdrop and lighting for our scene. The process of generating training data is now relatively simple: randomly select a HDRI background and then randomly choose a camera position and angle to render the model of the drone from:

Blenderproc provides a Python library that allows us to easily script this process, whilst also automating the process of annotating the rendered image. For this example, we generated 5k training images using 10 different backgrounds in approximately 1 hour on a NVIDIA RTX 3090 GPU. Here’s a sample of the generated data:

HDRI_rendered_annotated

Synthetic_training_data_annotated

Now that we have an annotated dataset, we can train a model against it.

By far the most common approach when training a model is to perform what is called “transfer learning”, a process that takes a pre-trained existing model and fine tunes it using a different dataset.

For this example, the Rowden ML team took an object detection model architecture called YOLOv5 that had been pre-trained against the 

 to identify and locate 80 different classes of everyday objects such as vehicles and animals. During transfer learning, we freeze the majority of the model and only allow the model to adjust the weights of the final layers of the network in order to tune the model to our drone dataset.

The idea here is that although we are applying the model to a different dataset, the majority of what the layers have learnt to identify will still be relevant (e.g. edge detection, identifying primitive shapes, etc.).

We were able to fine tune the pre-trained YOLOv5 model with good accuracy in 10 minutes using a NVIDIA 3090 RTX GPU. This model could be further improved with more time and data. Here’s a sample of its predictions made against the sample of data shown above:

Trained_model_synthetic_testdata

ML algorithms, and Neural Networks in particular, require large amounts of computational power to run, and it’s only in recent years that edge devices such as phones have been able to run the models locally, rather than transferring the data back to a centralised server.

NVIDIA’s Jetson series of hardware specifically targets ML edge use cases, providing a small form factor standalone device with an NVIDIA GPU. For this example, our team used the smallest of the Jetson devices, the Jetson Nano, but would typically target higher performing platforms such as the NVIDIA AGX for production environments.

NVIDIA provide a SDK called DeepStream that optimises the use of the on-board GPU for both video decoding and model inference, allowing a Jetson device to annotate multiple video streams simultaneously by feeding them through a Deep Neural Network in real-time. With everything happening locally on the device, a model can be deployed in internet-denied environments or scenarios that require fast responses to the model’s predictions.

To make the most of the onboard GPU’s processing power, we must first compile our trained model into a binary format that has been optimised to the GPU’s particular architecture. For this example, we exported the model to the .engine file format and configured the DeepStream pipeline to use it.

Here is how the model performed on a real video feed after being trained on the synthetic dataset:

Whilst there are a few false positives, this is an impressive performance for a model that has only been trained on synthetic data of a generic drone, and could easily be improved by generating a larger range of different drone images (and examples with no drones present).

With the current pace of development in Virtual and Augmented Reality, synthetic datasets and virtual environments will play an increasingly prominent role in Artificial Intelligence and Machine Learning.

Importantly, these approaches will help to accelerate the operationalisation of ML for users at the edge.

Drone_footage_Synthetic_training_Modelling_detection

Drone_footage_Synthetic_training_Modelling_detection_ROWDEN23_1872

Object detection with synthetic training data

The UK Government envisions its future defence-industrial base as a partnership between ‘smarter’ government customers and an ‘ecosystem of scientists, researchers, inventors, and innovators.’[1]

Traditionally, the evolution of military capability has been determined by the incumbent’s hardware design choices, yet future defence systems are increasingly dictated by software considerations.

To alleviate the challenges associated with the delivery of complex digital procurement programmes, the role of the System Integrator must evolve to bridge asymmetries and empower the Smarter Customer to succeed.

The Army are working on what we need and how we can retain the right skills in a marketplace that’s highly competitive and evolving quickly. We’ve experimented with apprenticeships in the DevSecOps world, but ultimately many leave us for better paid roles. The result is we are currently dominated by 'As A Service' contracts, but is that the right model?

Technological innovation is intrinsically tied to the efforts of a vast array of scientific research and engineering activities distributed across myriad commercial entities. These ‘technology systems’[3] are highly inter-dependent, complex, and extremely challenging for government to govern and integrate into existing ecosystems.

How is new technical expertise being generated and shared, by whom, and where can it be applied to defence problem sets? These questions are grow in importance as the delivery of future digital capability becomes increasingly dependent on the effective orchestration of these complex commercial technology systems.

The industrial base is responding, but dependency on more specialist software-centric expertise, rather than capital-intensive industry or generalist consultancy, is growing. The key challenge for government customers, then, becomes one of ‘finding the right expert’ to deliver material value to frontline operators at pace.[4]

Things get sold because they’re well marketed, not because they’re technically superior or meet some technical thresholds.

The underlying logic behind the government’s ‘smarter customer’ aspiration is that when the knowledge of government teams meets or exceeds that of the supply chain, they are in a better position to limit historically identified poor behaviours in digital procurement programmes, such as vendor lock-in and high costs of exit or change.

The role of a systems integrator (SI) is the key to ensuring customers can succeed when there are high degrees of uncertainty and risk. A SI should not slow down the delivery of value. Rather, they should alleviate some of the challenges government customers will face in delivering complex technology projects.

01. A rapidly changing technological landscape, creating performance and obsolescence risk.​

System integrators must create the conditions for component change whenever it is deemed necessary and ensure management of obsolescence through long term technology planning, not bespoke contracts for unsupported components. ​

02. Requirements that are either too explicit or too loose.​

To drive a better performing and more cost-effective solution, integrators must take a tailored approach to procurement of components on the merit of the individual requirement.

03. Historical instances of ‘vendor lock in’ leading to a high cost of exit and contributing to an unaffordable Equipment Programme.​

Commercial openness should be prioritised to the same extent as system openness. The industrial base’s incentive to succeed should be partly based on the possibility of replacement.

04. Poor contractor behaviours leading to a lack of trust and collaboration.

Collaboration and good behaviours must be driven by the integrator with technical expertise and conviction. SI's should support with building non-traditional supplier-customer relationships to build robust supply chains. Equally, government must be bold and work with SI’s who accept risk when incentivised to deliver against difficult challenges, not contractors who mitigate through ‘risk sharing’.

Historically, issues arise where the government owns its code but is blocked or delayed from accessing it by a contracted DevOps function. Government ownership is a necessity to drive better value and operational capability; SI’s should ensure the customer owns its source code with immediate and easy access to its hosting environments.


What we’ve got to do is actually pick some winners.

An understanding of recurrent and new informational asymmetries in the defence-industrial base can critically inform the development of innovation management, knowledge creation, and communication strategies within the enterprise. These strategies will be critical to realising defence initiatives like Multi-Domain Integration in the UK or Integrated Deterrence in the U.S, which are dependent on cross-sector collaboration across multiple geographic and military boundaries.

Integrators have a responsibility to bridge knowledge asymmetries, drive good behaviours, and champion ‘smarter customers’. Equally, the government must back the best athletes who are delivering and let them get on with creating value. Those who can’t, should be penalised.

[1] See Global Britain in a Competitive Age: the Integrated Review of Security, Defence, Development and Foreign Policy. (2021) https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/975077/Global_Britain_in_a_Competitive_Age-_the_Integrated_Review_of_Security__Defence__Development_and_Foreign_Policy.pdf

[2] Brigadier Crossfield, Stefan. (2021) https://defencedigital.blog.gov.uk/2021/07/30/three-of-many-challenges-for-becoming-a-digitally-transformed-british-army/

[3] Carlsson and Stanciwicz. (1991). On the Nature, Function and Composition of Technological Systems. Journal of Evolutionary Economics.

[4] See HM Government. (2021). ‘The Consultancy Playbook’, p. 16. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/987126/The_Consultancy_Playbook.pdf

[5] See interviews conducted by Bowne, Andrew S. 2022. National Contract Management Association. https://www.ncmahq.org/Shared_Content/CM-Magazine/CM-Magazine-July-2022/Innovations.aspx

[6] Gen. CQ Brown, speaking at the 2022 Reagan National Defense Forum. See: https://www.defensenews.com/industry/2022/12/04/air-force-chief-military-must-pick-some-winners-among-startups/

Whiteboard_working_Rowden_office_ROWDEN_1111

Overcoming the barriers to delivering software-centric capability

To successfully navigate the RF spectrum, frontline operators need to monitor it across different dimensions; frequency, time and location. The obvious approach to collecting more information is to provide more sensors to capture the RF picture in real-time across an environment. ​This approach will require sensors capable of spanning the growing frequency and geographical ranges that RF spectrum usage is proliferating, such as super high frequencies, that provide short range communications.​

The combination of the three differing dimensions creates an abundance of information that needs to be processed and fused. Currently, this is either post-processed or orchestrated by a central node, both of which require a coherent database that can be used to store, label, and access the data needed for analysis, SIGINT, and decision-making.​

The option to collect and store locally benefits collect where information lag is not important, but it increases the reliance on data storage, management, and processing on the sensor. When you consider that a one minute collect of a single Wi-Fi channel is 31GB, post-processing leaves analysts with a significant amount of data to sift through.​

Alternatively, some sensors collect and pass the data back to a central node for processing and analysis. This approach can reduce the need for processing and storage at the edge, but requires a resilient and large backhaul network. This is hard to maintain in congested environments (although agile radios are one avenue for solving this).​

In both approaches, users are reliant on being suitably trained to know when to collect and record. They might then be responsible for analysing and collating the data, or passing the information onto signal analysts to understand what useful information is contained within the spectrum, such as detecting, identifying, or intercepting communications, or monitoring changes in spectrum density.

The ability to do this in real time requires considerable skill, knowledge and processing power at the sensor nodes. This is where Machine Learning can help.


As modern sensors are being loaded with more processing power, there are opportunities for ML models and AI applications to be pushed to the edge​ to enable near-real time operations in the RF domain.

Pre-processing at the edge reduces the need to store and pass back large amounts of data. This saves precious storage and data bandwidth, allowing nodes to connect and share useful information in manageable quantities and removing the need for a central processing node.​

ML can be used to learn, detect and identify specific signals of interest, as well as a mechanism to only collect signals from specific devices.​ It can extract signals of interest from a larger collect, post processing, which can dramatically reduce analysis time in searching for specific signals or identifying patterns of life​.

By using ML to model normal RF behaviour in a local area, it can be used to detect deceptive targets, such as those hiding in the noise floor, or devices that spoof the transmissions, protocol and identities of others.


At Rowden, a significant amount of our ML work is focused on understanding and addressing legacy challenges in the RF and SIGINT space.

To accelerate joint research in this area, we identified early on the necessity to train our models by developing a methodology for collecting signals at mass, enabled by a RF-controlled environment in our on-site semi-anechoic chamber.​

Our ability to access and control elements of the set-up allow us to conduct rapid collection, testing, training, and modification of our pipeline, applying continuous development and new techniques to our methodology.

RF_Engineer_testing_semianechoic_chamber_sensor_processing_ML_

RF_Engineer_testing_semianechoic_chamber_sensor_processing_ML_ROWDEN23_1759

One of the areas we focus on in RF ML is in the development and deployment of Specification Emitter Identification (SEI) of wireless RF devices, such as push to talk radios like PMRs and DMRs.​

SEI is the challenge of pairing a received signal to a particular piece of hardware based purely on information captured at the RF physical layer. In ML terms, this is framed as a classification problem where a model has to pick between 1 of 10 possible classes for each packet received.

To make its decision, the model is given a single data packet in the form of the raw IQ values as received by SDR. ​The ML model can either use these values directly or extract higher-level features such as the statistical properties or an FFT of the signal.​

We have experimented with a range of different models, including classical methods such as KNNs and SVMs through to Deep Learning approaches such as CNNs and LSTMs, with a focus on deploying these models to run in real-time on edge-based hardware.

Another unique application of ML is in providing additional physical layer security by fingerprinting the unique signature of trusted devices. This approach can be used to assess received signals between communication devices, adding additional confidence that the signal being received is from a trusted device.

Our Anomaly Detection work uses an ML model to check if the transmission received really came from the emitter the device claims it came from.​ The premise behind this technique is to understand if devices within the local network are originating from trusted source. 

For the task of anomaly detection, the model’s training data only contains packets from known trusted devices. ​The model’s job is to use this data to build up an understanding of what is “normal”, such that it can identify anomalies during the testing phase. ​

Rowden has conducted experimentation and successfully demonstrated this on edge hardware for IoT devices, with variation in performance depending on the Machine Learning and Deep Learning method applied.

IoT_Anomaly_Detection

RF ML is an exciting area of research that Rowden will continue to invest in to better understand the limitations and operational potential for users at the edge.

To develop this field and further scrutinise the applicability and robustness of AI and ML models in this domain, specialists need to be able to legally collect representative real-world data or have access to controlled environments in which to conduct experiments.

Creating a multi-layered RF environment is highly complex. The real world has a tremendous impact on the way RF signals propagate. Multipath, fading and noise all impact the way in which signals are modified and received; developing such an environment requires significant time and investment​. 

Machine_Learning_sensor_processing_RF_engineer

Journal

A view on open-source software security

Object detection with synthetic training data

Overcoming the barriers to delivering software-centric capability

RF Machine Learning at Rowden