Spectral Autonomy

Machine learning for electronic warfare

Part of the Field Guide to Cognitive Electronic Warfare.

Machine learning can support signal recognition, state estimation, and decision-making in electronic warfare. Its usefulness depends on whether the training data and evaluation method represent the task and conditions in which the model will be used.

In this guide

Teaching a receiver to recognize a signal

Consider a receiver that must assign recorded signals to several transmitter categories. A machine-learning model can learn the relationship between examples of those signals and their assigned categories. Once trained, it applies that relationship to a new observation and produces a prediction. The engineering task is to make the learned relationship useful on the observations the receiver will actually encounter.

In supervised learning, an example consists of an input and a label, the answer supplied during training. A label might identify a modulation, a transmitter category, or an individual device. Modulation describes how information is represented in a signal; identifying it is a different task from identifying the equipment that produced the signal. Choosing the label therefore defines what the model is being asked to learn.

The input must also be chosen. It may contain receiver samples, a transformed representation of those samples, or selected features such as signal bandwidth and duration. A feature is a measured property used as an input to the model. Feature selection puts some engineering knowledge about the task into the representation; using less processed samples leaves more of that relationship for the model to learn.

The model's output should be interpreted in terms of these choices. A successful modulation classifier has learned evidence about signal structure. A device classifier is being asked to learn evidence that separates physical transmitters. Even when both operate on the same recordings, their results answer different questions.

How a dataset can teach the wrong relationship

A recording contains more than information about its transmitter. The propagation path, receiving equipment, background signals, and collection settings all affect the samples. A model can use any repeatable association in those samples to predict the label, including an association that will disappear outside the dataset.

Suppose every recording of transmitter category A was collected with one receiver and every recording of category B with another. A model may learn the receivers' differences. It could score well on recordings collected in the same way and fail when both transmitter categories are observed through a third receiver. The numerical score would be real, but the experiment would not have established the intended ability to distinguish transmitter categories across receivers.

Collection metadata helps expose this problem. Recording which device, receiver, session, and conditions produced each example allows the evaluator to separate those factors. Haigh's treatment of cognitive-EW data emphasizes the need to account for diversity and changing collection conditions when developing learned systems.

Designing the test around the intended use

Evaluation data is held out when it is reserved from fitting the model and selecting its settings. The important design choice is what gets held out together. Randomly separating samples from one recording tests performance on samples closely related to the training data. Reserving an entire collection session, receiver, or transmitter tests a different kind of generalization.

For the transmitter-category example, the test can reserve devices that belong to familiar categories but were not present during training. That asks whether the model learned properties of the category that extend to another device. A separate test using a different receiver asks whether the result survives a change in the measurement chain. Keeping those questions explicit makes the results interpretable.

Development and held-out evaluationDevelopment data are used to fit a model. Separate evaluation data are used to measure it. The evaluation set does not feed the fitting step.metricsDevelopment dataHeld-out evaluation dataModel fittingEvaluationReported result

Scroll horizontally to inspect the diagram.

Figure 1. Data used to fit the model and select its settings are separated from the final evaluation. The holdout design determines whether the result concerns new samples, devices, sessions, or conditions.

The same logic determines the baseline, the reference method used for comparison. If the proposed contribution is learning from new examples, one useful comparison is the same classifier with its updates disabled. If the contribution is a new representation, both representations can be evaluated on the same task and held-out data. The comparison should isolate the part of the method whose value is being tested.

Handling a signal outside the known categories

A classifier trained to choose among a fixed set of categories will normally select one of them even when the input belongs elsewhere. Open-set recognition adds a way to identify an input as unknown. This changes the task from choosing the best known label to deciding whether any known label is appropriate.

Unknown status and changed measurement conditions are related but distinct. An unfamiliar transmitter category may require rejection because it lies outside the label set. A familiar category observed through a new receiver may differ from the training data while still belonging to a valid known category. Out-of-distribution detection concerns the difference from the development data, which can arise in either situation.

The rejection rule creates a practical tradeoff. Rejecting more uncertain inputs can reduce incorrect known labels, but it can also discard usable observations of familiar signals. A test should show both effects so that the decision threshold can be related to the receiver's purpose.

Learning as observations arrive

Online learning updates a model during use. In the running example, new confirmed classifications could improve its treatment of a transmitter or measurement condition that was poorly represented during development. The new examples arrive in a sequence, so the information available to the model changes over time.

An evaluation of this process must preserve that sequence. The model makes a prediction using its current state, receives whatever feedback would actually become available, and is then updated. Testing on information from later in the sequence would give the model knowledge that it did not yet possess at the time of the decision.

The updated model also becomes a new system configuration. Retaining its version, update data, and evaluation results allows an engineer to investigate a later change in behavior. NIST's AI Risk Management Framework treats measurement and monitoring as continuing activities throughout a system's use, which is particularly relevant when its learned parameters change.

Reusing a learned representation

Pretraining approaches the data problem by learning a representation from a broader collection before adapting it to a specific task. A subsequent classifier may then need fewer task-specific labeled examples to make use of useful signal features. In self-supervised pretraining, the training objective is constructed from the data itself rather than from manually assigned category labels.

The question for the receiver designer is how well that representation transfers to the intended task. Comparing it with another method on the same labeled data and held-out conditions reveals whether it improves the result. The next chapter places such learned components back into systems whose sensing and decisions affect one another.