Project RED QUEEN
From Meritology
Introduction
I’m starting on an academic-oriented research project and I’m looking for collaborators, contributors, reviewers, etc. I'd also like to faciliate collaboration between other researchers doing work in this area.
If you'd like to participate, collaborate, or comment, please contact me: russell.thomas <at this domain name>.
Topic
The topic is the arms race between attackers and defenders from the perspective of innovation rates and “evolutionary success” – the Red Queen problem (running just to stand still). Here’s a sample research question: “can bureaucracies (defenders) keep up with a decentralized black market (attackers)?”, and similar. Answering these research questions would have policy implications on the effectiveness of regulation/mandates vs. incentive-based approaches, R&D policy, etc.
Focus
Personally, I want to focus primarily on theoretical models, but I’m also keen on grounding them in reality. If I can present some empirical data on the rate of innovation for various players as calibration, that would be superb.
On the theory side, I will be drawing from Evolutionary Ecology (host-parasite co-evolution, adaptive landscapes), Political Economy (models of real arms races), Computational Social Science (agent-based models, genetic algorithms, evolutionary game theory), and Economic-Engineering models of innovation and organization learning (risk/reward, optimal investment, etc.). I will also draw on “computable economics” that attempts to measure the information processing/learning capabilities of central planning vs. markets, etc.
Data Needed
Regarding empirical data, I would be interested in any of the following:
- Rate of innovation in the underlying information and IT environment
- What’s the half-life of the IT architecture in a large organization?
- What’s the product life for computing platforms?
- What’s the innovation rate for new forms of information or information standards (e.g. XML)?
- Rate of innovation in attacker tools, methods, and capabilities
- Timeline of major innovations (first appearance and widespread use)
- Time between discovery of vuln and widespread availability of exploit
- % of exploits that are Zero-day vs. known/resolved vulns
- Regime change in time series data that signals a major innovation (e.g. the phishing boom)
- Appearance rate of new monetization schemes, etc.
- Rate of innovation in defender tools, methods, controls, and capabilities
- Lifecycle of major technology solutions (products or products+services)
- What’s the half-life of corporate security policies? How often do policy manuals or training need to be completely redone?
- How long does it take to evaluate, test, and widely deploy some new capability? (e.g. web application security after 2000)
- Rate of innovation in regulations, standards (e.g. PCI-DSS), and other top-down mandates
- How long does it take to design and publish?
- How often are they updated and revised?
- How much forward-looking investigation do they do to anticipate future information security environments or threats?
- Evolution processes in the “Black Hat ecosystem
- Evolution processes information security technology and professional services ecosystem
Of course, this list is extremely broad. I’m all in favor of narrowing down to a particular security domain and ecosystem. Please make suggestions. Pointers to existing empirical reports are most welcome.
How to participate
Please email me privately (russell.thomas A-T meritology D-O-T com) if you are interested in collaborating or contributing in any way.
Resources
(humongous list of papers and books will appear here)
People
(list of people working on this topic, either collaborating via this wiki or elsewhere)
Draft Description The following is a first draft for a research project I've been considering. It may even become my PhD dissertation.
Information Security as a Red Queen Evolutionary Arms Race - A Computational Perspective
Abstract
Information security has been commonly viewed as a rivalry between attackers and defenders, and as being characterized by an “evolutionary arms race” where each side has incentives to continually create in new innovations to overcome the opponent’s capabilities. This, in turn, requires continual replacement of old solutions with new. In doing so, neither side gains a lasting advantage (i.e. the Red Queen effect). If this is true, then it will be important at a policy level to understand the dynamics of the evolutionary arms race in order to determine the intermediate and long-term trajectory of the game, and which policy decisions will swing the trajectory more in favour of defenders and for the greatest social welfare. This is especially true for government agencies that sponsor research and development (R&D), critical infrastructure companies, and regulatory bodies. In the context of cyber conflict and cyber deterrence , understanding arms race dynamics is critical because any initiative to create new offensive or deterrent capabilities by one faction could lead to a destabilizing arms race where other factions innovate to deter the deterrence, and so on. This could lead to counter productive results, including increasing instability and the likelihood of cyber conflict.
However, very little is known about the evolutionary dynamics of information security. To our knowledge, this paper is the first comprehensive research effort to create a history-friendly theoretical model using methods from several disciplines – evolutionary ecology, national security studies, computational economics and social sciences, and management science. Our goal is to introduce these new methods and show how they can endogenously model escalating innovation of both technical and economic solutions. We also aim to model arms race dynamics and the role of uncertainty and learning processes, especially mutual dependencies, counterproductive effect, and massively reciprocal uncertainty.
Outline
Our paper will have four major sections:
- Preliminaries – we will frame the research problem and the general problem of characterizing innovation arms races. We will briefly present a taxonomy of innovation arms races. We review alternative methods – game theory, dynamical systems, multi-agent systems, and other computational methods. Finally, we will include key assumptions, such as no clear boundary between ordinary cyber security (e.g. hacktivism and cyber crime) and cyber war.
- Theoretical Model – we will present our theoretical model (see below), and then demonstrate it using a relatively simple arms race – email spam vs. spam filtering. This analysis reveals surprising result: “good enough” spam filtering actually perpetuates email spam, given that spammers have been able to increase volume using botnets.
- Application to Cyber Deterrence – we will apply our theoretical model to complex, net-centric cyber security and cyber conflict scenarios. Specifically, we will describe how the theoretical model can be incorporated into a large-scale simulation model to support risk intelligence and policy decisions over three time scales – 1) long term, for the R&D policy and investment cycle, 2) intermediate term, corresponding to the months necessary to prepare for or respond to emerging threats or crisis situations, and 3) short term, on the time scale of unfolding cyber conflicts.
4. Discussion – we assess our model and results so far, and suggest directions for further research. Here is a summary of our theoretical model in its full design. (For practical reasons, we will start our research by implementing a subset of this full design and then expand it in later versions.) We draw on methods from computational social science, including multi-agent systems, adaptive automata and evolutionary computation, and computational organization theory. We use a multi-level approach and focus on two levels: 1) Capability-Action level and 2) Innovation Process level.
Capability-Action level
At the Capability-Action level, we define a multi-agent system based on algorithmic game theory over a graph topology . Each agent is a semi-autonomous strategic actor with local resources, capabilities, local and limited knowledge, boundedly rational capabilities for valuation, decision-making, and learning from experience, learning from signals, and learning from other actors. “Actors” represent people, either as individuals or organizations acting as a unit, or automated processes, or both. Actors can be purely reactive and behavioural, or fully strategic, or any combination. Compared to other multi-agent systems, our actors are cognitively complex.
Capabilities are formally defined for each actor, and encompass all possible actions for that actor, their resource requirements, and probability of success in a given context. Capabilities are formally modelled using stochastic automata, and represent arbitrary combinations of people, process, technology, and resources. Actors have a “folk knowledge” of the effectiveness of their capabilities, and can improve their knowledge through experience or from other actors. In addition to offensive or defensive cyber capabilities and resources, most actors will have a “crown jewel capability”, which represents their non-cyber raison d'etre (e.g. banking and insurance capabilities for a financial institution, war fighting capabilities for a military unit, etc.). For non-criminal actors, this “crown jewel capability” provides income for all cyber investments.
At each time step, actors can make strategic choices regarding their capabilities using their current knowledge, beliefs, and budget constraints – for improvements, augmentations, or replacements – drawing from the portfolio of available capabilities available to them. The portfolio of capabilities are subject to innovation, though the process of innovation is modelled on a higher level (see below). This approach allows us to model diffusion of innovations, “leakage” of innovations from “good guys” to “bad guys”, and other effects.
Motives of each actor are modelled by setting the functional form and parameters of their valuation system and also by setting parameters for their learning processes (used for sense-making, attention, framing, objective setting, and belief revisions). Actors can have simple models of the motives and capabilities of other actors.
Actors have a locally defined network of connections to other actors, which enable them to take actions, such as implement attacks or defenses, gather intelligence, or send signals. These connections also serve economic and organization purposes, allowing a variety of organization forms and trading relationships. Some actors have privileged positions and specialized capabilities to represent critical infrastructure – e.g. Internet Service Providers. Beyond local connections, actors are given general capabilities to interact with all other actors if they have their address (either direct address, analogous to a URL, or indirect address, similar to the old UUCP daisy chain approach). In addition to transactions through local connections, actors also have access to markets to acquire resources, products, services, and information. The details of these markets are exogenous to our model, but markets can spontaneously appear if conditions are right.
Actors can have affiliation in one or more groups or subgroups, which represent factions, allies, or merely shared interests. Groups can have positive, negative, or neutral alignment with other groups.
Cyber attack and defense are modelled at a campaign level. A “campaign” is an aggregate of moves and countermoves designed to achieve some cyber objectives, and may include one or more cyber attacks. At each time step, actors can initiate a cyber campaign, take action in a campaign, or terminate a campaign they initiated. The probability of success is an emergent function of offensive capabilities, defensive capabilities, and the environment. Success for the attacker is the partial or full achievement of the attacker’s objectives. The damage to defenders is reflected in through diminished cyber capabilities or though loss in value to the protected capability. (Though beyond our scope, this modelling approach is compatible with low-level, detailed models of cyber attacks, which would offer more realistic probability estimates and connect to empirical data.) The population of actors is dynamic and subject to evolutionary selection, which we implement using replicator dynamics. We also allow voluntary entry and exit of actors, based on the relative attractiveness of each group of actors. Success will attract new members, and lack of success will cause actors to leave the group.
Finally, We model actors in a fully symmetric fashion, meaning there is no fixed distinction between attackers, defenders, or neutral actors. Actors can change role or bias during a run, and can have mixed or ambiguous roles and also unintended or indirect effects. This net-centric approach has the advantage of allowing us to model a very wide range of scenarios, from simple cyber crime, to combined insider-outsider attacks, to richly complex cyber war or cyber terror scenarios.
Innovation Process Level
We model innovation on a separate level to manage complexity. The Innovation Process level is a separate multi-agent system where agents represent “products” within a state space of design possibilities. A “product” is a combination of features, functions, and performance capabilities, along with context interfaces. When this state space is combined with a performance or selection metric (a.k.a. “fitness”), it represents a generalized fitness landscape.
Innovation is modelled by the process of evolution in the fitness landscape via a variety of strategies, all with an aim to increase the fitness of each agent, and, in aggregate, to efficiently explore the landscape to find fitness peaks. Drawing on insights from host-parasite systems in evolutionary ecology, the fitness landscapes for attack and defense are coupled with a time delay. This allows us to model the Red Queen dynamic where innovations in either attack or defense can reduce the effectiveness or fitness of the other. Thus, finding a peak in fitness is not likely to be a lasting advantage, and might even result in much lower fitness than before the original innovation depending on competitive response. The topology of the generalized fitness landscape is based on Kaufman’s N-K model, as extended by Henderson and Clark and others .
Groups of Actors in the Capability-Actor level are associated with populations of agents in the Innovation Process level. The collective resources, capabilities, and organization of the actor groups determine their innovation capabilities. For example, actors which are isolated individuals innovate through simple processes – hill climbing, trial-and-error experiments, and imitation. Actor groups organized in networks can take on more sophisticated innovation strategies. Likewise, hierarchies of actors can implement the most sophisticated strategies, at the expense of agility, speed, and vulnerability to various biases and lock-in effects. In our Innovation Process model, there is no a priori advantage for any innovation strategy or level of sophistication. Instead, the success of any set of innovation strategies is dependent on the ever-changing shape of the fitness landscapes and the innovation capabilities of all the other actor groups.
By coupling the Capability-Action level with the Innovation Process level, we can model organization innovations such as collaboration networks, supply chains, principal-agent relationships, and hierarchies. We believe this is critical to understanding the dynamics of cyber arms races and the possible trajectories of innovation.
Research Method
Like all multi-agent models, our research method will involve running the model run many times over a range of parameters to evaluate the hypotheses . The goal is to identify characteristic patterns or regimes of behaviour that apply to a large class of scenarios and actor strategies. We will use formal inference methods, including statistical inference and analysis of competing hypothesis.
We will also seek to understand conditions of stability and instability and the interplay between the innovation arms race and the risk of cyber conflict. Finally, our intention is to design a theoretical model that can be adapted for empirical analysis, either using historical data, simulation games, or other methods. In turn, it should be possible to transform it into applied technology – a tool for a risk intelligence and decision analysis to guide policy makers.
