We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Beyond CAGE: Investigating Generalization of Learned Autonomous Network Defense Policies

Abstract: Advancements in reinforcement learning (RL) have inspired new directions in intelligent automation of network defense. However, many of these advancements have either outpaced their application to network security or have not considered the challenges associated with implementing them in the real-world. To understand these problems, this work evaluates several RL approaches implemented in the second edition of the CAGE Challenge, a public competition to build an autonomous network defender agent in a high-fidelity network simulator. Our approaches all build on the Proximal Policy Optimization (PPO) family of algorithms, and include hierarchical RL, action masking, custom training, and ensemble RL. We find that the ensemble RL technique performs strongest, outperforming our other models and taking second place in the competition. To understand applicability to real environments we evaluate each method's ability to generalize to unseen networks and against an unknown attack strategy. In unseen environments, all of our approaches perform worse, with degradation varied based on the type of environmental change. Against an unknown attacker strategy, we found that our models had reduced overall performance even though the new strategy was less efficient than the ones our models trained on. Together, these results highlight promising research directions for autonomous network defense in the real world.
Comments: NeurIPS 2022 Workshop: Reinforcement Learning for Real Life
Subjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR)
Cite as: arXiv:2211.15557 [cs.LG]
  (or arXiv:2211.15557v2 [cs.LG] for this version)

Submission history

From: Andy Applebaum [view email]
[v1] Mon, 28 Nov 2022 17:01:24 GMT (242kb,D)
[v2] Wed, 30 Nov 2022 14:35:42 GMT (242kb,D)

Link back to: arXiv, form interface, contact.