Martin Wendiggensen
Auto-Poking The Bear – Analytical Tradecraft In The AI Age
Analytical tradecraft and shared standards have transformed Cyber Threat Intelligence from a niche discipline into a collaborative industry-wide research endeavor. Researchers and analysts now routinely build on each other’s work, creating a foundation of trust and shared methodology.
AI is disrupting this ecosystem, as we increasingly delegate data preparation, analysis, and entire workflows to AI assistants. Doing so will make us more productive, but not without cost. While you may trust your own AI-assisted analysis, can you trust another researcher’s prompts/agent process? As questions about reliability and transparency persist, we will need to adapt our research methodology and develop a new joint understanding of the promises, pitfalls, and probabilities inherent in AI-assisted work.
We tackle these challenges through a concrete case study. We present our own LLM-based agentic system, developed to analyze Russian internet data leaked by Ukrainian cyber activists. We’ll walk through the system’s architecture and demonstrate its performance across tasks ranging from simple data collation to sophisticated analytical workflows to track adversaries.
Along the way, we outline how to understand the promises and limitations of this technology and more importantly, how to communicate them transparently to other researchers and audiences – so that we maintain transparency and accountability for published products.
Martin Wendiggensen is an AI Research Scientist at Dreadnode and PhD candidate at Johns Hopkins AIST. His research focuses on how AI is shifting the Cybersecurity Offensive-Defensive Balance. He received a Bachelor’s degree in Political Science with a focus on quantitative methods and Natural Language Processing from the University of Mannheim and studied in China, Israel, and Italy. After working as policy advisor to a member of the German National Parliament’s Foreign Affairs Committee, Martin received a Master’s degree in International Relations from Johns Hopkins SAIS. He has conducted research at NATO as well as the University of Mannheim and specialized in leveraging High Performance Computing for applied research on AI capabilities at Hopkins. Currently, he is a visiting researcher at ETH Zurich.
