First A submission (AAMAS): how much theory is enough when your experiments went sideways? [D]
Heat trend
Collecting trend data
The percentage is based on available heat signal, not comment count or independent people.
A PhD candidate is seeking advice for their first AAMAS submission, facing challenges with experimental results that only partially support their initial hypothesis. They are struggling with developing sufficient formal theory, having fallen into HARKing by building theory after seeing results. Issues include theory tied to specific training outputs rather than structural properties, and the discovery of incorrect hidden parameters in their codebase. They question how much theory is expected for empirical MARL papers at A* venues and how to recover from HARKing under publication pressure.
Hi everyone,
2nd-year PhD candidate here staring down my first A* submission deadline (AAMAS 2027). I could really use some perspective on theory expectations, especially since I think I’ve methodologically painted myself into a corner.
The setup
My project started with a clean hypothesis: if architecture X is more robust than Y to perturbation A, and B is a strictly harder version of A, then the X > Y ordering should hold under B as well. I isolated three variables I suspected were driving the effect, ran experiments, and… got results that only partially support the hypothesis, with clear boundary conditions.
Where I got stuck
Trying to explain the “why” mathematically sent me down a theory rabbit hole. I ended up with two bad options:
- Claims tied to specific training outputs rather than structural/architectural properties, or
- Weak, hand-wavy speculations that feel like post-hoc rationalizations.
I’m pretty sure I fell into HARKing.. I started building theory after seeing the results instead of deriving predictions beforehand.
Furthermore, my codebase is built on an undocumented public repo, and I recently found a bunch of hidden parameters set to wrong values for my setting. I’m currently re-running everything, which is why I’m being vague about specifics. My “insights” from the first round are probably garbage.
My actual questions
- For those who’ve reviewed for or published at AAMAS (or similar A* venues): how much formal theory is actually expected for an empirical MARL paper? Is “here’s the phenomenon, here’s the controlled experiments, here’s a plausible but incomplete theoretical sketch” a death sentence?
- If the theory ends up being training-dependent rather than structural, is that a sign I should pivot to a lower-tier venue, or can strong empirical characterization + limited theory still fly at A*?
- How do you recover from HARKing mid-project when you’re under pressure to publish in year 3/4 of a 4-year contract?
Any advice on how to salvage the timeline or reframe the narrative would be hugely appreciated.