When Janitors Go Bad: CCP Games Accidentally Pulls Plug on Epic Space Battle

EVE Online, yesterday:

In the midst of a potentially historic conflict in the Z9PP-H system of the war-torn Fountain region of New Eden, two formerly allied player groups clashed. The CFC, led by Goonswarm, attempted to thwart a TEST alliance and Pandemic Legion operation to claim sovereignty of the system.

The crescendo of battle was rising to epic proportions, when suddenly the plug was literally (if digitally) pulled.

The self-described janitors of New Eden, CCP Games, switched off a vital node and ended the battle, instantly disconnecting every participant.

You Won’t Like Them When They’re Angry

EVE players – especially those invested in demanding null-sec sovereignty wars – are not known for their mild-mannered tolerance. Unsurprisingly, social media and various forums exploded with frothing rage and gleeful Schadenfreude.

Accusations of foul play and CCP favouritism were levelled, but CCP representatives were quick to explain. CCP Explorer (@erlendur on Twitter) offering an apology to all, including the leader of the coalition who were denied their victory; 

@TheMittani Human error, forum thread incoming; we are very sorry for this mistake 🙁

— Erlendur (@erlendur) July 4, 2013

Further explanation was later offered on the official forums, explaining that in cases of extreme server load steps are taken to redistribute the resources of the single-shard Tranquility game server. CCP Falcon stated:

“Owing to the heavy load in Z9PP-H, caused by a fleet flight involving 2200+ pilots deeply embroiled in the Fountain War, the decision was made to remap all other systems away from the node on which Z9PP-H was located in order to improve responsiveness and playability. We were later than we wanted to be to try to reinforce the node (obviously) since it was a more organic-type battle without a fleet fight notification. Once we fully assessed the situation and what was at play, we figured it was time to go for it and the remap in preparation of what was set to be “Asakai 2.0”.” – CCP Falcon, EVE forum

However, that remap went wrong as the technician responsible for moving other star systems away from the conflict system of Z9PP-H, instead moved all the systems on the node, including Z9PP-H. The system was quickly returned to order, but by then it was too late.

Surprise Escape Route

The clash had already been ongoing for several hours and the tide had turned dramatically in favour of the CFC, with their sub-capital fleet of Megathron battleships ensnaring approximately 150 enemy Archon capital ships, holding the massive TEST carriers at their mercy. If nothing had upset the balance at that point, the CFC could have destroyed the entire fleet and deal a very painful blow to their enemies.

Based on the approximate in-game market value of 500 million ISK (InterStellar Kredits) for a $19.95 one-month “PLEX” subscription, those 150 carriers were worth an estimated $12,000 (or 9.6 Star Citizen Idris Corvettes).  Some scalp to be taken.

The node was subsequently restored but the formerly doomed carrier pilots had the good sense to stay logged off to avoid falling back into almost certain destruction.

CCP Falcon offered this apology;

“All pilots involved have our sincerest apologies for this error, and we are currently looking to review our policy of action in these types of situation to ensure that this is not repeated.” – CCP Falcon

Massive Damage

EVE Online is a game sold on the promise of huge fleet battles waged in ongoing wars fuelled by the grudge and greed of huge player-driven empires.

The CCP Games PR machine is not shy about extolling the virtues of their emergent gameplay sandbox which produces frequent clashes between hundreds – sometimes thousands – of valuable internet spaceships.

Earlier this year, the Battle of Asakai hit the headlines when player error triggered a vast feeding frenzy resulting in the convergence of over 3000 players in a single system.

The thousands of ships destroyed in the battle had a purchasable value equivalent to somewhere between $28,000 and $33,000. All of these ships will likely be replaced, ultimately meaning more revenue for CCP. Furthermore, the positive buzz that is generated by such events undoubtedly has a positive effect on subscriptions.

The battle of Z9PP-H was set to be just as lucrative, with a player-run live stream with an audience rising to 5000+ providing coverage and explanation of the events as they unfolded. This could potentially have put EVE’s fleet fights into a whole new shop window and demystified much about what goes on in that sector of EVE gameplay.

Breaking the flow of such a high profile battle was a real disappointment for players and CCP Games.