The real purpose of the E.U. is not economic, but political. It began as the ECSC, which was geared to making sure that Germany would not re-militarize by extracting iron from the Rhine region. The purpose of the E.U. is to obviate the sort of bloodshed that Britain saw in WWI and WWII. If the British people don't want to be in the E.U., then you should leave. I don't believe that even your own government should keep you from deciding such a matter as a people, directly. That said, with great power comes great responsibility, and this applies to popular sovereignty. In other words, the people taking up the mantle of direct democracy in a constitutional referendum should make an informed decision, looking beyond even the people's own immediate interests. The stakes are much, much higher than whether being in the E.U. is an economic net loss or gain to Britain on a yearly basis, or even whether the City is crimped or inconvenienced. Much more is at stake.
After watching Downton Abbey(season 2), War Horse, or Saving Private Ryan, you might want to reflect on the risks involved in secession. You might just be anticipating a future war between you and the continent. You might even be starting a chain reaction that could result in Europe itself being fractured. The Hungarian right, for instance, is urging Hungary to secede. Lest you think there is no long-term benefit in Eastern Europe being in the E.U., you might study Serbia a bit more than a century ago.
Oh, and by the way, the word from Switzerland is that the EEA is just about as restrictive as the E.U. because the EEA had to adopt most of the E.U.'s economic regulations in order to trade with the E.U.'s states. So be careful what you wish for—you might find that you have even less impact on trade regulations that bind you. You might be better off with a “multi-track” E.U. that, unlike the U.S., accommodates different preferences among the states. In other words—and I suspect this will not be lost on any of you—you can be part of something that outdoes the U.S. by improving immensely on modern federalism at the empire-level by showing the world how flexibility can be built in. Or, you can sit on the sidelines as the E.U. continues on to "ever closer union" while accommodating those states that want more room yet are nonetheless part of the union—part of the effort to obviate war in Europe for your generation and your posterity. The choice is yours.