I'm going act a little bit of a devils advocate here, as I think there are some key inconsistencies and omissions here which haven't really been explored and they could have an effect on how the OP would wish to pursue the issue. Others have been picked up on but not expanded upon. I'm going to look at this from the LL/Agent side.
It seems to me that the LL would have remedies available in a number of areas here:
1. Breach of Covenant (potentially twice).
2. Remedy in the Tort of Deceit, either by negligent or fraudulent misrepresentation.
3. Enforcement of an implied commercial contract (slightly tenuous).
Tackling breach of covenant not to sublet - we all know that an AST has an implied term prohibiting subletting without consent even if it is not expressly stated in the tenancy agreement (s15 HA 1988). The OP says she had permission to sublet to the first two lodgers but is silent on the third. If true then she is in the clear on the first two, but she would have to produce some evidence that she had consent imo. However the 3rd is the key issue here as upon taking the third 'tenant' she seems to have ceased occupying the property herself. She doesn't seem to have sought consent for this and the LL would be entitled to seek damages for the breach.
Misrepresentation. The LL could easily take the view that the OP has induced this AST by misrepresenting the facts either negligently or fraudulently. This is a bit more serious than breach of covenant as the pool of potential damages is wider. This comes down to the intention of the OP. If her intention was never to live in the property through the tenancy herself (to be an AST she must live at the property) and she represented otherwise (very much implied by signing an AST) then she could be liable here. The fact that she either a. never lived in the property, or b. lived in it for a short time in a sham arrangement; that she seems to have rented out the rooms in the nature of a business for commercial gain; that she lives nearby in another property - would all greatly help the LL's case. If proved then the tenancy is voidable.
Lastly there is tenancy but it is a commercial one. Here the OP is relying on protections built into a statutory regime that the LL cannot contract out of (the AST). But these protections are only available to someone fulfilling the AST criteria (one of which is being in actual occupation). In this case the LL could argue that in fact in the tenancy is in fact a commercial one and should have been negotiated on terms customary for this kind of agreement. That would mean the OP would become responsible for all the expenditure which the LL has covered which a commercial tenant would have picked up. The analogy here is consumer protection, if I go into Tesco's and buy 100 TVs for resale in my shop, I cannot claim the same protections (for returns, faults, etc) as a consumer. I doubt if this is needed as I think the LL would succeed on the first two.
So - although only in case 2 is the tenancy actually void, I think the agent is basically challenging the OP to try and claim their rent and deposit back, probably confident that the OPs position is sufficiently shaky to make both a straightforward claim difficult and the counter-claim and costs potentially high. The LLs position is made easier by the fact they are fulfilling their statutory duty by recognising the tenancies of the current occupants and by the fact that the OP is not in actual occupation - therefore no possession proceedings.
As I say - just being devils advocate!
If it sounds like I know what I am talking about........I don't.