Total bloody minefield, i looked into this a bit when we bought our house, our deeds say we are responsible for all the boundary fences, and i suspect the left side fence bit only applies if stated so.
If i understand your description correctly it sound like not only have they removed the existing fence, they have taken up a pathway that you own and damaged a piece of you drainage on your property, if so a solicitor is the next option,, but it will get expensive if they have the rights to the fence on their deeds, if they haven't you should be able to take them to court to rectify the damage, but again its likely to cost you more than the price of erecting a high fence on your side, if its between gardens i think there is no hight restrictions, its only next to a highway that has hight restrictions,
So bung a 8 foot job up, build a small wall against/under it to stop the drainage water that is also coming your way, with a bit of luck it will flood the buggers out
Oh and if the dogs bark all the time you can report that as well and they will have to do something about it, good luck.