It all depends if having a fence is such a big deal for you. If you want the fence to stay up, then repair it or put up a new one. If you dont care either way, take it down.
You are by no means required to keep the fence there.
If your neighbour has a problem with that, well then they can put up a fence themselves.
You are by no means required to keep the fence there.
If your neighbour has a problem with that, well then they can put up a fence themselves.


I suggest as others do, that as the original fixings are on your side, its down to you. Dont suggest anything that may upset the peace