What happens if an award is set aside — do I have to start the arbitration again?

Not necessarily. If the award is set aside on a procedural ground (e.g., denial of right to be heard), the court may remit the matter back to the same arbitrator to remedy the defect, or may direct that a new arbitration be commenced before a different arbitrator. If the agreement itself is void, the parties revert to court litigation.