What obstacles did James Madison face while in office?