Open the campaign in My campaigns and choose the option to end it. Its story and updates remain visible, but the page stops asking for support.
The current app also offers a restart flow for an ended campaign. It asks you to choose a new future end date before restarting.