Text this: Neutron star binary orbits in their host potential