Measurements of starlight polarized by aligned interstellar dust grains are used to probe the relation between the orientation of the ambient interstellar magnetic field (ISMF) and the ISMF traced by the ribbons of energetic neutral atoms discovered by the Interstellar Boundary Explorer spacecraft. We utilize polarization data, many acquired specifically for this study, to trace the configuration of the ISMF within 40 pc. A statistical analysis yields a best-fit ISMF orientation, B
magpol, aligned with Galactic coordinates ℓ = 42°, b = 49°. Further analysis shows the ISMF is more orderly for “downfield” stars located over 90° from B
magpol. The data subset of downfield stars yields an orientation for the nearby ISMF at ecliptic coordinates λ, β ≈ 219° ± 15°, 43° ± 9° (Galactic coordinates l, b ≈ 40°, 56°, ±17°). This best-fit ISMF orientation from polarization data is close to the field direction obtained from ribbon models. This agreement suggests that the ISMF shaping the heliosphere belongs to an extended ordered magnetic field. Extended filamentary structures are found throughout the sky. A previously discovered filament traversing the heliosphere nose region, “Filament A,” extends over 300° of the sky, and crosses the upwind direction of interstellar dust flowing into the heliosphere. Filament A overlaps the locations of the Voyager kilohertz emissions, three quasar intraday variables, cosmic microwave background (CMB) components, and the inflow direction of interstellar grains sampled by Ulysses and Galileo. These features are likely located in the upstream outer heliosheath where ISMF drapes over the heliosphere, suggesting Filament A coincides with a dusty magnetized plasma. A filament 55° long is aligned with a possible shock interface between local interstellar clouds. A dark spot in the CMB is seen within 5° of the filament and within 10° of the downfield ISMF direction. Two large magnetic arcs are centered on the directions of the heliotail. The overlap between CMB components and the aligned dust grains forming Filament A indicates the configuration of dust entrained in the ISMF interacting with the heliosphere provides a measurable foreground to the CMB.