Observations with the Interstellar Boundary Explorer (IBEX) have shown enhanced energetic neutral atom (ENA) emission from a narrow, circular ribbon likely centered on the direction of the local interstellar medium (LISM) magnetic field. Here, we show that recent determinations of the local interstellar velocity, based on interstellar atom measurements with IBEX, are consistent with the interstellar modulation of high-energy (tera-electron volts, TeV) cosmic rays and diffusive propagation from supernova sources revealed in global anisotropy maps of ground-based high-energy cosmic-ray observatories (Milagro, Asγ, and IceCube). Establishing a consistent local interstellar magnetic field direction using IBEX ENAs at hundreds to thousands of eV and galactic cosmic rays at tens of TeV has wide-ranging implications for the structure of our heliosphere and its interactions with the LISM, which is particularly important at the time when the Voyager spacecraft are leaving our heliosphere.