© 2017 Institute of Physics and Engineering in Medicine. The objective of this study was to demonstrate the potential benefits of using high energy x-rays in comparison with the conventional mammography imaging systems for phase sensitive imaging of breast tissues with varying glandular-adipose ratios. This study employed two modular phantoms simulating the glandular (G) and adipose (A) breast tissue composition in 50 G-50 A and 70 G-30 A percentage densities. Each phantom had a thickness of 5 cm with a contrast detail test pattern embedded in the middle. For both phantoms, the phase contrast images were acquired using a micro-focus x-ray source operated at 120 kVp and 4.5 mAs, with a magnification factor (M) of 2.5 and a detector with a 50 μm pixel pitch. The mean glandular dose delivered to the 50 G-50 A and 70 G-30 A phantom sets were 1.33 and 1.3 mGy, respectively. A phase retrieval algorithm based on the phase attenuation duality that required only a single phase contrast image was applied. Conventional low energy mammography images were acquired using GE Senographe DS and Hologic Selenia systems utilizing their automatic exposure control (AEC) settings. In addition, the automatic contrast mode (CNT) was also used for the acquisition with the GE system. The AEC mode applied higher dose settings for the 70 G-30 A phantom set. As compared to the phase contrast images, the dose levels for the AEC mode acquired images were similar while the dose levels for the CNT mode were almost double. The observer study, contrast-to-noise ratio and figure of merit comparisons indicated a large improvement with the phase retrieved images in comparison to the AEC mode images acquired with the clinical systems for both density levels. As the glandular composition increased, the detectability of smaller discs decreased with the clinical systems, particularly with the GE system, even at higher dose settings. As compared to the CNT mode (double dose) images, the observer study also indicated that the phase retrieved images provided similar or improved detection for all disc sizes except for the disk diameters of 2 mm and 1 mm for the 50 G-50 A phantom and 3 mm and 0.5 mm for the 70 G-30 A phantom. This study demonstrated the potential of utilizing a high energy phase sensitive x-ray imaging system to improve lesion detection and reduce radiation dose when imaging breast tissues with varying glandular compositions.