Purpose: To provide a comprehensive study on the accuracy of many commonly used Intensity Modulated Radiation Therapy (IMRT) treatment planning systems using the Radiological Physics Center's (RPC) anthropomorphic thorax phantom. Method and Materials: Treatment planning systems (TPSs) from Corvus, Eclipse, Pinnacle, and Tomotherapy were evaluated using the RPC anthropomorphic phantom. Treatment plans were designed using the same clinical constraints and prescriptions so that 96% of the planning target volume (PTV) was covered by the prescription dose. The phantom is equipped with TLD located in the tumor, heart, and spinal cord and radiochromic film located in three anatomical planes intersecting the tumor center and extending into the lung. IMRT QA was performed to adjust the calculated dose distributions in order to isolate the effects of heterogeneity. Comparisons were made between each TPS calculation and measurement. In two instances, re‐calculations of the original correction based pencil beam (PB) plans were performed using the superposition convolution (SC) method. Results: TPSs employing superposition convolution algorithms predicted dose within 3.6% of the target TLD, while TPSs using correction based pencil beam algorithms predicted dose within 5.0% of the target TLD. Both algorithm types showed variations (2% to 38% in the cord and heart) in predicting low dose to normal structures. The dose distributions within the PTV and penumbra lung regions showed good agreement when using an SC algorithm. However, TPSs using the PB type algorithm overestimated dose in the PTV and underestimated the extent of penumbra broadening corresponding to the surrounding lung. Conclusion: This work demonstrated that superposition convolution algorithms found in widely used IMRT treatment planning systems are able to calculate the dose accurately to the PTV and penumbra regions when low density heterogeneities are involved. Conflict of Interest: This work supported by PHS CA010953 and CA081647 awarded by NCI, DHHS. © 2007, American Association of Physicists in Medicine. All rights reserved.