Objective: To determine the relative importance weights of items and grades of a newly developed additive outcome measure called the juvenile idiopathic arthritis (JIA) magnetic resonance imaging (MRI) scoring system for the temporomandibular joint (TMJ) (JAMRIS-TMJ). Methods: An adaptive partial-profile, discrete choice experiment (DCE) survey using the 1000Minds platform was independently completed by members of an expert group consisting of radiologists and non-radiologist clinicians to determine the group-averaged relative weights for the JAMRIS-TMJ. Subsequently, an image-based vignette ranking exercise was done, during which experts individually rank ordered 14 patient vignettes for disease severity while blinded to the weights and unrestricted to JAMRIS-TMJ assessment criteria. Validity of the weighted JAMRIS-TMJ was tested by comparing the consensus-graded, DCE-weighted JAMRIS-TMJ score of the vignettes with their unrestricted image-based ranks provided by the experts. Results: Nineteen experts completed the DCE survey, and 21 completed the vignette ranking exercise. Synovial thickening and joint enhancement showed higher weights per raw score compared to bone marrow items and effusion in the inflammatory domain, while erosions and condylar flattening showed nonlinear and higher weights compared to disk abnormalities in the damage domain. The weighted JAMRIS-TMJ score of the vignettes correlated highly with the ranks from the unrestricted comparison method, with median Spearman's ρ of 0.92 (interquartile range [IQR] 0.87–0.95) for the inflammation and 0.93 (IQR 0.90–0.94) for the damage domain. Conclusion: A DCE survey was used to quantify the importance weights of the items and grades of the JAMRIS-TMJ. The weighted score showed high convergent validity with an unrestricted, holistic vignette ranking method.