Wearable sensors provide an exceptional opportunity in collecting real-Time behavioral data in free living conditions. However, wearable sensor data from observational studies often suffer from information bias, since participants' willingness to wear the monitoring devices may be associated with the underlying behavior of interest. The aim of this study was to introduce a semiparametric statistical approach for modeling wearable sensor-based physical activity monitoring data with informative device wear. Our simulation study indicated that estimates from the generalized estimating equations showed ignorable bias when device wear patterns were independent of the participants physical activity process, but incrementally more biased when the patterns of device non-wear times were increasingly associated with the physical activity process. The estimates from the proposed semiparametric modeling approach were unbiased both when the device wear patterns were (i) independent or (ii) dependent to the underlying physical activity process. We demonstrate an application of this method using data from the 2003-2004 National Health and Nutrition Examination Survey (N=4518), to examine gender differences in physical activity measured using accelerometers. The semiparametric model can be implemented using our R package acc, free software developed for reading, processing, simulating, visualizing, and analyzing accelerometer data, publicly available at the Comprehensive R Archive Network.