Acute myeloid leukemia (AML), a common hematological cancer of myeloid lineage cells, generally exhibits poor prognosis in the clinic and demands new treatment options. Recently, direct sequencing of samples from human AMLs and pre-leukemic diseases has unveiled their mutational landscapes and significantly advanced the molecular understanding of AML pathogenesis. The newly identified recurrent mutations frequently "hit" genes encoding epigenetic modulators, a wide range of chromatin-modifying enzymes and regulatory factors involved in gene expression regulation, supporting aberration of chromatin structure and epigenetic modification as a main oncogenic mechanism and cancer-initiating event. Increasing body of evidence demonstrates that chromatin modification aberrations underlying the formation of blood cancer can be reversed by pharmacological targeting of the responsible epigenetic modulators, thus providing new mechanism-based treatment strategies. Here, we summarize recent advances in development of small-molecule inhibitors specific to chromatin factors and their potential applications in the treatment of genetically defined AMLs. These compounds selectively inhibit various subclasses of "epigenetic writers" (such as histone methyltransferases MLL/KMT2A, G9A/KMT1C, EZH2/KMT6A, DOT1L/KMT4, and PRMT1), "epigenetic readers" (such as BRD4 and plant homeodomain finger proteins), and "epigenetic erasers" (such as histone demethylases LSD1/KDM1A and JMJD2C/KDM4C). We also discuss about the molecular mechanisms underpinning therapeutic effect of these epigenetic compounds in AML and favor their potential usage for combinational therapy and treatment of pre-leukemia diseases.