One-stage object detector has been the most widely used framework in modern object detection due to its excellent performance and high efficiency. Label assignment, which is designed to discriminate positive and negative samples in training process, is closely correlated to the detection performance of one-stage detectors. Previous works commonly utilize geometric prior such as anchor box or key point to determine positive samples. Despite its simplicity, the heuristic strategy is rigid and it might limit the upper bound of detection performance. By introducing extra semantic information, prediction-aware geometric score and sample re-weighting mechanism, we propose a novel strategy called Dynamic Label Assignment in this paper. To validate the effectiveness and generalization of our method, we conduct extensive experiments on the MS COCO dataset. Without bells and whistles, our best model with ResNeXt-101 as backbone achieves state-of-the-art 46.5 AP, surpassing other strong methods such as SAPD [30] (45.4 AP), ATSS [25] (45.6 AP), and GFL [11] (46.0 AP) by a large marigin.