Traffic states typically exhibit regular spatio-temporal patterns that support accurate prediction. However, traffic accidents can disrupt these patterns and introduce irregular fluctuations that challenge forecasting models. Previous studies have focused on predicting accident hotspots, risks, or traffic states using simple accident features. Yet, because traffic accidents are infrequent and exhibit spatio-temporal biases relative to collected traffic sensor data, they may significantly or only slightly affect traffic states, posing challenges for accurate prediction. In this study, we propose a large language model (LLM) enhanced Causality-aware Traffic accidents post-effects prediction (LLeCaT) to uncover the causal relationship between traffic states and accidents. Our model integrates fine-tuned LLMs to extract semantic information from accident records and estimate their effects on future traffic states following their occurrence. To validate the effectiveness of our proposed method, we collected and processed publicly available anonymized traffic accident records, traffic state datasets, and environmental information. We conducted extensive experiments demonstrating its superiority over all baselines. Additional ablation studies and case analyses further confirm the validity of the estimated causal effects.