I little late, since it is already Epiphany, but here goes....
Early Christians celebrated Our Lord's conception on March 25 (or April 6 - orthodox).
Why? Because that date corresponds to Nisan 10.
What is so special about Nisan 10? It is the date given in Exodus 12 that the lambs without blemish are to locked away until their sacrifice for the Passover.
So....9 months after March 25/Nisan 10 is December 25 (or January 6 orthodox Christmas).
We celebrate His Holy Conception with the Feast of the Annunciation on March 25.
It's not pagan. It's the liturgical calendar.