Gibbon outlined a bunch of reasons for Decline. I too did not agree with the Christianity claim.

In the empire building days, the soldiers could "earn" a nice piece of land after a couple of decades of service. That, to me, was the prime motivation of solders. When that carrot was gone, it was harder to find dedicated soldiers.