I have heard some non-theists claim that the universe never had a beginning – it's just always been (though not in its present form). However, in my article at the Alfred Society in response to Hume's Dialogue Concerning Natural Religion, I point out that such a position is deeply problematic. Just consider for a moment: the conditioned can never arrive until the conditions are complete; but if those conditions are infinite then they can never be complete (one could never reach the top of an infinite set of stairs no matter how long one climbed.) Now the conditions on which present cause and effects exist is that all the past causes and effects have been completed. It follows that if those conditions are infinite (that is, if the past series of cause and effect recedes infinitely into the past), then they can never be complete and we could never have reached the present moment.
Or consider again. An actual infinite, by definition, cannot be added to. However, if the past series of events forms an actually infinite then each new moment in time is adding to that infinite, which is impossible. Since, by definition, one cannot traverse an actual infinite, the past series of events cannot stretch backward to infinity otherwise we would never have reached the present moment.
(Such an argument only applies to properties within a time sequence. Thus, it does not bare relation on the infinity of God unless it were first assumed that God’s existence also flows in the time stream of past, present and future. But such an assumption is questionable.)