The 5 calendar year rule applies to D1. Divisions D2, D3, and NAIA follow a similar rule which offers more flexibility.
They use the 10 full-time semester rule-- a player has 10 full time semesters before his eligibility is exhausted. (He still only has 4 seasons of competition eligibity.) A player who takes time off from full-time studies doesn't accrue semesters against the rule. The extreme example of this was the 59 year old Mike Flynt returning to Sul Ross State as a linebacker in 2007.
So, if you weren't a full-time student (as defined by the college you were at) in any term, that term doesn't count toward the 10. It is also possible, but difficult, to to enroll part time in the fall--but you can't practice with the team.
There is a waiver possibility, but it's really tough to qualify unless you were unable to play in the first two years.